Joel Franusic ([info]joel) wrote,
@ 2007-12-27 13:17:00
Previous Entry  Add to memories!  Tell a Friend  Next Entry
Entry tags:idea, md5, password, sha1

Idea for an Amazon rainbow table super computer


  1. Run a program that will generate a wordlist, put the output in an queue.
  2. Have a bunch of EC2 instances that take input from the queue and generate MD5, SHA1, etc hashes from the queue. Store hash to password mappings in SimpleDB
  3. Depending on how fast inserts into SimpleDB are, it might make sense to store the hash/password mappings on disk or in S3 and stream them into SimpleDB
  4. ???
  5. Profit!


Since the Amazon AWS services have set prices, it should be pretty easy to calculate the dollar cost per n passwords. I'm guessing that number would be pretty low.

See also: http://twitter.com/jf/statuses/536491112

(3 comments) - (Post a new comment)


[info]rndmcnlly
2007-12-28 07:39 am UTC (link)
AWS is pretty heavyweight and expensive (compared to free, local resources).

Generating the next word in a sequence, hashing it, and storing it somewhere on a single machine is not much work (1us?). Adding an item to a SQS queue, removing it from the queue with a different program, hashing it and then storing it back to SimpleDB might take up to a whole second with packaging, handshaking, allocation, buffering, whatever. This sounds like a recipe for paying Amazon to do situps.

I think running a few monolithic processes on EC2 instances just once and then saving the resulting data once, un-indexed would be a good first step. Completely seperately you could decide how to provide easy and cheap access to the results.

(Reply to this)

Calculation?
(Anonymous)
2008-02-20 10:12 am UTC (link)
Did you ever calculate the cost of generating a rainbow table, say for MD5 of all possible 16-character strings, using EC2?

I would probably have a cluster of machines each using a different character at the start of the string, generating all the hashes from that seed and then posting the results back to a central store, rather than all the overhead of dealing with a queue.

(Reply to this) (Thread)
Re: Calculation?
[info]joel
2008-03-25 11:57 pm UTC (link)
No, I still need to do the calculation.

And yes, using a queue is probably a bad way to do this ... it's just so ... shiny.

(Reply to this) (Parent)

(3 comments) - (Post a new comment)

Create an Account
Forgot your login or password?
Log in with OpenID
English • Español • Deutsch • Русский…