I want to make a website that will periodically scrape sites like caltrain.org and then provide an API for programatically accessing that data.

As I'm working more on web applications I'm finding that there is a particular set of problems which are fixed in scope but that need constant tweaking in order to stay useful.

A possible solution for these sorts of problems might be a 3rd party online API. Here is the start of my list:

1: Email validation
2: Email verification
3: User Agent parsing

(I realize that an online API would have latency issues, but these are the sorts of things which should probably be performed asynchronously anyway)

Testing OpenID would be a lot more simple if there was an OpenID provider that was pre-filled with a lot of generated fake identities.

1. Run a program that will generate a wordlist, put the output in an queue.
   
2. Have a bunch of EC2 instances that take input from the queue and generate MD5, SHA1, etc hashes from the queue. Store hash to password mappings in SimpleDB
   
3. Depending on how fast inserts into SimpleDB are, it might make sense to store the hash/password mappings on disk or in S3 and stream them into SimpleDB
   
4. ???
   
5. Profit!
   

Since the Amazon AWS services have set prices, it should be pretty easy to calculate the dollar cost per n passwords. I'm guessing that number would be pretty low.

See also: http://twitter.com/jf/statuses/536491112

I've been fascinated by the "spam art" that manages to get through the gauntlet of spam filters at work.

It has been fun to watch the cat and mouse game that is played between the spam filterers and the spammers. The spam filterers develop methods to use OCR to scan images, and the spammers develop methods to subvert the OCR.

I just had this realization: the spammers are doing free testing for making CAPTCHAs.
One only needs to check their inbox to find the latest in OCR avoidance techniques!

I've spent the past few weeks learning about hospital bills and how to reduce them (I'll have to post more on this later, I've learned a lot of useful things). As part of this process I ended up reading about negotiation. I had no idea how fascinating this whole area is. While reading the Wikipedia article on negotiation, I realized that I would enjoy being able to play a game based on the negotiation process. progrium suggested that I write down some ideas, so here they are:

I would play a negotiation game.

Ideas:

This game will be to negotiation as Phoenix Wright is to law (without the bizarre representations of reality: No cross-examining parrots!)

You would play as a negotiator representing different organizations, and organization you would represent could give you a list of outcomes that they would want, and you would have to work out a deal making the other side happy and getting your client the outcome they want. Early "levels" might be negotiations among friends and family, eventually moving up to more difficult situations. In the end you could negotiate a trade dispute for an airline :-)

By definition, this game will not be focused on graphics, but on interactions with people, however it will be beneficial to convey emotion. A comic book style of display would be well suited to this sort of game. And I don't mean that this game should look like a cartoon, but more along the lines of Frank Miller or the Sandman comics.

Also, in negotiation there really isn't a "winning" condition. I picture that in this game, there wouldn't be a losing status, only re-negotiations.

And if you didn't get what you wanted from a negotiation, that would be a good time for your in game helper ("clippy"?) to teach you a new facet of negotiation theory.

I want a little device, a System on Chip perhaps, that has SATA on one end and a CAT-5 jack on the other. Add storage to your network by... adding storage TO your network. All you would need is some distributed filesystem to spread the love over all your directly attached network devices.

Many people seem to use sound as a way of identifying engine problems. They get used to the "sound" of their engine and get worried if something starts to sound "wrong".

I've noticed that some engines have very distinct "sound signatures". (Jeep and VW engines in particular) I've often wondered what exactly makes those engines sound so unique. Would it be possible to record the sound of each sound producing part in an engine? With recordings like those, I'm guessing that you could tweak the "sound" of each part until the mix matched your engine sound. "Oh, I guess my timing is off".

Another option is to identify "sound signatures" for specific engine failure scenarios. Then make a device that sits in your engine and listens for those signatures. This device could potentially identify failure before OBD-II would catch it. (I got this idea from my Dad, he helped make a device just like this which would detect eminent failure on electric motors)

A couple of years ago, when I was in Paris, I had to use computers in internet cafes. The possibility that my banking credentials would be intercepted by a keylogger was always on my mind. I came up with one possible solution for the problem of using secure credentials on an insecure terminal:

Summary: A secure website to change passwords for you.

My idea is this:

Make a website that can change passwords on your favorite websites (Gmail, your bank, and so on). Give this website a list of passwords and corresponding "change codes" to use. Keep this same list of passwords and "change codes" on your person. When you have used a password in an insecure, go to this website and click on the "Change my password" button. To verify that it really is you requesting the change (and not some random internet user) you would be prompted for your "change code". After verifying your change code, the website would change the password to the next password on your list.

I would love to be able to buy an antique with a history.

"Those are cool earrings! Where did you get them?"

 The answer to that question doesn't have to be a simple answer. What if you could answer that question with a story? What if you weren't the first person to have those earings?

This sort of thing happens often in large families. But what about people who have no family? What if somebody has so many things that they can't be absorbed by the extended family? Or what if you just want to spread a legacy outside of your family?

My idea is to take the idea of an antique store a little further. A store that doesn't just sell "old stuff", but that sells things with a history, an actual written history attached to each item. The store I'm envisioning would be a museum where you can buy what is on display, a museum where the displays are always changing.

I already generally despise consumerism, if I have to purchase something, I would much rather have something with a history, something with a story, something that had meaning to somebody else.

When I was searching for the lyrics to that song I'm currently obsessed with, I found that my search string resulted in suprisingly few results. I wonder if it would be possible to write software to find "statistically improbable lyrics" and then search for those lyrics. If the program doing the searches kept track of which websites had certain lyrics, I'm guessing that you could compile a list of favorite songs for certain people. An automatic last.fm I guess...

By now, I'm sure that most of you kids have seen the Postal Service "Such Great Heights" Music Video (if you haven't watch it now, it's good stuff).

Watching the video, it's pretty apparent that the footage comes from real equipment. Well it's true, there really is a company called "Skyworks" and the video was shot in their wafer fab.

What I'm wondering is why nobody has taken this great music video and turned it into a quick lesson in semi-conductor manufactcturing. It's such a perfect opportunity.

What is it like to be a trucker? What would my day be like if I was a researcher? What is it like to work for a meat packing plant?

The more time that passes, the more I realize how easy it is to have a myopic view of the world. I have the seed of an idea here - or maybe it is just a desire to learn more about the world that exists outside of the technology that I live and breathe. Maybe there are books I can read, maybe somebody has a blog that covers this area. Or maybe this really is a seed of an idea.

In any case whatever this seed is, Google seems like a good place to start.

I'm reading the latest Cringely article, and I'm wondering... has anybody made a Flash based video player that is also a Bit Torrent client? If/When YouTube dies, people are going to want to use something like this.

The world needs a program that can take IPA and pronounce it for you.

How the heck do you pronounce "jœrmuŋgandr"? Why do I have to use a manual lookup table to learn?