mult.ifario.us

Dan and I just finished up our talk at OSCON. You can download the slides or view it on Slideshare. I'll probably take it down at some point in the near future, but the sample system from the presentation is up and running for the moment.

We got started on the material for the talk several months back with the Twitter one-to-many publishing problem as a motivating problem to play with various non-relational data stores, and after some dabbling with Cassandra and HBase, we ended up focusing on Voldemort as an initial backend for the system. It is very likely that we'll craft some additional backends, and I'd particularly like to get to a more forgiving model for storing lists. (I'm already part way there on Dynomite with Osmos as the storage engine.)

The system described in the talk uses a small (two nodes) Voldemort cluster and a small cluster of web nodes (JAX-RS with a jQuery front-end) to implement enough microblogging functionality to be interesting — users, follow/followed, publishing — along with a simple dashboard implemented with Cacti and rudimentary deployment automation. The source is out on GitHub if you want to take a look. (Feel free to fork with it...)

Dan's blog entry on the presentation is here.

With Dan Diephouse, I'll be speaking at OSCON on July 23.

Taking the abstract literally, the talk looks like it is about building a Twitter clone with open source components, but it is not at all intended to be armchair quarterbacking about Twitter's early problems with availability. (We should all have these problems!) Rather, the talk is intended to be about some of the current crop of interesting open source distributed storage technologies — Cassandra, Voldemort, Redis (where the folks have already done some thinking about Twitter-like apps), CouchDB, HBase, Dynomite — as well as how to attack some of the operational problems (e.g., deployment, instrumentation, application updates) that come with using new tools in multi-node environments.

That's obviously quite a bit to fit into a relatively short speaking slot, but Dan and I plan to blog or otherwise publish material that won't fit.