up and running with cassandra
distributed systems primer, updated
object allocations on the web
How many objects does a Rails request allocate? Here are Twitter's numbers:
- API: 22,700 objects per request
- Website: 67,500 objects per request
- Daemons: 27,900 objects per action
I want them to be lower. Overall, we burn 20% of our front-end CPU on garbage collection, which seems high. Each process handles ~29,000 requests before getting killed by the memory...
scribe client
I've released Scribe 0.1, a Ruby client for the Scribe remote log server.
sudo gem install scribe
Usage is simple:
client = Scribe.new
client.log("I'm lonely in a crowded room.", "Rails")
Documentation is here.
about scribe
The primary benefit of Scribe over something like syslog-ng is increased scalability, because of Scribe's fundamentally distributed architecture. Scribe also does away with the...
ree
We recently migrated Twitter from a custom Ruby 1.8.6 build to a Ruby Enterprise Edition release candidate, courtesy of Phusion. Our primary motivation was the integration of Brent's MBARI patches, which increase memory stability.
Some features of REE have no effect on our codebase, but we definitely benefit from the MBARI patchset, the...
memcached gem release
One of the hardest gems to install is no more. It's now easy to install!
Memcached 0.15 features:
- Update to libmemcached 0.31.1
- Bundle libmemcached itself with the gem (antifuchs)
- UDP connection support
- Unix domain socket support (hellvinz)
AUTO_EJECT_HOSTSbugfixes (mattknox)
Install with gem install memcached. Since libmemcached is bundled in, there are no longer any dependencies.
on coordination
distributed systems primer
I've been reading a bunch of papers about distributed systems recently, in order to help systematize for myself the thing that we built over the last year. Many of them were originally passed to me by Toby DiPasquale. Here is an annotated list so everyone can benefit.
It helps if you have some algorithms...

