Cassandra is a hybrid non-relational database in the same class as Google's BigTable. It is more featureful
than a key/value store like Riak, but supports fewer query types than a document store like MongoDB.Cassandra was started by Face...
Read more »
Well, it's been a long time. But! I have four papers to add to my original distributed systems primer:coordination
CRDTs: Consistency Without Concurrency Control, Mihai Letia, Nuno Preguiça, and Marc Shapiro, 2009.
Guaranteeing eventual ...
Read more »
How many objects does a Rails request allocate? Here are Twitter's numbers:
I want them to be lower. Overall, we burn 20% of our front-end CPU on garbage collection, which seems high. Each process handles ~29,000 requests before getting killed by the...
Read more »
I've released Scribe 0.1, a Ruby client for the Scribe remote log server.
sudo gem install scribe
Usage is simple:
client = Scribe.new
client.log("I'm lonely in a crowded room.", "Rails")
Documentation is here.
The primary benefit of Scribe over something like syslog-ng is
increased scalability, because of Scribe's fundamentally distributed architecture. Scribe also does away with...
Read more »
We recently migrated Twitter from a custom Ruby 1.8.6 build to a Ruby Enterprise Edition release candidate, courtesy of Phusion. Our primary motivation was the integration of Brent's MBARI patches, which increase memory stability.
Some features of REE have no effect on our codebase, but we definitely benefit from the MBARI patchset,...
Read more »
One of the hardest gems to install is no more. It's now easy to install!
Memcached 0.15 features:
AUTO_EJECT_HOSTS bugfixes (mattknox)Install with gem install memcached. Since libmemcached is bundled in, there are no longer any dependencies.
I've been reading a bunch of papers about distributed systems recently, in order to help systematize for myself the thing that we built over the last year. Many of them were originally passed to me by Toby DiPasquale. Here is an annotated list so everyone can benefit.
It helps if you have some...
Read more »
Memcached is generally treated as a black box. But what if you really need to know what's in there? Not for runtime purposes, but for optimization and capacity planning?
demo$ sudo peep --pretty 2479time | exptime | nbytes | clsid | ...
Read more »
I'd like to call out something important from my QCon slides: the Railsbench GC settings.
quick study
In my experience, a typical production Rails app on Ruby 1.8 can recover 20% to 40% of user CPU by applying Stefan Kaes's Railsbench GC patch t...
Read more »
My QCon presentation is available.
Improving Running Components at TwitterSome choice Tweets:philwills: Evan Weaver on scaling twitter at #qcon was full of
interesting stuff and good questions from audience.
markhneedham: fascinating readin...
Read more »
#rails – Twitter Search