Benchmarking JRuby on Rails

Last night, while working on a project I found a really neat use of Rails Components, but I also noticed that this part of Rails is deprecated, among other reasons because it’s slow.

Well, how slow? During my quest to find out, I collected some interesting data, and even more importantly put JRuby and MRI Ruby face to face.

Disclaimer: the benchmarks were not done on a well isolated and specially configured test harness, but I did my best to gather data with informational value. All the components were used with OOB settings.

Setup

  • ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0] + Mongrel Web Server 1.1.4
  • jruby 1.1.6 (ruby 1.8.6 patchlevel 114) (2008-12-17 rev 8388) [x86_64-java] + GlassFish gem version: 0.9.2
  • common backend: mysql5 5.0.75 Source distribution (InnoDB table engine, Rails pool set to 30)

Benchmarks

Continue reading “Benchmarking JRuby on Rails”

Benchmarking JRuby on Rails

Last night, while working on a project I found a really neat use of Rails Components, but I also noticed that this part of Rails is deprecated, among other reasons because it’s slow.

Well, how slow? During my quest to find out, I collected some interesting data, and even more importantly put JRuby and MRI Ruby face to face.

Disclaimer: the benchmarks were not done on a well isolated and specially configured test harness, but I did my best to gather data with informational value. All the components were used with OOB settings.

Setup

  • ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0] + Mongrel Web Server 1.1.4
  • jruby 1.1.6 (ruby 1.8.6 patchlevel 114) (2008-12-17 rev 8388) [x86_64-java] + GlassFish gem version: 0.9.2
  • common backend: mysql5 5.0.75 Source distribution (InnoDB table engine, Rails pool set to 30)

Benchmarks

I used an excellent high quality benchmarking framework Faban for my tests. I was lazy, so I only used fhb (very similar to ab, but without its flaws) to invoke simple benchmarks:

  • simple request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/buckets/1
  • component request benchmark: bin/fhb -r 60/120/5 -c 10 http://localhost:3000/bucket1/object1

Both tests were run with JRuby as well as with RMI Ruby and in addition to that I ran the tests with Rails in single-threaded as well as multi-threaded modes. I didn’t use mongler clusters or glassfish pooled instances – there was always only one Ruby instance serving all the requests.

Results

ruby 1.8.6 + mongrel
---------------------------------
simple action + single-threaded:
ops/sec: 210.900
% errors: 0.0
avg. time: 0.047
max time: 0.382
90th %: 0.095

simple action + multi-threaded:
ops/sec: 226.483
% errors: 0.0
avg. time: 0.044
max time: 0.180
90th %: 0.095

component action + single-threaded:
ops/sec: 132.950
% errors: 0.0
avg. time: 0.075
max time: 0.214
90th %: 0.130

component action + multi-threaded:
ops/sec: 131.775
% errors: 0.0
avg. time: 0.076
max time: 0.279
90th %: 0.125

jruby 1.2.6 + glassfish gem 0.9.2
----------------------------------
simple action + single-threaded:
ops/sec: 141.417
% errors: 0.0
avg. time: 0.070
max time: 0.259
90th %: 0.115

simple action + multi-threaded:
ops/sec: 247.333
% errors: 0.0
avg. time: 0.040
max time: 0.318
90th %: 0.065

component action + single-threaded:
ops/sec: 107.858
% errors: 0.0
avg. time: 0.092
max time: 0.595
90th %: 0.145

component action + multi-threaded:
ops/sec: 179.042
% errors: 0.0
avg. time: 0.055
max time: 0.357
90th %: 0.085
Platform/Action Simple +/- Component +/-
Ruby ST 210ops 0% 132ops 0%
Ruby MT 226ops 7.62% 131ops -0.76%
JRuby ST 141ops -32.86% 107ops -18.94%
JRuby MT 247ops 17.62% 179ops 35.61%

(ST – single-threaded; MT – multi-threaded)

Conclusion

From my tests it appears that MRI is faster in single threaded mode, but JRuby makes up for the loss big time in the multi-threaded tests. It’s also interesting to see that the multi-threaded mode gives MRI(green threads) a performance boost, but it’s nowhere close to the boost that JRuby(native threads) can squeeze out from using multiple threads.

During the tests I noticed that rails was reporting more times spent in the db when using JRuby (2-80ms) compared to MRI (1-3ms). I don’t know how reliable this data is but I wonder if this is the bottleneck that is holding JRuby back in the single threaded mode.

Giles Bowkett – Ruby on Rails Podcast

Giles Bowkett, Ruby Inside Top Presenter of 2008, talks about success, startups, goals, and music.

Sponsor

Giles Bowkett – Ruby on Rails Podcast

Giles Bowkett, Ruby Inside Top Presenter of 2008, talks about success, startups, goals, and music.

Sponsor

What’s New in Edge Rails: HTTP Digest Authentication


This feature is scheduled for: Rails v2.3


Long ago, in your mother’s version of rails, we got a http basic authentication plugin. That functionality has since been rolled into Rails core, but it was always lacking HTTP digest authentication. Until this commit, that is.

For those that may now know the difference, basic authentication only base 64 encodes the authenticating username and password (making it easily decoded) whereas digest authentication sends an MD5 hash of your username and password. To simplify, digest is more secure than basic.

To request digest authentication in Rails, you’ll need to be able to retrieve the cleartext password for a given user (so the framework can hash and compare it using the nonce it created specifically for that request). This commit now allows you to also use a specific hashed format of the password. Here’s how this works if you have access to a cleartext password:

1
2
3
4
5
6
7
8
9
10
11
12
13
class ArticlesController < ApplicationController

  before_filter :digest_authenticate

  def digest_authenticate

    # Given this username, return the cleartext password (or nil if not found)
    authenticate_or_request_with_http_digest("Articles Administration") do |username|
      User.find_by_username(username).try(cleartext_password)
    end
  end

end

Most of us will want to do something with the result of the authentication and can do so with the boolean return value of authenticate_or_request_with_http_digest:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
class ArticlesController < ApplicationController

  before_filter :digest_authenticate

  def digest_authenticate

    success = authenticate_or_request_with_http_digest("Admin") do |username|
      (@user = User.find_by_username(username)).try(cleartext_password)
    end

    # If authentication succeeds, log the user in.  If not, kick back out a failure
    # message as the response body
    if success
      session[:user_id] = @user.id
    else
      request_http_digest_authentication("Admin", "Authentication failed")
    end
  end

end

If you don’t want to store clear text passwords you can return an MD5 hash from the authenticate_or_request_with_http_digest block as long as it’s in the format username:realm:password. You can get a password hash by using Digest::MD5::hexdigest.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class User < ActiveRecord::Base

  attr_accessor :password
  validates_presence_of :username, :crypted_password
  before_save :hash_password

  ...

  def hash_password
    if password_changed?
      self.crypted_password =
        Digest::MD5::hexdigest([username, "UserRealm", password].join(":"))
    end
  end
end

and then in your controller:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
class ArticlesController < ApplicationController

  before_filter :digest_authenticate

  def digest_authenticate

    # Just return the crypted (hashed) version of the password if it's in the supported
    # format.  Note that the realm here "UserRealm" should match the middle
    # argument of your password hash
    success = authenticate_or_request_with_http_digest("UserRealm") do |username|
      (@user = User.find_by_username(username)).try(crypted_password)
    end

    ...
  end

end

So there you have it, digest authentication in edge Rails.

tags: ruby,
rubyonrails



Rolling out new updates for Rails Boxcar

Alex, Director of Deployment Services, has been hard at work helping us get our new suite of hosting plans out for Rails Boxcar, a deployment environment that we’ve designed to help you get your Ruby on Rails applications running as painless and quickly as possible. With this new announcement, we’ve rebuilt the Boxcar image based on the feedback of our existing customers.

Additionally, we’ve been looking over some of early results from the Ruby on Rails Hosting in 2009 Survey that we’ve been running the past few weeks, which has further boosted our confidence that we’re on the right track with this big change.

What are some of the changes?

This means that with a Rails Boxcar, you can now get a pre-configured deployment environment using some of the most efficient platforms for hosting your Ruby on Rails applications. (REE has shown to increase performance by 33% in some cases)

We’re really excited about this new setup and would like to invite you all to check out our new plans and send us any questions that you might have.

The future of FuzzyFinder-TextMate

Back in October I released a Vim extension for mimicking TextMate’s cmd-T file lookup feature. I use it heavily now, and it works great for me.

Sadly, the author of the FuzzyFinder Vim script, upon which my extension depends, keeps changing internal implementation details that I had to hook into to make my extension work. The result? Every few weeks my extension breaks with the latest FuzzyFinder.

Needless to say, this is work I don’t need. The fuzzyfinder-textmate stuff works fine for me. It works fine for people on older versions of FuzzyFinder. And I really don’t care to support this anymore.

If you’re passionate about this, please feel free to fork the project on GitHub and release your changes independently. Feel free to post your changes on the vim script page, even! I hereby release that code into the public domain. Do with it as you please!

Rails Hosting Survey – 5 days left…

Wow. Thanks to all of you who have helped get the word out about the Ruby on Rails Hosting 2009 Survey. We just passed 900 people and we have about five more days left to hit the 1500 milestone that I set for myself.

If you can spare five minutes to help us reach this goal, we’d really appreciate it.

Here is a quick sample of the questions that we’re asking the community.

  • Where is your source code hosted?
  • Which database do you typically use in production?
  • which performance monitoring tool do you use?
  • How much of your monthly budget is allocated for deployment and hosting expenses?
  • So, can Rails scale? 😉

Don’t hesitate… we only have a few days left!


For more information, read the original post, Take the Ruby on Rails Hosting in 2009 Survey.

#146 PayPal Express Checkout

PayPal Express Checkout is easy to add to an existing ordering system. See how in this episode.

#146 PayPal Express Checkout

PayPal Express Checkout is easy to add to an existing ordering system. See how in this episode.

secret codes

Here are some secret codes I am involved with. They are some of the best codes recently coded.

kestrel

A replacement for Starling, the distributed message queue. Written on the JVM (Scala) because of the mature garbage collector. Has a constant performance profile regardless of the size of the queue.

memcached/libmemcached pre-builds

The C library and the Ruby client as a matched pair. Much improved failure handling over the public builds, and Ketama hashing is built-in as always. Switching from ruby-memcache to memcached at Twitter effectively halved our cluster CPU load. (These changes are getting upstreamed, so you can also just wait.)

cache-money

A Rails object cache layer for ActiveRecord. It can do primary key lookups and single index queries solely out of cache. Especially nice if you have replication lag.

peep

A heap inspector for live memcached server instances. They said it couldn’t be done.

thinking sphinx

Pat Allan has been working hard on his Sphinx plugin. He’s adding the missing enterprisey features from Ultrasphinx, so that it can finish its years sleeping gently in legacy apps. (Facets, and non-invasive delta indexing.)

bybusyness

Apache 2.2.10 got a new fair balancing algorithm. Smooths out latencies introduced by single-threaded backends with high standard deviation, such as Mongrel/Rails.

postscripts

Speaking of, I’m making sporadic progress on Mongrel 2, but don’t hold your breath. The main improvement will be Rack support.

A downside of my position at Twitter is meager open-source progress. But hey, we’re hiring (must be in SF or willing to move).

My RSI is slowly going away. The only real solution is to type less. And make sure to keep totally blasting your pecs.

Announcing Golden Gate Ruby Conference

This has been a long time coming, and I’m very happy to finally be able to announce the first ever Golden Gate Ruby Conference, here in San Francisco on April 17 and 18. There’s so much interest in Ruby here in SF, and tons of Ruby mojo too, so it’s about time we had our own conference! We’ll be bringing in some great speakers from all over so locals who don’t get to travel to conferences can get exposed to stuff they can’t usually see. And we’ll also have some of our high-powered local talent showing off their stuff too.

While this has been in the works for a while, there are still a few details to work out before we can talk about things like registration, price, our speaker lineup, etc. But we wanted to get the word out as soon as we had firm dates so people could get it on their calendars. And there’s still a lot we can talk about…

The organizers of the conference are yours truly, Josh Susser (“the has_many :through guy”), and Leah Silber, conference addict and the woman Yehuda Katz was lucky enough to marry. Yehuda is going to be helping me with the technical program too. Our employers, Pivotal Labs and Engine Yard are already committed to sponsor the conference, so we know we have plenty of support to make it happen. There are still a lot of opportunities to get involved as a volunteer or a sponsor, and we’re actively looking for both.

Our approach to this conference is not to take anything as a given. We’re rebels, we’re bucking the system, we’re using instance_eval to leave out explicit receivers! For starters, there will be no call for proposals for talks. The program will consist of invited talks, and talks selected by attendee voting. We’ll be setting up a site soon where talks can be proposed and voted on.

We’ve also got a GitHub account set up as a one-stop place where you can find all the code talked about at the conference, and a place to put stuff we may be hacking on over the weekend. Of course you can follow the conference on twitter, and we’ll have a blog set up soon enough.

We’re shooting for 150-200 people, single track, about a dozen talks, plenty of “hallway track” time, and a few surprises.

If you want to get involved, volunteer or sponsor, the email addresses on gogaruco.com are the best way to contact us.

Looking forward to this. You have no idea…

Do You Test Your Views?

        Those of you <a href="http://www.twitter.com/jeffcohen">following me on twitter</a> already know that I’m learning <a href="http://wiki.github.com/aslakhellesoy/cucumber">Cucumber</a>.


Traditionally I don’t test my views or controllers all that much.  I know, I know, it’s heresy.  But I’ve just never gotten much value from writing Rails functional tests nor integration tests.


Yet those of you who have been reading my articles or <a href="http://www.amazon.com/Rails-NET-Developers-Jeff-Cohen/dp/1934356204/">new book</a> know that I’ve been immersed in <span class="caps">TDD</span> for a long time and can’t imagine writing any application without a decent test suite to go with it.


So what do I test?  I test the models like no one’s business.  I use test/unit or shoulda to drive the design of all the models I write, and I write a lot of tests covering every possible edge case I can think of.


But the controllers and views?  Up until now it’s seemed like a waste of time, for several reasons.


First, <strong>most of my <div class="post-limited-image"><img src="http://feeds.feedburner.com/~f/SoftiesOnRails?i=klWnMdoJ" border="0"></div>

Continue reading “Do You Test Your Views?”

Do You Test Your Views?

Those of you following me on twitter already know that I’m learning Cucumber.

Traditionally I don’t test my views or controllers all that much. I know, I know, it’s heresy. But I’ve just never gotten much value from writing Rails functional tests nor integration tests.

Yet those of you who have been reading my articles or new book know that I’ve been immersed in TDD for a long time and can’t imagine writing any application without a decent test suite to go with it.

So what do I test? I test the models like no one’s business. I use test/unit or shoulda to drive the design of all the models I write, and I write a lot of tests covering every possible edge case I can think of.

But the controllers and views? Up until now it’s seemed like a waste of time, for several reasons.

First, most of my controllers are stupid-simple RESTful controllers. They’re as skinny as possible, because I move almost all of the real logic into the models. That way I can test my models with unit tests without trying to simulate a browser, and I find it easier to use script/console to play with models than with controllers.

Secondly, once the application gets to its first stabilization point, I find that HTML and CSS change way more often than 80% of the core application logic ever will. Writing tests for the views just means they’ll break after inconsequential layout changes. After all, views are just supposed to be a representation of your models (generally speaking), and other stuff (navigation, ajax effects, etc.) just aren’t worth my time writing a regression suite for.

As much as I’m digging Cucumber’s syntax, I’m desparately close to just going back to test/unit and forgetting about the controllers again.

Convince me otherwise. Do you test your controllers and views? If so, with what – Rails functional tests in test/unit? RSpec? Cucumber? What are you trying to verify in your tests, and why?


Is your team switching to Rails? Sign up for a fun-filled, one-day class in your own office: SwitchingToRails.com

Rails Envy Podcast – Episode #064: 01/21/2009

Episode 064. The awkwardness is sometimes palpable.

Subscribe via iTunes – iTunes only link.
Download the podcast ~16:30 mins MP3.
Subscribe to feed via RSS by copying the link to your RSS Reader


Sponsored by New Relic
The Rails Envy podcast is brought to you this week by NewRelic. NewRelic provides RPM which is a plugin for rails that allows you to monitor and quickly diagnose problems with your Rails application in real time. Check them out at NewRelic.com.

Call Us! You can reach the podcast voicemail line at 407 409-8440.

Flex On Rails talk in Denver – Thursday 22nd

Tony and I will be giving a talk on Flex on Rails at Derailed, the Denver Rails User Group. This will take place at Forest Room 5 in LODO at 6:30pm, this coming Thursday 22nd.

On the agenda.

  • 6:30 – 7:00: Meet and mingle
  • 7:00 – 8:30: Flex on Rails – From the authors Daniel Wanja and Tony Hillerson
  • 8:30 – … : Book raffle and spirits.

We will show highlights of each of the chapters and samples from the book.

I hope to see you there!
Daniel

#145 Integrating Active Merchant

In this episode I show how to add Active Merchant’s functionality to a Rails application to make a fully-functional checkout process.

#145 Integrating Active Merchant

In this episode I show how to add Active Merchant’s functionality to a Rails application to make a fully-functional checkout process.

Lessons through failure. Episode 1

I fucked up this last week.

On Monday, our primary contact for a large client sent over some last minute requirements and deadlines that were needed by end-of-day Wednesday. I didn’t have a lot of time to collect requirements and execute it without having to rearrange my priorities. But, I accepted the challenge.

The big change involved was that we were going to be supplied with a ton of data to be imported in to the database and approximately 20% of the data provided was new records, while the rest were duplicates. However, the other 80% wasn’t to be discarded as there were a few attributes that needed to be updated from the data file (which was supplied from the client’s parent company). In my haste to get the task done on time (didn’t get proper export file to be imported in our system until Wednesday morning)… I ended up running a few tests locally and pushed it out to production.

I managed to get the import file to run in production before leaving on Wednesday afternoon. The following morning, I came into the office to find out that my import process didn’t match up records properly and resulted in nearly all of the 80% side of that to be duplicated in the system. This resulted in lost productivity for our client, their vendors, and our team over a 12 hour period as people were confused about why reports were running weird, online transactions didn’t account for the duplicated, etc.

It took me most of Thursday and Friday to clean up the data that got skewed due to that oversight. Hi ho.

So, the take away from this? Sure, I could have blamed it on a lack of sufficient time to properly test things, but that’s bullshit. I should have had at least one other developer from our team review the problem and evaluate my proposed solution prior to me attempting to push into production.

Luckily, the client was happy that we were able to finish the last minute tasks, despite the unexpected headaches that cropped up.

If anything, I was just disappointed in myself, but Alex reminded me how important it was to fail early, fail often. It didn’t kill me (or anybody else for that matter), cost us the project, nor was it irreparable.

In the real world, deadlines and requirements change on a moments notice and it’s experiences like this that will make ourselves more confident that we can quickly respond to and execute.

What was your latest failure?

Jeremy McAnally – Ruby on Rails Podcast

Jeremy McAnally talks about his open source projects, documentation, and productivity.

Sponsor