Pair Programming & git & github & Gravatar & You & You

Or, how Hashrocket displays pair programming gravatars on github.

The Peculiar Problem of Pairs

At Hashrocket, we pair program all the time. We also use git exclusively for source control. This presents a problem: git only supports one “commiter,” but we’d like to give credit to both developers. Here’s how we hacked it.

Author vs. Committer

First, a bit of background: git does support the notion of an “author” as distinct from a “committer.” This feature handles the situation where some external third-party developer has submitted a patch to a source maintainer. The maintainer, as the authoritative and github-credentialed “committer,” can commit the changeset for the external developer, who is set as the “author.” Both the author and the committer are recorded in the commit data.

We did think about setting one pair developer as the author, and the other as the committer. But, then we thought of something we think is even sexier.

Set the Pairs in the Author Fields

Unfortunately, git only supports one “author” as well. But, the author fields do provide us a workable place to hack in our pair attribution solution: we concatenate and store both pair developers usernames in the those author fields. In the BASH shell, that goes a little something like this:

bash$ export GIT_AUTHOR_NAME='Jon Larkowski and Tim Pope'
bash$ export GIT_AUTHOR_EMAIL=dev+l4rk+tpope@example.com

The convention we use for GIT_AUTHOR_EMAIL is the form: dev+pair1+pair2@example.com where “dev” is short for “developer.” dev is just an email alias that goes to every developer at Hashrocket. The pairs are added in alphabetical order by github username. We hack our usernames into the email address using the “plus-notation” feature of email addresses.

Configure the Committer

Committer for a Developer’s Laptop

As for the committer field, it’s just the developer whose machine you’re pairing on. This allows us to tell which user this commit came from, while still indicating the pair authors. Also, it allows us to simply use that developer’s github credentials for pushing up changes. So the git log --pretty=full after a commit looks something like:


commit 3eb7b3a155f11e56cf406f2b09dc2fd98e92b532
Author: Jon Larkowski and Tim Pope <dev+l4rk+tpope@example.com>
Commit: Jon Larkowski <jon@example.com>

   Ebony and ivory, live together in perfect harmony.

Committer for a Shared Pairing Workstation

We do have one twist on the committer. More and more, we’re using communal desktop machines at Hashrocket. We have a couple Macs Pro and a couple Macs Mini (I assure you, those are the proper irregulars plural, look it up). On these shared machines, we name the committer after the machine’s name. This allows us to know which machine a commit came from, but it’s not attached to any particular developer’s name.

bash$ git config --get-regexp user\.*
user.email dev+some_workstation@example.com
user.name Some Workstation

Also, note that the shared development workstation will need to have its own public key registered on github, under your organization’s main github account, not your own personal one.

github & Gravatar Photos

github displays a Gravatar photo next to a username when displaying commit information. This helps add to the “social” feel of github.

Single Developer Gravatar Photo

For you as a lone developer, github just looks up that gravatar based on the email address you gave github when you signed up.

Developer Pair Gravatar Photo

Now for a pair, we need a unique email address that github can use to look up a photo on gravator. This is where our pair developer email address convention comes into play. We just add that dev+pair1+pair2@example.com as a new email address under our main corporate gravatar account, which is maybe signed up under info@example.com for instance.

Gravator lets you have as many email addresses as you one under one main umbrella account. It’s a bit spammy, in that every developer will receive a gravatar confirmation email whenever a new pair permutation signs up. But, that sounds like a “First World problem,” if I ever heard one.

Say, “Cheese!”

We typically just snap our pair photos using Photo Booth.app, since we’re likely pairing at that very moment on a Mac with a built-in iSight camera. Oh, and since we’re picky, we make sure that pair1 is on the left and pair2 is on the right of the photo.

Hitch

Setting git environment variables by hand every time you switch pairs could get tedious. Ro and Les wrote a little gem called “Hitch” that makes this easy.

Install it with:

sudo gem install therubymug-hitch

To set it up, follow the prompts and answer the questions:

hitchrc

When you start pairing with someone, just call:

hitch your_pairs_github_username

To code solo:

unhitch

To pick from the list of your previous pairs:

hitch -i

Just running hitch, with no arguments, will tell you who you are pairing with. Pro tip: Add it to your bash profile to see it when you start a shell.

Bonus, can hitch/unhitch from within vim, i.e.

:!hitch github_username

Hardcore BASH Scripting Action

Here’s a BASH script that accomplishes the same thing as hitch.

To set up a pair:

bash$ pair fry bender

Bender Bending Rodriguez and Philip J. Fry

To enter a pair that’s not a declared variable at the top of the script:

bash$ pair fry hypnotoad="All Hail the Hypnotoad"

Philip J. Fry and All Hail the Hypnotoad

To persist a pairing, even between shell invocations:

bash$ pair -w zoidberg bender

Bender Bending Rodriguez and Dr. John Zoidberg

Use pair with no arguments to see who’s paired. Use pair -w with no arguments to clear a persistent pairing.

RightSignature Featured on TechCrunch

grizzly-sendfile to Become an Official Grizzly Module

After a chat with JFA about grizzly-sendfile‘s future, I’m pleased to announce today that grizzly-sendfile 0.4 will be the first version of grizzly-sendfile released as an official module of grizzly. This is a huge news for grizzly-sendfile and I believe an equally important news for grizzly and its community.

What this “merger” means for grizzly-sendfile:

  • great opportunity to extend the reach
  • opportunity to become the default static file handler in Grizzly
  • aspiration to become the default static file handler in GlassFish v3
  • more testing and QA
  • easier and faster access to grizzly developers and contributors

What this “merger” means for grizzly:

  • contribution of 1 year of research, development and testing time in the area of static http downloads
  • several times better performance and scalability of http static file downloads
  • built-in X-Sendfile functionality
  • better JMX instrumentation for http downloads
  • and more

If you can’t wait for 0.4, go

Continue reading “grizzly-sendfile to Become an Official Grizzly Module”

grizzly-sendfile to Become an Official Grizzly Module

After a chat with JFA about grizzly-sendfile‘s future, I’m pleased to announce today that grizzly-sendfile 0.4 will be the first version of grizzly-sendfile released as an official module of grizzly. This is a huge news for grizzly-sendfile and I believe an equally important news for grizzly and its community.

What this “merger” means for grizzly-sendfile:

  • great opportunity to extend the reach
  • opportunity to become the default static file handler in Grizzly
  • aspiration to become the default static file handler in GlassFish v3
  • more testing and QA
  • easier and faster access to grizzly developers and contributors

What this “merger” means for grizzly:

  • contribution of 1 year of research, development and testing time in the area of static http downloads
  • several times better performance and scalability of http static file downloads
  • built-in X-Sendfile functionality
  • better JMX instrumentation for http downloads
  • and more

If you can’t wait for 0.4, go and get recently released version 0.3.

This is a great day for both projects! 🙂

Project site: http://grizzly-sendfile.kenai.com/

Merging Hash and HashWithIndifferentAccess

Ran into a gotcha the other day I thought I’d share. It came about in some code that was trying to give some default values in a hash, then merge in some params before creating an object. It looked something like this:

   Thing.new({:is_cool => true}.merge(params[:thing])
   

The gotcha here is that params is a HashWithIndifferentAccess, but {:is_cool => true} is not. Merging these two will actually drop the return value to just Hash. Observe.

   >> params = HashWithIndifferentAccess.new(:thing => {:is_cool => false, :name => 'Cool Thing'})
   => {"thing"=>{"name"=>"Cool Thing", "is_cool"=>false}}
   >> params.class
   => HashWithIndifferentAccess
   >> ({:is_class => true}.merge(params[:thing])).class
   => Hash
   

When you look at what is created when merging the two, you notice that there are actually two separate keys of the same name, one a symbol, one a string.

    >> {:is_cool => true}.merge(params[:thing])
    => {"name"=>"Cool Thing", :is_cool=>true, "is_cool"=>false}
   

And then when you pass this hash to your ActiveRecord model, you can see that it favors the symbol.

   >> thing = Thing.new({:is_cool => true}.merge(params[:thing]))
   => #<Thing id: nil, name: "Cool Thing", is_cool: true, created_at: nil, updated_at: nil>
   

The solution is to either use a string as your key:

   >> thing = Thing.new({'is_cool' => true}.merge(params[:thing]))
   => #<Thing id: nil, name: "Cool Thing", is_cool: false, created_at: nil, updated_at: nil>
   

Or make the initial hash a HashWithIndifferentAccess:

   >> thing = Thing.new(HashWithIndifferentAccess.new(:is_cool => true).merge(params[:thing]))
   => #<Thing id: nil, name: "Cool Thing", is_cool: false, created_at: nil, updated_at: nil>
   

So yea, keep an eye out for this gremlin.

Agile sighting in LAX

Well…sort of an agile sighting…I mean, not really but sort of 😉

Scrum_shirt

Copy S3 assets with right_aws

Lately, I’ve been using right_aws to interact with S3. One thing that I found helpful was copying assets between buckets and keeping the same permissions on them. However, it’s not as simple as just copying the assets over. You need to get the Access Control Policy from the source and put it in the copied asset.

Here’s a snippet of code that does the magic.

require 'rubygems' 
require 'right_aws' 

s3=RightAws::S3Interface.new(S3_KEY, S3_SECRET)
s3.copy(SOURCE_BUCKET, SOURCE_PATH, DESTINATION_BUCKET, DESTINATION_PATH, :copy, {"Cache-Control" => 'max-age=315360000', "Expires" => '315360000'})
acl_prop=s3.get_acl(SOURCE_BUCKET, SOURCE_PATH)
s3.put_acl(DESTINATION_BUCKET, DESTINATION, acl_prop[:object])

I Don’t Use RSpec

        It seems I’m in a super-minority among Rails developers.


First of all, I write tests.  My unscientific guess is that less than half of people who call themselves Rails developers actually write tests.


Second, I write tests first.  Of those that write tests, I would guess that less than half write them before writing the implementation code.


So already I’m in the group of “those crazy <span class="caps">TDD</span> developers”.  But you know what’s even worse?


In that very group of “crazy <span class="caps">TDD</span> developers”, I bet more than half use <a href="http://rspec.info/">RSpec</a>.


But I don’t.  Instead, I use a combination of <a href="http://www.thoughtbot.com/projects/shoulda/">shoulda</a>, <a href="http://github.com/brynary/webrat/tree/master">webrat</a>, and <a href="http://mocha.rubyforge.org/">mocha</a>.


Honestly I don’t know why I’ve never liked RSpec.  Lots of smart people use it, and maybe I just don’t understand it.  But I don’t think business users will write tests with the story runner.  The most they’ll ever do is <em>read</em> them, which is still <div class="post-limited-image"><img src="http://feeds.feedburner.com/~ff/SoftiesOnRails?i=wbjtrGw7aSM:eAdfFW8Y8aI:V_sGLiPBpWU" border="0"></div>

Continue reading “I Don’t Use RSpec”

I Don’t Use RSpec

It seems I’m in a super-minority among Rails developers.

First of all, I write tests. My unscientific guess is that less than half of people who call themselves Rails developers actually write tests.

Second, I write tests first. Of those that write tests, I would guess that less than half write them before writing the implementation code.

So already I’m in the group of “those crazy TDD developers”. But you know what’s even worse?

In that very group of “crazy TDD developers”, I bet more than half use RSpec.

But I don’t. Instead, I use a combination of shoulda, webrat, and mocha.

Honestly I don’t know why I’ve never liked RSpec. Lots of smart people use it, and maybe I just don’t understand it. But I don’t think business users will write tests with the story runner. The most they’ll ever do is read them, which is still valuable. But they can read my webrat tests well enough.

Also, RSpec just feels… heavy. There are spaces between words when I expect ruby-like underscores instead:

Model.should have(1).record

This just looks weird to me.

Unfortunately learning Mocha hasn’t been like falling off a log, either. There’s an excellent Google Group, but I don’t really know of a simple “Getting Started with Mocha” guide anywhere, so it’s taken me a while to get up to speed. Maybe I should take the time to write one? If enough people are interested I’d be willing to give it a shot.

So, there it is. I don’t use RSpec. So flame away and tell me know if I’m a genius or a moron.

Add-on: Wildcard Domains

Since we returned from a fun and successful Railsconf in Vegas, we have been in full swing completing the rollout of our paid services. The response has been enormous so far, and paid services are now available to all users.

If you’ve checked out the pricing page, you’ve undoubtedly noticed our line-up of a la carte add-ons. We’re really excited about add-ons becoming a key part of our platform, allowing us to seamlessly deliver popular application services and components with the built-in scalability and ease of use you’ve come to expect from Heroku.

We’ve had a solid first batch of add-ons in beta for a while, and today we’re happy to announce the first graduate: Wildcard Domains, which allows your app to respond on *.yourdomain.com. Since we started supporting custom domains on Heroku this has been high on the list of requested features, and now it’s here! To add it to your app for $5/month, simply go to your app’s Resources page and select turn it on. Enjoy!

Ruby 1.9 isn’t always faster: rake tab-completion revisited

Rake is awesome in many ways and we’re all using it for a plethora of tasks. It’s easy to use, fast to code and reliable.

A while back I blogged about tab completion for rake tasks and the other day I set out to speed it up, as the project I was working on had a huge amount of tasks and retrieving the list of tasks took a loooong while.

As it turns out, speeding up the task enumeration is not a trivial task. My first thought was to ParseTree but it was horrendously slow and quite complex. Then I thought I’d use a fast language, such as OCaml or perhaps even C to scan the files and become known worldwide for my wicked coding skillz.

Didn’t happen. As I’m sure you’ve already figured out, the rake library allows you to define raketasks in many different ways and people out there all seem to have different opinions on how to code up their tasks. Several important libraries define the rake tasks dynamically, ruling out any pattern matching approaches.

In other words I failed. But I also learned a lot and thanks to ruby-talk I also found a trick to make things spiffier: Ruby 1.9 isn’t always faster, Rake::Application is a singleton and String#hash isn’t what it used to be.

Read on for details.

Rake tab completion

The first step to setup rake tab completion is to tell bash to invoke your script when ‘rake’ is the command. Drop this in your ~/.bashrc:

  complete -C ~/bin/rake_completion.rb -o default rake

The next step is the ruby executable script that actually looks up the rake tasks. The script:

  #!/usr/bin/env ruby
 
def rake_silent_tasks
  dotcache = "#{File.join(File.expand_path('~'), ".raketabs-#{Dir.pwd.hash}")}"
  if File.exists?(dotcache)
    Marshal.load(File.read(dotcache))
  else
    require 'rubygems'
    require 'rake'
    load 'Rakefile'
    tasks = Rake.application.tasks.map(&:name)
    File.open(dotcache, 'w') { |f| f.puts Marshal.dump(tasks) }
    tasks || []
  end
end
 
exit 0 unless File.file?(File.join(Dir.pwd, 'Rakefile'))
exit 0 unless matches = ENV["COMP_LINE"].match(/^rake\s+(.*)/)
 
after_match = matches[1]
task_match = after_match.strip.empty? ? nil : after_match.strip.split.last
 
if task_match
  tasks = rake_silent_tasks.grep /^#{Regexp.escape task_match}/
 
  # handle namespaces
  if matches = task_match.match(/^([-\w:]+:)(.*)/)
    upto_last_colon = matches[1]
    after_match = matches[2]
    tasks = tasks.map { |t| (t =~ /^#{Regexp.escape upto_last_colon}([-\w:]+)$/) ? "#{$1}" : t }
  end
  puts tasks
end
 
exit 0  

We need to make this executable with:

  chmod +x ~/bin/rake_completion.rb

Let’s go through the code. First comes the ‘rake_silent_tasks’ top-level method. More on that later.
Next a couple of checks to ensure we’re somewhere where it makes sense to run ‘rake’ (i.e. a directory with a ‘Rakefile’ present). Then we check the $COMP_LINE environment variable. This is set by bash and we just need to match it against the string ‘rake’ at the beginning of the command line. Easy.

The remaining code is just your average regexp juggling to match the string you enter, e.g. ‘rake db:mi[TAB]’.

What is interesting here is all in the ‘rake_silent_tasks’ method. In order for lookups to be fast we really want to avoid invoking rake and parse the Rakefile (and all the .task files that are commonly present).

The code first checks if we’ve been running rake+TAB in this directory before and if so, loads an unmarshal the tasks into a Ruby Array. Using Marshal.load/.dump is a big speedup in itself.

If we need to lookup all the rake tasks from scratch, a full parse of all task defining ruby files is necessary. Previous versions of this code used a backtick invocation of rake like so:

  `rake --silent --tasks`

That ouputs the full list of tasks and their descriptions and is quite slow.

To bypass the necessity for backticks I read through the source code for rake and asked ruby-talk to help out and sure enough, there is a way to get around the necessity to spin off a whole new ruby process, and the following returns a list of tasks as an Array:

  require 'rake'
  load 'Rakefile'
  Rake.application.tasks

Pretty straightforward, uh? Sure, after the fact it’s obvious, but I spent a long while trying to instantiate a new Rake::Application object until told that rake is implemented as a singleton. This snippet will crash so badly that IRB dies and throws you back to the shell:

  require 'rake'
  a = Rake::Application.new
  a.init

(maybe it’s not even a crash, just plain wrong usage but it’s really hard to tell and can be quite confusing…)

Anyways, retrieving the rake tasks this way is a lot faster. How much?

  100 rake task enumerations for "/Users/david/projects/big_rails_app"
                    user     system      total        real
  backticks     0.160000   0.330000  59.680000 ( 61.323611)
  rake direct   0.590000   0.070000   0.660000 (  0.676324)

Two orders of magnitude! Yay!

Interestingly, Ruby 1.9 is quite a bit slower than good’ol 1.8:

    100 rake task enumerations for "/Users/david/projects/big_rails_app"
                      user     system      total        real
    backticks     0.230000   0.390000  59.940000 ( 61.596335)
    rake direct   0.960000   0.110000   1.070000 (  1.089892)
  

While I was investigating the various options to speed up tab completion I first of all grabbed for Ruby 1.9. After a while I discovered that the implementation above never hits the cache and the reason for that is a subtle change in the way the little used String#hash method works.

In 1.8 the hash for a given String is always the same, while in 1.9 the implementation changed to use MurmurHash and now the hash is identical only within the same ruby process. This change is not documented anywhere to my knowledge and is a deal breaker for ruby scripts used as command line executables (and maybe elsewhere as well).

I tried to replace the ‘Dir.pwd.hash’ snippet above with ‘Digest::MD5.hexdigest(Dir.pwd)’ so it would work under 1.9, but it’s a lot slower than Sting#hash, and we’re aiming for speed here, so…

All in all I’m happy with my investigation. As said, learned a lot and in the end I gained a fast-ish rake task tabber script. Many thanks go to Sebastian Hungerecker, Robert Klemme, Ryan Davis and of course to the original implementors of the rake tab completion script.

If you see any other means of speeding this up, please leave a comment! 🙂

Update: fixed escapes in code, as per commenters request (1 Jun 2009)

Grails, Clojure, New Android Updates

Grails: A Quick Start Guide is now available in beta and Programming Clojure is now in print and shipping. Updating Hello, Android for 1.5 Cupcake.

Rails Envy Podcast – Episode #080: 05/27/2009

Episode 80. Gregg sneezes on my dog in this episode. How messed up is that?

Subscribe via iTunes – iTunes only link.
Download the podcast ~22:00 mins MP3.
Subscribe to feed via RSS by copying the link to your RSS Reader


Want to Rejuvenate your legacy Rails application? Call Mocra as your first-choice team, led by Dr Nic. Mocra loves code, loves users, and they want to help you. Mention this advert to qualify for our largest discount consulting rate immediately. Check out Mocra.com for more info.

Sponsored by New Relic
NewRelic provides RPM which is a plugin for rails that allows you to monitor and quickly diagnose problems with your Rails application in real time. They also recently produced Rails Lab, which gives you expert advice on tuning and optimizing your Rails app.

Show Notes

Rails Envy Podcast – Episode #080: 05/27/2009

Episode 80. Gregg sneezes on my dog in this episode. How messed up is that?

Subscribe via iTunes – iTunes only link.
Download the podcast ~22:00 mins MP3.
Subscribe to feed via RSS by copying the link to your RSS Reader


Want to Rejuvenate your legacy Rails application? Call Mocra as your first-choice team, led by Dr Nic. Mocra loves code, loves users, and they want to help you. Mention this advert to qualify for our largest discount consulting rate immediately. Check out Mocra.com for more info.

Sponsored by New Relic
NewRelic provides RPM which is a plugin for rails that allows you to monitor and quickly diagnose problems with your Rails application in real time. They also recently produced Rails Lab, which gives you expert advice on tuning and optimizing your Rails app.

Show Notes

<style>.post ul { margin:0; text-indent: 0; padding: 0; } .post ul ul li { margin-left: 10px; } </style>

First RubyConf.China

Rubyconfchina

The first Ruby conference in Chinae was held at May 21, 2009. I have had the honor to be the host of the event, and it turned out better than I expected! Not only was Ruby’s Father, Matz, delivered an informative and educational speech, other illustrious developers such as @robbinfan, @rubbinlu, and YeZheng have also contributed their wealth of knowledge to the conference.

The conference was sponsored by JavaEye.com (the second largest tech community in China) and ShangHaionRails.org (a local ruby&rails organization located in ShangHai.) The conference room was originally booked to seat 200 people, and over 400 have attended this event. The event was a great success and you can check out more photos and videos on rubyconfchina.org and links below.

Photos

Videos

  • Matz — Why Ruby?

  • Design Pattern from java to ruby

  • Deep secret in JavaEye

  • Ruby and Rails Pitfall

  • Ruby/Rails for Enterprise Development

  • Open Source & Scrum

  • Blue Ruby* A Ruby VM for the ABAP Web Application Server

    Version 005 Now Available

    This time around we have the following:

    • Long chapter on RSpec
    • Messed with the cover some more
    • Many typos fixed, courtesy of Dana Jones

    The RSpec section turned out to be the longest in the book by far, and kind of crowded out anything else I was hoping to get in this iteration.

    Big thanks to Dana for catching a lot of typos and places where the text was unclear.

    Next time around, it’s looking good for Webrat, among other things.

    Update via http://www.railsprescriptions.com/products.

    Thanks.

    The missing first words from The Passionate Programmer

    While I was working on The Passionate Programmer I had the following Kurt Vonnegut quote (from A Man Without A Country) at the top of the introduction. Right before we published it, my editor reminded me that we had to get permission to print it. I was unable to reach anyone from Kurt Vonnegut’s estate on time, so I commented the quote out in the book’s source.

    This is how The Passionate Programmer was supposed to begin:

    "I urge you to please notice when you are happy, and exclaim or murmur or think at some point, 'If this isn't nice, I don't know what is.'" -- Kurt Vonnegut, Jr.

    #163 Self-Referential Association

    Creating a social networking site often requires a self-referential association on the User model to define friends/followers. In this episode I show how to do exactly that.

    #163 Self-Referential Association

    Creating a social networking site often requires a self-referential association on the User model to define friends/followers. In this episode I show how to do exactly that.

    Corrupt Attachments with AR_Mailer

    AR Mailer and Corrupt Truncated Attachments

    After changing our system to send out emails via ar_mailer, the attachments we were sending out were corrupted.

    The “mail” column of the ar_mailer schema is a TEXT column, but that won’t work for most attachments, thus, we simply bumped it up to a LONGTEXT (we’re using mysql on this project), and everything started working perfectly again.

    class EmailsMailTextToLongtext <  ActiveRecord::Migration                                                                                                                                           
      def self.up                                                                                                                                                                                      
        execute "ALTER TABLE emails CHANGE mail mail LONGTEXT"                                                                                                                                         
      end                                                                                                                                                                                              
                                                                                                                                                                                                       
      def self.down                                                                                                                                                                                    
        execute "ALTER TABLE emails CHANGE mail mail TEXT"                                                                                                                                             
      end                                                                                                                                                                                              
    end