Bundler: As Simple as What You Did Before

One thing we hear a lot by people who start to use bundler is that the workflow is more complicated than it used to be when they first start. Here’s one (anonymized) example: “Trying out Bundler to package my gems. Still of the opinion its over-complicating a relatively simple concept, I just want to install gems.”

Bundler has a lot of advanced features, and it’s definitely possible to model fairly complex workflows. However, we designed the simple case to be extremely simple, and to usually be even less work than what you did before. The problem often comes when trying to handle a slightly off-the-path problem, and using a much more complex solution than you need to. This can make *everything* much more complicated than it needs to be.

In this post, I’ll walk through the bundler happy path, and show some design decisions we made to keep things moving as smoothly as before, but with far fewer snags and problems than the approach you were using before. I should be clear that there are probably bugs in some cases when using a number of advanced features together, and we should fix those bugs. However, they don’t reflect core design decisions of bundler.

In the Beginning

When you create a Rails application for the first time, the Rails installer creates a Gemfile for you. You’ll note that you don’t actually have to run bundle install before starting the Rails server.

$ gem install rails
Successfully installed activesupport-3.0.0
Successfully installed builder-2.1.2
Successfully installed i18n-0.4.1
Successfully installed activemodel-3.0.0
Successfully installed rack-1.2.1
Successfully installed rack-test-0.5.6
Successfully installed rack-mount-0.6.13
Successfully installed tzinfo-0.3.23
Successfully installed abstract-1.0.0
Successfully installed erubis-2.6.6
Successfully installed actionpack-3.0.0
Successfully installed arel-1.0.1
Successfully installed activerecord-3.0.0
Successfully installed activeresource-3.0.0
Successfully installed mime-types-1.16
Successfully installed polyglot-0.3.1
Successfully installed treetop-1.4.8
Successfully installed mail-2.2.6.1
Successfully installed actionmailer-3.0.0
Successfully installed thor-0.14.2
Successfully installed railties-3.0.0
Successfully installed rails-3.0.0
22 gems installed
$ rails s
=> Booting WEBrick
=> Rails 3.0.0 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
[2010-09-30 12:25:51] INFO  WEBrick 1.3.1
[2010-09-30 12:25:51] INFO  ruby 1.9.2 (2010-08-18) [x86_64-darwin10.4.0]
[2010-09-30 12:25:51] INFO  WEBrick::HTTPServer#start: pid=26380 port=3000

If you take a look at your directory, you’ll see that Bundler noticed that you didn’t yet have a Gemfile.lock, saw that you had all the gems that you needed already in your system, and created a Gemfile.lock for you. If you didn’t have a necessary gem, the Rails server would error out. For instance, if you were missing Erubis, you’d get Could not find RubyGem erubis (~> 2.6.6). In this case, you’d run bundle install to install the gems.

As you can see, while bundler is providing some added features for you. If you take your application to a new development or production machine with the Gemfile.lock, you will always use exactly the same gems. Additionally, bundler will resolve all your dependencies at once, completely eliminating the problem exemplified by this thread on the Rails core mailing list. But you didn’t have to run bundle install or any bundle command unless you were missing some needed gems. This makes the core experience of bundler the same as “I just want to install gems”, while adding some new benefits.

Adding Gems

When you add a gem that is already installed on your system, you just add it to the Gemfile and start the server. Bundler will automatically pick it up and add it to the Gemfile.lock. Here again, you don’t need to run bundle install. To install the gem you added, you can run bundle install or you can even just use gem install to install the missing gems.

Once again, while bundler is handling a lot for you behind the scenes (ensuring compatibility of gems, tracking dependencies across machines), if you have all the gems that you need (and specified in your Gemfile) already installed on your system, no additional commands are required. If you don’t, a simple bundle install will get you up to date.

Deployment

After developing your application, you will probably need to deploy it. With bundler, if you are deploying your application with capistrano, you just add require "bundler/capistrano" to your deploy.rb. This will automatically install the gems in your Gemfile, using deployment-friendly settings. That’s it!

Before bundler, you had two options:

  • Most commonly, “make sure that all the gems I need are on the remote system”. This could involve including the gem install in the capistrano task, or even sshing into the remote system to install the needed gems. This solution would work, but would often leave no trail about what exactly happened. This was especially problematic for applications with a lot of dependencies, or applications with sporadic maintenance
  • Using the GemInstaller gem, which allowed you to list the gems you were using, and then ran gem install on each of the gems. Heroku used a similar approach, with a .gems manifest. While this solved the problem of top-level dependencies, it did not lock in dependencies of dependencies. This caused extremely common problems with rack and activesupport, and a long tail of other issues. One particularly egregious example is DreamHost, which directs users to use a brittle patch to Rails

In both of these cases, there was no guarantee that the gems you were using in development (including dependencies of dependencies like Rack and Active Support) remained the same after a deploy. One recent example that I saw was that carrierwave added an updated dependency on the newest activesupport, which depends on Ruby 1.8.7. Without making any change to his .gems manifest, Heroku failed to deploy his application on their Ruby 1.8.6 stack.

With bundler, this sort of problem can never happen, because you’re always running exactly the same versions of third-party code in production as you are in development and staging. This should increase your confidence in deployments, especially when you come back to an application months after it was originally developed.

Tweaked Gems

Some times, you will need to make small changes to one of your dependencies, and deploy your application with those changes. Of course, in some cases you can make the tweaks via monkey-patching, but in other cases, the only real solution is to change the gem itself.

With bundler, you can simply fork the gem on Github, make your changes, and change the gem line in your Gemfile to include a reference to the git repository, like this:

# change this
gem "devise"
 
# to this
gem "devise", :git => "git://github.com/plataformatec/devise.git"

This behaves exactly the same as a gem version, but uses your tweaks from Github. It participates in dependency resolution just like a regular gem, and can be installed with bundle install just like a regular gem. Once you’ve added a git repository, you run bundle install to install it (since the normal gem command can’t install git repositories).

You can use :branch, :tag, or :ref flags to specify a particular branch, tag or revision. By default, :git gems use the master branch. If you use master or a :branch, you can update the gem by running bundle update gem_name. This will check out the latest revision of the branch in question.

Conclusion

Using bundler for the sorts of things that you would have handled manually before should be easier than before. Bundler will handle almost all of the automatable record-keeping for you. While doing so, it offers the incredible guarantee that you will always run the same versions of third-party code.

There are a lot of advanced features of Bundler, and you can learn a lot more about them at the bundler web site.

RPCFN: Japanese Mosaic Problem (#14)

Ruby Programming Challenge For Newbies

RPCFN: Japanese Mosaic Problem (#14)

By Joseph Wilk

About Joseph Wilk

Joseph Wilk Joseph Wilk is a member of the core development team for Cucumber. He has been developing for the web for over 10 years in both big and small companies and as an entrepreneur. After stints working with Java and Python he finally found Ruby. He now spends his time in-between eating Cucumbers working at Songkick.com. Having more fun than is healthy working as a Software Gardener building web systems, working on open source projects and giving talks around the world about testing. He suffers from test obsession and has given up hope of any treatment.

Joseph Wilk has this to say about the challenge:

The Japanese Mosaic problem is a logic puzzle based on a grid with the cells potentially containing numbers ranging 0 to 9. The numbers reflect for a cell how many of its neighbours and itself are shaded in. The fun aspect of these problems is once solved you end up with some pretty ASCII art. You’re challenge is to write an algorithm to solve any mosaic puzzle. The challenge provides the opportunity to explore data structures to represent the problem space and looking at how we can navigate the structure in search of a solution. There are lots of opportunities to also think about the efficiency in searching for a solution. A Cucumber feature has already been written for you with lots of scenarios exploring through examples the rules of the Japanese Mosaic problem. If you are not familiar with Cucumber checkout http://cukes.info/. This Cucumber feature is both your specification and your executable tests. You’re mission is to get all the Scenarios passing. There are many ways to solve the problem and I look forward to seeing how people tackle these in the different languages. Good luck and safe Cuking.

Prizes

  • The participant with the best Ruby solution (if there is a tie between answers, then the one who posted first will be the winner) will be awarded any one of PeepCode’s Ruby on Rails screencasts.
  • From the remaining working Ruby solutions, three participants would be selected randomly and each one would be awarded any one of Pragmatic’s The Ruby Object Model and Metaprogramming screencasts.

The four persons who win, can’t win again in the next immediate challenge but can still participate.

The Ruby Challenge

RPCFN

The Challenge

All the instructions and the code to get you started are in a Github repository: http://github.com/josephwilk/japanese-mosaic-logic-puzzle

How to Enter the Challenge

Read the Challenge Rules. By participating in this challenge, you agree to be bound by these Challenge Rules. It’s free and registration is optional. You can enter the challenge just by posting the following as a comment to this blog post:

  1. Your name:
  2. Country of Residence:
  3. GIST URL of your Solution (i.e. Ruby code) with explanation and / or test cases:
  4. Code works with Ruby 1.8 / 1.9 / Both:
  5. Email address (will not be published):
  6. Brief description of what you do (will not be published):

Note:

  • As soon as we receive your GIST URL, we will fork your submission. This means that your solution is frozen and accepted. Please be sure that is the solution you want, as it is now recorded in time and is the version that will be evaluated.
  • All solutions posted would be hidden to allow participants to come up with their own solutions.
  • You should post your entries before midnight of 31st Oct. 2010 (Indian Standard Time). No new solutions will be accepted from 1st Nov. onwards.
  • On 1st Nov. 2010 all the solutions will be thrown open for everyone to see and comment upon.
  • The winning entries will be announced on this blog in the first week of Nov. 2010. The winners will be sent their prizes by email.

More details on the RPCFN?

Please refer to the RPCFN FAQ for answers to the following questions:

Donations

RPCFN is entirely financed by RubyLearning and sometimes sponsors, so if you enjoy solving Ruby problems and would like to give something back by helping with the running costs then any donations are gratefully received.

Click here to lend your support to: Support RubyLearning With Some Love and make a donation at www.pledgie.com !

Acknowledgements

Special thanks to:

  • Joseph Wilk.
  • GitHub, for giving us access to a private repository on GitHub to store all the submitted solutions.
  • The RubyLearning team.

Questions?

Contact Satish Talim at satish [dot] talim [at] gmail.com OR if you have any doubts / questions about the challenge (the current problem statement), please post them as comments to this post and the author will reply asap.

The Participants

There are two categories of participants. Some are vying for the prizes and some are participating for the fun of it.

In the competition

Just for Fun

Previous Challenge

RPCFN: Economics 101 (#13) by Dr. Bruce Scharlau.

Note: All the previous challenges, sponsors and winners can be seen on the Ruby Programming Challenge for Newbies page.

Update

  • The (#15) challenge by Julio Javier Cicchelli, Netherlands is scheduled for Nov. 2010.

Technorati Tags: , , , ,

An introduction to eventmachine, and how to avoid callback spaghetti

An introduction to eventmachine, and how to avoid callback spaghetti

This guest post is contributed by Martyn Loughran, who works at New Bamboo in London where he builds some very cool apps like Pusher. Pusher is a web service which finally makes web push easy using the brand new WebSocket protocol. When he’s not hacking away at the next great thing, he’s most likely to be found behind a cello; hacking in this context is less desirable. Alternatively find him on twitter as @mloughran where he tweets, reluctantly.

Martyn Loughran Evented programming has become cool lately, largely due to the very elegant node.js project. However, we’ve been evented in the Ruby world for many years thanks to eventmachine, which adds evented IO to Ruby. Writing evented code is often viewed as ‘hard’ or ‘back to front’, but it can actually be very elegant. You just need a few tricks up your sleeve.

One of the big challenges is understanding how to create clean abstractions. Since the structure of the code is so different to what you’re probably used to, you can quickly find yourself tied up in callback spaghetti. In this blog post I will explain some common patterns you can use while we combine the Twitter streaming API, Google’s language API, and a WebSocket server. No spaghetti is required, promise!

After getting started with eventmachine, we’ll discuss two common abstractions. Firstly deferrable objects which are like method calls which return asynchrnously. Secondly, how to abstract code that can trigger multiple events. Finally, we’ll add a WebSocket server into the same process to show off doing IO concurrently.

Getting started with eventmachine

Let’s start by installing eventmachine and checking that the basics work:

gem install eventmachine

You can run this ruby test1.rb as you normally would, but you’ll need to kill it when you’ve had enough.

# test1.rb

require 'rubygems'
require 'eventmachine'

EventMachine.run {
  EventMachine.add_periodic_timer(1) {
    puts "Hello world"
  }
}

With a bit of luck you should get a cheery message every second.

So how does this work? After requiring eventmachine, we call EventMachine.run, passing a block. The best advice for now is to completely ignore this and just focus on what’s inside the block, eventmachine can’t work without it. (If you’re curious, read up on the reactor pattern.) Inside the EventMachine.run block we call add_periodic_timer, passing another block. This tells eventmachine to fire an event every 1s, at which point it should call the block. This is the first of many interesting blocks. It’s known a callback.

You’re probably thinking you could have done this much easier with loop { puts 'hi'; sleep 1 }, and you’d be right! It’s gets better, I promise.

Using eventmachine for network IO

Efficient IO is the whole point of eventmachine, but you need to understand one thing. When you’re doing any kind of network IO with eventmachine, you must use eventmachine, or a library which uses eventmachine under the hood (you can usually spot them on github because they start with em-). You therefore need to be very careful when for example picking gems which talk to databases, apis, etc.

If you don’t do this you’ll block the reactor, which basically means that eventmachine will not be able to trigger any more events until the IO operation completes. So for example, if you used Net::HTTP from the standard library to request a URL which took 10s to return, the periodic timer you added earlier wouldn’t fire till it finished. You’ve thrown concurrency out of the window!

So let’s talk about the HTTP client. Although eventmachine actually comes with two different HTTP clients, both have issues, and it’s generally recommended to ignore them and install the very capable em-http-request instead

gem install em-http-request

Let’s make a quick http request to check that it works (note that EM is an alias for EventMachine, and by corollary I dislike typing):

require 'rubygems'
require 'eventmachine'

EM.run {
  require 'em-http'

  EM::HttpRequest.new('http://json-time.appspot.com/time.json').get.callback { |http|
    puts http.response
  }
}

Again we’re attaching a callback which is called once the request has completed. We’re attaching it in a slightly different way to the timer above, which we’ll discuss next.

Abstracting code that has a success or failure case

When designing APIs it’s extremely common to need to differentiate between successful and failure responses. In Ruby, two common ways to do this are to return nil, or to raise an exception (ActiveRecord::NotFound for example). Eventmachine provides quite an elegant abstraction for this: the deferrable.

A deferrable is an object to which you may attach a success and a failure callback, slightly confusingly named callback and errback respectively. If you’re so inclined you might like to look at the code, since there is not much to it.

The call to HttpRequest#get we looked at earlier actually returns a deferrable (in fact it returns an instance of EM::HttpClient which mixes in the EM::Deferrable module). It is also quite common to make use of the EM::DefaultDeferrable which you can use standalone.

As an excuse to use a deferrable ourselves I’ve decided that it would be a jolly marvelous idea to look up the language of tweets as they arrive from the twitter streaming API.

Handily, the Google AJAX Language API allows you to get the language of any piece of text. It’s designed to be used from the browser, but we won’t let a small matter like that stop us for now (although you should if it’s a real application).

When I’m trying out a new API I generally start with HTTParty (gem install httparty) since it’s just so quick and easy to use. Warning: you can’t use HTTParty with eventmachine since it uses Net::HTTP under the covers – this is just for testing!

require 'rubygems'
require 'httparty'
require 'json'

response = HTTParty.get("http://www.google.com/uds/GlangDetect", :query => {
  :v => '1.0',
  :q => "Sgwn i os yw google yn deall Cymraeg?"
})

p JSON.parse(response.body)["responseData"]

# => {"isReliable"=>true, "confidence"=>0.5834181, "language"=>"cy"}

Cool, Google understands Welsh!

Before we convert this to use em-http-request (HTTParty uses Net::HTTP under the covers), let’s wrap it up in a class so we can compare it to the eventmachine version we’ll write in a minute. I decided to return nil if the language could not be reliably determined.

require 'rubygems'
require 'httparty'
require 'json'

class LanguageDetector
  URL = "http://www.google.com/uds/GlangDetect"

  def initialize(text)
    @text = text
  end

  # Returns the language if it can be detected, otherwise nil
  def language
    response = HTTParty.get(URL, :query => {:v => '1.0', :q => @text})

    return nil unless response.code == 200

    info = JSON.parse(response.body)["responseData"]

    return info['isReliable'] ? info['language'] : nil
  end
end

puts LanguageDetector.new("Sgwn i os yw google yn deall Cymraeg?").language

We can now convert this code to use em-http-request:

require 'rubygems'
require 'em-http'
require 'json'

class LanguageDetector
  URL = "http://www.google.com/uds/GlangDetect"

  include EM::Deferrable

  def initialize(text)
    request = EM::HttpRequest.new(URL).get({
      :query => {:v => "1.0", :q => text}
    })

    # This is called if the request completes successfully (whatever the code)
    request.callback {
      if request.response_header.status == 200
        info = JSON.parse(request.response)["responseData"]
        if info['isReliable']
          self.succeed(info['language'])
        else
          self.fail("Language could not be reliably determined")
        end
      else
        self.fail("Call to fetch language failed")
      end
    }

    # This is called if the request totally failed
    request.errback {
      self.fail("Error making API call")
    }
  end
end

EM.run {
  detector = LanguageDetector.new("Sgwn i os yw google yn deall Cymraeg?")
  detector.callback { |lang| puts "The language was #{lang}" }
  detector.errback { |error| puts "Error: #{error}" }
}

This returns:

The language was cy

This looks pretty different. The important thing is that in all cases we’ve either called succeed or fail, which we can do since we included EM::Deferrable. Depending on which one we call, either the callback or errback block will be executed.

As an exercise you could try to extend this class to retry the api call up to 3 times on failure. You should be able to maintain exactly the same external interface, which means that we’ve successfully wrapped up this complexity in an object and can now forget how it works!

Abstracting code that returns multiple events multiple times

Now we arrive at territory where eventmachine really shines.

We’ll build a client that connects to Twitter’s streaming api, and emits events every time a tweet arrives.

To use Twitter’s streaming API you just need to open a long lived HTTP connection to stream.twitter.com, and wait for the deluge to arrive. Again we’ll use em-http-request.

Connecting to the API, and listening to all tweets that mention newtwitter is as easy as doing:

http = EventMachine::HttpRequest.new('http://stream.twitter.com/1/statuses/filter.json').post({
  :head => { 'Authorization' => [username, password] },
  :query => { "track" => "newtwitter" }
})

Although this returns a deferrable (which remember triggers its callback when the request completes), it’s actually got another trick up it’s sleeve. We can also register to be notified every time new data is received:

http.stream do |chunk|
  # This chunk may contain one or more tweets separated by \r\n
end

Let’s put this together and listen for some tweets:

require 'rubygems'
require 'em-http'
require 'json'

EM.run {
  username = 'yourtwitterusername'
  password = 'password'
  term = 'newtwitter'
  buffer = ""

  http = EventMachine::HttpRequest.new('http://stream.twitter.com/1/statuses/filter.json').post({
    :head => { 'Authorization' => [ username, password ] },
    :query => { "track" => term }
  })

  http.callback {
    unless http.response_header.status == 200
      puts "Call failed with response code #{http.response_header.status}"
    end
  }

  http.stream do |chunk|
    buffer += chunk
    while line = buffer.slice!(/.+\r\n/)
      tweet = JSON.parse(line)
      puts tweet['text']
    end
  end
}

This should return something like:

Hey @Twitter. When shall I be getting the #NewTwitter?
#NewTwitter #Perfect
WHAHOO WTF? #NewTwitter is weird!
Buenos días a todos! =) Estoy sola en la office, leyendo Le Monde y probando el #NewTwitter desde FireFox, que funciona de 10!
Curiosity and boredom got the better of me...I'm trying the #newtwitter

It works! Now say we wanted to lookup the language of each tweet using the class we built earlier. We could do this by adding further to our stream method, however this is the road to callback hell (and just imagine what it would be like if we hadn’t abstracted the language detection!).

http.stream do |chunk|
  buffer += chunk
  while line = buffer.slice!(/.+\r\n/)
    tweet = JSON.parse(line)
    text = tweet['text']
    detector = LanguageDetector.new(text)
    detector.callback { |lang|
      puts "Language: #{lang}, tweet: #{text}"
    }
    detector.errback { |error|
      puts "Language could not be determined for tweet: #{text}"
    }
  end
end

Let’s rewrite what we just did nicely.

Earlier we used a deferrable to abstract code that either succeeded or failed asynchronously. Another common technique in a lot of eventmachine code is to provide an onevent callback system. For example, it would be great if we could have an interface like this:

stream = TweetStream.new(username, password, term)
stream.ontweet { |tweet| puts tweet }

As if by magic here is the code!

require 'rubygems'
require 'em-http'
require 'json'

class TwitterStream
  URL = 'http://stream.twitter.com/1/statuses/filter.json'

  def initialize(username, password, term)
    @username, @password = username, password
    @term = term
    @callbacks = []
    @buffer = ""
    listen
  end

  def ontweet(&block)
    @callbacks << block
  end

  private

  def listen
    http = EventMachine::HttpRequest.new(URL).post({
      :head => { 'Authorization' => [ @username, @password ] },
      :query => { "track" => @term }
    })

    http.stream do |chunk|
      @buffer += chunk
      process_buffer
    end
  end

  def process_buffer
    while line = @buffer.slice!(/.+\r\n/)
      tweet = JSON.parse(line)

      @callbacks.each { |c| c.call(tweet['text']) }
    end
  end
end

So now we can write a nice bit of code like this.

EM.run {
  stream = TwitterStream.new('yourtwitterusername', 'pass', 'newtwitter')
  stream.ontweet { |tweet|
    LanguageDetector.new(tweet).callback { |lang|
      puts "New tweet in #{lang}: #{tweet}"
    }
  }
}

Mixing different kinds of IO in the same process

One of the great things about using eventmachine is that because none of the IO operations block, it’s possible and really easy to mix and match different types of IO in the same process.

So for example we could use WebSockets to push tweets to a browser in realtime and get something like this:

Fortunately there’s already a WebSocket server built using eventmachine, em-websocket (which I’m pretty familiar with since we use it in Pusher). Install it with gem install em-websocket.

Once we’ve started a websocket server (which is trivial), it’s as easy as:

stream.ontweet { |tweet|
  LanguageDetector.new(tweet).callback { |lang|
    puts "New tweet in #{lang}: #{tweet}"

    websocket_connections.each do |socket|
      socket.send(JSON.generate({
        :lang => lang,
        :tweet => tweet
      }))
    end
  }
}

See, no spaghetti!

All the code is contained in this gist, including the extremely basic HTML page which connects to the WebSocket. Please fork it and add the funky canvas visualisation I ran out of time to write!

Going further

If you’d like to learn more I recommend:

  • The wiki.
  • Aman Gupta’s slides (he knows what he’s talking about since he maintains eventmachine).
  • Joining the google group.
  • Reading the code.
  • Asking on the irc channel (#eventmachine).
  • Try node.js as well, it’s pretty cool and the same concepts apply.

Have fun!

I hope you found this article valuable and that it gives you an insight into the eventmachine. Feel free to ask questions and give feedback in the comments section of this post. Thanks!

Do read these awesome Guest Posts:

Technorati Tags: , , , , ,

Software developers: how do you know when it’s the right time to do a big software rewrite?

Yesterday I asked this question on twitter:


Software developers: how do you know when it’s the right time to do a big software rewrite?

I’ve written about this topic before (The Big Rewrite). I’m revisiting it for a presentation Rich Kilmer and I are doing this weekend at JRubyConf.

I got a bunch of replies. Some serious, some sarcastic, some obviously filtered through the jading effects of a few Big Rewrites played out. The answer ranged from “NEVER!” to “as soon as you feel like it’s necessary”. The one consistent theme is that software rewrites are scary and the decision shouldn’t be taken lightly.

Drivers include maintenance cost, programmer happiness, politics, and fear.

Here’s what the developers on Twitter said:

when the mud (from big ball of mud) falls into your eyes

seriously, though, i believe when you lose confidence implementing changes it’s probably a good time to consider a rewrite

when you find yourself monkey patching areas that you wrote at 2am on a Saturday thinking “ugh, I just want this done with” 🙂

the right time is after reading your infamous blog post.

for us the big rewrite was: Scale fail (Can’t support more than 100000 titles) and customers want linux and not windows

almost never. Incremental rewrites of specific components results in better isolation and less risky code.

My team did it once. The first dev team was gone, the code, design and database were a mess, there was no docs, no unit tests

the bugs were all over the place, the chances were slow and risky and we needed to add tons of features.

The CIO figured out that it would be more expensive to maintain the current product than to create a new one

when you’re singing “WTF! WTF! WTF! Holy Christ! WTF!”

when you look at a code made by you and curse the programmer behind it

when the pain starts to exceed the political pain required to get that kind of authority

– when you have enough in the bank to survive its possible failure to deliver on time and within budget

When the fundamental design algorithm is O(n!)

when you just can’t stand it anymore

About the same time you start asking that question 😛

By having a ‘feeling’ about it. However, big software rewrites mostly fail. Better to start something new!

When you spend more time trying to understand code than writing code

when only god know what are you trying to do

wasn’t the answer “Never”?

That’s a question searching for a rule that doesn’t/shouldn’t exist.

, it’s time to do a big sw rewrite when adding small features/fixes requires a lot of time fixing/changing the existing code.

I prefer never. I gradually employ targeted rewrites. The app will be eventually rewritten w/out the cost of stopping the world.

When I sit around thinking, “Gee, how can I screw myself and bankrupt my company in one action?”

you’re netscape, you have crazy market share, your code smells, and Microsoft decides the internet is important. Oh, wait.

, or when the cruft/ick factor in your code base causes good developers to quit.

it’s rarely the right thing imho.

I’d say when the creativity/crap ratio is about 1:10 and still declining.

I’ve never seen a good time to do it. Incremental change has always worked best, for me.

probably when you have to ask yourself that question + 1 week

When you haven’t written anything yet?

When Code comments warn me that I couldn’t possibly understand what follows, and that it can’t be changed.

when the features you’d be trying to preserve by avoiding a rewrite never should have been written in the first place.

I guess we don’t. But the turning point is usually when it gets too painful to keep it and too difficult/unstable to refactor.

When the overly-sensitive head architect of the project leaves the co. because adding a form field becomes a herculean task.

when it is painful to make even a minor change

The more defects become stories with architectural impact the closer you are to that point!

you should divide and conquer; unless it’s a new paradigm in a new language.

I would suggest it depends on how well the project works currently and whether it can be done a piece at a time RT big rewrite?

@RobotDeathSquad BigRewriteQuestion does have a useful side effect: the earnest responses tell you “where that person is at”.

when someone ponies up the dollars for it

When your velocity is very, very low because of design and architectural problems in the code. Its a big hump though.

90% of the time it makes more sense to refactor the existing code. Unless it’s a ColdFusion app 😉

It’s time for the big rewrite when the most experienced developer has lost all faith in the code base.

However, before beginning a big rewrite, you need to “fix” the management responsible for a codebase that needs a big rewrite.

> i do not wait for this moment, i try to refactor everytime 🙂

Big rewrites only occur when both product and engineering management act “unprofessionally” per @unclebobmartin’s definition

when i realize i’m spending more time compensating for existing code than writing new code.

when everyone you ask says “oh that class? Yeah it’s garbage. Nobody wants to touch it.” Chances are, it’s not the only one.

if the cost of adding value now is higher than rewriting and then adding value? I usually feel a rewrite is the scary option.

When I start to consistently feel loathing rather than excitement when I think of better designs.

@thomasfuchs I am pro-“abandon ship” as well. The need for big rewrites are signs of more sinister problems.

We rewrote our app once when the original app was unbearable to maintain.

For us it’s whenever the boss tells us to 🙂 We converted an app from VFP 9 to #rails in just under 2 yrs

That’s the hardest question I’ve ever had to answer in this biz. You never know, you just guess and hope you’re right.

It’s time for the big rewrite when things are screwed-up or messy AND nobody is bikeshedding – in other words: never

Secondly, if given the opportunity, I like walling in the existing app and building new apps around it to extend function.

I try to avoid big software rewrites as they tend to be a death march. Instead try to chip using a strangler application

When developers don’t look happy while coding. The less enjoyable it is for programmers the more rewrite is needed 🙂

Must consider context, so right answer is “it depends”. But the raising of the question itself is probably an indicator.

When maintaining the system in production gets too expensive (include running costs of hardware and admin time).

a rewrite takes less time and has less risks involved than a functional conversion but results in lower quality as well.
by functional conversion I mean a rewrite-from-scratch scenario, and by “rewrite” I meant “refactoring”

Software developers: how do you know when it’s the right time to do a big software rewrite?

Yesterday I asked this question on twitter:

Software developers: how do you know when it's the right time to do a big software rewrite?

I’ve written about this topic before (The Big Rewrite). I’m revisiting it for a presentation Rich Kilmer and I are doing this weekend at JRubyConf.

I got a bunch of replies. Some serious, some sarcastic, some obviously filtered through the jading effects of a few Big Rewrites played out. The answer ranged from “NEVER!” to “as soon as you feel like it’s necessary”. The one consistent theme is that software rewrites are scary and the decision shouldn’t be taken lightly.

Drivers include maintenance cost, programmer happiness, politics, and fear.

Here’s what the developers on Twitter said:

when the mud (from big ball of mud) falls into your eyes

seriously, though, i believe when you lose confidence implementing changes it’s probably a good time to consider a rewrite

when you find yourself monkey patching areas that you wrote at 2am on a Saturday thinking “ugh, I just want this done with” 🙂

the right time is after reading your infamous blog post.

for us the big rewrite was: Scale fail (Can’t support more than 100000 titles) and customers want linux and not windows

almost never. Incremental rewrites of specific components results in better isolation and less risky code.

My team did it once. The first dev team was gone, the code, design and database were a mess, there was no docs, no unit tests

the bugs were all over the place, the chances were slow and risky and we needed to add tons of features.

The CIO figured out that it would be more expensive to maintain the current product than to create a new one

when you’re singing “WTF! WTF! WTF! Holy Christ! WTF!”

when you look at a code made by you and curse the programmer behind it

when the pain starts to exceed the political pain required to get that kind of authority

@RobotDeathSquad BigRewriteQuestion does have a useful side effect: the earnest responses tell you “where that person is at”.

when someone ponies up the dollars for it

When your velocity is very, very low because of design and architectural problems in the code. Its a big hump though.

90% of the time it makes more sense to refactor the existing code. Unless it’s a ColdFusion app 😉

It’s time for the big rewrite when the most experienced developer has lost all faith in the code base.

However, before beginning a big rewrite, you need to “fix” the management responsible for a codebase that needs a big rewrite.

> i do not wait for this moment, i try to refactor everytime 🙂

Big rewrites only occur when both product and engineering management act “unprofessionally” per @unclebobmartin’s definition

when i realize i’m spending more time compensating for existing code than writing new code.

when everyone you ask says “oh that class? Yeah it’s garbage. Nobody wants to touch it.” Chances are, it’s not the only one.

if the cost of adding value now is higher than rewriting and then adding value? I usually feel a rewrite is the scary option.

When I start to consistently feel loathing rather than excitement when I think of better designs.

@thomasfuchs I am pro-”abandon ship” as well. The need for big rewrites are signs of more sinister problems.

We rewrote our app once when the original app was unbearable to maintain.

For us it’s whenever the boss tells us to 🙂 We converted an app from VFP 9 to #rails in just under 2 yrs

That’s the hardest question I’ve ever had to answer in this biz. You never know, you just guess and hope you’re right.

It’s time for the big rewrite when things are screwed-up or messy AND nobody is bikeshedding – in other words: never

Secondly, if given the opportunity, I like walling in the existing app and building new apps around it to extend function.

I try to avoid big software rewrites as they tend to be a death march. Instead try to chip using a strangler application

When developers don’t look happy while coding. The less enjoyable it is for programmers the more rewrite is needed 🙂

Must consider context, so right answer is “it depends”. But the raising of the question itself is probably an indicator.

When maintaining the system in production gets too expensive (include running costs of hardware and admin time).

a rewrite takes less time and has less risks involved than a functional conversion but results in lower quality as well.
by functional conversion I mean a rewrite-from-scratch scenario, and by “rewrite” I meant “refactoring”

Book Update

Brief update. Haven’t done this in a while.

The chapter on performance testing, and test performance is getting closer to done. I had hoped to finish it last week, but obviously didn’t get there. But it’s definitely getting there.

Filed under: Uncategorized

Solving the Kata

My post for the RubyLearning Blog is now available. It’s called “The Testing Mindset”, and it’s an extended work through of my TDD answer to the Ruby Kata from earlier this week.

Enjoy.

Filed under: Uncategorized

The Testing Mindset

The Testing Mindset

This guest post is contributed by Noel Rappin, who is a Senior Consultant with Obtiva, and has been a professional web developer for a dozen years. He is the author of four technical books. The most recent, Rails Test Prescriptions, is available for purchase at http://www.pragprog.com/titles/nrtest/rails-test-prescriptions. Noel has a Ph.D. from the Georgia Institute of Technology, where he studied educational technology and learner-centered design. You can find Noel on Twitter as @noelrap.

Noel Rappin If you want your Test-Driven Development (TDD) process to be effective, you need to have a testing mindset. Choosing the right tools helps, but the difference between using the TDD process well and using the TDD process poorly is much larger than any difference between tools.

If you’ve heard anything about testing, you’ve been told that the process is “red, green, refactor”, that is “write a failing test, make it pass, clean it up”. In order to use that process effectively, you need to get in the habit of “thinking TDD” as you attack a problem. Features of thinking TDD are:

  • The ability to break a task down into tiny, component pieces.
  • An embrace of the simplest solution to the problem.
  • A willingness to live with code that may feel unfinished until you write more tests.

The first one is not hard, and gets easier with practice. The second and third are the ones that I struggle with. The temptation to move your code just a little beyond your tests, or to put that little code filigree is very compelling, and almost always winds up being regretted.

I want to look at the testing mindset in solving a small problem that I first presented as a small Ruby Kata:

Find all the unique sequences of digits, like [1, 1, 2, 3, 8] that have the following properties:

  • Each element of the sequence is a digit between 1 and 9
  • The digits add to 15
  • There is at least digit that appears exactly twice
  • No digit appears more than twice
  • Order is irrelevant [1, 1, 2, 3, 8] and [1, 3, 2, 1, 8] are the same sequence and only count once.

There are 38 sequences.

At first glance, it seems like this problem would be hard to write in a test-driven way. If you start from the end result and try to do the whole thing in one try, then you wind up trying to test the final 38 number. That’s a hard way to go. The first step is to try and find a part of the problem that is small and verifiable.

After I little bit of analysis, I broke the problem into three parts. The first, and probably the easiest to understand, is how to verify whether a given sequence meets the parameters of the problem. The parameters are reasonably simple and easy to verify.

The second problem is to determine which sequences of numbers to test. I decided to solve this problem with a search through the potential sequences using a mechanism where each sequence would be able to say which sequence to test next. In other words, the problem starts by testing [1], determines that does not meet the requirements, then asks the sequence what to test next. The way I thought the problem through, that sequence is [1, 1]. This is far from the only way to solve the problem. However, by walking through the sequence this way, we have a simple, easy to verify, fact about each sequence — what comes after it in order.

The third part of the test is a loop that walks through the sequences and verifies each on in turn. We’ll come back to that in a moment.

I start by setting up an RSpec file, and write out my first specs. The first one is an easy one, just to make sure I can create my object and do the basics.

  it "should be able to sum its items" do
    sequence = Sequence.new(1, 2, 3, 4)
    sequence.sum.should == 10
  end

Getting there is pretty easy.

class Sequence

  attr_accessor :elements, :status

  def initialize(*elements)
    @elements = elements
  end

  def sum
    elements.inject { |sum, n| sum + n }
  end

end

The verification steps are all laid out in the problem. The first one, of course, is the sum.

  it "is valid if the sum is 15" do
    sequence = Sequence.new(1, 1, 7, 6)
    sequence.should be_valid
    sequence = Sequence.new(1, 1, 7, 7)
    sequence.should_not be_valid
  end

Two things to note. One is that I’m testing both the positive and negative conditions (strictly speaking, those should probably be in two different tests). Two is that I’ve picked a positive sequence that I know will stay valid even after I add the other constraints. That makes the test more stable as I add new requirements.

At this point, we pass with:

  def valid?
    sum == 15
  end

Gotta admit, that’s pretty simple. This is where the other two parts of the testing mindset come in. If you are a conscientious developer, then part of you is screaming on the inside that this is too simple, and it’s incomplete. The temptation to add the next piece of the code right now is pretty strong. Resist. Things are better if you go one step at a time.

The next step is this test:

  it "is invalid if there is no pair" do
    sequence = Sequence.new(8, 7)
    sequence.should_not have_digit_pair
    sequence.should_not be_valid
  end

I don’t need a positive test here, because my first test already covers it, though if you wanted to insist that I need a separate test for should have_digit_pair, I wouldn’t argue too strenuously.

At this point, I cheat a little bit, because I have this histogram method pre-tested from other projects, which makes the digit pair method easy.

  def histogram
    @histogram ||= begin
      @histogram = Hash.new(0)
      elements.each do |digit|
        @histogram[digit] += 1
      end
      @histogram
    end
  end

  def has_digit_pair?
    histogram.values.any? { |x| x == 2 }
  end

In this case, histogram is an implementation detail — if I didn’t have it, I would probably write it within the has_digit_pair? method and then refactor it to a separate method on the next step.

The digit trio test and code are similar

  it "is invalid if there is a digit trio" do
    sequence = Sequence.new(1, 1, 1, 5, 7)
    sequence.should have_digit_trio
    sequence.should_not be_valid
  end
  def has_digit_trio?
    histogram.values.any? { |x| x > 2 }
  end

At this point, the valid? method has gotten really, really complicated. Okay, not really:

  def valid?
    !has_digit_trio? && has_digit_pair? && sum == 15
  end

The need to test digit pair and digit trio as separate entities has eliminated any impulse I might have to write those methods inline in the valid method, which is an example of how writing in small pieces tends to give you short and independent code methods.

At this point, I’m comfortable with the verification and it’s time for the order of the sequences. One great feature of TDD that helps with ordering is that I can specify behavior without caring about implementation. I have no idea, as I start, what the final implementation is, but I can specify the behavior in a way that I can verify whatever implementation I wind up with.

The basic idea here is that each sequence will know what sequence to test next, and eventually we’ll cover the whole space with something like a while loop. It took a couple tries to get the rules for moving to the next sequence correct, I’ll spare you my back and forth. The first couple rules are easy.

  it "should start with 1" do
    sequence = Sequence.new
    sequence.next.elements.should == [1]
  end

  it "a sequence with a low sum should spawn a new element" do
    sequence = Sequence.new(1)
    sequence.next.elements.should == [1, 1]
  end

The implementation at this point is also pretty easy:

  def next
    return Sequence.new(1) if elements.empty?
    Sequence.new(*(elements << elements[-1]))
  end

So if the sequence is not empty, we append a duplicate of the last element to the end of the sequence. This helps guarantee that the digits in the sequence are in numerical order, which keeps us from having to worry about duplicate sequences in different orders, but that’s beside the immediate point (if we think there’s a problem later, we’ll write a test). The immediate point is that the specs pass, and it’s time to think up the next example of navigation behavior.

It actually might be helpful to code up the actual loop at this point. As much as I love TDD, it’s helpful to see the big picture — this example is as much an exploration of the problem as it is a normal feature, and having the loop in place makes it easy to see how the ordering method is behaving.

In Ruby 1.9 you can use an Enumerator to walk the sequence. While building the code, you can augment this with print statements to see what’s going on in a big picture sense, while using TDD to add new features. Technically, since I’ve refactored this into a class method, you could even write a test to verify the enumerator, although I didn’t when I originally solved the problem.

def self.sequences
  Enumerator.new do |y|
    sequence = Sequence.new
    while sequence
      sequence = sequence.next
      break unless sequence
      y << sequence
    end
  end
end

What’s cool about that is that invoking the enumerator is very simple:

if __FILE__ == $0
  result = Sequence.sequences.select { |s| s.valid? }
  p result.size
  p result.map(&:elements)
end

Okay, the logic at this point goes like this: following the rule we have so far, the first sequence we test is [1], the next one is [1, 1], and the next one is [1, 1, 1]. They all fail, which is fine. The relevant point is that there’s no need to then test [1, 1, 1, 2], because we know it will fail from the trio of 1′s. We can move directly on to [1, 1, 2].

We’re looking for a small, verifiable step, so to put it another way:

  it "a sequence with a trio should increase its last element" do
    sequence = Sequence.new(1, 1, 1)
    sequence.next.elements.should == [1, 1, 2]
  end

Which makes our developing program:

  def next
    return Sequence.new(1) if elements.empty?
    if has_digit_trio?
      elements[-1] += 1
      return Sequence.new(*elements)
    elsif sum < 15
      Sequence.new(*(elements << elements[-1]))
  end

Hey, we got a nice little side benefit from writing the digit trio method separately, we get to use it here.

At this point the program will hum along merrily until it gets to a sequence whose sum is over 15. (You can verify this by running the program via the enumerator). What we want when a sequence sum gets too high is to again stop, back up, and take the next to last digit and increase it. Or:

  it "a sequence whose sum is too high should back up an element" do
    sequence = Sequence.new(1, 1, 2, 2, 3, 3, 4)
    sequence.next.elements.should == [1, 1, 2, 2, 3, 4]
  end

Our next method is starting to get kind of complex:

def next
  return Sequence.new(1) if elements.empty?
  if has_digit_trio?
    elements[-1] += 1
    return Sequence.new(*elements)
  elsif sum < 15
    return Sequence.new(*(elements << elements[-1]))
  elsif sum >= 15
    new_elements = elements[0 .. -2]
    new_elements[-1] += 1
    return Sequence.new(*new_elements)
  end
end

Two more pieces, both of which work toward ending the sequence. We need to make sure that no digit goes over 9, and we need to make sure the sequence ends.

  it "a sequence should not let a digit get over 9" do
    sequence = Sequence.new(1, 9, 5)
    sequence.next.elements.should == [2]
  end

  it "the last sequence should end" do
    sequence = Sequence.new(9, 6)
    sequence.next.should be_nil
  end

Which we can solve with:

def next
  return Sequence.new(1) if elements.empty?
  if has_digit_trio?
    elements[-1] += 1
    return Sequence.new(*elements)
  elsif sum < 15
    return Sequence.new(*(elements << elements[-1]))
  elsif sum >= 15
    new_elements = elements[0 .. -2]
    if new_elements[-1] == 9
      return nil if new_elements.size == 1
      new_elements = new_elements[0 .. -2]
    end
    new_elements[-1] += 1
    return Sequence.new(*new_elements)
  end
end

It’s a little ugly, but I actually don’t see all that much in the way of necessary refactoring. I admit that the code as it stands is a little goofy in terms of whether Sequence objects are mutable or not, but it doesn’t interfere with solving the problem. Which is the point. If I need to worry about Sequence objects being immutable, I need to write additional tests to drive that change in logic. If I want to just refactor without changing logic, I can clean up as much as I want with some confidence that the tests will catch any errors.

Now, running the entire sequence will return the 38 sequences that solve this problem, with nearly the entire program under test. We were able to isolate small pieces of a more complex problem and turn them into concrete, verifiable pieces of code that enabled us to organically build the larger program.

If you want to see my complete solution, the tests are at http://gist.github.com/602077 and the program code is at http://gist.github.com/602076. Other solutions, with different assumptions and methods, are referenced in the original blog post at http://railsrx.com/2010/09/27/a-quick-ruby-kata/.

I hope you found this article valuable and that it gives you an insight into the “Testing Mindset”. Feel free to ask questions and give feedback in the comments section of this post. Thanks!

Technorati Tags: , , , , ,

Infinity Test: Flexible Continuous Testing with Multiple Ruby Implementations

Infinity Test is a new library by Tomas D’Stefano that pitches itself as a “flexible alternative to Autotest.” If you want your project’s tests (both RSpec or Test::Unit are supported) to be automatically run when changes are made, this is a great place to start.

The big benefit of Infinity Test is the support for testing across multiple Ruby implementations, powered by RVM (a candidate for Ruby project of the year, if ever there were one). For example, if you have some Test::Unit tests that you want to be run on Ruby 1.8.7, JRuby, Ruby Enterprise Edition, and Ruby 1.9.2, you could run:

infinity_test --test-unit --rubies=1.8.7,jruby,ree,1.9.2

There’s also a configuration file system with its own DSL to build more complex automated testing systems with customized notification systems and callbacks.

[ad] Ruby Inside’s newest sponsor is Recurly, a recurring billing service. They promise “subscription billing in 3 easy steps” and you can start a free trial right now. Their API is Ruby friendly and on GitHub!

iPad Programming: A Quick-Start Guide for iPhone Developers now In print and shipping

iPad Programming: A Quick-Start Guide for iPhone Developers now In print and shipping

An Introduction to Desktop Apps with Ruby

An Introduction to Desktop Apps with Ruby

This guest post is contributed by Martin Sadler, who has over 10 years experience in the web development industry working with a range of successful high profile businesses, public sector organisations, and individuals. He is best known in the Ruby community as the creator of Working With Rails and is a keen advocate of the Ruby on Rails Framework.

Introduction

Martin Sadler In this article I’m going to show you how easy it is to create a nifty little cross-platform system tray/menu bar application using JRuby.

The application enables a user to take a screenshot of their desktop and then do something interesting with it!

"Screenshot Tray Application"

What’s covered in this article

Some of the key areas covered include:

  • Using Java Classes within Ruby
  • Refactoring
  • Blocks

Note: This tutorial assumes no prior Java knowledge and covers basic Ruby concepts only.

What is JRuby?

JRuby is a version of Ruby that runs on top of the JVM. It runs just the same as regular MRI Ruby except it’s about twice as fast and you get access to existing Java libraries.

Getting started with JRuby

It’s never been easier to try JRuby out. RVM is your friend here. Assuming you have RVM already installed it’s simply a case of doing the following inside your terminal session:

rvm install jruby

and then (where x.y.z is the version of JRuby)

rvm use jruby-x.y.z

From this point all your normal ruby commands will be using JRuby in the current terminal. Sweet.

Part 1: Taking a screenshot

Robot

Robot does for the Desktop what Selenium and other automation tools do for the browser. The main difference is that you have far more access to a users machine. You can move the mouse around, control keystrokes, manipulate other programs, and most usefully take a screenshot.

What is Robot? It’s a Java class and we going to make good use of it in our first script.

Our first JRuby script

example1.rb

#----------------------------------------------------------------
# Summary: Takes a screenshot of the desktop and saves to disk
#----------------------------------------------------------------

# this gives us direct access to Java classes in Ruby
include Java

# here we are importing Java classes, just like you might require 'yaml' or 'date'
import java.awt.Robot
import java.awt.Toolkit
import java.awt.Rectangle
import javax.imageio.ImageIO

# Taking the screenshot
# 1) Create a new instance of the Robot class
# 2) Use the Toolkit class to get the size of the screen
# 3) and pass those dimensions to the Robot instance for capture
robot     = Robot.new
toolkit   = Toolkit.get_default_toolkit
dim       = toolkit.get_screen_size
rectangle = Rectangle.new(0, 0, dim.get_width, dim.get_height)
image     = robot.create_screen_capture(rectangle)

# Save the file to disk
file = java::io::File.new('test.png')
ImageIO::write(image, "png", file)

Running the above script is a simple case of:

ruby example1.rb

The resulting screenshot gets saved to the same directory you are in as ‘test.png’. Result.

Tip: One question you might be asking by this point is how do you know what classes are available when you include Java? Java API docs have the answer.
Now, although helpful, it’s still pure Java so you’ll need to do a bit of translating to the equivalent JRuby calls. The tree view makes it easier to see how all the classes fit together.

So we’ve got a script to take a screenshot, it does the job, but we can make it better.

Refactoring into a re-usable Ruby Class

screenshot.rb

class Screenshot
  include Java

  import java.awt.Robot
  import java.awt.Toolkit
  import java.awt.Rectangle
  import javax.imageio.ImageIO

  def self.capture(filename = 'screenshot.png')
    robot     = Robot.new
    toolkit   = Toolkit.get_default_toolkit
    dim       = toolkit.get_screen_size
    rectangle = Rectangle.new(0, 0, dim.get_width, dim.get_height)
    image     = robot.create_screen_capture(rectangle)
    file  = java::io::File.new(filename)
    ImageIO::write(image, "png", file)
  end

end

now fire up a copy of irb and enter the following:

require 'screenshot'
Screenshot.capture('screenshot1.png')

By moving the code into a class method it’s much tidier and easier to use for the next part of the application.

Gotcha: File naming is really important here. The Ruby class defined in the file must be the CamelCase version of the filename for the require to work otherwise you’ll get errors like :
LoadError: use ‘java_import’ to load normal Java classes
Note, this is only the case in JRuby not regular MRI Ruby

Part 2: Creating the system tray application

Now we have a class for taking a screenshot lets create the system tray application:

example2.rb

#-----------------------------------------------------------------------------
# Summary: Runs a system tray application with ability to take a screenshot
#-----------------------------------------------------------------------------

require 'screenshot'

# Import the Java class libraries we wish to use
include Java
import java.awt.TrayIcon
import java.awt.Toolkit

# Setup our menu items
exititem        = java.awt.MenuItem.new("Exit")
screenshotitem  = java.awt.MenuItem.new("Take Screenshot...")
aboutitem       = java.awt.MenuItem.new("About")

# Event handling
exititem.add_action_listener {java.lang.System::exit(0)}
screenshotitem.add_action_listener {Screenshot.capture}

# Add the items to the popup menu itself
popup = java.awt.PopupMenu.new
popup.add(aboutitem)
popup.add(screenshotitem)
popup.add(exititem)

# Give the tray an icon and attach the popup menu to it
image    = java.awt.Toolkit::default_toolkit.get_image("screenshot.gif")
tray_icon = TrayIcon.new(image, "Screenshot!", popup)
tray_icon.image_auto_size = true

# Finally add the tray icon to the tray
tray = java.awt.SystemTray::system_tray
tray.add(tray_icon)

Before you run this script make sure you have an icon named screenshot.gif in the same directory. Short of creating one you can use Google to find an icon.

On running the application you’ll see the menu appear in your system tray. Click the screenshot menu item and the screenshot gets saved to disk!

"Screenshot Tray Application"

Refactor

The application is neat but it feels quite bulky, lets condense it down in to a much tighter API:

example3.rb

require 'tray_application'; require 'screenshot';
app = TrayApplication.new("Deskshot")
app.icon_filename = 'screenshot.png'
app.item('Take Screenshot')  {Screenshot.capture}
app.item('Exit')              {java.lang.System::exit(0)}
app.run

Not bad for an application of 6 lines! Where did all that code go?

tray_application.rb

class TrayApplication

  include Java
  import java.awt.TrayIcon
  import java.awt.Toolkit

  attr_accessor :icon_filename
  attr_accessor :menu_items

  def initialize(name = 'Tray Application')
    @menu_items = []
    @name       = name
  end

  def item(label, &block)
    item = java.awt.MenuItem.new(label)
    item.add_action_listener(block)
    @menu_items << item
  end

  def run
    popup = java.awt.PopupMenu.new
    @menu_items.each{|i| popup.add(i)}

    # Give the tray an icon and attach the popup menu to it
    image    = java.awt.Toolkit::default_toolkit.get_image(@icon_filename)
    tray_icon = TrayIcon.new(image, @name, popup)
    tray_icon.image_auto_size = true

    # Finally add the tray icon to the tray
    tray = java.awt.SystemTray::system_tray
    tray.add(tray_icon)
  end

end

A similar pattern of refactoring was carried out here as per the Screenshot class.

Probably the most interesting point is the way you can use Ruby to capture a block of code. In the case of the Screenshot application there are certain actions we only want to be executed when the user clicks a menu item. You can see these as code surrounded by curly braces. Alternatively they could have been written as follows:

app.item('Take Screenshot') do
  Screenshot.capture
end

Which does more or less the same job. Usually the do/end format is preferred for multi-line code blocks.

What is handy for us is that we can capture these code blocks using the &block parameter in the method definition. This makes the block available as a local variable inside the method and it can be simply pushed onto an array (@menu_items) for later recall.

Curious as to how a block is executed once stored? Here is a simple example:

def hello(&block)
  block.call # execute the contents of the hello block using call
end

hello do
  p "hello there"
end

Tip: Always try to keep your code tidy as you go along. Look for patterns and bake them into classes if need be. Think about how the end product will look like and work out how to get there. See also: Readme Driven Development.

Part 4: One further improvement

So now we have a tray application that takes a screenshot and saves to disk. How about we get it to open in an image browser too. No problem.

Lets make a small addition by making use of the Desktop Java Class.

screenshot.rb (revisited)

class Screenshot
  include Java

  import java.awt.Desktop # added Desktop to import
  import java.awt.Robot
  import java.awt.Toolkit
  import java.awt.Rectangle
  import javax.imageio.ImageIO

  def self.capture(filename = 'screenshot.png')
    robot     = Robot.new
    toolkit   = Toolkit.get_default_toolkit
    dim       = toolkit.get_screen_size
    rectangle = Rectangle.new(0, 0, dim.get_width, dim.get_height)
    image     = robot.create_screen_capture(rectangle)

    file  = java::io::File.new(filename)
    ImageIO::write(image, "png", file)

    # Open the file in the users default application for the given file type
    desktop = Desktop.get_desktop
    desktop.open(file)
   end
  end

end

So now when you click ‘Take Screenshot..’ up pops Preview (on Mac) with your Desktop image.

Beyond the screenshot

So we have put together the foundations of a screenshot tray application and in the process built a skeleton GUI JRuby framework. So where could we go from here?

  • Send the screenshot to a web service

    This would make for a handy utility for a support web app. Take the file and post it via http to your app for processing.

  • Add an About Box

    Sadly the amount of code to show this here would have detracted from the main article. However it is still pretty straight forward. JFrame is your friend.

  • HotKey for taking the screenshot

  • Package! Rawr. Distribute

    So currently you’ll need JRuby in order to run this application. One of the main benefits of JRuby is it compiles down to Java bytecode and will run on any Java enabled platform. Mac, Windows, and Unix support in one go!

  • Variations on the theme

    Why stop at screenshots? Here are few more ideas:

    • Drag and drop copy and upload of files to a web service
    • A notification service e.g. new support messages, tweets
    • System analytics tool
    • Copy and Paste share bin

Reference

Attribution

In Summary

I hope you found this article valuable and that it gives you an insight into the world of possibilities with JRuby and desktop applications. Feel free to ask questions and give feedback in the comments section of this post. Thanks!

Technorati Tags: , , , , ,

Imagine you’re hit by a car

I was scrolling through articles on this site earlier, and I decided to re-read some of my own encouraging words for a shot of inspiration. On the post called What Would You Rather Be Doing? I came across this comment by Derek Sivers and wanted to make sure everyone saw it.

  Derek Sivers Says: 
  May 13th, 2009 at 05:39 PM

  Imagine you’re hit by a car today, wake up in the hospital, and the doctor is saying, “Sorry. We can’t stop the bleeding. You’ve only got an hour to live.”

  You’re laying on what you now realize is your death bed. Oh no! No no no! All those things you wanted to do! You never went to Italy! You never started       
  that business you knew would be a huge success! You never asked-out that gorgeous person you flirt with! You never flew ZeroG. You never lost 30 
  pounds and got proud of your looks. You never had kids.

  Make that list.

  Then make it your to-do list.

  Eliminate all those potential regrets.

Thanks, Derek!

Win a Prize by Telling us about Your Top 3 Most Useful Ruby Gems

Win a Prize by Telling us about Your Top 3 Most Useful Ruby Gems

Today I’d like to try something a little different on the blog and open up a topic for some disucssion. This one could cause some ‘energetic’ conversation but I’d love to do an informal poll on it anyway.

Top 3 Most Useful Ruby Gems – and why?

I’d love to hear your opinion and some of the reasoning for your choice.

There’s no right or wrong ultimately (although I know some readers hold strong opinions on the matter) – but hopefully in the discussion and reasons for your answer we’ll have some good learning.

Post your comments and go into the draw to win $25 worth of screencasts (Episode 1 to 5) from The Pragmatic Bookshelf.

PS: I’ll kick things off – I asked some Rubyists on their choice of the “Top 3 Most Useful Gems” and here is what they said:

  • Bruce Tate
    • Mongoid: The nosql movement is in full swing, and mongoid is our mapper of choice. The criteria API is excellent, start up is easy and efficient, and you can peel away the API to get straight to the MongoDB API beneath.
    • Shoulda: I know this gem is going to be taking a back seat to rspec, but we love it. The matchers are brilliant, but I really love the lack of ceremony in this gem set. We maintain full testing coverage, and shoulda makes it easy.
    • Rails 3: I need to tip my hat to the team. They’ve done a great job simplifying routing, active record, and opening up the architecture at the same time.
  • Christopher Haupt
    • nokogiri: We do a lot of HTML and XML parsing/digesting for our app. Having tried many other tools for the job over time, I keep finding nokogiri gets the job done with simplicity, speed, and stability.
    • haml: It is actually painful for me to use ERB now that all of my projects use haml. The number of programming mistakes due to typos is much lower. Amazing what a little forced indentation can do.
    • redcar: This may be cheating a little bit, but it is delivered as a gem! I constantly get asked about tools for Ruby development, and choice of editor can be something of a religious choice. I use TextMate, vim, e-TextEditor, and occasionally RubyMine. Redcar holds a lot of promise, is very hackable, and is open source!
  • Lucas Carlson
    • andand: Nil error bugs are among the most common kind you encounter with imperative languages and the andand library bridges the gap in and easy and straightforward way. Not as good as functional programming, but a good compromise.
    • aasm: State machines are a great way to deal with highly state-full interactions. Keeping track of whether an email account has been validated or if a blog post is published or drafted are great examples of ways that state machines can make your life easier.
    • paperclip: Such an elegant solution to storing uploaded files with various backend data storage methods, this gem is one of the true gems of using Ruby.
  • Paul Barry
    • Nokogiri: It has picked up the torch from Hpricot as the best way to parse HTML/XML. Nokogiri makes pulling data off the web so easy that powerful, useful programs can be written in just a few lines of code.
    • Resque: Having a queue that can be used to process background jobs is essential for any web application. Resque uses Redis to make that easy and provides a useful web interface to check in on your background jobs and see what’s going on.
    • Cucumber: Cucumber has completely changed the way I test applications by providing a way to write clear high-level tests that everyone involved in the project can understand. Having re-usable step definitions makes it fast and easy to write maintainable tests that only need to change when the business logic changes.
  • Sandip Ransing
    • Rails 3: ROR is an open-source web framework which follows the MVC pattern and agile methodologies built over Ruby language. I am working with Rails for the past 3 years and have built many Rails applications in a very short period. Many thanks to David (creator of Rails) and the Rails team for their awesome work. The open-source community is playing a fantastic role in adding new features to Rails and by contributing through gems and plugins. Recently Rails 3 has released many more features. Let’s ride Rails 3 and definitely it would me more fun :)
    • Thinking Sphinx: (Lines copied as it is from the sphinx site) Sphinx is a very fast search engine that indexes data and provides flexible ways of searching it. Thinking Sphinx allows you to link up your models into Sphinx simply and painlessly -– because let’s face it, searching across multiple fields using SQL is a pain in the neck. There are other search options which we have used earlier before migrating to sphinx, such as acts_as_ferret (Lack of proper documentation and takes time to index, therefore performance wise not effective) and acts_as_solr (requires Java as an overhead. Good to use with JRuby, though). Sphinx (high performance, tons of records can be indexed within minutes, easy to use because of proper documentation).
    • Prawn: PDF Library built over Ruby language to simplify pdf generation. Easy to use, neat, proper documentation, predefined layouts, faster, continuous development and support makes prawn the best among other pdf libraries.
  • Sébastien Grosjean
    • devise: An highly flexible Authentication management. Baked with full features as an engine, it’s really quick to get started with default views, … but also really easy to update to your desired need and style.
    • will_paginate: Does an introduction is still needed? Simply the best solution I’ve found to handle any model pagination.
    • routing_filter: Working mainly in Europe, I have extensive needs to develop multilingual applications. On my side, routing_filter helps me handle locales in URL and URL helpers. It gives a great flexibility, and allow you much more than that, can be really helpful for clean paginated URLs, showing extensions.
  • Sethupathi Asokan
    • Josh-splat: Ruby gem is initiated and developed by Josh Software for SMS platform. It acts as a platform for different SMS service providers so that the user/developers don’t need to worry about different type of formats. In the recently concluded LSRC 2010 our Gautam Rege presented and got a wonderful response.
    • Rails 3: Josh Software is exclusive into RoR development and Rails 3 is a breeze for any web base application. It is evolving and widely accepted by everyone. There are growing number of hosting companies supporting Rails 3.
    • delayed_job: One of the best worker thread we have worked on. It eases the developer’s work by providing various useful options (main one is retry).
  • Victor H. Goff III
    • RVM: for allowing me to test various versions of Ruby, and for the gemsets that let you keep your development and application environment clean. For BSD/Linux based systems.
    • bundler: for allowing you to ‘package’ your dependencies easily.
    • pik: for multiple Ruby versions for Windows users.

Top 3 Most Useful Ruby Gems as on 3rd Oct. 2010 at 6.00 hrs:

Charting Software ChartGo.com

Update (1st Oct.): Congrats to Samnang for winning $25 worth of screencasts (Episode 1 to 5) from The Pragmatic Bookshelf. The winner was selected from a random draw.

Looking forward to reading your answers. Please post them as comments here.

Technorati Tags: , ,

Good RobotLeg presentation at 360Flex

Paul Robertson gave a RobotLeg presentation at 360Flex and put his slides online. I had to dive into a RobotLeg application the other and wanted a refresher and found his slides really great so I’ve extracted for your (and my) convenience some useful bullets points:

  • View
    • the View is represented by your view components and their Mediators
    • Mediators provide API for view components
    • Mediators listen for view component events
    • Mediators listen for framework events
    • Mediators dispatch framework events
    • view components are not coupled to their Mediators
    • Mediators are coupled to their view components
    • Mediators can access Service and Model classes directly
  • Actor
    • Actor is the base class for Model and Service classes
    • eventDispatcher is injected into Actor
    • Actor provides a dispatch(event) method
    • Actor is for your convenience

The Ruby movement

The Ruby movement

This guest post is contributed by Matt Aimonetti, a Senior Engineer at Sony Playstation in San Diego, CA. Matt has been active in the Ruby community for many years, he developed or contributed to a lot OSS libraries and frameworks, spoke at users groups and conferences in the U.S. and abroad. Working with startups, fortune 100 companies and traditional companies, he had the opportunity to be involved with really captivating Ruby, MacRuby, noSQL/lessSQL, Rails and Merb projects. Matt is currently writing a MacRuby book for O’Reilly, available for free online.

Matt Aimonetti An art movement is a tendency or style in art with a specific common philosophy or goal, followed by a group of artists during a restricted period of time, or, at least, with the hay-day of the movement defined within usually a number of years.1

The programming world is much closer to the art world than you might think. Painters, sculptors, architects, singers, writers, cinematographers and photographers are recognized as artists, while programmers/coders/hackers are not there yet. One could argue that programming is more of a craft than an art, but instead of getting into semantics, letís look at “programming movements” the same way we look at “art movements”.

Art is about expressing and generating feelings. Various styles and techniques can be used and when artists work on a piece, they do not target the entire world but often limit their focus. Authors often try to express something personal and communicate their own world view to their audience. Keeping that in mind, letís see how this applies to the Ruby language and its creator: Matt Aimonetti, Matsumoto Yukihiro aka Matz.

Matsumoto Yukihiro
Matsumoto Yukihiro

Any good designer/artist starts by learning from exemplars. In the dead cold of a late German autumn in 1705, an impoverished young musician named J.S. Bach took a 6 month, unpaid leave from his job and walked 250 miles to study the work and ideas of maestro: Dietrich Buxtehude. Now fast forward to 2010 and take a look at well known Japanese artist Takashi Murakami’s creations.

Superflat creation for Louis Vuitton - Takashi Murakami, 2003
Superflat creation for Louis Vuitton – Takashi Murakami, 2003

You can see how masters like Henri Matisse and Andy Warhol have helped him create his own style.

La gerbe' Henri Matisse, 1953
La gerbe’ Henri Matisse, 1953

Murakami’s Superflat creation for Louis Vuitton, clearly follows Warhol’s pop-art steps with a strong influence from previous artists like Matisse, Tezuka and others. Watch one of the video clips Murakami produced for LV.

The same thing goes for programming languages. When designing his programming language: Ruby, Matz was strongly influenced by Perl, Smalltalk, Python, Eiffel, Dylan, Lisp and many more languages. But at the same time he created his own programming language with its own design values, own desiderata/objectives and own priorities.

Programming is not a religion. And neither are art movements. When you reflect on art movements, you might like impressionism better than expressionism but you know they are both art movements. When you study their contexts a bit more, you understand the motivations behind each movement better. Don’t believe even for a second that Ruby is a perfect language nor that it will be the last programming language you will need to learn. Programming languages, very much like art movements, are not set in stone, they evolve and spawn new movements. But regardless of what happens, if they grow to be important enough, programming movements influence new languages and keep on living through them.

Alright, enough armchair philosophy. Let’s look at the Ruby movement values and the problems its designer was trying to address.

In his OSCON 2003 presentation, Matz explained what his motivations in creating Ruby were. He started by stating that languages influence human thoughts much more than we think. And that the good programming languages should help developers:

  • program better.
  • think smarter.
  • finish your job faster.

He defined a few principles that he felt needed to be put in place to help program better/reduce stress in programming:

  • Principle of least surprise (the developer should not be surprised by the way an API works).
  • Principle of succinctness (shorter code is easier to write, read and maintain).
  • Principle of human interface (the language should be written for the developer, not the machine).

This, in a way, is the Ruby manifesto. It is not uncommon for art movements to define their rules, perspective and even publish manifestos. The futurists, surrealists and dadaists for instance had their own manifestos while others like the cubists let art critics explain their movements. In his presentation, Matz clearly defined the core values that he refers to when he is thinking about making a change somewhere. Each language has its own set of values and they are sometimes in opposition from one language to the other. It’s up to you, the developer, to see how these values fit your project and your team. Unfortunately, there is not (yet) a perfect language that everyone you can use for everything.

If you are reading this article, you are probably already part of the Ruby movement or you are looking into it. The two first things you should probably remember are:

  • to learn and understand Ruby’s values.
  • to be curious about what other “movements” value and how they approach challenges.

f you do that, you will first understand the pros and cons of the language, and you will understand why some aspects of the language can seem odd to you. (For more info, read the post I wrote about the discussion I had with an ex-Java developer).

Look at other engineering movements, not only programming languages:

Look at how architects design buildings, how musicians compose music, how NASA designs rockets. By understanding the main design goal, the list of objectives and the scope of each approach, you will be able to understand, value and criticize each movement.

Don’t limit yourself to using idioms and pushing keys on your keyboard without understanding the “why” behind the “how”. Ruby wants you to become a better programmer, that is part of the language’s objectives. Better Ruby developers mean a stronger influence on the Ruby movement and improvement brought by the community synergy. If you find something that is not right with Ruby or its community (I have my own list), you should try to understand why it is like that and ask yourself what you can do to help. Don’t think for a second that you are not smart or expert enough. Be passionate and get involved to improve yourself and the movement as a whole. Take for example artists learning from outside their paradigms. Picasso and Matisse were friendly rivals, they shared the same interest in primitivism and African art and influenced each other while being part of different movements. Directors like Quentin Tarantino, Martin Scorsese, John Woo, Steven Soderbergh, Brian De Palma, Wim Wenders, Oliver Stone all have different styles but all admit to have been directly influenced by Jean-Luc Godard and the Nouvelle Vague movement. Most modern music movements love to borrow from each other to create interesting new trends.

So when you work on your code, when you are looking at other programming language, think about art and remember that a movement always benefits from borrowing ideas from other movements. Don’t forget that one’s own little programming world is transient, metamorphic and therefore should remain fun, challenging and welcoming. Educate yourself, stay open minded and help take the Ruby movement to the next level.

Recommended reading (not Ruby specific):

The Design of Design The Design of Design

Hackers & Painters Hackers & Painters

The Pragmatic Programmer The Pragmatic Programmer: From Journeyman to Master

The Passionate Programmer The Passionate Programmer

So – what do you think? If you have ideas, sites, resources, etc. that I haven’t mentioned, please post them as comments here.

Post supported by Sticker Mule: Sticker Mule prints custom stickers starting at $69 for 100. They aspire to be every Ruby developer’s favorite sticker printing service. A $25 off coupon (RL01) is available for Ruby Learning readers through October 31st, 2010. Enter code RL01 during checkout.

Technorati Tags: , ,

Good RobotLeg presentation at 360Flex

Paul Robertson gave a RobotLeg presentation at 360Flex and put his slides online. I had to dive into a RobotLeg application the other and wanted a refresher and found his slides really great so I’ve extracted for your (and my) convenience some useful bullets points:

  • View
    • the View is represented by your view components and their Mediators
    • Mediators provide API for view components
    • Mediators listen for view component events
    • Mediators listen for framework events
    • Mediators dispatch framework events
    • view components are not coupled to their Mediators
    • Mediators are coupled to their view components
    • Mediators can access Service and Model classes directly
  • Actor
    • Actor is the base class for Model and Service classes
    • eventDispatcher is injected into Actor
    • Actor provides a dispatch(event) method
    • Actor is for your convenience
  • Model
    • Models extend Actor
    • Models provide an API for data
    • Models sit between application data and other actors
    • Models should not listen for framework events
    • Models dispatch framework events
  • Service
    • Services extend Actor
    • Services usually implement an interface
    • Services communicate with the outside world and provide an API to external services
    • Services can parse results from external services
    • Services do not store data
    • Services do not receive framework events
    • Services dispatch framework events
  • Controller
    • represented by the Command class
    • Commands are executed in response to framework events
    • Commands are stateless
    • Commands perform work on Service and Model classes and dispatch events (call other commands)
    • Commands receive data from the events that trigger them
    • Commands dispatch framework events
    • Commands do not receive framework events

You can get the slides here and also checkout his blog entry for more info.

These days everyone has his preferred Flex framework. Mine is still Swiz as it doesn’t impose a structure like Cairgorm or RobotLeg. I’ve also just started on a new project that uses Parsley, so I’ve soon done them all!

Enjoy!
Daniel

A Quick Ruby Kata

This is, I have on good authority, actual homework from a 4th grade gifted program, paraphrased to make it more code-kata like.

Find all the unique sequences of digits, like [1, 1, 2, 3, 8] that have the following properties:

  • Each element of the sequence is a digit between 1 and 9
  • The digits add to 15
  • There is at least digit that appears exactly twice
  • No digit appears more than twice
  • Order is irrelevant [1, 1, 2, 3, 8] and [1, 3, 2, 1, 8] are the same sequence and only count once.

I believe the original problem did specify that there are 38 sequences that match.

I’m interested in seeing solutions to this if anybody goes ahead and does it. I’m pretty sure there’s an elegant solution using Ruby 1.9 enumerations, but that’s not the way I went on my first try.

Filed under: Uncategorized

Building your own Fake Sinatra with Rack and Metaprogramming

http://teachmetocode.com/screencasts/faking-sinatra-with-rack-and-metaprogramming/ (or on Ruby Inside)

Charles Max Wood of Teach Me To Code Screencasts has put together a 13 minute screencast demonstrating how to use some Ruby metaprogramming magic along with Rack in order to build a small Sinatra-esque webapp framework. Watch the HD version at Vimeo or download the video to get the best quality.

Charles is also trying to raise a little money to help him get to RubyConf and do some coverage from there. If you enjoy this screencast, I’m sure he’d love a few bucks (and might sing your praises into the bargain).

#233 Engage with Devise

Janrain Engage (aka RPX) makes it easy to support many authentication solutions through their one service.

#233 Engage with Devise

Janrain Engage (aka RPX) makes it easy to support many authentication solutions through their one service.