Raptor: A Forthcoming Ruby Web Server for Faster App Deployment

Raptor bills itself as a new Ruby “app server” and it claims to blow everything else out of the water performance-wise (by between 2-4x!) whether that’s Unicorn, Puma, Passenger, or even TorqueBox on JRuby. The bad news for now is there’s no source or repo yet and only a handful of people (including me) have been given a sneak peek, although a public beta is promised on November 25th.

The history of Ruby webapp deployment

The deployment of Ruby (and therefore Rails) webapps was a painful mess for years, a state I lamented 7 years ago in No True ‘mod_ruby’ is Damaging Ruby’s Viability on the Web. Thankfully, shortly thereafter a number of projects came out to make life easier, the most famous being Phusion Passenger (then known as mod_rails) in April 2008.

Things have continued to improve gradually over the years, with Passenger getting consistently better, and new approaches such as those offered by Unicorn and Puma, using JRuby, as well as proxying through Nginx, coming into the picture.

Enter Raptor

Raptor, a new entry to the burgeoning world of Ruby Web servers, boasts some compelling features. “Visibility” is cited as a key feature so that you can look ‘into’ your app and analyze its performance as easily as possible using a JSON API (so building your own tools around the API should be simple). Raptor also uses the HTTP parser from Node which itself was derived from Nginx’s HTTP parser; both are renowned for their speed and stability. Raptor boasts a zero-copy, concurrent, evented architecture which makes it efficient memory and IO-wise – so even if you have slow clients or a slow network, these won’t bring your app server to a stuttering standstill.

Another feature that jumped out at me is integrated caching. Raptor doesn’t rely on an external services like memcached or Redis at all, but is truly internal and optimized specifically for Web workloads. If you’ve never set up caching before, this could provide a big boost as with Raptor it’ll be available “out of the box”.

The initial results seem promising. Fabio Akita has already shared some early benchmark results which broadly mirror my own experience (disclaimer: as someone with rather little experience and authority in benchmarking, my benchmarks are oriented around Raptor’s own benchmarking suite) but, as always, YMMV and such benchmarks are often criticized.

The waiting game..

The team behind Raptor promise they’ll be releasing some interesting blog posts soon about the technology behind it, including how the cache is implemented and has been optimized, how the zero-copy system works and how it’ll benefit your code, and similar things. So keep an eye on rubyraptor.org, especially around November 25th.

A Simple Tour of the Ruby MRI Source Code with Pat Shaughnessy

I’m not in Ruby core or, well, even a confident C coder anymore, but I’ve long enjoyed digging in the Ruby MRI source code to understand weird behavior and to pick up stuff for my Ruby course.

Pat Shaughnessy is also a fan of digging around in Ruby’s internals and has written some great posts like How Ruby Executes Your Code, Objects, Classes and Modules, and Exploring Ruby’s Regular Expression Algorithm.

When Pat released his Ruby Under a Microscope book, I knew it would be right up my street! He digs into how objects are represented internally, why MRI, Rubinius and JRuby act in certain ways and, of course, “lots more.”

I invited Pat to take a very high level cruise through the MRI codebase with me so we could share that knowledge with Ruby programmers who haven’t dared take a look ‘under the hood’ and to

Continue reading “A Simple Tour of the Ruby MRI Source Code with Pat Shaughnessy”

A Simple Tour of the Ruby MRI Source Code with Pat Shaughnessy

I’m not in Ruby core or, well, even a confident C coder anymore, but I’ve long enjoyed digging in the Ruby MRI source code to understand weird behavior and to pick up stuff for my Ruby course.

Pat Shaughnessy is also a fan of digging around in Ruby’s internals and has written some great posts like How Ruby Executes Your Code, Objects, Classes and Modules, and Exploring Ruby’s Regular Expression Algorithm.

When Pat released his Ruby Under a Microscope book, I knew it would be right up my street! He digs into how objects are represented internally, why MRI, Rubinius and JRuby act in certain ways and, of course, “lots more.”

I invited Pat to take a very high level cruise through the MRI codebase with me so we could share that knowledge with Ruby programmers who haven’t dared take a look ‘under the hood’ and to show it’s not as scary or pointless as it may seem.

It’s 100% free so enjoy it above or on YouTube in 720p HD.

P.S. Pat is happy to do another video digging deeper into how Ruby actually takes your code and executes it and he’s able to walk through the actual virtual machine for us. If the reaction to this video is good, we’ll sit down again and see if we can do it! 🙂

Let’s Build a Simple Video Game with JRuby: A Tutorial

Ruby isn’t known for its game development chops despite having a handful of interesting libraries suited to it. Java, on the other hand, has a thriving and popular game development scene flooded with powerful libraries, tutorials and forums. Can we drag some of Java’s thunder kicking and screaming over to the world of Ruby? Yep! – thanks to JRuby. Let’s run through the steps to build a simple ‘bat and ball’ game now.

The Technologies We’ll Be Using

JRuby

If you’re part of the “meh, JRuby” brigade, suspend your disbelief for a minute. JRuby is easy to install, easy to use, and isn’t going to trample all over your system or suck up all your memory. It will be OK!

One of JRuby’s killer features is its ability to use Java libraries and generally dwell as a first class citizen on the JVM. JRuby lets us use performant Java powered game development libraries in a Rubyesque way, lean on Java-based tutorials, and basically have our cake and eat it too.

To install JRuby, I recommend RVM (Ruby Version Manager). I think the JRuby core team prefer you to use their own installer but rvm install jruby has always proven quick and effective for me. Once you get it installed, rvm use jruby and you’re done.

Slick and LWJGL

The Slick library is a thin layer of structural classes over the top of LWJGL (Lightweight Java Game Library), a mature and popular library that abstracts away most of the boring system level work.

Out of the box LWJGL gives us OpenGL for graphics, OpenAL for audio, controller inputs, and even OpenCL if we wanted to do heavy parallelism or throw work out to the GPU. Slick gives us constructs like game states, geometry, particle effects, and SVG integration, while allowing us to drop down to using LWJGL for anything we like.

Getting Started: Installing Slick and LWJGL

Rather than waste precious time on theory, let’s get down to the nitty gritty of getting a basic window and some graphics on screen:

  • First, create a folder in which to store your game and its associated files. From here I’ll assume it’s /mygame
  • Go to the Slick homepage and choose “Download Full Distribution” (direct link to .zip here).
  • Unzip the download and copy the lib folder into your /mygame as /mygame/lib – this folder includes both LWGWL and Slick.
  • In /mygame/lib, we need to unpack the natives-[your os].jar file and move its contents directly into /mygame.

    Mac OS X: Right click on the natives-mac.jar file and select to unarchive it (if you have a problem, grab the awesome free The Unarchiver from the App Store) then drag the files in /mygame/lib/native-mac/* directly into /mygame.

    Linux and Windows: Running jar -xf natives-linux.jar or jar -xf natives-win32.jar and copying the extracted files back to /mygame should do the trick.

  • Now your project folder should look a little like this:

    If so, we’re ready to code.

A Bare Bones Example

Leaping in with a bare bones example, create /mygame/verybasic.rb and include this code:

$:.push File.expand_path('../lib', __FILE__)

require 'java'
require 'lwjgl.jar'
require 'slick.jar'

java_import org.newdawn.slick.BasicGame
java_import org.newdawn.slick.GameContainer
java_import org.newdawn.slick.Graphics
java_import org.newdawn.slick.Input
java_import org.newdawn.slick.SlickException
java_import org.newdawn.slick.AppGameContainer

class Demo < BasicGame
  def render(container, graphics)
    graphics.draw_string('JRuby Demo (ESC to exit)', 8, container.height - 30)
  end

  # Due to how Java decides which method to call based on its
  # method prototype, it's good practice to fill out all necessary
  # methods even with empty definitions.
  def init(container)
  end

  def update(container, delta)
    # Grab input and exit if escape is pressed
    input = container.get_input
    container.exit if input.is_key_down(Input::KEY_ESCAPE)
  end
end

app = AppGameContainer.new(Demo.new('SlickDemo'))
app.set_display_mode(640, 480, false)
app.start

Ensure that ruby actually runs JRuby (using ruby -v) and then run it from the command line with ruby verybasic.rb. Assuming all goes well, you’ll see this:

If you don’t see something like the above, feel free to comment here, but your problems most likely orient around not having the right ‘native’ libraries in the current directory or from not running the game in its own directory in the first place (if you get probable missing dependency: no lwjgl in java.library.path – bingo).

Explanation of the demo code

  • $:.push File.expand_path('../lib', __FILE__) pushes the ‘lib’ folder onto the load path. (I’ve used push because my preferred << approach breaks WordPress ;-))
  • require 'java' enables a lot of JRuby’s Java integration functionality.
  • Note that we can use require to load the .jar files from the lib directory.
  • The java_import lines bring the named classes into play. It’s a little like include, but not quite.
  • We lean on Slick’s BasicGame class by subclassing it and adding our own functionality.
  • render is called frequently by the underlying game engine. All activities relevant to rendering the game window go here.
  • init is called when a game is started.
  • update is called frequently by the underlying game engine. Activities related to updating game data or processing input can go here.
  • The code at the end of the file creates a new AppGameContainer which in turn is given an instance of our game. We set the resolution to 640×480, ensure it’s not in full screen mode, and start the game.

Fleshing Out a Bat and Ball Game

The demo above is something but there are no graphics or a game mechanic, so it’s far from being a ‘video game.’ Let’s flesh it out to include some images and a simple pong-style bat and ball mechanic.

Note: I’m going to ignore most structural and object oriented concerns to flesh out this basic prototype. The aim is to get a game running and to understand how to use some of Slick and LWJGL’s features. We can do it again properly later 🙂

All of the assets and code files demonstrated here are also available in an archive if you get stuck. Doing it all by hand to start with will definitely help though.

A New Code File

Start a new game file called pong.rb and start off with this new bootstrap code (very much like the demo above but with some key tweaks):

$:.push File.expand_path('../lib', __FILE__)

require 'java'
require 'lwjgl.jar'
require 'slick.jar'

java_import org.newdawn.slick.BasicGame
java_import org.newdawn.slick.GameContainer
java_import org.newdawn.slick.Graphics
java_import org.newdawn.slick.Image
java_import org.newdawn.slick.Input
java_import org.newdawn.slick.SlickException
java_import org.newdawn.slick.AppGameContainer

class PongGame < BasicGame
  def render(container, graphics)
    graphics.draw_string('RubyPong (ESC to exit)', 8, container.height - 30)
  end

  def init(container)
  end

  def update(container, delta)
    input = container.get_input
    container.exit if input.is_key_down(Input::KEY_ESCAPE)
  end
end

app = AppGameContainer.new(PongGame.new('RubyPong'))
app.set_display_mode(640, 480, false)
app.start

Make sure it runs, then move on to fleshing it out.

A Background Image

It’d be nice for our game to have an elegant background. I’ve created one called bg.png which you can drag or copy and paste from here (so it becomes /mygame/bg.png):

Now we want to load the background image when the game starts and render it constantly.

To load the game at game start, update the init and render methods like so:

def render(container, graphics)
  @bg.draw(0, 0)
  graphics.draw_string('RubyPong (ESC to exit)', 8, container.height - 30)
end

def init(container)
  @bg = Image.new('bg.png')
end

The @bg instance variable picks up an image and then we issue its draw method to draw it on to the window every time the game engine demands that the game render itself. Run pong.rb and check it out.

Adding A Ball and Paddle

Adding a ball and paddle is similar to doing the background. So let’s give it a go:

def render(container, graphics)
  @bg.draw(0, 0)
  @ball.draw(@ball_x, @ball_y)
  @paddle.draw(@paddle_x, 400)
  graphics.draw_string('RubyPong (ESC to exit)', 8, container.height - 30)
end

def init(container)
  @bg = Image.new('bg.png')
  @ball = Image.new('ball.png')
  @paddle = Image.new('paddle.png')
  @paddle_x = 200
  @ball_x = 200
  @ball_y = 200
  @ball_angle = 45
end

The graphics for ball.png and paddle.png are here. Place them directly in /mygame.

We now have this:

Note: As I said previously, we’re ignoring good OO practices and structural concerns here but in the long run having separate classes for paddles and balls would be useful since we could encapsulate the position information and sprites all together. For now, we’ll ‘rough it’ for speed.

Making the Paddle Move

Making the paddle move is pretty easy. We already have an input handler in update dealing with the Escape key. Let’s extend it to allowing use of the arrow keys to update @paddle_x too:

def update(container, delta)
  input = container.get_input
  container.exit if input.is_key_down(Input::KEY_ESCAPE)

  if input.is_key_down(Input::KEY_LEFT) and @paddle_x > 0
    @paddle_x -= 0.3 * delta
  end

  if input.is_key_down(Input::KEY_RIGHT) and @paddle_x < container.width - @paddle.width
    @paddle_x += 0.3 * delta
  end
end

It’s crude but it works! (P.S. I’d normally use && instead of and but WordPress is being a bastard – I swear I’m switching one day.)

If the left arrow key is detected and the paddle isn’t off the left hand side of the screen, @paddle_x is reduced by 0.3 * delta and vice versa for the right arrow.

The reason for using delta is because we don’t know how often update is being called. delta contains the number of milliseconds since update was last called so we can use it to ‘weight’ the changes we make. In this case I want to limit the paddle to moving at 300 pixels per second and 0.3 * 1000 (1000ms = 1s) == 300.

Making the Ball Move

Making the ball move is similar to the paddle but we’ll be basing the @ball_x and @ball_y changes on @ball_angle using a little basic trigonometry.

If you stretch your mind back to high school, you might recall that we can use sines and cosines to work out the offset of a point at a certain angle within a unit circle. For example, our ball is currently moving at an angle of 45, so:

Math.sin(45 * Math::PI / 180)   # => 0.707106781186547
Math.cos(45 * Math::PI / 180)   # => 0.707106781186548

Note: The * Math::PI / 180 is to convert degrees into radians.

We can use these figures as deltas by which to move our ball based upon a chosen ball speed and the delta time variable that Slick gives us.

Add this code to the end of update:

@ball_x += 0.3 * delta * Math.cos(@ball_angle * Math::PI / 180)
@ball_y -= 0.3 * delta * Math.sin(@ball_angle * Math::PI / 180)

If you run the game now, the ball will move up and right at an angle of 45 degrees, though it will continue past the game edge and never return. We have more logic to do!

Note: We use -= with @ball_y because sines and cosines use regular cartesian coordinates where the y axis goes from bottom to top, not top to bottom as screen coordinates do.

Add some more code to update to deal with ball reflections:

if (@ball_x > container.width - @ball.width) || (@ball_y < 0) || (@ball_x < 0)
  @ball_angle = (@ball_angle + 90) % 360
end

This code is butt ugly and pretty naive (get ready for a nice OO design assignment later) but it’ll do the trick for now. Run the game again and you’ll notice the ball hop through a couple of bounces off of the walls and then off of the bottom of the screen.

Resetting the Game on Failure

When the ball flies off of the bottom of the screen, we want the game to restart. Let’s add this to update:

if @ball_y > container.height
  @paddle_x = 200
  @ball_x = 200
  @ball_y = 200
  @ball_angle = 45
end

It’s pretty naive again, but does the trick. Ideally, we would have a method specifically designed to reset the game environment, but our game is so simple that we’ll stick to the basics.

Paddle and Ball Action

We want our paddle to hit the ball! All we need to do is cram another check into update (poor method – promise to refactor it later!) to get things going:

if @ball_x >= @paddle_x and @ball_x < = (@paddle_x + @paddle.width) and @ball_y.round >= (400 - @ball.height)
  @ball_angle = (@ball_angle + 90) % 360
end

Note: WordPress has borked the less than operator in the code above. Eugh. Fix that by hand 😉

And bingo, we have it. Run the game and give it a go. We have a simple, but performant, video game running on JRuby.

If you’d prefer everything packaged up and ready to go, grab this archive file of my /mygame directory.

What Next?

Object orientation

As I’ve taken pains to note throughout this article, the techniques outlined above for maintaining the ball and paddle are naive – an almost C-esque approach.

Building separate classes to maintain the sprite, position, and the logic associated with them (such as bouncing) will clean up the update method significantly. I leave this as a task for you, dear reader!

Stateful Games

Games typically have multiple states, including menus, game play, levels, high score screens, and so forth. Slick includes a StateBasedGame class to help with this, although you could rig up your own on top of BasicGame if you really wanted to.

The Slick wiki has some great tutorials that go through various elements of the library, including a Tetris clone that uses game states. The tutorials are written in Java, naturally, but the API calls and method names are all directly transferrable (I’ll be writing an article about ‘reading’ Java code for porting to Ruby soon).

Packaging for Distribtion

One of the main reasons I chose JRuby over the Ruby alternatives was the ability to package up games easily in a .jar file for distribution. The Ludum Dare contest involves having other participants judge your game and since most participants are probably not running Ruby, I wanted it to be relatively easy for them to run my game.

Warbler is a handy tool that can produce .jar files from a Ruby app. I’ve only done basic experiments so far but will be writing up an article once I have it all nailed.

Ludum Dare

I was inspired to start looking into JRuby and Java game libraries by the Ludum Dare game development contest. They take place every few months and you get 48 hours to build your own game from scratch. I’m hoping to enter for the first time in just a couple of days and would love to see more Rubyists taking part.

Amazon Releases aws-sdk, An Official AWS SDK for Ruby Developers

Amazon has unveiled an official Ruby SDK for AWS! Amazon.com’s Amazon Web Services has been a rip-roaring success since its first publicly-available service, S3 (Simple Storage Storage), was released in 2006. It has since expanded to about 20 services in all, the most popular being S3 and the “elastic compute cloud” EC2.

There have previously been unofficial Ruby libraries for interfacing with Amazon’s many services, including PoolParty, right_aws, and Marcel Molina’s awesome aws-s3, and Amazon even released some bits and pieces of Ruby code before, but the new aws-sdk gem represents a stronger effort to have a single, official cohesive library for Rubyists using AWS.

What Does aws-sdk Support?

A quick laundry list:

  • EC2
  • S3
  • SQS (queue service)
  • SNS (notifications service)
  • SES (Simple E-mail Service)
  • SimpleDB (in the form of an ORM)

This covers the most used items of the AWS suite though some further investigation needs to be done to see if Elastic MapReduce and Elastic Block Store has been baked into the EC2 stuff. There’s no Route 53 or Mechanical Turk support as yet, though Mark Percival’s RTurk library is still being updated and supports the latter.

Getting Started With aws-sdk

Installing the new official AWS SDK gem is a piece of cake:

gem install aws-sdk

(Note: aws-sdk has the Nokogiri XML/HTML parsing library as a dependency so if you’re using MRI, you’ll need to be in an environment where that can be compiled. It isn’t pure Ruby.)

Now, let’s roll out a simple file upload system using the S3 support available in aws-sdk:

require 'aws-sdk'

bucket_name = 'mytestbucket'
source_filename = '/tmp/something.txt'

AWS.config(
  :access_key_id => ACCESS_KEY_ID,
  :secret_access_key => SECRET_ACCESS_KEY
)

# Create the basic S3 object
s3 = AWS::S3.new

# Load up the 'bucket' we want to store things in
bucket = s3.buckets[bucket_name]

# If the bucket doesn't exist, create it
unless bucket.exists?
  puts "Need to make bucket #{bucket_name}.."
  s3.buckets.create(bucket_name)
end

# Grab a reference to an object in the bucket with the name we require
object = bucket.objects[File.basename(source_filename)]

# Write a local file to the aforementioned object on S3
object.write(:file => source_filename)

Note that you’ll need to set ACCESS_KEY_ID and SECRET_ACCESS_KEY somehow. You could do this with a YAML file you load in, hard code into your script (not recommended for security reasons), or perhaps use environment variables (a common approach with the Java tools, I believe). You can get your own AWS Access Key ID and the Secret Access Key from your Security Credentials page in your AWS settings.

If you are familiar with the already existing AWS::S3 library, you should note that the new aws-sdk library is not compatible with it and it cannot be loaded at the same time. You should also note that the code used to make everything work is quite different with aws-sdk and while porting a script from one to the other shouldn’t be a lengthy job, it’s not a straightforward search and replace either.

Further Reading

This was just a flying visit with a basic overview of using aws-sdk for S3. To learn more, hit up the aws-sdk-for-ruby repository on GitHub to see more examples and sample code, as well as the official AWS SDK for Ruby page by Amazon.

One thing I noticed is that if you open up the aws-sdk gem and snoop around the source code, you can learn a lot more than from Amazon’s own articles. The code is well documented and I had to refer to it a few times already. So definitely crack open that gem and have a dig around, you’ll find useful stuff in there.

[sponsor]RubyMine is a popular and powerful Ruby and Rails IDE with the full stack of essential developer tools, all tightly integrated into a convenient and smart development environment. Download it now for a free 30 day trial.

Clever Algorithms: A Free Book of Nature-Inspired Ruby Recipes

cleveralgorithms.pngClever Algorithms is a newly released book by Jason Brownlee PhD that describes 45 algorithms from the Artificial Intelligence (AI) field with Ruby-based examples. It’s well produced and, notably, free in its PDF and online formats. A print copy is available at a small cost.

The book kicks off with a chapter of background regarding AI and its problem domains and moves on to an array of algorithms in the probabilistic, neural networking, stochastic, swarm, and evolutionary spaces.

Ruby purists will note that even though the demonstrations are in Ruby, they’re not very Ruby like. Classes are rarely defined and using methods defined in the main context as functions is the order of the day. Nonetheless, the book remains well written and interesting and the Ruby code – as generic as it is – will nonetheless help Rubyists get the idea behind many of the processes demonstrated.

This book provides a handbook of algorithmic recipes from the fields of Metaheuristics, Biologically Inspired Computation and Computational Intelligence that have been described in a complete, consistent, and centralized manner. These standardized descriptions were carefully designed to be accessible, usable, and understandable.

Most of the algorithms described in this book were originally inspired by biological and natural systems, such as the adaptive capabilities of genetic evolution and the acquired immune system, and the foraging behaviors of birds, bees, ants and bacteria. An encyclopedic algorithm reference, this book is intended for research scientists, engineers, students, and interested amateurs.

Jason Brownlee

Check out Jason’s book at cleveralgorithms.com and the content and code are in this GitHub repository.

[ad] Jaconda is a chat system designed for teams working on projects. It works from the browser, your IM app, or your phone, and lets you chat, share files, and keep up with tickets and project activity you can have sent automatically to your rooms.

A WebKit Plugin Written in MacRuby

webkit.pngEloy Duran (of the Dutch Rails consultancy Fingertips) has put together an interesting side project: a WebKit plugin written in MacRuby. His ‘MacRubyWebKitPluginExample’ project on GitHub is a short, self contained example of how to pull it off, so it’s worth checking out if you want to do something similar. Eloy’s example simply allows Ruby code to be supplied by a text box in a WebView and then executed by MacRuby on the back end.

Here’s a video showing it in action:

Before you get too excited, there are some significant provisos. Eloy explains:

Whoa, hold your horses, son. Because this brings us to the most important note, and will most probably be sad news to you, which is that none of the common WebKit based browsers support this… This means no Safari nor Chrome. [Due to their lack of support for garbage collection.]

The latter could possibly be compiled with garbage collection support, it’s completely open-source after all, but I was not able to do so in my quick attempts. In case you’d like to have a go, I could not get ‘Chromium Framework.framework’ to support it. Let me know if you are successful.

However, it’s possible to use the plugin from a Web browser because, well, that’s what a WebView is. You can build a simple Web browser

Eloy Duran

So it’s early days, but these problems seem surmountable, and in the interim it could be a useful technique for those of you building MacRuby OS X apps with custom WebKit WebViews.

Thanks to Ilya Grigorik for pointing this project out on Twitter.

Parslet – A Simple PEG Parser Framework for Ruby

Screen shot 2011-01-12 at 2.32.49 AM.pngParslet is a new “simple parser framework” for Ruby built by Kaspar Schiess. It follows the PEG (parsing expression grammar) style of parsing and its primary goals are to be simple, testable, and to have extensive and powerful error reporting features (something that Kaspar states Treetop sorely lacks).

If you’ve already used Treetop, you might wonder what the hoopla is about. The key difference is that Parslet does not generate code to represent the parsing grammars you create as Treetop does – it works directly from rules defined using standard Ruby syntax (Treetop has its own Ruby-esque grammar language). Parslet’s error messages are also significantly nicer than Treetop’s when you inevitably run into trouble with your grammar as they’re generated by Parslet itself and don’t spring from Treetop’s internally generated code.

ast.png

So if you need to build a parser (or think you might), check out Parslet – it has a great homepage with some useful documentation. Installation is simple (gem install parslet) and Kaspar has put together a “getting started” tutorial that walks through building a basic grammar for a teeny-tiny programming language. It concludes with a complete example of a programming language interpreter in a mere 49 lines.

VCR: A Recorder For All Your Tests’ HTTP Interactions

haha-funny.pngVCR is a library by Myron Marston that records your test suite’s HTTP interactions so that they can be quickly replayed during future test runs. The big win is that you get predictable, quick and accurate tests. If you need to update the data, just delete the fixtures VCR generates and you’re good to go.

On the surface, VCR sounds like it copies the work of libraries like FakeWeb and Webmock, but VCR uses those libraries to provide a cleaner, more abstracted experience. VCR supports the mocking features of FakeWeb, Webmock, Typhoeus and Faraday out of the box, and further, supports multiple HTTP client libraries including Net::HTTP, Typhoeus, HTTPClient, Curb, Patron, em-http-request and Net::HTTP-based clients like HTTParty, RESTClient and Mechanize.

You can learn more at the VCR GitHub page (it has a good, basic README) or give it a try right now with the following:

sudo gem install webmock vcr

And then run the following code:

require 'rubygems'
require 'test/unit'
require 'vcr'

VCR.config do |c|
  c.cassette_library_dir = 'fixtures/vcr_cassettes'
  c.stub_with :webmock # or :fakeweb
end

class VCRTest <  Test::Unit::TestCase
  def test_example_dot_com
    VCR.use_cassette('synopsis', :record => :new_episodes) do
      response = Net::HTTP.get_response(URI.parse('http://example.com/'))
      assert_match /You have reached this web page by typing.*example\.com/, response.body
    end
  end
end

The first run will fetch example.com and save the completed response into a fixtures file generated at ./fixtures/vcr_cassettes/synopsis.yml. Future runs will then look for that fixture file and use the earlier response if present.

[ad] New Relic is the #1 on-demand application performance management app in the Ruby world and now they’ve expanded to PHP. Yep, you can now ensure the health and availability of your PHP apps too and the RPM Lite package is free.

A Tender, Loving RubyConf Presentation on Performance Analysis

tendertalk.png

At RubyConf X in November 2010, Aaron Patterson (a.k.a tenderlove) gave a 45 minute talk called ZOMG WHY IS THIS CODE SO SLOW? The video of this conference, as recorded by the always awesome Confreaks, is now online. It’s a must watch, if only to learn how he got commit access to both Ruby and Rails (a rare feat indeed). In the talk Aaron examines his work on Arel (as used to build up SQL queries in Rails 3.0) and how he boosted its performance, a topic he also covered in an October 2010 blog post.

The official synopsis:

In this talk, we’ll take a look at different tips and tricks for writing speedy code in Ruby. We’ll look at techniques for analyzing our Ruby code to find the slow spots, and ways we can speed up those hot spots. MRI’s VM will be analyzed in order to understand how to take maximum advantage of our Virtual Machine. Tools for uncovering the hidden algorithms used in our legacy code will be on display! Learning to mathematically represent those algorithms will be discussed so that we can slice away at our slow code! Bring your helmets and seat belts and get ready to strap yourself in for some EXTREME programming! If you attend this talk we’ll give you the whole seat, but you’ll only need the edge! Space is limited, so ACT NOW!

Aaron Patterson

Just go watch it folks.

RubyDrop: A Dropbox Clone in Ruby

rubydrop.png Ever used Dropbox? It’s awesome. A cross-platform (Windows, Mac, Linux, and even mobile) file syncing and backup service with 2GB for free (or 2.25GB if you sign up with this link). Well, if you’d like to roll out your own system on your own infrastructure, send some thanks to Ryan LeFevre, the creator of RubyDrop, an open source Dropbox clone based on Ruby and git.

Cloning Dropbox sounds like a tall order and in reality, “Dropbox clone” is a big stretch. RubyDrop is quite simple under the hood and focuses on the file syncing part of the problem. A Ruby process monitors a folder for changes and uses Git to do the heavy lifting and change management between your clients and a defined central server.

RubyDrop is not without its flaws. What happens when files are changed on multiple nodes and then clash? Your mileage may vary. What’s good, though, is that Ryan seems keen to improve the system and has already planned a centralized server and is considering integrating rsync to make file syncing smoother. Ryan is a systems engineer at TwitPic with a background in PHP, JavaScript, and C but RubyDrop is his first Ruby project. I look forward to seeing more both from him and RubyDrop.

[sponsor] MinuteDock is a fast and streamlined time tracking/logging app that makes tracking time and sending invoices painless and easy. Check it out.

The Why, What, and How of Rubinius 1.0’s Release

Rubinius or GitHub repo, an alternative Ruby implementation that’s built in Ruby itself – as much as possible, has this last weekend hit the coding equivalent of a Bar Mitzvah.. its 1.0 release! Congratulations to the Rubinius team, past and present, and everyone who has helped with its release – I didn’t know if you were going to make it for a moment there..

The Why: The History

Back in 2006, Geoffrey Grosenbach (of PeepCode) claimed to have made a bet by donating $1,000 to the then-new Rubinius Ruby implementation project founded by Evan Phoenix. It seemed more like an act of charity at the time, but Geoffrey had the last laugh in securing the naming rights to Rubinius 1.0, also known as Fabius.

Rubinius’ goal was inspired by LISP and Smalltalk compilers/interpreters where all (or as much as practically possible) of the implementation was written in the language it aimed to support. The benefits of this are clear; you wouldn’t need to be a wizened-old C guru to make significant improvements or changes to the interpreter.. you could just write Ruby.

By the end of 2007, and thanks to a lot of help from Engine Yard, there were 5 full-time staff “solely” working on Rubinius and progress was thick and fast, with Rubinius becoming only the third Ruby implementation to run Rails in May 2008.

In November 2008, disaster struck. Not only was the world plunged into a financial crisis, but Evan had to lay off most of his Rubinius team, leaving just two in-house Rubinius developers. The goal, now, was to get more people involved with Rubinius from the traditional open source world, donating their time for free. Regardless, the future didn’t seem particularly bright, and in May 2009, Evan announced that “the rumors of our demise are greatly exaggerated” (in reference to Mark Twain) and that development was continuing (though seemingly at a less industrious pace).

The What: What’s It Got?

It’s great, then, that Rubinius has made it to the hallowed “version 1.0.” Evan’s a straight-up guy, and if he’s putting the 1.0 seal on it, it must be pretty good to go. Want some bulletpoints on what it’s got?

  • Works with both Rails 3.x and 2.3.5
  • Works with Sinatra
  • It has a generational garbage collector
  • Support for popular extensions including sqlite3-ruby, mysql, Nokogiri, yajl-ruby (and the obligatory “much, much more.”)
  • Integrated profiler
  • JIT compilation (a la JRuby) for über-performance
  • It doesn’t work on Windows (yet)

The How: Getting Rubinius 1.0 Running

Sorry to sound like a broken record but.. the best way to get Rubinius 1.0 running is to use Wayne E Seguin’s RVM (Ruby Version Manager)! RVM has already been updated to support Rubinius 1.0, so it’s very easy to get up and running if you’re on OS X or Linux (though you need to make sure you get HEAD first):

# rvm update --head
.. time passes, then you close your terminal and open a new one ..
# rvm install rbx
.. time passes, Rubinius 1.0 gets installed ..
# rvm rbx
# ruby -v
rubinius 1.0.0 (1.8.7 release 2010-05-14 JI) [x86_64-apple-darwin10.3.0]

If you prefer to live life on the wild side, you can hit up the Rubinius GitHub repository, check out the code and perform a manual install (by following the instructions in the README). If you’re interested in tinkering, rather than just trying your code on Rubinius, this is the route I’d advise, since you can more easily dig into Rubinius’s source code and see just how it’s implemented (or, of course, you could just browse the source on GitHub, as I tend to do).

RDropbox: A Ruby Client Library for Dropbox

Dropbox is a popular file hosting service (4m+ users) that provides synced backup and file hosting to OS X, Windows, and Linux users. You get up to 2GB of space for free. RDropbox is a library by Tim Morgan (of Autumn fame) that takes advantage of the official Dropbox API from Ruby.

With RDropbox you can log into a Dropbox account using OAuth and then upload and download files. A requirement, however, is that you apply for Dropbox API access and are approved, as the API is not fully open to the public without going through the approval process (this appears to be in order to avoid overloading their service). The API was, notably, worked on by notable Ruby alumnus, Zed Shaw.

Once you’ve made it into the Dropbox API program, RDropbox gives you the benefit of writing code as simple as:

# STEP 1: Authorize the user
session = Dropbox::Session.new('your_consumer_key', 'your_consumer_secret')
puts "Visit #{session.authorize_url} to log in to Dropbox. Hit enter when you have done this."
gets
session.authorize
session.sandbox = true

# STEP 2: Play!
session.upload('testfile.txt')
uploaded_file = session.file('testfile.txt')
puts uploaded_file.metadata.size

uploaded_file.move 'new_name.txt'
uploaded_file.delete

An alternative, unofficial route: tvongaza’s DropBox

If you want to be using Dropbox from Ruby right now, there’s an alternative: tvongaza/DropBox. This alternative library predates the official Dropbox API and uses the same techniques as the official Dropbox clients instead. For this you’ll need to use your Dropbox e-mail username and password to log in (this could be a problem if you want to use third party Dropbox accounts!) and you can then create, delete and rename folders and files, as well as check usage statistics.

3 New Date and Time Libraries for Rubyists

In the UK there’s a cliché that goes: “You wait hours for a bus, and then three come along at once!” So it went with these three Ruby date and time libraries. They all made an appearance on RubyFlow last week and are all useful in their own ways, depending on how you’re working with dates and times.

ice_cube – Fast querying and expansion of event recurrence rules

ice_cube is a library by John Crepezzi that provides “fast querying and expansion of recurrence rules in Ruby.” What this means is that you can create schedules powered by date recurrence rules that can be quite complex (e.g. every 4 years on a Tuesday in the first week of November). Rules like these are defined by chaining methods together, rather than using natural language.

To install:

gem install ice_cube

To use:

require 'ice_cube'
rule = IceCube::Rule.yearly(4).month_of_year(:november).day(:tuesday).day_of_month(2, 3, 4, 5, 6, 7, 8)
schedule = IceCube::Schedule.new(Time.now)
schedule.add_recurrence_rule rule
schedule.first(3)
# => [Tue Nov 02 05:04:38 +0000 2010, Tue Nov 04 05:04:38 +0000 2014, Tue Nov 06 05:04:38 +0000 2018]

ice_cube also supports exporting rules into iCal and YAML formats as well as a natural language equivalent.

John has put together a PDF presentation that shows off more usage, and there are some simple examples on the official site too.

tickle – A natural language parser for recurring events

tickle is a natural language parser for recurring events by Joshua Lippiner that stands in contrast to ice_cube’s method driven approach. It depends on the popular chronic natural language date parser and appears (through my experience) to be for Ruby 1.9+ only.

tickle lets you throw it things like every 4 days starting next saturday, every other week, the tenth of the month and similar (there are a lot of examples on tickle’s GitHub page). You pass these to the Tickle.parse method and you get the next occurrence of the rule.

To install:

gem install tickle

To use:

require 'tickle'
Tickle.parse('every 4 days starting next saturday')
# => 2010-05-01 12:00:00 +0000

tickle isn’t particularly mature yet and it only makes it easy to get the next occurrence of your rule, but the developer suggests that once an event has occurred, you automatically run Tickle again to get the next date. In this way, it seems tickle is well suited for situations where only the next occurrence needs to be stored and the rule can be kept in a separate database column or similar.

business_time – Time and date offsets based on “business time/hours”

business_time is a new library that works with the concept of “business time” or “business hours.” Rather than just letting you perform operations on dates by absolute numbers of days or hours, you can now work with business days and hours of your own definition. business_time depends heavily on Active Support.

To install:

gem install business_time

Note: business_time depends on Active Support (gem: activesupport)

To use:

require 'active_support'
require 'business_time'
# Examples "from now"
4.business_hours.from_now
5.business_days.from_now
# Using user supplied dates
my_birthday = Date.parse("August 4th, 2010")
10.business_days.before(my_birthday)
# Add a day to not count as a business day
BusinessTime::Config.holidays << my_birthday
# Overlapping days are OK
6.business_hours.after(Time.parse("August 3rd, 3:00pm"))

Nestful: A Simple Ruby HTTP/REST Client Library

Nestful is a simple HTTP/REST client library for Ruby, developed by Alex MacCaw (of Juggernaut) fame. Nestful allows you to consume basic Web services easily, usually in a single line of code. It can deal with JSON, buffered downloads, and callbacks out of the box.

HTTParty is the current, de-facto simple HTTP/REST client library used by most Rubyists (when net/http won’t do or when Typhoeus is too overkill) but Nestful differs enough from HTTParty to live alongside it. While HTTParty encourages you to build up some structure and separate the types of resources you’re accessing into classes (that HTTParty then extends), Nestful offers a simpler, “just call a method from anywhere” approach.

Some basic Nestful examples:

Nestful.get 'http://example.com' #=> "body"
Nestful.post 'http://example.com', :format => :form #=> "body"
Nestful.get 'http://example.com', :params => {:nestled => {:params => 1}}
Nestful.get 'http://example.com', :format => :json #=> {:json_hash => 1}
Nestful::Resource.new('http://example.com')['assets'][1].get(:format => :xml) #=> {:xml_hash => 1}

Nestful’s features include:

  • Simple API
  • File buffering
  • Before/Progress/After Callbacks
  • JSON & XML requests
  • Multipart requests (file uploading)
  • Resource API
  • Proxy support
  • SSL support

Cinch: A Ruby IRC Bot Building Framework

Cinch (or GitHub repo) is a new Ruby “microframework” for creating IRC bots. Effectively, Cinch is a library that both abstracts away all of the complexities of dealing with IRC servers and presents a DSL for rolling out your own functionality.

Cinch’s Hello Bot example demonstrates how you can easily create a bot that connects to an IRC server (irc.freenode.org), joins a channel (#cinch) and then replies to greetings:

irc = Cinch.setup :verbose => true do
  server "irc.freenode.org"
  nick "Cinchbot"
  channels %w(#cinch)
end

irc.plugin "hello" do |m|
  m.reply "Hello, #{m.nick}!"
end

irc.run

Cinch isn’t the first attempt at building a DSL for creating bots in Ruby. Autumn is perhaps the most famous existing library, but it’s extremely heavy compared to Cinch. Cinch vs Autumn is almost like Sinatra vs Rails. Other libraries include Rbot and Butler.

Update: I’ve been reminded in the comments that Isaac is another IRC bot DSL that’s very much like Cinch.

If you prefer to get nearer the wire and see how the IRC protocol works, this Ruby code snippet provides basic IRC functionality.

awesome_print: A New Pretty Printer for your Ruby Objects

awesome_print is a Ruby tool that provides “pretty printing” support for your objects. It’s a bit like p, pp or, if you prefer, puts obj.inspect, but with significantly improved, contextual, colored output. Its creator and maintainer is Michael Dvorkin of Fat Free CRM fame.

Being able to see “inside” Ruby objects on the fly can prove useful whether you’re debugging some code your tests did not dare reach or you’re just playing around in irb. The most common way to examine objects is with p or the inspect method, but these don’t format their output in a particularly easy-to-read way. pp – part of the standard library – is a pretty printer that improves matters but it still leaves a lot to be desired. awesome_print takes it all to the next level.

A visual comparison between pp and awesome_print proves compelling:

awesome_print‘s most compelling features are showing you the index of array elements within its output, inheritance for the classes of embedded objects, and color coding. Further, it’s highly customizable, with the ability to set indent levels as well as the colors for each type of data shown.

To get up and running, gem install awesome_print and then require 'ap' and use ap in place on anywhere you’d usually use p. Yep, that’s it.

Padrino: A Webapp Framework Wrapped Around Sinatra

Hot on the heels of Sinatra 1.0 comes the official release of Padrino (or GitHub repo), a webapp framework that provides an extra layer of functionality on top of Sinatra (like helpers, generators, admin interface, and internationalization). Padrino is Sinatra 1.0 compatible.

Developers Davide D’Agostino, Nathan Esquenazi and Arthur Chiu love Sinatra and its development philosophy, but want to provide a deeper, standardized layer of functionality on top of the typical Sinatra stack. True to form, Padrino extends Sinatra with a wealth of extra features:

  • Namespaced route aliases
  • Nested routes
  • Controllers
  • i18n / Internationalization
  • Mailer system
  • Django-esque admin interface
  • Unified logging
  • Tag, asset, for, and tag helpers for use in views
  • App, model and controller generators

Most of these features can be added to Sinatra already, either manually or by selecting from a wide assortment of independent plugins. Padrino, on the other hand, provides a standard suite of functionality that, hopefully, will continue to be improved as a whole over time. It feels a lot like Ramaze but with the similar functionality wrapped around Sinatra instead.

The Padrino team have done a great job putting together its official Web site and there’s a lot of documentation to peruse. They’ve also put together a guide to contributing to the project if you want to get your hands dirty. There’s also a 12 minute screencast:

Supermodel: Simple ActiveModel-Powered In-Memory Models

Supermodel is a new library by Alex Maccaw that uses the Rails 3.0 ActiveModel library to provide ActiveRecord-esque in-memory “database” storage in Ruby.

Supermodel is best demonstrated with a basic example:

require 'supermodel'

class Person < SuperModel::Base; end

a = Person.new( :name => "Jim" )
a.save

Person.find_by_name('Jim') # => #<Person>
Person.all # => [#<Person>]

This is just the start! Out of the box, Supermodel supports validations, callbacks, observers, dirty change tracking, and serialization. It also allows you, with only a little magic, to go beyond ephemeral memory-only structures and marshall your SuperModel-based objects to disk or even to a Redis store.

A more complex example that includes randomly generated IDs and validations:

require 'supermodel'

class Person < SuperModel::Base
  include SuperModel::RandomID
  attributes :name
  validates_presence_of :name
end

a = Person.new
a.valid? # => false
a.name = "Jim"
a.valid? # => true
a.save
a.id # => "6481a4fcd834e567836587c6da"

It’s early days for Supermodel, but I can see it becoming a big deal for Rubyists away from the Rails stack. The gemified version doesn’t have support for relationships yet, but the edge version on GitHub has early support for belongs_to and has_many included.Alex has written the code in a well structured way and creating modules or subclasses to add support for interacting with other backends (such as, say, Tokyo Cabinet) doesn’t look like it’d be too hard.

Supermodel shows a lot of promise and is what I was originally hoping ActiveModel was going to be. It provides just the right level of abstraction and separation from the database, but without losing the goodness we came to enjoy from ActiveRecord over the past few years.

Sinatra 1.0 Released: Major Milestone for Ruby’s Best Webapp DSL

In November 2007, we casually mentioned a new Ruby webapp library called Sinatra. It took a year to capture the imagination of the Ruby community as a whole and we eventually covered it in more depth but today we’re proud to (exclusively) announce that Sinatra has today reached its landmark 1.0 release!

Impromptu release party in the official #sinatra channel on irc.freenode.net, anyone? 🙂

Sinatra is well known in the Ruby community for providing developers with a simple way to put together Web apps big and small. The canonical ultra-simple example:

require 'rubygems'
require 'sinatra'

get '/hi' do
  "Hello World!"
end

Sinatra’s lead developers — Ryan Tomayko, Simon Rozet, and Blake Mizerany — have done a great job, along with about 50 other contributors, to produce a slick and powerful Web application DSL for Rubyists. Their ideas have even inspired similar frameworks in other languages (such as Sammy in JavaScript). Satish Talim put together a great piece, 20+ Rubyists are using Sinatra – Do you?, last year that got a good feel for how Sinatra’s being used for Web apps both big and small by a collection of Rubyists.

What’s New?

As an avid user of Sinatra 0.9, I asked Blake Mizerany what the biggest changes were going to 1.0:

I think the biggest changes are what we cleaned up. Tilt is a great new under-the-hood addition. Sinatra has matured; we’re done messing around. It’s super solid. The extension API has matured to something really killer. Extensions are ridiculously simple to create now – you can easily install helper methods, DSL methods, and install new routes on apps, and the user need only require 'sinatra/your-extension' – that’s it.

Unsurprisingly, the official changelog also provides useful information to existing Sinatra developers.

A more significant longer-term change for Sinatra over the past year has been in how it integrates with Rack. In Sinatra Rack And Middleware, Ben Schwarz looks at how Sinatra interacts with Rack and how you can use Rack’s middleware and multi-application features with Sinatra. A key aspect of this is the ability to produce “modular” Sinatra applications by subclassing Sinatra::Base to separate discrete applications or application portions:

require 'rubygems'
require 'sinatra/base'

class MyApp < Sinatra::Base
  get '/' do
    "Hello world!"
  end
end

And then, in a Rackup file:

require 'my_app'
run MyApp

Installing and Trying Sinatra

If you’re not yet using Sinatra and want to give it a quick try, you can install it with RubyGems:

gem install sinatra

Put the following basic example into a file, say example.rb:

require 'rubygems'
require 'sinatra'

get '/hi' do
  "Hello World!"
end

Then run the Ruby file in the usual way, and a request to http://localhost:4567/hi should get you a “Hello World!” response.

Learn More Online and on IRC

The examples shown above are as basic as Sinatra gets, but it goes a lot deeper than that (such as embedding Sinatra apps inside Rails apps) and the Sinatra Web site has some great code examples of where you can go next. There’s also solid documentation available and if you want to take a look at lots of existing Sinatra projects to get a feel for the patterns and techniques involved, there’s a great list of Sinatra apps and extensions too.

Lastly, if you’re interested in asking questions about Sinatra or just hanging out with Sinatra’s developers and users, head along to the #sinatra channel on irc.freenode.net. There’s usually between 50-100 people hanging out there.