This Week in Ruby: Rails 4 Sneak Peeks, A Gist Drama, and a CoffeeScript Book (by a Rubyist!)

Welcome to this week’s Web-based syndication of Ruby Weekly, the Ruby e-mail newsletter.

Highlights include: MagicRuby 2012, a code structure/patterns ‘drama’, Mark Bates’ new CoffeeScript book, minitest 3.2.0, and the Ruby Rogues chatting with Jim Weirich (who I met for the first time this week, ironically at a JavaScript conference!)

Headlines

‘Programming in CoffeeScript’ by Mark Bates, Now Available
Mark Bates, a Ruby developer, was won over by CoffeeScript and has written a book for Addison-Wesley which has now been published. Despite being a language that compiles down to JavaScript, CoffeeScript has become relevant to Rails developers in recent times given its inclusion in Rails 3.1+.

Reading

Am I Doing It Wrong?
A month ago, Rubyist Justin Ko asked fellow Rubyists if his code in a Rails project was poorly designed or not. The initial consensus was that he’d done a good job but then last week DHH weighed in and.. it got a bit ‘dramatic.’

My (new) Favourite RSpec Pattern
Jordan Maguire shares an interesting RSpec technique and there’s some interesting debate in the comments. It seems popular, though Geoffrey Grosenbach raises doubts about its readability.

Tutorial: Reddit in 10 Lines of Code
A curious walkthrough of taking an existing ‘Reddit in Sinatra’ and using some ‘seldom used features of Ruby’ to cram it down to 10 lines of dense Ruby code.

Generating Random URL-safe tokens with SecureRandom
Burke Libbey shows off a single liner that can generate a URL safe token using something from the Ruby standard library.

Rails 4.0 Sneak Peek: Queueing
A quick look at the new queueing API in edge Rails (to become Rails 4.0).

Watching and Listening

Facebook Authentication in Rails (RailsCasts)
Ryan Bates shows us how to create a new Facebook application, configure it, add authentication with the omniauth-facebook gem and top it off with client-side authentication using the JavaScript SDK.

Complex Made Simple: Sleep Better with TorqueBox
At RailsConf 2012, Lance Ball spoke about TorqueBox, a popular Ruby application server built on JRuby and JBoss AS7 that provides asynchronous messaging, scheduled jobs, long-running processes, caching, and more.

The Ruby Rogues Discuss SOLID with Jim Weirich
The Rogues sit down with Ruby old-hand and all round nice guy Jim Weirich to talk about the ‘SOLID’ principles of object oriented design.

The Changelog Interviews Laurent Sansonetti about Ruby Motion, MacRuby and More
The Changelog guys caught up with RubyMotion creator Laurent Sansonetti to talk about MacRuby, RubyMotion, MonoTouch, and more.

Libraries and Code

Rack Canonical Host: Middleware for Defining A Canonical Host Name
Rack middleware for defining a single hostname as the canonical host for your app. Requests for other hostnames will get redirected to the canonical host.

SoundCord: Perform Comparisons for Phonetically Similar Terms
A phonetic algorithm matches two different words with similar pronunciation to the same code, which allows phonetic similarity based word set comparison and indexing. SoundCord is initially aimed at Brazilian Portuguese but English and Spanish support is not far behind.

Jist: Publish GitHub ‘gists’ from Ruby
Jist is a gem that allows you to publish a gist from Ruby or the command line including file uploads and OAuth2 authentication.

ffi-http-parser: Ruby FFI Bindings to the ‘http-parser’ Library
http-parser is a high performance HTTP message parser written in C by Joyent. ffi-http-parser can hook your Ruby scripts up to it for parsing both HTTP requests and responses.

Jobs

Back End Developer, SoundCloud, Berlin, Germany
Soundcloud is looking for an über-talented back end web developer with knowledge of the latest web development tools and methodologies. As part of our A-team, you will be faced with the challenges of developing the world’s leading audio sharing platform.

Sr. Ruby on Rails Software Engineer
Do you live to work hard, play hard, live life big? Then consider working at G5. G5 has created a digital experience management platform designed to allow clients to optimize their online experience. Our mission is to help our customers and employees thrive.

Last but not least..

Sublime Text 2 for Ruby
A look at the Sublime Text 2 programmers’ text editor including why it’s a good fit for Rubyists and how to set it up in a productive manner.

Archimedes Labs mentioned in the Wall Street Journal

Today’s Wall Street Journal has an article written by Timothy Hay about the practices of Law firms here in Silicon Valley in dealing with start-ups. Archimedes Labs gets a nice mention.

Dart for Hipsters

Dart for Hipsters now available on the Pragmatic exPress

#360 Facebook Authentication

This will show how to create a new facebook application and configure it. Then add some authentication with the omniauth-facebook gem and top it off with a client-side authentication using the JavaScript SDK.

The Last 2 Weeks in Ruby: Rails 3.2.6, Savon 1.0, RailsInstaller for OS X, Pow 0.4.0, Rails 2 Asset Pipeline

Welcome to this week’s Web-based syndication of Ruby Weekly, the Ruby e-mail newsletter. While I have you, be sure to follow @RubyInside on Twitter as I’m going to be posting news more frequently there than on the Web site in future.


The latest highlights include: Rails 3.2.6, RailsInstaller for OS X, Pow 0.4.0, Rails 2 Asset Pipeline, Sidekiq 2.0, and Savon 1.0.

Headlines

Rails 3.2.6 Released: Fixes More Vulnerabilities
Rails 3.2.4 was released a few weeks ago fixing two serious security vulnerabilities (explained in another post below). Rails 3.2.5 then followed a day later as 3.2.4 introduced a nasty scoping bug.. and.. now Rails 3.2.6 to continue to fix those bugs. They’re tricky blighters.

RailsInstaller for OS X: Ruby, Rails, Git, and More in One Download
Engine Yard’s RailsInstaller has long been a popular way to install Rails, Ruby, Git, Sqlite and other tools in one hit on Windows. Now it’s here for OS X too.

Announcing Pow 0.4.0 With xip.io Support
37signals’ Sam Stephenson has unveiled the latest version of the popular Mac OS X zero-configuration web server for Ruby webapp development. Port proxying is one of the new features.

Take My 30 Second Ruby Webcast Survey
I’m planning to run some Ruby oriented ‘webcasts’ (live, 30-60 minute presentations + live coding) and have a handful of ideas. I want to see which are the most popular and would appreciate your help (or even your own ideas).

Early Access to Avdi Grimm’s ‘Confident Ruby’ Available
Avdi Grimm, author of the popular Exceptional Ruby and Objects on Rails, is working on his new book, Confident Ruby. It’s still under development but you can get access now in return for some money or, intriguingly, a postcard.

JewelryBox 1.3 Released: The Official OS X RVM GUI
JewelryBox is a GUI app that brings OS X and RVM together in a loving embrace. 1.3 brings improved usability, gemset management, and Mountain Lion compatibility (it’s a signed app, so no GateKeeper complaints either).

From our sponsor

Michael Hartl’s Rails Tutorial 2nd Edition, Out Now
Michael Hartl has just announced the official release of the 2nd edition of his popular Rails Tutorial screencast series. It’s 15 hours of ‘over the shoulder’ training in Rails 3.2 using Twitter Bootstrap, Ruby 1.9, and full on test driven BDD.

Reading

RSpec’s New Expectation Syntax (Coming in 2.11)
Myron Marston shows off a new syntax for expectations in RSpec. It looks like “expect(foo).to eq(bar)” versus the more typical “foo.should eq(bar)”. What’s the motivation? Learn more here.

Removing config.threadsafe! from Rails
Core Rails developer Aaron ‘tenderlove’ Patterson looks at why the ‘config.threadsafe!’ directive can be removed from Rails and demonstrates the issues involved.

Rails Pull Request: Enable threadsafe! By Default
A 2 character pull request that triggered a big discussion about the role of ‘threadsafe!’ in future Rails apps, best summarized by the Aaron Patterson article linked above.

Setting Up an Ubuntu 12.04 Rails Development Environment
Includes Postgres, RVM, and some handy bash aliases.

Forgotten Rails Features: HTTP Streaming
One of a series of posts looking at interesting Rails features that are no longer in the spotlight. Here, Robert May looks at the HTTP streaming features introduced in Rails 3.1.

Using vim-slime with Pry for REPL Perfection
Alan MacDougall demonstrates how you can set up vim so that you can send Ruby code straight from the editor into an interactive Pry session.

A Write Up of the SQL Injection Vulnerability in Rails
Louis Nyffenegger gives a scrappy yet practical explanation of the latest vulnerability in Rails (which has been fixed by Rails 3.2.4/5).

PhoneGap vs RubyMotion
A somewhat chalk and cheese comparison framework wise but a reasonable discussion of the pros and cons of deploying a similar app using both technologies. An interesting sidenote is it compelled one of the creators of PhoneGap to write a post in response.

Future Versions of Pow Won’t Automatically Support RVM
It seems automatic support for .rvmrc files to set the Ruby environment is going away in Pow. There’s a simple workaround (using the .powrc file for each project) but it’s worth noting for the future.

Ruby Is Faster Than Python, PHP and Perl(?)
Uh-oh, controversy alert! Tony Arcieri attacks the myth that Ruby is the slowest language out there with a look at the Alioth Programming Languages Shootout. Sadly, the shootout is rather flawed but the real take away is that all four languages aren’t far apart.

Watching and Listening

RailsCasts on Brakeman: The Rails Vulnerability Scanner
Ryan Bates looks at the Brakeman gem, which will scan the Ruby code of a Rails application and alert you to common security vulnerabilities.

Ruby Rogues 59: Security with Rein Henrichs
The Rogues sit down with Rein Henrichs to talk about Ruby and Rails app security from password storage and network sniffing through to the GitHub hack and Rails vulnerabilities.

An Intro to Sinatra
At the Pittsburgh Ruby Brigade Language User Group, Carol Nichols gave a 45 minute introduction to Sinatra, the lightweight DSL for building webapps in Ruby.

Creating A Blog Application in Rails 3.2
An hour long screencast by John Ash walking through the development of a blogging webapp in Rails 3.2.

Evented Ruby vs Node.js
A 40 minute talk by Jerry Cheung from RailsConf 2012.

Dangers of Session Hijacking (RailsCasts)
Ryan Bates of RailsCasts takes a look at the dangers of ‘session hijacking’ and shows off a way to improve the situation on a Rails app.

Introducing the ‘Giant Robots Smashing into other Giant Robots’ Podcast
Courtesy of Ben Orenstein and Thoughtbot comes a new development oriented podcast based around Thoughtbot’s popular Ruby and Rails blog.

Matz’s Keynote at Euruko 2012
This year, the annual European Ruby Conference landed in Amsterdam and here’s Matz’s keynote to enjoy on Vimeo.

Geoffrey Grosenbach’s Euruko Keynote
Let the always mellifluous murmurs of PeepCode’s Geoffrey Grosenbach wash over you in this 40 minute keynote.

Using Ruby to Craft and Test Beautiful Command Line Applications
From RubyConf India comes a talk by Shishir Das and Nikhil Mungel about building elegant command line apps with Ruby (on Unix-like systems, naturally). Video good, audio not so good.

Libraries and Code

API Taster: Visually Test Your Rails Application’s API
A handy tool that works out endpoints from your routes file and then lets you perform requests direct from the Web browser. A clever idea.

Rails 2 Asset Pipeline: Familiar Asset Handling for Those On Rails 2
An interesting backport of some of Rails 3.1+’s most interesting functionality by Michael Grosser.

cod: IPC Made Simple
A small library striving to ‘change the way people do IPC in Ruby.’ It abstracts away the toll of using IO.pipe, Sockets and other primitives and allows message oriented communication via a simple interface.

Slacker: Test Framework for SQL Server 2005 and 2008 Programmable Objects
An RSpec-based transacted BDD framework/engine for the automated testing of SQL Server programmable objects (think stored procedures, table/scalar functions, etc.) Runs on both Windows and Linux.

Redis Store: Redis Stores for Ruby Frameworks
Redis Store provides a full set of stores (Cache, I18n, Session, HTTP Cache) for all the modern Ruby frameworks like: Ruby on Rails, Sinatra, Rack, Rack::Cache and I18n. It natively supports object marshalling, timeouts, single or multiple nodes and namespaces.

Talks: Let Your Ruby Programs Talk To You
A gem that hooks into the text to speech tools available on both Mac OS X and Linux.

Savon 1.0 Released: ‘Heavy Metal’ SOAP Client
Okay, SOAP might not be the big deal it once was, but many APIs and enterprise technologies still use it, and Savon, even pre-1.0, has been a popular choice for using SOAP from Ruby.

BubbleWrap: Cocoa Wrappers and Helpers for RubyMotion
BubbleWrap is a collection of (tested) helpers and wrappers used to wrap CocoaTouch code and provide more Ruby-like APIs.

Sinew: Collect Structured Data From Web Sites (by Scraping)
Sinew collects structured data from the Web (using scraping techniques) via a Ruby DSL built for crawling, a robust caching system, and integration with Nokogiri.

Announcing Virtus 0.5.0 (Part of DataMapper 2 Is Done)
Virtus is a key part of the forthcoming DataMapper 2 which provides attributes for plain Ruby objects. You can now use Virtus in modules, dynamically extend objects on the fly, and Structs can be used as an embedded value in other objects.

Doodle: Extended Attribute Accessors for Objects
A Ruby gem for creating extended attribute accessors with defaults, conversions and validations (along similar lines to last week’s ‘SmartProperties’ item).

nanoc 3.4 Released
The popular Web site generation / publishing tool (think a more flexible Jekyll) gets an update.

Markdown Engine Wrapper
An abstraction and wrapper library that gives access to several different Ruby Markdown libraries both in Ruby and via a command line app called ‘markdown’.

Mina: Fast Deployment and Server Automation Tool
Mina lets you build and run scripts to manage your app deployments on servers via SSH. Ruby based and framework/project agnostic.

Sidekiq 2.0 Released, Gets Scheduled Jobs
Sidekiq is a simple but efficient background job processor for Ruby (and Rails.) It’s designed to be Resque-compatible but is faster as many jobs can execute in parallel. Version 2 adds scheduled jobs so you can say ‘send this email 3 days from now.’

Parts of RubyMotion Open Sourced
RubyMotion is a Ruby compiler/framework for iOS that has recently taken the Ruby world by storm. Its creator, Laurent Sansonetti, has now open sourced parts (but not all) of the framework so that the community can contribute to and extend the platform.

IProcess: Transport Ruby Objects Between Processes
IProcess, short for Inter Process Communication(IPC) Process, is a collection of classes you can use to transport Ruby objects between processes running on UNIX-like operating systems.

SmartProperties: Ruby Accessors ‘on Steroids’
Bored of plain old attr_accessors? Include the SmartProperties module and use the property method along with a name and optional configuration parameters to define new properties along with input conversion, validation, and default values.

Jobs

Rails Software Engineer at Mulu.me (Los Angeles)
Join a killer engineering team to build out a social shopping application where users earn for a cause. You’ll be working with Ruby, Rails, JavaScript, PostgreSQL, and building a product you’re proud of with an awesome team.

Rails Developer at IBM [Emeryville, California]
As interested in technical verve as years of experience, IBM has positions for junior and senior engineers with a focus on those experienced with Rails, JavaScript, CSS and jQuery.

Head of Tech or Lead Developer – Purpose (New York, NY)
Manage a new technology initiative for an established agency. The project aims to build tools for the “ethical consumer” and the hands-on role includes all aspects of development, management and deployment. Competitive comp + stock options.

RoR Developers at Quri (San Francisco, CA)
Quri, a San Francisco-based mobile and web startup that’s transforming the $350 billion trade marketing industry, is seeking an experienced Ruby on Rails engineer to join our growing team.

Web Developer – Threespot (Washington DC)
Responsible for more than development, testing, & documentation of web-based software & technical solutions, our Developers are involved in all stages of client engagements, from application design through specifications documentation & production programming. (Ruby welcome!)

Want to post a job? Click here for more info.

Last but not least..

Why Our Code Smells
A well put together slide deck by Brandon Keepers of GitHub with some interesting points on code smells, primarily focusing on testing.

Ruby Fiddle: Ruby Snippets in the Browser
If you’ve done any JavaScript work, you might be familiar with ‘JSFiddle’ a popular site that previews HTML, JS, and CSS you supply in the browser. Ruby Fiddle aims to do similarly for Ruby, though the code runs remotely.

Cedar is the Default Heroku Stack

The Heroku Cedar stack went public beta last year with a series of blog posts. Since then, over 80,000 developers have deployed over 4.5 million times, to apps written in dozens of different programming languages and frameworks. Today, over 75 percent of Heroku app development activity is on the Cedar stack. Production apps like Banjo, Rapportive, PageLever, do.com, and Project Zebra run on Cedar; some of these serve hundreds of millions or even billions of requests per month.

Cedar features a streamlined HTTP stack allowing for advanced HTTP capabilities, heroku run for execution of arbitrary one-off dynos, Procfile and the process model for execution of any type of worker process. Most importantly, Cedar is a polyglot platform with official support for Clojure, Java, Node.js, Python, Ruby, and Scala, and extensibility for unlimited others via buildpacks.

You can still create applications on one of our other stacks using heroku create --stack, but we recommend Cedar for all new apps. If you have applications under active development running on Aspen or Bamboo, we recommend migrating to Cedar.

Setting Akamai Edge-Control headers with Ruby on Rails

Just a short and sweet little tip.

Several months ago we moved one of our clients over to Akamai&#8217;s Content Delivery Network (CDN). Ww were previously using a combination of Amazon S3 and CloudFront with <em>some</em> benefits, but we were finding several key areas of the world were not s covered by Amazon (yet) for asset delivery. Along with that, we really wanted to take advantage of the <span class="caps">CDN</span> for more of our <span class="caps">HTML</span> content with a lot of complex rules that related to geo-targeting and regionalization of content.


I&#8217;ll try to cover those topics in another post, but wanted to share a few tidbits of code that we are using to manage Akamai&#8217;s Edge-control caches from within our Rails application.


With Akamai, we&#8217;re able to tell their Edge servers whether it should hold on to the response so it can try to avoid an extra request to the origin <div class="post-limited-image"><img src="http://feeds.feedburner.com/~ff/RobbyOnRails?d=7Q72WNTAKBA" border="0"></div>

Continue reading “Setting Akamai Edge-Control headers with Ruby on Rails”

Setting Akamai Edge-Control headers with Ruby on Rails

Just a short and sweet little tip.

Several months ago we moved one of our clients over to Akamai’s Content Delivery Network (CDN). Ww were previously using a combination of Amazon S3 and CloudFront with some benefits, but we were finding several key areas of the world were not s covered by Amazon (yet) for asset delivery. Along with that, we really wanted to take advantage of the CDN for more of our HTML content with a lot of complex rules that related to geo-targeting and regionalization of content.

I’ll try to cover those topics in another post, but wanted to share a few tidbits of code that we are using to manage Akamai’s Edge-control caches from within our Rails application.

With Akamai, we’re able to tell their Edge servers whether it should hold on to the response so it can try to avoid an extra request to the origin (aka our Rails application). From Rails, we just added a few helper methods to our controllers so that we can litter our application with various expiration times.

  # Sets the headers for Akamai
  # acceptable formats include:
  #   1m, 10m, 90m, 2h, 5d
  def set_cache_control_for(maxage="20m")
    headers['Edge-control'] = "!no-store, max-age=#{maxage}"
  end

This allows us to do things like:

  class ProductsController < ApplicationController
    def show
      set_cache_control_for('4h')
      @product = Product.find(params[:id])
    end
  end

Then when Akamai gets a request for http://domain.com/products/20-foo-bar, it’ll try to keep a cached copy around for four hours before it hits our server again.

#358 Brakeman

The Brakeman gem will scan the Ruby code of a Rails application and alert you to common security vulnerabilities.

Rails 3.2.5

After a brief hiatus, Peter and Jason discuss the new releases of Rails, JRuby 1.7.0, RailsInstaller for OS X, and more.

The Mega Ruby News and Release Roundup for May 2012

Welcome to the bumper pick’n’mix of Ruby and Rails news and releases for May 2012, fresh from the pages of Ruby Weekly (which, unsurprisingly, comes out once every week – on Thursdays).

Highlights include: RubyMotion’s release, the JRuby guys are moving to Red Hat, Spree 1.1, DHH’s RailsConf keynote, and JRuby 1.7.0 preview 1.

Headlines

RubyMotion Released: It’s Ruby, but for iOS
The biggest Ruby news of the month! RubyMotion is a Ruby implementation and toolchain for iOS by Laurent Sansonetti, the former MacRuby lead developer, that lets you build iPhone and iPad apps.

DHH’s RailsConf 2012 Keynote (YouTube)
DHH rails against conservatism, people who think about the newbies, and ex-hippies, while celebrating progress and getting the audience to chant “I will not fear change, I will not fight progress”. “Stay hippie,” he closes.

Spree 1.1 (Rails E-commerce Platform) Released
Spree is a popular Rails-based e-commerce system and the 1.1.0 release brings support for Rails 3.2, an API overhaul, and more.

Read A Chapter From ‘Ruby Under a Microscope’, A Forthcoming E-Book
Prolific Ruby blogger Pat Shaughnessy has been working on a new e-book about Ruby language and MRI internals. It’s not finished yet but he’s sharing a single preview chapter for free right now. Enjoy.

Everyday Rails Testing with RSpec by Aaron Sumner
A new e-book written by Aaron Sumner that digs into building confidence with testing Rails apps using RSpec. It comes in PDF, EPUB and Mobi formats. Always handy to have up to date efforts in this regard.

OpenShift (Red Hat’s Answer to Heroku?)
OpenShift is Red Hat’s new, free auto-scaling ‘platform as a service’ for webapps. It supports Java, Ruby, Node, Python, PHP and Perl and seems to be a reasonable entry into a world made popular by Ruby’s own Heroku.

Startups Court Dev Bootcamp’s Ruby Grads with $79K Average Salary
A rare human interest story for Ruby Weekly! Dev Bootcamp is a 10 week $12K bootcamp program for people to get up to speed with Rails development and of their latest batch, 14 of 17 developers ready to get a job got an offer at an average salary of $79K.

The Bastards Book of Ruby
Billing itself as ‘a programming primer for counting and other unconventional tasks,’ the online Bastards Book teaches Ruby programming via practical data processing tasks. It’s aimed at non-programmers and especially journalists, researchers, scientists, etc.

JRuby Core Team Members Moving to Red Hat
At JRubyConf this week, it was revealed that Thomas Enebo and Charles Nutter – key members of the JRuby core team – are moving from Engine Yard to Red Hat. Learn more about what they’ll be up to here.

First Alpha of ruby_parser 3.0 Released
ruby_parser (RP) is a long standing Ruby parser written in pure Ruby but till now it had the disadvantage of not supporting Ruby 1.9 parsing. Now it does! Still only an alpha but a good release to see.

MRI Accepts Google Patch for Google NativeClient (NaCl) Support
Still early days on this but Google has been working on a proof of concept port of Ruby to its NativeClient technology (which lets you compile software to run in Chrome).

Vermont’s First Ruby Conference, July 28 – 29, 2012
Burlington Ruby Conf 2012 will take place in Burlington, Vermont on July 28 – 29th. Join like-minded Ruby enthusiasts for Vermont’s first Ruby conference, hosted on beautiful Lake Champlain.

JRuby 1.7.0 Preview 1 Released
1.7 is proving to be a big release for JRuby and the first preview is out. A key change is that Ruby 1.9.3 is now the default runtime mode (but 1.8 is still available as an option) and support for Java 7’s invokedynamic brings some major performance boosts (when running on the right JVM.)

Heroku’s ‘Cedar’ Stack Now in General Availability
The popular ‘Cedar’ stack on Heroku that allows you to run multiple processes, multiple languages, and more is now out of beta and is ‘generally available’.

Reading

What Makes an Awesome Command-line Application?
David Copeland, the author of Build Awesome Command-Line Applications with Ruby, shares some pointers to building good command line apps in Ruby.

Ruby HTTP Client Performance Shootout Redux
Jonathan Rochkind puts several HTTP client libraries through their paces. Eric Hodel, author of net-http-persistent, gives the write up a thumbs up so that’s good enough for me.

Exhibit vs Presenter
Mike Pack explores the ‘exhibit’ and ‘presenter’ decorator patterns. And if all this sounds like buzzword bingo to you, this is a pretty accessible introduction!

A Vision for Cucumber 2.0
Cucumber is a popular natural language accepting testing toolkit for Ruby and in this post Matt Wynne has collected together ideas from various Cucumber maintainers to see what’s coming in Cucumber’s future.

Schemaless SQL
The slides from a talk from Railsconf 2012 by Heroku developer Will Leinweber who shows off some powerful new features in Postgresql for working with schemaless data as well as JSON datatypes and embedded JavaScript.

Meet 6 Misunderstood Ruby Features
Targeting C++ developers who are learning Ruby, Arpan Sen looks at six areas of Ruby that might trip them up: class hierarchy, singleton methods, self, method_missing, exception handling, and threading. If you already know your stuff though, steer clear.

Multiple Ruby Version Support on Heroku
With the release of Bundler 1.2.0 beta and the Ruby ‘build pack’ you can now select the version of Ruby you want to run on Heroku. Learn more here.

Mixins: A Refactoring Anti-Pattern
An interesting stroll through measuring complexity and refactoring with Steve Klabnik.

Five Common Rails Mistakes
Experienced Rails developer Mike Perham shares a handful of common mistakes he sees in most Rails apps.

How To Get 4x the Performance Out of Heroku with Unicorn
A look at using Unicorn and a Procfile to increase the number of processes that are handling Web requests when using Heroku.

Getting Started with Ember.js and Ruby on Rails
Fresh from the official Ember site comes a guide to building a simple, personal photoblog application using Ember.js and Rails 3.2.

Benchmark Your Bundle
See how long it takes to load each individual library referenced by your Gemfile.

Passing Open Files Between Processes with UNIX Sockets
Jesse Storimer shares a neat technique for using UNIX sockets as a conduit to passing file descriptors between processes.

Backbone.js, CoffeeScript, Jasmine, HAML and Rails Working Together
A holistic walkthrough by Jeremy Walker tying together a bunch of modern Web development technologies.

A Better Way to Set Up Rails on Windows
Don’t flame me about this, but what’s the best way to get a Rails environment running on Windows for development work? Use VirtualBox and a VM says Roshan Choxi of programming training company Bloc.

Three Quick Rails Console Tips from 37signals
Dig into your app with the ‘app’ method, play with helpers, and find out where certain methods are defined (more a general 1.9 trick though).

Anaphora in Ruby, 2012 Edition
Esteemed essayist Reg Braithwaite updates a popular 2009 article about ‘anaphora’ in Ruby, namely language elements that can refer back to earlier used objects or results. A lot of geekery to enjoy here.

Integrating Rails and jQuery Mobile
There are some rough edges when integrating Ruby on Rails and jQuery Mobile for your mobile application and jQuery Mobile core contributor John Bender shows you how to smooth these out.

Objectify: A Better Way to Build Rails Applications
Objectify is a framework that codifies good object oriented design practices for building maintainable Rails applications. In this post, James Golick explains the motivations.

You Don’t Have to Use Bundler to Create New RubyGems
Not a new or provocative opinion, but nonetheless an interesting introduction to the ‘Ore’ project generator which I didn’t know about before.

Injection Vulnerabilities in Ruby Apps Through Bad Regexes
A pretty simple issue but one that I suspect is commonplace.

Tossing Out IRB for Pry
Pry is a powerful and popular alternative to the IRB Ruby console. In this post, Phil Aquilina shows off a few reasons why to give it a go.

CoffeeScript Love: Testing CoffeeScript in Rails
MochaRails is a mountable Rails engine that serves a browser-based Mocha test suite, along with your development JavaScript files, via the Asset Pipeline.

The Development of Sequel
A slidedeck by Jeremy Evans about his work on the popular database interaction library.

Why Our Code Smells
A great slide deck by Brandon Keepers.

Watching and Listening

The Rails API Gem – RailsCasts
Ryan Bates looks at the ‘Rails API’ gem (recently linked in Ruby Weekly) and demonstrates how it can be used to build an API-only Rails app.

10 Minutes on Rails Engines
Ryan Bigg presents a well-recorded ten minute tour of Rails engines, what they are, how they work in different versions of Rails, how to build them, and examples of their usage in Forem and Spree.

Sandboxing Ruby Code: Lessons from the Battlefield
A talk by Tejas Dinkar and Jasim Basheer from RubyConf India 2012 looking at the process of running untrusted code on your server using various sandboxing techniques.

rbenv and Bundler
An hour long talk from the Pittsburgh Ruby Brigade presenting a history of rbenv and Bundler, debating the merits of rbenv and RVM, and showing off rbenv-bundler, a new plugin which ties rbenv and Bundler together.

Making Little Classes Out of Big Ones with Avdi Grimm
In a ‘Lunch n’ Learn’ session with Avdi Grimm, he takes a look at the pros and cons of four different techniques for breaking a too-big class into smaller pieces. 50 minutes long.

RubyMotion with Laurent Sansonetti
As Ruby Weekly may have shown, the RubyMotion Ruby + iOS developer tools have been a big hit lately and their creator Laurent Sansonetti dropped in to chat with the Ruby Rogues about what it all means.

RailsCasts on ‘Squeel’ – a SQL DSL
Squeel provides a comprehensive DSL for writing SQL queries in Ruby, all built upon Arel. Ryan Bates gives us a tour in just 9 minutes.

Libraries and Code

Focused Controller: Bringing Real OOP to Rails Controllers
Jon Leighton notes that classical Rails controllers violate the Single Responsibility Principle (part of the SOLID group of OO programming principles). Focused Controller aims to resolve that by splitting up responsibilities into an object each. The usage example shows it off best.

Initial Queue Implementation on Edge Rails
It seems Rails 4 is getting a built-in unified API for queueing! Enjoy this GitHub commit jam packed with comments and discussion on the development of this feature.

Sublime Text 2 Guard Plugin
This project provides integration of Guard (a tool that handles and responds to events on file system modifications) into the Sublime Text 2 editor. Handy if you want your tests to run automatically but then see the results directly in ST2.

Thor: Toolkit for Building Command Line Interfaces
Thor is not a new tool but it has a new Web site so check it out. It makes an interesting alternative to Rake for certain use cases (although ultimately Rake and Thor should serve different purposes).

Firehose: Build Real-Time Web Applications
Firehose is both a Rack application and JavaScript library that makes building scalable real-time web applications easy. You’ll need to dig in a bit though as it’s a little buzzwordy on the surface.

Kandan: A Slick, Rails-Powered Open Source Live Chat App
A fan of online chat apps like Talker or Campfire? Kandan is an open source Rails app that claims to be ‘the slickest chat app out there!’ I had it up and running on Heroku within a couple of minutes.

DataMiner: Download, Parse and Import Many Data Formats into AR Models
DataMiner by Seamus Abshere is a Ruby library that can download data archives, extract them, parse the data contained, and then import that data into ActiveRecord models.

Monologue: Mountable Rails Blogging Engine
Monologue is a basic mountable blogging engine in Rails built to be easily mounted in an already existing Rails app, but it can also be used alone.

Mechanize 2.5 Released
The popular Web request and Web site interaction library has been updated.

Ampex: A(nother) Practical Use Of Ruby’s & Operator
The ampex library takes the idea from ‘&:symbol’, and adds a little more flexibility. An interesting idea.

Introducing Live for RubyMotion
Live is a gem for RubyMotion implementing some of the ideas presented in Bret Victor’s awesome ‘Inventing on Principle’ talk. It interfaces with the REPL and allows you to control it from the comfort of your preferred text editor.

ActiveRecord Reputation System (from Twitter)
The Reputation System gem makes it easy to integrate ‘reputation systems’ into Rails applications and decouple the system from the main application.

rake-rails: Bringing Rails Commands to Rake
Ever get confused between which things you should use ‘rake’ for and which you should use the ‘rails’ command for? Andy Lindeman brings them both together with rake-rails.

CLAWS: Command Line AWS
A command line based AWS (Amazon Web Services) console with Capistrano integration.

Sextant: Bringing Your Rails Routes to a View
Sextant is a simple addon to give you a /rails/routes URL while in development on a Rails app, as a way to see the routes your app implements. Interestingly, this functionality is being merged into the forthcoming Rails 4.0 as standard.

Jobs

Senior Developer at AT&T [Palo Alto, California]
An opportunity to join a new and rapidly growing group within AT&T called ‘The Foundry’ that works at the cutting edge of new technologies and turns them into new opportunities for AT&T. A love of programming and proficiency with technologies like Ruby, JavaScript, and HTML5 is essential.

Senior Software Engineer at Zendesk [San Francisco]
Popular hosted helpdesk service Zendesk is looking for a senior engineer with 2+ years of Rails experience.

Senior Web Technologist at Holiday Extras UK
Do you spend your leisure time tinkering with the latest NoSQL du jour or playing with the latest AWS toy? Do you dream of big data? Is GitHub the first thing you think of when someone says the word fork? If so, Holiday Extras is looking for techies like you.

Upcoming Me

Updates, schedules, things, and stuff.

Scottish Ruby

The Scottish Ruby conference is having a charity workshop June 28, and I’m presenting my “Advanced Rails Design” workshop. This is the extended dance mix version of the workshop I did at Mountain West Ruby earlier this year. I thought it went really well (so did the attendees, I’m sure), and I’m very excited about this one. Details at http://scottishrubyconference.com/charity/ — you don’t need to be attending Scottish Ruby, but you do need to register in advance.

There are three workshops that day — a UX workshop that has sold out. A Dave Thomas advance Ruby workshop that hasn’t, and mine. Let’s just say there are more tickets available right now for mine than I’d like, and I hope that if you are in the neighborhood, you’ll stop by. It’ll be worth it.

Windy City Rails

Much closer to home, I’m speaking at Windy City Rails this year. According to their schedule, my talk will be “Let’s Make Testing Fun Again”. This conference is always great, the venue this year looks outstanding, and the speaker list is — myself excluded — top-notch. Hope to see you there.

Ignite Rails

My IgniteRailsConf talk: Manage Your Development Environment / Never Burn Another Burger by Noel Rappin is now available on line at http://www.youtube.com/watch?v=VLCLBdFsSOE. I don’t think my other RailsConf thing is up yet, but I’m sure I’ll let you know.

Master Space and Time With JavaScript

It’s near schedule. I think that converting all the text for the first two parts of the book will be done next week. Leaving me with a) a serious edit b) cover and incidental design, c) cleanup for epub and mobi, and d) product sale logistics. Still hoping that’ll go out in early July. If you’ve signed up at http://www.noelrappin.com/mstjs-form, you’ll probably get an early look/chance to help me work the kinks out.

And Hey,

You can still buy Rails Test Prescriptions. Much of it is still up-to-date…

Upcoming Me

Updates, schedules, things, and stuff.

Scottish Ruby

The Scottish Ruby conference is having a charity workshop June 28, and I’m presenting my “Advanced Rails Design” workshop. This is the extended dance mix version of the workshop I did at Mountain West Ruby earlier this year. I thought it went really well (so did the attendees, I’m sure), and I’m very excited about this one. Details at http://scottishrubyconference.com/charity/ — you don’t need to be attending Scottish Ruby, but you do need to register in advance.

There are three workshops that day — a UX workshop that has sold out. A Dave Thomas advance Ruby workshop that hasn’t, and mine. Let’s just say there are more tickets available right now for mine than I’d like, and I hope that if you are in the neighborhood, you’ll stop by. It’ll be worth it.

Windy City Rails

Much closer to home, I’m speaking at Windy City Rails this year. According to their schedule, my talk will be “Let’s Make Testing Fun Again”. This conference is always great, the venue this year looks outstanding, and the speaker list is — myself excluded — top-notch. Hope to see you there.

Ignite Rails

My IgniteRailsConf talk: Manage Your Development Environment / Never Burn Another Burger by Noel Rappin is now available on line at http://www.youtube.com/watch?v=VLCLBdFsSOE. I don’t think my other RailsConf thing is up yet, but I’m sure I’ll let you know.

Master Space and Time With JavaScript

It’s near schedule. I think that converting all the text for the first two parts of the book will be done next week. Leaving me with a) a serious edit b) cover and incidental design, c) cleanup for epub and mobi, and d) product sale logistics. Still hoping that’ll go out in early July. If you’ve signed up at http://www.noelrappin.com/mstjs-form, you’ll probably get an early look/chance to help me work the kinks out.

And Hey,

You can still buy Rails Test Prescriptions. Much of it is still up-to-date…

#356 Dangers of Session Hijacking

If a user’s authentication cookie is sent over an insecure connection it is vulnerable to session hijacking, or more specifically, sidejacking. Learn how this is done, and how you can prevent it.

Automator + Bash = Yay

At it’s best, working in Mac OS X combines the power of the Unix shell with the convenience of an actual interface.

Here’s a best case scenario:

As I may have mentioned here a few times, I’m writing a book. As part of my current workflow, I need to convert my text from it’s old format to my new format, which is Markdown. The old format is a custom XML-based language the details of which don’t matter beyond the fact that it’s XML-based.

Moving the text over has two issues:

  • The obvious one is that there are XML tags in the body of the text for things like code literals and text italics that I want to replace with either the Markdown backtick or the Markdown underscore.
  • The less obvious one is that when writing XML text, I treat it like code, meaning I’m ridiculously insane about text layout. In my XML source, I really do still insist on an 80-character line, with hard returns and indentation. I’ve decided that I don’t need to do this when I write in Markdown, so when I move the text over, I need to get rid of all the hard returns and spacing.

What I’d like is a workflow where I can copy a paragraph of text to the clipboard, do magic, and then paste cleaned-up text in the new book file. Going a paragraph at a time is not a problem — it’s actually preferable, since I’m editing as I go, so moving a whole file at a time is not really what I want.

I toyed with the idea of doing this in Ruby, but that seemed like a pain, so I wrote a short shell script.

Understand: I never do this, and I pretty much know beans about the shell programs involved. But by googling things like “shell remove newlines” and with some helpful man pages, I cobbled together the following, with details of the XML fuzzed a bit.

    pbpaste | tr -d '\n' | tr -s ' ' | 
    sed -E 's/<\/*(literal|lit)>/\`/g' | 
    sed -E 's/<\/*(italic|bold)>/\*/g' | pbcopy

All of you who actually know shell scripting are invited to have a hearty laugh. While you are off chuckling, I’ll explain what this line noise does…

  • pbpaste takes the text from the clipboard, which is piped to…
  • tr -d '\n', which removes all the newlines, and pipes to…
  • tr -s ' ', which removes duplicate spaces, and pipes to…
  • sed -E 's/<\/*(literal|lit)>/\/g’`, which takes all the XML tags for things that I want replaced with Markdown literal syntax and puts in a back tick, then pipes to…
  • sed -E 's/<\/*(italic|bold)>/\*/g', takes the XML tags I want to replace with Markdown emphasis syntax, then pipes to…
  • pbcopy, which copies the final text back into the clipboard

Works great. A bit of a mouthful to type, if I can mix metaphors. (The actual one is even more complicated, because I strip out some XML entities as well). It’s a bit much to type at the terminal in my workflow. Creating an alias is a possibility, but still, requires a terminal.

There’s another option in Mac OS X, though: Automator.

You may not have played with Automator, because you do not fully appreciate Automator. Automator is an OS X application that lets you chain together predefined actions (very similar to the actions exposed via apple script), using a GUI interface, and save the result as an application or an OS X service, among other options.

This isn’t an Automator tutorial, because what I want to do here is really simple. One of the available actions in Automator is “Run Shell Script”.

Hmm…

I created a new Automator document as a Service, added the Run Shell Script action to it, pasted my big shell script into the body of the action. The action doesn’t need input. Even better, I can have it work not on the clipboard, but on the selected text in the open application, which saves me a step in my workflow. The shell script is already putting the output on the clipboard, so I don’t need to deal with that in automator either.

Okay, big deal, it runs a shell script. But, since I’ve saved it as a service, I can assign a global key command to it. In the System Preferences for Keyboard, my new service is somewhere in the Keyboard Shortcuts tab (under services). From there, I can assign a keyboard shortcut, which is available in any application that exposes itself to services, which is many applications.

Now I just select a paragraph of old text in my text editor, hit my new key combination, and I can paste the cleaned up text in my new editor window. A little Unix, a little Mac, and a lot of time saved.

June PragPub Magazine

June PragPub Magazine now available

Tokaido Status Update: Implementation Details

Hey guys!

Since my last update, Tokaido was fully funded, and I’ve been hard at work planning, researching and working on Tokaido.

So far, we have a working binary build of Ruby, but no setup chrome. Because the binary build already exists, Terence Lee was able to experiment with it at a recent Rails Girls event, with great success:

Great thanks to Terence to put together a simple installer script that we could use to test whether the core build worked on a wide variety of OSX systems.

One thing that I mentioned in my original proposal was a desire to work closely with others working on related projects. Very soon after my project was announced, I teamed up with Michal Papis of the rvm team to make the core statically compiled distribution something that would work outside of the GUI part of Tokaido.

We decided to use the sm scripting framework to build Tokaido, to make it easy to share code between rvm2, Tokaido, and the Unix Rails Installer. The majority of the work I have done so far has been in researching how to properly build a portable Ruby, and working with Michal to build the solution in terms of the sm framework. The rest of this blog post discusses the results of that research, for those interested.

The discussion in this blog post is specific to Mac OSX.

Portable Build

The hardest technical part of the project is creating a portable binary build of Ruby that can be moved around to various machines. What do I mean by that?

When you compile Ruby using the normal ./configure && make, the resulting binary is not portable between machines for a number of reasons.

Hard-Coded Paths

By default, the compiled Ruby comes with a binary .bundle file for each compiled part of the standard library. For example, Aaron Patterson wrote the psych library as a C library. When you compile Ruby, you get psych.bundle as part of the distribution. When a Ruby program invokes require "psych", the system’s dynamic loader will load in psych.bundle.

By default, Ruby hard-codes the path to the Ruby dynamic library (libruby.1.9.1.dylib) into psych.bundle. Since Psych uses C functions from Ruby, this path is used by the dynamic loader to make sure that the require Ruby dependency is available. We can use a tool called otool to see all of the dependencies for a particular library:

$ otool -L psych.bundle 
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.3.0/psych.bundle:
    /Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/libruby.1.9.1.dylib (compatibility version 1.9.1, current version 1.9.1)
    /Users/wycats/.rvm/usr/lib/libyaml-0.2.dylib (compatibility version 3.0.0, current version 3.2.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

The second line in the output references the libruby.1.9.1.dylib using an absolute path on my local machine. If I take these binaries and give them to you, the linker won’t be able to find libruby and won’t load Psych.

In addition to the problem with the compiled .bundle files, the compiler also hardcodes the paths in the Ruby binary itself:

$ otool -L `which ruby`
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/bin/ruby:
    /Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/libruby.1.9.1.dylib (compatibility version 1.9.1, current version 1.9.1)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

Finally, the location of the standard library is hardcoded into the Ruby binary:

$ strings /Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/libruby.1.9.1.dylib | grep rvm
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/x86_64-darwin11.3.0
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/vendor_ruby/1.9.1
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/vendor_ruby/1.9.1/x86_64-darwin11.3.0
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/vendor_ruby
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.3.0

Fortunately, the C Ruby folks know about this problem, and include an (undocumented?) flag that you can pass to ./configure, --enable-load-relative. This flag fixes the problems with hardcoded paths:

Instead of creating a separate libruby.1.9.1.dylib that the ruby executable links to, this flag includes the compiled binary code inside of the ruby executable.

$ ./configure --enable-load-relative --prefix=/Users/wycats/Code/ruby/build
... snip ...
$ make && make install
... snip ...
$ otool -L build/bin/ruby                                                  
build/bin/ruby:
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

You can see that Ruby still links against a few system dynamic libraries. These dynamic libraries are extremely stable in OSX, and aren’t a problem for binary distributions.

In order to enable compilation of native extensions, this build of Ruby distributes an archive file instead of a dylib. As we will see later, the OSX linker knows how to automatically handle this.

This flag also affects psych.bundle:

$ otool -L build/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle                      
build/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle:
    /usr/local/lib/libyaml-0.2.dylib (compatibility version 3.0.0, current version 3.2.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

External Dependencies

In addition to the general problem of hardcoded paths, there’s another issue lurking in the above otool output for Psych. After eliminating the hardcoded path to a local Ruby, we are still left with a link to /usr/local/lib/libyaml-0.2.dylib. Unfortunately, libyaml doesn’t come with OSX, so if I take this distribution of Ruby and hand it off to a fresh system, Psych will fail to find libyaml at runtime and fail to load.

A number of the .bundle files that Ruby ships with have similar external dependencies. In general, these dependencies ship with OSX, but some, like openssl, may not last more than another release or two. In addition, the specific versions of these dependencies shipped with OSX may change over time, possibly resulting in different behavior on different systems.

In general, we can eliminate these problems by including the binaries we need into the .bundle files, instead of asking the operating system’s dynamic loader to find them at runtime.

The OSX linker’s (ld) behavior in this respect is interesting:

  • The linker starts with a list of paths to search for libraries
  • When compiling a program, it may need a particular dependency (psych needs libyaml)
  • It searches through the path for that library. Both libyaml.dylib and libyaml.a will suffice.
  • If the linker finds a .dylib first, it will dynamically link that dependency.
  • If the linker finds a .a first, it will statically link that dependency. By statically linking, we mean that it simply includes the binary into the outputted compiled file
  • If the linker finds a directory containing both a .a and a .dylib, it will dynamically link the dependency

In this case, our goal is to get the linker to statically link libyaml. In order to do this, we will need to build a libyaml.a and get the directory containing that file to the front of the linker’s path.

In the case of libyaml, getting a .a looks like this:

$ wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
... snip ...
$ tar -xzf yaml-0.1.4.tar.gz
$ cd yaml-0.1.4 
$ ./configure --disable-shared
... snip ...
$ make
... snip ...
$ otool -L src/.libs/libyaml.a
Archive : src/.libs/libyaml.a
src/.libs/libyaml.a(api.o):
src/.libs/libyaml.a(reader.o):
src/.libs/libyaml.a(scanner.o):
src/.libs/libyaml.a(parser.o):
src/.libs/libyaml.a(loader.o):
src/.libs/libyaml.a(writer.o):
src/.libs/libyaml.a(emitter.o):
src/.libs/libyaml.a(dumper.o):

We now have a libyaml.a. Note that the configure flag for getting a .a for a given library is not particularly standardized. Three popular ones: --static, --enable-static, --disable-shared.

Next, we need to move libyaml.a into a directory with any other .a files we want to use and pass them to the compilation process:

$ LDFLAGS="-L/Users/wycats/Code/ruby/deps" ./configure --enable-load-relative --prefix=/Users/wycats/Code/ruby/build
... snip ...
$ make && make install
... snip ...
$ otool -L build/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundlebuild/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle:
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

And voila! We now have a psych.bundle that does not depend on libyaml.dylib. Instead, libyaml is now included in psych.bundle itself. This moves us a step closer to having a portable Ruby build.

We will want to repeat this process for every part of the Ruby standard library with external dependencies (openssl, readline, and zlib are some others). Even though libyaml is the only library that does not ship with OSX, eliminating external dependencies on the operating system insulates our build from changes that Apple makes in the future. Of the dependencies, OpenSSL is the most problematic, as it has already been deprecated in Lion.

The sm Framework

This is where the sm (scripting management) framework comes into play. The goal of the sm framework is to encapsulate solutions to these concerns into reusable libraries. In particular, it abstracts the idea of downloading and compiling a package, and common requirements, like static compilation.

For example, let’s take a look at the libyaml library.

The first important file here is config/defaults:

version=0.1.4
base_url=http://pyyaml.org/download/libyaml
configure_flag_static=--disable-shared

This specifies the current version, the URL to download the tarball from, and importantly for us, the configure flag that libyaml expects in order to build a .a file. We added that third line because we needed it for Tokaido. This satisfies one of the major goals of the project: to get as much of the code as possible into shared code instead of code that is specific to Tokaido.

The other important file in the libyaml library is shell/functions:

#!/bin/sh
 
libyaml_prefetch()
{
  package define \
    file "yaml-${package_version}.${archive_format}" \
    dir "yaml-${package_version}"
}
 
libyaml_preconfigure()
{
  os is darwin || autoreconf -is --force > autoreconf.log 2>&1 ||
    __sm.package.error "Autoreconf of ${package_name} ${package_version} failed! " "$PWD/autoreconf.log"
}

The sm framework defines a series of steps that a package install goes through:

# preinstall
#
#   prefetch
# fetch
#   postgetch
#   preextract
# extract
#   prepatch
# patch
#   preconfigure
# configure
#   postconfigure
#   prebuild
# build
#   preinstall
# install
#   preactivate
# activate
#   postactivate
#
# postinstall

The indented functions above are user-defined. Functions like fetch and configure are defined by sm.

In our case, the libyaml library defines two of those steps: prefetch and preconfigure. The prefetch function allows us to provide extra information to the fetch method, which specifically allows the prefetch to override the package_file (${package_file:="${package_name}-${package_version}.${archive_format}"}). In our case, even though the package name is libyaml, we want to download the file yaml-1.1.4.tar.gz.

The openssl library is somewhat more complicated. As with libyaml, we needed to teach sm how to install openssl statically. You can check out the commit to see how easy that was.

The great thing about getting this stuff into sm is that there is now a shared resource to answer questions like “how do you statically build openssl”. The work I did with Michal to improve the information for the libaries that Ruby depends on can now be used by anyone else trying to build Ruby (or anything else with those dependencies for that matter).

Tokaido is an sm library

Tokaido itself is an sm library! This means that the core Tokaido build will work on Linux, so it can be used to create a standalone distribution of Ruby for Linux, and maybe even the core of a Tokaido for Linux!

The Tokaido package implements a lot of the sm hooks, so check it out to learn more about what you can do using sm’s package API.

In my next post, I’ll talk about the architecture of the Tokaido UI component.

Tokaido Status Update: Implementation Details

Hey guys!

Since my last update, Tokaido was fully funded, and I’ve been hard at work planning, researching and working on Tokaido.

So far, we have a working binary build of Ruby, but no setup chrome. Because the binary build already exists, Terence Lee was able to experiment with it at a recent Rails Girls event, with great success:

Great thanks to Terence to put together a simple installer script that we could use to test whether the core build worked on a wide variety of OSX systems.

One thing that I mentioned in my original proposal was a desire to work closely with others working on related projects. Very soon after my project was announced, I teamed up with Michal Papis of the rvm team to make the core statically compiled distribution something that would work outside of the GUI part of Tokaido.

We decided to use the sm scripting framework to build Tokaido, to make it easy to share code between rvm2, Tokaido, and the Unix Rails Installer. The majority of the work I have done so far has been in researching how to properly build a portable Ruby, and working with Michal to build the solution in terms of the sm framework. The rest of this blog post discusses the results of that research, for those interested.

The discussion in this blog post is specific to Mac OSX.

Portable Build

The hardest technical part of the project is creating a portable binary build of Ruby that can be moved around to various machines. What do I mean by that?

When you compile Ruby using the normal ./configure && make, the resulting binary is not portable between machines for a number of reasons.

Hard-Coded Paths

By default, the compiled Ruby comes with a binary .bundle file for each compiled part of the standard library. For example, Aaron Patterson wrote the psych library as a C library. When you compile Ruby, you get psych.bundle as part of the distribution. When a Ruby program invokes require "psych", the system’s dynamic loader will load in psych.bundle.

By default, Ruby hard-codes the path to the Ruby dynamic library (libruby.1.9.1.dylib) into psych.bundle. Since Psych uses C functions from Ruby, this path is used by the dynamic loader to make sure that the require Ruby dependency is available. We can use a tool called otool to see all of the dependencies for a particular library:

$ otool -L psych.bundle 
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.3.0/psych.bundle:
	/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/libruby.1.9.1.dylib (compatibility version 1.9.1, current version 1.9.1)
	/Users/wycats/.rvm/usr/lib/libyaml-0.2.dylib (compatibility version 3.0.0, current version 3.2.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

The second line in the output references the libruby.1.9.1.dylib using an absolute path on my local machine. If I take these binaries and give them to you, the linker won’t be able to find libruby and won’t load Psych.

In addition to the problem with the compiled .bundle files, the compiler also hardcodes the paths in the Ruby binary itself:

$ otool -L `which ruby`
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/bin/ruby:
	/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/libruby.1.9.1.dylib (compatibility version 1.9.1, current version 1.9.1)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

Finally, the location of the standard library is hardcoded into the Ruby binary:

$ strings /Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/libruby.1.9.1.dylib | grep rvm
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/x86_64-darwin11.3.0
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/vendor_ruby/1.9.1
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/vendor_ruby/1.9.1/x86_64-darwin11.3.0
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/vendor_ruby
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1
/Users/wycats/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/x86_64-darwin11.3.0

Fortunately, the C Ruby folks know about this problem, and include an (undocumented?) flag that you can pass to ./configure, --enable-load-relative. This flag fixes the problems with hardcoded paths:

Instead of creating a separate libruby.1.9.1.dylib that the ruby executable links to, this flag includes the compiled binary code inside of the ruby executable.

$ ./configure --enable-load-relative --prefix=/Users/wycats/Code/ruby/build
... snip ...
$ make && make install
... snip ...
$ otool -L build/bin/ruby                                                  
build/bin/ruby:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

You can see that Ruby still links against a few system dynamic libraries. These dynamic libraries are extremely stable in OSX, and aren’t a problem for binary distributions.

In order to enable compilation of native extensions, this build of Ruby distributes an archive file instead of a dylib. As we will see later, the OSX linker knows how to automatically handle this.

This flag also affects psych.bundle:

$ otool -L build/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle                      
build/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle:
	/usr/local/lib/libyaml-0.2.dylib (compatibility version 3.0.0, current version 3.2.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

External Dependencies

In addition to the general problem of hardcoded paths, there’s another issue lurking in the above otool output for Psych. After eliminating the hardcoded path to a local Ruby, we are still left with a link to /usr/local/lib/libyaml-0.2.dylib. Unfortunately, libyaml doesn’t come with OSX, so if I take this distribution of Ruby and hand it off to a fresh system, Psych will fail to find libyaml at runtime and fail to load.

A number of the .bundle files that Ruby ships with have similar external dependencies. In general, these dependencies ship with OSX, but some, like openssl, may not last more than another release or two. In addition, the specific versions of these dependencies shipped with OSX may change over time, possibly resulting in different behavior on different systems.

In general, we can eliminate these problems by including the binaries we need into the .bundle files, instead of asking the operating system’s dynamic loader to find them at runtime.

The OSX linker’s (ld) behavior in this respect is interesting:

  • The linker starts with a list of paths to search for libraries
  • When compiling a program, it may need a particular dependency (psych needs libyaml)
  • It searches through the path for that library. Both libyaml.dylib and libyaml.a will suffice.
  • If the linker finds a .dylib first, it will dynamically link that dependency.
  • If the linker finds a .a first, it will statically link that dependency. By statically linking, we mean that it simply includes the binary into the outputted compiled file
  • If the linker finds a directory containing both a .a and a .dylib, it will dynamically link the dependency

In this case, our goal is to get the linker to statically link libyaml. In order to do this, we will need to build a libyaml.a and get the directory containing that file to the front of the linker’s path.

In the case of libyaml, getting a .a looks like this:

$ wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
... snip ...
$ tar -xzf yaml-0.1.4.tar.gz
$ cd yaml-0.1.4 
$ ./configure --disable-shared
... snip ...
$ make
... snip ...
$ otool -L src/.libs/libyaml.a
Archive : src/.libs/libyaml.a
src/.libs/libyaml.a(api.o):
src/.libs/libyaml.a(reader.o):
src/.libs/libyaml.a(scanner.o):
src/.libs/libyaml.a(parser.o):
src/.libs/libyaml.a(loader.o):
src/.libs/libyaml.a(writer.o):
src/.libs/libyaml.a(emitter.o):
src/.libs/libyaml.a(dumper.o):

We now have a libyaml.a. Note that the configure flag for getting a .a for a given library is not particularly standardized. Three popular ones: --static, --enable-static, --disable-shared.

Next, we need to move libyaml.a into a directory with any other .a files we want to use and pass them to the compilation process:

$ LDFLAGS="-L/Users/wycats/Code/ruby/deps" ./configure --enable-load-relative --prefix=/Users/wycats/Code/ruby/build
... snip ...
$ make && make install
... snip ...
$ otool -L build/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundlebuild/lib/ruby/1.9.1/x86_64-darwin11.4.0/psych.bundle:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 159.1.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

And voila! We now have a psych.bundle that does not depend on libyaml.dylib. Instead, libyaml is now included in psych.bundle itself. This moves us a step closer to having a portable Ruby build.

We will want to repeat this process for every part of the Ruby standard library with external dependencies (openssl, readline, and zlib are some others). Even though libyaml is the only library that does not ship with OSX, eliminating external dependencies on the operating system insulates our build from changes that Apple makes in the future. Of the dependencies, OpenSSL is the most problematic, as it has already been deprecated in Lion.

The sm Framework

This is where the sm (scripting management) framework comes into play. The goal of the sm framework is to encapsulate solutions to these concerns into reusable libraries. In particular, it abstracts the idea of downloading and compiling a package, and common requirements, like static compilation.

For example, let’s take a look at the libyaml library.

The first important file here is config/defaults:

version=0.1.4
base_url=http://pyyaml.org/download/libyaml
configure_flag_static=--disable-shared

This specifies the current version, the URL to download the tarball from, and importantly for us, the configure flag that libyaml expects in order to build a .a file. We added that third line because we needed it for Tokaido. This satisfies one of the major goals of the project: to get as much of the code as possible into shared code instead of code that is specific to Tokaido.

The other important file in the libyaml library is shell/functions:

#!/bin/sh
 
libyaml_prefetch()
{
  package define \
    file "yaml-${package_version}.${archive_format}" \
    dir "yaml-${package_version}"
}
 
libyaml_preconfigure()
{
  os is darwin || autoreconf -is --force > autoreconf.log 2>&1 ||
    __sm.package.error "Autoreconf of ${package_name} ${package_version} failed! " "$PWD/autoreconf.log"
}

The sm framework defines a series of steps that a package install goes through:

# preinstall
#
#   prefetch
# fetch
#   postgetch
#   preextract
# extract
#   prepatch
# patch
#   preconfigure
# configure
#   postconfigure
#   prebuild
# build
#   preinstall
# install
#   preactivate
# activate
#   postactivate
#
# postinstall

The indented functions above are user-defined. Functions like fetch and configure are defined by sm.

In our case, the libyaml library defines two of those steps: prefetch and preconfigure. The prefetch function allows us to provide extra information to the fetch method, which specifically allows the prefetch to override the package_file (${package_file:="${package_name}-${package_version}.${archive_format}"}). In our case, even though the package name is libyaml, we want to download the file yaml-1.1.4.tar.gz.

The openssl library is somewhat more complicated. As with libyaml, we needed to teach sm how to install openssl statically. You can check out the commit to see how easy that was.

The great thing about getting this stuff into sm is that there is now a shared resource to answer questions like “how do you statically build openssl”. The work I did with Michal to improve the information for the libaries that Ruby depends on can now be used by anyone else trying to build Ruby (or anything else with those dependencies for that matter).

Tokaido is an sm library

Tokaido itself is an sm library! This means that the core Tokaido build will work on Linux, so it can be used to create a standalone distribution of Ruby for Linux, and maybe even the core of a Tokaido for Linux!

The Tokaido package implements a lot of the sm hooks, so check it out to learn more about what you can do using sm’s package API.

In my next post, I’ll talk about the architecture of the Tokaido UI component.

Tokaido Status Update: Implementation Details

Hey guys!

Since my last update, Tokaido was fully funded, and I’ve been hard at work planning, researching and working on Tokaido.

So far, we have a working binary build of Ruby, but no setup chrome. Because the binary build already exists, Terence Lee was able to experiment with it at a recent Rails Girls event, with great success:

Great thanks to Terence to put together a simple installer script that we could use to test whether the core build worked on a wide variety of OSX systems.

One thing that I mentioned in my original proposal was a desire to work closely with others working on related projects. Very soon after my project was announced, I teamed up with Michal Papis Continue reading “Tokaido Status Update: Implementation Details”

Tokaido Status Update: Implementation Details

Hey guys!

Since my last update, Tokaido was fully funded, and I’ve been hard at work planning, researching and working on Tokaido.

So far, we have a working binary build of Ruby, but no setup chrome. Because the binary build already exists, Terence Lee was able to experiment with it at a recent Rails Girls event, with great success:

Great thanks to Terence to put together a simple installer script that we could use to test whether the core build worked on a wide variety of OSX systems.

One thing that I mentioned in my original proposal was a desire to work closely with others working on related projects. Very soon after my project was announced, I teamed up with Michal Papis Continue reading “Tokaido Status Update: Implementation Details”