What is docrails?

Over the years I have seen some confusion about what is exactly
docrails
and how it relates to the documentation of Ruby on Rails.

This post explains everything you want to know about this aspect of the project.

What is docrails?

docrails is a branch of Ruby on Rails with
public write access where anyone can push doc fixes.

If you see a typo, you’d like to correct a factual error, complement some
existing documentation, add a useful example… before docrails existed you had
to open a pull request (or the equivalent in those days) and follow the ordinary
workflow to get it accepted. docrails allows you to clone the repo, edit, and
push. Done!

ZOMG, that’s awesome! Tell me more!

Changes to the code base need review before they are pushed. Each individual new
feature or bug fix needs the perspective and responsability of core team
members to take a decision about it.

Documentation fixes, though, are much more likely to be fine as they are. So,
docrails has a public write policy to ease the workflow for contributors.

All commits have to be reviewed anyway, so docrails needs the same effort from
Rails committers than going through pull requests, please everyone give big
props to Vijay Dev who is nowadays in charge
of this time consuming task.

The point of docrails is to provide a way to contribute to the Rails documentation
that is fast and easy for contributors.

But wait, I am editing some separate thing?

docrails is a separate branch because it has a different access policy, but
you are editing the actual Ruby on Rails documentation.

Every few days, once all new commits are reviewed docrails is merged into
master, and master is merged into docrails. Also, very important edits may be
cherry-picked into stable branches at the discrection of who merges.

What is allowed in docrails?

You can freely push changes to any RDoc, guides, and READMEs.

No code can be touched at all. That’s a hard rule. No matter how insignificant,
not even a one character typo in a string literal.

CHANGELOGs cannot be edited either.

Is docrails a documentation project?

No, Ruby on Rails has no documentation project. Treating documentation as a
separate aspect of the project would be similar to treating testing as an
external part of the project.

Documentation is an integral part of the development of Ruby on Rails.
Contributing a feature or bug fix means contributing its code, test coverage,
and documentation.

I am preparing a pull request, should I document later via docrails?

No, docrails is meant only for quick doc fixes.

Pull requests should be complete: code, tests, and docs. If a pull request lacks
any of those in general it won’t be accepted as is.

Also, updating docs does not only mean that you edit the RDoc next to the code
you are touching. Often the change needs grepping the project tree to find
instances of what the pull request is about, to update examples, revise
explanations affected by your change, or writing new documentation.

Tidbit: run ack -a to have guides included in the search.

I made a doc fix, when is it going to be online?

Rails releases are a complete set. The documentation itself is part of the
release. The fix is going to be online in the stable API or
guides websites when
the branch that contains the fix gets released.

Edits merged into master are always online in the edge API
and edge guides,
which are regenerated after every push to master. Thus, edits done via docrails
are online in the edge docs website after the next docrails/master cross-merge.

Can I open pull requests for documentation fixes in Ruby on Rails?

Absolutely. Specially if you are unsure about the fix. But if you feel confident
just push to docrails.

Please do not open pull requests in docrails.

Note that docrails has no issues tab. The reason is docrails is not a project,
as explained above, only a way to bypass pull requests. Documentation issues are
Ruby on Rails issues and belong to the Ruby on Rails project just as any other
kind of issue.

Does Ruby on Rails has documenters?

No, documentation comes with each push to master. Everyone documents Rails.

The only exception is guide authors. Guide authors take the task to write an
entire new guide about a certain topic, and they are allowed to push early
drafts to docrails for convenience (only guides in the public index are
considered to be published).

That’s for new guides. Once published, guides maintenance happens in master as
everything else.

Edge Rails: PATCH is the new primary HTTP method for updates

What is PATCH?

The HTTP method PUT means resource creation or replacement at some given URL.

Think files, for example. If you upload a file to S3 at some URL, you want
either to create the file at that URL or replace an existing file if there’s
one. That is PUT.

Now let’s say a web application has an Invoice model with a paid flag that
indicates whether the invoice has been paid. How do you set that flag in a
RESTful way? Submitting paid=1 via PUT to /invoices/:id does not conform to
HTTP semantics, because such request would not be sending a complete representation of the invoice for replacement.

With the constraints of the methods GET, POST, PUT, DELETE, the traditional answer
is to define the paid flag of a given invoice to be a resource by itself. So,
you define a route to be able to PUT paid=1 to /invoices/:id/paid. You have
to do that because PUT does not allow partial updates to a resource.

Now let’s think about ordinary edit forms in typical Ruby on Rails applications.
How many times are we sending a complete representation for replacement? Not
always, perhaps we could say that it is even rare in practice that you do so.
For example, the conventional created_at and updated_at timestamps normally
can’t be set by end-users, though they are often considered to belong to the
representation of resources that map to records.

PUT in addition is an idempotent method. You should be able to replay a request
as many times as you want and get the same resource, something that sometimes
is violated by conventional idioms for creating children resources using
nested attributes while updating a parent resource.

There’s nothing theoretical preventing PUT from doing partial updates, but when
HTTP was being standarized the replacement semantics were already deployed.

Because of that, the PATCH method was defined in 1995 and standarized later.
PATCH is a method that is not safe,
nor idempotent, and allows full and partial updates and side-effects on other resources.

In practice, as you see, PATCH suits everyday web programming way better than
PUT for updating resources. In Ruby on Rails it corresponds naturally to the way
we use update_attributes for updating records.

Thus, PATCH is going to be the primary method for updates in Rails 4.0.

Routing

This is an important change, but we plan to do it in a way that is backwards
compatible.

When a resource is declared in config/routes.rb, for example,

resources :users

the action in UsersController to update a user is still update in Rails 4.0.

PUT requests to /users/:id in Rails 4.0 get routed to update as they are
today. So, if you have an API that gets real PUT requests it is going to work.

In Rails 4.0, though, the router also routes PATCH requests to /users/:id to
the update action.

So, in Rails 4.0 both PUT and PATCH are routed to update.

Forms

Forms of persisted resources:

form_for @user

get “patch” in the hidden field “_method”. The RFC is deliberately vague about
the way to represent changes in a PATCH request. Submitting a form is
perfectly valid, client and server must simply agree on the accepted ways
to update a resource.

Let me emphasize that the “_method” hack is a workaround for the limitations in
web browsers. As you probably know Rails routes real HTTP methods. That is, actual
PUT, DELETE, and now, PATCH requests are routed to their respective actions.

General availability

PATCH requests are available in all places where the rest of the methods are
available today. There is a patch macro for the routes DSL, :via understands
the symbol :patch. Tests can issue PATCH requests, request objects respond to
patch?, etc. Please see the original commit for details (with an important
followup here).

Will my web server understand PATCH?

Yes, it should. I have personally tried Apache, nginx, Phusion Passenger,
Unicorn, Thin, and WEBrick. They all understood PATCH requests out of the box.

Also, HTTP clients should be in general able to issue PATCH requests. For example
in curl(1) you’d execute:

curl -d'user[name]=wadus' -X PATCH http://localhost:3000/users/1

Credits

We would like to thank David Lee for this
contribution and endless patience to keep interested in this even after months
of discussion
.

Also I would like to
highlight the quality of the patch itself. It is excellent: code, tests, all
docs revised, comments in code revised. Everything carefully and
thoroughly updated. An exemplar patch.

Rails 3.2.1 has been released

Rails 3.2.1 is out, with some fixes and doc improvements. Please check the CHANGELOGs gist for details.

What’s new in Edge Rails: EXPLAIN

There are some new features related to EXPLAIN in the forthcoming Ruby on Rails 3.2 we’d like
to share:

  • Running EXPLAIN manually
  • Automatic EXPLAIN for slow queries
  • Silencing automatic EXPLAIN

As of this writing they are available for the adapters sqlite3, mysql2, and
postgresql.

Running EXPLAIN Manually

You can now run EXPLAIN on the SQL generated by a relation this way:

User.where(:id => 1).joins(:posts).explain

The result of that method call is a string that carefully imitates the output of
database shells. For example, under MySQL you get something similar to

EXPLAIN for: SELECT `users`.* FROM `users` INNER JOIN `posts` ON `posts`.`user_id` = `users`.`id` WHERE `users`.`id` = 1
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
| id | select_type | table | type  | possible_keys | key     | key_len | ref   | rows | Extra       |
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
|  1 | SIMPLE      | users | const | PRIMARY       | PRIMARY | 4       | const |    1 |             |
|  1 | SIMPLE      | posts | ALL   | NULL          | NULL    | NULL    | NULL  |    1 | Using where |
+----+-------------+-------+-------+---------------+---------+---------+-------+------+-------------+
2 rows in set (0.00 sec)

and under PostgreSQL the same call yields something like

EXPLAIN for: SELECT "users".* FROM "users" INNER JOIN "posts" ON "posts"."user_id" = "users"."id" WHERE "users"."id" = 1
                                  QUERY PLAN
------------------------------------------------------------------------------
 Nested Loop Left Join  (cost=0.00..37.24 rows=8 width=0)
   Join Filter: (posts.user_id = users.id)
   ->  Index Scan using users_pkey on users  (cost=0.00..8.27 rows=1 width=4)
         Index Cond: (id = 1)
   ->  Seq Scan on posts  (cost=0.00..28.88 rows=8 width=4)
         Filter: (posts.user_id = 1)
(6 rows)

Please notes that explain runs the query or queries and asks the
database for their respective query plan afterwards. This is because due to eager loading a relation may trigger several queries to fetch the records and their associations, and in such cases, some queries need the result of
the previous ones.

If the relation triggers several queries the method still returns a single
string with all the query plans. This is an output meant for human consumption so we preferred to present
everything as a string in a format which is familiar right away rather than a structure.

Automatic EXPLAIN For Slow Queries

Rails 3.2 has the ability to help in detecting slow queries.

New applications get

config.active_record.auto_explain_threshold_in_seconds = 0.5

in config/environments/development.rb. Active Record monitors queries and if
they take more than that threshold their query plan will be logged using warn.

That works for anything running find_by_sql (which is almost everything, since
most of Active Record ends up calling that method). In the particular case of
relations, the threshold is compared against the total time needed to fetch the
records, not against the individual time taken by each involved query. Because
conceptually we are concerned with the cost of the call

User.where(:id => 1).joins(:posts).explain

rather than the cost of the different queries that call may trigger due to the
implementation.

By default the threshold is nil in the test and production environments, which
means the feature is disabled.

The value of that parameter is nil also if the threshold is not set, so
existing applications will need to add it by hand if they migrate to 3.2 to be
able to have automatic EXPLAIN enabled.

Silencing Automatic EXPLAIN

With automatic EXPLAIN enabled, it could still be the case that some queries are
just slow and you know they have to be. For example, a heavyweight report in the
backoffice.

The macro silence_auto_explain allows you to avoid having EXPLAIN run
repeatedly in those areas of code:

ActiveRecord::Base.silence_auto_explain do
  # no automatic EXPLAIN here
end

Interpreting Query Plans

The interpretation of the query plans is another topic, these are some pointers:

Happy debugging!

The Ruby on Rails API Switches to SDoc

The Ruby on Rails API is switching to SDoc starting with 3.1.

SDoc is a RDoc format created by Володя Колесников (@voloko) that has been powering railsapi.com for a long time.

Among other things, SDoc provides a search box with fuzzy match completion, tree browsing, keyboard navigation, and a really nice template.

You can already see the upgrade in the edge API.

We’d like to thank very much Володя for his work updating SDoc for the official API. Thanks man!

News from the Documentation Front

New Configuration Guide

Rails 3.1 will come with a new comprehensive guide about configuring Rails applications written by Ryan Bigg (@ryanbigg). The current draft is already available in the edge guides.

Rails Documentation Team

The documentation team, which was created some three years ago and consisted of Pratik Naik (@lifo), Mike Gunderloy (@MikeG1), and me, played a key role at bootstrapping docrails. Together with lots of API contributors and guides authors. Kudos to Pratik and Mike, their effort was outstanding and gave a definitive push to this aspect of the project.

After all these years, documentation maintenance happens regularly in master. Because of that, we are no longer going to have a separate documentation team. The same way we do not have a separate testing team. Tests and docs are an integral part of Ruby on Rails and complete patches have or should have proper tests and docs coverage.

Rails Guides Reviewers

Reviewing guides needs a special profile and dedication that has its own standards and pace. There’s going to be a team of guides reviewers that will take care of new material. I am very glad to announce that Vijay Dev (@vijay_dev) is going to be the first member of this team. Vijay has done an awesome work in docrails in the last months. Welcome aboard :).

Next Steps

The Rails documentation has improved a lot in the last years, it has more content, and it has better editorial quality. But there’s still a lot to do. Here are some ideas to work on:

  • A new documentation generator that evaluates the source tree and introspects to generate the API, mixed with a parser to extract documentation snippets.

  • Methods in the API have a link that toggles the visibility of their source code. Wouldn’t it be awesome if there was a toggler that disclosed their tests?

  • Test coverage for the guides.

  • What about a gorgeus template design?

If you’d like to hack on any of them please go ahead!

jQuery: New Default

In Rails 3.1 jQuery is going to be the default JavaScript library. Also, RJS has been extracted out. This post explains what that means for new applications, and what to look for while upgrading existing applications.

New Applications

Starting with Rails 3.1

rails new my_app

generates an application with jQuery.

The -j option of the generator lets you choose Prototype and Scriptaculous instead:

rails new my_app -j prototype

Such an application does not have RJS yet available though. From now on prototype-rails is needed for RJS, see below.

Upgrading Applications Using No RJS

Existing applications using no RJS should remove any references to ActionView::Base.debug_rjs in the project. Typically that means deleting

config.action_view.debug_rjs = true

from config/environments/development.rb. Other than that, upgrading should work out of the box as far as these changes is concerned.

Upgrading Applications Using RJS

Existing application using RJS should work out of the box with prototype-rails.

prototype-rails

RJS has been extracted to prototype-rails.

Applications using RJS have to add this line to their Gemfile while working against Rails master before the 3.1 release:

gem 'prototype-rails', :git => 'git://github.com/rails/prototype-rails.git'

prototype-rails is gonna be a gem when Rails 3.1 is out.

prototype-rails is the one who provides now the RJS template handler; the configuration flag ActionView::Base.debug_rjs; the ability to pass a block to link_to_function and button_to_function; the :update option of render, both in controllers and views; and the modules ActionView::Helpers::PrototypeHelper and ActionView::Helpers::ScriptaculousHelper.

Extraction has taken backwards compatibility into account as much as possible. With that goal in mind, everything is put back where it was before, either by reopening classes or modules, or via alias_method_chain when the refactor involved cutting some part of a method out, as happened with render :update.

There’s an exception: ActionView::Helpers::PrototypeHelper and ActionView::Helpers::ScriptaculousHelper are no longer ancestors of ActionView::Helpers. They are now injected into ActionView::Base and ActionView::TestCase directly.

When you include a module M into a class C, the class stores a proxy to M in its ancestor chain. After inclusion, if you add methods to M instances of C respond to them. But if you include another module N into M, the ancestor chain of C is not updated. Instances of C won’t respond to N‘s methods. Although a complete dynamic method dispatch would treat both ways of augmenting M equal, that’s the way it works as of today.

By the time prototype-rails is loaded ActionView::Helpers has already been included into ActionView::Base and ActionView::TestCase, so we need to inject them directly where they are needed. Standard usage just works, but please take into account that change in case you reopened or used ActionView::Helpers assuming those ancestors.

Before release, prototype-rails is going to provide also a way to bring Prototype and Scriptaculous to any application.

One Last Detail: The xhr Test Helper

The xml_http_request/xhr test helper is a simple convenience method that sets the X_REQUESTED_WITH header to “XMLHttpRequest”. If the test request has no Accept header, a fixed value of

[Mime::JS, Mime::HTML, Mime::XML, 'text/xml', Mime::ALL].join(', ')

serves as default.

The Accept header in Prototype calls corresponds to that value, but in jQuery it depends on the dataType attribute. Also, in other JavaScript libraries the Accept header may behave differently. What to do with the default?

This method has been left as is. In particular, if your application uses jQuery the default Accept header in the test will be different from the Accept header sent by jQuery. That’s some vinegar: Rails encourages you to base interfaces on explicit format parameters. So, for example, best practice is to provide a /users.json endpoint, rather than a /users endpoint with a JSON representation chosen depending on the Accept header.

If you desperately need a matching Accept header in tests you are still able to pass it. And you can also write your own convenience test helpers easily, since a XHR request is just one whose X_REQUESTED_WITH header matches /XMLHttpRequest/i.

Try It Out!

If your application uses RJS it would be really helpful that you test it against master with prototype-rails and report any issues you may find. Thanks!

Why HTTP Streaming?

Rails 3.1 is going to support HTTP streaming, aka chunked responses, this post explains what’s all about.

What Is HTTP Streaming?

Ordinary dynamic HTTP responses need a Content-Length header. Their timeline look like this:

HTTP request -> dynamic content generation -> HTTP response

Those are three serial steps because normally you need to generate the content in order to be able to know its size, and thus fill the Content-Length header of the response.

HTTP provides an alternative to this schema to be able to flush data as it is produced, known as chunked transfer encoding. That’s what we are referring to as streaming in recent commits.

Streamed responses have no Content-Length header. Rather, they have a Transfer-Encoding header with a value of “chunked”, and a body consisting of a series of chunks you write to the socket preceded by their individual sizes. Modulus details.

This is an example taken from Wikipedia:

HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked

25
This is the data in the first chunk

1C
and this is the second one

3
con
8
sequence
0

Point is, you are able to flush chunks to the socket as soon as you have them, no need to wait for the whole thing to be generated.

When Do Web Browsers Fetch Assets?

Web broswers parse documents as their content is received. When they find an asset referenced, think an image, stylesheet, or script, a request to fetch them is fired. That happens in parallel while the document is being received and processed, no matter whether the content comes chunked or not.

Browsers have limits on the number of concurrent requests they are allowed to do, a global one (typically +30), and another per domain (nowadays typically 4 or 6), but within those limits, requests for getting assets happen as the content is parsed.

Modern clients do not even block on JavaScript files as old ones did, they implement scanners that look ahead for asset nodes and request them. For example, this is the preload scanner of WebKit.

Trivia: While investigating this I discovered by accident that if the MIME type is unclear, for example “text/html” without an explicit charset, then web browsers buffer 1 KB of data firing no asset requests to be able to peek at the content and do an educated guess.

So What’s The Benefit Of Streaming?

Streaming doesn’t cut latency, neither it cuts the time a dynamic response needs to be generated. But since the application sends content right away instead of waiting for the whole response to be rendered, the client is able to request assets sooner. In particular, if you flush the head of an HTML document CSS and JavaScript files are going to be fetched in parallel, while the server works on generating content. The consequence is that pages load faster.

Followup

Streaming is still being polished for Rails 3.1, expect another post in the future covering its practical aspects in Ruby on Rails applications.

Thanks

Tony Gentilcore provided his insider’s guidance into this, thank you very much Tony! Also, thanks a lot to the Browserscope project for their really useful tables.