Made by DnA

Ajax in Rails 3.1 - A Roadmap

2011-12-05T00:00:00-08:00

By adopting jQuery and including the UJS adapter, Rails 3.1 has made initiating Ajax requests as easy as adding a data-remote attribute to a DOM element.

In order to get the most out of this new infrastructure however, you do need to be a bit aware of how things are working under the hood. In this article, we'll take a look together at the parts of the Ajax request that the UJS adapter handles and at the parts that it doesn't. In addition, we'll explore ways the Ajax call can be customized to accomodate those instances where you need more fine-grained control.

Lastly, after discussing the technical details, I'll provide common use-case examples illustrating how the Rails UJS adapter can be harnessed to facilitate various types of Ajax calls.

jQuery and the `$.ajax()` Function

Before we dive into the UJS Ajax helper functions, here is a quick refresher on how jQuery itself handles asynchronous calls to the server. If you're familiar with implementing Ajax with jQuery, you might want to skip this section, but note that we'll be referring later on to a lot of the terms introduced here.

The special sauce that makes the asynchronous communication between the browser and the server possible is the XMLHttpRequest (XHR) object. For each Ajax request, an instance of this object needs to be created in order to encapsulate the different details of the request and to adequately handle the response.

In practice, instantiating the XHR object is rather involved because of browser discrepancies, but fortunately jQuery abstracts away the process of setting up the Ajax request and dealing with the server response. It does this by means of the general utility function $.ajax() which lies at the heart of every jQuery Ajax call. This function exposes the configurable properties of the XHR object and also allows you to hook into different stages of the Ajax lifecycle via callbacks.

The $.ajax() function takes a single parameter, an options object. This object is comprised of a set of key-value pairs used for configuration. You can find a full list of possible settings here. The more commonly used ones are:

url: where to send the request to (default is the current page)
type: usually GET or POST (default is GET)
dataType: the format of the data we expect back from the server. Sets the value of the "Accept" attribute in the HTTP request header object.
Callback functions to hook into various stages of the request, such beforeSend, complete, error and success

jQuery also provides some convenience methods you can avail yourself of to make an ajax call. These include $.get(), $.getScript(), $.getJSON(), $.post() and .load(), all of which pre-configure some of the above mentioned properties. In the end though, all of these methods call the $.ajax() function in the background.

Rails 3.1 and the jQuery UJS Adapter

As convenient as jQuery makes Ajax calls, the Rails UJS adapter goes one step further in facilitating asynchronous requests.

Note that the jQuery UJS adapter is part of the

jquery-rails

gem that is included in every new Rails 3.1 project. The file we're referring to here -

jquery_ujs.js

- is located inside that gem. It is available in the search path for Sprockets and not part of the base application tree. To check it out you can either open the jquery-rails gem and navigate to

vendor/assets/javascripts/jquery_ujs.js

, or else take a look at the file on Github.

Before the advent of Rails 3.1, a typical Ajax request with jQuery would look something like the code below. Here, we attach a listener to the click event of a link. When the event fires, an Ajax request is initiated, causing a comment form to be loaded into the DOM:

  // adds a comment form upon click
  $('a#add_comment').live('click', function(event) {
     $.ajax({
        url: $(this).href,
        dataType: 'script'
     });
     return false;
  });

Things are much simpler in Rails 3.1. The UJS adapter allows us to make exactly the same Ajax call with just one line of code:

<%= link_to "Add a Comment", new_comment_path, :remote => true %>

This will generate the following HTML:

<a href="/comments/new" data-remote="true">Add a Comment</a>

The UJS adapter accomplishes this by finding all DOM elements with the HTML5 attribute data-remote set to true and binds common events associated with those elements, such as the click event for links or the submit event for forms. When the event fires, an $.ajax() call is executed in the background.

In HTML5

data-*

attributes allow us to embed custom data on all HTML elements. These custom attributes need to be prefixed by

data-

followed by the name of the attribute. The value can be anything. An example is

data-remote="true"

. Data attributes are most often interacted with via JavaScript. In fact, jQuery has a convenient

$.data( key )

method that returns the value for the

data-[key]

attribute on the element in question. For more information, see the jQuery API reference for the .data() method.

Let's now take a look at the parts of the UJS adapter that make this automagic possible.

Setting Up the Ajax Request: Configuration Defaults and How to Override Them

To get the most out of the UJS helpers, we first need to familiarize ourselves with the types of event listeners available and what kind of elements they're bound to. Additionally we need to know what Ajax configuration options are automatically set and in what manner they can be customized.

At first glance the UJS file might appear a bit daunting, but it is very well documented and is in fact not that hard to decipher. Basically, it defines a jQuery plugin function named $.rails. The main purpose of this function is to set up Ajax requests and other common JavaScript tasks, such as confirm dialogs, in an unobtrusive manner. In our discussion, we will be focusing on the Ajax-related aspects of the UJS adapter.

Near the beginning of the file, you'll find a bunch of different jQuery selectors grouped together and assigned to properties of the $.rails object:

// Link elements bound by jquery-ujs
linkClickSelector: 'a[data-confirm], a[data-method], a[data-remote], a[data-disable-with]',

// Select elements bound by jquery-ujs
inputChangeSelector: 'select[data-remote], input[data-remote], textarea[data-remote]',

// Form elements bound by jquery-ujs
formSubmitSelector: 'form',

// Form input elements bound by jquery-ujs
formInputClickSelector: 'form input[type=submit], form input[type=image], form button[type=submit], form button:not(button[type])'

Each property defines a jQuery selector collection. For example, the linkClickSelector property is composed of all link tags on the DOM containing specific data-* attributes like a[data-remote], a[data-method], etc.

This initial setup tells us that the UJS adapter is aware of:

links with data-* attributes
input, select and textarea elements, that have the data-remote attribute
forms and the different elements by which forms can be submitted

There is a reason those selectors have been assembled in this way. As we will see, they all will get specific event listeners attached to them.

In jQuery, as in other JavaScript frameworks, there are many ways to bind event listeners to DOM elements. As of jQuery 1.7, the preferred way is via the

.on()

method. Other methods to attach event handlers are

.bind()

.live()

and

.delegate()

. To find out more about how they differ, see this blog post on The Difference Between jQuery’s .bind(), .live(), and .delegate()

As an example of what this means, all links with a data-remote attribute set to true - and therefore part of the linkClickSelector group - get a click event listener attached to them. (Actually, it's the click.rails event, which is an example of a namespaced event).

In the code snippet below you can see how this is done. After checking for a few things like whether the link is supposed to generate a confirm popup, has been disabled or was right clicked, the callback function invokes the rails.handleRemote() method sending on the clicked DOM link as an argument.

  // https://github.com/rails/jquery-rails/blob/master/vendor/assets/javascripts/jquery_ujs.js

  $(document).delegate(rails.linkClickSelector, 'click.rails', function(e) {
    var link = $(this),
    method = link.data('method'),
    data = link.data('params');

    // This has to do with the :confirm attribute that you can set
    if (!rails.allowAction(link)) return rails.stopEverything(e);

    // check whether link is disabled
    if (link.is(rails.linkDisableSelector)) rails.disableElement(link);

    if (link.data('remote') !== undefined) {
      // Ajax call won't happen on right click
      if ( (e.metaKey || e.ctrlKey) && (!method || method === 'GET') && !data ) { return true; }

      if (rails.handleRemote(link) === false) { rails.enableElement(link); }
      return false;

    } else if (link.data('method')) {
      rails.handleMethod(link);
      return false;
    }
  });

Similar things take place for "ajaxified" input, select and textarea elements, which listen for the change event, as well as for forms which fire an Ajax call on submit.

The callback functions associated with these event listeners ultimately end up at the rails.handleRemote() function which is responsible for setting up and initializing the Ajax request. There is quite a bit going on in that function, so I won't paste it in its entirety, but you should check it out for yourself.

You will notice that the actual Ajax call only takes place at the very end of the function, while its main body is concerned with building up the Ajax options object that is destined for jQuery's $.ajax() method.

In following three tables, I've assembled some of the default configuration values for this options object based on the type of DOM element that has initiated the Ajax request. Also included are some notes on how these properties can be customized.

$.ajax() properties for links (on event type: "click")
url	Read from the href attribute of the link.
method	GET by default but can be overridden by setting the data-method attribute
dataType	The default is script , i.e. the server is expected to return valid JavaScript. Can be overridden by setting the data-type attribute.
data	Used for sending additional query parameters not included in the url string. You define extra parameters with the data-params attribute, e.g. data-params="data1=value1&data2=value2"

$.ajax() properties for input, select, textarea (on event type: "change")
url	By default the current page, but can be overridden with the data-url attribute.
method	GET by default but can be overridden by setting the data-method attribute
dataType	The default dataType is script , i.e. the server is expected to return valid JavaScript. Can be overridden by setting the data-type attribute.
data	Parameters are sent by serializing the element via element.serialize();

$.ajax() properties for forms (on event type: "submit")
url	Picked up from the action attribute on forms.
method	Rails usually sets the correct method attribute on forms to POST or PUT if you use the form_for view helper. You can also set the method attribute yourself.
dataType	The default dataType is script , i.e. the server is expected to return valid JavaScript. Can be overridden by setting the data-type attribute.
data	Form data is sent to the server by serializing the form via element.serializeArray();

Receiving the Ajax Response: UJS Custom Events

Note that the handleRemote() described above only takes care of initiating the Ajax request. You are still responsible for making sure that the server responds appropriately and for processing the data returned. In this context, you will sometimes need to make use of callback functions.

The $.ajax() method allows for user-defined callbacks to hook into the different stages of the Ajax request. As we have seen, the UJS adapter has wrapped the $.ajax() call, in which we would normally define the callbacks. However, it does provide alternate ways for us to declare callbacks as needed.

This is accomplished by firing custom events that tie into the regular jQuery callbacks in the lifecycle of the Ajax request. The names of those custom events mirror the ones they correspond to, but with an ajax: prefix, e.g. ajax:beforeSend.

You can view a full list of those events and when they are fired here.

To better understand that those custom events are basically a proxy for the actual $.ajax() callbacks, let's take a quick look at the part of the handleRemote() function that is responsible for setting them up.

// The options object will be eventually passed to the ajax() function
options = {
          // stopping the "ajax:beforeSend" event will cancel the ajax request
          beforeSend: function(xhr, settings) {
            if (settings.dataType === undefined) {
              xhr.setRequestHeader('accept', '*/*;q=0.5, ' + settings.accepts.script);
            }
            return rails.fire(element, 'ajax:beforeSend', [xhr, settings]);
          },
          success: function(data, status, xhr) {
            element.trigger('ajax:success', [data, status, xhr]);
          },
          complete: function(xhr, status) {
            element.trigger('ajax:complete', [xhr, status]);
          },
          error: function(xhr, status, error) {
            element.trigger('ajax:error', [xhr, status, error]);
          }
        };

For example, you can see that the ajax:success event gets triggered in the callback function for success. So by subscribing your DOM element to this event, you get the chance to define a callback function like so:

$(".delete_comment").live('ajax:success', function(evt, data, status, xhr){
  $(this).closest('div').fadeOut();
});

The Format of the Server Response

As we have seen, setting the dataType property tells the server what type of response data the client is requesting.

Deciding what this format should be is probably one of the most debated parts of Ajax in Rails and perhaps Ajax in general. In Rails you can choose convention over configuration and let the dataType default to script. That way you can handle everything to do with the response in a .js.erb or .js.coffee file and thereby avoid the need for callbacks, etc.

While I was researching this blog post, I cam across an article by Chad Fowler that grew out of a survey he had posted on Twitter: How Rails Developers do Ajax (with jQuery) in 2011.

From the survey results and the comments that followed, it would appear that there is no consensus amongst Rails developers as to whether the server should respond with JavaScript and/or HTML or whether it should be sending raw data in form of JSON and delegate the responsibility to the browser to deal with the data.

One can argue for both approaches. I agree with Chad when he says that while sending JSON might be better from a purist point of view (after all the server is not supposed to deal with the nitty gritty of DOM structure), sending JavaScript or HTML is often the more pragmatic approach.

And in fact, the UJS adapter in Rails sets the stage in a way that actually favors this latter approach and is the least hassle.

Philosophical bias aside, there probably will be occasions when you need the server return to something other than JavaScript. So it's good to know your options when it comes to the dataType property.

Setting the dataType property

The dataTypes that jQuery recognizes are these:

text: simple strings
html: blocks of html
script: a snippet of JavaScript, executed on the page
json: JSON-formatted data
jsonp: "padded" JSON for cross-domain requests
xml: XML-formatted data

As we will see a little later on, these names are short forms that jQuery understands and can translate into the correct HTTP Accept header values.

Looking at how the handleRemote() method tries to set the dataType property, we can see that we have three opportunities to customize the setting. Here is the code from the UJS adapter:

// Submits "remote" forms and links with ajax
handleRemote: function(element) {
  // ....
  dataType = element.data('type') || ($.ajaxSettings && $.ajaxSettings.dataType);


  options = {
    beforeSend: function(xhr, settings) {
      if (settings.dataType === undefined) {
        xhr.setRequestHeader('accept', '*/*;q=0.5, ' + settings.accepts.script);
      }
      return rails.fire(element, 'ajax:beforeSend', [xhr, settings]);
    }
  };

  rails.ajax(options);
}

The first way is by adding a data-type attribute to the DOM element in question. Doing this will take precedence over everything else. For example, here is how you'd set the dataType to accept json:

<%= link_to "Add a new task", new_project_task_path(@project), "data-type" => "json", :id => "add_task_btn" %>

You can also define a default dataType via jQuery's global $.ajaxSetup() function. This will change the default dataType for all Ajax requests:

$.ajaxSetup({
  dataType: 'json'
});

Finally, in the beforeSend callback, you have one last chance to override the dataType property. Note that in this case you cannot use the short form, but have to use the correct string that will be sent as the HTTP Accept header. The ability to reset the dataType in this fashion comes in handy for example when for some reason you don't have direct access to the DOM element itself.

  $("#add_task_button").live("ajax:beforeSend", function(e, xhr, settings){
    new_data_type = "application/json, text/javascript, */*; q=0.01";
    xhr.setRequestHeader('accept', new_data_type);
  })

The dataType property and HTTP Headers

The dataType setting ends up as the "Accept" Header, or more precisely, the value of the Accept key in the Request Header. The value is a comma-separated string of formats that will be accepted by the client. For instance:

*/*;q=0.5, text/javascript

The individual values represent acceptable MIME types and are sometimes accompanied by a value for the relative degree of preference q. The q is a number between 0 and 1 and, unless specified, is implicitly 1.

*/* represents a generic request that tells the server to return whatever it has. In practice, Rails will return the first format defined in the controller action.

So this setting - */*;q=0.5, text/javascript -, which happens to be the default in the UJS adapter, tells the server that javascript is preferred (implicit q=1), but failing that any other format will do, though relatively less preferred (q=0.5).

There are historic reasons for this default setting. The original default was simply text/javascript, but that meant that unless the javascript MIME type was handled by the controller, i.e. the js format was defined, Rails would throw a MissingTemplate error. By adding the generic request as the second preference, Rails will return any other defined response. Source

Sending the right data format back from the server

It is your task to ensure that the Controller returns the expected data type, or else something that is compatible.

In practice, by leaving the default dataType as script you can handle the Ajax request with a js.erb or a js.coffee file in which you can intermingle Ruby and JavaScript to determine the response. The server will send this back as pure JavaScript, which will be automatically evaluated by the browser.

Generally speaking, several things have to play nicely together for there to be a smooth transaction between browser and server:

First, Ajax sends the request with the Accept header set to the desired MIME type.

Next, the Rails Controller checks the Accept header to determine what it should (ideally) return. To do that, it has to first translate the content of the Accept header into known MIME types.

Lastly, the Controller will determine whether the action in question handles the particular MIME type. Before Rails 3 this was typically done in a respond_to block in the Controller action. With Rails 3 you can take advantage of the respond_with method to save you some typing.

For the rest of the article, we'll look at a variety of ways in which Ajax can be implemented in Rails 3.1 apps.

Example 1: Click Event: Inserting DOM Elements

Let's say we have a link that prompts users to leave a comment. When it is clicked, we want to insert a comment form via Ajax.

We can take advantage of the convenience provided by the UJS adapter to easily get the job done. Let's first look at the actual link that triggers the Ajax request when clicked:

<%= link_to "Add a Comment", new_comment_path, :remote => true %>

Recall that by adding the :remote => true attribute the UJS adapter will attach an event listener for the click event which when fired will initiate an Ajax request with the following properties:

url: '/comments/new' (read from the href attribute)
method: 'GET' (this is the default for links)
dataType: 'script' (also the default)

Very little code is needed in the Controller:

class CommentsController < ApplicationController
  respond_to :html, :js

  def new
    @comment = Comment.new
    respond_with(@comment)
  end
end

Finally, the app/views/comments/new.js.erb template:

var comment_form = $('<%= j(render(:partial => "form"))%>');

$('#new_comment').html(comment_form);

Note:

j()

is an alias for

escape_javascript()

And that's it!

Example 2: Submitting a Form via Ajax

Submitting a form via Ajax is not much more involved. For this example we'll submit a comment form.

Again, we just need to add the :remote => true attribute to the form:

<div id="new_comment">
  <%= form_for @comment, :remote => true do |f| %>
  <div class="field">
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </div>
  <div class="field">
    <%= f.label :body, "Comment" %><br />
    <%= f.text_area :body %>
  </div>
  <div class="actions">
    <%= submit_tag "Submit" %>
  </div>
 <% end %>
</div>
<div id="comments-container">
  <%= render :partial => @comments %>
</div>

The Controller side of things is fairly routine. Note that we are using the :location option of the respond_with method. While this does not affect the Ajax request, for an HTML request it would tell Rails to redirect the user to the comments/index page (instead of the default comments/show/:id) after saving the comment successfully.

class CommentsController < ApplicationController
  respond_to :html, :js

  def create
    @comment = Comment.new(params[:comment])
    flash[:notice] = 'Comment was successfully created.' if @comment.save
    respond_with(@comment, :location => comments_path)
  end
end

Let's assume that we'd like to avoid blank comment submissions and that the model has some validations in place to ensure that. How would we handle the case of a blank form submission? It turns out that error handling in .js.erb templates is a cinch:

var form = $('#new_comment');

<% if @comment.errors.any? %>

  // Create a list of errors
  var errors = $('<ul />');

  <% @comment.errors.full_messages.each do |error| %>
    errors.append('<li><%= escape_javascript(error) %></li>');
  <% end %>

  // Display errors before the form
  $('<div />').addClass('.errors').html(errors).prependTo("#new_comment");

<% else %>

  // Add the new comment to the beginning of the #comment-container division
  $("#comments-container").prepend('<%= j(render(:partial => "comment", :object => @comment))%>');

  // empty the form for new submission
  form.find('input:text,textarea').val('');

<% end %>

Example 3: Deleting via Ajax

So far we've been using the default Ajax setup taking advantage of the UJS adapter and the respond_with feature in Rails. Now we're going to be looking at a scenario where you might want to diverge from the default, even though you don't strictly need to.

We want to allow for a comment to be deleted a without triggering a full page refresh. The comment should simply disappear when we click the corresponding "Delete" link. Here is what the view code looks like:

<div id="comment-<%= comment.id %>">
  <p><%= comment.name %></p>
  <div><%= comment.body %></div>
  <%= link_to "Delete", comment_path(comment), :remote => true, :method => :delete, :confirm => 'Are you sure?', :class => "delete_comment" %>
  <hr />
</div>

We could again just simply handle the server response with a js.erb template. However, in this case, generating a new template file seems like overkill.

If you think about it, we don't need the server to return anything. The resource, in a REST-ful sense, has been deleted. It is unlikely that the deletion process will be unsuccessful, since the delete links all have the correct url with the right comment id. So unless the user tampers with the url everything should go as planned. Ideally, we just need the server to let us know that it has finished its business. We'll do that by overriding the default template rendering behavior of the controller by telling it to render "nothing":

class CommentsController < ApplicationController
  respond_to :html, :js

  def destroy
    @comment = Comment.find(params[:id])
    @comment.destroy
    respond_with(@comment) do |format|
      format.js { render :nothing => true }
    end
  end
end

Once we get the green light from the server, we'll remove every trace of that comment from the page. As we are not rendering a template, we need to handle a successful response in a callback function. As discussed, with the UJS adapter this means subscribing to custom events:

$(document).ready(function() {
  $(".delete_comment").live('ajax:success', function(evt, data, status, xhr){
    $(this).closest('div').fadeOut();
  });
});

In the code above, we simply fade out the division directly surrounding the comment that has just been deleted.

Example 4: Client-side Validation with Ajax (Using data-remote on an input field)

Links and forms are not the only elements that when given a data-remote attribute will trigger an Ajax call. As we have seen in the UJS adapter file, certain form elements such as inputs, selects and text areas can just as easily be "ajaxified". The event that is attached to them is change, which means that the Ajax call gets triggered when the user changes the value of the input field and leaves it.

As an example use case of this feature, let's check whether a username has already been taken before a sign up form is submitted. To accomplish this client-side validation task, we need to perform an asynchronous server roundtrip via Ajax.

To provide feedback to the user, we'll display a message next to the input field indicating whether the username is taken or not.

Again, the question arises as to what type of data the server should return. We could theoretically leave it to the server to display the right message in the DOM by reverting to the use of js.erb templates. Again however, this seems like overkill. One alternative would be to let the server search for a user by the username and return its search results as a JSON object.

But first things first. Here is a user signup form with an input field wired up to respond to the change event:

<%= form_for @user do |f| %>
  <div class="field">
    <%= f.label :username %><br />
    <%= f.text_field :username, :remote => true, "data-url" => "/users/check_username", "data-type" => :json %>
    <span id="username_check">&nbsp;</span>
  </div>
  <div class="field">
    <%= f.label :password %><br />
    <%= f.password_field :password %>
  </div>
  <div class="field">
    <%= f.label :password_confirmation %><br />
    <%= f.password_field :password_confirmation %>
  </div>
  <div class="actions">
    <%= f.submit "Sign Up" %>
  </div>
<% end %>

Notice that in addition to the data-remote attribute, the input field also sports two more custom data- attributes, namely data-type and data-url.

The data-url attribute will override the UJS default setting of sending the request to the current page. In addition, we also need to create a separate route to handle the username check with a url of /users/check_username:

// config/routes.rb

resources :users do
  collection do
    get 'check_username'
  end
end

Using the convenience of respond_with, this is all we need to do in the Controller to ensure that the @user resource is returned as a JSON object:

class UsersController < ApplicationController
  respond_to :html, :json

  def check_username
    @user = User.find_by_username(params[:user][:username])
    respond_with(@user)
  end

end

We can handle the response from the server in a success callback. The data returned from the server will be the @user object in JSON format. If the username is still available, @user will be nil. nil.to_json returns null. We therefore check for null in our success callback.

On the other hand, if the @user does exist, we can parse the JSON response and extract the username:

$(document).ready(function() {

  $("#user_username").bind('ajax:success', function(evt, data, status, xhr){
    if (data !== null) {
      $('#username_check').html(data.username + ' is already taken');
    } else {
      $('#username_check').html('Username is available!');
    }
  });

  // cancel Ajax call if input field empty
  $("#user_username").live('ajax:before', function(){
    if ($(this).val() == '') {
      return false;
    }
  });

  $("#user_username").focus(function(){
    $('#username_check').empty();
  });

});

The rest of the code handles the case when the change event is triggered by a blank input field. To deal with this scenario, we bind the ajax:before event and return false, which serves to cancel the Ajax call. Finally, we also need to make sure that the message about username availability disappears should the user click in the input field afresh.

Example 5: Adding Dynamic Selects (Using data-remote on a select field)

Like the input field from Example 4, select fields with a data-remote attribute also trigger Ajax calls on change.

To demonstrate how to make use of this setup, we will create a dynamic, dependent select in a user signup form. When a user selects her country, a second select will become active making the states in that country selectable. Since in many countries the state is not a required part of the address, the state select will remain deactivated when one of those countries is chosen.

For the actual geographical data needed to populate the various dropdowns, we'll make use of a gem named carmen. It touts itself as a "simple collection of geographic names and abbreviations for Ruby". Among other things, carmen replaces the official Rails country_select view helper and supports states for a number of countries like Australia, Brazil, Canada, China, Cuba, Denmark, Germany, India, Italy, Mexico, the Netherlands, New Zealand, Norway, Spain, Ukraine, and United States.

To set this up, you simply need to add gem "carmen" to your Gemfile. You can then use the following code to the user signup form:

<%= form_for @user do |f| %>
  <!-- other form fields here -->
  <div class="field">
    <%= f.label :country %><br />
    <%= f.country_select :country, nil, {:include_blank => true}, "data-remote" => true, "data-url" => "/users/get_states", "data-type" => :json %>
  </div>
  <div class="field">
    <%= f.label :state %><br />
    <%= f.select :state, [], { :include_blank => true }, :disabled => true %>
  </div>
  <!-- other form fields here -->
  <div class="actions">
    <%= f.submit "Sign Up" %>
  </div>
<% end %>

Notice that the state select is disabled by default and contains no options. The country_select has three data-* attributes defined: data-url to tell UJS where to send the request, data-type => :json to override the default script data format and as always data-remote to trigger the Ajax call when a country is selected.

Here we elect once again to return the states as JSON and, in a success callback, add them as options to the state select. Since this is a bit of an oddball request, we'll add a new, non-RESTful route called get_states to the User resource. The Controller code:

class UsersController < ApplicationController
  respond_to :html, :json

  rescue_from Carmen::StatesNotSupported, Carmen::NonexistentCountry, :with => :state_not_supported

  def get_states
    @states = Carmen.states(params[:user][:country])
    respond_with(@states)
  end

  protected

  def state_not_supported
    @states = nil
    respond_with(@states)
  end

end

One side effect of using the Carmen gem was that I had to rescue a special type of error raised if states are requested for a country that doesn't have any. (I guess the gem was not designed for dependent selects.) Also, I had to handle the case where the country code was a blank string; this triggers a Carmen::NonexistentCountry error. This can happen for example when the user selects a country and afterwards resets the select to the blank option.

To get the states for a particular country, we use the class method Carmen.states which takes a country code and returns an array structure:

Carmen.states('US') => [['Alabama', 'AL'], ['Arkansas', 'AR'], ... ]

The JSON representation of this array looks like this:

{0: ["Alabama","AL"], 1: ["Alaska","AK"], 2: ["Arizona","AZ"], ...}

In the success callback we parse the JSON structure and construct select options for inserting into the state select, which then gets re-enabled:

$(document).ready(function() {

    $("#user_country").bind('ajax:success', function(evt, data, status, xhr){
      var select = $('#user_state');

      if (data !== null) {
        select.removeAttr('disabled');
        $.each(data, function(key, value) {
          $("<option/>").val(value[1]).text(value[0]).appendTo(select);
        });
      } else {
        select.empty();
        select.attr('disabled', 'disabled');
      }
    });
});

Voilå, a functioning dynamic country and state drop down pair with very little headache!

Example 6: Ajax Pagination - Returning HTML directly

There are many ways to do pagination via Ajax. For demo purposes, though, I'll use it as an example of where we could return an HTML partial rather than JavaScript or JSON.

Pagination in Rails is usually done with the help of a gem. I considered using the will_paginate gem for this demo, but unfortunately adding extra attributes, such as data-remote, to the underlying links generated by the will_paginate view helper method turned out to be harder than expected. If you're curious, it involves creating a custom link renderer that inherits from WillPaginate::LinkRenderer.

The alternative pagination gem that has gained some traction lately is Kaminari, which from a setup and interaction perspective is much like will_paginate and happens to support Ajax links for its paginate helper method.

In this example, we'll paginate a list of users, each displayed with their name, avatar and location info. When we click on a pagination link, we'd like to render an HTML partial containing the next page of users along with the updated pagination links.

To begin with, you need to add gem "kaminari" to your Gemfile. Next, set up the view page in such a way that the varying content is contained in a partial that is surrounded by an identifiable division. Something like this would work:

<div id="container">
 <%= render :partial => "users" %>
</div>

And this is the content of the _users.html.erb partial including the kaminari paginate helper:

<%= paginate @users, :remote => true %>

<%= @users.each do |user| %>
<dl>
  <dd class='img'>
   <%= gravatar_image_tag(user.email) %>
  </dd>
  <dt>
    <%= link_to user.username, user_path(user) %>
  </dt>
  <dd>
    <%= [user.country, user.state].compact.join(", ") %>
  </dd>
</dl>
<% end %>

It was at this point that I realized that while the Kaminari pagination helper supports the data-remote attribute, it doesn't recognize other custom data-* attributes. For this to work as an HTML data type example, we would need to set the data-type attribute. I got around this limitation by setting that attribute in a ajax:beforeSend callback:

$(document).ready(function() {
  $(".pagination span a").live('ajax:beforeSend', function(event, xhr, settings){
    xhr.setRequestHeader('accept', 'text/html');
  });
});

In the Controller we have a new sort of situation: Ajax has requested HTML as the response MIME type. However, we only want to render the partial if the request came via Ajax. Otherwise, that is in the case of a non-Ajax request we want to render the regular HTML template. Luckily, there is a simple way to distinguish between the two types of requests:

class UsersController < ApplicationController
  respond_to :html

  def index
    @users = @users = User.order(:username).page(params[:page]).per(10)
    respond_with(@users) do |format|
      format.html {
        if request.xhr?
          render :partial => "users"
        end
      }
    end
  end

The last piece of the puzzle is to actually insert the contents of the partial in the right place. Here is the code for the success callback that accomplishes this:

$(document).ready(function() {
    $(".pagination span a").live('ajax:success', function(evt, data, status, xhr){
      $("#container").html(data);
    });
});

And with that we've come to the end of this longer-than-intended post on Ajax and Rails 3.1. Hope you found it useful!

Feedback, comments, suggestions? Please leave them below.

Representing Tabular Data in Ruby - A Mini E-Book

2011-09-29T00:00:00-07:00

Mendicant University (MU) regularly releases the programming assignments used in its three-week Ruby software development courses. In addition to publicizing the assignments, students who took the course often open up their solutions. So anyone interested in getting better at Ruby can use this material for self-study.

To increase the educational value of this material, Felipe Doria and I decided to create a "digest" of one of the exercises. It eventually grew into something the size of a mini E-Book, mainly because of the large number of code samples.

The assignment we picked was about building a general purpose table structure that could be used in a wide range of data processing scenarios. In the E-Booklet we outline step by step the process taken to arrive at the final solution, stopping along the way to explain the reasoning behind the various design decisions.

The concepts covered include:

TDD approach to a complex problem
Modeling tabular data (rows, columns, cells)
Efficiently handling exceptions and bad user input
API design in line with the Single Responsibility Principle
Mixing in core modules like Enumerable

You can download the E-Book in PDF format.

Ruby's Eigenclasses Demystified

2011-06-24T00:00:00-07:00

In Ruby, classes and objects are intricately connected in ways that might not be immediately obvious. Owing to their special use-case, it is easy to lose sight of the fact that classes are themselves objects in Ruby. What makes them different, vis-à-vis regular objects, is that they serve as a blueprint for object creation and also that they are part of a hierarchy of classes. That is, classes can have instances (objects), superclasses (parents) and subclasses (children).

If classes are also objects it follows that they must have a class of their own. In Ruby the class of all class objects is the class Class:

# one way to instantiate a new class
Dog = Class.new
    
# a more conventional way of class construction
class Dog
    # some doggie behavior here
end

Dog.class
=> Class

The Method Lookup Path

The relationship between classes and objects becomes important in the context of the method lookup path. When you call a method on an object, Ruby first looks for that method in the object's class. Failing to find it there, it next checks the superclass of the object's class and keeps on going up through the class hierarchy until it hits the topmost class which as of Ruby 1.9 is BasicObject. The diagram below illustrates the way Ruby traverses the class hierarchy in search of a given method:

The method lookup path for classes works in a similar fashion. When you call a method on a class object, it will first look in its class, which as we have established is Class, and then continue on to its superclass, Module, all the way up to BasicObject:

Dog.class
=> Class
Class.superclass
=> Module
Module.superclass
=> Object
Object.superclass
=> BasicObject

And again visually:

As we can see BasicObject and Object lie at the root of the class hierarchy, so this means that all objects, whether they be regular objects or class objects, will eventually inherit the instance methods defined in those classes.

Singleton Methods

The primary function of classes is to define the behavior of their instances. For example, the behavior of dog instances in general is placed in the Dog class. However, in Ruby we can also confer unique behavior to individual objects. Meaning that we can create methods for a specific dog object that are not shared by other instances of the Dog class. These methods are often referred to as singleton methods as they belong to a single object only.

To re-iterate, singleton methods are defined on the object itself instead of in its class:

snoopy = Dog.new
def snoopy.alter_ego
  "Red Baron"
end

snoopy.alter_ego
# => "Red Baron"

fido.alter_ego
# => NoMethodError: undefined method `alter_ego' for #<Dog:0x0000000190cee0>

Eigenclasses

As noted above, when a method gets called on an object, Ruby first looks in the object's class for the method definition. In the case of singleton methods, the method is obviously not added to the object's class, since that method is not available to other instances of that class. So where the heck is it? And, how does Ruby manage to find both the object's singleton methods and also the general instance methods without disrupting the normal method lookup path?

As it turns out, Ruby accomplishes all this in typical neat fashion. First a new anonymous class is created to hold the object's singleton methods. Then this anonymous class assumes the role of the object's class and the original class is re-designated as the superclass of that anonymous class. As such, the normal method lookup pattern is unaltered - both the singleton methods (in the anonymous class) and the instance methods (in the superclass) will be found along the method lookup path when the object receives a method call (see diagram below).

There are many alternate names for this dynamically created anonymous class that Ruby inserts into the method lookup path: metaclasses, ghost classes, singleton classes and eigenclasses. The word "eigen" comes from the German, roughly connoting something like "one's very own". Eigenclass seems to have found favor in the Ruby community, so I'll stick to that.

Note: In addition to defining singleton methods using the object name (e.g. def snoopy.alter_ego), you can also add methods to eigenclasses using this special syntax:

class << snoopy
  def alter_ego
    "Red Baron"
  end
end

The class << syntax opens the eigenclass of whatever object you pass to it and lets you define a method straight in there.

Class Methods

Being objects, classes too can have singleton methods. In fact, what we commonly think of as class methods are nothing more than singleton methods for classes - methods owned by a singular object that incidentally happens to be a class. As with all singleton methods, class methods are also housed in an eigenclass.

Class singleton methods can be created in a variety of ways:

class Dog
  def self.closest_relative
    "wolf"
  end
end

class Dog
  class << self
    def closest_relative
      "wolf"
    end
  end
end

def Dog.closest_relative
  "wolf"
end

class << Dog
  def closest_relative
    "wolf"
  end
end

All the examples above are identical in that they all open the eigenclass of the object Dog and define a (class) method directly in there.

Eigenclasses Revealed

Eigenclasses are not only anonymous; under normal circumstances they are hidden from view. This, despite the fact that they have been designated as the first stop in the method lookup path:

class << Dog
  def closest_relative
   "wolf"
  end
end

Dog.class
# => Class

In the code above, the Dog class is still reporting its class as the class Class even though we have added a class method, which should have caused the class to be changed to be the eigenclass. Nonetheless, there is a way we can get objects to reveal their eigenclass:

class Object 
  def eigenclass 
    class << self
      self
    end 
  end 
end

This code might appear a bit puzzling at first glance, so let's see if we can figure out what's going on here.

First off, we know that the Object class is one of the ultimate ancestors of all classes in Ruby, which is why it is a good place to define methods that you want to be universally available.

To understand this code we have to keep track of where and how the value of self changes. Immediately inside the eigenclass() instance method, self is the object that the method was originally called on. We then open the eigenclass of this receiver object with the class << self syntax. Now that we are in the scope of the eigenclass, the value of self changes and now refers to the eigenclass object itself. By returning the value of self from within this eigenclass, we trigger Ruby's built-in to_s object identifier - allowing us get a glimpse of the elusive eigenclass.

Here's how we can make actual use of this eigenclass() instance method to bring eigenclasses out into the open.

class Dog
end

snoopy = Dog.new
=> #<Dog:0x00000001c4a170>

snoopy.eigenclass
=> #<Class:#<Dog:0x00000001c4a170>>

snoopy.eigenclass.superclass
=> Dog

Ruby's to_s Method

The to_s method is defined in the Object class and returns a string representing the object it was called on. This string is a composite of the object's class name followed by a unique object ID. For example, #<Mouse:0x00000001c4a170>.

Class objects (instances of the class Class) such as Class, Object or String override the to_s method and simply return their name instead.

Dog.to_s
=> "Dog"

In actuality, the overriding of the to_s method for classes is done in Module which is the superclass of Class. Here is how the Ruby documentation describes the Module to_s method:

Returns a string representing the module or class. For basic classes and modules this is the name. For singletons, we show information on the thing we are attached to as well.

Note the bit about singletons including information about the attached object as well. This explains why when we call to_s on the eigenclass of snoopy we get back the anonymously named "Class" followed by the identifier for its attached object: #<Class:#<Dog:0x00000001c4a170>>. Calling to_s on the superclass of the eigenclass, however, references the original (non-singleton) class Dog and so simply returns the name "Dog".

Eigenclasses and Class Inheritance

Just as an object has access to instance methods defined in the superclasses of its class, so do classes have access to the class methods defined in their ancestor classes. Take a look at the code below:

class Mammal
  def self.warm_blooded?
    true
  end
end
 
class Dog < Mammal
  def self.closest_relative
    "wolf"
  end
end
 
Dog.closest_relative
# => "wolf"
Dog.warm_blooded?
# => true

This is another puzzler. How is it that classes manage to inherit the class methods of their superclasses? After all, the superclass of a class object is in the method lookup path of its instances but not in its own method lookup path:

Method lookup path of a Dog instance: fido --> Dog --> Mammal --> Object ...

Method lookup path of a Dog Object: Dog --> Class --> Module --> Object...

Here again some fancy footwork is done to ensure that the method lookup path pattern remains unaltered and yet allow access to the class methods of the superclass.

First off, when a class (aka singleton) method is defined on Dog, an eigenclass is created. For all intents and purposes, this eigenclass becomes the class of the Dog object in lieu of Class. Class gets "pushed up" the lookup chain and becomes a superclass. Now, when a class method is defined on the superclass of Dog (e.g, Mammal), the resulting eigenclass is designated as the immediate superclass of Dog's own eigenclass and Class gets pushed up even higher. As this description is probably as clear as mud, hopefully the following diagram will shed some light on the matter:

So there you have it. The Ruby object method lookup path in all its glory. Well almost. We haven't looked at where module mixins fit into the picture. Hopefully I'll have the time to cover that in a later blogpost.

Resources

The Ruby Object Model and Metaprogramming, Screencasts by Dave Thomas
Metaprogramming Ruby by Paolo Perrotta

Dynamically Nesting Deeply Nested Forms

2010-12-31T00:00:00-08:00

This is an update to my previous post about dynamic nested forms. It was brought to my attention that the functionality of the nested_form gem on github breaks down when used with a form that is nested multiple levels deep.

I remedied this with some tweaks to the code and the gem is now ready to be used on deeply nested forms. I have only tested it for forms up to four levels deep, but there should be no reason why it wouldn't work with even deeper nesting. Of course why anyone would want to nest their forms deeper than that is beyond me. Even four levels can be a real challenge from a usability perspective.

Below is a quick demo on how to set up a deeply nested Rails form for a Project --> Tasks --> Milestones --> Steps:

Nothing special on the Model side of things:

class Project < ActiveRecord::Base
  has_many :tasks, :dependent => :destroy
  validates_presence_of :name
  validates_associated :tasks
  
  accepts_nested_attributes_for :tasks, 
                                :allow_destroy => true, 
                                :reject_if => :all_blank
end

class Task < ActiveRecord::Base
  belongs_to :project
  has_many :milestones, :dependent => :destroy
  
  accepts_nested_attributes_for :milestones, 
                                :allow_destroy => true, 
                                :reject_if => :all_blank
  validates_presence_of :name
  validates_associated :milestones
end

class Milestone < ActiveRecord::Base
  belongs_to :task
  has_many :steps, :dependent => :destroy
  
  accepts_nested_attributes_for :steps, 
                                :allow_destroy => true, 
                                :reject_if => :all_blank
  
  validates_presence_of :name
  validates_associated :steps
end


class Step < ActiveRecord::Base
  belongs_to :milestone
  
  validates_presence_of :name
end

On entering a new Project we would like to be able to dynamically add fields for the project's tasks, as well as the task's milestones and finally for the steps that belong to a milestone. Way easier than it sounds.

The controller code only needs to instantiate a new project instance variable. Note that it is not necessary to pre-build an instance of any of the nested objects.

class ProjectsController < ApplicationController

  def new
    @project = Project.new
  end

end

And here's what the form could look like. I wrote the view code in Haml, not only because I use it all the time, but also because the lack of opening/closing html tags and the indentation might be visually helpful in this case:

= javascript_include_tag 'jquery', 'nested_form'
= nested_form_for @project do |form|
  %fieldset
    %p
      = form.label :name
      %br/
      = form.text_field :name
    = form.fields_for :tasks do |task_form|
      %h4 Task
      = task_form.text_field :name
      = task_form.fields_for :milestones do |milestone_form|
        %h4 Milestone   
        %p
          = milestone_form.text_field :name
        = milestone_form.fields_for :steps do |step_form|
          %h4 Step
          %p
            = step_form.text_field :name
          %p
            = step_form.link_to_remove "[-] Remove this step"
        %p
          = milestone_form.link_to_add "[+] Add A Step", :steps
        %p
          = milestone_form.link_to_remove "[-] Remove this milestone" 
      %p
        = task_form.link_to_add "[+] Add A Milestone", :milestones
      %p      
        = task_form.link_to_remove "[-] Remove this task"
      %hr
    %p
      = form.link_to_add "[+] Add a task", :tasks  

    .actions
      = form.submit 'Save'

As you can see, the only complication here is to organize the form in such a way that it makes sense to the user and to keep track of the different Form Builder instances (form, task_form, milestone_form, step_form) in the nested forms. I'm also including a screenshot of what the form actually looks like. I'm aware of the lame design and the confusing number and placement of add and remove links, but I hope that it still manages to illustrate what is possible.

By the way, the indentation of the nested form visible in the picture is accomplished with a single line of CSS:

.fields {
  margin-left: 15px;
}

This is facilitated by the fact that every newly inserted nested form part will always be wrapped in a div.fields.

Rails Named Bind Variables

2010-11-08T00:00:00-08:00

Let's say you have a form that searches various colums in a database table for a particular term. Your query might look something like this:

User.paginate(:page => params[:page],
:conditions => ["field1 LIKE ? OR field_2 LIKE ? OR field_3 LIKE ?",
params[:search], params[:search], params[:search]])

Not very DRY, is it? Turns out there is a much more elegant way to do this in Rails. You can use named bind variables instead. Named bind variables allow you to replace the question mark placeholders in the conditions array with a symbol. The values corresponding to these symbol placeholders can be then accessed via a hash.

To refactor the simple User search example cited above, first wire up the controller like so:

User.search(params[:search], params[:page])

Then in the User model, create a class method called search using a named bind variable:

def self.search(search, page)
  sql_condition = %w(email real_name nickname
                     twitter_account_name
                     github_account_name).
                  map {|field| "#{field} LIKE :search"}.join(" OR ")

  paginate :per_page => 20, :page => page,
           :conditions => [sql_condition, {:search => "%#{search}%"}],
           :order => 'email'
end

The value of the named bind variable (:search) is accessed through the hash using the matching symbol key.

Another possible use case for named bind variables is in those cases where you have multiple parameters in your conditions array. If we are dealing with four or five parameters, it takes some looking to figure out which question mark matches which parameter. With named bind variables you can do something like the following:

Company.where(
  "id = :id AND name = :name AND division = :division AND
  created_at > :accounting_date",
  { :id => 3, :name => "37signals", :division => "First",
    :accounting_date => '2005-01-01' }
).first

Cooking Up A Custom Rails 3 Template

2010-10-11T00:00:00-07:00

Rails 3 templates allow you to set up your work environment just the way you want it. For example, I prefer jQuery to Prototype, Haml to erb templates, and these days am using RSpec for testing. I also use Compass with Sass and Factory Girl for fixture replacement. Repeating the steps required to swap out these parts every time I generate a new rails app gets old pretty quickly. Templates are plain ruby files that automate these type of setup tasks.

Rails 3 template files can install plugins or gems, run rake tasks or migrations and even place boilerplate code into any file you specify. So besides setting up your basic Rails environment, you can instruct your template file to set up a user system, for example, by automatically installing the devise gem and running its generate command.

Templates have been completely overhauled in Rails 3. Besides sporting more commands, templates now share the API with generators. This makes sense seeing that they both involve generating new files or modifying existing ones. The internals have been rebuilt and now leverage the Thor gem in the background.

Available Generator Methods

To take full advantage of the new Rails 3 template API, you need to be familiar with what methods are available to you. For reference purposes, it is also beneficial to know where these methods are defined. The more general purpose methods now live in the Thor::Actions class. Some of these are:

copy_file(source, *args, &block)
create_file(destination, *args, &block)
get(source, *args, &block) -- download files from a remote url
prepend_file(path, *args, &block)
remove_file(path, config = {})
run(command, config = {}) -- used to run bash commands

Console Options

Thor::Shell::Basic provides several methods that enable you to capture and process user input:

ask(statement, color = nil) -- Example: "Which design framework? [none(default), compass]: "
no?(statement, color = nil) -- Returns true if the user has answered the question with no
yes?(statement, color = nil) -- Returns true if the user has answered the question with yes

Injecting Code Into Files

The Rails specific methods can be found in the module Rails::Generators::Actions. The following methods find and write to a particular file:

add_source(source, options={}) -- adds a given source to the Gemfile
gem(*args) -- Adds an entry for the given gem to the Gemfile
route(routing_code) -- adds a route
environment(data=nil, options={}, &block) -- insert line into application.rb
application(data=nil, options={}, &block) -- Alias for environment

Creating Files

The methods below call Thor's create_file method under the hood. The second argument determines the content of the newly created file. This argument can either be a string or a block:

vendor(filename, data=nil, &block) -- Creates a file in the /vendor directory
lib(filename, data=nil, &block) -- Creates a file in the /lib directory
rakefile(filename, data=nil, &block) -- Creates a rakefile in the /lib/tasks directory
initializer(filename, data=nil, &block) -- Creates a file in the /config/initializers directory

For example, the following illustrates the rakefile method:

rakefile("bootstrap.rake") do
    project = ask("What is the UNIX name of your project?")

    <<-TASK
      namespace :#{project} do
         task :bootstrap do
          puts "i like boots!"
       end
      end
    TASK
end

rakefile("seed.rake", "puts 'im plantin ur seedz'")

Executing stuff

Finally there are methods that execute a command:

git(command={}) -- Runs a command in git
generate(what, *args) -- Runs a generator from Rails or a plugin. script/rails generate #{what} [flattened args] is run in the background
rake(command, options={}) -- Runs the supplied rake task
readme(path) -- Reads the file at the given path and prints out its contents
plugin(name, options) -- Installs a plugin from github

Template Design

Now that we have an overview of the building blocks, we need to consider the base composition of the template. Because most templates are very opinionated in their choices of what to swap out, tweak, generate, etc., chances are slim that you will find one that perfectly suits your needs right out of the box. However, Rails 3 has made it very easy to customize existing templates for different setups without needing to create an entirely new one for each configuration.

Use Recipes for Modularity and Organization

Much of the flexibility for mixing and remixing template modules derives from the apply method in the Thor::Actions module.

# File 'lib/thor/actions.rb', line 191

def apply(path, config={})
  verbose = config.fetch(:verbose, true)
  is_uri  = path =~ /^https?\:\/\//
  path    = find_in_source_paths(path) unless is_uri

  say_status :apply, path, verbose
  shell.padding += 1 if verbose

  if is_uri
    contents = open(path, "Accept" => "application/x-thor-template") {|io| io.read }
  else
    contents = open(path) {|io| io.read }
  end

  instance_eval(contents, path)
  shell.padding -= 1 if verbose
end

The apply method takes a path variable which can point to either an external url or a local file. The contents of the file are read and executed using instance_eval. This allows you to separate out functionality you want to invoke into different component or "recipe" files. You make a directory to house each of the recipe files, e.g. one for "jquery", "cucumber", "haml", etc. In the template file you can then pick and choose which of these recipes you want to apply. Since it uses instance_eval, the apply method executes the content of the recipe file within the scope of the template file. This has the same effect as if you had placed the recipe code snippet directly into the template file.

As a basis for my own template, I forked an existing template on github that appealed to me because it was fairly simple and used this recipe approach. It came very close to the setup I needed, but more importantly, it afforded the flexibility for customizing the template.

Out of the box, this is the starting directory structure:

rails-templater
  \_bootstrap.rb (to bootstap entire app building process using RVM gemset) 
  \_core_extensions.rb (extending Rails::Generators::Actions)
  \_recipes (houses individual recipe files)
    \_design.rb
    \_default.rb
    \_cucumber.rb
    \_haml.rb
    \_etc...
  \_snippets (misc. snippets)
  \_templater.rb (the actual template file)
  \_templates
    \_git (template for .gitignore file)
    \_haml (template for views/layouts/application.html.haml in haml)
    \_mongoid

To create a new Rails app based off of this template you would need to use the -m switch with the path to the templater.rb file. This is the actual template file that orchestrates all the things that happen after the default files and folders of the new app have been generated.

Defining And Modifying Recipes

In the main template file you can add new or remove existing recipes. The template code loops through the recipes you specify and applies each of them in turn.

required_recipes = %w(default jquery haml rspec factory_girl remarkable)
required_recipes.each {|required_recipe| apply recipe(required_recipe)}

What if you have recipes that you want to apply sometimes, but not other times? It is very easy to make a recipe optional or allow for live choices between different recipes.

The file core_extensions.rb extends the regular Rails::Generators::Actions with some custom methods. One of those additional methods is called load_options and this is where the template defines some choices and saves them to a Hash called @template_options:

# in core_extensions.rb

def load_options
  @template_options[:design] = ask("Which design framework? 
                               [none(default), compass]: ").downcase
  @template_options[:design] = "none" if @template_options[:design].nil?

  @template_options[:orm] = options["skip_active_record"] ? "mongoid" : "active_record"
end

In the template or recipe files this @template_options can later be queried to make decisions. In simpler cases, the same effect can also be accomplished with a simple yes/no question:

apply(recipe('cucumber')) if yes?("Do you want to some cukes?")

Delayed Code Execution Using Lambdas

Some of the code in the recipe files depends on the presence of certain gems, meaning that it can only be executed after the gem in question has been installed. However, the recipes are being read (applied) before the gems are loaded. So how can you define the gem dependent code in the recipe file, but only run it after the gem is available? Lambdas to the rescue! Take the example of the recipe for compass:

gem 'compass'

compass_sass_dir = "app/stylesheets"
compass_css_dir = "public/stylesheets/compiled"

compass_command = "compass init rails . --using blueprint/semantic --css-dir=#{compass_css_dir} --sass-dir=#{compass_sass_dir} "

stategies << lambda do
  puts "Beginning Compass setup"
  run compass_command
  puts "Compass has been setup"
end

As you can see in the above code snippet, a piece of executable code is being added to the strategies array which was initialized in the beginning. After the template file runs its bundle install command, we invoke a method called execute_strategies that will loop through the lambdas originating from different recipes and call each of them in turn.

def execute_stategies
  stategies.each {|stategy| stategy.call }
end

Using RVM

I love RVM (Ruby Version Manager). For each Ruby version installed, RVM allows a specific groups of gems (Gemsets) to be associated with a project. This is a highly useful feature as it ensures that the gems of one application are isolated from those of another.

To utilize the Gemset feature, these are the commands you need to run on the console before actually generating the Rails project:

$ rvm gemset create [app_name]
$ rvm gemset use [app_name]
$ gem install bundler
$ gem install rails

I wanted to automate these steps and somehow merge it into the template generation process. To accomplish this, I created a bootstrap.rb file that employs the RVM API.

rvm_lib_path = "#{`echo $rvm_path`.strip}/lib"
$LOAD_PATH.unshift(rvm_lib_path) unless $LOAD_PATH.include?(rvm_lib_path)
require 'rvm'

rvm_ruby = ARGV[0]
app_name = ARGV[1]

unless rvm_ruby
  puts "\n You need to specify a which rvm ruby to use."
end

unless app_name
  puts "\n You need to name your app."
end

@env = RVM::Environment.new(rvm_ruby)

puts "Creating gemset #{app_name} in #{rvm_ruby}"
@env.gemset_create(app_name)
puts "Now using gemset #{app_name}"
@env.gemset_use!(app_name)

puts "Installing bundler gem."
puts "Successfully installed bundler" if @env.system("gem", "install", "bundler")
puts "Installing rails gem."
puts "Successfully installed rails" if @env.system("gem", "install", "rails")

template_file = File.join(File.expand_path(File.dirname(__FILE__)), 'templater.rb')
system("rails new #{app_name} -JT -d mysql -m #{template_file}")

The bootstrap.rb file first runs the RVM setup steps and then loads the rails template file. So to create your new Rails app using RVM, simply run

$ ruby /path/to/rails-templater/bootstrap.rb [rvm-ruby] [name of new app]

If you'd like to use my template as a starting place for your own, you can get it here.

Dynamic Nested Forms in Rails 3

2010-10-07T00:00:00-07:00

Update: The nested_form gem now also works for deeply nested forms. For more details, read Dynamically Nesting Deeply Nested Forms

Some time ago Ryan Bates did a Railscasts episode on dynamic nested forms. He later packaged the code into a plugin that was able to add and remove nested model fields dynamically through JavaScript. Not only is this plugin really light weight, but once installed, its functionality can be incorporated into any form in your app. Some folks have forked the project and started porting it to Rails 3.

After making a few minor tweaks, I converted the plugin to a gem that is fully tested and compatible with Rails 3. If you'd to use this gem in one your Rails 3 apps, you can either clone the gem locally or link to my github repository by adding the following line to your Gemfile:

#Gemfile
gem "nested_form", :git => "git://github.com/madebydna/nested_form.git"

After running bundle:install, issue following command from the root of your app:

$ rails generate nested_form:install

This will generate the nested_form js file which needs to be included after the main jQuery file in your layout:

<%= javascript_include_tag 'jquery', 'nested_form' %>

Whenever you have a nested form that you wish to "ajaxify" (it actually doesn't use ajax at all, since the server never gets called), just use the new form helper called nested_form_for instead of form_for.

<%= nested_form_for @project do |f| %>

Don't worry, this does not override any default behaviour of the form helper. After merging a custom form builder into the options hash, nested_form_for forwards the call to the regular form_for method . What you do get by using nested_form_for, however, are two new methods that enable the dynamic behavior. Those are:

link_to_add
link_to_remove

And here's how to use them in your nested_form helper:

<%= f.fields_for :tasks do |task_form| %>
  <%= task_form.text_field :name %>
  <%= task_form.link_to_remove "Remove this task" %>
<% end %>
<%= f.link_to_add "Add a task", :tasks %>

Note that link_to_remove needs to be called on the form builder instance of the nested form (here called task_form), while link_to_add gets called on the form builder instance of the parent form (here called f) and is placed outside the nested form.

Even though the gem is meant to just be plugged and played, I found it intriguing to peek under the hood to dissect how it works. A plugin such as this one adds an additional layer of magic and convenience on top of Rails. However, it could not have been built without a solid understanding of how the Rails source code generates its forms.

A Detour Into Source Code For Generating Rails Forms

The form helpers in Rails automagically generate the HTML necessary to ultimately produce a neat params hash, so that the model object can be saved to database.

The very much simplified diagram below attempts to introduce the major players in generating forms with Rails.

Whenever you create a new form with the form_for method, the code for which can be found in rails/actionpack/lib/action_view/helpers/form_helper.rb, the method figures out what model object the form is meant for and extracts any options, such as the url the form should be submitted to. However, it only extracts what is needed to create the form tag. The rest of the arguments, including the block which defines the actual form fields are sent on to a fields_for method.

The fields_for method of the FormHelper module

This fields_for method is not the one we commonly see in nested forms view templates. The nested fields_for is an instance method of the FormBuilder class whereas the fields_for method that the parent form_for delegates to is housed in the FormHelper module (see diagram). Here is how the Rails source documentation explains its function:

"[fields_for] creates a scope around a specific model object like form_for, but doesn't create the form tags themselves. This makes fields_for suitable for specifying additional model objects in the same form."

  
module ActionView::Helpers::FormHelper
# .....

def fields_for(record_or_name_or_array, *args, &block)
  raise ArgumentError, "Missing block" unless block_given?
  options = args.extract_options!

  # figure out object and object_name form record_or_name_or_array

  builder = options[:builder] || ActionView::Base.default_form_builder
  capture(builder.new(object_name, object, self, options, block), &block)
end

# ....
end

From the code above we can see that the parent fields_for method instantiates a FormBuilder object with a bunch of arguments, including self, which is an instance of ActionView::Base, the class that the FormHelper module will be included into. Note that if you have specified a custom form builder in the form_for call (:builder => MyCustomBuilder), it will be used instead of the default form builder.

The last line of the fields_for method "captures" the entire html of the form after evaluating the block that is defined between the do ... end of our form_for invocation.

The first argument passed to capture is the instance of the form builder and the second is the block itself. The capture method yields the FormBuilder instance to the block as the only argument (see diagram). This FormBuilder object - by convention often named f or form - comes equipped with all the form helper methods, such as text_field, label, select that allow for the construction of form tags customized to the model object that the FormBuilder represents.

The fields_for method of the FormBuilder class

One of the FormBuilder's methods is the familiar nested fields_for used to generate nested form fields. The two fields_for methods share their name because they have a similar purpose. Both are responsible for changing scope, that is changing the model object that the form tags are being built for.

Where they differ is that the FormBuilder's fields_for method also needs to determine the relationship between the model object currently in scope and the other one passed in as a parameter.

Once it has done that, it passes its arguments right back to the fields_for method of the FormHelper, which will now yield a new form builder to the block of the nested form. This also allows to switch FormBuilders mid-stream, for instance.

module ActionView::Helpers  
  class FormBuilder  
   def fields_for(record_or_name_or_array, *args, &block)
     # ....
  
     # option to switch form builders for nested form  
     if options[:builder]
       args << {} unless args.last.is_a?(Hash)
       args.last[:builder] ||= options[:builder]
     end

     case record_or_name_or_array
     # ....
     # code to check how record_or_name_or_array is associated with the model 
     # currently in scope
     # constructs the name variable as a string depending on the findings
     # ...
     end
 
     # @template is an instance of the ActionView::Base class
     @template.fields_for(name, *args, &block)
   end
 end
end

How the nested_form Plugin Works

The nested_form basically creates a blueprint for the nested form fields and places it in a hidden division right after the form. A click on the link created by link_to_add will insert an additional nested form through jQuery after the last nested form of its kind.

As we have seen, the nested_form plugin has a custom form builder that sports two additional methods link_to_add and link_to_remove. To employ this alternate form builder, we call upon the services of a new view helper, called nested_form_for.

module NestedForm::ViewHelper

  def nested_form_for(*args, &block)
    options = args.extract_options!.
              reverse_merge(:builder => NestedForm::Builder)
    output = form_for(*(args << options), &block)
    @after_nested_form_callbacks ||= []
    fields = @after_nested_form_callbacks.collect do |callback|
      callback.call
    end
    output << fields.join('').html_safe
  end

  def after_nested_form(&block)
    @after_nested_form_callbacks ||= []
    @after_nested_form_callbacks << block
  end

end

In the code above, we can see that nested_form_for basically merges the :builder option into the form and then passes the arguments on to the regular form_for method. It also iterates over an array of proc objects, calling each each of them in turn. The resulting output is captured and displayed in the template after the form closes.

It is this hidden blueprint of the nested form fields that will get tacked on after the form closes. But how is this accomplished or, to be precise, where does @after_nested_form_callbacks get populated and with what?

To answer these questions we need to look at the code for link_to_add in the custom FormBuilder. In the end, it returns a link that will be picked up by jQuery and will trigger the dynamic behavior of inserting nested forms. Before that happens, though, the method also calls @template.after_nested_form (a method in NestedForm::ViewHelper) which effectively populates the @after_nested_form_callbacks in our custom view helper with the content of the nested form.

module NestedForm
  class Builder < ActionView::Helpers::FormBuilder

    def link_to_add(name, association)
      @fields ||= {}
      @template.after_nested_form do
        model_object = object.class.reflect_on_association(association).klass.new
        output = %Q[<div id="#{association}_fields_blueprint" style="display: none">].html_safe
        output << fields_for(association, model_object, 
                  :child_index => "new_#{association}", 
                  &@fields[association])
        output.safe_concat('</div>')
        output
      end
      @template.link_to(name, "javascript:void(0)", 
                        :class => "add_nested_fields", 
                        "data-association" => association)
    end

  end
end

A Neat Trick

If we look closely, we can see that the most important part of the code, namely the block to be sent into the fields_for method depends on @fields[association], which we've initialized as an empty Hash. The process through which @fields[association] gets a value is based on a simple slight of hand but one that requires familiarity with the ActionView source code.

module NestedForm
  class Builder < ActionView::Helpers::FormBuilder

    def fields_for_with_nested_attributes(association, args, block)
      @fields ||= {}
      @fields[association] = block
      super
    end


    def fields_for_nested_model(name, association, args, block)
      output = '<div class="fields">'.html_safe
      output << super
      output.safe_concat('</div>')
      output
    end

  end
end

The two methods in the code snippet above, fields_for_with_nested_attributes and fields_for_nested_model override the default behavior of FormBuilder objects. They are both being invoked in the course of switching the scope to the child model when fields_for is called. The responsibility of these methods is basically to figure out names and indexes of the child models.

The way they are used here has nothing to do with their core functionality, however. That part gets taken care of by a call to the super class. The crux is that they also afford an opportunity to slurp up the block that was sent to fields_for and save it for later use in the @fields Hash, properly keyed with the name of the child association. So while the form goes about its business creating form fields for the nested model we sneakily snatched a copy of the block and saved it. Note that this happens before link_to_add is called. When it is called eventually it is however equipped with the information it needs to create the hidden blueprint for the nested form. Quite slick indeed.

Introduction to the Pathname Ruby Standard Library

2010-09-23T00:00:00-07:00

This tutorial on the Pathname Ruby Standard Library was originally created as an assignment for the Ruby Mendicant University (RMU), organized by Gregory Brown. For those who don't know yet, RMU is an awesome, community funded school aimed at helping intermediate Ruby developers on their path towards excellence. You can gain "alumni" status after an intense three week course that is sure to push the current limits of your Ruby mojo.

Other students in my RMU session were also tasked with producing tutorials on different, less documented Ruby Standard Libraries. You can view those tutorials on github.

Looks Like a String, Works Like a Path

As its name implies, a Pathname object facilitates access to files and directories on the file system. It does this by holding a string pointer - effectively a @path instance variable - in memory that can be used to query or manipulate the files or directories that the instance represents. Pathname is at heart a very simple class with few core methods of its own. The majority of these core methods don't actually access the file system, but provide specialized string parsing functionality. The root? method, for example, basically checks to see whether the pathname begins with consecutive slashes.

However, in addition to these core methods, the Pathname class sports a slew of convenience methods that essentially serve as a proxy for the powerful family of Ruby IO related classes and libraries, such as File, FileTest, Dir and FileUtils. If you know your way around these libraries, then many methods of Pathname will be familiar to you, as they share the same names as those to which they are delegating the method calls to. By way of example, the instance method Pathname#exist? is a wrapper for FileTest#exist?:

def exist?() FileTest.exist?(@path) end

Keeping this method forwarding in mind, most of the time you can simply guess what methods are available and how they work. However there are some gotchas, so it's always advisable to have the documentation handy. The following does not work as expected:

p = Pathname.new('/tmp')
p.mkdir('dir123')

TypeError: can't convert String into Integer
from /usr/lib/ruby/1.9/pathname.rb:974:in `mkdir'
# ... etc

So what went wrong? We have an instance of Pathname and mkdir is a viable instance method, courtesy of the Dir class. However if we look at how mkdir is implemented, the reason for the error becomes obvious:

def mkdir(*args) Dir.mkdir(@path, *args) end

The arguments that the Pathname version of mkdir are equivalent to the second parameter to the Dir.mkdir method, which optionally expects directory permissions as integers. So here's how we can make it work as we intend to:

p = Pathname.new('/tmp/dir123')
p.mkdir

How Pathname Works

Pathname is a simple yet powerful convenience class that unifies the functionality of file handling libraries in a way that is supposed to be logical and intuitive. While you would usually have to switch back and forth from manipulating strings and calling upon the class methods of File, Dir and others, with Pathname you have it all in one neat package.

1. Core Methods and String Manipulation

As mentioned above, the core methods are for the most part clever algorithms to query and manipulate strings that are assumed to represent a path. They range from the most simple, like the + and the join methods, to the seriously cool, like for example ascend, descend or cleanpath. Here's a quick demo of some of these at work:

# Joining paths with the "+" operator

path = Pathname.new("/home/andev")
path2 = path + "/tmp/file.txt"
# => #<Pathname:/home/andev/tmp/file.txt>

# Joining paths with the "join" method
path3 = path.join("tmp/file/", "one.txt")
# => #<Pathname:/home/andev/tmp/file/one.txt>

# Ascend 
path3.ascend do |dir|
  puts "Next directory up: #{dir}"
end

Next directory up: /home/andev/tmp/file/one.txt
Next directory up: /home/andev/tmp/file
Next directory up: /home/andev/tmp
Next directory up: /home/andev
Next directory up: /home
Next directory up: /
# => nil
# Descend is basically just the reverse of ascend

# Cleanpath
path4 = path + "rails_sites/../sinatra_sites"
=> #<Pathname:/home/andev/rails_sites/../sinatra_sites>
path4.cleanpath
=> #<Pathname:/home/andev/sinatra_sites>
# Is that neat, or what? "cleanpath" is smart enough to remove 
# the redundancy of going in and back out of a folder

Note that since these core methods only manipulate strings, they don't check whether the file represented by the string actually exists or not. That comes into play when we trigger the wrapper methods to the other IO libraries.

2. File and Directory Status, Property and Manipulation Methods

When the time comes that you want to use the Pathname instance to actually access the file system, whether it be to inquire about the existence or properties of a file or directory, or maybe to create some new folders or write to a file, then the full arsenal of File and FileTest methods and many of Dir and FileUtils are at your fingertips. This makes it invaluable for any file heavy scripting tasks that require a lot of back and forth between directories.

# Querying File Objects

path = Pathname.new("/home/andev")
path.directory?
# => true
path.file?
# => false

# Creating directories and files

path2 = path + "tmp/one"
# => #<Pathname:/home/andev/tmp/one>
path2.mkpath
# This will create two new nested directories "tmp/one"

# This will in one step create a new file, open it in write mode 
# and write "Hello from Pathname" into it
(path2 + "text.txt").open("w") do |file|
  file.write "Hello from Pathname"
end

Before we move on to a complete scripting example, it is interesting to note where the versatility of Pathname comes from. One oddity about Pathname is that the instance variable @path that it holds actually never changes, no matter what you do with the instance. The + and join methods don't concatenate to the @path variable, as one might expect, but instead return a new Pathname object. The same goes for the ascend and descend methods. Methods like Pathname#entries returns the files and subdirectories in the directory that @path points to, all as new Pathname objects.

Why is this important? This becomes clear if we consider what would happen if join for instance did not return another Pathname object, but a string that represents the result of the joining, for instance. To work with this new string as a path, one would need to either assign it to a new variable or create a whole new Pathname object. But since the method already returns a new Pathname object, it is ready to be used or queried as one. In other words, this enables chainability, which turns out to be very convenient.

A Practical Example

Imagine the following scenario. You have thought about changing your rails apps over to Sass from CSS, but resisted it because it would have meant translating your existing CSS files into the Sass syntax. With the advent of Sass 3, however, a new syntax known as SCSS has been introduced, which is fully compatible with the syntax of CSS3, while still supporting the full power of Sass. Now all you need to do is change the file extension of your CSS files from ".css" to ".scss".

The following simple script demonstrates how you can leverage the Pathname library to achieve just that.

# scss.rb

require 'pathname'

CURRENT_DIR = Pathname.getwd
project = Pathname.new(ARGV[0])

test = CURRENT_DIR.join(project) 

unless test.exist?
  puts "\nThe path you supplied doesn't seem to exist."
  exit
end 

Pathname.glob(test + "public/stylesheets/**/*.css") do |dir|
    base = dir.realpath.parent + dir.basename(".css") 
    dir.rename("#{base}.scss")
end

You use the script like this:

ruby scss.rb path/to/my_rails_app

This script is quite trivial, but might give you an idea where Pathname would be useful. The Pathname.getwd method gets the absolute location of the script. The argument to the script is expected to be the relative path from this directory to your rails app. After joining the CURRENT_DIR and the user supplied path, we run a quick check to make sure the path is correct.

Pathname.glob then begins searching for files that end in .css and are located in the public/stylesheets folder of the rails app. The notation '**/*' is used to tell Pathname that we want to search recursively through every subdirectory, scanning for the pattern as we go. When supplied a block, the glob method yields the newly minted Pathname object for further processing.

# File pathname.rb, line 762
  def Pathname.glob(*args) # :yield: p
    if block_given?
      Dir.glob(*args) {|f| yield Pathname.new(f) }
    else
      Dir.glob(*args).map {|f| Pathname.new(f) }
    end
  end

The rename method destructively renames the file pointed to by the @path variable to the new name that is expected as the only argument. One caveat is that the new "name" needs to be the absolute path to the file and not just a basename with an extension. dir.basename(".css") returns the name of the file, minus the extention, e.g. style, while dir.realpath.parent gets the absolute path up to the directory that the stylesheet is located in.

base = dir.realpath.parent + dir.basename(".css") 
dir.rename("#{base}.scss")

Image Batch Resize Using ImageMagick

2010-07-22T00:00:00-07:00

This is a useful little shell script to resize an entire folder of images, using ImageMagick. If you don't specify a target folder, this command will overwrite your existing images.

$ find . -name "*.jpg" -exec convert -scale 50% -quality 75 {} ~/target_folder/{} \;

A Ruby Helper To Cleanly Truncate HTML Text

2010-06-04T00:00:00-07:00

A problem that occasionally crops up, is how to best truncate text with HTML markup, e.g. in order to display the first lines of a blog post or the initial sentences of a product description. The Rails helper method #truncate does not take care of the problem caused by chopped HTML tags or missing end tags.

Things are relatively simple if the markup does not need to be taken into consideration. Then it is just a matter of efficiently stripping out all HTML tags and sending the remainder off to the truncate method.

One way to accomplish this is through regular expressions:

module TextHelper

  TAG_PATTERN = %r{(</?.*?>)}

  def truncate_html(text, max_length = 30, ellipsis = "...")
    tag_free = text.gsub(TAG_PATTERN, '')
    truncate(tag_free, :length => max_length, :omission => ellipsis)
  end

end

In certain circumstances, though, it is desirable to retain the HTML formatting. So we need an HTML-aware truncator, one that would only use the actual text to determine content length, make sure that all the HTML tags are properly closed and that HTML entities, such as &, are left intact.

For example, if we wanted to truncate the following markup

  This text is <strong>bold</strong> and <i>beautiful</i>.

to 15 characters, then we expect this as a result:

  This text is <strong>bo</strong>

Googling for a solution, I found a blog post Rails truncate helper that handles HTML tags and entities by Henrik Nyh with a great looking code snippet, that uses Hpricot to accomplish just that. However I had some problems getting this code to work in some test cases, specifically in scenarios involving nested ul and li tags. In the end I could not discern any specific pattern for the failures, so I decided to try and port the same code to Nokogiri and lo and behold everything worked as it should.

There are actually some advantages to traversing the text fragment with Nokogiri: first, it is purportedly faster than Hpricot and secondly, it natively takes care of HTML entities, while Hpricot does not recognize them and thus requires the use of regular expressions. An added bonus with both HTML parsers is that they automatically fix malformed HTML.

So here's the final Nokogiri version:

require "rubygems"
require "nokogiri"

module TextHelper

  def truncate_html(text, max_length, ellipsis = "...")
    ellipsis_length = ellipsis.length     
    doc = Nokogiri::HTML::DocumentFragment.parse text
    content_length = doc.inner_text.length
    actual_length = max_length - ellipsis_length
    content_length > actual_length ? doc.truncate(actual_length).inner_html + ellipsis : text.to_s
  end

end

module NokogiriTruncator
  module NodeWithChildren
    def truncate(max_length)
      return self if inner_text.length <= max_length
      truncated_node = self.dup
      truncated_node.children.remove

      self.children.each do |node|
        remaining_length = max_length - truncated_node.inner_text.length
        break if remaining_length <= 0
        truncated_node.add_child node.truncate(remaining_length)
      end
      truncated_node
    end
  end

  module TextNode
    def truncate(max_length)
      Nokogiri::XML::Text.new(content[0..(max_length - 1)], parent)
    end
  end

end

Nokogiri::HTML::DocumentFragment.send(:include, NokogiriTruncator::NodeWithChildren)
Nokogiri::XML::Element.send(:include, NokogiriTruncator::NodeWithChildren)
Nokogiri::XML::Text.send(:include, NokogiriTruncator::TextNode)

You can add this code as a helper file in RAILS_ROOT/app/helpers/text_helper.rb

Setting Up USPS Shipping In Spree

2010-05-26T00:00:00-07:00

Integrating shipping via a carrier API (USPS, Fedex, UPS, etc.) with the rails e-commerce platform Spree can be a bit daunting at first. However, once it is understood how the different parts fit together it turns out to be quite easy. The active_shipping extension does all the heavy lifting and it is just a matter of figuring out how to configure it for your particular needs.

In this post I will outline how I set up the active_shipping extension for USPS, but it should be very similar for other carriers.

Shipping Methods, Zones and Calculators

To handle shipping at checkout you need to make one or more shipping methods available. A shipping method has a customer-facing descriptive name, is associated with a particular zone and it uses a calculator. A zone is for the most part a grouping of countries that you pre-define in the admin section. It is used to determine which shipping methods are available for individual orders. A calculator houses the business logic that is responsible for computing the final shipping total.

You can either write your own calculator, enable one of the basic calculators that ship with the Spree gem (for example Calculator::FlatRate) or you can use the calculators provided by an extension, such as the active_shipping extension. The USPS exposes its various delivery services, such as First Class, Media Mail, Priority Mail and so on, through an API. The active_shipping extension uses the active_shipping gem to interface with carrier APIs and ultimately creates usable Spree-type calculators.

For each USPS delivery service you want to offer (e.g. "USPS Media Mail") then, you need a corresponding Shipping Method (set up through the admin panel) and a matching calculator (registered in the active_shipping extension) that ties the delivery service and the shipping method together.

Enabling Shipping Via USPS

Once you've installed the active_shipping extension, you will have several USPS shipping calculators at your disposal that you can now associate with shipping methods. Here is the list of the USPS calculators that the spree-active-shipping extension comes with out of the box:

    
  #in vendor/extensions/active_shipping/active_shipping_extension.rb 
  def activate
    [ 
    #... calculators for Fedex and UPS ...
        Calculator::Usps::MediaMail,
        Calculator::Usps::ExpressMail,
        Calculator::Usps::PriorityMail,
        Calculator::Usps::PriorityMailSmallFlatRateBox,
        Calculator::Usps::PriorityMailRegularMediumFlatRateBoxes,
        Calculator::Usps::PriorityMailLargeFlatRateBox
      ].each(&:register)
  end

Before you can add shipping methods you need to set up zones. I set up two zones, one for domestic shipping - containing only the United States - and a second one for international shipping consisting of all other countries. This setup reflects the basic distinction that is used by the USPS.

With zones in place we can now start adding some shipping methods through the admin panel. The only other essential requirement to calculate the shipping total at checkout is that each product and variant be assigned a weight.

The active_shipping gem needs some configuration variables set in order to consume the carrier web service. Among other things, it needs the API username and the origin location:

  Spree::ActiveShipping::Config.set(:usps_login => "YOUR_USPS_LOGIN")
  Spree::ActiveShipping::Config.set(:origin_country => "US")
  Spree::ActiveShipping::Config.set(:origin_state => "HI")
  Spree::ActiveShipping::Config.set(:origin_city => "Pahoa")
  Spree::ActiveShipping::Config.set(:origin_zip => "96778")
  # these can be set in an initializer in your site extension

Adding Additional USPS Calculators

If the active_shipping extension has not set up a calculator for a particular USPS delivery service that you would like to offer, you can easily add it yourself. For example I needed a First Class domestic and some international calculators, which are not available by default.

The first step is to create a calculator class for each delivery service you want to add, making sure it inherits from Calculator::Usps::Base:

#in vendor/extensions/site/app/models/calculator/usps/first_class_mail_international_parcels.rb
class Calculator::Usps::FirstClassMailInternationalParcels < Calculator::Usps::Base
  def self.description
    "USPS First-Class Mail International Package"
  end
end

Unlike calculators that you write yourself, these additional calculators do not have to implement a #compute instance method that returns a shipping amount, but only need the one description class method, since the superclasses take care of the rest.

However, there is one gotcha to bear in mind. The string returned by the description method must exactly match the name of the USPS delivery service returned by the API call. So how do you find out the exact name of the delivery service? One way to do this is to inspect what gets returned by the API call. Take a look at this excerpt of the code where most of the action takes place:

#vendor/extensions/active_shipping/app/models/calculator/active_shipping.rb  
class Calculator::ActiveShipping < Calculator
  # ...
  def compute(line_items)
   #....  
   rates = retrieve_rates(origin, destination, packages(order))
   # here you can raise the rates hash  

   return nil unless rates
   rate = rates[self.description].to_f + (Spree::ActiveShipping::Config[:handling_fee].to_f || 0.0)
   return nil unless rate
   # divide by 100 since active_shipping rates are expressed as cents

   return rate/100.0
  end  

  def retrieve_rates(origin, destination, packages)
   #....
   # carrier is an instance of ActiveMerchant::Shipping::USPS
   response = carrier.find_rates(origin, destination, packages)
   # turn this beastly array into a nice little hash
   h = Hash[*response.rates.collect { |rate| [rate.service_name, rate.price] }.flatten]
   #....
  end
  # ...
end

Depending on the carrier, in our case USPS, the active_shipping gem returns an array with the name of the service and the price, which the retrieve_rates method converts into a nice hash. The key equals the name of the shipping service and the value is set to the shipping cost. Here is an example of what gets returned for an order with an international destination:

#rates  
{"USPS Priority Mail International Flat Rate Envelope"=>1345, 
"USPS First-Class Mail International Large Envelope"=>376, 
"USPS USPS GXG Envelopes"=>4295, 
"USPS Express Mail International Flat Rate Envelope"=>2895, 
"USPS First-Class Mail International Package"=>396, 
"USPS Priority Mail International Medium Flat Rate Box"=>4345, 
"USPS Priority Mail International"=>2800, 
"USPS Priority Mail International Large Flat Rate Box"=>5595, 
"USPS Global Express Guaranteed Non-Document Non-Rectangular"=>4295, 
"USPS Global Express Guaranteed Non-Document Rectangular"=>4295, 
"USPS Global Express Guaranteed (GXG)"=>4295, 
"USPS Express Mail International"=>2895, 
"USPS Priority Mail International Small Flat Rate Box"=>1345}

From all the possible shipping services, only the one that matches the description of the calculator gets selected. At this point an optional flat handling fee (set via preferences) can be added:

rate = rates[self.description].to_f + (Spree::ActiveShipping::Config[:handling_fee].to_f || 0.0)

Finally, don't forget to register the new calculators you added. In extensions, this is accomplished in the activate method:

# vendor/extensions/site_extension.rb 
def activate
  [ Calculator::Usps::FirstClassMailInternationalParcels,
    Calculator::Usps::PriorityMailInternational,
    Calculator::Usps::FirstClassMailParcels
  ].each(&:register)
end

Changing Availability of Shipping Methods on Criteria Other Than the Zone

Ordinarily it is the zone of the shipping address that determines which shipping methods are displayed to a customer at checkout. Here is how is the availability of a shipping method is determined:

  class Checkout < ActiveRecord::Base
    #...
    def shipping_methods 
      return [] unless ship_address
      ShippingMethod.all_available(order)
    end
    #...
  end

  class ShippingMethod < ActiveRecord::Base
  #.....
    def available?(order)
      calculator.available?(order)
    end

    def available_to_order?(order)
      available?(order) &amp;&amp; zone &amp;&amp; zone.include?(order.ship_address)
    end

    def self.all_available(order)
      all.select { |method| method.available_to_order?(order)}
    end
  end

Unless overridden, the calculator's #available? method returns true by default. It is the zone of the destination address that filters out the shipping methods. However, in some circumstances it may be necessary to filter out additional shipping methods.

For example, I needed to remove the First Class domestic shipping service for orders where the combined shipping weight was over 13oz, since First Class is only available up to 13oz. Even though the USPS API does not return First Class in its array of options, nonetheless First Class will appear as an option in the checkout view with a value of 0, since it has been set as a Shipping Method.

In order to remove the First Class shipping method from orders that weigh more than 13oz, I needed to override the calculator's #available? method:

  class Calculator::Usps::FirstClassMailParcels < Calculator::Usps::Base
    def self.description
      "USPS First-Class Mail Parcel"
    end

    def available?(order)
      multiplier = Spree::ActiveShipping::Config[:unit_multiplier]
      weight = order.line_items.inject(0) do |weight, line_item|
        weight + (line_item.variant.weight ? (line_item.quantity * line_item.variant.weight * multiplier) : 0)
      end
      #if weight in ounces > 13, then First Class Mail is not available for the order
      weight > 13 ? false : true
    end
  end