Reusable filtering plug-in in Rails

Introduction

Filtering large sets of data is a common interaction in today's web applications. Examples include filtering web pages (e.g. Google), filtering e-mails (e.g. Gmail), filtering flights (e.g. Kayak) and filtering jobs (e.g. monster). The user is usually interested in a subset of the data, which can be specified by various types of constraints. The most common one is probably finding items matching a set of search terms. But other constraints are also found, such as constraining the result set by a category or a set of categories (e.g. jobs in IT) or by ranges (e.g. gifts with a price between $10 and $20, flights leaving SFO between 4pm and 6pm).

HTML provides various form elements used to build these interactions. The text input is probably the most used element, but check boxes, radio buttons and drop-down lists are also available. Interestingly, there is no built-in element to select ranges. The main reason is probably that there is no right way to represent a range. For instance, selecting a range of dates should be done with a calendar-like widget while selecting a price should be done with a simple slider. Furthermore, rendering an element to select ranges usually requires some knowledge about the data it selects (e.g. the bounds). In this project, we are building a histogram-based slider for selecting numeric ranges. We are especially interested in studying how the element can be packaged and reused in different applications. Is it possible to make it as easy to use as a built-in form element such as a text input?

Interaction

Histogram-based sliders have an advantage over simple sliders. They provide the user some information about the data distribution as well the number of results he is filtering.

Compare

to

Implementation

Our interaction is implemented in a combination of JavaScript and ruby on the Rails 2.2 platform. The interaction is packaged as Rails plug-in for ease of installation and maintenance.

Client side

The client side is primarily JavaScript built on the Prototype framework to handle the user interface. The client side code detects when a user starts a click near one of the handles and starts the interaction. As the user drags the mouse, the overlay is redrawn, snapping to bar boundaries to the underlying histogram. When the user releases the mouse, the client side code updates hidden form fields and optionally submits the form. In addition, the JavaScript code provides some neat usability features such as a label showing the value of the current boundary as well as a special mouse cursor hinting that the boundaries are draggable.

The JavaScript code is approximately 300 lines long. A fair amount of the code is there to handle corner cases properly (such as avoiding to move a boundary past the other one or out of the chart) and to snap a boundary to the right position. Furthermore, we have been especially careful to cache computations to avoid traversing the DOM or computing boundary positions to frequently.

Server side

Extensions to ActiveRecord

• The new_search_with_histogram method processes the POST and GET parameters and returns the filtered results. It is called in the controller, and the result is passed to the view.
• The histogram method calculates the bucket sizes by issuing SELECT COUNT(*)... SQL statements for a given model and column. It is used by our helper method. The user doesn't have to know about this method, unless he has to compute the bucket sizes with a different method (e.g. without SQL).

Extensions to ActionView

• The sparkline helper method generates the DIVs required to render the histogram and calls the constructor of our JavaScript class. It is used in the view.

Google Chart API

The background image of the histogram is rendered using the Google Chart API. The advantages and disadvantages of this approach are discussed in the last section.

Encapsulation and reusability

Our interaction is relatively complicated to encapsulate since it spans the server-client divide. We need to add custom code to the Model, the Controller and the View as well as package JavaScript and CSS files. Packaging files is easy as we simply have to copy and include them. Adding code to the various MVC components could be painful for the user if we were using a less dynamic language than Ruby (i.e. he would have to wrap of lot of the objects with custom wrappers). To solve these two problems, we are taking advantage of the advanced language features of Ruby and the Rails plug-in architecture, as describe below.

Benefits from the framework facilities

• Rails has a plug-in architecture with a console script script/plugin that allows installation directly from a svn repository, as shown during the class presentation. This makes retrieving and installing a plug-in easy. The plug-in system also provides callbacks on install and uninstall (i.e. install.rb and uninstall.rb are run at appropriate times). This allows us to copy our JavaScript and CSS files into Rails' public directory.

• Ruby does not close classes. This means that we can add methods to a previously defined class by simply reopening the class. We make extensive use of this feature to add our code to preexisting classes. Data related methods (i.e. histogram generation and filtering) are added to the ActiveRecord class. The HTML and JavaScript code required to initialize a histogram is generated by a method added to the ActionView class. The plug-in system provides a callback when the application is loading (i.e. init.rb is run at the appropriate time). This allows us to modify the ActiveRecord and ActionView classes when the server is starting.

• Ruby's metaclass support (which allows methods to be added to particular instances of a class) is used to pass data without the explicit knowledge of the developer. In order to hide the internal details of our system, we do not want to force the developer to explicitly pass variables between various components (e.g. from the model to the view through the controller). This problem is made especially difficult given the shared-nothing architecture of Rails. For example, we must keep the histogram boundary locations in addition to the values at those locations in order to preserve the state of the control after the user executes a search. We do this by adding methods and variables to the instance of the Model returned from our ActiveRecord methods. As the developer passes the results to the view, our state information is similarly passed.

Usage

Assuming that the main Model we want to filter is called Student.

1. Install the plug-in into your Rails application:

script/plugin install http://rails-histogram.googlecode.com/svn/trunk/

2. Include the following in the desired method of the controller:

@search = Student.new_search_with_histogram(params)

3. Include the following in view:

<%= sparkline @search, :gpa, options... %>

Optional parameters (options)

key	type	default	description
:buckets	int	10	Number of buckets for histogram
:width	int	400	Width of histogram image in px
:height	int	100	Height of histogram image in px
:space	int	0	Space between bars in the histogram image
:rescale	boolean	true	Use log scale for y-axis when true, linear axis otherwise
:submit	string	'search_submit'	id of submit button to use after histogram is manipulated

4. Include /public/javascript/sparkline.js and /public/stylesheet/sparkline.css in layout

These files are copied when the plug-in is installed.

Critique

Strengths

• Our interaction is easy to install. Server side code, JavaScript, and CSS are all modular and encapsulated by our Rails plug-in. As a result, the developer does not need to write a single line of JavaScript or CSS to make the interaction work out of the box.
• Our interaction is as decoupled from the form than a built-in element such as a text input. We don't need to use a special kind of form, nor do we have to do some complex wiring between the histogram and the form. The histogram can be added to any existing form without modifying the rest of the form. Furthermore, the histogram is completely decoupled from the way the form is submitted. If it is an AJAX form, then our interaction is asynchronous. If it is a traditional form, then our interaction is synchronous.
• The complex logic required to generate the histogram buckets is encapsulated in our plug-in and added to ActiveRecord for free. Even though 90% of our code is client side, the extra 10 percents make the integration of our component a lot easier for the developer.
• Histogram sliders are useful in a lot of applications.

Weaknesses

• The user interface is tightly coupled to the histogram which makes it difficult to change the look of the histogram without having to change the user interface code. Because the overlay snaps to the boundaries of the underlying histogram, the JavaScript must know intimate details about the histogram underneath (e.g. width of each bucket, space between buckets, borders around each bar, margins, etc.). If the user changes the histogram generation (or the Google Chart API changes) then the JavaScript must be changed accordingly, which defeats the encapsulation. Two solutions to this problem are available:
  • Instead of relying on the Google Chart API, we could provide some server side code generating the image. It would basically do exactly what the Google Chart API is doing, but would provide fine grained control on how the histogram is rendered. The disadvantage of this approach is that generating the image requires the usage of some complex image generation libraries such as ImageMagick.
  • Instead of rendering an image, we could create the histogram using DIVs. This approach is very fast and probably quite easy to implement. Furthermore, it could be entirely configured using CSS, which is nice. However, if the number of buckets is high or if there are many histograms on the page, this approach could potentially use a lot of memory and slow down the browser.
• Our interaction is platform dependent. Do the extra 10 percents aforementioned worth the fact that it only works on Rails? One could reuse only our JavaScript and write his own back-end, though.
• Histogram sliders are useful in a lot of applications, but not in all applications. This input method is more specialized than, say, a text box and has thus less use cases. What the outcome of this project is hopefully proving is that it is possible to create reusable form elements requiring some knowledge about the back-end data and, thus, spanning the server-client divide. The histogram-based slider is only an example of such reusable elements.