Ruby on Rails or: How I Learned to Stop Worrying and Love Web Application Testing

Matthew Williams Department of Computer Sciences

Florida Institute of Technology mwilli02@fit.edu

Ruby on Rails [RoR], a fairly new web development framework that utilizes the MVC design pattern, includes a very important and powerful set of testing tools. RoR pushes the developer to use these tools by automatically generating test files which can be immediately used to design test cases with. A feature like this is what sets RoR apart from aging web technologies. With more and more applications being ported to the web, it is time to begin treating them like true applications which means extensive testing.

Keywords: Web Development, Ruby, Ruby on Rails,

Testing Frameworks, Web Testing

1. Introduction

Ruby on Rails [RoR], an accidental web

development framework developed by 37Signals

(http://www.37signals.com), has gained much popularity

since its conception in 2004. The term accidental is used

because unlike many frameworks which are built from

the ground up, RoR was extracted from a 37Signals

product called Basecamp. The developers at 37Signals

decided to build their own development toolset while

building Basecamp and when the application was

complete, they discovered that the framework could

easily be extracted and used by any web application and

then open sourced it under the MIT license.[1]

The main focus of RoR is convention over

configuration. In most cases you’re doing 80% of what

everyone else is doing and the other 20% is what makes

you stand out. The developers built RoR because there

weren’t any tools available at the time that could

accomplish their tasks in such a way that it would make

them happy. Developers who are happy and who enjoy

developing in the environment they are in will produce

great things.

RoR was developed with a lot of tasks in mind. One

specific functionality that you don’t always find in other

frameworks, but you do in RoR, is the inclusion of

automated test functionality.

Note: Ruby code ahead!

1.1 All thanks to Ruby

RoR as you might be able to guess is built on top of

the Ruby programming language. It’s an interpretive

language that was developed in Japan in 1995. It is

heavily influenced by older languages such as Lisp,

ADA and Perl. It’s also a very beautiful and elegant

language, words you don’t often see when describing a

programming language. 37Signals choose to build Rails

on top of Ruby for these reasons. Ruby is also

completely object oriented. Everything you encounter in

Ruby can be treated as an object which makes it very

powerful and very easy to use, not to mention very

readable.[8]

Ruby comes standard with testing libraries.

Test/unit provides extensive unit testing and shares many

of the same attributes as other unit testing frameworks

available for other programming languages. Not only

are you able to develop extensive unit tests but you are

open to functional testing and integration testing as well.

Unit testing will generally be to test the logic

throughout your application. Calculations and

validations will most often be the most prevalent in your

tests as they are easy to develop to get your application

tests growing.

Functional tests will be used against your

controllers. You may have methods to post form data or

generate session data, with RoR you can easily generate

functional tests to verify they work.

Integration testing ties everything together. Once

you have core functionality in your application, you can

SWE 5411 (NG) Florida Institute of Technology Summer 2007

write integration tests to check against various

scenarios.[9]

Because all of this capability is a part of every

standard Ruby install, it is available to your RoR

application. Ruby is the key factor in making this all

happen.

2. Automated File Generation

RoR does a great amount of file generation. Using a

variety of scripts, you can easily build your application

while RoR maintains all of your files in a standardized

directory structure. This not only helps out with

readability but also helps from applications going awry

on the developer.

$ruby script/generate model user

exists app/models/

exists test/unit/

exists test/fixtures/

create app/models/user.rb

create test/unit/users_test.rb

create test/fixtures/users.yml

Table 1 - Automated file generation example

2.1 Automated Test Generation

One of the crucial files generated in Table 1 is the

users_test.rb file. RoR does its best to enforce good

development habits, one of which is a large focus on

testing. Each model in your application has a

corresponding test file.

require ‘/../test_helper’

Class UserTest < Test::Unit::TestCase

fixtures :users

# Replace this with your rest tests.

def test_truth

assert true

end

end

Table 2 - Automated test script example

Although this doesn’t generate any test cases for

you, it does provide the ground work to get you started.

At this point you now have access to your model from

the test script and you can begin designing test cases.

3 Ruby on Rails and Test Driven

Development

RoR is very much considered a great platform for

test driven development [TDD]. Before you even begin

writing code, you can start writing your test cases at

which then you can start coding your controller in such a

way that it will pass all your tests. Many developers

adopt a very agile programming methodology based

around the automated tests. By using such a simple

methodology as not committing modified code to a

version control system until that code passes all of its

tests. In doing so you’re maintaining a bug free branch

of your application that other developers can work off of.

3.1 Automating Tests with Rake

Another benefit of RoR is the inclusion of Rake, the

Ruby equivalent of Make. Rake will allow you to

automate many redundant tasks. These tasks could be

basic database maintenance or backups or more

importantly, running tests.

task :runtests do

tests = FileList['test/**/*_test.rb']

tests.each do |file|

m = %r".*/([^/].*)_test.rb".match(file)

puts m[1]+" should:\n"

test_definitions = File::readlines(file).select {|line| line =~ /.*def

test.*/}

test_definitions.each do |definition|

m = %r"test_(should_)?(.*)".match(definition)

puts " - "+m[2].gsub(/_/," ")

end

puts "\n"

end

end

Table 3 - Rake task that automatically runs all

tests[2]

The rake task in Table 3 automatically handles all

your tests in a single location. Running your RakeFile

periodically while actively developing will provide you

feedback on your tests. What’s even better is that it will

run any tests created in the future; this follows the “don’t

repeat yourself” [DRY] methodology which is highly

preached in the RoR community.[2]

3.2 Test Examples

Let’s take for example the case of handling a

password input field when a user registers with our

application. There are a variety of scenarios that we

would want to look out for:

• Is the password more than 4 characters?

• Is the field Nil? (Rubys equivalent to Null)

• Is the password the same as their username?

These are pretty straight forward requirements and

before we begin we can develop our test cases for them.

def test_these_passwords

assert !User.new("name","abc").is_password_safe?

assert !User.new("name","").is_password_safe?

assert !User.new("name","name").is_password_safe?

assert User.new("name","JiEhf").is_password_safe?

end

Table 4 - Example method containing test cases

Notice the beginning of the method is “test”. The

layout the test file is such that any method beginning

with “test” as the first four letters will be run. You could

include as many test methods as you would like inside a

single script and they will each be executed. At this

point we could then create our is_password_safe?

method inside our controller which would test again each

of our cases. Running the test would produce the

following output.

Started

.

Finished in 0.01 seconds.

1 tests, 4 assertions, 0 failures, 0 errors

Table 5 - Test output

Because we were able to implement the correct

checks against the password inside our controller, our

four assertions passed and we could continue

development. If however we forgot to check for a

password that is less than 4 characters we would have

received a failure with a description containing the error

and the line that it occurred at. At that point we could

review our code and make the desired changes so our

code passes our tests.[3]

4 ActiveRecord and Security

One of the key features of RoR is ActiveRecord.

ActiveRecord is a layer that sits between your models

and your database tables. This methodology is referred

to as Object-Relational Mapping [ORM]. ActiveRecord

eliminates the need to write any raw SQL queries against

your database. This makes all RoR applications

completely database independent. More importantly it

makes it more secure.

One of the most exploited features that are difficult

to test against in today’s web applications are SQL

injections. A SQL injection is when a user inserts

standard SQL into a web form, when the form is

submitted the user is then returned more data, often data

that shouldn’t be seen by the user. ActiveRecord

protects RoR applications against this attack. Any raw

SQL that would need to developed could easily be done

inside a model method, protecting it from public access.

This doesn’t mean that RoR applications are 100%

protected against SQL injection attacks but it’s one less

security concern.

Today’s more popular web development languages

such as PHP and Java use raw SQL queries for most all

application that interact with a database. Because of the

potential threat, you are left with having to find a third

party testing tool which can test for such threats. This is

where RoR excels because of its inclusion of testing

frameworks. But what the key issue is SQL injections

are not a concern with a standard RoR application, but

advanced applications do need to beware.[4]

4.1 Testing without ActiveRecord; quickly Finished in 11.541093 seconds.

3001 tests, 5325 assertions, 0 failures, 0 errors

Table 6 – Very fast testing when removing

ActiveRecord from the picture [5]

At Ruby Hoedown, a recent Ruby focused

conference; the concept of running tests without

interacting with ActiveRecord was demonstrated. The

speaker, Dan Manges, describes how they do their

testing without hitting their database, the result is a

highly efficient set of tests that can perform very quickly

and without the expense of database transactions.

“Testing the ActiveRecord framework in addition to your

business logic is unnecessary - ActiveRecord has its own

suite of tests. Therefore, disconnecting from the database

allows you to test the business logic in isolation from the

persistence logic and have really fast tests.” [5] The

testing methodology is quite simple and Dan has

released the framework, UnitRecord, which is available

at http://unit-test-ar.rubyforge.org/.

5 AJAX Without Headaches

Despite being a functionality of browsers for many

years, Asynchronous JavaScript with XML [AJAX] has

become the new fad when it comes to “Web 2.0”

applications. The difficulty with AJAX is the complex

JavaScript it takes to execute it. The purpose of AJAX is

to make a remote call to either POST or GET data

without refreshing the page you are visiting. All of this

is done as separate connections by your browser in the

background.

Because of its popularity, there have been many

AJAX frameworks popping up on the Internet. These

frameworks cut down the tedious JavaScript calls down

to a few lines of code that almost anyone can understand.

RoR takes these frameworks one step further. With the

inclusion of Prototype.js, one of the most popular AJAX

frameworks available to date, RoR has dozens of

methods that are typically a single line of code that can

perform very complex AJAX calls and visual effects.

5.1 RJS Templates

The RoR approach to AJAX is through RJS

templates. These interact with your views to generate

the necessary JavaScript to perform AJAX actions. The

inclusion of RJS templates within your application can

easily make your application more enjoyable to use,

easier to navigate and a much more pleasurable

experience on the eyes.

View:

<h1 id='header'>RJS Template Test</h1>

<ul id='list'>

<li>Foo</li>

</ul>

<%= link_to_remote("Add bar",:url =>{ :action => :add }) %>

Action:

def add

end

add.rjs

page.insert_html :bottom, 'list',

content_tag("li", "Bar")

page.visual_effect :highlight, 'list', :duration => 3

Table 7 - RJS template example

As you can see in Table 7, simply creating an RJS

template that’s named after an action in your controller

will execute when triggered with a click on the page

being viewed. In the example, when the user clicks

“Add bar”, the RJS template inserts a new list item with

the text “Bar” at the end of the unordered list. It then

highlights the list element for three seconds; all without a

page refresh. Doing this by writing raw JavaScript could

easily be dozens, if not hundreds lines of code but the

simplicity of RoR handles the hard work for you.

5.2 Testing RJS One benefit of the RoR community is the amount of

plugins developed that can easily be integrated into any

application at any given time. Out of the box, RoR

cannot effectively test AJAX actions done within an RJS

template. However, a third party plugin, Another RJS

Test System [ARTS] has become available that can test

and verify your AJAX calls are returning the appropriate

data or rendering aspects of the page correctly.[7]

def test_create_rjs

xhr :post, :create, :post => {:title => "Yet Another Post", :body =>

"This is yet another post"}

assert_rjs :insert_html, :bottom, 'posts'

assert_rjs :visual_effect, :highlight, "post_#{assigns(:post).id}"

end

Table 8 - Verifying AJAX functionality with ARTS[7]

ARTS has the functionality to test ~17 RJS functions and

because it is open source you could easily add new RJS

tests to the suite. You can download ARTS at

http://thar.be/svn/projects/plugins/arts/.

6 Code coverage

Code coverage is an integral part of software testing.

With a simple glimpse you can determine how much of

your code is being tested against and provide you per file

test information with detailed information on what lines

in each of your files are being hit the most.

Like many of the plugins previously mentioned,

rcov, an open source tool for Ruby code coverage can be

implemented into your RoR application very easily. It

provides a great amount of detail on your application.

Some of the detail provided is total lines of code per file,

total coverage and code coverage. With the tool you are

able to drill down into each file where you’re presented

with a nicely formatted and color coded version of the

file containing highlighted code that is being called in

tests and corresponding number counts for how many

times that specific line is hit.[10]

Rcov is freely available at

http://eigenclass.org/hiki.rb?rcov and can immediately

be implemented into any existing RoR application so

you can instantly get code coverage information for your

existing tests or to keep track as you develop tests.

7 Beyond Bundled Test Frameworks

The integrated test suite is certainly not everything

and the kitchen sink as far as testing frameworks go.

With RoR as active as it is, many wanted an even better

solution. And so, RSpec was born. As described on its

website, RSpec is “a framework which provides

programmers with a Domain Specific Language to

describe the behavior of Ruby code with readable,

executable examples that guide you in the design process

and serve well as both documentation and tests.”[11]

RSpec is for the advanced RoR developer or those

who might not be satisfied with the included testing

framework. Test cases are extremely literal and anyone

joining the development team could easily determine

what the test is for and why.

it "should return the top item when sent #peek" do

@stack.peek.should == @last_item_added

end

Table 9 - Example RSpec test case[11]

8 Summary

RoR is redefining the way developers write web

applications. It is removing the bloat that you would

most often see in other languages such as J2EE or PHP,

it provides immediate access to almost any database

management system and it comes with a full testing

framework. Many applications are moving from the

desktop and straight to the web. With more capabilities

being added to web browsers and more bandwidth

available to the average user, there isn’t much that can be

done on the desktop that can’t be done on the web. This

is why that developers need to embrace software

development strategies that you would normally see in

use for any given desktop application and apply those

same rules to the web.

When dealing with web applications you need to

assume that the user is completely clueless, or that it’s

completely the other way around and the user is more

advanced than they need to be. Either of these cases

could lead to poorly entered data in your forms or

various forms of exploitation. Following standard

development practices and using a test driven

development methodology will prevent these users from

breaking your application, resulting in a rich and

flawless user experience.

Large companies like Oracle are even almost

completely abandoning the desktop altogether. They

have recently embraced J2EE for developing application

front ends and they have even begun publishing tutorials

on using Oracle with RoR, exposing the option to their

customers as an alternative to J2EE.

Companies like Sun and Apple are noticing the

importance of RoR as well. Sun now supports full RoR

development in their IDE, NetBeans. Developers have

even gone as far as to port the Ruby interpreter to Java

which is now allowing RoR applications to run on a

standard Tomcat server with little to no extra

configuration. Apple will be including RoR in their next

revision of the OS X Operating System, Leopard.

Despite the advancement of many programming

languages, there will almost always be a need for

designing test cases. As projects grow and grow, code

can become more difficult to maintain and simple

changes could easily interfere with the functionality of

another area of the application. Having tests in place

will prevent these scenarios from happening.

Lastly, the development of these applications is no

longer done by computer scientists or those with

technology related backgrounds. The technologies have

so vastly grown that just about anyone can pick up a

book and start developing. But not having that

background and software development mindset can

easily flaw their applications. What RoR has done to

web development will ultimately revolutionize it, if it

hasn’t already done so. With only three years of

Figure 1 - Sample rcov output [10]

exposure and a growing user base who are becoming

more educated, we can only hope to see an increase in

bug free web applications in the future, all thanks to the

testing capabilities and elegance of RoR.

References

[1] Curt Hibbs. What Is Ruby on Rails. 2005 ONLamp.com http://www.onlamp.com/pub/a/onlamp/2005/10/13/what_is_rails.html

[2] http://wiki.rubyonrails.org/rails/pages/HowToDoTestDrivenDevelopmentInRails

[3] 37Signals. A Guide to Testing Rails. http://manuals.rubyonrails.com/read/chapter/22

[4] 37Signals. Securing your Rails applications http://manuals.rubyonrails.com/read/chapter/43

[5] Dan Manges. Rails: UnitRecord – Test Without the Database http://www.dcmanges.com/blog/rails-unit-record-test-without-the-database

[6] Cody Fauser - Rails RJS Templates http://www.codyfauser.com/2005/11/20/rails-rjs-templates

[7] Guide: Test Driven Development with ARTS http://glu.ttono.us/articles/2006/05/29/guide-test-driven-rjs-with-arts

[8] About Ruby http://www.ruby-lang.org/en/about/

[9] Gregory Brown. Rails Testing: Not just for the paranoid http://www.oreilly.de/artikel/2007/06/rails.html

[10] rcov: Code Coverage for Ruby http://eigenclass.org/hiki.rb?rcov

[11] RSpec Home http://rspec.rubyforge.org/