[This was originally posted on Manilla’s tech blog, which is no longer up. RIP Manilla]

Most Rails developers have a love/hate relationship with their tests.  We love testing, we love writing tests, we love the confidence that a robust test suite provides when refactoring or adding new features.  But sometimes, especially with a large test suite, running the tests takes a disruptive and discouraging amount of time.

At Manilla, it takes about 20 minutes to run the complete test suite.  This is not terrible, but it’s far too much to run during development.  So we usually only run the spec file for the code file we’re working on, then we’re done developing, we run the whole suite before committing to our master branch.

My previous employer was a much larger company with many times more tests.  It was basically impossible to run the test suite on a developer machine so we had a massive parallelized CI-farm that everyone used every time they were committing code.  So while the situation at Manilla is still tolerable, I know what the future holds.

(most of what I’m about to say is paraphrased from Corey Haines’ excellent talk on Fast Rails Tests)

There are two approaches to making your test suite run faster.

The first is to improve your test infrastructure.  There are lots of strategies for this, like using fixtures instead of factories, using fixture_builder to ease the pain of maintaining fixtures, running spork to speed up running individual tests, etc.  These are attractive because let you continue writing tests in the way most people already do, but they require occasional bursts of non-development work to keep running.

The second approach is what Corey simply calls “changing the design of your code”.  He shows a lot of examples and benefits, but he glossed over the philosophical difference that underlies his approach.  I didn’t really get his talk until I grasped this distinction.

State Verification vs Behavior Verification

The best description of this is in Martin Fowler’s essay Mocks Aren’t Stubs.  Here’s a quick example for those of you who didn’t read that essay like I just told you to.  Imagine you’re writing a class for an ATM, and you want to test the deposit function.

Here’s your function:

## atm.rb
def deposit(account, amount)
  account.create_transaction(amount)
end

With state verification, your test code looks like this:

## atm_spec.rb
it "#deposit - should increase the Account balance" do
  account = Account.new(:balance => 100)
  Atm.new.deposit(account, 100)
  account.balance.should == 200
end

The test ends up being a mini-integration test that not only does your Atm.deposit work, but the Account.deposit works too.

With behavior verification testing, the test code looks like this:

## atm_spec.rb
class FakeAccount
end
it "#deposit - should tell Account to create a transaction" do
  account = FakeAccount.new
  account.should_receive(:create_transaction).with(100)
  Atm.new.deposit(account, 100)
end

In this case, you’re not testing that the balance of the account actually increased, you’re just verifying that you called the account object the way you expect to be calling it.  With behavior verification testing, you treat any class outside the class you’re currently testing as an API.  Most developers already do this with things like email, geocoding, batch processing, etc.

There are two huge implications of behavior verification testing.

Your unit tests for one class or module can’t be broken by code changes in other model.  This is a good thing because a change in one class won’t break hundreds of dependent tests across the entire suite.  But it also makes your integration tests that much more important.  If the interface to a class changes, your behavior verification tests won’t break even though the code no longer works.  State verification tests do test integration as deep as the coupling between the classes, but still need to be backed by real end-to-end integration tests.

The other point, relevant to test speed, is that you don’t need to load any resources that aren’t used by the class or module you’re testing.  So just like you don’t need to actually send emails when you run your email tests, you don’t need to load external classes to make sure you’re calling them correctly.

In the context of Rails, this means that through some slight restructuring of your code, you can avoid calling one big huge heavy API: Rails itself.  Loading any ActiveRecord class requires loading Rails, which takes several to many seconds, and including spec_helper takes many more seconds.

But how do avoid using Rails and ActiveRecord?

Strategies for Writing Fast Tests in Rails

Don’t get me wrong – ActiveRecord is a wonderful thing.  Rails is great.  If lines of code were pollutants, Rails would be the love-child of Al Gore and Captain Planet.  But it has a tendency to take over and get into all of your code.  Rails calls your code, rather than you calling Rails code.  Here’s how to slightly change your code to isolate your business logic from Rails and greatly speed up your test execution:

First, continue using ActiveRecord for persistence, validation, relationships, etc.  It’s wonderful for that.

For class Foo, create a module called FooBehavior (or some more targeted subset, like FooDepositBehavior) and include that in your class.

Move your business logic methods to that module and change them so they do not assume internal knowledge of the class.  For instance, change this:

## account.rb
def approve_loan_request?(withdrawal_amount)
  self.balance > withdrawal_amount
end

to this:

## account_bahavior.rb
def approve_loan_request?(withdrawal_amount, current_balance)
  current_balance > withdrawal_amount
end

The code looks almost identical, but in the first version, the innocent looking ‘self.balance’ requires the whole infrastructure of Rails to be in memory, it might make a database call unless it has already loaded the object, etc.  The second version is just about the easiest line of Ruby code ever written.

Here are some tips to write fast behavoir verification testable code:

• Extract business logic code into its own module where appropriate
• In business logic methods, pass all arguments and compute based on those (this also makes your tests more meaningful because they’re not dependent on outside state – see Functional Programming For the Rest of Us for more details)
• When you need to lookup data using ActiveRecord’s API, use scopes or create your own methods that only contain AR queries and only return results.  This makes stubbing much easier.

And in your tests:

• Require only the specific file(s) you’re testing
• Put a dummy class at the top of your file for any external resources you’re calling
• Use doubles or mocks, not factories or fixtures.  If you’ve separated your business logic from your AR classes, then your code and tests don’t need to know the difference between a full ActiveRecord object and a mock.
• Put your “fast tests” in a separate directory (e.g. spec/fast/).  This makes them easier to run all at once.
• Write a script (not a rake task, rake loads Rails) to run all your fast tests.  If you write tests like this, it’s faster to run the whole fast test suite than it is to autocomplete the filename that you’re working on. Ours looks like this:

#!/usr/bin/ruby
def test_files(dir)
 Dir["#{dir}**/*_spec.rb"].join " "
end
exec "rspec #{test_files('spec/fast/')}"

Results

We just started down this road so right now there’s only one file of fast tests, but there’s a look at the difference it makes:

Without Spork running:

$ time rspec spec/models/interstitial_announcement_spec.rb 
…
  11/11:       100% |==========================================| Time: 00:00:03

Finished in 3.47 seconds
11 examples, 0 failures
real    1m40.845s
user    1m29.785s
sys     0m4.165s

With Spork running:

$ time rspec spec/models/interstitial_announcement_spec.rb 
...
  11/11:       100% |==========================================| Time: 00:00:02

Finished in 2.57 seconds
11 examples, 0 failures 
real    0m10.528s
user    0m2.269s
sys     0m0.196s

After converted to fast tests:

$ time fast_tests
…
  12/12:       100% |==========================================| Time: 00:00:00

Finished in 0.07578 seconds
12 examples, 0 failures 
real    0m2.834s
user    0m2.519s
sys     0m0.246s

Summary:

Without Spork 1:40.8

With Spork 0:10.5

Fast Style 0:02.8

Conclusion

If you’re still reading, then you have an amazing attention span.  This journey felt a lot like tracking down an odd bug, where one symptom leads to a whole different root cause and solution.  I did not expect to end up studying testing philosophies when I was grumbling bad thoughts while waiting for my tests to run.  But even without the side benefit of faster tests, I’m grateful I learned a new way to look at testing.

Like so many things in programming, state verification vs behavior verification is “it depends”, not “right or wrong”.  There is some code that just works better with state verification.  Also, I bet most of your tests are already state verification, and there’s no reason to do a big-bang rewrite of everything at once.  Just convert the appropriate parts of a class to use behavior verification testing as you’re working on that class, over time, 5%, 20%, 50% of your test suite will be fast.

The real benefit is only partly to your full suite execution time.  A fast test suite can run thousands of tests per second, so instead of running only the test file you’re working on while you develop, you can run a big fraction of all your test suite. Every. Time. You. Write. Code.  Then, testing can be something you do while you develop instead of after it.