{"id":628,"date":"2014-05-17T22:14:41","date_gmt":"2014-05-18T05:14:41","guid":{"rendered":"http:\/\/pchristensen.com\/blog\/?p=628"},"modified":"2014-05-17T22:14:41","modified_gmt":"2014-05-18T05:14:41","slug":"fast-rails-tests","status":"publish","type":"post","link":"http:\/\/pchristensen.com\/blog\/articles\/fast-rails-tests\/","title":{"rendered":"Fast Rails Tests Through Behavior Verification"},"content":{"rendered":"<p style=\"color: #7a7a7a;\">[This was originally posted on Manilla&#8217;s tech blog, which is no longer up. RIP Manilla]<\/p>\n<p style=\"color: #7a7a7a;\">Most Rails developers have a love\/hate relationship with their tests. \u00a0We love testing, we love writing tests, we love the confidence that a robust test suite provides when refactoring or adding new features. \u00a0But sometimes, especially with a large test suite, running the tests takes a disruptive and discouraging amount of time.<\/p>\n<p style=\"color: #7a7a7a;\">At Manilla, it takes about 20 minutes to run the complete test suite. \u00a0This is not terrible, but it\u2019s far too much to run during development. \u00a0So we usually only run the spec file for the code file we\u2019re working on, then we\u2019re done developing, we run the whole suite before committing to our master branch.<\/p>\n<p style=\"color: #7a7a7a;\">My previous employer was a much larger company with many times more tests. \u00a0It was basically impossible to run the test suite on a developer machine so we had a massive parallelized CI-farm that\u00a0<i>everyone<\/i>\u00a0used\u00a0<i>every time<\/i>\u00a0they were committing code. \u00a0So while the situation at Manilla is still tolerable, I know what the future holds.<\/p>\n<p style=\"color: #7a7a7a;\">(most of what I\u2019m about to say is paraphrased from\u00a0<a style=\"color: #f78e1e;\" href=\"http:\/\/www.youtube.com\/watch?v=bNn6M2vqxHE\">Corey Haines\u2019 excellent talk on Fast Rails Tests<\/a>)<br \/>\n<span class=\"embed-youtube\"><iframe loading=\"lazy\" class=\"youtube-player\" src=\"http:\/\/www.youtube.com\/embed\/bNn6M2vqxHE?version=3&amp;rel=1&amp;fs=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" width=\"490\" height=\"306\" frameborder=\"0\"><\/iframe><\/span><\/p>\n<p style=\"color: #7a7a7a;\">There are two approaches to making your test suite run faster.<\/p>\n<p style=\"color: #7a7a7a;\">The first is to improve your test infrastructure. \u00a0There are lots of strategies for this, like using fixtures instead of factories, using fixture_builder to ease the pain of maintaining fixtures, running spork to speed up running individual tests, etc. \u00a0These are attractive because let you continue writing tests in the way most people already do, but they require occasional bursts of non-development work to keep running.<\/p>\n<p style=\"color: #7a7a7a;\">The second approach is what Corey simply calls \u201cchanging the design of your code\u201d. \u00a0He shows a lot of examples and benefits, but he glossed over the philosophical difference that underlies his approach. \u00a0I didn\u2019t really get his talk until I grasped this distinction.<\/p>\n<h4 style=\"color: #7a7a7a;\">State Verification vs Behavior Verification<\/h4>\n<p style=\"color: #7a7a7a;\">The best description of this is in Martin Fowler\u2019s essay\u00a0<a style=\"color: #f78e1e;\" href=\"http:\/\/martinfowler.com\/articles\/mocksArentStubs.html\">Mocks Aren\u2019t Stubs<\/a>. \u00a0Here\u2019s a quick example for those of you who didn\u2019t read that essay like I just told you to. \u00a0Imagine you\u2019re writing a class for an ATM, and you want to test the deposit function.<\/p>\n<p style=\"color: #7a7a7a;\">Here\u2019s your function:<\/p>\n<pre style=\"color: #7a7a7a;\">## atm.rb\r\ndef deposit(account, amount)\r\n\u00a0 account.create_transaction(amount)\r\nend<\/pre>\n<p style=\"color: #7a7a7a;\">With state verification, your test code looks like this:<\/p>\n<pre style=\"color: #7a7a7a;\">## atm_spec.rb\r\nit \"#deposit - should increase the Account balance\" do\r\n\u00a0 account = Account.new(:balance =&gt; 100)\r\n\u00a0 Atm.new.deposit(account, 100)\r\n\u00a0 account.balance.should == 200\r\nend<\/pre>\n<p style=\"color: #7a7a7a;\">The test ends up being a mini-integration test that not only does your Atm.deposit work, but the Account.deposit works too.<\/p>\n<p style=\"color: #7a7a7a;\">With behavior verification testing, the test code looks like this:<\/p>\n<pre style=\"color: #7a7a7a;\">## atm_spec.rb\r\nclass FakeAccount\r\nend\r\nit \"#deposit - should tell Account to create a transaction\" do\r\n\u00a0 account = FakeAccount.new\r\n\u00a0 account.should_receive(:create_transaction).with(100)\r\n\u00a0 Atm.new.deposit(account, 100)\r\nend<\/pre>\n<p style=\"color: #7a7a7a;\">In this case, you\u2019re not testing that the balance of the account actually increased, you\u2019re just verifying that you called the account object the way you expect to be calling it. \u00a0With behavior verification testing, you treat\u00a0<i>any class outside the class you\u2019re currently testing as an API<\/i>. \u00a0Most developers already do this with things like email, geocoding, batch processing, etc.<\/p>\n<p style=\"color: #7a7a7a;\">There are two huge implications of behavior verification testing.<\/p>\n<p style=\"color: #7a7a7a;\">Your unit tests for one class or module can\u2019t be broken by code changes in other model. \u00a0This is a good thing because a change in one class won\u2019t break hundreds of dependent tests across the entire suite. \u00a0But it also makes your integration tests that much more important. \u00a0If the interface to a class changes, your behavior verification tests won\u2019t break even though the code no longer works. \u00a0State verification tests do test integration as deep as the coupling between the classes, but still need to be backed by real end-to-end integration tests.<\/p>\n<p style=\"color: #7a7a7a;\">The other point, relevant to test speed, is that you don\u2019t need to load any resources that aren\u2019t used by the class or module you\u2019re testing. \u00a0So just like you don\u2019t need to actually send emails when you run your email tests, you don\u2019t need to load external classes to make sure you\u2019re calling them correctly.<\/p>\n<p style=\"color: #7a7a7a;\">In the context of Rails, this means that through some slight restructuring of your code, you can avoid calling one big huge heavy API: Rails itself. \u00a0Loading any ActiveRecord class requires loading Rails, which takes several to many seconds, and including spec_helper takes many more seconds.<\/p>\n<p style=\"color: #7a7a7a;\">But how do avoid using Rails and ActiveRecord?<\/p>\n<h4 style=\"color: #7a7a7a;\">Strategies for Writing Fast Tests in Rails<\/h4>\n<p style=\"color: #7a7a7a;\">Don\u2019t get me wrong \u2013 ActiveRecord is a wonderful thing. \u00a0Rails is great. \u00a0If lines of code were pollutants, Rails would be the love-child of Al Gore and Captain Planet. \u00a0But it has a tendency to take over and get into all of your code. \u00a0Rails calls your code, rather than you calling Rails code. \u00a0Here\u2019s how to slightly change your code to isolate your business logic from Rails and greatly speed up your test execution:<\/p>\n<p style=\"color: #7a7a7a;\">First, continue using ActiveRecord for persistence, validation, relationships, etc. \u00a0It\u2019s wonderful for that.<\/p>\n<p style=\"color: #7a7a7a;\">For class Foo, create a module called FooBehavior (or some more targeted subset, like FooDepositBehavior) and include that in your class.<\/p>\n<p style=\"color: #7a7a7a;\">Move your business logic methods to that module and change them so they do not assume internal knowledge of the class. \u00a0For instance, change this:<\/p>\n<pre style=\"color: #7a7a7a;\">## account.rb\r\ndef approve_loan_request?(withdrawal_amount)\r\n\u00a0 self.balance &gt; withdrawal_amount\r\nend<\/pre>\n<p style=\"color: #7a7a7a;\">to this:<\/p>\n<pre style=\"color: #7a7a7a;\">## account_bahavior.rb\r\ndef approve_loan_request?(withdrawal_amount, current_balance)\r\n\u00a0 current_balance &gt; withdrawal_amount\r\nend<\/pre>\n<p style=\"color: #7a7a7a;\">The code looks almost identical, but in the first version, the innocent looking \u2018self.balance\u2019 requires the whole infrastructure of Rails to be in memory, it might make a database call unless it has already loaded the object, etc. \u00a0The second version is just about the easiest line of Ruby code ever written.<\/p>\n<p style=\"color: #7a7a7a;\">Here are some tips to write fast behavoir verification testable code:<\/p>\n<ul style=\"color: #7a7a7a;\">\n<li>Extract business logic code into its own module where appropriate<\/li>\n<li>In business logic methods, pass all arguments and compute based on those (this also makes your tests more meaningful because they\u2019re not dependent on outside state \u2013 see\u00a0<a style=\"color: #f78e1e;\" href=\"http:\/\/www.defmacro.org\/ramblings\/fp.html\">Functional Programming For the Rest of Us<\/a>\u00a0for more details)<\/li>\n<li>When you need to lookup data using ActiveRecord\u2019s API, use scopes or create your own methods that only contain AR queries and only return results. \u00a0This makes stubbing much easier.<\/li>\n<\/ul>\n<p style=\"color: #7a7a7a;\">And in your tests:<\/p>\n<ul style=\"color: #7a7a7a;\">\n<li>Require only the specific file(s) you\u2019re testing<\/li>\n<li>Put a dummy class at the top of your file for any external resources you\u2019re calling<\/li>\n<li>Use doubles or mocks, not factories or fixtures. \u00a0If you\u2019ve separated your business logic from your AR classes, then your code and tests don\u2019t need to know the difference between a full ActiveRecord object and a mock.<\/li>\n<li>Put your \u201cfast tests\u201d in a separate directory (e.g. spec\/fast\/). \u00a0This makes them easier to run all at once.<\/li>\n<li>Write a script (not a rake task, rake loads Rails) to run all your fast tests. \u00a0If you write tests like this, it\u2019s faster to run the whole fast test suite than it is to autocomplete the filename that you\u2019re working on. Ours looks like this:<\/li>\n<\/ul>\n<pre style=\"color: #7a7a7a;\">#!\/usr\/bin\/ruby\r\ndef test_files(dir)\r\n Dir[\"#{dir}**\/*_spec.rb\"].join \" \"\r\nend\r\nexec \"rspec #{test_files('spec\/fast\/')}\"<\/pre>\n<h4 style=\"color: #7a7a7a;\">Results<\/h4>\n<p style=\"color: #7a7a7a;\">We just started down this road so right now there\u2019s only one file of fast tests, but there\u2019s a look at the difference it makes:<\/p>\n<p style=\"color: #7a7a7a;\">Without Spork running:<\/p>\n<pre style=\"color: #7a7a7a;\">$ time rspec spec\/models\/interstitial_announcement_spec.rb\u00a0\r\n\u2026\r\n\u00a0 11\/11: \u00a0 \u00a0 \u00a0 100% |==========================================| Time: 00:00:03\r\n\r\nFinished in 3.47 seconds\r\n11 examples, 0 failures\r\nreal\u00a0 \u00a0 1m40.845s\r\nuser\u00a0 \u00a0 1m29.785s\r\nsys \u00a0 \u00a0 0m4.165s<\/pre>\n<p style=\"color: #7a7a7a;\">With Spork running:<\/p>\n<pre style=\"color: #7a7a7a;\">$ time rspec spec\/models\/interstitial_announcement_spec.rb\u00a0\r\n...\r\n\u00a0 11\/11: \u00a0 \u00a0 \u00a0 100% |==========================================| Time: 00:00:02\r\n\r\nFinished in 2.57 seconds\r\n11 examples, 0 failures\u00a0\r\nreal\u00a0 \u00a0 0m10.528s\r\nuser\u00a0 \u00a0 0m2.269s\r\nsys \u00a0 \u00a0 0m0.196s<\/pre>\n<p style=\"color: #7a7a7a;\">After converted to fast tests:<\/p>\n<pre style=\"color: #7a7a7a;\">$ time fast_tests\r\n\u2026\r\n\u00a0 12\/12: \u00a0 \u00a0 \u00a0 100% |==========================================| Time: 00:00:00\r\n\r\nFinished in 0.07578 seconds\r\n12 examples, 0 failures\u00a0\r\nreal\u00a0 \u00a0 0m2.834s\r\nuser\u00a0 \u00a0 0m2.519s\r\nsys \u00a0 \u00a0 0m0.246s<\/pre>\n<p style=\"color: #7a7a7a;\">Summary:<\/p>\n<p style=\"color: #7a7a7a;\">Without Spork 1:40.8<\/p>\n<p style=\"color: #7a7a7a;\">With Spork 0:10.5<\/p>\n<p style=\"color: #7a7a7a;\">Fast Style 0:02.8<\/p>\n<h4 style=\"color: #7a7a7a;\">Conclusion<\/h4>\n<p style=\"color: #7a7a7a;\">If you\u2019re still reading, then you have an amazing attention span. \u00a0This journey felt a lot like tracking down an odd bug, where one symptom leads to a whole different root cause and solution. \u00a0I did not expect to end up studying testing philosophies when I was grumbling bad thoughts while waiting for my tests to run. \u00a0But even without the side benefit of faster tests, I\u2019m grateful I learned a new way to look at testing.<\/p>\n<p style=\"color: #7a7a7a;\">Like so many things in programming, state verification vs behavior verification is \u201cit depends\u201d, not \u201cright or wrong\u201d. \u00a0There is some code that just works better with state verification. \u00a0Also, I bet most of your tests are already state verification, and there\u2019s no reason to do a big-bang rewrite of everything at once. \u00a0Just convert the appropriate parts of a class to use behavior verification testing as you\u2019re working on that class, over time, 5%, 20%, 50% of your test suite will be fast.<\/p>\n<p style=\"color: #7a7a7a;\">The real benefit is only partly to your full suite execution time. \u00a0A fast test suite can run thousands of tests per second, so instead of running only the test file you\u2019re working on while you develop, you can run a big fraction of all your test suite. Every. Time. You. Write. Code. \u00a0Then, testing can be something you do while you develop instead of after it.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>[This was originally posted on Manilla&#8217;s tech blog, which is no longer up. RIP Manilla] Most Rails developers have a love\/hate relationship with their tests. \u00a0We love testing, we love writing tests, we love the confidence that a robust test suite provides when refactoring or adding new features. \u00a0But sometimes, especially with a large test [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[10],"tags":[],"class_list":{"0":"post-628","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-programming","7":"entry"},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pazgP-a8","_links":{"self":[{"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/posts\/628","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/comments?post=628"}],"version-history":[{"count":0,"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/posts\/628\/revisions"}],"wp:attachment":[{"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/media?parent=628"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/categories?post=628"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/pchristensen.com\/blog\/wp-json\/wp\/v2\/tags?post=628"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}