Blog Archives

We have a test suite here that now is rapidly approaching 2 hours using a single core. Let me just repeat that. A developer realistically would have to leave their machine testing overnight to see if the suite is working. That’s really not good enough.

Specjour has been a bit of a turn-key miracle worker with our RSpec suite, however lately we’ve started to require some custom database setup that we do in a seeds.rb file as well as some custom bundler install parameters as most of our devs don’t have MySQL installed. Both of our needs were being nicely stomped on by Specjour so I thought it was time to look elsewhere.

I took a trip down Hydra lane and while getting to the point of having a working, local, dual runner system was a piece of cake, getting something working remotely via SSH took me hours of pain. Debugging the remote SSH workers was a nightmare and I spent a couple of hours running through code before deciding it was probably better to update our existing solution rather than tooling up a brand new one.

Back to the Specjour code. Specjour includes a rails directory inside which is an init.rb which Rails will run at initialisation (it’s part of what Rails does) but the Specjour initialiser will always just run the default database setup task no matter what initialiser you’ve got setup. We had a specjour initialiser that runs if ENV['PREPARE_DB'] was populated, which it is by Specjour, the problem was that the Specjour initialiser ran in the Rails after_initialization hook and therefore stomped all over our database setup.

The first step was just to have our initialiser write to another ENV element and then to have the Specjour after_initialize handler respect this. This isn’t too hard to implement as the after_initialize handler is just a block that is attached and so inside of this block you just need to check that ENV element. In my case I created a new ENV['DB_PREPPED'] element when my database setup had completed and then when the after_initialize block runs it checks for ENV['DB_PREPPED'] and will do nothing if that’s been set to true.

Easy. I now had Specjour respecting our database setup task.

The next step was to try and test this outside of a Rails application, not only that but to test the operation of a block (anonymous function?). To do this I setup a stub on a mock Rails class and let it capture the after_initialize block and then I ran a number of specs against this block.

module Specjour
  module DbScrub
  end
end

DO_NOT_REQUIRE = true

describe "Rails Initialiser" do
  before :all do
    ENV['PREPARE_DB'] = "true"

    stub(Specjour::DbScrub).scrub

    class Rails
      class << self; attr_accessor :configuration; end
      class << self; attr_accessor :test_block; end
    end

    config = Object.new
    stub(config).after_initialize { |args|
      object = Object.new
      Rails.test_block = args
      object
    }
    Rails.configuration = config

    require 'rails/init'
  end

... tests ...

This code essentially mocks up Rails.configuration and then stubs the after_initialize method. This stub then places the block that after_initialize yields to into Rails.test_block. When I require 'rails/init' it sequentially processes the file (as with all Ruby) and the stub will capture the block. After this is a bunch of tests I run an whether the Specjour::DbScrub.scrub method is called or not, so it’s nothing special.

I felt like at this stage I had fairly well tested the main aspects of the database setup.

The next issue was with how bundler was being handled. We have a situation where we would like to install sometimes without some gems. Some of the gems we use and have written use applications we’d rather not maintain in development and get tested in our staging and production environments. We generally will run a bundle install in development without the production or metrics groups so I wanted to have the ability to pass through a custom bundler command. That’s pretty easy now with my gem. Inside .specjour/bundler.yml there is a command property. I think this is more complex than what’s required, but I can foresee us needing a number of custom rake tasks and shell scripts so this bundler.yml should have probably started life as a settings/commands/something_generic.yml

To test this part of my changes was pretty simple. I basically just stubbed the system calls to bundler to give certain return values and checked to make sure the correct program flow happened.

describe ".bundle_install" do
    let :manager do
      stub.instance_of(Specjour::Manager).project_path { "/tmp" }

      stub(Dir).chdir(anything) { |args|
        args.last.call # This yields to the block for Dir.chdir()
      }

      manager = Specjour::Manager.new
      stub(manager).project_path { "blah" }
      mock(manager).system('bundle lock')

      manager
    end

    it "should perform a bundle lock" do
      stub(manager).system('bundle check > /dev/null') { true }

      manager.bundle_install
    end

    it "should check if there are gems required" do
      mock(manager).system('bundle check > /dev/null') { true }

      manager.bundle_install
    end

    context "when gems are required" do
      before :each do
        # Not a before :all as it needs to hook into the let hook above

        stub(manager).system('bundle check > /dev/null') { false }
      end

      context "and there is a bundler YAML file" do
        before :each do
          config_file = ".specjour/bundler.yml"

          mock(File).exists?(config_file) { true }
          mock(File).read(config_file) { "" }
          mock(YAML).load(anything) {
            { 'command' => "do it" }
          }
        end

        it "should get the bundle command from the YAML file" do
          mock(manager).system('do it > /dev/null')
          manager.bundle_install
        end
      end

      context "and there is no bundler YAML file" do
        before :each do
          mock(File).exists?(".specjour/bundler.yml") { false }
        end

        it "should perform a bundle install" do
          mock(manager).system('bundle install > /dev/null')
          manager.bundle_install
        end
      end
    end
  end

You can see that I stubbed our the Dir.chdir block to just yield directly to the call, otherwise it’ll throw an exception. Then I stubbed and mocked out the Kernel.system calls as necessary. Kernel methods are generally included into Ruby objects so you don’t stub Kernel, you stub the object that has the Kernel methods. Most of the testing is pretty basic, but I’d be keen to hear if I’m doing anything incorrectly!

This was my first major venture into adding functionality to a public project and it was good fun. I think it made me do a little better work than I might normally, it’s a great motivation to potentially have peers look at how you do things.

After bundling it all up and testing it here with over a dozen developers and even more machines I’m pretty happy with how it functions. I’ve made a pull request back to the original gem creator and hopefully he’ll like what I’ve done. In the meantime if you want to check it out then my Specjour is available on Git Hub.