Just to cut to the chase, Specjour will easily allow you to run your tests in parallel on your local machine, or in a distributed way on your network. It cut our test suite from over 11 minutes to under 2.5 minutes. We went from one Macbook Pro to a Macbook Pro plus an iMac. Specjour is gooood, but continuing …
We were running into issues with our testing suite running past 15 minutes to run. It caused a number of issues, firstly it disuaded people from running the suite for small changes they thought would be okay, secondly people tended to slow down or not work at all when the suite was running. Given we were building tests into an existing code base and coverage was running generally under 30% this was already really worrying.
We’ve been concentrating primarily on functional testing with RSpec so I went looking for solutions for trying to parallelise RSpec specifically. I tried parallel-tests but I couldn’t get it to work. Then I heard about Specjour on a Hashrocket video so I decided to give that a try. It worked pretty well out of the box, I just had to plug up a wayward plugin, update the rsync config file for the project and change one of the files in the gem to call a different rake task.
Specjour essentially uses Bonjour to find workers on the network and dispatch tests to them and it uses rsync to synchronise the project you’re testing to the remote server. Once the sync has happened then runs the rake task to setup the databases, loads the environment and starts accepting the tests that are dispatched to it.
The instructions will work for the majority of people, I only had some hiccups due to our project. Essentially I gem installed specjour which gave me a specjour executable which I ran to create a dRb server that communicates via Bonjour. Then in my project I added specjour to my Gemfile (all managed via Bundler!) and ran rake specjour.
For me, I ended up having a few hundred failing tests, all to do with the Ruby version of a stack overflow (stack level too deep). It turned out it was caused by a plugin calling alias method chain twice so that the original method ended up being aliased to the replacement method and so there was an infinite loop. Once I applied a fix then we were only down to a few failing tests to do with seed data.
I changed the db_scrub.rb file to perform a different rake task for us as we do require some seed data and then we were on our way.
Previously on my Macbook Pro the suite ran in just over 11 minutes, after turning on a four core iMac and my Macbook Pro I had six workers ready to go and it took about 2.5 minutes to run our tests. I’d really like to crank up the other few iMacs and the 10 or so Macbooks and see what happens then.
I think the only changes we’ll require is to be able to specify which task is run to create the databases, other than that it works perfectly.
In closing I’d say I tried a number of distributed and parallel methods for running our tests and I like Specjour the best. Apparently it will run Cucumber tests as well, I’m hoping to move on to that soon enough.
I used this post today as a guide on how to get another iMac up and running so I thought it was probably a good point to chuck this up here, if only for our own reference.
We are starting with a 27″ iMac and a base Snow Leopard install.
The first step was to install Xcode from the Snow Leopard disc, you’ll find it in the Optional Installs part of the disc.
Ruby and RubyGems comes with Snow Leopard by default so we’ll use them. You’ll likely need to update the RubyGems system :
sudo gem update --system
Now we can install Rails :
sudo gem install rails
This will install the latest stable version of Rails. We need a specific version for some of our applications (until we’ve tested it under the newer version) and you can do this by adding the -v switch :
sudo gem install rails -v=2.3.4
After this we can switch on Apache and install passenger, the module that runs Rails on Apache. So turn on Web Sharing in the Sharing panel of your System Preferences.
This will compile the Apache module and give you some text to paste into your apache conf file to load it. I keep this in a separate config file in /etc/apache2/other/passenger.conf. It should look something like :
Next download the Passenger Preference Pane so you can control your Rails sites from the OS X Preferences area. You can get it from http://www.fngtps.com/passenger-preference-pane. You should now be able to run Rails websites on your local machine very easily.
I like to use a Pomodoro timer when pairing and the one I like the best for OS X is just called Pomodoro Desktop. It will keep people focussed and also can serve to keep the control swapping in a fairly regular way. You can find out about how Pomodoro works here.
We use Github for our repository and exchange is done via SSH, so you’ll need to generate an SSH key and put that into your Github account.
ssh-keygen
cat ~/.ssh/id_rsa.pub
Copy the output and put that into your public SSH keys in Git Hub.
When pairing with Git the ability to have both parties responsible for the work they’re doing is appreciated sometimes. Especially twelve months from now when you can’t remember why you did something. Not that you don’t comment your code, or always write beautifully understandable code. There is a nice gem called hitch. It comes out of Hashrocket and handles the ‘hitching’ and ‘unhitching’ of partners on a machine. We have a pairing station so generally we just want to change the pair that’s in operation, so it works well. You’ll need to be inside a git repository to do the initial setup. I cloned one of our projects and then you issue :
hitch -m
Which will prompt you for your main email address, this is the email address that all devs in your team receive, or a group email of some sort.
Now to hitch two devs you’d do :
hitch dev1 dev2
This will prompt you for names of the dev1 and dev2 users so that they’ll be displayed in your commit. It’ll also associate your commits to the email address devs+dev1+dev2@company.com if you setup devs@company.com as the email address when you issued the hitch -m command. If you setup gravatars for all your devs+user1+user2@company.com email addresses then you’ll get a nice picture in Github too.
I think that’s about it really. We obviously just install gems as required and make customisations on a per developer need. We generally use Textmate for editing so this means installing a theme (we use the Railscast theme) and the relevant bundles for Rspec, Haml and Sass.
I’ve been converting a Rails 2.3.5 project to using Bundler and ran into a bit of a snag this week. I was getting a over 300 failing tests with errors such as :
I couldn’t find anything that I’d missed, I’d moved all my gem declarations from the various environment files into my Gemfile and hadn’t had many issues (many) but this was a really odd problem. It was as if part of Rails wasn’t being loaded correctly.
It turns out that you need to include a tzinfo gem for some reason. Don’t ask me how I ran across the solution for this issue, I really can’t remember. It works though.
So if you’re having issues with your app, or tests not passing due to time zone issues in Rails 2.3.x using Bundler then just include :
gem "tzinfo"
In your Gemfile, run another bundle install and away you go. It worked for me, I hope it works for you!
I’m currently getting into using RSpec for testing our controllers on what is turning into a large project. It’s been more than handy because we have a lot of complex scoping to take into account whenever retrieving data. People don’t like to see other peoples’ financial data, mostly because it implies that someone is probably looking at theirs. With this in mind it’s more than important that we know the right data is going to the right places and hence the need for controller testing.
Now most of our controllers require the user to be logged in so writing tests to check this for every controller is annoying and time consuming, more than that it feels dirty. I think this is what some people call a code smell though I’m not up to speed on buzz words. There are also other tasks that are done quite often such as setting up the various types of users we’d like to test as, it would be nice if this were easily put in one place and could be easily pulled in. I guess I was looking for a template of tests that I could share.
It seems that the solution to it is found in Shared Example Groups which I hadn’t heard very much discussion about and it kind of leaves working out how they work to you rather than documenting it too much.
So far I’ve used it simply to make sure that controllers that require are redirecting users appropriately and also for setting up a specific type of user for our system before testing.
I created a directory under /spec/support called example_groups and in there I have a file called login_groups.rb. In that file I have something like the following :
shared_examples_for "customer is logged in" do
before(:all) do
@user = Factory(:customer_user)
@user.customers.push Factory(:customer)
end
before(:each) do
activate_authlogic
OperatorSession.create(@operator)
end
end
Now in my spec files when I have a bunch of tests requiring a logged in customer I will include this little snippet :
it_should_behave_like "customer is logged in"
I get a logged in customer to start playing around with. I have the spec/support/example_groups directory in my include paths for Rspec and so it just all works.
My tests can then start to look like :
describe MerchantsController do
it_should_behave_like "areas requiring login"
context "customer logged in" do
it_should_behave_like "customer is logged in"
... insert other tests here ...
end
end
It means I can swap in another authentication gem/plugin pretty easily and also encapsulates the logic about creating customers, or whatever type of item you want to use, so that if that changes you can swap things in and out with a minimum of fuss.
Just to be clear, example groups aren’t limited to setup tasks or connecting to before/after hooks, you can also include a bunch of tests as well. This allows me to have a bunch of tests to run to make sure that a user does have to be logged in for various controllers and include these tests in on line.
I hope this helps someone, it took a bit of searching and trial and error myself this morning to get it working and find the uses for it that I’ve found. I’m definitely open to better solutions to this sort of issue though.
We’ve been using Selenium, Webrat, Cucumber and any number of other gems together for quite a while with very few problems. Just recently our Selenium tests just stopped working on the login step, and given our application requires users to login to do anything, it meant out test suite was basically broken from step one.
Given that we thought very little of any consequence had changed it was annoying trying to troubleshoot the problem. Typically you start off thinking it’s something you did, then maybe it’s something someone you know did, then you start to branch out and think that maybe, just maybe, it was a bug elsewhere!
I initially started to check the logs and they seemed perfect, but it wasn’t finding the user record in the database. So I loaded the database in sqlite and lo-and-behold there were no user records to be found. I started a debug session and it reported that the user had in fact been created. I thought it had to be something to do with transactions being used. Trying to insert a record into the database and being told the database was locked confirmed this.
I turned out that the Cucumber gem writer determined that there should be no more global setting for turning on or off transaction support when running tests. Previously transactions were turned off by default in Cucumber, and you’d generally turn them back on if you needed them. This change is annoying because a good chunk of our suite uses Selnium for testing and it requires transactions to be turned off. It’s also a bit odd to reverse a default option and have no way to enable it at all.
Apparently the work around is to tag all the features and scenarios that rely on non-transactional database operations with this new tag of @no-txn. After implementing this, we’ve found it does work and so our tests are back to usual condition.
So if you’re running into problems with your Selenium tests using Cucumber with Rails and your database isn’t updating within a test then check out the ‘@no-txn’ tag and see what happens after you use that.