Recently we launched the iPhone app for ClimateWatch. The ClimateWatch project was developed by Earthwatch with the Bureau of Meteorology and The University of Melbourne to understand how changes in temperature and rainfall are affecting the behaviour of Australia’s plants and animals.
Components of the project built by The Frontier Group include a web application for viewing species and managing your observations, as well as an iPhone application for convenient observation recording.
The iPhone app launch has had some press which you can view with the links below:
Earlier this year we were introduced to PetRescue, who were looking for a development company to rebuild their web application that had initially been built and tweaked over the past 8 years by one of the founders.
PetRescue is a not-for-profit organisation that finds new permanent homes for rescue pets. They are Australia’s largest on-line directory of homeless animals, and to date we have helped find new, loving families for over 140,000 pets. PetRescue started in 2004 as a small not-for-profit team dedicated to finding new homes for lost and abandoned pets, with the belief they had something to contribute to improving outcomes for rescue animals across the country.
PetRescue is a web application for both the general public looking to adopt a pet, and rescue groups around the country to list their pets for adoption. Most of the functionality is in the admin interface of the app that makes the website self sufficient, enabling rescue groups complete control over their adoption listings and process.
The PetRescue website receives ~200,000 pageviews per day and had nearly a decade of legacy data, making it a huge task to design and rebuild from the ground up, while maintaining integrity of the existing data. The app was heavily tuned for performance in the launch week, and the site also consists of import/export scripts and an API for various third party websites.
Overall the launch was a success, and PetRescue now have a much improved platform to help rehome the thousands of pets listed each month.
Recently, I switched over from using Cucumber to Spinach in my Rails and mobile projects. I have been enjoying using Spinach so far and thought I’d share a pattern I’ve been using that seems to work for me.
Now I would like to preface this by saying that:
I’m not an expert on the topic
I’m not saying Spinach is better than Cucumber (or vice versa)
One of the things I found myself doing often when I was naming Cucumber steps was writing names that became very convoluted. I did this because Cucumber scopes all steps globally and naming two steps identically will cause the feature to fail with “Ambiguous match of “step name”". This meant I had to name almost identical steps slightly differently which would get compounded the more similar steps I had.
For example:
And I fill out the object create form with valid details
And I fill out the object create form with valid details using the google auto complete
And I fill out the object create form with valid details using the google auto complete and marking it as draft
When what I really wanted to say was:
And I fill out the form with valid details
All of that other information was covered in the feature name and/or scenario description.
When I moved to Spinach I found that I wasn’t having that problem because each feature’s steps were available only to that feature. I thought this was great… until I found myself defining identical steps in 20 different files.
My friend Tony Issakov then mentioned that you could define modules, include the Spinach DSL and share them between features. I was also warned, by my friend Jordan Maguire, that if you do this too much, you can end up with step definition files that are just includes and aren’t very readable.
For Example:
Feature: Updating an existing object
Background:
Given I am logged in as an admin
And there is an object in the system
And I am on the object index page
Scenario: Successfully updating object
When I click the object's name
And I fill out the form with valid details
Then I should be be on the object's show page
And I should see the object's details have been update
Scenario: Failed to update object
When I click the object's name
And I fill out the form with invalid details
Then I should see the form again
And I should see any form errors
class UpdatingAnExistingObject < Spinach::FeatureSteps
include SharedLinks
include SharedPath
include SharedForm
include SharedObject
include SharedViews
end
While this is an extreme example you can see that it quickly becomes difficult to know what step is defined where.
After a bit of playing around I found a happy balance between too much and too little step sharing. My general rule of thumb is to only share the following types of steps:
Authentication
module SharedAuthentication
include Spinach::DSL
Given 'I am logged in as an admin' do
visit new_user_session_path
user = FactoryGirl.create :admin
login_with user.email, 'secret'
end
def login_with email, password
fill_in 'user[email]', with: email
fill_in 'user[password]', with: password
click_button :submit
end
end
Paths
module SharedPath
include Spinach::DSL
And 'I am on the objects index page' do
visit objects_path
end
And "I am on the object's show page for my venue" do
current_path.should == object_path(@object)
end
Then 'I should be on the objects index page' do
current_path.should == objects_path
end
end
Forms
Note that I don’t have any of the “And I fill in the form” steps in here. This is because I want separate them out and just have ‘And I fill out the form with valid details’ in my step definition.
module SharedForm
include Spinach::DSL
And 'I should see any form errors' do
page.should have_css 'p.inline-errors'
end
def fill_in_object_edit_form attributes = nil
if attributes.present?
fill_in 'object[name]', with: attributes[:name]
# fill in other attributes
else
fill_in 'object[name]', with: ''
end
click_button :submit
end
end
Flash
module SharedFlash
include Spinach::DSL
And 'I should see a confirmation message' do
page.should have_css 'p.notice'
end
And 'I should see a flash alert' do
page.should have_css 'p.alert'
end
end
Object creation
I generally like to have one per model that handles basic object creation.
module SharedObject
include Spinach::DSL
And 'there is object in the system' do
@object = FactoryGirl.create :object
end
## Where mk_two_object is another factory for Object
And 'there is a mark two object in the system' do
@mk_two_object = FactoryGirl.create :mk_two_object
end
end
If we put it all together using the above example we get something that looks like this:
For Example:
Feature: Updating an existing object
Background:
Given I am logged in as an admin
And there is an object in the system
And I am on the object index page
Scenario: Successfully updating object
When I click the object's name
And I fill out the form with valid details
Then I should be be on the object's show page
And I should see the details have been update
Scenario: Failed to update object
When I click the object's name
And I fill out the form with invalid details
Then I should see the form again
And I should see any form errors
class UpdatingAnExistingObject < Spinach::FeatureSteps
include SharedPath
include SharedForm
include SharedObject
include SharedAuthentication
When "I click the object's name" do
click_link @object.name
end
And 'I fill out the form with valid details' do
@attributes = FactoryGirl.attributes_for :object
fill_in_object_edit_form @attributes
end
And 'I should see the details have been update' do
page.should have_content @attributes[:name]
## ...
end
And 'I fill out the form with invalid details' do
fill_in_object_edit_form
end
Then 'I should see the form again' do
page.should have_css "form#edit_object_#{@object.id}"
end
end
Using this pattern I’ve found that I get the step separation that I was looking for in Spinach while keeping the shared functionality that I enjoyed in Cucumber.
These days there’s a lot of money in mobile applications – and where there’s money and new technologies there’s web developers primed and ready to argue about how best to implement these new technologies.
As one of this dedicated crowd I’ve found myself recently working with a lot of Rails RESTful APIs that talk to my mobile applications. This has given me the opportunity to both create my own RESTful APIs and use RESTful APIs written by other developers.
As a result, I’ve gained a much better understanding of the HTTP Status Codes and how they can help you write both better APIs and client applications.
For clarity, I use the term ‘client’ to refer to the mobile application that is communicating with the Rails RESTful API backend.
When I say API I’m referring to a Rails RESTful API.
Before I begin, here are some good sites for reading up on the HTTP Status Codes:
Here’s a good article by Alex Rogriguez over at IBM from 2008 on REST APIs. If you aren’t clear on what a RESTful API actually is give Alex’s article a read.
The status code works hand-in-hand with the response body to help the client application process the request.
Status codes tell the client application what it needs to do with the response in general terms.
For example, if your API returns a 401 you know you need a logged in user. If the API returns a 403 you know the user you’re working with doesn’t have sufficient authorization. If the API returns a 418 you know that the API is telling you it’s a teapot and was probably written by knobs.
The response body will tell your client application in specifics what it needs to do with the response. If the client is trying to update a record and the API responds with a 400, the response body will inform the client why the request failed. On the other hand, if the API responds with a 200 the response body will provide the client with the updated entity.
That seems pretty straight forward, right? The difficulty lies in when the API returns a status code and a response body that don’t match up.
I worked with an API that returned a 200 status code but the response body had an errors array in it. That was an absolute trip – I had to parse the response body and look for the presence of content in an errors array first. Then I called an error handler from within the success callback of a jQuery ajax function. This is an example of a bad RESTful API. I hope by reading the remainder of this article you can avoid doing that.
Which status codes should I be using?
As long as you’re not too pedantic status codes are easy to work with when writing an API.
Here’s a rough guide to how I handle mine:
(For the purpose of all of the examples below, any response content I refer to will be as JSON.)
(2xx) Success Codes
200 OK
The 200 OK status code can be your go-to for any successful response. There are many success codes but for a very basic API I’ve found the 200 *can* be expressive enough. The distinction between 200 and the other success codes hasn’t proved valuable in my APIs so far.
However, some other Success Codes you might be interested in:
201 Created for when you’re creating a new resource.
202 Accepted for when you’ve successfully set the request to be performed in a background task. Useful if your client is requesting something on the API that is time-consuming and you don’t want the client to have to wait.
What should I return in the 200 response content?
The content of the response is dependent on the HTTP Method used in the request.
This JSON should allow the client to update all the records they have that match the records provided in the response. Possibly, it will also allow the client to determine which records have been deleted and can prune its local storage accordingly.
Now, consider an example where you are updating a resource with a request like:
PUT api.example.com/1/users/1
Your response JSON will look like:
{id: 1, name: 'Jordan'}
This allows the client to update its data for the record.
(4xx) Client Error Codes
Here I’ll cover the 3 client error codes I find myself using most in my APIs:
400 Bad Request
The 400 Bad Request status indicates that the request ‘could not be understood by the server due to malformed syntax’. See the IETF docs
There is also the 422 Unprocessable Entity status code that appears to be popular for roughly the same purpose. I’ve read from a couple of sources that since the 422 is part of the WebDAV extension that 400 is preferable over 422. I don’t have the knowledge to weigh in on this argument. Perhaps someone can enlighten us in the comments section.
Some examples scenarios I’ve used an API to return a 400 for:
The client is trying to create a resource with data that fails validation rules on the API
The client is requesting resources using invalid params (e.g.: the date is malformed)
The client request is missing fields the API requires as indicated in the thoroughly written docs! (e.g.: a search might require an end date is a start date is passed through)
What should I return in the 400 response content?
The response of a 400 will depend on why the request failed.
Likely, you’ll want to keep a consistent format across all API end-points. I nest my failure reasons in an errors attribute in the JSON.
As an example, let’s say the client request failed because the name provided was too long. The API will respond with:
{errors: {name: "must be fewer than 18 characters"}}
In fact, given this is being returned with a 400 status code, you can probably do away with the errors wrapper entirely and only return:
{name: "must be fewer than 18 characters"}
The only important thing is to be consistent across your API in how you represent these errors.
401 Unauthorized
The 401 Unauthorized status indicates that the request had missing/invalid authentication credentials. See the IETF docs.
To demonstrate the use of the 401 in an API, I’ll use one of my apps as an example.
In the client application we have a login screen for entering email/password to log in. On all other pages in the application we use token-based authentication (each request must have an token passed through that is checked by the API before performing any request). When the client successfully logs in the API passes a token back that can be persisted on the client device.
In this scenario, the API will return a 401 if:
The client app attempts to authenticate a user by passing invalid username or password
The client app makes a request to an action that requires authentication without providing the token (To be honest, I think this has only ever happened to me in development)
The client app makes a request to an action that requires authentication providing an invalid token (For example: the user tries to log in to the client app after their user has been disabled on the API)
What should I return in the 401 response content?
I’ve found that so far I’ll just return one of:
{errors: {invalid_credentials: 'Authentication credentials provided were invalid'}}
or
{errors: {no_token: 'No authentication token was provided in request'}}
or
{errors: {invalid_token: 'Token provided was invalid'}}
403 Forbidden
The 403 Forbidden status indicates that although the request was valid the action requested failed authorization constraints. See the IETF docs.
Pretty straight forward scenario: user x tries to update resource y. User x is not authorized to update resource y.
As a note, if you are making a client where you strictly control the interactions with the API I feel that your client ever getting an authorization error is at best a code smell but more likely it’s actually just a bug.
Consider an interface that has a button that the user can interact with which returns a message like ‘You are not authorized to perform this action’. Ideally, in a mobile interface the user should never see an authorization error message since it’s not something the user can do anything to fix.
What should I return in the 401 response content?
The API will return a string with a simple error message like:
{errors: {authorization: 'You are not authorized to perform this action'}}
What about the rest of the client error status codes?
So I’ve given you the top 3 error status codes I use in a bit more detail but that’s not an exhaustive guide to the error codes. Here’s a whirlwind summary of the remaining error codes that I’ve used in my APIs:
404 Not Found for when the URL is wrong or the requested resource does not exist.
410 Gone for when the resource has been permanently (and intentionally) deleted. This informs the client application that it should remove any references to the resource in question.
Summary
I hope I’ve provided some insight into the various status codes you can leverage to create an accurate, informative API. I also hope I’ve illustrated the strong relationship between the status code and the response body and the importance of providing a descriptive response body.
If you have any thoughts or disagree with me on this let me know in the comments and we can discuss!
For the last few months we have been working hard building the iPhone and web apps for the Big Help Mob in Perth. It has been a tremendous effort culminating in both a TV feature and launch event this week.
Well done on everyone who contributed to the project. A fitting end to the hard work of the last few months.
