Blog Archives

Simple Database Export and Import With Character Encoding Conversion

Posted in Code, Ruby on Rails

There is a gem called ydd that offers really simple import and export of smallish databases. It exports to YAML and then imports to whatever database Rails can connect to. After using YDD a few times I’ve found it easier to pinpoint the cause of problems that occur using taps.

It doesn’t handle character encodings though so I went about adding that. With the handy rchardet gem and IConv, detecting the character encoding of the incoming string and converting it to UTF-8 was pretty simple. I’ve created a pull request for the gem that will hopefully be accepted.

The essential code is below, and revolves mainly around the detection and conversion. Using //TRANSLIT causes IConv to try and convert the incoming character code to something that exists in the UTF8 character set, and then //IGNORE will ignore any characters that don’t exist in the UTF8 character set. Chaining //TRANSLIT and then //IGNORE will make IConv try a conversion first and then ignore anything it cannot convert.

I used this gem after the above changes to convert about 400,000 records of text data with ASCII, windows-1252, IBM866 and other character encodings from an old SQLite installation to a new postgres database without any issues.

Allowing Multiple Users to Use The Pivotal Tracker Gem

Posted in Code, Tips and Tricks

I’ve been using the Pivotal Tracker gem by Justin Smestad on a project to do various things but I ran into a problem using it with multiple API keys. The gem would create a single connection object and continue to mimic the user the first connection was made with, even when you’d updated the token to different user’s token. The Tracker API relies on you passing your API key for access to the projects and stories your account has access to, so I had the problem that the first user to use my app would determin access for everyone using the app. Not good!

After checking out the project the change was pretty simple but it highlighted an issue that I’ve seen in plenty of code where caching is done too broadly. In this case the cacheing (or memoization) was performed without taking into account the variable that was sent to establish the connection in the first place. The offending method looked like this :

Can’t see this Gist? View it on Github!

This was set on the class and as such @connection would never be created after the first call, whether the token was reset or not. I just changed it so that the connection is cached for each @token the system uses. For us it’s a small number or users, but in a larger context of course you’d need to implement some invalidation protocols to stop the following code from (slowly) swallowing the world :

Can’t see this Gist? View it on Github!

NoToken is just a class inheriting from StandardError that I chucked in there, the only thing that makes the gem now work properly come afterwards. I setup a hash and then store a new connection for each new incoming token, the connection is keyed by the token in the @connections hash. All pretty simple.

I’ve created a pull request that will hopefully be accepted soon, I think it fixes an unexpected problem with the gem.

Accessing ActiveRecord and ActiveModel Callback Chains in Rails 3

Posted in Code, Ruby on Rails, Tips and Tricks

It seems that access to the ActiveRecord callback chain has changed recently, however the documentation still contains the following :

To list the methods and procs registered with a particular callback, append _callback_chain to the callback name that you wish to list and send that to your class from the Rails console.

This used to work in Rails 2.3.x and I’d used it a few times before, however trying to use it in Rails 3 gave me problems :


HourEntry.before_save_callback_chain
NoMethodError: undefined method `before_save_callback_chain' for #

After some digging around ActiveRecord then ActiveModel and ActiveSupport I figured I could just access the _#{type}_callbacks method and then reject and select these arrays to get what was required. This is what I came up with to get the before_save callbacks that I was interested in.


HourEntry._save_callbacks.select { |callback| callback.kind.eql?(:before) }

This returns me the array of callback objects that are chained before I call save on the HourEntry model. If I want to check for a particular proc being in that chain then I can do :


HourEntry._save_callbacks.select { |callback| callback.kind.eql?(:after) }.collect(&:filter).include?(:archive_if_no_users)

It’s not optimal chaining a number of enumerable methods but I think it seems pretty readable.

I am hoping that the callback_chain methods return at some point (though I only used them sparingly) but more than that I’m hoping someone might be able to set me straight on how I’m going about getting around the callback chains on ActiveRecord and ActiveModel.

Composing Scopes On Multiple Models with Rails 3

Posted in Code, Ruby on Rails

Today I had some knowledge delivered straight to the top of my dome, by Darcy. The issue was I had two models that I wanted to join and then apply scopes to. I wanted to retrieve Story objects that were joined to HourEntry objects, that were from this week. However when I tried to use the Time scope on the query it was attempting to call those methods on Story.

ree-1.8.7-2011.02 :132 > Story.joins(:hour_entries).this_week
NoMethodError: undefined method `this_week' for #

The problem was that the .this_week call was being sent to Story rather than to HourEntry. Here’s where Darcy chimed in with the scoped.merge call. It went a little something like this :

Story.scoped.merge HourEntry.this_week

This successfully merged the scope defined by Story with the scope defined by the HourEntry.this_week call and I get all the stories that have HourEntry objects that are returned by the .this_week scope.

The final stage for this call was :

Story.scoped.merge HourEntry.worked_on_by(user).this_week

This then returns all the stories you’ve worked on this week. Add a bit of grouping/distinct and it’s done and done. The additional nice part about this is composing these scopes will allow me to do a lot more in the future, very quickly.

Reloading Factory Girl Factories in the Rails 3 Console

Posted in Inside TFG, Ruby on Rails

Updated : August 2011 to work with Factory Girl 2.0.5

It’s been annoying me for some time that when I hit reload! in the Rails console my factories stop working, or point to the wrong class. This wasn’t a real issue until I started using Devise.

Devise uses a mapping between classes and routes, so when a factory built object comes through to Devise after a console reload, or a class redefinition then it will fail. This is commonly the case in development and test environments.

I sat down with a colleague today (thanks Darcy) and we found the appropriate place to reload Factories to have it all work.

This is the code we put into my application.rb :

    ActionDispatch::Callbacks.after do      
      # Reload the factories
      return unless (Rails.env.development? || Rails.env.test?)
      
      unless FactoryGirl.factories.blank? # first init will load factories, this should only run on subsequent reloads
        FactoryGirl.factories.clear
        FactoryGirl.find_definitions
      end

    end    

After that everything just worked and I can hit reload! in the console without any issue.

Search Posts

Featured Posts

Categories

Archives

View more archives