Rails 4.1: ActiveRecord::Base.find_each Enumerator
Making use of a new feature in Edge Rails to build a simple rate limiter
I was working with several maintenance tasks that query external webservices for a collection of ActiveRecord objects. In order to avoid hitting the webservices' rate limit, we pause every other iteration for a fraction of a second before we continue.
The code looks something like this:
I didn't like that the information of our rate limit guard clauses was so scattered: there were bits before the block and others in the block. I had the urge to refactor it into a more concise form using a Ruby block itself.
My idea was to create a method that takes the rate limit, the sleep time and an enumeration and then yield the elements of the enum to a block.
I wrapped that method into a module and out came this:
This is how the specs look like:
Unfortunately and unlike other enumerable methods, the batch finder
ActiveRecord::Base.find_each does not return an Enumerator when
called without a block. Not in Rails 3.2 and not in Rails 4.0.
Luckily, it has been solved in Edge Rails (see 840c552) and I quickly created a Rails 4.1.0.beta app to go for the following code:
ActiveRecord::Base.find_each now returns a proper
Enumerator that can be passed into the rate limiter. If you want
to learn more about Enumerators in Ruby (or if you'd like to refresh your
memory), I can recommend this excellent Ruby Tapas
episode by Avdi Grimm.
You can also find all the code examples on Github.comments powered by Disqus