I often need to execute really large computations on really large data sets. I usually end up writing a rake task to do it, which calls methods in my models.

But something about the process bugged me.

Each time I had to re-implement my 'batching code' that allowed me to not chew up GB after GB of memory due to klass.find(:all, :include => [:everythingunderthe_sun]).

Re-implementation of the same logic over and over across many projects is not very DRY, so I got out my blow torch and lit it up. The difficulty was that the part that was different each time I batched was at the center of the code, right in the middle of the batch loop. But I didn't let that stop me!

Please refer to the homepage for current documentation! (Trying to keep it DRY :)