In a company I’m currently working we have been using resque to do some heavy asynchronous work. And well, that’s like obvious choice in Rails world - just go with resque, fork some processes and relax ;). But when you get to a point, when you need few hundred workers and few servers, you start to ask yourself - can’t you do it better?

You probably can. There are some pretty smart folks out there, who were so kindly to share their knowledge. And that’s how you can find sidekiq build on top of celluloid.

Sidekiq uses multi-threading, so you can leverage that even if you use MRI (that have un-famous GIL) - if you have a lot of I/O bound work you can still benefit from this great piece of software (even without need to migrate to JRuby or Rubinius).

I ran some simple benchmarks and said to myself - hell yeah, let’s just do it! About 70 commits later I completely integrated sidekiq, updated test suite and created some custom sidekiq middleware that I needed (man it’s great, just take a look at the docs). Within 24 hours it processed over 1000000 jobs using 20x less amount of workers. How cool is that? I would say - pretty cool.

Few general tips and notes to all of you and to future-myself ;-)