Thoughts and hacks about dev

A few days ago, I found quite a disturbing bug in Mongoid:

user.groups.public_send(:find) # => <Enumerator#...> user.groups.send(:find) # => nil

So what is that? send and public_send does not have the same behavior for a public method!

I found this only after some debugging and lost even more time to find the solution...

Actually user.groups is an embedded collection, which in Mongoid is a Mongoid::Relations::Proxy. This proxy delegates almost everything to its target, in this case an array.

To create the proxy, Mongoid undefine almost every method inside the object and then only add some specialized methods.

Great, but if you ever have to do this, either undef both send and public_send or don't undefine them.

In the case of Mongoid, my problem can easily be fixed with the following patch:

Today I was reading a random post on Stackoverflow about recurring jobs using the new ActiveJob framework.

Antarr Byrd was trying to reschedule a job using the after_perform callback

I read an answer from bcd who said:

But that's not a reliable way of doing it as if an exception occurs the chain breaks. Also you must have the initial job scheduled. If you use resque you can use activejob-scheduler

So I learned about ActiveJob Scheduler which claims to be "A background job scheduler for any queue backend".

ActiveJob::Scheduler

Unlike bcd said, it is actually possible to enqueue jobs with any queuing adapter, not only resque.

The project is really great, and it actually lets you schedule jobs even with ActiveJob's default inline adapter!

It uses rufus-scheduler under the hood, so you can even directly use rufus to delay a job with the inline adapter.

As bcd said, one great advantage of this solution is that it will reschedule the job even if the last run failed, which is not the case of the after_perform callback.

As I'm not writing an apology of ActiveJob-Scheduler, here come the downsides.

This is a lie.

Like its predecessors, ActiveJob::Scheduler is built with Rufus::Scheduler, an immensely powerful task scheduling library to make sure your jobs get kicked off at exactly the right time.

I've worked with a lot of different programming language including ruby but also some low level stuff like C and assembly. I don't know any solution to actually run something at exactly the specified time, do you?

On a single process operating system, you can maybe do it, but not on any common multi-processing OS we use everyday. That OS will let your process run once in a while, not when you request it.

This gem is great, but not magic, so please don't lie.

Scheduling in-memory? No, thanks

So ActiveJob-Scheduler uses rufus-scheduler, "an in-process, in-memory scheduler".

Great project too, but for scheduling jobs? Really? How about I want to schedule an email in 1 month but I update my app every week?

At every update, the application will be restarted, so a new process will be spawn with his own memory. As an in-memory scheduler, rufus will lose every information.

That's not a problem for a lot of cases, but when you want to schedule a job you may not want it to disappear at every update, and even less on unexpected restart!

What about my queue adapter?

When I started using ActiveJob, I took a look at the different adapters available and choose one according to its features.

If you use ActiveJob::Scheduler, you can pick any adapter is really does not matter is the end, because all the scheduling in done by rufus, not by your adapter!

So you may as well use the inline adapter, it will run your job directly at the desired time, instead of doing unnecessary work first (sending to the backend which will in most case run it directly).

Your adapter persists delayed jobs across restarts? No problem, ActiveJob-Scheduler will not use this feature and won't tell you unless you try to look how it works!

Initial scheduling

ActiveJob-Scheduler initial scheduling is great, you can simply define your jobs in a YAML file.

But then, you have to launch a rake task or an a binary to start it!

The schedule must run as a separate process, but it's very light...it delegates all the real processing to your queue workers, and simply enqueues jobs as the specified time rolls around.

This means you have one more worker just for the scheduling, this is insane! Okay, it won't take much ressources, but it'll still inscrease the worker number, and consequently the bill...

Useless in most cases

If you use a backend which supports delay (and you should), then ActiveJob-Scheduler is useless.

You can schedule initial jobs on worker start (check if it does not exists first maybe if you can).

after_perform is great, and great to reschedule job even if it does not get called on error. Why would you reschedule a job which just failed?

If you still want to do so, then rescue_from error to also reschedule or simply ensure that the job is rescheduled.

Whatever way does your backend works, it won't be restarted if you update your app, so even in-memory persistence is okay. Some backends also support file persistence.

It does not need an additional worker to run.