Long-Running Tasks In Rails
Without Much Effort
Andy Stewart
April 2008
Not me
Still not me
Why?
User starts
something
lengthy
Delayed
task
Regular
task
Example
Synchronous link to Campaign Monitor
10
BackgroundJob (BJ) Spawn
Sparrow
Delayed Job (DJ)
AM4R
Conveyor
BackgrounDRb
Beanstalkd &
async-observer
Workling
BackgroundFu
Factors
Durability
Progress reporting
Scheduling
Serial vs. Parallel
Process management
Execution environment
Error handling
Learning curve
Installation burden
Constraints on task code
Durability
Durable Hopeful
BackgrounDRb
Background Job
Beanstalk
BackgroundFu
Spawn
Delayed Job
Sparrow
Workling
Workling
0% 50% 100%
Tool Method
Background Job Process’s stats
BackgroundFu Incremental
BackgrounDRb Ask workers
Workling DIY “return store”
Scheduling
BackgrounDRb
cron + {rake, script/runner,
whatever}
Delaying
(on purpose)
Delayed Job
Beanstalk
Process
Management
No Problem Hassle
Background Job
Everything else
Spawn
Examples
BackgrounDRb
# Lots of configuration
...
# Your worker code
class BillingWorker < BackgrounDRb::MetaWorker
set_worker_name :billing_worker
# Called when worker is loaded for the first time
def create(args = nil)
end
# The lengthy task.
def charge_customer(customer_id = nil)
# ... do stuff ...
end
end
# Your invocation code
class CustomersController < ApplicationController
def upgrade_account
# ...
[Link](:billing_worker).charge_customer(@[Link])
end
end
Beanstalk
# Some configuration
...
# Comments controller
def create
@comment = [Link](params[:comment])
if @[Link]
[Link]({:type => "comment", :id => @[Link]}) rescue nil
# Then redirect and return
end
end
# Worker - Rake task
loop do
job = [Link]
job_hash = [Link] # ybody deserializes the job
case job_hash[:type]
when "comment"
if Comment.check_for_spam(job_hash[:id])
[Link]
else
[Link]
end
else
puts "Don't know what type of job this is: #{job_hash.inspect}"
end
end
Source: [Link]
Async Observer
# Some configuration
...
# Start some workers
$ ./vendor/plugins/async_observer/bin/worker
# Your code
class Person < ActiveRecord::Base
async_after_create do |person|
SiteStats.increment_members()
end
def befriend(other_person)
Friendship.async_send(:create, self, other_person)
end
end
Delayed Job
# Small config - db migration
# A job is a Ruby object with #perform
class NewsletterJob < [Link](:text, :emails)
def perform
[Link] { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end
# Stick in queue
Delayed::[Link] [Link]('lorem ipsum...',
[Link](:all).collect(&:email))
# Delayed job
[Link]([Link](1)).send_later(:import_massive_csv, massive_csv)
# Running tasks
$ rake jobs:work <CTRL-C to cancel>
Background Job
# Command line
$ [Link] "./jobs/background_job_to_run"
$ [Link] "./script/runner ./jobs/background_job_to_run"
# Within Rails
def upload_to_s3
[Link] "./script/runner ./jobs/s3_uploader.rb #{[Link]}"
end
Source: [Link]
1
Background Job
Two-line installation
Zero configuration
No job-code constraints
Automatic* process management
Durable, prioritised, taggable jobs
Comprehensive job results
Clustering
* Manual if you want
Feed my spaniel
Buy my PeepCode PDF
Photo Credits:
1. BBC
2. Forever in Song (CD cover)
3. Me
4. Me
My Photos’ Licence:
Creative Commons Attribution-Noncommercial-No Derivative Works 2.0
UK: England & Wales Licence
Text’s Licence:
Creative Commons Attribution-Share Alike 2.0 UK: England & Wales Licence