daeltar-clockwork 0.2.4
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +142 -0
- data/Rakefile +23 -0
- data/VERSION +1 -0
- data/bin/clockwork +20 -0
- data/lib/clockwork.rb +123 -0
- metadata +69 -0
data/README.md
ADDED
@@ -0,0 +1,142 @@
|
|
1
|
+
Clockwork - a clock process to replace cron
|
2
|
+
===========================================
|
3
|
+
|
4
|
+
Cron is non-ideal for running scheduled application tasks, especially in an app
|
5
|
+
deployed to multiple machines. [More details.](http://adam.heroku.com/past/2010/4/13/rethinking_cron/)
|
6
|
+
|
7
|
+
Clockwork is a cron replacement. It runs as a lightweight, long-running Ruby
|
8
|
+
process which sits alongside your web processes (Mongrel/Thin) and your worker
|
9
|
+
processes (DJ/Resque/Minion/Stalker) to schedule recurring work at particular
|
10
|
+
times or dates. For example, refreshing feeds on an hourly basis, or send
|
11
|
+
reminder emails on a nightly basis, or generating invoices once a month on the
|
12
|
+
1st.
|
13
|
+
|
14
|
+
Quickstart
|
15
|
+
----------
|
16
|
+
|
17
|
+
Create clock.rb:
|
18
|
+
|
19
|
+
require 'clockwork'
|
20
|
+
include Clockwork
|
21
|
+
|
22
|
+
handler do |job|
|
23
|
+
puts "Running #{job}"
|
24
|
+
end
|
25
|
+
|
26
|
+
every(10.seconds, 'frequent.job')
|
27
|
+
every(3.minutes, 'less.frequent.job')
|
28
|
+
every(1.hour, 'hourly.job')
|
29
|
+
|
30
|
+
every(1.day, 'midnight.job', :at => '00:00')
|
31
|
+
|
32
|
+
Run it with the clockwork binary:
|
33
|
+
|
34
|
+
$ clockwork clock.rb
|
35
|
+
Starting clock for 4 events: [ frequent.job less.frequent.job hourly.job midnight.job ]
|
36
|
+
Triggering frequent.job
|
37
|
+
|
38
|
+
Use with queueing
|
39
|
+
-----------------
|
40
|
+
|
41
|
+
The clock process only makes sense as a place to schedule work to be done, not
|
42
|
+
to do the work. It avoids locking by running as a single process, but this
|
43
|
+
makes it impossible to parallelize. For doing the work, you should be using a
|
44
|
+
job queueing system, such as
|
45
|
+
[Delayed Job](http://www.therailsway.com/2009/7/22/do-it-later-with-delayed-job),
|
46
|
+
[Beanstalk/Stalker](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend/),
|
47
|
+
[RabbitMQ/Minion](http://adamblog.heroku.com/past/2009/9/28/background_jobs_with_rabbitmq_and_minion/), or
|
48
|
+
[Resque](http://github.com/blog/542-introducing-resque). This design allows a
|
49
|
+
simple clock process with no locks, but also offers near infinite horizontal
|
50
|
+
scalability.
|
51
|
+
|
52
|
+
For example, if you're using Beanstalk/Staker:
|
53
|
+
|
54
|
+
require 'stalker'
|
55
|
+
|
56
|
+
handler { |job| Stalker.enqueue(job) }
|
57
|
+
|
58
|
+
every(1.hour, 'feeds.refresh')
|
59
|
+
every(1.day, 'reminders.send', :at => '01:30')
|
60
|
+
|
61
|
+
Using a queueing system which doesn't require that your full application be
|
62
|
+
loaded is preferable, because the clock process can keep a tiny memory
|
63
|
+
footprint. If you're using DJ or Resque, however, you can go ahead and load
|
64
|
+
your full application enviroment, and use per-event blocks to call DJ or Resque
|
65
|
+
enqueue methods. For example, with DJ/Rails:
|
66
|
+
|
67
|
+
require 'config/boot'
|
68
|
+
require 'config/environment'
|
69
|
+
|
70
|
+
every(1.hour, 'feeds.refresh') { Feed.send_later(:refresh) }
|
71
|
+
every(1.day, 'reminders.send', :at => '01:30') { Reminder.send_later(:send_reminders) }
|
72
|
+
|
73
|
+
Anatomy of a clock file
|
74
|
+
-----------------------
|
75
|
+
|
76
|
+
clock.rb is standard Ruby. Since we include the Clockwork module (the
|
77
|
+
clockwork binary does this automatically, or you can do it explicitly), this
|
78
|
+
exposes a small DSL ("handler" and "every") to define the handler for events,
|
79
|
+
and then the events themselves.
|
80
|
+
|
81
|
+
The handler typically looks like this:
|
82
|
+
|
83
|
+
handler { |job| enqueue_your_job(job) }
|
84
|
+
|
85
|
+
This block will be invoked every time an event is triggered, with the job name
|
86
|
+
passed in. In most cases, you should be able to pass the job name directly
|
87
|
+
through to your queueing system.
|
88
|
+
|
89
|
+
The second part of the file are the events, which roughly resembles a crontab:
|
90
|
+
|
91
|
+
every(5.minutes, 'thing.do')
|
92
|
+
every(1.hour, 'otherthing.do')
|
93
|
+
|
94
|
+
In the first line of this example, an event will be triggered once every five
|
95
|
+
minutes, passing the job name 'thing.do' into the handler. The handler shown
|
96
|
+
above would thus call enqueue_your_job('thing.do').
|
97
|
+
|
98
|
+
You can also pass a custom block to the handler, for job queueing systems that
|
99
|
+
rely on classes rather than job names (i.e. DJ and Resque). In this case, you
|
100
|
+
need not define a general event handler, and instead provide one with each
|
101
|
+
event:
|
102
|
+
|
103
|
+
every(5.minutes, 'thing.do') { Thing.send_later(:do) }
|
104
|
+
|
105
|
+
If you provide a custom handler for the block, the job name is used only for
|
106
|
+
logging.
|
107
|
+
|
108
|
+
You can also use blocks to do more complex checks:
|
109
|
+
|
110
|
+
every(1.day, 'check.leap.year') do
|
111
|
+
Stalker.enqueue('leap.year.party') if Time.now.year % 4 == 0
|
112
|
+
end
|
113
|
+
|
114
|
+
In production
|
115
|
+
-------------
|
116
|
+
|
117
|
+
Only one clock process should ever be running across your whole application
|
118
|
+
deployment. For example, if your app is running on three VPS machines (two app
|
119
|
+
servers and one database), your app machines might have the following process
|
120
|
+
topography:
|
121
|
+
|
122
|
+
* App server 1: 3 web (thin start), 3 workers (rake jobs:work), 1 clock (clockwork clock.rb)
|
123
|
+
* App server 2: 3 web (thin start), 3 workers (rake jobs:work)
|
124
|
+
|
125
|
+
You should use Monit, God, Upstart, or Inittab to keep your clock process
|
126
|
+
running the same way you keep your web and workers running.
|
127
|
+
|
128
|
+
Meta
|
129
|
+
----
|
130
|
+
|
131
|
+
Created by Adam Wiggins
|
132
|
+
|
133
|
+
Inspired by [rufus-scheduler](http://rufus.rubyforge.org/rufus-scheduler/) and [http://github.com/bvandenbos/resque-scheduler](resque-scehduler)
|
134
|
+
|
135
|
+
Design assistance from Peter van Hardenberg and Matthew Soldo
|
136
|
+
|
137
|
+
Patches contributed by Mark McGranaghan
|
138
|
+
|
139
|
+
Released under the MIT License: http://www.opensource.org/licenses/mit-license.php
|
140
|
+
|
141
|
+
http://github.com/adamwiggins/clockwork
|
142
|
+
|
data/Rakefile
ADDED
@@ -0,0 +1,23 @@
|
|
1
|
+
require 'bundler/setup'
|
2
|
+
require 'jeweler'
|
3
|
+
|
4
|
+
Jeweler::Tasks.new do |s|
|
5
|
+
s.name = "daeltar-clockwork"
|
6
|
+
s.summary = "A scheduler process to replace cron."
|
7
|
+
s.description = "A scheduler process to replace cron, using a more flexible Ruby syntax running as a single long-running process. Inspired by rufus-scheduler and resque-scheduler."
|
8
|
+
s.author = "Adam Wiggins"
|
9
|
+
s.email = "adam@heroku.com"
|
10
|
+
s.homepage = "http://github.com/adamwiggins/clockwork"
|
11
|
+
s.executables = [ "clockwork" ]
|
12
|
+
s.rubyforge_project = "daeltar-clockwork"
|
13
|
+
|
14
|
+
s.files = FileList["[A-Z]*", "{bin,lib}/**/*"]
|
15
|
+
end
|
16
|
+
|
17
|
+
Jeweler::GemcutterTasks.new
|
18
|
+
|
19
|
+
task 'test' do
|
20
|
+
sh "ruby test/clockwork_test.rb"
|
21
|
+
end
|
22
|
+
|
23
|
+
task :build => :test
|
data/VERSION
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
0.2.4
|
data/bin/clockwork
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
STDERR.sync = STDOUT.sync = true
|
4
|
+
|
5
|
+
require File.expand_path('../../lib/clockwork', __FILE__)
|
6
|
+
include Clockwork
|
7
|
+
|
8
|
+
usage = "clockwork <clock.rb>"
|
9
|
+
file = ARGV.shift or abort usage
|
10
|
+
|
11
|
+
file = "./#{file}" unless file.match(/^[\/.]/)
|
12
|
+
|
13
|
+
require file
|
14
|
+
|
15
|
+
trap('INT') do
|
16
|
+
puts "\rExiting"
|
17
|
+
exit
|
18
|
+
end
|
19
|
+
|
20
|
+
run
|
data/lib/clockwork.rb
ADDED
@@ -0,0 +1,123 @@
|
|
1
|
+
module Clockwork
|
2
|
+
class FailedToParse < StandardError; end;
|
3
|
+
|
4
|
+
class ClockworkEvent
|
5
|
+
attr_accessor :job, :last
|
6
|
+
|
7
|
+
def initialize(period, job, block, options={})
|
8
|
+
@period = period
|
9
|
+
@job = job
|
10
|
+
@at = parse_at(options[:at])
|
11
|
+
@last = nil
|
12
|
+
@block = block
|
13
|
+
end
|
14
|
+
|
15
|
+
def to_s
|
16
|
+
@job
|
17
|
+
end
|
18
|
+
|
19
|
+
def time?(t)
|
20
|
+
ellapsed_ready = (@last.nil? or (t - @last).to_i >= @period)
|
21
|
+
time_ready = (@at.nil? or (t.hour == @at[0] and t.min == @at[1]))
|
22
|
+
ellapsed_ready and time_ready
|
23
|
+
end
|
24
|
+
|
25
|
+
def run(t)
|
26
|
+
@last = t
|
27
|
+
@block.call(@job)
|
28
|
+
rescue => e
|
29
|
+
log_error(e)
|
30
|
+
end
|
31
|
+
|
32
|
+
def log_error(e)
|
33
|
+
STDERR.puts exception_message(e)
|
34
|
+
end
|
35
|
+
|
36
|
+
def exception_message(e)
|
37
|
+
msg = [ "Exception #{e.class} -> #{e.message}" ]
|
38
|
+
|
39
|
+
base = File.expand_path(Dir.pwd) + '/'
|
40
|
+
e.backtrace.each do |t|
|
41
|
+
msg << " #{File.expand_path(t).gsub(/#{base}/, '')}"
|
42
|
+
end
|
43
|
+
|
44
|
+
msg.join("\n")
|
45
|
+
end
|
46
|
+
|
47
|
+
def parse_at(at)
|
48
|
+
return unless at
|
49
|
+
m = at.match(/^(\d\d):(\d\d)$/)
|
50
|
+
raise FailedToParse, at unless m
|
51
|
+
hour, min = m[1].to_i, m[2].to_i
|
52
|
+
raise FailedToParse, at if hour >= 24 or min >= 60
|
53
|
+
[ hour, min ]
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
extend self
|
58
|
+
|
59
|
+
def handler(&block)
|
60
|
+
@@handler = block
|
61
|
+
end
|
62
|
+
|
63
|
+
class NoHandlerDefined < RuntimeError; end
|
64
|
+
|
65
|
+
def get_handler
|
66
|
+
raise NoHandlerDefined unless (defined?(@@handler) and @@handler)
|
67
|
+
@@handler
|
68
|
+
end
|
69
|
+
|
70
|
+
def every(period, job, options={}, &block)
|
71
|
+
event = ClockworkEvent.new(period, job, block || get_handler, options)
|
72
|
+
@@events ||= []
|
73
|
+
@@events << event
|
74
|
+
event
|
75
|
+
end
|
76
|
+
|
77
|
+
def run
|
78
|
+
log "Starting clock for #{@@events.size} events: [ " + @@events.map { |e| e.to_s }.join(' ') + " ]"
|
79
|
+
loop do
|
80
|
+
tick
|
81
|
+
sleep 1
|
82
|
+
end
|
83
|
+
end
|
84
|
+
|
85
|
+
def log(msg)
|
86
|
+
puts msg
|
87
|
+
end
|
88
|
+
|
89
|
+
def tick(t=Time.now)
|
90
|
+
to_run = @@events.select do |event|
|
91
|
+
event.time?(t)
|
92
|
+
end
|
93
|
+
|
94
|
+
to_run.each do |event|
|
95
|
+
log "Triggering #{event}"
|
96
|
+
event.run(t)
|
97
|
+
end
|
98
|
+
|
99
|
+
to_run
|
100
|
+
end
|
101
|
+
|
102
|
+
def clear!
|
103
|
+
@@events = []
|
104
|
+
@@handler = nil
|
105
|
+
end
|
106
|
+
|
107
|
+
end
|
108
|
+
|
109
|
+
unless 1.respond_to?(:seconds)
|
110
|
+
class Numeric
|
111
|
+
def seconds; self; end
|
112
|
+
alias :second :seconds
|
113
|
+
|
114
|
+
def minutes; self * 60; end
|
115
|
+
alias :minute :minutes
|
116
|
+
|
117
|
+
def hours; self * 3600; end
|
118
|
+
alias :hour :hours
|
119
|
+
|
120
|
+
def days; self * 86400; end
|
121
|
+
alias :day :days
|
122
|
+
end
|
123
|
+
end
|
metadata
ADDED
@@ -0,0 +1,69 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: daeltar-clockwork
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
hash: 31
|
5
|
+
prerelease:
|
6
|
+
segments:
|
7
|
+
- 0
|
8
|
+
- 2
|
9
|
+
- 4
|
10
|
+
version: 0.2.4
|
11
|
+
platform: ruby
|
12
|
+
authors:
|
13
|
+
- Adam Wiggins
|
14
|
+
autorequire:
|
15
|
+
bindir: bin
|
16
|
+
cert_chain: []
|
17
|
+
|
18
|
+
date: 2011-07-30 00:00:00 Z
|
19
|
+
dependencies: []
|
20
|
+
|
21
|
+
description: A scheduler process to replace cron, using a more flexible Ruby syntax running as a single long-running process. Inspired by rufus-scheduler and resque-scheduler.
|
22
|
+
email: adam@heroku.com
|
23
|
+
executables:
|
24
|
+
- clockwork
|
25
|
+
extensions: []
|
26
|
+
|
27
|
+
extra_rdoc_files:
|
28
|
+
- README.md
|
29
|
+
files:
|
30
|
+
- README.md
|
31
|
+
- Rakefile
|
32
|
+
- VERSION
|
33
|
+
- bin/clockwork
|
34
|
+
- lib/clockwork.rb
|
35
|
+
homepage: http://github.com/adamwiggins/clockwork
|
36
|
+
licenses: []
|
37
|
+
|
38
|
+
post_install_message:
|
39
|
+
rdoc_options: []
|
40
|
+
|
41
|
+
require_paths:
|
42
|
+
- lib
|
43
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
44
|
+
none: false
|
45
|
+
requirements:
|
46
|
+
- - ">="
|
47
|
+
- !ruby/object:Gem::Version
|
48
|
+
hash: 3
|
49
|
+
segments:
|
50
|
+
- 0
|
51
|
+
version: "0"
|
52
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
53
|
+
none: false
|
54
|
+
requirements:
|
55
|
+
- - ">="
|
56
|
+
- !ruby/object:Gem::Version
|
57
|
+
hash: 3
|
58
|
+
segments:
|
59
|
+
- 0
|
60
|
+
version: "0"
|
61
|
+
requirements: []
|
62
|
+
|
63
|
+
rubyforge_project: daeltar-clockwork
|
64
|
+
rubygems_version: 1.8.6
|
65
|
+
signing_key:
|
66
|
+
specification_version: 3
|
67
|
+
summary: A scheduler process to replace cron.
|
68
|
+
test_files: []
|
69
|
+
|