simple_job 0.0.3 → 0.1.0
Sign up to get free protection for your applications and to get access to all the features.
- data/CHANGELOG.rdoc +6 -0
- data/README.rdoc +46 -5
- data/lib/simple_job/job_definition.rb +7 -2
- data/lib/simple_job/sqs_job_queue.rb +186 -7
- data/lib/simple_job/version.rb +1 -1
- metadata +18 -3
data/CHANGELOG.rdoc
CHANGED
@@ -1,5 +1,11 @@
|
|
1
1
|
= Simple Job
|
2
2
|
|
3
|
+
== Version 0.1.0
|
4
|
+
* Added AWS CloudWatch monitors for job execution
|
5
|
+
* Added :replace_existing option to register_simple_job
|
6
|
+
* Added signal handling to SQSJobQueue#poll to allow clean exit
|
7
|
+
* New SQSJobQueue#poll implementation to allow metric gathering
|
8
|
+
|
3
9
|
== Version 0.0.3
|
4
10
|
* Properly handling SystemExit and SignalException in SQSJobQueue#poll so that process can be daemonized
|
5
11
|
|
data/README.rdoc
CHANGED
@@ -6,6 +6,8 @@ A gem containing libraries that support running background jobs or tasks. It's d
|
|
6
6
|
* Enqueue a job
|
7
7
|
* Poll for and execute jobs
|
8
8
|
|
9
|
+
It fits seamlessly into a rails environment (utilizing its configuration), but does not require one.
|
10
|
+
|
9
11
|
It is architected to support multiple types of queue implementations, but currently, it only includes an implementation using AWS SQS (http://aws.amazon.com/sqs/). Alternate queue implementations could be plugged in by the client using lib/simple_job/sqs_job_queue.rb as an example.
|
10
12
|
|
11
13
|
The AWS SQS queue implementation requires the aws-sdk gem, which must be initialized (by calling AWS.config) for this API to be capable of enqueuing or polling for jobs.
|
@@ -24,17 +26,54 @@ SQSJobQueue is the queue implementation used by default. This may be overridden
|
|
24
26
|
SimpleJob::SQSJobQueue.config :queue_prefix => 'my-job'
|
25
27
|
SimpleJob::SQSJobQueue.define_queue 'normal', :default => true
|
26
28
|
|
27
|
-
=== Complex configuration with explicit queue implementation, non-rails-defined environment, and
|
29
|
+
=== Complex configuration with explicit queue implementation, non-rails-defined environment, multiple queues, and CloudWatch monitoring
|
28
30
|
|
29
31
|
SimpleJob::JobQueue.config :implementation => 'sqs'
|
30
|
-
SimpleJob::SQSJobQueue.config :queue_prefix => 'my-job', :environment => 'production'
|
32
|
+
SimpleJob::SQSJobQueue.config :queue_prefix => 'my-job', :environment => 'production', :cloud_watch_namespace => 'SimpleJob'
|
31
33
|
SimpleJob::SQSJobQueue.define_queue 'normal', :visibility_timeout => 60, :default => true
|
32
34
|
SimpleJob::SQSJobQueue.define_queue 'high-priority', :visibility_timeout => 10
|
33
35
|
SimpleJob::SQSJobQueue.define_queue 'long-running', :visibility_timeout => 3600
|
34
36
|
|
37
|
+
=== AWS CloudWatch monitoring
|
38
|
+
|
39
|
+
If you provide a CloudWatch namespace to SimpleJob::SQSJobQueue.config, then a series of metrics will be saved to AWS CloudWatch by the queue's poll method:
|
40
|
+
|
41
|
+
SimpleJob::SQSJobQueue.config :cloud_watch_namespace => 'MyNamespace'
|
42
|
+
|
43
|
+
The following metrics will be published:
|
44
|
+
|
45
|
+
MessageCheckCount:: A count of the number of poll attempts
|
46
|
+
MessageReceivedCount:: A count of the number of poll attempts that return a message
|
47
|
+
MessageMissCount:: A count of the number of poll attempts that do not return a message
|
48
|
+
ExecutionCount:: A count of the number of jobs that are executed
|
49
|
+
SuccessCount:: A count of the number of jobs that are executed without error
|
50
|
+
ErrorCount:: A count of the number of jobs that raise an exception during execution
|
51
|
+
ExecutionTime:: The number of milliseconds it takes the job to execute
|
52
|
+
TimeToCompletion:: The number milliseconds between when a job is requested and when it is successfully completed
|
53
|
+
ExecutionAttempts:: The number of executions that occur before a job is successfully completed
|
54
|
+
|
55
|
+
The metrics above are published using the following dimensions:
|
56
|
+
|
57
|
+
Environment:: The current environment (ie. development vs. production)
|
58
|
+
SQSQueueName:: The name of the SQS queue being polled
|
59
|
+
Host:: The hostname of the machine running the poll operation
|
60
|
+
JobType:: The string label for the type of job being executed
|
61
|
+
|
62
|
+
Note that the JobType dimension will only be present on the ExecutionCount, SuccessCount, ErrorCount, ExecutionTime, TimeToCompletion, and ExecutionAttempts metrics.
|
63
|
+
|
64
|
+
If no namespace is provided to SimpleJob::SQSJobQueue.config, then no cloudwatch metrics will be gathered or published.
|
65
|
+
|
35
66
|
|
36
67
|
== Job Definition
|
37
68
|
|
69
|
+
To define a job, simply include the SimpleJob::JobDefinition module, and implement its #execute method.
|
70
|
+
|
71
|
+
Attributes declared using simple_job_attribute will be automatically serialized into the JSON message that's enqueued, and re-populated into the object once it's dequeued. They each have standard getters/setters defined by attr_accessor, and may be set upon object initialization by passing them as hash keys/values to #new.
|
72
|
+
|
73
|
+
ActiveModel validation is supported and included automatically.
|
74
|
+
|
75
|
+
=== Synopsis
|
76
|
+
|
38
77
|
class FooSender
|
39
78
|
include SimpleJob::JobDefinition
|
40
79
|
|
@@ -53,7 +92,9 @@ SQSJobQueue is the queue implementation used by default. This may be overridden
|
|
53
92
|
|
54
93
|
=== Typical usage of default queue
|
55
94
|
|
56
|
-
You may call #enqueue with no arguments, in which case JobQueue.default will be used.
|
95
|
+
You may call #enqueue with no arguments, in which case JobQueue.default will be used (defined by passing the :default option when defining the queue as documented in the QueueConfiguration section).
|
96
|
+
|
97
|
+
Similar to ActiveRecord's save method, enqueue will return true or false depending on whether the object passes validation.
|
57
98
|
|
58
99
|
f = FooSender.new(:target => 'joe', :foo_content => 'foo!') # can also assign attributes with f.target=, f.foo_content=
|
59
100
|
if f.enqueue
|
@@ -87,11 +128,11 @@ Alternatively, the queue may be attached to the job upfront for multiple enqueue
|
|
87
128
|
|
88
129
|
== Job Server Usage
|
89
130
|
|
90
|
-
Calling #poll on a queue instance will, by default, dispatch each job to the proper registered class based on type and version. Its options are passed through to the underlying implementation.
|
131
|
+
Calling #poll on a queue instance will, by default, dispatch each job to the proper registered class based on type and version. Its options are passed through to the underlying implementation. See SQSJobQueue::poll for documentation of the options it accepts.
|
91
132
|
|
92
133
|
JobQueue.default.poll(:poll_interval => 1, :idle_timeout => 60)
|
93
134
|
|
94
|
-
You can override the default behavior by passing a block to #poll that accepts both a job definition instance and an SQS message.
|
135
|
+
You can override the default behavior by passing a block to #poll that accepts both a job definition instance and an SQS message. In this case, you must call "definition.execute" in the block. The default handler simply calls definition.execute.
|
95
136
|
|
96
137
|
JobQueue.default.poll(:poll_interval => 1, :idle_timeout => 60) do |definition, message|
|
97
138
|
logger.debug "received message: #{message.inspect}"
|
@@ -127,7 +127,9 @@ module JobDefinition
|
|
127
127
|
end
|
128
128
|
|
129
129
|
def register_simple_job(options = {})
|
130
|
-
default_type = self.name.underscore.to_sym
|
130
|
+
default_type = self.name.split('::').last.underscore.to_sym
|
131
|
+
|
132
|
+
replace_existing = options.delete(:replace_existing) || true
|
131
133
|
|
132
134
|
@definition = {
|
133
135
|
:class => self,
|
@@ -139,7 +141,10 @@ module JobDefinition
|
|
139
141
|
@definition[:versions] = Array(@definition[:versions])
|
140
142
|
@definition[:versions].collect! { |value| value.to_s }
|
141
143
|
|
142
|
-
|
144
|
+
if replace_existing
|
145
|
+
::SimpleJob::JobDefinition.job_definitions.delete_if { |item| item[:type] == default_type }
|
146
|
+
end
|
147
|
+
|
143
148
|
::SimpleJob::JobDefinition.job_definitions << @definition
|
144
149
|
end
|
145
150
|
|
@@ -1,6 +1,13 @@
|
|
1
|
+
require 'socket'
|
2
|
+
require 'fog'
|
3
|
+
|
1
4
|
module SimpleJob
|
5
|
+
|
6
|
+
# A SimpleJob::JobQueue implementation that uses AWS SQS
|
2
7
|
class SQSJobQueue < JobQueue
|
3
8
|
|
9
|
+
DEFAULT_POLL_INTERVAL = 1
|
10
|
+
|
4
11
|
# Registers this queue implementation with SimpleJob::JobQueue with identifier "sqs".
|
5
12
|
register_job_queue 'sqs', self
|
6
13
|
|
@@ -9,6 +16,7 @@ class SQSJobQueue < JobQueue
|
|
9
16
|
:queue_prefix => ENV['SIMPLE_JOB_SQS_JOB_QUEUE_PREFIX'],
|
10
17
|
:default_visibility_timeout => 60,
|
11
18
|
:environment => (defined?(Rails) && Rails.env) || 'development',
|
19
|
+
:cloud_watch_namespace => nil,
|
12
20
|
}
|
13
21
|
|
14
22
|
@config.merge!(options) if options
|
@@ -49,33 +57,88 @@ class SQSJobQueue < JobQueue
|
|
49
57
|
sqs_queue.send_message(message)
|
50
58
|
end
|
51
59
|
|
60
|
+
# Polls the queue, matching incoming messages with registered jobs, and
|
61
|
+
# executing the proper job/version.
|
62
|
+
#
|
63
|
+
# If called without a block, it will simply call the #execute method of the
|
64
|
+
# matched job. A block may be passed to add custom logic, but in this case
|
65
|
+
# the caller is responsible for calling #execute. The block will be passed
|
66
|
+
# two arguments, the matching job definition (already populated with the
|
67
|
+
# contents of the message) and the raw AWS message.
|
68
|
+
#
|
69
|
+
# The queue's configured visibility timeout will be used unless the
|
70
|
+
# :visibility_timeout option is passed (as a number of seconds).
|
71
|
+
#
|
72
|
+
# By default, the message's 'sent_at', 'receive_count', and
|
73
|
+
# 'first_received_at' attributes will be populated in the AWS message, but
|
74
|
+
# this may be overridden by passing an array of symbols to the :attributes
|
75
|
+
# option.
|
76
|
+
#
|
77
|
+
# By default, errors during job execution or message polling will be logged
|
78
|
+
# and the polling will continue, but that behavior may be changed by setting
|
79
|
+
# the :raise_exceptions option to true.
|
80
|
+
#
|
81
|
+
# By defult, this method will poll indefinitely. If you pass an :idle_timeout
|
82
|
+
# option, the polling will stop and this method will return if that number
|
83
|
+
# of seconds passes without receiving a message. In both cases, the method
|
84
|
+
# will safely complete processing the current message and return if a HUP,
|
85
|
+
# INT, or TERM signal is sent to the process.
|
86
|
+
#
|
87
|
+
# Note that this method will override any signal handlers for the HUP, INT,
|
88
|
+
# or TERM signals during its execution, but the previous handlers will be
|
89
|
+
# restored once the method returns.
|
52
90
|
def poll(options = {}, &block)
|
53
91
|
options = {
|
54
92
|
:visibility_timeout => visibility_timeout,
|
55
93
|
:attributes => [ :sent_at, :receive_count, :first_received_at ],
|
56
94
|
:raise_exceptions => false,
|
95
|
+
:idle_timeout => nil,
|
96
|
+
:poll_interval => DEFAULT_POLL_INTERVAL
|
57
97
|
}.merge(options)
|
58
98
|
|
59
99
|
message_handler = block || lambda do |definition, message|
|
60
100
|
definition.execute
|
61
101
|
end
|
62
102
|
|
103
|
+
exit_next = false
|
104
|
+
|
105
|
+
logger.debug 'trapping terminate signals with function to exit loop'
|
106
|
+
signal_exit = lambda do
|
107
|
+
logger.info "caught signal to shutdown; finishing current message and quitting..."
|
108
|
+
exit_next = true
|
109
|
+
end
|
110
|
+
previous_traps = {}
|
111
|
+
['HUP', 'INT', 'TERM'].each do |signal|
|
112
|
+
previous_traps[signal] = Signal.trap(signal, signal_exit)
|
113
|
+
end
|
114
|
+
|
115
|
+
last_message_at = Time.now
|
116
|
+
|
63
117
|
loop do
|
64
118
|
last_message = nil
|
119
|
+
current_start_milliseconds = get_milliseconds
|
120
|
+
current_job_type = 'unknown'
|
65
121
|
begin
|
66
|
-
sqs_queue.
|
122
|
+
sqs_queue.receive_messages(options) do |message|
|
67
123
|
last_message = message
|
68
124
|
raw_message = JSON.parse(message.body)
|
125
|
+
current_job_type = raw_message['type']
|
69
126
|
definition_class = JobDefinition.job_definition_class_for(raw_message['type'], raw_message['version'])
|
70
127
|
raise('no definition found') if !definition_class
|
71
128
|
definition = definition_class.new.from_json(message.body)
|
72
129
|
message_handler.call(definition, message)
|
73
130
|
end
|
74
|
-
|
75
|
-
|
76
|
-
|
77
|
-
|
131
|
+
|
132
|
+
log_execution(true, last_message, current_job_type, current_start_milliseconds)
|
133
|
+
|
134
|
+
break if options[:idle_timeout] && ((Time.now - last_message_at) > options[:idle_timeout])
|
135
|
+
|
136
|
+
unless last_message
|
137
|
+
Kernel.sleep(options[:poll_interval]) unless options[:poll_interval] == 0
|
138
|
+
end
|
78
139
|
rescue Exception => e
|
140
|
+
log_execution(false, last_message, current_job_type, current_start_milliseconds)
|
141
|
+
|
79
142
|
if options[:raise_exceptions]
|
80
143
|
raise e
|
81
144
|
else
|
@@ -84,7 +147,15 @@ class SQSJobQueue < JobQueue
|
|
84
147
|
logger.error(e.backtrace.join("\n "))
|
85
148
|
end
|
86
149
|
end
|
150
|
+
break if exit_next
|
151
|
+
end
|
152
|
+
|
153
|
+
logger.debug 'restoring previous signal traps'
|
154
|
+
previous_traps.each do |signal, command|
|
155
|
+
Signal.trap(signal, command)
|
87
156
|
end
|
157
|
+
|
158
|
+
logger.info "shutdown successful"
|
88
159
|
end
|
89
160
|
|
90
161
|
private
|
@@ -93,17 +164,125 @@ class SQSJobQueue < JobQueue
|
|
93
164
|
attr_accessor :queues
|
94
165
|
end
|
95
166
|
|
96
|
-
attr_accessor :sqs_queue, :visibility_timeout
|
167
|
+
attr_accessor :queue_name, :sqs_queue, :visibility_timeout, :cloud_watch
|
97
168
|
|
98
169
|
def initialize(type, visibility_timeout)
|
99
170
|
sqs = ::AWS::SQS.new
|
100
|
-
self.
|
171
|
+
self.queue_name = "#{self.class.config[:queue_prefix]}-#{type}-#{self.class.config[:environment]}"
|
172
|
+
self.sqs_queue = sqs.queues.create(queue_name)
|
101
173
|
self.visibility_timeout = visibility_timeout
|
174
|
+
self.cloud_watch = Fog::AWS::CloudWatch.new(
|
175
|
+
:aws_access_key_id => AWS.config.access_key_id,
|
176
|
+
:aws_secret_access_key => AWS.config.secret_access_key
|
177
|
+
)
|
102
178
|
end
|
103
179
|
|
104
180
|
def logger
|
105
181
|
JobQueue.config[:logger]
|
106
182
|
end
|
107
183
|
|
184
|
+
def get_milliseconds(time = Time.now)
|
185
|
+
(time.to_f * 1000).round
|
186
|
+
end
|
187
|
+
|
188
|
+
def log_execution(successful, message, job_type, start_milliseconds)
|
189
|
+
if self.class.config[:cloud_watch_namespace]
|
190
|
+
timestamp = DateTime.now.to_s
|
191
|
+
environment = self.class.config[:environment]
|
192
|
+
hostname = Socket.gethostbyname(Socket.gethostname).first
|
193
|
+
|
194
|
+
message_dimensions = [
|
195
|
+
{ 'Name' => 'Environment', 'Value' => environment },
|
196
|
+
{ 'Name' => 'SQSQueueName', 'Value' => queue_name },
|
197
|
+
{ 'Name' => 'Host', 'Value' => hostname },
|
198
|
+
]
|
199
|
+
|
200
|
+
job_dimensions = message_dimensions + [
|
201
|
+
{ 'Name' => 'JobType', 'Value' => job_type },
|
202
|
+
]
|
203
|
+
|
204
|
+
metric_data = [
|
205
|
+
{
|
206
|
+
'MetricName' => 'MessageCheckCount',
|
207
|
+
'Timestamp' => timestamp,
|
208
|
+
'Unit' => 'Count',
|
209
|
+
'Value' => 1,
|
210
|
+
'Dimensions' => message_dimensions
|
211
|
+
},
|
212
|
+
{
|
213
|
+
'MetricName' => 'MessageReceivedCount',
|
214
|
+
'Timestamp' => timestamp,
|
215
|
+
'Unit' => 'Count',
|
216
|
+
'Value' => message ? 1 : 0,
|
217
|
+
'Dimensions' => message_dimensions
|
218
|
+
},
|
219
|
+
{
|
220
|
+
'MetricName' => 'MessageMissCount',
|
221
|
+
'Timestamp' => timestamp,
|
222
|
+
'Unit' => 'Count',
|
223
|
+
'Value' => message ? 0 : 1,
|
224
|
+
'Dimensions' => message_dimensions
|
225
|
+
}
|
226
|
+
]
|
227
|
+
|
228
|
+
if message
|
229
|
+
now = get_milliseconds
|
230
|
+
|
231
|
+
metric_data.concat([
|
232
|
+
{
|
233
|
+
'MetricName' => 'ExecutionCount',
|
234
|
+
'Timestamp' => timestamp,
|
235
|
+
'Unit' => 'Count',
|
236
|
+
'Value' => 1,
|
237
|
+
'Dimensions' => job_dimensions
|
238
|
+
},
|
239
|
+
{
|
240
|
+
'MetricName' => 'SuccessCount',
|
241
|
+
'Timestamp' => timestamp,
|
242
|
+
'Unit' => 'Count',
|
243
|
+
'Value' => successful ? 1 : 0,
|
244
|
+
'Dimensions' => job_dimensions
|
245
|
+
},
|
246
|
+
{
|
247
|
+
'MetricName' => 'ErrorCount',
|
248
|
+
'Timestamp' => timestamp,
|
249
|
+
'Unit' => 'Count',
|
250
|
+
'Value' => successful ? 0 : 1,
|
251
|
+
'Dimensions' => job_dimensions
|
252
|
+
},
|
253
|
+
{
|
254
|
+
'MetricName' => 'ExecutionTime',
|
255
|
+
'Timestamp' => timestamp,
|
256
|
+
'Unit' => 'Milliseconds',
|
257
|
+
'Value' => now - start_milliseconds,
|
258
|
+
'Dimensions' => job_dimensions
|
259
|
+
}
|
260
|
+
])
|
261
|
+
|
262
|
+
if successful
|
263
|
+
metric_data.concat([
|
264
|
+
{
|
265
|
+
'MetricName' => 'TimeToCompletion',
|
266
|
+
'Timestamp' => timestamp,
|
267
|
+
'Unit' => 'Milliseconds',
|
268
|
+
'Value' => now - get_milliseconds(message.sent_at),
|
269
|
+
'Dimensions' => job_dimensions
|
270
|
+
},
|
271
|
+
{
|
272
|
+
'MetricName' => 'ExecutionAttempts',
|
273
|
+
'Timestamp' => timestamp,
|
274
|
+
'Unit' => 'Count',
|
275
|
+
'Value' => message.receive_count,
|
276
|
+
'Dimensions' => job_dimensions
|
277
|
+
}
|
278
|
+
])
|
279
|
+
end
|
280
|
+
end
|
281
|
+
|
282
|
+
cloud_watch.put_metric_data(self.class.config[:cloud_watch_namespace], metric_data)
|
283
|
+
end
|
284
|
+
end
|
285
|
+
|
108
286
|
end
|
109
287
|
end
|
288
|
+
|
data/lib/simple_job/version.rb
CHANGED
metadata
CHANGED
@@ -1,13 +1,13 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: simple_job
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
hash:
|
4
|
+
hash: 27
|
5
5
|
prerelease:
|
6
6
|
segments:
|
7
7
|
- 0
|
8
|
+
- 1
|
8
9
|
- 0
|
9
|
-
|
10
|
-
version: 0.0.3
|
10
|
+
version: 0.1.0
|
11
11
|
platform: ruby
|
12
12
|
authors:
|
13
13
|
- David Dawson
|
@@ -62,6 +62,21 @@ dependencies:
|
|
62
62
|
name: aws-sdk
|
63
63
|
prerelease: false
|
64
64
|
type: :runtime
|
65
|
+
- !ruby/object:Gem::Dependency
|
66
|
+
requirement: &id004 !ruby/object:Gem::Requirement
|
67
|
+
none: false
|
68
|
+
requirements:
|
69
|
+
- - ~>
|
70
|
+
- !ruby/object:Gem::Version
|
71
|
+
hash: 13
|
72
|
+
segments:
|
73
|
+
- 1
|
74
|
+
- 1
|
75
|
+
version: "1.1"
|
76
|
+
version_requirements: *id004
|
77
|
+
name: fog
|
78
|
+
prerelease: false
|
79
|
+
type: :runtime
|
65
80
|
description: Contains libraries that support defining, queueing, and executing jobs.
|
66
81
|
email: daws23@gmail.com
|
67
82
|
executables: []
|