procrastinator 1.0.0.pre.rc3 → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +285 -294
- data/RELEASE_NOTES.md +10 -5
- data/Rakefile +7 -0
- data/lib/procrastinator/config.rb +34 -3
- data/lib/procrastinator/logged_task.rb +2 -4
- data/lib/procrastinator/queue.rb +26 -2
- data/lib/procrastinator/queue_worker.rb +4 -0
- data/lib/procrastinator/rake/daemon_tasks.rb +16 -9
- data/lib/procrastinator/rspec/matchers.rb +30 -0
- data/lib/procrastinator/scheduler.rb +19 -12
- data/lib/procrastinator/task.rb +12 -0
- data/lib/procrastinator/task_meta_data.rb +12 -0
- data/lib/procrastinator/task_store/file_transaction.rb +53 -55
- data/lib/procrastinator/task_store/simple_comma_store.rb +33 -9
- data/lib/procrastinator/test/mocks.rb +9 -0
- data/lib/procrastinator/version.rb +2 -1
- data/procrastinator.gemspec +1 -0
- metadata +19 -4
data/README.md
CHANGED
@@ -1,78 +1,74 @@
|
|
1
1
|
# Procrastinator
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
Tasks are can be rescheduled and retried after failures, and you can use whatever storage mechanism is needed.
|
3
|
+
A storage-agnostic job queue gem in plain Ruby.
|
6
4
|
|
7
5
|
## Big Picture
|
8
6
|
|
9
|
-
|
7
|
+
Define **Task Handler** classes like this:
|
10
8
|
|
11
9
|
```ruby
|
12
10
|
# Sends a welcome email
|
13
11
|
class SendWelcomeEmail
|
12
|
+
attr_accessor :container, :logger, :scheduler
|
13
|
+
|
14
14
|
def run
|
15
15
|
# ... etc
|
16
16
|
end
|
17
17
|
end
|
18
18
|
```
|
19
19
|
|
20
|
-
|
20
|
+
Then build a task **Scheduler**:
|
21
21
|
|
22
22
|
```ruby
|
23
|
-
scheduler = Procrastinator.setup do |
|
24
|
-
|
25
|
-
|
26
|
-
|
23
|
+
scheduler = Procrastinator.setup do |config|
|
24
|
+
config.with_store some_email_task_database do
|
25
|
+
config.define_queue :welcome, SendWelcomeEmail
|
26
|
+
config.define_queue :birthday, SendBirthdayEmail, max_attempts: 3
|
27
27
|
end
|
28
28
|
|
29
|
-
|
29
|
+
config.define_queue :thumbnail, GenerateThumbnail, store: 'imgtasks.csv', timeout: 60
|
30
30
|
end
|
31
31
|
```
|
32
32
|
|
33
|
-
|
33
|
+
And **defer** tasks:
|
34
34
|
|
35
35
|
```ruby
|
36
|
-
scheduler.
|
36
|
+
scheduler.defer(:welcome, data: 'elanor@example.com')
|
37
37
|
|
38
|
-
scheduler.
|
38
|
+
scheduler.defer(:thumbnail, data: {file: 'forcett.png', width: 100, height: 150})
|
39
39
|
|
40
|
-
scheduler.
|
41
|
-
```
|
42
|
-
|
43
|
-
And tell a process to actually do them:
|
44
|
-
|
45
|
-
```ruby
|
46
|
-
# Starts a daemon with a thread for each queue.
|
47
|
-
# Other options are direct control (for testing) or threaded (for terminals)
|
48
|
-
scheduler.work.daemonized!
|
40
|
+
scheduler.defer(:birthday, run_at: Time.now + 3600, data: {user_id: 5})
|
49
41
|
```
|
50
42
|
|
51
43
|
## Contents
|
52
44
|
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
- [
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
|
45
|
+
* [Installation](#installation)
|
46
|
+
* [Task Handlers](#task-handlers)
|
47
|
+
+ [Attribute Accessors](#attribute-accessors)
|
48
|
+
+ [Errors & Logging](#errors---logging)
|
49
|
+
* [Configuration](#configuration)
|
50
|
+
+ [Defining Queues](#defining-queues)
|
51
|
+
+ [Task Store](#task-store)
|
52
|
+
- [Data Fields](#data-fields)
|
53
|
+
- [CSV Task Store](#csv-task-store)
|
54
|
+
- [Shared Task Stores](#shared-task-stores)
|
55
|
+
+ [Task Container](#task-container)
|
56
|
+
* [Deferring Tasks](#deferring-tasks)
|
57
|
+
+ [Timing](#timing)
|
58
|
+
+ [Rescheduling Existing Tasks](#rescheduling-existing-tasks)
|
59
|
+
+ [Retries](#retries)
|
60
|
+
+ [Cancelling](#cancelling)
|
61
|
+
* [Running Tasks](#running-tasks)
|
62
|
+
+ [In Testing](#in-testing)
|
63
|
+
- [RSpec Matchers](#rspec-matchers)
|
64
|
+
+ [In Production](#in-production)
|
65
|
+
* [Similar Tools](#similar-tools)
|
66
|
+
+ [Linux etc: Cron and At](#linux-etc--cron-and-at)
|
67
|
+
+ [Gem: Resque](#gem--resque)
|
68
|
+
+ [Gem: Rails ActiveJob / DelayedJob](#gem--rails-activejob---delayedjob)
|
69
|
+
* [Contributing](#contributing)
|
70
|
+
+ [Developers](#developers)
|
71
|
+
* [License](#license)
|
76
72
|
|
77
73
|
<!-- ToC generated with http://ecotrust-canada.github.io/markdown-toc/ -->
|
78
74
|
|
@@ -84,13 +80,118 @@ Add this line to your application's Gemfile:
|
|
84
80
|
gem 'procrastinator'
|
85
81
|
```
|
86
82
|
|
87
|
-
And then run:
|
83
|
+
And then run in a terminal:
|
88
84
|
|
89
85
|
bundle install
|
90
86
|
|
91
|
-
##
|
87
|
+
## Task Handlers
|
88
|
+
|
89
|
+
Task Handlers are what actually get run on the task queue. They'll look like this:
|
90
|
+
|
91
|
+
```ruby
|
92
|
+
# This is an example task handler
|
93
|
+
class MyTask
|
94
|
+
# These attributes will be assigned by Procrastinator when the task is run.
|
95
|
+
# :data is optional
|
96
|
+
attr_accessor :container, :logger, :scheduler, :data
|
97
|
+
|
98
|
+
# Performs the core work of the task.
|
99
|
+
def run
|
100
|
+
# ... perform your task ...
|
101
|
+
end
|
102
|
+
|
103
|
+
# ==================================
|
104
|
+
# OPTIONAL HOOKS
|
105
|
+
# ==================================
|
106
|
+
#
|
107
|
+
# You can always omit any of the methods below. Only #run is mandatory.
|
108
|
+
##
|
109
|
+
|
110
|
+
# Called after the task has completed successfully.
|
111
|
+
#
|
112
|
+
# @param run_result [Object] The result of #run.
|
113
|
+
def success(run_result)
|
114
|
+
# ...
|
115
|
+
end
|
116
|
+
|
117
|
+
# Called after #run raises any StandardError (or subclass).
|
118
|
+
#
|
119
|
+
# @param error [StandardError] Error raised by #run
|
120
|
+
def fail(error)
|
121
|
+
# ...
|
122
|
+
end
|
123
|
+
|
124
|
+
# Called after a permanent failure, either because:
|
125
|
+
# 1. the current time is after the task's expire_at time.
|
126
|
+
# 2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`.
|
127
|
+
#
|
128
|
+
# If #final_fail is executed, then #fail will not.
|
129
|
+
#
|
130
|
+
# @param error [StandardError] Error raised by #run
|
131
|
+
def final_fail(error)
|
132
|
+
# ...
|
133
|
+
end
|
134
|
+
end
|
135
|
+
```
|
136
|
+
|
137
|
+
### Attribute Accessors
|
138
|
+
|
139
|
+
Task Handlers have attributes that are set after the Handler is created. The attributes are enforced early on to prevent
|
140
|
+
the tasks from referencing unknown variables at whatever time they are run - if they're missing, you'll get
|
141
|
+
a `MalformedTaskError`.
|
142
|
+
|
143
|
+
| Attribute | Required | Description |
|
144
|
+
|------------|----------|-------------|
|
145
|
+
|`:container`| Yes | Container declared in `#setup` from the currently running instance |
|
146
|
+
|`:logger` | Yes | Logger object for the Queue |
|
147
|
+
|`:scheduler`| Yes | A scheduler object that you can use to schedule new tasks (eg. with `#defer`)|
|
148
|
+
|`:data` | No | Data provided to `#defer`. Calls to `#defer` will error if they do not provide data when expected and vice-versa. |
|
149
|
+
|
150
|
+
### Errors & Logging
|
151
|
+
|
152
|
+
Errors that trigger `#fail` or `#final_fail` are saved to the task storage under keywords `last_error` and
|
153
|
+
`last_fail_at`.
|
154
|
+
|
155
|
+
Each queue worker also keeps a logfile log using the Ruby
|
156
|
+
[Logger class](https://ruby-doc.org/stdlib-2.7.1/libdoc/logger/rdoc/Logger.html). Log files are named after the queue (
|
157
|
+
eg. `log/welcome-queue-worker.log`).
|
158
|
+
|
159
|
+
```ruby
|
160
|
+
scheduler = Procrastinator.setup do |config|
|
161
|
+
# you can set custom log location and level:
|
162
|
+
config.log_with(directory: '/var/log/myapp/', level: Logger::DEBUG)
|
163
|
+
|
164
|
+
# you can also set the log rotation age or size (see Logger docs for details)
|
165
|
+
config.log_with(shift: 1024, age: 5)
|
166
|
+
|
167
|
+
# use a falsey log level to disable logging entirely:
|
168
|
+
config.log_with(level: false)
|
169
|
+
end
|
170
|
+
```
|
171
|
+
|
172
|
+
The logger can be accessed in your tasks by calling `logger` or `@logger`.
|
173
|
+
|
174
|
+
```ruby
|
175
|
+
# Example handler with logging
|
176
|
+
class MyTask
|
177
|
+
attr_accessor :container, :logger, :scheduler
|
178
|
+
|
179
|
+
def run
|
180
|
+
logger.info('This task got run. Hooray!')
|
181
|
+
end
|
182
|
+
end
|
183
|
+
```
|
184
|
+
|
185
|
+
Some events are always logged by default:
|
92
186
|
|
93
|
-
|
187
|
+
|event |level |
|
188
|
+
|--------------------|-------|
|
189
|
+
|Task completed | INFO |
|
190
|
+
|Task cailure | ERROR |
|
191
|
+
|
192
|
+
## Configuration
|
193
|
+
|
194
|
+
`Procrastinator.setup` allows you to define which queues are available and other general settings.
|
94
195
|
|
95
196
|
```ruby
|
96
197
|
require 'procrastinator'
|
@@ -100,14 +201,11 @@ scheduler = Procrastinator.setup do |config|
|
|
100
201
|
end
|
101
202
|
```
|
102
203
|
|
103
|
-
It then returns a
|
104
|
-
|
105
|
-
* See [Scheduling Tasks](#scheduling-tasks)
|
106
|
-
* See [Start Working](#start-working)
|
204
|
+
It then returns a **Task Scheduler** that your code can use to defer tasks.
|
107
205
|
|
108
206
|
### Defining Queues
|
109
207
|
|
110
|
-
In setup, call `#define_queue` with a symbol name and
|
208
|
+
In setup, call `#define_queue` with a symbol name and that queue's Task Handler class:
|
111
209
|
|
112
210
|
```ruby
|
113
211
|
# You must provide a queue name and the class that handles those jobs
|
@@ -132,7 +230,7 @@ Description of keyword options:
|
|
132
230
|
### Task Store
|
133
231
|
|
134
232
|
A task store is a [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) pattern object that knows how to read and
|
135
|
-
write tasks in your data storage (eg. database, CSV file, etc).
|
233
|
+
write tasks in your data storage (eg. database, HTTP API, CSV file, microdot, etc).
|
136
234
|
|
137
235
|
```ruby
|
138
236
|
task_store = ReminderStore.new # eg. some SQL task storage class you wrote
|
@@ -151,17 +249,17 @@ A task store is required to implement *all* of the following methods or else it
|
|
151
249
|
1. `#read(attributes)`
|
152
250
|
|
153
251
|
Returns a list of hashes from your datastore that match the given attributes hash. The search attributes will be in
|
154
|
-
their final form (eg. `:data` will already be serialized). Each hash must contain the properties listed
|
155
|
-
|
252
|
+
their final form (eg. `:data` will already be serialized). Each hash must contain the properties listed in
|
253
|
+
the [Data Fields](#data-fields) table.
|
156
254
|
|
157
255
|
2. `#create(queue:, run_at:, initial_run_at:, expire_at:, data:)`
|
158
256
|
|
159
|
-
Saves a task in your storage. Receives a hash with [
|
257
|
+
Saves a task in your storage. Receives a hash with [Data Fields](#data-fields) keys:
|
160
258
|
`:queue`, `:run_at`, `:initial_run_at`, `:expire_at`, and `:data`.
|
161
259
|
|
162
260
|
3. `#update(id, new_data)`
|
163
261
|
|
164
|
-
Saves the provided full [
|
262
|
+
Saves the provided full [Data Fields](#data-fields) hash to your datastore.
|
165
263
|
|
166
264
|
4. `#delete(id)`
|
167
265
|
|
@@ -177,32 +275,32 @@ _Warning_: Task stores shared between queues **must** be thread-safe if using th
|
|
177
275
|
These are the data fields for each individual scheduled task. When using the built-in task store, these are the field
|
178
276
|
names. If you have a database, use this to inform your table schema.
|
179
277
|
|
180
|
-
| Hash Key | Type
|
181
|
-
|
182
|
-
| `:id` |
|
183
|
-
| `:queue` | symbol
|
184
|
-
| `:run_at` |
|
185
|
-
| `:initial_run_at` |
|
186
|
-
| `:expire_at` |
|
187
|
-
| `:attempts` |
|
188
|
-
| `:last_fail_at` |
|
189
|
-
| `:last_error` | string
|
190
|
-
| `:data` |
|
191
|
-
|
192
|
-
|
193
|
-
|
194
|
-
|
195
|
-
|
278
|
+
| Hash Key | Type | Description |
|
279
|
+
|-------------------|----------| ------------------------------------------------------------------------|
|
280
|
+
| `:id` | integer | Unique identifier for this exact task |
|
281
|
+
| `:queue` | symbol | Name of the queue the task is inside |
|
282
|
+
| `:run_at` | datetime | Time to attempt running the task next. Updated for retries¹ |
|
283
|
+
| `:initial_run_at` | datetime | Original `run_at` value. Reset if `#reschedule` is called. |
|
284
|
+
| `:expire_at` | datetime | Time to permanently fail the task because it is too late to be useful |
|
285
|
+
| `:attempts` | integer | Number of times the task has tried to run |
|
286
|
+
| `:last_fail_at` | datetime | Time of the most recent failure |
|
287
|
+
| `:last_error` | string | Error message + backtrace of the most recent failure. May be very long. |
|
288
|
+
| `:data` | JSON | Data to be provided to the task handler, serialized² to JSON. |
|
289
|
+
|
290
|
+
¹ `nil` indicates that it is permanently failed and will never run, either due to expiry or too many attempts.
|
291
|
+
|
292
|
+
² Serialized using `JSON.dump` and `JSON.parse` with **symbolized keys**. It is strongly recommended to only supply
|
293
|
+
simple data types (eg. id numbers) to reduce storage space, eliminate redundancy, and reduce the chance of a
|
196
294
|
serialization error.
|
197
295
|
|
198
|
-
Times are all handled as
|
296
|
+
Times are all handled as Ruby stdlib Time objects.
|
199
297
|
|
200
|
-
####
|
298
|
+
#### CSV Task Store
|
201
299
|
|
202
300
|
Specifying no storage will cause Procrastinator to save tasks using the very basic built-in CSV storage. It is not
|
203
301
|
designed for heavy loads, so you should replace it in a production environment.
|
204
302
|
|
205
|
-
The file path is defined in `Procrastinator::Store::SimpleCommaStore::DEFAULT_FILE`.
|
303
|
+
The default file path is defined in `Procrastinator::Store::SimpleCommaStore::DEFAULT_FILE`.
|
206
304
|
|
207
305
|
```ruby
|
208
306
|
Procrastinator.setup do |config|
|
@@ -213,7 +311,7 @@ end
|
|
213
311
|
|
214
312
|
#### Shared Task Stores
|
215
313
|
|
216
|
-
When there are tasks use the same storage, you can wrap them in a `with_store` block.
|
314
|
+
When there are tasks that use the same storage, you can wrap them in a `with_store` block.
|
217
315
|
|
218
316
|
```ruby
|
219
317
|
email_task_store = EmailTaskStore.new # eg. some SQL task storage class you wrote
|
@@ -232,20 +330,19 @@ end
|
|
232
330
|
|
233
331
|
### Task Container
|
234
332
|
|
235
|
-
Whatever is given to `#provide_container` will available to
|
236
|
-
|
237
|
-
This can be useful for things like app containers, but you can use it for whatever you like.
|
333
|
+
Whatever is given to `#provide_container` will be available to Task Handlers via the `:container` attribute and it is
|
334
|
+
intended for dependency injection.
|
238
335
|
|
239
336
|
```ruby
|
240
|
-
Procrastinator.setup do |
|
241
|
-
|
337
|
+
Procrastinator.setup do |config|
|
338
|
+
config.provide_container lunch: 'Lasagna'
|
242
339
|
|
243
340
|
# .. other setup stuff ...
|
244
341
|
end
|
245
342
|
|
246
343
|
# ... and in your task ...
|
247
344
|
class LunchTask
|
248
|
-
attr_accessor :
|
345
|
+
attr_accessor :container, :logger, :scheduler
|
249
346
|
|
250
347
|
def run
|
251
348
|
logger.info("Today's Lunch is: #{ container[:lunch] }")
|
@@ -253,243 +350,121 @@ class LunchTask
|
|
253
350
|
end
|
254
351
|
```
|
255
352
|
|
256
|
-
## Tasks
|
257
|
-
|
258
|
-
Your task class is what actually gets run on the task queue. They'll look like this:
|
259
|
-
|
260
|
-
```ruby
|
261
|
-
|
262
|
-
class MyTask
|
263
|
-
# These attributes will be assigned by Procrastinator when the task is run.
|
264
|
-
attr_accessor :logger, :scheduler, :container, :data
|
265
|
-
|
266
|
-
# Performs the core work of the task.
|
267
|
-
def run
|
268
|
-
# ... perform your task ...
|
269
|
-
end
|
270
|
-
|
271
|
-
# ========================================
|
272
|
-
# OPTIONAL HOOKS
|
273
|
-
#
|
274
|
-
# You can always omit any of the methods
|
275
|
-
# below. Only #run is mandatory.
|
276
|
-
#
|
277
|
-
# ========================================
|
278
|
-
|
279
|
-
# Called after the task has completed successfully.
|
280
|
-
# Receives the result of #run.
|
281
|
-
def success(run_result)
|
282
|
-
# ...
|
283
|
-
end
|
284
|
-
|
285
|
-
# Called after #run raises any StandardError (or subclass).
|
286
|
-
# Receives the raised error.
|
287
|
-
def fail(error)
|
288
|
-
# ...
|
289
|
-
end
|
290
|
-
|
291
|
-
# Called after either is true:
|
292
|
-
# 1. the time reported by Time.now is past the task's expire_at time.
|
293
|
-
# 2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`.
|
294
|
-
# In this case, #fail will not be executed, only #final_fail.
|
295
|
-
#
|
296
|
-
# When called, the task will be marked to never be run again.
|
297
|
-
# Receives the raised error.
|
298
|
-
def final_fail(error)
|
299
|
-
# ...
|
300
|
-
end
|
301
|
-
end
|
302
|
-
```
|
303
|
-
|
304
|
-
### Attribute Accessors
|
305
|
-
|
306
|
-
Tasks must provide accessors for `:logger`, `:container`, and `:scheduler`, while the `:data` accessor is semi-optional
|
307
|
-
(see below). This is to prevent the tasks from referencing unknown variables once they actually get run.
|
308
|
-
|
309
|
-
* `:data` The data you provided in the call to `#delay`. Any task with a `:data` accessor will require data be passed
|
310
|
-
to `#delay`, and vice versa. See [Task Data](#task-data) for more.
|
311
|
-
|
312
|
-
* `:container` The container you've provided in your setup. See [Task Container](#task-container) for more.
|
313
|
-
|
314
|
-
* `:logger` The queue's Logger object. See [Logging](#logging) for more.
|
315
|
-
|
316
|
-
* `:scheduler` A scheduler object that you can use to schedule new tasks (eg. with `#delay`).
|
317
|
-
|
318
|
-
### Errors & Logging
|
319
|
-
|
320
|
-
Errors that trigger `#fail` or `#final_fail` are saved to the task storage under columns `last_error` and
|
321
|
-
`last_error_at`.
|
322
|
-
|
323
|
-
Each queue worker also keeps a logfile log using the Ruby
|
324
|
-
[Logger class](https://ruby-doc.org/stdlib-2.7.1/libdoc/logger/rdoc/Logger.html). Log files are named after the queue (
|
325
|
-
eg. `log/welcome-queue-worker.log`).
|
326
|
-
|
327
|
-
```ruby
|
328
|
-
scheduler = Procrastinator.setup do |env|
|
329
|
-
# you can set custom log location and level:
|
330
|
-
env.log_with(directory: '/var/log/myapp/', level: Logger::DEBUG)
|
331
|
-
|
332
|
-
# you can also set the log rotation age or size (see Logger docs for details)
|
333
|
-
env.log_with(shift: 1024, age: 5)
|
334
|
-
|
335
|
-
# use a falsey log level to disable logging entirely:
|
336
|
-
env.log_with(level: false)
|
337
|
-
end
|
338
|
-
```
|
339
|
-
|
340
|
-
The logger can be accessed in your tasks by calling `logger` or `@logger`.
|
341
|
-
|
342
|
-
```ruby
|
343
|
-
|
344
|
-
class MyTask
|
345
|
-
attr_accessor :logger, :scheduler, :container
|
346
|
-
|
347
|
-
def run
|
348
|
-
logger.info('This task got run. Hooray!')
|
349
|
-
end
|
350
|
-
end
|
351
|
-
```
|
352
|
-
|
353
|
-
Some events are always logged by default:
|
354
|
-
|
355
|
-
|event |level |
|
356
|
-
|--------------------|-------|
|
357
|
-
|process started | INFO |
|
358
|
-
|#success called | DEBUG |
|
359
|
-
|#fail called | DEBUG |
|
360
|
-
|#final_fail called | DEBUG |
|
361
|
-
|
362
|
-
## Scheduling Tasks
|
353
|
+
## Deferring Tasks
|
363
354
|
|
364
|
-
To
|
355
|
+
To add tasks to a queue, call `#defer` on the scheduler returned by `Procrastinator.setup`:
|
365
356
|
|
366
357
|
```ruby
|
367
|
-
scheduler = Procrastinator.setup do |
|
368
|
-
|
369
|
-
|
358
|
+
scheduler = Procrastinator.setup do |config|
|
359
|
+
config.define_queue :reminder, EmailEveryone
|
360
|
+
config.define_queue :thumbnail, CreateThumbnail
|
370
361
|
end
|
371
362
|
|
372
|
-
# Provide the queue name and any data you want passed in
|
373
|
-
scheduler.
|
363
|
+
# Provide the queue name and any data you want passed in, if needed
|
364
|
+
scheduler.defer(:reminder)
|
365
|
+
scheduler.defer(:thumbnail, data: 'forcett.png')
|
374
366
|
```
|
375
367
|
|
376
368
|
If there is only one queue, you may omit the queue name:
|
377
369
|
|
378
370
|
```ruby
|
379
|
-
|
380
|
-
|
371
|
+
thumbnailer = Procrastinator.setup do |config|
|
372
|
+
config.define_queue :thumbnail, CreateThumbnail
|
381
373
|
end
|
382
374
|
|
383
|
-
|
375
|
+
thumbnailer.defer(data: 'forcett.png')
|
384
376
|
```
|
385
377
|
|
386
|
-
###
|
387
|
-
|
388
|
-
Most tasks need some additional information to complete their work, like id numbers,
|
378
|
+
### Timing
|
389
379
|
|
390
|
-
|
391
|
-
you have a database instead of passing in a complex Ruby object, pass in just the primary key and reload it in the
|
392
|
-
task's `#run`. This will require less space in your database and avoids obsolete or duplicated information.
|
380
|
+
You can specify a particular timeframe that a task may be run. The default is to run immediately and never expire.
|
393
381
|
|
394
|
-
|
382
|
+
Be aware that the task is not guaranteed to run at a precise time; the only promise is that the task won't be tried *
|
383
|
+
before* `run_at` nor *after* `expire_at`.
|
395
384
|
|
396
|
-
|
397
|
-
|
385
|
+
Tasks attempted after `expire_at` will be final-failed. Setting `expire_at` to `nil`
|
386
|
+
means it will never expire (but may still fail permanently if, say, `max_attempts` is reached).
|
398
387
|
|
399
388
|
```ruby
|
400
|
-
|
401
|
-
|
389
|
+
run_time = Time.new(2016, 9, 19)
|
390
|
+
expire_time = Time.new(2016, 9, 20)
|
402
391
|
|
403
|
-
#
|
404
|
-
scheduler.
|
405
|
-
```
|
392
|
+
# runs on or after 2016 Sept 19, never expires
|
393
|
+
scheduler.defer(:greeting, run_at: run_time, data: 'elanor@example.com')
|
406
394
|
|
407
|
-
|
408
|
-
|
409
|
-
fail permanently if, say, `max_attempts` is reached).
|
410
|
-
|
411
|
-
```ruby
|
412
|
-
# will not run at or after
|
413
|
-
scheduler.delay(:happy_birthday, expire_at: Time.new(2018, 03, 17, 12, 00, '-06:00'), data: 'contact@tenjin.ca')
|
395
|
+
# can run immediately but not after 2016 Sept 20
|
396
|
+
scheduler.defer(:greeting, expire_at: expire_time, data: 'mendoza@example.com')
|
414
397
|
|
415
|
-
#
|
416
|
-
scheduler.
|
398
|
+
# can run immediately but not after 2016 Sept 20
|
399
|
+
scheduler.defer(:greeting, run_at: run_time, expire_at: expire_time, data: 'tahani@example.com')
|
417
400
|
```
|
418
401
|
|
419
|
-
### Rescheduling
|
402
|
+
### Rescheduling Existing Tasks
|
420
403
|
|
421
|
-
Call `#reschedule` with the queue name and some identifying information
|
422
|
-
time.
|
404
|
+
Call `#reschedule` with the queue name and some task-identifying information and then chain `#to` with the new time.
|
423
405
|
|
424
406
|
```ruby
|
425
|
-
|
426
|
-
|
427
|
-
end
|
407
|
+
run_time = Time.new(2016, 9, 19)
|
408
|
+
expire_time = Time.new(2016, 9, 20)
|
428
409
|
|
429
|
-
scheduler.
|
410
|
+
scheduler.defer(:reminder, run_at: Time.at(0), data: 'chidi@example.com')
|
430
411
|
|
431
|
-
# we can reschedule the task
|
432
|
-
scheduler.reschedule(:reminder, data: '
|
412
|
+
# we can reschedule the task that matches this data
|
413
|
+
scheduler.reschedule(:reminder, data: 'chidi@example.com').to(run_at: run_time)
|
433
414
|
|
434
415
|
# we can also change the expiry time
|
435
|
-
scheduler.reschedule(:reminder, data: '
|
416
|
+
scheduler.reschedule(:reminder, data: 'chidi@example.com').to(expire_at: expire_time)
|
436
417
|
|
437
418
|
# or both
|
438
|
-
scheduler.reschedule(:reminder, data: '
|
439
|
-
|
419
|
+
scheduler.reschedule(:reminder, data: 'chidi@example.com').to(run_at: run_time,
|
420
|
+
expire_at: expire_time)
|
440
421
|
```
|
441
422
|
|
442
|
-
Rescheduling
|
423
|
+
Rescheduling changes the task's...
|
443
424
|
|
444
425
|
* `:run_at` and `:initial_run_at` to a new value, if provided
|
445
426
|
* `:expire_at` to a new value if provided.
|
446
427
|
* `:attempts` to `0`
|
447
428
|
* `:last_error` and `:last_error_at` to `nil`.
|
448
429
|
|
449
|
-
Rescheduling will not change `:id`, `:queue` or `:data`.
|
430
|
+
Rescheduling will not change `:id`, `:queue` or `:data`.
|
431
|
+
|
432
|
+
A `RuntimeError` is raised if the new run_at is after expire_at.
|
450
433
|
|
451
434
|
### Retries
|
452
435
|
|
453
|
-
Failed tasks
|
436
|
+
Failed tasks are automatically retried, with their `run_at` updated on an increasing delay (in seconds) according to
|
437
|
+
this formula:
|
454
438
|
|
455
|
-
> 30 +
|
439
|
+
> 30 + number_of_attempts<sup>4</sup>
|
456
440
|
|
457
441
|
Situations that call `#fail` or `#final_fail` will cause the error timestamp and reason to be stored in `:last_fail_at`
|
458
442
|
and `:last_error`.
|
459
443
|
|
460
444
|
### Cancelling
|
461
445
|
|
462
|
-
Call `#cancel` with the queue name and some identifying information to narrow the search to a single task.
|
446
|
+
Call `#cancel` with the queue name and some task-identifying information to narrow the search to a single task.
|
463
447
|
|
464
448
|
```ruby
|
465
|
-
|
466
|
-
|
467
|
-
end
|
449
|
+
run_time = Time.parse('April 1')
|
450
|
+
scheduler.defer(:reminder, run_at: run_time, data: 'derek@example.com')
|
468
451
|
|
469
|
-
|
470
|
-
|
471
|
-
|
472
|
-
scheduler.reschedule(:reminder, data: 'bob@example.com')
|
473
|
-
|
474
|
-
# or multiple attributes
|
475
|
-
scheduler.reschedule(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
|
452
|
+
# we can cancel the task made above using whatever we know about it
|
453
|
+
# An error will be raised if it matches multiple tasks or finds none
|
454
|
+
scheduler.cancel(:reminder, run_at: run_time, data: 'derek@example.com')
|
476
455
|
|
477
456
|
# you could also use the id number directly, if you have it
|
478
|
-
scheduler.
|
457
|
+
scheduler.cancel(:reminder, id: 137)
|
479
458
|
```
|
480
459
|
|
481
|
-
##
|
482
|
-
|
483
|
-
Use the scheduler object returned by setup to `#work` queues **serially**, **threaded**, or **daemonized**.
|
460
|
+
## Testing with Procrastinator
|
484
461
|
|
485
|
-
|
462
|
+
Working serially performs tasks from each queue sequentially. There is no multithreading or daemonizing.
|
486
463
|
|
487
|
-
|
488
|
-
|
489
|
-
Work serially for TDD tests or other situations you need close direct control.
|
464
|
+
Call `work` on the Scheduler with an optional list of queues to filter by.
|
490
465
|
|
491
466
|
```ruby
|
492
|
-
# work just one task
|
467
|
+
# work just one task
|
493
468
|
scheduler.work.serially
|
494
469
|
|
495
470
|
# work the first five tasks
|
@@ -499,61 +474,75 @@ scheduler.work.serially(steps: 5)
|
|
499
474
|
scheduler.work(:greeting, :reminders).serially(steps: 2)
|
500
475
|
```
|
501
476
|
|
502
|
-
###
|
503
|
-
|
504
|
-
Threaded working will spawn a worker thread per queue.
|
477
|
+
### RSpec Matchers
|
505
478
|
|
506
|
-
|
507
|
-
caveats around multithreading, so proceed with caution.
|
479
|
+
A `have_task` RSpec matcher is defined to make testing task scheduling a little easier.
|
508
480
|
|
509
481
|
```ruby
|
510
|
-
#
|
511
|
-
|
482
|
+
# Note: you must require the matcher file separately
|
483
|
+
require 'procrastinator'
|
484
|
+
require 'procrastinator/rspec/matchers'
|
512
485
|
|
513
|
-
|
514
|
-
scheduler.work.threaded(timeout: 5)
|
486
|
+
task_storage = TaskStore.new
|
515
487
|
|
516
|
-
|
517
|
-
|
488
|
+
scheduler = Procrastinator.setup do |config|
|
489
|
+
config.define_queue :welcome, SendWelcome, store: task_storage
|
490
|
+
end
|
491
|
+
|
492
|
+
scheduler.defer(data: 'tahani@example.com')
|
493
|
+
|
494
|
+
expect(task_storage).to have_task(data: 'tahani@example.com')
|
518
495
|
```
|
519
496
|
|
520
|
-
|
497
|
+
## Running Tasks
|
521
498
|
|
522
|
-
|
499
|
+
When you are ready to run a Procrastinator daemon in production, you may use some provided Rake tasks.
|
523
500
|
|
524
|
-
|
525
|
-
like [Monit](https://mmonit.com/monit/). Provide a block to daemonized! to get
|
501
|
+
In your Rake file call `DaemonTasks.define` with a block that constructs a scheduler instance.
|
526
502
|
|
527
503
|
```ruby
|
528
|
-
#
|
529
|
-
|
504
|
+
# Rakefile
|
505
|
+
require 'rake'
|
506
|
+
require 'procrastinator/rake/daemon_tasks'
|
507
|
+
|
508
|
+
# Defines a set of tasks that will control a Procrastinator daemon
|
509
|
+
# Default pid_path is /tmp/procrastinator.pid
|
510
|
+
Procrastinator::Rake::DaemonTasks.define do
|
511
|
+
Procrastinator.setup do
|
512
|
+
# ... etc ...
|
513
|
+
end
|
514
|
+
end
|
515
|
+
```
|
530
516
|
|
531
|
-
|
532
|
-
|
517
|
+
You can name the daemon process by specifying the pid_path with a specific .pid file. If does not end with '.pid' it is
|
518
|
+
assumed to be a directory name, and `procrastinator.pid` is appended.
|
533
519
|
|
534
|
-
|
535
|
-
|
520
|
+
```ruby
|
521
|
+
# Rakefile
|
536
522
|
|
537
|
-
#
|
538
|
-
|
523
|
+
# This would define a process titled my-app
|
524
|
+
Procrastinator::Rake::DaemonTasks.define(pid_path: 'my-app.pid') do
|
525
|
+
# ... build a Procrastinator instance here ...
|
526
|
+
end
|
539
527
|
|
540
|
-
#
|
541
|
-
|
542
|
-
#
|
543
|
-
task_store.reconnect_mysql
|
528
|
+
# equivalent to ./pids/procrastinator.pid
|
529
|
+
Procrastinator::Rake::DaemonTasks.define(pid_path: 'pids') do
|
530
|
+
# ... build a Procrastinator instance here ...
|
544
531
|
end
|
545
532
|
```
|
546
533
|
|
547
|
-
|
548
|
-
combinations.
|
549
|
-
|
550
|
-
Expected use is a single process with one thread per queue. More complex use is possible but Procrastinator can't
|
551
|
-
guarantee concurrency in your Task Store.
|
534
|
+
Either run the generated Rake tasks in a terminal or with your daemon monitoring tool of choice (eg. Monit, systemd)
|
552
535
|
|
553
|
-
|
536
|
+
```bash
|
537
|
+
# In terminal
|
538
|
+
bundle exec rake procrastinator:start
|
539
|
+
bundle exec rake procrastinator:status
|
540
|
+
bundle exec rake procrastinator:restart
|
541
|
+
bundle exec rake procrastinator:stop
|
542
|
+
```
|
554
543
|
|
555
|
-
|
556
|
-
|
544
|
+
There are instructions for using Procrastinator with Monit in
|
545
|
+
the [github wiki](https://github.com/TenjinInc/procrastinator/wiki/Monit-Configuration).
|
557
546
|
|
558
547
|
## Similar Tools
|
559
548
|
|
@@ -597,6 +586,8 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
|
|
597
586
|
version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
|
598
587
|
push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
599
588
|
|
589
|
+
Docs are generated using YARD. Run `rake yard` to generate a local copy.
|
590
|
+
|
600
591
|
## License
|
601
592
|
|
602
593
|
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|