procrastinator 1.0.0.pre.rc4 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +285 -294
- data/RELEASE_NOTES.md +10 -5
- data/Rakefile +7 -0
- data/lib/procrastinator/config.rb +34 -3
- data/lib/procrastinator/logged_task.rb +2 -4
- data/lib/procrastinator/queue.rb +26 -2
- data/lib/procrastinator/queue_worker.rb +4 -0
- data/lib/procrastinator/rake/daemon_tasks.rb +16 -9
- data/lib/procrastinator/rspec/matchers.rb +30 -0
- data/lib/procrastinator/scheduler.rb +16 -9
- data/lib/procrastinator/task.rb +12 -0
- data/lib/procrastinator/task_meta_data.rb +12 -0
- data/lib/procrastinator/task_store/file_transaction.rb +53 -55
- data/lib/procrastinator/task_store/simple_comma_store.rb +33 -9
- data/lib/procrastinator/test/mocks.rb +9 -0
- data/lib/procrastinator/version.rb +2 -1
- data/procrastinator.gemspec +1 -0
- metadata +19 -4
data/README.md
CHANGED
@@ -1,78 +1,74 @@
|
|
1
1
|
# Procrastinator
|
2
2
|
|
3
|
-
|
4
|
-
|
5
|
-
Tasks are can be rescheduled and retried after failures, and you can use whatever storage mechanism is needed.
|
3
|
+
A storage-agnostic job queue gem in plain Ruby.
|
6
4
|
|
7
5
|
## Big Picture
|
8
6
|
|
9
|
-
|
7
|
+
Define **Task Handler** classes like this:
|
10
8
|
|
11
9
|
```ruby
|
12
10
|
# Sends a welcome email
|
13
11
|
class SendWelcomeEmail
|
12
|
+
attr_accessor :container, :logger, :scheduler
|
13
|
+
|
14
14
|
def run
|
15
15
|
# ... etc
|
16
16
|
end
|
17
17
|
end
|
18
18
|
```
|
19
19
|
|
20
|
-
|
20
|
+
Then build a task **Scheduler**:
|
21
21
|
|
22
22
|
```ruby
|
23
|
-
scheduler = Procrastinator.setup do |
|
24
|
-
|
25
|
-
|
26
|
-
|
23
|
+
scheduler = Procrastinator.setup do |config|
|
24
|
+
config.with_store some_email_task_database do
|
25
|
+
config.define_queue :welcome, SendWelcomeEmail
|
26
|
+
config.define_queue :birthday, SendBirthdayEmail, max_attempts: 3
|
27
27
|
end
|
28
28
|
|
29
|
-
|
29
|
+
config.define_queue :thumbnail, GenerateThumbnail, store: 'imgtasks.csv', timeout: 60
|
30
30
|
end
|
31
31
|
```
|
32
32
|
|
33
|
-
|
33
|
+
And **defer** tasks:
|
34
34
|
|
35
35
|
```ruby
|
36
|
-
scheduler.
|
36
|
+
scheduler.defer(:welcome, data: 'elanor@example.com')
|
37
37
|
|
38
|
-
scheduler.
|
38
|
+
scheduler.defer(:thumbnail, data: {file: 'forcett.png', width: 100, height: 150})
|
39
39
|
|
40
|
-
scheduler.
|
41
|
-
```
|
42
|
-
|
43
|
-
And tell a process to actually do them:
|
44
|
-
|
45
|
-
```ruby
|
46
|
-
# Starts a daemon with a thread for each queue.
|
47
|
-
# Other options are direct control (for testing) or threaded (for terminals)
|
48
|
-
scheduler.work.daemonized!
|
40
|
+
scheduler.defer(:birthday, run_at: Time.now + 3600, data: {user_id: 5})
|
49
41
|
```
|
50
42
|
|
51
43
|
## Contents
|
52
44
|
|
53
|
-
|
54
|
-
|
55
|
-
|
56
|
-
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
- [
|
63
|
-
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
69
|
-
|
70
|
-
|
71
|
-
|
72
|
-
|
73
|
-
|
74
|
-
|
75
|
-
|
45
|
+
* [Installation](#installation)
|
46
|
+
* [Task Handlers](#task-handlers)
|
47
|
+
+ [Attribute Accessors](#attribute-accessors)
|
48
|
+
+ [Errors & Logging](#errors---logging)
|
49
|
+
* [Configuration](#configuration)
|
50
|
+
+ [Defining Queues](#defining-queues)
|
51
|
+
+ [Task Store](#task-store)
|
52
|
+
- [Data Fields](#data-fields)
|
53
|
+
- [CSV Task Store](#csv-task-store)
|
54
|
+
- [Shared Task Stores](#shared-task-stores)
|
55
|
+
+ [Task Container](#task-container)
|
56
|
+
* [Deferring Tasks](#deferring-tasks)
|
57
|
+
+ [Timing](#timing)
|
58
|
+
+ [Rescheduling Existing Tasks](#rescheduling-existing-tasks)
|
59
|
+
+ [Retries](#retries)
|
60
|
+
+ [Cancelling](#cancelling)
|
61
|
+
* [Running Tasks](#running-tasks)
|
62
|
+
+ [In Testing](#in-testing)
|
63
|
+
- [RSpec Matchers](#rspec-matchers)
|
64
|
+
+ [In Production](#in-production)
|
65
|
+
* [Similar Tools](#similar-tools)
|
66
|
+
+ [Linux etc: Cron and At](#linux-etc--cron-and-at)
|
67
|
+
+ [Gem: Resque](#gem--resque)
|
68
|
+
+ [Gem: Rails ActiveJob / DelayedJob](#gem--rails-activejob---delayedjob)
|
69
|
+
* [Contributing](#contributing)
|
70
|
+
+ [Developers](#developers)
|
71
|
+
* [License](#license)
|
76
72
|
|
77
73
|
<!-- ToC generated with http://ecotrust-canada.github.io/markdown-toc/ -->
|
78
74
|
|
@@ -84,13 +80,118 @@ Add this line to your application's Gemfile:
|
|
84
80
|
gem 'procrastinator'
|
85
81
|
```
|
86
82
|
|
87
|
-
And then run:
|
83
|
+
And then run in a terminal:
|
88
84
|
|
89
85
|
bundle install
|
90
86
|
|
91
|
-
##
|
87
|
+
## Task Handlers
|
88
|
+
|
89
|
+
Task Handlers are what actually get run on the task queue. They'll look like this:
|
90
|
+
|
91
|
+
```ruby
|
92
|
+
# This is an example task handler
|
93
|
+
class MyTask
|
94
|
+
# These attributes will be assigned by Procrastinator when the task is run.
|
95
|
+
# :data is optional
|
96
|
+
attr_accessor :container, :logger, :scheduler, :data
|
97
|
+
|
98
|
+
# Performs the core work of the task.
|
99
|
+
def run
|
100
|
+
# ... perform your task ...
|
101
|
+
end
|
102
|
+
|
103
|
+
# ==================================
|
104
|
+
# OPTIONAL HOOKS
|
105
|
+
# ==================================
|
106
|
+
#
|
107
|
+
# You can always omit any of the methods below. Only #run is mandatory.
|
108
|
+
##
|
109
|
+
|
110
|
+
# Called after the task has completed successfully.
|
111
|
+
#
|
112
|
+
# @param run_result [Object] The result of #run.
|
113
|
+
def success(run_result)
|
114
|
+
# ...
|
115
|
+
end
|
116
|
+
|
117
|
+
# Called after #run raises any StandardError (or subclass).
|
118
|
+
#
|
119
|
+
# @param error [StandardError] Error raised by #run
|
120
|
+
def fail(error)
|
121
|
+
# ...
|
122
|
+
end
|
123
|
+
|
124
|
+
# Called after a permanent failure, either because:
|
125
|
+
# 1. the current time is after the task's expire_at time.
|
126
|
+
# 2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`.
|
127
|
+
#
|
128
|
+
# If #final_fail is executed, then #fail will not.
|
129
|
+
#
|
130
|
+
# @param error [StandardError] Error raised by #run
|
131
|
+
def final_fail(error)
|
132
|
+
# ...
|
133
|
+
end
|
134
|
+
end
|
135
|
+
```
|
136
|
+
|
137
|
+
### Attribute Accessors
|
138
|
+
|
139
|
+
Task Handlers have attributes that are set after the Handler is created. The attributes are enforced early on to prevent
|
140
|
+
the tasks from referencing unknown variables at whatever time they are run - if they're missing, you'll get
|
141
|
+
a `MalformedTaskError`.
|
142
|
+
|
143
|
+
| Attribute | Required | Description |
|
144
|
+
|------------|----------|-------------|
|
145
|
+
|`:container`| Yes | Container declared in `#setup` from the currently running instance |
|
146
|
+
|`:logger` | Yes | Logger object for the Queue |
|
147
|
+
|`:scheduler`| Yes | A scheduler object that you can use to schedule new tasks (eg. with `#defer`)|
|
148
|
+
|`:data` | No | Data provided to `#defer`. Calls to `#defer` will error if they do not provide data when expected and vice-versa. |
|
149
|
+
|
150
|
+
### Errors & Logging
|
151
|
+
|
152
|
+
Errors that trigger `#fail` or `#final_fail` are saved to the task storage under keywords `last_error` and
|
153
|
+
`last_fail_at`.
|
154
|
+
|
155
|
+
Each queue worker also keeps a logfile log using the Ruby
|
156
|
+
[Logger class](https://ruby-doc.org/stdlib-2.7.1/libdoc/logger/rdoc/Logger.html). Log files are named after the queue (
|
157
|
+
eg. `log/welcome-queue-worker.log`).
|
158
|
+
|
159
|
+
```ruby
|
160
|
+
scheduler = Procrastinator.setup do |config|
|
161
|
+
# you can set custom log location and level:
|
162
|
+
config.log_with(directory: '/var/log/myapp/', level: Logger::DEBUG)
|
163
|
+
|
164
|
+
# you can also set the log rotation age or size (see Logger docs for details)
|
165
|
+
config.log_with(shift: 1024, age: 5)
|
166
|
+
|
167
|
+
# use a falsey log level to disable logging entirely:
|
168
|
+
config.log_with(level: false)
|
169
|
+
end
|
170
|
+
```
|
171
|
+
|
172
|
+
The logger can be accessed in your tasks by calling `logger` or `@logger`.
|
173
|
+
|
174
|
+
```ruby
|
175
|
+
# Example handler with logging
|
176
|
+
class MyTask
|
177
|
+
attr_accessor :container, :logger, :scheduler
|
178
|
+
|
179
|
+
def run
|
180
|
+
logger.info('This task got run. Hooray!')
|
181
|
+
end
|
182
|
+
end
|
183
|
+
```
|
184
|
+
|
185
|
+
Some events are always logged by default:
|
92
186
|
|
93
|
-
|
187
|
+
|event |level |
|
188
|
+
|--------------------|-------|
|
189
|
+
|Task completed | INFO |
|
190
|
+
|Task cailure | ERROR |
|
191
|
+
|
192
|
+
## Configuration
|
193
|
+
|
194
|
+
`Procrastinator.setup` allows you to define which queues are available and other general settings.
|
94
195
|
|
95
196
|
```ruby
|
96
197
|
require 'procrastinator'
|
@@ -100,14 +201,11 @@ scheduler = Procrastinator.setup do |config|
|
|
100
201
|
end
|
101
202
|
```
|
102
203
|
|
103
|
-
It then returns a
|
104
|
-
|
105
|
-
* See [Scheduling Tasks](#scheduling-tasks)
|
106
|
-
* See [Start Working](#start-working)
|
204
|
+
It then returns a **Task Scheduler** that your code can use to defer tasks.
|
107
205
|
|
108
206
|
### Defining Queues
|
109
207
|
|
110
|
-
In setup, call `#define_queue` with a symbol name and
|
208
|
+
In setup, call `#define_queue` with a symbol name and that queue's Task Handler class:
|
111
209
|
|
112
210
|
```ruby
|
113
211
|
# You must provide a queue name and the class that handles those jobs
|
@@ -132,7 +230,7 @@ Description of keyword options:
|
|
132
230
|
### Task Store
|
133
231
|
|
134
232
|
A task store is a [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) pattern object that knows how to read and
|
135
|
-
write tasks in your data storage (eg. database, CSV file, etc).
|
233
|
+
write tasks in your data storage (eg. database, HTTP API, CSV file, microdot, etc).
|
136
234
|
|
137
235
|
```ruby
|
138
236
|
task_store = ReminderStore.new # eg. some SQL task storage class you wrote
|
@@ -151,17 +249,17 @@ A task store is required to implement *all* of the following methods or else it
|
|
151
249
|
1. `#read(attributes)`
|
152
250
|
|
153
251
|
Returns a list of hashes from your datastore that match the given attributes hash. The search attributes will be in
|
154
|
-
their final form (eg. `:data` will already be serialized). Each hash must contain the properties listed
|
155
|
-
|
252
|
+
their final form (eg. `:data` will already be serialized). Each hash must contain the properties listed in
|
253
|
+
the [Data Fields](#data-fields) table.
|
156
254
|
|
157
255
|
2. `#create(queue:, run_at:, initial_run_at:, expire_at:, data:)`
|
158
256
|
|
159
|
-
Saves a task in your storage. Receives a hash with [
|
257
|
+
Saves a task in your storage. Receives a hash with [Data Fields](#data-fields) keys:
|
160
258
|
`:queue`, `:run_at`, `:initial_run_at`, `:expire_at`, and `:data`.
|
161
259
|
|
162
260
|
3. `#update(id, new_data)`
|
163
261
|
|
164
|
-
Saves the provided full [
|
262
|
+
Saves the provided full [Data Fields](#data-fields) hash to your datastore.
|
165
263
|
|
166
264
|
4. `#delete(id)`
|
167
265
|
|
@@ -177,32 +275,32 @@ _Warning_: Task stores shared between queues **must** be thread-safe if using th
|
|
177
275
|
These are the data fields for each individual scheduled task. When using the built-in task store, these are the field
|
178
276
|
names. If you have a database, use this to inform your table schema.
|
179
277
|
|
180
|
-
| Hash Key | Type
|
181
|
-
|
182
|
-
| `:id` |
|
183
|
-
| `:queue` | symbol
|
184
|
-
| `:run_at` |
|
185
|
-
| `:initial_run_at` |
|
186
|
-
| `:expire_at` |
|
187
|
-
| `:attempts` |
|
188
|
-
| `:last_fail_at` |
|
189
|
-
| `:last_error` | string
|
190
|
-
| `:data` |
|
191
|
-
|
192
|
-
|
193
|
-
|
194
|
-
|
195
|
-
|
278
|
+
| Hash Key | Type | Description |
|
279
|
+
|-------------------|----------| ------------------------------------------------------------------------|
|
280
|
+
| `:id` | integer | Unique identifier for this exact task |
|
281
|
+
| `:queue` | symbol | Name of the queue the task is inside |
|
282
|
+
| `:run_at` | datetime | Time to attempt running the task next. Updated for retries¹ |
|
283
|
+
| `:initial_run_at` | datetime | Original `run_at` value. Reset if `#reschedule` is called. |
|
284
|
+
| `:expire_at` | datetime | Time to permanently fail the task because it is too late to be useful |
|
285
|
+
| `:attempts` | integer | Number of times the task has tried to run |
|
286
|
+
| `:last_fail_at` | datetime | Time of the most recent failure |
|
287
|
+
| `:last_error` | string | Error message + backtrace of the most recent failure. May be very long. |
|
288
|
+
| `:data` | JSON | Data to be provided to the task handler, serialized² to JSON. |
|
289
|
+
|
290
|
+
¹ `nil` indicates that it is permanently failed and will never run, either due to expiry or too many attempts.
|
291
|
+
|
292
|
+
² Serialized using `JSON.dump` and `JSON.parse` with **symbolized keys**. It is strongly recommended to only supply
|
293
|
+
simple data types (eg. id numbers) to reduce storage space, eliminate redundancy, and reduce the chance of a
|
196
294
|
serialization error.
|
197
295
|
|
198
|
-
Times are all handled as
|
296
|
+
Times are all handled as Ruby stdlib Time objects.
|
199
297
|
|
200
|
-
####
|
298
|
+
#### CSV Task Store
|
201
299
|
|
202
300
|
Specifying no storage will cause Procrastinator to save tasks using the very basic built-in CSV storage. It is not
|
203
301
|
designed for heavy loads, so you should replace it in a production environment.
|
204
302
|
|
205
|
-
The file path is defined in `Procrastinator::Store::SimpleCommaStore::DEFAULT_FILE`.
|
303
|
+
The default file path is defined in `Procrastinator::Store::SimpleCommaStore::DEFAULT_FILE`.
|
206
304
|
|
207
305
|
```ruby
|
208
306
|
Procrastinator.setup do |config|
|
@@ -213,7 +311,7 @@ end
|
|
213
311
|
|
214
312
|
#### Shared Task Stores
|
215
313
|
|
216
|
-
When there are tasks use the same storage, you can wrap them in a `with_store` block.
|
314
|
+
When there are tasks that use the same storage, you can wrap them in a `with_store` block.
|
217
315
|
|
218
316
|
```ruby
|
219
317
|
email_task_store = EmailTaskStore.new # eg. some SQL task storage class you wrote
|
@@ -232,20 +330,19 @@ end
|
|
232
330
|
|
233
331
|
### Task Container
|
234
332
|
|
235
|
-
Whatever is given to `#provide_container` will available to
|
236
|
-
|
237
|
-
This can be useful for things like app containers, but you can use it for whatever you like.
|
333
|
+
Whatever is given to `#provide_container` will be available to Task Handlers via the `:container` attribute and it is
|
334
|
+
intended for dependency injection.
|
238
335
|
|
239
336
|
```ruby
|
240
|
-
Procrastinator.setup do |
|
241
|
-
|
337
|
+
Procrastinator.setup do |config|
|
338
|
+
config.provide_container lunch: 'Lasagna'
|
242
339
|
|
243
340
|
# .. other setup stuff ...
|
244
341
|
end
|
245
342
|
|
246
343
|
# ... and in your task ...
|
247
344
|
class LunchTask
|
248
|
-
attr_accessor :
|
345
|
+
attr_accessor :container, :logger, :scheduler
|
249
346
|
|
250
347
|
def run
|
251
348
|
logger.info("Today's Lunch is: #{ container[:lunch] }")
|
@@ -253,243 +350,121 @@ class LunchTask
|
|
253
350
|
end
|
254
351
|
```
|
255
352
|
|
256
|
-
## Tasks
|
257
|
-
|
258
|
-
Your task class is what actually gets run on the task queue. They'll look like this:
|
259
|
-
|
260
|
-
```ruby
|
261
|
-
|
262
|
-
class MyTask
|
263
|
-
# These attributes will be assigned by Procrastinator when the task is run.
|
264
|
-
attr_accessor :logger, :scheduler, :container, :data
|
265
|
-
|
266
|
-
# Performs the core work of the task.
|
267
|
-
def run
|
268
|
-
# ... perform your task ...
|
269
|
-
end
|
270
|
-
|
271
|
-
# ========================================
|
272
|
-
# OPTIONAL HOOKS
|
273
|
-
#
|
274
|
-
# You can always omit any of the methods
|
275
|
-
# below. Only #run is mandatory.
|
276
|
-
#
|
277
|
-
# ========================================
|
278
|
-
|
279
|
-
# Called after the task has completed successfully.
|
280
|
-
# Receives the result of #run.
|
281
|
-
def success(run_result)
|
282
|
-
# ...
|
283
|
-
end
|
284
|
-
|
285
|
-
# Called after #run raises any StandardError (or subclass).
|
286
|
-
# Receives the raised error.
|
287
|
-
def fail(error)
|
288
|
-
# ...
|
289
|
-
end
|
290
|
-
|
291
|
-
# Called after either is true:
|
292
|
-
# 1. the time reported by Time.now is past the task's expire_at time.
|
293
|
-
# 2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`.
|
294
|
-
# In this case, #fail will not be executed, only #final_fail.
|
295
|
-
#
|
296
|
-
# When called, the task will be marked to never be run again.
|
297
|
-
# Receives the raised error.
|
298
|
-
def final_fail(error)
|
299
|
-
# ...
|
300
|
-
end
|
301
|
-
end
|
302
|
-
```
|
303
|
-
|
304
|
-
### Attribute Accessors
|
305
|
-
|
306
|
-
Tasks must provide accessors for `:logger`, `:container`, and `:scheduler`, while the `:data` accessor is semi-optional
|
307
|
-
(see below). This is to prevent the tasks from referencing unknown variables once they actually get run.
|
308
|
-
|
309
|
-
* `:data` The data you provided in the call to `#delay`. Any task with a `:data` accessor will require data be passed
|
310
|
-
to `#delay`, and vice versa. See [Task Data](#task-data) for more.
|
311
|
-
|
312
|
-
* `:container` The container you've provided in your setup. See [Task Container](#task-container) for more.
|
313
|
-
|
314
|
-
* `:logger` The queue's Logger object. See [Logging](#logging) for more.
|
315
|
-
|
316
|
-
* `:scheduler` A scheduler object that you can use to schedule new tasks (eg. with `#delay`).
|
317
|
-
|
318
|
-
### Errors & Logging
|
319
|
-
|
320
|
-
Errors that trigger `#fail` or `#final_fail` are saved to the task storage under columns `last_error` and
|
321
|
-
`last_error_at`.
|
322
|
-
|
323
|
-
Each queue worker also keeps a logfile log using the Ruby
|
324
|
-
[Logger class](https://ruby-doc.org/stdlib-2.7.1/libdoc/logger/rdoc/Logger.html). Log files are named after the queue (
|
325
|
-
eg. `log/welcome-queue-worker.log`).
|
326
|
-
|
327
|
-
```ruby
|
328
|
-
scheduler = Procrastinator.setup do |env|
|
329
|
-
# you can set custom log location and level:
|
330
|
-
env.log_with(directory: '/var/log/myapp/', level: Logger::DEBUG)
|
331
|
-
|
332
|
-
# you can also set the log rotation age or size (see Logger docs for details)
|
333
|
-
env.log_with(shift: 1024, age: 5)
|
334
|
-
|
335
|
-
# use a falsey log level to disable logging entirely:
|
336
|
-
env.log_with(level: false)
|
337
|
-
end
|
338
|
-
```
|
339
|
-
|
340
|
-
The logger can be accessed in your tasks by calling `logger` or `@logger`.
|
341
|
-
|
342
|
-
```ruby
|
343
|
-
|
344
|
-
class MyTask
|
345
|
-
attr_accessor :logger, :scheduler, :container
|
346
|
-
|
347
|
-
def run
|
348
|
-
logger.info('This task got run. Hooray!')
|
349
|
-
end
|
350
|
-
end
|
351
|
-
```
|
352
|
-
|
353
|
-
Some events are always logged by default:
|
354
|
-
|
355
|
-
|event |level |
|
356
|
-
|--------------------|-------|
|
357
|
-
|process started | INFO |
|
358
|
-
|#success called | DEBUG |
|
359
|
-
|#fail called | DEBUG |
|
360
|
-
|#final_fail called | DEBUG |
|
361
|
-
|
362
|
-
## Scheduling Tasks
|
353
|
+
## Deferring Tasks
|
363
354
|
|
364
|
-
To
|
355
|
+
To add tasks to a queue, call `#defer` on the scheduler returned by `Procrastinator.setup`:
|
365
356
|
|
366
357
|
```ruby
|
367
|
-
scheduler = Procrastinator.setup do |
|
368
|
-
|
369
|
-
|
358
|
+
scheduler = Procrastinator.setup do |config|
|
359
|
+
config.define_queue :reminder, EmailEveryone
|
360
|
+
config.define_queue :thumbnail, CreateThumbnail
|
370
361
|
end
|
371
362
|
|
372
|
-
# Provide the queue name and any data you want passed in
|
373
|
-
scheduler.
|
363
|
+
# Provide the queue name and any data you want passed in, if needed
|
364
|
+
scheduler.defer(:reminder)
|
365
|
+
scheduler.defer(:thumbnail, data: 'forcett.png')
|
374
366
|
```
|
375
367
|
|
376
368
|
If there is only one queue, you may omit the queue name:
|
377
369
|
|
378
370
|
```ruby
|
379
|
-
|
380
|
-
|
371
|
+
thumbnailer = Procrastinator.setup do |config|
|
372
|
+
config.define_queue :thumbnail, CreateThumbnail
|
381
373
|
end
|
382
374
|
|
383
|
-
|
375
|
+
thumbnailer.defer(data: 'forcett.png')
|
384
376
|
```
|
385
377
|
|
386
|
-
###
|
387
|
-
|
388
|
-
Most tasks need some additional information to complete their work, like id numbers,
|
378
|
+
### Timing
|
389
379
|
|
390
|
-
|
391
|
-
you have a database instead of passing in a complex Ruby object, pass in just the primary key and reload it in the
|
392
|
-
task's `#run`. This will require less space in your database and avoids obsolete or duplicated information.
|
380
|
+
You can specify a particular timeframe that a task may be run. The default is to run immediately and never expire.
|
393
381
|
|
394
|
-
|
382
|
+
Be aware that the task is not guaranteed to run at a precise time; the only promise is that the task won't be tried *
|
383
|
+
before* `run_at` nor *after* `expire_at`.
|
395
384
|
|
396
|
-
|
397
|
-
|
385
|
+
Tasks attempted after `expire_at` will be final-failed. Setting `expire_at` to `nil`
|
386
|
+
means it will never expire (but may still fail permanently if, say, `max_attempts` is reached).
|
398
387
|
|
399
388
|
```ruby
|
400
|
-
|
401
|
-
|
389
|
+
run_time = Time.new(2016, 9, 19)
|
390
|
+
expire_time = Time.new(2016, 9, 20)
|
402
391
|
|
403
|
-
#
|
404
|
-
scheduler.
|
405
|
-
```
|
392
|
+
# runs on or after 2016 Sept 19, never expires
|
393
|
+
scheduler.defer(:greeting, run_at: run_time, data: 'elanor@example.com')
|
406
394
|
|
407
|
-
|
408
|
-
|
409
|
-
fail permanently if, say, `max_attempts` is reached).
|
410
|
-
|
411
|
-
```ruby
|
412
|
-
# will not run at or after
|
413
|
-
scheduler.delay(:happy_birthday, expire_at: Time.new(2018, 03, 17, 12, 00, '-06:00'), data: 'contact@tenjin.ca')
|
395
|
+
# can run immediately but not after 2016 Sept 20
|
396
|
+
scheduler.defer(:greeting, expire_at: expire_time, data: 'mendoza@example.com')
|
414
397
|
|
415
|
-
#
|
416
|
-
scheduler.
|
398
|
+
# can run immediately but not after 2016 Sept 20
|
399
|
+
scheduler.defer(:greeting, run_at: run_time, expire_at: expire_time, data: 'tahani@example.com')
|
417
400
|
```
|
418
401
|
|
419
|
-
### Rescheduling
|
402
|
+
### Rescheduling Existing Tasks
|
420
403
|
|
421
|
-
Call `#reschedule` with the queue name and some identifying information
|
422
|
-
time.
|
404
|
+
Call `#reschedule` with the queue name and some task-identifying information and then chain `#to` with the new time.
|
423
405
|
|
424
406
|
```ruby
|
425
|
-
|
426
|
-
|
427
|
-
end
|
407
|
+
run_time = Time.new(2016, 9, 19)
|
408
|
+
expire_time = Time.new(2016, 9, 20)
|
428
409
|
|
429
|
-
scheduler.
|
410
|
+
scheduler.defer(:reminder, run_at: Time.at(0), data: 'chidi@example.com')
|
430
411
|
|
431
|
-
# we can reschedule the task
|
432
|
-
scheduler.reschedule(:reminder, data: '
|
412
|
+
# we can reschedule the task that matches this data
|
413
|
+
scheduler.reschedule(:reminder, data: 'chidi@example.com').to(run_at: run_time)
|
433
414
|
|
434
415
|
# we can also change the expiry time
|
435
|
-
scheduler.reschedule(:reminder, data: '
|
416
|
+
scheduler.reschedule(:reminder, data: 'chidi@example.com').to(expire_at: expire_time)
|
436
417
|
|
437
418
|
# or both
|
438
|
-
scheduler.reschedule(:reminder, data: '
|
439
|
-
|
419
|
+
scheduler.reschedule(:reminder, data: 'chidi@example.com').to(run_at: run_time,
|
420
|
+
expire_at: expire_time)
|
440
421
|
```
|
441
422
|
|
442
|
-
Rescheduling
|
423
|
+
Rescheduling changes the task's...
|
443
424
|
|
444
425
|
* `:run_at` and `:initial_run_at` to a new value, if provided
|
445
426
|
* `:expire_at` to a new value if provided.
|
446
427
|
* `:attempts` to `0`
|
447
428
|
* `:last_error` and `:last_error_at` to `nil`.
|
448
429
|
|
449
|
-
Rescheduling will not change `:id`, `:queue` or `:data`.
|
430
|
+
Rescheduling will not change `:id`, `:queue` or `:data`.
|
431
|
+
|
432
|
+
A `RuntimeError` is raised if the new run_at is after expire_at.
|
450
433
|
|
451
434
|
### Retries
|
452
435
|
|
453
|
-
Failed tasks
|
436
|
+
Failed tasks are automatically retried, with their `run_at` updated on an increasing delay (in seconds) according to
|
437
|
+
this formula:
|
454
438
|
|
455
|
-
> 30 +
|
439
|
+
> 30 + number_of_attempts<sup>4</sup>
|
456
440
|
|
457
441
|
Situations that call `#fail` or `#final_fail` will cause the error timestamp and reason to be stored in `:last_fail_at`
|
458
442
|
and `:last_error`.
|
459
443
|
|
460
444
|
### Cancelling
|
461
445
|
|
462
|
-
Call `#cancel` with the queue name and some identifying information to narrow the search to a single task.
|
446
|
+
Call `#cancel` with the queue name and some task-identifying information to narrow the search to a single task.
|
463
447
|
|
464
448
|
```ruby
|
465
|
-
|
466
|
-
|
467
|
-
end
|
449
|
+
run_time = Time.parse('April 1')
|
450
|
+
scheduler.defer(:reminder, run_at: run_time, data: 'derek@example.com')
|
468
451
|
|
469
|
-
|
470
|
-
|
471
|
-
|
472
|
-
scheduler.reschedule(:reminder, data: 'bob@example.com')
|
473
|
-
|
474
|
-
# or multiple attributes
|
475
|
-
scheduler.reschedule(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
|
452
|
+
# we can cancel the task made above using whatever we know about it
|
453
|
+
# An error will be raised if it matches multiple tasks or finds none
|
454
|
+
scheduler.cancel(:reminder, run_at: run_time, data: 'derek@example.com')
|
476
455
|
|
477
456
|
# you could also use the id number directly, if you have it
|
478
|
-
scheduler.
|
457
|
+
scheduler.cancel(:reminder, id: 137)
|
479
458
|
```
|
480
459
|
|
481
|
-
##
|
482
|
-
|
483
|
-
Use the scheduler object returned by setup to `#work` queues **serially**, **threaded**, or **daemonized**.
|
460
|
+
## Testing with Procrastinator
|
484
461
|
|
485
|
-
|
462
|
+
Working serially performs tasks from each queue sequentially. There is no multithreading or daemonizing.
|
486
463
|
|
487
|
-
|
488
|
-
|
489
|
-
Work serially for TDD tests or other situations you need close direct control.
|
464
|
+
Call `work` on the Scheduler with an optional list of queues to filter by.
|
490
465
|
|
491
466
|
```ruby
|
492
|
-
# work just one task
|
467
|
+
# work just one task
|
493
468
|
scheduler.work.serially
|
494
469
|
|
495
470
|
# work the first five tasks
|
@@ -499,61 +474,75 @@ scheduler.work.serially(steps: 5)
|
|
499
474
|
scheduler.work(:greeting, :reminders).serially(steps: 2)
|
500
475
|
```
|
501
476
|
|
502
|
-
###
|
503
|
-
|
504
|
-
Threaded working will spawn a worker thread per queue.
|
477
|
+
### RSpec Matchers
|
505
478
|
|
506
|
-
|
507
|
-
caveats around multithreading, so proceed with caution.
|
479
|
+
A `have_task` RSpec matcher is defined to make testing task scheduling a little easier.
|
508
480
|
|
509
481
|
```ruby
|
510
|
-
#
|
511
|
-
|
482
|
+
# Note: you must require the matcher file separately
|
483
|
+
require 'procrastinator'
|
484
|
+
require 'procrastinator/rspec/matchers'
|
512
485
|
|
513
|
-
|
514
|
-
scheduler.work.threaded(timeout: 5)
|
486
|
+
task_storage = TaskStore.new
|
515
487
|
|
516
|
-
|
517
|
-
|
488
|
+
scheduler = Procrastinator.setup do |config|
|
489
|
+
config.define_queue :welcome, SendWelcome, store: task_storage
|
490
|
+
end
|
491
|
+
|
492
|
+
scheduler.defer(data: 'tahani@example.com')
|
493
|
+
|
494
|
+
expect(task_storage).to have_task(data: 'tahani@example.com')
|
518
495
|
```
|
519
496
|
|
520
|
-
|
497
|
+
## Running Tasks
|
521
498
|
|
522
|
-
|
499
|
+
When you are ready to run a Procrastinator daemon in production, you may use some provided Rake tasks.
|
523
500
|
|
524
|
-
|
525
|
-
like [Monit](https://mmonit.com/monit/). Provide a block to daemonized! to get
|
501
|
+
In your Rake file call `DaemonTasks.define` with a block that constructs a scheduler instance.
|
526
502
|
|
527
503
|
```ruby
|
528
|
-
#
|
529
|
-
|
504
|
+
# Rakefile
|
505
|
+
require 'rake'
|
506
|
+
require 'procrastinator/rake/daemon_tasks'
|
507
|
+
|
508
|
+
# Defines a set of tasks that will control a Procrastinator daemon
|
509
|
+
# Default pid_path is /tmp/procrastinator.pid
|
510
|
+
Procrastinator::Rake::DaemonTasks.define do
|
511
|
+
Procrastinator.setup do
|
512
|
+
# ... etc ...
|
513
|
+
end
|
514
|
+
end
|
515
|
+
```
|
530
516
|
|
531
|
-
|
532
|
-
|
517
|
+
You can name the daemon process by specifying the pid_path with a specific .pid file. If does not end with '.pid' it is
|
518
|
+
assumed to be a directory name, and `procrastinator.pid` is appended.
|
533
519
|
|
534
|
-
|
535
|
-
|
520
|
+
```ruby
|
521
|
+
# Rakefile
|
536
522
|
|
537
|
-
#
|
538
|
-
|
523
|
+
# This would define a process titled my-app
|
524
|
+
Procrastinator::Rake::DaemonTasks.define(pid_path: 'my-app.pid') do
|
525
|
+
# ... build a Procrastinator instance here ...
|
526
|
+
end
|
539
527
|
|
540
|
-
#
|
541
|
-
|
542
|
-
#
|
543
|
-
task_store.reconnect_mysql
|
528
|
+
# equivalent to ./pids/procrastinator.pid
|
529
|
+
Procrastinator::Rake::DaemonTasks.define(pid_path: 'pids') do
|
530
|
+
# ... build a Procrastinator instance here ...
|
544
531
|
end
|
545
532
|
```
|
546
533
|
|
547
|
-
|
548
|
-
combinations.
|
549
|
-
|
550
|
-
Expected use is a single process with one thread per queue. More complex use is possible but Procrastinator can't
|
551
|
-
guarantee concurrency in your Task Store.
|
534
|
+
Either run the generated Rake tasks in a terminal or with your daemon monitoring tool of choice (eg. Monit, systemd)
|
552
535
|
|
553
|
-
|
536
|
+
```bash
|
537
|
+
# In terminal
|
538
|
+
bundle exec rake procrastinator:start
|
539
|
+
bundle exec rake procrastinator:status
|
540
|
+
bundle exec rake procrastinator:restart
|
541
|
+
bundle exec rake procrastinator:stop
|
542
|
+
```
|
554
543
|
|
555
|
-
|
556
|
-
|
544
|
+
There are instructions for using Procrastinator with Monit in
|
545
|
+
the [github wiki](https://github.com/TenjinInc/procrastinator/wiki/Monit-Configuration).
|
557
546
|
|
558
547
|
## Similar Tools
|
559
548
|
|
@@ -597,6 +586,8 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
|
|
597
586
|
version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
|
598
587
|
push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
599
588
|
|
589
|
+
Docs are generated using YARD. Run `rake yard` to generate a local copy.
|
590
|
+
|
600
591
|
## License
|
601
592
|
|
602
593
|
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).
|