procrastinator 1.0.0.pre.rc4 → 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
data/README.md CHANGED
@@ -1,78 +1,74 @@
1
1
  # Procrastinator
2
2
 
3
- Procrastinator is a pure Ruby job scheduling gem. Put off tasks until later or for another process to handle.
4
-
5
- Tasks are can be rescheduled and retried after failures, and you can use whatever storage mechanism is needed.
3
+ A storage-agnostic job queue gem in plain Ruby.
6
4
 
7
5
  ## Big Picture
8
6
 
9
- If you have tasks like this:
7
+ Define **Task Handler** classes like this:
10
8
 
11
9
  ```ruby
12
10
  # Sends a welcome email
13
11
  class SendWelcomeEmail
12
+ attr_accessor :container, :logger, :scheduler
13
+
14
14
  def run
15
15
  # ... etc
16
16
  end
17
17
  end
18
18
  ```
19
19
 
20
- Setup a procrastination environment like:
20
+ Then build a task **Scheduler**:
21
21
 
22
22
  ```ruby
23
- scheduler = Procrastinator.setup do |env|
24
- env.with_store some_email_task_database do
25
- env.define_queue :greeting, SendWelcomeEmail
26
- env.define_queue :birthday, SendBirthdayEmail, max_attempts: 3
23
+ scheduler = Procrastinator.setup do |config|
24
+ config.with_store some_email_task_database do
25
+ config.define_queue :welcome, SendWelcomeEmail
26
+ config.define_queue :birthday, SendBirthdayEmail, max_attempts: 3
27
27
  end
28
28
 
29
- env.define_queue :thumbnail, GenerateThumbnail, store: 'imgtasks.csv', timeout: 60
29
+ config.define_queue :thumbnail, GenerateThumbnail, store: 'imgtasks.csv', timeout: 60
30
30
  end
31
31
  ```
32
32
 
33
- Put jobs off until later:
33
+ And **defer** tasks:
34
34
 
35
35
  ```ruby
36
- scheduler.delay(:greeting, data: 'bob@example.com')
36
+ scheduler.defer(:welcome, data: 'elanor@example.com')
37
37
 
38
- scheduler.delay(:thumbnail, data: {file: 'full_image.png', width: 100, height: 100})
38
+ scheduler.defer(:thumbnail, data: {file: 'forcett.png', width: 100, height: 150})
39
39
 
40
- scheduler.delay(:send_birthday_email, run_at: Time.now + 3600, data: {user_id: 5})
41
- ```
42
-
43
- And tell a process to actually do them:
44
-
45
- ```ruby
46
- # Starts a daemon with a thread for each queue.
47
- # Other options are direct control (for testing) or threaded (for terminals)
48
- scheduler.work.daemonized!
40
+ scheduler.defer(:birthday, run_at: Time.now + 3600, data: {user_id: 5})
49
41
  ```
50
42
 
51
43
  ## Contents
52
44
 
53
- - [Installation](#installation)
54
- - [Setup](#setup)
55
- * [Defining Queues](#defining-queues)
56
- * [Task Store](#task-store)
57
- + [Data Fields](#data-fields)
58
- * [Task Container](#task-container)
59
- - [Tasks](#tasks)
60
- * [Accessing Task Attributes](#accessing-task-attributes)
61
- * [Errors & Logging](#errors-logging)
62
- - [Scheduling Tasks](#scheduling-tasks)
63
- * [Providing Data](#providing-data)
64
- * [Scheduling](#scheduling)
65
- * [Rescheduling](#rescheduling)
66
- * [Retries](#retries)
67
- * [Cancelling](#cancelling)
68
- - [Working on Tasks](#working-on-tasks)
69
- * [Stepwise Working](#stepwise-working)
70
- * [Threaded Working](#threaded-working)
71
- * [Daemonized Working](#daemonized-working)
72
- + [PID Files](#pid-files)
73
- - [Contributing](#contributing)
74
- * [Developers](#developers)
75
- - [License](#license)
45
+ * [Installation](#installation)
46
+ * [Task Handlers](#task-handlers)
47
+ + [Attribute Accessors](#attribute-accessors)
48
+ + [Errors & Logging](#errors---logging)
49
+ * [Configuration](#configuration)
50
+ + [Defining Queues](#defining-queues)
51
+ + [Task Store](#task-store)
52
+ - [Data Fields](#data-fields)
53
+ - [CSV Task Store](#csv-task-store)
54
+ - [Shared Task Stores](#shared-task-stores)
55
+ + [Task Container](#task-container)
56
+ * [Deferring Tasks](#deferring-tasks)
57
+ + [Timing](#timing)
58
+ + [Rescheduling Existing Tasks](#rescheduling-existing-tasks)
59
+ + [Retries](#retries)
60
+ + [Cancelling](#cancelling)
61
+ * [Running Tasks](#running-tasks)
62
+ + [In Testing](#in-testing)
63
+ - [RSpec Matchers](#rspec-matchers)
64
+ + [In Production](#in-production)
65
+ * [Similar Tools](#similar-tools)
66
+ + [Linux etc: Cron and At](#linux-etc--cron-and-at)
67
+ + [Gem: Resque](#gem--resque)
68
+ + [Gem: Rails ActiveJob / DelayedJob](#gem--rails-activejob---delayedjob)
69
+ * [Contributing](#contributing)
70
+ + [Developers](#developers)
71
+ * [License](#license)
76
72
 
77
73
  <!-- ToC generated with http://ecotrust-canada.github.io/markdown-toc/ -->
78
74
 
@@ -84,13 +80,118 @@ Add this line to your application's Gemfile:
84
80
  gem 'procrastinator'
85
81
  ```
86
82
 
87
- And then run:
83
+ And then run in a terminal:
88
84
 
89
85
  bundle install
90
86
 
91
- ## Setup
87
+ ## Task Handlers
88
+
89
+ Task Handlers are what actually get run on the task queue. They'll look like this:
90
+
91
+ ```ruby
92
+ # This is an example task handler
93
+ class MyTask
94
+ # These attributes will be assigned by Procrastinator when the task is run.
95
+ # :data is optional
96
+ attr_accessor :container, :logger, :scheduler, :data
97
+
98
+ # Performs the core work of the task.
99
+ def run
100
+ # ... perform your task ...
101
+ end
102
+
103
+ # ==================================
104
+ # OPTIONAL HOOKS
105
+ # ==================================
106
+ #
107
+ # You can always omit any of the methods below. Only #run is mandatory.
108
+ ##
109
+
110
+ # Called after the task has completed successfully.
111
+ #
112
+ # @param run_result [Object] The result of #run.
113
+ def success(run_result)
114
+ # ...
115
+ end
116
+
117
+ # Called after #run raises any StandardError (or subclass).
118
+ #
119
+ # @param error [StandardError] Error raised by #run
120
+ def fail(error)
121
+ # ...
122
+ end
123
+
124
+ # Called after a permanent failure, either because:
125
+ # 1. the current time is after the task's expire_at time.
126
+ # 2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`.
127
+ #
128
+ # If #final_fail is executed, then #fail will not.
129
+ #
130
+ # @param error [StandardError] Error raised by #run
131
+ def final_fail(error)
132
+ # ...
133
+ end
134
+ end
135
+ ```
136
+
137
+ ### Attribute Accessors
138
+
139
+ Task Handlers have attributes that are set after the Handler is created. The attributes are enforced early on to prevent
140
+ the tasks from referencing unknown variables at whatever time they are run - if they're missing, you'll get
141
+ a `MalformedTaskError`.
142
+
143
+ | Attribute | Required | Description |
144
+ |------------|----------|-------------|
145
+ |`:container`| Yes | Container declared in `#setup` from the currently running instance |
146
+ |`:logger` | Yes | Logger object for the Queue |
147
+ |`:scheduler`| Yes | A scheduler object that you can use to schedule new tasks (eg. with `#defer`)|
148
+ |`:data` | No | Data provided to `#defer`. Calls to `#defer` will error if they do not provide data when expected and vice-versa. |
149
+
150
+ ### Errors & Logging
151
+
152
+ Errors that trigger `#fail` or `#final_fail` are saved to the task storage under keywords `last_error` and
153
+ `last_fail_at`.
154
+
155
+ Each queue worker also keeps a logfile log using the Ruby
156
+ [Logger class](https://ruby-doc.org/stdlib-2.7.1/libdoc/logger/rdoc/Logger.html). Log files are named after the queue (
157
+ eg. `log/welcome-queue-worker.log`).
158
+
159
+ ```ruby
160
+ scheduler = Procrastinator.setup do |config|
161
+ # you can set custom log location and level:
162
+ config.log_with(directory: '/var/log/myapp/', level: Logger::DEBUG)
163
+
164
+ # you can also set the log rotation age or size (see Logger docs for details)
165
+ config.log_with(shift: 1024, age: 5)
166
+
167
+ # use a falsey log level to disable logging entirely:
168
+ config.log_with(level: false)
169
+ end
170
+ ```
171
+
172
+ The logger can be accessed in your tasks by calling `logger` or `@logger`.
173
+
174
+ ```ruby
175
+ # Example handler with logging
176
+ class MyTask
177
+ attr_accessor :container, :logger, :scheduler
178
+
179
+ def run
180
+ logger.info('This task got run. Hooray!')
181
+ end
182
+ end
183
+ ```
184
+
185
+ Some events are always logged by default:
92
186
 
93
- `Procrastinator.setup` allows you to define which queues are available and other settings.
187
+ |event |level |
188
+ |--------------------|-------|
189
+ |Task completed | INFO |
190
+ |Task cailure | ERROR |
191
+
192
+ ## Configuration
193
+
194
+ `Procrastinator.setup` allows you to define which queues are available and other general settings.
94
195
 
95
196
  ```ruby
96
197
  require 'procrastinator'
@@ -100,14 +201,11 @@ scheduler = Procrastinator.setup do |config|
100
201
  end
101
202
  ```
102
203
 
103
- It then returns a `Scheduler` that your code can use to schedule tasks or tell to start working.
104
-
105
- * See [Scheduling Tasks](#scheduling-tasks)
106
- * See [Start Working](#start-working)
204
+ It then returns a **Task Scheduler** that your code can use to defer tasks.
107
205
 
108
206
  ### Defining Queues
109
207
 
110
- In setup, call `#define_queue` with a symbol name and the class that performs those jobs:
208
+ In setup, call `#define_queue` with a symbol name and that queue's Task Handler class:
111
209
 
112
210
  ```ruby
113
211
  # You must provide a queue name and the class that handles those jobs
@@ -132,7 +230,7 @@ Description of keyword options:
132
230
  ### Task Store
133
231
 
134
232
  A task store is a [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) pattern object that knows how to read and
135
- write tasks in your data storage (eg. database, CSV file, etc).
233
+ write tasks in your data storage (eg. database, HTTP API, CSV file, microdot, etc).
136
234
 
137
235
  ```ruby
138
236
  task_store = ReminderStore.new # eg. some SQL task storage class you wrote
@@ -151,17 +249,17 @@ A task store is required to implement *all* of the following methods or else it
151
249
  1. `#read(attributes)`
152
250
 
153
251
  Returns a list of hashes from your datastore that match the given attributes hash. The search attributes will be in
154
- their final form (eg. `:data` will already be serialized). Each hash must contain the properties listed
155
- in [Task Data](#task-data) below.
252
+ their final form (eg. `:data` will already be serialized). Each hash must contain the properties listed in
253
+ the [Data Fields](#data-fields) table.
156
254
 
157
255
  2. `#create(queue:, run_at:, initial_run_at:, expire_at:, data:)`
158
256
 
159
- Saves a task in your storage. Receives a hash with [Task Data](#task-data) keys:
257
+ Saves a task in your storage. Receives a hash with [Data Fields](#data-fields) keys:
160
258
  `:queue`, `:run_at`, `:initial_run_at`, `:expire_at`, and `:data`.
161
259
 
162
260
  3. `#update(id, new_data)`
163
261
 
164
- Saves the provided full [Task Data](#task-data) hash to your datastore.
262
+ Saves the provided full [Data Fields](#data-fields) hash to your datastore.
165
263
 
166
264
  4. `#delete(id)`
167
265
 
@@ -177,32 +275,32 @@ _Warning_: Task stores shared between queues **must** be thread-safe if using th
177
275
  These are the data fields for each individual scheduled task. When using the built-in task store, these are the field
178
276
  names. If you have a database, use this to inform your table schema.
179
277
 
180
- | Hash Key | Type | Description |
181
- |-------------------|---------| ----------------------------------------------------------------------------------------|
182
- | `:id` | int | Unique identifier for this exact task |
183
- | `:queue` | symbol | Name of the queue the task is inside |
184
- | `:run_at` | iso8601 | Time to attempt running the task next. ¹ |
185
- | `:initial_run_at` | iso8601 | Originally requested run_at. Reset when rescheduled. |
186
- | `:expire_at` | iso8601 | Time to permanently fail the task because it is too late to be useful |
187
- | `:attempts` | int | Number of times the task has tried to run; this should only be > 0 if the task fails |
188
- | `:last_fail_at` | iso8601 | Unix timestamp of when the most recent failure happened |
189
- | `:last_error` | string | Error message + bracktrace of the most recent failure. May be very long. |
190
- | `:data` | string | Serialized data accessible in the task instance. |
191
-
192
- > ¹ `nil` indicates that it is permanently failed and will never run, either due to expiry or too many attempts.
193
-
194
- Data is serialized using `JSON.dump` and `JSON.parse` with **symbolized keys**. It is strongly recommended to only
195
- supply simple data types (eg. id numbers) to reduce storage space, eliminate redundancy, and reduce the chance of a
278
+ | Hash Key | Type | Description |
279
+ |-------------------|----------| ------------------------------------------------------------------------|
280
+ | `:id` | integer | Unique identifier for this exact task |
281
+ | `:queue` | symbol | Name of the queue the task is inside |
282
+ | `:run_at` | datetime | Time to attempt running the task next. Updated for retries¹ |
283
+ | `:initial_run_at` | datetime | Original `run_at` value. Reset if `#reschedule` is called. |
284
+ | `:expire_at` | datetime | Time to permanently fail the task because it is too late to be useful |
285
+ | `:attempts` | integer | Number of times the task has tried to run |
286
+ | `:last_fail_at` | datetime | Time of the most recent failure |
287
+ | `:last_error` | string | Error message + backtrace of the most recent failure. May be very long. |
288
+ | `:data` | JSON | Data to be provided to the task handler, serialized² to JSON. |
289
+
290
+ ¹ `nil` indicates that it is permanently failed and will never run, either due to expiry or too many attempts.
291
+
292
+ ² Serialized using `JSON.dump` and `JSON.parse` with **symbolized keys**. It is strongly recommended to only supply
293
+ simple data types (eg. id numbers) to reduce storage space, eliminate redundancy, and reduce the chance of a
196
294
  serialization error.
197
295
 
198
- Times are all handled as [ISO8601](https://en.wikipedia.org/wiki/ISO_8601) formatted strings.
296
+ Times are all handled as Ruby stdlib Time objects.
199
297
 
200
- #### Default Task Store
298
+ #### CSV Task Store
201
299
 
202
300
  Specifying no storage will cause Procrastinator to save tasks using the very basic built-in CSV storage. It is not
203
301
  designed for heavy loads, so you should replace it in a production environment.
204
302
 
205
- The file path is defined in `Procrastinator::Store::SimpleCommaStore::DEFAULT_FILE`.
303
+ The default file path is defined in `Procrastinator::Store::SimpleCommaStore::DEFAULT_FILE`.
206
304
 
207
305
  ```ruby
208
306
  Procrastinator.setup do |config|
@@ -213,7 +311,7 @@ end
213
311
 
214
312
  #### Shared Task Stores
215
313
 
216
- When there are tasks use the same storage, you can wrap them in a `with_store` block.
314
+ When there are tasks that use the same storage, you can wrap them in a `with_store` block.
217
315
 
218
316
  ```ruby
219
317
  email_task_store = EmailTaskStore.new # eg. some SQL task storage class you wrote
@@ -232,20 +330,19 @@ end
232
330
 
233
331
  ### Task Container
234
332
 
235
- Whatever is given to `#provide_container` will available to Tasks through the task attribute `:container`.
236
-
237
- This can be useful for things like app containers, but you can use it for whatever you like.
333
+ Whatever is given to `#provide_container` will be available to Task Handlers via the `:container` attribute and it is
334
+ intended for dependency injection.
238
335
 
239
336
  ```ruby
240
- Procrastinator.setup do |env|
241
- env.provide_container lunch: 'Lasagna'
337
+ Procrastinator.setup do |config|
338
+ config.provide_container lunch: 'Lasagna'
242
339
 
243
340
  # .. other setup stuff ...
244
341
  end
245
342
 
246
343
  # ... and in your task ...
247
344
  class LunchTask
248
- attr_accessor :logger, :scheduler, :container
345
+ attr_accessor :container, :logger, :scheduler
249
346
 
250
347
  def run
251
348
  logger.info("Today's Lunch is: #{ container[:lunch] }")
@@ -253,243 +350,121 @@ class LunchTask
253
350
  end
254
351
  ```
255
352
 
256
- ## Tasks
257
-
258
- Your task class is what actually gets run on the task queue. They'll look like this:
259
-
260
- ```ruby
261
-
262
- class MyTask
263
- # These attributes will be assigned by Procrastinator when the task is run.
264
- attr_accessor :logger, :scheduler, :container, :data
265
-
266
- # Performs the core work of the task.
267
- def run
268
- # ... perform your task ...
269
- end
270
-
271
- # ========================================
272
- # OPTIONAL HOOKS
273
- #
274
- # You can always omit any of the methods
275
- # below. Only #run is mandatory.
276
- #
277
- # ========================================
278
-
279
- # Called after the task has completed successfully.
280
- # Receives the result of #run.
281
- def success(run_result)
282
- # ...
283
- end
284
-
285
- # Called after #run raises any StandardError (or subclass).
286
- # Receives the raised error.
287
- def fail(error)
288
- # ...
289
- end
290
-
291
- # Called after either is true:
292
- # 1. the time reported by Time.now is past the task's expire_at time.
293
- # 2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`.
294
- # In this case, #fail will not be executed, only #final_fail.
295
- #
296
- # When called, the task will be marked to never be run again.
297
- # Receives the raised error.
298
- def final_fail(error)
299
- # ...
300
- end
301
- end
302
- ```
303
-
304
- ### Attribute Accessors
305
-
306
- Tasks must provide accessors for `:logger`, `:container`, and `:scheduler`, while the `:data` accessor is semi-optional
307
- (see below). This is to prevent the tasks from referencing unknown variables once they actually get run.
308
-
309
- * `:data` The data you provided in the call to `#delay`. Any task with a `:data` accessor will require data be passed
310
- to `#delay`, and vice versa. See [Task Data](#task-data) for more.
311
-
312
- * `:container` The container you've provided in your setup. See [Task Container](#task-container) for more.
313
-
314
- * `:logger` The queue's Logger object. See [Logging](#logging) for more.
315
-
316
- * `:scheduler` A scheduler object that you can use to schedule new tasks (eg. with `#delay`).
317
-
318
- ### Errors & Logging
319
-
320
- Errors that trigger `#fail` or `#final_fail` are saved to the task storage under columns `last_error` and
321
- `last_error_at`.
322
-
323
- Each queue worker also keeps a logfile log using the Ruby
324
- [Logger class](https://ruby-doc.org/stdlib-2.7.1/libdoc/logger/rdoc/Logger.html). Log files are named after the queue (
325
- eg. `log/welcome-queue-worker.log`).
326
-
327
- ```ruby
328
- scheduler = Procrastinator.setup do |env|
329
- # you can set custom log location and level:
330
- env.log_with(directory: '/var/log/myapp/', level: Logger::DEBUG)
331
-
332
- # you can also set the log rotation age or size (see Logger docs for details)
333
- env.log_with(shift: 1024, age: 5)
334
-
335
- # use a falsey log level to disable logging entirely:
336
- env.log_with(level: false)
337
- end
338
- ```
339
-
340
- The logger can be accessed in your tasks by calling `logger` or `@logger`.
341
-
342
- ```ruby
343
-
344
- class MyTask
345
- attr_accessor :logger, :scheduler, :container
346
-
347
- def run
348
- logger.info('This task got run. Hooray!')
349
- end
350
- end
351
- ```
352
-
353
- Some events are always logged by default:
354
-
355
- |event |level |
356
- |--------------------|-------|
357
- |process started | INFO |
358
- |#success called | DEBUG |
359
- |#fail called | DEBUG |
360
- |#final_fail called | DEBUG |
361
-
362
- ## Scheduling Tasks
353
+ ## Deferring Tasks
363
354
 
364
- To schedule tasks, just call `#delay` on the environment returned from `Procrastinator.setup`:
355
+ To add tasks to a queue, call `#defer` on the scheduler returned by `Procrastinator.setup`:
365
356
 
366
357
  ```ruby
367
- scheduler = Procrastinator.setup do |env|
368
- env.define_queue :reminder, EmailReminder
369
- env.define_queue :thumbnail, CreateThumbnail
358
+ scheduler = Procrastinator.setup do |config|
359
+ config.define_queue :reminder, EmailEveryone
360
+ config.define_queue :thumbnail, CreateThumbnail
370
361
  end
371
362
 
372
- # Provide the queue name and any data you want passed in
373
- scheduler.delay(:reminder, data: 'bob@example.com')
363
+ # Provide the queue name and any data you want passed in, if needed
364
+ scheduler.defer(:reminder)
365
+ scheduler.defer(:thumbnail, data: 'forcett.png')
374
366
  ```
375
367
 
376
368
  If there is only one queue, you may omit the queue name:
377
369
 
378
370
  ```ruby
379
- scheduler = Procrastinator.setup do |env|
380
- env.define_queue :reminder, EmailReminder
371
+ thumbnailer = Procrastinator.setup do |config|
372
+ config.define_queue :thumbnail, CreateThumbnail
381
373
  end
382
374
 
383
- scheduler.delay(data: 'bob@example.com')
375
+ thumbnailer.defer(data: 'forcett.png')
384
376
  ```
385
377
 
386
- ### Providing Data
387
-
388
- Most tasks need some additional information to complete their work, like id numbers,
378
+ ### Timing
389
379
 
390
- The `:data` parameter is serialized to string as YAML, so it's better to keep it as simple as possible. For example, if
391
- you have a database instead of passing in a complex Ruby object, pass in just the primary key and reload it in the
392
- task's `#run`. This will require less space in your database and avoids obsolete or duplicated information.
380
+ You can specify a particular timeframe that a task may be run. The default is to run immediately and never expire.
393
381
 
394
- ### Scheduling
382
+ Be aware that the task is not guaranteed to run at a precise time; the only promise is that the task won't be tried *
383
+ before* `run_at` nor *after* `expire_at`.
395
384
 
396
- You can set when the particular task is to be run and/or when it should expire. Be aware that the task is not guaranteed
397
- to run at a precise time; the only promise is that the task will be attempted *after* `run_at` and before `expire_at`.
385
+ Tasks attempted after `expire_at` will be final-failed. Setting `expire_at` to `nil`
386
+ means it will never expire (but may still fail permanently if, say, `max_attempts` is reached).
398
387
 
399
388
  ```ruby
400
- # runs on or after 1 January 3000
401
- scheduler.delay(:greeting, run_at: Time.new(3000, 1, 1), data: 'philip_j_fry@example.com')
389
+ run_time = Time.new(2016, 9, 19)
390
+ expire_time = Time.new(2016, 9, 20)
402
391
 
403
- # run_at defaults to right now:
404
- scheduler.delay(:thumbnail, run_at: Time.now, data: 'shut_up_and_take_my_money.gif')
405
- ```
392
+ # runs on or after 2016 Sept 19, never expires
393
+ scheduler.defer(:greeting, run_at: run_time, data: 'elanor@example.com')
406
394
 
407
- You can also set an `expire_at` deadline. If the task has not been run before `expire_at` is passed, then it will be
408
- final-failed the next time it would be attempted. Setting `expire_at` to `nil` means it will never expire (but may still
409
- fail permanently if, say, `max_attempts` is reached).
410
-
411
- ```ruby
412
- # will not run at or after
413
- scheduler.delay(:happy_birthday, expire_at: Time.new(2018, 03, 17, 12, 00, '-06:00'), data: 'contact@tenjin.ca')
395
+ # can run immediately but not after 2016 Sept 20
396
+ scheduler.defer(:greeting, expire_at: expire_time, data: 'mendoza@example.com')
414
397
 
415
- # expire_at defaults to nil:
416
- scheduler.delay(:greeting, expire_at: nil, data: 'bob@example.com')
398
+ # can run immediately but not after 2016 Sept 20
399
+ scheduler.defer(:greeting, run_at: run_time, expire_at: expire_time, data: 'tahani@example.com')
417
400
  ```
418
401
 
419
- ### Rescheduling
402
+ ### Rescheduling Existing Tasks
420
403
 
421
- Call `#reschedule` with the queue name and some identifying information, and then calling #to on that to provide the new
422
- time.
404
+ Call `#reschedule` with the queue name and some task-identifying information and then chain `#to` with the new time.
423
405
 
424
406
  ```ruby
425
- scheduler = Procrastinator.setup do |env|
426
- env.define_queue :reminder, EmailReminder
427
- end
407
+ run_time = Time.new(2016, 9, 19)
408
+ expire_time = Time.new(2016, 9, 20)
428
409
 
429
- scheduler.delay(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
410
+ scheduler.defer(:reminder, run_at: Time.at(0), data: 'chidi@example.com')
430
411
 
431
- # we can reschedule the task made above
432
- scheduler.reschedule(:reminder, data: 'bob@example.com').to(run_at: Time.parse('June 20 12:00'))
412
+ # we can reschedule the task that matches this data
413
+ scheduler.reschedule(:reminder, data: 'chidi@example.com').to(run_at: run_time)
433
414
 
434
415
  # we can also change the expiry time
435
- scheduler.reschedule(:reminder, data: 'bob@example.com').to(expire_at: Time.parse('June 23 12:00'))
416
+ scheduler.reschedule(:reminder, data: 'chidi@example.com').to(expire_at: expire_time)
436
417
 
437
418
  # or both
438
- scheduler.reschedule(:reminder, data: 'bob@example.com').to(run_at: Time.parse('June 20 12:00'),
439
- expire_at: Time.parse('June 23 12:00'))
419
+ scheduler.reschedule(:reminder, data: 'chidi@example.com').to(run_at: run_time,
420
+ expire_at: expire_time)
440
421
  ```
441
422
 
442
- Rescheduling sets the task's:
423
+ Rescheduling changes the task's...
443
424
 
444
425
  * `:run_at` and `:initial_run_at` to a new value, if provided
445
426
  * `:expire_at` to a new value if provided.
446
427
  * `:attempts` to `0`
447
428
  * `:last_error` and `:last_error_at` to `nil`.
448
429
 
449
- Rescheduling will not change `:id`, `:queue` or `:data`. A `RuntimeError` is raised if the runtime is after the expiry.
430
+ Rescheduling will not change `:id`, `:queue` or `:data`.
431
+
432
+ A `RuntimeError` is raised if the new run_at is after expire_at.
450
433
 
451
434
  ### Retries
452
435
 
453
- Failed tasks have their `run_at` rescheduled on an increasing delay (in seconds) according to this formula:
436
+ Failed tasks are automatically retried, with their `run_at` updated on an increasing delay (in seconds) according to
437
+ this formula:
454
438
 
455
- > 30 + (number_of_attempts)<sup>4</sup>
439
+ > 30 + number_of_attempts<sup>4</sup>
456
440
 
457
441
  Situations that call `#fail` or `#final_fail` will cause the error timestamp and reason to be stored in `:last_fail_at`
458
442
  and `:last_error`.
459
443
 
460
444
  ### Cancelling
461
445
 
462
- Call `#cancel` with the queue name and some identifying information to narrow the search to a single task.
446
+ Call `#cancel` with the queue name and some task-identifying information to narrow the search to a single task.
463
447
 
464
448
  ```ruby
465
- scheduler = Procrastinator.setup do |env|
466
- env.define_queue :reminder, EmailReminder
467
- end
449
+ run_time = Time.parse('April 1')
450
+ scheduler.defer(:reminder, run_at: run_time, data: 'derek@example.com')
468
451
 
469
- scheduler.delay(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
470
-
471
- # we can cancel the task made above using whatever we know about it, like the saved :data
472
- scheduler.reschedule(:reminder, data: 'bob@example.com')
473
-
474
- # or multiple attributes
475
- scheduler.reschedule(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
452
+ # we can cancel the task made above using whatever we know about it
453
+ # An error will be raised if it matches multiple tasks or finds none
454
+ scheduler.cancel(:reminder, run_at: run_time, data: 'derek@example.com')
476
455
 
477
456
  # you could also use the id number directly, if you have it
478
- scheduler.reschedule(:reminder, id: 137)
457
+ scheduler.cancel(:reminder, id: 137)
479
458
  ```
480
459
 
481
- ## Working on Tasks
482
-
483
- Use the scheduler object returned by setup to `#work` queues **serially**, **threaded**, or **daemonized**.
460
+ ## Testing with Procrastinator
484
461
 
485
- ### Serial Working
462
+ Working serially performs tasks from each queue sequentially. There is no multithreading or daemonizing.
486
463
 
487
- Working serially performs a task from each queue directly. There is no multithreading or daemonizing.
488
-
489
- Work serially for TDD tests or other situations you need close direct control.
464
+ Call `work` on the Scheduler with an optional list of queues to filter by.
490
465
 
491
466
  ```ruby
492
- # work just one task, no threading
467
+ # work just one task
493
468
  scheduler.work.serially
494
469
 
495
470
  # work the first five tasks
@@ -499,61 +474,75 @@ scheduler.work.serially(steps: 5)
499
474
  scheduler.work(:greeting, :reminders).serially(steps: 2)
500
475
  ```
501
476
 
502
- ### Threaded Working
503
-
504
- Threaded working will spawn a worker thread per queue.
477
+ ### RSpec Matchers
505
478
 
506
- Use threaded working for task queues that should only run while the main application is running. This includes the usual
507
- caveats around multithreading, so proceed with caution.
479
+ A `have_task` RSpec matcher is defined to make testing task scheduling a little easier.
508
480
 
509
481
  ```ruby
510
- # work tasks until the application exits
511
- scheduler.work.threaded
482
+ # Note: you must require the matcher file separately
483
+ require 'procrastinator'
484
+ require 'procrastinator/rspec/matchers'
512
485
 
513
- # work tasks for 5 seconds
514
- scheduler.work.threaded(timeout: 5)
486
+ task_storage = TaskStore.new
515
487
 
516
- # only work tasks on greeting and reminder queues
517
- scheduler.work(:greeting, :reminders).threaded
488
+ scheduler = Procrastinator.setup do |config|
489
+ config.define_queue :welcome, SendWelcome, store: task_storage
490
+ end
491
+
492
+ scheduler.defer(data: 'tahani@example.com')
493
+
494
+ expect(task_storage).to have_task(data: 'tahani@example.com')
518
495
  ```
519
496
 
520
- ### Daemonized Working
497
+ ## Running Tasks
521
498
 
522
- Daemonized working **consumes the current process** and then proceeds with threaded working in the new daemon.
499
+ When you are ready to run a Procrastinator daemon in production, you may use some provided Rake tasks.
523
500
 
524
- Use daemonized working for production environments, especially in conjunction with daemon monitors
525
- like [Monit](https://mmonit.com/monit/). Provide a block to daemonized! to get
501
+ In your Rake file call `DaemonTasks.define` with a block that constructs a scheduler instance.
526
502
 
527
503
  ```ruby
528
- # work tasks forever as a headless daemon process.
529
- scheduler.work.daemonized!
504
+ # Rakefile
505
+ require 'rake'
506
+ require 'procrastinator/rake/daemon_tasks'
507
+
508
+ # Defines a set of tasks that will control a Procrastinator daemon
509
+ # Default pid_path is /tmp/procrastinator.pid
510
+ Procrastinator::Rake::DaemonTasks.define do
511
+ Procrastinator.setup do
512
+ # ... etc ...
513
+ end
514
+ end
515
+ ```
530
516
 
531
- # you can specify the new process name and the directory to save the procrastinator.pid file
532
- scheduler.work.daemonized!(name: 'myapp-queue', pid_path: '/var/run')
517
+ You can name the daemon process by specifying the pid_path with a specific .pid file. If does not end with '.pid' it is
518
+ assumed to be a directory name, and `procrastinator.pid` is appended.
533
519
 
534
- # ... or set the pid file name precisely by giving a .pid path
535
- scheduler.work.daemonized!(pid_path: '/var/run/myapp.pid')
520
+ ```ruby
521
+ # Rakefile
536
522
 
537
- # only work tasks in the 'greeting' and 'reminder' queues
538
- scheduler.work(:greeting, :reminders).daemonized!
523
+ # This would define a process titled my-app
524
+ Procrastinator::Rake::DaemonTasks.define(pid_path: 'my-app.pid') do
525
+ # ... build a Procrastinator instance here ...
526
+ end
539
527
 
540
- # supply a block to run code after the daemon subprocess has forked off
541
- scheduler.work.daemonized! do
542
- # this gets run after the daemon is spawned
543
- task_store.reconnect_mysql
528
+ # equivalent to ./pids/procrastinator.pid
529
+ Procrastinator::Rake::DaemonTasks.define(pid_path: 'pids') do
530
+ # ... build a Procrastinator instance here ...
544
531
  end
545
532
  ```
546
533
 
547
- Procrastinator endeavours to be thread-safe and support concurrency, but this flexibility allows for many possible
548
- combinations.
549
-
550
- Expected use is a single process with one thread per queue. More complex use is possible but Procrastinator can't
551
- guarantee concurrency in your Task Store.
534
+ Either run the generated Rake tasks in a terminal or with your daemon monitoring tool of choice (eg. Monit, systemd)
552
535
 
553
- #### PID Files
536
+ ```bash
537
+ # In terminal
538
+ bundle exec rake procrastinator:start
539
+ bundle exec rake procrastinator:status
540
+ bundle exec rake procrastinator:restart
541
+ bundle exec rake procrastinator:stop
542
+ ```
554
543
 
555
- Process ID files are a single-line file that saves the daemon's process ID number. It's saved to the directory given
556
- by `:pid_dir`. The default location is `pids/` relative to the file that called `#daemonized!`.
544
+ There are instructions for using Procrastinator with Monit in
545
+ the [github wiki](https://github.com/TenjinInc/procrastinator/wiki/Monit-Configuration).
557
546
 
558
547
  ## Similar Tools
559
548
 
@@ -597,6 +586,8 @@ To install this gem onto your local machine, run `bundle exec rake install`. To
597
586
  version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
598
587
  push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
599
588
 
589
+ Docs are generated using YARD. Run `rake yard` to generate a local copy.
590
+
600
591
  ## License
601
592
 
602
593
  The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).