procrastinator 0.6.1 → 0.9.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 7162a00d304b7e97d03893a92fbe45b11534e305
-  data.tar.gz: e6e0fe48cf1cc1940aed6f85ccf889e36d40d0ff
+  metadata.gz: b3428d03ab5288918a2d1305ca9e8234303ba0fd
+  data.tar.gz: 75ab4bae088f10401a976a29ad84fbfb1b9a6bf7
 SHA512:
-  metadata.gz: 3d23c3bba5d8df4963a7a33cbe70606dba64b86cba3f0c5861d37a9a878728e180bbea1afa196431fef7f2121e2a073cb2450989970a3b1c899378bd82f5a101
-  data.tar.gz: b2c35bafb8c18af846ab8199a923ef9bc9d09d81c731c3d9c0b46a7ca7f1ab1c87adf8f9f0ff1768341c86d9a59044b6f94db8230bcba6da15c68f0e849831df
+  metadata.gz: e9e3ebd949697326212b199bbf17e9424a8dac47e4ac7ebbfb3154764330462fc46931ccc67aee4fc6c81a70d4e5513f7c45b66f9ac63d272987a1dc79f62519
+  data.tar.gz: 99df6bdc69e8af68572fa1619f5ecaaeb0083853e60c2ec2f9da7f5a644f5f9bfffc4e54ff64ac0eda3004c93ca7c4a4cb4c438816dc9b044025f8f2da53c311

data/.rubocop.yml

@@ -0,0 +1,7 @@
+inherit_from: ../.rubocop.yml
+
+AllCops:
+  Exclude:
+    - 'bin/*'
+
+  TargetRubyVersion: 2.3

data/.ruby-version

@@ -1 +1 @@
-ruby-2.1.2
+ruby-2.4.2

data/Gemfile

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 source 'https://rubygems.org'
 
 # Specify your gem's dependencies in procrastinator.gemspec

data/README.md

@@ -1,9 +1,67 @@
 # Procrastinator
+Procrastinator is a pure ruby job scheduling gem to allow your app to put off work for later. 
+Tasks are scheduled in queues and those queues are monitored by separate worker subprocesses. 
+Once the scheduled time arrives, the queue worker performs that task. 
 
-Procrastinator is a framework-independent job scheduling gem to allow your app to put stuff of until later. It creates 
-a subprocess for each queue to performs tasks at the designated times. Or maybe later, depending on how busy it is. 
+If the task fails to complete or takes too long, it delays it and tries again later.
 
-Don't worry, it'll get done eventually. 
+## Big Picture
+If you have tasks like this:
+
+```ruby
+class SendWelcomeEmail
+   def run
+      # ... email stuff ...
+   end
+end
+```
+
+Setup a procrastination environment:
+```ruby
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :greeting, SendWelcomeEmail
+   env.define_queue :thumbnail, GenerateThumbnail, timeout: 60
+   env.define_queue :birthday, SendBirthdayEmail, max_attempts: 3
+end
+```
+
+And then get your lazy on:
+
+```ruby
+scheduler.delay(:greeting, data: 'bob@example.com')
+
+scheduler.delay(:thumbnail, data: {file: 'full_image.png', width: 100, height: 100})
+
+scheduler.delay(:send_birthday_email, run_at: Time.now + 3600, data: {user_id: 5})
+```
+
+## Contents
+  * [Installation](#installation)
+  * [Setup](#setup)
+    + [Defining Queues: `#define_queue`](#defining-queues----define-queue-)
+    + [The Task Loader: `#load_with`](#the-task-loader----load-with-)
+      - [Task Data](#task-data)
+    + [The Task Context: `#provide_context`](#the-task-context----provide-context-)
+    + [Controlling Queue Processes: `#each_process`](#controlling-queue-processes----each-process-)
+      - [The Subprocess Hook](#the-subprocess-hook)
+      - [Naming Processes](#naming-processes)
+      - [Tracking Process IDs](#tracking-process-ids)
+  * [Tasks](#tasks)
+    + [Accessing Task Attributes](#accessing-task-attributes)
+    + [Retries](#retries)
+  * [Scheduling Tasks](#scheduling-tasks)
+    + [Providing Data](#providing-data)
+    + [Controlling Timing](#controlling-timing)
+    + [Rescheduling](#rescheduling)
+    + [Cancelling](#cancelling)
+  * [Preventing Queue Workers](#preventing-queue-workers)
+  * [Test Mode](#test-mode)
+  * [Errors & Logging](#errors---logging)
+  * [Contributing](#contributing)
+    + [Developers](#developers)
+  * [License](#license)
+
+<!-- ToC generated with http://ecotrust-canada.github.io/markdown-toc/ -->
 
 ## Installation
 
@@ -17,224 +75,518 @@ And then run:
 
     bundle install
 
-## Usage
-Setup a procrastination environment:
+## Setup
+`Procrastinator.setup` allows you to define which queues are available. You can also optionally
+specify a task loader IO object, task context, and other settings.
 
 ```ruby
-procrastinator = Procrastinator.setup(TaskPersister.new) do |env| 
-  env.define_queue(:email)
-  env.define_queue(:cleanup, max_attempts: 3)
+Procrastinator.setup do |env|
+   # call methods on env to set configurations
 end
 ```
 
-And then delay some tasks:
+When setup is complete, Procrastinator spins off a sub process to work on each queue and 
+returns the configured scheduler, which is used to `#delay` tasks. 
+
+If there are old queue processes, Procrastinator will kill them before spawning a replacement. 
+
+### Defining Queues: `#define_queue`
+In the setup block, you can call `#define_queue` on the environment: 
 
 ```ruby
-procrastinator.delay(queue: :email, task: EmailGreeting.new('bob@example.com'))
-procrastinator.delay(queue: :cleanup, run_at: Time.now + 3600, task: ClearTempData.new)
+Procrastinator.setup do |env|
+   env.define_queue :greeting, SendWelcomeEmail
+end
 ```
 
-Read on for more details on each step. 
+The first two parameters are the queue name symbol and the task class to run on that queue. 
 
-### Setup Phase
-The setup phase first defines which queues are available and the persistence strategy to use for reading 
-and writing tasks. It then spins off a sub process for working on each queue within that environment. 
+You can also provide these keyword arguments:
 
+ * `:timeout`
+ 
+   Duration (seconds) after which tasks in this queue should fail for taking too long.
+    
+ * `:max_attempts` 
+ 
+   Maximum number of attempts for tasks in this queue. Once `attempts` meets or exceeds `max_attempts`, the task will 
+   be permanently failed.
+    
+ * `:update_period`
+  
+   Delay (seconds) between reloads of all tasks from the task loader.
+   
+ * `:max_tasks`
+ 
+   The maximum number of tasks to run concurrently within a queue worker process.
+
+
+```ruby 
+# all defaults set explicitly:
+env.define_queue :queue_name, YourTaskClass, timeout: 3600, max_attempts: 20, update_period: 10, max_tasks: 10
+```
 
-#### Declaring a Persistence Strategy
-The persister instance is the last step between Procrastinator and your data storage (eg. database). As core Procrastinator is framework-agnostic, it needs you to provide an object that knows how to read and write task data. 
+### The Task Loader: `#load_with`
+The task loader is a [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) pattern object
+that knows how to read and write tasks in your data storage (eg. file, database, etc).
 
-Your [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) class is required to provide the following methods: 
+Procrastinator comes with a simple CSV file task loader by default, but you are encouraged to build one that suits
+your situation. 
 
-* `#read_tasks(queue_name)` - Returns a list of hashes from your data storage. Each hash must contains the properites of one task, as seen in the *Attributes Hash* 
-* `#create_task(data)` - Creates a task in your datastore. Receives a hash with keys `:queue`, `:run_at`, `:initial_run_at`, `:expire_at`, and `:task` as described in *Attributes Hash* 
-* `#update_task(attributes)` - Receives the Attributes Hash as the data to be saved
-* `#delete_task(id)` - Deletes the task with the given id. 
+In setup, the environment's `#load_with` method expects an task loader instance: 
 
-If the strategy does not have all of these methods, Procrastinator will explode with a `MalformedPersisterError`  and you will be sad. 
+```ruby
+loader = MyTaskLoader.new('tasks.csv')
 
-***Attributes Hash***
+scheduler = Procrastinator.setup do |env|
+   # ... other setup stuff ...
 
-|  Hash Key         | Type   | Description                                                                           |
-|-------------------|--------| --------------------------------------------------------------------------------------|
-| `:id`             | int    | Unique identifier for this exact task                                                 |
-| `:queue`          | symbol | Name of the queue the task is inside                                                      | 
-| `:run_at`         | int    | Unix timestamp of when to next attempt running the task                               |
-| `:initial_run_at` | int    | Unix timestamp of the original run_at; before the first attempt, this is equal to run_at |
+   env.load_with loader
+   
+   # if you're using the default CSV loader and want to set where it saves the CSV,
+   # provide the keyword argument :location with your path or filename
+   env.load_with location: '/var/myapp/'
+end
+```
+
+For per-process resource management (eg. independent database connections), you can build and assign a 
+task loader in the `#each_process` block.
+
+```ruby
+connection = SomeDatabaseLibrary.connect('my_app_development')
+
+scheduler = Procrastinator.setup do |env|
+   env.load_with MyTaskLoader.new(connection)
+   
+   env.each_process do
+      # make a fresh connection
+      connection.reconnect
+      env.load_with MyTaskLoader.new(connection)
+   end
+   
+   # .. other setup stuff ...
+end
+```
+
+A task loader class is required to implement *all* of the following four methods: 
+
+1. `#read(attributes)`
+
+   Returns a list of hashes from your datastore that match the given attributes hash. The search
+   attributes will be in their final form (eg. `:data` will already be serialized).
+   Each hash must contain the properties listed in [Task Data](#task-data) below.
+     
+2. `#create(queue:, run_at:, initial_run_at:, expire_at:, data:)`
+
+   Creates a task in your datastore. Receives a hash with [Task Data](#task-data) keys: 
+   `:queue`, `:run_at`, `:initial_run_at`, `:expire_at`, and `:data`.
+    
+3. `#update(id, new_data)`
+ 
+   Saves the provided full [Task Data](#task-data) hash to your datastore.
+   
+4. `#delete(id)`
+ 
+   Deletes the task with the given identifier in your datastore.
+
+<!-- This paragraph is here to allow people to google for the error keyword -->
+If your loader is missing any of the above methods, Procrastinator will explode 
+with a `MalformedPersisterError` and you will be sad. 
+
+#### Task Data
+These are the data fields for each individual scheduled task. When using the built-in task loader,
+these are the field names. If you have a database, use this to inform your table schema. 
+
+|  Hash Key         | Type   | Description                                                                             |
+|-------------------|--------| ----------------------------------------------------------------------------------------|
+| `:id`             | int    | Unique identifier for this exact task                                                   |
+| `:queue`          | symbol | Name of the queue the task is inside                                                    | 
+| `:run_at`         | int    | Unix timestamp of when to next attempt running the task. ¹                              |
+| `:initial_run_at` | int    | Unix timestamp of the originally requested run                                          |
 | `:expire_at`      | int    | Unix timestamp of when to permanently fail the task because it is too late to be useful |
-| `:attempts`       | int    | Number of times the task has tried to run; this should only be > 0 if the task fails  |
-| `:last_fail_at`   | int    | Unix timestamp of when the most recent failure happened                               |
-| `:last_error`     | string | Error message + bracktrace of the most recent failure. May be very long.              |
-| `:task`           | string | YAML-dumped ruby object definition of the task.                                       |
+| `:attempts`       | int    | Number of times the task has tried to run; this should only be > 0 if the task fails    |
+| `:last_fail_at`   | int    | Unix timestamp of when the most recent failure happened                                 |
+| `:last_error`     | string | Error message + bracktrace of the most recent failure. May be very long.                |
+| `:data`           | string | Serialized data accessible in the task instance.²                                       |
 
-Notice that the times are all given as unix epoch timestamps. This is to avoid any confusion with timezones, and it is recommended that you store times in this manner for the same reason. 
+> ¹ If `nil`, that indicates that it is permanently failed and will never run, either due to expiry or too many attempts.
 
-#### Defining Queues
-`Procrastinator.setup` requires a block be provided, and that in the block call `#define_queue` be called on the provided environment. Define queue takes a queue name symbol and these properies as a hash
+> ² Serialized using YAML.dump and unserialized with YAML.safe_load. Keep to simple data types (eg. id numbers) to 
+> reduce storage space, eliminate redundancy, and reduce the chance of a serialization error.
 
- * :timeout - Time, in seconds, after which it should fail tasks in this queue for taking too long to execute.
- * :max_attempts - Maximum number of attempts for tasks in this queue. If attempts is >= max_attempts, the task will be final_failed and marked to never run again
- * :update_period - Delay, in seconds, between refreshing the task list from the persister
- * :max_tasks - The maximum number of tasks to run concurrently with multi-threading. 
+Notice that the times are all stored as unix epoch integer timestamps. This is to avoid confusion or conversion
+errors with timezones or daylight savings.
 
-**Examples**
-```ruby 
-Procrastinator.setup(some_persister) do |env|
-   env.define_queue(:email)
+### The Task Context: `#provide_context`
+Whatever you give to `#provide_context` will be made available to your Task through the task attribute `:context`. 
+
+This can be useful for things like app containers, but you can use it for whatever you like.  
+
+```ruby
+Procrastinator.setup do |env|
+   # .. other setup stuff ...
+ 
+   env.provide_context some_key: "This hash will be passed into your task's methods"
+end
+
+# ... and in your task ...
+class MyTask
+   include Procrastinator::Task
+   
+   task_attr :context
    
-   # with all defaults set explicitly
-   env.define_queue(:email, timeout: 3600, max_attempts: 20, update_period: 10, max_tasks: 10)
+   def run
+      puts "My context is: #{context}"
+   end
 end
 ```
-  
-#### Sub-Processes
-Each queue is worked in a separate process.  
 
-<!-- , and each process multi-threaded to handle more than one task at a time. This should help prevent a single task from clogging up the whole queue, or a single queue clogging up the entire system. -->
+### Controlling Queue Processes: `#each_process`
+In the setup block, use `#each_process` to configure details about the queue subprocesses. 
+
+#### The Subprocess Hook
+If you pass a block to `#each_process`, it will be run after the process is forked, 
+but before the queue worker starts working on tasks.
+
+```ruby
+Procrastinator.setup do |env|
+   # ... other setup stuff ...
+
+   env.each_process do 
+      # create process-specific resources here, like database connections 
+      # (the parent process's connection could disappear, because they're asychnronous)
+      connection = SomeDatabase.connect('bob@mainframe/my_database')
+      
+      # these two are the configuration methods you're most likely to use in #each_process
+      env.provide_context MyApp.build_task_package
+      env.load_with MyDatabase.new(connection)
+   end
+end
+```
+
+NB: The `each_process` block **does not run in Test Mode**. 
+
+#### Naming Processes
+Each queue subprocess is named after the queue it's working on, eg. `greeting-queue-worker` or 
+`thumbnails-queue-worker`.
 
-The sub-processes checks that the parent process is still alive every 5 seconds. If there is no process with the parent's PID, the sub-process will self-exit. 
+If you're running multiple apps on the same machine, then you may want to distinguish which app the queue worker 
+was spawned for. You can provide a `prefix:` argument to `#each_process` with a string that will be prepended to the 
+process command name.  
 
-Sub-processes can be given a name prefix with the process_prefix method: 
+```ruby
+scheduler = Procrastinator.setup do |env|
+   # ... other setup stuff ...
+   
+   env.each_process(prefix: 'myapp')
+end
+```
+
+#### Tracking Process IDs
+The sub processes are tracked by saving their PIDs in files. You can set the directory where those files are saved: 
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.process_prefix('myapp')
+scheduler = Procrastinator.setup do |env|
+   # ... other setup stuff ...
+   
+   env.each_process(pid_dir: '/var/run/myapp/')
 end
 ```
 
-###Scheduling Tasks For Later
-Procrastinator will let you be lazy: 
+Any PIDs found in those files are killed right before Procrastinator spawns queues, 
+so that it can replace them with new code.  
+
+## Tasks
+Your task class is what actually gets run on the task queue. They will look something like this: 
+
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
+class MyTask
+   include Procrastinator::Task
+   
+   # Give any of these symbols to task_attr and they will become available as methods
+   # task_attr :data, :logger, :context, :scheduler
+   
+   # Performs the core work of the task. 
+   def run
+      # ... perform your task ...
+   end
+   
+   
+   # ========================================
+   #             OPTIONAL HOOKS
+   #
+   # You can always omit any of the methods
+   # below. Only #run is mandatory.
+   #
+   # ========================================
+   
+   # Called after the task has completed successfully. 
+   # Receives the result of #run.
+   def success(run_result)
+      # ...
+   end
+   
+   # Called after #run raises any StandardError (or subclass).
+   # Receives the raised error.
+   def fail(error)
+      # ...
+   end
+   
+   # Called after either is true: 
+   #   1. the time reported by Time.now is past the task's expire_at time.
+   #   2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`. 
+   #      In this case, #fail will not be executed, only #final_fail. 
+   #
+   # When called, the task will be marked to never be run again.
+   # Receives the raised error.
+   def final_fail(error)
+      # ...
+   end
 end
+```
+
+### Accessing Task Attributes
+Include `Procrastinator::Task` in your task class and then use `task_attr` to register which task attributes your 
+task wants access to. 
 
-procrastinator.delay(task: EmailReminder.new('bob@example.com'))
+```ruby
+class MyTask
+   include Procrastinator::Task
+   
+   # declare the task attributes you care about by calling task_attr. 
+   # You can use any of these symbols: 
+   #             :data, :logger, :context, :scheduler
+   task_attr :data, :logger
+   
+   def run
+      # the attributes listed in task_attr become methods like attr_accessor
+      logger.info("The data for this task is #{data}")
+   end
+end
 ```
 
-... unless there are multiple queues defined. Thne you must provide a queue name with your task:
+ * `:data`
+   This is the data that you provided in the call to `#delay`. Any task that registers `:data` as a task 
+   attribute will require data be passed to `#delay`.
+   See [Task Data](#task-data) for more.
+   
+ * `:context`
+  
+   The context you've provided in your setup. See [Task Context](#task-context-provide_context) for more.
+ 
+ * `:logger`
+    
+   The queue's Logger object. See [Logging](#logging) for more.
+    
+ * `:scheduler`
+ 
+   A scheduler object that you can use to schedule new tasks (eg. with `#delay`).
+
+
+### Retries
+Failed tasks have their `run_at` rescheduled on an increasing delay (in seconds) according to this formula: 
+
+> 30 + (number_of_attempts)<sup>4</sup>
+
+Situations that call `#fail` or `#final_fail` will cause the error timestamp and reason to be stored in `:last_fail_at` 
+and `:last_error`.
+
+
+## Scheduling Tasks
+To schedule tasks, just call `#delay` on the environment returned from `Procrastinator.setup`: 
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
-   env.define_queue(:cleanup)
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
+   env.define_queue :thumbnail, CreateThumbnail
 end
 
-procrastinator.delay(:email, task: EmailReminder.new('bob@example.com'))
+# Provide the queue name and any data you want passed in
+scheduler.delay(:reminder, data: 'bob@example.com')
+``` 
+
+If you have only one queue, you can omit the queue name: 
+
+```ruby
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
+end
+
+scheduler.delay(data: 'bob@example.com')
 ```
 
+### Providing Data
+Most tasks need some additional information to complete their work, like id numbers, 
+
+The `:data` parameter is serialized to string as YAML, so it's better to keep it as simple as possible. For example, if 
+you have a database instead of passing in a complex Ruby object, pass in just the primary key and reload it in the 
+task's `#run`. This will require less space in your database and avoids obsolete or duplicated information.    
+
+### Controlling Timing
 You can set when the particular task is to be run and/or when it should expire. Be aware that the task is not guaranteed 
-to run at a precise time; the only promise is that the task will get run some time after `run_at`, unless it's after `expire_at`. 
+to run at a precise time; the only promise is that the task will be attempted *after* `run_at` and before `expire_at`.
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
-end
+# runs on or after 1 January 3000
+scheduler.delay(:greeting, run_at: Time.new(3000, 1, 1), data: 'philip_j_fry@example.com')
+
+# run_at defaults to right now:
+scheduler.delay(:thumbnail, run_at: Time.now, data: 'shut_up_and_take_my_money.gif')
+```
 
-# run on or after 1 January 3000
-procrastinator.delay(run_at: Time.new(3000, 1, 1), task: EmailGreeting.new('philip_j_fry@example.com'))
+You can also set an `expire_at` deadline. If the task has not been run before `expire_at` is passed, then it will be 
+final-failed the next time it would be attempted.
+Setting `expire_at` to `nil` means it will never expire (but may still fail permanently if, 
+say, `max_attempts` is reached).
 
-# explicitly setting default run_at
-procrastinator.delay(run_at: Time.now, task: EmailReminder.new('bob@example.com'))
+```ruby
+# will not run at or after 
+scheduler.delay(:happy_birthday, expire_at: Time.new(2018, 03, 17, 12, 00, '-06:00'),  data: 'contact@tenjin.ca'))
+
+# expire_at defaults to nil:
+scheduler.delay(:greeting, expire_at: nil, data: 'bob@example.com')
 ```
 
-You can also set an `expire_at` deadline on when to run a task. If the task has not been run before `expire_at` is passed, then it will be final-failed the next time it is attempted. Setting `expire_at` to `nil` will mean it will never expire (but may still fail permanently if, say, `max_attempts` is reached).
+### Rescheduling
+Call `#reschedule` with the queue name and some identifying 
+information, and then calling #to on that to provide the new time.
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
 end
 
-procrastinator.delay(expire_at: , task: EmailGreeting.new('bob@example.com'))
+scheduler.delay(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
+
+# we can reschedule the task made above 
+scheduler.reschedule(:reminder, data: 'bob@example.com').to(run_at: Time.parse('June 20 12:00'))
+
+# we can also change the expiry time
+scheduler.reschedule(:reminder, data: 'bob@example.com').to(expire_at: Time.parse('June 23 12:00'))
 
-# explicitly setting default
-procrastinator.delay(expire_at: nil, task: EmailGreeting.new('bob@example.com'))
+# or both
+scheduler.reschedule(:reminder, data: 'bob@example.com').to(run_at:    Time.parse('June 20 12:00'), 
+                                                            expire_at: Time.parse('June 23 12:00'))
 ```
 
-#### Task Definition
-Like the persister provided to `.setup`, your task is a strategy object that fills in the details of what to do. For this, 
-your task **must provide** a `#run` method:
+Rescheduling updates the task's `:run_at` and `:initial_run_at` to a new value, if provided and/or 
+`:expire_at` to a new value if provided. A `RuntimeError` is raised if the resulting runtime is after the expiry. 
 
- * `#run` - Performs the core work of the task.
+It also resets `:attempts` to `0` and clears both `:last_error` and `:last_error_at` to `nil`.
 
-You may also optionally provide these hook methods, which are run during different points in the process:
+Rescheduling will not change `:id`, `:queue` or `:data`.
 
- * `#success(logger)` - run after the task has completed successfully 
- * `#fail(logger, error)` - run after the task has failed due to `#run` producing a `StandardError` or subclass.
- * `#final_fail(logger, error)` - run after the task has failed for the last time because either:
-    1. the number of attempts is >= the `max_attempts` defined for the queue; or
-    2. the time reported by `Time.now` is past the task's `expire_at` time.
+### Cancelling
+Call `#cancel` with the queue name and some identifying information to narrow the search to a single task.
 
-If a task reaches `#final_fail` it will be marked to never be run again.
+```ruby
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
+end
 
-***Task Failure & Rescheduling***
+scheduler.delay(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
 
-Tasks that fail have their `run_at` rescheduled on an increasing delay **(in seconds)** according to this formula: 
- * 30 + n<sup>4</sup>
-  
-n = the number of attempts 
+# we can cancel the task made above using whatever we know about it, like the saved :data
+scheduler.reschedule(:reminder, data: 'bob@example.com')
 
-Both failing and final_failing will cause the error timestamp and reason to be stored in `:last_fail_at` and `:last_error`.
+# or multiple attributes
+scheduler.reschedule(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
 
-### Testing With Procrastinator
-Procrastinator uses multi-threading and multi-processing internally, which is a nightmare for testing. Fortunately for you, 
-Test Mode will disable all of that, and rely on your tests to tell it when to tick. 
+# you could also use the id number directly, if you have it
+scheduler.reschedule(:reminder, id: 137)
+```
 
-Enable Test Mode by setting `Procrastinator.test_mode` to `true` before setting up, or by calling enable_test_mode on 
-the procrastination environment. 
+## Preventing Queue Workers
+Sometimes you want to be able to put your app into a maintenance mode temporarily. 
+If Procrastinator sees the environment variable `PROCRASTINATOR_STOP` is set, it
+will not spawn child process queue workers at all. 
+
+To kill workers, set `PROCRASTINATOR_STOP` to 1 and then restart your app. It will
+find all the old subprocesses and halt them, but not create new ones.  
+
+This is distinct from Test Mode, because Test Mode will still create workers, but on the
+same process as the original caller to setup. 
+
+## Test Mode
+Procrastinator uses multi-threading and multi-processing internally, which is a nightmare for automated testing. 
+Test Mode will disable all of that and rely on your tests to tell it when to act. 
+
+Set `Procrastinator.test_mode = true` before setup, or call `#enable_test_mode` on 
+the procrastination environment:
 
 ```ruby
 # all further calls to `Procrastinator.setup` will produce a procrastination environment where Test Mode is enabled
 Procrastinator.test_mode = true
  
-# or you can also enable it directly in the setup
-env = Procrastinator.setup do |env|
+# or you can also enable it in the setup
+scheduler = Procrastinator.setup do |env|
    env.enable_test_mode
     
-   # other settings...
+   # ... other settings...
 end
 ```
 
-In your tests, tell the procrastinator environment to work off one item from its queues: 
+Then in your tests, tell the procrastinator environment to work off one item: 
 
 ```
-# works one task on all queues
+# execute one task on all queues
 env.act
 
-# provide queue names to works one task on just those queues
-env.act(:cleanup, :email)
+# or provide queue names to execute one task on those specific queues
+scheduler.act(:cleanup, :email)
 ```
 
-### Logging
-Logging is crucial to knowing what went wrong in an application after the fact, and because Procrastinator runs workers
-in separate processes, providing a logger instance isn't really an option. 
- 
-Instead, provide a directory that your Procrastinator instance should write log entries into:
+## Errors & Logging
+Errors that trigger #fail or #final_fail are saved in the task persistence (database, file, etc) under `last_error` and 
+`last_error_at`.
+
+Each queue worker also writes its own log using the Ruby 
+[Logger class](https://ruby-doc.org/stdlib-2.5.1/libdoc/logger/rdoc/Logger.html).
+The log files are named after its queue process name (eg. `log/welcome-queue-worker.log`) and
+they are saved in the log directory defined in setup. 
 
 ```ruby
-procrastinator = Procrastinator.setup do |env|
-   env.log_dir('log/')
+scheduler = Procrastinator.setup do |env|
+   # ... other setup stuff ... 
+
+   # you can set custom log directory and level:
+   env.log_inside '/var/log/myapp/'
+   env.log_at_level Logger::DEBUG
+   
+   # these are the defaults:
+   env.log_inside 'log/' # relative to the running directory
+   env.log_at_level Logger::INFO
+   
+   # use nil to disable logging entirely:
+   env.log_inside nil
 end
 ```
- 
-Each worker creates its own log named after the queue it is working on (eg. `log/email-queue-worker.log`). The default 
-directory is `./log/`, relative to wherever the application is running. Logging will not occur at all if `log_dir` is 
-assigned a falsey value. 
 
-The logging level can be set using `log_level` and a value from the Ruby standard library 
-[Logger class](https://ruby-doc.org/stdlib-2.2.3/libdoc/logger/rdoc/Logger.html) (eg. `Logger::WARN`, `Logger::DEBUG`, etc.). 
-
-It logs process start at level `INFO`, process termination due to parent disppearance at level `ERROR` and task hooks 
-`#success`, `#fail`, and `#final_fail` are at a level `DEBUG`. 
+The logger can be accessed in your tasks by including Procrastinator::Task in your task class and then calling
+`task_attr :logger`. 
 
 ```ruby
-procrastinator = Procrastinator.setup do |env|
-   env.log_dir('log/')
-   env.log_level(Logger::INFO) # setting the default explicity
+class MyTask
+   include Procrastinator::Task
+   
+   task_attr :logger
+
+   def run
+      logger.info('This task got run. Hooray!')
+   end
 end
 ```
 
+**Default Log Messages**
+
+|event               |level  |
+|--------------------|-------|
+|process started     | INFO  |
+|#success called     | DEBUG |
+|#fail called        | DEBUG |
+|#final_fail called  | DEBUG |
+
 ## Contributing
 Bug reports and pull requests are welcome on GitHub at 
 [https://github.com/TenjinInc/procrastinator](https://github.com/TenjinInc/procrastinator).
@@ -242,7 +594,9 @@ Bug reports and pull requests are welcome on GitHub at
 This project is intended to be a friendly space for collaboration, and contributors are expected to adhere to the 
 [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
-### Core Developers
+Play nice.
+
+### Developers
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can 
 also run `bin/console` for an interactive prompt that will allow you to experiment.