procrastinator 0.6.1 → 1.0.0.pre.rc2

data/README.md

@@ -1,9 +1,80 @@
 # Procrastinator
 
-Procrastinator is a framework-independent job scheduling gem to allow your app to put stuff of until later. It creates 
-a subprocess for each queue to performs tasks at the designated times. Or maybe later, depending on how busy it is. 
+Procrastinator is a pure Ruby job scheduling gem. Put off tasks until later or for another process to handle.
 
-Don't worry, it'll get done eventually. 
+Tasks are can be rescheduled and retried after failures, and you can use whatever storage mechanism is needed.
+
+## Big Picture
+
+If you have tasks like this:
+
+```ruby
+# Sends a welcome email
+class SendWelcomeEmail
+   def run
+      # ... etc
+   end
+end
+```
+
+Setup a procrastination environment like:
+
+```ruby
+scheduler = Procrastinator.setup do |env|
+   env.with_store some_email_task_database do
+      env.define_queue :greeting, SendWelcomeEmail
+      env.define_queue :birthday, SendBirthdayEmail, max_attempts: 3
+   end
+
+   env.define_queue :thumbnail, GenerateThumbnail, store: 'imgtasks.csv', timeout: 60
+end
+```
+
+Put jobs off until later:
+
+```ruby
+scheduler.delay(:greeting, data: 'bob@example.com')
+
+scheduler.delay(:thumbnail, data: {file: 'full_image.png', width: 100, height: 100})
+
+scheduler.delay(:send_birthday_email, run_at: Time.now + 3600, data: {user_id: 5})
+```
+
+And tell a process to actually do them:
+
+```ruby
+# Starts a daemon with a thread for each queue. 
+# Other options are direct control (for testing) or threaded (for terminals)
+scheduler.work.daemonized!
+```
+
+## Contents
+
+- [Installation](#installation)
+- [Setup](#setup)
+    * [Defining Queues](#defining-queues)
+    * [Task Store](#task-store)
+        + [Data Fields](#data-fields)
+    * [Task Container](#task-container)
+- [Tasks](#tasks)
+    * [Accessing Task Attributes](#accessing-task-attributes)
+    * [Errors & Logging](#errors-logging)
+- [Scheduling Tasks](#scheduling-tasks)
+    * [Providing Data](#providing-data)
+    * [Scheduling](#scheduling)
+    * [Rescheduling](#rescheduling)
+    * [Retries](#retries)
+    * [Cancelling](#cancelling)
+- [Working on Tasks](#working-on-tasks)
+    * [Stepwise Working](#stepwise-working)
+    * [Threaded Working](#threaded-working)
+    * [Daemonized Working](#daemonized-working)
+        + [PID Files](#pid-files)
+- [Contributing](#contributing)
+    * [Developers](#developers)
+- [License](#license)
+
+<!-- ToC generated with http://ecotrust-canada.github.io/markdown-toc/ -->
 
 ## Installation
 
@@ -17,238 +88,515 @@ And then run:
 
     bundle install
 
-## Usage
-Setup a procrastination environment:
+## Setup
+
+`Procrastinator.setup` allows you to define which queues are available and other settings.
 
 ```ruby
-procrastinator = Procrastinator.setup(TaskPersister.new) do |env| 
-  env.define_queue(:email)
-  env.define_queue(:cleanup, max_attempts: 3)
+require 'procrastinator'
+
+scheduler = Procrastinator.setup do |config|
+   # ...
 end
 ```
 
-And then delay some tasks:
+It then returns a `Scheduler` that your code can use to schedule tasks or tell to start working.
+
+* See [Scheduling Tasks](#scheduling-tasks)
+* See [Start Working](#start-working)
+
+### Defining Queues
+
+In setup, call `#define_queue` with a symbol name and the class that performs those jobs:
+
+```ruby
+# You must provide a queue name and the class that handles those jobs
+config.define_queue :greeting, SendWelcomeEmail
+
+# but queues have some optional settings, too
+config.define_queue :greeting, SendWelcomeEmail, store: 'tasks.csv', timeout: 60, max_attempts: 2, update_period: 1
+
+# all defaults set explicitly
+config.define_queue :greeting, SendWelcomeEmail, store: 'procrastinator.csv', timeout: 3600, max_attempts: 20, update_period: 10
+```
+
+Description of keyword options:
+
+| Option           | Description   |
+|------------------| ------------- |
+| `:store`         | Storage IO object for tasks. See [Task Store](#task-store)     |
+| `:timeout`       | Max duration (seconds) before tasks are failed for taking too long  |
+| `:max_attempts`  | Once a task has been attempted `max_attempts` times, it will be permanently failed. |
+| `:update_period` | Delay (seconds) between reloads of all tasks from the task store  |
+
+### Task Store
+
+A task store is a [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) pattern object that knows how to read and
+write tasks in your data storage (eg. database, CSV file, etc).
 
 ```ruby
-procrastinator.delay(queue: :email, task: EmailGreeting.new('bob@example.com'))
-procrastinator.delay(queue: :cleanup, run_at: Time.now + 3600, task: ClearTempData.new)
+task_store = ReminderStore.new # eg. some SQL task storage class you wrote
+
+Procrastinator.setup do |config|
+   config.define_queue(:reminder, ReminderTask, store: task_store)
+
+   # to use the default CSV storage, provide :store with a string or Pathname
+   config.define_queue(:reminder, ReminderTask, store: '/var/myapp/tasks.csv')
+end
 ```
 
-Read on for more details on each step. 
+A task store is required to implement *all* of the following methods or else it will raise a
+`MalformedPersisterError`:
+
+1. `#read(attributes)`
+
+   Returns a list of hashes from your datastore that match the given attributes hash. The search attributes will be in
+   their final form (eg. `:data` will already be serialized). Each hash must contain the properties listed
+   in [Task Data](#task-data) below.
+
+2. `#create(queue:, run_at:, initial_run_at:, expire_at:, data:)`
+
+   Saves a task in your storage. Receives a hash with [Task Data](#task-data) keys:
+   `:queue`, `:run_at`, `:initial_run_at`, `:expire_at`, and `:data`.
+
+3. `#update(id, new_data)`
 
-### Setup Phase
-The setup phase first defines which queues are available and the persistence strategy to use for reading 
-and writing tasks. It then spins off a sub process for working on each queue within that environment. 
+   Saves the provided full [Task Data](#task-data) hash to your datastore.
 
+4. `#delete(id)`
 
-#### Declaring a Persistence Strategy
-The persister instance is the last step between Procrastinator and your data storage (eg. database). As core Procrastinator is framework-agnostic, it needs you to provide an object that knows how to read and write task data. 
+   Deletes the task with the given identifier from storage
 
-Your [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) class is required to provide the following methods: 
+Procrastinator comes with a simple CSV file task store by default, but you are encouraged to build one that suits your
+situation.
 
-* `#read_tasks(queue_name)` - Returns a list of hashes from your data storage. Each hash must contains the properites of one task, as seen in the *Attributes Hash* 
-* `#create_task(data)` - Creates a task in your datastore. Receives a hash with keys `:queue`, `:run_at`, `:initial_run_at`, `:expire_at`, and `:task` as described in *Attributes Hash* 
-* `#update_task(attributes)` - Receives the Attributes Hash as the data to be saved
-* `#delete_task(id)` - Deletes the task with the given id. 
+_Warning_: Task stores shared between queues **must** be thread-safe if using threaded or daemonized work modes.
 
-If the strategy does not have all of these methods, Procrastinator will explode with a `MalformedPersisterError`  and you will be sad. 
+#### Data Fields
 
-***Attributes Hash***
+These are the data fields for each individual scheduled task. When using the built-in task store, these are the field
+names. If you have a database, use this to inform your table schema.
 
-|  Hash Key         | Type   | Description                                                                           |
-|-------------------|--------| --------------------------------------------------------------------------------------|
-| `:id`             | int    | Unique identifier for this exact task                                                 |
-| `:queue`          | symbol | Name of the queue the task is inside                                                      | 
-| `:run_at`         | int    | Unix timestamp of when to next attempt running the task                               |
-| `:initial_run_at` | int    | Unix timestamp of the original run_at; before the first attempt, this is equal to run_at |
-| `:expire_at`      | int    | Unix timestamp of when to permanently fail the task because it is too late to be useful |
-| `:attempts`       | int    | Number of times the task has tried to run; this should only be > 0 if the task fails  |
-| `:last_fail_at`   | int    | Unix timestamp of when the most recent failure happened                               |
-| `:last_error`     | string | Error message + bracktrace of the most recent failure. May be very long.              |
-| `:task`           | string | YAML-dumped ruby object definition of the task.                                       |
+|  Hash Key         | Type    | Description                                                                             |
+|-------------------|---------| ----------------------------------------------------------------------------------------|
+| `:id`             | int     | Unique identifier for this exact task                                                   |
+| `:queue`          | symbol  | Name of the queue the task is inside                                                    | 
+| `:run_at`         | iso8601 | Time to attempt running the task next. ¹                                      |
+| `:initial_run_at` | iso8601 | Originally requested run_at. Reset when rescheduled.                                    |
+| `:expire_at`      | iso8601 | Time to permanently fail the task because it is too late to be useful |
+| `:attempts`       | int     | Number of times the task has tried to run; this should only be > 0 if the task fails    |
+| `:last_fail_at`   | iso8601 | Unix timestamp of when the most recent failure happened                                 |
+| `:last_error`     | string  | Error message + bracktrace of the most recent failure. May be very long.                |
+| `:data`           | string  | Serialized data accessible in the task instance.                                      |
 
-Notice that the times are all given as unix epoch timestamps. This is to avoid any confusion with timezones, and it is recommended that you store times in this manner for the same reason. 
+> ¹ `nil` indicates that it is permanently failed and will never run, either due to expiry or too many attempts.
 
-#### Defining Queues
-`Procrastinator.setup` requires a block be provided, and that in the block call `#define_queue` be called on the provided environment. Define queue takes a queue name symbol and these properies as a hash
+Data is serialized using `JSON.dump` and `JSON.parse` with **symbolized keys**. It is strongly recommended to only
+supply simple data types (eg. id numbers) to reduce storage space, eliminate redundancy, and reduce the chance of a
+serialization error.
 
- * :timeout - Time, in seconds, after which it should fail tasks in this queue for taking too long to execute.
- * :max_attempts - Maximum number of attempts for tasks in this queue. If attempts is >= max_attempts, the task will be final_failed and marked to never run again
- * :update_period - Delay, in seconds, between refreshing the task list from the persister
- * :max_tasks - The maximum number of tasks to run concurrently with multi-threading. 
+Times are all handled as [ISO8601](https://en.wikipedia.org/wiki/ISO_8601) formatted strings.
 
-**Examples**
-```ruby 
-Procrastinator.setup(some_persister) do |env|
-   env.define_queue(:email)
-   
-   # with all defaults set explicitly
-   env.define_queue(:email, timeout: 3600, max_attempts: 20, update_period: 10, max_tasks: 10)
+#### Default Task Store
+
+Specifying no storage will cause Procrastinator to save tasks using the very basic built-in CSV storage. It is not
+designed for heavy loads, so you should replace it in a production environment.
+
+The file path is defined in `Procrastinator::Store::SimpleCommaStore::DEFAULT_FILE`.
+
+```ruby
+Procrastinator.setup do |config|
+   # this will use the default CSV task store. 
+   config.define_queue(:reminder, ReminderTask)
 end
 ```
-  
-#### Sub-Processes
-Each queue is worked in a separate process.  
-
-<!-- , and each process multi-threaded to handle more than one task at a time. This should help prevent a single task from clogging up the whole queue, or a single queue clogging up the entire system. -->
 
-The sub-processes checks that the parent process is still alive every 5 seconds. If there is no process with the parent's PID, the sub-process will self-exit. 
+#### Shared Task Stores
 
-Sub-processes can be given a name prefix with the process_prefix method: 
+When there are tasks use the same storage, you can wrap them in a `with_store` block.
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.process_prefix('myapp')
+email_task_store = EmailTaskStore.new # eg. some SQL task storage class you wrote
+
+Procrastinator.setup do |config|
+   with_store(email_task_store) do
+      # queues defined inside this block will use the email task store
+      config.define_queue(:welcome, WelcomeTask)
+      config.define_queue(:reminder, ReminderTask)
+   end
+
+   # and this will not use it
+   config.define_queue(:thumbnails, ThumbnailTask)
 end
 ```
 
-###Scheduling Tasks For Later
-Procrastinator will let you be lazy: 
+### Task Container
+
+Whatever is given to `#provide_container` will available to Tasks through the task attribute `:container`.
+
+This can be useful for things like app containers, but you can use it for whatever you like.
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
+Procrastinator.setup do |env|
+   env.provide_container lunch: 'Lasagna'
+
+   # .. other setup stuff ...
 end
 
-procrastinator.delay(task: EmailReminder.new('bob@example.com'))
+# ... and in your task ...
+class LunchTask
+   attr_accessor :logger, :scheduler, :container
+
+   def run
+      logger.info("Today's Lunch is: #{ container[:lunch] }")
+   end
+end
 ```
 
-... unless there are multiple queues defined. Thne you must provide a queue name with your task:
+## Tasks
+
+Your task class is what actually gets run on the task queue. They'll look like this:
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
-   env.define_queue(:cleanup)
-end
 
-procrastinator.delay(:email, task: EmailReminder.new('bob@example.com'))
+class MyTask
+   # These attributes will be assigned by Procrastinator when the task is run.
+   attr_accessor :logger, :scheduler, :container, :data
+
+   # Performs the core work of the task. 
+   def run
+      # ... perform your task ...
+   end
+
+   # ========================================
+   #             OPTIONAL HOOKS
+   #
+   # You can always omit any of the methods
+   # below. Only #run is mandatory.
+   #
+   # ========================================
+
+   # Called after the task has completed successfully. 
+   # Receives the result of #run.
+   def success(run_result)
+      # ...
+   end
+
+   # Called after #run raises any StandardError (or subclass).
+   # Receives the raised error.
+   def fail(error)
+      # ...
+   end
+
+   # Called after either is true: 
+   #   1. the time reported by Time.now is past the task's expire_at time.
+   #   2. the task has failed and the number of attempts is equal to or greater than the queue's `max_attempts`. 
+   #      In this case, #fail will not be executed, only #final_fail. 
+   #
+   # When called, the task will be marked to never be run again.
+   # Receives the raised error.
+   def final_fail(error)
+      # ...
+   end
+end
 ```
 
-You can set when the particular task is to be run and/or when it should expire. Be aware that the task is not guaranteed 
-to run at a precise time; the only promise is that the task will get run some time after `run_at`, unless it's after `expire_at`. 
+### Attribute Accessors
+
+Tasks must provide accessors for `:logger`, `:container`, and `:scheduler`, while the `:data` accessor is semi-optional
+(see below). This is to prevent the tasks from referencing unknown variables once they actually get run.
+
+* `:data`  The data you provided in the call to `#delay`. Any task with a `:data` accessor will require data be passed
+  to `#delay`, and vice versa. See [Task Data](#task-data) for more.
+
+* `:container` The container you've provided in your setup. See [Task Container](#task-container) for more.
+
+* `:logger` The queue's Logger object. See [Logging](#logging) for more.
+
+* `:scheduler` A scheduler object that you can use to schedule new tasks (eg. with `#delay`).
+
+### Errors & Logging
+
+Errors that trigger `#fail` or `#final_fail` are saved to the task storage under columns `last_error` and
+`last_error_at`.
+
+Each queue worker also keeps a logfile log using the Ruby
+[Logger class](https://ruby-doc.org/stdlib-2.7.1/libdoc/logger/rdoc/Logger.html). Log files are named after the queue (
+eg. `log/welcome-queue-worker.log`).
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
+scheduler = Procrastinator.setup do |env|
+   # you can set custom log location and level:
+   env.log_with(directory: '/var/log/myapp/', level: Logger::DEBUG)
+
+   # you can also set the log rotation age or size (see Logger docs for details)
+   env.log_with(shift: 1024, age: 5)
+
+   # use a falsey log level to disable logging entirely:
+   env.log_with(level: false)
 end
+```
 
-# run on or after 1 January 3000
-procrastinator.delay(run_at: Time.new(3000, 1, 1), task: EmailGreeting.new('philip_j_fry@example.com'))
+The logger can be accessed in your tasks by calling `logger` or `@logger`.
 
-# explicitly setting default run_at
-procrastinator.delay(run_at: Time.now, task: EmailReminder.new('bob@example.com'))
+```ruby
+
+class MyTask
+   attr_accessor :logger, :scheduler, :container
+
+   def run
+      logger.info('This task got run. Hooray!')
+   end
+end
 ```
 
-You can also set an `expire_at` deadline on when to run a task. If the task has not been run before `expire_at` is passed, then it will be final-failed the next time it is attempted. Setting `expire_at` to `nil` will mean it will never expire (but may still fail permanently if, say, `max_attempts` is reached).
+Some events are always logged by default:
+
+|event               |level  |
+|--------------------|-------|
+|process started     | INFO  |
+|#success called     | DEBUG |
+|#fail called        | DEBUG |
+|#final_fail called  | DEBUG |
+
+## Scheduling Tasks
+
+To schedule tasks, just call `#delay` on the environment returned from `Procrastinator.setup`:
 
 ```ruby
-procrastinator = Procrastinator.setup(task_persister) do |env|
-   env.define_queue(:email)
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
+   env.define_queue :thumbnail, CreateThumbnail
 end
 
-procrastinator.delay(expire_at: , task: EmailGreeting.new('bob@example.com'))
+# Provide the queue name and any data you want passed in
+scheduler.delay(:reminder, data: 'bob@example.com')
+``` 
+
+If there is only one queue, you may omit the queue name:
+
+```ruby
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
+end
 
-# explicitly setting default
-procrastinator.delay(expire_at: nil, task: EmailGreeting.new('bob@example.com'))
+scheduler.delay(data: 'bob@example.com')
 ```
 
-#### Task Definition
-Like the persister provided to `.setup`, your task is a strategy object that fills in the details of what to do. For this, 
-your task **must provide** a `#run` method:
+### Providing Data
 
- * `#run` - Performs the core work of the task.
+Most tasks need some additional information to complete their work, like id numbers,
 
-You may also optionally provide these hook methods, which are run during different points in the process:
+The `:data` parameter is serialized to string as YAML, so it's better to keep it as simple as possible. For example, if
+you have a database instead of passing in a complex Ruby object, pass in just the primary key and reload it in the
+task's `#run`. This will require less space in your database and avoids obsolete or duplicated information.
 
- * `#success(logger)` - run after the task has completed successfully 
- * `#fail(logger, error)` - run after the task has failed due to `#run` producing a `StandardError` or subclass.
- * `#final_fail(logger, error)` - run after the task has failed for the last time because either:
-    1. the number of attempts is >= the `max_attempts` defined for the queue; or
-    2. the time reported by `Time.now` is past the task's `expire_at` time.
+### Scheduling
 
-If a task reaches `#final_fail` it will be marked to never be run again.
+You can set when the particular task is to be run and/or when it should expire. Be aware that the task is not guaranteed
+to run at a precise time; the only promise is that the task will be attempted *after* `run_at` and before `expire_at`.
 
-***Task Failure & Rescheduling***
+```ruby
+# runs on or after 1 January 3000
+scheduler.delay(:greeting, run_at: Time.new(3000, 1, 1), data: 'philip_j_fry@example.com')
 
-Tasks that fail have their `run_at` rescheduled on an increasing delay **(in seconds)** according to this formula: 
- * 30 + n<sup>4</sup>
-  
-n = the number of attempts 
+# run_at defaults to right now:
+scheduler.delay(:thumbnail, run_at: Time.now, data: 'shut_up_and_take_my_money.gif')
+```
 
-Both failing and final_failing will cause the error timestamp and reason to be stored in `:last_fail_at` and `:last_error`.
+You can also set an `expire_at` deadline. If the task has not been run before `expire_at` is passed, then it will be
+final-failed the next time it would be attempted. Setting `expire_at` to `nil` means it will never expire (but may still
+fail permanently if, say, `max_attempts` is reached).
 
-### Testing With Procrastinator
-Procrastinator uses multi-threading and multi-processing internally, which is a nightmare for testing. Fortunately for you, 
-Test Mode will disable all of that, and rely on your tests to tell it when to tick. 
+```ruby
+# will not run at or after 
+scheduler.delay(:happy_birthday, expire_at: Time.new(2018, 03, 17, 12, 00, '-06:00'), data: 'contact@tenjin.ca')
 
-Enable Test Mode by setting `Procrastinator.test_mode` to `true` before setting up, or by calling enable_test_mode on 
-the procrastination environment. 
+# expire_at defaults to nil:
+scheduler.delay(:greeting, expire_at: nil, data: 'bob@example.com')
+```
+
+### Rescheduling
+
+Call `#reschedule` with the queue name and some identifying information, and then calling #to on that to provide the new
+time.
 
 ```ruby
-# all further calls to `Procrastinator.setup` will produce a procrastination environment where Test Mode is enabled
-Procrastinator.test_mode = true
- 
-# or you can also enable it directly in the setup
-env = Procrastinator.setup do |env|
-   env.enable_test_mode
-    
-   # other settings...
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
 end
+
+scheduler.delay(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
+
+# we can reschedule the task made above 
+scheduler.reschedule(:reminder, data: 'bob@example.com').to(run_at: Time.parse('June 20 12:00'))
+
+# we can also change the expiry time
+scheduler.reschedule(:reminder, data: 'bob@example.com').to(expire_at: Time.parse('June 23 12:00'))
+
+# or both
+scheduler.reschedule(:reminder, data: 'bob@example.com').to(run_at:    Time.parse('June 20 12:00'),
+                                                            expire_at: Time.parse('June 23 12:00'))
 ```
 
-In your tests, tell the procrastinator environment to work off one item from its queues: 
+Rescheduling sets the task's:
+
+* `:run_at` and `:initial_run_at` to a new value, if provided
+* `:expire_at` to a new value if provided.
+* `:attempts` to `0`
+* `:last_error` and `:last_error_at` to `nil`.
+
+Rescheduling will not change `:id`, `:queue` or `:data`. A `RuntimeError` is raised if the runtime is after the expiry.
+
+### Retries
+
+Failed tasks have their `run_at` rescheduled on an increasing delay (in seconds) according to this formula:
+
+> 30 + (number_of_attempts)<sup>4</sup>
+
+Situations that call `#fail` or `#final_fail` will cause the error timestamp and reason to be stored in `:last_fail_at`
+and `:last_error`.
+
+### Cancelling
 
+Call `#cancel` with the queue name and some identifying information to narrow the search to a single task.
+
+```ruby
+scheduler = Procrastinator.setup do |env|
+   env.define_queue :reminder, EmailReminder
+end
+
+scheduler.delay(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
+
+# we can cancel the task made above using whatever we know about it, like the saved :data
+scheduler.reschedule(:reminder, data: 'bob@example.com')
+
+# or multiple attributes
+scheduler.reschedule(:reminder, run_at: Time.parse('June 1'), data: 'bob@example.com')
+
+# you could also use the id number directly, if you have it
+scheduler.reschedule(:reminder, id: 137)
 ```
-# works one task on all queues
-env.act
 
-# provide queue names to works one task on just those queues
-env.act(:cleanup, :email)
+## Working on Tasks
+
+Use the scheduler object returned by setup to `#work` queues **serially**, **threaded**, or **daemonized**.
+
+### Serial Working
+
+Working serially performs a task from each queue directly. There is no multithreading or daemonizing.
+
+Work serially for TDD tests or other situations you need close direct control.
+
+```ruby
+# work just one task, no threading
+scheduler.work.serially
+
+# work the first five tasks
+scheduler.work.serially(steps: 5)
+
+# only work tasks on greeting and reminder queues
+scheduler.work(:greeting, :reminders).serially(steps: 2)
 ```
 
-### Logging
-Logging is crucial to knowing what went wrong in an application after the fact, and because Procrastinator runs workers
-in separate processes, providing a logger instance isn't really an option. 
- 
-Instead, provide a directory that your Procrastinator instance should write log entries into:
+### Threaded Working
+
+Threaded working will spawn a worker thread per queue.
+
+Use threaded working for task queues that should only run while the main application is running. This includes the usual
+caveats around multithreading, so proceed with caution.
 
 ```ruby
-procrastinator = Procrastinator.setup do |env|
-   env.log_dir('log/')
-end
+# work tasks until the application exits
+scheduler.work.threaded
+
+# work tasks for 5 seconds
+scheduler.work.threaded(timeout: 5)
+
+# only work tasks on greeting and reminder queues
+scheduler.work(:greeting, :reminders).threaded
 ```
- 
-Each worker creates its own log named after the queue it is working on (eg. `log/email-queue-worker.log`). The default 
-directory is `./log/`, relative to wherever the application is running. Logging will not occur at all if `log_dir` is 
-assigned a falsey value. 
 
-The logging level can be set using `log_level` and a value from the Ruby standard library 
-[Logger class](https://ruby-doc.org/stdlib-2.2.3/libdoc/logger/rdoc/Logger.html) (eg. `Logger::WARN`, `Logger::DEBUG`, etc.). 
+### Daemonized Working
+
+Daemonized working **consumes the current process** and then proceeds with threaded working in the new daemon.
 
-It logs process start at level `INFO`, process termination due to parent disppearance at level `ERROR` and task hooks 
-`#success`, `#fail`, and `#final_fail` are at a level `DEBUG`. 
+Use daemonized working for production environments, especially in conjunction with daemon monitors
+like [Monit](https://mmonit.com/monit/). Provide a block to daemonized! to get
 
 ```ruby
-procrastinator = Procrastinator.setup do |env|
-   env.log_dir('log/')
-   env.log_level(Logger::INFO) # setting the default explicity
+# work tasks forever as a headless daemon process.
+scheduler.work.daemonized!
+
+# you can specify the new process name and the directory to save the procrastinator.pid file 
+scheduler.work.daemonized!(name: 'myapp-queue', pid_path: '/var/run')
+
+# ... or set the pid file name precisely by giving a .pid path
+scheduler.work.daemonized!(pid_path: '/var/run/myapp.pid')
+
+# only work tasks in the 'greeting' and 'reminder' queues
+scheduler.work(:greeting, :reminders).daemonized!
+
+# supply a block to run code after the daemon subprocess has forked off
+scheduler.work.daemonized! do
+   # this gets run after the daemon is spawned
+   task_store.reconnect_mysql
 end
 ```
 
+Procrastinator endeavours to be thread-safe and support concurrency, but this flexibility allows for many possible
+combinations.
+
+Expected use is a single process with one thread per queue. More complex use is possible but Procrastinator can't
+guarantee concurrency in your Task Store.
+
+#### PID Files
+
+Process ID files are a single-line file that saves the daemon's process ID number. It's saved to the directory given
+by `:pid_dir`. The default location is `pids/` relative to the file that called `#daemonized!`.
+
+## Similar Tools
+
+Procrastinator is a library that exists to enable job queues with flexibility in storage mechanism and minimal
+dependencies. It's neat but it is specifically intended for smaller datasets. Some other approaches include:
+
+### Linux etc: Cron and At
+
+Consider [Cron](https://en.wikipedia.org/wiki/Cron) for tasks that run on a regular schedule.
+
+Consider [At](https://en.wikipedia.org/wiki/At_(command)) for tasks that run once at a particular time.
+
+While neither tool natively supports retry, they can be great solutions for simple situations.
+
+### Gem: Resque
+
+Consider [Resque](https://rubygems.org/gems/resque) for larger datasets (eg. 10,000+ jobs) where performance
+optimization becomes relevant.
+
+### Gem: Rails ActiveJob / DelayedJob
+
+Consider [DelayedJob](https://rubygems.org/gems/delayed_job) for projects that are tightly integrated with Rails and
+fully commit to live in that ecosystem.
+
 ## Contributing
-Bug reports and pull requests are welcome on GitHub at 
+
+Bug reports and pull requests are welcome on GitHub at
 [https://github.com/TenjinInc/procrastinator](https://github.com/TenjinInc/procrastinator).
- 
-This project is intended to be a friendly space for collaboration, and contributors are expected to adhere to the 
+
+This project is intended to be a friendly space for collaboration, and contributors are expected to adhere to the
 [Contributor Covenant](http://contributor-covenant.org) code of conduct.
 
-### Core Developers
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can 
+Play nice.
+
+### Developers
+
+After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can
 also run `bin/console` for an interactive prompt that will allow you to experiment.
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the 
-version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, 
+To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
+version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
 push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
 ## License
+
 The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).