postburner 0.9.0.rc.1 → 1.0.0.pre.2

data/README.md

@@ -1,771 +1,1346 @@
 # Postburner
-An ActiveRecord layer on top of [Backburner](https://github.com/nesquena/backburner) for inspecting and auditing the
-queue, especially for delayed jobs. It isn't meant to be outperform other queues, but be safe (and inspectable).
 
-It is meant to be complementary to [Backburner](https://github.com/nesquena/backburner). Use Backburner as the default
-ActiveJob processor for mailers, active storage, and the like. Use a
-`Postburner::Job` for things that you want to track. See [Comparison to Backburner](#comparison-to-backburner) for more.
+Fast Beanstalkd-backed job queue with **optional PostgreSQL audit trail** for ActiveJob.
 
-Postburner meant to be a replacement/upgrade for [Que](https://github.com/que-rb/que).
-If you need something faster, check out Que - we love it! Postburner is built for a slightly different purpose: to be
-simple, safe, and inspectable. All of which Que has (or can be added), but are included by default with some architecture
-differences. See [Comparison to Que](#comparison-to-que) for more.
+Postburner provides dual-mode job execution:
+- **Default jobs**: Fast execution via Beanstalkd only (no PostgreSQL overhead)
+- **Tracked jobs**: Full audit trail with logs, timing, errors, and statistics
 
-## Queue Strategies
+Built for production environments where you want fast background processing for most jobs, but comprehensive auditing for critical operations.
 
-Postburner uses different strategies to control how jobs are queued and executed:
+## Features
 
-| Strategy | When to Use | Behavior | Requires Beanstalkd |
-|----------|-------------|----------|---------------------|
-| **NiceQueue** (default) | Production | Async via Beanstalkd, gracefully re-queues premature jobs | Yes |
-| **Queue** (suggested) | Production | Async via Beanstalkd, raises error on premature execution | Yes |
-| **TestQueue** | Testing with explicit time control | Inline execution, raises error for scheduled jobs | No |
-| **ImmediateTestQueue** | Testing with automatic time travel | Inline execution, auto time-travels for scheduled jobs | No |
-| **NullQueue** | Batch processing / deferred execution | Jobs created but not queued, manual execution via `Postburner::Job.perform(id)` | No |
+- **ActiveJob native** - Works seamlessly with Rails, ActionMailer, ActiveStorage
+- **Dual-mode execution** - Default or tracked (database backed)
+- **Rich audit trail** - Logs, timing, errors, retry tracking (tracked jobs only)
+- **ActiveRecord** - Query jobs with ActiveRecord (Tracked jobs only)
+- **Beanstalkd** - Fast, reliable queue separate from your database, peristent storage
+- **Process isolation** - Forking workers with optional threading for throughput
+- **Test-friendly** - Inline execution without Beanstalkd in tests
+
+## Quick Start
+
+**Installation:**
+
+```bash
+sudo apt-get install beanstalkd # OR brew install beanstalkd
+
+# Start beanstalkd (in-memory only)
+beanstalkd -l 127.0.0.1 -p 11300
+
+# OR with persistence (recommended for production)
+mkdir -p /var/lib/beanstalkd
+beanstalkd -l 127.0.0.1 -p 11300 -b /var/lib/beanstalkd
+```
 
 ```ruby
-# Switch strategies
-Postburner.nice_async_strategy!           # Default production (NiceQueue)
-Postburner.async_strategy!                # Strict production (Queue)
-Postburner.inline_test_strategy!          # Testing (TestQueue)
-Postburner.inline_immediate_test_strategy! # Testing with time travel (ImmediateTestQueue)
-Postburner.null_strategy!                 # Deferred execution (NullQueue)
+# Gemfile
+gem 'postburner', '~> 1.0.0.pre.1'
 ```
 
-## Usage
+```bash
+bundle install
+rails generate postburner:install
+rails db:migrate
+```
+
+**Configuration:**
 
 ```ruby
-class RunDonation < Postburner::Job
-  queue 'critical'
-  queue_priority 0 # 0 is highest priority
-  queue_max_job_retries 0 # don't retry
+# config/application.rb
+config.active_job.queue_adapter = :postburner
+```
 
-  def perform(args)
-    # do long tasks here
-    # also have access to self.args
+```yaml
+# config/postburner.yml
+production:
+  beanstalk_url: <%= ENV['BEANSTALK_URL'] || 'beanstalk://localhost:11300' %>
+  worker_type: simple
+  queues:
+    default: {}
+    critical: {}
+    mailers: {}
+```
+
+**Usage:**
+
+```ruby
+# Default job (fast, no PostgreSQL overhead)
+class SendEmail < ApplicationJob
+  def perform(user_id)
+    UserMailer.welcome(user_id).deliver_now
   end
 end
 
-# RunDonation#create! is the `ActiveRecord` method, so it returns a model,
-# you can manipulate, add columns to the table to, reference with foreign
-# keys, etc.
-job = RunDonation.create!(args: {donation_id: 123})
-# NOTE Make sure use use an after_commit or after_save_commit, etc to avoid
-#      any race conditions of Postburner trying to process before a required
-#      database mutation is commited.
-job.queue!
-=> {:status=>"INSERTED", :id=>"1139"}
+# Default job with Beanstalkd configuration
+class SendEmail < ApplicationJob
+  include Postburner::Beanstalkd
+
+  queue_priority 100  # Custom priority
+  queue_ttr 300       # 5 minute timeout
+
+  def perform(user_id)
+    UserMailer.welcome(user_id).deliver_now
+  end
+end
+
+# Tracked job (full audit trail, includes Beanstalkd automatically)
+class ProcessPayment < ApplicationJob
+  include Postburner::Tracked  # ← Opt-in to tracking (includes Beanstalkd)
 
-# queue for later with `:at`
-RunDonation.create!(args: {donation_id: 123}).queue! at: Time.zone.now + 2.days
-=> {:status=>"INSERTED", :id=>"1140"}
+  queue_priority 0  # Highest priority
+  queue_ttr 600     # 10 minute timeout
+
+  def perform(payment_id)
+    log "Processing payment #{payment_id}"
+    Payment.find(payment_id).process!
+    log! "Payment processed successfully"
+  end
+end
 
-# queue for later with `:delay`
-#
-# `:delay` takes priority over `:at`takes priority over `:at` because the
-# beanstalkd protocol uses uses `delay`
-RunDonation.create!(args: {donation_id: 123}).queue! delay: 1.hour
-=> {:status=>"INSERTED", :id=>"1141"}
+# Run worker
+bin/postburner --config config/postburner.yml --env production
 ```
 
-### Backburner Queue Configuration
+## Table of Contents
 
-Postburner jobs inherit from `Backburner::Queue`, giving you access to Backburner's queue configuration methods. These are **optional** - Backburner provides sensible defaults.
+- [Why Beanstalkd?](#why-beanstalkd)
+- [Installation](#installation)
+- [Usage](#usage)
+  - [Standard Jobs](#standard-jobs)
+  - [Tracked Jobs](#tracked-jobs)
+  - [Postburner::Job](#postburnerjob)
+- [Workers](#workers)
+- [Configuration](#configuration)
+- [Queue Strategies](#queue-strategies)
+- [Testing](#testing)
+- [Job Management](#job-management)
+- [Beanstalkd Integration](#beanstalkd-integration)
+- [Web UI](#web-ui)
 
-```ruby
-class CriticalJob < Postburner::Job
-  # Specify which Beanstalkd tube to use (default: 'backburner-jobs')
-  queue 'critical'
+## Why Beanstalkd?
 
-  # Set job priority - lower numbers = higher priority (default: 65536)
-  # Most urgent priority is 0
-  queue_priority 0
+Beanstalkd is a simple, fast, and reliable queue system. It is a good choice for production environments where you want fast background processing for most jobs, but comprehensive auditing for critical operations.
 
-  # Set maximum retry attempts before burying (default: from Backburner config)
-  queue_max_job_retries 3
+The [protocol](https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt) reads more like a README than a protocol. Check it out and you will instantly understand how it works.
 
-  # Set job timeout in seconds (default: from Backburner config)
-  queue_respond_timeout 300
+### Binlogs
 
-  def perform(args)
-    # your job logic
-  end
-end
+Beanstalkd lets you persist jobs to disk in case of a crash or restart. Just restart beanstalkd and the jobs will be back in the queue.
+
+**Setup:**
+
+```bash
+# Create binlog directory
+sudo mkdir -p /var/lib/beanstalkd
+sudo chown beanstalkd:beanstalkd /var/lib/beanstalkd  # If running as service
+
+# Start beanstalkd with binlog persistence
+beanstalkd -l 127.0.0.1 -p 11300 -b /var/lib/beanstalkd
+```
+
+**Configuration options:**
+
+```bash
+# Basic persistence
+beanstalkd -b /var/lib/beanstalkd
+
+# With custom binlog file size (default varies)
+beanstalkd -b /var/lib/beanstalkd -s 10485760  # 10MB per binlog file
+
+# Reduce fsync frequency for better performance (trades safety for speed)
+beanstalkd -b /var/lib/beanstalkd -f 200  # fsync at most every 200ms (default: 50ms)
+
+# Never fsync (maximum performance, higher risk of data loss on power failure)
+beanstalkd -b /var/lib/beanstalkd -F
 ```
 
-**Common configurations:**
+**Options:**
+- `-b <dir>` - Enable binlog persistence in specified directory
+- `-s <bytes>` - Maximum size of each binlog file (requires `-b`)
+- `-f <ms>` - Call fsync at most once every N milliseconds (requires `-b`, default: 50ms)
+- `-F` - Never call fsync (requires `-b`, maximum performance but higher data loss risk)
+
+**Note:** The fsync options control the trade-off between performance and safety. A power failure could result in loss of jobs added within the fsync interval.
+
+### Priority
+
+Priority controls the order in which jobs are processed from the queue. Beanstalkd always processes the highest priority (lowest number) jobs first.
+
+**Priority Range:** 0 to 4,294,967,295 (unsigned 32-bit integer)
+- `0` = Highest priority (processed first)
+- `4,294,967,295` = Lowest priority (processed last)
+- Default: `65536` (configurable in `config/postburner.yml`)
+
+When a worker reserves a job, Beanstalkd returns the highest priority job available in the tube. Jobs with the same priority are processed in FIFO order.
+
+**ActiveJob with Postburner::Beanstalkd:**
 
 ```ruby
-# High priority job with no retries
-class PaymentJob < Postburner::Job
-  queue 'payments'
-  queue_priority 0
-  queue_max_job_retries 0  # Don't retry - handle failures manually
-end
+class ProcessPayment < ApplicationJob
+  include Postburner::Beanstalkd
 
-# Background job with retries
-class EmailJob < Postburner::Job
-  queue 'emails'
-  queue_priority 1000  # Lower priority
-  queue_max_job_retries 5  # Retry up to 5 times
+  queue_as :critical
+  queue_priority 0  # Highest priority - processed immediately
 end
 
-# Long-running job
-class DataExportJob < Postburner::Job
-  queue 'exports'
-  queue_respond_timeout 3600  # 1 hour timeout
+class SendEmail < ApplicationJob
+  include Postburner::Beanstalkd
+
+  queue_as :mailers
+  queue_priority 1000  # Lower priority - processed after critical jobs
 end
 ```
 
-These methods come from [Backburner](https://github.com/nesquena/backburner#configure-backburner) and work seamlessly with Postburner. See Backburner's documentation for all available options.
-
-### Mailers
+**Postburner::Job:**
 
 ```ruby
-j = Postburner::Mailer.
-  delivery(UserMailer, :welcome)
-  .with(name: 'Freddy')
+class CriticalJob < Postburner::Job
+  queue 'critical'
+  queue_priority 0  # Highest priority
 
-j.queue!
-=> {:status=>"INSERTED", :id=>"1139"}
-```
+  def perform(args)
+    # Critical business logic
+  end
+end
 
-### Job Management Methods
+class BackgroundTask < Postburner::Job
+  queue 'default'
+  queue_priority 5000  # Lower priority
 
-Postburner provides several methods for managing jobs in Beanstalkd and the database:
+  def perform(args)
+    # Non-urgent background work
+  end
+end
+```
 
-#### delete! - Remove from Beanstalkd only
+**Recommended Priority Ranges:**
 
-Deletes the job from the Beanstalkd queue but **keeps the database record**. Use this when you want to cancel a queued job while maintaining its audit trail.
+| Priority Range | Use Case | Examples |
+|---------------|----------|----------|
+| 0-256 | Critical business operations | Payments, transactions, real-time notifications |
+| 256-4096 | High priority tasks | Password resets, order confirmations |
+| 4096-65536 | Standard background jobs | Email sending, data exports (65536 is default) |
+| 65536-262144 | Low priority maintenance | Cache warming, log cleanup |
+| 262144+ | Deferred/bulk operations | Analytics, batch reports |
 
-```ruby
-job = MyJob.create!(args: {})
-job.queue!(delay: 1.hour)
-job.bkid  # => 12345
+**Configuration:**
 
-# Remove from Beanstalkd but keep in database
-job.delete!
+Set default priority in `config/postburner.yml`:
 
-job.reload
-job.bkid  # => 12345 (still has bkid)
-job.persisted?  # => true (still in database)
-# Job won't execute, but you can still inspect it
+```yaml
+production:
+  default_priority: 65536  # Default for jobs without explicit priority
+  default_ttr: 300
 ```
 
-#### remove! - Soft delete (Beanstalkd + removed_at)
-
-Removes the job from Beanstalkd **and** sets `removed_at` timestamp. The database record is preserved for audit trails. This is the recommended way to cancel jobs.
+**Dynamic Priority (Postburner::Job only):**
 
 ```ruby
-job = MyJob.create!(args: {})
-job.queue!(delay: 1.hour)
+class ProcessOrder < Postburner::Job
+  queue 'orders'
+  queue_priority 100  # Default
 
-# Soft delete - removes from queue and marks as removed
-job.remove!
+  before_enqueue :adjust_priority
+
+  def perform(args)
+    order = Order.find(args['order_id'])
+    order.process!
+  end
 
-job.reload
-job.removed_at  # => 2025-11-01 12:34:56 UTC
-job.persisted?  # => true (still in database)
-# Job won't execute and is marked as removed
+  private
+
+  def adjust_priority
+    if args['urgent']
+      self.queue_priority = 0  # Override to highest priority
+    end
+  end
+end
+
+# Usage
+ProcessOrder.create!(args: { 'order_id' => 123, 'urgent' => true }).queue!
 ```
 
-#### destroy / destroy! - Delete from both
+### Time to Run (TTR)
 
-Destroys the database record **and** removes from Beanstalkd via `before_destroy` callback. Use when you want to completely eliminate the job.
+TTR (Time-to-Run) is the maximum duration in seconds that a worker has to process a reserved job before Beanstalkd automatically releases it back to the ready queue.
+
+1. Worker reserves a job from Beanstalkd
+2. TTR countdown begins immediately upon reservation
+3. If job completes within TTR, worker deletes/buries the job (success)
+4. If TTR expires before completion, Beanstalkd automatically releases the job back to ready queue
+5. Another worker can then reserve and process the same job
+
+This mechanism protects against workers crashing or hanging—jobs won't be stuck indefinitely.
+
+**TTR Range:** 1 to 4,294,967,295 seconds (unsigned 32-bit integer)
+- Minimum: `1` second (Beanstalkd silently increases 0 to 1)
+- Default: `300` seconds (5 minutes, configurable in `config/postburner.yml`)
+
+**ActiveJob with Postburner::Beanstalkd:**
 
 ```ruby
-job = MyJob.create!(args: {})
-job.queue!(delay: 1.hour)
+class ProcessPayment < ApplicationJob
+  include Postburner::Beanstalkd
 
-# Completely remove job from database and Beanstalkd
-job.destroy!
+  queue_as :critical
+  queue_priority 0
+  queue_ttr 300  # 5 minutes to complete
+end
 
-# Job no longer exists in database or Beanstalkd
-```
+class QuickEmail < ApplicationJob
+  include Postburner::Beanstalkd
 
-#### kick! - Retry buried jobs
+  queue_as :mailers
+  queue_ttr 60  # 1 minute for fast email jobs
+end
 
-Moves a buried job back into the ready queue. Buried jobs are those that failed too many times or were explicitly buried by Beanstalkd.
+class LongRunningReport < ApplicationJob
+  include Postburner::Beanstalkd
 
-```ruby
-# Job failed and was buried
-job.beanstalk_job.stats['state']  # => 'buried'
+  queue_as :reports
+  queue_ttr 3600  # 1 hour for complex reports
+end
+```
 
-# Move it back to ready queue for retry
-job.kick!
+**Postburner::Job:**
 
-job.beanstalk_job.stats['state']  # => 'ready'
+```ruby
+class DataImport < Postburner::Job
+  queue 'imports'
+  queue_ttr 1800  # 30 minutes (equivalent to queue_ttr)
+
+  def perform(args)
+    # Long-running import logic
+  end
+end
 ```
 
-**Summary:**
-- **`delete!`** - Removes from Beanstalkd, keeps full database record (keeps `bkid`)
-- **`remove!`** - Removes from Beanstalkd, marks `removed_at`, keeps database record (**recommended for canceling jobs**)
-- **`destroy!`** - Removes from both Beanstalkd and database (complete deletion)
-- **`kick!`** - Moves buried job back to ready queue (for retrying failed jobs)
+**Extending TTR with `extend!` (Touch Command):**
 
-### [Beaneater](https://github.com/beanstalkd/beaneater) and [beanstalkd](https://raw.githubusercontent.com/beanstalkd/beanstalkd/master/doc/protocol.txt) Integration
+For jobs that may take longer than expected, you can extend the TTR dynamically using `extend!`. This calls Beanstalkd's `touch` command, which resets the TTR countdown.
 
-Access Beanstalkd directly for advanced queue operations:
+**Available for:**
+- `Postburner::Job` subclasses (always available via `bk.extend!`)
+- `Postburner::Tracked` ActiveJob classes (includes `extend!` method)
 
 ```ruby
-# Get the beanstalkd job id
-job.bkid
-=> 1104
+class ProcessImport < ApplicationJob
+  include Postburner::Tracked  # Includes Beanstalkd and enables extend!
 
-# Get the beaneater job object - call any beaneater methods
-job.beanstalk_job
+  queue_as :imports
+  queue_ttr 300  # 5 minutes initial TTR
 
-# Get beanstalkd stats for this job
-job.beanstalk_job.stats
-=> {"id"=>1104, "tube"=>"critical", "state"=>"ready", "pri"=>0, ...}
-
-# Get a cached Backburner connection
-c = Postburner.connection
-c.beanstalk.tubes.to_a
-c.beanstalk.tubes.to_a.map{|t| c.tubes[t.name].peek(:buried)}
-c.beanstalk.tubes['ss.development.caching'].stats
-c.beanstalk.tubes['ss.development.caching'].peek(:buried).kick
-c.beanstalk.tubes['ss.development.caching'].kick(3)
-c.close
-
-# Automatically close connection (recommended)
-Postburner.connected do |connection|
-  # do stuff with connection
-  connection.tubes['my.tube'].stats
+  def perform(file_id)
+    file = ImportFile.find(file_id)
+
+    file.each_batch(size: 100) do |batch|
+      batch.each do |row|
+        process_row(row)
+      end
+
+      extend!  # Reset TTR to 300 seconds from now
+      log "Processed batch, extended TTR"
+    end
+  end
 end
-# Connection automatically closed
 ```
 
-Read about the [beanstalkd protocol](https://raw.githubusercontent.com/beanstalkd/beanstalkd/master/doc/protocol.txt).
+**For Postburner::Job (via `bk` accessor):**
 
-### Basic model fields
 ```ruby
-# ActiveRecord primary key
-job.id
+class LargeDataProcessor < Postburner::Job
+  queue 'processing'
+  queue_ttr 600  # 10 minutes
 
-# int id of the beankstalkd job
-job.bkid
+  def perform(args)
+    dataset = Dataset.find(args['dataset_id'])
 
-# string uuid
-job.sid
+    dataset.each_chunk do |chunk|
+      process_chunk(chunk)
 
-# ActiveRecord STI job subtype
-job.type
+      bk.extend!  # Reset TTR via beanstalkd job accessor
+      log "Chunk processed, TTR extended"
+    end
+  end
+end
+```
+
+**`extend!` and Beanstalkd `touch`**
+
+- Calls Beanstalkd's `touch` command on the reserved job
+- Resets the TTR countdown to the original TTR value from the current moment
+- Example: Job has `queue_ttr 300`. After 200 seconds, calling `extend!` gives you another 300 seconds (not just 100)
+- Can be called multiple times during job execution
+- Useful for iterative processing where each iteration is quick but total time is unpredictable
 
-# jsonb arguments for use in job
-job.args
+**Configuration:**
 
-# time job should run - not intended to be changed
-# TODO run_at should be readonly after create
-job.run_at
+Set default TTR in `config/postburner.yml`:
+
+```yaml
+production:
+  default_priority: 65536
+  default_ttr: 300  # 5 minutes default for all jobs
 ```
 
-### Job statistics
+**Recommended TTR Values:**
+
+| Job Type | TTR | Reasoning |
+|----------|-----|-----------|
+| Email sending | 60-120s | Network operations, should be fast |
+| API calls | 30-60s | External services, fast or timeout |
+| File processing | 600-1800s | Variable based on file size, use `extend!` |
+| Reports | 1800-3600s | Complex queries and aggregations |
+| Imports/Exports | 3600s+ | Large datasets, use `extend!` in loops |
+
+**Best Practices:**
+
+1. **Set realistic TTR values** based on expected job duration plus buffer
+2. **Use `extend!` for iterative work** where total time is unpredictable but each iteration is bounded
+3. **Don't set TTR too high** - it delays recovery from crashed workers
+4. **Monitor TTR timeouts** - frequent timeouts indicate jobs need more time or optimization
+5. **For Default jobs** (ActiveJob without `Postburner::Beanstalkd`), include the module to set custom TTR
+
+**What happens on TTR timeout:**
+
 ```ruby
-# when job was inserted into beankstalkd
-job.queued_at
+# Job starts processing
+job = ProcessImport.perform_later(file_id)
+
+# Worker crashes or hangs after 200 seconds
+# At 300 seconds (TTR), Beanstalkd automatically:
+# 1. Releases job back to ready queue
+# 2. Increments the job's reserve count
+# 3. Makes job available for another worker
 
-# last time attempted
-job.attempting_at
+# Another worker picks up the job and retries
+# (ActiveJob retry_on/discard_on handlers still apply)
+```
 
-# last time processing started
-job.processing_at
+## Installation
 
-# when completed
-job.processed_at
+### 1. Install Beanstalkd
 
-# when removed, may be nil
-job.removed_at
+First [install beanstalkd](https://beanstalkd.github.io/download.html):
 
-# lag in ms from run_at/queued_at to attempting_at
-job.lag
+```bash
+sudo apt-get install beanstalkd
+brew install beanstalkd # for macOS/Linux
 
-# duration of processing in ms
-job.duration
+# Start beanstalkd (in-memory only)
+beanstalkd -l 127.0.0.1 -p 11300
 
-# number of attempts
-job.attempt_count
+# OR with persistence (recommended for production)
+mkdir -p /var/lib/beanstalkd
+beanstalkd -l 127.0.0.1 -p 11300 -b /var/lib/beanstalkd
+```
 
-# number of errors (length of errata)
-job.error_count
+### 2. Add Gem
 
-# number of log entries (length of logs)
-job.log_count
+```ruby
+# Gemfile
+gem 'postburner', '~> 1.0.0.pre.1'
+```
 
-# array of attempting_at times
-job.attempts
+```bash
+bundle install
+```
 
-# array of errors
-job.errata
+### 3. Install Migration
 
-# array of log messages
-job.logs
+```bash
+rails generate postburner:install
+rails db:migrate
 ```
 
-### Job logging and exceptions
+This creates the `postburner_jobs` table for tracked jobs.
 
-Optionally, you can:
-1. Add log messages to the job during processing to `logs`
-1. Add log your own exceptions to `errata`
+### 4. Configure ActiveJob
 
 ```ruby
-class RunDonation < Postburner::Job
-  queue 'critical'
-  queue_priority 0 # 0 is highest priority
-  queue_max_job_retries 0 # don't retry
+# config/application.rb
+config.active_job.queue_adapter = :postburner
+```
 
-  def perform(args)
-    # log at task, defaults to `:info`, but `:debug`, `:warning`, `:error`
-    log "Log bad condition...", level: :error
+### 5. Create Worker Configuration
 
-    begin
-      # danger
-    rescue Exception => e
-      log_exception e
-    end
-  end
-end
+```bash
+cp config/postburner.yml.example config/postburner.yml
 ```
 
-### Callbacks
+Edit `config/postburner.yml` for your environment (see [Configuration](#configuration)).
+
+## Usage
 
-Postburner provides ActiveJob-style lifecycle callbacks for job execution:
+### Default Jobs
+
+Default jobs execute quickly via Beanstalkd without PostgreSQL overhead. Perfect for emails, cache warming, notifications, etc.
 
 ```ruby
-class ProcessPayment < Postburner::Job
-  queue 'critical'
+class SendWelcomeEmail < ApplicationJob
+  queue_as :mailers
 
-  before_enqueue :validate_payment
-  after_enqueue :send_confirmation
+  def perform(user_id)
+    UserMailer.welcome(user_id).deliver_now
+  end
+end
 
-  before_attempt :log_attempt
-  after_attempt :update_metrics
+# Enqueue immediately
+SendWelcomeEmail.perform_later(123)
 
-  before_processing :acquire_lock
-  after_processing :release_lock
+# Enqueue with delay
+SendWelcomeEmail.set(wait: 1.hour).perform_later(123)
 
-  after_processed :send_receipt  # Only runs on success
+# Enqueue at specific time
+SendWelcomeEmail.set(wait_until: Date.tomorrow.noon).perform_later(123)
+```
 
-  def perform(args)
-    payment = Payment.find(args['payment_id'])
-    payment.process!
-  end
+**Overview:**
+- Fast execution (no database writes)
+- Low overhead
+- Standard ActiveJob retry_on/discard_on support
+- No audit trail
+- No logging or statistics
 
-  private
+#### Configuring Beanstalkd Priority and TTR
 
-  def validate_payment
-    raise "Invalid payment" unless args['payment_id'].present?
-  end
+For default jobs that need custom Beanstalkd configuration, include `Postburner::Beanstalkd`:
 
-  def send_confirmation
-    PaymentMailer.queued(args['payment_id']).deliver_later
-  end
+```ruby
+class SendWelcomeEmail < ApplicationJob
+  include Postburner::Beanstalkd
 
-  def send_receipt
-    PaymentMailer.receipt(args['payment_id']).deliver_later
+  queue_as :mailers
+  queue_priority 100    # Lower = higher priority (0 is highest)
+  queue_ttr 300         # Time-to-run in seconds (5 minutes)
+
+  def perform(user_id)
+    UserMailer.welcome(user_id).deliver_now
   end
 end
 ```
 
-**Available callbacks:**
-- `before_enqueue`, `around_enqueue`, `after_enqueue` - Run when `queue!` is called
-- `before_attempt`, `around_attempt`, `after_attempt` - Run on every execution attempt (including retries)
-- `before_processing`, `around_processing`, `after_processing` - Run during job execution
-- `after_processed` - Only runs after successful completion (not on errors)
+**Configuration options:**
+- `queue_priority` - Beanstalkd priority (0-4294967295, lower = higher priority)
+- `queue_ttr` - Time-to-run in seconds before job times out
+
+Jobs without `Postburner::Beanstalkd` use defaults from `config/postburner.yml`:
+- `default_priority: 65536`
+- `default_ttr: 300`
 
-### Optionally, mount the engine
+### Tracked Jobs
+
+Tracked jobs store full execution details in PostgreSQL, providing comprehensive audit trails (i.e. logging, timing, errors, retry tracking) for critical operations.
+
+**Note:** `Postburner::Tracked` automatically includes `Postburner::Beanstalkd`, giving you access to `queue_priority`, `queue_ttr`, `bk`, and `extend!`.
 
 ```ruby
-mount Postburner::Engine => "/postburner"
+class ProcessPayment < ApplicationJob
+  include Postburner::Tracked  # ← Opt-in to PostgreSQL tracking (includes Beanstalkd)
 
-# mount only for development inspection
-mount Postburner::Engine => "/postburner" if Rails.env.development?
-```
+  queue_as :critical
+  queue_priority 0              # Highest priority (Beanstalkd included automatically)
+  queue_ttr 600                 # 10 minute timeout
+  retry_on StandardError, wait: :exponentially_longer, attempts: 5
 
-[Open the controller](https://guides.rubyonrails.org/engines.html#overriding-models-and-controllers)
-to add your own authentication or changes - or just create your own routes, controllers, and views.
+  def perform(payment_id)
+    payment = Payment.find(payment_id)
 
-[Override the views](https://guides.rubyonrails.org/engines.html#overriding-views) to make them
-prettier - or follow the suggestion above and use your own.
+    log "Starting payment processing for $#{payment.amount}"
 
-## Testing
+    begin
+      payment.charge!
+      log! "Payment charged successfully"
+    rescue PaymentError => e
+      log_exception!(e)
+      raise
+    end
+
+    log "Payment complete", level: :info
+  end
+end
+```
 
-Postburner provides multiple testing strategies to make your tests fast and predictable without requiring Beanstalkd. Tests automatically use `TestQueue` when Rails test mode is detected.
+**Tracking provides:**
+- Complete execution history (queued_at, processing_at, processed_at)
+- Custom logs with `log` and `log!`
+- Exception tracking with `log_exception()` and `log_exception!()`
+- Timing statistics (lag, duration)
+- Retry attempts tracking
+- Query with ActiveRecord
+- Foreign key relationships
+- Beanstalkd operations via `bk` accessor
+- TTR extension via `extend!` method
 
-### Rails and ActiveJob Test Configuration
+**Query tracked jobs:**
 
-Rails provides several configuration options that affect how background jobs behave in test environments:
+```ruby
+# Find by state
+Postburner::TrackedJob.where(processed_at: nil)
+Postburner::TrackedJob.where.not(removed_at: nil)
+
+# Find with errors
+Postburner::TrackedJob.where("error_count > 0")
+
+# Statistics
+Postburner::TrackedJob.average(:duration)  # Average execution time
+Postburner::TrackedJob.maximum(:lag)       # Worst lag
+
+# Inspect execution
+job = Postburner::TrackedJob.last
+job.logs      # Array of log entries with timestamps
+job.errata    # Array of exceptions with backtraces
+job.attempts  # Array of attempt timestamps
+job.duration  # Execution time in milliseconds
+job.lag       # Queue lag in milliseconds
+```
 
-#### ActiveJob Queue Adapter
+### Direct Postburner::Job Usage
 
-The `config.active_job.queue_adapter` setting controls how ActiveJob queues jobs. Rails supports several built-in adapters:
+Direct `Postburner::Job` subclasses are **always tracked**:
 
 ```ruby
-# config/environments/test.rb (or config/application.rb)
+class RunDonation < Postburner::Job
+  queue 'critical'
+  queue_priority 0
+  queue_max_job_retries 0
 
-# :test adapter (Rails default for test environment)
-# - Jobs are NOT executed automatically
-# - Jobs accumulate in ActiveJob::Base.queue_adapter.enqueued_jobs
-# - You must explicitly call perform_enqueued_jobs to execute them
-config.active_job.queue_adapter = :test
+  def perform(args)
+    donation = Donation.find(args['donation_id'])
+    donation.process!
+    log "Processed donation #{donation.id}"
+  end
+end
 
-# :inline adapter
-# - Jobs execute immediately in the same process
-# - No queueing, synchronous execution like Postburner's test strategies
-config.active_job.queue_adapter = :inline
+# Create and enqueue
+job = RunDonation.create!(args: { 'donation_id' => 123 })
+job.queue!
 
-# :async adapter
-# - Jobs execute asynchronously in a thread pool
-# - Not recommended for tests due to race conditions
-config.active_job.queue_adapter = :async
+# With delay
+job.queue!(delay: 1.hour)
 
-# :backburner adapter (for production use with Postburner)
-# - Requires Beanstalkd to be running
-# - Jobs queued to Beanstalkd tubes
-config.active_job.queue_adapter = :backburner
+# At specific time
+job.queue!(at: 2.days.from_now)
 ```
 
-#### ActiveJob Test Helpers
+#### Instance-Level Queue Configuration
 
-When using the `:test` adapter, Rails provides test helpers in `ActiveJob::TestHelper`:
+Override queue priority and TTR per job instance for dynamic behavior:
 
 ```ruby
-# test/test_helper.rb
-require 'test_helper'
+# Set priority during creation
+job = RunDonation.create!(
+  args: { 'donation_id' => 123 },
+  queue_priority: 1500,  # Override class-level priority
+  queue_ttr: 300         # Override class-level TTR
+)
+job.queue!
 
-class MyTest < ActiveSupport::TestCase
-  include ActiveJob::TestHelper
+# Set dynamically after creation
+job = RunDonation.create!(args: { 'donation_id' => 456 })
+job.queue_priority = 2000
+job.queue_ttr = 300
+job.queue!
 
-  test "job is enqueued" do
-    assert_enqueued_with(job: MyJob, args: [{user_id: 123}]) do
-      MyJob.perform_later(user_id: 123)
-    end
+# Use in before_enqueue callback for conditional behavior
+class ProcessOrder < Postburner::Job
+  queue 'orders'
+  queue_priority 100   # Default priority
+  queue_ttr 120  # Default TTR
+
+  before_enqueue :set_priority_based_on_urgency
+
+  def perform(args)
+    order = Order.find(args['order_id'])
+    order.process!
   end
 
-  test "job is performed" do
-    perform_enqueued_jobs do
-      MyJob.perform_later(user_id: 123)
+  private
+
+  def set_priority_based_on_urgency
+    if args['express_shipping']
+      self.queue_priority = 0    # High priority for express orders
+      self.queue_ttr = 600       # Allow 10 minutes to complete
+    else
+      self.queue_priority = 1000 # Low priority for standard orders
+      self.queue_ttr = 120       # Standard 2 minute timeout
     end
-    # Job has now executed
   end
 end
+
+# Express order gets high priority automatically
+ProcessOrder.create!(args: { 'order_id' => 789, 'express_shipping' => true }).queue!
 ```
 
-**Available helpers:**
-- `perform_enqueued_jobs` - Executes all enqueued jobs
-- `assert_enqueued_jobs` - Asserts number of jobs enqueued
-- `assert_enqueued_with` - Asserts specific job was enqueued with args
-- `assert_no_enqueued_jobs` - Asserts no jobs were enqueued
-- `assert_performed_jobs` - Asserts number of jobs performed
-- `queue_adapter.enqueued_jobs` - Array of enqueued jobs for inspection
+**Overview:**
+- Same as Tracked jobs
+- Access the ActiveRecord Postburner::Job directly.
+
+## Workers
+
+Postburner uses named worker configurations to support different deployment patterns. Each worker can have different fork/thread settings and process different queues, enabling flexible production deployments.
+
+### Named Workers Configuration
+
+Configure multiple named workers with different concurrency profiles:
+
+```yaml
+production:
+  beanstalk_url: <%= ENV['BEANSTALK_URL'] %>
+
+  workers:
+    # Heavy, memory-intensive jobs - more processes, fewer threads
+    imports:
+      default_forks: 4
+      default_threads: 1
+      default_gc_limit: 500
+      queues:
+        - imports
+        - data_processing
+
+    # General jobs - fewer processes, many threads
+    general:
+      default_forks: 2
+      default_threads: 100
+      default_gc_limit: 5000
+      queues:
+        - default
+        - mailers
+        - notifications
+```
 
-#### Other ActiveJob Configuration Options
+### Puma-Style Architecture
 
-```ruby
-# config/application.rb or config/environments/test.rb
+The worker supports two modes based on fork configuration:
+- **`forks: 0`** - Single process with thread pools (development/staging)
+- **`forks: 1+`** - Multiple processes with thread pools (production, Puma-style)
 
-# Set default queue name for all jobs
-config.active_job.queue_name_prefix = "myapp_#{Rails.env}"
+This gives you a natural progression from simple single-threaded development to high-concurrency production.
 
-# Skip after_enqueue callbacks if before_enqueue halts
-config.active_job.skip_after_callbacks_if_terminated = true
+### Configuration Examples
 
-# Log arguments when jobs are enqueued (useful for debugging)
-config.active_job.log_arguments = true  # default: true
+**Development (single worker, single-threaded):**
+```yaml
+development:
+  beanstalk_url: beanstalk://localhost:11300
 
-# Set queue priority (if adapter supports it)
-config.active_job.default_queue_name = 'default'
+  workers:
+    default:
+      # Defaults: forks=0, threads=1, gc_limit=nil
+      queues:
+        - default
+        - mailers
+```
 
-# Configure job serialization
-config.active_job.custom_serializers << MyCustomSerializer
+**Staging (single worker, multi-threaded):**
+```yaml
+staging:
+  beanstalk_url: beanstalk://localhost:11300
+
+  workers:
+    default:
+      default_threads: 10
+      default_gc_limit: 5000
+      queues:
+        - critical
+        - default
+        - mailers
 ```
 
-### Postburner Test Modes vs ActiveJob Adapters
+**Production (multiple workers with different profiles):**
+```yaml
+production:
+  beanstalk_url: <%= ENV['BEANSTALK_URL'] %>
+
+  workers:
+    imports:
+      default_forks: 4      # 4 processes
+      default_threads: 1    # 1 thread per process = 4 concurrent jobs
+      default_gc_limit: 500
+      queues:
+        - imports
+        - data_processing
+
+    general:
+      default_forks: 2      # 2 processes
+      default_threads: 100  # 100 threads per process = 200 concurrent jobs
+      default_gc_limit: 5000
+      queues:
+        - default
+        - mailers
+        - notifications
+```
 
-**Key Difference:** Postburner jobs (subclasses of `Postburner::Job`) do NOT use ActiveJob's queue adapter. Postburner has its own queue strategies that control execution independently of ActiveJob settings.
+### Running Workers
 
-| Postburner Strategy | Execution | Use Case | ActiveJob Equivalent |
-|---------------------|-----------|----------|----------------------|
-| `TestQueue` | Inline, raises on scheduled | Explicit time control | `:inline` adapter |
-| `ImmediateTestQueue` | Inline with auto time-travel | Convenience testing | `:inline` adapter + `travel_to` |
-| `NullQueue` | Deferred, manual execution | Batch processing | `:test` adapter (manual `perform_enqueued_jobs`) |
-| `Queue` / `NiceQueue` | Async via Beanstalkd | Production | `:backburner` adapter |
+**Single worker (auto-selected):**
+```bash
+bin/postburner
+```
+If only one worker is defined, it's automatically selected.
 
-**When to use what:**
-- **ActiveJob with `:test` adapter:** For testing standard ActiveJob classes (mailers, ActiveStorage, etc.)
-- **Postburner test strategies:** For testing `Postburner::Job` subclasses
-- Both can coexist in the same test suite with different behaviors
+**Multiple workers (must specify):**
+```bash
+bin/postburner --worker imports    # Run the 'imports' worker
+bin/postburner --worker general    # Run the 'general' worker
+```
 
-### Automatic Test Mode Detection
+**Filter queues (override config):**
+```bash
+bin/postburner --worker general --queues default,mailers  # Only process specific queues
+```
 
-In Rails test environments, Postburner automatically switches to `TestQueue`:
+### Running Workers in Separate Processes
+
+For production deployments, run different workers in separate OS processes for isolation and resource allocation:
+
+**Docker Compose example:**
+```yaml
+services:
+  # Process 1: Imports worker (forks=4, threads=1)
+  worker-imports:
+    build: .
+    command: bin/postburner --worker imports
+    environment:
+      RAILS_ENV: production
+      BEANSTALK_URL: beanstalk://beanstalkd:11300
+    depends_on:
+      - beanstalkd
+      - postgres
+
+  # Process 2: General worker (forks=2, threads=100)
+  worker-general:
+    build: .
+    command: bin/postburner --worker general
+    environment:
+      RAILS_ENV: production
+      BEANSTALK_URL: beanstalk://beanstalkd:11300
+    depends_on:
+      - beanstalkd
+      - postgres
+```
 
-```ruby
-# Happens automatically when:
-# - Rails.env.test? is true
-# - ActiveJob::Base.queue_adapter_name == :test
+**Procfile example (Heroku/Foreman):**
+```
+worker_imports: bin/postburner --worker imports
+worker_general: bin/postburner --worker general
+```
 
-# Verify you're in test mode
-Postburner.testing?  # => true in tests
+### Worker Architecture
+
+**Single Process Mode (forks: 0):**
+```
+Main Process
+├─ Queue 'critical' Thread Pool (1 thread)
+├─ Queue 'default' Thread Pool (10 threads)
+└─ Queue 'mailers' Thread Pool (5 threads)
 ```
 
-**Note:** Even though `TestQueue` is set automatically in Rails test environments, you can override it at any time to use a different test strategy:
+**Multi-Process Mode (forks: 1+):**
+```
+Parent Process
+├─ Fork 0 (queue: critical)
+│   └─ Thread 1
+├─ Fork 0 (queue: default)
+│   ├─ Thread 1-10
+├─ Fork 1 (queue: default)  # Puma-style: multiple forks of same queue
+│   ├─ Thread 1-10
+├─ Fork 2 (queue: default)
+│   └─ Thread 1-10
+├─ Fork 3 (queue: default)
+│   └─ Thread 1-10
+│   └─ Total: 40 concurrent jobs for 'default' (4 forks × 10 threads)
+└─ Fork 0 (queue: mailers)
+    └─ Thread 1-5
+```
 
-```ruby
-# In test_helper.rb or specific tests
-Postburner.inline_immediate_test_strategy!  # Use automatic time travel
-# OR
-Postburner.null_strategy!  # Use deferred execution
+### Benefits
+
+**Single Process Mode (forks: 0):**
+- Simplest deployment
+- Easy debugging
+- Lower memory footprint
+- Best for development and moderate load
+
+**Multi-Process Mode (forks: 1+):**
+- Horizontal scaling per queue
+- Process isolation
+- Automatic memory cleanup via GC limits
+- Crash isolation (one fork down doesn't affect others)
+- Best for production high-volume workloads
+
+### GC Limits
+
+Set `default_gc_limit` per worker to automatically restart after processing N jobs.
+
+- Worker processes N jobs
+- Worker exits with code 99
+- In single-process mode (forks: 0): process manager restarts the entire process
+- In multi-process mode (forks: 1+): parent process automatically restarts just that fork
+
+```yaml
+production:
+  workers:
+    imports:
+      default_forks: 4
+      default_threads: 1
+      default_gc_limit: 500  # Restart after 500 jobs (memory-intensive)
+      queues:
+        - imports
+        - data_processing
+
+    general:
+      default_forks: 2
+      default_threads: 100
+      default_gc_limit: 5000  # Restart after 5000 jobs
+      queues:
+        - default
+        - mailers
+```
 
-# The auto-detection only sets the initial strategy
-# You can change it anytime during tests
+**Why?**
+- Prevents memory bloat in long-running workers
+- Automatic cleanup without manual intervention
+- Works seamlessly with Docker and Kubernetes restarts
+
+## Configuration
+
+### YAML Configuration
+
+```yaml
+# config/postburner.yml
+default: &default
+  beanstalk_url: <%= ENV['BEANSTALK_URL'] || 'beanstalk://localhost:11300' %>
+
+development:
+  <<: *default
+  # Defaults: forks: 0, threads: 1, gc_limit: nil
+  queues:
+    default: {}
+    mailers: {}
+
+test:
+  <<: *default
+  # Defaults: forks: 0, threads: 1, gc_limit: nil
+  queues:
+    default: {}
+
+staging:
+  <<: *default
+  default_threads: 10      # Multi-threaded, single process
+  default_gc_limit: 5000
+  queues:
+    default: {}
+    mailers: {}
+
+production:
+  <<: *default
+  default_forks: 4         # Puma-style: multiple processes
+  default_threads: 10      # 10 threads per fork
+  default_gc_limit: 5000
+
+  queues:
+    critical:
+      forks: 1
+      threads: 1           # 1 concurrent job
+      gc_limit: 100
+
+    default:
+      forks: 4
+      threads: 10          # 40 total concurrent jobs (4 × 10)
+      gc_limit: 1000
+
+    mailers:
+      forks: 2
+      threads: 5           # 10 total email senders (2 × 5)
+      gc_limit: 500
 ```
 
-### TestQueue: Explicit Time Control (Default for Tests)
+### Queue Configuration Methods
 
-The `TestQueue` strategy forces explicit time management, helping catch scheduling bugs:
+For `Postburner::Job` subclasses (and backwards compatibility):
 
 ```ruby
-test "processes job immediately" do
-  job = MyJob.create!(args: { user_id: 123 })
-  job.queue!
-  assert job.reload.processed_at
+class CriticalJob < Postburner::Job
+  queue 'critical'                  # Beanstalkd tube name
+  queue_priority 0                  # Lower = higher priority (0 is highest)
+  queue_ttr 300                     # TTR (time-to-run) in seconds
+  queue_max_job_retries 3           # Max retry attempts
+  queue_retry_delay 5               # Fixed delay: 5 seconds between retries
+
+  def perform(args)
+    # ...
+  end
 end
+```
 
-test "processes scheduled job with time travel" do
-  job = MyJob.create!(args: { user_id: 123 })
-  job.queue!(delay: 2.hours)
+**Exponential backoff with proc:**
 
-  # Must use travel_to for scheduled jobs
-  travel_to(job.run_at) do
-    assert job.reload.processed_at
+```ruby
+class BackgroundTask < Postburner::Job
+  queue 'default'
+  queue_max_job_retries 5
+  queue_retry_delay ->(retries) { 2 ** retries }  # 2s, 4s, 8s, 16s, 32s
+
+  def perform(args)
+    # ...
   end
 end
 ```
 
-### ImmediateTestQueue: Automatic Time Travel
+### ActiveJob Configuration
 
-For convenience in integration tests, `ImmediateTestQueue` automatically handles scheduled jobs:
+For ActiveJob classes, use standard ActiveJob configuration:
 
 ```ruby
-test "scheduled jobs execute immediately" do
-  Postburner.inline_immediate_test_strategy!
+class MyJob < ApplicationJob
+  include Postburner::Tracked  # Optional: for audit trail
 
-  job = SendReminderEmail.create!(args: { user_id: 123 })
-  job.queue!(delay: 24.hours)
+  queue_as :default
 
-  # Automatically time-travels to scheduled time
-  assert job.reload.processed_at
-  assert_emails 1
+  retry_on StandardError, wait: :exponentially_longer, attempts: 5
+  retry_on CustomError, wait: 10.seconds
+
+  discard_on ActiveJob::DeserializationError
+
+  def perform(user_id)
+    # ...
+  end
 end
 ```
 
-### NullQueue: Deferred Execution
+## Callbacks
 
-Create jobs without queueing, then manually execute them:
+Postburner provides lifecycle callbacks:
 
 ```ruby
-test "batch processes jobs" do
-  Postburner.null_strategy!
+class ProcessPayment < ApplicationJob
+  include Postburner::Tracked
+
+  before_enqueue :validate_payment
+  after_enqueue :send_confirmation
+
+  before_attempt :log_attempt       # Runs on every retry
+  after_attempt :update_metrics
+
+  before_processing :acquire_lock
+  after_processing :release_lock
+
+  after_processed :send_receipt     # Only on success
 
-  jobs = 5.times.map do |i|
-    job = ProcessRecord.create!(args: { id: i })
-    job.queue!
-    job
+  def perform(payment_id)
+    # ...
   end
 
-  # Manually execute when ready
-  jobs.each { |job| Postburner::Job.perform(job.id) }
+  private
 
-  assert jobs.all? { |j| j.reload.processed_at }
+  def validate_payment
+    raise "Invalid" unless payment_valid?
+  end
+
+  def send_receipt
+    PaymentMailer.receipt(arguments.first).deliver_later
+  end
 end
 ```
 
-## Installation
+**Available callbacks:**
+- `before_enqueue`, `around_enqueue`, `after_enqueue` - When queued
+- `before_attempt`, `around_attempt`, `after_attempt` - Every execution (including retries)
+- `before_processing`, `around_processing`, `after_processing` - During execution
+- `after_processed` - Only after successful completion
 
-First [install beanstalkd](https://beanstalkd.github.io/download.html). On Debian-based systems, that goes like this:
-```bash
-sudo apt-get install beanstalkd
-```
+## Queue Strategies
+
+Postburner uses different strategies to control job execution. These affect `Postburner::Job` subclasses (not ActiveJob classes).
+
+| Strategy | When to Use | Behavior | Requires Beanstalkd |
+|----------|-------------|----------|---------------------|
+| **NiceQueue** (default) | Production | Async via Beanstalkd, gracefully re-queues premature jobs | Yes |
+| **Queue** | Production (strict) | Async via Beanstalkd, raises error on premature execution | Yes |
+| **TestQueue** | Testing with explicit time control | Inline execution, raises error for scheduled jobs | No |
+| **ImmediateTestQueue** | Testing with automatic time travel | Inline execution, auto time-travels for scheduled jobs | No |
+| **NullQueue** | Batch processing / deferred execution | Jobs created but not queued, manual execution | No |
 
-Then add to your Gemfile.
 ```ruby
-gem 'postburner'
+# Switch strategies
+Postburner.nice_async_strategy!           # Default production (NiceQueue)
+Postburner.async_strategy!                # Strict production (Queue)
+Postburner.inline_test_strategy!          # Testing (TestQueue)
+Postburner.inline_immediate_test_strategy! # Testing with time travel
+Postburner.null_strategy!                 # Deferred execution
 ```
 
-Install...
-```bash
-$ bundle
-```
+**Note:** These strategies only affect `Postburner::Job` subclasses. ActiveJob classes execute according to the ActiveJob adapter configuration.
 
-After bundling, install the migration.
-```
-# install migration, possible to edit and add attributes or indexes as needed.
-$ bundle exec rails g postburner:install
+## Testing
+
+Postburner provides test-friendly execution modes that don't require Beanstalkd.
+
+### Automatic Test Mode
+
+In Rails test environments, Postburner automatically uses inline execution:
+
+```ruby
+# test/test_helper.rb - automatic!
+Postburner.testing?  # => true in tests
 ```
 
-Inspect `XXX_create_postburner_jobs.rb`. (The template is [here](-/blob/master/lib/generators/postburner/install/templates/migrations/create_postburner_jobs.rb.erb)).The required attributes are in there, but add more if you would like to use them,
-or do it in a separate migration. By default, several indexes are added 
+### Testing Default Jobs (ActiveJob)
+
+Use standard ActiveJob test helpers:
 
-Because `Postburner` is built on `Backburner`, add a `config/initializers/backburner.rb` with option found [here](https://github.com/nesquena/backburner#configuration).
 ```ruby
-Backburner.configure do |config|
-  config.beanstalk_url       = "beanstalk://127.0.0.1"
-  config.tube_namespace      = "some.app.production"
-  config.namespace_separator = "."
-  config.on_error            = lambda { |e| puts e }
-  config.max_job_retries     = 3 # default 0 retries
-  config.retry_delay         = 2 # default 5 seconds
-  config.retry_delay_proc    = lambda { |min_retry_delay, num_retries| min_retry_delay + (num_retries ** 3) }
-  config.default_priority    = 65536
-  config.respond_timeout     = 120
-  config.default_worker      = Backburner::Workers::Simple
-  config.logger              = Logger.new(STDOUT)
-  config.primary_queue       = "backburner-jobs"
-  config.priority_labels     = { :custom => 50, :useless => 1000 }
-  config.reserve_timeout     = nil
-  config.job_serializer_proc = lambda { |body| JSON.dump(body) }
-  config.job_parser_proc     = lambda { |body| JSON.parse(body) }
+class MyTest < ActiveSupport::TestCase
+  test "job executes" do
+    SendEmail.perform_later(123)
+    # Job executes immediately in test mode
+  end
+
+  test "job with delay" do
+    travel 1.hour do
+      SendEmail.set(wait: 1.hour).perform_later(123)
+    end
+  end
 end
 ```
 
-Finally, set `Backburner` for `ActiveJob`
-```
-# config/application.rb
-config.active_job.queue_adapter = :backburner
-```
+### Testing Tracked Jobs
 
-Postburner may later provide an adapter, but we recommend using `Postburner::Job` classes
-directly.
+Tracked jobs create database records you can inspect:
 
-Add jobs to `app/jobs/`. There currently is no generator.
+```ruby
+test "tracked job logs execution" do
+  ProcessPayment.perform_later(456)
+
+  job = Postburner::TrackedJob.last
+  assert job.processed_at.present?
+  assert_includes job.logs.map { |l| l[1]['message'] }, "Processing payment 456"
+end
+```
+
+### Testing Legacy Postburner::Job
 
 ```ruby
-# app/jobs/run_donation.rb
-class RunDonation < Postburner::Job
-  queue 'critical'
-  queue_priority 0 # 0 is highest priority
-  queue_max_job_retries 0 # don't retry
+test "processes immediately" do
+  job = RunDonation.create!(args: { 'donation_id' => 123 })
+  job.queue!
+
+  assert job.reload.processed_at
+end
 
-  def process(args)
-    # do long tasks here
-    # also have access to self.args
+test "scheduled job with time travel" do
+  job = RunDonation.create!(args: { 'donation_id' => 123 })
+
+  travel_to(2.hours.from_now) do
+    job.queue!(delay: 2.hours)
+    assert job.reload.processed_at
   end
 end
 ```
 
-### Comparison to Backburner
+## Job Management
 
-Compared to plain [Backburner](https://github.com/nesquena/backburner),
-Postburner adds:
-- Database Jobs for inspection, linking, auditing, removal (and deletion)
-- Direct access to associated [Beanstalk](https://beanstalkd.github.io/) (via [beaneater](https://github.com/beanstalkd/beaneater))
-- Job Statistics (lag, attempts, logs, tracked errors)
-- Convenience methods to clear tubes, stats, and connections for Beanstalk.
+### Canceling Jobs
 
-Otherwise, Postburner tries to be a super simple layer on `Backburner::Queue`
-and `ActiveRecord`. Every tool with either of those are available in
-`Postburner::Job`s.
+**For Postburner::Job subclasses:**
 
-Comes with a mountable interface that can be password protected with whatever
-authentication you use in your project.
+```ruby
+job = RunDonation.create!(args: { 'donation_id' => 123 })
+job.queue!(delay: 1.hour)
 
-### Comparison to Que
+# Soft delete (recommended) - removes from queue, sets removed_at
+job.remove!
+job.removed_at  # => 2025-11-19 12:34:56 UTC
+job.persisted?  # => true (still in database for audit)
 
-Postburner meant to be a replacement/upgrade for [Que](https://github.com/que-rb/que).
-However, if you need something faster and backed with ACID compliance, check out Que.  
+# Hard delete - removes from both queue and database
+job.destroy!
 
-Postburner has some additional features such as retained jobs after processing,
-stats, per job logging, etc.
+# Delete from Beanstalkd only (keeps database record)
+job.delete!
+```
 
-Postburner is meant to be simpler than `Que`. Que is incredible, but jobs should
-be simple so the logic and history can be transparent.
+**For tracked ActiveJob classes:**
 
-## Contributing
-Submit a pull request. Follow conventions of the project. Be nice.
+```ruby
+# Find the TrackedJob record
+job = Postburner::TrackedJob.find(123)
+job.remove!  # Cancel execution
+```
 
-### V1 TODO
-- Basic tests
-- Add Authentication modules for engine mount.
+### Retrying Buried Jobs
 
-### V1+ TODO
-- Install generator
-  - Sub to backburner
-- Job generator
-  - Build file in app/jobs
-  - Inherit from Postburner::Job
-- Add before/after/around hooks
-- Add destroy, and remove actions on show page
-- Clear tubes.
-- Document how/when to use activerecord hooks
-- Document how/when to use backburner hooks
-- Document how/when to use postburner hooks
-- Add logging with Job.args in backburner logs
-- MAYBE - ActiveJob adapter
+Jobs that fail repeatedly get "buried" in Beanstalkd:
 
+```ruby
+job = Postburner::Job.find(123)
+job.kick!  # Moves buried job back to ready queue
+```
 
-### Running in Development
+### Inspecting Job State
 
-```
-cd test/dummy
-bundle exec backburner
+```ruby
+job = Postburner::TrackedJob.find(123)
+
+# Timestamps
+job.queued_at      # When queued to Beanstalkd
+job.run_at         # Scheduled execution time
+job.attempting_at  # First attempt started
+job.processing_at  # Current/last processing started
+job.processed_at   # Completed successfully
+job.removed_at     # Soft deleted
+
+# Statistics
+job.lag       # Milliseconds between scheduled and actual execution
+job.duration  # Milliseconds to execute
+job.attempt_count  # Number of attempts
+job.error_count    # Number of errors
+
+# Audit trail
+job.logs    # Array of log entries with timestamps
+job.errata  # Array of exceptions with backtraces
+job.attempts # Array of attempt timestamps
 ```
 
-### Releasing
+## Beanstalkd Integration
 
-1. `gem bump -v minor -t`
-  Where <minor> can be: major|minor|patch|pre|release or a version number
-2. Edit `CHANGELOG.md` with details from `git log --oneline`
-3. `git commit --amend`
-4. `gem release -k nac -p`
-  Where <nac> is an authorized key with push capabilities.
+Direct access to Beanstalkd for advanced operations:
 
+```ruby
+# Get Beanstalkd job ID
+job.bkid  # => 12345
 
-## FAQ
+# Access Beaneater job object
+job.beanstalk_job.stats
+# => {"id"=>12345, "tube"=>"critical", "state"=>"ready", ...}
 
-### Can I save a Postburner::Job without enqueuing it?
+# Connection management
+Postburner.connected do |conn|
+  conn.tubes.to_a  # List all tubes
+  conn.tubes['postburner.production.critical'].stats
+  conn.tubes['postburner.production.critical'].kick(10)  # Kick 10 buried jobs
+end
+```
 
-Yes! You can absolutely save a `Postburner::Job` without enqueuing it. Looking at the code, here's how it works:
 
-#### Creating vs. Queueing
+## Web UI
 
-When you use `create!` or `save!`, the job is persisted to the database but **not** sent to Beanstalkd:
+Mount the inspection interface:
 
 ```ruby
-# Creates job in database but does NOT enqueue
-job = RunDonation.create!(args: {donation_id: 123})
-job.persisted?  # => true
-job.queued_at   # => nil
-job.bkid        # => nil (no Beanstalkd ID)
+# config/routes.rb
+mount Postburner::Engine => "/postburner"
+
+# Only in development
+mount Postburner::Engine => "/postburner" if Rails.env.development?
 ```
 
-To actually enqueue it, you must explicitly call `queue!`:
+Add your own authentication:
 
 ```ruby
-# Now it's enqueued to Beanstalkd
-job.queue!(delay: 1.hour)
-job.queued_at  # => Time.zone.now
-job.bkid       # => 12345 (Beanstalkd job ID)
+# config/routes.rb
+authenticate :user, ->(user) { user.admin? } do
+  mount Postburner::Engine => "/postburner"
+end
+```
+
+## Deployment
+
+### Docker
+
+```dockerfile
+# Dockerfile.worker
+FROM ruby:3.3
+
+WORKDIR /app
+COPY . .
+RUN bundle install
+
+CMD ["bundle", "exec", "postburner", "--config", "config/postburner.yml", "--env", "production"]
+```
+
+```yaml
+# docker-compose.yml
+services:
+  beanstalkd:
+    image: schickling/beanstalkd
+    command: ["-l", "0.0.0.0", "-p", "11300", "-b", "/var/lib/beanstalkd"]
+    ports:
+      - "11300:11300"
+    volumes:
+      - beanstalkd_data:/var/lib/beanstalkd
+
+  worker:
+    build:
+      context: .
+      dockerfile: Dockerfile.worker
+    depends_on:
+      - beanstalkd
+      - postgres
+    environment:
+      BEANSTALK_URL: beanstalk://beanstalkd:11300
+      DATABASE_URL: postgres://postgres@postgres/myapp_production
+
+volumes:
+  beanstalkd_data:
 ```
 
-#### How it Works
+## Migration from v0.x
 
-The implementation uses a clever pattern (app/models/postburner/job.rb:86-724):
+Key changes in v1.0:
 
-1. **after_save_commit callback** (`insert_if_queued!` at line 86) runs on every save
-2. **Conditional insertion** (line 722) - only inserts if `will_insert?` returns true
-3. **Flag check** (lines 242-244) - `will_insert?` only returns true if `@_insert_options` is set
-4. **Flag set** (line 199) - `@_insert_options` is only set inside the `queue!` method
+### Removed
+- Backburner dependency
+- `config/initializers/backburner.rb`
+- Backburner worker (`bundle exec backburner`)
 
-So the job is safely persisted without queueing until you explicitly call `queue!`.
+### Added
+- ActiveJob adapter (first-class Rails integration)
+- Default jobs (Beanstalkd only, no PostgreSQL)
+- `bin/postburner` executable
+- `config/postburner.yml` configuration
+- Multiple worker types (Simple, Forking, ThreadsOnFork)
 
-#### Use Cases
+### Migration Steps
 
-This is useful for:
-- Creating jobs conditionally queued based on business logic
-- Building jobs within transactions that might roll back
-- Testing job creation without running workers
-- Creating foreign key relationships before queueing
+1. **Update Gemfile:**
+   ```ruby
+   gem 'postburner', '~> 1.0.0.pre.1'
+   ```
+
+2. **Remove Backburner config:**
+   ```bash
+   rm config/initializers/backburner.rb
+   ```
+
+3. **Create Postburner config:**
+   ```bash
+   cp config/postburner.yml.example config/postburner.yml
+   ```
+
+4. **Update ActiveJob adapter:**
+   ```ruby
+   # config/application.rb
+   config.active_job.queue_adapter = :postburner  # was :backburner
+   ```
+
+5. **Update worker command:**
+   ```bash
+   # Old
+   bundle exec backburner
+
+   # New
+   bundle exec postburner --config config/postburner.yml
+   ```
+
+6. **Existing Jobs:** Direct `Postburner::Job` subclasses continue to work without changes!
+
+## Contributing
+
+Submit a pull request. Follow project conventions. Be nice.
 
 ## License
+
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).