postburner 1.0.0.pre.14 → 1.0.0.pre.15

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 71c6a81826b32021b0f0083c1771b27423e1b25198d7c00669cbab1150c431e7
-  data.tar.gz: 26403dba984c15f4bad94d81f1f5833d6a6bb16c0bb216a236f6bcaf07c0d019
+  metadata.gz: 971531132068ea4bddfdb5b5a8aaae4c788ea576aae268010fb6cd4dbd2b82f1
+  data.tar.gz: 44053985f2bebd385bc2a17eb1eb655d89de17d77a8c25b4a1210177cefead66
 SHA512:
-  metadata.gz: bc088b5d6e5b36b514562635ee3ed75955dfe809efe981e3ddbdc47c5f5387715233b7eb0eec8b4e78f97cf715202beabb7f8b90df553c605eec058eee462a9c
-  data.tar.gz: 40dbfc6f0979226d5c67780ffe31c73700a6a7b7faf97bf8e7fd127a2c889e708db91835bc9389032191ae2fae6460f3cc1b21f28ee17a6a3f00dbc84c31fbea
+  metadata.gz: 19c023063d9eb06913fcb59eaaf581a537b8a944b8ab8713b95210475517223a42887f239bd8788cd313919ffe9bfb89c6c9f80c8b24869239b943bfd2c300ad
+  data.tar.gz: 0e0ed766a67ddd9ef02c0b97eff3a25c1e923c79068608a4d31db779ae23187eb85a5b075e8100644faf1c196f0bafaf1a7711e09db8a948b0ab09e9138f9b49

data/README.md

@@ -2,24 +2,20 @@
 
 Fast Beanstalkd-backed job queue with **optional PostgreSQL records**.
 
-Depends on Beanstalkd, Postgres, ActiveRecord, and ActiveJob (1).
-
 Postburner provides dual-mode job execution:
 - **Fast jobs**: Fast execution via Beanstalkd
 - **Tracked jobs**: Audited jobs logs, timing, errors, and statistics
 
 Built for production environments where you want fast background processing for most jobs, but comprehensive auditing for critical operations.
 
-- **ActiveJob native** - Works seamlessly with Rails, ActionMailer, ActiveStorage
-- **Dual-mode execution** - Default or tracked (database backed)
-- **Rich audit trail** - Logs, timing, errors, retry tracking (tracked jobs only)
-- **ActiveRecord** - Query jobs with ActiveRecord (Tracked jobs only)
+- **ActiveJob Adapter** - To use with Rails, ActionMailer, ActiveStorage
+- **Dual-mode execution** - Beanstalkd only or tracked (database backed)
+- **Rich audit trail** - Logs, timing, errors, instrumentation, retry tracking (tracked jobs only)
+- **ActiveRecord** - Query jobs with ActiveRecord (on opt in)
 - **Scheduler** - Schedule jobs at fixed intervals, cron expressions, and calendar-aware anchor points.
-- **Beanstalkd** - Fast, reliable queue separate from your database, peristent storage
+- **Test-friendly** - Testing jobs can be tricky, so we go beyond just inline.
 - **Process isolation** - Forking workers with optional threading for throughput
-- **Test-friendly** - Inline execution without Beanstalkd in tests
-
-(1) An ActiveJob adapter is provided for seamless integration with the default Rails stack, you can use Postburner without ActiveJob by using the `Postburner::Job` class directly.
+- **Beanstalkd** - Fast, reliable queue separate from your database, persistent storage
 
 ```ruby
 # Default job (fast, no PostgreSQL overhead)
@@ -75,7 +71,7 @@ end
 # Scheduled job (fixed interval, anchor based)
 Postburner::Schedule.create!(
   name: 'daily_event_retention',
-  job_class: EventRetentionJob, # either Postburner::Job or ActiveJob
+  job_class: EventRetentionJob, # either Postburner::Job or ActiveJob ancestor
   anchor: Time.zone.parse('2025-01-01 09:30:00'),
   interval: 1,
   interval_unit: 'days',
@@ -132,7 +128,7 @@ Postburner [beanstalkd](https://beanstalkd.github.io/) is used with PostgreSQL t
 
 ```ruby
 # Gemfile
-gem 'postburner', '~> 1.0.0.pre.11'
+gem 'postburner', '~> 1.0.0.pre.14'
 
 # config/application.rb
 config.active_job.queue_adapter = :postburner
@@ -140,28 +136,28 @@ config.active_job.queue_adapter = :postburner
 
 ```yaml
 # config/postburner.yml
-development:  # <- environment config, i.e. defaults
+development:  # <- environment config
   beanstalk_url: <%= ENV['BEANSTALK_URL'] || 'beanstalk://localhost:11300' %>
-  default_forks: 2
-  default_threads: 10
-  default_gc_limit: 500
+  forks: 2
+  threads: 10
+  gc_limit: 500
 
-  workers:  # <- worker config, i.e. overrides
+  workers:  # <- worker config (overrides env-level)
     default:
-      threads: 16        # Overrides default_threads
+      threads: 16        # Overrides env-level threads
       queues:
         - default
         - mailers
     critical:
-      forks: 4           # Overrides default_forks
-      threads: 8         # Overrides default_threads
-      gc_limit: 24       # Overrides default_gc_limit
+      forks: 4           # Overrides env-level forks
+      threads: 8         # Overrides env-level threads
+      gc_limit: 24       # Overrides env-level gc_limit
       queues:
         - payments
     slow:
-      forks: 4           # Overrides default_forks
-      threads: 1         # Overrides default_threads
-      gc_limit: 1        # Overrides default_gc_limit
+      forks: 4           # Overrides env-level forks
+      threads: 1         # Overrides env-level threads
+      gc_limit: 1        # Overrides env-level gc_limit
       queues:
         - imports
         - video
@@ -241,8 +237,8 @@ end
 ```
 
 **Configuration options:**
-- `queue_priority` - Beanstalkd priority (0-4294967295, lower = higher priority)
-- `queue_ttr` - Time-to-run in seconds before job times out
+- `priority` - Beanstalkd priority (0-4294967295, lower = higher priority)
+- `ttr` - Time-to-run in seconds before job times out
 
 Jobs without `Postburner::Beanstalkd` use defaults from `config/postburner.yml`:
 - `default_priority: 65536`
@@ -276,7 +272,7 @@ end
 
 Tracked jobs store full execution details in PostgreSQL, providing comprehensive audit trails (i.e. logging, timing, errors, retry tracking) for critical operations.
 
-**Note:** `Postburner::Tracked` automatically includes `Postburner::Beanstalkd`, giving you access to `queue_priority`, `queue_ttr`, `bk`, and `extend!`.
+**Note:** `Postburner::Tracked` automatically includes `Postburner::Beanstalkd`, giving you access to `priority`, `ttr`, `bk`, and `extend!`.
 
 ```ruby
 class ProcessPayment < ApplicationJob
@@ -375,8 +371,8 @@ Override queue priority and TTR per job instance for dynamic behavior:
 # Set priority during creation
 job = ProcessPayment.create!(
   args: { 'payment_id' => 123 },
-  queue_priority: 1500,  # Override class-level priority
-  queue_ttr: 300         # Override class-level TTR
+  priority: 1500,  # Override class-level priority
+  ttr: 300         # Override class-level TTR
 )
 job.queue!
 
@@ -427,7 +423,7 @@ Postburner includes a lightweight, fixed-rate scheduler for recurring jobs. Perf
 The scheduler uses **immediate enqueue** combined with a **watchdog safety net**:
 
 1. When an execution is created, it's immediately enqueued to Beanstalkd's delayed queue with the appropriate delay until `run_at`
-2. For `Postburner::Job` and `ActiveJob` with `Postburner::Tracked` schedules the next execution when the current job runs - providing immediate pickup without waiting for the watchdog. Normal `ActiveJob` schedules need to rely on the watchdog to create the next execution, so set the `default_scheduler_interval` to pick up exections appropriately.
+2. For `Postburner::Job` and `ActiveJob` with `Postburner::Tracked` schedules the next execution when the current job runs - providing immediate pickup without waiting for the watchdog. Normal `ActiveJob` schedules need to rely on the watchdog to create the next execution, so set the `scheduler_interval` to pick up executions appropriately.
 3. A lightweight watchdog job in the `scheduler` tube acts as a safety net:
    ```json
    { "scheduler": true, "interval": 300 }
@@ -464,8 +460,8 @@ Add scheduler settings to `config/postburner.yml`:
 ```yaml
 production:
   beanstalk_url: <%= ENV['BEANSTALK_URL'] %>
-  default_scheduler_interval: 300  # Pickup new schedules every 5 minutes
-  default_scheduler_priority: 100  # Scheduler jobs run at priority 100
+  scheduler_interval: 300  # Pickup new schedules every 5 minutes
+  scheduler_priority: 100  # Scheduler jobs run at priority 100
 
   workers:
     default:
@@ -475,8 +471,8 @@ production:
 ```
 
 **Configuration options:**
-- `default_scheduler_interval` - How often (in seconds) to check for due schedules (default: 300)
-- `default_scheduler_priority` - Beanstalkd priority for watchdog jobs (default: 100)
+- `scheduler_interval` - How often (in seconds) to check for due schedules (default: 300)
+- `scheduler_priority` - Beanstalkd priority for watchdog jobs (default: 100)
 
 **Choosing an interval:** Since executions are enqueued immediately to Beanstalkd's delayed queue, the watchdog interval primarily affects:
 - How quickly new schedules are auto-bootstrapped (if you don't call `start!`)
@@ -636,8 +632,8 @@ schedule.next_run_at_times(5)  # Next 5 run times
 
 # View executions
 schedule.executions.pending
-schedule.executions.completed
-schedule.executions.failed
+schedule.executions.scheduled
+schedule.executions.skipped
 ```
 
 #### Starting Schedules
@@ -664,7 +660,7 @@ The job is immediately in Beanstalkd's delayed queue and will run at the schedul
 
 **Option 2: Auto-bootstrap (eventual pickup)**
 
-If you don't call `start!`, the scheduler watchdog will automatically bootstrap the schedule on its next run. This adds up to one `default_scheduler_interval` of delay before the first execution is enqueued:
+If you don't call `start!`, the scheduler watchdog will automatically bootstrap the schedule on its next run. This adds up to one `scheduler_interval` of delay before the first execution is enqueued:
 
 ```ruby
 schedule = Postburner::Schedule.create!(...)
@@ -683,10 +679,9 @@ Each scheduled run creates an execution record for tracking:
 ```ruby
 execution = Postburner::ScheduleExecution.find(123)
 
-execution.status        # pending, running, completed, failed, skipped
+execution.status        # pending, scheduled, skipped
 execution.run_at        # Scheduled time
 execution.enqueued_at   # When job was queued
-execution.completed_at  # When job finished
 execution.beanstalk_job_id  # Beanstalkd job ID
 execution.job_id            # Postburner::Job ID (if using Postburner::Job)
 ```
@@ -905,25 +900,25 @@ Postburner uses named worker configurations to support different deployment patt
 Configure multiple named workers with different concurrency profiles:
 
 ```yaml
-production:  # <- environment config, i.e. defaults
+production:  # <- environment config
   beanstalk_url: <%= ENV['BEANSTALK_URL'] %>
-  default_forks: 2
-  default_threads: 10
-  default_gc_limit: 5000
+  forks: 2
+  threads: 10
+  gc_limit: 5000
 
-  workers:  # <- worker config, i.e. overrides
+  workers:  # <- worker config (overrides env-level)
     # Heavy, memory-intensive jobs - more processes, fewer threads
     imports:
-      forks: 4           # Overrides default_forks
-      threads: 1         # Overrides default_threads
-      gc_limit: 500      # Overrides default_gc_limit
+      forks: 4           # Overrides env-level forks
+      threads: 1         # Overrides env-level threads
+      gc_limit: 500      # Overrides env-level gc_limit
       queues:
         - imports
         - data_processing
 
     # General jobs - fewer processes, many threads
     general:
-      threads: 100       # Overrides default_threads (forks uses default_forks=2)
+      threads: 100       # Overrides env-level threads (forks uses env-level forks=2)
       queues:
         - default
         - mailers
@@ -957,12 +952,12 @@ development:  # <- environment config
 ```yaml
 staging:  # <- environment config
   beanstalk_url: beanstalk://localhost:11300
-  default_threads: 10
-  default_gc_limit: 5000
+  threads: 10
+  gc_limit: 5000
 
   workers:  # <- worker config
     default:
-      # Uses env defaults: default_threads=10, default_gc_limit=5000
+      # Uses env-level: threads=10, gc_limit=5000
       queues:
         - critical
         - default
@@ -971,25 +966,25 @@ staging:  # <- environment config
 
 **Production (multiple workers with different profiles):**
 ```yaml
-production:  # <- environment config, i.e. defaults
+production:  # <- environment config
   beanstalk_url: <%= ENV['BEANSTALK_URL'] %>
-  default_forks: 2
-  default_threads: 10
-  default_gc_limit: 5000
+  forks: 2
+  threads: 10
+  gc_limit: 5000
 
-  workers:  # <- worker config, i.e. overrides
+  workers:  # <- worker config (overrides env-level)
     imports:
-      forks: 4           # Overrides default_forks (4 processes)
-      threads: 1         # Overrides default_threads (1 thread per process = 4 concurrent jobs)
-      gc_limit: 500      # Overrides default_gc_limit
+      forks: 4           # Overrides env-level forks (4 processes)
+      threads: 1         # Overrides env-level threads (1 thread per process = 4 concurrent jobs)
+      gc_limit: 500      # Overrides env-level gc_limit
       queues:
         - imports
         - data_processing
 
     general:
-      # forks uses default_forks=2 (2 processes)
-      threads: 100       # Overrides default_threads (100 threads per process = 200 concurrent jobs)
-      # gc_limit uses default_gc_limit=5000
+      # forks uses env-level forks=2 (2 processes)
+      threads: 100       # Overrides env-level threads (100 threads per process = 200 concurrent jobs)
+      # gc_limit uses env-level gc_limit=5000
       queues:
         - default
         - mailers
@@ -1148,8 +1143,8 @@ The `shutdown_timeout` controls how long workers wait for in-flight jobs to comp
 
 ```yaml
 production:
-  default_ttr: 300              # Default TTR for jobs
-  default_shutdown_timeout: 300 # Defaults to default_ttr if not specified
+  default_ttr: 300        # Default TTR for jobs
+  shutdown_timeout: 300   # Defaults to default_ttr if not specified
 
   workers:
     imports:
@@ -1164,7 +1159,7 @@ production:
 
 ### GC Limits
 
-Set `default_gc_limit` at environment level or `gc_limit` per worker to automatically restart after processing N jobs.
+Set `gc_limit` at environment level or per worker to automatically restart after processing N jobs.
 
 - Worker processes N jobs
 - Worker exits with code 99
@@ -1173,22 +1168,22 @@ Set `default_gc_limit` at environment level or `gc_limit` per worker to automati
 
 ```yaml
 production:  # <- environment config
-  default_forks: 2
-  default_threads: 10
-  default_gc_limit: 5000
+  forks: 2
+  threads: 10
+  gc_limit: 5000
 
   workers:  # <- worker config
     imports:
-      forks: 4           # Overrides default_forks
-      threads: 1         # Overrides default_threads
-      gc_limit: 500      # Overrides default_gc_limit (restart after 500 jobs, memory-intensive)
+      forks: 4           # Overrides env-level forks
+      threads: 1         # Overrides env-level threads
+      gc_limit: 500      # Overrides env-level gc_limit (restart after 500 jobs, memory-intensive)
       queues:
         - imports
         - data_processing
 
     general:
-      # Uses default_forks=2, default_threads=100, default_gc_limit=5000 (restart after 5000 jobs)
-      threads: 100       # Overrides default_threads
+      # Uses env-level forks=2, threads=100, gc_limit=5000 (restart after 5000 jobs)
+      threads: 100       # Overrides env-level threads
       queues:
         - default
         - mailers
@@ -1229,41 +1224,41 @@ test:  # <- environment config
 
 staging:  # <- environment config
   <<: *default
-  default_threads: 10      # Multi-threaded, single process
-  default_gc_limit: 5000
+  threads: 10      # Multi-threaded, single process
+  gc_limit: 5000
 
   workers:  # <- worker config
     default:
-      # Uses env defaults: default_threads=10, default_gc_limit=5000
+      # Uses env-level: threads=10, gc_limit=5000
       queues:
         - default
         - mailers
 
-production:  # <- environment config, i.e. defaults
+production:  # <- environment config
   <<: *default
-  default_forks: 2         # Default for workers
-  default_threads: 10      # Default for workers
-  default_gc_limit: 5000   # Default for workers
+  forks: 2         # Inherited by workers
+  threads: 10      # Inherited by workers
+  gc_limit: 5000   # Inherited by workers
 
-  workers:  # <- worker config, i.e. overrides
+  workers:  # <- worker config (overrides env-level)
     critical:
-      forks: 1             # Overrides default_forks
-      threads: 1           # Overrides default_threads (1 concurrent job)
-      gc_limit: 100        # Overrides default_gc_limit
+      forks: 1             # Overrides env-level forks
+      threads: 1           # Overrides env-level threads (1 concurrent job)
+      gc_limit: 100        # Overrides env-level gc_limit
       queues:
         - critical
 
     default:
-      forks: 4             # Overrides default_forks
-      threads: 10          # Overrides default_threads (40 total concurrent jobs: 4 × 10)
-      gc_limit: 1000       # Overrides default_gc_limit
+      forks: 4             # Overrides env-level forks
+      threads: 10          # Overrides env-level threads (40 total concurrent jobs: 4 × 10)
+      gc_limit: 1000       # Overrides env-level gc_limit
       queues:
         - default
 
     mailers:
-      # forks uses default_forks=2
-      threads: 5           # Overrides default_threads (10 total email senders: 2 × 5)
-      gc_limit: 500        # Overrides default_gc_limit
+      # forks uses env-level forks=2
+      threads: 5           # Overrides env-level threads (10 total email senders: 2 × 5)
+      gc_limit: 500        # Overrides env-level gc_limit
       queues:
         - mailers
 ```
@@ -1311,7 +1306,7 @@ Both refer to the Beanstalkd tube `postburner.production.mailers`.
 
 **Naming convention:** Use underscores for multi-word queue names (e.g., `background_jobs`, `high_priority`). Avoid hyphens as they can cause issues with some Beanstalkd client libraries.
 
-**Important:** No need to set `config.active_job.queue_name_prefix` - Postburner handles prefixing automatically, when jobs are enqueued with postburner.
+**Important:** No need to set `config.active_job.queue_name_prefix` - Postburner handles prefixing automatically when jobs are enqueued with postburner.
 
 ### Queue Configuration Methods
 
@@ -1534,16 +1529,16 @@ end
 
 Beanstalkd is a simple, fast, and reliable queue system. It is a good choice for production environments where you want fast background processing for most jobs, but comprehensive auditing for critical operations.
 
-The [protocol](https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt) reads more like a README than a protocol. Check it out and you will instantly understand how it works. Here is a help
+The [protocol](https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt) reads more like a README than a protocol. Check it out and you will instantly understand how it works.
 
-Here is a picture of the typical job lifecycle:
+Diagram of the typical job lifecycle:
 
 ```
    put            reserve               delete
   -----> [READY] ---------> [RESERVED] --------> *poof*`
 ```
 
-Here is a picture with more possibilities:
+Diagram with all lifecycle:
 
 ```
    put with delay               release with delay
@@ -1748,7 +1743,7 @@ end
 ```ruby
 class DataImport < Postburner::Job
   queue 'imports'
-  ttr 1800  # 30 minutes (equivalent to queue_ttr)
+  ttr 1800  # 30 minutes
 
   def perform(args)
     # Long-running import logic
@@ -1761,8 +1756,8 @@ end
 For jobs that may take longer than expected, you can extend the TTR dynamically using `extend!`. This calls Beanstalkd's `touch` command, which resets the TTR countdown.
 
 **Available for:**
-- `Postburner::Job` subclasses (always available via `bk.extend!`)
-- `Postburner::Tracked` ActiveJob classes (includes `extend!` method)
+- `Postburner::Job` subclasses (via `extend!` method)
+- `Postburner::Tracked` ActiveJob classes (via `extend!` method)
 
 ```ruby
 class ProcessImport < ApplicationJob
@@ -1799,7 +1794,7 @@ class LargeDataProcessor < Postburner::Job
     dataset.each_chunk do |chunk|
       process_chunk(chunk)
 
-      bk.extend!  # Reset TTR via beanstalkd job accessor
+      extend!  # Reset TTR (calls bk.touch internally)
       log "Chunk processed, TTR extended"
     end
   end
@@ -2000,7 +1995,7 @@ Postburner.connected do |conn|
 end
 ```
 
-## Web UI -- Dated
+## Web UI - v2 Coming Soon
 
 Mount the inspection interface:
 
@@ -2043,7 +2038,7 @@ beanstalkd -l 127.0.0.1 -p 11300 -b /var/lib/beanstalkd
 
 ```ruby
 # Gemfile
-gem 'postburner', '~> 1.0.0.pre.1'
+gem 'postburner', '~> 1.0.0.pre.14'
 ```
 
 ```bash
@@ -2094,7 +2089,7 @@ Key changes in v1.0:
 
 1. **Update Gemfile:**
    ```ruby
-   gem 'postburner', '~> 1.0.0.pre.1'
+   gem 'postburner', '~> 1.0.0.pre.14'
    ```
 
 2. **Remove Backburner config:**
@@ -2138,7 +2133,7 @@ We encourage AI tools, but do not vibe, as the code must look like it was writte
 ```bash
 bundle install
 bundle exec rails test # must have beanstalkd on 11300 by default
-bundle exec rails app:postburner:work # if you want to run the worker
+bundle exec rails app:postburner:work # run worker from engine root (uses test/dummy app)
 ```
 
 ## License

data/app/controllers/postburner/application_controller.rb

@@ -1,4 +1,5 @@
 module Postburner
   class ApplicationController < ActionController::Base
+    protect_from_forgery with: :exception
   end
 end

data/bin/postburner

@@ -18,7 +18,7 @@
 #   bin/postburner
 #   bin/postburner --worker default
 #   bin/postburner --env production --worker imports
-#   bin/postburner --worker default --queues default,mailers
+#   bin/postburner --worker default --queues default,mailers # just using --worker is preferred
 #
 
 require 'optparse'

data/lib/generators/postburner/install/templates/config/postburner.yml

@@ -33,6 +33,6 @@ test:
 # Multi-process, multi-thread (forks: 2, threads: 4)
 production:
   <<: *shared
-  default_forks: 2
-  default_threads: 4
-  default_gc_limit: 64
+  forks: 2
+  threads: 4
+  gc_limit: 64

data/lib/postburner/configuration.rb

@@ -21,7 +21,7 @@ module Postburner
   #
   class Configuration
     # Global settings
-    attr_accessor :beanstalk_url, :logger, :default_queue, :default_priority, :default_ttr
+    attr_accessor :beanstalk_url, :logger, :default_priority, :default_ttr
     attr_accessor :default_scheduler_interval, :default_scheduler_priority
     attr_accessor :enqueue_options
 
@@ -31,7 +31,6 @@ module Postburner
     # @param options [Hash] Configuration options
     # @option options [String] :beanstalk_url Beanstalkd URL (default: ENV['BEANSTALK_URL'] or localhost)
     # @option options [Logger] :logger Logger instance (default: Rails.logger)
-    # @option options [String] :default_queue Default queue name (default: 'default')
     # @option options [Integer] :default_priority Default job priority (default: 65536, lower = higher priority)
     # @option options [Integer] :default_ttr Default time-to-run in seconds (default: 300)
     # @option options [Integer] :default_scheduler_interval Scheduler check interval in seconds (default: 300)
@@ -51,7 +50,6 @@ module Postburner
     def initialize(options = {})
       @beanstalk_url = options[:beanstalk_url] || ENV['BEANSTALK_URL'] || 'beanstalk://localhost:11300'
       @logger = options[:logger] || (defined?(Rails) ? Rails.logger : Logger.new(STDOUT))
-      @default_queue = options[:default_queue] || 'default'
       @default_priority = options[:default_priority] || 65536
       @default_ttr = options[:default_ttr] || 300
       @default_scheduler_interval = options[:default_scheduler_interval] || 300
@@ -153,20 +151,19 @@ module Postburner
       worker_config = {
         name: worker_name,
         queues: worker_yaml['queues'] || ['default'],
-        forks: worker_yaml['forks'] || env_config['default_forks'] || 0,
-        threads: worker_yaml['threads'] || env_config['default_threads'] || 1,
-        gc_limit: worker_yaml['gc_limit'] || env_config['default_gc_limit'],
+        forks: worker_yaml['forks'] || env_config['forks'] || 0,
+        threads: worker_yaml['threads'] || env_config['threads'] || 1,
+        gc_limit: worker_yaml['gc_limit'] || env_config['gc_limit'],
         timeout: worker_yaml['timeout'] || 3,
-        shutdown_timeout: worker_yaml['shutdown_timeout'] || env_config['default_shutdown_timeout'] || default_ttr
+        shutdown_timeout: worker_yaml['shutdown_timeout'] || env_config['shutdown_timeout'] || default_ttr
       }
 
       options = {
         beanstalk_url: env_config['beanstalk_url'],
-        default_queue: env_config['default_queue'],
         default_priority: env_config['default_priority'],
         default_ttr: env_config['default_ttr'],
-        default_scheduler_interval: env_config['default_scheduler_interval'],
-        default_scheduler_priority: env_config['default_scheduler_priority'],
+        default_scheduler_interval: env_config['scheduler_interval'],
+        default_scheduler_priority: env_config['scheduler_priority'],
         worker_config: worker_config
       }
 
@@ -190,8 +187,8 @@ module Postburner
     # Beanstalkd tube name with environment namespace (e.g.,
     # 'postburner.production.critical').
     #
-    # @param queue_name [String, Symbol, nil] Base queue name (defaults to configured default_queue)
-    # @param env [String, Symbol, nil] Environment name (defaults to Rails.env or 'development')
+    # @param queue_name [String, Symbol] Base queue name
+    # @param env [String, Symbol, nil] Environment name (defaults to Rails.env)
     #
     # @return [String] Full tube name with environment prefix
     #
@@ -199,14 +196,11 @@ module Postburner
     #   config.expand_tube_name('critical', 'production')
     #   # => "postburner.production.critical"
     #
-    # @example With configured default
-    #   config.default_queue = 'background'
-    #   config.expand_tube_name
-    #   # => "postburner.development.background"
+    #   config.expand_tube_name('default')
+    #   # => "postburner.test.default" (in test env)
     #
-    def expand_tube_name(queue_name = nil, env = nil)
+    def expand_tube_name(queue_name, env = nil)
       env ||= defined?(Rails) ? Rails.env : nil
-      queue_name ||= @default_queue
       [
         tube_prefix(env),
         queue_name,

data/lib/postburner/scheduler.rb

@@ -45,8 +45,8 @@ module Postburner
   # Add to config/postburner.yml:
   #
   #   production:
-  #     default_scheduler_interval: 300  # Check every 5 minutes (default)
-  #     default_scheduler_priority: 100  # Watchdog priority (default)
+  #     scheduler_interval: 300  # Check every 5 minutes (default)
+  #     scheduler_priority: 100  # Watchdog priority (default)
   #
   # The interval primarily affects:
   # - How quickly new schedules are auto-bootstrapped (if you don't call start!)
@@ -247,8 +247,8 @@ module Postburner
     # The watchdog will execute after the delay, process all schedules, and re-queue itself.
     #
     # Reads interval and priority from configuration if not provided:
-    # - default_scheduler_interval (default: 300 seconds)
-    # - default_scheduler_priority (default: 100)
+    # - scheduler_interval (default: 300 seconds)
+    # - scheduler_priority (default: 100)
     #
     # @param interval [Integer, nil] Seconds until next run (default: from config)
     # @param priority [Integer, nil] Beanstalkd priority (default: from config)

data/lib/postburner/version.rb

@@ -1,3 +1,3 @@
 module Postburner
-  VERSION = '1.0.0.pre.14'
+  VERSION = '1.0.0.pre.15'
 end

data/lib/postburner/worker.rb

@@ -110,6 +110,7 @@ module Postburner
       all_tubes = config.expanded_tube_names + [config.scheduler_tube_name]
       logger.info "[Postburner] #{config.beanstalk_url} known tubes: #{all_tubes.join(', ')}"
       log_next_scheduler_watchdog
+      ensure_watchdog_on_startup!
 
       if worker_config[:forks] > 0
         start_forked_mode
@@ -678,6 +679,29 @@ module Postburner
       Postburner::Scheduler.ensure_watchdog!(connection: connection)
     end
 
+    # Ensure watchdog exists on worker startup.
+    #
+    # Called before spawning forks or threads to ensure the scheduler watchdog
+    # exists. In forked mode, this prevents a race condition where multiple forks
+    # all see no watchdog and each create one. In single-process mode, this
+    # ensures the watchdog exists immediately rather than waiting for first timeout.
+    #
+    # @return [void]
+    # @api private
+    def ensure_watchdog_on_startup!
+      Postburner.connected do |conn|
+        if Postburner::Scheduler.watchdog_exists?(connection: conn)
+          logger.debug "[Postburner::Worker] Watchdog already exists"
+        else
+          logger.info "[Postburner::Worker] Creating scheduler watchdog"
+          Postburner::Scheduler.enqueue_watchdog
+        end
+      end
+    rescue => e
+      logger.warn "[Postburner::Worker] Failed to ensure watchdog on startup: #{e.message}"
+      # Non-fatal - threads will create it on timeout, though may create duplicates in forked mode
+    end
+
     # Deletes a job from Beanstalkd with one retry on failure.
     #
     # If delete fails (e.g., network blip), waits 1 second and retries once.

data/lib/postburner.rb

@@ -428,6 +428,30 @@ module Postburner
     result
   end
 
+  # Clears all configured tubes including scheduler.
+  #
+  # Convenience method that clears all watched tubes plus the scheduler tube
+  # in a single call. This is the equivalent of:
+  #
+  #   Postburner.clear_jobs!(Postburner.watched_tube_names + [Postburner.scheduler_tube_name])
+  #
+  # @param silent [Boolean] If true, suppress output to stdout (default: false)
+  #
+  # @return [Hash] Statistics and results (see Connection#clear_tubes!)
+  #
+  # @example Clear everything
+  #   Postburner.clear_all!
+  #
+  # @example Silent mode
+  #   result = Postburner.clear_all!(silent: true)
+  #
+  # @see #clear_jobs!
+  #
+  def self.clear_all!(silent: false)
+    all_tubes = watched_tube_names + [scheduler_tube_name]
+    clear_jobs!(all_tubes, silent: silent)
+  end
+
   # Returns array of watched tube names with environment prefix.
   #
   # Expands configured queue names to full tube names and memoizes the result.

data/lib/tasks/postburner.rake

@@ -15,4 +15,108 @@ namespace :postburner do
     runner = Postburner::Runner.new(options)
     runner.run
   end
+
+  desc "Show Beanstalkd tube statistics"
+  task stats: :environment do
+    puts "Postburner Tube Statistics (#{Rails.env})"
+    puts "=" * 60
+    Postburner.clear_jobs!
+  end
+
+  # Clear tasks remove jobs from Beanstalkd only. PostgreSQL records (TrackedJob,
+  # Postburner::Job) are not affected. Tracked jobs can be requeued after clearing
+  # if needed:
+  #
+  #   Postburner::TrackedJob.where(processed_at: nil).find_each(&:requeue!)
+  #   Postburner::Job.where(processed_at: nil).find_each(&:requeue!)
+  #
+  namespace :clear do
+    desc "Clear Postburner tubes (jobs + scheduler). Prompts for selective clearing. Beanstalkd only - DB records preserved."
+    task all: :environment do
+      tubes = Postburner.watched_tube_names + [Postburner.scheduler_tube_name]
+      clear_tubes_with_confirmation(tubes, "all Postburner tubes")
+    end
+
+    desc "Clear job tubes only (excludes scheduler). Prompts for selective clearing. Beanstalkd only - DB records preserved."
+    task jobs: :environment do
+      tubes = Postburner.watched_tube_names
+      clear_tubes_with_confirmation(tubes, "job tubes")
+    end
+
+    desc "Clear scheduler watchdog tube only. Beanstalkd only - will be recreated on next worker timeout."
+    task scheduler: :environment do
+      tubes = [Postburner.scheduler_tube_name]
+      clear_tubes_with_confirmation(tubes, "scheduler tube")
+    end
+  end
+
+  # Convenience alias
+  desc "Clear Postburner tubes (alias for clear:all). Beanstalkd only - DB records preserved."
+  task clear: "clear:all"
+
+  def clear_tubes_with_confirmation(tubes, description)
+    puts "Postburner Clear Tubes (#{Rails.env})"
+    puts "=" * 60
+    puts
+    puts "NOTE: This clears Beanstalkd only. Database records are preserved"
+    puts "      and tracked jobs can be requeued if needed."
+    puts
+
+    # Show current stats
+    stats = Postburner.stats(tubes)
+    total_jobs = stats[:totals][:total]
+
+    if total_jobs == 0
+      puts "No jobs to clear in #{description}."
+      return
+    end
+
+    # Collect tubes that actually have jobs
+    tubes_with_jobs = stats[:tubes].select { |t| t[:total] > 0 }
+
+    puts "Tubes with jobs:"
+    tubes_with_jobs.each do |tube|
+      puts "  #{tube[:name]}: #{tube[:total]} jobs (ready: #{tube[:ready]}, delayed: #{tube[:delayed]}, buried: #{tube[:buried]})"
+    end
+    puts
+    puts "Total: #{total_jobs} jobs will be deleted"
+    puts
+
+    # Confirmation prompt - require typing tube short names (last segment)
+    if ENV['FORCE'] == 'true'
+      confirmed_tubes = tubes_with_jobs.map { |t| t[:name] }
+    else
+      # Build mapping of short name -> full name
+      tube_short_names = tubes_with_jobs.map { |t| t[:name].split('.').last }
+      puts "Type the queue name(s) you want to clear (space-separated):"
+      puts "  #{tube_short_names.join(' ')}"
+      puts
+      puts "Only the queues you type will be cleared. Press Enter to abort."
+      puts
+      print "> "
+      response = $stdin.gets&.strip || ""
+      entered_names = response.split(/\s+/).map(&:strip).reject(&:empty?)
+
+      # Match entered short names to full tube names
+      confirmed_tubes = tubes_with_jobs
+        .select { |t| entered_names.include?(t[:name].split('.').last) }
+        .map { |t| t[:name] }
+
+      unrecognized = entered_names - tube_short_names
+      if unrecognized.any?
+        puts
+        puts "Unrecognized queues ignored: #{unrecognized.join(', ')}"
+      end
+    end
+
+    if confirmed_tubes.any?
+      jobs_to_clear = tubes_with_jobs.select { |t| confirmed_tubes.include?(t[:name]) }.sum { |t| t[:total] }
+      puts
+      puts "Clearing #{confirmed_tubes.size} tube(s)..."
+      Postburner.clear_jobs!(confirmed_tubes, silent: true)
+      puts "Done. Cleared #{jobs_to_clear} jobs."
+    else
+      puts "Aborted. No tubes cleared."
+    end
+  end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: postburner
 version: !ruby/object:Gem::Version
-  version: 1.0.0.pre.14
+  version: 1.0.0.pre.15
 platform: ruby
 authors:
 - Matt Smith
@@ -133,8 +133,6 @@ files:
 - bin/postburner
 - bin/rails
 - config/environment.rb
-- config/postburner.yml
-- config/postburner.yml.example
 - config/routes.rb
 - lib/generators/postburner/install/USAGE
 - lib/generators/postburner/install/install_generator.rb

data/config/postburner.yml

@@ -1,22 +0,0 @@
-development:
-  beanstalk_url: beanstalk://localhost:11300
-  worker_type: simple
-  queues:
-    default: {}
-
-test:
-  beanstalk_url: beanstalk://localhost:11300
-  worker_type: simple
-  queues:
-    default: {}
-
-production:
-  beanstalk_url: beanstalk://localhost:11300
-  worker_type: threads_on_fork
-  queues:
-    critical:
-      threads: 1
-      gc_limit: 100
-    default:
-      threads: 5
-      gc_limit: 500

data/config/postburner.yml.example

@@ -1,144 +0,0 @@
-# Postburner Configuration Example
-#
-# Copy this file to config/postburner.yml and customize for your environment.
-#
-# ## Named Workers Configuration
-#
-# Postburner uses named worker configurations to support different deployment patterns:
-# - Single worker: bin/postburner (auto-selects the single worker)
-# - Multiple workers: bin/postburner --worker <name> (must specify which worker)
-#
-# Each worker can have different fork/thread settings and process different queues.
-# This enables running different queue groups in separate OS processes with distinct
-# concurrency profiles.
-#
-# ## Puma-Style Architecture
-#
-# - **forks: 0** = Single process with thread pool (development/staging)
-# - **forks: 1+** = Multiple processes with thread pools (production)
-#
-# Scale by adjusting forks and threads per worker:
-# - Development: forks=0, threads=1 (single-threaded, easiest debugging)
-# - Staging: forks=0, threads=10 (multi-threaded, moderate load)
-# - Production: forks=4, threads=10 (40 concurrent jobs per worker)
-#
-
-default: &default
-  # Beanstalkd connection URL
-  # Override with ENV['BEANSTALK_URL'] if set
-  beanstalk_url: <%= ENV['BEANSTALK_URL'] || 'beanstalk://localhost:11300' %>
-
-development:
-  <<: *default
-
-  workers:
-    default:
-      # Single-threaded, single process (simplest for debugging)
-      # If not specified, uses env-level defaults
-      queues:
-        - default
-        - mailers
-
-test:
-  <<: *default
-
-  workers:
-    default:
-      # Test mode uses inline strategies automatically
-      queues:
-        - default
-
-staging:  # <- environment config
-  <<: *default
-
-  # Env-level defaults (use default_ prefix)
-  default_threads: 10
-  default_gc_limit: 5000
-
-  workers:  # <- worker config overrides
-    default:
-      # Multi-threaded, single process (moderate concurrency)
-      # Uses env-level defaults: default_threads=10, default_gc_limit=5000
-      queues:
-        - critical
-        - default
-        - mailers
-
-production:  # <- environment config, i.e. defaults, NOT worker config
-  <<: *default
-
-  # Env-level defaults (use default_ prefix)
-  default_forks: 2
-  default_threads: 10
-  default_gc_limit: 5000
-
-  # Example 1: Single worker using env defaults
-  # Run: bin/postburner
-  #
-  # workers:
-  #   default:
-  #     # Uses default_forks=2, default_threads=10
-  #     queues:
-  #       - critical
-  #       - default
-  #       - mailers
-  #       - imports
-
-  # Example 2: Multiple workers with different concurrency profiles
-  # Run separate processes:
-  #   bin/postburner --worker imports    (4 forks, 1 thread each)
-  #   bin/postburner --worker general    (2 forks, 100 threads each)
-  #
-  workers:  # <- worker config, i.e. overrides, NOT environment config
-    # Heavy, memory-intensive jobs - more processes, fewer threads
-    imports:
-      forks: 4              # Overrides default_forks
-      threads: 1            # Overrides default_threads
-      gc_limit: 500         # Overrides default_gc_limit
-      queues:
-        - imports
-        - data_processing
-
-    # General jobs - uses env defaults (forks=2, threads=10)
-    # Override threads for higher concurrency
-    general:
-      threads: 100          # Overrides default_threads (forks uses default_forks=2)
-      queues:
-        - default
-        - mailers
-        - notifications
-
-  # Example 3: Fine-grained control with multiple specialized workers
-  # Run separate processes:
-  #   bin/postburner --worker critical
-  #   bin/postburner --worker default
-  #   bin/postburner --worker mailers
-  #
-  # workers:
-  #   critical:
-  #     forks: 1
-  #     threads: 1
-  #     gc_limit: 100
-  #     queues:
-  #       - critical
-  #
-  #   default:
-  #     forks: 4
-  #     threads: 10
-  #     queues:
-  #       - default
-  #
-  #   mailers:
-  #     forks: 2
-  #     threads: 5
-  #     queues:
-  #       - mailers
-
-# Env-Level Defaults (can be overridden per worker):
-#
-# default_queue: default           # Default queue name (optional)
-# default_priority: 65536          # Lower = higher priority (optional, 0 is highest)
-# default_ttr: 300                 # Time-to-run in seconds (optional)
-# default_threads: 1               # Thread count per fork (optional, defaults to 1)
-# default_forks: 0                 # Fork count (optional, defaults to 0 = single process)
-# default_gc_limit: nil            # Exit after N jobs for restart (optional, nil = no limit)