postburner 1.0.0.rc.5 → 1.0.0.rc.7

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 62187fb2fd681e87e1028c52cbd6f3756b32c895c05390bffca05d51e1287ec7
-  data.tar.gz: 31ab083d1e9d00bfb7dc6fabbf09d7065f5062d73a6cd738835a0b76f54426b6
+  metadata.gz: 41b3c95f8abde17bb19b725dd0c39ce65b97a53e73afb77c7237e2e51f37d7e9
+  data.tar.gz: 7e5de5424a027d59c9552624bd555ae38da4f30823f3ffc28ef86601c96301e0
 SHA512:
-  metadata.gz: d2e7a422e53852490b5e2ec8502f3dd69a34f17975ee808f463658f20f2ca228c5467d149987caaf7e905433b364b30c7f92142e07e7f7da3bbd7763d99c6284
-  data.tar.gz: aa6ce11d8f045be5b259a1869339117ddf40fca8e6d3579ced469355d2c89fbd7f19da96903154dac11a0f0efed57db840c1581ba9512c2665300d58d0453e32
+  metadata.gz: d2f90b3d590adfc16247f285f8ea27cef03d9dd72af5671bc92117d2418442dcf6f331c68d5380a2917687ea2353524f972fdd7edeca4fe9a8e1b6b965dff9ef
+  data.tar.gz: 06accd7ecd5c01aa83532e7f8cd69e2aca53459e7b8f8f8c0a290163555a2f24f47df8ecb4407e39d7401680547b7f016b9111221688e5dc10c2fea43eb49ba6

data/CHANGELOG.md

@@ -1,5 +1,92 @@
 # Changelog
 
+## v1.0.0.rc.7 - 2026-08-03
+
+### Fixed
+
+- **Delayed jobs can release before their `run_at`.** Beanstalkd delays are whole seconds, so a fractional delay has to round. Insertions are now computed as `(at - now).ceil`.
+- **Premature re-insert could spin.** `NiceQueue`/`DefaultQueue` re-inserted a premature job with a fractional second value. Re-insert delay is now `[early.ceil, 1].max`.
+
+### Changed
+
+- Premature-job messages now report how early the job arrived: the `PREMATURE; RE-INSERTED` audit log entry and the `Postburner::Job::PrematurePerform` message both include the delta in seconds, which makes sub-second scheduling drift visible instead of silent.
+
+## v1.0.0.rc.6 - 2026-06-27
+
+### Highlights
+
+- `Postburner::Schedule#reconcile!` — guarantees exactly one live future execution per enabled schedule and zero future executions for a disabled schedule. The watchdog delegates to it.
+- `Postburner::Schedule#enable!` / `#disable!` — convenience wrappers that drive reconciliation.
+- Auto-reconcile on schedule edits — an `after_update_commit` hook reconciles whenever a scheduling/snapshot attribute changes.
+- `Postburner::ScheduleExecution#supersede!` — tears down an execution (Beanstalkd job + `Postburner::Job` AR row) and marks it `superseded`.
+- `superseded` status (enum value `201`) on `Postburner::ScheduleExecution`, distinct from `skipped`. Neither counts as a live future execution.
+- `live` and `future_live` scopes on `Postburner::ScheduleExecution` (`pending`/`scheduled` only; `future_live` adds `run_at > now`).
+- Instrumentation events `supersede.schedule_execution.postburner` and `reconcile.schedule.postburner`.
+- **`ScheduleExecution#skip!` now skips one occurrence, then resumes -- tearing down down the associated `Postburner::Job` **and** the Beanstalkd job. The skipped row is retained as history.
+- **`schedule.destroy` now tears down each execution's Beanstalkd job and `Postburner::Job` (no orphans)
+- The watchdog sweeps **disabled** schedules with a lingering live future execution and supersedes them.
+
+### Fixed
+
+- Scheduled **tracked ActiveJob** executions had a `nil` `bkid`: the adapter stored the entire Beanstalkd put response hash (`{status:, id:}`) instead of the id, which cast to `nil` in the `bigint` column. Scheduled **non-tracked ActiveJob** executions discarded the Beanstalkd id entirely (`beanstalk_job_id` was always `nil`), so they could not be cancelled. Both now capture the real id via the adapter's `provider_job_id`.
+- Scheduled **`Postburner::Job`** executions stored the wrong value in `ScheduleExecution#beanstalk_job_id` (the return value of `queue!`, not the Beanstalkd id). It now holds the real `job.bkid`.
+- `Postburner::OrphanedJob#destroy` can be soft-removed (`remove!`) and destroyed (`destroy`), with Beanstalkd job, while `readonly?` still blocks changes.
+
+### Upgrade notes
+
+- This is a **template mutation** (like rc.4), not an additive migration. Existing installs must add a migration that:
+  1. Replaces the total unique index on `(schedule_id, run_at)` with a **live-only partial** unique index (`WHERE status IN (0, 11)`), so a superseded/skipped row can coexist with a freshly recreated live execution at the same `run_at`.
+  2. Drops `ON DELETE CASCADE` on the `schedule_id` foreign key (teardown is now owned by `dependent: :destroy` + `before_destroy`).
+- **Behavior change — deleting a schedule.** With `ON DELETE CASCADE` removed, deleting a schedule by any path other than ActiveRecord `destroy` — raw SQL, `Schedule.delete`, `where(...).delete_all` — now raises a foreign-key violation when the schedule has executions, instead of silently cascading. Always use `schedule.destroy` (which runs `dependent: :destroy` + the `before_destroy` teardown). This is intentional: it prevents orphaned Beanstalkd jobs and `Postburner::Job` rows.
+
+Standalone upgrade migration (written for zero-downtime on large `postburner_schedule_executions` tables — builds the index `CONCURRENTLY` and adds the FK unvalidated then validates it, neither of which blocks writes; requires `disable_ddl_transaction!`):
+
+```ruby
+class UpgradePostburnerSchedulesForReconcile < ActiveRecord::Migration[7.2]
+  disable_ddl_transaction!
+
+  # Existing installs created the unique index via the generator with Rails'
+  # default name. Adjust if your install named it differently.
+  OLD_INDEX = 'index_postburner_schedule_executions_on_schedule_id_and_run_at'
+  NEW_INDEX = 'index_pb_sched_exec_live_schedule_run_at'
+
+  def up
+    # 1. Add the live-only partial unique index CONCURRENTLY (no write lock) so a
+    #    superseded/skipped row can coexist with a live recreate at the same run_at.
+    #    Narrowing uniqueness can't conflict with existing data (the old total
+    #    index was stricter), so this build cannot fail on valid rows.
+    add_index :postburner_schedule_executions, [:schedule_id, :run_at],
+      unique: true, where: 'status IN (0, 11)',
+      name: NEW_INDEX, algorithm: :concurrently, if_not_exists: true
+
+    # Drop the old total unique index. Target by NAME: with the new index added,
+    # two indexes now match the (schedule_id, run_at) columns.
+    remove_index :postburner_schedule_executions,
+      name: OLD_INDEX, algorithm: :concurrently, if_exists: true
+
+    # 2. Replace the cascading FK with a plain one. Add unvalidated first (brief
+    #    metadata-only lock), then validate separately (does not block writes).
+    #    dependent: :destroy + before_destroy now own teardown.
+    remove_foreign_key :postburner_schedule_executions, column: :schedule_id
+    add_foreign_key :postburner_schedule_executions, :postburner_schedules,
+      column: :schedule_id, validate: false
+    validate_foreign_key :postburner_schedule_executions, column: :schedule_id
+  end
+
+  def down
+    remove_foreign_key :postburner_schedule_executions, column: :schedule_id
+    add_foreign_key :postburner_schedule_executions, :postburner_schedules,
+      column: :schedule_id, on_delete: :cascade, validate: false
+    validate_foreign_key :postburner_schedule_executions, column: :schedule_id
+
+    add_index :postburner_schedule_executions, [:schedule_id, :run_at],
+      unique: true, name: OLD_INDEX, algorithm: :concurrently, if_not_exists: true
+    remove_index :postburner_schedule_executions,
+      name: NEW_INDEX, algorithm: :concurrently, if_exists: true
+  end
+end
+```
+
 ## v1.0.0.rc.5 - 2026-06-26
 
 ### Fixed

data/README.md

@@ -126,7 +126,7 @@ Postburner [beanstalkd](https://beanstalkd.github.io/) is used with PostgreSQL t
 
 ```ruby
 # Gemfile
-gem 'postburner', '~> 1.0.0.pre.18'
+gem 'postburner', '~> 1.0.0.rc.6'
 
 # config/application.rb
 config.active_job.queue_adapter = :postburner
@@ -499,8 +499,7 @@ The scheduler uses **immediate enqueue** combined with a **watchdog safety net**
    ```
 4. When a worker reserves the watchdog, it instantiates `Postburner::Scheduler` which:
    - Acquires a PostgreSQL advisory lock for coordination
-   - Auto-bootstraps any unstarted schedules
-   - Ensures each schedule has a future execution queued
+   - Reconciles every schedule (see [Editing & Pausing Schedules](#editing--pausing-schedules)) — auto-bootstrapping unstarted schedules, ensuring each enabled schedule has exactly one future execution queued, and tearing down lingering futures on disabled schedules
    - Re-queues a new watchdog with delay for the next interval
 
 NOTE: The watchdog is ephemeral data in Beanstalkd, not a database record. `Postburner::Scheduler` is the handler class that does the work. This design requires no dedicated scheduler process - existing workers handle everything.
@@ -687,8 +686,9 @@ Postburner::Schedule.create!(
 schedule = Postburner::Schedule.find_by(name: 'daily_cleanup')
 Postburner::Schedule.enabled  # All enabled schedules
 
-# Disable temporarily
-schedule.update!(enabled: false)
+# Pause / resume (preferred — see "Editing & Pausing Schedules" below)
+schedule.disable!  # Tears down the queued future execution
+schedule.enable!   # Resumes from the next grid point after now
 
 # Change catch-up policy
 schedule.update!(catch_up: true)
@@ -700,11 +700,68 @@ schedule.next_run_at  # => 2025-01-02 09:00:00 -0500
 schedule.next_run_at_times(5)  # Next 5 run times
 
 # View executions
+schedule.executions.live        # pending + scheduled (real queued work)
+schedule.executions.future_live # the live future execution(s)
 schedule.executions.pending
 schedule.executions.scheduled
-schedule.executions.skipped
+schedule.executions.skipped     # cancelled occurrences (history)
+schedule.executions.superseded  # replaced by a recreate (history)
+
+# Delete a schedule (ALWAYS use destroy — see note below)
+schedule.destroy  # Tears down every execution's Beanstalkd job + Postburner::Job row
 ```
 
+**Deleting schedules — always use `destroy`.** `schedule.destroy` runs `dependent: :destroy` plus a `before_destroy` teardown that removes each execution's Beanstalkd job and any `Postburner::Job` row, so nothing is orphaned. Deleting a schedule by any other path — raw SQL, `Schedule.delete`, `where(...).delete_all` — raises a foreign-key violation when the schedule has executions, by design. Reach for `schedule.destroy` (or `disable!` to keep the record and history).
+
+#### Editing & Pausing Schedules
+
+Postburner keeps a schedule's queued work in sync with its configuration through a single, idempotent convergence path: **reconciliation**. The guarantee is simple:
+
+- An **enabled** schedule always has **exactly one** live future execution, sitting on the current grid with a snapshot matching the live config.
+- A **disabled** schedule has **zero** future executions.
+
+You rarely call reconciliation directly (`schedule.reconcile!` exists if you need it). Instead it runs for you in three places: the watchdog reconciles every schedule on each pass, schedule edits trigger it automatically, and `enable!` / `disable!` drive it.
+
+##### Pausing & resuming
+
+Use `disable!` / `enable!` rather than `update!(enabled: ...)` (both work, but these read better and make intent obvious):
+
+```ruby
+schedule.disable!  # Supersedes the queued future execution → zero future executions
+schedule.enable!   # Recreates a future execution at the next grid point after now
+```
+
+Disabling now actually tears down the queued future execution in Beanstalkd — it doesn't just stop creating new ones. Enabling resumes from the **next grid point after now**; it does **not** backfill the gap while the schedule was paused (`catch_up` only governs the running-job chaining path, not resume).
+
+##### Editing a schedule
+
+Editing a scheduling/snapshot attribute — `args`, `queue`, `priority`, the grid (`anchor`, `interval`, `interval_unit`, `cron`), `timezone`, `enabled`, or `name` — automatically supersedes the stale queued execution and recreates it from the current config (via an `after_update_commit` hook):
+
+```ruby
+# Payload-only edit: recreates the future execution at the SAME run_at
+schedule.update!(args: { report_type: 'weekly' })
+
+# Grid edit: recreates the future execution at the NEW grid point
+schedule.update!(interval: 2, interval_unit: 'hours')
+```
+
+> **Best-effort, not synchronous.** This inline reconcile is non-blocking: it *tries* the global scheduler advisory lock without waiting, so it never holds up a web request. If the watchdog happens to hold the lock mid-sweep, the inline reconcile is skipped and the watchdog converges on its next pass. For UI-managed schedules, **don't assume the next run is updated immediately** — read it back after the watchdog interval (`scheduler_interval`) rather than right after the save.
+
+##### Skipping a single occurrence
+
+`skip!` cancels **one** upcoming occurrence and then resumes at the next grid point after the skipped slot. It tears down both the Beanstalkd job and the associated `Postburner::Job` row, marks the execution `skipped` (kept as history), and triggers a best-effort reconcile. The skipped occurrence does not run and is not recreated:
+
+```ruby
+execution = schedule.executions.future_live.first
+execution.skip!  # This one won't run; the schedule continues at the next grid point
+```
+
+To pause every upcoming run instead of just one, use `disable!`.
+
+##### Superseding (internal)
+
+`ScheduleExecution#supersede!` performs the same teardown as `skip!` but records a `superseded` status, meaning the execution was replaced by a recreate (config/grid drift resolved by reconciliation) rather than cancelled by an operator. Reconciliation uses it internally; you'll mostly encounter `superseded` rows when reading execution history. Like skipped rows, superseded rows are excluded from `live` / `future_live`.
+
 #### Starting Schedules
 
 When you create a schedule, it won't run until the first execution is created. You have two options:
@@ -748,19 +805,27 @@ Each scheduled run creates an execution record for tracking:
 ```ruby
 execution = Postburner::ScheduleExecution.find(123)
 
-execution.status        # pending, scheduled, skipped
+execution.status        # pending, scheduled, skipped, superseded
 execution.run_at        # Scheduled time
 execution.enqueued_at   # When job was queued
 execution.beanstalk_job_id  # Beanstalkd job ID
 execution.job_id            # Postburner::Job ID (if using Postburner::Job)
 ```
 
+**Status values:**
+- `pending` — created, not yet enqueued to Beanstalkd
+- `scheduled` — enqueued to Beanstalkd, waiting for `run_at`
+- `skipped` — one occurrence cancelled by an operator (`skip!`); retained as history
+- `superseded` — replaced by a recreate during reconciliation (`supersede!`); retained as history
+
+Only `pending` and `scheduled` count as a live future execution (the `live` / `future_live` scopes). `skipped` and `superseded` are inert history.
+
 **Execution lifecycle:**
 1. Execution created with `pending` status and immediately enqueued to Beanstalkd
 2. Status changes to `scheduled` once enqueued
 3. At `run_at` time, Beanstalkd releases job to worker
 4. For `Postburner::Job` (and `ActiveJob` with `Postburner::Tracked`) schedules: `before_attempt` callback creates next execution
-5. Watchdog periodically verifies future executions exist (safety net)
+5. Watchdog periodically reconciles each schedule, guaranteeing future executions exist (safety net)
 
 #### Timezone Handling
 
@@ -1572,7 +1637,8 @@ Postburner emits ActiveSupport::Notifications events following Rails conventions
 |-------|------|--------------|
 | `create.schedule.postburner` | When schedule is created | `:schedule` |
 | `update.schedule.postburner` | When schedule is updated | `:schedule`, `:changes` |
-| `audit.schedule.postburner` | When scheduler audits a schedule | `:schedule` |
+| `audit.schedule.postburner` | When scheduler audits a schedule | `:schedule`, `:execution_created`, `:orphans_enqueued` |
+| `reconcile.schedule.postburner` | When a schedule is reconciled (executions converged to config) | `:schedule`, `:superseded`, `:created` |
 
 **Schedule Payload Structure:**
 
@@ -1595,7 +1661,8 @@ Postburner emits ActiveSupport::Notifications events following Rails conventions
 |-------|------|--------------|
 | `create.schedule_execution.postburner` | When execution is created | `:schedule`, `:execution` |
 | `enqueue.schedule_execution.postburner` | When execution is enqueued to Beanstalkd | `:schedule`, `:execution`, `:beanstalk_job_id` |
-| `skip.schedule_execution.postburner` | When execution is skipped | `:schedule`, `:execution` |
+| `skip.schedule_execution.postburner` | When an operator skips one occurrence | `:schedule`, `:execution` |
+| `supersede.schedule_execution.postburner` | When an execution is superseded (replaced by a recreate during reconciliation) | `:schedule`, `:execution` |
 
 **Execution Payload Structure:**
 
@@ -2368,7 +2435,7 @@ Key changes in v1.0:
 
 1. **Update Gemfile:**
    ```ruby
-   gem 'postburner', '~> 1.0.0.pre.18'
+   gem 'postburner', '~> 1.0.0.rc.6'
    ```
 
 2. **Remove Backburner config:**

data/app/concerns/postburner/insertion.rb

@@ -85,7 +85,15 @@ module Postburner
       self.run_at = case
                     when at.present?
                       # this is rudimentary, add error handling
-                      options[:delay] ||= at.to_i - now.to_i
+                      #
+                      # Ceil, don't floor. Beanstalkd delays are whole seconds,
+                      # so some rounding is unavoidable -- but it must round
+                      # *away* from now. Flooring both sides (`at.to_i -
+                      # now.to_i`) drops frac(at) and frac(now) independently,
+                      # releasing the job early by frac(at) - frac(now)
+                      # whenever that is positive. Consumers then see a job
+                      # arrive before its own run_at.
+                      options[:delay] ||= [(at.to_time - now).ceil, 0].max
                       at
                     when options[:delay].present?
                       now + options[:delay].seconds

data/app/models/postburner/orphaned_job.rb

@@ -33,12 +33,12 @@ module Postburner
   # application code.
   #
   # @example Typical lifecycle
-  #   # A DB row exists with type: 'Flex::TriggerNoShowDeliveriesJob' (class deleted)
+  #   # A DB row exists with type: 'MyApp::RemovedJob' (class deleted)
   #   job = Postburner::Job.find(42)
   #   job.class         # => Postburner::OrphanedJob
-  #   job.type          # => 'Flex::TriggerNoShowDeliveriesJob'
+  #   job.type          # => 'MyApp::RemovedJob'
   #   job.orphaned?     # => true
-  #   job.missing_class_name  # => 'Flex::TriggerNoShowDeliveriesJob'
+  #   job.missing_class_name  # => 'MyApp::RemovedJob'
   #   job.perform             # => raises Postburner::Job::OrphanedJobError
   #   job.remove!             # soft-deletes fine despite readonly?
   #
@@ -84,6 +84,10 @@ module Postburner
     # through instance save/update and therefore does not trigger
     # `ensure_proper_type` either).
     #
+    # Uses the STI base class for the relation so the row's missing `type` value
+    # (which differs from OrphanedJob's sti_name) does not scope it out — a
+    # `self.class.where` would match zero rows and silently fail to persist.
+    #
     # Idempotent: does nothing if already removed.
     #
     # @return [void]
@@ -93,10 +97,31 @@ module Postburner
 
       self.delete!
       now = Time.current
-      self.class.where(id: self.id).update_all(removed_at: now)
+      self.class.base_class.where(id: self.id).update_all(removed_at: now)
       self.removed_at = now
     end
 
+    # Hard-deletes this orphaned row, removing BOTH its Beanstalkd job and its AR
+    # row, while keeping `readonly?` true for saves/updates.
+    #
+    # The base ActiveRecord#destroy raises ReadOnlyRecord on a readonly record,
+    # which would make ScheduleExecution#teardown_job! (which calls `job.destroy`
+    # uniformly across job shapes) fail for an execution whose job class was
+    # deleted/renamed. This override mirrors {#remove!}: it removes the Beanstalkd
+    # job via {#delete!} and deletes the row at the relation level — using the STI
+    # base class so the missing `type` value doesn't scope the row out — bypassing
+    # both `readonly?` and instance callbacks. The row is genuinely removed (not a
+    # soft-delete), matching destroy semantics.
+    #
+    # @return [self] frozen, with destroyed? == true
+    #
+    def destroy
+      self.delete!
+      self.class.base_class.where(id: self.id).delete_all
+      @destroyed = true
+      freeze
+    end
+
     # Prevents saves that would allow Rails' ensure_proper_type to overwrite the
     # original `type` value with 'Postburner::OrphanedJob'.
     #