pgbus 0.20.1 → 0.21.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +58 -8
  3. data/README.md +1 -1
  4. data/app/helpers/pgbus/workflow_insights_helper.rb +8 -0
  5. data/app/models/pgbus/workflow_entry.rb +3 -2
  6. data/app/views/pgbus/workflows/definitions/_step_card.html.erb +3 -0
  7. data/config/locales/da.yml +3 -0
  8. data/config/locales/de.yml +3 -0
  9. data/config/locales/en.yml +3 -0
  10. data/config/locales/es.yml +3 -0
  11. data/config/locales/fi.yml +3 -0
  12. data/config/locales/fr.yml +3 -0
  13. data/config/locales/it.yml +3 -0
  14. data/config/locales/ja.yml +3 -0
  15. data/config/locales/nb.yml +3 -0
  16. data/config/locales/nl.yml +3 -0
  17. data/config/locales/pt.yml +3 -0
  18. data/config/locales/sv.yml +3 -0
  19. data/lib/generators/pgbus/templates/add_workflows.rb.erb +3 -1
  20. data/lib/generators/pgbus/templates/migration.rb.erb +3 -1
  21. data/lib/pgbus/configuration/workflow_settings.rb +1 -1
  22. data/lib/pgbus/mcp/tools/workflow_stats_tool.rb +1 -1
  23. data/lib/pgbus/mcp/tools/workflows_attention_tool.rb +1 -1
  24. data/lib/pgbus/version.rb +1 -1
  25. data/lib/pgbus/web/data_source/workflow_list.rb +6 -4
  26. data/lib/pgbus/web/workflow_insights_report.rb +11 -3
  27. data/lib/pgbus/web/workflow_state.rb +5 -2
  28. data/lib/pgbus/workflow/instance.rb +36 -3
  29. data/lib/pgbus/workflow/runner.rb +2 -2
  30. data/lib/pgbus/workflow/signal_peek.rb +47 -0
  31. data/lib/pgbus/workflow/signals.rb +31 -10
  32. data/lib/pgbus/workflow/step_definition.rb +45 -3
  33. data/lib/pgbus/workflow/step_execution.rb +27 -10
  34. data/lib/pgbus/workflow/wait_timeout.rb +38 -7
  35. data/lib/pgbus/workflow.rb +11 -5
  36. data/sig/pgbus.rbs +43 -0
  37. metadata +2 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c2c853eab253c7e6bbf570e0cd92ada59c071752fb3d6d8a0a5f796790aedfa3
4
- data.tar.gz: 8922bdb2140718d03a2bf1cf3600c714fc2b181c566f1213091f83b28385b74c
3
+ metadata.gz: 1c3c6921f17cfc002f7177240a3d1b2c202a78c7127e8f0da75a75ebcaf365d1
4
+ data.tar.gz: b73396771876a230b1b62b6b30dceb87329be7259810c5346fd4a3fc78f59991
5
5
  SHA512:
6
- metadata.gz: d9d38d6bb97efc078db1afa031b51cfe2512aac285d664d09e23ce8a4a3b1a3eda075134a8b7e8d234834c31c543a6095c3cdb3bbef77cec3f3f53744e7019eb
7
- data.tar.gz: 0e15d3375c20815ea3f0fbc34381e824c308f7210b67189fae56a1f11abeb3381d9b71b2b418e46c4d24bf5f5b80df3360adfccf5c7b77a02a6533d963b30699
6
+ metadata.gz: 5667ec2a6ebd1ef9e09f5f89818e69784267b3aa0c13a0b5cef2782b346af2361c106454fbb58d67a688b48f451483327fa4f3edc0a1d9a82664aa599af52454
7
+ data.tar.gz: 83d7a44001fe09db593caacc788307198361279c023b7ed2ed8397fdec17fb353fa44087d9a973857951835d6cb1206314ade872a99d85586988561be2477a85
data/CHANGELOG.md CHANGED
@@ -1,15 +1,68 @@
1
1
  ## [Unreleased]
2
2
 
3
+ ## [0.21.0] - 2026-10-19
4
+
5
+ ### Added
6
+
7
+ - **feat(workflow): `wait_timeout:` takes a per-run deadline — a method name or a lambda, returning a duration or a Time (issue #574).** `step :await_reply, wait_for: :replied, wait_timeout: -> { subject.expires_at }, on_timeout: :run` (or `wait_timeout: :reply_deadline`, an instance method) is evaluated once per attempt, when the step parks, on the workflow instance (`instance_exec`, as `skip_if:` is, with `subject` and `arguments`). It returns a `Time` (or anything with `#to_time` that is not a String): the deadline itself, so `timeout_at` no longer drifts by the time the run took to reach the wait (the steps before it, queue latency, a reattempt); a deadline already past is `timeout_at = now`, and the next sweep applies `on_timeout`. Or a Duration or a number of seconds, counted from park and required to be positive. Or `nil`: no timeout for this attempt. Anything else raises `ArgumentError` at the gate, which, like an error the callable raises, goes through the step's exception policies (the default pauses the run), so the run is never left parked silently. The callable runs under the run's row lock in a savepoint, so a query of its own that fails in the database (a dropped column, a `statement_timeout`) is handled the same way instead of aborting the run's transaction. `on_timeout:` still needs `wait_timeout:`, and a callable counts as given (its default is `:pause!`). Under `on_timeout: :notify` the callable is evaluated again at each notice: a `Time` after now is the next deadline, a duration moves the deadline one period on as a static `wait_timeout:` does, and `nil` or a `Time` not after now (the same `expires_at`) clears `timeout_at`, so the wait is reported once and keeps waiting with no deadline instead of re-firing on a past time (the Workflows list and the run's page still say since when it is late, and that no further late notice is due); one that raises or returns anything else at a notice pauses the run with that error (the sweep holds the run's row lock, under which no `on_exception` block may run). A pause still moves the deadline by its length, as for a static one. `StepDefinition#wait_timeout` stays the static seconds (nil for a callable); `#wait_timeout_callable` and `#dynamic_wait_timeout?` read the per-run one. The definitions page (`/pgbus/workflows/definitions/:workflow_class`) shows each signal step's wait timeout, `dynamic (per run)` for a callable, and `pgbus_workflow_stats` sends it per step as `wait_timeout` (seconds or `"dynamic"`). A static `wait_timeout:` (seconds or a Duration) is unchanged, validated when the class loads and counted from park. Refs #574.
8
+ - **feat(workflow): `woken?` and `signal_ready?` — a `wait_for:` step's guards can tell a signalled step from one that has not waited yet (issue #572).** Guards run before the signal gate, on every claim of the step, so a `skip_if:` on the state the signal reports (skip a wait the record already passed) recorded a step `signal!` woke as `skipped` and never ran its body, and a signal sent before the step's first claim stayed `pending` for good because the gate never ran. Two step readers, usable in `skip_if:`, `cancel_if` and the body, answer it without claiming anything (the gate still claims, after the guards): `signal_ready?` is true when the gate runs this attempt with a signal, the one the step holds or a pending one its `wait_for:` matcher takes; `woken?` is true when this attempt holds a signal before its gate (`signal!` woke it, or an earlier attempt took one and reattempted or failed into a pause), false on a first claim even with a buffered signal. Both keep one answer for the whole attempt, are false after an `on_timeout: :run` wake and in a step without `wait_for:`, and raise `Pgbus::Workflow::NotInStep` outside a step. The recipe: `step :await_shipped, wait_for: :shipped, skip_if: -> { subject.shipped? && !signal_ready? } do … end` skips a run that started after the order shipped and completes a woken step or one whose signal came first, consuming the signal. `skip_if` still runs on every claim, and no step outcome changes for an existing class. A guard that skips takes no signal: a pending one stays pending for a later `wait_for:` step, as with an outside `skip!`. Refs #572.
9
+ - **feat(workflow): `start!` takes `actor:` and `reason:` for the audit trail (issue #571).** `Pgbus::Workflow.start!(subject = nil, idempotency_key: nil, actor: nil, reason: nil, **arguments)`: the run's `started` event records the actor and the reason, and its first `step_scheduled` event the actor, as `rerun!` and every outside verb already did, so a run started by a person says who started it and why (`ReportWorkflow.start!(actor: "alice", reason: "monthly close")`). Without them the trail says `"app"` with no reason, as before; the default is `actor || "app"`, the outside verbs' convention, so `actor: ""` is kept as given. Neither is stored in the run's `arguments` or changes the run row. A duplicate start (same `idempotency_key`, `duplicate_start?` true) still writes no events, so the first caller's actor stays on the trail. Refs #571.
10
+
11
+ ### Changed
12
+
13
+ - **`actor:` and `reason:` are reserved keywords of `Pgbus::Workflow.start!` (issue #571).** Like `idempotency_key:`, they are taken for the audit trail and no longer reach the run's `arguments`. A workflow that receives an argument literally named `actor` or `reason` through `start!` renames it, e.g. `start!(subject, requested_by: "alice")` read as `arguments[:requested_by]`. Runs already started keep their stored arguments, and `rerun!` copies them as stored. Refs #571.
14
+
15
+ ### Fixed
16
+
17
+ - **The workflows one-ongoing index predicate dumps the same `schema.rb` line from a migrated and from a schema-loaded database (issue #573).** `pgbus:add_workflows` and the fresh-install migration declared `idx_pgbus_workflows_one_ongoing` with `where: "status IN ('ready','performing','waiting','paused')"`. `status` is a `varchar`, so Postgres stored the list as `varchar` literals cast to `text[]`; `db:schema:load` parses that dumped string into a different expression, and each later dump flipped the line depending on which kind of database it came from. Both templates now write `status::text IN (…)`, whose stored form replays to itself. The index means the same either way, so existing installs need nothing. An app that wants a stable dump can recreate the index in a migration of its own: `remove_index :pgbus_workflows, name: "idx_pgbus_workflows_one_ongoing"`, then `add_index :pgbus_workflows, [:workflow_class, :subject_type, :subject_id], unique: true, where: "status::text IN ('ready','performing','waiting','paused')", name: "idx_pgbus_workflows_one_ongoing"` (both run in the migration's transaction, so the constraint never lapses; `remove_index` takes an ACCESS EXCLUSIVE lock, so `pgbus_workflows` is locked against reads and writes (workers, workflow starts and the dashboard wait) until the migration commits), and dump the schema again. Refs #573.
18
+ - **A guard that ends a `wait_for:` step after a reattempt consumes the signal the step holds (issue #572).** A woken attempt that called `reattempt!` (or failed into a pause and was resumed) kept its signal `claimed` for the next attempt, whose row carries no `signal_id` until its gate runs. When `cancel_if` or `skip_if` ended that next attempt (or a guard raised and its `on_exception` policy skipped or cancelled, or a deploy had declared the step a `removed_step` meanwhile), the signal stayed `claimed` forever. The attempt now records the held signal before its outcome, so it is consumed when the step ends and kept when it reattempts or pauses, as for any attempt. Refs #572.
19
+
20
+ ## [0.20.1] - 2026-10-18
21
+
3
22
  ### Added
4
23
 
5
24
  - **feat(workflow): `on_timeout: :notify` — a soft wait deadline that reports a late signal and keeps waiting.** A fifth `on_timeout:` value for a `wait_for:` step with `wait_timeout:`. When the sweep (or `expire_wait`) finds the wait past `timeout_at`, it records the `wait_timed_out` event and instruments `pgbus.workflow.wait_timed_out` with `action: "notify"`, as the other values do, and then leaves the run and the step as they are: same attempt, still `waiting`, no step outcome, no StepJob. Only `timeout_at` moves, to the next deadline on the `wait_timeout:` grid (one `wait_timeout:` on; a sweep that missed several deadlines reports them once and moves to the first one ahead of now), so a late signal still wakes that attempt and its body runs with `received_signal` set and `timed_out?` false. A paused run's wait still does not fire while paused. Replaces the `on_timeout: :run` + `reattempt!` workaround, whose `reattempted` outcomes counted as failures in `pgbus_workflow_steps_total`. New counter `pgbus_workflow_wait_overdue_total{workflow_class, step}` (Prometheus, StatsD, AppSignal), one per late notice (a sweep that missed several deadlines sends one). The dashboard says "Waiting for signal `approved` since 3h ago (late since 2h ago). The next late notice goes out in 1h" once a deadline passed (and "A late notice goes out …" before), and the timeline marks every `wait_timed_out` with a dashed line, listed as text below the chart; the workflow rows (and so the `pgbus_workflows` / `pgbus_workflow` MCP tools) gain `on_timeout` and `late_since` (the first missed deadline). The bundled AppSignal dashboard gets a "Late workflow waits" graph. Refs #565.
25
+
26
+ ### Fixed
27
+
28
+ - **An `:until_executed` job keeps its uniqueness key while its retry waits (issue #564).** A job that rescheduled itself with `retry_job(wait:)` (or `retry_on … wait:`) enqueued the retry through the key it held (#333), but the key row kept pointing at the running attempt, and the executor deleted the row by key when that attempt archived. For the whole wait nothing held the key: a fresh `perform_later` with the same key ran beside the retry, and the retry's own success then deleted the newcomer's row. Now the row is handed to the retry the way a first enqueue takes one: before the retry is sent or parked it is unbound onto the retry's queue and re-stamped (`UniquenessKey.unbind!`), and once sent it is bound to the retry (`UniquenessKey.bind!(…, acquired_at:)`). A retry parked behind its `limits_concurrency` slot (with `to: 1, on_conflict: :block` the attempt holds that slot, so its retry always parks) keeps the unbound row until it runs; a retry dropped by its limit, or whose send failed before any message existed, hands the row back to the attempt. A finishing message (success, dead-letter, or a failed `:while_executing` attempt) releases only the row it holds (`UniquenessKey.release_held_by!`): the row bound to it, or an unbound row acquired no later than the message was sent, so a retry that finishes before its bind still releases it; a row handed to a retry, or that a successor took after an operator released it, stays. The dashboard's discards (a job, a failed job, a dead-lettered copy, an event, the bulk discards and queue purges) release the same way, through `UniquenessKey.release_all_held_by!` for the bulk ones, judged by when the job's message was sent: a failed job's queued message gives its own `enqueued_at`, and the dead-letter header gains `source_msg_id` and `source_enqueued_at` so a DLQ copy names its original and when it was sent. Without a send time only a lock bound to the message is freed (the reaper takes the rest). The Locks page's Release still deletes by key. A retry enqueued outside the executor, or by a job running inline inside another job, moves nothing; one enqueued by an attempt that no longer holds the key moves nothing and logs a warning. The dispatcher's reaper now counts a key carried by a job parked in `pgbus_blocked_executions` (legacy double-encoded rows included) as live: a parked `:until_executed` job's unbound row was reaped once it was older than twice the visibility timeout. A parked job's `created_at` now comes from the database clock, like its lock's, so discarding it from the dashboard frees its lock even when the app's clock runs behind the database's. A job's lock now records its queue the way the queue's table spells it (`Uniqueness.lock_queue_name`: `orders-handler` → `orders_handler`), at every place a job lock is written or released (adapter, executor, recurring scheduler), so a release that knows only the table (the dashboard) or a worker that spells the queue the table's way still finds it; a multi-queue worker also hands the executor the configured queue name instead of one cut from the table name. For such a queue, a job in flight during the upgrade holds its lock under the old spelling: its first retry after the upgrade does not take the lock (logged), so it and that job's later retries run without one until the job finishes, and the stale lock is reclaimed by the reaper once its message is gone. `config.queue_prefix` is now stored the way the queue tables spell it (`"my-app"` reads back as `"my_app"`): table names are unchanged, and every place that strips the prefix off a table name (the dashboard, the worker, the lock queue above) strips it whole. A prefix with a leading, trailing or doubled underscore (`"myapp_"`) is kept as written and now warned about at boot: `queue_name` collapses the separator after it only for a queue name that needs normalizing (`myapp__orders`, but `myapp_orders_handler` for `orders-handler`), so the reaper cannot resolve such a queue's lock back to its table: a lock orphaned there (a worker crash before its release) is never reclaimed and has to be released by key. A job's own release and the dashboard still match. `Pgbus::ActiveJob::Delivery.current` gains `enqueued_at` and `job_id`. Refs #564.
29
+ - **Presence statements run under the ActiveRecord adapter's lock (issue #566).** `Pgbus::Streams::Presence` ran every statement on `connection.raw_connection`, and `raw_connection` holds the adapter's lock only long enough to hand the `PG::Connection` out. On a pinned pool, as in transactional tests, every thread shares one adapter and that lock is all that serialises them, so the streamer's seat sweep (`expire_seats!` on the dispatcher thread) could be in flight on the same libpq connection a request thread was about to use. Every presence statement (`join`, `leave`, `touch`, `set_state`, `members`, `count`, `sweep!`, `expire_seats!` and the seat-schema probe) now goes through `Presence.exec_params`, which holds `adapter.lock` for the whole statement. The adapter lock is reentrant, so a request already inside a transaction re-enters it; unpinned, it is a `NullLock` and nothing changes. The seat-schema probe also no longer runs while holding the probe's own mutex: a sweep that probed under it while a pinned request inside `ActiveRecord::Base.transaction` (which holds the adapter's lock for its whole block) asked `seats?` deadlocked both threads. Two threads may now probe at once: a true answer one of them stored is never taken back, and a probe that a `reset_schema_cache!` overtook stores nothing. `Presence.connection` still returns the raw `PG::Connection`.
30
+
31
+ ## [0.20.0] - 2026-10-18
32
+
33
+ ### Added
34
+
6
35
  - **feat(mcp): workflow write tools, opt-in — resume, cancel, skip step, resume at, run now, signal (issue #550).** `config.mcp_write_tools = true` (default `false`, validated as a boolean, read when `Pgbus::MCP::Server.build` runs) adds `pgbus_workflow_resume`, `pgbus_workflow_cancel`, `pgbus_workflow_skip_step`, `pgbus_workflow_resume_at` (`step`), `pgbus_workflow_run_now` and `pgbus_workflow_signal` (`name`, a JSON object `payload` capped at `workflow_max_payload_size`, an optional `idempotency_key`; name and key at most 255 characters) to the MCP server; off, they are never constructed and `tools/list` does not show them, and the read tools are the same either way. Each takes `workflow_id`, a required `reason` (1–500 characters after trimming) and an optional `expected_status` (a run that has moved on by the time the call starts answers `ok: false` and nothing is done; the check runs just before the action, whose transition is guarded again under the run's row lock); a verb the run's `available_actions` does not allow answers `ok: false` with the allowed list; otherwise the call goes through the dashboard's `Web::DataSource` action, is recorded on the run's audit trail with its reason as actor `mcp` (`mcp:<client name>` when the transport knows the client; the name is stripped of control characters and capped at 100), and answers `{ ok:, reason:, workflow: }` with the `pgbus_workflow` detail as the run is after the call (refusals included), payloads always redacted, plus `available_actions` for a disallowed verb and `code`/`status` when the guarded transition refused it; an unexpected failure answers a fixed reason rather than the exception's text (the DataSource logged it). The tools live on their own base class, `Pgbus::MCP::WriteTool` (not `BaseTool`), annotated `readOnlyHint: false, destructiveHint: true` (`idempotentHint: true` on cancel only: a retried skip or resume at can act again once the run parked or paused again). No bulk tools: `pgbus_workflows_attention` groups the stuck runs and `pgbus_workflows` lists a group's runs, one call per run. The server's instructions name them when they are on, `pgbus mcp` logs it at start, and an unauthenticated `Pgbus::MCP.rack_app` with them on warns that anyone who reaches it can change workflow runs. `Workflow#signal!`, `Workflow.signal!` (by subject), `Signals.deliver` and `DataSource#signal_workflow` take an optional `reason:`, recorded on the `signal_received` event. Refs #550.
7
36
  - **feat(dashboard): workflow insights — per-class pipeline, step performance, funnel, per-revision numbers, MCP stats (issue #549).** `/pgbus/workflows/definitions` (linked from the Workflows page as "Insights by class") has one row per workflow class — every registered `Pgbus::Workflow` subclass and every class name still in `pgbus_workflows` — with its ongoing, paused and stuck runs, the runs that finished or were cancelled in the window and their p50/p95 end-to-end duration, a "definition not loaded" badge for a class that no longer loads and a count of ongoing runs on removed steps. `/pgbus/workflows/definitions/:workflow_class` (`OrderWorkflow` or `Billing::OrderWorkflow`) shows the class's steps in order as a pipeline: each step with the runs sitting at it now (scheduled, running, waiting, paused), the p50/p95/p99 run time and pickup latency (scheduled → started) of its attempts that finished in the window and their failure (an error was recorded), reattempt and skip rates, with a "few samples" note under 20 attempts; then runs started, finished and cancelled per bucket and the end-to-end duration distribution (ApexCharts, each with a caption and a data table), a funnel (of the runs started in the window, how many reached each step and where each one is now) and the same numbers for the last five app revisions, newest first, marking a step whose failure rate rose more than five points over the revision before. Window 1 h, 24 h (default), 7 d or 30 d, capped at `workflow_retention`. Every count of runs a list filter can express links to the Workflows list filtered to exactly those runs (percentiles, rates, the funnel's "reached" and the per-revision rows are text), which gains `ended_from`/`ended_to` filters (when a run finished or was cancelled; also on `pgbus_workflows`); need-attention counts open `/pgbus/workflows/attention?workflow_class=…`, the attention groups narrowed to the class (also `pgbus_workflows_attention(workflow_class:)`), and each removed step a run still sits on links to those runs. Throughput buckets follow the dashboard zone's wall clock: a day bucket starts at local midnight on both sides of a DST change (at its first hour where a DST change skips midnight). Runs and attempts dated after now are not counted. Read from the step and run rows on the existing indexes (`Web::WorkflowInsightsQueries`, one query per block): on ~1 M step rows over 20 classes a class's page reads in 26 ms and the list in 86 ms over 24 h (median; budgets 250 / 500 ms, `rake bench:workflow_insights`), about 300 ms over 30 d (reported, no budget); `WorkflowEntry.stuck_reason_counts` takes `workflow_class:` so a class's page counts only its own stuck runs. A class name from the URL that no class or run has is never constantized. `Web::DataSource` gains `workflow_definitions(window:)` and `workflow_definition(workflow_class, window:)`; the MCP server gains the read-only `pgbus_workflow_definitions` and `pgbus_workflow_stats`. The workflows guide gains a Monitoring section with example Prometheus alerts. Refs #549.
8
37
  - **feat(dashboard): the Workflows pages — list and filters, needs attention, step timeline, audit trail, operator actions, MCP read tools (issue #542).** `/pgbus/workflows` lists every run newest first (ongoing by default, each state, or all) with its class, subject, status badge, current step, start and last activity, and says in words what it is doing and what happens next ("Waiting for batch `OrderWorkflow#notify_vendors` (7 of 12 jobs done)", "Paused on `capture`: Timeout::Error — resume or cancel", "Overdue: `welcome` was due 2h ago; the sweep sends it again"); filters on class, current step, error class, subject type and id, workflow id and a created-at range combine and survive paging. `/pgbus/workflows/:workflow_id` shows the arguments, an ApexCharts range-bar timeline (one row per attempt split into waited, queued and ran, coloured by outcome, with waits and pauses as their own segments, zoomable), every attempt with the same durations as text, its worker, app revision, error, backtrace and cause chain, the signals, the audit trail ("Resumed by dashboard:alice: SMTP fixed"), whether it started on an older definition, rerun links both ways, and the actions `available_actions` allows: Resume (labelled "retry `capture`" or "continue with `receipt`"), Resume at a chosen step, Run now, Skip step, Send signal (a JSON object, sent once per form render), Rerun and Cancel, each with a confirm and an optional reason, each reporting the real outcome ("Not done: the run is cancelled now."). `/pgbus/workflows/attention` groups every stuck run by class, step, reason and error class with a count, first/last seen and the newest message, with bulk Resume/Cancel (500 runs per press, each through its own guarded transition: "Resumed 212; 2 had already moved on"). The home page gains a Workflows card ("N ongoing (M paused) · K need attention") and the nav a Workflows link. New settings: `config.web_actor` (`->(request) { name }`, recorded as `"dashboard:<name>"`, or `"dashboard"`) and `config.workflow_subject_url` (`->(subject_type, subject_id) { url }`, linked only when it is an http(s) URL or a same-site path). `Web::DataSource` gains `workflows`, `workflows_count`, `workflow_detail`, `workflow_attention_groups`, the home counts and the actions; `Pgbus::WorkflowEntry.attention_groups` / `attention_ids` share the stuck-reason `CASE` with the gauge. The MCP server gains the read-only `pgbus_workflows`, `pgbus_workflow` and `pgbus_workflows_attention`, with arguments, outputs, signal payloads and error text behind `include_payloads`. Refs #542.
9
38
  - **feat(workflows): waiting steps — `fan_out` over a batch, signals, wait timeouts (issue #541).** `fan_out :notify_vendors do |b| b.add(*jobs) end` runs its body with a recording builder, then opens one batch (`Pgbus.batch`, described `Class#step workflow_id`) whose `on_complete` is `Pgbus::Workflow::BatchFinishedJob`; the step and the run park as `waiting` and the callback resumes the run once every member finished: the step completes and the next is scheduled, or, with a dead-lettered member, `on_batch_failure:` (`:pause!` default, `:cancel!`, `:skip!`) applies with `Pgbus::Workflow::FanOutFailed` ("1 of 3 jobs dead-lettered") recorded. A verb or an exception in the body ends the step before any batch exists, the body cannot set the batch's callbacks, an empty batch resolves at once, and a reattempt or `resume!` re-runs the whole body on a fresh batch (members must be idempotent). `step :capture, wait_for: :payment_confirmed` parks the step at its gate until `wf.signal!(:payment_confirmed, payload: {…}, idempotency_key: event.id, actor:)` arrives, or `OrderWorkflow.signal!(order, :payment_confirmed, …, missing: :raise | :ignore)` by subject; signals live in `pgbus_workflow_signals` (`Pgbus::WorkflowSignal`, `pending → claimed → consumed`), early ones are buffered, a redelivery with the same key stores nothing and answers `duplicate_delivery?`, `wait_for:` takes a name, a `Pgbus::Workflow::SignalMatcher.new(:shipped) { |payload| … }` or any `matches?(signal)` object, the body reads `received_signal.payload` with indifferent access, and a claimed signal survives a `reattempt!` or a failure that pauses. The wake is a step-row write, so its StepJob goes out only after the signal commits; a paused run takes the signal but stays paused until `resume!`, and `resume!` of a run paused while waiting puts it back to `waiting`. `wait_timeout: 2.days, on_timeout: :pause! | :cancel! | :skip! | :run` bounds a wait (`timeout:` still bounds the run): the dispatcher's sweep applies it under the row lock (`Pgbus::Workflow::WaitTimedOut` for `:pause!`; `:run` runs the body with `timed_out?` true), a late signal stays pending, and the time a run spends paused does not count against the wait. The sweep also resolves a fan_out step whose batch finished but whose callback was lost, and pauses one whose batch row is gone (outcome unknown, whatever `on_batch_failure:` says). `skip!` outside a step skips the step a waiting run waits on. New runner verbs `park`, `wake`, `resolve`, `rewait` and outside verbs `skip`, `signal` in `States`; events `step_waiting`, `signal_received`, `signal_claimed`, `wait_timed_out`, `fan_out_opened`, `fan_out_finished`; instrumentation `pgbus.workflow.signal` and `pgbus.workflow.wait_timed_out`; stuck reason `waiting_overdue` past the new `workflow_waiting_visibility_threshold` (7 days). `TestHelpers` gains `deliver_signal`, `complete_fan_out` and `expire_wait`, and `speedrun_workflow` raises `Pgbus::Workflow::StepWaiting` naming what the step waits for. Schema: `pgbus:add_workflows` and `pgbus:install` carry the five step columns and the signals table; installs that ran the core migration add them with `rails generate pgbus:add_workflow_signals [--database=pgbus]` (`pgbus:update` detects it), and `start!` raises `WaitingStepsUnmigrated` (or `BatchCallbacksUnmigrated` for a `fan_out` before `pgbus:add_batch_callback_jobs`) until then. Refs #541.
10
39
  - **feat(workflows): `Pgbus::Workflow`, durable multi-step workflows.** A workflow class declares steps (`step :name, wait:, skip_if:, timeout:, job_options:, on_exception:`, anonymous `step { … }`, `step :method`, `step def …`, `before_step:`/`after_step:`), `cancel_if`, class-level `on_exception` policies and `set_step_job_options`; `SignupWorkflow.start!(user, plan: "pro")` writes the run and its first step row in one transaction, and each step attempt is a `pgbus_workflow_steps` row and one `Pgbus::Workflow::StepJob`, sent only by the step row's after-commit callback (so `start!` inside an app transaction on the same database sends nothing until it commits). Inside a step, `subject` (GlobalID, nil once the record is gone), `arguments` (ActiveJob::Arguments, indifferent access, capped at `workflow_max_payload_size`) and the verbs `pause!`, `cancel!`, `reattempt!(wait:)`, `skip!`, `finished!`; outside, `resume!`, `resume_at!(:step)`, `pause!`, `cancel!`, `run_now!`, `reschedule!(at:)` and `rerun!`, each with `actor:`/`reason:`, all checked against one transition table, `Pgbus::Workflow::States` (`available_actions`). An unhandled exception never re-raises into ActiveJob: the step row records the class, message, cleaned and capped backtrace, cause chain, `hostname:pid` and app revision, `Pgbus::ErrorReporter` gets it with the workflow, step and attempt as context, and the run pauses unless a policy (`action:`, `matching:`, `wait:`, `max_reattempts:`, `terminal_action:`, `report:`, or an imperative block) says otherwise. Idempotent by contract: `output(**values)` is written in the UPDATE that completes the step and `output_of(:step)` reads the latest completed attempt only; `step_idempotency_key` is the same on every attempt (`attempt_idempotency_key` differs); `start!(…, idempotency_key:)` returns the existing run with `duplicate_start?`. At most one ongoing run per `(class, subject)` (`Pgbus::Workflow::AlreadyRunning`); subject-less runs are not limited. `removed_step :name` gravestones skip a step a live run still names; an unknown step pauses with `StepNotDefined`; `definition_digest` marks runs that started on an older definition. Every transition writes a `pgbus_workflow_events` row (actor, reason, details) in its own transaction (`wf.events`). Housekeeping from the dispatcher: a redelivered StepJob whose row is still in progress means its worker died, and `workflow_stall_action` (`:pause`, `:reattempt`, `:cancel`) applies at once; the sweep (`workflow_sweep_interval`) stalls a step past its `timeout:` or `workflow_stall_threshold`, re-sends lost scheduled steps and emits `pgbus.workflow.gauge`; finished/cancelled runs are deleted after `workflow_retention` (30 days). Instrumented as `pgbus.workflow.step`, `pgbus.workflow_finished`, `pgbus.workflow_paused`, `pgbus.workflow_cancelled`, `pgbus.workflow.stall_detected`; Prometheus/StatsD and AppSignal get `pgbus_workflow_step_duration_seconds`, `pgbus_workflow_steps_total`, run counters and the `pgbus_workflows`/`pgbus_workflows_stuck` gauges (the AppSignal dashboard gains a workflows row; the StatsD backend sends any `*_seconds` histogram as a `*_ms` timer, × 1000). Instrumentation and error reports of a transition made inside an app's transaction go out when that transaction commits, never on its rollback. `Pgbus::Workflow::TestHelpers` (`perform_next_step`, `run_step_inline`, `speedrun_workflow`) runs steps inline; `Pgbus::Workflow::StepContext` and Pgbus's log context tag a running step; `Pgbus::ActiveJob::Delivery.current` exposes the running job's `msg_id`/`read_ct`. Set up with `rails generate pgbus:add_workflows` (`pgbus:update` detects the missing tables; fresh `pgbus:install` includes them). New configuration: `workflow_queue`, `workflow_retention`, `workflow_sweep_interval`, `workflow_stall_threshold`, `workflow_stall_action`, `workflow_max_payload_size`, `workflow_error_backtrace_lines`. Refs #540.
40
+
41
+ ### Fixed
42
+
43
+ - **A blank signal `idempotency_key` is no key (issue #550).** `wf.signal!(…, idempotency_key: "")` (or whitespace) stored `""` as a real key, so every later signal sent with a blank key dedup'd against the first: it answered `duplicate_delivery?` and was never stored, leaving a step waiting for it. `Signals.deliver` now treats a blank key as none, for app code, the dashboard and the MCP tool alike. Refs #550.
44
+
45
+ ## [0.19.4] - 2026-10-17
46
+
47
+ ### Fixed
48
+
49
+ - **`Pgbus.batch` inside a database transaction counts the jobs that enqueue after commit, and waits for them.** A job with Active Job's `enqueue_after_transaction_commit` added inside an open transaction is enqueued only when the transaction commits, after the block has returned and its thread tag is gone. The batch counted nothing, finished empty when the block returned and sent its `on_complete` (and `on_success`) callback before any member job existed; the jobs then ran outside the batch. Even a rolled-back transaction sent the callback. Now `b.add`, `batch.add` and `perform_later` record the batch on the job instance (`enqueue_batch_id`, transient: never serialized, cleared once the enqueue is done), the adapter tags and counts each job from it at the real enqueue, and the batch starts processing (or an open batch re-checks for finish) through `ActiveRecord.after_all_transactions_commit`, after every deferred enqueue has counted itself in. On a rollback nothing is sent, no callback fires and the batch row is rolled back with the transaction. A deferred first attempt discarded at enqueue (uniqueness or concurrency `on_conflict: :discard`) is uncounted as before, and a `retry_on` re-enqueue still re-joins without a second count. Outside a transaction nothing changes, including the queries a batch runs. Known gaps: a bare `ActiveJob.perform_all_later` inside the block does not record the batch (use `b.add`); a `batch.add` inside a transaction raises `AlreadyFinished` when the transaction commits, not at the `add` line, if the batch finished in the meantime; a batch row on a separate `connects_to` database is not part of the caller's transaction. Refs #553.
50
+ - **AppSignal: the job's transaction is current while the job runs, and carries its error (issue #552).** The AppSignal subscriber opened the `background_job` transaction from a finish-only notification callback, after the job had run, and completed it in the same call. `pgbus.job_failed` fired after that, so its `Appsignal.set_error` found no current transaction and reported nothing: a job whose class no longer exists (`ActiveJob::UnknownJobClassError`) or whose arguments cannot be deserialized retried and dead-lettered with nothing in AppSignal but the counters, an error raised inside `perform` arrived only as an action-less sample from the Rails error reporter, and job code calling `Appsignal.add_tags`, `set_error` or `instrument` attached to nothing. Event handlers had the same shape. The subscriber now listens to `pgbus.executor.execute` and `pgbus.event_processed` on both sides (`start` and `finish`): the `<JobClass>#perform` / `<Handler>#handle` transaction opens before the job or handler runs, is current for the whole execution (deserialize, perform, archive), records the exception the execution raised and completes at the end, so an error raised inside `perform` lands on it once, with its duration and queue start. A failure raised before that span opens (a message body that is not JSON, an event that cannot be built, a dead-letter move that failed) is reported from `pgbus.job_failed` / `pgbus.event_failed` on a transaction of its own with the same action and tags; an error the span already recorded is not reported twice. A transaction that is already current is reused and left to its owner to complete, as AppSignal's own ActiveJob hook does, and a span inside another of its kind restores the outer one when it finishes. Like that hook, the span records any exception that ends the execution, so a job interrupted between `perform` and the archive (an `Async::Stop` on shutdown under `execution_mode: :async`, an `Interrupt`) now shows up as an error sample too. The counters and the `pgbus_job_duration_ms` / `pgbus_event_duration_ms` distributions are recorded as before, once per execution. Refs #552.
51
+
52
+ ## [0.19.3] - 2026-10-17
53
+
54
+ ### Added
55
+
11
56
  - **`Pgbus.stream_key` works inside a non-main Ractor.** `Pgbus.stream_key`, `Pgbus.stream_key!` and `Pgbus.stream_name_budget` (and `Streams::Key`) check the queue-name budget against the `queue_prefix`, which they read from `Pgbus.configuration`; a non-main Ractor cannot read that (it holds the logger and the app's callables), so they raised `Ractor::IsolationError` there. Each `Pgbus.finalize!` and finalizing `Pgbus.configure` (so also the engine's `after_initialize` and `pgbus start`) now publishes `Pgbus.configuration_snapshot`, a Ractor-shareable `Pgbus::Configuration::Snapshot` of the settings a non-main Ractor reads (the `queue_prefix` for now), and the stream keys read it there. On the main Ractor nothing changes: they read the live configuration, so a `queue_prefix` set through a setter still applies at once; a non-main Ractor sees it from the next finalize. Refs #527.
12
- - **Deploy tasks: one-time work owed by a release, run once the fleet has rolled over.** Declare a backfill, data fix or cache warm in `config/deploy_tasks.yml` (the shape of `recurring.yml` minus `schedule:`, plus `version:` and `every_deploy:`) or `config.deploy_tasks`; the scheduler enqueues it once per `(key, version)` (or once per revision for an `every_deploy:` task) after every live pgbus process reports the new app revision and a worker on it is alive, so it never runs on old code during the overlap window and two schedulers never both fire it. It is not exactly-once: a failure between the enqueue and recording the run can repeat a task after a scheduler restart, and a scheduler killed between claiming and enqueueing loses that arm, so deploy tasks must be idempotent. Every pgbus process stamps the revision (`config.app_revision`, `PGBUS_REVISION`, `DASH_VERSION`, `KAMAL_VERSION` or the `REVISION` file) into its heartbeat. The dashboard gets a "Deploy tasks" page with the rollover status, Run now and Enable/Disable; run history lives in `pgbus_deploy_task_runs` (`deploy_task_run_retention`, default 90 days); `pgbus.deploy_task.enqueue` is instrumented and the MCP server gains `pgbus_deploy_tasks`. Set up with `rails generate pgbus:add_deploy_tasks` (`pgbus:update` detects the missing tables). Refs #536.
57
+
58
+ ### Fixed
59
+
60
+ - **The dashboard's confirm and alert dialogs open centred again (issue #529).** The engine's stylesheet is built with Tailwind 4, whose preflight sets `margin: 0` on every element. A modal `<dialog>` keeps the browser's `inset: 0` but lost the `margin: auto` that centres it inside that inset, so every Pause, Purge and Delete confirmation opened in the top-left corner. Both dialogs now carry `m-auto`. The confirm dialog's Confirm button was white on yellow (1.91:1). It now uses the dark-on-yellow warning colours (`bg-yellow-400 text-yellow-950`), and the axe gate audits an open confirm dialog in light and dark mode. Refs #529.
61
+
62
+ ## [0.19.2] - 2026-10-17
63
+
64
+ ### Added
65
+
13
66
  - **`pgbus.event_dead_lettered`: the event consumer says when it dead-letters an event.** A job that exhausted `max_retries` emitted `pgbus.job_dead_lettered`; an event moved to `<queue>_dlq` the same way emitted nothing, so a pager, an error tracker or a metrics counter had no push signal for dead events. `Process::Consumer` now emits `pgbus.event_dead_lettered` once per dead-lettered event, after `move_to_dead_letter` returns and the failed-event row is cleared, with `queue_name` (logical), `routing_key`, `handler` (the class that owns the queue in this process, or nil), `event_id`, `read_ct`, `msg_id` and `error` (the class of the last recorded failure, or nil). `handler` is nil unless exactly one subscriber owns the queue. A body that is not a JSON object (unparseable text, or a jsonb scalar or array written straight to the queue) is still dead-lettered, with `routing_key`, `handler` and `event_id` nil, and its `error` is the consumer's own failure to read it (`JSON::ParserError`, `NoMethodError` or `TypeError`): no handler ran. A move that raises emits nothing. A subscriber that raises is logged at debug level and does not change the outcome: the message stays dead-lettered, with one `dead_lettered` stat, no `failed` stat and no circuit-breaker failure. The AppSignal and metrics subscribers count it as `pgbus_event_count` with `status: "dead_lettered"`, tagged `handler` and `routing_key`. `pgbus.job_dead_lettered` and the consumer's success path are unchanged: the envelope is parsed a second time only on the dead-letter branch. Refs #543.
14
67
  - **feat(batches): `Pgbus.batch do |b| … end`, a builder for starting a batch.** `b.on_complete`, `b.on_success` and `b.on_failure` take an ActiveJob class or a configured instance; `b.add(*jobs)` takes job instances (each keeps its own `.set(wait:, queue:, priority:)`) and sends them in one bulk enqueue. The batch row exists, and a `uniqueness_key` is held, before the block runs, and the batch stays pending until the block returns, so the calls can come in any order and a job that finishes while the block is still adding cannot finish the batch early. Every `perform_later` inside the block joins the batch, as inside `Batch#enqueue`. With `on_conflict: :discard` or `:log` and the key held, the block does not run. Inside a job, `batch(description: …) { |b| … }` opens a new, independent batch; `batch` without a block still reads the job's own batch. A member job adds siblings with `batch.add(NextJob.new(id))`, the open-batch form of `batch.enqueue { … }`. `Pgbus::Batch.start` is the same as `Pgbus.batch`, and `Pgbus::Batch.new` + `#enqueue` stay as the lower-level API. Refs #538.
15
68
 
@@ -17,14 +70,11 @@
17
70
 
18
71
  - **The batch completion callback is `on_complete`.** `Pgbus::Batch.new(on_complete:)`, the builder's `b.on_complete` and the `#on_complete` reader replace `on_finish`, which keeps working as a deprecated alias until 1.0 and logs one warning per process. The database columns stay `on_finish_class` / `on_finish_job` until the 1.0 schema cleanup, so batches created before upgrading still fire, and the `pgbus.batch_finished` instrumentation event keeps its name. The dashboard's batch page labels the callback "On Complete". Refs #538.
19
72
 
20
- ### Fixed
73
+ ## [0.19.1] - 2026-10-17
21
74
 
22
- - **An `:until_executed` job keeps its uniqueness key while its retry waits (issue #564).** A job that rescheduled itself with `retry_job(wait:)` (or `retry_on … wait:`) enqueued the retry through the key it held (#333), but the key row kept pointing at the running attempt, and the executor deleted the row by key when that attempt archived. For the whole wait nothing held the key: a fresh `perform_later` with the same key ran beside the retry, and the retry's own success then deleted the newcomer's row. Now the row is handed to the retry the way a first enqueue takes one: before the retry is sent or parked it is unbound onto the retry's queue and re-stamped (`UniquenessKey.unbind!`), and once sent it is bound to the retry (`UniquenessKey.bind!(…, acquired_at:)`). A retry parked behind its `limits_concurrency` slot (with `to: 1, on_conflict: :block` the attempt holds that slot, so its retry always parks) keeps the unbound row until it runs; a retry dropped by its limit, or whose send failed before any message existed, hands the row back to the attempt. A finishing message (success, dead-letter, or a failed `:while_executing` attempt) releases only the row it holds (`UniquenessKey.release_held_by!`): the row bound to it, or an unbound row acquired no later than the message was sent, so a retry that finishes before its bind still releases it; a row handed to a retry, or that a successor took after an operator released it, stays. The dashboard's discards (a job, a failed job, a dead-lettered copy, an event, the bulk discards and queue purges) release the same way, through `UniquenessKey.release_all_held_by!` for the bulk ones, judged by when the job's message was sent: a failed job's queued message gives its own `enqueued_at`, and the dead-letter header gains `source_msg_id` and `source_enqueued_at` so a DLQ copy names its original and when it was sent. Without a send time only a lock bound to the message is freed (the reaper takes the rest). The Locks page's Release still deletes by key. A retry enqueued outside the executor, or by a job running inline inside another job, moves nothing; one enqueued by an attempt that no longer holds the key moves nothing and logs a warning. The dispatcher's reaper now counts a key carried by a job parked in `pgbus_blocked_executions` (legacy double-encoded rows included) as live: a parked `:until_executed` job's unbound row was reaped once it was older than twice the visibility timeout. A parked job's `created_at` now comes from the database clock, like its lock's, so discarding it from the dashboard frees its lock even when the app's clock runs behind the database's. A job's lock now records its queue the way the queue's table spells it (`Uniqueness.lock_queue_name`: `orders-handler` → `orders_handler`), at every place a job lock is written or released (adapter, executor, recurring scheduler), so a release that knows only the table (the dashboard) or a worker that spells the queue the table's way still finds it; a multi-queue worker also hands the executor the configured queue name instead of one cut from the table name. For such a queue, a job in flight during the upgrade holds its lock under the old spelling: its first retry after the upgrade does not take the lock (logged), so it and that job's later retries run without one until the job finishes, and the stale lock is reclaimed by the reaper once its message is gone. `config.queue_prefix` is now stored the way the queue tables spell it (`"my-app"` reads back as `"my_app"`): table names are unchanged, and every place that strips the prefix off a table name (the dashboard, the worker, the lock queue above) strips it whole. A prefix with a leading, trailing or doubled underscore (`"myapp_"`) is kept as written and now warned about at boot: `queue_name` collapses the separator after it only for a queue name that needs normalizing (`myapp__orders`, but `myapp_orders_handler` for `orders-handler`), so the reaper cannot resolve such a queue's lock back to its table: a lock orphaned there (a worker crash before its release) is never reclaimed and has to be released by key. A job's own release and the dashboard still match. `Pgbus::ActiveJob::Delivery.current` gains `enqueued_at` and `job_id`. Refs #564.
23
- - **Presence statements run under the ActiveRecord adapter's lock (issue #566).** `Pgbus::Streams::Presence` ran every statement on `connection.raw_connection`, and `raw_connection` holds the adapter's lock only long enough to hand the `PG::Connection` out. On a pinned pool, as in transactional tests, every thread shares one adapter and that lock is all that serialises them, so the streamer's seat sweep (`expire_seats!` on the dispatcher thread) could be in flight on the same libpq connection a request thread was about to use. Every presence statement (`join`, `leave`, `touch`, `set_state`, `members`, `count`, `sweep!`, `expire_seats!` and the seat-schema probe) now goes through `Presence.exec_params`, which holds `adapter.lock` for the whole statement. The adapter lock is reentrant, so a request already inside a transaction re-enters it; unpinned, it is a `NullLock` and nothing changes. The seat-schema probe also no longer runs while holding the probe's own mutex: a sweep that probed under it while a pinned request inside `ActiveRecord::Base.transaction` (which holds the adapter's lock for its whole block) asked `seats?` deadlocked both threads. Two threads may now probe at once: a true answer one of them stored is never taken back, and a probe that a `reset_schema_cache!` overtook stores nothing. `Presence.connection` still returns the raw `PG::Connection`.
24
- - **A blank signal `idempotency_key` is no key (issue #550).** `wf.signal!(…, idempotency_key: "")` (or whitespace) stored `""` as a real key, so every later signal sent with a blank key dedup'd against the first: it answered `duplicate_delivery?` and was never stored, leaving a step waiting for it. `Signals.deliver` now treats a blank key as none, for app code, the dashboard and the MCP tool alike. Refs #550.
25
- - **`Pgbus.batch` inside a database transaction counts the jobs that enqueue after commit, and waits for them.** A job with Active Job's `enqueue_after_transaction_commit` added inside an open transaction is enqueued only when the transaction commits, after the block has returned and its thread tag is gone. The batch counted nothing, finished empty when the block returned and sent its `on_complete` (and `on_success`) callback before any member job existed; the jobs then ran outside the batch. Even a rolled-back transaction sent the callback. Now `b.add`, `batch.add` and `perform_later` record the batch on the job instance (`enqueue_batch_id`, transient: never serialized, cleared once the enqueue is done), the adapter tags and counts each job from it at the real enqueue, and the batch starts processing (or an open batch re-checks for finish) through `ActiveRecord.after_all_transactions_commit`, after every deferred enqueue has counted itself in. On a rollback nothing is sent, no callback fires and the batch row is rolled back with the transaction. A deferred first attempt discarded at enqueue (uniqueness or concurrency `on_conflict: :discard`) is uncounted as before, and a `retry_on` re-enqueue still re-joins without a second count. Outside a transaction nothing changes, including the queries a batch runs. Known gaps: a bare `ActiveJob.perform_all_later` inside the block does not record the batch (use `b.add`); a `batch.add` inside a transaction raises `AlreadyFinished` when the transaction commits, not at the `add` line, if the batch finished in the meantime; a batch row on a separate `connects_to` database is not part of the caller's transaction. Refs #553.
26
- - **AppSignal: the job's transaction is current while the job runs, and carries its error (issue #552).** The AppSignal subscriber opened the `background_job` transaction from a finish-only notification callback, after the job had run, and completed it in the same call. `pgbus.job_failed` fired after that, so its `Appsignal.set_error` found no current transaction and reported nothing: a job whose class no longer exists (`ActiveJob::UnknownJobClassError`) or whose arguments cannot be deserialized retried and dead-lettered with nothing in AppSignal but the counters, an error raised inside `perform` arrived only as an action-less sample from the Rails error reporter, and job code calling `Appsignal.add_tags`, `set_error` or `instrument` attached to nothing. Event handlers had the same shape. The subscriber now listens to `pgbus.executor.execute` and `pgbus.event_processed` on both sides (`start` and `finish`): the `<JobClass>#perform` / `<Handler>#handle` transaction opens before the job or handler runs, is current for the whole execution (deserialize, perform, archive), records the exception the execution raised and completes at the end, so an error raised inside `perform` lands on it once, with its duration and queue start. A failure raised before that span opens (a message body that is not JSON, an event that cannot be built, a dead-letter move that failed) is reported from `pgbus.job_failed` / `pgbus.event_failed` on a transaction of its own with the same action and tags; an error the span already recorded is not reported twice. A transaction that is already current is reused and left to its owner to complete, as AppSignal's own ActiveJob hook does, and a span inside another of its kind restores the outer one when it finishes. Like that hook, the span records any exception that ends the execution, so a job interrupted between `perform` and the archive (an `Async::Stop` on shutdown under `execution_mode: :async`, an `Interrupt`) now shows up as an error sample too. The counters and the `pgbus_job_duration_ms` / `pgbus_event_duration_ms` distributions are recorded as before, once per execution. Refs #552.
27
- - **The dashboard's confirm and alert dialogs open centred again (issue #529).** The engine's stylesheet is built with Tailwind 4, whose preflight sets `margin: 0` on every element. A modal `<dialog>` keeps the browser's `inset: 0` but lost the `margin: auto` that centres it inside that inset, so every Pause, Purge and Delete confirmation opened in the top-left corner. Both dialogs now carry `m-auto`. The confirm dialog's Confirm button was white on yellow (1.91:1). It now uses the dark-on-yellow warning colours (`bg-yellow-400 text-yellow-950`), and the axe gate audits an open confirm dialog in light and dark mode. Refs #529.
75
+ ### Added
76
+
77
+ - **Deploy tasks: one-time work owed by a release, run once the fleet has rolled over.** Declare a backfill, data fix or cache warm in `config/deploy_tasks.yml` (the shape of `recurring.yml` minus `schedule:`, plus `version:` and `every_deploy:`) or `config.deploy_tasks`; the scheduler enqueues it once per `(key, version)` (or once per revision for an `every_deploy:` task) after every live pgbus process reports the new app revision and a worker on it is alive, so it never runs on old code during the overlap window and two schedulers never both fire it. It is not exactly-once: a failure between the enqueue and recording the run can repeat a task after a scheduler restart, and a scheduler killed between claiming and enqueueing loses that arm, so deploy tasks must be idempotent. Every pgbus process stamps the revision (`config.app_revision`, `PGBUS_REVISION`, `DASH_VERSION`, `KAMAL_VERSION` or the `REVISION` file) into its heartbeat. The dashboard gets a "Deploy tasks" page with the rollover status, Run now and Enable/Disable; run history lives in `pgbus_deploy_task_runs` (`deploy_task_run_retention`, default 90 days); `pgbus.deploy_task.enqueue` is instrumented and the MCP server gains `pgbus_deploy_tasks`. Set up with `rails generate pgbus:add_deploy_tasks` (`pgbus:update` detects the missing tables). Refs #536.
28
78
 
29
79
  ## [0.19.0] - 2026-10-16
30
80
 
data/README.md CHANGED
@@ -887,7 +887,7 @@ wf.events.map(&:event) # ["started", "step_schedule
887
887
  - **Idempotent by contract**: `output(charge_id: …)` is durable once the step completes and later steps read it with `output_of(:charge)`; `step_idempotency_key` is the same on every attempt, for the external API; `start!(order, idempotency_key: "checkout-#{order.id}")` returns the existing run.
888
888
  - **One ongoing run per subject** and class: a second `start!` raises `Pgbus::Workflow::AlreadyRunning`. Subject-less runs are not limited.
889
889
  - **Fan-out**: `fan_out :notify_vendors do |b| b.add(*jobs) end` opens one batch for the jobs and parks the step until every member finished; the batch's `on_complete` resumes the run, and a dead-lettered member applies `on_batch_failure:` (`:pause!` by default) with `Pgbus::Workflow::FanOutFailed` recorded. A reattempt, or `resume!` of a failed fan-out step, re-runs the whole body on a fresh batch, so members must be idempotent; a run resumed while its batch is still running keeps waiting on that batch.
890
- - **Signals**: `step :capture, wait_for: :payment_confirmed do … received_signal.payload["amount"] … end` parks until `wf.signal!(:payment_confirmed, payload: {…}, idempotency_key: event.id)`, or by subject, `OrderWorkflow.signal!(order, :payment_confirmed, …, missing: :ignore)`. Early signals are buffered, a redelivery with the same key is a no-op (`duplicate_delivery?`), and matchers (`Pgbus::Workflow::SignalMatcher.new(:shipped) { |payload| … }`) narrow what a step takes. `wait_timeout: 2.days, on_timeout: :pause!` (or `:cancel!`, `:skip!`, `:run`) puts a deadline on the wait, fired by the dispatcher's sweep; `on_timeout: :notify` makes it a soft deadline that reports the late signal (a `wait_timed_out` event and `pgbus_workflow_wait_overdue_total`) and keeps waiting for it, its deadline moving one `wait_timeout:` on each time (a sweep that missed several reports them once).
890
+ - **Signals**: `step :capture, wait_for: :payment_confirmed do … received_signal.payload["amount"] … end` parks until `wf.signal!(:payment_confirmed, payload: {…}, idempotency_key: event.id)`, or by subject, `OrderWorkflow.signal!(order, :payment_confirmed, …, missing: :ignore)`. Early signals are buffered, a redelivery with the same key is a no-op (`duplicate_delivery?`), and matchers (`Pgbus::Workflow::SignalMatcher.new(:shipped) { |payload| … }`) narrow what a step takes. `wait_timeout: 2.days, on_timeout: :pause!` (or `:cancel!`, `:skip!`, `:run`) puts a deadline on the wait, fired by the dispatcher's sweep; `on_timeout: :notify` makes it a soft deadline that reports the late signal (a `wait_timed_out` event and `pgbus_workflow_wait_overdue_total`) and keeps waiting for it, its deadline moving one `wait_timeout:` on each time (a sweep that missed several reports them once). `wait_timeout:` also takes a method name or a lambda, evaluated on the run when the step parks: `wait_timeout: -> { subject.expires_at }` returns a `Time` (or any non-String value with `#to_time`, such as a DateTime or a TimeWithZone: the deadline itself, however long the run took to reach the wait), a positive Duration or number of seconds counted from park, or `nil` for no timeout.
891
891
  - **Housekeeping**: a worker that dies mid-step is noticed when its message comes back (the visibility heartbeat keeps a live one invisible); a hung step stalls after its `timeout:` or `workflow_stall_threshold`; the dispatcher re-sends lost steps and deletes finished runs after `workflow_retention` (30 days).
892
892
  - **Metrics** for Prometheus/StatsD and AppSignal: per-step duration histograms and counters, and gauges of runs by status and by stuck reason.
893
893
  - **Dashboard**: `/pgbus/workflows` lists runs with what each one does next, filters by state, class, step, error, subject, id and creation date, and shows a run's timeline, attempts, signals and audit trail with its actions (Resume, Resume at, Run now, Skip step, Send signal, Rerun, Cancel); `/pgbus/workflows/attention` groups the stuck runs for bulk Resume/Cancel. `config.web_actor = ->(request) { … }` names the operator on the audit trail and `config.workflow_subject_url = ->(type, id) { … }` links the subject. The MCP server reads the same through `pgbus_workflows`, `pgbus_workflow` and `pgbus_workflows_attention`, and with `config.mcp_write_tools = true` acts on one run per call through `pgbus_workflow_resume`, `_cancel`, `_skip_step`, `_resume_at`, `_run_now` and `_signal` ([write tools](#write-tools-opt-in)).
@@ -72,6 +72,14 @@ module Pgbus
72
72
 
73
73
  def pgbus_insight_definition_path(name, window) = pgbus.workflow_definition_path(name, window: window.name)
74
74
 
75
+ # A signal step's wait timeout: its length, or "dynamic (per run)" for a
76
+ # wait_timeout: lambda or method name (issue #574).
77
+ def pgbus_insight_wait_timeout(seconds)
78
+ return t("pgbus.workflow_insights.show.pipeline.wait_timeout_dynamic") if seconds == "dynamic"
79
+
80
+ pgbus_duration(seconds)
81
+ end
82
+
75
83
  def pgbus_insight_kind(step)
76
84
  key = step[:declared] ? step[:kind] : "undeclared"
77
85
  return unless KIND_TONES.key?(key)
@@ -55,8 +55,9 @@ module Pgbus
55
55
  # paused_manual paused otherwise (the step called pause!, or an outside pause!)
56
56
  # overdue ready, its scheduled step due more than +threshold+ ago
57
57
  # waiting_overdue waiting longer than +waiting_threshold+ on a step
58
- # without a wait_timeout: (only with +waiting_threshold+,
59
- # which needs the waiting-step columns)
58
+ # without a deadline (no timeout_at: no wait_timeout:, or
59
+ # a per-run one that set none or cleared it; only with
60
+ # +waiting_threshold+, which needs the waiting-step columns)
60
61
  # With +workflow_class+, that class's runs only (the workflow insights,
61
62
  # issue #549).
62
63
  def self.stuck_reason_counts(threshold:, now: Time.current, waiting_threshold: nil, workflow_class: nil)
@@ -10,6 +10,9 @@
10
10
  </h3>
11
11
  <%= pgbus_insight_kind(step) %>
12
12
  </div>
13
+ <% if step[:wait_timeout] %>
14
+ <p class="mt-2 text-sm"><span class="text-gray-600 dark:text-gray-300"><%= t("pgbus.workflow_insights.show.pipeline.wait_timeout") %></span> <span class="font-mono text-gray-900 dark:text-white"><%= pgbus_insight_wait_timeout(step[:wait_timeout]) %></span></p>
15
+ <% end %>
13
16
  <ul class="mt-3 grid grid-cols-2 gap-x-3 gap-y-1 text-sm" aria-label="<%= t("pgbus.workflow_insights.show.pipeline.now_label") %>">
14
17
  <% Pgbus::WorkflowInsightsHelper::LIVE_STATUSES.each do |status| %>
15
18
  <% count = step[:counts][status.to_sym] %>
@@ -1167,6 +1167,8 @@ da:
1167
1167
  skipped: Sprunget over
1168
1168
  step_link: Igangværende kørsler ved %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Ventetidsgrænse
1171
+ wait_timeout_dynamic: dynamisk (pr. kørsel)
1170
1172
  window_label: Forsøg der sluttede i vinduet
1171
1173
  summary:
1172
1174
  attention: Kræver handling
@@ -1350,6 +1352,7 @@ da:
1350
1352
  waiting_batch_html: Venter på batch %{batch} (%{done} af %{total} jobs færdige).
1351
1353
  waiting_signal_html: Venter på signalet %{signal} siden %{ago}.
1352
1354
  waiting_signal_late_html: Venter på signalet %{signal} siden %{ago} (forsinket siden %{late}). Næste forsinkelsesbesked sendes %{time}.
1355
+ waiting_signal_late_no_deadline_html: Venter på signalet %{signal} siden %{ago} (forsinket siden %{late}). Der sendes ikke flere forsinkelsesbeskeder.
1353
1356
  waiting_signal_notify_html: Venter på signalet %{signal} siden %{ago}. En forsinkelsesbesked sendes %{time}, og ventetiden fortsætter.
1354
1357
  waiting_signal_timeout_html: Venter på signalet %{signal} siden %{ago}. Ventetiden udløber %{time}.
1355
1358
  waiting_step_html: Venter på %{step}.
@@ -1167,6 +1167,8 @@ de:
1167
1167
  skipped: Übersprungen
1168
1168
  step_link: Laufende Läufe bei %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Warte-Timeout
1171
+ wait_timeout_dynamic: dynamisch (pro Lauf)
1170
1172
  window_label: Im Zeitfenster beendete Versuche
1171
1173
  summary:
1172
1174
  attention: Brauchen Aufmerksamkeit
@@ -1350,6 +1352,7 @@ de:
1350
1352
  waiting_batch_html: Wartet auf Stapel %{batch} (%{done} von %{total} Jobs erledigt).
1351
1353
  waiting_signal_html: Wartet auf Signal %{signal} seit %{ago}.
1352
1354
  waiting_signal_late_html: Wartet auf Signal %{signal} seit %{ago} (verspätet seit %{late}). Die nächste Verspätungsmeldung folgt %{time}.
1355
+ waiting_signal_late_no_deadline_html: Wartet auf Signal %{signal} seit %{ago} (verspätet seit %{late}). Es folgt keine weitere Verspätungsmeldung.
1353
1356
  waiting_signal_notify_html: Wartet auf Signal %{signal} seit %{ago}. Eine Verspätungsmeldung folgt %{time}, und das Warten geht weiter.
1354
1357
  waiting_signal_timeout_html: Wartet auf Signal %{signal} seit %{ago}. Das Warten läuft ab %{time}.
1355
1358
  waiting_step_html: Wartet auf %{step}.
@@ -1167,6 +1167,8 @@ en:
1167
1167
  skipped: Skipped
1168
1168
  step_link: Ongoing runs at %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Wait timeout
1171
+ wait_timeout_dynamic: dynamic (per run)
1170
1172
  window_label: Attempts that finished in the window
1171
1173
  summary:
1172
1174
  attention: Need attention
@@ -1350,6 +1352,7 @@ en:
1350
1352
  waiting_batch_html: Waiting for batch %{batch} (%{done} of %{total} jobs done).
1351
1353
  waiting_signal_html: Waiting for signal %{signal} since %{ago}.
1352
1354
  waiting_signal_late_html: Waiting for signal %{signal} since %{ago} (late since %{late}). The next late notice goes out %{time}.
1355
+ waiting_signal_late_no_deadline_html: Waiting for signal %{signal} since %{ago} (late since %{late}). No further late notice is due.
1353
1356
  waiting_signal_notify_html: Waiting for signal %{signal} since %{ago}. A late notice goes out %{time}, and the wait goes on.
1354
1357
  waiting_signal_timeout_html: Waiting for signal %{signal} since %{ago}. The wait times out %{time}.
1355
1358
  waiting_step_html: Waiting on %{step}.
@@ -1167,6 +1167,8 @@ es:
1167
1167
  skipped: Omitidos
1168
1168
  step_link: Ejecuciones en curso en %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Tiempo de espera
1171
+ wait_timeout_dynamic: dinámico (por ejecución)
1170
1172
  window_label: Intentos terminados en la ventana
1171
1173
  summary:
1172
1174
  attention: Requieren atención
@@ -1350,6 +1352,7 @@ es:
1350
1352
  waiting_batch_html: Esperando el lote %{batch} (%{done} de %{total} trabajos terminados).
1351
1353
  waiting_signal_html: Esperando la señal %{signal} desde %{ago}.
1352
1354
  waiting_signal_late_html: Esperando la señal %{signal} desde %{ago} (con retraso desde %{late}). El próximo aviso de retraso sale %{time}.
1355
+ waiting_signal_late_no_deadline_html: Esperando la señal %{signal} desde %{ago} (con retraso desde %{late}). No sale ningún otro aviso de retraso.
1353
1356
  waiting_signal_notify_html: Esperando la señal %{signal} desde %{ago}. Un aviso de retraso sale %{time} y la espera continúa.
1354
1357
  waiting_signal_timeout_html: Esperando la señal %{signal} desde %{ago}. La espera caduca %{time}.
1355
1358
  waiting_step_html: Esperando a %{step}.
@@ -1167,6 +1167,8 @@ fi:
1167
1167
  skipped: Ohitettu
1168
1168
  step_link: Käynnissä olevat ajot vaiheessa %{step}
1169
1169
  title: Putki
1170
+ wait_timeout: Odotuksen aikaraja
1171
+ wait_timeout_dynamic: dynaaminen (ajokohtainen)
1170
1172
  window_label: Ikkunassa päättyneet yritykset
1171
1173
  summary:
1172
1174
  attention: Vaatii toimia
@@ -1350,6 +1352,7 @@ fi:
1350
1352
  waiting_batch_html: Odottaa erää %{batch} (%{done} / %{total} tehtävää valmis).
1351
1353
  waiting_signal_html: Odottaa signaalia %{signal}, alkoi %{ago}.
1352
1354
  waiting_signal_late_html: Odottaa signaalia %{signal}, alkoi %{ago} (myöhässä alkaen %{late}). Seuraava myöhästymisilmoitus lähtee %{time}.
1355
+ waiting_signal_late_no_deadline_html: Odottaa signaalia %{signal}, alkoi %{ago} (myöhässä alkaen %{late}). Uusia myöhästymisilmoituksia ei lähde.
1353
1356
  waiting_signal_notify_html: Odottaa signaalia %{signal}, alkoi %{ago}. Myöhästymisilmoitus lähtee %{time}, ja odotus jatkuu.
1354
1357
  waiting_signal_timeout_html: Odottaa signaalia %{signal}, alkoi %{ago}. Odotus aikakatkaistaan %{time}.
1355
1358
  waiting_step_html: Odottaa vaihetta %{step}.
@@ -1167,6 +1167,8 @@ fr:
1167
1167
  skipped: Ignorées
1168
1168
  step_link: Exécutions en cours à %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Délai d'attente
1171
+ wait_timeout_dynamic: dynamique (par exécution)
1170
1172
  window_label: Tentatives terminées dans la fenêtre
1171
1173
  summary:
1172
1174
  attention: À traiter
@@ -1350,6 +1352,7 @@ fr:
1350
1352
  waiting_batch_html: En attente du lot %{batch} (%{done} travaux sur %{total} terminés).
1351
1353
  waiting_signal_html: En attente du signal %{signal} depuis %{ago}.
1352
1354
  waiting_signal_late_html: En attente du signal %{signal} depuis %{ago} (en retard depuis %{late}). Le prochain avis de retard part %{time}.
1355
+ waiting_signal_late_no_deadline_html: En attente du signal %{signal} depuis %{ago} (en retard depuis %{late}). Aucun autre avis de retard ne part.
1353
1356
  waiting_signal_notify_html: En attente du signal %{signal} depuis %{ago}. Un avis de retard part %{time} et l'attente continue.
1354
1357
  waiting_signal_timeout_html: En attente du signal %{signal} depuis %{ago}. L'attente expire %{time}.
1355
1358
  waiting_step_html: En attente de %{step}.
@@ -1167,6 +1167,8 @@ it:
1167
1167
  skipped: Saltati
1168
1168
  step_link: Esecuzioni in corso a %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Timeout di attesa
1171
+ wait_timeout_dynamic: dinamico (per esecuzione)
1170
1172
  window_label: Tentativi terminati nella finestra
1171
1173
  summary:
1172
1174
  attention: Richiedono attenzione
@@ -1350,6 +1352,7 @@ it:
1350
1352
  waiting_batch_html: In attesa del lotto %{batch} (%{done} lavori su %{total} completati).
1351
1353
  waiting_signal_html: In attesa del segnale %{signal} da %{ago}.
1352
1354
  waiting_signal_late_html: In attesa del segnale %{signal} da %{ago} (in ritardo da %{late}). Il prossimo avviso di ritardo parte %{time}.
1355
+ waiting_signal_late_no_deadline_html: In attesa del segnale %{signal} da %{ago} (in ritardo da %{late}). Non parte nessun altro avviso di ritardo.
1353
1356
  waiting_signal_notify_html: In attesa del segnale %{signal} da %{ago}. Un avviso di ritardo parte %{time} e l'attesa continua.
1354
1357
  waiting_signal_timeout_html: In attesa del segnale %{signal} da %{ago}. L'attesa scade %{time}.
1355
1358
  waiting_step_html: In attesa di %{step}.
@@ -1167,6 +1167,8 @@ ja:
1167
1167
  skipped: スキップ
1168
1168
  step_link: "%{step} の進行中の実行"
1169
1169
  title: パイプライン
1170
+ wait_timeout: 待機タイムアウト
1171
+ wait_timeout_dynamic: 動的(実行ごと)
1170
1172
  window_label: 期間内に終了した試行
1171
1173
  summary:
1172
1174
  attention: 要対応
@@ -1350,6 +1352,7 @@ ja:
1350
1352
  waiting_batch_html: バッチ %{batch} を待機中です(%{total} 件中 %{done} 件のジョブが完了)。
1351
1353
  waiting_signal_html: シグナル %{signal} を %{ago} から待機中です。
1352
1354
  waiting_signal_late_html: シグナル %{signal} を %{ago} から待機中です(%{late} から遅延)。次の遅延通知は %{time} に送られます。
1355
+ waiting_signal_late_no_deadline_html: シグナル %{signal} を %{ago} から待機中です(%{late} から遅延)。これ以上の遅延通知は送られません。
1353
1356
  waiting_signal_notify_html: シグナル %{signal} を %{ago} から待機中です。遅延通知は %{time} に送られ、待機は続きます。
1354
1357
  waiting_signal_timeout_html: シグナル %{signal} を %{ago} から待機中です。待機は %{time} にタイムアウトします。
1355
1358
  waiting_step_html: "%{step} を待機中です。"
@@ -1167,6 +1167,8 @@ nb:
1167
1167
  skipped: Hoppet over
1168
1168
  step_link: Pågående kjøringer ved %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Tidsgrense for venting
1171
+ wait_timeout_dynamic: dynamisk (per kjøring)
1170
1172
  window_label: Forsøk som ble avsluttet i vinduet
1171
1173
  summary:
1172
1174
  attention: Trenger tiltak
@@ -1350,6 +1352,7 @@ nb:
1350
1352
  waiting_batch_html: Venter på gruppen %{batch} (%{done} av %{total} jobber ferdige).
1351
1353
  waiting_signal_html: Venter på signalet %{signal} siden %{ago}.
1352
1354
  waiting_signal_late_html: Venter på signalet %{signal} siden %{ago} (forsinket siden %{late}). Neste forsinkelsesvarsel sendes %{time}.
1355
+ waiting_signal_late_no_deadline_html: Venter på signalet %{signal} siden %{ago} (forsinket siden %{late}). Det sendes ingen flere forsinkelsesvarsler.
1353
1356
  waiting_signal_notify_html: Venter på signalet %{signal} siden %{ago}. Et forsinkelsesvarsel sendes %{time}, og ventingen fortsetter.
1354
1357
  waiting_signal_timeout_html: Venter på signalet %{signal} siden %{ago}. Ventetiden utløper %{time}.
1355
1358
  waiting_step_html: Venter på %{step}.
@@ -1167,6 +1167,8 @@ nl:
1167
1167
  skipped: Overgeslagen
1168
1168
  step_link: Lopende runs bij %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Wachttime-out
1171
+ wait_timeout_dynamic: dynamisch (per run)
1170
1172
  window_label: Pogingen die in het venster eindigden
1171
1173
  summary:
1172
1174
  attention: Aandacht nodig
@@ -1350,6 +1352,7 @@ nl:
1350
1352
  waiting_batch_html: Wacht op groep %{batch} (%{done} van %{total} taken klaar).
1351
1353
  waiting_signal_html: Wacht op signaal %{signal}, gestart %{ago}.
1352
1354
  waiting_signal_late_html: Wacht op signaal %{signal}, gestart %{ago} (te laat sinds %{late}). De volgende melding over de vertraging gaat %{time} uit.
1355
+ waiting_signal_late_no_deadline_html: Wacht op signaal %{signal}, gestart %{ago} (te laat sinds %{late}). Er gaat geen nieuwe melding over de vertraging meer uit.
1353
1356
  waiting_signal_notify_html: Wacht op signaal %{signal}, gestart %{ago}. Een melding over de vertraging gaat %{time} uit, en het wachten gaat door.
1354
1357
  waiting_signal_timeout_html: Wacht op signaal %{signal}, gestart %{ago}. De wachttijd verloopt %{time}.
1355
1358
  waiting_step_html: Wacht op %{step}.
@@ -1167,6 +1167,8 @@ pt:
1167
1167
  skipped: Ignoradas
1168
1168
  step_link: Execuções em curso em %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Tempo limite de espera
1171
+ wait_timeout_dynamic: dinâmico (por execução)
1170
1172
  window_label: Tentativas terminadas na janela
1171
1173
  summary:
1172
1174
  attention: Precisam de atenção
@@ -1350,6 +1352,7 @@ pt:
1350
1352
  waiting_batch_html: Aguardando o lote %{batch} (%{done} de %{total} trabalhos concluídos).
1351
1353
  waiting_signal_html: Aguardando o sinal %{signal} desde %{ago}.
1352
1354
  waiting_signal_late_html: Aguardando o sinal %{signal} desde %{ago} (atrasado desde %{late}). O próximo aviso de atraso sai %{time}.
1355
+ waiting_signal_late_no_deadline_html: Aguardando o sinal %{signal} desde %{ago} (atrasado desde %{late}). Não sai mais nenhum aviso de atraso.
1353
1356
  waiting_signal_notify_html: Aguardando o sinal %{signal} desde %{ago}. Um aviso de atraso sai %{time} e a espera continua.
1354
1357
  waiting_signal_timeout_html: Aguardando o sinal %{signal} desde %{ago}. A espera expira %{time}.
1355
1358
  waiting_step_html: Aguardando %{step}.
@@ -1167,6 +1167,8 @@ sv:
1167
1167
  skipped: Överhoppade
1168
1168
  step_link: Pågående körningar vid %{step}
1169
1169
  title: Pipeline
1170
+ wait_timeout: Väntetidsgräns
1171
+ wait_timeout_dynamic: dynamisk (per körning)
1170
1172
  window_label: Försök som avslutades i fönstret
1171
1173
  summary:
1172
1174
  attention: Behöver åtgärd
@@ -1350,6 +1352,7 @@ sv:
1350
1352
  waiting_batch_html: Väntar på gruppen %{batch} (%{done} av %{total} jobb klara).
1351
1353
  waiting_signal_html: Väntar på signalen %{signal} sedan %{ago}.
1352
1354
  waiting_signal_late_html: Väntar på signalen %{signal} sedan %{ago} (sen sedan %{late}). Nästa förseningsavisering går ut %{time}.
1355
+ waiting_signal_late_no_deadline_html: Väntar på signalen %{signal} sedan %{ago} (sen sedan %{late}). Ingen fler förseningsavisering går ut.
1353
1356
  waiting_signal_notify_html: Väntar på signalen %{signal} sedan %{ago}. En förseningsavisering går ut %{time}, och väntan fortsätter.
1354
1357
  waiting_signal_timeout_html: Väntar på signalen %{signal} sedan %{ago}. Väntan tar slut %{time}.
1355
1358
  waiting_step_html: Väntar på %{step}.
@@ -32,8 +32,10 @@ class AddPgbusWorkflows < ActiveRecord::Migration<%= migration_version %>
32
32
 
33
33
  add_index :pgbus_workflows, :workflow_id,
34
34
  unique: true, name: "idx_pgbus_workflows_workflow_id", if_not_exists: true
35
+ # status::text keeps the predicate stable in schema.rb: without the cast, a
36
+ # migrated and a schema-loaded database dump it differently (issue #573).
35
37
  add_index :pgbus_workflows, [:workflow_class, :subject_type, :subject_id],
36
- unique: true, where: "status IN ('ready','performing','waiting','paused')",
38
+ unique: true, where: "status::text IN ('ready','performing','waiting','paused')",
37
39
  name: "idx_pgbus_workflows_one_ongoing", if_not_exists: true
38
40
  add_index :pgbus_workflows, [:workflow_class, :idempotency_key],
39
41
  unique: true, where: "idempotency_key IS NOT NULL",
@@ -238,8 +238,10 @@ class CreatePgbusTables < ActiveRecord::Migration<%= migration_version %>
238
238
 
239
239
  add_index :pgbus_workflows, :workflow_id,
240
240
  unique: true, name: "idx_pgbus_workflows_workflow_id"
241
+ # status::text keeps the predicate stable in schema.rb: without the cast, a
242
+ # migrated and a schema-loaded database dump it differently (issue #573).
241
243
  add_index :pgbus_workflows, [:workflow_class, :subject_type, :subject_id],
242
- unique: true, where: "status IN ('ready','performing','waiting','paused')",
244
+ unique: true, where: "status::text IN ('ready','performing','waiting','paused')",
243
245
  name: "idx_pgbus_workflows_one_ongoing"
244
246
  add_index :pgbus_workflows, [:workflow_class, :idempotency_key],
245
247
  unique: true, where: "idempotency_key IS NOT NULL",
@@ -12,7 +12,7 @@ module Pgbus
12
12
  # config.workflow_max_payload_size = 131_072 # bytes: arguments and step outputs
13
13
  # config.workflow_error_backtrace_lines = 50 # backtrace lines kept on a failed step
14
14
  # config.workflow_waiting_visibility_threshold = 7.days # a step waiting this long without a
15
- # # wait_timeout: counts as stuck (waiting_overdue)
15
+ # # deadline counts as stuck (waiting_overdue)
16
16
  # config.workflow_subject_url = ->(type, id) { "/admin/#{type.underscore.pluralize}/#{id}" }
17
17
  # # the dashboard's subject link (nil: plain text)
18
18
  module WorkflowSettings
@@ -11,7 +11,7 @@ module Pgbus
11
11
  tool_name "pgbus_workflow_stats"
12
12
  title "Pgbus Workflow Stats"
13
13
  description <<~DESC
14
- One Pgbus::Workflow class over a window (1h, 24h default, 7d or 30d, capped at workflow_retention). steps, in declaration order (gravestones as kind "removed"; names only the rows hold as declared: false): counts (runs sitting at the step now by run status ready/performing/waiting/paused), run_ms (p50/p95/p99 of finished_at - started_at over completed or failed attempts) and pickup_ms (p50/p95 of scheduled → started), rates (failed: an error was recorded, reattempted, skipped) and outcomes over the sample_size attempts that finished in the window; percentiles over fewer than 20 samples are noisy. Also: started/finished/cancelled, e2e_ms, duration_histogram (finished runs per created → finished range), throughput (runs started/finished/cancelled per bucket_seconds), funnel (of the runs started in the window: reached = runs with a row for the step, stopped = where each run is now by status) and by_revision (the five app revisions that finished an attempt last, newest first, totals and per step).
14
+ One Pgbus::Workflow class over a window (1h, 24h default, 7d or 30d, capped at workflow_retention). steps, in declaration order (gravestones as kind "removed"; names only the rows hold as declared: false): wait_timeout (a wait_for: step's wait_timeout: in seconds, "dynamic" when a lambda or a method name sets it per run, null without one), counts (runs sitting at the step now by run status ready/performing/waiting/paused), run_ms (p50/p95/p99 of finished_at - started_at over completed or failed attempts) and pickup_ms (p50/p95 of scheduled → started), rates (failed: an error was recorded, reattempted, skipped) and outcomes over the sample_size attempts that finished in the window; percentiles over fewer than 20 samples are noisy. Also: started/finished/cancelled, e2e_ms, duration_histogram (finished runs per created → finished range), throughput (runs started/finished/cancelled per bucket_seconds), funnel (of the runs started in the window: reached = runs with a row for the step, stopped = where each run is now by status) and by_revision (the five app revisions that finished an attempt last, newest first, totals and per step).
15
15
  DESC
16
16
 
17
17
  input_schema(
@@ -9,7 +9,7 @@ module Pgbus
9
9
  tool_name "pgbus_workflows_attention"
10
10
  title "Pgbus Workflows Needing Attention"
11
11
  description <<~DESC
12
- Ongoing Pgbus::Workflow runs that need a person, grouped by (workflow_class, step_name, reason, error_class): reason is paused_error, stalled, paused_manual, overdue (due but not picked up past workflow_stall_threshold) or waiting_overdue (waiting past workflow_waiting_visibility_threshold without a wait_timeout). Each group has its run count, first_seen_at/last_seen_at, the status its runs are in and the actions that status allows; total_runs sums the groups (at most 200 groups, biggest first). workflow_class narrows them to one class. The newest error message is redacted unless the server allows payloads and include_payloads is set.
12
+ Ongoing Pgbus::Workflow runs that need a person, grouped by (workflow_class, step_name, reason, error_class): reason is paused_error, stalled, paused_manual, overdue (due but not picked up past workflow_stall_threshold) or waiting_overdue (waiting past workflow_waiting_visibility_threshold without a deadline: no wait_timeout, or a per-run one that set none or cleared it after a :notify notice). Each group has its run count, first_seen_at/last_seen_at, the status its runs are in and the actions that status allows; total_runs sums the groups (at most 200 groups, biggest first). workflow_class narrows them to one class. The newest error message is redacted unless the server allows payloads and include_payloads is set.
13
13
  DESC
14
14
 
15
15
  input_schema(
data/lib/pgbus/version.rb CHANGED
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Pgbus
4
- VERSION = "0.20.1"
4
+ VERSION = "0.21.0"
5
5
  end
@@ -158,12 +158,14 @@ module Pgbus
158
158
  end
159
159
  end
160
160
 
161
- # { step_id => the first deadline its wait missed } for waiting steps
162
- # with a deadline, one query: the first wait_timed_out event of each.
161
+ # { step_id => the first deadline its wait missed } for steps waiting
162
+ # on a signal, one query: the first wait_timed_out event of each.
163
163
  # Only on_timeout: :notify leaves a step waiting after that event
164
- # (issue #565): every other on_timeout ends the attempt.
164
+ # (issue #565): every other on_timeout ends the attempt. A per-run
165
+ # wait_timeout: may have cleared the deadline at that notice (issue
166
+ # #574), so a step without one is looked up too.
165
167
  def late_since(waiting)
166
- steps = waiting.select { |step| step_attribute(step, :timeout_at) }
168
+ steps = waiting.select { |step| step_attribute(step, :wait_for) }
167
169
  return {} if steps.empty?
168
170
 
169
171
  WorkflowEvent.where(workflow_id: steps.map(&:workflow_id), step_id: steps.map(&:id),
@@ -47,14 +47,14 @@ module Pgbus
47
47
  )
48
48
  end
49
49
 
50
- # [{ name:, kind:, removed:, declared: }] in pipeline order.
50
+ # [{ name:, kind:, removed:, declared:, wait_timeout: }] in pipeline order.
51
51
  def step_list(klass, stats, ongoing, reached, stopped)
52
52
  seen = (stats.keys + ongoing.map { |row| row[:step] } + reached.keys + stopped.keys.map(&:first)).compact.uniq
53
53
  declared = klass ? klass.workflow_steps.entries.map { |entry| declared_step(entry) } : []
54
54
  extra = (seen - declared.map { |entry| entry[:name] }).sort_by do |name|
55
55
  [reached.dig(name, :position) || Float::INFINITY, stats.dig(name, :first_seen_at)&.to_f || Float::INFINITY, name]
56
56
  end
57
- declared + extra.map { |name| { name: name, kind: nil, removed: false, declared: false } }
57
+ declared + extra.map { |name| { name: name, kind: nil, removed: false, declared: false, wait_timeout: nil } }
58
58
  end
59
59
 
60
60
  def declared_step(entry)
@@ -63,7 +63,15 @@ module Pgbus
63
63
  elsif entry.waits_for_signal? then "signal"
64
64
  else "step"
65
65
  end
66
- { name: entry.name, kind: kind, removed: entry.removed?, declared: true }
66
+ { name: entry.name, kind: kind, removed: entry.removed?, declared: true, wait_timeout: wait_timeout(entry) }
67
+ end
68
+
69
+ # A signal step's wait_timeout: in seconds, "dynamic" for a per-run one
70
+ # (issue #574), nil without one.
71
+ def wait_timeout(entry)
72
+ return nil if entry.removed?
73
+
74
+ entry.dynamic_wait_timeout? ? "dynamic" : entry.wait_timeout
67
75
  end
68
76
 
69
77
  def step(entry, stats, ongoing)
@@ -51,9 +51,12 @@ module Pgbus
51
51
  return ["waiting", "waiting_step", { step: step }] unless row[:waiting_for]
52
52
 
53
53
  args = { signal: row[:waiting_for], ago: row[:waiting_since] }
54
- return ["waiting", "waiting_signal", args] unless row[:timeout_at]
54
+ return signal_deadline(row, args.merge(time: row[:timeout_at])) if row[:timeout_at]
55
+ # A per-run wait_timeout: reported once under on_timeout: :notify,
56
+ # then left without a deadline (issue #574).
57
+ return ["waiting", "waiting_signal_late_no_deadline", args.merge(late: row[:late_since])] if row[:late_since]
55
58
 
56
- signal_deadline(row, args.merge(time: row[:timeout_at]))
59
+ ["waiting", "waiting_signal", args]
57
60
  end
58
61
 
59
62
  # An on_timeout: :notify deadline sends a late notice and the wait goes
@@ -94,6 +94,33 @@ module Pgbus
94
94
  @timed_out == true
95
95
  end
96
96
 
97
+ # Inside a step (its guards and its body): true when this attempt holds
98
+ # a signal before its gate runs: signal! woke the step, or an earlier
99
+ # attempt of it took one and reattempted or failed into a pause. False
100
+ # on an attempt that has not waited yet (even one whose gate then takes
101
+ # a buffered signal), after an on_timeout: :run wake, and in a step
102
+ # without wait_for:. The same answer for the whole attempt.
103
+ def woken?
104
+ step_context!(:woken?)
105
+ @woken = !@signal_peek&.held.nil? if @woken.nil?
106
+ @woken
107
+ end
108
+
109
+ # Inside a step (its guards and its body): true when the gate runs this
110
+ # attempt with a signal: the one it holds (woken?), or a pending one its
111
+ # wait_for: matcher takes. The guard for "skip a wait the record already
112
+ # passed", which neither skips a signalled step nor leaves its signal
113
+ # pending. False after an on_timeout: :run wake (the gate takes no
114
+ # signal then) and in a step without wait_for:. Reads only: the gate
115
+ # claims the signal after the guards.
116
+ #
117
+ # step :await_shipped, skip_if: -> { subject.shipped? && !signal_ready? }, wait_for: :shipped do … end
118
+ def signal_ready?
119
+ step_context!(:signal_ready?)
120
+ @signal_ready = !@signal_peek&.ready.nil? if @signal_ready.nil?
121
+ @signal_ready
122
+ end
123
+
97
124
  # Deliver a signal to this run. Stored under the run's lock; a step
98
125
  # waiting for it (wait_for:) is woken at once, otherwise it waits,
99
126
  # pending, for the step that claims it. With an idempotency_key a
@@ -164,19 +191,25 @@ module Pgbus
164
191
  @guarding = false
165
192
  end
166
193
 
167
- # Used by the StepJob only.
168
- def enter_step(context)
194
+ # Used by the StepJob only. +signals+ is the attempt's SignalPeek, which
195
+ # woken? and signal_ready? read until the gate answers them.
196
+ def enter_step(context, signals: nil)
169
197
  @step_context = context
198
+ @signal_peek = signals
170
199
  @output_values = nil
171
200
  @pending_output = nil
172
201
  @received_signal = nil
173
202
  @timed_out = false
203
+ @woken = nil
204
+ @signal_ready = nil
174
205
  end
175
206
 
176
207
  # Used by the StepJob's signal gate only.
177
- def enter_wait(signal:, timed_out:)
208
+ def enter_wait(signal:, timed_out:, woken: false)
178
209
  @received_signal = signal
179
210
  @timed_out = timed_out
211
+ @woken = woken
212
+ @signal_ready = !signal.nil?
180
213
  end
181
214
  end
182
215
  end
@@ -68,11 +68,11 @@ module Pgbus
68
68
 
69
69
  # --- start! and rerun! --------------------------------------------
70
70
 
71
- def start(klass, subject:, arguments:, idempotency_key: nil)
71
+ def start(klass, subject:, arguments:, idempotency_key: nil, actor: DEFAULT_ACTOR, reason: nil)
72
72
  attrs = run_attributes(klass, idempotency_key: idempotency_key)
73
73
  .merge(Payload.subject_attributes(subject))
74
74
  .merge(arguments: Payload.dump(arguments, what: "arguments of #{klass.name}"))
75
- insert_run(klass, attrs, actor: DEFAULT_ACTOR)
75
+ insert_run(klass, attrs, actor: actor, reason: reason)
76
76
  end
77
77
 
78
78
  # A finished or cancelled run started again: same subject and
@@ -0,0 +1,47 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Pgbus
4
+ class Workflow
5
+ # What a wait_for: step's attempt holds and what its gate would run it
6
+ # with (issue #572), read without claiming anything. The StepJob builds
7
+ # one per attempt: the step readers woken? and signal_ready? read it in
8
+ # the guards, and the gate decides with the same one, so the gate takes
9
+ # exactly the signal signal_ready? promised. All of it runs under the
10
+ # run's row lock, so each answer is looked up (and a matcher run) once.
11
+ class SignalPeek
12
+ def initialize(record, step, definition, now: Time.current)
13
+ @record = record
14
+ @step = step
15
+ @definition = definition
16
+ @now = now
17
+ end
18
+
19
+ # The claimed signal the step holds (signal! woke it, or an earlier
20
+ # attempt took one and reattempted or paused), or nil. Always nil for a
21
+ # step without wait_for:.
22
+ def held
23
+ return @held if defined?(@held)
24
+
25
+ @held = waits? ? WorkflowSignal.held_by_step(@record.workflow_id, @step.step_name) : nil
26
+ end
27
+
28
+ # The sweep woke this attempt with on_timeout: :run: its body runs
29
+ # without a signal (Signals.timed_out?).
30
+ def timed_out? = Signals.timed_out?(@step, @now)
31
+
32
+ # The signal the gate runs this attempt with: the one the step holds,
33
+ # else the oldest pending one its matcher takes. nil when the wait timed
34
+ # out (the body runs without one), when nothing matches (the step
35
+ # parks), and for a step without wait_for:.
36
+ def ready
37
+ return @ready if defined?(@ready)
38
+
39
+ @ready = waits? && !timed_out? ? held || Signals.match(@record, @definition.wait_for) : nil
40
+ end
41
+
42
+ private
43
+
44
+ def waits? = @definition.waits_for_signal?
45
+ end
46
+ end
47
+ end
@@ -12,6 +12,8 @@ module Pgbus
12
12
  # lock: a signal this step already holds (woken by signal!, or claimed by
13
13
  # an earlier attempt that reattempted or paused on a failure), else the
14
14
  # oldest pending one its matcher takes, else the step parks as waiting.
15
+ # The guards run first; woken? and signal_ready? (issue #572) show them
16
+ # what the gate holds and would take, through SignalPeek, claiming nothing.
15
17
  #
16
18
  # A claimed signal is consumed when its step ends, except when that
17
19
  # attempt reattempts or fails into a pause: the next attempt runs with it.
@@ -38,23 +40,38 @@ module Pgbus
38
40
  end
39
41
 
40
42
  # The gate of a wait_for: step. True when the body runs (with a signal,
41
- # or timed out), false when the step parked. Sets received_signal and
42
- # timed_out? on +workflow+.
43
- def gate(record, step, definition, workflow, now: Time.current)
44
- if timed_out?(step, now)
43
+ # or timed out), false when the step parked. Sets received_signal,
44
+ # timed_out?, woken? and signal_ready? on +workflow+. Which signal it
45
+ # takes is SignalPeek#ready: given the attempt's +peek+ (the StepJob
46
+ # passes the one its guards read), the very answer signal_ready? gave.
47
+ def gate(record, step, definition, workflow, now: Time.current, peek: nil)
48
+ peek ||= SignalPeek.new(record, step, definition, now: now)
49
+ if peek.timed_out?
45
50
  workflow.send(:enter_wait, signal: nil, timed_out: true)
46
51
  return true
47
52
  end
48
53
 
49
- signal = WorkflowSignal.held_by_step(record.workflow_id, step.step_name) || match(record, definition.wait_for)
50
- return park(record, step, definition, now) unless signal
54
+ signal = peek.ready
55
+ return park(record, step, definition, workflow, now) unless signal
51
56
 
57
+ woken = !peek.held.nil?
52
58
  claim!(record, step, signal, actor: Runner::SYSTEM_ACTOR) if signal.status == "pending"
53
59
  step.update_columns(signal_id: signal.id) unless step.signal_id == signal.id
54
- workflow.send(:enter_wait, signal: signal, timed_out: false)
60
+ workflow.send(:enter_wait, signal: signal, timed_out: false, woken: woken)
55
61
  true
56
62
  end
57
63
 
64
+ # An attempt that ended before its gate (cancel_if, skip_if, a guard
65
+ # that raised, a removed_step) records +held+, the signal the step
66
+ # holds, on its row, so settle treats it as for any attempt: consumed
67
+ # when the step ends, kept when it reattempts or fails into a pause. A
68
+ # post-reattempt row carries no signal_id until its gate runs. A
69
+ # pending signal stays pending: no step took it, and a later wait_for:
70
+ # step may.
71
+ def adopt_held(step, held)
72
+ step.update_columns(signal_id: held.id) if held && step.signal_id != held.id
73
+ end
74
+
58
75
  # The step's attempt ended (StepOutcome.apply): consume the signal it
59
76
  # holds, unless the next attempt runs with it.
60
77
  def settle(step, status, column, now:)
@@ -116,14 +133,18 @@ module Pgbus
116
133
  nil
117
134
  end
118
135
 
119
- def park(record, step, definition, now) # rubocop:disable Naming/PredicateMethod -- the gate's false: the body does not run
120
- timeout_at = definition.wait_timeout && (now + definition.wait_timeout)
136
+ # A per-run wait_timeout: is evaluated here, before Waiting.park! writes
137
+ # the waiting state: one that raises (an invalid return raises
138
+ # ArgumentError) leaves the attempt in_progress for the step's exception
139
+ # policies; nil parks it without a deadline.
140
+ def park(record, step, definition, workflow, now) # rubocop:disable Naming/PredicateMethod -- the gate's false: the body does not run
141
+ timeout_at = definition.timeout_at_for(workflow, now)
121
142
  Waiting.park!(record, step, now: now, wait_for: definition.wait_for_name, timeout_at: timeout_at)
122
143
  false
123
144
  end
124
145
 
125
146
  # This attempt waited, holds no signal, and its wait_timeout passed: the
126
- # sweep woke it with on_timeout: :run.
147
+ # sweep woke it with on_timeout: :run (SignalPeek#timed_out?).
127
148
  def timed_out?(step, now)
128
149
  !step.waiting_since.nil? && step.signal_id.nil? && !step.timeout_at.nil? && step.timeout_at <= now + 1
129
150
  end
@@ -15,8 +15,11 @@ module Pgbus
15
15
  # (job_options:) may set. Anything else raises at class load.
16
16
  JOB_OPTIONS = %i[queue priority wait wait_until].freeze
17
17
 
18
+ # wait_timeout is the static wait_timeout: in seconds (nil for a per-run
19
+ # one); wait_timeout_callable the per-run one as given, a lambda or a
20
+ # method name (issue #574).
18
21
  attr_reader :name, :body, :wait, :skip_if, :job_options, :timeout, :exception_policies,
19
- :wait_for, :wait_for_name, :wait_timeout, :on_timeout
22
+ :wait_for, :wait_for_name, :wait_timeout, :wait_timeout_callable, :on_timeout
20
23
 
21
24
  def self.validate_job_options!(options)
22
25
  raise ArgumentError, "job_options: must be a Hash, got #{options.inspect}" unless options.is_a?(Hash)
@@ -104,6 +107,44 @@ module Pgbus
104
107
  result ? true : false
105
108
  end
106
109
 
110
+ # True when wait_timeout: is a lambda or a method name, evaluated per run.
111
+ def dynamic_wait_timeout? = !wait_timeout_callable.nil?
112
+
113
+ # The timeout_at of the step parking now on +workflow+: now plus a
114
+ # static wait_timeout:, what a per-run one returns (a deadline already
115
+ # past is now, so the next sweep applies on_timeout), or nil.
116
+ def timeout_at_for(workflow, now)
117
+ return now + wait_timeout if wait_timeout
118
+ return nil unless dynamic_wait_timeout?
119
+
120
+ deadline = resolve_wait_timeout(workflow)
121
+ deadline.is_a?(Time) ? [deadline, now].max : deadline && (now + deadline)
122
+ end
123
+
124
+ # What the per-run wait_timeout: returns on +workflow+: a Time (an
125
+ # absolute deadline), seconds from now, or nil (no timeout). It runs as
126
+ # a guard does (a flow-control verb raises NotInStep), in a savepoint:
127
+ # a query of its own that fails in the database must not abort the
128
+ # run's transaction, which the step's outcome is written in. Anything
129
+ # else raises ArgumentError, as an error it raises goes up, for the
130
+ # step's exception policies.
131
+ def resolve_wait_timeout(workflow)
132
+ callable = wait_timeout_callable
133
+ value = WorkflowEntry.transaction(requires_new: true) do
134
+ workflow.send(:guarding) { callable.is_a?(Symbol) ? workflow.send(callable) : workflow.instance_exec(&callable) }
135
+ end
136
+ return nil if value.nil?
137
+ if value.is_a?(Numeric) || self.class.duration?(value)
138
+ return self.class.seconds(value, "wait_timeout (step #{name})", positive: true)
139
+ end
140
+
141
+ time = value.to_time if value.respond_to?(:to_time) && !value.is_a?(String)
142
+ return time if time.is_a?(Time)
143
+
144
+ raise ArgumentError, "wait_timeout: of step #{name} must return a Time, a Duration, a number of seconds or nil, " \
145
+ "got #{value.inspect}"
146
+ end
147
+
107
148
  private
108
149
 
109
150
  def assign_wait(wait_for, wait_timeout, on_timeout)
@@ -112,8 +153,9 @@ module Pgbus
112
153
  raise ArgumentError, "on_timeout: needs wait_for: and wait_timeout:" if !on_timeout.nil? && (@wait_for.nil? || wait_timeout.nil?)
113
154
  raise ArgumentError, "wait_timeout: needs wait_for: (it bounds the wait for a signal)" if wait_timeout && @wait_for.nil?
114
155
 
115
- @wait_timeout = self.class.seconds(wait_timeout, "wait_timeout", positive: true)
116
- @on_timeout = @wait_timeout && normalize_on_timeout(on_timeout.nil? ? :pause! : on_timeout)
156
+ @wait_timeout_callable = wait_timeout if wait_timeout.is_a?(Symbol) || wait_timeout.is_a?(Proc)
157
+ @wait_timeout = @wait_timeout_callable ? nil : self.class.seconds(wait_timeout, "wait_timeout", positive: true)
158
+ @on_timeout = (@wait_timeout || @wait_timeout_callable) && normalize_on_timeout(on_timeout.nil? ? :pause! : on_timeout)
117
159
  end
118
160
 
119
161
  def normalize_on_timeout(value)
@@ -13,7 +13,8 @@ module Pgbus
13
13
  # heartbeat is on: its worker died, and the stall action applies now.
14
14
  # 2. Guards, still under the lock: the class or the step no longer
15
15
  # defined (pause with StepNotDefined, or skip a removed_step),
16
- # cancel_if, skip_if.
16
+ # cancel_if, skip_if (which read woken? and signal_ready? through the
17
+ # attempt's SignalPeek), then a wait_for: step's signal gate.
17
18
  # 3. The body, outside any transaction, inside `catch` for the
18
19
  # flow-control verbs and `rescue StandardError` for the policies.
19
20
  # 4. The outcome, under the lock again (StepOutcome).
@@ -21,7 +22,9 @@ module Pgbus
21
22
  # Whatever the step's outcome, the job itself completes normally: the
22
23
  # step row and the pgbus.workflow.* events carry the truth.
23
24
  class StepExecution
24
- Claim = Data.define(:record, :step, :klass, :definition, :workflow, :guard_error)
25
+ # +adopt_held+: a guard raised before the gate, so the signal the step
26
+ # holds is put on this attempt's row before its outcome (Signals.adopt_held).
27
+ Claim = Data.define(:record, :step, :klass, :definition, :workflow, :guard_error, :adopt_held)
25
28
 
26
29
  def initialize(step_id, delivery: Pgbus::ActiveJob::Delivery.current, config: Pgbus.configuration)
27
30
  @step_id = step_id
@@ -103,15 +106,23 @@ module Pgbus
103
106
  return undefined!(record, step, notes, "#{record.workflow_class} is not a Pgbus::Workflow class any more") unless klass
104
107
 
105
108
  unless definition
106
- return skip!(record, step, notes, klass) if klass.workflow_steps.removed?(step.step_name)
109
+ if klass.workflow_steps.removed?(step.step_name)
110
+ # The definition is gone, so whether it waited is unknown: look for a
111
+ # held signal only where the signal schema exists.
112
+ Signals.adopt_held(step, WorkflowSignal.held_by_step(record.workflow_id, step.step_name)) if Sweep.waiting_ready?
113
+ return skip!(record, step, notes, klass)
114
+ end
107
115
 
108
116
  return undefined!(record, step, notes, "#{klass.name} has no step #{step.step_name} " \
109
117
  "(declare `removed_step :#{step.step_name}` to skip it)")
110
118
  end
111
119
 
112
120
  workflow = klass.new(record)
113
- workflow.send(:enter_step, StepContext::Attempt.new(step_id: step.id, step_name: step.step_name, attempt: step.attempt))
114
- claim = Claim.new(record: record, step: step, klass: klass, definition: definition, workflow: workflow, guard_error: nil)
121
+ peek = SignalPeek.new(record, step, definition)
122
+ workflow.send(:enter_step, StepContext::Attempt.new(step_id: step.id, step_name: step.step_name, attempt: step.attempt),
123
+ signals: peek)
124
+ claim = Claim.new(record: record, step: step, klass: klass, definition: definition, workflow: workflow, guard_error: nil,
125
+ adopt_held: false)
115
126
  begin
116
127
  verdict = workflow.send(:guarding) do
117
128
  next FlowControl::Outcome.cancel if cancel?(klass, workflow)
@@ -119,20 +130,23 @@ module Pgbus
119
130
  FlowControl::Outcome.skip if definition.skip?(workflow)
120
131
  end
121
132
  rescue StandardError => e
122
- return claim.with(guard_error: e)
133
+ return claim.with(guard_error: e, adopt_held: definition.waits_for_signal?)
134
+ end
135
+ if verdict
136
+ Signals.adopt_held(step, peek.held)
137
+ return outcome!(record, step, notes, klass, verdict)
123
138
  end
124
- return outcome!(record, step, notes, klass, verdict) if verdict
125
139
 
126
- gate(record, step, definition, workflow, claim)
140
+ gate(record, step, definition, workflow, claim, peek)
127
141
  end
128
142
 
129
143
  # A wait_for: step runs only with a signal (or once its wait timed
130
144
  # out); without one it parks and this delivery ends. A matcher that
131
145
  # raises is handed to the exception policies, like a guard.
132
- def gate(record, step, definition, workflow, claim)
146
+ def gate(record, step, definition, workflow, claim, peek)
133
147
  return claim unless definition.waits_for_signal?
134
148
 
135
- Signals.gate(record, step, definition, workflow) ? claim : nil
149
+ Signals.gate(record, step, definition, workflow, peek: peek) ? claim : nil
136
150
  rescue StandardError => e
137
151
  claim.with(guard_error: e)
138
152
  end
@@ -185,6 +199,9 @@ module Pgbus
185
199
  next park_fan_out(record, claim, outcome, notes) if outcome&.action == :fan_out
186
200
 
187
201
  notes.report(decision.handler_error, handler_context(claim)) if decision&.handler_error
202
+ # Here rather than in the claim's transaction, which the statement
203
+ # a guard failed on may have aborted.
204
+ Signals.adopt_held(claim.step, SignalPeek.new(record, claim.step, claim.definition).held) if claim.adopt_held
188
205
  StepOutcome.apply(record, claim.step, outcome, notes: notes, error: error, output: claim.workflow.pending_output,
189
206
  report: decision ? decision.report : true, duration: duration,
190
207
  klass: claim.klass)
@@ -14,7 +14,8 @@ module Pgbus
14
14
  # received_signal nil and timed_out? true
15
15
  # :notify only the event and the instrumentation (issue #565): the
16
16
  # same attempt keeps waiting and timeout_at moves to its next
17
- # deadline, one wait_timeout on (see next_deadline)
17
+ # deadline, one wait_timeout on (see next_deadline); a per-run
18
+ # wait_timeout: is evaluated again for it (see notify_deadline)
18
19
  #
19
20
  # A signal that arrives after the timeout fired stays pending, like any
20
21
  # signal no step waits for, except under :notify, whose step still waits
@@ -56,12 +57,12 @@ module Pgbus
56
57
  timeout_at: step.timeout_at&.utc&.iso8601(3) }.compact)
57
58
  notes.instrument("pgbus.workflow.wait_timed_out",
58
59
  Runner.payload_for(record, step_name: step.step_name, attempt: step.attempt, action: action.to_s))
59
- apply(record, step, action, notes: notes, now: now, wait_timeout: definition&.wait_timeout)
60
+ apply(record, step, action, notes: notes, now: now, definition: definition)
60
61
  action
61
62
  end
62
63
 
63
- def apply(record, step, action, notes:, now:, wait_timeout: nil)
64
- return notify!(step, wait_timeout, now: now) if action == :notify
64
+ def apply(record, step, action, notes:, now:, definition: nil)
65
+ return notify!(record, step, definition, notes: notes, now: now) if action == :notify
65
66
  return Waiting.wake!(record, step, now: now, timeout_at: [step.timeout_at, now].compact.min) if action == :run
66
67
 
67
68
  error = action == :pause! ? timed_out_error(step, now) : nil
@@ -69,12 +70,42 @@ module Pgbus
69
70
  end
70
71
 
71
72
  # The run and the step row stay as they are (no outcome, no StepJob);
72
- # only timeout_at moves.
73
- def notify!(step, wait_timeout, now:)
74
- timeout_at = next_deadline(step.timeout_at, wait_timeout, now)
73
+ # only timeout_at moves. A per-run wait_timeout: that raises here, or
74
+ # returns what is no deadline, pauses the run with that error: the
75
+ # sweep holds the run's row lock, under which no on_exception block
76
+ # may run, so the exception policies do not apply.
77
+ def notify!(record, step, definition, notes:, now:)
78
+ timeout_at = notify_deadline(record, step, definition, now)
79
+ rescue StandardError => e
80
+ Waiting.resolve!(record, step, FlowControl::Outcome.pause, notes: notes, error: e)
81
+ else
75
82
  step.timeout_at = timeout_at if WorkflowStep.move_deadline!(step.id, timeout_at: timeout_at).positive?
76
83
  end
77
84
 
85
+ # The next deadline of a :notify wait. A per-run wait_timeout: (issue
86
+ # #574) is evaluated again: a Time after now is the next deadline, a
87
+ # duration moves it one period on as a static one does, and nil or a
88
+ # Time not after now leaves the wait without a deadline (it was
89
+ # reported once and keeps waiting).
90
+ def notify_deadline(record, step, definition, now)
91
+ return next_deadline(step.timeout_at, definition.wait_timeout, now) unless definition.dynamic_wait_timeout?
92
+
93
+ deadline = definition.resolve_wait_timeout(instance_for(record, step))
94
+ case deadline
95
+ when nil then nil
96
+ when Time then deadline if deadline > now
97
+ else next_deadline(step.timeout_at, deadline, now)
98
+ end
99
+ end
100
+
101
+ # The workflow instance a per-run wait_timeout: is evaluated on, as the
102
+ # StepJob builds it for the attempt.
103
+ def instance_for(record, step)
104
+ record.workflow_klass.new(record).tap do |workflow|
105
+ workflow.send(:enter_step, StepContext::Attempt.new(step_id: step.id, step_name: step.step_name, attempt: step.attempt))
106
+ end
107
+ end
108
+
78
109
  # The first deadline after +now+ on the wait's wait_timeout grid: one
79
110
  # wait_timeout on when the sweep is within a period of the deadline
80
111
  # (and for expire_wait ahead of it). A sweep that missed several
@@ -111,12 +111,18 @@ module Pgbus
111
111
  #
112
112
  # SignupWorkflow.start!(user, plan: "pro") # arguments["plan"]
113
113
  # CheckoutWorkflow.start!(order, idempotency_key: "checkout-#{order.id}")
114
+ # ReportWorkflow.start!(actor: "alice", reason: "monthly close")
114
115
  #
115
- # +idempotency_key+ is reserved: with one, a second start! with the same
116
- # key returns the first run (any status) with duplicate_start? true.
117
- # Raises AlreadyRunning when the subject already has an ongoing run.
118
- def start!(subject = nil, idempotency_key: nil, **arguments)
119
- Runner.start(self, subject: subject, arguments: arguments, idempotency_key: idempotency_key)
116
+ # +idempotency_key+, +actor+ and +reason+ are reserved and never reach
117
+ # the arguments. With an idempotency_key, a second start! with the same
118
+ # key returns the first run (any status) with duplicate_start? true and
119
+ # records nothing. +actor+ (default "app") and +reason+ go on the
120
+ # started event, and the actor on the first step_scheduled, as on every
121
+ # outside verb. Raises AlreadyRunning when the subject already has an
122
+ # ongoing run.
123
+ def start!(subject = nil, idempotency_key: nil, actor: nil, reason: nil, **arguments)
124
+ Runner.start(self, subject: subject, arguments: arguments, idempotency_key: idempotency_key,
125
+ actor: actor || Runner::DEFAULT_ACTOR, reason: reason)
120
126
  end
121
127
 
122
128
  # The run with this workflow_id, as an instance of the class that
data/sig/pgbus.rbs CHANGED
@@ -56,6 +56,49 @@ module Pgbus
56
56
  end
57
57
  end
58
58
 
59
+ # Partial: only the readers a workflow step's guards and body use to see
60
+ # its signal (issue #572), and a step definition's wait_timeout: (issue
61
+ # #574). The rest of Workflow is not covered by spec/pgbus/rbs_spec.rb yet.
62
+ class Workflow
63
+ include Instance
64
+
65
+ class Error < Pgbus::Error
66
+ end
67
+
68
+ # A step reader (woken?, signal_ready?, ...) called outside a step.
69
+ class NotInStep < Error
70
+ end
71
+
72
+ module Instance
73
+ # The Pgbus::WorkflowSignal a wait_for: step took, or nil when its wait timed out.
74
+ def received_signal: () -> untyped
75
+ # The body runs because the wait timed out (on_timeout: :run).
76
+ def timed_out?: () -> bool
77
+ # This attempt holds a signal before its gate: signal! woke it, or an
78
+ # earlier attempt took one and reattempted or paused.
79
+ def woken?: () -> bool
80
+ # The gate runs this attempt with a signal: the held one, or a pending
81
+ # one its wait_for: matcher takes.
82
+ def signal_ready?: () -> bool
83
+ end
84
+
85
+ # One declared step (klass.workflow_steps.fetch(:name)): its wait_timeout:.
86
+ class StepDefinition
87
+ # A static wait_timeout: in seconds; nil without one, or for a per-run one.
88
+ def wait_timeout: () -> Numeric?
89
+ # A per-run wait_timeout: as declared: a method name or a lambda.
90
+ def wait_timeout_callable: () -> (Symbol | Proc)?
91
+ # wait_timeout: is a method name or a lambda, evaluated per run.
92
+ def dynamic_wait_timeout?: () -> bool
93
+ # The timeout_at of the step parking at +now+ on +workflow+; nil for no timeout.
94
+ # +now+ and the result are a Time or an ActiveSupport::TimeWithZone (Time.current
95
+ # in an app with a time zone), which sig/ does not declare: untyped.
96
+ def timeout_at_for: (Workflow workflow, untyped now) -> untyped
97
+ # What a per-run wait_timeout: returns on +workflow+: a deadline, seconds from now, or nil.
98
+ def resolve_wait_timeout: (Workflow workflow) -> (Time | Numeric)?
99
+ end
100
+ end
101
+
59
102
  def self.configuration: () -> Configuration
60
103
  def self.configuration_snapshot: () -> Configuration::Snapshot
61
104
  def self.configure: () { (Configuration) -> void } -> Configuration
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pgbus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.20.1
4
+ version: 0.21.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Mikael Henriksson
@@ -588,6 +588,7 @@ files:
588
588
  - lib/pgbus/workflow/removed_step.rb
589
589
  - lib/pgbus/workflow/runner.rb
590
590
  - lib/pgbus/workflow/signal_matcher.rb
591
+ - lib/pgbus/workflow/signal_peek.rb
591
592
  - lib/pgbus/workflow/signals.rb
592
593
  - lib/pgbus/workflow/states.rb
593
594
  - lib/pgbus/workflow/step_context.rb