geneva_drive 0.6.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a4a73216fcf8355550c3b5e26e76eee14468eb301e8eb315a979bc8f34bdb670
-  data.tar.gz: a9b9cc0d8998f6c479806158dd12725774431e7dba8486c5781ed68f238175b5
+  metadata.gz: dd1756c2bb26aeb5fcf15ef77232f06314335ce3427aa6053085476e3cd78042
+  data.tar.gz: 0a445074432320b0040d6e9c69f6b77586cd3275a7bbf034096a5ef0dedc3334
 SHA512:
-  metadata.gz: b77f44d451d256d081af6d7b30d62bcaa46bfd41b0606cdcd1d07529e18db7ff5e2d1753d52f6bd24f600f92b7e14587146072a53e46e1052728c760cf6ded78
-  data.tar.gz: 9bdd9f474cd8370ebcc77b4126617833a249b6e1237ed8021463b66d082db63e17bc024fa8718ec3aee1d031731c72b3a9c90903d2ce40ced2f01eb0b1c421ef
+  metadata.gz: 1113718f861068971a284fcfd54ccabb2b9cdfc3fb7bbff6a75d505f9d5741ba0cdcbe0b3ce62d50b4441d5be8901e505d68bef2e95e197d7c00d7297a27e8b6
+  data.tar.gz: bbc4dc5c73a5c77d19725b60c2951fa1a7d4192cf97739d3cda4f021c4d892e05458df5feb6313372541d1f2a36e840ca51dfe7add125bb0b215cc5d9539392f

data/CHANGELOG.md

@@ -1,6 +1,15 @@
 # Changelog
 
-## [Unreleased]
+## [0.8.0]
+
+- Add `removed_step :name` for taking a step out of a workflow that has instances in flight. Deleting a `step` outright strands every workflow holding a scheduled execution for it: the executor finds no definition and pauses with `StepNotDefinedError`, which during a rolling deploy can hit every workflow sitting on that step at once. The gem cannot recover from this on its own — it knows the step is gone, but not whether the steps after it were ever meant to run, and a step's position says nothing about reachability (a `capture_payment` that normally ends the workflow by calling `finished!` sits directly before `refund`, so skipping forward would refund a payment that was never captured). `removed_step` is how the author of the removal records that judgement in advance: declared in the position the step used to occupy, it keeps the name resolvable so executions scheduled before the removal are skipped and the workflow continues to the next runnable step, while scheduling never spools an execution for the name again and `before_step:`/`after_step:` references keep working. An execution parked on the step waiting for a signal is released on `resume!` rather than waiting for an event that can no longer match. `Workflow.removed_steps_in_flight` reports which gravestones step execution rows still reference (ignoring finished and canceled workflows), so a `removed_step` line can be deleted once it comes back empty. The default for an undeclared removal is unchanged: the workflow still pauses.
+- Add `workflow.resume_at!(:step_name)` for workflows already stranded on a step that was deleted without a `removed_step` gravestone. Plain `resume!` had nowhere to send them and died with a `NoMethodError` on nil deep inside `create_step_execution`; it now raises `StepNotDefinedError` naming the missing step, the steps still defined, and the way out. Where to land is deliberately the operator's call, for the same reason the executor will not guess it.
+- Support MariaDB for the JSON columns (resumable step `cursor`, signal `payload`). MariaDB has no JSON type: `json` there is an alias for LONGTEXT with an auto-generated CHECK constraint named after the column, and since `cursor` is reserved in MariaDB, loading a schema dump that re-emits that constraint died on a syntax error. Rails also types such a column as text, so a Hash written to it was stored as `#inspect` output, which the same CHECK constraint then rejected. The installer migrations now ask for plain LONGTEXT on MariaDB via the new `geneva_drive_json_column` migration helper, and the models encode and decode JSON themselves whenever the column is text. The check is on the column, not the adapter, so installs that already ran the old migration on MariaDB get the encoding without having to rewrite the column.
+- Test against Rails 7.2, 8.0 and 8.1 on SQLite, PostgreSQL, MySQL and MariaDB. Previously the appraisals stopped at Rails 8.0, and the MySQL CI job could not build `mysql2`.
+
+## [0.7.0]
+
+- Add signals: external events delivered to a workflow instance with `workflow.signal!(:payment_confirmed, payload: {...}, idempotency_key: event["id"])`, and a step primitive that parks until one arrives with `step :capture, wait_for: :payment_confirmed`. Signals are rows in a new `geneva_drive_signals` table, persisted before any processing, so arrival order does not matter: a signal that lands before its waiting step exists is buffered and claimed when the step runs, and one that lands after the step parked wakes it immediately. Both sides of the rendezvous run under the workflow row lock, so no signal can slip through the gap, and delivery is one transaction: the insert, the waking of parked executions and every counter bump land together or not at all, so a crashed `signal!` leaves nothing half-delivered (the job enqueue stays after-commit, with the existing stuck-scheduled housekeeping sweep as its backstop). Waiting is a first-class step execution state (`waiting`) that holds no queue slot and no timer — nothing polls. Matching is by name - `wait_for:` also takes a `GenevaDrive::SignalMatcher`, whose optional block narrows by payload and is evaluated on the workflow the signal was sent to (so `hero` is in scope), or any other object responding to `#matches?(signal)`. Matchers are plain reusable objects, so a rule can live in a constant and be shared across steps. Signals move `pending` → `claimed` → `consumed`, consumed only once every attached execution chain has finished cleanly — where skipping counts as finishing, so any flavor of skip (`skip_if:` at wake, `skip!` in a step, an exception policy, or an operator `skip!` on a ready or paused workflow) settles the signal that step was delivered, while skips of steps that never attached leave buffered signals alone — and in the same transaction as the step's own completion, so a reattempt or a `resume!` re-reads the same payload and the audit trail never claims an event was handled when it was not. Two counters on the row, `claimed` and `consumed`, count attachments and clean resolutions (with `claimed?` / `consumed?` reading those counters rather than the lifecycle state). Redelivery is deduplicated by a unique index on `(workflow_id, name, idempotency_key)` — the duplicate call returns the original row flagged `duplicate_delivery?` without dispatching. Payloads go through ActiveJob serializers, come back with indifferent access, and are bounded by `GenevaDrive.max_signal_payload_size` (128 KB, `nil` disables). `signal!` on a finished or canceled workflow raises `GenevaDrive::WorkflowNotOngoing` unless it is a redelivery of a known event; on a paused workflow the row is buffered and delivered by `resume!`. Timeouts are deliberately out of scope for now; waiting is instead made loud through `waiting_since` and two housekeeping gauges (`geneva_drive.waiting_step_executions`, `geneva_drive.waiting_overdue` past `GenevaDrive.waiting_visibility_threshold`, 7 days by default), with `skip!` and `cancel!` as operator escape hatches. Needs one new installer migration (the signals table plus two columns on step executions); until it runs everything else is unaffected and executing a `wait_for:` step raises a configuration error pointing at the missing migration. Test helpers: `assert_waiting_for_signal`, `assert_signal_state`, and the step-driving helpers now raise instead of spinning on a parked execution. Mixing up `wait:` and `wait_for:` now raises `StepConfigurationError` at class load in both directions, which also tightens `wait:` to reject Strings (previously `wait: "later"` silently meant zero seconds).
 
 ## [0.6.0]
 

data/MANUAL.md

@@ -325,6 +325,80 @@ end
 
 The referenced step must already be defined — you can only insert before or after steps that appear earlier in the class body.
 
+### Removing a Step with `removed_step`
+
+Deleting a `step` is not safe while workflows are in flight. Step executions are rows scheduled ahead of time, so at the moment a deploy lands there are workflows holding a row that names a step the new code no longer has. When such a row's job runs, the executor finds no definition and pauses the workflow with `StepNotDefinedError`. During a rolling deploy this can happen to every workflow sitting on that step at once.
+
+GenevaDrive cannot recover from this by itself, and the reason is worth being precise about. It knows the step is gone. It does not know whether the work that step did still needs doing, and — more importantly — it does not know which of the remaining steps would have run had it executed. A step's position in the sequence says nothing about reachability: a `capture_payment` that normally ends the workflow by calling `finished!` sits directly before `refund`, and nothing in the database distinguishes "carry on to the next one" from "stop here". Skipping forward on the workflow's behalf would, in exactly that case, issue a refund for a payment that was never captured. So the executor pauses and waits for a human, which is the only correct thing it can do on its own.
+
+`removed_step` is how you tell it the answer in advance. Declare it in the position the step used to occupy:
+
+```ruby
+class PaymentWorkflow < GenevaDrive::Workflow
+  step :authorize do
+    PaymentGateway.authorize(hero)
+  end
+
+  removed_step :capture_payment
+
+  step :send_receipt do
+    ReceiptMailer.receipt(hero).deliver_later
+  end
+end
+```
+
+The declaration is an assertion only you can make: *this step no longer needs to happen, carry on past it.* Having been told, GenevaDrive can act without guessing:
+
+- Executions already scheduled under the old name resolve, are marked `skipped`, and the workflow proceeds to the next runnable step.
+- Scheduling never spools a new execution for the name again — a new `PaymentWorkflow` goes `authorize` → `send_receipt`.
+- `before_step:` / `after_step:` references to the name keep working.
+- An execution parked on the step waiting for a signal is released on `resume!` instead of waiting for an event that can no longer match.
+
+A removed step takes a name and nothing else. There is nothing to configure about a step that does not run.
+
+Because the gravestone is in the source, in order, the decision is visible in the diff that removes the step — which is where a reviewer can still object to it.
+
+### Retiring the Gravestone
+
+A `removed_step` line is temporary. It can go once no step execution row can land on it any more, which is a question you can ask directly:
+
+```ruby
+PaymentWorkflow.removed_steps_in_flight
+# => {"capture_payment" => 3}   # three rows still point at it — leave the line in place
+# => {}                          # drained — the removed_step line can be deleted
+```
+
+Rows belonging to finished and canceled workflows are ignored, since they will never execute again. The full lifecycle is: replace `step` with `removed_step` and deploy; let in-flight workflows drain past it; delete the line in a later deploy once `removed_steps_in_flight` comes back empty.
+
+### Recovering a Workflow Stranded by a Deleted Step
+
+If a step was deleted without leaving a gravestone, the workflows that were sitting on it are paused and pointed at a name that no longer resolves. Plain `resume!` has nowhere to send them and says so:
+
+```ruby
+workflow.resume!
+# GenevaDrive::StepNotDefinedError: Cannot resume PaymentWorkflow #42: step 'capture_payment'
+# is no longer defined in PaymentWorkflow. Pick the step to continue from with
+# resume_at!(:step_name) — the steps still defined are authorize, send_receipt — or cancel!
+# the workflow. ...
+```
+
+`resume_at!` is the escape hatch. You name the step to continue from, because only you know what the deleted step would have done:
+
+```ruby
+workflow.resume_at!(:send_receipt)
+```
+
+It cancels any stray execution, points the workflow at the named step, and continues from there. It refuses names that are not defined, and refuses to land on a `removed_step`. To find the affected workflows:
+
+```ruby
+GenevaDrive::Workflow.paused
+  .joins(:step_executions)
+  .where(geneva_drive_step_executions: {error_class_name: "GenevaDrive::StepNotDefinedError"})
+  .distinct
+```
+
+If the workflow should not continue at all, `cancel!` it.
+
 ## Conditional Execution
 
 ### Skipping Steps with `skip_if:`
@@ -679,6 +753,179 @@ test "keeps its place across interruptions" do
 end
 ```
 
+## Waiting for Signals
+
+Some steps cannot finish on their own. A payment needs the provider's webhook, a contract needs a countersignature, a shipment needs the carrier to scan the parcel. The usual Rails answer is to poll — schedule the step, check whether the thing happened, `reattempt!(wait: 5.minutes)` if it did not — and that works, but it burns a worker slot on every check and it puts a floor under how fast the workflow can react.
+
+A **signal** inverts that. It is an event record addressed to one workflow instance:
+
+```ruby
+# app/controllers/payment_webhooks_controller.rb
+workflow = OrderFulfillmentWorkflow.ongoing.for_hero(order).first
+workflow.signal!(:payment_confirmed,
+  payload: {amount_cents: event["amount"]},
+  idempotency_key: event["id"])
+```
+
+And a step declares that it waits for one with `wait_for:`:
+
+```ruby
+class OrderFulfillmentWorkflow < GenevaDrive::Workflow
+  step :reserve_stock do
+    hero.reserve_stock!
+  end
+
+  step :capture_payment, wait_for: :payment_confirmed do
+    hero.capture!(received_signal.payload[:amount_cents])
+  end
+
+  step :ship do
+    ShippingLabel.create!(order: hero)
+  end
+end
+```
+
+When `:capture_payment` runs and no matching signal has arrived, its execution parks in the `waiting` state and the job ends. Nothing is enqueued, nothing polls, and the parked row costs nothing until `signal!` wakes it. When the webhook lands, `signal!` wakes the execution and enqueues its job again.
+
+Signals need one migration — it creates the `geneva_drive_signals` table and adds two columns to `geneva_drive_step_executions` — so re-run `bin/rails generate geneva_drive:install` on an existing installation to pick it up. Until then everything else keeps working, and executing a step that declares `wait_for:` fails with a configuration error pointing at the missing migration.
+
+### Arrival Order Does Not Matter
+
+The signal row is written before anything is dispatched, which is what makes the timing irrelevant. There are exactly two places where sender and receiver meet, and both run under the workflow row lock:
+
+- When the waiting step's job runs, it looks for a matching unconsumed signal. Found: attach and run. Not found: park.
+- When `signal!` is called, it looks for parked executions. Found: attach, reschedule, enqueue. Not found: the row simply sits there.
+
+So a webhook that arrives while the workflow is still three steps away from the waiter is not lost and not early — it is buffered, and claimed when the waiting step finally runs. A webhook that arrives a week after the step parked wakes it immediately.
+
+Delivery is one database transaction: the signal row, the waking of the parked execution and the bookkeeping around it either all land or none of them do. If `signal!` raises, nothing was written and the sender can simply try again.
+
+> [!IMPORTANT]
+> `signal!` is an instance method: you find the workflow the way you find anything in Rails, usually `SomeWorkflow.ongoing.for_hero(record).first`. Delivering a brand-new event to a workflow that has already finished or been canceled raises `GenevaDrive::WorkflowNotOngoing` rather than silently doing nothing.
+
+### Deduplicating Redelivery
+
+Webhook providers redeliver. Pass an `idempotency_key:` — the provider's event id is the natural choice — and the second delivery becomes a database-level no-op that returns the row the first one wrote:
+
+```ruby
+signal = workflow.signal!(:payment_confirmed, idempotency_key: event["id"])
+Rails.logger.info("Ignoring redelivered #{event["id"]}") if signal.duplicate_delivery?
+```
+
+Redelivering the very event that finished the workflow is also a no-op, not an error — that specific case is what the deduplication is for. Signals sent without an idempotency key never deduplicate: two calls make two rows, the older one is delivered first, and the newer one stays in the table as an auditable record of what arrived.
+
+### Narrowing What Counts as a Match
+
+`wait_for: :document_signed` is shorthand for `wait_for: GenevaDrive::SignalMatcher.new(:document_signed)`, which matches on name alone. Give the matcher a block and it also has to like the payload:
+
+```ruby
+class ContractWorkflow < GenevaDrive::Workflow
+  # The block receives the payload and runs on the workflow the signal was
+  # sent to, so `hero` and the workflow's own methods are in scope.
+  step :await_countersignature,
+    wait_for: GenevaDrive::SignalMatcher.new(:document_signed) { |payload|
+      payload[:document_id] == hero.contract_id
+    } do
+    hero.mark_countersigned!
+  end
+end
+```
+
+A matcher is a plain object with no ties to the step that uses it, so a rule worth repeating can live in a constant and be shared across steps and workflows:
+
+```ruby
+SETTLED = GenevaDrive::SignalMatcher.new(:payment_confirmed) { |payload| payload[:amount_cents].to_i > 0 }
+
+step :capture_payment, wait_for: SETTLED do
+  hero.capture!
+end
+```
+
+Anything responding to `#matches?(signal)` works too, and owns the whole predicate — which is how a single step waits on either of two names:
+
+```ruby
+class PaymentSettled
+  def initialize(min_cents:) = @min_cents = min_cents
+
+  def matches?(signal)
+    %w[payment_confirmed payment_captured].include?(signal.name) &&
+      signal.payload[:amount_cents].to_i >= @min_cents
+  end
+end
+
+step :capture_payment, wait_for: PaymentSettled.new(min_cents: 100) do
+  hero.capture!
+end
+```
+
+Payloads are serialized with ActiveJob serializers, so `Date`, `Time` and ActiveRecord objects survive the round trip. Hashes come back with indifferent access, because a webhook sender produces string keys and a Ruby caller produces symbols and a matcher should not have to care. The serialized payload is limited to 128 KB (`GenevaDrive.max_signal_payload_size`, `nil` disables) — a payload describes the event; the data your workflow works on belongs on the hero.
+
+> [!WARNING]
+> Matchers run at the gate *and* inside `signal!`, in the sender's process. Keep them cheap and free of side effects, exactly like `skip_if:`. A matcher that raises during `signal!` raises to whoever sent the signal — and because `signal!` is a single transaction, the delivery rolls back whole: no signal row is recorded, nothing is attached or woken, no counter moves. The sender (or the webhook provider's retry) re-delivers once the matcher is fixed.
+
+### How Waiting Composes
+
+- `wait: 2.days, wait_for: :payment_confirmed` means "no earlier than two days from now, and only once signaled". A signal arriving during the delay is buffered and claimed when the step finally runs.
+- `skip_if:` is evaluated before waiting, so "wait for the signature — unless the contract was pre-signed" skips immediately instead of parking forever. Both `skip_if:` and `cancel_if` are re-evaluated when a signal wakes the step; a parked workflow does not re-check them while it sleeps. A step skipped at wake consumes the signal that woke it, so the next waiter does not inherit a stale event.
+- `resumable_step` accepts `wait_for:`. The wait happens once, at the start of the chain, and `received_signal` stays the same across every successor execution.
+- A reattempt — yours or an exception policy's — re-reads the *same* signal. Reattempting means "process this event again", not "wait for another event".
+
+The signal itself moves through `pending` → `claimed` → `consumed`. It is claimed when an execution attaches to it, and consumed once every attached execution has finished cleanly, in the same transaction as the step's own completion. A step that fails, pauses, or gets canceled leaves its signal claimed, so the retry picks up the same payload and the audit trail does not claim an event was handled when it was not.
+
+Skipping counts as finishing: skipping a step that was delivered a signal consumes that signal, whether the skip came from `skip_if:`, from `skip!` inside the step, from an exception policy, or from an operator calling `workflow.skip!` on a ready or paused workflow. Moving past a step deliberately settles the event it was given. Skipping a step that never attached — one still parked, or one whose `skip_if:` fired before it ever looked for a signal — leaves buffered signals exactly as they were.
+
+Two counters on the row make that legible: `signal.claimed` counts attachments (a step that failed and was retried attaches twice), and `signal.consumed` counts cleanly resolved ones. In a linear workflow they end at one apiece. The predicates `signal.claimed?` and `signal.consumed?` read those counters, so they answer "was this event ever picked up" and "was it ever handled" — which is not the same question as `signal.state`, and is usually the one worth asking.
+
+> [!NOTE]
+> Two sequential steps waiting on `:payment_confirmed` need two signals. The first step consumes the first signal, so the second step parks until another one arrives.
+
+### Getting Unstuck
+
+There is no `timeout:` option. Waiting forever is a legitimate state — a contract may genuinely sit unsigned for months — so instead of a timer, waiting is made loud: a distinct `waiting` state, a `waiting_since` timestamp, and two housekeeping gauges, `geneva_drive.waiting_step_executions` and `geneva_drive.waiting_overdue` (parked longer than `GenevaDrive.waiting_visibility_threshold`, seven days by default). Alert on the overdue gauge.
+
+When something does stall, three escape hatches already exist:
+
+- `workflow.skip!` abandons the wait and moves to the next step.
+- `workflow.cancel!` gives up on the workflow.
+- The queue is already a clock: schedule a job that signals a deadline, and let the step decide.
+
+```ruby
+# Send the workflow its own deadline; the step reads it off the payload
+class ContractDeadlineJob < ApplicationJob
+  def perform(contract)
+    workflow = ContractWorkflow.ongoing.for_hero(contract).first
+    workflow&.signal!(:document_signed,
+      payload: {timed_out: true},
+      idempotency_key: "deadline-#{workflow.id}")
+  end
+end
+
+step :await_countersignature, wait_for: :document_signed do
+  cancel! if received_signal.payload[:timed_out]
+  hero.mark_countersigned!
+end
+```
+
+### Testing Waiting Steps
+
+The step-driving helpers refuse to spin on a parked execution — they raise and tell you which signal is missing, because a test that hangs on a rendezvous is far worse than one that fails:
+
+```ruby
+test "captures once the payment is confirmed" do
+  workflow = OrderFulfillmentWorkflow.create!(hero: order)
+  perform_next_step(workflow)                    # :reserve_stock
+  perform_next_step(workflow)                    # :capture_payment parks
+
+  assert_waiting_for_signal(workflow, :payment_confirmed)
+
+  workflow.signal!(:payment_confirmed, payload: {amount_cents: 12_500})
+  speedrun_workflow(workflow)
+
+  assert_signal_state(workflow, :payment_confirmed, :consumed)
+  assert workflow.finished?
+end
+```
+
 ## Exception Handling
 
 ### Default Behavior
@@ -1054,8 +1301,13 @@ end
 # Resume a specific workflow
 workflow = GenevaDrive::Workflow.find(id)
 workflow.resume!  # Re-enqueues existing scheduled step or creates new one
+
+# Or continue from a specific step, when the step it was pointed at is gone
+workflow.resume_at!(:send_receipt)
 ```
 
+`resume_at!` exists for workflows paused on a step that no longer exists in the code. See [Recovering a Workflow Stranded by a Deleted Step](#recovering-a-workflow-stranded-by-a-deleted-step).
+
 ## Workflow States
 
 ### State Machine Diagram
@@ -1080,6 +1332,7 @@ Step executions have their own state machine:
 | State | Meaning |
 |-------|---------|
 | `scheduled` | Waiting to run |
+| `waiting` | Parked until a matching signal arrives |
 | `in_progress` | Currently executing |
 | `completed` | Finished successfully |
 | `failed` | Exception occurred |
@@ -1783,7 +2036,11 @@ The workflow will sit in `paused` state — holding the slot, preventing duplica
 
 ### Pause on missing step definitions
 
-There is one more scenario where pause happens automatically. If a workflow references a step name that no longer exists in the class definition — typically after a deploy that removed or renamed a step — the executor pauses the workflow rather than silently skipping or crashing. This gives operators a chance to notice the mismatch and decide whether to skip, cancel, or deploy a fix.
+There is one more scenario where pause happens automatically. If a workflow references a step name that no longer exists in the class definition — typically after a deploy that removed or renamed a step — the executor pauses the workflow rather than silently skipping or crashing.
+
+This is not a limitation to be worked around. The executor knows the step is gone but not whether the steps after it were ever meant to run, and guessing wrong means performing work the missing step would have prevented. Pausing hands that judgement to someone who has it.
+
+Declare `removed_step :name` when you take a step out and the judgement is recorded ahead of time, so nothing pauses; see [Removing a Step with `removed_step`](#removing-a-step-with-removed_step). For workflows already stranded by a step deleted without one, `resume_at!` is the way back — see [Recovering a Workflow Stranded by a Deleted Step](#recovering-a-workflow-stranded-by-a-deleted-step).
 
 ## Complete Example Workflows
 
@@ -1980,6 +2237,7 @@ end
 | State | Meaning |
 |-------|---------|
 | `scheduled` | Waiting to run |
+| `waiting` | Parked until a matching signal arrives |
 | `in_progress` | Currently executing |
 | `completed` | Finished successfully |
 | `failed` | Exception occurred |
@@ -1993,6 +2251,7 @@ A resumable step interrupted mid-iteration completes its execution with outcome
 | Option | Type | Description |
 |--------|------|-------------|
 | `wait:` | Duration | Delay before step executes |
+| `wait_for:` | Symbol, String, `SignalMatcher`, matcher object | Park until a matching signal arrives |
 | `job_options:` | Hash | Options passed to Active Job's `set` method for this step |
 | `skip_if:` | Proc, Symbol, Boolean | Condition to skip step |
 | `on_exception:` | Symbol | Exception handler (`:pause!`, `:cancel!`, `:reattempt!`, `:skip!`) |

data/README.md

@@ -8,6 +8,8 @@ GenevaDrive provides a clean DSL for defining multi-step workflows that execute
 
 - **Hero-oriented**: Workflows are associated with a polymorphic "hero" (the subject of the workflow)
 - **Step-based execution**: Define workflows as a series of steps with optional wait times
+- **Resumable steps**: Iterate over large collections with a database-checkpointed cursor, surviving interruptions
+- **Signals**: Steps can park until an external event arrives (`wait_for:`), woken by `workflow.signal!` from a webhook or controller
 - **Durable**: Steps are persisted to the database, surviving process restarts
 - **Idempotent**: Database constraints ensure each step runs exactly once
 - **Flow control**: Methods like `cancel!`, `pause!`, `skip!`, `reattempt!`, and `finished!`

data/Rakefile

@@ -1,7 +1,6 @@
 require "bundler/setup"
 require "bundler/gem_tasks"
 
-require "rake/testtask"
 require "standard/rake"
 require "yard"
 
@@ -15,3 +14,17 @@ end
 
 APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
 load "rails/tasks/engine.rake"
+
+# Rails' own plugin test runner rather than a Rake::TestTask: it is what sets up
+# and tears down the per-worker databases that ActiveSupport::TestCase.parallelize
+# creates, and a plain Rake::TestTask leaves those workers hanging at exit.
+desc "Run the test suite"
+task :test do
+  ruby "-Itest", "bin/test"
+end
+
+# The root Gemfile resolves to the newest released Rails and the dummy app
+# defaults to SQLite, so a plain `rake` is the "latest Rails on SQLite" run.
+# The appraisal gemfiles cover the older Rails minors, and CI additionally
+# points DATABASE_URL at PostgreSQL and MySQL.
+task default: [:standard, :test]

data/bin/matrix

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+# Local CI: run the suite for every Rails appraisal against every database
+# engine the CI matrix covers, so that a one-engine failure can be reproduced
+# without pushing. Brings compose.yaml up first and leaves it running.
+#
+#   bin/matrix                    # the whole matrix
+#   bin/matrix rails_8_1          # one Rails version, all engines
+#   bin/matrix rails_8_1 sqlite   # one cell
+set -uo pipefail
+cd "$(dirname "$0")/.."
+
+# The gemfiles are generated from Appraisals, which makes them the one place
+# the supported Rails versions are written down. CI reads the same list off
+# them, so neither side can drift from the other.
+RAILS_VERSIONS=${1:-$(ls gemfiles/*.gemfile | xargs -n1 basename | sed 's/\.gemfile$//' | tr '\n' ' ')}
+ENGINES=${2:-"sqlite postgres mysql mariadb"}
+
+for engine in $ENGINES; do
+  case "$engine" in
+    sqlite|postgres|mysql|mariadb) ;;
+    *) echo "Unknown engine: $engine (pick from sqlite, postgres, mysql, mariadb)" >&2; exit 1 ;;
+  esac
+done
+
+# Only the engines actually asked for - CI runs one cell per job and should not
+# be pulling a MySQL image to test against PostgreSQL.
+services=""
+for engine in $ENGINES; do
+  case "$engine" in
+    postgres|mysql|mariadb) services="$services $engine" ;;
+  esac
+done
+if [ -n "$services" ]; then
+  docker compose up -d --wait $services || exit 1
+fi
+
+# Compose publishes to an ephemeral host port, so ask it which one it got.
+port_of() {
+  docker compose port "$1" "$2" | sed 's/.*://'
+}
+
+# SQLite gets no DATABASE_URL, exactly as in CI: the dummy app defaults to it
+# and resolves its own path relative to Rails.root.
+url_for() {
+  case "$1" in
+    postgres) echo "postgresql://postgres:postgres@127.0.0.1:$(port_of postgres 5432)/geneva_drive_test" ;;
+    mysql)    echo "mysql2://root:root@127.0.0.1:$(port_of mysql 3306)/geneva_drive_test?encoding=utf8mb4" ;;
+    mariadb)  echo "mysql2://root:root@127.0.0.1:$(port_of mariadb 3306)/geneva_drive_test?encoding=utf8mb4" ;;
+  esac
+}
+
+failed=0
+results=()
+for rails in $RAILS_VERSIONS; do
+  gemfile="gemfiles/$rails.gemfile"
+  [ -f "$gemfile" ] || { echo "No such appraisal: $gemfile" >&2; exit 1; }
+  BUNDLE_GEMFILE=$gemfile bundle check >/dev/null 2>&1 || BUNDLE_GEMFILE=$gemfile bundle install || exit 1
+  for engine in $ENGINES; do
+    echo "==> $rails / $engine"
+    if [ "$engine" = sqlite ]; then
+      env -u DATABASE_URL BUNDLE_GEMFILE=$gemfile RAILS_ENV=test GENEVA_DRIVE_ENGINE=$engine bundle exec rake test
+    else
+      env BUNDLE_GEMFILE=$gemfile RAILS_ENV=test GENEVA_DRIVE_ENGINE=$engine DATABASE_URL="$(url_for "$engine")" bundle exec rake test
+    fi
+    if [ $? -eq 0 ]; then
+      results+=("ok    $rails / $engine")
+    else
+      results+=("FAIL  $rails / $engine")
+      failed=1
+    fi
+  done
+done
+
+printf '\n'
+printf '%s\n' "${results[@]}"
+exit $failed

data/bin/md2html

@@ -42,6 +42,98 @@ class SyntaxHighlightingHTML < Redcarpet::Render::HTML
   end
 end
 
+# GitHub-Flavored Markdown alerts: `> [!NOTE]` etc. Redcarpet emits these
+# as ordinary blockquotes, so we post-process the HTML to spot the marker,
+# strip it, and rewrite the blockquote with an alert class + title.
+# GitHub knows exactly these five and matches them case-sensitively —
+# `[!note]` or `[!FOO]` stay literal text there, and so must they here.
+GFM_ALERT_TYPES = %w[NOTE TIP IMPORTANT WARNING CAUTION].freeze
+# Only spaces/tabs and one line ending may follow the marker. `\s*` would
+# swallow the blank line separating the marker from a following paragraph.
+GFM_ALERT_PATTERN = /\A\[!(#{GFM_ALERT_TYPES.join("|")})\][ \t]*\r?\n?/
+
+# The marker opens the text of a paragraph. GitHub insists it sit alone on the
+# blockquote's first line; we are laxer and also accept body text trailing it
+# on the same line, rather than leaving a half-parsed `[!NOTE]` on the page.
+def alert_marker_of(child)
+  return nil unless child.element? && child.name == "p"
+  first_text = child.children.first
+  return nil unless first_text&.text?
+  match = first_text.content.match(GFM_ALERT_PATTERN)
+  match && match[1]
+end
+
+# Take the marker out of the paragraph it introduces and keep everything else,
+# element children included — that is the whole point: an alert whose body
+# opens with `**bold**` has the marker and a <strong> in the same <p>, so we
+# may only edit that leading text node, never replace the paragraph.
+def strip_alert_marker!(paragraph)
+  first_text = paragraph.children.first
+  first_text.content = first_text.content.sub(GFM_ALERT_PATTERN, "") if first_text&.text?
+
+  # A marker line ending in two spaces is a Markdown hard break, so Redcarpet
+  # leaves a <br> where the marker was. Drop that leading padding, otherwise
+  # the alert body opens with a blank line.
+  while (node = paragraph.children.first)
+    blank_text = node.text? && node.content.strip.empty?
+    break unless blank_text || (node.element? && node.name == "br")
+    node.remove
+  end
+
+  # The marker was alone on its line: the paragraph has nothing left to say.
+  paragraph.remove if paragraph.children.empty?
+end
+
+def transform_gfm_alerts(html_body)
+  doc = Nokogiri::HTML::DocumentFragment.parse(html_body)
+  doc.css("blockquote").each do |bq|
+    # Redcarpet collapses adjacent blockquotes into one, so a single
+    # <blockquote> may carry several `[!TYPE]` markers. Partition its
+    # children into segments, each introduced by a marker (or the
+    # untagged preamble before any marker).
+    segments = [[nil, []]]
+    bq.children.to_a.each do |child|
+      marker = alert_marker_of(child)
+      if marker
+        segments << [marker, [child]]
+      else
+        segments.last[1] << child
+      end
+    end
+    segments.reject! do |_, nodes|
+      nodes.all? { |n| n.text? && n.content.strip.empty? }
+    end
+
+    # Nothing to do if no markers appeared at all.
+    next if segments.all? { |type, _| type.nil? }
+
+    prev = bq
+    segments.each_with_index do |(type, children), i|
+      new_bq = Nokogiri::XML::Node.new("blockquote", bq.document)
+      children.each { |c| new_bq.add_child(c) }
+
+      if type
+        first_p = new_bq.children.find(&:element?)
+        strip_alert_marker!(first_p) if first_p && first_p.name == "p"
+
+        title = Nokogiri::XML::Node.new("p", bq.document)
+        title["class"] = "markdown-alert-title"
+        title.content = type.capitalize
+        new_bq.prepend_child(title)
+        new_bq["class"] = "markdown-alert markdown-alert-#{type.downcase}"
+      end
+
+      if i == 0
+        bq.replace(new_bq)
+      else
+        prev.add_next_sibling(new_bq)
+      end
+      prev = new_bq
+    end
+  end
+  doc.to_html
+end
+
 def build_toc(html_body)
   doc = Nokogiri::HTML::DocumentFragment.parse(html_body)
   headings = doc.css("h1, h2")
@@ -82,6 +174,11 @@ def html_template(title, content, toc: "")
       --table-border: #dfe2e5;
       --blockquote-color: #6a737d;
       --blockquote-border: #dfe2e5;
+      --alert-note: #0969da;
+      --alert-tip: #1a7f37;
+      --alert-important: #8250df;
+      --alert-warning: #9a6700;
+      --alert-caution: #cf222e;
       --toc-width: 280px;
     }
 
@@ -247,6 +344,22 @@ def html_template(title, content, toc: "")
       margin-bottom: 0;
     }
 
+    /* GitHub-Flavored Markdown alerts (> [!NOTE] etc.) */
+    .markdown-alert {
+      color: var(--text-color);
+      padding: 0.5em 1em;
+    }
+    .markdown-alert-title {
+      font-weight: 600;
+      margin: 0 0 0.25em;
+      color: var(--alert-accent, inherit);
+    }
+    .markdown-alert-note      { border-left-color: var(--alert-note);      --alert-accent: var(--alert-note); }
+    .markdown-alert-tip       { border-left-color: var(--alert-tip);       --alert-accent: var(--alert-tip); }
+    .markdown-alert-important { border-left-color: var(--alert-important); --alert-accent: var(--alert-important); }
+    .markdown-alert-warning   { border-left-color: var(--alert-warning);   --alert-accent: var(--alert-warning); }
+    .markdown-alert-caution   { border-left-color: var(--alert-caution);   --alert-accent: var(--alert-caution); }
+
     /* Horizontal rules */
     hr {
       border: 0;
@@ -444,6 +557,7 @@ def convert_markdown_to_html(input_path, output_path = nil, toc: true)
   })
 
   html_body = markdown.render(markdown_content)
+  html_body = transform_gfm_alerts(html_body)
   toc_html = toc ? build_toc(html_body) : ""
   html_document = html_template(title, html_body, toc: toc_html)
 

data/lib/generators/geneva_drive/install/install_generator.rb

@@ -65,6 +65,11 @@ module GenevaDrive
           "add_resumable_step_support.rb",
           "db/migrate/add_resumable_step_support_to_geneva_drive_step_executions.rb"
         )
+
+        migration_template(
+          "add_signals_support.rb",
+          "db/migrate/add_signals_support_to_geneva_drive.rb"
+        )
       end
 
       # Creates the initializer file.

data/lib/generators/geneva_drive/install/templates/add_resumable_step_support.rb

@@ -5,15 +5,11 @@ class AddResumableStepSupportToGenevaDriveStepExecutions < ActiveRecord::Migrati
 
   def change
     unless column_exists?(:geneva_drive_step_executions, :cursor)
-      # Cursor for resumable steps. Use database-native JSON type:
-      # - PostgreSQL: jsonb (indexed, efficient, supports containment queries)
-      # - MySQL 5.7+: json (native validation and storage)
-      # - SQLite: json (Rails handles as TEXT with serialization)
-      if connection.adapter_name.downcase.include?("postgresql")
-        add_column :geneva_drive_step_executions, :cursor, :jsonb
-      else
-        add_column :geneva_drive_step_executions, :cursor, :json
-      end
+      # Cursor for resumable steps. Database-native JSON where there is one -
+      # jsonb on PostgreSQL, json on MySQL and SQLite - and plain LONGTEXT on
+      # MariaDB, which has no JSON type. See geneva_drive_json_column.
+      cursor_type, cursor_options = geneva_drive_json_column
+      add_column :geneva_drive_step_executions, :cursor, cursor_type, **cursor_options
     end
 
     unless column_exists?(:geneva_drive_step_executions, :continues_from_id)