@pikku/core 0.12.72 → 0.12.74

package/knowledge/decisions/internals/workflow-queues-are-per-workflow-by-default.md

@@ -0,0 +1,42 @@
+---
+type: decision
+title: Workflows get their own queues by default, and queue names are resolved from queue meta
+description: Per-workflow queues stop one slow step head-of-line-blocking every other workflow; `shared-groups` trades that for one set of pollers
+tags: workflow
+---
+
+# Workflows get their own queues by default, and queue names are resolved from queue meta
+
+`WorkflowQueueOptions.queueStrategy` defaults to `'per-workflow'`: each workflow
+gets its own `wf-orchestrator-*` and `wf-step-*` queue. That gives complete
+isolation and is what lets serverless providers deploy one unit per workflow,
+at the cost of one set of pollers per queue — which adds up on pull-based
+backends. `'shared-groups'` puts every workflow on the shared orchestrator and
+step-worker queues and keeps them isolated with a per-group concurrency cap, so
+there is one set of pollers for the whole system. It is only for single-process
+runtimes; a per-unit serverless deploy needs the per-workflow queues to route to
+its units. The options are passed to the service constructor rather than read
+from `config.workflow` because the queues are wired during construction, before
+singleton services — and so before `config` — exist.
+
+Under `'shared-groups'`, `getJobGroup` returns a group keyed by workflow name
+(steps group by step function, mirroring how per-step queues split them). It
+returns `undefined` under `'per-workflow'`, because the queue name already
+isolates the workflow and a group would cap it inside its own dedicated queue.
+The group tier repeats the id so a workflow can be given its own limit purely
+from config; note tiers can only lower a limit, never raise it above `default`,
+because the backend applies its pre-fetch exclusion per group using `default`
+alone.
+
+`getOrchestratorQueueName` and `getStepWorkerQueueName` read `queue.meta`, which
+is always populated globally, rather than `queue.registrations`, which only
+covers queues this unit consumes. In a per-unit deploy the orchestrator unit
+produces to the per-step queues without consuming them, so registrations would
+miss them. `wireQueueWorkers` warns loudly when workflows exist but no
+`wf-orchestrator-*` queue is in meta: everything still "works" via the shared
+fallback, but the isolation is gone and one slow step head-of-line-blocks every
+other workflow — invisible until a queue starves.
+
+**What this rules out:** defaulting to `'shared-groups'`, returning a job group
+under `'per-workflow'`, resolving queue names from `queue.registrations`, or
+dropping the wiring-time warning as noise.

package/knowledge/decisions/internals/workflow-repeated-step-names-get-an-ordinal-suffix.md

@@ -0,0 +1,33 @@
+---
+type: decision
+title: A workflow step name repeated in one run gets an ordinal suffix, and the first reach stays bare
+description: `name`, `name#1`, `name#2` keys each reach separately without changing the durable key of any existing run
+tags: workflow
+---
+
+# A workflow step name repeated in one run gets an ordinal suffix, and the first reach stays bare
+
+`nextStepKey` in `pikku-workflow-service.ts` turns a logical step name into the
+physical, replay-stable key for the Nth time the run reaches it: the bare name
+for ordinal 0, `name#N` for repeats. Keeping the first reach unsuffixed is what
+makes the scheme backward compatible — existing runs and their persisted step
+rows are untouched — while still letting the same literal step name be invoked
+several times (a loop, a helper called twice) without the rows clobbering each
+other.
+
+The counters live in the per-replay `RunContext.replay.ordinals` and are reset
+by `beginReplay` on every orchestrator tick. That reset is what makes the keys
+deterministic: given a deterministic workflow body, the second replay assigns
+exactly the same key to the same reach and therefore finds the cached row rather
+than minting `name#1` for a step that already succeeded. `nextStepKey` also
+records the key it just produced as `replay.lastStep`, which is how the next
+step learns its predecessor (`fromStepName`).
+
+Graph runs use the same convention for cycle revisits, and
+`stripInstanceOrdinal` / `executeWorkflowStep` map `node#N` back to the logical
+node id, which is not a literal key in the graph's `nodes` map.
+
+**What this rules out:** suffixing the first reach too (it re-keys every step of
+every existing run), advancing the ordinal counters anywhere other than a fresh
+replay pass, deriving step keys from a non-deterministic source, or treating a
+physical step key as a graph node id without stripping the ordinal.

package/knowledge/decisions/internals/workflow-replay-reads-its-steps-once-and-caches-only-the-immutable-half.md

@@ -0,0 +1,32 @@
+---
+type: decision
+title: A workflow replay reads its steps once and caches only the run's immutable half
+description: The per-replay snapshot collapses O(N^2) step reads to one, but caching mutable run fields would make the replay read a lie
+tags: workflow
+---
+
+# A workflow replay reads its steps once and caches only the run's immutable half
+
+A DSL replay walks the workflow body from the top, and each step it passes asks
+for its own row — so a run of N steps costs N reads per replay and O(N^2) over
+its lifetime. `beginReplay` in `pikku-workflow-service.ts` opens a pass by
+calling `listStepStates`, which backends able to answer in a single query
+override; `loadOrCreateStep` then serves every step from that snapshot. This is
+safe because a pass reaches each step key at most once and the steps it replays
+past are `succeeded`, and therefore immutable.
+
+`getRunIdentity` caches the run for the same pass, but only for its immutable
+half — which workflow it is, the wire it was started on, its input. Anything
+needing `status`, `output`, `error` or `state` must call `getRun`: those move
+while the run executes and a cached copy would be a lie. The `replay` half of
+`RunContext` is rebuilt from scratch on every orchestrator tick and torn down by
+`endReplay`.
+
+`loadOrCreateStep` also handles a concurrent replay creating the row after the
+snapshot was taken: if `create()` throws, it re-reads the step, and only if that
+read also fails does it rethrow the original — because in that case the insert
+failed for its own reasons and that error is the one worth seeing.
+
+**What this rules out:** widening the snapshot to cover mutable run fields,
+keeping it across orchestrator ticks, or removing the re-read fallback in
+`loadOrCreateStep` on the grounds that a duplicate insert "cannot happen".

package/knowledge/decisions/internals/workflow-retries-are-owned-by-the-workflow-not-the-queue.md

@@ -0,0 +1,31 @@
+---
+type: decision
+title: Workflow step retries are owned by the workflow, never by the queue
+description: A step's retry count is resolved once and always passed to the queue as `attempts`, so the queue can never apply its own default
+tags: workflow
+---
+
+# Workflow step retries are owned by the workflow, never by the queue
+
+A step's retry policy comes from the step itself, falling back to
+`DEFAULT_STEP_RETRIES` (5) when unset. The default is deliberately greater than
+zero so a transient failure — a DB blip, a downstream restart, a deploy — is
+ridden out without the author asking for it; that is safe only because every
+step gets a stable `invocationId` to dedupe on. An explicit `retries: 0` means
+exactly once and must be honoured.
+
+`resolveStepJobOptions` in `pikku-workflow-service.ts` therefore ALWAYS emits
+`attempts`, even when it is 1. Backoff defaults to exponential whenever at least
+one retry remains, so retries ride out an outage instead of firing instantly; a
+concrete `retryDelay` (`15000`, `'15s'`) selects fixed backoff, and only the
+literal `'exponential'` selects exponential explicitly. `rpcStep` resolves the
+policy once, before both persistence and dispatch, so the value stored on the
+step row — which drives `retriesExhausted` — is the same number the queue turns
+into `attempts`. `queueGraphNode` in `graph/graph-runner.ts` defaults node
+retries to `DEFAULT_STEP_RETRIES` for the same reason.
+
+**What this rules out:** dropping `attempts` when it equals 1 "because that is
+the default anyway", resolving retries separately at the persistence and
+dispatch sites, or letting the queue adapter's own `retry_limit` decide. Any of
+those lets the queue re-run a step the workflow said to run once, or lets the
+engine believe retries are exhausted while the queue keeps retrying.

package/knowledge/decisions/internals/workflow-run-capabilities-are-extensions-not-subclasses.md

@@ -0,0 +1,39 @@
+---
+type: decision
+title: Workflow run capabilities are extensions, not subclasses
+description: Scenario support lives in a separate module behind `setRunExtension` because a bundler drops an unused module but never an unused class member
+tags: workflow
+---
+
+# Workflow run capabilities are extensions, not subclasses
+
+`PikkuWorkflowService` names nothing about what a `WorkflowRunExtension` is for.
+An extension is installed with `setRunExtension(create)` and receives a narrow
+`WorkflowRunEngine` handle — `inlineStep`, `updateRunStatus`,
+`onChildWorkflowFailed`, `verifyStepName` — so recording a durable step stays
+available to it without becoming public API on every workflow service a
+production app instantiates.
+
+The reason it is not a subclass is bundle size: a bundler drops an unused
+*module* but never an unused class member. Anything declared on
+`PikkuWorkflowService` ships in every server built on Pikku, along with
+everything it imports. `PikkuScenarioService` (`pikku-scenario-service.ts`) is
+the one implementation today — steps, actors, lifecycle hooks, the browser
+provider and the assertion wire members — and scenarios only ever run from
+`pikku scenario run`, so the whole surface stays behind an import that only the
+runner makes. It is not a workflow service in its own right because a scenario
+is not a different kind of run; it is the same durable run with a step
+vocabulary on top. `scenario-service.test.ts` asserts that none of those members
+leak onto the base class.
+
+`createScenarioRunner` in the same file bundles the two lines
+`pikku scenario run` needs so no caller has to remember the capability is
+installed rather than inherited. It pairs the extension with
+`InMemoryWorkflowService`, which is the right engine because a scenario run is a
+single external process driving a deployed app over its real transport: there
+is nothing to persist and no second worker to resume it.
+
+**What this rules out:** moving scenario steps, actors, hooks or the browser
+provider onto `PikkuWorkflowService`, making `PikkuScenarioService` extend it,
+or widening `WorkflowRunEngine` into "just pass the service" — each one puts the
+scenario runtime, and its transitive imports, into every deployed Pikku server.

package/knowledge/decisions/internals/workflow-run-mirror-is-never-a-source-of-truth.md

@@ -0,0 +1,29 @@
+---
+type: decision
+title: The workflow run mirror is an observability sink, never a second source of truth
+description: Every mirrored write happens after the authoritative write lands, and a mirror failure can never fail the workflow
+tags: workflow
+---
+
+# The workflow run mirror is an observability sink, never a second source of truth
+
+`WorkflowRunMirror` lets an executor shadow every state write to an external
+read store — typically a DB the console UI queries while the run itself lives in
+a Durable Object or Redis. Both properties that make it safe follow from the
+single shape of `PikkuWorkflowService.mirrored()` in
+`pikku-workflow-service.ts`: the mirror is only ever told about a write that has
+already landed in the canonical store, and a mirror that is down or throwing is
+caught, logged at warn, and otherwise invisible to the workflow. Even the log
+call is wrapped, because singleton services may not be initialised yet.
+
+Every public write method is a thin `mirrored(() => …Impl(...), (mirror) => …)`
+pair over a `protected abstract` implementation. That pairing is the reason a
+mirror method added to the interface but never wired into the service would
+write nothing at runtime — `workflow-mirror.test.ts` spells out the full method
+list to catch exactly that.
+
+**What this rules out:** reading run or step state back from the mirror,
+awaiting the mirror before returning the authoritative result, letting a mirror
+rejection propagate, or writing to the mirror first so the two "stay in sync" —
+any of which turns an index into a second, divergent source of truth that can
+take the workflow down with it.

package/knowledge/decisions/internals/workflow-run-polling-backs-off-to-the-callers-ceiling.md

@@ -0,0 +1,33 @@
+---
+type: decision
+title: Workflow run polling starts short and backs off to the caller's ceiling
+description: `pollIntervalMs` is a ceiling, not a cadence, and the wait lives in its own method so the schedule can be asserted without the clock
+tags: workflow
+---
+
+# Workflow run polling starts short and backs off to the caller's ceiling
+
+`awaitRunEnd` in `pikku-workflow-service.ts` reads a run until it reaches an end
+state, starting at `WORKFLOW_POLL_MIN_MS` (10ms) and multiplying by
+`WORKFLOW_POLL_FACTOR` (1.6) up to the caller's `maxIntervalMs`. A fixed
+interval is wrong at both ends: it makes a workflow that finished in
+milliseconds wait out the whole interval anyway, and it keeps reading a
+long-running one at full rate for its entire life. Backing off returns quick
+runs promptly while a slow run's read cost grows logarithmically rather than
+linearly with duration. So `runToCompletion`'s `pollIntervalMs` is a ceiling,
+not a cadence.
+
+An inline sub-workflow uses a lower ceiling, `WORKFLOW_CHILD_POLL_MAX_MS`
+(500ms), because the parent step is blocked on it: every wait there is added
+latency in the middle of a workflow rather than at its edge.
+
+`waitBeforeNextRead` exists as its own method purely so the schedule can be
+asserted directly (`workflow-run-polling.test.ts`). Timing a poll loop by the
+clock measures the host's scheduler as much as the policy — `setTimeout(40)`
+routinely returns late on a loaded CI runner — which makes the obvious test both
+slow and flaky.
+
+**What this rules out:** replacing the backoff with a fixed `pollIntervalMs`
+sleep, reading `pollIntervalMs` as the first wait, giving child runs the
+top-level ceiling, or inlining `waitBeforeNextRead` back into the loop as a bare
+`setTimeout`.

package/knowledge/decisions/internals/workflow-run-timeline-is-a-pure-fold-over-durable-history.md

@@ -0,0 +1,37 @@
+---
+type: decision
+title: The workflow run timeline is a pure fold over durable history, with the row's status as the authority
+description: No IO in the fold keeps time-travel transport-independent; the terminal event comes from `status`, not from a timestamp every backend populates
+tags: workflow
+---
+
+# The workflow run timeline is a pure fold over durable history, with the row's status as the authority
+
+`run-timeline.ts` turns `getRunHistory` — one row per step *attempt*, each with
+lifecycle timestamps — into a flat, chronologically ordered event stream, and
+folds that stream up to any point to recover what the run "knew" then: the same
+step cache a replay would hold, plus the walked path. Both functions are pure,
+with no IO, so they are trivially testable and the same fold works for the
+Redis, Kysely and in-memory stores alike.
+
+The terminal event is driven by the row's authoritative `status`, falling back
+to `updatedAt` when the specific stamp is absent, because the lifecycle
+timestamps are not populated by every backend — Kysely leaves
+`succeededAt`/`runningAt` null. The intermediate `scheduled` and `running`
+events are optional enrichment, emitted only when the backend recorded them; the
+`pending` event always exists and is the one that carries provenance
+(`fromStepName`).
+
+Ordering is defensive: events sort by timestamp, then by `LIFECYCLE_ORDER`, then
+by original index, so rows sharing an instant stay deterministic across
+backends. In the fold, a retry's `pending` event reopens the step and drops the
+prior result and error.
+
+The index conventions are fixed and callers depend on them: `TimelineEvent.seq`
+is a monotonic 0-based position, `attempt` is 1-based, and
+`reconstructStateAt(at)` is inclusive whether `at` is a seq or a `Date` — a
+point before the first event yields the empty initial state with `seq` of `-1`.
+
+**What this rules out:** reading state inside the fold, deriving the terminal
+event from `succeededAt`/`failedAt` alone, trusting history to arrive sorted, or
+letting a retry's reopening event keep the previous attempt's outcome.

package/knowledge/decisions/internals/workflow-scenario-assertions-never-retry-and-record-one-step.md

@@ -0,0 +1,50 @@
+---
+type: decision
+title: Scenario steps default to no retries, and a whole poll is one durable step
+description: Retrying a failed assertion is wrong for a test primitive; recording the poll as one step means replay returns the outcome, not the loop
+tags: workflow
+---
+
+# Scenario steps default to no retries, and a whole poll is one durable step
+
+`ScenarioStepOptions.retries` defaults to 0, unlike an ordinary workflow step
+which inherits `DEFAULT_STEP_RETRIES`. `PikkuScenarioService.scenarioStep`
+applies that default explicitly (`options?.retries ?? 0`). Retrying a failed
+assertion is the wrong behaviour for a test primitive: it converts a real
+failure into a slow flake and hides the very race the scenario was written to
+catch.
+
+`expectEventually` is the sanctioned way to wait. The entire poll — invoke,
+check the predicate, sleep, repeat until `within` elapses — runs inside a single
+`inlineStep`, so it is ONE recorded step and a replay returns the cached
+outcome rather than re-polling. `expectError` and `expectService` are the same
+shape: one durable step whose body contains the whole assertion, with a failure
+message that names the step, the target, and what was actually seen.
+
+`pollUntil` in `scenario-poll.ts` is the step-level equivalent, and its
+`undefined` convention is deliberate: `undefined` means "not yet" and nothing
+else, because `false`, `0` and `''` are all real answers — a probe asking
+whether something happened reports `false` when it did not. It returns
+`undefined` at the deadline rather than throwing, leaving the error to the
+caller, who is the only one who knows what was being waited for. Its own
+defaults are shorter than the assertion wrappers': `within` 15s, `interval`
+250ms.
+
+`requireActor` and `requireScenarioEnv` (`scenario-step-guards.ts`) exist for
+the mirror-image reason: `actor` and `env` are optional on the step wire because
+a pure assertion step needs neither, so the narrowing happens in one place with
+a message that says what to do, instead of every step file writing its own
+guard. `scenarioStep`'s `verifyStepName` call doubles as the guard for `then`
+being a wire member — an accidental `await scenario` calls it with a resolve
+function, which lands as a loud named error instead of a silent hang.
+
+The assertion options are lenient by design so a scenario reads as intent
+rather than as an exact-match harness: `matches` is a substring test when it is
+a string and a full match only when it is a `RegExp`; `calledWith` deep-equals
+the first argument only; `times` means at-least-once unless a count is given;
+`within` defaults to `'30s'` and `interval` to `'1s'`.
+
+**What this rules out:** giving scenario steps the workflow retry default,
+recording each poll iteration as its own step, treating a falsy `pollUntil`
+result as "not yet", making `pollUntil` throw on timeout, or narrowing `actor`
+by hand inside individual step files.

package/knowledge/decisions/internals/workflow-scenario-hooks-are-a-scenario-only-affordance.md

@@ -0,0 +1,43 @@
+---
+type: decision
+title: Scenario lifecycle hooks are a scenario-only affordance and never mask the failure they follow
+description: A durable workflow replays, so a callback that reruns each replay has no honest meaning there
+tags: workflow
+---
+
+# Scenario lifecycle hooks are a scenario-only affordance and never mask the failure they follow
+
+`PikkuScenarioService.scenarioHooks` (`pikku-scenario-service.ts`) returns hooks
+only when `workflowMeta.source === 'scenario'`. A plain workflow is durable and
+resumable, so a `before`/`after` callback that reran on every replay would have
+no honest meaning — hooks exist for scenarios because a scenario run is a single
+pass by a single external process.
+
+A hook is not a pikku function: it has no id, no meta and no schema, so it
+cannot go through `runPikkuFunc` and the runner records nothing for it. It gets
+exactly what the scenario body gets — the same wire (which is how it reaches the
+app through `actors`) and singleton services composed with this invocation's
+wire services — and nothing else. `ScenarioHookError` keeps the original error
+as `cause`, so the failure that actually happened is never lost behind the label
+saying which phase it happened in.
+
+`onAfterRunFunc` returns early on the `interrupted` outcome, because the run is
+suspended or waiting and teardown would fire mid-flight. When the scenario has
+already failed for its own reason, an `after`-hook failure is attached as
+`cause` and logged rather than replacing the headline; only a teardown failure
+after a *passing* scenario fails the run.
+
+At the feature level (`workflow.types.ts`, `CoreFeature`), hooks run once around
+the whole group — `before → a → b → c → after` — not per scenario. That is the
+one thing a feature deliberately cannot express: gherkin's `Background:` runs
+per scenario, and per-scenario setup is the scenario's own `before`.
+
+`setScenarioEnvironment` on the same service is per-service, not per-run, for a
+related reason: a runner process targets exactly one environment for every
+scenario it executes, so threading it through each run would only create ways
+for two runs in one process to disagree.
+
+**What this rules out:** offering hooks on plain workflows, routing a hook
+through `runPikkuFunc` so it gets recorded as a step, running `after` on an
+interrupted run, letting a teardown failure overwrite the scenario's own
+failure, or reinterpreting feature hooks as per-scenario `Background:`.

package/knowledge/decisions/internals/workflow-scenario-prose-is-rendered-from-typed-calls-not-parsed-from-english.md

@@ -0,0 +1,32 @@
+---
+type: decision
+title: Scenario prose is rendered out of typed calls, not parsed into them
+description: The inversion of cucumber — a readable report with no regex step registry to maintain
+tags: workflow
+---
+
+# Scenario prose is rendered out of typed calls, not parsed into them
+
+`scenario-prose.ts` renders the English sentence a reporter shows for a scenario
+step. It is the inversion of cucumber: instead of parsing English into a call
+through a registry of regexes, the call is typed and the English is rendered out
+of it. The readable report survives without anyone maintaining a step-definition
+registry, and a step that no longer exists cannot leave a dangling phrase behind.
+
+`renderScenarioProse` fills a step's `template` from the input the step was
+actually called with, so the sentence names the values under test — "sees
+@pikku/addon-todos" rather than "sees an addon in the gallery" three times over.
+A placeholder with no recorded value renders as nothing and the surrounding
+whitespace collapses, so an omitted optional input reads as a shorter sentence
+rather than leaking a literal `{state}` into the report. `template` is
+deliberately distinct from `description`: `description` documents what the step
+does, `template` is what a reader of the report sees, and it falls back to
+`description` when absent.
+
+It lives in `@pikku/core` rather than in the CLI so the CLI reporter and the
+console render the same sentence for the same step.
+
+**What this rules out:** adding a gherkin parser or regex step registry,
+rendering prose in the CLI reporter or console instead of core (the two would
+drift), collapsing `template` into `description`, or making a missing
+placeholder value render as the raw placeholder.

package/knowledge/decisions/internals/workflow-scenario-quarantine-reason-lives-in-code.md

@@ -0,0 +1,18 @@
+---
+type: decision
+title: A quarantined scenario states its reason in code, not in a CI invocation
+description: `skip` carries the why next to the scenario it applies to, and naming the scenario explicitly still runs it
+tags: workflow
+---
+
+# A quarantined scenario states its reason in code, not in a CI invocation
+
+`WorkflowsMeta`'s `skip` field (`workflow.types.ts`) is a string, not a boolean.
+A scenario held out of a default run has to say why, and stating the reason in
+code keeps the quarantine next to the scenario it applies to rather than buried
+in a CI invocation nobody reads. A skipped scenario is still runnable — naming
+it explicitly with `--flows` runs it regardless — so quarantine is a default,
+not a disablement.
+
+**What this rules out:** turning `skip` into a boolean flag, moving the skip
+list into CI configuration, or making a skipped scenario unrunnable.

package/knowledge/decisions/internals/workflow-scenario-step-targets-are-string-literals-for-the-inspector.md

@@ -0,0 +1,34 @@
+---
+type: decision
+title: Scenario step targets are string literals so the inspector can read them statically
+description: `step/given/when/then` mirror `do`'s RPC shape because the extractor reads a literal, not an imported symbol
+tags: workflow
+---
+
+# Scenario step targets are string literals so the inspector can read them statically
+
+`ScenarioStepInvocation` in `dsl/workflow-dsl.types.ts`, and the `step`,
+`given`, `when` and `then` members of `PikkuScenarioWire`, all take
+`(stepName, stepFunc: string, data?, options?)` — deliberately the same shape as
+`WorkflowWireDoRPC`. The target is a string, not an imported symbol, because
+the Pikku inspector is static analysis: it reads the argument as a literal and
+does not resolve identifiers. Type safety comes back at the edges — the
+generated `TypedScenario` narrows these over `FlattenedScenarioStepMap`.
+
+`given`/`when`/`then` are pure sugar over `step`; the phase only changes the
+prose a reporter renders (`scenario-prose.ts`), never what executes.
+
+The prose direction is itself the decision: rather than parsing English into a
+call the way cucumber does, `composeStepProse` renders English out of a typed
+call, so a readable report survives without a regex registry paying for it. It
+lives in core so the CLI reporter and the console render the same sentence for
+the same step, and `renderStepTemplate` fills `{placeholders}` from the input
+the step was actually called with — "sees @pikku/addon-todos" rather than the
+same generic sentence three times. A placeholder with no recorded value renders
+as nothing and the surrounding whitespace collapses, so an omitted optional
+input reads as a shorter sentence rather than leaking a literal `{state}`.
+
+**What this rules out:** changing `stepFunc` to accept the imported step config,
+reordering the arguments away from `do`'s shape, giving `given`/`when`/`then`
+behaviour of their own, or moving prose rendering into the CLI where the console
+would drift from it.

package/knowledge/decisions/internals/workflow-step-compensation-runs-as-its-own-durable-step.md

@@ -0,0 +1,26 @@
+---
+type: decision
+title: A step's compensation handler runs as a durable step of its own, and never compensates itself
+description: A refund or rollback must not fire twice on replay, so `onError` is recorded as `<step>:onError` with retries disabled
+tags: workflow
+---
+
+# A step's compensation handler runs as a durable step of its own, and never compensates itself
+
+`runStepCompensation` in `pikku-workflow-service.ts` invokes a failed step's
+`onError` RPC through `rpcStep` under the name `<stepName>:onError` rather than
+calling it directly. Durability is the point: a compensation handler is
+typically a refund or a rollback, and a bare invoke would fire again on every
+replay that walks past the failed step. Recorded as a step, the second replay
+finds it `succeeded` and returns the cached result.
+
+It runs with `{ retries: 0 }` and its own `onError` is deliberately not
+forwarded — a compensation handler cannot itself compensate. `onError` mirrors a
+graph node's: the handler receives `{ error: { message } }`, and the original
+error is still thrown afterwards, so the workflow fails either way. This is
+compensation, not recovery.
+
+**What this rules out:** invoking the `onError` RPC inline "since it's just
+cleanup", giving it the workflow's default retry count, chaining a second
+`onError` onto it, or swallowing the original error because the handler
+succeeded.

package/knowledge/decisions/internals/workflow-step-dispatch-failure-is-transient-not-a-run-failure.md

@@ -0,0 +1,33 @@
+---
+type: decision
+title: A failed workflow step dispatch is transient infrastructure, not a run failure
+description: Queue-unreachable errors leave the run running and the step pending so the orchestrator replays; marking the run failed loses it
+tags: workflow
+---
+
+# A failed workflow step dispatch is transient infrastructure, not a run failure
+
+`WorkflowDispatchException` (`pikku-workflow-service.ts`) means the queue itself
+could not accept the job — pg-boss momentarily down, the transport unreachable —
+not that the step's own logic failed. Everywhere it surfaces (`dispatchStep`,
+`sleepStep`, `startWorkflow`, `orchestrateWorkflow`) the run is left untouched:
+the step stays `pending`, the run stays `running`, and the orchestrator job is
+rethrown so the queue redelivers it and the workflow replays from its snapshot.
+
+The ordering inside `rpcStep` and `sleepStep` is load-bearing: dispatch happens
+BEFORE the step is marked `scheduled`. If the step were marked `scheduled`
+first and the dispatch then failed, the next replay would see `scheduled`,
+pause, and wait forever for a job that was never enqueued.
+
+Redelivery is always safe because an orchestrator job is idempotent: it replays
+the workflow from the snapshot and every already-completed step returns its
+cached result rather than running again. The per-job `attempts` that
+`resolveStepJobOptions` always emits is what makes redelivery happen at all — it
+overrides a queue configured with `retry_limit 0`, so the workflow's retry
+policy survives a conservative queue configuration.
+
+**What this rules out:** folding `WorkflowDispatchException` into the generic
+catch that calls `updateRunStatus(runId, 'failed', ...)`, swallowing it so the
+run silently stalls, or reordering `setStepScheduled` above `dispatchStep` /
+`scheduleSleep` to "keep the status writes together". Each strands or kills a
+run that a redelivered orchestrator tick would have recovered on its own.

package/knowledge/decisions/internals/workflow-step-lock-is-held-only-to-claim-the-step.md

@@ -0,0 +1,27 @@
+---
+type: decision
+title: A workflow step lock is held only to claim the step, never across its execution
+description: Holding the advisory lock — and its pooled connection — across step work exhausted the connection pool and self-deadlocked
+tags: workflow
+---
+
+# A workflow step lock is held only to claim the step, never across its execution
+
+`executeWorkflowStep` in `pikku-workflow-service.ts` takes `withStepLock` for an
+atomic check-and-mark-running only: it reads the step, returns `null` if the
+step already `succeeded` or is already `running` (another worker owns it),
+starts a fresh attempt if it `failed`, and otherwise marks it `running`. The
+lock is then released, and the actual work plus result persistence run outside
+it.
+
+The guard is what makes that safe — once a step is `running`, any concurrent
+worker returns early. The alternative was tried and failed: holding the advisory
+lock, and therefore its pooled connection, across `executeGraphStep` (network
+I/O plus further pool queries) let concurrent steps exhaust the connection pool
+and self-deadlock.
+
+**What this rules out:** widening the `withStepLock` callback to cover RPC
+invocation, child-workflow start, `setStepResult` or `resumeWorkflow` — the
+"obviously safer" refactor that reintroduces the deadlock. If a stronger
+guarantee is ever needed it has to come from the claim itself, not from a longer
+lock hold.

package/knowledge/decisions/internals/workflow-step-rpc-name-is-provenance-only.md

@@ -0,0 +1,34 @@
+---
+type: decision
+title: A workflow step's recorded `rpcName` is provenance only — nothing dispatches off it
+description: It exists so a reader can join a runtime step row back to the declaration that produced it, especially when the durable name was built in a loop
+tags: workflow
+---
+
+# A workflow step's recorded `rpcName` is provenance only — nothing dispatches off it
+
+`insertStepState` and `inlineStep` in `pikku-workflow-service.ts` record the
+name a step was dispatched by: an RPC for a `workflow.do` step, a step function
+for a scenario step, `null` for a closure. Nothing in the engine dispatches off
+that value — it is stored so a reader can join a step back to the function that
+ran it.
+
+It earns its keep when the durable step name was built at runtime. A scenario
+step called in a loop reaches the run as, say, `sees @pikku/addon-todos` while
+it was declared as ``sees ${packageName}``; the recorded step-function name is
+then the only way to join that row back to its declaration. `inlineStep` also
+records the `data` the step was called with for the same reason: a reporter
+renders each step's prose from it, so two calls to one step stay
+distinguishable by what they were asked to check.
+
+Step lineage is recorded alongside it. `fromStepName` is the predecessor that
+scheduled a step — the walked transition — captured by `rpcStep`/`inlineStep`
+*before* `nextStepKey` advances the lineage, and surfaced to a step as
+`fromInvocationId`. In a cyclic graph `a → b → a → c`, the second `a` therefore
+carries `b`'s id, which is what lets the walked path be reconstructed from the
+chain alone.
+
+**What this rules out:** treating the recorded name as the dispatch target,
+dropping it for closure steps "since it is always null", capturing
+`lastStepName` after `nextStepKey` has run, or omitting `data` on inline steps —
+each breaks either the join back to source or the reconstructed path.

package/knowledge/decisions/internals/workflow-suspend-and-approval-reasons-are-durable-step-identities.md

@@ -0,0 +1,38 @@
+---
+type: decision
+title: A suspend or approval `reason` is the step's durable identity, not just a message
+description: The reason is namespaced and used raw as the step key, so it must be derived deterministically across replays
+tags: workflow
+---
+
+# A suspend or approval `reason` is the step's durable identity, not just a message
+
+`getSuspendStepName` and `getApprovalStepName` in `pikku-workflow-service.ts`
+derive a step key from the `reason` string — `__workflow_suspend:<reason>` and
+`__workflow_approval:<reason>`. Each distinct reason is therefore its own step
+row, which is what lets one workflow hold several independent suspends
+(wait-for-build, then wait-for-approval) and lets a dynamic reason inside a loop
+work exactly like a dynamic `do()` step name. The two prefixes are separate
+namespaces so a suspend, an approval, and a `do`/`sleep` step of the same name
+cannot collide.
+
+Because the reason IS the identity, it must be derived deterministically: the
+same replay must produce the same reason at the same point, or the run mints a
+new suspend instead of finding the one it is waiting on. This is the same
+contract as `do()` and `sleep()` step names.
+
+An approval additionally stores its record under a run-state key built by
+`approvalStateKey`, which hex-encodes the step name. The Mongo backend restricts
+state keys to `/^[a-zA-Z0-9_]+$/` while a reason is arbitrary human text, and
+one key per gate means two gates resolving concurrently cannot clobber each
+other through a read-modify-write.
+
+One consequence to know about: `approveStep`'s optional `reason` argument
+addresses the *first* reach of a gate only. If a gate is reached again on a
+later loop iteration, `nextStepKey` gives that row a `#N` suffix, and there is
+currently no way for a caller to name it.
+
+**What this rules out:** deriving a reason from a timestamp, a random id or
+anything else that varies between replays; sharing one namespace (or one
+run-state key) between suspends and approvals; and storing the reason raw as a
+run-state key, which breaks on any backend that constrains key characters.