@pikku/core 0.12.72 → 0.12.77

package/knowledge/decisions/internals/workflow-core-never-imports-a-browser-driver.md

@@ -0,0 +1,42 @@
+---
+type: decision
+title: Core declares the scenario browser surface structurally and never imports a driver
+description: `@pikku/core` must stay dependency-free for edge runtimes, so playwright augments the interface instead of being imported by it
+tags: workflow
+---
+
+# Core declares the scenario browser surface structurally and never imports a driver
+
+`PikkuBrowserWire`, `TestIdSelector`, `ScenarioBrowserProvider` and
+`ScenarioBrowserFailure` all live in `scenario-step.types.ts` as plain
+structural types. `@pikku/core` deliberately never imports playwright — it has
+to stay dependency-free for edge runtimes — so `@pikku/playwright` augments
+`PikkuBrowserWire` via `declare module`, and `wire.browser.page` becomes a fully
+typed Playwright `Page` only in a project that installs it. Declaring the
+provider contract here is also what lets the CLI depend on core alone.
+
+`reset` and `captureFailure` are optional on `ScenarioBrowserProvider` so a
+driver written against an earlier version keeps compiling; the runner treats a
+driver without them as one offering no isolation and no diagnostics.
+`captureFailure` must never throw — a failure to capture must not replace the
+failure being captured — and exists because a browser step fails with a selector
+timeout that says nothing about *why* the page never rendered; the answer is
+almost always in the page's own console and request errors, which the driver has
+been collecting all along.
+
+`TestIdSelector` is richer than a bare `data-testid` because one rarely names
+exactly one element: `where` matches the element's own data attributes (so a
+step asserts a status without reading translated copy back to the app), `prefix`
+matches a family of ids, `containing` picks the match holding a piece of text,
+and `within` scopes the lookup to one row or section. Core defines the shape;
+the driver resolves it against a real page.
+
+The same dependency discipline applies at runtime: `resolveScenarioActors` in
+`pikku-scenario-service.ts` imports the HTTP actor client lazily, so even a
+runner bundle only pays for the AI persona conversation loop when a scenario
+actually signs an actor in.
+
+**What this rules out:** importing playwright (or any driver) from core,
+tightening `reset`/`captureFailure` to required, letting `captureFailure` throw,
+reducing `TestIdSelector` to a plain string, or making the actor-client import
+static.

package/knowledge/decisions/internals/workflow-dsl-meta-separates-runtime-expressions-from-literals.md

@@ -0,0 +1,38 @@
+---
+type: decision
+title: Workflow DSL meta keeps runtime expressions in their own field, apart from literal values
+description: A string `value` regenerates as a string literal; an `expression` regenerates as code, so the two can never share a field
+tags: workflow
+---
+
+# Workflow DSL meta keeps runtime expressions in their own field, apart from literal values
+
+Several step metas in `dsl/workflow-dsl.types.ts` carry a literal field and a
+parallel `expression` field: `SetStepMeta` has `value` and `expression`
+(`count + 1`), `SleepStepMeta` has `duration` and `expression` (a duration known
+only at runtime, e.g. a loop variable). They are mutually exclusive by
+construction because regenerated code has to emit them differently — a string
+`value` becomes a string literal, an `expression` becomes raw code. Collapsing
+them would make every computed assignment regenerate as a quoted string.
+
+Two related shapes exist for the same "the extractor cannot see this
+statically" reason. `ReturnStepMeta.spread` records variables spread into a
+returned object (`return { ...r }`) or a sole returned variable (`return r`) by
+name, because their fields are not enumerable statically and cannot be expanded
+into `outputs`. `FeatureMeta.unresolvedEntries` counts feature entries that
+could not be read statically (a spread, a `.map()`), so a non-zero count marks
+the listing as partial rather than pretending it is complete.
+
+`FanoutStepMeta.stepName` is optional for a different structural reason: a
+fanout is not itself a cached step, and node ids are step names — borrowing a
+body step's name would give the loop and that step the same id, collapsing one
+onto the other.
+
+Free-text documentation (`GraphNodeConfig.notes`,
+`PikkuWorkflowGraphConfig.notes`) is excluded from the graph topology hash, so
+editing a note never marks the workflow as changed and never triggers a version
+mismatch on in-flight runs.
+
+**What this rules out:** merging `expression` into `value`/`duration`, expanding
+`spread` into concrete `outputs`, giving a fanout a required `stepName` taken
+from its body, and folding `notes` into `graphHash`.

package/knowledge/decisions/internals/workflow-features-resolve-scenarios-by-object-identity.md

@@ -0,0 +1,29 @@
+---
+type: decision
+title: A feature resolves its scenarios by object identity, never by name or shape
+description: An unregistered scenario comes back explicitly unresolved rather than silently running as something else
+tags: workflow
+---
+
+# A feature resolves its scenarios by object identity, never by name or shape
+
+`resolveFeatureScenarios` in `feature.ts` builds a `Map` keyed by the registered
+config object itself and looks each feature entry up in it. `pikkuScenario`
+returns its config verbatim and `addWorkflow` registers that same object, so a
+feature holding the imported identifier holds the very object that was
+registered. Nothing is matched by shape, by name, or by any other guess.
+
+That is also why a scenario built inline inside a feature — and therefore never
+registered — comes back in `unresolved` rather than silently running as
+something else. Because scenarios are referenced by imported identifier, a
+renamed or deleted scenario is a compile error rather than a silent skip.
+
+Entries are returned in declaration order and features in registration order,
+since a feature's reading order is its declaration order. A scenario's effective
+tags are its own unioned with the containing feature's, so a tag filter selects
+through the feature.
+
+**What this rules out:** falling back to name matching when identity lookup
+misses, structurally comparing configs, or dropping unresolved entries silently
+instead of reporting them — each turns "this scenario is not registered" into
+"some other scenario ran instead".

package/knowledge/decisions/internals/workflow-graph-inline-and-queued-runs-share-one-planner.md

@@ -0,0 +1,42 @@
+---
+type: decision
+title: Inline and queued workflow graph runs share one transition planner
+description: A second, weaker inline traversal would lose joins, cycle revisits and step provenance that the queued path has
+tags: workflow
+---
+
+# Inline and queued workflow graph runs share one transition planner
+
+`planGraphTransitions` in `graph/graph-runner.ts` is the single place that
+decides which nodes fire next. `continueGraph` (queued) and
+`continueGraphInline` (in-process loop) both call it; they differ only in
+whether the planned wave is dispatched via `queueGraphNode` or executed by
+`executeGraphNodeInline`. Sharing the planner is what gives the inline path
+joins, cycle revisits and `fromStepName` provenance identical to the queue,
+instead of a second and weaker traversal. `executeGraphNodeInline` persists
+under the same physical instance key and records the same predecessor as
+`queueGraphNode` for the same reason.
+
+The planner distinguishes two kinds of edge. A forward edge is node-once: the
+target fires only if it has no instance yet, so converging edges (joins)
+collapse to a single run. A back-edge — one whose target can reach the source,
+detected by `closesCycle` — is a revisit: it fires a fresh ordinal instance
+(`target#1`, …) and is edge-once on `from → target` so it does not re-fire every
+tick. Cycles terminate when branch routing stops looping back, and every
+instance records the predecessor it was reached from.
+
+`remapStepNamesToNodeIds` and `remapBranchKeys` are called on the completed and
+branch sets even where their results are discarded: planning keys steps
+physically, so those calls exist only to surface an ambiguous template-node
+config as an error.
+
+On the queued path a node that sets no `retries` falls back to
+`DEFAULT_STEP_RETRIES` rather than to zero, so the persisted step's retry count
+matches the queue job's `attempts` (see `resolveStepJobOptions`). A step row
+claiming one attempt while the queue silently delivers five is the kind of
+disagreement that makes a retry bug unreadable from the outside.
+
+**What this rules out:** writing a separate traversal for the inline path,
+making forward edges fire per-edge (which breaks joins) or back-edges fire
+node-once (which breaks loops), dropping `fromStepName` from either path, and
+deleting the "unused" remap calls as dead code.

package/knowledge/decisions/internals/workflow-graph-node-notes-are-excluded-from-the-graph-hash.md

@@ -0,0 +1,25 @@
+---
+type: decision
+title: Workflow graph node notes are non-semantic and excluded from the graph hash
+description: Documentation on a node must not count as a topology change, or editing a comment redeploys the workflow
+tags: workflow
+---
+
+# Workflow graph node notes are non-semantic and excluded from the graph hash
+
+`GraphNodeConfig.notes` (`graph/workflow-graph.types.ts`, mirrored on the typed
+builder in `graph/graph-node.ts`) is free-text documentation attached to a node.
+It is deliberately excluded from the graph topology hash (`graphHash`), so
+editing a note never marks the workflow as changed. The graph-level `notes`
+field on `wireWorkflowGraph` (`graph/wire-workflow-graph.ts`) — which carries
+things like imported sticky notes — is excluded for the same reason.
+
+The hash exists to answer "is this the same workflow the running instances were
+started against?". Prose about a node has no bearing on that. If notes were
+hashed, adding a sentence of documentation would register as a topology change
+and the only safe habit would be to never document a node.
+
+**What this rules out:** folding `notes` into `graphHash` "for completeness",
+or using `notes` to carry anything semantic — a routing hint, a version marker,
+a flag some other code reads — since nothing that changes behaviour may live in
+a field the change-detection hash ignores.

package/knowledge/decisions/internals/workflow-inline-runs-report-their-run-id-before-they-can-fail.md

@@ -0,0 +1,29 @@
+---
+type: decision
+title: An inline workflow run reports its run id the moment it exists, because a failure throws instead of returning
+description: `onRunCreated` is the only moment guaranteed to happen whether the run passes, fails or suspends
+tags: workflow
+---
+
+# An inline workflow run reports its run id the moment it exists, because a failure throws instead of returning
+
+`startWorkflow` in `pikku-workflow-service.ts` calls `options.onRunCreated(runId)`
+immediately after `createRun`, before any execution. An inline run that fails
+throws rather than returning `{ runId }`, so a caller wanting to read the run
+back — its steps, which one failed, what it was called with — would otherwise
+have nothing to read it by. Run creation is the only moment guaranteed to happen
+whether the run goes on to pass, fail or suspend.
+
+The failure path is equally deliberate. `WorkflowAsyncException`,
+`WorkflowCancelledException`, `WorkflowSuspendedException` and
+`WorkflowDispatchException` are all excluded from the "mark the run failed"
+branch: the first three already recorded their own status, and the fourth is
+transient. When a run does fail, an *expected* error (a `PikkuError`, e.g. a
+build gate tripping) logs only its message — the message is the whole story, and
+the `expected` flag survives the step-boundary rehydration that strips the
+class. Anything else is logged in full so the trace is there to debug.
+
+**What this rules out:** moving `onRunCreated` to after execution or into the
+success path, returning a sentinel run id instead of throwing, folding the four
+control-flow exceptions into the generic failure branch, or dumping a stack for
+every expected error.

package/knowledge/decisions/internals/workflow-invocation-id-is-the-dedupe-key-not-step-id.md

@@ -0,0 +1,43 @@
+---
+type: decision
+title: `invocationId` is a workflow step's dedupe key; `stepId` is store-specific and must never be used as one
+description: The invocation id is a frozen UUIDv5 of runId + stepName, identical across retries on every backend
+tags: workflow
+---
+
+# `invocationId` is a workflow step's dedupe key; `stepId` is store-specific and must never be used as one
+
+`deriveInvocationId` (`workflow-invocation-id.ts`) is `uuidv5(runId:stepName)`.
+Because both inputs are stable across replays, the same call yields the same
+UUID on every attempt and on every storage backend — so a step can
+`INSERT … ON CONFLICT (invocation_id)` or pass it as an external idempotency
+key (a Stripe key, say) and have a retry of a half-applied side effect collapse
+onto the first attempt.
+
+`stepId` cannot do that job. Whether it stays the same or is minted fresh per
+attempt is store-specific: the in-memory store mints a new one each attempt
+while the SQL store reuses the row. `WorkflowStepWire` documents `invocationId`
+as the dedupe key for exactly this reason.
+
+`PIKKU_WORKFLOW_NAMESPACE` in `workflow-invocation-id.ts` is frozen. Changing it
+would alter every derived invocation id and break dedupe across a deploy — steps
+that already ran would look new. The v5 implementation is hand-rolled (SHA-1
+plus the version and variant bit twiddling) rather than pulled from the `uuid`
+package, and `workflow-invocation-id.test.ts` pins it against the known
+`www.example.com`-in-DNS-namespace vector.
+
+Calling the same step name more than once in a run *is* disambiguated. Ordinals
+are already in: `nextStepKey` (`pikku-workflow-service.ts`) mints a physical key
+per reach — `name`, then `name#1`, `name#2` — and every step entry point routes
+through it, so `deriveInvocationId` hashes the physical key, not the logical
+name. Each reach therefore gets its own row and its own invocation id. The first
+reach keeps the bare name, so ids minted before ordinals landed are unchanged,
+and the ordinal counters reset at the start of each replay, so a given call site
+resolves to the same key on every attempt. `workflow-step-ordinal.test.ts` pins
+all three properties.
+
+**What this rules out:** using `stepId` as an idempotency key, regenerating the
+namespace UUID, swapping in a different hash or a random id, passing the logical
+step name to `deriveInvocationId` instead of the physical key from
+`nextStepKey`, or "simplifying" the derivation to include anything that varies
+between replays.

package/knowledge/decisions/internals/workflow-queued-step-dispatch-requires-an-explicit-opt-in.md

@@ -0,0 +1,29 @@
+---
+type: decision
+title: A workflow step goes through the queue only if its function opts in, and there is no inline fallback
+description: `workflowQueued: true` is the whole decision; a missing queue service is a hard error, not a silent downgrade
+tags: workflow
+---
+
+# A workflow step goes through the queue only if its function opts in, and there is no inline fallback
+
+`dispatchStep` in `pikku-workflow-service.ts` decides queued-vs-inline purely
+from the step function's `workflowQueued` flag in function meta, which defaults
+to false. If the flag is not set the method returns false and the caller runs
+the step inline. If it IS set and no queue service is configured, that is a hard
+error naming the step and the function — never a quiet downgrade to inline
+execution, because a function marked `workflowQueued` was marked that way for a
+reason (isolation, a long tail latency, a resource limit) that inline execution
+does not honour.
+
+Inline execution is not merely a degraded queue path. `runInlineRetryLoop` wraps
+the same `running → result` / `fail → retry-attempt → backoff → retry`
+scaffolding around a step-specific body and stays O(K) — no suspend, no replay —
+which is what makes an inline run cheap. Its optional `onError` hook exists for
+terminal errors that must NOT retry (RPC-not-found suspends the run for
+redeploy); if the hook throws, the loop exits immediately without recording a
+step error or retrying.
+
+**What this rules out:** inferring "should queue" from the presence of a queue
+service, falling back to inline when the queue is missing, and reusing
+`runInlineRetryLoop` for anything that needs to suspend between attempts.

package/knowledge/decisions/internals/workflow-queues-are-per-workflow-by-default.md

@@ -0,0 +1,42 @@
+---
+type: decision
+title: Workflows get their own queues by default, and queue names are resolved from queue meta
+description: Per-workflow queues stop one slow step head-of-line-blocking every other workflow; `shared-groups` trades that for one set of pollers
+tags: workflow
+---
+
+# Workflows get their own queues by default, and queue names are resolved from queue meta
+
+`WorkflowQueueOptions.queueStrategy` defaults to `'per-workflow'`: each workflow
+gets its own `wf-orchestrator-*` and `wf-step-*` queue. That gives complete
+isolation and is what lets serverless providers deploy one unit per workflow,
+at the cost of one set of pollers per queue — which adds up on pull-based
+backends. `'shared-groups'` puts every workflow on the shared orchestrator and
+step-worker queues and keeps them isolated with a per-group concurrency cap, so
+there is one set of pollers for the whole system. It is only for single-process
+runtimes; a per-unit serverless deploy needs the per-workflow queues to route to
+its units. The options are passed to the service constructor rather than read
+from `config.workflow` because the queues are wired during construction, before
+singleton services — and so before `config` — exist.
+
+Under `'shared-groups'`, `getJobGroup` returns a group keyed by workflow name
+(steps group by step function, mirroring how per-step queues split them). It
+returns `undefined` under `'per-workflow'`, because the queue name already
+isolates the workflow and a group would cap it inside its own dedicated queue.
+The group tier repeats the id so a workflow can be given its own limit purely
+from config; note tiers can only lower a limit, never raise it above `default`,
+because the backend applies its pre-fetch exclusion per group using `default`
+alone.
+
+`getOrchestratorQueueName` and `getStepWorkerQueueName` read `queue.meta`, which
+is always populated globally, rather than `queue.registrations`, which only
+covers queues this unit consumes. In a per-unit deploy the orchestrator unit
+produces to the per-step queues without consuming them, so registrations would
+miss them. `wireQueueWorkers` warns loudly when workflows exist but no
+`wf-orchestrator-*` queue is in meta: everything still "works" via the shared
+fallback, but the isolation is gone and one slow step head-of-line-blocks every
+other workflow — invisible until a queue starves.
+
+**What this rules out:** defaulting to `'shared-groups'`, returning a job group
+under `'per-workflow'`, resolving queue names from `queue.registrations`, or
+dropping the wiring-time warning as noise.

package/knowledge/decisions/internals/workflow-repeated-step-names-get-an-ordinal-suffix.md

@@ -0,0 +1,33 @@
+---
+type: decision
+title: A workflow step name repeated in one run gets an ordinal suffix, and the first reach stays bare
+description: `name`, `name#1`, `name#2` keys each reach separately without changing the durable key of any existing run
+tags: workflow
+---
+
+# A workflow step name repeated in one run gets an ordinal suffix, and the first reach stays bare
+
+`nextStepKey` in `pikku-workflow-service.ts` turns a logical step name into the
+physical, replay-stable key for the Nth time the run reaches it: the bare name
+for ordinal 0, `name#N` for repeats. Keeping the first reach unsuffixed is what
+makes the scheme backward compatible — existing runs and their persisted step
+rows are untouched — while still letting the same literal step name be invoked
+several times (a loop, a helper called twice) without the rows clobbering each
+other.
+
+The counters live in the per-replay `RunContext.replay.ordinals` and are reset
+by `beginReplay` on every orchestrator tick. That reset is what makes the keys
+deterministic: given a deterministic workflow body, the second replay assigns
+exactly the same key to the same reach and therefore finds the cached row rather
+than minting `name#1` for a step that already succeeded. `nextStepKey` also
+records the key it just produced as `replay.lastStep`, which is how the next
+step learns its predecessor (`fromStepName`).
+
+Graph runs use the same convention for cycle revisits, and
+`stripInstanceOrdinal` / `executeWorkflowStep` map `node#N` back to the logical
+node id, which is not a literal key in the graph's `nodes` map.
+
+**What this rules out:** suffixing the first reach too (it re-keys every step of
+every existing run), advancing the ordinal counters anywhere other than a fresh
+replay pass, deriving step keys from a non-deterministic source, or treating a
+physical step key as a graph node id without stripping the ordinal.

package/knowledge/decisions/internals/workflow-replay-reads-its-steps-once-and-caches-only-the-immutable-half.md

@@ -0,0 +1,32 @@
+---
+type: decision
+title: A workflow replay reads its steps once and caches only the run's immutable half
+description: The per-replay snapshot collapses O(N^2) step reads to one, but caching mutable run fields would make the replay read a lie
+tags: workflow
+---
+
+# A workflow replay reads its steps once and caches only the run's immutable half
+
+A DSL replay walks the workflow body from the top, and each step it passes asks
+for its own row — so a run of N steps costs N reads per replay and O(N^2) over
+its lifetime. `beginReplay` in `pikku-workflow-service.ts` opens a pass by
+calling `listStepStates`, which backends able to answer in a single query
+override; `loadOrCreateStep` then serves every step from that snapshot. This is
+safe because a pass reaches each step key at most once and the steps it replays
+past are `succeeded`, and therefore immutable.
+
+`getRunIdentity` caches the run for the same pass, but only for its immutable
+half — which workflow it is, the wire it was started on, its input. Anything
+needing `status`, `output`, `error` or `state` must call `getRun`: those move
+while the run executes and a cached copy would be a lie. The `replay` half of
+`RunContext` is rebuilt from scratch on every orchestrator tick and torn down by
+`endReplay`.
+
+`loadOrCreateStep` also handles a concurrent replay creating the row after the
+snapshot was taken: if `create()` throws, it re-reads the step, and only if that
+read also fails does it rethrow the original — because in that case the insert
+failed for its own reasons and that error is the one worth seeing.
+
+**What this rules out:** widening the snapshot to cover mutable run fields,
+keeping it across orchestrator ticks, or removing the re-read fallback in
+`loadOrCreateStep` on the grounds that a duplicate insert "cannot happen".

package/knowledge/decisions/internals/workflow-retries-are-owned-by-the-workflow-not-the-queue.md

@@ -0,0 +1,31 @@
+---
+type: decision
+title: Workflow step retries are owned by the workflow, never by the queue
+description: A step's retry count is resolved once and always passed to the queue as `attempts`, so the queue can never apply its own default
+tags: workflow
+---
+
+# Workflow step retries are owned by the workflow, never by the queue
+
+A step's retry policy comes from the step itself, falling back to
+`DEFAULT_STEP_RETRIES` (5) when unset. The default is deliberately greater than
+zero so a transient failure — a DB blip, a downstream restart, a deploy — is
+ridden out without the author asking for it; that is safe only because every
+step gets a stable `invocationId` to dedupe on. An explicit `retries: 0` means
+exactly once and must be honoured.
+
+`resolveStepJobOptions` in `pikku-workflow-service.ts` therefore ALWAYS emits
+`attempts`, even when it is 1. Backoff defaults to exponential whenever at least
+one retry remains, so retries ride out an outage instead of firing instantly; a
+concrete `retryDelay` (`15000`, `'15s'`) selects fixed backoff, and only the
+literal `'exponential'` selects exponential explicitly. `rpcStep` resolves the
+policy once, before both persistence and dispatch, so the value stored on the
+step row — which drives `retriesExhausted` — is the same number the queue turns
+into `attempts`. `queueGraphNode` in `graph/graph-runner.ts` defaults node
+retries to `DEFAULT_STEP_RETRIES` for the same reason.
+
+**What this rules out:** dropping `attempts` when it equals 1 "because that is
+the default anyway", resolving retries separately at the persistence and
+dispatch sites, or letting the queue adapter's own `retry_limit` decide. Any of
+those lets the queue re-run a step the workflow said to run once, or lets the
+engine believe retries are exhausted while the queue keeps retrying.

package/knowledge/decisions/internals/workflow-run-capabilities-are-extensions-not-subclasses.md

@@ -0,0 +1,39 @@
+---
+type: decision
+title: Workflow run capabilities are extensions, not subclasses
+description: Scenario support lives in a separate module behind `setRunExtension` because a bundler drops an unused module but never an unused class member
+tags: workflow
+---
+
+# Workflow run capabilities are extensions, not subclasses
+
+`PikkuWorkflowService` names nothing about what a `WorkflowRunExtension` is for.
+An extension is installed with `setRunExtension(create)` and receives a narrow
+`WorkflowRunEngine` handle — `inlineStep`, `updateRunStatus`,
+`onChildWorkflowFailed`, `verifyStepName` — so recording a durable step stays
+available to it without becoming public API on every workflow service a
+production app instantiates.
+
+The reason it is not a subclass is bundle size: a bundler drops an unused
+*module* but never an unused class member. Anything declared on
+`PikkuWorkflowService` ships in every server built on Pikku, along with
+everything it imports. `PikkuScenarioService` (`pikku-scenario-service.ts`) is
+the one implementation today — steps, actors, lifecycle hooks, the browser
+provider and the assertion wire members — and scenarios only ever run from
+`pikku scenario run`, so the whole surface stays behind an import that only the
+runner makes. It is not a workflow service in its own right because a scenario
+is not a different kind of run; it is the same durable run with a step
+vocabulary on top. `scenario-service.test.ts` asserts that none of those members
+leak onto the base class.
+
+`createScenarioRunner` in the same file bundles the two lines
+`pikku scenario run` needs so no caller has to remember the capability is
+installed rather than inherited. It pairs the extension with
+`InMemoryWorkflowService`, which is the right engine because a scenario run is a
+single external process driving a deployed app over its real transport: there
+is nothing to persist and no second worker to resume it.
+
+**What this rules out:** moving scenario steps, actors, hooks or the browser
+provider onto `PikkuWorkflowService`, making `PikkuScenarioService` extend it,
+or widening `WorkflowRunEngine` into "just pass the service" — each one puts the
+scenario runtime, and its transitive imports, into every deployed Pikku server.

package/knowledge/decisions/internals/workflow-run-mirror-is-never-a-source-of-truth.md

@@ -0,0 +1,29 @@
+---
+type: decision
+title: The workflow run mirror is an observability sink, never a second source of truth
+description: Every mirrored write happens after the authoritative write lands, and a mirror failure can never fail the workflow
+tags: workflow
+---
+
+# The workflow run mirror is an observability sink, never a second source of truth
+
+`WorkflowRunMirror` lets an executor shadow every state write to an external
+read store — typically a DB the console UI queries while the run itself lives in
+a Durable Object or Redis. Both properties that make it safe follow from the
+single shape of `PikkuWorkflowService.mirrored()` in
+`pikku-workflow-service.ts`: the mirror is only ever told about a write that has
+already landed in the canonical store, and a mirror that is down or throwing is
+caught, logged at warn, and otherwise invisible to the workflow. Even the log
+call is wrapped, because singleton services may not be initialised yet.
+
+Every public write method is a thin `mirrored(() => …Impl(...), (mirror) => …)`
+pair over a `protected abstract` implementation. That pairing is the reason a
+mirror method added to the interface but never wired into the service would
+write nothing at runtime — `workflow-mirror.test.ts` spells out the full method
+list to catch exactly that.
+
+**What this rules out:** reading run or step state back from the mirror,
+awaiting the mirror before returning the authoritative result, letting a mirror
+rejection propagate, or writing to the mirror first so the two "stay in sync" —
+any of which turns an index into a second, divergent source of truth that can
+take the workflow down with it.

package/knowledge/decisions/internals/workflow-run-polling-backs-off-to-the-callers-ceiling.md

@@ -0,0 +1,33 @@
+---
+type: decision
+title: Workflow run polling starts short and backs off to the caller's ceiling
+description: `pollIntervalMs` is a ceiling, not a cadence, and the wait lives in its own method so the schedule can be asserted without the clock
+tags: workflow
+---
+
+# Workflow run polling starts short and backs off to the caller's ceiling
+
+`awaitRunEnd` in `pikku-workflow-service.ts` reads a run until it reaches an end
+state, starting at `WORKFLOW_POLL_MIN_MS` (10ms) and multiplying by
+`WORKFLOW_POLL_FACTOR` (1.6) up to the caller's `maxIntervalMs`. A fixed
+interval is wrong at both ends: it makes a workflow that finished in
+milliseconds wait out the whole interval anyway, and it keeps reading a
+long-running one at full rate for its entire life. Backing off returns quick
+runs promptly while a slow run's read cost grows logarithmically rather than
+linearly with duration. So `runToCompletion`'s `pollIntervalMs` is a ceiling,
+not a cadence.
+
+An inline sub-workflow uses a lower ceiling, `WORKFLOW_CHILD_POLL_MAX_MS`
+(500ms), because the parent step is blocked on it: every wait there is added
+latency in the middle of a workflow rather than at its edge.
+
+`waitBeforeNextRead` exists as its own method purely so the schedule can be
+asserted directly (`workflow-run-polling.test.ts`). Timing a poll loop by the
+clock measures the host's scheduler as much as the policy — `setTimeout(40)`
+routinely returns late on a loaded CI runner — which makes the obvious test both
+slow and flaky.
+
+**What this rules out:** replacing the backoff with a fixed `pollIntervalMs`
+sleep, reading `pollIntervalMs` as the first wait, giving child runs the
+top-level ceiling, or inlining `waitBeforeNextRead` back into the loop as a bare
+`setTimeout`.

package/knowledge/decisions/internals/workflow-run-timeline-is-a-pure-fold-over-durable-history.md

@@ -0,0 +1,37 @@
+---
+type: decision
+title: The workflow run timeline is a pure fold over durable history, with the row's status as the authority
+description: No IO in the fold keeps time-travel transport-independent; the terminal event comes from `status`, not from a timestamp every backend populates
+tags: workflow
+---
+
+# The workflow run timeline is a pure fold over durable history, with the row's status as the authority
+
+`run-timeline.ts` turns `getRunHistory` — one row per step *attempt*, each with
+lifecycle timestamps — into a flat, chronologically ordered event stream, and
+folds that stream up to any point to recover what the run "knew" then: the same
+step cache a replay would hold, plus the walked path. Both functions are pure,
+with no IO, so they are trivially testable and the same fold works for the
+Redis, Kysely and in-memory stores alike.
+
+The terminal event is driven by the row's authoritative `status`, falling back
+to `updatedAt` when the specific stamp is absent, because the lifecycle
+timestamps are not populated by every backend — Kysely leaves
+`succeededAt`/`runningAt` null. The intermediate `scheduled` and `running`
+events are optional enrichment, emitted only when the backend recorded them; the
+`pending` event always exists and is the one that carries provenance
+(`fromStepName`).
+
+Ordering is defensive: events sort by timestamp, then by `LIFECYCLE_ORDER`, then
+by original index, so rows sharing an instant stay deterministic across
+backends. In the fold, a retry's `pending` event reopens the step and drops the
+prior result and error.
+
+The index conventions are fixed and callers depend on them: `TimelineEvent.seq`
+is a monotonic 0-based position, `attempt` is 1-based, and
+`reconstructStateAt(at)` is inclusive whether `at` is a seq or a `Date` — a
+point before the first event yields the empty initial state with `seq` of `-1`.
+
+**What this rules out:** reading state inside the fold, deriving the terminal
+event from `succeededAt`/`failedAt` alone, trusting history to arrive sorted, or
+letting a retry's reopening event keep the previous attempt's outcome.

package/knowledge/decisions/internals/workflow-scenario-assertions-never-retry-and-record-one-step.md

@@ -0,0 +1,50 @@
+---
+type: decision
+title: Scenario steps default to no retries, and a whole poll is one durable step
+description: Retrying a failed assertion is wrong for a test primitive; recording the poll as one step means replay returns the outcome, not the loop
+tags: workflow
+---
+
+# Scenario steps default to no retries, and a whole poll is one durable step
+
+`ScenarioStepOptions.retries` defaults to 0, unlike an ordinary workflow step
+which inherits `DEFAULT_STEP_RETRIES`. `PikkuScenarioService.scenarioStep`
+applies that default explicitly (`options?.retries ?? 0`). Retrying a failed
+assertion is the wrong behaviour for a test primitive: it converts a real
+failure into a slow flake and hides the very race the scenario was written to
+catch.
+
+`expectEventually` is the sanctioned way to wait. The entire poll — invoke,
+check the predicate, sleep, repeat until `within` elapses — runs inside a single
+`inlineStep`, so it is ONE recorded step and a replay returns the cached
+outcome rather than re-polling. `expectError` and `expectService` are the same
+shape: one durable step whose body contains the whole assertion, with a failure
+message that names the step, the target, and what was actually seen.
+
+`pollUntil` in `scenario-poll.ts` is the step-level equivalent, and its
+`undefined` convention is deliberate: `undefined` means "not yet" and nothing
+else, because `false`, `0` and `''` are all real answers — a probe asking
+whether something happened reports `false` when it did not. It returns
+`undefined` at the deadline rather than throwing, leaving the error to the
+caller, who is the only one who knows what was being waited for. Its own
+defaults are shorter than the assertion wrappers': `within` 15s, `interval`
+250ms.
+
+`requireActor` and `requireScenarioEnv` (`scenario-step-guards.ts`) exist for
+the mirror-image reason: `actor` and `env` are optional on the step wire because
+a pure assertion step needs neither, so the narrowing happens in one place with
+a message that says what to do, instead of every step file writing its own
+guard. `scenarioStep`'s `verifyStepName` call doubles as the guard for `then`
+being a wire member — an accidental `await scenario` calls it with a resolve
+function, which lands as a loud named error instead of a silent hang.
+
+The assertion options are lenient by design so a scenario reads as intent
+rather than as an exact-match harness: `matches` is a substring test when it is
+a string and a full match only when it is a `RegExp`; `calledWith` deep-equals
+the first argument only; `times` means at-least-once unless a count is given;
+`within` defaults to `'30s'` and `interval` to `'1s'`.
+
+**What this rules out:** giving scenario steps the workflow retry default,
+recording each poll iteration as its own step, treating a falsy `pollUntil`
+result as "not yet", making `pollUntil` throw on timeout, or narrowing `actor`
+by hand inside individual step files.