workflow 5.0.0-beta.4 → 5.0.0-beta.40

package/docs/changelog/eager-processing.mdx

@@ -0,0 +1,269 @@
+---
+title: Eager Processing of Steps & Incremental Event Replay
+description: Combine workflow event replay and step bundles to do work inline where possible, only deferring to queue for parallelism
+type: overview
+---
+
+# Eager Processing of Steps & Incremental Event Replay
+
+**Date**: March 2026
+
+This is a major internal architecture change to how Workflow DevKit executes workflows and steps. It reduces function invocations and queue overhead by executing steps _inline_ within the same function invocation as the workflow replay, rather than dispatching every step to a separate function via the queue.
+
+## Previous Architecture
+
+The previous architecture used two separate routes, each backed by its own queue trigger:
+
+```
+Queue: __wkf_workflow_*  -->  /.well-known/workflow/v1/flow   (workflow replay in VM)
+                                |
+                          suspension (step needed)
+                                |
+                          queue step to __wkf_step_*
+                                |
+Queue: __wkf_step_*      -->  /.well-known/workflow/v1/step   (step execution in Node.js)
+                                |
+                          step completes
+                                |
+                          queue continuation to __wkf_workflow_*
+                                v
+                          (cycle repeats for each step)
+```
+
+Each step required **2 queue messages** (step invoke + workflow continuation) and **2 function invocations**, plus cold start overhead for each. A serial workflow with 10 steps needed ~21 function invocations.
+
+## New Architecture
+
+The two routes are merged into a single handler at `/.well-known/workflow/v1/flow` using `workflowEntrypoint()`. The step route is no longer generated.
+
+The handler runs an inline execution loop:
+
+```
+receive queue message
+  |
+  +-- if message has stepId+stepName: execute that step, queue workflow continuation, exit
+  |
+  v
+replay workflow in VM
+  |
+  +-- workflow completed --> create run_completed event, exit
+  +-- workflow failed   --> create run_failed event, exit
+  |
+  v
+suspension with pending operations
+  |
+  +-- process hooks and waits (unchanged)
+  |
+  +-- 0 pending steps  --> return (waits/hooks only)
+  +-- 1 pending step   --> execute inline, loop back to replay
+  +-- N pending steps  --> queue N-1 to self (with stepId),
+  |                        execute 1 inline, loop back to replay
+  |
+  +-- timeout check: if wall-clock time >= threshold,
+  |   re-schedule self via queue and exit
+  |
+  v
+(loop continues until completion, timeout, or non-step suspension)
+```
+
+A serial workflow with 10 steps now completes in **1 function invocation**.
+
+## Background Steps (Parallel Execution)
+
+When a workflow suspends with multiple pending steps (e.g., from `Promise.all`), the handler creates `step_created` events for all of them, queues N-1 back to `__wkf_workflow_*` with `stepId` and `stepName` in the message payload, executes 1 step inline, and loops back to replay.
+
+Each background step message is handled by a separate function invocation of the same handler. When a message arrives with `stepId` and `stepName`, the handler executes that specific step, then checks if all parallel steps from the batch are done by comparing `step_created` events against terminal events (`step_completed`/`step_failed`):
+
+- **All steps done**: The handler replays the workflow inline, continuing the execution loop without a queue roundtrip.
+- **Steps still pending**: The handler returns without queuing a continuation. The last handler to complete its step will see all steps done and replay inline.
+- **Pending ops (stream writes)**: The handler queues a continuation and returns, so `waitUntil` can flush the pending stream data.
+
+### Convergence After Parallel Steps
+
+When multiple background steps complete near-simultaneously, multiple handlers may observe "all steps done" and attempt to advance the workflow concurrently. The event-sourced architecture plus the invariants below ensure safe convergence:
+
+- **`step_created` idempotency** — duplicate creates return 409; exactly one handler owns each step
+- **`step_completed` / `step_failed` idempotency** — only the first invocation to record a terminal result wins
+- **Queue idempotency keys** — background step messages use `correlationId` as idempotency key
+- **Deterministic replay** — all invocations produce the same result given the same event log
+
+### Single Inline Executor Per Step
+
+Inline step execution combined with background-step dispatch introduces a new coordination requirement: when multiple handlers reach the same `Promise.all` batch concurrently, we need to guarantee that each step body runs at most once via the inline path. Without that guarantee, the event log accumulates duplicate `step_started` events (including some written *after* `step_completed`, which orphans them on replay) and step bodies run redundantly.
+
+The design enforces a simple invariant: **exactly one handler owns each step, and only the owner may execute it inline**. Ownership is established by the atomicity of `step_created`:
+
+1. **Atomic `step_created`** — `events.create('step_created', correlationId=X)` is serialized per-correlationId in every world. Exactly one concurrent caller succeeds; the rest receive `EntityConflictError`.
+2. **Suspension handler reports ownership** — only `step_created` writes that actually succeeded (not those that caught 409) count toward ownership.
+3. **Inline execution is gated on ownership** — a handler that didn't win any `step_created` race performs no inline execution.
+4. **Queueing is unconditional** — for every pending step except the one being inline-executed, the handler enqueues a background step message with `idempotencyKey: correlationId`. This is what makes crash recovery work: if a prior handler wrote `step_created` but crashed before enqueueing, a later handler will enqueue the orphaned step. Concurrent handlers' redundant enqueues dedupe on the idempotency key.
+
+Together these give: every `step_created` event has exactly one inline executor **and** at least one queued dispatch. Step bodies are never executed concurrently, and `step_started` events never land in the log after `step_completed` for the same step.
+
+**Retry semantics are preserved**: a step that is currently running (status=`running`) still accepts a second `step_started` write with an incremented attempt counter — this is how queue redelivery after a SIGKILL mid-execution legitimately re-runs the step.
+
+## Incremental Event Loading
+
+The handler caches the event log in memory across loop iterations. Instead of re-fetching the entire event log on each replay:
+
+1. **First iteration**: full load, returning both the events and the final pagination cursor
+2. **Subsequent iterations**: fetch only events created after the saved cursor and append them to the cached array
+
+For a 10-step serial workflow completing in one invocation, the 10th replay loads ~2 new events instead of re-fetching all ~30.
+
+Incremental loading depends on the World returning a cursor even on the final page of results. If a World implementation does not return a cursor after the initial load, the handler logs an error and falls back to a full reload.
+
+## Timeout Handling
+
+The inline execution loop checks wall-clock time before each replay iteration. If the elapsed time exceeds a configurable threshold (default: 110 seconds, for a 120-second function limit), the handler re-schedules itself via the queue and returns. Configurable via `WORKFLOW_V2_TIMEOUT_MS`.
+
+If a single step takes longer than the timeout threshold, the step runs to completion (or SIGKILL) — there is no interruption mechanism for in-progress step execution. This is the same behavior as the previous architecture.
+
+## Queue Message Changes
+
+The `WorkflowInvokePayload` schema has two new optional fields: `stepId` and `stepName`. When `stepId` is present, the handler executes that specific step before (or instead of) replaying the workflow. Background steps are queued with both set, so the handler knows which step function to call without loading the event log. Previously, `stepName` was resolved by loading all events and searching for the `step_created` event matching the `stepId` — an O(N) operation on the full event history for every background step arrival.
+
+The queue trigger configuration uses `WORKFLOW_QUEUE_TRIGGER` on the `__wkf_workflow_*` topic. The `__wkf_step_*` topic and its separate trigger are no longer generated.
+
+## Generated File Layout
+
+```
+.well-known/workflow/v1/
+  flow/
+    route.js                  # Handler (workflowEntrypoint)
+    __step_registrations.js   # Step function registrations (side effects)
+  webhook/
+    [token]/
+      route.js                # Webhook handler (unchanged)
+  manifest.json               # Workflow/step/class manifest (unchanged)
+  config.json                 # Functions config (single trigger)
+```
+
+The `step/` directory is no longer generated.
+
+## Design Notes and Tradeoffs
+
+### Parent→Child Polling Holds Worker Slots
+
+`Run#returnValue` is implemented as a polling step: the workflow awaits the child run's terminal status inside a step body. In worker-based worlds (notably `world-postgres`), each such poll occupies a queue worker slot until the child run finishes. Parent workflows that fan out to many child runs — recursive workflows like `fibonacciWorkflow` are the obvious case — can therefore consume a large fraction of available workers just holding positions in `Promise.all([...children.map(c => c.returnValue)])`.
+
+If `queueConcurrency` is smaller than the peak number of concurrent parent polls plus the workers needed for any in-flight children, the system deadlocks: every slot is held by a parent waiting on a child, but no child can acquire a slot to start.
+
+For `world-postgres`, the default `queueConcurrency` is set to **50**. Workflows that fan out more aggressively must raise this ceiling.
+
+To prevent deadlock when polling is executed inline by the step executor, `Run#pollReturnValue()` detects when it's running inside a step executor and throws `TooEarlyError` instead of polling in a blocking loop. The step executor handles `TooEarlyError` by re-queueing the step with a 1-second delay, freeing the worker. Unlike `RetryableError`, `TooEarlyError` does NOT count against `maxRetries`, so polling steps can retry indefinitely until the child completes.
+
+**Follow-up**: Replace the worker-pool sizing requirement with a polling design that does not occupy a worker slot. Options under consideration: moving child-completion polling out of the step body into the suspension layer, or emitting a `run_completed` notification on the parent's stream/queue so the parent only resumes when the child actually finishes.
+
+### Mixed Suspensions
+
+A suspension may contain steps, hooks, and waits simultaneously. The handler creates events for all, then dispatches everything we are not running inline as a single parallel batch of queue messages:
+
+```
+ownedPendingSteps = pendingSteps.filter(owned by this handler)
+inlineStep        = ownedPendingSteps[0]   // optional
+
+dispatches = [
+  ...for each non-inline pendingStep: queue stepId message (idempotency=correlationId),
+  ...if soonest pending wait:          queue delayed continuation
+                                       (delaySeconds=min(remaining, maxDelay),
+                                        idempotency=waitCorrelationId[:hop|:secondBucket]),
+]
+await Promise.all(dispatches)
+
+if (!inlineStep) return
+await executeStep(inlineStep)
+```
+
+The wait timer is queued as its own continuation rather than encoded in the handler's return value (`{ timeoutSeconds }`). This is what makes `Promise.race(step, sleep)` behave correctly: even when the inline step blocks the handler for the full step duration, the wait continuation fires in a separate function invocation. If the sleep wins, that parallel invocation observes `wait_completed` via the "complete elapsed waits" pass and finishes the run; if the step wins, the wait continuation fires later and no-ops on the terminal run via the existing terminal-event check.
+
+Step queueing remains unconditional (covers crash recovery: if a prior handler wrote `step_created` but crashed before queueing, a later handler will queue it; idempotency keys dedupe redundant queues across concurrent handlers).
+
+Wait continuations are likewise deduplicated, keyed on the wait's correlation ID: while a wait is pending, every replay pass over the run re-observes it and would otherwise enqueue another delayed continuation. A key is attached in all cases, since some worlds serialize key-less workflow messages per run, which would park the continuation behind the handler's own inline step execution.
+
+Two situations deliver a continuation while its wait is still pending, and each varies the key so the re-enqueue isn't dropped by a world's dedupe window (which outlives the first delivery): waits longer than the maximum queue delay (23h, bounded by VQS's 24h message retention) are clamped and chained across hops, with the hop index suffixed to the key so each hop dedupes within its window but the chain always advances; and near-elapsed waits (≤2s remaining) use a second-bucketed key suffix so a continuation delivered marginally early (clock skew) can enqueue a fresh short-delay retry. See `runtime/wait-continuation.ts` for the full selection logic.
+
+The retry/throttle and hook-conflict paths still return `{ timeoutSeconds }` since their semantics are "redeliver THIS message after a delay" rather than "schedule a fresh wait timer." Those can be unified in a follow-up.
+
+The unified dispatch requires `world-local` to honor `delaySeconds` on the queue (added in the same PR series). Without it, the wait continuation would fire instantly in dev and trigger a spurious replay before the wait elapsed (recoverable via redelivery, but inefficient and observable as duplicate `step_started` events under contention).
+
+### VM Sandboxing
+
+Workflow code still runs in a Node.js VM for determinism and sandboxing. Step code runs in the Node.js host context. The only change is that both happen within the same function invocation.
+
+### Bundle Size and Cold Start
+
+The combined bundle is larger (contains both step code and workflow VM code). Cold start time increases slightly. The reduction in total function invocations more than compensates.
+
+### Step Retries
+
+When an inline step fails with retries remaining:
+
+- `RetryableError` with explicit `retryAfter` delay: re-queue to self with `stepId` and delay
+- Transient errors with immediate retry: re-queue to self with `stepId` (delay = 1s)
+- `FatalError`: fail immediately
+
+### Encryption Key Resolution
+
+Encryption keys are resolved once before the inline execution loop starts (after the run status is confirmed as `running`) and reused across all iterations. Background step executions resolve the key independently. The key does not change within a run.
+
+### Module Scope Duplication in Re-Bundled Output
+
+Builders that re-bundle the combined output into a single file (standalone CLI, Vercel Build Output API, NestJS) produce a layout where esbuild creates isolated module scopes for each source module, even within the same output file. Without intervention this means `registerStepFunction` and `getStepFunction` operate on different `Map` instances — steps are registered into one Map but looked up from another.
+
+The step function registry and the step context storage are `globalThis` singletons (via `Symbol.for`) to ensure all module scopes share the same instances. The same pattern is used for the World singleton and the class serialization registry.
+
+### Inline Step Execution with Pending Stream Operations
+
+When a step's arguments or return value include serialized streams (e.g., `WritableStream` from `getWritable()`, or AI SDK streaming steps), the serialization layer creates background `flushablePipe` operations that pipe data to S3. These ops are tracked in an `ops` array and need to complete before the stream data is readable by external consumers.
+
+In V1, each step ran in a separate function invocation. After the step completed, `waitUntil(ops)` kept the function alive to flush the ops. In V2, the inline execution loop continues immediately after the step body returns — so we need to know whether to keep looping or break out and let `waitUntil` flush.
+
+`executeStep()` attempts a 500ms `Promise.race` between the ops settling and a timeout. If ops settle in time (data confirmed on server), it returns `hasPendingOps: false` and the V2 handler continues the inline loop. If ops don't settle in 500ms (e.g., `WritableStream` kept open across steps), it returns `hasPendingOps: true` and the V2 handler breaks the loop and queues a continuation so `waitUntil` can flush them.
+
+**Follow-up**: Shrink the 500ms inline-ops budget once we have confidence that the flush-waiter path settles deterministically across all worlds. A signaled "ops drained" event from the world layer would let `executeStep()` proceed without the timeout in the common case.
+
+### Buffered Stream Flush with Waiter Promises
+
+`WorkflowServerWritableStream` buffers writes and flushes via a 10ms `setTimeout` for batching. Naively, `write()` could return immediately after buffering, but that would cause the `flushablePipe`'s `pendingOps` counter to reach 0 before data actually reached the server — the V2 inline loop would see ops as settled prematurely and produce data-loss races on every step with `WritableStream` serialization.
+
+`write()` returns a promise that resolves only after the scheduled flush completes. Multiple writes within the 10ms window still share a single batched HTTP request (the batching optimization is preserved). Each write registers a `{resolve, reject}` pair in a `flushWaiters` array. When the `setTimeout` fires and `flush()` completes the HTTP round-trip, all waiters are resolved (or rejected on error). This makes `pendingOps` accurately reflect server-side data state while keeping network-efficient batching.
+
+### Lock-Release Polling Interval
+
+`flushablePipe`'s `pollWritableLock` / `pollReadableLock` use `setInterval` to detect when a user releases their stream lock without closing the stream — the Web Streams API has no event for that state. The V2 step executor's `opsSettled` race waits for this poll to resolve after each writable-bearing step body returns, so the polling interval sits on the critical path of every streaming step.
+
+The interval was lowered from 100ms to 10ms. Per-step wait drops from ~50ms average to ~5ms (scaling linearly with the number of writable-bearing steps in a workflow). Per-tick work is just `writable.locked` plus a `getWriter()`/`releaseLock()` probe — microsecond-scale, so 10× more ticks is not measurable in practice.
+
+**Follow-up**: Replace polling entirely with an event-driven release signal — wrap the writable returned from the `WritableStream` reviver with a writer that fires on `releaseLock()` — bringing the wait to ~0ms. This would also remove a source of timing drift between worlds with synchronous storage (`world-local`) and worlds with HTTP-deferred storage (`world-vercel`).
+
+### Concurrent `step_started` and Attempt Counter
+
+When the V2 handler dispatches N parallel steps as background messages, each background step completion queues a workflow continuation. Up to N continuations may replay concurrently, and each may attempt to start the same not-yet-completed step (since `step_started` succeeds for already-running steps). Each call atomically increments the `attempt` counter — so with N=5 parallel steps, the counter can reach 5 on the first genuine execution.
+
+The max retries check in `executeStep()` only enforces when `step.error` exists — distinguishing actual retries (failed → retry with error) from concurrent first-attempt races (multiple handlers start the same step simultaneously without any prior failure). Concurrent starts are harmless since `step_completed` idempotency ensures only the first completion wins.
+
+### Unconsumed Event Check Two-Phase Drain
+
+The `EventsConsumer`'s unconsumed event check uses a two-phase promise queue drain: yield once after the first drain (via `setTimeout(0)`) so cross-VM promise chains can append follow-up async work, then re-drain before checking. This improves timing for scenarios like `step_completed` → for-await loop resume → next hook hydration.
+
+The check additionally arms a `DEFERRED_CHECK_DELAY_MS = 100` `setTimeout` after the second drain, since Node.js does not guarantee that `setTimeout(0)` fires after all cross-context microtasks settle. Any `subscribe()` call arriving during that 100ms window cancels the check via version invalidation + `clearTimeout`, so the delay only adds latency to genuine corruption — never to the happy path.
+
+**Follow-up**: 100ms is a heuristic chosen empirically. A deterministic settlement signal — for example, a "VM idle" callback exposed by the workflow VM bridge that fires only after all pending cross-context promise chains have resolved — would let the consumer fire the unconsumed-event check immediately on quiescence instead of waiting for a wall-clock timeout.
+
+### Lazy World Loading
+
+Static imports of `world-local` and `world-vercel` from the runtime caused two distinct build-time issues: Next.js production builds pulled both worlds (including their Node-only deps like `debug`'s `tty` requires) into the route module, and Turbopack's NFT (Node File Trace) errored on `process.cwd()` and dynamic `import()` patterns it couldn't statically analyze.
+
+A `getWorldLazy()` accessor (backed by a `globalThis` `Symbol.for` cache) replaces the static import in step-side modules. This breaks the static import chain from step code to `world.ts`, preventing both worlds from being bundled into the step registrations.
+
+Because tree-shaking can otherwise drop `world.ts`'s module-load registration entirely, a server-only side-effect module (`@workflow/core/runtime/world-init`) imports `./world.js` purely for its module-load side effect. It's wired via package conditions:
+
+- `default` → real, loads `world.ts`
+- `workflow` → empty stub, used by VM/step bundles
+
+This guarantees the world is loaded for routes that consume `start()` without going through the queue-driven flow handler first, while keeping `world.ts` and its server-only deps out of the workflow sandbox bundle.
+
+### Community Worlds and the `world.streams` API
+
+Community world adapters must implement the `world.streams.*` interface. The runtime legacy stream normalization was removed as part of this work.

package/docs/changelog/index.mdx

@@ -12,4 +12,6 @@ Stay up to date with the latest changes to Workflow SDK.
 
 ## 2026
 
-- TBD
+- [Resilient hook resume](/docs/changelog/resilient-resume) — July 2026
+- [Eager processing of steps and incremental event replay](/docs/changelog/eager-processing) — March 2026
+- Serializable AbortController and AbortSignal — March 12, 2026

package/docs/changelog/lazy-event-creation.md

@@ -0,0 +1,127 @@
+---
+title: Lazy inline step start
+description: Defer step_created for the inline step and fold it into a single step_started, saving one world round-trip per inline step.
+---
+
+# Lazy inline step start
+
+## Motivation
+
+The owned-inline runtime path used to write two separate world events for a step it already owns and is about to run inline:
+
+1. `step_created` — written by the suspension handler (`suspension-handler.ts`)
+2. `step_started` — written by `executeStep` (`step-executor.ts`)
+3. `step_completed` / `step_failed` — written by `executeStep`
+
+On the Vercel world each `world.events.create` is a network round-trip, so for a simple sequential `"use step"` workflow this is pure latency between steps. Steps (1) and (2) are two round-trips for a step we already own and are about to execute in the same invocation.
+
+This change defers `step_created` for that one inline step: `executeStep` sends a single `step_started` carrying the step input, and the world creates the step on the fly — materializing the step entity **and** a synthetic `step_created` event so replay still observes it. **Two writes per inline step instead of three.** It mirrors the existing [resilient `run_started` → `run_created`](./resilient-start) pattern.
+
+Steps that are *queued* (not run inline) keep their eager `step_created` and are unchanged. Only the single inline step per suspension is made lazy.
+
+## Design
+
+### Suspension handler
+
+- `handleSuspension` selects exactly one step to defer — the first uncreated step (`stepItems.find(item => stepsNeedingCreation.has(...))`), which matches the inline candidate the caller would have picked.
+- For that step it **skips** the `step_created` write and instead returns it as `lazyInlineStep = { correlationId, stepName, dehydratedInput }`. It is **not** added to `createdStepCorrelationIds` — ownership is no longer decided here.
+- A `lazyInlineStep` is designated only when there is no `hook.getConflict()` awaiter (`hasAwaitedHookCreation === false`). With an awaiter present nothing runs inline, so nothing is deferred.
+
+### `executeStep`
+
+- A new `lazyStepInput` parameter carries the already-dehydrated input. When present, the `step_started` event includes `input` so the world creates the step on the fly. When absent, `step_started` carries only `stepName` (the legacy contract).
+- For an **unregistered** step on the lazy path, `executeStep` sends the lazy `step_started` first to materialize the step, *then* writes `step_failed` (see [Materialize before failing](#materialize-before-failing-unregistered-steps) below).
+
+### World contract
+
+- `step_started` accepts an optional `input`. When provided for a non-existent step, the world creates the step entity plus a **synthetic `step_created`** event (so the event log reads `created → started → completed`), then records `step_started` — atomically.
+- A `stepCreated` signal is added to the event result so callers can tell whether the lazy `step_started` created the step or attached to an existing one.
+- Worlds updated: `world-local`, `world-postgres`, `world-vercel`.
+
+## Interactions with inline execution
+
+### Exactly-one-owner is preserved (the race moved)
+
+The guarantee that exactly one handler runs a step's body inline is intact — it just resolves at a different event:
+
+- **Before:** ownership was won at the atomic `step_created` claim in the suspension handler; the loser caught `EntityConflictError` and queued instead.
+- **After:** two concurrent handlers may both *select* the same `lazyInlineStep` (selection is optimistic, before any race). The race is now the world's atomic create-claim inside the lazy `step_started` (lock file in world-local, `onConflictDoNothing` + unique index in world-postgres, `attribute_not_exists` on the server). The loser gets `EntityConflictError`, which `executeStep` maps to `{ type: 'skipped' }`, so it never runs the body.
+
+This is safe **because a lazy `step_started` is only ever sent for a brand-new step** — the suspension handler defers only steps with no prior `step_created` (`!hasCreatedEvent`).
+
+### Crash recovery is unchanged
+
+On crash recovery the step already has a `step_created` event in the log (`hasCreatedEvent === true`), so it is **not** a lazy candidate. It is re-queued and re-run via a normal **non-lazy** `step_started`, which re-starts a `running` step. At-least-once execution is preserved exactly as before.
+
+### Materialize before failing unregistered steps
+
+The inline path previously assumed an invariant: *by the time `executeStep` runs, the step entity already exists* (the suspension handler created it). The unregistered-step branch relied on this — it writes `step_failed` directly, with no preceding `step_started`.
+
+With the `step_created` deferred, the entity no longer exists when `executeStep` bails for an unregistered step, so the `step_failed` write hits the world's "step must exist" ordering guard and is rejected — wedging the run until it times out. The fix: on the lazy path, send the lazy `step_started` first (creating the entity + synthetic `step_created`), then write `step_failed`. The lazy `step_started`'s atomic create-claim still preserves exactly-one-owner — a concurrent winner makes our create reject with `EntityConflictError` → `skipped`, so the failure is never written twice.
+
+This is the general rule the deferral introduces: **any inline path that writes a terminal step event must first ensure the deferred step has been materialized.**
+
+### Replay correctness and the inline-delta fast path
+
+The client step consumer (`step.ts`) sets `hasCreatedEvent` only when it observes a `step_created` event, and checks step-name divergence against `stepName`. The lazy path stays replay-correct only because the world writes a **synthetic `step_created`**: replay still sees `created → started → completed`. The input lives on the synthetic `step_created`; the `step_started` row drops the input but keeps `stepName` for the divergence check.
+
+This intersects with the inline-delta optimization. The delta returned on the `step_completed` write is consumed by the *next* replay in place of an `events.list`, diffed against `preInlineWriteCursor` (snapshotted before replay). Because the synthetic `step_created`, `step_started`, and `step_completed` are all written *after* that cursor, the world's "events since cursor" delta carries the full triple — so the next replay does not diverge. The delta gate (one step, no hooks/waits, the lone pending step is the inline one) is unchanged.
+
+### Pre-emption by attributes / hook conflicts
+
+When `attr_set` events force an immediate in-process replay, or a hook conflict forces a re-invocation, the handler skips the dispatch loop for that pass. The deferred step is therefore **neither created nor queued** on that pass; it is recreated and run on the following replay (where it is still a lazy candidate).
+
+This is a small behavioral improvement: previously the eager `step_created` left an orphan "created but never started" event when a step lost an attribute/hook race (e.g. `Promise.race([setAttributes(), step()])` where the attribute write wins and completes the run). With deferral, a step that loses the race is never created at all — less event-log garbage.
+
+### `hook.getConflict()` awaiter
+
+When a `hook.getConflict()` awaiter is present, no `lazyInlineStep` is designated, nothing runs inline, every step gets its eager `step_created` and is queued, and the handler re-invokes immediately so replay resolves the awaiter. This is identical to the pre-change behavior — the deferral never serializes the awaiter's parallel continuation behind an inline step.
+
+## Rollout and compatibility
+
+Server-first. The matching world-vercel backend change must deploy before this ships; the Vercel world targets a single backend whose spec version is always at least the SDK's, so the new SDK only ever talks to an already-upgraded backend. An old SDK against a new backend is safe because the lazy path is strictly additive — it triggers only when `step_started` carries both `stepName` and `input`, which old SDKs never send. For `world-local` / `world-postgres` the world ships in the same package as the runtime, so there is no version skew. Detection is by `input` presence on the event, mirroring resilient `run_started` — no capability negotiation is needed.
+
+# Parallel inline steps + optimistic start
+
+A follow-up builds two more latency wins on top of lazy inline start. Both are client-side only — they reuse the world's lazy create-on-`step_started` support and need no further world/backend changes.
+
+## Inline up to N steps in parallel
+
+Previously the owned-inline path ran **exactly one** step inline per suspension and queued the rest. For a `Promise.all([stepA(), stepB(), stepC()])` fan-out that meant one branch ran inline while the others paid a queue round-trip each before making any progress.
+
+The suspension handler now defers `step_created` for up to **`WORKFLOW_MAX_INLINE_STEPS` (default 3)** steps and returns them as `lazyInlineSteps`. The runtime runs that batch inline **in parallel** (`Promise.all`), each via its own lazy `step_started`, and queues only the steps beyond the cap.
+
+- **Selection:** the first N uncreated steps, matching the previous single-step inline candidate. Steps beyond N keep their eager `step_created` and are queued exactly as before.
+- **Result aggregation:** `retry` steps (whose `step_started` succeeded, so the step exists) are re-queued per-step as background steps with their own delay. `throttled` steps are different: a throttle rejects the lazy `step_started` on the create-claim, so the step was *never created* and has no recoverable input — re-queuing it as an input-less background step would make the world reject the bare `step_started` with "Step not found" and redeliver until it fails. So any throttle instead **defers redelivery of the orchestrator** (by the longest throttle backoff in the batch), which re-runs the throttled step inline *with its input* on replay. The runtime only loops back to replay in-process once every inline step has reached a terminal state.
+- **Inline-delta fast path:** still used only for the single-step sequential case (`lazyInlineSteps.length === 1`). With more than one inline step each writes its own events, so a per-write delta would be partial; multi-step batches fall back to a normal incremental `events.list`.
+- **Config:** `WORKFLOW_MAX_INLINE_STEPS` is clamped to 1..16. Setting it to `1` reproduces the previous single-inline-step behavior exactly (a useful kill-switch). Inline bodies run in parallel within one function invocation, so the cap also bounds per-handler memory/CPU fan-out.
+
+## Optimistic inline start (opt-in, off by default)
+
+Normally `executeStep` awaits `step_started` (the lazy create-claim round-trip) before running the body. Because the inline path already holds the step input locally, it doesn't actually need that round-trip to begin.
+
+When `WORKFLOW_OPTIMISTIC_INLINE_START` is enabled (set it to `1`/`true` — it is **off by default**), an inline step fires `step_started` **without awaiting it** and starts running the body immediately against locally-synthesized state. A lazy step is always brand-new, so attempt is 1, there is no prior error, and `startedAt` is now — all known without the server. The in-flight `step_started` is reconciled just before the terminal write:
+
+- **Lost the create-claim (409 / `EntityConflictError`)** → discard the body result and return `skipped`; the winning handler owns the terminal write.
+- **Run gone / throttled / too-early** → discard the body result and surface `gone` / `throttled` / `retry`.
+- **Transient (non-translatable) failure** → propagate it, so the queue redelivers — exactly as the await path does today.
+- **Success** → write `step_completed` / `step_failed` / `step_retrying` as usual. Awaiting `step_started` before the terminal write keeps the event log ordered (`created → started → completed`).
+
+### Safety and the idempotency tradeoff
+
+- **Exactly-one terminal write is preserved.** Optimistic start changes only *when the body runs*, never who writes the terminal event — that is still gated by the lazy `step_started` create-claim, which is awaited before the terminal write. Losers return `skipped`.
+- **Bounded to attempt 1.** Only brand-new (`!hasCreatedEvent`) steps are lazy; a retried step already has a `step_created`, so it takes the normal await-then-run path with the real attempt counter. Synthesizing `attempt = 1` locally is therefore always correct.
+- **Wider double-execution — why it's off by default.** Running the body before confirming ownership means two handlers racing for the same step's create-claim can *both* run the side effects before either wins (previously the loser 409'd on `step_created` and skipped before running anything). This is unsafe for non-idempotent steps: in particular, two concurrent runs of a step that writes to the **workflow stream** (e.g. an AI agent streaming tokens) can interleave and **corrupt the stream data**. So the optimization ships **disabled**; enable it (`WORKFLOW_OPTIMISTIC_INLINE_START=1`) only for deployments whose inline step bodies are idempotent and stream-safe.
+
+## Queue messages: inline steps don't pay a round-trip
+
+Inline steps that **complete** never enqueue a per-step flow-route message. When every step in an inline batch reaches a terminal state with no pending background ops, the runtime simply continues its in-process loop and replays — so a sequential chain (or a clean parallel fan-out) of inline steps runs entirely within one invocation with **zero** queue messages. Verified: a workflow whose only work is three parallel inline steps issues no `queue()` calls.
+
+The only flow-route messages produced around an inline batch are:
+
+- **Pre-batch dispatch** — the steps *beyond* the inline cap (and any pending wait/sleep continuation). Inline steps are explicitly excluded from this dispatch.
+- **`retry` results** — one delayed message per retried step. A retry *is* the step becoming its own background invocation, so this is expected.
+- **`throttled` results** — a single deferral of the orchestrator message (see above).
+- **Pending background ops** — if any inline step left unflushed stream writes (e.g. output streams to blob storage), the loop breaks and enqueues **one** continuation (aggregated across the batch, not per-step) so `waitUntil` can flush before the next replay reads them.
+
+In other words: completed inline steps cost no queue round-trips; only steps that genuinely run as their own background invocations create new flow-route messages.

package/docs/changelog/meta.json

@@ -1,5 +1,13 @@
 {
   "title": "Changelog",
-  "pages": ["index", "eager-processing", "resilient-start"],
+  "pages": [
+    "index",
+    "eager-processing",
+    "resilient-resume",
+    "resilient-start",
+    "lazy-event-creation",
+    "turbo-mode",
+    "step-message-ownership"
+  ],
   "defaultOpen": false
 }

package/docs/changelog/resilient-resume.mdx

@@ -0,0 +1,22 @@
+---
+title: Resilient hook resume
+description: resumeHook() now tolerates transient event storage failures, as long as the queue is healthy, instead of failing the resume.
+---
+
+# Resilient `resumeHook()`
+
+## Motivation
+
+`resumeHook()` used to write the `hook_received` event and dispatch the workflow queue message strictly one after the other, so every resume paid two sequential round trips and a transient event-storage failure failed the whole resume even when the queue was healthy. This change runs both writes **concurrently** — cutting a round trip off resume latency — and, on the same path, brings `resumeHook()` to parity with [resilient `start()`](/docs/changelog/resilient-start): a transient event-write failure no longer fails the resume when the payload can still be delivered through the queue.
+
+## Design
+
+- On the fast path, `resumeHook()` writes the `hook_received` event and dispatches the workflow queue message **concurrently** (`Promise.allSettled`). The queue message carries a `hookInput` payload — the dehydrated hook payload plus a client-minted `resumeId` idempotency key, the hook token, and a payload digest.
+- A `(runId, resumeId)` dedup constraint keeps the two writers converging on **exactly one** `hook_received` event: whichever lands first wins, and the other is resolved server-side as success rather than a duplicate. The queue consumer idempotently re-ensures the event from `hookInput` before replay, so the resume is guaranteed even if the direct write never commits.
+- Replay also deduplicates: `hook_received` events sharing a `resumeId` belong to the same resume attempt, and only the first in the event log is delivered to workflow code. Even if a redelivery materializes the event twice, the payload reaches the workflow exactly once.
+- **Queue dispatch failure is fatal** — the run was not re-triggered, so no consumer will re-ensure the event, and `resumeHook()` throws. A transient event-write failure (429/5xx, a transport error, or an expected `(runId, resumeId)` conflict with the consumer's own re-ensure) is swallowed because the queue delivery still guarantees the resume; a terminal run surfaces as `HookNotFoundError`, and any other event-write error is rethrown.
+- `resumeHook()` returns `ResumedHook` (exported from `workflow/api`), which extends `Hook` with an optional `resilientResume` flag. The flag is `true` only when the direct write failed transiently and the resume was recovered through the queue; on the happy path and the sequential fallback it is absent.
+
+## Compatibility
+
+The parallel fast path is gated per resume: it activates only when both the target run's queue consumer and the live backend independently attest dedup support (re-checked on every resume, so rollout and rollback both degrade safely). Otherwise — for oversized payloads, legacy runs, or with `WORKFLOW_DISABLE_LAZY_HOOK_RESUME=1` — `resumeHook()` falls back to the original sequential write-then-dispatch path. Because runs keep executing on the deployment they were created on, a resume targeting a run from an older deployment simply uses the sequential path.