workflow 5.0.0-beta.5 → 5.0.0-beta.50

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (256) hide show
  1. package/README.md +68 -23
  2. package/dist/api-workflow.d.ts +1 -1
  3. package/dist/api-workflow.d.ts.map +1 -1
  4. package/dist/api-workflow.js +1 -1
  5. package/dist/api.d.ts +3 -3
  6. package/dist/api.d.ts.map +1 -1
  7. package/dist/api.js +5 -7
  8. package/dist/index.d.ts +1 -0
  9. package/dist/index.d.ts.map +1 -1
  10. package/dist/index.js +6 -1
  11. package/dist/internal/builtins.d.ts +20 -3
  12. package/dist/internal/builtins.d.ts.map +1 -1
  13. package/dist/internal/builtins.js +68 -4
  14. package/dist/internal/errors.d.ts +1 -1
  15. package/dist/internal/errors.d.ts.map +1 -1
  16. package/dist/internal/errors.js +2 -2
  17. package/dist/nest-builder.d.ts +2 -0
  18. package/dist/nest-builder.d.ts.map +1 -0
  19. package/dist/nest-builder.js +2 -0
  20. package/dist/nest-vercel-builder.d.ts +2 -0
  21. package/dist/nest-vercel-builder.d.ts.map +1 -0
  22. package/dist/nest-vercel-builder.js +2 -0
  23. package/dist/observability.d.ts +1 -1
  24. package/dist/observability.js +2 -2
  25. package/dist/runtime.d.ts +2 -1
  26. package/dist/runtime.d.ts.map +1 -1
  27. package/dist/runtime.js +4 -1
  28. package/docs/ai/chat-session-modeling.mdx +29 -26
  29. package/docs/ai/defining-tools.mdx +6 -7
  30. package/docs/ai/human-in-the-loop.mdx +11 -11
  31. package/docs/ai/index.mdx +50 -45
  32. package/docs/ai/message-queueing.mdx +16 -16
  33. package/docs/ai/meta.json +1 -0
  34. package/docs/ai/resumable-streams.mdx +40 -28
  35. package/docs/ai/sleep-and-delays.mdx +10 -10
  36. package/docs/ai/streaming-updates-from-tools.mdx +6 -6
  37. package/docs/api-reference/index.mdx +24 -0
  38. package/docs/api-reference/meta.json +8 -0
  39. package/docs/api-reference/vitest/index.mdx +9 -15
  40. package/docs/api-reference/workflow/create-hook.mdx +89 -10
  41. package/docs/api-reference/workflow/create-webhook.mdx +16 -15
  42. package/docs/api-reference/workflow/define-hook.mdx +35 -33
  43. package/docs/api-reference/workflow/fatal-error.mdx +30 -8
  44. package/docs/api-reference/workflow/fetch.mdx +14 -10
  45. package/docs/api-reference/workflow/get-step-metadata.mdx +2 -2
  46. package/docs/api-reference/workflow/get-workflow-metadata.mdx +3 -3
  47. package/docs/api-reference/workflow/get-writable.mdx +7 -7
  48. package/docs/api-reference/workflow/index.mdx +4 -1
  49. package/docs/api-reference/workflow/retryable-error.mdx +1 -1
  50. package/docs/api-reference/workflow/set-attributes.mdx +61 -0
  51. package/docs/api-reference/workflow/sleep.mdx +4 -4
  52. package/docs/api-reference/workflow-ai/durable-agent.mdx +48 -86
  53. package/docs/api-reference/workflow-ai/index.mdx +3 -3
  54. package/docs/api-reference/workflow-ai/workflow-chat-transport.mdx +67 -24
  55. package/docs/api-reference/workflow-api/get-hook-by-token.mdx +26 -12
  56. package/docs/api-reference/workflow-api/get-run.mdx +43 -8
  57. package/docs/api-reference/workflow-api/index.mdx +6 -10
  58. package/docs/api-reference/workflow-api/resume-hook.mdx +73 -12
  59. package/docs/api-reference/workflow-api/resume-webhook.mdx +11 -9
  60. package/docs/api-reference/workflow-api/start.mdx +60 -13
  61. package/docs/api-reference/workflow-astro/index.mdx +18 -0
  62. package/docs/api-reference/workflow-astro/meta.json +4 -0
  63. package/docs/api-reference/workflow-astro/workflow.mdx +45 -0
  64. package/docs/api-reference/workflow-errors/entity-conflict-error.mdx +4 -4
  65. package/docs/api-reference/workflow-errors/hook-conflict-error.mdx +60 -0
  66. package/docs/api-reference/workflow-errors/hook-not-found-error.mdx +8 -8
  67. package/docs/api-reference/workflow-errors/index.mdx +88 -0
  68. package/docs/api-reference/workflow-errors/meta.json +6 -0
  69. package/docs/api-reference/workflow-errors/precondition-failed-error.mdx +68 -0
  70. package/docs/api-reference/workflow-errors/run-expired-error.mdx +2 -2
  71. package/docs/api-reference/workflow-errors/run-not-supported-error.mdx +58 -0
  72. package/docs/api-reference/workflow-errors/step-not-registered-error.mdx +5 -5
  73. package/docs/api-reference/workflow-errors/throttle-error.mdx +2 -2
  74. package/docs/api-reference/workflow-errors/too-early-error.mdx +2 -2
  75. package/docs/api-reference/workflow-errors/workflow-error.mdx +52 -0
  76. package/docs/api-reference/workflow-errors/workflow-not-registered-error.mdx +5 -6
  77. package/docs/api-reference/workflow-errors/workflow-run-cancelled-error.mdx +6 -6
  78. package/docs/api-reference/workflow-errors/workflow-run-failed-error.mdx +5 -5
  79. package/docs/api-reference/workflow-errors/workflow-run-not-completed-error.mdx +58 -0
  80. package/docs/api-reference/workflow-errors/workflow-run-not-found-error.mdx +4 -4
  81. package/docs/api-reference/workflow-errors/workflow-runtime-error.mdx +58 -0
  82. package/docs/api-reference/workflow-errors/workflow-world-error.mdx +8 -8
  83. package/docs/api-reference/workflow-globals.mdx +14 -10
  84. package/docs/api-reference/workflow-nest/configure-workflow-controller.mdx +33 -0
  85. package/docs/api-reference/workflow-nest/index.mdx +31 -0
  86. package/docs/api-reference/workflow-nest/meta.json +9 -0
  87. package/docs/api-reference/workflow-nest/nest-local-builder.mdx +64 -0
  88. package/docs/api-reference/workflow-nest/workflow-controller.mdx +40 -0
  89. package/docs/api-reference/workflow-nest/workflow-module.mdx +74 -0
  90. package/docs/api-reference/workflow-next/with-workflow.mdx +39 -17
  91. package/docs/api-reference/workflow-nitro/index.mdx +60 -0
  92. package/docs/api-reference/workflow-nuxt/index.mdx +48 -0
  93. package/docs/api-reference/workflow-observability/hydrate-data.mdx +35 -0
  94. package/docs/api-reference/workflow-observability/hydrate-resource-io.mdx +62 -0
  95. package/docs/api-reference/workflow-observability/index.mdx +62 -0
  96. package/docs/api-reference/workflow-observability/meta.json +11 -0
  97. package/docs/api-reference/workflow-observability/observability-revivers.mdx +50 -0
  98. package/docs/api-reference/workflow-observability/parse-class-name.mdx +41 -0
  99. package/docs/api-reference/workflow-observability/parse-step-name.mdx +40 -0
  100. package/docs/api-reference/workflow-observability/parse-workflow-name.mdx +55 -0
  101. package/docs/api-reference/workflow-runtime/create-world.mdx +39 -0
  102. package/docs/api-reference/workflow-runtime/get-world-handlers.mdx +44 -0
  103. package/docs/api-reference/{workflow-api → workflow-runtime}/get-world.mdx +11 -14
  104. package/docs/api-reference/workflow-runtime/health-check.mdx +50 -0
  105. package/docs/api-reference/workflow-runtime/index.mdx +41 -0
  106. package/docs/api-reference/workflow-runtime/meta.json +12 -0
  107. package/docs/api-reference/workflow-runtime/set-world.mdx +51 -0
  108. package/docs/api-reference/workflow-runtime/workflow-entrypoint.mdx +43 -0
  109. package/docs/api-reference/workflow-runtime/world/analytics.mdx +315 -0
  110. package/docs/api-reference/workflow-runtime/world/index.mdx +60 -0
  111. package/docs/api-reference/workflow-runtime/world/meta.json +4 -0
  112. package/docs/api-reference/workflow-runtime/world/queue.mdx +88 -0
  113. package/docs/api-reference/{workflow-api → workflow-runtime}/world/storage.mdx +98 -34
  114. package/docs/api-reference/{workflow-api → workflow-runtime}/world/streams.mdx +8 -8
  115. package/docs/api-reference/workflow-serde/index.mdx +1 -2
  116. package/docs/api-reference/workflow-serde/workflow-deserialize.mdx +3 -4
  117. package/docs/api-reference/workflow-serde/workflow-serialize.mdx +8 -8
  118. package/docs/api-reference/workflow-sveltekit/index.mdx +18 -0
  119. package/docs/api-reference/workflow-sveltekit/meta.json +4 -0
  120. package/docs/api-reference/workflow-sveltekit/workflow-plugin.mdx +42 -0
  121. package/docs/api-reference/workflow-vite/index.mdx +18 -0
  122. package/docs/api-reference/workflow-vite/meta.json +4 -0
  123. package/docs/api-reference/workflow-vite/workflow.mdx +48 -0
  124. package/docs/changelog/attributes-mvp.mdx +380 -0
  125. package/docs/changelog/batched-event-writes.mdx +79 -0
  126. package/docs/changelog/eager-processing.mdx +110 -436
  127. package/docs/changelog/index.mdx +4 -2
  128. package/docs/changelog/lazy-event-creation.md +127 -0
  129. package/docs/changelog/lazy-hook-resume.mdx +78 -0
  130. package/docs/changelog/meta.json +11 -1
  131. package/docs/changelog/resilient-resume.mdx +32 -0
  132. package/docs/changelog/resilient-start.mdx +33 -285
  133. package/docs/changelog/step-message-ownership.mdx +360 -0
  134. package/docs/changelog/turbo-mode.md +87 -0
  135. package/docs/comparisons/index.mdx +66 -0
  136. package/docs/comparisons/meta.json +11 -0
  137. package/docs/comparisons/workflow-sdk-vs-aws-agentcore.mdx +55 -0
  138. package/docs/comparisons/workflow-sdk-vs-aws-step-functions.mdx +111 -0
  139. package/docs/comparisons/workflow-sdk-vs-cloudflare-workflows.mdx +71 -0
  140. package/docs/comparisons/workflow-sdk-vs-inngest.mdx +102 -0
  141. package/docs/comparisons/workflow-sdk-vs-temporal.mdx +123 -0
  142. package/docs/comparisons/workflow-sdk-vs-trigger-dev.mdx +104 -0
  143. package/docs/configuration/build-and-diagnostics.mdx +70 -0
  144. package/docs/configuration/cli-and-web-ui.mdx +241 -0
  145. package/docs/configuration/framework-options.mdx +165 -0
  146. package/docs/configuration/index.mdx +32 -0
  147. package/docs/configuration/meta.json +12 -0
  148. package/docs/configuration/runtime-tuning.mdx +376 -0
  149. package/docs/configuration/worlds.mdx +313 -0
  150. package/docs/cookbook/advanced/child-workflows.mdx +211 -264
  151. package/docs/cookbook/advanced/meta.json +6 -1
  152. package/docs/cookbook/advanced/publishing-libraries.mdx +65 -56
  153. package/docs/cookbook/advanced/serializable-steps.mdx +28 -20
  154. package/docs/cookbook/advanced/upgrading-workflows.mdx +199 -0
  155. package/docs/cookbook/agent-patterns/agent-cancellation.mdx +27 -19
  156. package/docs/cookbook/agent-patterns/durable-agent.mdx +14 -142
  157. package/docs/cookbook/agent-patterns/human-in-the-loop.mdx +30 -22
  158. package/docs/cookbook/common-patterns/batching.mdx +18 -14
  159. package/docs/cookbook/common-patterns/idempotency.mdx +41 -53
  160. package/docs/cookbook/common-patterns/rate-limiting.mdx +8 -4
  161. package/docs/cookbook/common-patterns/saga.mdx +23 -19
  162. package/docs/cookbook/common-patterns/scheduling.mdx +34 -22
  163. package/docs/cookbook/common-patterns/sequential-and-parallel.mdx +29 -25
  164. package/docs/cookbook/common-patterns/timeouts.mdx +26 -21
  165. package/docs/cookbook/common-patterns/webhooks.mdx +10 -6
  166. package/docs/cookbook/common-patterns/workflow-composition.mdx +30 -27
  167. package/docs/cookbook/index.mdx +22 -21
  168. package/docs/cookbook/integrations/ai-sdk.mdx +85 -47
  169. package/docs/cookbook/integrations/chat-sdk.mdx +50 -33
  170. package/docs/cookbook/integrations/sandbox.mdx +62 -45
  171. package/docs/deploying.mdx +95 -0
  172. package/docs/errors/abort-signal-timeout-in-workflow.mdx +16 -12
  173. package/docs/errors/corrupted-event-log.mdx +29 -18
  174. package/docs/errors/deployment-mismatch.mdx +71 -0
  175. package/docs/errors/fetch-in-workflow.mdx +11 -7
  176. package/docs/errors/hook-conflict.mdx +69 -13
  177. package/docs/errors/index.mdx +2 -36
  178. package/docs/errors/node-js-module-in-workflow.mdx +9 -5
  179. package/docs/errors/replay-divergence.mdx +27 -0
  180. package/docs/errors/run-expired.mdx +85 -0
  181. package/docs/errors/runtime-decryption-failed.mdx +77 -0
  182. package/docs/errors/serialization-failed.mdx +44 -12
  183. package/docs/errors/start-invalid-workflow-function.mdx +9 -5
  184. package/docs/errors/step-executed-multiple-times.mdx +23 -0
  185. package/docs/errors/step-not-registered.mdx +6 -6
  186. package/docs/errors/timeout-in-workflow.mdx +12 -8
  187. package/docs/errors/webhook-invalid-respond-with-value.mdx +18 -18
  188. package/docs/errors/webhook-response-not-sent.mdx +20 -16
  189. package/docs/errors/workflow-not-registered.mdx +5 -5
  190. package/docs/foundations/cancellation.mdx +31 -32
  191. package/docs/foundations/errors-and-retries.mdx +42 -11
  192. package/docs/foundations/hooks.mdx +64 -35
  193. package/docs/foundations/idempotency.mdx +244 -12
  194. package/docs/foundations/index.mdx +1 -23
  195. package/docs/foundations/meta.json +2 -1
  196. package/docs/foundations/serialization.mdx +21 -22
  197. package/docs/foundations/starting-workflows.mdx +106 -30
  198. package/docs/foundations/streaming.mdx +107 -59
  199. package/docs/foundations/versioning.mdx +263 -0
  200. package/docs/foundations/workflows-and-steps.mdx +9 -9
  201. package/docs/getting-started/astro.mdx +22 -18
  202. package/docs/getting-started/express.mdx +15 -11
  203. package/docs/getting-started/fastify.mdx +15 -11
  204. package/docs/getting-started/hono.mdx +15 -11
  205. package/docs/getting-started/index.mdx +10 -3
  206. package/docs/getting-started/meta.json +3 -1
  207. package/docs/getting-started/nestjs.mdx +87 -20
  208. package/docs/getting-started/next.mdx +22 -16
  209. package/docs/getting-started/nitro.mdx +22 -18
  210. package/docs/getting-started/nuxt.mdx +15 -11
  211. package/docs/getting-started/python.mdx +135 -40
  212. package/docs/getting-started/react-router/index.mdx +33 -0
  213. package/docs/getting-started/react-router/meta.json +5 -0
  214. package/docs/getting-started/react-router/v7.mdx +237 -0
  215. package/docs/getting-started/react-router/v8.mdx +232 -0
  216. package/docs/getting-started/sveltekit.mdx +20 -16
  217. package/docs/getting-started/tanstack-start.mdx +17 -13
  218. package/docs/getting-started/vite.mdx +15 -11
  219. package/docs/how-it-works/cancellation.mdx +63 -63
  220. package/docs/how-it-works/code-transform.mdx +83 -67
  221. package/docs/how-it-works/encryption.mdx +30 -26
  222. package/docs/how-it-works/event-sourcing.mdx +98 -34
  223. package/docs/how-it-works/framework-integrations.mdx +96 -337
  224. package/docs/how-it-works/understanding-directives.mdx +22 -22
  225. package/docs/internal/index.mdx +6 -4
  226. package/docs/internal/meta.json +6 -1
  227. package/docs/internal/nitro-native-build.mdx +38 -0
  228. package/docs/internal/nitro-web-ui.mdx +24 -0
  229. package/docs/internal/serializable-abort-controller.mdx +7 -7
  230. package/docs/meta.json +3 -2
  231. package/docs/observability/attributes.mdx +134 -0
  232. package/docs/observability/index.mdx +32 -10
  233. package/docs/observability/meta.json +1 -1
  234. package/docs/observability/retention.mdx +93 -0
  235. package/docs/observability/tracing.mdx +124 -0
  236. package/docs/testing/index.mdx +36 -36
  237. package/docs/testing/server-based.mdx +10 -10
  238. package/docs/whats-new.mdx +186 -0
  239. package/package.json +17 -14
  240. package/docs/api-reference/workflow-api/world/index.mdx +0 -58
  241. package/docs/api-reference/workflow-api/world/meta.json +0 -4
  242. package/docs/api-reference/workflow-api/world/observability.mdx +0 -164
  243. package/docs/api-reference/workflow-api/world/queue.mdx +0 -86
  244. package/docs/deploying/building-a-world.mdx +0 -251
  245. package/docs/deploying/index.mdx +0 -95
  246. package/docs/deploying/meta.json +0 -4
  247. package/docs/deploying/world/local-world.mdx +0 -84
  248. package/docs/deploying/world/meta.json +0 -4
  249. package/docs/deploying/world/postgres-world.mdx +0 -224
  250. package/docs/deploying/world/vercel-world.mdx +0 -179
  251. package/docs/migration-guides/index.mdx +0 -34
  252. package/docs/migration-guides/meta.json +0 -9
  253. package/docs/migration-guides/migrating-from-aws-step-functions.mdx +0 -363
  254. package/docs/migration-guides/migrating-from-inngest.mdx +0 -314
  255. package/docs/migration-guides/migrating-from-temporal.mdx +0 -318
  256. package/docs/migration-guides/migrating-from-trigger-dev.mdx +0 -337
@@ -1,21 +1,20 @@
1
1
  ---
2
- title: Eager Processing of Steps & Incremental Event Replay
3
- description: Combine workflow event replay and step bundles to do work inline where possible, only deferring to queue for parallelism
2
+ title: Eager processing of steps and incremental event replay
3
+ description: Combine workflow event replay and step bundles to do work inline where possible, only deferring to the queue for parallelism.
4
4
  type: overview
5
5
  ---
6
6
 
7
- # Eager Processing of Steps & Incremental Event Replay
7
+ # Eager processing of steps and incremental event replay
8
8
 
9
9
  **Date**: March 2026
10
10
 
11
- This is a major internal architecture change to how Workflow DevKit executes workflows and steps on the Vercel platform. It reduces function invocations and queue overhead by executing steps _inline_ within the same function invocation as the workflow replay, rather than dispatching every step to a separate function via the queue.
11
+ This is a major internal architecture change to how Workflow DevKit executes workflows and steps. It reduces function invocations and queue overhead by executing steps _inline_ within the same function invocation as the workflow replay, rather than dispatching every step to a separate function via the queue.
12
12
 
13
- ## Previous Architecture
13
+ ## Previous architecture
14
14
 
15
- The previous architecture used two separate routes,
16
- each backed by its own queue trigger:
15
+ The previous architecture used two separate routes, each backed by its own queue trigger:
17
16
 
18
- ```
17
+ ```text
19
18
  Queue: __wkf_workflow_* --> /.well-known/workflow/v1/flow (workflow replay in VM)
20
19
  |
21
20
  suspension (step needed)
@@ -31,15 +30,15 @@ Queue: __wkf_step_* --> /.well-known/workflow/v1/step (step execution in
31
30
  (cycle repeats for each step)
32
31
  ```
33
32
 
34
- Each step required **2 queue messages** (step invoke + workflow continuation) and **2 function invocations**, plus cold start overhead for each. A serial workflow with 10 steps needed ~21 function invocations.
33
+ Each step required **2 queue messages** (step invocation and workflow continuation) and **2 function invocations**, plus cold-start overhead for each. A serial workflow with 10 steps needed approximately 21 function invocations.
35
34
 
36
- ## New Architecture
35
+ ## New architecture
37
36
 
38
37
  The two routes are merged into a single handler at `/.well-known/workflow/v1/flow` using `workflowEntrypoint()`. The step route is no longer generated.
39
38
 
40
39
  The handler runs an inline execution loop:
41
40
 
42
- ```
41
+ ```text
43
42
  receive queue message
44
43
  |
45
44
  +-- if message has stepId+stepName: execute that step, queue workflow continuation, exit
@@ -69,120 +68,66 @@ suspension with pending operations
69
68
 
70
69
  A serial workflow with 10 steps now completes in **1 function invocation**.
71
70
 
72
- ## Inline Step Execution
73
-
74
- After the workflow suspends with pending steps, the handler executes one step inline:
75
-
76
- 1. Create `step_started` event
77
- 2. Hydrate step input from the event log
78
- 3. Look up the step function via `getStepFunction(stepName)`
79
- 4. Execute the step function
80
- 5. Create `step_completed` or `step_failed` event
81
- 6. Loop back to workflow replay
82
-
83
- This logic lives in `executeStep()` in `packages/core/src/runtime/step-executor.ts`.
71
+ ## Background steps (parallel execution)
84
72
 
85
- ## Background Steps (Parallel Execution)
73
+ When a workflow suspends with multiple pending steps (for example, from `Promise.all`), the handler creates `step_created` events for all of them, queues N-1 back to `__wkf_workflow_*` with `stepId` and `stepName` in the message payload, executes 1 step inline, and loops back to replay.
86
74
 
87
- When a workflow suspends with multiple pending steps (e.g., from `Promise.all`), the handler:
75
+ Each background step message is handled by a separate function invocation of the same handler. When a message arrives with `stepId` and `stepName`, the handler executes that specific step, then checks if all parallel steps from the batch are done by comparing `step_created` events against terminal events (`step_completed`/`step_failed`):
88
76
 
89
- 1. Creates `step_created` events for all pending steps
90
- 2. Queues N-1 steps back to `__wkf_workflow_*` with `stepId` and `stepName` in the message payload
91
- 3. Executes 1 step inline
92
- 4. Loops back to replay
93
-
94
- Each background step message is handled by a separate function invocation of the same handler. When a message arrives with `stepId` and `stepName`, the handler executes that specific step, then checks if all parallel steps from the batch are done by loading the event log and comparing `step_created` events against terminal events (`step_completed`/`step_failed`):
95
-
96
- - **All steps done**: The handler replays the workflow inline, continuing the execution loop without a queue roundtrip. Events are loaded with a cursor so subsequent loop iterations can use incremental loading.
77
+ - **All steps done**: The handler replays the workflow inline, continuing the execution loop without a queue roundtrip.
97
78
  - **Steps still pending**: The handler returns without queuing a continuation. The last handler to complete its step will see all steps done and replay inline.
98
- - **Pending ops (stream writes)**: The handler queues a continuation and returns, so `waitUntil` can flush the pending stream data to the server.
79
+ - **Pending ops (stream writes)**: The handler queues a continuation and returns, so `waitUntil` can flush the pending stream data.
99
80
 
100
- ### Convergence After Parallel Steps
81
+ ### Convergence after parallel steps
101
82
 
102
- When multiple background steps complete near-simultaneously, multiple handlers may observe "all steps done" and attempt to advance the workflow concurrently. The event-sourced architecture plus the invariants described below ensure safe convergence:
83
+ When multiple background steps complete near-simultaneously, multiple handlers may observe "all steps done" and attempt to advance the workflow concurrently. The event-sourced architecture plus the invariants below ensure safe convergence:
103
84
 
104
- - **`step_created` idempotency** --- duplicate creates return 409; exactly one handler owns each step
105
- - **`step_completed` / `step_failed` idempotency** --- only the first invocation to record a terminal result wins
106
- - **Queue idempotency keys** --- background step messages use `correlationId` as idempotency key
107
- - **Deterministic replay** --- all invocations produce the same result given the same event log
85
+ - **`step_created` idempotency**: Duplicate creates return 409; exactly one handler owns each step.
86
+ - **`step_completed` / `step_failed` idempotency**: Only the first invocation to record a terminal result wins.
87
+ - **Queue idempotency keys**: Background step messages use `correlationId` as the idempotency key.
88
+ - **Deterministic replay**: All invocations produce the same result given the same event log.
108
89
 
109
- ### Single Inline Executor Per Step
90
+ ### Single inline executor per step
110
91
 
111
- Inline step execution combined with background-step dispatch introduces a new coordination requirement that the V1 handler did not face: when multiple handlers reach the same `Promise.all` batch concurrently, we need to guarantee that each step body runs at most once via the inline path. Without that guarantee, the event log accumulates duplicate `step_started` events (including some written *after* `step_completed`, which orphans them on replay) and step bodies run redundantly.
92
+ Inline step execution combined with background-step dispatch introduces a new coordination requirement: when multiple handlers reach the same `Promise.all` batch concurrently, we need to guarantee that each step body runs at most once via the inline path. Without that guarantee, the event log accumulates duplicate `step_started` events (including some written *after* `step_completed`, which orphans them on replay) and step bodies run redundantly.
112
93
 
113
- The design enforces a simple invariant: **exactly one handler owns each step, and only the owner may execute it inline**. Ownership is established by the atomicity of `step_created`:
94
+ The design enforces one invariant: **exactly one handler owns each step, and only the owner may execute it inline**. Ownership is established by the atomicity of `step_created`:
114
95
 
115
- 1. **Atomic `step_created`** --- the world's `events.create('step_created', correlationId=X)` is serialized per-correlationId. Exactly one concurrent caller succeeds; the rest receive `EntityConflictError`. In production worlds (Postgres, Vercel) this is enforced at the SQL/DB layer. In `world-local`, a per-step in-process async mutex in `packages/world-local/src/storage/events-storage.ts` wraps every step lifecycle event's check-and-write so the same guarantee holds for dev.
116
- 2. **Suspension handler reports ownership** --- `handleSuspension()` returns `createdStepCorrelationIds: Set<string>`, populated only for `step_created` writes that actually succeeded (not those that caught 409).
117
- 3. **Inline execution is gated on ownership** --- the runtime loop in `packages/core/src/runtime.ts` picks its inline step from `pendingSteps.filter(s => createdStepCorrelationIds.has(s.correlationId))`. A handler that didn't win any `step_created` race performs no inline execution.
118
- 4. **Queueing is unconditional** --- for every pending step except the one being inline-executed, the handler enqueues a background step message with `idempotencyKey: correlationId`. This matches V1's enqueue pattern and is what makes crash recovery work: if a prior handler wrote `step_created` but crashed before enqueueing, a later handler (from flow-message redelivery or `reenqueueActiveRuns`) will enqueue the orphaned step. Concurrent handlers' redundant enqueues dedupe on the idempotency key.
96
+ 1. **Atomic `step_created`**: `events.create('step_created', correlationId=X)` is serialized per correlation ID in every world. Exactly one concurrent caller succeeds; the rest receive `EntityConflictError`.
97
+ 2. **Suspension handler reports ownership**: Only `step_created` writes that succeeded (not those that caught 409) count toward ownership.
98
+ 3. **Inline execution is gated on ownership**: A handler that didn't win any `step_created` race performs no inline execution.
99
+ 4. **Queueing is unconditional**: For every pending step except the one being executed inline, the handler enqueues a background step message with `idempotencyKey: correlationId`. This makes crash recovery work: if a prior handler wrote `step_created` but crashed before enqueueing, a later handler will enqueue the orphaned step. Concurrent handlers' redundant enqueues deduplicate on the idempotency key.
119
100
 
120
- Together these give: every `step_created` event has exactly one inline executor (possibly zero if the inline path was skipped due to crash) **and** at least one queued dispatch (from whichever handler first reaches the suspension path after the `step_created` is visible). Step bodies are never executed concurrently, and `step_started` events never land in the log after `step_completed` for the same step. Event-log replay sees clean subscriber-matched sequences. With this invariant in place, the earlier `onUnconsumedEvent` skip logic for step/hook/wait lifecycle events was removed — any unconsumed event now immediately fatals as a corrupted event log (its original purpose before the V2 work).
101
+ Together these give: every `step_created` event has exactly one inline executor **and** at least one queued dispatch. Step bodies are never executed concurrently, and `step_started` events never land in the log after `step_completed` for the same step.
121
102
 
122
- **Retry semantics are preserved**: the per-step mutex in `world-local` only rejects `step_started` when the step is already in a *terminal* state (`completed` / `failed`). A step that is currently running (status=`running`) still accepts a second `step_started` write with an incremented attempt counter — this is how queue redelivery after a SIGKILL mid-execution legitimately re-runs the step. The previously-documented "attempt counter inflation" failure mode is therefore no longer reachable via the concurrent-inline path; see "Concurrent `step_started` Inflating Attempt Counter" below for the complementary executor-side guard that still catches edge cases (e.g., postgres retries under high contention).
103
+ **Retry semantics are preserved**: a step that is currently running (status=`running`) still accepts a second `step_started` write with an incremented attempt counter: this is how queue redelivery after a SIGKILL mid-execution legitimately re-runs the step.
123
104
 
124
- ## Incremental Event Loading
105
+ ## Incremental event loading
125
106
 
126
107
  The handler caches the event log in memory across loop iterations. Instead of re-fetching the entire event log on each replay:
127
108
 
128
- 1. **First iteration**: full load via `getAllWorkflowRunEventsWithCursor()`, which returns both the events and the final pagination cursor
129
- 2. **Subsequent iterations**: `getNewWorkflowRunEvents(runId, cursor)` fetches only events created after the saved cursor and appends them to the cached array
109
+ 1. **First iteration**: Load all events and return the final pagination cursor.
110
+ 2. **Subsequent iterations**: Fetch only events created after the saved cursor and append them to the cached array.
130
111
 
131
112
  For a 10-step serial workflow completing in one invocation, the 10th replay loads ~2 new events instead of re-fetching all ~30.
132
113
 
133
- ### Server-Side Cursor Fix
134
-
135
- The incremental loading depends on the server returning a cursor even on the final page of results (`hasMore: false`). Previously, `workflow-server` returned `cursor: null` when there were no more pages. This was fixed in the `peter/fix-end-cursor` branch to always return an `eid:<eventId>` cursor when there are events, aligning with `world-local` and `world-postgres` behavior.
136
-
137
- If a World implementation does not return a cursor after the initial load, the handler logs an error and falls back to a full reload.
138
-
139
- ## Timeout Handling
114
+ Incremental loading depends on the World returning a cursor even on the final page of results. If a World implementation does not return a cursor after the initial load, the handler logs an error and falls back to a full reload.
140
115
 
141
- The inline execution loop checks wall-clock time before each replay iteration. If the elapsed time exceeds a configurable threshold (default: 110 seconds, for a 120-second function limit), the handler re-schedules itself via the queue and returns.
116
+ ## Timeout handling
142
117
 
143
- The threshold is configurable via the `WORKFLOW_V2_TIMEOUT_MS` environment variable.
118
+ The inline execution loop checks wall-clock time before each replay iteration. If the elapsed time exceeds a configurable threshold (110s by default for a 120-second function limit), the handler reschedules itself through the queue and returns. Configure the threshold with `WORKFLOW_V2_TIMEOUT_MS`.
144
119
 
145
- If a single step takes longer than the timeout threshold, the step runs to completion (or SIGKILL) — there is no interruption mechanism for in-progress step execution. This is the same behavior as the previous architecture.
120
+ If a single step takes longer than the timeout threshold, the step runs to completion (or SIGKILL). There is no interruption mechanism for in-progress step execution. This is the same behavior as the previous architecture.
146
121
 
147
- ## Queue Message Changes
122
+ ## Queue message changes
148
123
 
149
- The `WorkflowInvokePayload` schema has two new optional fields:
150
-
151
- {/*@skip-typecheck - snippet, not runnable code*/}
152
-
153
- ```typescript
154
- stepId: z.string().optional()
155
- stepName: z.string().optional()
156
- ```
157
-
158
- When `stepId` is present, the handler executes that specific step before (or instead of) replaying the workflow. Background steps are queued with both `stepId` and `stepName` set, so the handler knows which step function to call without loading the event log. Previously, `stepName` was resolved by loading all events and searching for the `step_created` event matching the `stepId` — an O(N) operation on the full event history for every background step arrival.
124
+ The `WorkflowInvokePayload` schema has two new optional fields: `stepId` and `stepName`. When `stepId` is present, the handler executes that specific step before (or instead of) replaying the workflow. Background steps are queued with both set, so the handler knows which step function to call without loading the event log. Previously, `stepName` was resolved by loading all events and searching for the `step_created` event matching the `stepId`, an O(N) operation on the full event history for every background step arrival.
159
125
 
160
126
  The queue trigger configuration uses `WORKFLOW_QUEUE_TRIGGER` on the `__wkf_workflow_*` topic. The `__wkf_step_*` topic and its separate trigger are no longer generated.
161
127
 
162
- ## Builder Changes
163
-
164
- ### Base Builder
165
-
166
- New method `createCombinedBundle()` in `packages/builders/src/base-builder.ts`:
167
-
168
- 1. Builds the step registrations bundle (same esbuild + SWC step mode as before)
169
- 2. Builds the workflow VM code string (same esbuild + SWC workflow mode as before)
170
- 3. Generates a combined route file that imports the step registrations and uses `workflowEntrypoint(workflowCode)`
171
-
172
- No changes to the SWC plugin were needed. The two-pass build approach (separate step and workflow SWC modes) still applies.
173
-
174
- ### Framework Builders
175
-
176
- All framework builders were updated to use `createCombinedBundle()`:
128
+ ## Generated file layout
177
129
 
178
- - **Next.js** (eager and deferred/lazyDiscovery): replaces separate step + flow route generation
179
- - **NestJS, Nitro, Standalone**: replaces separate `createStepsBundle()` + `createWorkflowsBundle()` calls
180
- - **SvelteKit, Astro**: same, plus post-processing regex updated to match `workflowEntrypoint`
181
- - **Vercel Build Output API** (used by Nitro/Astro production): single `flow.func/` with `WORKFLOW_QUEUE_TRIGGER`
182
-
183
- ### Generated File Layout
184
-
185
- ```
130
+ ```text
186
131
  .well-known/workflow/v1/
187
132
  flow/
188
133
  route.js # Handler (workflowEntrypoint)
@@ -196,400 +141,129 @@ All framework builders were updated to use `createCombinedBundle()`:
196
141
 
197
142
  The `step/` directory is no longer generated.
198
143
 
199
- ## Suspension Handler
200
-
201
- `handleSuspension()` in `packages/core/src/runtime/suspension-handler.ts` creates events for all pending operations (hooks, step events, wait events) but does **not** queue step messages. It returns the pending step items so the handler can decide which to execute inline vs. queue to background.
202
-
203
- ## Concerns and Edge Cases
204
-
205
- ### Parent→Child Polling Holds Worker Slots
206
-
207
- `Run#returnValue` is implemented as a polling step: the workflow awaits the child run's terminal status inside a step body. In worker-based worlds (notably `world-postgres`), each such poll occupies a queue worker slot until the child run finishes. Parent workflows that fan out to many child runs — recursive workflows like `fibonacciWorkflow` are the obvious case — can therefore consume a large fraction of available workers just holding positions in `Promise.all([...children.map(c => c.returnValue)])`.
208
-
209
- If `queueConcurrency` is smaller than the peak number of concurrent parent polls plus the workers needed for any in-flight children, the system deadlocks: every slot is held by a parent waiting on a child, but no child can acquire a slot to start. An earlier iteration of this work mitigated the deadlock runtime-side by detecting step context (via `contextStorage.getStore()`) and throwing `TooEarlyError` to re-enqueue the polling step, freeing the worker. That runtime guard has been removed — the responsibility now lies with worker-pool sizing.
210
-
211
- For `world-postgres`, the default `queueConcurrency` is set to **50**, which is comfortably above the ~24 concurrent polls `fibonacciWorkflow(6)` produces at peak. Workflows that fan out more aggressively must raise this ceiling. `packages/core/src/runtime/run.ts`, the `queueConcurrency` option on `createWorld()` for `world-postgres`, and the `fibonacciWorkflow` fixture in `workbench/example/workflows/99_e2e.ts` all carry pointers to this caveat.
212
-
213
- **Follow-up**: Replace the worker-pool sizing requirement with a polling design that does not occupy a worker slot. Options under consideration: (a) restore a runtime-side `TooEarlyError` re-enqueue path but make it visible to the user (rather than the silent guard the earlier iteration shipped), (b) move child-completion polling out of the step body into the suspension layer so a parent waiting on a child does not consume queue capacity at all, or (c) emit a `run_completed` notification on the parent's stream/queue so the parent only resumes when the child actually finishes. The current `queueConcurrency=50` default is a workaround, not a long-term answer — workflows with deep recursion or large fan-out can still exhaust workers regardless of how high we set the ceiling.
214
-
215
- ### VM Sandboxing
216
-
217
- Workflow code still runs in a Node.js VM for determinism and sandboxing. Step code runs in the Node.js host context. The only change is that both happen within the same function invocation.
218
-
219
- ### Bundle Size and Cold Start
220
-
221
- The combined bundle is larger (contains both step code and workflow VM code). Cold start time increases slightly. The reduction in total function invocations more than compensates.
222
-
223
- ### Step Retries
224
-
225
- When an inline step fails with retries remaining:
226
-
227
- - `RetryableError` with explicit `retryAfter` delay: re-queue to self with `stepId` and delay
228
- - Transient errors with immediate retry: re-queue to self with `stepId` (delay = 1s)
229
- - `FatalError`: fail immediately
144
+ ## Design notes and tradeoffs
230
145
 
231
- ### Mixed Suspensions
146
+ ### Parent→child polling holds worker slots
232
147
 
233
- A suspension may contain steps, hooks, and waits simultaneously. The handler creates events for all, then chooses between inline execution and queue dispatch:
148
+ `Run#returnValue` is implemented as a polling step: the workflow awaits the child run's terminal status inside a step body. In worker-based Worlds (notably `world-postgres`), each such poll occupies a queue worker slot until the child run finishes. Parent workflows that fan out to many child runs, such as recursive workflows like `fibonacciWorkflow`, can therefore consume a large fraction of available workers while holding positions in `Promise.all([...children.map(c => c.returnValue)])`.
234
149
 
235
- - **Steps only** (no waits): one owned step is executed inline; the rest are queued. The loop continues after the inline step completes.
236
- - **Steps + at least one wait**: every step is queued (no inline execution). The handler returns with the wait timeout. Whichever lands first — a step's continuation or the wait timer — drives the next replay.
237
- - **Hooks / waits only**: handler returns with the wait timeout (or no timeout, for hook-only suspensions). The next continuation is driven by external resume or the wait timer.
150
+ If `queueConcurrency` is smaller than the peak number of concurrent parent polls plus the workers needed for any in-flight children, the system deadlocks: every slot is held by a parent waiting on a child, but no child can acquire a slot to start.
238
151
 
239
- The "no inline when there's a wait" carve-out is necessary to preserve `Promise.race(step, sleep)` semantics. Inline `await executeStep(...)` blocks the handler for the full step duration, and `wait_completed` events are only created on the *next* loop iteration's "complete elapsed waits" pass — so a longer-running step would always swallow the shorter sleep and `Promise.race` would resolve incorrectly. Queueing the step in this case lets the wait timer drive a continuation in parallel, matching V1's behavior where each step ran in a separate function invocation.
152
+ For `world-postgres`, the default `queueConcurrency` is set to **50**. Workflows that fan out more aggressively must raise this ceiling.
240
153
 
241
- Pure step suspensions (without waits) still benefit from inline execution; the carve-out only costs an extra queue roundtrip when a step and a sleep coexist.
154
+ To prevent deadlock when polling is executed inline by the step executor, `Run#pollReturnValue()` detects when it's running inside a step executor and throws `TooEarlyError` instead of polling in a blocking loop. The step executor handles `TooEarlyError` by re-queueing the step with a 1-second delay, freeing the worker. Unlike `RetryableError`, `TooEarlyError` does NOT count against `maxRetries`, so polling steps can retry indefinitely until the child completes.
242
155
 
243
- ### Hook Conflicts
156
+ **Follow-up**: Replace the worker-pool sizing requirement with a polling design that does not occupy a worker slot. Options under consideration: moving child-completion polling out of the step body into the suspension layer, or emitting a `run_completed` notification on the parent's stream/queue so the parent only resumes when the child actually finishes.
244
157
 
245
- If a hook conflict is detected during suspension handling, the handler breaks the loop and returns `{ timeoutSeconds: 0 }` for immediate re-invocation, same as the previous behavior.
246
-
247
- ### Encryption Key Resolution
248
-
249
- Encryption keys are resolved once before the inline execution loop starts (after the run status is confirmed as `running`) and reused across all iterations. Background step executions resolve the key independently. The key does not change within a run.
250
-
251
- ## Framework Support
252
-
253
- All framework integrations have been updated: Next.js (eager and deferred/lazyDiscovery), NestJS, SvelteKit, Astro, Nitro/Nuxt/Hono/Express/Vite, and CLI standalone. The Vercel Build Output API builder (used by Nitro and Astro for production deploys) also uses the combined bundle with `WORKFLOW_QUEUE_TRIGGER`.
254
-
255
- ## Non-Next.js Integration Challenges
256
-
257
- ### Module Scope Duplication in Re-Bundled Output
258
-
259
- Builders that use `bundleFinalOutput: true` (standalone CLI, Vercel Build Output API, NestJS) produce a single file where esbuild re-bundles the step registrations and the workflow runtime together. esbuild creates isolated module scopes for each source module, even within the same output file. This meant `registerStepFunction` and `getStepFunction` operated on different `Map` instances — steps were registered into one Map but looked up from another.
260
-
261
- **Fix**: The step function registry (`registeredSteps` Map in `@workflow/core/private`) and the step context storage (`contextStorage` AsyncLocalStorage in `@workflow/core/step/context-storage`) were changed from module-scoped variables to `globalThis` singletons using `Symbol.for`. This ensures all esbuild module scopes share the same instances. The pattern was already used in the codebase for the World singleton and the class serialization registry.
262
-
263
- ### Workflow Package CJS Export Condition
264
-
265
- The `workflow` package's root export has `"require": "./dist/typescript-plugin.cjs"` for TypeScript editor plugin loading. When esbuild bundles with CJS format, it resolves `import { defineHook } from 'workflow'` via the `require` condition, getting the TS plugin instead of the API.
266
-
267
- **Fix**: Added a `"node"` condition (`"node": "./dist/index.js"`) before the `"require"` condition in the workflow package's exports. esbuild with `conditions: ['node']` matches `"node"` first and uses the correct API entry. TypeScript's plugin loader doesn't use `conditions: ['node']`, so it still falls through to `"require"` for the TS plugin.
268
-
269
- ### Local World Concurrent Replay Interference
270
-
271
- The local development world (`world-local`) processes queue messages with high concurrency (default: 1000). With the V2 combined handler, parallel steps generate multiple workflow continuation messages. When these are processed concurrently, each triggers a replay that sees in-flight events from other concurrent replays. This causes "unconsumed event" errors because the event consumer encounters events that don't match any subscriber in the current replay state.
272
-
273
- In production (Vercel), this doesn't happen — each function invocation is isolated with its own event loading.
274
-
275
- **Fix**: The `EventsConsumer`'s `onUnconsumedEvent` callback (see "Concurrent Replay Interference with Multi-Batch Workflows" below) handles the concurrent event visibility issue. The V2 inline replay optimization (where the last background step to complete replays inline instead of queuing) further reduces concurrent replays. Redundant step executions from concurrent handlers are harmless due to `step_completed` idempotency — only the first completion wins.
276
-
277
- ### ESM `bundleFinalOutput` and Dynamic Require Errors
278
-
279
- When `bundleFinalOutput: true` is used with ESM format, esbuild bundles CJS dependencies (like `debug`) into the output. CJS `require()` calls are wrapped in esbuild's `__require` polyfill, which throws "Dynamic require of X is not supported" in ESM contexts where `require` is undefined. This affected all ESM-based framework builders (Nitro, NestJS, SvelteKit, Astro) that were switched to `bundleFinalOutput: true` during the V2 migration.
280
-
281
- **Fix**: ESM builders use `bundleFinalOutput: false` with `externalizeNonSteps: true`, matching the pre-V2 behavior. The framework's own bundler (Vite, Rollup, Turbopack) handles dependency resolution. The standalone CLI and Vercel Build Output API builders use `bundleFinalOutput: true` with ESM output plus a `createRequire(import.meta.url)` banner (see "V2 Combined Bundle Switched from CJS to ESM" below) so CJS dependencies can still call `require()` for Node.js builtins.
282
-
283
- ### Rollup Tree-Shaking of Step Registrations
284
-
285
- When `bundleFinalOutput: false` is used with Nitro's rollup pipeline, the step registrations bundle (`steps.mjs`) only contains side-effect code (`registerStepFunction` calls) with no exports. Rollup tree-shakes the entire module because it has no used exports, removing all step registrations from the production bundle. This causes "Step not found" errors at runtime.
286
-
287
- **Fix**: The steps bundle now exports a sentinel value (`export const __steps_registered = true`), and the combined route file imports it (`import { __steps_registered } from './steps.mjs'`). This gives rollup a used binding to track, preventing it from dropping the module and its side effects.
288
-
289
- ### Concurrent Replay Interference with Multi-Batch Workflows (historic)
290
-
291
- An earlier iteration of the V2 work hit "Unconsumed event in event log" errors when multiple concurrent handlers raced into the same batch boundary. The diagnosis at the time was that concurrent handlers could see events the current replay hadn't reached yet, and the mitigation was a skip path in `onUnconsumedEvent` that tolerated step/hook/wait lifecycle events whose correlationId had a matching `step_created` / `hook_created` / earlier `wait_completed` in the log.
292
-
293
- Later work on the "Single Inline Executor Per Step" invariant (described above) identified the actual root cause: duplicate `step_started` events were being written *after* `step_completed` on the same step, because the local world's `step_started` was not atomic w.r.t. terminal state and the main loop was re-picking already-queued steps for inline execution. Fixing those at the source (per-step mutex in `world-local` + ownership-gated inline dispatch + unconditional queueing with idempotency keys) eliminated the unconsumed-step-event path entirely, and fixing the `wait_completed` cursor bug (the main loop manually pushed `wait_completed` events without advancing `eventsCursor`, so the next incremental fetch re-returned them as local-array duplicates) eliminated the wait case.
294
-
295
- **Current state**: the `onUnconsumedEvent` skip logic has been removed. Any unconsumed event now fatals the run with `CORRUPTED_EVENT_LOG`, matching the original contract from PR #1055. Incremental event loading in `runtime.ts` dedupes by `eventId` to tolerate any residual manual pushes.
296
-
297
- ### Stale V1 Artifacts in Build Caches
298
-
299
- SvelteKit and Astro's build caches (including Vercel's) may preserve the old V1 `step/` route directory from previous builds. When the V2 builder runs, it no longer generates step routes, but the stale files remain and cause build failures (e.g., importing the removed `stepEntrypoint`). Additionally, SvelteKit's `beforeExit` hook that patches `.vc-config.json` files for Vercel deployments was still trying to configure the non-existent `step.func/` directory.
300
-
301
- **Fix**: SvelteKit and Astro builders now clean up stale V1 step route directories during build. SvelteKit's Vercel deployment hook was updated to only configure the combined `flow.func/` directory.
302
-
303
- ### Next.js Canary Turbopack and Temp Files
304
-
305
- The deferred (lazyDiscovery) Next.js builder writes build artifacts with a `.temp` extension to avoid HMR churn, then copies them to their final names. The V2 migration created `__step_registrations.route.js.temp` in the `app/` directory. Canary Turbopack rejects this file as an "Unknown module type" because the `.temp` extension has no associated loader.
306
-
307
- **Fix**: The step registrations file is written directly to its final name (`__step_registrations.js`) since it doesn't need the temp-file HMR mechanism. Only the route file uses temp naming.
308
-
309
- ### Concurrent `step_started` Inflating Attempt Counter
310
-
311
- When the V2 handler dispatches N parallel steps as background messages, each background step completion queues a workflow continuation. Up to N continuations may replay concurrently, and each may attempt to start the same not-yet-completed step (since `step_started` succeeds for already-running steps). Each call atomically increments the `attempt` counter. With N=5 parallel steps, the attempt counter can reach 5 on the first genuine execution — exceeding the default `maxRetries + 1 = 4` threshold and prematurely failing the step with "exceeded max retries".
312
-
313
- This is the same known limitation described in "Convergence After Parallel Steps" above, but with a concrete failure mode: `promiseRaceStressTestWorkflow` (which uses 5 parallel steps with `Promise.race`) consistently failed on Postgres tests.
314
-
315
- **Fix**: The max retries check in `executeStep()` now only enforces when `step.error` exists — distinguishing actual retries (failed → retry with error) from concurrent first-attempt races (multiple handlers start the same step simultaneously without any prior failure). Concurrent starts are harmless since `step_completed` idempotency ensures only the first completion wins.
316
-
317
- ### Inline Step Execution with Pending Stream Operations
318
-
319
- When a step's arguments or return value include serialized streams (e.g., `WritableStream` from `getWritable()`, or AI SDK streaming steps), the serialization layer creates background `flushablePipe` operations that pipe data to S3. These ops are tracked in an `ops` array and need to complete before the stream data is readable by external consumers.
158
+ ### Mixed suspensions
320
159
 
321
- In V1, each step ran in a separate function invocation. After the step completed, `waitUntil(ops)` kept the function alive to flush the ops. The function then returned, giving `waitUntil` exclusive event loop time.
160
+ A suspension may contain steps, hooks, and waits simultaneously. The handler creates events for all, then dispatches everything we are not running inline as a single parallel batch of queue messages:
322
161
 
323
- **Current state**: `executeStep()` attempts a 500ms `Promise.race` between the ops settling and a timeout. If ops settle in time (data confirmed on server), it returns `hasPendingOps: false` and the V2 handler continues the inline loop. If ops don't settle in 500ms (e.g., `WritableStream` kept open across steps), it returns `hasPendingOps: true` and the V2 handler breaks the loop and queues a continuation so `waitUntil` can flush them.
162
+ ```text
163
+ ownedPendingSteps = pendingSteps.filter(owned by this handler)
164
+ inlineStep = ownedPendingSteps[0] // optional
324
165
 
325
- **Earlier attempts that failed** (before the flush waiter fix below):
166
+ dispatches = [
167
+ ...for each non-inline pendingStep: queue stepId message (idempotency=correlationId),
168
+ ...if soonest pending wait: queue delayed continuation
169
+ (delaySeconds=min(remaining, maxDelay),
170
+ idempotency=waitCorrelationId[:hop|:secondBucket]),
171
+ ]
172
+ await Promise.all(dispatches)
326
173
 
327
- 1. **500ms inline ops await without flush waiters** — The same 500ms race, but `WorkflowServerWritableStream` used a buffered 10ms flush timer: the `flushablePipe`'s `pendingOps` reached 0 when the buffered `write()` returned (instant), but the actual S3 HTTP write hadn't started yet. The ops appeared settled but data wasn't on S3. Multiple approaches to fix the timing (delaying `pollWritableLock`, closing the writable to trigger flush, adding a post-settle delay) all failed or caused other issues (deadlocks, premature stream closure).
328
-
329
- 2. **Root cause of the buffered write issue**: `WorkflowServerWritableStream.write()` buffers chunks and schedules a flush via `setTimeout(flush, 10ms)`. The `flushablePipe` calls `await writer.write(chunk)` which returns immediately (data buffered). `pendingOps--` fires before the 10ms timer. The `pollWritableLock` sees `pendingOps === 0` and resolves `state.promise`. The ops appear settled, but data is still in the buffer.
330
-
331
- 3. **Why this only affects Vercel Prod**: On local (world-local), stream writes go to the filesystem — effectively instant. On Vercel (world-vercel), writes go through HTTP to workflow-server → S3, adding 50-100ms latency. The buffered write returns instantly but the HTTP round-trip is deferred. When the V2 loop continues and the function eventually returns, `waitUntil` may not have enough time to flush.
332
-
333
- **Follow-up**: The flush-waiter design described under "Buffered Stream Flush with Waiter Promises" below is the landed fix and resolves the buffered-write race. The remaining work is to shrink the 500ms inline-ops budget once we have confidence that the flush-waiter path settles deterministically across all worlds (today the budget is a defensive ceiling, not a tuned latency target), and to surface a stronger contract for "ops settled" — currently a 500ms timeout means "probably settled, give up and queue a continuation", which is correct but coarse. A signaled "ops drained" event from the world layer would let `executeStep()` proceed without the timeout in the common case, removing latency for streaming workflows whose ops settle in well under 500ms.
334
-
335
- ### CJS `module.exports` Collision in BOA Bundles (RESOLVED)
336
-
337
- The Vercel Build Output API (BOA) builder creates a single CJS bundle via `createCombinedBundle` with `bundleFinalOutput: true`. The combined route file imports the steps bundle:
338
-
339
- ```js
340
- import { __steps_registered } from './__step_registrations.js';
341
- import { workflowEntrypoint } from 'workflow/runtime';
342
- export const POST = workflowEntrypoint(workflowCode);
174
+ if (!inlineStep) return
175
+ await executeStep(inlineStep)
343
176
  ```
344
177
 
345
- When esbuild re-bundles this into CJS, the steps bundle's code is inlined. If the steps bundle is also CJS format, it contains its own `module.exports = __toCommonJS(...)` at the top level. esbuild sometimes inlines CJS modules **without** a `__commonJS()` wrapper (the heuristic depends on the module's detected format). When unwrapped, the steps bundle's `module.exports` assignment executes at the top level and **overwrites** the combined route's `module.exports`, removing the `POST` handler export.
346
-
347
- **Symptoms**: The Vercel deployment builds and starts successfully, but the `POST` handler is missing from the function's exports. Queue messages are delivered to the function but nothing processes them. All e2e tests hang indefinitely.
348
-
349
- **Debugging steps that led to the root cause**:
350
-
351
- 1. Tested the CJS bundle locally with `node -e "require('./index.js')"` — confirmed 92 steps registered, but `module.exports` only contained `{ __steps_registered }`, not `{ POST }`.
352
- 2. Found two `module.exports` assignments in the bundle: line ~45K (from the combined route, exporting `POST`) and line ~95K (from the inlined steps bundle, exporting `__steps_registered`). The second overwrites the first.
353
- 3. Compared with the standalone builder's bundle which had the same steps code wrapped in `__commonJS()` — esbuild's wrapper prevents the inner `module.exports` from leaking.
354
-
355
- **Fix**: When `bundleFinalOutput` is true, build the steps bundle in **ESM format** regardless of the final output format. The final esbuild pass converts everything to CJS correctly. ESM steps don't have `module.exports`, so there's no collision. The combined route's `export const POST` becomes the sole `module.exports` entry.
356
-
357
- ### Step Error Source Maps on BOA Deployments
358
-
359
- The V2 combined CJS bundle (`bundleFinalOutput: true`) loses original source file names during re-bundling. Error stack traces show `/var/task/index.js` instead of `99_e2e.ts`. The `hasStepSourceMaps()` utility was updated to return `false` for BOA-builder frameworks (Express, Fastify, Hono, Nitro, Nuxt, Vite, Astro, Example) on Vercel preview, aligning test expectations with the actual bundle behavior.
360
-
361
- ### CLI Health Check Port Mismatch
362
-
363
- The CLI `health` command defaults to `http://localhost:3000` when `WORKFLOW_LOCAL_BASE_URL` is not set. Different frameworks use different ports (Astro: 4321, SvelteKit: 5173). The e2e test passed `WORKFLOW_LOCAL_BASE_URL` via the spawn env, but the CLI's `getEnvVars()` function had a fixed list of env vars that didn't include `WORKFLOW_LOCAL_BASE_URL`. The env var was set but never read.
364
-
365
- **Fix**: Added `WORKFLOW_LOCAL_BASE_URL` to the CLI's `getEnvVars()` return object.
366
-
367
- ### Buffered Stream Flush with Waiter Promises
368
-
369
- `WorkflowServerWritableStream` buffers writes and flushes via a 10ms `setTimeout` for batching. Previously, `write()` returned immediately after buffering, causing the `flushablePipe`'s `pendingOps` counter to reach 0 before data actually reached the server. The V2 inline loop saw ops as settled prematurely and broke on every step with `WritableStream` serialization.
370
-
371
- **Fix**: `write()` now returns a promise that resolves only after the scheduled flush completes. Multiple writes within the 10ms window still share a single batched HTTP request (the batching optimization is preserved). Each write registers a `{resolve, reject}` pair in a `flushWaiters` array. When the `setTimeout` fires and `flush()` completes the HTTP round-trip, all waiters are resolved (or rejected on error). This makes `pendingOps` accurately reflect server-side data state while keeping network-efficient batching.
372
-
373
- The 500ms inline ops await in the step executor can now distinguish between:
374
-
375
- - **Steps where ops settle** (data on server, ~200ms after lock release + flush) → continue loop inline
376
- - **Steps where ops don't settle** (WritableStream kept open across steps) → break loop
377
-
378
- ### Lock-Release Polling Interval Lowered to 10ms
379
-
380
- `flushablePipe`'s `pollWritableLock` / `pollReadableLock` use `setInterval` to detect when a user releases their stream lock without closing the stream — the Web Streams API has no event for that state. The V2 step executor's `opsSettled` race waits for this poll to resolve after each writable-bearing step body returns, so the polling interval sits on the critical path of every streaming step.
381
-
382
- The interval was originally 100ms. Measuring a synthetic workflow with 5 sequential streaming steps (each step receives a shared `WritableStream` argument, writes a few chunks, releases the writer with the stream still open — the same pattern `doStreamStep` / `writeToolOutputToUI` / `writeFinishChunk` use in `DurableAgent.chat`) produced a per-step wait distribution clustered between 22–100ms with a mean of ~58ms. That matches the analytical prediction for a periodic poll with uniformly random offset relative to step return: ~half the interval. Across the 5 steps, polling alone added ~290ms of latency to the workflow even though no step actually had pending I/O — the writes were already flushed, the writer lock was already released, and we were just waiting for the next tick to notice.
383
-
384
- **Fix**: dropped the polling interval from 100ms to 10ms in `packages/core/src/flushable-stream.ts`. Per-step wait drops from ~50ms average to ~5ms (a 10× improvement, expected to scale linearly with the number of writable-bearing steps in a workflow). For `DurableAgent.chat` with one tool call (4 writable-bearing steps), this removes ~180ms from the streaming chat response's critical path. Per-tick work is just `writable.locked` plus a `getWriter()`/`releaseLock()` probe — both microsecond-scale, so 10× more ticks during a stream's lifetime is not measurable in practice.
385
-
386
- **Follow-up**: Replace the polling entirely with an event-driven release signal — wrap the writable returned from the `WritableStream` reviver with a writer that fires on `releaseLock()` — bringing the wait to ~0ms. The 10ms polling interval is the cheap path that captures most of the available win without the structural change, but every writable-bearing step still pays a ~5ms tax that the event-driven design would eliminate. The structural change is also worth pursuing because it removes a source of timing drift between `world-local` (filesystem-instant) and `world-vercel` (HTTP-deferred) — both would see truly synchronous lock-release detection rather than periodic-poll detection.
387
-
388
- ### Event Consumer Skip Logic Was Too Broad For Wait Replays
389
-
390
- The V2 handler needs some tolerance for out-of-order replay, especially around step events created by concurrent continuations. An early follow-up broadened that fallback to all wait lifecycle events too, so `onUnconsumedEvent` would skip `wait_created` and the first `wait_completed` whenever they matched a known wait. In the BOA-backed previews that broke `hookDisposeTestWorkflow`: once the first run disposed its hook and went into `sleep('5s')`, a replay could skip the live wait event before `sleep()` registered its subscriber, leaving the run stuck forever at `wait_created`.
391
-
392
- **Fix**: Keep the step/hook replay tolerance, but narrow the wait fallback to the one case we actually need: duplicate `wait_completed` events that appear *after* an earlier completion for the same wait. The `hookDispose` e2e was also updated to poll for hook registration/disposal instead of relying on fixed 3-5 second sleeps, which made the Vercel preview timing less brittle.
393
-
394
- ### TooEarlyError Retry Delay in Step Executor
395
-
396
- The `executeStep()` function handles `TooEarlyError` (thrown when a step's `retryAfter` timestamp hasn't been reached yet) by returning a `retry` result with a timeout. The original implementation used a stale access pattern `(err as any).meta?.retryAfter` copied from an older error shape. The `TooEarlyError` class (from `@workflow/errors`) has `retryAfter` as a direct property (number of seconds), not nested under `.meta`. The stale pattern always evaluated to `undefined`, falling back to a 1-second delay regardless of the server's actual retry-after value.
397
-
398
- **Fix**: Changed to `err.retryAfter ?? 1`, matching the correct pattern used in `step-handler.ts`.
399
-
400
- ### Health Check Endpoint JSON Response
401
-
402
- The `withHealthCheck()` wrapper in `helpers.ts` was updated (on main) to return a JSON response with `{ healthy, endpoint, specVersion, workflowCoreVersion }` instead of a plain text string. The V2 branch's e2e test still expected `Content-Type: text/plain` and a text body after merging main, causing the "health check endpoint (HTTP)" test to fail across all frameworks and environments.
403
-
404
- **Fix**: Updated the e2e test to expect `Content-Type: application/json` and validate the JSON body structure, including a `specVersion >= SPEC_VERSION_CURRENT` range assertion.
405
-
406
- ### V2 Combined Bundle Switched from CJS to ESM
407
-
408
- The V2 combined bundle was initially emitted as CJS by the standalone CLI and Vercel Build Output API builders, while `main` had already moved those outputs to ESM in [#1562](https://github.com/vercel/workflow/pull/1562). Staying on CJS meant `import.meta.url` was polyfilled (often producing the wrong path in re-bundled contexts), and the `world-testing` server had to import from `flow.js` via `createRequire` to force CJS semantics on what was really a CJS bundle.
409
-
410
- **Fix**: Align V2 with `main`'s ESM defaults:
411
-
412
- 1. The BOA builder emits `__step_registrations.mjs` and `index.mjs`, writes `"type": "module"` in `package.json`, and sets `handler: "index.mjs"` in `.vc-config.json`.
413
- 2. The standalone builder no longer overrides `format`; it inherits the base builder's `'esm'` default.
414
- 3. The standalone config outputs `step.mjs` / `flow.mjs` instead of `.js`.
415
- 4. The `world-testing` server uses a native `import { POST } from '../.well-known/workflow/v1/flow.mjs'` instead of `createRequire`.
416
- 5. `createCombinedBundle`'s final esbuild pass (for `bundleFinalOutput: true`) now prepends the same `createRequire(import.meta.url)` banner used by the workflow/webhook bundles so CJS dependencies that call `require()` for Node.js builtins (for example the `events` module referenced by bundled libraries) still resolve at runtime.
417
- 6. To avoid a duplicate `__createRequire` declaration, the inner steps bundle that gets inlined by the final pass skips the banner — only the outer bundle emits it. This is threaded through via a new `skipEsmRequireBanner` option on `createStepsBundle`.
418
-
419
- ### World specVersion in Health Check Responses
178
+ The wait timer is queued as its own continuation rather than encoded in the handler's return value (`{ timeoutSeconds }`). This is what makes `Promise.race(step, sleep)` behave correctly: even when the inline step blocks the handler for the full step duration, the wait continuation fires in a separate function invocation. If the sleep wins, that parallel invocation observes `wait_completed` via the "complete elapsed waits" pass and finishes the run; if the step wins, the wait continuation fires later and no-ops on the terminal run via the existing terminal-event check.
420
179
 
421
- The `getWorldHandlers()` return value was updated on main to include `specVersion` (the World's declared spec version). The V2 handler destructures this as `worldSpecVersion` and passes it to `handleHealthCheckMessage()` for inclusion in queue-based health check responses. This was merged alongside the V2 timeout configuration.
180
+ Step queueing remains unconditional (covers crash recovery: if a prior handler wrote `step_created` but crashed before queueing, a later handler will queue it; idempotency keys dedupe redundant queues across concurrent handlers).
422
181
 
423
- ### Async World Singleton Drift After Merge
182
+ Wait continuations are likewise deduplicated, keyed on the wait's correlation ID: while a wait is pending, every replay pass over the run re-observes it and would otherwise enqueue another delayed continuation. A key is attached in all cases, since some worlds serialize key-less workflow messages per run, which would park the continuation behind the handler's own inline step execution.
424
183
 
425
- The later `main` merge changed `getWorld()` and `getWorldHandlers()` to be asynchronous promise-backed singletons, but the eager-processing branch still had synchronous call sites in the V2 runtime path. That left `packages/core/src/runtime.ts` and `packages/core/src/runtime/helpers.ts` trying to access `.events` on a `Promise<World>`, which failed typecheck immediately after the merge.
184
+ Two situations deliver a continuation while its wait is still pending, and each varies the key so the re-enqueue isn't dropped by a world's dedupe window (which outlives the first delivery): waits longer than the maximum queue delay (23h, bounded by VQS's 24h message retention) are clamped and chained across hops, with the hop index suffixed to the key so each hop dedupes within its window but the chain always advances; and near-elapsed waits (≤2s remaining) use a second-bucketed key suffix so a continuation delivered marginally early (clock skew) can enqueue a fresh short-delay retry. See `runtime/wait-continuation.ts` for the full selection logic.
426
185
 
427
- **Fix**: Rebases the V2 workflow entrypoint onto the async world API by lazily awaiting `getWorldHandlers()` when wiring the queue handler and awaiting `getWorld()` at the remaining runtime/helper call sites. This preserves the inline replay loop while matching `main`'s new world initialization contract.
186
+ The retry/throttle and hook-conflict paths still return `{ timeoutSeconds }` since their semantics are "redeliver THIS message after a delay" rather than "schedule a fresh wait timer." Those can be unified in a follow-up.
428
187
 
429
- ### Lazy World Loading for Next.js Production Builds
188
+ The unified dispatch requires `world-local` to honor `delaySeconds` on the queue (added in the same PR series). Without it, the wait continuation would fire instantly in dev and trigger a spurious replay before the wait elapsed (recoverable via redelivery, but inefficient and observable as duplicate `step_started` events under contention).
430
189
 
431
- After the async world merge, `packages/core/src/runtime/world.ts` still eagerly imported both `@workflow/world-local` and `@workflow/world-vercel`, and it initialized `createRequire()` from `process.cwd() + '/package.json'` at module load time. In the Next.js production build jobs that caused the generated flow route to pull `@workflow/world-vercel` and its `debug` dependency into local builds, then fail during page-data collection with `module.createRequire failed parsing argument` and `Dynamic require of "tty" is not supported`.
190
+ ### VM sandboxing
432
191
 
433
- **Fix**: Switched the runtime world loader to use `createRequire(import.meta.url)` and moved the local/Vercel world imports behind the existing async `createWorld()` branches. Local Next.js builds now only load the selected world implementation at runtime instead of bundling both worlds eagerly into the route module.
434
-
435
- ### Deferred Next.js Builds Re-Ran Eager Discovery
436
-
437
- The later merge also pulled `BaseBuilder.createCombinedBundle()` into the deferred Next.js path without a way to pass the already-discovered workflow/step/serde entry sets. As a result, `packages/next/src/builder-deferred.ts` quietly fell back to `discoverEntries()` during production builds, re-emitting `Discovering workflow directives ...` and failing the local build tests that assert deferred mode avoids eager input-graph scans.
438
-
439
- **Fix**: Threaded explicit `discoveredEntries` through `createCombinedBundle()` and passed the deferred builder's tracked workflow/step/serde file sets into that call. Deferred Next.js builds now reuse the socket/cache-driven discovery state instead of re-running the base eager discovery pass.
440
-
441
- ### Deferred Package Steps Fell Back to Compiled `dist/` Files
442
-
443
- Once deferred discovery stopped re-running the base eager scan, some package-provided steps were only being rediscovered from built artifacts such as `packages/ai/dist/agent/durable-agent.js`. Those compiled files no longer carried every nested `'use step'` directive, so local production Next.js builds could miss registrations like `@workflow/ai/agent`'s `closeStream` helper and fail at runtime with "step is not registered in the current deployment".
444
-
445
- **Fix**: The deferred Next.js builder now rewrites discovered workspace package paths from `dist/` back to their matching `src/` files when those sources exist. That keeps deferred bundling pointed at the directive-bearing source modules instead of their compiled output.
446
-
447
- ### Workspace Source Step IDs Lost Export Subpaths
448
-
449
- Switching deferred builds over to workspace `src/` files fixed the missing nested directives, but it exposed a second mismatch in the SWC manifest path logic. `packages/builders/src/module-specifier.ts` only matched package exports against the on-disk file being transformed, so `packages/ai/src/agent/durable-agent.ts` was assigned `@workflow/ai@...` while the runtime still referenced the exported subpath id `@workflow/ai/agent@...`. Local Next.js agent runs then failed with "Step `step//@workflow/ai/agent@...//closeStream` is not registered" even though the source file was finally back in the bundle.
450
-
451
- **Fix**: `resolveModuleSpecifier()` now treats workspace source files as the source-backed form of their exported `dist/` targets when deriving step ids. That preserves package export subpaths like `@workflow/ai/agent` for id generation while still bundling the directive-bearing `src/` modules.
452
-
453
- ### Tarball-Staged Next.js Builds Still Lost Package Step Sources
454
-
455
- The local production and Postgres Next.js jobs stage the workbenches by packing workspace packages into tarballs and installing those tarballs into a temporary `node_modules` tree. Deferred discovery was already willing to rewrite workspace `packages/*/dist/*` files back to `src/*`, but the tarballed `@workflow/ai` package did not publish its `src/` tree and the base builder still treated `node_modules/@workflow/*/src/*` as ordinary package imports. That meant the staged CI path fell back to `dist/` again and dropped nested steps like `@workflow/ai/agent`'s `closeStream`, even after the workspace build path had been fixed.
456
-
457
- **Fix**: Publish `packages/ai/src` in the tarball, treat source-backed `node_modules/@workflow/*/src/*` files like external workspace source files when generating bundle imports, and extend deferred transitive step discovery to follow bare `workflow` / `@workflow/*` package imports during non-watch builds.
458
-
459
- ### Vercel Step Source Map Expectations Were Too Optimistic
460
-
461
- Merging `main` also pulled in a newer `hasStepSourceMaps()` expectation for Vercel preview deployments. On this branch, the non-Next workbench previews still emit step stacks without source filenames like `99_e2e.ts` or `helpers.ts`, so the Vercel step-error assertions regressed across the BOA-backed workbench matrix even though runtime behavior was otherwise unchanged.
462
-
463
- **Fix**: Revert the Vercel step source map expectation to the conservative branch behavior so preview e2e only asserts source filenames where this branch actually preserves them. Concretely, `hasStepSourceMaps()` returns `false` for *every* framework on Vercel deployments. Re-applying the blanket Vercel carve-out is what keeps the 11-framework `e2e-vercel-prod` matrix green while V2 source-map coverage catches up. The same pipeline regression also affects `nextjs-webpack` in local dev: pre-V2, webpack dev mode imported step sources directly so error stacks named `99_e2e.ts` / `helpers.ts`; under V2 the step bundle is inlined into the combined flow route and webpack's re-bundling collapses those filenames out of the dev-mode source maps. The helper now returns `false` for `nextjs-webpack` regardless of `DEV_TEST_CONFIG`.
464
-
465
- **Follow-up**: Wire up consumable inline source maps in the V2 step bundle across the framework integrations — the BOA-backed ones (Astro, Express, Fastify, Hono, Nitro, Nuxt, Vite, plus the standalone `example`), `nextjs-turbopack`, and `nextjs-webpack` (both dev and prod). The plan is to let each builder's `createCombinedBundle()` call carry an esbuild source-map pipeline that survives the framework's downstream re-bundling step, and then re-introduce the per-framework matrix in `hasStepSourceMaps()` so error stacks correctly point at `99_e2e.ts` / `helpers.ts` everywhere. Tracking this as a deferred follow-up rather than blocking the V2 cutover, since the runtime behavior is unaffected — only the surfaced filenames in step error stacks differ.
466
-
467
- ### Community Worlds Still Used The Pre-`world.streams` API
468
-
469
- The Redis community-world benchmark still loads an external world package that has not adopted the newer `world.streams.*` interface yet. Once the eager-processing changes exercised stream writes through the modern namespace consistently, that adapter started failing with `Cannot read properties of undefined (reading 'writeMulti')` before the benchmark could even start.
470
-
471
- **Decision**: Community world adapters must implement the `world.streams.*` interface. The runtime legacy stream normalization (`normalizeLegacyWorld`) was removed. Community world e2e tests are skipped until the adapters are updated.
472
-
473
- ### Build Output API Flow Handler Drift
474
-
475
- The Vercel Build Output API builder still emitted the combined flow function as `index.js`, but the surrounding metadata kept pointing at `index.mjs`. That mismatch meant BOA-based preview deployments published neither `/.well-known/workflow/v1/flow` nor the public manifest, so the Vercel production e2e suite collapsed into manifest `404` errors immediately after deployment.
476
-
477
- **Fix**: Updated `packages/builders/src/vercel-build-output-api.ts` to point both `.vc-config.json` and manifest extraction at `flow.func/index.js`, which matches the CommonJS file the builder actually writes.
478
-
479
- ### Async World Loading Broke Custom Target Worlds
480
-
481
- The first lazy-world-loading fix switched package resolution over to `createRequire(import.meta.url)` globally. That solved the Next.js bundling problem for built-in worlds, but it also made custom targets like `@workflow/world-postgres` resolve relative to `@workflow/core` instead of the consuming app. Local Postgres tests then failed at startup with `Cannot find module '@workflow/world-postgres'`.
482
-
483
- **Fix**: The runtime now creates the package resolver lazily from `process.cwd()/package.json` when possible, falling back to `import.meta.url` only when the app root cannot be resolved. That keeps custom world modules app-relative without reintroducing the eager module-load failure in Next.js builds.
484
-
485
- ### Core Logger Still Pulled `debug` Into Webpack Flow Routes
486
-
487
- Even after lazy world loading stopped eagerly importing `@workflow/world-vercel`, the generated Next.js webpack flow route still evaluated `packages/core/src/logger.ts` at module load. That file had a top-level `import debug from 'debug'`, which in turn pulled `debug/src/node` and its `tty` dynamic require into `/.well-known/workflow/v1/flow`. Webpack then failed during page-data collection with `Dynamic require of "tty" is not supported`.
488
-
489
- **Fix**: Replace the static `debug` dependency in the core logger with lightweight `process.env.DEBUG` matching plus `console.debug`. That keeps verbose opt-in logging for local debugging without forcing webpack to bundle `debug` and its Node-only terminal helpers into the flow route.
490
-
491
- ### Deferred Next.js Builder Helper Drift After Merge
492
-
493
- Merging `main` into the eager-processing branch pulled in a set of helper methods for copied-step import rewriting in `packages/next/src/builder-deferred.ts`, but the corresponding call sites were not present on this branch yet. That left `getRelativeImportSpecifier`, `getStepCopyFileName`, and `rewriteRelativeImportsForCopiedStep` orphaned, and `@workflow/next` failed to build with `TS6133` unused-private-member errors immediately after the merge.
494
-
495
- **Fix**: Removed the orphaned helper methods during merge resolution and kept the existing deferred-builder behavior unchanged. The copied-step import-rewrite work should land as a complete change set rather than a partial backport from `main`.
496
-
497
- ### Next.js React Step Fixture and `eval('require(...)')`
498
-
499
- The `nextjs-webpack` e2e suite still failed after the merge in `workflows/8_react_render.tsx`, where the step intentionally did `eval('require("react-dom/server")')` to avoid Next.js linting rules around importing `react-dom/server` directly. That pattern was brittle under webpack rebundling: even though the intermediate step bundle had a `createRequire(import.meta.url)` banner, the rebundled route still failed at runtime with `TypeError: require is not a function`.
500
-
501
- **Fix**: Updated the React-rendering step fixture in both Next.js workbenches to use `await import('react-dom/server')` instead. The test still exercises server-side React rendering inside a step, but no longer depends on bundler-specific `eval('require(...)')` behavior.
192
+ Workflow code still runs in a Node.js VM for determinism and sandboxing. Step code runs in the Node.js host context. The only change is that both happen within the same function invocation.
502
193
 
503
- ## Inline Execution Verification Tests
194
+ ### Bundle size and cold start
504
195
 
505
- The `@workflow/world-testing` package includes invocation-counting tests that verify the V2 inline loop behavior for each workflow pattern:
196
+ The combined bundle is larger (contains both step code and workflow VM code). Cold start time increases slightly. The reduction in total function invocations more than compensates.
506
197
 
507
- | Workflow Pattern | Expected Invocations | Why |
508
- |-----------------|---------------------|-----|
509
- | Sequential steps (3 adds) | **1** | All steps execute inline |
510
- | Sequential steps + WritableStream | **1** | Ops settle via flush waiter promises (500ms race) |
511
- | Sleep (1s) + step | **2** | Sleep requires queue round-trip |
512
- | Promise.all (2 steps) | **2-3** | Background step + inline replay after all steps done |
198
+ ### Step retries
513
199
 
514
- The test server tracks flow handler invocations per `runId` via an internal counter. Each test asserts the exact invocation count after the workflow completes.
200
+ When an inline step fails with retries remaining:
515
201
 
516
- ### world-testing Flow Invocation Counting Missed Wrapped Queue Payloads
202
+ - `RetryableError` with explicit `retryAfter` delay: Requeue to self with `stepId` and a delay.
203
+ - Transient errors with immediate retry: Requeue to self with `stepId` and a 1s delay.
204
+ - `FatalError`: Fail immediately.
517
205
 
518
- The inline-execution assertions in `packages/world-testing` count how many times the flow handler runs by inspecting the queue callback body and extracting `runId`. After the queue callback shape drifted, some worlds were only exposing the workflow payload under `body.payload.runId`, so the helper recorded `0` invocations even when the workflow completed correctly. That showed up in CI as the Postgres inline-execution spec failing its "single flow invocation" assertion.
206
+ ### Encryption key resolution
519
207
 
520
- **Fix**: Accept both top-level `runId` and nested `payload.runId` when tracking flow invocations in the embedded test server.
208
+ Encryption keys are resolved once before the inline execution loop starts (after the run status is confirmed as `running`) and reused across all iterations. Background step executions resolve the key independently. The key does not change within a run.
521
209
 
522
- ### Turbopack NFT Tracing Errors in V2 Combined Flow Route
210
+ ### Module scope duplication in re-bundled output
523
211
 
524
- The V2 combined flow route imports the step registrations bundle (`__step_registrations.js`), which esbuild produces as a monolithic file. On `main`, step registrations live in a separate route (`step/route.js`), so Turbopack traces them independently. In V2, Turbopack traces the step registrations through the flow route's import graph, encountering `world.ts` code with `process.cwd()`, dynamic `import()` calls to `@workflow/world-local`/`@workflow/world-vercel`, and `createRequire()` patterns — all of which trigger fatal NFT (Node File Trace) errors.
212
+ Builders that rebundle the combined output into a single file (standalone Workflow CLI, Vercel Build Output API, and NestJS) produce a layout where esbuild creates isolated module scopes for each source module, even within the same output file. Without intervention, `registerStepFunction` and `getStepFunction` operate on different `Map` instances: steps are registered into one `Map` but looked up from another.
525
213
 
526
- **Fix**: Introduced `get-world-lazy.ts`, a globalThis `Symbol.for`-based accessor that replaces the static `import { getWorld } from './runtime/world.js'` in all step-side modules (`serialization.ts`, `run.ts`, `helpers.ts`, `start.ts`, `resume-hook.ts`). This breaks the static import chain from step code to `world.ts`, preventing esbuild from bundling `world.ts` (and its transitive deps) into the step registrations. The step registrations bundle dropped from ~37k lines to ~6.6k lines (matching `main`), with zero `process.cwd()` or world package references.
214
+ The step function registry and the step context storage are `globalThis` singletons (via `Symbol.for`) to ensure all module scopes share the same instances. The same pattern is used for the World singleton and the class serialization registry.
527
215
 
528
- The `getWorldLazy()` function reads from the globalThis world singleton cache (populated by the runtime's `getWorld()` on first call). When the cache is empty (e.g., `start()` called from application code before any workflow runs), it falls back to a dynamic `import()` of `world.js` to initialize the world.
216
+ ### Inline step execution with pending stream operations
529
217
 
530
- Additional changes for Turbopack compatibility:
531
- - Removed `stepEntrypoint` re-export from `runtime.ts` (V2 doesn't use separate step routes)
532
- - Lazy-loaded `getPort` via `createRequire` with opaque specifier to prevent `@workflow/utils/get-port` filesystem operations from being traced
533
- - `getRuntimeRequire()` uses `process.cwd()` as primary resolution base (for custom world packages like `@workflow/world-postgres` that are app-level deps, not `@workflow/core` deps), with `import.meta.url` fallback
218
+ When a step's arguments or return value include serialized streams (e.g., `WritableStream` from `getWritable()`, or AI SDK streaming steps), the serialization layer creates background `flushablePipe` operations that pipe data to S3. These ops are tracked in an `ops` array and need to complete before the stream data is readable by external consumers.
534
219
 
535
- ### Cold-Start `MODULE_NOT_FOUND: './world.js'` From `getWorldLazy` Fallback
220
+ In V1, each step ran in a separate function invocation. After the step completed, `waitUntil(ops)` kept the function alive to flush the ops. In V2, the inline execution loop continues immediately after the step body returns, so we need to know whether to keep looping or break out and let `waitUntil` flush.
536
221
 
537
- The `getWorldLazy()` design assumed one of two paths would always succeed: either `globalThis[GetWorldFnKey]` is populated (because some prior code reached `world.ts`'s module body), or the dynamic `import('./world.js')` fallback resolves at runtime.
222
+ `executeStep()` attempts a 500ms `Promise.race` between the ops settling and a timeout. If ops settle in time (data confirmed on server), it returns `hasPendingOps: false` and the V2 handler continues the inline loop. If ops don't settle in 500ms (e.g., `WritableStream` kept open across steps), it returns `hasPendingOps: true` and the V2 handler breaks the loop and queues a continuation so `waitUntil` can flush them.
538
223
 
539
- Both assumptions break for routes that consume `start` (or any other `getWorldLazy` consumer) without going through the queue-driven flow handler first:
224
+ **Follow-up**: Shrink the 500ms inline-ops budget once we have confidence that the flush-waiter path settles deterministically across all worlds. A signaled "ops drained" event from the world layer would let `executeStep()` proceed without the timeout in the common case.
540
225
 
541
- 1. Webpack and Turbopack tree-shake the named import `{ getWorld } from './runtime/world.js'` out of `runtime.ts` once a consumer only uses `start`. `world.ts` is dropped from the bundle entirely, so its module-load `globalThis[GetWorldFnKey] ??= getWorld` registration never fires.
542
- 2. The dynamic-import fallback inside `get-world-lazy.ts` builds the specifier `./world.js` at runtime to evade bundler tracing — but webpack inlines `get-world-lazy.js` into the route bundle, so the relative specifier resolves against `/var/task/<app>/.next/server/app/<route>/route.js` where no sibling `world.js` exists. Node throws `MODULE_NOT_FOUND`.
226
+ ### Buffered stream flush with waiter promises
543
227
 
544
- The symptom: the very first request that goes through `start()` on a cold serverless invocation fails. Once any other code path (typically the queue-driven `/.well-known/workflow/v1/flow` route, which uses `getWorld` directly via `workflowEntrypoint`) has loaded `world.ts`, subsequent `start()` calls succeed for the rest of the process lifetime — making the failure flake-shaped: hard to reproduce in dev where everything tends to be warmed, but reliable on first user traffic into a fresh function instance.
228
+ `WorkflowServerWritableStream` buffers writes and flushes via a 10ms `setTimeout` for batching. Naively, `write()` could return immediately after buffering, but that would cause the `flushablePipe`'s `pendingOps` counter to reach 0 before data actually reached the server: the V2 inline loop would see ops as settled prematurely and produce data-loss races on every step with `WritableStream` serialization.
545
229
 
546
- **Fix**: Added `@workflow/core/runtime/world-init`, a server-only side-effect module that imports `./world.js` purely for its module-load side effect (the globalThis registration). It's exported via package conditions:
230
+ `write()` returns a promise that resolves only after the scheduled flush completes. Multiple writes within the 10ms window still share a single batched HTTP request (the batching optimization is preserved). Each write registers a `{resolve, reject}` pair in a `flushWaiters` array. When the `setTimeout` fires and `flush()` completes the HTTP round-trip, all waiters are resolved (or rejected on error). This makes `pendingOps` accurately reflect server-side data state while keeping network-efficient batching.
547
231
 
548
- - `default` → `./dist/runtime/world-init.js` (real, loads `world.ts`)
549
- - `workflow` → `./dist/workflow/world-init-stub.js` (empty, used by VM/step bundles)
232
+ ### Lock-release polling interval
550
233
 
551
- `packages/workflow/src/api.ts` (the host file behind `workflow/api`'s `default` condition) imports it for its side effect. The matching VM/step entry `api-workflow.ts` does not, so `world.ts` and its server-only deps (`@workflow/world-local`, `@workflow/world-vercel`, `cbor-x`, …) stay out of the workflow sandbox bundle.
234
+ `flushablePipe`'s `pollWritableLock` / `pollReadableLock` use `setInterval` to detect when a user releases their stream lock without closing the stream, since the Web Streams API has no event for that state. The V2 step executor's `opsSettled` race waits for this poll to resolve after each writable-bearing step body returns, so the polling interval sits on the critical path of every streaming step.
552
235
 
553
- Reverification: built bundles for `vade-review` (Next.js webpack) show `createLocalWorld`/`createVercelWorld`/`GetWorldFnKey` present in the route's vendor chunk for `@workflow/core` (zero before the fix), and the workflow VM bundle's `flow/route.js` and `__step_registrations.js` continue to have zero references to either the world-init module or `world.ts`. Cold-start `POST /api/review/submit` succeeds on the first request after a fresh server boot — the regression case.
236
+ The interval was lowered from 100ms to 10ms. Per-step wait drops from ~50ms average to ~5ms (scaling linearly with the number of writable-bearing steps in a workflow). Per-tick work is `writable.locked` plus a `getWriter()`/`releaseLock()` probe (microsecond-scale), so 10× more ticks is not measurable in practice.
554
237
 
555
- The dynamic-import fallback in `get-world-lazy.ts` is preserved as defense-in-depth for environments outside the documented configurations (CJS test runners, scripts that import deeply into `@workflow/core` without going through `workflow/api`).
238
+ **Follow-up**: Replace polling entirely with an event-driven release signal (wrap the writable returned from the `WritableStream` reviver with a writer that fires on `releaseLock()`), bringing the wait to ~0ms. This would also remove a source of timing drift between worlds with synchronous storage (`world-local`) and worlds with HTTP-deferred storage (`world-vercel`).
556
239
 
557
- ### Run#returnValue Worker Deadlock in V2 Inline Execution
240
+ ### Concurrent `step_started` and attempt counter
558
241
 
559
- When a workflow calls `start()` to spawn child workflows (e.g., `fibonacciWorkflow`), the parent's `Run#returnValue` step polls the child's completion status in a blocking loop (`while (true) { ... sleep(1000) ... }`). In V2, this step is executed inline by the step executor, holding a worker thread slot. If the child workflow's queue message is waiting for the same worker pool, the parent blocks the child from starting — a classic deadlock.
242
+ When the V2 handler dispatches N parallel steps as background messages, each background step completion queues a workflow continuation. Up to N continuations may replay concurrently, and each may attempt to start the same not-yet-completed step (since `step_started` succeeds for already-running steps). Each call atomically increments the `attempt` counter, so with N=5 parallel steps, the counter can reach 5 on the first genuine execution.
560
243
 
561
- **Fix**: `Run#pollReturnValue()` detects whether it's running inside a step executor (via `contextStorage.getStore()`) and, if so, throws `TooEarlyError` instead of polling in a blocking loop. `TooEarlyError` is handled specially by the step executor — it returns `{ type: 'retry', timeoutSeconds }` which re-queues the step with a 1-second delay, freeing the worker to process child workflows. Unlike `RetryableError`, `TooEarlyError` does NOT count against `maxRetries`, so polling steps can retry indefinitely until the child completes.
244
+ The max retries check in `executeStep()` only enforces when `step.error` exists, distinguishing actual retries (failed → retry with error) from concurrent first-attempt races (multiple handlers start the same step simultaneously without any prior failure). Concurrent starts are harmless since `step_completed` idempotency ensures only the first completion wins.
562
245
 
563
- When called from outside a step (e.g., test code, API routes), `pollReturnValue()` retains the original blocking loop behavior for backward compatibility.
246
+ ### Unconsumed event check two-phase drain
564
247
 
565
- ### Unconsumed Event Check Two-Phase Drain
248
+ The `EventsConsumer`'s unconsumed event check uses a two-phase promise queue drain: yield once after the first drain (via `setTimeout(0)`) so cross-VM promise chains can append follow-up async work, then re-drain before checking. This improves timing for scenarios like `step_completed` → for-await loop resume → next hook hydration.
566
249
 
567
- After merging `main`, the `EventsConsumer`'s unconsumed event check was updated with a two-phase promise queue drain: yield once after the first drain (via `setTimeout(0)`) so cross-VM promise chains can append follow-up async work, then re-drain before checking. This improves timing for scenarios like `step_completed` → for-await loop resume → next hook hydration. The V2 `onUnconsumedEvent` skip logic (returning `true` to advance past known-safe events) was preserved through the merge.
250
+ The check additionally arms a `DEFERRED_CHECK_DELAY_MS = 100` `setTimeout` after the second drain, since Node.js does not guarantee that `setTimeout(0)` fires after all cross-context microtasks settle. Any `subscribe()` call arriving during that 100ms window cancels the check via version invalidation + `clearTimeout`, so the delay only adds latency to genuine corruption, never to the happy path.
568
251
 
569
- The check additionally arms a `DEFERRED_CHECK_DELAY_MS = 100` `setTimeout` after the second drain, since Node.js does not guarantee that `setTimeout(0)` fires after all cross-context microtasks settle. Any `subscribe()` call arriving during that 100ms window cancels the check via version invalidation + `clearTimeout`, so the delay only adds latency to genuine corruption — never to the happy path.
252
+ **Follow-up**: 100ms is a heuristic chosen empirically. A deterministic settlement signal (for example, a "VM idle" callback exposed by the workflow VM bridge that fires only after all pending cross-context promise chains have resolved) would let the consumer fire the unconsumed-event check immediately on quiescence instead of waiting for a wall-clock timeout.
570
253
 
571
- **Follow-up**: 100ms is a heuristic chosen empirically to cover cross-VM microtask propagation under the workflow runtime's worst-case scheduling. A deterministic settlement signal — for example, a "VM idle" callback exposed by the workflow VM bridge that fires only after all pending cross-context promise chains have resolved — would let the consumer fire the unconsumed-event check immediately on quiescence instead of waiting for a wall-clock timeout. That would tighten corruption detection (no spurious 100ms wait) and remove the only remaining wall-clock heuristic from the V2 inline loop's correctness path.
254
+ ### Lazy World loading
572
255
 
573
- ### Nitro Builder Atomic File Writes
256
+ Static imports of `world-local` and `world-vercel` from the runtime caused two distinct build-time issues: Next.js production builds pulled both worlds (including their Node-only deps like `debug`'s `tty` requires) into the route module, and Turbopack's NFT (Node File Trace) errored on `process.cwd()` and dynamic `import()` patterns it couldn't statically analyze.
574
257
 
575
- After merging `main`, the Nitro builder now uses atomic temporary files (UUID-suffixed `.tmp` files) for build output, renaming them into place only after all builds succeed. This prevents partial/inconsistent output during dev HMR when a build fails mid-way. The V2 `createCombinedBundle` call was adapted to use this pattern.
258
+ A `getWorldLazy()` accessor (backed by a `globalThis` `Symbol.for` cache) replaces the static import in step-side modules. This breaks the static import chain from step code to `world.ts`, preventing both worlds from being bundled into the step registrations.
576
259
 
577
- ## Final Status
260
+ Because tree-shaking can otherwise drop `world.ts`'s module-load registration entirely, a server-only side-effect module (`@workflow/core/runtime/world-init`) imports `./world.js` purely for its module-load side effect. It's wired via package conditions:
578
261
 
579
- All framework integrations pass across all test environments:
262
+ - `default` maps to the real module and loads `world.ts`.
263
+ - `workflow` maps to an empty stub used by VM and step bundles.
580
264
 
581
- | Test Suite | Frameworks | Status |
582
- |-----------|-----------|--------|
583
- | Unit Tests | core | 581/581 |
584
- | Embedded Tests | world-testing | 9/9 (including inline execution) |
585
- | Local Dev | 14 frameworks | All pass |
586
- | Local Prod | 14 configurations | All pass |
587
- | Postgres | 14 frameworks | All pass |
588
- | Vercel Prod | 11 frameworks | All pass |
589
- | Vercel Deployments | 15 projects | All succeed |
590
- | Community Worlds | Turso, MongoDB, Redis | All pass |
591
- | Windows | e2e | Pass |
265
+ This guarantees the world is loaded for routes that consume `start()` without going through the queue-driven flow handler first, while keeping `world.ts` and its server-only deps out of the workflow sandbox bundle.
592
266
 
593
- Known remaining flakes (same as main):
267
+ ### Community Worlds and the `world.streams` API
594
268
 
595
- - `webhookWorkflow` / `hookWorkflow` — timing-sensitive hook delivery
269
+ Community world adapters must implement the `world.streams.*` interface. The runtime legacy stream normalization was removed as part of this work.