workflow 5.0.0-beta.42 → 5.0.0-beta.44
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -4
- package/dist/internal/builtins.d.ts +4 -4
- package/dist/internal/builtins.js +6 -6
- package/docs/ai/chat-session-modeling.mdx +23 -24
- package/docs/ai/defining-tools.mdx +5 -6
- package/docs/ai/human-in-the-loop.mdx +11 -11
- package/docs/ai/index.mdx +20 -20
- package/docs/ai/message-queueing.mdx +6 -6
- package/docs/ai/meta.json +1 -0
- package/docs/ai/resumable-streams.mdx +32 -28
- package/docs/ai/sleep-and-delays.mdx +9 -9
- package/docs/ai/streaming-updates-from-tools.mdx +4 -4
- package/docs/api-reference/vitest/index.mdx +8 -8
- package/docs/api-reference/workflow/create-hook.mdx +15 -15
- package/docs/api-reference/workflow/create-webhook.mdx +15 -15
- package/docs/api-reference/workflow/define-hook.mdx +10 -10
- package/docs/api-reference/workflow/fatal-error.mdx +2 -2
- package/docs/api-reference/workflow/fetch.mdx +7 -7
- package/docs/api-reference/workflow/get-step-metadata.mdx +2 -2
- package/docs/api-reference/workflow/get-workflow-metadata.mdx +3 -3
- package/docs/api-reference/workflow/get-writable.mdx +7 -7
- package/docs/api-reference/workflow/index.mdx +1 -1
- package/docs/api-reference/workflow/retryable-error.mdx +1 -1
- package/docs/api-reference/workflow/set-attributes.mdx +2 -2
- package/docs/api-reference/workflow/sleep.mdx +3 -3
- package/docs/api-reference/workflow-ai/durable-agent.mdx +42 -42
- package/docs/api-reference/workflow-ai/index.mdx +3 -3
- package/docs/api-reference/workflow-ai/workflow-chat-transport.mdx +28 -28
- package/docs/api-reference/workflow-api/get-hook-by-token.mdx +11 -11
- package/docs/api-reference/workflow-api/get-run.mdx +10 -10
- package/docs/api-reference/workflow-api/index.mdx +2 -4
- package/docs/api-reference/workflow-api/resume-hook.mdx +14 -14
- package/docs/api-reference/workflow-api/resume-webhook.mdx +4 -4
- package/docs/api-reference/workflow-api/start.mdx +16 -15
- package/docs/api-reference/workflow-astro/workflow.mdx +3 -3
- package/docs/api-reference/workflow-errors/entity-conflict-error.mdx +4 -4
- package/docs/api-reference/workflow-errors/hook-conflict-error.mdx +4 -4
- package/docs/api-reference/workflow-errors/hook-not-found-error.mdx +8 -8
- package/docs/api-reference/workflow-errors/index.mdx +6 -6
- package/docs/api-reference/workflow-errors/precondition-failed-error.mdx +9 -9
- package/docs/api-reference/workflow-errors/run-expired-error.mdx +2 -2
- package/docs/api-reference/workflow-errors/run-not-supported-error.mdx +4 -4
- package/docs/api-reference/workflow-errors/step-not-registered-error.mdx +5 -5
- package/docs/api-reference/workflow-errors/throttle-error.mdx +2 -2
- package/docs/api-reference/workflow-errors/too-early-error.mdx +2 -2
- package/docs/api-reference/workflow-errors/workflow-error.mdx +4 -4
- package/docs/api-reference/workflow-errors/workflow-not-registered-error.mdx +5 -6
- package/docs/api-reference/workflow-errors/workflow-run-cancelled-error.mdx +6 -6
- package/docs/api-reference/workflow-errors/workflow-run-failed-error.mdx +5 -5
- package/docs/api-reference/workflow-errors/workflow-run-not-completed-error.mdx +4 -4
- package/docs/api-reference/workflow-errors/workflow-run-not-found-error.mdx +4 -4
- package/docs/api-reference/workflow-errors/workflow-runtime-error.mdx +2 -2
- package/docs/api-reference/workflow-errors/workflow-world-error.mdx +8 -8
- package/docs/api-reference/workflow-globals.mdx +12 -11
- package/docs/api-reference/workflow-nest/configure-workflow-controller.mdx +2 -2
- package/docs/api-reference/workflow-nest/nest-local-builder.mdx +4 -4
- package/docs/api-reference/workflow-nest/workflow-controller.mdx +1 -1
- package/docs/api-reference/workflow-nest/workflow-module.mdx +4 -4
- package/docs/api-reference/workflow-next/with-workflow.mdx +16 -16
- package/docs/api-reference/workflow-nitro/index.mdx +6 -6
- package/docs/api-reference/workflow-nuxt/index.mdx +4 -4
- package/docs/api-reference/workflow-observability/hydrate-data.mdx +5 -5
- package/docs/api-reference/workflow-observability/hydrate-resource-io.mdx +4 -4
- package/docs/api-reference/workflow-observability/index.mdx +6 -8
- package/docs/api-reference/workflow-observability/observability-revivers.mdx +2 -2
- package/docs/api-reference/workflow-observability/parse-class-name.mdx +3 -3
- package/docs/api-reference/workflow-observability/parse-step-name.mdx +4 -4
- package/docs/api-reference/workflow-observability/parse-workflow-name.mdx +4 -4
- package/docs/api-reference/workflow-runtime/create-world.mdx +8 -8
- package/docs/api-reference/workflow-runtime/get-world-handlers.mdx +6 -6
- package/docs/api-reference/workflow-runtime/get-world.mdx +4 -4
- package/docs/api-reference/workflow-runtime/health-check.mdx +1 -1
- package/docs/api-reference/workflow-runtime/index.mdx +2 -4
- package/docs/api-reference/workflow-runtime/set-world.mdx +15 -13
- package/docs/api-reference/workflow-runtime/workflow-entrypoint.mdx +6 -5
- package/docs/api-reference/workflow-runtime/world/analytics.mdx +9 -9
- package/docs/api-reference/workflow-runtime/world/index.mdx +5 -3
- package/docs/api-reference/workflow-runtime/world/queue.mdx +11 -11
- package/docs/api-reference/workflow-runtime/world/storage.mdx +62 -28
- package/docs/api-reference/workflow-runtime/world/streams.mdx +7 -7
- package/docs/api-reference/workflow-serde/index.mdx +1 -1
- package/docs/api-reference/workflow-serde/workflow-deserialize.mdx +2 -2
- package/docs/api-reference/workflow-serde/workflow-serialize.mdx +7 -7
- package/docs/api-reference/workflow-sveltekit/workflow-plugin.mdx +3 -3
- package/docs/api-reference/workflow-vite/workflow.mdx +5 -5
- package/docs/changelog/attributes-mvp.mdx +39 -39
- package/docs/changelog/batched-event-writes.mdx +79 -0
- package/docs/changelog/eager-processing.mdx +63 -63
- package/docs/changelog/index.mdx +3 -3
- package/docs/changelog/lazy-event-creation.md +27 -27
- package/docs/changelog/meta.json +2 -1
- package/docs/changelog/resilient-resume.mdx +5 -5
- package/docs/changelog/resilient-start.mdx +14 -14
- package/docs/changelog/step-message-ownership.mdx +47 -47
- package/docs/changelog/turbo-mode.md +20 -20
- package/docs/comparisons/index.mdx +13 -13
- package/docs/comparisons/workflow-sdk-vs-aws-agentcore.mdx +15 -15
- package/docs/comparisons/workflow-sdk-vs-aws-step-functions.mdx +12 -12
- package/docs/comparisons/workflow-sdk-vs-cloudflare-workflows.mdx +11 -11
- package/docs/comparisons/workflow-sdk-vs-inngest.mdx +19 -19
- package/docs/comparisons/workflow-sdk-vs-temporal.mdx +23 -23
- package/docs/comparisons/workflow-sdk-vs-trigger-dev.mdx +18 -17
- package/docs/configuration/build-and-diagnostics.mdx +5 -5
- package/docs/configuration/cli-and-web-ui.mdx +4 -4
- package/docs/configuration/runtime-tuning.mdx +102 -21
- package/docs/configuration/worlds.mdx +34 -12
- package/docs/cookbook/advanced/child-workflows.mdx +25 -25
- package/docs/cookbook/advanced/publishing-libraries.mdx +40 -40
- package/docs/cookbook/advanced/serializable-steps.mdx +21 -21
- package/docs/cookbook/advanced/upgrading-workflows.mdx +31 -31
- package/docs/cookbook/agent-patterns/agent-cancellation.mdx +20 -20
- package/docs/cookbook/agent-patterns/human-in-the-loop.mdx +22 -22
- package/docs/cookbook/common-patterns/batching.mdx +14 -14
- package/docs/cookbook/common-patterns/idempotency.mdx +9 -9
- package/docs/cookbook/common-patterns/rate-limiting.mdx +3 -3
- package/docs/cookbook/common-patterns/saga.mdx +19 -19
- package/docs/cookbook/common-patterns/scheduling.mdx +23 -23
- package/docs/cookbook/common-patterns/sequential-and-parallel.mdx +26 -26
- package/docs/cookbook/common-patterns/timeouts.mdx +23 -23
- package/docs/cookbook/common-patterns/webhooks.mdx +6 -6
- package/docs/cookbook/common-patterns/workflow-composition.mdx +19 -19
- package/docs/cookbook/index.mdx +22 -22
- package/docs/cookbook/integrations/ai-sdk.mdx +43 -41
- package/docs/cookbook/integrations/chat-sdk.mdx +34 -34
- package/docs/cookbook/integrations/sandbox.mdx +46 -46
- package/docs/deploying.mdx +15 -15
- package/docs/errors/abort-signal-timeout-in-workflow.mdx +12 -12
- package/docs/errors/corrupted-event-log.mdx +13 -14
- package/docs/errors/deployment-mismatch.mdx +14 -14
- package/docs/errors/fetch-in-workflow.mdx +8 -8
- package/docs/errors/hook-conflict.mdx +11 -11
- package/docs/errors/index.mdx +1 -1
- package/docs/errors/node-js-module-in-workflow.mdx +5 -5
- package/docs/errors/replay-divergence.mdx +3 -3
- package/docs/errors/runtime-decryption-failed.mdx +12 -12
- package/docs/errors/serialization-failed.mdx +40 -12
- package/docs/errors/start-invalid-workflow-function.mdx +5 -5
- package/docs/errors/step-executed-multiple-times.mdx +2 -2
- package/docs/errors/step-not-registered.mdx +5 -5
- package/docs/errors/timeout-in-workflow.mdx +8 -8
- package/docs/errors/webhook-invalid-respond-with-value.mdx +18 -18
- package/docs/errors/webhook-response-not-sent.mdx +16 -16
- package/docs/errors/workflow-not-registered.mdx +5 -5
- package/docs/foundations/cancellation.mdx +31 -31
- package/docs/foundations/errors-and-retries.mdx +42 -11
- package/docs/foundations/hooks.mdx +37 -37
- package/docs/foundations/idempotency.mdx +9 -9
- package/docs/foundations/serialization.mdx +21 -22
- package/docs/foundations/starting-workflows.mdx +36 -37
- package/docs/foundations/streaming.mdx +68 -39
- package/docs/foundations/versioning.mdx +3 -3
- package/docs/foundations/workflows-and-steps.mdx +9 -9
- package/docs/getting-started/astro.mdx +16 -16
- package/docs/getting-started/express.mdx +8 -8
- package/docs/getting-started/fastify.mdx +8 -8
- package/docs/getting-started/hono.mdx +8 -8
- package/docs/getting-started/nestjs.mdx +18 -17
- package/docs/getting-started/next.mdx +11 -11
- package/docs/getting-started/nitro.mdx +16 -16
- package/docs/getting-started/nuxt.mdx +8 -8
- package/docs/getting-started/python.mdx +4 -4
- package/docs/getting-started/react-router/v7.mdx +1 -1
- package/docs/getting-started/react-router/v8.mdx +1 -1
- package/docs/getting-started/sveltekit.mdx +14 -14
- package/docs/getting-started/tanstack-start.mdx +12 -12
- package/docs/getting-started/vite.mdx +8 -8
- package/docs/how-it-works/cancellation.mdx +62 -62
- package/docs/how-it-works/code-transform.mdx +66 -54
- package/docs/how-it-works/encryption.mdx +25 -21
- package/docs/how-it-works/event-sourcing.mdx +84 -28
- package/docs/how-it-works/framework-integrations.mdx +12 -12
- package/docs/how-it-works/understanding-directives.mdx +21 -21
- package/docs/internal/index.mdx +6 -6
- package/docs/internal/nitro-native-build.mdx +2 -2
- package/docs/internal/nitro-web-ui.mdx +4 -4
- package/docs/internal/serializable-abort-controller.mdx +7 -7
- package/docs/observability/attributes.mdx +3 -3
- package/docs/observability/index.mdx +14 -10
- package/docs/observability/tracing.mdx +10 -10
- package/docs/testing/index.mdx +35 -35
- package/docs/testing/server-based.mdx +10 -10
- package/docs/whats-new.mdx +185 -0
- package/package.json +12 -12
|
@@ -5,7 +5,7 @@ description: A minimal, write-only subset of the planned Workflow Attributes fea
|
|
|
5
5
|
|
|
6
6
|
# Workflow Attributes (MVP)
|
|
7
7
|
|
|
8
|
-
This
|
|
8
|
+
This **experimental** minimum viable product (MVP) implements a subset of the [planned Workflow Attributes feature for 5.0.0](https://github.com/vercel/workflow/pull/1933). See [discussion #132](https://github.com/vercel/workflow/discussions/132) for broader background on the use cases and the full design space.
|
|
9
9
|
|
|
10
10
|
The MVP lets workflow code attach plaintext `string → string` metadata to a run, viewable in any observability surface that reads the `WorkflowRun` entity. It is deliberately narrow: write-only, no reads from inside a run, no list/filter endpoints, no event-log representation. The wire format and SDK surface are chosen so the full 5.0.0 implementation replaces this without source-level breaking changes for end users.
|
|
11
11
|
|
|
@@ -24,7 +24,7 @@ The remainder of this page documents the original MVP motivation and implementat
|
|
|
24
24
|
## What MVP supports
|
|
25
25
|
|
|
26
26
|
- `experimental_setAttributes(record)` callable from a **workflow body** (`"use workflow"` function), dispatched via an internal `__builtin_set_attributes` step bridge so the mutation gets a `step_created → step_completed` event pair
|
|
27
|
-
- Attributes are materialized onto the `WorkflowRun` entity, plaintext, and visible
|
|
27
|
+
- Attributes are materialized onto the `WorkflowRun` entity, plaintext, and visible through `world.runs.get()` / `world.runs.list()` and any observability user interface (UI) built on top of those
|
|
28
28
|
- World implementations emit a side-channel observability record per successful write (in `world-vercel`, this hooks into the same observability/analytics pipeline already used for other run lifecycle events)
|
|
29
29
|
|
|
30
30
|
Calling `experimental_setAttributes` from a step body was intentionally not supported in the MVP, but step-body calls are now supported as a follow-up. Plain host code remains unsupported because there is no active workflow run to attach attributes to.
|
|
@@ -42,15 +42,15 @@ See [PR #1933](https://github.com/vercel/workflow/pull/1933) for the design of t
|
|
|
42
42
|
|
|
43
43
|
## Why the MVP defers an `attr_set` event type
|
|
44
44
|
|
|
45
|
-
The full design represents attribute changes as a new `attr_set` event type in the event log, replayed by the workflow runtime VM to reconstruct the attribute snapshot. That requires bumping `SPEC_VERSION_CURRENT`, because every world implementation (including community worlds) needs to handle the new event during replay, and the runtime's reconstruction logic gains a new case.
|
|
45
|
+
The full design represents attribute changes as a new `attr_set` event type in the event log, replayed by the workflow runtime virtual machine (VM) to reconstruct the attribute snapshot. That requires bumping `SPEC_VERSION_CURRENT`, because every world implementation (including community worlds) needs to handle the new event during replay, and the runtime's reconstruction logic gains a new case.
|
|
46
46
|
|
|
47
|
-
A spec version bump is expensive: it gates every world adapter and ties the rollout to coordinated upgrades. We do not want to pay that cost twice
|
|
47
|
+
A spec version bump is expensive: it gates every world adapter and ties the rollout to coordinated upgrades. We do not want to pay that cost twice: once for the MVP, again for the full feature.
|
|
48
48
|
|
|
49
49
|
The MVP instead writes attributes via a direct entity-mutation path (outside the event log) which does not require a spec version bump. The downside is that MVP-era attributes have **no representation in the event log** and will not be visible to event-based reconstruction (e.g. a materialization rebuild). When the full feature ships, new writes use `attr_set` events; old runs created during the MVP window retain whatever attributes were materialized at the time, but their history is not recoverable.
|
|
50
50
|
|
|
51
51
|
## Implementation plan
|
|
52
52
|
|
|
53
|
-
### 1. `@workflow/world
|
|
53
|
+
### 1. `@workflow/world`: storage interface addition
|
|
54
54
|
|
|
55
55
|
Add an `experimentalSetAttributes` method to `Storage.runs`:
|
|
56
56
|
|
|
@@ -85,7 +85,7 @@ runs: {
|
|
|
85
85
|
}
|
|
86
86
|
```
|
|
87
87
|
|
|
88
|
-
The method is **optional** to avoid forcing every World implementation (especially community-maintained adapters such as Redis, MongoDB, Turso, and similar) to ship support before the API
|
|
88
|
+
The method is **optional** to avoid forcing every World implementation (especially community-maintained adapters such as Redis, MongoDB, Turso, and similar) to ship support before the API stabilizes. World implementations that do support it return the post-merge attribute snapshot so callers (notably the SDK helper and world adapters emitting observability records) have it without a follow-up read.
|
|
89
89
|
|
|
90
90
|
Add `attributes?: Record<string, string>` to `WorkflowRunBaseSchema` in `packages/world/src/runs.ts`. Optional for backward compatibility: runs created before this field landed have no `attributes` and read as `undefined`.
|
|
91
91
|
|
|
@@ -93,7 +93,7 @@ Add `attributes?: Record<string, string>` to `WorkflowRunBaseSchema` in `package
|
|
|
93
93
|
|
|
94
94
|
`world-vercel` calls into a remote endpoint to persist attributes:
|
|
95
95
|
|
|
96
|
-
```
|
|
96
|
+
```text
|
|
97
97
|
POST /v2/runs/:runId/attributes
|
|
98
98
|
|
|
99
99
|
{
|
|
@@ -107,9 +107,9 @@ POST /v2/runs/:runId/attributes
|
|
|
107
107
|
|
|
108
108
|
Response: `{ "attributes": { "phase": "done", "tenant": "t1" } }`.
|
|
109
109
|
|
|
110
|
-
The `changes` field **deliberately mirrors** the eventual `attr_set` event's `eventData.changes`. `allowReservedAttributes` is optional and framework-only; omit it for user-authored attributes. When the full feature ships, this endpoint goes away
|
|
110
|
+
The `changes` field **deliberately mirrors** the eventual `attr_set` event's `eventData.changes`. `allowReservedAttributes` is optional and framework-only; omit it for user-authored attributes. When the full feature ships, this endpoint goes away: the same `changes` shape is posted to `POST /v2/runs/:id/events` with `eventType: 'attr_set'` (plus a `writer` discriminator). No SDK signature change, no client-side migration.
|
|
111
111
|
|
|
112
|
-
### 3. `@workflow/core
|
|
112
|
+
### 3. `@workflow/core`: SDK surface
|
|
113
113
|
|
|
114
114
|
A new export from `@workflow/core` (re-exported by `workflow`):
|
|
115
115
|
|
|
@@ -122,11 +122,11 @@ function experimental_setAttributes(
|
|
|
122
122
|
): Promise<void>
|
|
123
123
|
```
|
|
124
124
|
|
|
125
|
-
`undefined` is normalized to `null` (unset). An empty object is a no-op
|
|
125
|
+
`undefined` is normalized to `null` (unset). An empty object is a no-op. It makes no remote procedure call (RPC) and creates no events.
|
|
126
126
|
|
|
127
127
|
Validation (shared helper, applied both client-side and server-side):
|
|
128
128
|
|
|
129
|
-
- Key: 1–256
|
|
129
|
+
- Key: 1–256 characters, must not start with `$` (reserved, see "Reserved `$` namespace" below)
|
|
130
130
|
- Value: ≤ 256 bytes UTF-8
|
|
131
131
|
- Maximum 64 attributes per run (validated against the post-merge snapshot when the server applies)
|
|
132
132
|
- SDK-side violations throw `FatalError` from `@workflow/errors` before the internal step is dispatched; worlds revalidate as the final authority before mutating storage
|
|
@@ -146,7 +146,7 @@ await experimental_setAttributes(
|
|
|
146
146
|
);
|
|
147
147
|
```
|
|
148
148
|
|
|
149
|
-
The flag is per-call (no run-level "this run accepts reserved keys" mode), so each framework call site explicitly declares intent. Don't enable it from user code
|
|
149
|
+
The flag is per-call (no run-level "this run accepts reserved keys" mode), so each framework call site explicitly declares intent. Don't enable it from user code: misuse can conflict with observability surfaces, agent dashboards, or future platform features that rely on the reserved namespace.
|
|
150
150
|
|
|
151
151
|
`experimental_setAttributes` is callable from a workflow body:
|
|
152
152
|
|
|
@@ -161,7 +161,7 @@ export async function myWorkflow(orderId: string) {
|
|
|
161
161
|
}
|
|
162
162
|
```
|
|
163
163
|
|
|
164
|
-
The workflow-body path validates input inside the VM and then dispatches the canonical `AttributeChange[]` through an internal `__builtin_set_attributes` step bridge
|
|
164
|
+
The workflow-body path validates input inside the VM and then dispatches the canonical `AttributeChange[]` through an internal `__builtin_set_attributes` step bridge. See "How workflow-body dispatch works" below. The mutation is materialized on the run entity by the step body.
|
|
165
165
|
|
|
166
166
|
Step-body calls resolve to the host-side export. In a step context, that export validates the input and posts the attribute changes directly to `world.runs.experimentalSetAttributes(runId, changes)`. Plain host code still throws `FatalError`.
|
|
167
167
|
|
|
@@ -180,7 +180,7 @@ const result = await processOrder();
|
|
|
180
180
|
await experimental_setAttributes({ phase: 'done', orderId: result.id });
|
|
181
181
|
```
|
|
182
182
|
|
|
183
|
-
**Fire-and-forget (`void`).** Drop the `await` to let the workflow proceed without blocking. The pending step queues on the workflow's next suspension (any `await` on a runtime primitive
|
|
183
|
+
**Fire-and-forget (`void`).** Drop the `await` to let the workflow proceed without blocking. The pending step queues on the workflow's next suspension (any `await` on a runtime primitive: a step, `sleep`, a hook). This is the canonical pattern for observability / tracking metadata where the workflow doesn't depend on the write.
|
|
184
184
|
|
|
185
185
|
{/*@skip-typecheck - snippet, not runnable code*/}
|
|
186
186
|
|
|
@@ -195,18 +195,18 @@ return result;
|
|
|
195
195
|
|
|
196
196
|
Two trade-offs to know about:
|
|
197
197
|
|
|
198
|
-
1. **Order of arrival at the world is not workflow-source order.** Fire-and-forget steps run out-of-band on the queue worker. A `void` write to one key followed by an `await` write to the same key may race; LWW
|
|
198
|
+
1. **Order of arrival at the world is not workflow-source order.** Fire-and-forget steps run out-of-band on the queue worker. A `void` write to one key followed by an `await` write to the same key may race; last-write-wins (LWW) by arrival applies (see "Concurrent writes" below).
|
|
199
199
|
|
|
200
|
-
2. **The last `void` before `return` may not land.** If you place a `void experimental_setAttributes(...)` immediately before returning, with no intervening `await` on a runtime primitive, drain-on-completion commits the `step_created` event but the step body is not reliably dispatched before the run transitions to its terminal status
|
|
200
|
+
2. **The last `void` before `return` may not land.** If you place a `void experimental_setAttributes(...)` immediately before returning, with no intervening `await` on a runtime primitive, drain-on-completion commits the `step_created` event but the step body is not reliably dispatched before the run transitions to its terminal status. See the architectural note in the "Implementation notes" section. In practice workflows almost always have an `await` after the last fire-and-forget call (a step, a sleep, a hook); if you don't, add `await sleep('0s')` before returning, or use the awaited form for that final write.
|
|
201
201
|
|
|
202
|
-
**Parallel (`Promise.all`).** Multiple calls dispatch concurrently. Writes to disjoint keys all land. Writes to the same key resolve last-write-wins by *arrival order at the world* (not the order the workflow body issued the calls)
|
|
202
|
+
**Parallel (`Promise.all`).** Multiple calls dispatch concurrently. Writes to disjoint keys all land. Writes to the same key resolve last-write-wins by *arrival order at the world* (not the order the workflow body issued the calls), so don't use `Promise.all` for writes that must observe a specific order to the same key.
|
|
203
203
|
|
|
204
204
|
{/*@skip-typecheck - snippet, not runnable code*/}
|
|
205
205
|
|
|
206
206
|
```ts
|
|
207
207
|
'use workflow';
|
|
208
208
|
await Promise.all([
|
|
209
|
-
experimental_setAttributes({ phase: 'init' }), // disjoint keys
|
|
209
|
+
experimental_setAttributes({ phase: 'init' }), // disjoint keys: all land
|
|
210
210
|
experimental_setAttributes({ orderId: 'ord_123' }),
|
|
211
211
|
experimental_setAttributes({ tenant: 't1' }),
|
|
212
212
|
]);
|
|
@@ -224,11 +224,11 @@ Implement `experimentalSetAttributes(runId, changes)` by reading the run's JSON
|
|
|
224
224
|
|
|
225
225
|
#### `world-postgres`
|
|
226
226
|
|
|
227
|
-
Add an `attributes JSONB` column to the runs table (default `'{}'::jsonb`, NOT NULL). Apply the merge in SQL using `jsonb_set` / `jsonb_strip_nulls` so the database does the merge atomically without a read-modify-write cycle, returning the post-merge map
|
|
227
|
+
Add an `attributes JSONB` column to the runs table (default `'{}'::jsonb`, NOT NULL). Apply the merge in Structured Query Language (SQL) using `jsonb_set` / `jsonb_strip_nulls` so the database does the merge atomically without a read-modify-write cycle, returning the post-merge map through `RETURNING attributes`.
|
|
228
228
|
|
|
229
229
|
#### `world-vercel`
|
|
230
230
|
|
|
231
|
-
Pure HTTP wrapper. Calls the wire endpoint described in §2 and returns the response's `attributes`. The backing service materializes the attribute map onto its run-row storage; where the underlying data store supports atomic per-key map updates, the merge is a single atomic operation rather than a read-modify-write cycle
|
|
231
|
+
Pure HTTP wrapper. Calls the wire endpoint described in §2 and returns the response's `attributes`. The backing service materializes the attribute map onto its run-row storage; where the underlying data store supports atomic per-key map updates, the merge is a single atomic operation rather than a read-modify-write cycle, the same shape the future `attr_set` event handler will use, so the storage layout is forward-compatible.
|
|
232
232
|
|
|
233
233
|
After the persistence ack, the service emits a side-channel observability record carrying the post-merge attribute snapshot, decoupled from the request path so the runtime never waits on analytics emission.
|
|
234
234
|
|
|
@@ -245,8 +245,8 @@ The MVP applies **last-write-wins by arrival order at the world**. Two concurren
|
|
|
245
245
|
Writes from a single `await`-ed call chain are serialized by the workflow VM and land in workflow-source order. The concurrent / racy case applies to:
|
|
246
246
|
|
|
247
247
|
- Multiple `experimental_setAttributes` calls inside one `Promise.all` writing the same key (the workflow VM dispatches them concurrently; the world sees them in scheduler order, not source order).
|
|
248
|
-
- `void experimental_setAttributes(...)` followed by another call to the same key
|
|
249
|
-
- Multiple workflows writing the same key on the same run (rare
|
|
248
|
+
- `void experimental_setAttributes(...)` followed by another call to the same key: the fire-and-forget step may still be in flight when the next call lands.
|
|
249
|
+
- Multiple workflows writing the same key on the same run (rare: usually one workflow owns a run).
|
|
250
250
|
|
|
251
251
|
Disjoint-key writes are unaffected: every call lands, regardless of pattern. Applications that need conditional semantics on a shared key should wait for the 5.0.0 release; we will not retrofit conditional writes onto the MVP path.
|
|
252
252
|
|
|
@@ -254,11 +254,11 @@ Disjoint-key writes are unaffected: every call lands, regardless of pattern. App
|
|
|
254
254
|
|
|
255
255
|
Attribute changes do not appear in `world.events.list(runId)`. There is no record of *when* a key changed or *which step attempt* set it. The current snapshot on the run entity is authoritative; the history is lost.
|
|
256
256
|
|
|
257
|
-
When the full feature ships, new writes carry writer attribution (`writer: { type: 'workflow' }` or `writer: { type: 'step', stepId, attempt }`) in their `attr_set` events. MVP-era writes will not have this
|
|
257
|
+
When the full feature ships, new writes carry writer attribution (`writer: { type: 'workflow' }` or `writer: { type: 'step', stepId, attempt }`) in their `attr_set` events. MVP-era writes will not have this. History starts at the `attr_set` cutover.
|
|
258
258
|
|
|
259
259
|
### MVP attributes do not survive materialization rebuild
|
|
260
260
|
|
|
261
|
-
Any tooling that reconstructs the run entity from the event log (disaster recovery, debugging, audit) will see no attributes on MVP-era runs, because the writes are not in the event log. This is the chief reason `experimentalSetAttributes` is named "experimental"
|
|
261
|
+
Any tooling that reconstructs the run entity from the event log (disaster recovery, debugging, audit) will see no attributes on MVP-era runs, because the writes are not in the event log. This is the chief reason `experimentalSetAttributes` is named "experimental": it is a known break from the otherwise-strict event-sourced model.
|
|
262
262
|
|
|
263
263
|
The 5.0.0 path closes this gap.
|
|
264
264
|
|
|
@@ -272,8 +272,8 @@ If you need behavior the MVP does not provide (read, list, filter, initial attri
|
|
|
272
272
|
|
|
273
273
|
Unit tests in `@workflow/world` (validation surface) and `@workflow/core` (VM-side dispatch + host-side stub):
|
|
274
274
|
|
|
275
|
-
- Validation rules
|
|
276
|
-
- Reserved `$` namespace
|
|
275
|
+
- Validation rules: key length, value byte cap, `$` prefix, per-batch duplicates, post-merge count cap (with `existingKeys` so updates of present keys don't falsely trip the cap)
|
|
276
|
+
- Reserved `$` namespace: rejected by default, accepted when `allowReservedAttributes: true` is passed (both for `validateAttributeKey` and at the batch level via `validateAttributeChanges`)
|
|
277
277
|
- `experimental_setAttributes({})` is a no-op (no dispatch, no events)
|
|
278
278
|
- `undefined` value normalizes to a `null`-valued change on the wire
|
|
279
279
|
- The `{ allowReservedAttributes: true }` opt-in is forwarded through the step bridge so the world receives the flag
|
|
@@ -305,16 +305,16 @@ End-to-end in `workbench/nextjs-turbopack` (exercises the full SWC plugin + work
|
|
|
305
305
|
|
|
306
306
|
- Awaited workflow-body calls dispatch through the `__builtin_set_attributes` step bridge and merge correctly (the test inspects the run's event log to confirm a `step_created` / `step_completed` pair was emitted)
|
|
307
307
|
- Fire-and-forget (`void experimental_setAttributes`) attributes land before the run terminates
|
|
308
|
-
- `Promise.all` of disjoint-key writes
|
|
309
|
-
- Workflow throws after an awaited `experimental_setAttributes
|
|
308
|
+
- `Promise.all` of disjoint-key writes: every key persists
|
|
309
|
+
- Workflow throws after an awaited `experimental_setAttributes`: the attribute persists on the now-`failed` run (the per-run file lock on `run_failed` re-reads inside the critical section so the attribute snapshot survives the lifecycle write)
|
|
310
310
|
|
|
311
311
|
## Migration to native v4
|
|
312
312
|
|
|
313
313
|
The native attributes follow-up applies this migration:
|
|
314
314
|
|
|
315
|
-
- `experimental_setAttributes` (SDK)
|
|
316
|
-
- `runs.experimentalSetAttributes` (world interface)
|
|
317
|
-
- Wire endpoint
|
|
315
|
+
- `experimental_setAttributes` (SDK): unchanged signature, new dispatch path
|
|
316
|
+
- `runs.experimentalSetAttributes` (world interface): deprecated, then removed; replaced by `events.create(runId, { eventType: 'attr_set', eventData: { changes, writer } })`
|
|
317
|
+
- Wire endpoint: `POST /v2/runs/:runId/attributes` is deprecated but temporarily retained for older clients; new calls post the same `changes` shape to events as `attr_set`
|
|
318
318
|
- Pre-existing attribute values on MVP-era runs remain on the run entity but are not represented in the event log
|
|
319
319
|
|
|
320
320
|
Skew protection means workflows started under the MVP will continue to run with the MVP dispatch path on their original deployment. New deployments use the new path. No in-place data migration is needed.
|
|
@@ -330,10 +330,10 @@ This section records concrete decisions taken while landing the MVP that weren't
|
|
|
330
330
|
1. The workflow VM bundle resolves `experimental_setAttributes` to `packages/core/src/workflow/set-attributes.ts` (via the `workflow` package-exports condition).
|
|
331
331
|
2. That helper validates the input record inline (no shared helper, no cross-file dependency from a 'use step' file) and produces canonical `AttributeChange[]`.
|
|
332
332
|
3. It dispatches through the standard workflow-VM step mechanism: `globalThis[WORKFLOW_USE_STEP]('__builtin_set_attributes')(changes)`. The `useStep` dispatcher is the same one used by every other step call from a workflow body, populated by `packages/core/src/workflow.ts` at VM bootstrap.
|
|
333
|
-
4. The dispatch queues a step (`step_created`), the host runs `__builtin_set_attributes(changes, options)` from `packages/workflow/src/internal/builtins.ts`. The step body reads the active world, current run id, and attempt number directly from `globalThis` symbols (`Symbol.for('@workflow/world//cache')` and `Symbol.for('WORKFLOW_STEP_CONTEXT_STORAGE')`)
|
|
333
|
+
4. The dispatch queues a step (`step_created`), the host runs `__builtin_set_attributes(changes, options)` from `packages/workflow/src/internal/builtins.ts`. The step body reads the active world, current run id, and attempt number directly from `globalThis` symbols (`Symbol.for('@workflow/world//cache')` and `Symbol.for('WORKFLOW_STEP_CONTEXT_STORAGE')`), populated by the host runtime, and calls `world.runs.experimentalSetAttributes(runId, changes, options)`.
|
|
334
334
|
5. The step completes (`step_completed`), the workflow resumes.
|
|
335
335
|
|
|
336
|
-
This puts the mutation on the event log as a normal `step_created → step_completed` pair without inventing a new event type
|
|
336
|
+
This puts the mutation on the event log as a normal `step_created → step_completed` pair without inventing a new event type. That stays for the full 5.0.0 cutover.
|
|
337
337
|
|
|
338
338
|
The internal step is best-effort during the experimental phase. It sets `maxRetries = 2`, for three total attempts. If `world.runs.experimentalSetAttributes` fails on attempts 1 or 2, the error is rethrown so the runtime retries the step normally. If it still fails on attempt 3, the step logs `console.error` and returns; the workflow run continues instead of receiving a retry-exhaustion `FatalError` for failed tag posting.
|
|
339
339
|
|
|
@@ -347,15 +347,15 @@ When the full 5.0.0 attributes feature lands, `__builtin_set_attributes` is repl
|
|
|
347
347
|
|
|
348
348
|
### Endpoint lives under `v2`, not a fresh namespace
|
|
349
349
|
|
|
350
|
-
The initial draft placed the new endpoint at `POST /v3/runs/:runId/attributes`, on the assumption that introducing a new wire feature warranted a major namespace bump. In practice `world-vercel` mixes `/v1/...` and `/v2/...` endpoints already, and creating a `v3Api` subrouter
|
|
350
|
+
The initial draft placed the new endpoint at `POST /v3/runs/:runId/attributes`, on the assumption that introducing a new wire feature warranted a major namespace bump. In practice `world-vercel` mixes `/v1/...` and `/v2/...` endpoints already, and creating a `v3Api` subrouter for a single endpoint would have required duplicating the auth / flags / rate-limit middleware stack. The MVP endpoint is therefore mounted under the existing `v2Api`. The wire body shape is unchanged, so the migration path described above (rerouting from `/v2/runs/:runId/attributes` to `/v2/runs/:id/events`) still holds, only within the same namespace.
|
|
351
351
|
|
|
352
352
|
### Concurrent writes: read-modify-write, not per-key atomic
|
|
353
353
|
|
|
354
354
|
The plan called for per-key atomic `UpdateExpression` updates (`SET #attrs.#k = :v` / `REMOVE #attrs.#k`) in the `world-vercel` backing store, on the basis that it eliminates the read-modify-write race. The MVP ships with the simpler read-modify-write path instead:
|
|
355
355
|
|
|
356
|
-
- **In `world-postgres
|
|
357
|
-
- **In the `world-vercel` backing service
|
|
358
|
-
- **In `world-local
|
|
356
|
+
- **In `world-postgres`**: The SQL-side `jsonb_set` / `-` chain *is* used and is genuinely atomic on the run row, so the only race is the cap check (a separate `SELECT`). Documented as LWW-by-arrival for the cap; the merge itself is atomic.
|
|
357
|
+
- **In the `world-vercel` backing service**: The attributes column is laid out as a native key-addressable map so the atomic `UpdateItem` variant can be enabled later without a data migration. The MVP commits the merged map through the existing entity update path. Two concurrent writers therefore race; whichever lands second wins on shared keys, and any write to a non-overlapping key is preserved.
|
|
358
|
+
- **In `world-local`**: An in-process per-run mutex serializes the read-merge-write sequence so parallel `experimental_setAttributes` calls from concurrent steps do not lose writes within a single process. There is a corresponding test that exercises 20 parallel writes to the same run.
|
|
359
359
|
|
|
360
360
|
This is consistent with the original "concurrent writes are LWW by arrival" caveat. Promoting to per-key atomic writes is a no-API-break change once the event-sourced path lands.
|
|
361
361
|
|
|
@@ -367,7 +367,7 @@ For the MVP the endpoint reuses the existing `WORKFLOW_EVENT` fact with `eventTy
|
|
|
367
367
|
|
|
368
368
|
### Validation rules are shared between SDK and world
|
|
369
369
|
|
|
370
|
-
Validation lives in a single helper exported from `@workflow/world` (`validateAttributeChanges`, `validateAttributeKey`, `validateAttributeValue`). Both the SDK `experimental_setAttributes` helper and the `world-local` / `world-postgres` implementations call it; the `world-vercel` backing service applies the same rules independently. The shared module is the authoritative spec for the limits (256-char keys, 256-byte values, max 64 attributes per run, `$`-prefixed keys reserved)
|
|
370
|
+
Validation lives in a single helper exported from `@workflow/world` (`validateAttributeChanges`, `validateAttributeKey`, `validateAttributeValue`). Both the SDK `experimental_setAttributes` helper and the `world-local` / `world-postgres` implementations call it; the `world-vercel` backing service applies the same rules independently. The shared module is the authoritative spec for the limits (256-char keys, 256-byte values, max 64 attributes per run, `$`-prefixed keys reserved), so any future change goes through one file.
|
|
371
371
|
|
|
372
372
|
### Run row reconstruction had to thread `attributes` through
|
|
373
373
|
|
|
@@ -375,6 +375,6 @@ Validation lives in a single helper exported from `@workflow/world` (`validateAt
|
|
|
375
375
|
|
|
376
376
|
### Optional world method: feature-detect, warn once
|
|
377
377
|
|
|
378
|
-
`runs.experimentalSetAttributes` is optional on the `World` interface so community worlds (Redis, MongoDB, Turso, etc.) continue to build and run without adopting the experimental API. The SDK helper feature-detects the method's presence on first dispatch; if absent, it logs a single `console.warn` for the lifetime of the process and resolves silently for that call and all subsequent calls. Users do not need to feature-detect in their own code
|
|
378
|
+
`runs.experimentalSetAttributes` is optional on the `World` interface so community worlds (Redis, MongoDB, Turso, etc.) continue to build and run without adopting the experimental API. The SDK helper feature-detects the method's presence on first dispatch; if absent, it logs a single `console.warn` for the lifetime of the process and resolves silently for that call and all subsequent calls. Users do not need to feature-detect in their own code: calling `experimental_setAttributes` against an unsupporting world is safe but ineffective.
|
|
379
379
|
|
|
380
380
|
See "Test coverage" above for the full test surface that ships with this change.
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Batched event writes
|
|
3
|
+
description: An optional World API (events.createBatch) that appends an ordered set of events in one durable write with per-event outcomes, and a suspension fan-out fold that uses it.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Batched event writes (`events.createBatch`)
|
|
7
|
+
|
|
8
|
+
## Motivation
|
|
9
|
+
|
|
10
|
+
A workflow suspension that schedules several steps and waits previously wrote one event per entity: one `world.events.create` call per `step_created` and `wait_created`. Against a remote World each write is its own network round trip and its own crash boundary. Batching folds a suspension's schedule into **one durable write** with per-event outcomes, cutting request count and making the whole fan-out land atomically per attempt.
|
|
11
|
+
|
|
12
|
+
## The World spec addition
|
|
13
|
+
|
|
14
|
+
`Storage['events']` gains one **optional** method:
|
|
15
|
+
|
|
16
|
+
```ts
|
|
17
|
+
import type {
|
|
18
|
+
BatchEventRequest,
|
|
19
|
+
CreateEventBatchParams,
|
|
20
|
+
EventBatchResult,
|
|
21
|
+
} from '@workflow/world';
|
|
22
|
+
|
|
23
|
+
interface BatchCapableEvents {
|
|
24
|
+
createBatch?(
|
|
25
|
+
runId: string,
|
|
26
|
+
events: BatchEventRequest[],
|
|
27
|
+
params?: CreateEventBatchParams
|
|
28
|
+
): Promise<EventBatchResult>;
|
|
29
|
+
}
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
The supporting types, excerpted (canonical definitions live in `@workflow/world`):
|
|
33
|
+
|
|
34
|
+
{/* @skip-typecheck illustrative excerpts of the canonical @workflow/world types */}
|
|
35
|
+
```ts
|
|
36
|
+
interface BatchEventRequest {
|
|
37
|
+
/** The event: the same discriminated union the single `create` takes. */
|
|
38
|
+
event: CreateEventRequest;
|
|
39
|
+
/** Client event time; under slot identity, the source of the durable createdAt. */
|
|
40
|
+
occurredAt?: Date;
|
|
41
|
+
/** Per-event compute attribution, same as the single create's CreateEventParams. */
|
|
42
|
+
computeInstanceId?: string;
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
type BatchEventItemResult =
|
|
46
|
+
| { status: 200; event: Event; run?: WorkflowRun; step?: Step; wait?: Wait }
|
|
47
|
+
| { status: number; error: string; message: string };
|
|
48
|
+
|
|
49
|
+
interface EventBatchResult {
|
|
50
|
+
/** One entry per submitted event, in request order. */
|
|
51
|
+
results: BatchEventItemResult[];
|
|
52
|
+
}
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
The contract:
|
|
56
|
+
|
|
57
|
+
- **Ordered**: events land in the run's log in request order at consecutive slots. A concurrent writer may push the whole batch to slots above the caller's view; no skipped-event report accompanies the batch result, so a position-tracking caller compares committed slots against its expectation and reloads to observe what interleaved (its local view stays a strict prefix of the log, never a hole).
|
|
58
|
+
- **Per-event outcomes**: the batch is processed as a whole, and each event reports what its own single `create` would have returned: `200` plus the materialized entity, or the single-path status/code (`409`/`conflict` for an event an earlier delivery already applied). Callers reuse their single-path conflict handling per event.
|
|
59
|
+
- **Idempotent on retry, for entity-conditioned shapes**: creates, terminal transitions, and the born-running pair are each guarded by their own entity condition, so retrying a batch of them that (partially) committed converges to per-event `409`s with nothing written twice. A standalone bare `step_started` or a `step_retrying` re-patches its step instead of converging, so `world-vercel` only auto-retries batches whose every event is retry-convergent (everything the runtime folds today is), and rejects `hook_received` in a batch outright.
|
|
60
|
+
- **Method presence is the capability declaration.** A World that doesn't implement it keeps the single-event path; a World that implements it must make each attempt atomic (a lost race leaves nothing behind). `world-vercel` implements it against `POST /v4/runs/:runId/events/batch` for slot-identity runs with `specVersion >= 6`. `world-local` and `world-postgres` deliberately do not because batching provides no benefit for a local write.
|
|
61
|
+
- **Not batchable** (Worlds reject the request): `run_created`, `run_started`, `run_cancelled`, `hook_created`, `hook_disposed`, `attr_set`, and multiple events targeting one entity, except `step_created` followed by `step_started` for the same step, which creates the step born-running.
|
|
62
|
+
|
|
63
|
+
## The runtime integration (suspension fan-out fold)
|
|
64
|
+
|
|
65
|
+
**On by default.** The suspension handler folds a **clean fan-out** (the suspension's eager `step_created` and `wait_created` writes) into `createBatch` calls of at most 32 events (mirroring the server's transaction budgets). Chunks of a larger fan-out commit **concurrently**: slot assignment is the World's, so parallel chunks race for slot ranges exactly like the pre-fold path's parallel single writes did, and per-entity conditions, not commit order, carry correctness. The fold only engages when the World implements `createBatch`, the run is on slot identity, and the suspension carries no attribute writes, no hook writes, and no resilient step dispatch; everything else keeps the single-event path byte-for-byte.
|
|
66
|
+
|
|
67
|
+
**Per-chunk continuation.** Each chunk's follow-on work starts the moment **that chunk** commits, not when the whole fold does: a chunk's step-execution queue messages publish right off its own commit (publish-after-create holds per step), and only the chunk carrying the inline pairs gates the replay's continuation: trailing chunks' commits and publishes are joined before the invocation can acknowledge its message, so the durability contract ("every create durable before ack") is unchanged.
|
|
68
|
+
|
|
69
|
+
**Pre-claimed inline pairs.** When the fold engages and has company for them (at least two inline steps, or one plus other batchable events), the steps the runtime is about to execute inline join the batch as adjacent `[step_created, step_started]` pairs: the created row carrying the input, the started row a bare ownership-stamped claim the World folds into a born-running create. The inline bodies start straight off the pair chunk's commit (in parallel with the queue publishes and any trailing chunks) with no per-step claim POST at all, and a pair that loses its atomic create-claim to a concurrent delivery skips its body exactly as a lost lazy claim does. A lone inline step with nothing else to batch keeps the optimistic lazy-start path, whose claim overlaps the body.
|
|
70
|
+
|
|
71
|
+
Per-event `409`s are tolerated the same way the single path tolerates `EntityConflictError` (a concurrent delivery already created the entity); any other per-event failure fails the suspension write the way a single-path rejection would. A batch carrying a `step_started` (that is, any batch with inline pairs) is **not** retried in-process on a transport blip: a pair's `409` cannot be told apart from the caller's own earlier attempt having committed it, so recovery goes through queue redelivery instead, where the step's ownership stamp routes it back to the same invocation.
|
|
72
|
+
|
|
73
|
+
`createBatch` is optional, so today only the Vercel World folds at all: every other World keeps the single-event path and never sends a pair.
|
|
74
|
+
|
|
75
|
+
**Escape hatch**: Set `WORKFLOW_BATCH_TRANSITIONS=0` (or `false`) to disable batching and restore the exact prior one-write-per-event path. See [`WORKFLOW_BATCH_TRANSITIONS`](/docs/configuration/worlds#workflow_batch_transitions).
|
|
76
|
+
|
|
77
|
+
## Follow-up
|
|
78
|
+
|
|
79
|
+
The deferred sequential transition (holding `step_completed(N)` across the replay turn and committing `[step_completed(N), step_created(N+1), step_started(N+1)]` as one batch at the next lazy start) builds on this contract and ships separately.
|