workflow 5.0.0-beta.2 → 5.0.0-beta.21

package/docs/cookbook/common-patterns/idempotency.mdx

@@ -1,45 +1,25 @@
 ---
 title: Idempotency
-description: Ensure external side effects happen exactly once, even when steps are retried or workflows are replayed.
+description: Make step retries safe and coordinate duplicate workflow starts with hook tokens.
 type: guide
-summary: Use step IDs as idempotency keys for external APIs like Stripe so that retries and replays don't create duplicate charges.
+summary: Use step IDs for retry-safe external calls, and use deterministic hook tokens when duplicate requests must route to one active workflow.
 ---
 
-Workflow steps can be retried (on failure) and replayed (on cold start). If a step calls an external API that isn't idempotent, retries could create duplicate charges, send duplicate emails, or double-process records. Use idempotency keys to make these operations safe.
+Use idempotency when a retry or duplicate request should not repeat the underlying work. In Workflow, there are two common patterns: use the step ID for retry-safe external calls, and use hook tokens to coordinate duplicate workflow starts.
 
 ## When to use this
 
-- Charging a payment (Stripe, PayPal)
-- Sending transactional emails or SMS
-- Creating records in external systems where duplicates are harmful
-- Any step that has side effects in systems you don't control
+- A step charges a payment, sends an email, enqueues work, or creates an external record.
+- A route may receive duplicate requests that should map to one active workflow run.
 
-## Pattern: Step ID as idempotency key
+## Step idempotency
 
 Every step has a unique, deterministic `stepId` available via `getStepMetadata()`. Pass this as the idempotency key to external APIs:
 
 ```typescript
 import { getStepMetadata } from "workflow";
 
-declare function createCharge(customerId: string, amount: number): Promise<{ id: string }>; // @setup
-declare function sendReceipt(customerId: string, chargeId: string): Promise<void>; // @setup
-
-export async function chargeCustomer(customerId: string, amount: number) {
-  "use workflow";
-
-  const charge = await createCharge(customerId, amount);
-  await sendReceipt(customerId, charge.id);
-
-  return { customerId, chargeId: charge.id, status: "completed" };
-}
-```
-
-### Step function with idempotency key
-
-```typescript
-import { getStepMetadata } from "workflow";
-
-async function createCharge(
+export async function createCharge(
   customerId: string,
   amount: number
 ): Promise<{ id: string }> {
@@ -69,39 +49,43 @@ async function createCharge(
 
   return charge.json();
 }
-
-async function sendReceipt(customerId: string, chargeId: string): Promise<void> {
-  "use step";
-
-  const { stepId } = getStepMetadata();
-
-  await fetch("https://api.example.com/receipts", {
-    method: "POST",
-    headers: { "Idempotency-Key": stepId },
-    body: JSON.stringify({ customerId, chargeId }),
-  });
-}
 ```
 
-## Race condition caveats
+See [Step Idempotency](/docs/foundations/idempotency#step-idempotency) for why `stepId` is stable across retries and how to think about external API conflicts.
 
-Workflow does not currently provide distributed locking or true exactly-once delivery across concurrent runs. If two workflow runs could process the same entity concurrently:
+## Run idempotency
 
-- **Rely on the external API's idempotency** (like Stripe's `Idempotency-Key`) rather than checking a local flag.
-- **Don't use check-then-act patterns** like "read a flag, then write if not set" -- another run could read the same flag between your read and write.
+For duplicate workflow-start requests, derive a hook token from your domain key. You can avoid obvious duplicate starts by checking whether an active hook already owns that token before calling `start()`:
 
-If your external API doesn't support idempotency keys natively, consider adding a deduplication layer (e.g., a database unique constraint on the operation ID).
+```typescript
+import { getHookByToken, start } from "workflow/api";
+import { HookNotFoundError } from "workflow/errors";
+import { processOrder } from "./workflows/process-order";
+
+export async function POST(request: Request) {
+  const { orderId } = await request.json();
+  const token = `order:${orderId}`;
+
+  try {
+    const hook = await getHookByToken(token); // [!code highlight]
+    return Response.json({ runId: hook.runId, reused: true });
+  } catch (error) {
+    if (!HookNotFoundError.is(error)) throw error;
+  }
 
-## Tips
+  const run = await start(processOrder, [orderId]); // [!code highlight]
+  return Response.json({ runId: run.runId, reused: false });
+}
+```
 
-- **`stepId` is deterministic.** It's the same value across retries and replays of the same step, making it a reliable idempotency key.
-- **Always provide idempotency keys for non-idempotent external calls.** Even if you think a step won't be retried, cold-start replay will re-execute it.
-- **Handle 409/conflict as success.** If an external API returns "already processed," treat that as a successful result, not an error.
-- **Make your own APIs idempotent** where possible. Accept an idempotency key and return the cached result on duplicate requests.
+The workflow should create the deterministic hook and check `await hook.getConflict()` before duplicate-sensitive work — awaiting `getConflict()` suspends the workflow to commit the hook registration and resolves with the conflicting run when another active run already owns the token (or `null` once the hook is registered). See [Run idempotency](/docs/foundations/idempotency#run-idempotency) for the full pattern, including how to steer an active run with `resumeHook()` and how to handle the current race between `start()` and hook registration.
 
 ## Key APIs
 
-- [`"use workflow"`](/docs/api-reference/workflow/use-workflow) -- declares the orchestrator function
-- [`"use step"`](/docs/api-reference/workflow/use-step) -- declares step functions with full Node.js access
-- [`getStepMetadata()`](/docs/api-reference/step/get-step-metadata) -- provides the deterministic `stepId` for idempotency keys
+- [`"use workflow"`](/docs/foundations/workflows-and-steps#workflow-functions) -- declares the orchestrator function
+- [`"use step"`](/docs/foundations/workflows-and-steps#step-functions) -- declares step functions with full Node.js access
+- [`getStepMetadata()`](/docs/api-reference/workflow/get-step-metadata) -- provides the deterministic `stepId` for idempotency keys
+- [`createHook()`](/docs/api-reference/workflow/create-hook) -- creates a hook with an optional deterministic token
+- [`getHookByToken()`](/docs/api-reference/workflow-api/get-hook-by-token) -- finds the active hook for a token
+- [`resumeHook()`](/docs/api-reference/workflow-api/resume-hook) -- resumes the active hook when the duplicate request carries data
 - [`start()`](/docs/api-reference/workflow-api/start) -- starts a new workflow run

package/docs/cookbook/common-patterns/meta.json

@@ -2,14 +2,14 @@
   "title": "Common Patterns",
   "defaultOpen": true,
   "pages": [
+    "sequential-and-parallel",
+    "workflow-composition",
     "saga",
     "batching",
     "rate-limiting",
-    "fan-out",
     "scheduling",
+    "timeouts",
     "idempotency",
-    "webhooks",
-    "content-router",
-    "child-workflows"
+    "webhooks"
   ]
 }

package/docs/cookbook/common-patterns/rate-limiting.mdx

@@ -224,5 +224,5 @@ export async function downloadWithRetry(url: string) {
 - [`"use step"`](/docs/foundations/workflows-and-steps) -- marks functions that run with full Node.js access
 - [`RetryableError`](/docs/api-reference/workflow/retryable-error) -- signals the runtime to retry after a delay
 - [`FatalError`](/docs/api-reference/workflow/fatal-error) -- signals a permanent failure, skipping retries
-- [`getStepMetadata()`](/docs/api-reference/step/get-step-metadata) -- provides the current attempt number and step ID
+- [`getStepMetadata()`](/docs/api-reference/workflow/get-step-metadata) -- provides the current attempt number and step ID
 - [`sleep()`](/docs/api-reference/workflow/sleep) -- durable pause for circuit breaker cooldowns

package/docs/cookbook/common-patterns/saga.mdx

@@ -13,6 +13,12 @@ Use the saga pattern when a business transaction spans multiple services and you
 - Any sequence where partial completion leaves the system in an inconsistent state
 - Operations that need "all or nothing" semantics across external APIs
 
+## How it works
+
+1. Each forward step does work and registers a compensation function.
+2. If any step throws `FatalError`, the catch block runs compensations in reverse (LIFO) order to restore consistency.
+3. Regular errors are retried automatically (up to 3x by default). Use `FatalError` only for permanent failures where retrying won't help.
+
 ## Pattern
 
 Each step returns a result and pushes a compensation handler onto a stack. If a later step throws a `FatalError`, the workflow catches it and executes compensations in LIFO order.
@@ -34,23 +40,21 @@ export async function subscriptionUpgradeSaga(accountId: string, seats: number)
   const compensations: Array<() => Promise<void>> = [];
 
   try {
-    // Step 1: Reserve seats
     const reservationId = await reserveSeats(accountId, seats);
     compensations.push(() => releaseSeats(accountId, reservationId)); // [!code highlight]
 
-    // Step 2: Capture payment
     const invoiceId = await captureInvoice(accountId, seats);
     compensations.push(() => refundInvoice(accountId, invoiceId)); // [!code highlight]
 
-    // Step 3: Provision access
     const entitlementId = await provisionSeats(accountId, seats);
     compensations.push(() => deprovisionSeats(accountId, entitlementId)); // [!code highlight]
 
-    // Step 4: Notify
+    // No compensation — notifications are fire-and-forget
     await sendConfirmation(accountId, invoiceId, entitlementId);
+
     return { status: "completed" };
   } catch (error) {
-    // Unwind compensations in reverse order
+    // Unwind compensations in reverse (LIFO) order
     for (const compensate of compensations.reverse()) { // [!code highlight]
       await compensate(); // [!code highlight]
     }
@@ -62,11 +66,13 @@ export async function subscriptionUpgradeSaga(accountId: string, seats: number)
 
 ### Step functions
 
-Each step is a `"use step"` function with full Node.js access. Forward steps do the work; compensation steps undo it.
+Each step is a `"use step"` function with full Node.js access (fetch, fs, npm packages). Forward steps do the work and throw `FatalError` on permanent failure; compensation steps undo it and must be idempotent — safe to call multiple times if the workflow restarts mid-rollback.
 
 ```typescript
 import { FatalError } from "workflow";
 
+// Forward steps
+
 async function reserveSeats(accountId: string, seats: number): Promise<string> {
   "use step";
   const res = await fetch(`https://api.example.com/seats/reserve`, {
@@ -78,15 +84,6 @@ async function reserveSeats(accountId: string, seats: number): Promise<string> {
   return reservationId;
 }
 
-async function releaseSeats(accountId: string, reservationId: string): Promise<void> {
-  "use step";
-  // Compensations should be idempotent — safe to call twice
-  await fetch(`https://api.example.com/seats/release`, {
-    method: "POST",
-    body: JSON.stringify({ accountId, reservationId }),
-  });
-}
-
 async function captureInvoice(accountId: string, seats: number): Promise<string> {
   "use step";
   const res = await fetch(`https://api.example.com/invoices`, {
@@ -98,14 +95,6 @@ async function captureInvoice(accountId: string, seats: number): Promise<string>
   return invoiceId;
 }
 
-async function refundInvoice(accountId: string, invoiceId: string): Promise<void> {
-  "use step";
-  await fetch(`https://api.example.com/invoices/${invoiceId}/refund`, {
-    method: "POST",
-    body: JSON.stringify({ accountId }),
-  });
-}
-
 async function provisionSeats(accountId: string, seats: number): Promise<string> {
   "use step";
   const res = await fetch(`https://api.example.com/entitlements`, {
@@ -117,14 +106,6 @@ async function provisionSeats(accountId: string, seats: number): Promise<string>
   return entitlementId;
 }
 
-async function deprovisionSeats(accountId: string, entitlementId: string): Promise<void> {
-  "use step";
-  await fetch(`https://api.example.com/entitlements/${entitlementId}`, {
-    method: "DELETE",
-    body: JSON.stringify({ accountId }),
-  });
-}
-
 async function sendConfirmation(
   accountId: string,
   invoiceId: string,
@@ -136,17 +117,131 @@ async function sendConfirmation(
     body: JSON.stringify({ accountId, invoiceId, entitlementId, template: "upgrade-complete" }),
   });
 }
+
+// Compensation steps — must be idempotent
+
+async function releaseSeats(accountId: string, reservationId: string): Promise<void> {
+  "use step";
+  await fetch(`https://api.example.com/seats/release`, {
+    method: "POST",
+    body: JSON.stringify({ accountId, reservationId }),
+  });
+}
+
+async function refundInvoice(accountId: string, invoiceId: string): Promise<void> {
+  "use step";
+  await fetch(`https://api.example.com/invoices/${invoiceId}/refund`, {
+    method: "POST",
+    body: JSON.stringify({ accountId }),
+  });
+}
+
+async function deprovisionSeats(accountId: string, entitlementId: string): Promise<void> {
+  "use step";
+  await fetch(`https://api.example.com/entitlements/${entitlementId}`, {
+    method: "DELETE",
+    body: JSON.stringify({ accountId }),
+  });
+}
+```
+
+### Streaming step progress (optional)
+
+Use `getWritable()` to stream progress events to a UI so users can see each step execute in real time.
+
+```typescript
+import { FatalError } from "workflow";
+import { getWritable } from "workflow";
+
+type SagaEvent =
+  | { type: "step_start"; step: string }
+  | { type: "step_done"; step: string; detail: string }
+  | { type: "step_failed"; step: string; error: string }
+  | { type: "compensating"; step: string }
+  | { type: "compensated"; step: string }
+  | { type: "result"; status: "completed" | "rolled_back" };
+
+async function emit(event: SagaEvent) {
+  "use step";
+  const writer = getWritable<SagaEvent>().getWriter();
+  try {
+    await writer.write(event);
+  } finally {
+    writer.releaseLock();
+  }
+}
+
+declare function reserveSeats(accountId: string, seats: number): Promise<string>; // @setup
+declare function releaseSeats(accountId: string, reservationId: string): Promise<void>; // @setup
+declare function captureInvoice(accountId: string, seats: number): Promise<string>; // @setup
+declare function refundInvoice(accountId: string, invoiceId: string): Promise<void>; // @setup
+declare function provisionSeats(accountId: string, seats: number): Promise<string>; // @setup
+declare function deprovisionSeats(accountId: string, entitlementId: string): Promise<void>; // @setup
+declare function sendConfirmation(accountId: string, invoiceId: string, entitlementId: string): Promise<void>; // @setup
+
+export async function subscriptionUpgradeSaga(accountId: string, seats: number) {
+  "use workflow";
+
+  const compensations: Array<{ name: string; execute: () => Promise<void> }> = [];
+
+  try {
+    await emit({ type: "step_start", step: "Reserve Seats" });
+    const reservationId = await reserveSeats(accountId, seats);
+    compensations.push({ name: "Release Seats", execute: () => releaseSeats(accountId, reservationId) });
+    await emit({ type: "step_done", step: "Reserve Seats", detail: reservationId });
+
+    await emit({ type: "step_start", step: "Capture Invoice" });
+    const invoiceId = await captureInvoice(accountId, seats);
+    compensations.push({ name: "Refund Invoice", execute: () => refundInvoice(accountId, invoiceId) });
+    await emit({ type: "step_done", step: "Capture Invoice", detail: invoiceId });
+
+    await emit({ type: "step_start", step: "Provision Seats" });
+    const entitlementId = await provisionSeats(accountId, seats);
+    compensations.push({ name: "Deprovision Seats", execute: () => deprovisionSeats(accountId, entitlementId) });
+    await emit({ type: "step_done", step: "Provision Seats", detail: entitlementId });
+
+    // No compensation — notifications are fire-and-forget
+    await emit({ type: "step_start", step: "Send Confirmation" });
+    await sendConfirmation(accountId, invoiceId, entitlementId);
+    await emit({ type: "step_done", step: "Send Confirmation", detail: "sent" });
+
+    await emit({ type: "result", status: "completed" });
+    return { status: "completed" };
+  } catch (error) {
+    const errorMessage = error instanceof Error ? error.message : "Unknown error";
+    await emit({ type: "step_failed", step: "failed", error: errorMessage });
+
+    // Unwind compensations in reverse (LIFO) order
+    for (const comp of compensations.reverse()) {
+      await emit({ type: "compensating", step: comp.name });
+      await comp.execute();
+      await emit({ type: "compensated", step: comp.name });
+    }
+
+    await emit({ type: "result", status: "rolled_back" });
+    return { status: "rolled_back" };
+  }
+}
 ```
 
+## Adapting to your use case
+
+- Replace the step functions with real API calls. Each `"use step"` function has full Node.js access.
+- Add or remove steps as needed — the pattern scales to any number of steps.
+- Make compensations idempotent — they may be retried if the workflow restarts mid-rollback.
+- The `emit()` calls and `SagaEvent` type are optional — remove them if you don't need real-time UI progress.
+
 ## Tips
 
 - **Use `FatalError` for permanent failures.** Regular errors trigger automatic retries (up to 3 by default). Throw `FatalError` when retrying won't help (e.g., insufficient funds, invalid input).
 - **Make compensations idempotent.** If a compensation step is retried, it should produce the same result. Check whether the resource was already released before releasing it again.
 - **Compensation steps are also `"use step"` functions.** This makes them durable — if the workflow restarts mid-rollback, it resumes where it left off.
 - **Capture values in closures carefully.** Use block-scoped variables or copy values before pushing compensations to avoid referencing stale state.
+- **Notifications don't need compensations.** Fire-and-forget steps like sending emails or Slack messages typically don't register a compensation.
 
 ## Key APIs
 
-- [`"use workflow"`](/docs/api-reference/workflow/use-workflow) -- declares the orchestrator function
-- [`"use step"`](/docs/api-reference/workflow/use-step) -- declares step functions with full Node.js access
+- [`"use workflow"`](/docs/foundations/workflows-and-steps#workflow-functions) -- declares the orchestrator function
+- [`"use step"`](/docs/foundations/workflows-and-steps#step-functions) -- declares step functions with full Node.js access
 - [`FatalError`](/docs/api-reference/workflow/fatal-error) -- non-retryable error that triggers compensation
+- [`getWritable()`](/docs/api-reference/workflow/get-writable) -- streams data from workflows for real-time UI updates