workflow 5.0.0-beta.4 → 5.0.0-beta.41

package/docs/cookbook/advanced/child-workflows.mdx

@@ -1,10 +1,16 @@
 ---
 title: Child Workflows
-description: Spawn child workflows from a parent and poll their progress for batch processing, report generation, and other multi-workflow orchestration scenarios.
+description: Spawn child workflows from a parent and wait for completion via hook resume.
 type: guide
-summary: Orchestrate independent child workflows from a parent workflow using start(), sleep(), and getRun() to fan out work with isolated failure boundaries.
+summary: Orchestrate independent child workflows from a parent using start(), defineHook(), and startAndWait() — the child resumes the parent's hook when done instead of polling getRun().status.
+related:
+  - /docs/api-reference/workflow-api/start
 ---
 
+<CopyPrompt
+  text="Refactor this workflow to use child workflows. Keep the parent as an exported `&quot;use workflow&quot;` function. Move independent units of durable work into separate exported child workflow functions. From the parent, call `start(childWorkflow, [args])` from `workflow/api` or the documented `startAndWait`/hook pattern where completion must resume the parent. Pass only serializable state to children. For fan-out, start children in parallel with `Promise.all` or bounded batches, collect run IDs, handle partial failures with `Promise.allSettled`, and use `getRun(runId)` when status, cancellation, streams, or return values are needed. Verify child start, completion, failure, and parent resume behavior."
+/>
+
 Use child workflows when a single workflow needs to orchestrate many independent units of work. Each child runs as its own workflow with a separate event log, retry boundary, and failure scope -- if one child fails, it doesn't take down the parent or siblings.
 
 ## When to use child workflows
@@ -18,20 +24,68 @@ Child workflows are the right choice when:
 
 For simpler cases where steps share a single event log, use [direct await composition](/cookbook/common-patterns/workflow-composition#direct-await-flattening) instead.
 
-## Basic pattern: spawn and poll
+## Basic pattern: spawn and wait via hook
 
-The core pattern has three parts:
+The recommended pattern has four parts:
 
-1. A **step** that calls `start()` to spawn a child workflow and returns the run ID
-2. A **polling loop** in the parent workflow that checks child status with `getRun()`
-3. A **step** that retrieves the child's return value once it completes
+1. A **completion hook** the parent creates and awaits — zero compute while waiting
+2. A **wrapped child export** that runs the real child in try/catch/finally and resumes the parent's hook from a step in `finally`
+3. A **`start()` call** that spawns the wrapped child with the hook token (directly from the workflow in v5)
+4. A **`startAndWait()` helper** that ties the hook, spawn, and typed result together
 
 ```typescript
-import { sleep } from "workflow";
-import { getRun, start } from "workflow/api";
+import { defineHook, getWorkflowMetadata } from "workflow";
+import { start } from "workflow/api";
+import { z } from "zod";
+
+declare function fetchDocument(documentId: string): Promise<string>; // @setup
+declare function analyzeContent(content: string): Promise<string>; // @setup
+declare function generateSummary(analysis: string): Promise<string>; // @setup
 
-declare function pollUntilComplete(runIds: string[]): Promise<void>; // @setup
-declare function collectResults(runIds: string[]): Promise<Array<{ documentId: string; summary: string }>>; // @setup
+const childCompletionHook = defineHook({
+  schema: z.discriminatedUnion("status", [
+    z.object({ status: z.literal("completed"), value: z.unknown() }),
+    z.object({ status: z.literal("failed"), error: z.string() }),
+  ]),
+});
+
+function completionToken(parentRunId: string, key: string) {
+  return `child-completion:${parentRunId}:${key}`;
+}
+
+async function resumeParentCompletion(
+  token: string,
+  result:
+    | { status: "completed"; value: unknown }
+    | { status: "failed"; error: string }
+) {
+  "use step";
+  await childCompletionHook.resume(token, result);
+}
+
+async function withChildCompletionHook<TResult>(
+  runChild: () => Promise<TResult>,
+  completionTokenArg: string
+) {
+  let result:
+    | { status: "completed"; value: TResult }
+    | { status: "failed"; error: string }
+    | undefined;
+
+  try {
+    const value = await runChild();
+    result = { status: "completed", value };
+  } catch (error) {
+    result = {
+      status: "failed",
+      error: error instanceof Error ? error.message : String(error),
+    };
+  } finally {
+    if (result) {
+      await resumeParentCompletion(completionTokenArg, result);
+    }
+  }
+}
 
 // Child workflow -- processes a single document
 export async function processDocument(documentId: string) {
@@ -44,163 +98,113 @@ export async function processDocument(documentId: string) {
   return { documentId, summary };
 }
 
-async function fetchDocument(documentId: string): Promise<string> {
-  "use step";
-  const res = await fetch(`https://docs.example.com/api/${documentId}`);
-  return res.text();
-}
+// Spawnable wrapper -- explicit export so `start()` can register it
+export async function processDocumentWithCompletion(
+  documentId: string,
+  completionTokenArg: string
+) {
+  "use workflow";
 
-async function analyzeContent(content: string): Promise<string> {
-  "use step";
-  // Call analysis API
-  return `analysis of ${content.length} chars`;
+  await withChildCompletionHook(
+    () => processDocument(documentId),
+    completionTokenArg
+  );
 }
 
-async function generateSummary(analysis: string): Promise<string> {
-  "use step";
-  // Generate summary from analysis
-  return `Summary: ${analysis}`;
+async function startAndWait<TResult>(
+  key: string,
+  startChild: (completionTokenArg: string) => Promise<void>
+): Promise<TResult> {
+  const { workflowRunId } = getWorkflowMetadata();
+  const token = completionToken(workflowRunId, key);
+  const hook = childCompletionHook.create({ token }); // [!code highlight]
+
+  await startChild(token);
+
+  const completion = await hook; // [!code highlight]
+  if (completion.status === "failed") {
+    throw new Error(completion.error);
+  }
+  return completion.value as TResult;
 }
 
 // Parent workflow -- orchestrates document processing
 export async function processDocumentBatch(documentIds: string[]) {
   "use workflow";
 
-  // Spawn a child workflow for each document
-  const runIds = await spawnChildren(documentIds);
-
-  // Poll until all children complete
-  await pollUntilComplete(runIds);
-
-  // Collect results
-  const results = await collectResults(runIds);
+  const results = await Promise.all(
+    documentIds.map((documentId) =>
+      startAndWait<{ documentId: string; summary: string }>(documentId, (token) =>
+        start(processDocumentWithCompletion, [documentId, token]).then(() => undefined) // [!code highlight]
+      )
+    )
+  );
 
   return { processed: results.length, results };
 }
-
-async function spawnChildren(
-  documentIds: string[]
-): Promise<string[]> {
-  "use step"; // [!code highlight]
-
-  const runIds: string[] = [];
-  for (const docId of documentIds) {
-    const run = await start(processDocument, [docId]); // [!code highlight]
-    runIds.push(run.runId);
-  }
-  return runIds;
-}
 ```
 
-### Polling loop
+### Why hooks instead of polling?
 
-The parent workflow polls child statuses in a loop, sleeping between checks. This is durable -- if the parent replays, the sleep and status checks replay from the event log.
+Polling with `getRun().status` in a `sleep()` loop works, but hook resume is preferable because:
 
-```typescript
-import { sleep } from "workflow";
-import { getRun } from "workflow/api";
-
-const POLL_INTERVAL = "30s";
-const MAX_POLL_ITERATIONS = 120; // 60 minutes at 30s intervals
-
-async function pollUntilComplete(runIds: string[]): Promise<void> {
-  let iteration = 0;
-
-  while (iteration < MAX_POLL_ITERATIONS) {
-    const status = await checkStatuses(runIds); // [!code highlight]
-
-    if (status.running === 0) {
-      if (status.failed > 0) {
-        throw new Error(
-          `${status.failed} of ${runIds.length} children failed`
-        );
-      }
-      return; // All completed successfully
-    }
+- **Zero compute while waiting** — the parent suspends on the hook instead of waking every poll interval
+- **Immediate wake-up** — the parent resumes as soon as the child finishes, not on the next poll tick
+- **Typed payloads** — the child sends `{ status, value | error }` directly; no separate `returnValue` fetch step
+- **No worker-pool pressure** — `Run#returnValue` polling inside steps can hold worker slots while waiting for children (see [Eager Processing](/docs/changelog/eager-processing))
 
-    iteration += 1;
-    await sleep(POLL_INTERVAL); // [!code highlight]
-  }
-
-  throw new Error("Timed out waiting for children to complete");
-}
-
-async function checkStatuses(
-  runIds: string[]
-): Promise<{ running: number; completed: number; failed: number }> {
-  "use step"; // [!code highlight]
-
-  let running = 0;
-  let completed = 0;
-  let failed = 0;
-
-  for (const runId of runIds) {
-    const run = getRun(runId); // [!code highlight]
-    const status = await run.status; // [!code highlight]
-
-    if (status === "completed") completed += 1;
-    else if (status === "failed" || status === "cancelled") failed += 1;
-    else running += 1; // pending, running
-  }
-
-  return { running, completed, failed };
-}
-
-async function collectResults(
-  runIds: string[]
-): Promise<Array<{ documentId: string; summary: string }>> {
-  "use step";
-
-  const results = [];
-  for (const runId of runIds) {
-    const run = getRun(runId);
-    const value = await run.returnValue;
-    results.push(value as { documentId: string; summary: string });
-  }
-  return results;
-}
-```
+When a parent calls a child workflow inline with `await` (flattened into the same run), the same wrapper and hook handshake still works — pass the token and `await processDocumentWithCompletion(...)` inside `startAndWait()` instead of calling `start()`.
 
 ## Fan-out pattern: chunked spawning
 
-When spawning hundreds of children, batch the `start()` calls to avoid overwhelming the system. Use multiple spawn steps, each launching a chunk of children.
+When spawning hundreds of children, batch the `start()` calls to avoid overwhelming the system. Each child still gets its own completion hook keyed by a stable identifier (document ID, report ID, index).
 
 ```typescript
 import { start } from "workflow/api";
 
-declare function pollUntilComplete(runIds: string[]): Promise<void>; // @setup
+declare function startAndWait<TResult>(
+  key: string,
+  startChild: (completionTokenArg: string) => Promise<void>
+): Promise<TResult>; // @setup
 
 const CHUNK_SIZE = 10;
 
-export async function largeReportBatch(reportConfigs: Array<{ id: string; query: string }>) {
+export async function largeReportBatch(
+  reportConfigs: Array<{ id: string; query: string }>
+) {
   "use workflow";
 
-  // Spawn children in chunks
-  const allRunIds: string[] = [];
+  const results = [];
   for (let i = 0; i < reportConfigs.length; i += CHUNK_SIZE) {
     const chunk = reportConfigs.slice(i, i + CHUNK_SIZE);
-    const runIds = await spawnReportChunk(chunk); // [!code highlight]
-    allRunIds.push(...runIds);
+    const chunkResults = await Promise.all(
+      chunk.map((config) =>
+        startAndWait<{ reportId: string; formatted: string }>(config.id, (token) =>
+          start(generateReportWithCompletion, [
+            config.id,
+            config.query,
+            token,
+          ]).then(() => undefined)
+        )
+      )
+    );
+    results.push(...chunkResults);
   }
 
-  // Poll until all complete
-  await pollUntilComplete(allRunIds);
-
-  const results = await collectReportResults(allRunIds);
   return { total: results.length, results };
 }
 
-async function spawnReportChunk(
-  configs: Array<{ id: string; query: string }>
-): Promise<string[]> {
-  "use step";
+async function generateReportWithCompletion(
+  reportId: string,
+  query: string,
+  completionTokenArg: string
+) {
+  "use workflow";
 
-  const runIds: string[] = [];
-  for (const config of configs) {
-    const run = await start(generateReport, [config.id, config.query]);
-    runIds.push(run.runId);
-  }
-  return runIds;
+  await withChildCompletionHook(
+    () => generateReport(reportId, query),
+    completionTokenArg
+  );
 }
 
 async function generateReport(reportId: string, query: string) {
@@ -213,160 +217,103 @@ async function generateReport(reportId: string, query: string) {
 
 declare function queryDatabase(reportId: string, query: string): Promise<string>; // @setup
 declare function formatReport(reportId: string, data: string): Promise<string>; // @setup
-
-declare function collectReportResults(
-  runIds: string[]
-): Promise<Array<{ reportId: string; formatted: string }>>; // @setup
+declare function withChildCompletionHook<TResult>(
+  runChild: () => Promise<TResult>,
+  completionTokenArg: string
+): Promise<void>; // @setup
 ```
 
 ## Error handling
 
 ### Tolerating partial failures
 
-Not every batch requires 100% success. Use `allowFailures` logic to let the parent continue when some children fail, while still surfacing the failures.
+Use `Promise.allSettled` with `startAndWait()` so one failing child doesn't abort siblings. The hook payload already carries `{ status: "failed", error }` — no status polling required.
 
 ```typescript
-import { sleep } from "workflow";
-import { getRun } from "workflow/api";
-
-const POLL_INTERVAL = "30s";
-const MAX_POLL_ITERATIONS = 120;
-
-async function pollWithPartialFailures(
-  runIds: string[],
-  maxFailureRate: number
-): Promise<{ completed: string[]; failed: string[] }> {
-  let iteration = 0;
-  const completedIds: string[] = [];
-  const failedIds: string[] = [];
-
-  while (iteration < MAX_POLL_ITERATIONS) {
-    const status = await checkDetailedStatuses(runIds);
-
-    completedIds.length = 0;
-    failedIds.length = 0;
-
-    for (const entry of status) {
-      if (entry.status === "completed") completedIds.push(entry.runId);
-      else if (entry.status === "failed" || entry.status === "cancelled")
-        failedIds.push(entry.runId);
-    }
-
-    const active = runIds.length - completedIds.length - failedIds.length;
-
-    // Check if failure rate exceeds threshold
-    const failureRate = failedIds.length / Math.max(1, runIds.length); // [!code highlight]
-    if (failureRate > maxFailureRate) { // [!code highlight]
-      throw new Error( // [!code highlight]
-        `Failure rate ${(failureRate * 100).toFixed(1)}% exceeds ` + // [!code highlight]
-        `threshold of ${(maxFailureRate * 100).toFixed(1)}%` // [!code highlight]
-      ); // [!code highlight]
-    } // [!code highlight]
-
-    if (active === 0) {
-      return { completed: completedIds, failed: failedIds };
-    }
-
-    iteration += 1;
-    await sleep(POLL_INTERVAL);
-  }
+import { start } from "workflow/api";
 
-  throw new Error("Timed out waiting for children");
-}
+declare function startAndWait<TResult>(
+  key: string,
+  startChild: (completionTokenArg: string) => Promise<void>
+): Promise<TResult>; // @setup
+declare function processDocumentWithCompletion(
+  documentId: string,
+  completionTokenArg: string
+): Promise<void>; // @setup
 
-async function checkDetailedStatuses(
-  runIds: string[]
-): Promise<Array<{ runId: string; status: string }>> {
-  "use step";
+export async function processDocumentBatchTolerant(documentIds: string[]) {
+  "use workflow";
 
-  const statuses = [];
-  for (const runId of runIds) {
-    const run = getRun(runId);
-    const status = await run.status;
-    statuses.push({ runId, status });
-  }
-  return statuses;
+  const settled = await Promise.allSettled(
+    documentIds.map((documentId) =>
+      startAndWait<{ documentId: string; summary: string }>(documentId, (token) =>
+        start(processDocumentWithCompletion, [documentId, token]).then(
+          () => undefined
+        )
+      )
+    )
+  );
+
+  const results = settled
+    .filter(
+      (entry): entry is PromiseFulfilledResult<{ documentId: string; summary: string }> =>
+        entry.status === "fulfilled"
+    )
+    .map((entry) => entry.value);
+
+  const failed = settled.filter((entry) => entry.status === "rejected").length;
+
+  return { processed: results.length, failed, results };
 }
 ```
 
 ### Retrying failed children
 
-When a child fails, the parent can spawn a replacement and continue polling. Track restart counts to prevent infinite retry loops.
+When a child fails, spawn a replacement with a fresh hook token. Track restart counts to prevent infinite retry loops.
 
 ```typescript
-import { sleep } from "workflow";
-
-declare function checkDetailedStatuses(runIds: string[]): Promise<Array<{ runId: string; status: string }>>; // @setup
-
-const POLL_INTERVAL = "30s";
-const MAX_POLL_ITERATIONS = 120;
-
-async function pollWithRetries(
-  initialRunIds: string[],
-  maxRestartsPerChild: number,
-  spawnReplacement: (index: number) => Promise<string>
-): Promise<void> {
-  const activeRuns = new Map<number, string>();
-  const restartCounts = new Map<number, number>();
-
-  initialRunIds.forEach((runId, index) => activeRuns.set(index, runId));
-
-  let iteration = 0;
-
-  while (iteration < MAX_POLL_ITERATIONS) {
-    const statuses = await checkDetailedStatuses(
-      Array.from(activeRuns.values())
-    );
-    const statusByRunId = new Map(
-      statuses.map((s) => [s.runId, s.status])
-    );
-
-    for (const [index, runId] of activeRuns.entries()) {
-      const status = statusByRunId.get(runId) ?? "running";
-
-      if (status === "completed") {
-        activeRuns.delete(index);
-        continue;
-      }
-
-      if (status === "failed" || status === "cancelled") {
-        const restarts = (restartCounts.get(index) ?? 0) + 1; // [!code highlight]
-        restartCounts.set(index, restarts); // [!code highlight]
-
-        if (restarts > maxRestartsPerChild) { // [!code highlight]
-          throw new Error( // [!code highlight]
-            `Child ${index} exceeded restart limit (${maxRestartsPerChild})` // [!code highlight]
-          ); // [!code highlight]
-        } // [!code highlight]
-
-        const newRunId = await spawnReplacement(index); // [!code highlight]
-        activeRuns.set(index, newRunId); // [!code highlight]
-      }
+declare function startAndWait<TResult>(
+  key: string,
+  startChild: (completionTokenArg: string) => Promise<void>
+): Promise<TResult>; // @setup
+declare function spawnProcessDocument(
+  documentId: string,
+  completionTokenArg: string
+): Promise<void>; // @setup
+
+async function startAndWaitWithRetries(
+  documentId: string,
+  maxRestarts: number
+): Promise<{ documentId: string; summary: string }> {
+  for (let attempt = 0; attempt <= maxRestarts; attempt++) {
+    try {
+      return await startAndWait<{ documentId: string; summary: string }>(
+        `${documentId}:${attempt}`,
+        (token) => spawnProcessDocument(documentId, token)
+      );
+    } catch (error) {
+      if (attempt === maxRestarts) throw error;
     }
-
-    if (activeRuns.size === 0) return;
-
-    iteration += 1;
-    await sleep(POLL_INTERVAL);
   }
 
-  throw new Error("Timed out waiting for children");
+  throw new Error("unreachable");
 }
 ```
 
 ## Tips
 
-- **`start()` must be called from a step**, not directly from a workflow function. Wrap it in a `"use step"` function.
-- **`getRun()` must also be called from a step.** The polling loop lives in the workflow, but the actual status check is a step.
-- **Set a max iteration count on polling loops** to prevent runaway workflows. Calculate the count from your expected max duration and poll interval.
-- **Use chunked spawning for large batches.** Spawning 500 children in a single step can time out. Break it into chunks of 10-50.
-- **Each child has its own retry semantics.** Steps inside child workflows retry independently. The parent only sees the child's final status.
-- **Use `deploymentId: "latest"`** if children should run on the most recent deployment. See the [`start()` API reference](/docs/api-reference/workflow-api/start#using-deploymentid-latest) for compatibility considerations.
+- **`defineHook().resume()` must be called from a step.** The wrapped child's `finally` block calls a step that resumes the parent hook.
+- **Export wrapped children at module scope.** The SDK registers `"use workflow"` functions statically — a runtime higher-order function returned from `withChildCompletionHook()` cannot be passed to `start()`.
+- **Use stable hook keys** — document ID, job ID, or index — so parallel children inside one parent run don't collide on tokens.
+- **Use chunked spawning for large batches.** Starting 500 children at once can create a large burst of work. Break it into chunks of 10-50.
+- **Each child has its own retry semantics.** Steps inside child workflows retry independently. The parent sees the final `{ status, value | error }` payload from the hook.
+- **Use `deploymentId: "latest"`** if children should run on the most recent deployment. See [Versioning](/docs/foundations/versioning) for the full model and the [`start()` API reference](/docs/api-reference/workflow-api/start#using-deploymentid-latest) for compatibility considerations.
 
 ## Key APIs
 
 - [`start()`](/docs/api-reference/workflow-api/start) -- spawn a new workflow run and get its run ID
-- [`getRun()`](/docs/api-reference/workflow-api/get-run) -- retrieve a workflow run's status and return value
-- [`sleep()`](/docs/api-reference/workflow/sleep) -- durably pause between polling iterations
+- [`defineHook()`](/docs/api-reference/workflow/define-hook) -- typed hook for parent/child completion handshakes
+- [`resumeHook()`](/docs/api-reference/workflow-api/resume-hook) -- resume a waiting parent from a step (called by the child wrapper)
+- [`getWorkflowMetadata()`](/docs/api-reference/workflow/get-workflow-metadata) -- read the parent run ID for deterministic hook tokens
 - [`"use workflow"`](/docs/foundations/workflows-and-steps) -- marks the orchestrator function
 - [`"use step"`](/docs/foundations/workflows-and-steps) -- marks functions with full Node.js access

package/docs/cookbook/advanced/meta.json

@@ -2,7 +2,7 @@
   "title": "Advanced",
   "pages": [
     "child-workflows",
-    "distributed-abort-controller",
+    "upgrading-workflows",
     "serializable-steps",
     "publishing-libraries"
   ]