@smthrs/flow 0.0.0-stage → 1.0.0-rc.3

package/docs/guides/queue-work-to-a-worker.md

@@ -0,0 +1,122 @@
+---
+title: "Hand work to a durable worker"
+description: "Declare a durable queue, suspend a flow until a worker records its result, and run the worker as a layer with bounded concurrency."
+sidebar:
+  order: 10
+---
+
+A `DurableQueue` moves work out of the flow that asked for it. The flow offers a
+payload, parks, and resumes when a worker persists a result. Requests and recorded
+results survive the process that made them.
+
+## Declare the queue
+
+```ts
+import { DurableQueue } from "@smthrs/flow"
+import * as Schema from "effect/Schema"
+
+export const Renders = DurableQueue.make({
+  name: "Renders",
+  payload: Schema.Struct({ page: Schema.String }),
+  success: Schema.String,
+  idempotencyKey: ({ page }) => page
+})
+```
+
+`idempotencyKey` is required. The payload key scopes an occurrence counter;
+the execution ID and occurrence ordinal also contribute to the queue item ID.
+Distinct offers of the same page create distinct items, including within one
+execution. Replaying the same occurrence derives the same ID for queue deduplication.
+Business-level deduplication across distinct offers or runs is the caller's job.
+`success` defaults to `Schema.Void` and `error` to `Schema.Never`.
+
+## Offer work from a flow
+
+```ts
+const rendered = DurableQueue.process(Renders, { page: "home" })
+```
+
+`process` offers the payload under the name `DurableQueue/<name>`, attaches a
+completion token, and suspends the execution until a worker records the handler's
+exit against it. The success and error channels are the queue's declared schemas,
+so the caller sees the worker's own outcome. While suspended, the execution's
+waiting annotation has reason `event` and the item's completion token.
+
+Invalid payloads and offer encoding failures die with `SchemaError` without
+retrying. The final offer failure of an exhausted retry schedule also becomes a
+defect. The typed error channel is reserved for the worker's declared error.
+
+`retrySchedule` bounds how a failing offer is retried. The default retries with
+exponential delays capped at one minute and never gives up, so a caller that
+wants a bound supplies its own:
+
+```ts
+import * as Schedule from "effect/Schedule"
+
+const bounded = DurableQueue.process(Renders, { page: "home" }, {
+  retrySchedule: Schedule.recurs(5)
+})
+```
+
+## Run the worker
+
+`DurableQueue.worker` is the layer form: it forks the worker into the layer's
+scope, so it starts with the composition and stops with it.
+
+```ts
+import * as Effect from "effect/Effect"
+
+export const RendersWorker = DurableQueue.worker(
+  Renders,
+  ({ page }) => Effect.succeed(`<html>${page}</html>`),
+  { concurrency: 4, maxAttempts: 10 }
+)
+```
+
+`concurrency` defaults to `1` and must be a positive safe integer;
+`DurableQueue.makeWorker` and `DurableQueue.worker` throw a `RangeError`
+otherwise. `makeWorker` is the underlying effect, for a host that wants to fork
+it itself.
+
+The handler's exit is what the waiting flow receives. A handler that fails with
+the queue's declared error type resumes the caller with that typed failure; a
+handler that dies resumes it with a defect. An interrupt-only handler exit records
+nothing and requeues the item without consuming an attempt. Mixed failures are
+recorded with interrupt reasons removed.
+
+Handler execution is at least once: if recording its result fails, a subsequent
+take runs the handler again. Make side effects idempotent using a business key in
+the payload. `maxAttempts` is passed to the persisted queue and defaults to `10`
+on both worker constructors. Failed takes consume attempts; interrupt-only exits
+do not. A declared handler failure that is successfully recorded completes the
+item instead of retrying it.
+
+A worker completes only the wait its own `process` call created: an item stored
+under ID `key` must carry a token for `DurableQueue/<name>/<key>`. An item whose
+token is malformed or names any other wait point is logged at error level and
+acknowledged without running the handler. The worker links its span to the
+offering span only when the stored trace and span ids are 32 and 16 lowercase hex
+characters.
+
+A take failure after the handler produces a recordable exit is logged at error
+level with the queue name, item ID, completion token, attempt, attempt limit, and
+whether that limit was reached. At exhaustion the store drops or quarantines the
+item according to its policy. If completion writes exhaust the attempts without
+recording a result, the caller remains suspended and requires recovery by the
+host. Raising `maxAttempts` allows more retries but does not guarantee completion.
+
+## When to reach for it
+
+A durable queue is the right shape when the work has a different lifetime or a
+different scaling story from the flow that asked for it: a render farm, a batch
+of uploads, anything where the caller should park rather than hold a slot. When
+the work belongs to the same run, an action is simpler and keeps the step in the
+plan. When it is its own unit of work with its own journal,
+[a child execution](./run-a-child-flow.md) is the closer fit.
+
+## Related pages
+
+- [Wait for an external signal](./wait-for-an-external-signal.md): the deferred
+  and token machinery the queue's reply channel is built on.
+- [Suspension and replay](../concepts/suspension-and-replay.md): what the caller
+  is doing while the worker works.

package/docs/guides/retry-a-failing-action.md

@@ -0,0 +1,243 @@
+---
+title: "Retry a failing action"
+description: "Declare a retry policy, understand where the engine makes its retry decision, keep integrity failures unretried, and read the two terminal defects."
+sidebar:
+  order: 3
+---
+
+A `RetryPolicy` is plain data, not a `Schedule`. That is deliberate: the next
+delay is derived from a **persisted** attempt count rather than from fiber-local
+state, so a backoff ladder survives a park, a resume, and the death of the
+process that started it.
+
+## Declare a policy
+
+```ts
+import { RetryPolicy } from "@smthrs/flow"
+
+const policy = RetryPolicy.make({
+  initialMs: 200,
+  factor: 2,
+  maxMs: 10_000,
+  maxAttempts: 5,
+  expirationMs: 5 * 60_000,
+  jitterRatio: 0.2,
+  nonRetryable: ["payments/ChargeDeclined"]
+})
+```
+
+`RetryPolicy.make` checks every bound and throws a `RangeError` naming the field
+that is wrong: `initialMs` finite and greater than zero, `factor` finite and
+positive, `maxMs` finite and not below `initialMs`, `maxAttempts` a safe integer
+of at least one, `expirationMs` finite and positive, and `jitterRatio` finite
+and within zero and one inclusive. `initialMs: 0` is refused rather than read as
+"retry immediately": every computed delay would be zero, which `nextDelay`
+treats as exhausted, so the policy would give up on the first failure whatever
+`maxAttempts` promised. Use `initialMs: 1` for a near-immediate retry.
+`jitterRatio: 0` disables jitter. The
+`nonRetryable` array is copied and frozen, so mutating your array later cannot
+change what a parked policy means.
+
+`RetryPolicy.defaultRetryPolicy` is `{ initialMs: 200, factor: 1.5, maxMs: 30000 }`.
+It declares neither `maxAttempts` nor `expirationMs`, so it never gives up. Bound
+a long-lived retry with `expirationMs` when a wall-clock give-up is required.
+
+## Attach it to the work
+
+The engine reads `action.retryPolicy` at dispatch. Both forms of `Action.make`
+take it; the [`Action.make` reference](../reference/flow.md#actionmake) lists
+every option. The inline form:
+
+```ts
+import { Action } from "@smthrs/flow"
+import * as Effect from "effect/Effect"
+import * as Schema from "effect/Schema"
+
+const settle = Action.make({
+  name: "payments/Settle",
+  success: Schema.String,
+  error: Schema.String,
+  tier: "irreversible",
+  idempotencyKey: "settle:checkout-v1",
+  retryPolicy: policy,
+  execute: Effect.suspend(() => callProvider())
+})
+```
+
+A **declared** action, the kind a body names, takes the same option:
+
+```ts
+const Settle = Action.make("payments/Settle", {
+  payload: { orderId: Schema.String },
+  success: Schema.String,
+  error: Schema.String,
+  retryPolicy: policy
+})
+```
+
+Two other tools cover different needs:
+
+- Dispatch an inline action from inside an implementation when a nested step
+  needs its own record and its own attempt sequence.
+- Wrap work in `Action.retry`, which is `Effect.retry` with the durable attempt
+  context threaded through it, for a retry the policy does not describe.
+
+```ts
+const attempts = Action.retry(settle, { times: 3 })
+```
+
+`Action.retry` updates `Action.CurrentAttempt` on each attempt and pins ordinals
+per allocation scope, so every attempt of one sequence reuses its own action's
+ordinals instead of drawing new step keys. A nested block shares the enclosing
+block's pinned slot, so a completed inner dispatch replays rather than
+re-executing.
+
+## Infrastructure failures without a declared policy
+
+When no `retryPolicy` is declared, Dispatch uses `RetryPolicy.transient` only
+for a typed failure whose `Fault.of(error).class` is `infra`, and only when the
+action has an `idempotencyKey` or explicitly declares an empty `effects.writes`.
+A missing effect declaration does not promise repeat safety. Other failures
+return immediately. A declared policy retains its existing behavior.
+
+The transient policy starts at 5 seconds, doubles to a 5-minute cap, and expires
+2 hours after the first attempt. Durable engines retain the origin and attempt
+count across recovery. Exhaustion preserves the final error and its fault class.
+An unkeyed irreversible action still cannot retry under the irreversible guard.
+
+For plain effects that the caller knows are safe to repeat:
+
+```ts
+import { Fault } from "@smthrs/flow"
+
+const recovered = Fault.retryTransient(readStatus)
+```
+
+This helper retries typed infrastructure failures under the same bounded policy.
+It preserves the final typed error; defects and cancellation propagate directly.
+Host command adapters can use `Unreachable.classifyExit(stderr)` from
+`@smthrs/kernel` to turn DNS, connection, HTTP 429, and HTTP 5xx failures into
+typed infrastructure errors. Ordinary command failures remain domain errors.
+
+## What the engine decides, and when
+
+`RetryPolicy.decide` is the engine's single retry decision point, and
+non-retryable classification happens there and nowhere else. On a failing
+attempt the engine asks the policy, passing the current attempt, the error, and
+the elapsed time since the first attempt:
+
+| Decision                 | What happens                                                                                                                                                   |
+| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `RetryAfter(delayMs)`    | The engine sleeps and dispatches attempt `n + 1`. An irreversible action with no `idempotencyKey` dies with `IrreversibleRetryRequiresIdempotencyKey` instead. |
+| `GiveUp("nonRetryable")` | The original typed failure propagates unchanged.                                                                                                               |
+| `GiveUp("exhausted")`    | The final declared failure propagates unchanged.                                                                                                               |
+| `GiveUp("expired")`      | The final declared failure propagates unchanged.                                                                                                               |
+
+A spent retry policy preserves the action's error channel. `Node.catch` and
+ordinary typed Effect recovery can handle the final failure. The execution span
+records `retry.stopReason` and `retry.attempt`; adding retries does not turn a
+business failure into a defect. Previously persisted exhaustion defects remain
+historical outcomes and are not rewritten.
+
+A durable engine persists the first attempt's start time and the attempt
+sequence, so `expirationMs` is measured from the true first attempt and a
+resumed run does not re-sleep the ladder from attempt one.
+
+## Failures that are never retried
+
+`RetryPolicy.defaultNonRetryable` lists error tags that are non-retryable under
+**every** policy, with no per-callsite opt-out. They are integrity verdicts from
+[`@smthrs/engine-store`](/api/engine-store), and they must reach the driver
+without an action-level retry hiding the first detection.
+
+Your own tags go in `nonRetryable`. `RetryPolicy.errorTag` is how a tag is read
+off an error: an own string `_tag` when present, otherwise the first own `name`
+descriptor found while walking a bounded prototype chain.
+
+## Bound a hung attempt
+
+A retry policy only sees attempts that end. Declare `attemptTimeoutMs` to end
+one that hangs, or `heartbeatTimeoutMs` to end one that stops reporting
+progress through `Action.heartbeat`. Both forms of `Action.make` take them, as
+positive integers:
+
+```ts
+const Fetch = Action.make("sandbox/Fetch", {
+  payload: { url: Schema.String },
+  success: Schema.String,
+  retryPolicy: policy,
+  attemptTimeoutMs: 30_000,
+  heartbeatTimeoutMs: 5_000
+})
+
+const layer = Fetch.toLayer(({ url }) =>
+  Effect.gen(function*() {
+    const stream = yield* openStream(url)
+    // Each heartbeat restarts the 5 second gap.
+    return yield* readChunks(stream, { onChunk: () => Action.heartbeat })
+  })
+)
+```
+
+An expired bound interrupts the body and dies with `Action.AttemptTimedOut`,
+naming the action, the attempt, the `bound` that expired (`"attempt"` or
+`"heartbeat"`), and its `timeoutMs`. The engine counts that defect as a failed
+attempt: `retryPolicy` decides on it like a typed failure, a durable engine
+records it as a failed attempt row, and listing `"@smthrs/flow/AttemptTimedOut"`
+in `nonRetryable` stops the retries. Without a retry policy, or once the policy
+gives up, the defect is the action's outcome. Every other defect stays
+unretried.
+
+## Infrastructure interrupts
+
+An action implementation or transport adapter may fail with
+`Action.InfraInterrupt` when it can identify a retryable infrastructure event.
+It is not an ordinary domain failure and `retryPolicy` does not see it. Only an
+inline action's `interruptRetryPolicy`, an Effect `Schedule`, retries the
+explicit marker. The shipped engines do not convert owner loss, host shutdown,
+or ordinary fiber interruption into it:
+
+```ts
+import * as Schedule from "effect/Schedule"
+
+const resilient = Action.make({
+  name: "payments/Settle",
+  success: Schema.String,
+  error: Schema.String,
+  interruptRetryPolicy: Schedule.recurs(3),
+  execute: callProvider().pipe(
+    Effect.catchTag(
+      "TransportUnavailable",
+      (error) => Effect.fail(new Action.InfraInterrupt({ reason: error.message }))
+    )
+  )
+})
+```
+
+Spending that schedule without reaching an ordinary success or failure dies with
+`Action.InfraInterruptRetriesExhausted`, carrying the action name, the attempt
+count, and the final interrupt.
+
+## Deciding without an engine
+
+The policy functions are pure and exported, so a host can ask the same questions
+the engine asks:
+
+```ts
+RetryPolicy.nextDelay(policy, 2) // Option<number>; None means give up
+RetryPolicy.isNonRetryable(policy, error)
+RetryPolicy.decide(policy, { attempt: 2, error })
+```
+
+`nextDelay` is total even for a policy decoded from a persisted row: a
+non-finite attempt, elapsed time, bound, or computed delay answers `None` rather
+than handing a caller a negative or `NaN` duration. `RetryPolicy.nextDelayEffect`
+and `RetryPolicy.decideEffect` are the variants that sample the `Random` service
+for jitter.
+
+## Related pages
+
+- [Attach an implementation to an action](./implement-an-action.md): the tiers a
+  retry has to respect.
+- [Suspension and replay](../concepts/suspension-and-replay.md): why
+  `suspendedRetryPolicy` on a flow is a different thing entirely.

package/docs/guides/reuse-a-recorded-result.md

@@ -0,0 +1,122 @@
+---
+title: "Reuse a recorded result"
+description: "Declare a cache policy on a sealed action, supply the composition's complete cache environment, and declare the file boundary that makes a result shareable across runs."
+sidebar:
+  order: 12
+---
+
+Every sealed action's result is recorded and replayed inside its own execution.
+Reusing one across executions is a stronger claim, and it takes three
+declarations. Missing any one of them scopes the key to the run that produced it,
+which is safe and does nothing.
+
+1. **An identity a second run can derive.** The action's `idempotencyKey`.
+2. **A hermetic boundary.** The action's declared `FileBoundary`, with
+   `boundaryMode: "hard"`.
+3. **A complete environment.** The composition's `Action.CacheEnvironment`.
+
+## Declare how long the result stays good
+
+`Action.withCache` returns a copy of the action annotated with a `CachePolicy`.
+The action is not mutated: `annotate` returns a separate declaration, and that
+copy is what a plan captures.
+
+```ts
+import { Action } from "@smthrs/flow"
+import * as Effect from "effect/Effect"
+import * as Schema from "effect/Schema"
+
+const compile = Action.withCache(
+  Action.make({
+    name: "build/Compile",
+    success: Schema.String,
+    tier: "sealed",
+    idempotencyKey: "build/compile/v1",
+    fileBoundary: { readSet: [], writeSet: [], boundaryMode: "hard" },
+    execute: Effect.sync(() => "dist/server.js")
+  }),
+  { ttlMs: 15 * 60_000, scope: "shared" }
+)
+```
+
+Both policy fields are optional and both defaults are the behavior you get
+without a policy: no age bound, and the reach the composition's environment
+already granted.
+
+| Field   | Meaning                                                                                                                                                                                                                                    |
+| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `ttlMs` | A positive whole number of milliseconds bounding the age of a row the engine may serve. Past it the dispatch executes again.                                                                                                               |
+| `scope` | `shared` names the inputs and the environment, so any run on any host that would have produced the same bytes may reuse the row. `flow` and `run` narrow that, folding the flow or the execution into the key so a sibling never reads it. |
+
+The engine journals its age judgement as a `flows.engine.cache-provenance`
+record before acting on it, so a reader can tell an expiry from a plain miss, and
+a resumed run reads the verdict it already took instead of re-judging it against
+a fresh clock.
+
+`Action.cachePolicyOf(annotations)` reads a policy back off an annotation bag,
+and `Action.CachePolicyAnnotation` is the key it is carried under.
+
+## Declare the file boundary
+
+An action's typed `fileBoundary` declares its `Action.FileBoundary`, and the engine folds
+that boundary into the key rather than trusting a caller-supplied field:
+
+| Field          | Meaning                                                                                                                                   |
+| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
+| `readSet`      | Exact files already measured (`Action.FileInput`, a workspace-relative path plus its content digest), or globs to expand while preparing. |
+| `writeSet`     | The files or patterns the action may write.                                                                                               |
+| `removes`      | The paths the action may delete. Optional, empty by default, and disjoint from `writeSet`: a path cannot be both promised and disclaimed. |
+| `boundaryMode` | `hard` rejects undeclared access immediately. `expected` validates it later.                                                              |
+
+`removes` exists because a declared write that is absent when the action
+finishes is a defect: recording the absence as valid evidence would cache the
+claim that the file should not exist, and a later replay would act on it by
+deleting the path. A declared removal is what makes an absent path legitimate.
+
+`Action.Filegroup`, `Action.Glob`, and `Action.TreeArtifact` are the reusable
+vocabulary those sets are written in: named groups of declarations,
+Bazel-style include and exclude patterns, and a directory output captured and
+replayed as one tree artifact.
+
+Both inline and declared actions accept `fileBoundary`. Declared actions may
+compute it from decoded payload fields, and can derive `idempotencyKey` from
+the same payload. They also accept `retryPolicy` and `interruptRetryPolicy`;
+see the [`Action.make` reference](../reference/flow.md#actionmake) for every
+option. A new typed boundary is validated before dispatch. Historical
+inline `metadata` boundary declarations remain readable for existing callers.
+
+## Declare the composition's environment
+
+```ts
+const environment = Action.layerCacheEnvironment({
+  layers: ["node@26.4.0", "toolchain@3.1.0"],
+  capabilities: { filesystem: ["read", "write"] }
+})
+```
+
+`Action.CacheEnvironment` is `{ layers, capabilities }`: the ordered semantic
+runtime layers, including versions and configuration, and the complete effective
+capability groups. A composition either provides a **complete** value or leaves
+it absent. When it is absent, the engine scopes action keys to the current
+execution rather than presenting incomplete environment data as reusable
+identity, and `Action.CurrentCacheEnvironment` reads `undefined`.
+
+That is the safe default and it is doing real work: a half-declared environment
+would let a result computed under one toolchain be served to a run under
+another.
+
+## What cannot enter a key
+
+An object-form `idempotencyKey` is caller-owned canonical JSON. Material that
+canonical serialization rejects, such as a `Date`, an `undefined`, a class
+instance, or a `Redacted`, is refused with `Action.UncanonicalIdempotencyKey`
+naming the offending path. The refusal is a typed, non-retryable recorded
+completion rather than an untyped defect, because the same declaration derives
+the same rejection on every attempt and the body never runs.
+
+## Related pages
+
+- [Execution identity](../concepts/execution-identity.md): the full account of
+  what enters a key, and the one limit of schema-shaped key material.
+- [Attach an implementation to an action](./implement-an-action.md): tiers, and
+  when an inline action is the right form.

package/docs/guides/run-a-child-flow.md

@@ -0,0 +1,108 @@
+---
+title: "Run a flow as a child execution"
+description: "Choose between splicing a callee inline, opening a real child execution, and handing off to the next round, and know which build refusals force a boundary."
+sidebar:
+  order: 8
+---
+
+A flow can reach another flow in three ways, and they are three different things.
+
+| Call                    | What the caller's plan holds                                                             | Requirements             | Identity                                                                              |
+| ----------------------- | ---------------------------------------------------------------------------------------- | ------------------------ | ------------------------------------------------------------------------------------- |
+| `callee.call(payload)`  | The callee's whole body, spliced in. Every inner step is visible and individually keyed. | Propagate to the caller. | The caller's execution.                                                               |
+| `callee.child(payload)` | One leaf.                                                                                | Dropped.                 | A separate execution, with its own row, journal lineage, retry policy, and placement. |
+| `callee.to(payload)`    | The settlement of this round.                                                            | Dropped.                 | The next round of this trampoline lineage.                                            |
+
+## Splice a callee inline
+
+```ts
+import { Action, Flow } from "@smthrs/flow"
+import { Node } from "@smthrs/plan"
+import * as Schema from "effect/Schema"
+
+const Build = Action.make("release/Build", {
+  payload: { target: Schema.String },
+  success: Schema.String
+})
+
+export const Compile = Flow.make("release/Compile", {
+  payload: { target: Schema.String },
+  success: Schema.String,
+  body: (payload) => Build.call(payload)
+})
+
+export const Release = Flow.make("release/Release", {
+  payload: { target: Schema.String },
+  success: Schema.String,
+  body: ({ target }) => Compile.call({ target })
+})
+```
+
+Inline is the default choice. One plan, one run, and every step of the callee is
+a node the caller's operator can see, key, and replay. The callee's obligations
+become the caller's, which is exactly right: they are running in the caller's
+execution.
+
+## Open a real child
+
+```ts
+export const Fanout = Flow.make("release/Fanout", {
+  payload: { targets: Schema.Array(Schema.String) },
+  success: Schema.Struct({ web: Schema.String, api: Schema.String }),
+  body: () =>
+    Node.all({
+      web: Compile.child({ target: "web" }),
+      api: Compile.child({ target: "api" })
+    })
+})
+```
+
+Choose `child` when the callee should be its own unit of work: its own run row to
+watch, its own journal lineage, its own retry and placement decisions. The
+parent's plan holds one leaf naming it, the parent suspends while the child is
+unsettled, and it resumes when the child settles.
+
+The identity is derived rather than minted. `Interpreter.childExecutionId`
+combines the parent execution id, the node's address, the callee tag, and a digest
+of the payload, so a re-driven parent lands on the child it already started
+instead of opening a second one. That is what makes a child at most once without
+the caller naming an id.
+
+Requirements are dropped at the boundary because the child runs under its own
+driver, which provides its own context. The caller's composition does not have to
+carry the child's implementations, and the child's driver does.
+
+## Hand off to the next round
+
+`callee.to(payload)` ends the current round and names the next one. It is the
+loop construct, not a call: nothing comes back, because this execution is over.
+See [Trampoline rounds](../concepts/trampoline-rounds.md).
+
+## When a boundary is required
+
+Two build refusals exist to send you to `child` or `to`, and both throw from
+`Graph.build` rather than landing in diagnostics:
+
+- `recursion_requires_boundary`: a flow called itself inline. Inline expansion
+  would never terminate. Use `to` for a loop or `child` for a nested run.
+- `placement_requires_boundary`: an inline callee declares a `Flow.Placement`
+  the enclosing flow cannot satisfy. A placement is a property of an execution,
+  so a callee that needs a different one needs its own execution.
+
+At run time, `FlowRuntime.FlowCycleDetected` is the separate refusal for a child
+whose execution would close a cycle in the persisted parent chain. Its `path`
+holds the ordered execution ids from the cycle's target back to itself.
+
+## Cancellation and cleanup travel
+
+A parent's interruption reaches its children, and a child's suspension reaches
+its parent, because the engine records the lineage edge. Cleanup follows the
+same edges: see
+[Cancel a run and undo its effects](./cancel-and-roll-back.md).
+
+## Related pages
+
+- [Flows and actions](../concepts/flows-and-actions.md): the propagation rules,
+  stated once.
+- [Execution identity](../concepts/execution-identity.md): how a child's id is
+  derived.

package/docs/guides/run-an-external-job.md

@@ -0,0 +1,86 @@
+---
+title: "Run an external job"
+description: "Start external work once, park between probes, and reattach after a host restart."
+sidebar:
+  order: 8
+---
+
+`ExternalJob.make` composes recorded Start, Status, Collect and Cancel actions
+with ordinary flow handoffs and durable timers. The adapter owns the external
+worker; a parked Smithers execution owns no worker process scope.
+
+```ts
+import { Action, ExternalJob, Flow, Interpreter } from "@smthrs/flow"
+import { Node } from "@smthrs/plan"
+import { RemoteChildProcessSpawner, Sandbox } from "@smthrs/sandbox"
+import { Effect, Layer, Schema } from "effect"
+
+const Build = ExternalJob.make("build/external", {
+  payload: { source: Schema.String },
+  handle: Sandbox.JobHandle,
+  success: Sandbox.JobResult,
+  error: Schema.Union([RemoteChildProcessSpawner.ProviderError, Sandbox.CaptureError]),
+  probe: { every: "15 seconds", max: "2 minutes" },
+  timeout: "2 hours",
+  restarts: 1
+})
+
+// The provider adapter supplies create-or-get, observation, collection and
+// cancellation. Convert cleanup transport errors to visible defects.
+const adapter = Sandbox.job(provider, { command: "make" })
+const buildLayer = Build.toLayer({
+  ...adapter,
+  cancel: (handle, key) => adapter.cancel(handle, key).pipe(Effect.orDie)
+})
+
+const Pipeline = Flow.make("build/pipeline", {
+  payload: { source: Schema.String },
+  success: Sandbox.JobResult,
+  error: Build.errorSchema,
+  body: Node.capture({ version: "build/pipeline/v1", build: Build._tag }, (payload) => Build.call(payload))
+})
+
+const implementations = Layer.mergeAll(buildLayer, Interpreter.layer(Pipeline))
+  .pipe(Layer.provideMerge(Action.layerImplementations))
+```
+
+Use a retained provider. Supply the runtime and the adapter's services, including durable capture receipt
+storage, to that layer. Declare the complete adapter failure union in `error`;
+errors are encoded in durable action outcomes.
+
+## Identity and replay
+
+`Build.call` uses the existing child boundary, so each call has its own durable
+execution identity. It does not inline the job into its caller. `.child` and
+`.execute` also retain the ordinary flow APIs. Two calls with identical payloads
+start separate jobs. `.execute` with the same execution id rejoins its lineage.
+
+The initial action journals the enclosing job execution id and start time.
+Adapters receive `<execution id>#g1`, then `#g2` for one allowed replacement.
+The public payload cannot select an identity or supply a continuation handle.
+`start` must atomically create-or-get by that key. Smithers replay alone cannot
+make a provider's non-idempotent spawn safe.
+
+Status actions have separate probe keys, while the provider receives the same
+job key throughout one generation. Delays double from `probe.every` up to
+`probe.max`, and stop at the original timeout deadline. That deadline is carried
+through every generation and survives a host restart.
+
+## Replacement and cancellation
+
+Status returns `Running`, `Exited` with `exitCode`, or `Lost`. `Exited` calls
+`collect`. A collected exit can fail with `ExternalJob.Again` to request a
+replacement. Both Lost and Again complete Cancel before a durable backoff and
+the next Start. A failed cancellation cannot launch another worker.
+`Again` is a dependency fault if it escapes the replacement boundary. A flow's
+own `DeadlineExceeded` is a policy fault: its elapsed deadline is terminal.
+
+Exhaustion fails with `ExternalJobLost` (`infra`). Timeout completes Cancel and
+fails with `ExternalJobTimedOut` (`dependency`). `restarts: 0`, the default,
+permits only the first generation. Adapter infrastructure errors retain their
+declared type and use the runtime's repeat-safe retry policy.
+
+The actual flow scope retains `Flow.withRollback` cancellation. Timer parking,
+replay and host release preserve the external worker. Explicit cancellation
+cancels it, including a retained parked scope. The rollback is process-local;
+providers still need their retained-machine lease or reaper after a host dies.