@astrale-os/sdk 0.6.0-beta.39 → 0.6.0-beta.40

package/docs/durable-workflows.md

@@ -0,0 +1,418 @@
+# Durable Workflows
+
+A callable whose Schema declares `execution: { durable: true }` runs on the Kernel's workflow
+engine. A call returns at once with the `K.Execution` it recorded; the engine journals the run,
+keeps its timers and promises, retries it and survives restarts of the Domain, the Kernel and the
+engine itself.
+
+```ts
+// schema/schema.ts, in the Invoice Class
+remind: method({
+  auth: 'authenticated',
+  input: z.object({ note: z.string() }),
+  output: z.object({ reminder: z.string() }),
+  execution: { durable: true },
+})
+```
+
+A durable callable needs `authenticated` or `authorized` auth, a value input and output, and a
+remote binding. Implement it with `defineWorkflow`; `defineAction` refuses a durable callable.
+
+```ts
+export const remind = defineWorkflow<typeof schema>()(
+  'Invoice.remind',
+  async ({ input, self, execution, step, sleep, awaitSignal }) => {
+    const sent = await step.query('count-reminders', remindersOf, { invoice: self })
+    const reminder = await step.mutate('record', recordReminder, {
+      invoice: self,
+      rank: sent.length + 1,
+      note: input.note,
+      execution: execution.id,
+    })
+    await sleep(24 * 60 * 60_000)
+    const approval = await awaitSignal<{ readonly by: string }>('approval')
+    return { reminder: `${reminder}:${approval.by}` }
+  },
+)
+```
+
+## Context
+
+| Member | Meaning |
+| --- | --- |
+| `input`, `self`, `domain` | As for any Workflow; `self` is the receiver of an instance Method. |
+| `execution` | Attested facts: `id`, `function {id, version}`, `kernel`, `initiator {principal, caller}`, `receiver`. Data, never authority. |
+| `step.run(step, execute)` | Runs `execute` once and journals its portable output; a replay returns the journaled output. `execute` receives the capabilities of an Action, acting as the Domain alone: `kernel`, `graph`, `query`, `mutate`, `dependencies` and `integrations`. `step` is the Step's id, or `{ id, retry, timeoutMs }` (see [Retry budgets and timeouts](#retry-budgets-and-timeouts)). |
+| `step.query(step, query, input?)` | One journaled Query; a replay returns the result of its first run. |
+| `step.mutate(step, mutation, input)` | One journaled Mutation. |
+| `step.parallel(declare)` | Runs the Steps `declare` declares together, in one round trip to the engine; see [Parallel Steps](#parallel-steps). |
+| `sleep(ms)` | Durable timer, kept by the engine even while the Domain is down. |
+| `awaitSignal(name)` | Waits for the value `Execution.signal({ name, value })` delivers once under `name`, sent before or during the wait. |
+| `awaitSignal(name, { timeoutMs })` | Waits at most `timeoutMs`: `{ kind: 'signal', value }` or `{ kind: 'timeout' }`. |
+| `cancelled(error)` | Whether `error` is the engine's cooperative cancellation; catch it to compensate. |
+| `exhausted(error)` | Whether `error` is a Step that spent its retry budget; catch it to take another path. |
+| `retryAfter(ms, cause?)` | The failure to throw inside a Step whose next attempt must wait `ms` (at most one hour), as an HTTP `Retry-After` asks. |
+| `awaitExecution(ref)` | Waits durably for another Execution by journaled polling of its `result`. A poll has no Step budget: while the Kernel cannot answer, the run is retried until it pauses. |
+| `error(code, details)` | Declared business errors, as for any Workflow; they become the `failed` outcome. |
+
+A durable run acts as its Domain, never as its caller: the Workflow and its Steps have no `caller`,
+and `kernel`, `graph` and `dependencies` offer only `self`. The Kernel is reachable only inside a
+Step, so every Kernel call is journaled and a replay never repeats it. A Step whose outcome was not
+journaled runs again. What a retried Step repeats depends on the call:
+
+| Call inside a Step | On a retry of the Step |
+| --- | --- |
+| Mutation (`step.mutate`, `mutate`, `graph.self.mutate`, `kernel.mutate`) | Carries a key derived from the Execution, the Step and its position. The Kernel commits a receipt with the write and answers with the first result, without writing again. |
+| `kernel.invoke` or `dependencies.<alias>.self.invoke` of a durable callable | Carries such a key too, so it reaches the same child Execution. The call answers with that Execution's `ExecutionRef`. |
+| `kernel.invoke` or `dependencies.<alias>.self.invoke` of any other callable | Carries no key, because the Kernel accepts one only for a durable Function. The call runs again, so its effects must be idempotent: declare the callee durable, or give it an identity in its input. |
+| Integration operation, `keyed` with source `execution` | Its call id is the next key of the same sequence, and the Provider receives it as `idempotencyKey`: every attempt sends the same key. |
+| Integration operation, `keyed` with source `input` | The Provider receives the key the decision takes from the input. |
+| Integration operation, `safe` | Runs again. |
+| Integration operation that may be `unsafe` | Absent from `integrations`: a retry would repeat it. An operation whose decision turns out `unsafe` is refused before its Provider runs. |
+
+A Provider called from a Step receives no `invoke`, since no caller exists, and its `auth.mint` and
+deadline are bounded by the Step's deadline. An operation whose replay is a function counts as
+replayable only when that function never answers `unsafe`.
+
+The Host keeps receipts longer than the engine keeps a run, so a Step is never retried after its
+receipt is gone. A callable is known durable from the Schema closure of the Domain: an untyped Path,
+or a callable outside that closure, is keyed only when the call passes its own `idempotencyKey`.
+
+This requires a Kernel that keeps Mutation receipts. An older Kernel refuses the Mutation key.
+
+Step ids are unique within a run. Steps and waits run one after another: the engine follows one
+awaited operation at a time, and loses track of a run that awaits several together through
+`Promise.all`, `allSettled`, `race` or `any`. A Step, `sleep`, `awaitSignal` or `awaitExecution`
+started while another runs is therefore refused with a `TypeError`, which pauses the run as any
+error between Steps does. Independent reads may share one Step, whose Kernel calls are ordinary
+calls that `Promise.all` may combine:
+
+```ts
+const [invoice, reminders] = await step.run('read-invoice', ({ query }) =>
+  Promise.all([query(invoiceOf, { invoice: self }), query(remindersOf, { invoice: self })]),
+)
+```
+
+Code between Steps must be deterministic.
+
+### Parallel Steps
+
+Each Step awaited alone costs one round trip to the engine. `step.parallel` is the one way to run
+several Steps together: the engine follows them as one operation, through its own combinator, and
+they cost one round trip together.
+
+```ts
+const [invoice, sent, receipt] = await step.parallel((s) => [
+  s.query('read-invoice', invoiceOf, { invoice: self }),
+  s.query({ id: 'count-reminders', retry: { maxAttempts: 5 } }, remindersOf, { invoice: self }),
+  s.run('charge', ({ integrations }) => integrations.ledger.charge({ amount: 12 })),
+])
+```
+
+`s` declares Steps with the arguments of `step.run`, `step.query` and `step.mutate`, and starts
+nothing; `declare` returns exactly the Steps it declares, as an array. Each Step keeps its id,
+retry budget, timeout and keys, as if it ran alone. Once every Step has settled, `step.parallel`
+resolves with their outputs in declaration order, or throws the failure of the first failed Step in
+that order, as that Step alone would throw it: a declared error, a `STEP_EXHAUSTED` that `exhausted`
+recognizes, or a cancellation. The other Steps have run by then, and their effects stand. An empty
+list resolves with `[]`. Parallel Steps do not nest, and a Step that depends on another's output
+runs after it.
+
+Run reads and independent effects together. Keep in sequence the effects a failure would make you
+compensate, so the Workflow knows which ones committed.
+
+## From an inline Workflow
+
+The same `defineWorkflow` implements an inline and a durable Workflow; the Schema decides which.
+Declaring `execution: { durable: true }` on a callable changes the context its Workflow receives:
+
+| Inline Workflow | Durable Workflow |
+| --- | --- |
+| `query` and `mutate` in the context, as the Domain; `kernel`, `graph` and `dependencies`, as the Domain or the caller | Only inside a Step, as the Domain alone: `step.query`, `step.mutate`, or `step.run(id, ({ query, mutate, kernel, graph, dependencies }) => …)` |
+| `integrations` in the context, with every operation | Inside `step.run` only, without the operations that may be `unsafe` |
+| `caller` | Absent; `execution.initiator` records who started the run, as data |
+| `execution`: `signal`, `deadline`, `idempotencyKey`, `background`, `sleep`, `request` | `execution`: `id`, `function`, `kernel`, `initiator`, `receiver`; `sleep` is a member of the context |
+| `step.run(id, () => …)`, not journaled | `step.run`, `step.query`, `step.mutate` and `step.parallel`, journaled, each Step under its retry budget |
+| A failing Step throws into the Workflow at once | A failing Step is attempted again within its budget, then throws `STEP_EXHAUSTED` into the Workflow |
+| Callers receive the output | Callers receive `ExecutionRef<Output>`; the Workflow still returns the output |
+
+The type checker reports every use of a member the durable context lacks, and every caller that
+reads the output. Three changes it cannot see:
+
+- A business refusal a Step throws as anything but `error(...)` is attempted again until the Step
+  spends its budget, and the Execution ends `STEP_EXHAUSTED` instead of the refusal. Turn a refusal
+  into its declared error inside the Step that detects it, not around the Step.
+- Code between Steps runs again on every replay. A value it reads from the clock or randomness
+  differs on the next attempt; read it in a Step. `FNC-REPLAY-VALUES` warns on such a read.
+- A caller that ignores the output, such as a View that refreshes after the call, compiles unchanged
+  but refreshes before the Execution ends. Follow the Execution, and refresh when it ends.
+
+`createTestEngine` (see [Testing](#testing)) shows the first two: a refusal path ends
+`STEP_EXHAUSTED`, with what the last attempt threw in `failure`, and code whose path differs on
+replay fails with `JournalMismatch`.
+
+## Reads and effects
+
+The engine retries a Step as a whole, so running it again must be safe. A Step may make several
+reads, and the keys of the table above make its effects safe to repeat. What a Step must not do is
+read before its effect: an effect Step builds its request only from the Workflow input and the
+results of earlier Steps, so it reads what it needs with `step.query` first:
+
+```ts
+const sent = await step.query('count-reminders', remindersOf, { invoice: self })
+await step.mutate('record', recordReminder, { invoice: self, rank: sent.length + 1 })
+```
+
+A Step can commit its effect and lose its journal entry; the engine then runs it again with the same
+keys. The Kernel answers a Mutation from its receipt only when the request is the same. Rebuilt from a
+read made earlier in the same Step, the request can differ, since that read now sees the Step's own
+write: the Kernel refuses it with `MUTATION_IDEMPOTENCY_CONFLICT` on every attempt until the Step
+spends its budget and the Execution fails, or a guard on the read takes the other branch and the
+Execution fails although its effect committed. A read from an earlier Step replays unchanged, so the retried request is
+identical. A read after the effect sees the same write on every attempt.
+
+Two effects in one Step replay safely, each under its own key, but when the second fails after the
+first committed, the Step reports only its failure and the Workflow cannot tell what to compensate.
+Give each effect its own Step; two Mutations that can commit together are one Mutation.
+
+The Domain linter checks these rules in a durable Workflow. `FNC-STEP-READS` reports a Step
+that reads before its effect, unless the two take exclusive branches. `FNC-EFFECT-STEP` warns on a
+Step where one run can make two effects, or repeat one in a loop. `FNC-STEP-RETRY` warns on a call a retried
+Step repeats with no key: a callable the types prove not durable, an untyped Path, or `fetch`. A call
+whose kind the source does not prove is not reported. `FNC-REPLAY-VALUES` warns on a read of the clock
+or randomness (`Date.now()`, `new Date()`, `Math.random()`, `crypto.randomUUID()`) in the code a replay
+runs again: the Workflow's body outside its Steps, the arguments of a Step call included.
+
+A journaled read is the value at the moment its Step ran; a Step that runs later, after retries or a
+pause, does not read it again. A fact that must still hold when the effect commits is a precondition
+of the Mutation (`mutation.expect`). Map `MUTATION_PRECONDITION_FAILED` in the Mutation's `reject` to
+a result the Workflow handles, such as a declared error; unmapped, the engine retries it until the
+Step spends its budget. A retried Step whose Mutation already committed still receives that commit's
+receipt, even though its precondition no longer holds.
+
+## Testing
+
+`createTestEngine` from `@astrale-os/sdk/testing` runs a durable Workflow in memory with the
+receiver a Host serves; only the engine is simulated. Each Step journals its result, every attempt
+replays the journal, timers follow a virtual clock, signals are buffered, a failing Step is retried
+within its budget and then fails into the Workflow, a run failing outside any Step is retried until it
+pauses, and cancellation is cooperative.
+
+```ts
+import { createTestEngine } from '@astrale-os/sdk/testing'
+
+const engine = createTestEngine()
+const execution = await engine.start(remind, {
+  schema: Billing,                      // the resolved Schema
+  input: { note: 'soon' },
+  self: invoice,                        // receiver of an instance Method
+  capabilities: {
+    query: (query, input) => 2,
+    mutate: (mutation, input, { idempotencyKey }) => 'reminder-3',
+    kernel: { invoke: async (reference, input, options) => ({ id: 'execution-child' }) },
+  },
+  providers: { ledger: testLedger },    // Providers of the Workflow's Integrations
+})
+await engine.advance(24 * 60 * 60_000) // wake the sleeping run
+await execution.signal('approval', { by: 'alice' })
+expect(execution.outcome).toEqual({ kind: 'completed', value: { reminder: 'reminder-3' } })
+await execution.replay()                // the journal alone reaches the same outcome
+```
+
+| Member | Meaning |
+| --- | --- |
+| `capabilities` | What a Step reaches instead of a Kernel. `query` and `mutate` answer the Step's Queries and Mutations; `mutate` receives the key a Host sends. `kernel` is the Step's Session: `kernel.invoke` and every `dependencies` call reach its `invoke`, keyed for a durable callee. |
+| `providers` | Providers bound through the Steps' key sequence, as on a Host. |
+| `retry` option | The deployment's `durable.retry`: the budget of every Step that declares none, and with `maxAttempts` the attempts of a run failing outside any Step. |
+| `state`, `outcome`, `failure` | `running`, `waiting`, `paused`, `completed`, `failed` or `cancelled`; the outcome as `Execution.result` answers it; what the last failed attempt threw, which a `STEP_EXHAUSTED` outcome does not carry. |
+| `signal`, `cancel`, `resume` | As the `K.Execution` Methods; each settles the run again. |
+| `advance(ms)` | Moves the virtual clock and settles every waiting run. |
+| `replay()` | Runs the Workflow from its journal alone and fails if it runs a Step again, diverges or ends differently. |
+| `interruptAfter(id)`, or the `interruptAfter` option | Loses Step `id`'s first result before the journal records it, as a crash after its effect: the engine retries the Step, which presents the same keys. |
+| `journal`, `attempts` | The recorded entries, each Step with the times it ran, and the attempts of the run. |
+
+A Workflow whose code between Steps is not deterministic fails its next attempt with
+`JournalMismatch` and pauses. Retries do not wait out a backoff, but the backoff a retry skips, or the
+delay its failure asks for, counts toward the Step's `maxDurationMs`. Code should await only Steps and
+the context's own operations; the engine counts a run as waiting once it waits on a timer or a signal
+and nothing else runs.
+
+## Signals
+
+A signal is delivered once per name: `Execution.signal` resolves it, a second signal of the same name
+is refused, and a signal sent before the wait is kept for it. A bounded wait races the signal against
+a durable timer:
+
+```ts
+const consent = await awaitSignal<{ readonly code: string }>('consent', { timeoutMs: 15 * 60_000 })
+if (consent.kind === 'timeout') throw error('CONSENT_EXPIRED', {})
+```
+
+The engine journals which came first, so a replay answers the same even when the signal arrived
+after the timeout; such a signal is kept for a later wait under the same name. Race durable waits
+only through `awaitSignal`: `Promise.race` over `sleep` and `awaitSignal` is not journaled and does
+not settle on the engine.
+
+## Retry budgets and timeouts
+
+A declared business error ends the Execution as `failed { family, code, message, details }`, with no
+retry. Any other error a Step throws fails that attempt, and the engine attempts the Step again within
+its retry budget. A Step that spends its budget throws into the Workflow; uncaught, the Execution
+ends `failed` with code `STEP_EXHAUSTED`:
+
+```ts
+{ kind: 'failed', error: { code: 'STEP_EXHAUSTED', message: 'Workflow Step notify exhausted its retries.', details: { step: 'notify' } } }
+```
+
+The outcome never carries what the last attempt threw. When the Kernel refused that attempt, the
+outcome names the refusal by its codes alone, never its message:
+
+```ts
+details: { step: 'archive', refusal: { code: 2002, reason: 'AUTH_INVALID' } }
+```
+
+`code` is the Kernel error code; `reason`, when the Kernel gave one, its reason code. A refusal
+behind another error, as its `cause`, counts as well; any other failure leaves `refusal` out. A failed Execution is never resumed: a new
+attempt is a new Execution, under a new key, so its Steps run their effects again under new keys.
+
+Each Step runs under the deployment's budget and timeout, or under the ones it declares:
+
+```ts
+await step.mutate('record', recordReminder, input) // the deployment's budget and timeout
+
+try {
+  await step.run({ id: 'notify', retry: { maxAttempts: 5 }, timeoutMs: 120_000 }, ({ integrations }) =>
+    integrations.mailer.send(reminder),
+  )
+} catch (cause) {
+  if (!exhausted(cause)) throw cause
+  await step.mutate('follow-up', recordManualFollowUp, { invoice: self })
+}
+```
+
+| `retry` field | Meaning | Bounds |
+| --- | --- | --- |
+| `maxAttempts` | Attempts, the first one included. | 1 to 1,000 |
+| `maxDurationMs` | Time from the first attempt after which a failed attempt is not retried; an attempt already scheduled still runs. | 1 ms to 365 days |
+| `initialIntervalMs` | Delay before the first retry, doubled on each failure. | 1 ms to one hour |
+| `maxIntervalMs` | Longest delay between two attempts. | `initialIntervalMs` to one day |
+
+The limits, `maxAttempts` and `maxDurationMs`, go together: a Step that declares either replaces both
+limits of the deployment, and the one it omits does not bound it. The budget is spent at the first
+limit reached. An interval the Step leaves out is the deployment's. `timeoutMs` bounds one attempt,
+its Kernel calls included, from one second to ten minutes: past it, the attempt fails, what it holds
+is aborted, and the failure counts toward the budget. Each Workflow service declares to the engine
+an abort timeout of fifteen minutes, so a Step's own timeout always ends its attempt before the
+engine would cut it.
+
+A Kernel refusal that prescribes a delay (capacity, draining, maintenance) waits that delay before
+the next attempt, instead of the backoff. Throw `retryAfter(ms, cause)` inside a Step for the same
+behavior, for instance on an HTTP `429` with `Retry-After`. Both attempts count toward the budget.
+
+A run that fails outside any Step keeps the engine's pause. An input or receiver the code cannot
+read, an output the Schema refuses, a journal whose Steps the code no longer takes, or an error
+thrown between two Steps fails every attempt; after the deployment's `maxAttempts` the Execution is
+`paused` until `Execution.resume()` or `Execution.cancel()`. `resume` attempts the run again with
+the code served at its URL: it helps only once that code can replay the journal. A run that fails
+again before it awaits anything never sees a cooperative cancel; `cancel({ force: true })`, which
+the owner Domain holds, ends it without compensation.
+
+## Redeployments
+
+A new Release gives new calls a new engine service; running Executions keep theirs, with its URL,
+and every attempt of a run is served by the code at that URL. A run started on one Release ends on
+it, under the following rules.
+
+**Serve each Release at its own URL.** `adapter-cloudflare` and `adapter-astrale` deploy each
+Release at an immutable URL, so a run keeps the code that started it, whatever is deployed after.
+Code redeployed in place at the same URL replays the journals of the runs in flight: the engine
+refuses a journal whose Steps no longer match, in order and by id, and pauses the run; code with the
+same Steps continues the run. Renaming, reordering or removing a Step that runs have journaled is
+therefore an incompatible change in place, and no option of the Workflow declares it.
+
+**Keep the Schema additive while runs are in flight.** A Step calls the Kernel as installed when it calls,
+not when its run started: a Class, Function or Edge the run uses and a later install
+removed or narrowed refuses its calls until the Step spends its budget, with `STEP_EXHAUSTED`.
+Remove them once the runs that use them have ended.
+
+**Do not retire a Release, or remove a durable callable, while runs are in flight.** A retired
+Release no longer answers its URL: its runs no longer progress, a signal to them times out, and a
+cooperative `cancel` leaves them `paused`; only `cancel({ force: true })` ends them. A durable
+callable a new install no longer declares has its runs cancelled by the Kernel, without
+compensation, within minutes.
+
+**Keep runs that call the Kernel shorter than the drain bound.** After an issuer change, the
+replaced issuer remains accepted for the runs in flight at the change for at most the Host drain
+bound (7 days by default), described below; a Step of such a run calling the Kernel later is
+refused until it spends its budget, with `STEP_EXHAUSTED`. A run that only waits and journals is
+not bounded.
+
+A Step's credential names its Execution (the `execution` claim, `EXECUTION_CLAIM` of
+`@astrale-os/kernel-protocol/issuer/claims`). After a consented issuer change, the Kernel keeps
+accepting the replaced issuer for any call during the Host window (300 s by default), then only for
+the Steps of runs that were in flight at the change and that the engine still holds, for at most
+the Host drain bound (7 days by default). While the engine is unreachable those Steps are refused
+as unavailable and retried. A run started after the change is never accepted on the replaced
+issuer. The replaced deployment signs this claim itself, so it can keep calling for its in-flight
+runs until they end or the drain bound passes; consent with `previous: 'revoke'` to cut it at once.
+
+## Serving
+
+`serve({ durable })` serves the engine at
+`<public URL>/invoke/restate/<encoded Kernel issuer>/<declaration>/<service>`. Retained engine
+services registered before issuer-prefixed paths keep their `<declaration>/<service>` callbacks.
+Canonical requests discover their engine's signing keys at
+`<Kernel issuer>/.well-known/astrale/workflow-engine.json` over HTTPS. The discovered document must
+name that issuer, and the signed run must name the same Kernel before any Workflow code runs.
+Discovery follows no redirects, has a ten-second deadline and a 16 KiB document limit, and retains
+at most 64 issuer documents for one minute. An authentication refusal expires its cached authority
+for the next request, at most once per minute; the receiver never replays a consumed request.
+Invalid or unavailable discovery fails the canonical request without falling back to static keys.
+Provide the Restate SDK Fetch build of the platform; every engine request requires its signature.
+
+```ts
+serve({
+  // runtime, load, identity, server…
+  durable: async () => ({
+    engine: await import('@restatedev/restate-sdk/fetch'),
+  }),
+})
+```
+
+The provider runs only on the first engine request. Concurrent requests share initialization;
+a successful receiver is reused, and a failed or unconfigured provider can be retried by a later
+request. Health, release and ordinary invocation requests do not load the engine. An already
+loaded engine can still be supplied synchronously.
+
+| Option | Default |
+| --- | --- |
+| `retentionMs` | 7 days; at least the Host minimum (one day by default). |
+| `retry` | The budget of every Step that declares none: 20 attempts from 1 s to 60 s. `maxAttempts` also bounds a run failing outside any Step, which then pauses. The Host caps them. |
+| `stepTimeoutMs` | 60 s per Step attempt that declares no `timeoutMs`, including its Kernel calls; ten minutes at most. |
+
+The provider receives the durable `Request`. Its route identifies the HTTPS Kernel whose
+workflow-engine discovery document supplies signing authority. `DurableServing.fetch` supplies
+that discovery capability and is captured when the receiver initializes. Static signing keys and
+routes without a Kernel issuer are refused; they do not initialize the engine.
+
+Without `durable`, the Domain refuses the engine and only durable calls fail. An ordinary
+invocation of a durable callable is refused: it never runs inline.
+
+### Cloudflare Workers
+
+The Cloudflare adapter serves the engine itself when the Schema declares a durable callable: it
+retains the Restate Workers build and its WebAssembly core in a separate deployed module. The
+module is imported only on the first engine request, so frontend, health, release and ordinary
+invocations do not initialize Restate. Callbacks discover signing authority from their Kernel issuer. Cloudflare owns
+compilation of declared WebAssembly modules; this separation defers their instantiation and the
+JavaScript engine's evaluation. The options above go in the adapter's `durable` parameter.
+
+```ts
+cloudflare({
+  namespace: { name: 'deployments', routingDomain: 'deployments.example.com' },
+  durable: { retry: { maxAttempts: 10 } },
+})
+```
+
+Each Release is an immutable deployment with its own URL: a new Release gives new calls a new engine
+service, and running Executions keep being served by the deployment that started them.

package/docs/handler-dependencies.md

@@ -0,0 +1,213 @@
+# Handler authority and dependency calls
+
+Actions and Workflows receive `kernel`, `graph`, `query`, `mutate`, and `dependencies`.
+Local capabilities default to the executing Domain's self authority. Direct dependency calls
+select `caller` or `self` explicitly.
+
+| Surface | Default authority | Explicit selection |
+| --- | --- | --- |
+| `kernel` | Executing Domain | `.caller`, `.self` |
+| `graph` | Executing Domain | `.caller`, `.self` |
+| `query`, `mutate` | Executing Domain | Through `graph` |
+| `dependencies.alias` | Explicit selection required | `.caller.invoke`, `.self.invoke` |
+
+`self` executes as the Domain. `caller` executes with the Domain as principal on behalf of the one
+incoming caller Identity. These defaults belong to the API surface. A foreign target does not
+silently change the authority selected by `kernel.invoke`.
+
+## The caller
+
+An authenticated handler receives `caller.identity` and `caller.principal`. `identity` is the one
+Identity whose authority the invocation exercises: use it for ownership, attribution, and
+per-user lookups. `principal` is the executor whose credential authenticated the call, never
+the user once a Domain acts for them: after token exchange it is the Domain that exchanged the
+token (with the Session `exchange` option, this Domain), and after a `caller` dependency call it is
+the calling Domain. A caller is always exactly one Identity.
+
+## Public callables
+
+A callable declared `auth: 'anonymous'` admits a call that carries no credential. Such a call has
+no caller, so its handler acts as the installed Domain alone:
+
+```ts
+export const submitLead = defineAction<typeof schema>()('submitLead', ({ input, mutate }) =>
+  mutate(createLead, input),
+)
+```
+
+- `caller` is `{ kind: 'anonymous' }`: there is no Identity to attribute the call to.
+- `kernel`, `graph`, `query`, `mutate`, and `dependencies.alias.self` are the same Domain self
+  authority an authenticated handler receives. Installation requirements and ownership bound it;
+  nothing is granted to whoever sent the call.
+- `kernel.caller`, `graph.caller`, `dependencies.alias.caller`, and a Provider's `execution.invoke`
+  are absent, and typed as possibly absent in an `anonymous` callable. Reach them through the
+  selection itself: `graph.caller?.query(...)`, or test `graph.caller !== undefined` first. A check
+  on `caller.kind` does not narrow them.
+- The same callable called with a credential is authenticated as usual and receives its caller. An
+  invalid credential is refused; it never falls back to an anonymous call.
+
+The Kernel admits a public call without asking who sent it, and the handler then runs with the
+Domain's authority. The callable's input contract and its handler are therefore the only gate:
+admit only what the public may submit, and return only what the public may read.
+
+## Declaring and invoking a dependency
+
+Declare the exact remote Schema and each consumed callable requirement in Domain composition:
+
+```ts
+import { DirectorySchema } from '@example/directory-domain'
+import { defineDomain, requirements } from '@astrale-os/sdk/domain'
+import { defineSchema, schema as language } from '@astrale-os/sdk/schema'
+
+const schema = defineSchema('consumer.example', {
+  name: 'Consumer',
+  dependencies: { directory: DirectorySchema },
+  // Compose local callable declarations here.
+})
+const directory = language.resolve(DirectorySchema)
+export const domain = defineDomain({
+  schema,
+  runtime,
+  requirements: requirements({ functions: [directory.functions.lookup] }),
+})
+```
+
+The handler uses the declared alias without importing the remote Domain:
+
+```ts
+export const lookup = defineAction<typeof schema>()('lookup', ({ dependencies, input }) =>
+  dependencies.directory.caller.invoke(d => d.functions.lookup, input),
+)
+```
+
+Other callable shapes use the same invoker and require their own exact Domain requirements:
+
+```ts
+await dependencies.directory.caller.invoke(d => d.classes.Employee.static.methods.create, input)
+await dependencies.directory.self.invoke(d => d.classes.Employee.methods.rename, instance, input)
+```
+
+`instance` is a canonical `NodeId`. Input is always explicit, including `null` for a null-input
+contract. The selector runs once against the loaded dependency and must return its exact owned,
+concrete callable object. Copied, stale, foreign, abstract, or asynchronous selections fail before
+transport. Current published Class resolution may expose an inherited alias to the same owning
+Method object; invocation still uses that exact owner. It never invents a new inherited target.
+
+The invoker preserves Function and Method input/output inference, including value streams and binary
+responses. A durable callable answers with the Execution it starts, so its call types as
+`ExecutionRef<Output>`. For value input, its optional third argument (fourth for instance Methods) accepts only
+`timeoutMs` and `idempotencyKey`, using the canonical Client session types. The Kernel accepts an
+`idempotencyKey` only for a durable callable, where it names the Execution the call starts, and
+refuses it for any other. Cancellation and the parent deadline are inherited. A timeout can shorten
+the deadline; a child does not inherit the parent's idempotency key. Detached
+`const { invoke } = dependencies.directory.caller` keeps its types and authority binding.
+
+## Binary Function input
+
+A Function can declare `input.binary(metadataSchema, { maximumBytes })`. The handler receives
+validated JSON metadata in `ctx.input` and a single-consumer `ReadableStream<Uint8Array>` plus its
+declared byte `length` in `ctx.body`. Consume the stream through EOF before returning a successful
+result. The bytes stay outside `Call.input` and are sent as the HTTP request body.
+
+```ts
+const receipt = await dependencies.files.caller.invoke(
+  d => d.functions.upload,
+  { mediaType: file.type },
+  { body: file },
+)
+await kernel.invoke(uploadReference, { mediaType: file.type }, { body: file })
+await kernel.call({ target: uploadReference.target, input: { mediaType: file.type } }, { body: file })
+```
+
+The typed `invoke` overload requires `body` for a binary Function and forbids it for a value-input
+Function. Raw `call` and `dispatch` also accept `{ body }`; the Server checks the resolved Function
+contract. A binary input cannot use `idempotencyKey`.
+
+`domain.dependencies` remains immutable loaded Schema metadata. `dependencies` is a separate frozen
+map containing only direct non-Kernel aliases. Each alias exposes exactly `.caller` and `.self`,
+each with its own `invoke`. Selectors do not nest, and there is no default `invoke` on the alias.
+
+## Explicit authority
+
+```ts
+import { reference } from '@astrale-os/sdk/client/session'
+import { Path } from '@astrale-os/sdk/graph/path'
+
+const directory = domain.dependencies.directory
+const lookup = reference(directory, directory.functions.lookup)
+await kernel.invoke(lookup, input)       // executing Domain
+await kernel.caller.invoke(lookup, input) // incoming caller Identity
+await kernel.self.invoke(lookup, input)   // executing Domain
+
+const rename = reference(directory, directory.classes.Employee.methods.rename)(Path.id(instance))
+await kernel.self.invoke(rename, input)
+```
+
+`graph.query` and `graph.mutate` execute named SDK definitions. `kernel.graph` is the canonical raw
+Kernel graph capability. Explicit named graph operations use `graph.caller.query` or
+`graph.self.mutate`. Bare `query` and `mutate` are ordinary functions.
+
+For U → A → B, a `caller` dependency call carries the incoming caller U and identifies A as the
+outbound principal. Inside B, local defaults use B's self authority; B's next `caller` dependency
+call forwards the same caller U with principal B. A caller is always one Identity. Existing
+Kernel authentication, delegation, callable capability, and Policy gates still apply. This wrapper
+neither grants permission nor retries denials under stronger authority.
+
+## Workflow and lifecycle
+
+A direct dependency invocation is one semantic effect. Workflows place it in one named step:
+
+```ts
+const employee = await step.run('lookup-employee', () =>
+  dependencies.directory.caller.invoke(d => d.functions.lookup, input),
+)
+await step.run('record-employee', () => mutate(recordEmployee, employee))
+```
+
+In a durable Workflow, the Step callback receives `dependencies` with the rest of its capabilities,
+acting as the Domain alone: only `self` exists. A call to a durable callable carries the Step's
+derived key, so a retried Step reaches the same child Execution; see
+[Durable Workflows](durable-workflows.md).
+
+```ts
+const shipment = await step.run('ship', ({ dependencies }) =>
+  dependencies.shipping.self.invoke(d => d.functions.ship, { parcel }),
+)
+```
+
+Inline Steps do not imply a distributed transaction, durable replay, or remote deduplication. Stream and
+binary objects cannot become portable step results; a native streaming response belongs in an
+Action. Invocation sessions remain open until a returned stream or binary body settles.
+
+Integrations remain useful for consumer-owned substitution, replay metadata, or failure translation.
+They are optional for direct declared Domain calls. Provider `execution.invoke` remains caller-bound,
+and Provider auth minting retains its existing self binding.
+
+## Migration
+
+The handler `client` alias is removed. Replace former `client` use with `kernel.caller` to preserve
+caller authority, or choose the new self default deliberately. Custom `InvocationAuthority`
+implementations now return `{ caller, graph, close }` with
+`caller: { kind, principal, identity, authentication }` and `graph: { caller, self }`; remove the
+redundant `client` member and the former `caller.authority` Grant. Replace `caller.authority` reads
+with `caller.identity`. The standalone ClientSession remains the canonical invocation surface,
+including the binary-input `invoke`, `dispatch`, and `call` overloads described above.
+
+The `union` authority mode is removed from `kernel`, `graph`, and `dependencies`. Replace
+`kernel.union`, `graph.union`, and `dependencies.alias.union` with `self` to act as the executing
+Domain, or with `caller` when the call must be authorized as the incoming caller. Admission rejects
+an `InvocationAuthority` whose `graph` still carries a `union` session.
+
+A call to an `anonymous` callable that carries no credential acts as the executing Domain: it
+receives `kernel`, `graph`, `query`, `mutate`, and `dependencies.alias.self`, and no `caller`
+selection (see [Public callables](#public-callables)). Before, `kernel`, `graph`, and `dependencies`
+were `null` there, `query` and `mutate` were absent, and a Provider's `execution.auth` was absent.
+A handler or Provider that used that absence to refuse calls without a credential still compiles and
+no longer refuses them. Replace such a test (`kernel === null`, `graph === null`,
+`dependencies === null`, `query === undefined`) with `caller.kind === 'anonymous'`, and a Provider's
+`execution.auth === undefined` with `execution.invoke === undefined`. Custom `InvocationAuthority`
+implementations add `self({ via, domain, context })`, which resolves to `{ graph: { self }, close }`.
+
+The linter accepts direct declared dependencies and canonical Kernel references. It checks exact
+callable requirements, rejects foreign handler imports, and reports opaque selectors as
+indeterminate. Requirement analysis never executes selectors during compilation.