stitchkit 0.93.0 → 0.94.1

package/llms/observability.txt

@@ -0,0 +1,893 @@
+# stitchkit/observability — stitchkit docs slice
+
+> Generated from the `docs/` tree by `bun run gen:llms`. The index of all slices is
+> `llms.txt` in this package.
+
+Contains: Guide: Observability  (docs/guide/observability.md) · API reference: `stitchkit/observability`  (docs/api/reference.md)
+
+==============================================================================
+# Guide: Observability  (docs/guide/observability.md)
+==============================================================================
+
+# Observability
+
+A request reaches your app through one of two surfaces — an HTTP route or a
+tool call (MCP / agent). Observability is the same question on both: *what
+happened, how long did it take, who made it, did it fail.*
+
+stitchkit answers this at two levels.
+
+- **The raw hooks** — `LifecycleHooks` and `ToolCallHooks`. Every request and
+  every tool call passes through a point you can observe. The lowest level;
+  always available. [Jump ↓](#the-raw-hooks)
+- **`stitchkit/observability`** — framework-owned HTTP completion plus canonical
+  tool hooks: W3C trace context, an `AsyncLocalStorage` request context, payload
+  sanitisation and `createObservability` with independent request/tool sinks.
+  [Start here ↓](#the-observability-module)
+
+stitchkit still ships no logger and no audit store, and auditing stays opt-in —
+those are the app's choice (→ ADR 0012, reaffirmed in ADR 0184). What it ships is
+the machinery that turns a completed call into a clean, normalised record, keeps
+the ones worth keeping, and does not lose them when the store is down.
+
+## The observability module
+
+`stitchkit/observability` is server-only. It has three parts — a trace context,
+a request context, and event projections — and you usually touch only the last.
+
+### createObservability
+
+`createObservability` configures request and tool projections independently.
+Every completed application call is normalised into one `RequestEvent`; HTTP
+completion is owned directly by `createHandler`, while MCP/Agent completion uses
+the canonical `ToolCallHooks` runner. Confirmed HTTP client cancellation stays
+in the access log by default and becomes a structured event only when the
+request sink opts in. There is no nested HTTP audit wrapper.
+
+```ts
+import { createObservability } from 'stitchkit/observability'
+
+const write = (event) => db.auditLog.create({ data: {
+  traceId: event.traceId,
+  source: event.source,
+  method: event.method,
+  path: event.path,
+  ok: event.ok,
+  statusCode: event.statusCode,
+  durationMs: event.durationMs,
+  userId: event.userId,
+  payload: event.payload,
+}})
+
+export const observability = createObservability({
+  request: {
+    write,
+    includePayload: false, // default: no Request.clone(), payload is null
+    includeCancelled: false, // default: keep client closes in access logs only
+    filter: (event) => event.method !== 'GET', // see the note on jobs below
+  },
+  tools: {
+    write,
+    filter: (event) => event.source === 'mcp' || event.source === 'agent',
+  },
+})
+```
+
+Wire each projection where its completion is owned:
+
+```ts
+createServer({ services, observability: observability.request })
+
+createMcpHandler({ /* … */ hooks: observability.toolCall })
+mountAgent(service, { hooks: observability.toolCall })
+```
+
+Each sink runs fire-and-forget and fails independently: a slow or broken request
+sink cannot block the response, suppress operational logging or break the tool
+sink.
+
+#### Client cancellation
+
+A physical client close is classified only when the request's own signal is
+aborted and the thrown value is either an `AbortError` or contains the exact
+`request.signal.reason` by identity at the top level or within at most eight
+standard `cause` links. Cause traversal is cycle-safe. No message or error-code
+matching is used; active requests and unrelated/deeper causes remain application
+failures. The access completion is always `499/info`, without application error
+fields or project `onError`.
+
+Structured request sinks are default-preserving: existing sinks receive no row
+for this outcome. Opt in when cancellation frequency belongs in the durable
+stream:
+
+```ts
+const observability = createObservability({
+  request: {
+    includeCancelled: true,
+    write: (event) => {
+      if (event.outcome === 'cancelled') return recordClientClose(event)
+      return recordApplicationRequest(event)
+    },
+  },
+})
+```
+
+An opted-in row has `outcome: 'cancelled'`, `ok: false`, `statusCode: 499`, the
+ordinary identity/trace/timing fields and no `errorCode`, `errorMessage` or
+`errorDetail`. `ok` remains the legacy success bit: branch on `outcome` first.
+Cancellation rows use the same filter, capacity, diagnostics, ordering,
+`flush()` and `close()` machinery as every request event. MCP protocol
+cancellation remains represented by `event.mcp.outcome`; Agent and CLI have no
+generic client-disconnect signal and are not inferred from error text.
+
+The fire-and-forget work has an explicit bounded lifecycle:
+
+```ts
+export const observability = createObservability({
+  request: {
+    maxPending: 1_000,
+    write: persistRequestEvent,
+    onSinkError: ({ error, event }) => reportAuditFailure(error, event),
+    onDrop: ({ reason, event, pending }) => {
+      reportAuditDrop({ reason, traceId: event.traceId, pending })
+    },
+  },
+  tools: {
+    maxPending: 500,
+    write: persistToolEvent,
+    onSinkError: ({ error, event }) => reportToolAuditFailure(error, event),
+    onDrop: reportToolAuditDrop,
+  },
+})
+
+const status = observability.getStatus()
+const report = await observability.close({ timeoutMs: 5_000 })
+```
+
+`maxPending` defaults to `1000` per sink and must be a positive safe integer.
+Filtering and sanitisation happen before a write occupies a pending slot. At
+capacity, a new event is rejected with `onDrop({ reason: 'capacity', ... })`;
+after close, admission reports `reason: 'closed'`. Existing writes are never
+cancelled. Sink, filter and diagnostic-callback failures remain isolated from
+the observed request or tool call and are reported through `onSinkError` when
+configured; `onSinkError`/`onDrop` cannot create unhandled rejections.
+
+`getStatus()` returns immutable per-surface and aggregate counters: capacity,
+received, accepted, filtered, completed, dropped, failed, preparation failures,
+preparing/pending work and the closed flag. Read it from readiness or metrics
+without parsing callback logs.
+
+`flush()` snapshots the current generation and waits only for events admitted
+up to that call. `close()` atomically stops admission, drains every accepted
+generation and returns the final counters plus `durationMs`; repeated unbounded
+calls return the same report. Graceful shutdown order is therefore:
+
+1. stop HTTP/MCP admission;
+2. wait for active requests and tool calls;
+3. `await observability.close({ timeoutMs })`;
+4. close the database/storage connection used by the sinks.
+
+### Give the drain a bound in a shutdown budget
+
+Unbounded, `close()` waits for every accepted event however long the sink takes.
+That is right when the sink is healthy and fatal when it is not: one write that
+never settles — a database that has stopped answering — holds the drain forever,
+and a shutdown that gives every other step a deadline then spends its entire
+budget here and exits by force. A drain that cannot be bounded cannot take part
+in a shutdown budget: it either fits, or it cancels the budget.
+
+Both `flush` and `close` take `{ timeoutMs?, signal? }`:
+
+```ts
+const report = await observability.close({ timeoutMs: 5_000 })
+if (!report.drained) {
+  const t = report.total
+  logger.warn('audit drain incomplete', {
+    stillWriting: t.pending + t.preparing,
+    neverWritten: t.received - t.filtered - t.completed,
+  })
+}
+```
+
+Those are two different numbers and the difference matters. `pending +
+preparing` is what the drain was still waiting for when the bound expired —
+events handed to `write` that may yet succeed. `received - filtered -
+completed` is every event the sink has not written **for any reason**: those,
+plus the ones already lost to `failed`, `dropped` and `preparationFailed`. A
+shutdown log that wants to state how much audit was lost wants the second; one
+that wants to explain why the drain did not finish wants the first.
+
+`drained` is read from the counters, not from which side of the race won, so a
+bound that expires on a sink that has in fact finished — a shutdown signal
+already aborted by an earlier step — reports `true` rather than a false alarm.
+
+The bound ends the **waiting**, not the writes: a sink's `write` is handed no
+cancellation, so an outstanding one keeps running against whatever the consumer
+closes next. It is a bound on waiting for I/O, not on wall time — no bound can
+preempt a `write` that occupies the event loop. Racing `close()` against your
+own timer looks equivalent and is not: it discards the report along with the
+wait.
+
+`flush(bound?)` takes the same bound and returns whether the generation it
+waited on settled. That outcome is returned rather than left to `getStatus()`,
+which cannot answer it: flush waits on the events admitted before the call while
+the status counts everything alive right now, so a complete flush and an expired
+one look identical there.
+
+Every other sink in the framework takes the same bound —
+`createApplicationEventSink`, `createApplicationSnapshotSink`,
+`createAgentRuntimeEventSink`, `createAgentObservability`. The hazard was never
+specific to the audit sink; it was specific to waiting without a limit.
+
+The drain itself is started once and shared, so a second `close` with a shorter
+bound observes the same drain under its own limit rather than starting another.
+Passing no bound keeps the previous behaviour exactly, including the identical
+returned promise.
+
+Stitchkit manages only in-process delivery. If process-crash durability matters,
+make `write` enqueue into a consumer-owned durable outbox and let that adapter
+own retry, replay and storage policy:
+
+```ts
+interface AuditOutbox {
+  enqueue(event: RequestEvent): Promise<void>
+}
+
+const observability = createObservability({
+  request: { write: (event) => outbox.enqueue(event) },
+  tools: { write: (event) => outbox.enqueue(event) },
+})
+```
+
+The core intentionally contains no retry scheduler, database dependency or
+disk queue. A durable adapter can use `(traceId, spanId, source, toolPhase)` as
+its idempotency identity.
+
+### RequestEvent
+
+Every surface produces the same shape — so a single audit table stays
+queryable across all three:
+
+| Field | Notes |
+|-------|-------|
+| `source` | `http` \| `mcp` \| `agent` \| `job` |
+| `kind` | `request` \| `job` — written on **every** row, so a filter never has to read it off an absent `method` |
+| `name` | what the work is, on a job (`agent-loop`, `broadcast-send`); absent on a request, which its method and path name |
+| `method` / `path` | the verb + path, or `TOOL` + `/{source}/{tool}` — **absent** on a job, which arrived over no transport |
+| `serviceName` / `action` | stable contract identity of the operation (→ ADR 0022) — from the contract, not parsed from `path`; set on every surface, present even on a pre-handler 400 |
+| `toolName` | tool calls only |
+| `httpMethod` | the contract verb on **tool** events (their `method` is `TOOL`) — filter reads vs writes across both surfaces with `(event.httpMethod ?? event.method) !== 'GET'` |
+| `dimensions` | app-defined domain dimensions (tenant / project / entity id) — see [request context](#request-context) |
+| `traceId` / `spanId` / `parentSpanId` | [W3C trace context](#trace-context) |
+| `outcome` | optional `'cancelled'` on explicitly enabled HTTP client-close rows; ordinary rows omit it |
+| `ok` / `statusCode` | legacy success bit plus real HTTP status, or `200`/`400` for a tool; an opted-in cancellation is `false` / `499`. `statusCode` is **absent** on a job — it had no transport to have a status on, and inventing one would be the same defect as the `method: 'AGENT'` this replaced. A job's outcome is `ok`, plus `errorCode` when it failed |
+| `durationMs` / `startedAt` | timing |
+| `errorCode` / `errorMessage` / `errorDetail` | failures only — `errorDetail` carries the structure the message flattens (e.g. Zod issues) |
+| `payload` | sanitised tool arguments; HTTP is `null` unless request `includePayload` is enabled |
+| `resultSize` / `responseBytes` | result item count + serialised size |
+| `userId` / `ipAddress` / `userAgent` | identity |
+
+### Work that is not a request — `runUnitOfWork`
+
+An agent loop launched fire-and-forget, a scheduled broadcast, a scenario engine
+tick: each has a start, a duration and an outcome, and none of them arrived over
+a transport. Run one inside a context and it is audited like everything else:
+
+```ts
+await runUnitOfWork({ name: 'broadcast-send', observability }, async () => {
+  setRequestDimensions({ projectId })
+  await send()
+})
+```
+
+The row carries `kind: 'job'`, the `name`, the duration and the outcome, and
+carries **no** `method`, `path` or `statusCode`, because the work had none. That
+absence is the point. The alternative is what consuming code had to do before —
+write `method: 'AGENT'` into the field for verbs — and a row that names a verb
+nobody used is worse than a row that names none, because a filter reading
+`method !== 'GET'` counts it as a write and nothing looks wrong.
+
+Which is why **a filter on `method` needs a decision about jobs.** `auditChanges`
+makes it explicitly: a row with no verb is kept, because whether it changed
+anything cannot be answered from the row, and an audit that drops what it did not
+examine reports zero and looks exactly like a system in which nothing happened.
+A hand-written `event.method !== 'GET'` keeps jobs too (`undefined !== 'GET'`);
+`event.kind === 'request' && event.method !== 'GET'` is how you exclude them on
+purpose.
+
+Without an `observability` the work still runs inside a context, so trace ids,
+`setRequestDimensions` and the bounded logger behave — recording is a separate
+decision from having a context. `traceparent` continues the trace of whatever
+scheduled the work, instead of starting an unrelated one.
+
+### Write the calls that changed something
+
+`auditChanges` is the filter most projects end up writing, shipped so they do not
+have to write it six ways:
+
+```ts
+import { auditChanges, createObservability } from 'stitchkit/observability';
+
+createObservability({
+  request: { write: saveAuditRow, filter: auditChanges },
+  tools: { write: saveAuditRow, filter: auditChanges },
+});
+```
+
+It drops `GET`, `HEAD` and `OPTIONS`, keeps everything else, and keeps `401` and
+`403` **whatever the verb was** — a refused read is the row an audit exists to
+hold, and a filter that drops every `GET` drops exactly that. An unrecognised
+verb is kept: an extra row costs bytes, a missing one costs the answer to "who
+changed this", silently and only later.
+
+One filter across HTTP, MCP and agent calls, because a tool call carries its
+contract verb in `httpMethod` while its `method` is the literal `TOOL`.
+
+Narrower policy stays yours — compose it:
+
+```ts
+filter: (event) => auditChanges(event) && event.serviceName !== 'health',
+```
+
+### Keeping the row when the store is down
+
+A sink is fire-and-forget, so an unreachable store means the event is simply
+gone — and a store is most likely to be unreachable during an incident, which is
+the window whose rows someone will later want most.
+
+`createSpooledSink` writes the row to a local append-only file first, offers it
+to the store second, and marks it delivered third. What a previous process left
+unmarked is replayed:
+
+```ts
+import { createSpooledSink } from 'stitchkit/observability';
+
+const audit = createSpooledSink({
+  path: '/var/lib/app/audit.ndjson',
+  write: saveAuditRow,
+});
+
+// Once, at startup, before the sink is wired — and await it.
+const { replayed, failed } = await audit.recover();
+
+createObservability({ request: { write: audit.write, filter: auditChanges } });
+```
+
+The guarantee is **at least once**. A crash between the store accepting a row and
+the file recording that it did replays the row, so **the store must be idempotent
+on the record key** — `event.spanId` by default, unique per call. A unique index
+on it turns the duplicate into a no-op; without one, replay writes the row twice.
+Exactly-once would need the file and the database to share a transaction. They do
+not.
+
+One process, one path. Two processes pointed at the same file replay each other's
+records: harmless against an idempotent store, wasteful always.
+
+### Request context
+
+When request observability is configured, `createHandler` establishes the
+`AsyncLocalStorage` request context itself and uses the same completion snapshot
+for operational logging and `RequestEvent`. No `wrapFetch` composition is
+needed:
+
+```ts
+createServer({ services, logging, observability: observability.request })
+```
+
+`wrapInRequestContext` remains available for a custom fetch pipeline that does
+not use `createHandler`; it is no longer part of built-in HTTP audit wiring:
+
+```ts
+Bun.serve({ fetch: wrapInRequestContext(customFetch) })
+```
+
+Some fields are filled in late. Set them from the hooks that know:
+
+```ts
+import {
+  setRequestDimensions,
+  setRequestError,
+  setRequestUser,
+} from 'stitchkit/observability'
+
+createAuthHook({ /* … */ inject: (ctx, user) => user && setRequestUser(user.id) })
+// ↑ on the HTTP path. Inside a tool call this writes to that call's own context
+//   (→ ADR 0045); a tool row takes its identity from the mount's `context`.
+// only to override what the framework already recorded — see below:
+setRequestError({ code: err.code, message: err.message, details: err.issues })
+```
+
+**The failure is recorded for you.** Every error travels one path inside the
+framework, and that path writes `{ code, message, details }` onto the context —
+so an audited failure names its cause whether or not you wrote an `onError`, and
+whether or not your `onError` returns its own `Response`. Where the envelope was
+scrubbed to `INTERNAL_SERVER_ERROR`, the row gets the **real** message rather than
+the placeholder; the caller still receives the scrubbed one. → ADR 0043
+
+`setRequestError` is therefore an **override**, not the wiring: call it when you
+want the row to say something other than what the framework derived (a domain
+code, a curated message, structured issues). The framework writes only when the
+context carries nothing yet, so your value always wins.
+
+**Endpoint identity is automatic — for contract operations.** The framework
+writes the matched operation's `(serviceName, action)` into the context at
+route-match, *before* validation, so both are present on every event for a
+request that matched a contract route, including a pre-handler 400. Nothing to
+wire.
+
+Raw routes may opt into the same context with `serviceName` and optional
+`action` on their `RawRoute` declaration. Without that declaration they remain
+absent, as they do for a request short-circuited by `onRequest` and an unmatched
+404 or 405. Both fields therefore remain optional in `RequestEvent`; a sink
+with a `NOT NULL` column on either still has to account for unattributed paths.
+
+When failure attribution itself is asynchronous, use `createErrorHook`'s
+`onError(error, info, ctx, endpoint)` observer. The framework awaits it before
+calling `render`, so identity or audit enrichment is visible to both the final
+error envelope and the request event. `endpoint` is the matched `MethodDef`, or
+`undefined` when routing failed before a method was selected.
+
+**Domain dimensions** — attach your own tenant / project / entity id with
+`setRequestDimensions`. It is an opaque `Record<string, string>` the core gives no
+meaning to (→ ADR 0021). Resolve it cheaply from `ctx.params` / headers in
+`beforeHandle` (success) or `onError` (a pre-handler failure — `ctx.params` /
+`ctx.req` are available there) and it lands on `event.dimensions` for the request
+either way, so your sink reads it as a column instead of re-parsing the path:
+
+> **On the tool path it lands on that call's own event, not the request's.** The
+> same hooks object is assignable to `ToolLifecycle`, so this recipe is the one
+> people apply to tools — and since ADR 0045 each tool call runs in its own
+> context. Read the value off the tool row (`event.toolName != null`); both rows
+> carry the same `traceId`, so recovering the enclosing request is one filter:
+>
+> ```bash
+> jq -s '[.[] | select(.dimensions.botId == "B7") | .traceId] as $t
+>        | .[] | select(.traceId | IN($t[]))' audit.jsonl
+> ```
+>
+> Worth checking your **runbooks** as well as your code when this lands: a
+> dimension filter written against request rows keeps parsing and starts
+> returning nothing.
+
+```ts
+// beforeHandle (success) and onError (failure) alike:
+const projectId = ctx.req?.headers.get('x-project') ?? String(ctx.params?.projectId ?? '')
+if (projectId) setRequestDimensions({ projectId })
+```
+
+Make the framework router share this trace id — so request logs and your
+application logs carry one id — by passing `getTraceId` as the resolver:
+
+```ts
+createHandler({ /* … */ traceId: getTraceId })
+```
+
+`getTraceId` returns `undefined` outside an active context, and the framework
+falls back to its own resolver — a trusted inbound `x-request-id` / `x-trace-id`,
+else a fresh id — so the line never carries the string `"undefined"`.
+
+Inside a **tool call** the context is that call's own (→ ADR 0045), so the
+enrichment a request log picks up describes the request, not the call.
+
+`getRequestContext()` / `getTraceId()` then return the active values from
+anywhere in the call — stamp `getTraceId()` onto every line your logger writes.
+The **request log picks the context up on its own**: with a context active, each
+completion line carries `userId`, `serviceName`, `action` and `dimensions`
+without any configuration.
+
+⚠️ In the **structured** output only — the `json` line and a custom `logger`'s
+`data`. The `pretty` `←` line is a line to read, not a record to query, and
+never carries them (nor `enrich`'s fields). Set
+[`logging.format: 'json'`](./server.md#two-formats-and-who-chooses) to see them
+locally; you do not need to change `NODE_ENV` or deploy.
+
+### Correlating with a reverse proxy
+
+Every response the stitchkit handler produces carries the resolved id as
+**`x-request-id`**. With `cors` configured it is in the default
+`Access-Control-Expose-Headers`, so a browser client can read it and quote it in
+a bug report. Stitchkit browser clients retain it as `ApiError.traceId` on
+response-backed failures. Note the deliberate asymmetry: inbound the id may arrive as
+`X-Trace-Id` *or* `X-Request-Id`; outbound there is one name and no alias.
+
+Log the same id from nginx and the two logs join on one key. `log_format` and
+`map` are `http {}`-context directives — put the block there, not inside
+`server {}`:
+
+```nginx
+log_format  stitch  '$remote_addr $status $request_time rid=$rid "$request"';
+
+# Fall back to nginx's own id for responses stitchkit never produced — Bun's
+# native `routes`, a throwing `onRequest` (which escapes before any response
+# exists), a redirect whose headers are immutable.
+map $upstream_http_x_request_id $rid {
+    ""      $request_id;
+    default $upstream_http_x_request_id;
+}
+
+access_log /var/log/nginx/access.log stitch;
+```
+
+`grep rid=<id>` across both logs then reconstructs the whole request. Tool calls
+join too: `createToolLogger`'s record carries `traceId`, so an MCP or agent call
+made inside a request lines up with it.
+
+### Trace context
+
+stitchkit speaks [W3C Trace Context](https://www.w3.org/TR/trace-context/). An
+inbound `traceparent` header is continued — its trace id is kept, its span
+becomes the parent — so a trace spans the front-end call, your HTTP handler and
+every tool call underneath it. With no inbound header a fresh root trace is
+minted. Each tool call opens a [`childSpan`](#trace-context) of the request it
+runs in.
+
+You rarely call the trace functions directly — `createHandler` request
+observability and `wrapInRequestContext` use them for you. They are exported (`resolveTraceContext`,
+`parseTraceparent`, `formatTraceparent`, `childSpan`) for when you need to
+propagate a `traceparent` onward to another service.
+
+The browser side starts the trace:
+[`createHttpClient({ trace: true })`](./client.md#httpclientconfig) emits a
+fresh root `traceparent` on every request, which the server then continues. The
+trace helpers themselves are browser-safe and also exported from the root
+`stitchkit` entry — a custom client can format its own header.
+
+The default CORS allow-list already permits `traceparent` / `tracestate`, so a
+cross-origin `trace: true` client works out of the box. If you set a custom
+`cors.headers`, extend `DEFAULT_CORS_ALLOW_HEADERS` rather than replacing it, or
+the preflight will reject the trace header.
+
+MCP tools use the same context. On a modern HTTP or stdio `tools/call`, the
+official SDK exposes W3C propagation values from request `_meta`; Stitchkit
+continues a valid `_meta.traceparent` before validation, lifecycle and hooks.
+If MCP metadata omits `traceparent`, an HTTP call keeps its ambient HTTP trace
+and a stdio call starts a new root. If the key is present but malformed or
+all-zero, the call still runs under a fresh local trace instead of trusting the
+ambient one. This makes explicit MCP propagation authoritative without allowing
+invalid input to break the tool.
+
+Bounded `tracestate` and `baggage` are available through
+`getRequestContext()?.trace` for outbound propagation. Their contents are not
+copied into the standard `RequestEvent`; never use them for auth, RBAC, tenant
+selection or rate-limit identity. Every MRTR attempt reads its current request
+metadata, so callers that want one trace across rounds must propagate it on
+each request.
+
+> **Span ids live in the request context, not on `ctx`.** The handler `ctx`
+> carries a single `traceId`; the full
+> `{ traceId, spanId, parentSpanId, tracestate?, baggage? }` is on
+> the observability request context. To stamp `spanId` / `parentSpanId` into an
+> audit row read `getRequestContext()?.trace`, not `ctx.spanId`:
+>
+> ```ts
+> const trace = getRequestContext()?.trace
+> audit({ traceId: trace?.traceId, spanId: trace?.spanId, parentSpanId: trace?.parentSpanId })
+> ```
+
+### Sanitisation
+
+A payload goes into an audit row only after `sanitizePayload`:
+
+- **secret-named keys are masked** — a key whose words include a secret term is
+  redacted (`password`, `sessionToken`, `X-Api-Key`, `authorization`, `cookie`,
+  …), while identifiers that merely contain one (`authorId`, `sessionCount`,
+  `tokenizer`) survive. This applies wherever a key exists — an object field and
+  a `Map` entry alike, at any depth. A `Set` member has no key, so nothing there
+  is masked by name;
+- **binary blobs** (`Uint8Array`, `Blob`, `FormData`) collapse to metadata —
+  never the bytes;
+- the result is **capped** — anything over the byte limit becomes a preview.
+
+`createObservability` runs it on every emitted event; tune each sink separately:
+
+```ts
+createObservability({
+  request: {
+    write,
+    includePayload: true,
+    sanitize: { maxBytes: 8_000, sensitiveKeys: /password|token|pin/i },
+  },
+  tools: {
+    write,
+    sanitize: { maxBytes: 8_000, sensitiveKeys: /password|token|pin/i },
+  },
+})
+```
+
+`redact`, `truncatePreview` and `measureSize` are exported on their own if you
+need to sanitise something outside the audit path.
+
+## The raw hooks
+
+Tool observability is built on hooks you can also use directly — for a one-off
+metric, a custom log line, anything that is not a full audit row.
+
+| Surface | Hook | Fires |
+|---------|------|-------|
+| HTTP | `LifecycleHooks.afterHandle` / `onError` | after each HTTP request |
+| MCP & agent tools | `ToolCallHooks.afterToolCall` | after each tool call |
+| MCP & agent tools | `ToolCallHooks.onToolError` | when executable parsing, extension resolution, lifecycle or the handler throws |
+
+`afterHandle(ctx, result, endpoint)` runs after a handler returns;
+`onError(ctx, error, endpoint)` when one throws. `afterToolCall(options)` runs
+after every tool call —
+success and error alike — carrying the tool name, the arguments, the result, the
+duration, the call context, the endpoint identity, and (only when the call failed
+by throwing) the raw thrown value.
+
+A confirmed HTTP client disconnect is the exception to the raw-hook table: it
+is transport cancellation, so neither `afterHandle` nor project `onError` runs.
+Use the `499/info` access completion or opt-in request cancellation event above.
+
+```ts
+createMcpHandler({
+  serverInfo, auth, services,
+  hooks: {
+    afterToolCall: ({ toolName, result, durationMs }) => {
+      metrics.timing(`tool.${toolName}`, durationMs, { ok: String(result.ok) })
+    },
+  },
+})
+```
+
+Log **after** completion — a record is of a *finished* call; you need the
+outcome and the duration, neither of which exists before the handler runs.
+
+### The cause behind a failed tool call
+
+`afterToolCall` gives you the `ToolResult`, and for a thrown `AppError` that is
+the whole story — `code` and `details` are yours to route. For anything else it
+is not: an unexpected throw (a dropped connection, a `TypeError`) is scrubbed to
+a bare `INTERNAL_SERVER_ERROR` with the message `Internal server error`, because
+a raw `Error.message` can carry a connection string or a file path. The result
+your sink receives says nothing about why.
+
+`onToolError` is where that value still exists — **as thrown**, before
+normalisation, stack and `cause` intact:
+
+```ts
+createMcpHandler({
+  serverInfo, auth, services,
+  hooks: {
+    onToolError: ({ toolName, error, endpoint }) => {
+      reportToolFailure({
+        tool: toolName,
+        action: endpoint.key,
+        message: error instanceof Error ? error.message : String(error),
+        stack: error instanceof Error ? error.stack : undefined,
+      })
+    },
+  },
+})
+```
+
+It fires for a throw from executable params/input/`ToolExtend` parsing,
+extension resolution, `beforeHandle`, the handler or `afterHandle` — every span
+where information is destroyed — and runs **before** `afterToolCall`. It does
+not fire for an ordinary validation result, an output-schema mismatch or a
+`beforeToolCall` rejection: each is already described in full by its
+`ToolResult`.
+
+It observes, it does not handle: the tool envelope is always the framework's, a
+returned value is ignored, and a throw from the hook itself is reported to
+`console.error` and swallowed rather than replacing the failure it was called to
+observe. (This is also why it lives on `ToolCallHooks` rather than being an
+`onError` twin on `ToolLifecycle` — `LifecycleHooks.onError` returns a
+`Response`, which a tool call has no use for, and a whole `createServer` hooks
+object must stay assignable to `ToolLifecycle`.)
+
+### Compose lifecycle policy explicitly
+
+When auth, a feature policy and audit identity are independent modules, compose
+their existing hooks instead of forwarding each phase by hand:
+
+```ts
+import { composeLifecycleHooks } from 'stitchkit/server'
+import { composeToolLifecycle } from 'stitchkit/tools'
+
+const httpLifecycle = composeLifecycleHooks(authLifecycle, featurePolicy, auditIdentity)
+const toolLifecycle = composeToolLifecycle(authHook, featureToolPolicy, auditToolIdentity)
+```
+
+Phases run left to right. `onRequest` stops on the first `Response`; `onError`
+falls through until a hook handles it; after transforms feed the current value
+forward and preserve it when a hook returns `undefined`. Throws remain owned by
+the existing outer error boundary, and the original context/signal identity is
+never replaced.
+
+**Do not reach for `setRequestError` here.** It writes to the *request* context,
+which the built-in **tool** row does not read: a tool event takes
+`errorCode` / `errorMessage` / `errorDetail` from the `ToolResult`, and only
+identity and `dimensions` from the context. Calling it in `onToolError` would
+leave the tool row exactly as scrubbed as before. It is right for the **HTTP**
+path, where the request *is* the record. (Since ADR 0045 a tool call runs in its
+own context, so it can no longer write into the enclosing `/mcp` request's row
+either — the call is simply not where that helper belongs.)
+
+### One row that names the cause
+
+You do not need to correlate the two hooks yourself. `afterToolCall` receives the
+same raw value as the named `error` field, so one hook can build one record:
+
+```ts
+hooks: {
+  afterToolCall: ({ toolName, result, durationMs, endpoint, error }) => {
+    void writeRow({ toolName, result, durationMs, cause: error, endpoint })
+  },
+}
+```
+
+`error` is present only when the call failed by **throwing** — a
+validation failure or a `beforeToolCall` rejection leaves it `undefined`, because
+neither ever had a raw value to lose. Consumers destructure only the fields they
+use; future optional fields do not change callback arity.
+
+`createObservability({ tools })` uses it already. Where the envelope was scrubbed to
+`INTERNAL_SERVER_ERROR`, the row's `errorMessage` becomes the real message
+instead of the placeholder; a truthful envelope (a thrown `AppError`, a
+`ZodError`) is left alone, `errorCode` and `errorDetail` are untouched, and the
+stack is not written — that is `onToolError`'s job, for a tracker that wants it.
+The caller still receives the scrubbed envelope in every case: the raw text
+reaches your server-side record, never the response. → ADR 0042
+
+So the two hooks divide by **purpose**, not by capability — `onToolError` for a
+sink of your own (a tracker, a stack, an alert), `afterToolCall` for the record.
+
+### Keying a row on (service, action)
+
+Built-in observability keys every event by **service** and **action**
+(`event.serviceName` / `event.action`, → ADR 0029) — reach for the raw hook only
+when you also need the handler **output**, which the audit wrapper never sees. For
+that, read the endpoint identity off the `OperationIdentity` the tool hook
+receives — `endpoint.serviceName` and `endpoint.key` (the endpoint key or native
+action,
+e.g. `updatePartial`). They are stable and always present (→ ADR 0022); the action
+is not in the URL and `toolName` is absent on HTTP-only endpoints, so this is the
+only reliable pair. Contract operations are full `MethodDef` values; native MCP
+operations intentionally have no HTTP `path`. `afterHandle` also gives you the handler `result` — so it is
+the home for a rich mutation audit that records output:
+
+```ts
+hooks: {
+  afterHandle: (ctx, result, endpoint) => {
+    void logMutation({
+      service: endpoint.serviceName,
+      action: endpoint.key,
+      input: ctx.input,
+      output: result,
+      userId: ctx.userId,
+      traceId: ctx.traceId,
+    })
+    return result   // don't transform
+  },
+}
+```
+
+> **Why HTTP observability is framework-owned, not a lifecycle hook.**
+> `LifecycleHooks` has a single `onError`; an audit built on it would compete
+> with the app's error renderer and miss raw/unmatched exits. `createHandler`
+> sees the final response on every path and emits one completion without
+> consuming an application hook.
+
+Keep any sink **asynchronous and self-contained**: a slow or failing write must
+never block or break the request. Swallow the sink's own errors.
+
+## Typed dimensions and bounded loggers
+
+`createDimensionsProjector` keeps endpoint metadata opaque. Configure typed
+request, result and error projections, then call its phase methods from the
+corresponding lifecycle hooks. Every projection writes through
+`setRequestDimensions`; repeated keys follow the declared `overwrite`,
+`preserve` or `error` policy. Two of its inputs are programming errors and
+throw, which fails the request rather than recording a lie: a repeated key
+under `collision: 'error'`, and a key named `__proto__` (it would set the
+bag's prototype — it can only arrive from parsed input, never from an object
+literal, which has no such own key).
+
+`createBoundedLogger` decorates the existing `StitchLogger` interface, so Pino,
+console and application sinks keep their ownership. It reuses the same
+sanitizer as request audit: sensitive keys, configured paths, URL fragments and
+`Error` fields are redacted before string/collection/depth/total-byte bounds are
+applied. Active trace, span, user and dimensions are reserved framework fields;
+caller data cannot overwrite them. A throwing getter, cyclic value or failing
+sink never escapes into application code.
+
+One string bound for the whole record cuts an error's stack from the bottom —
+which is exactly where your own frames are. Raise the ceiling for that key alone,
+wherever it sits (`stack` at the top of one record and under `errorDetail` in the
+next is the same field), without touching the bound for everything else:
+
+```ts
+createBoundedLogger({
+  sink,
+  bounds: { stringLength: 4_000, stringLengthByKey: { stack: 32_000 } },
+})
+// or directly: redact(value, { maxStringLength: 4_000, maxStringLengthByKey: { stack: 32_000 } })
+```
+
+Keyed by name, not by path — the question is "what is this field", not "where is
+it". The entry ceiling (`entryBytes`) keeps the last word: a record that does not
+fit still collapses, but the decision is made for the record, not by cutting one
+field short in advance.
+
+==============================================================================
+# API reference: `stitchkit/observability`  (docs/api/reference.md)
+==============================================================================
+
+## `stitchkit/observability`
+
+Server-only. The audit layer one level above the raw hooks — W3C trace context,
+an `AsyncLocalStorage` request context, payload sanitisation and a normalised
+audit event. See the [Observability guide](../guide/observability.md).
+
+### Events
+
+| Export | Kind | Summary |
+|--------|------|---------|
+| `createObservability` | function | configure framework-owned request completion and canonical tool event sinks — [guide](../guide/observability.md#createobservability) |
+| `auditChanges` | function | the sink `filter` most projects write themselves — drops `GET`/`HEAD`/`OPTIONS`, keeps `401`/`403` whatever the verb, keeps an unrecognised verb |
+| `createSpooledSink` / `SpooledSink` / `SpooledSinkConfig` / `SpoolRecovery` | function / _types_ | spool the row to an append-only file before the store and replay what a previous process left undelivered — at least once, keyed on `spanId` |
+| `createDimensionsProjector` / `DimensionsProjector` / `DimensionsProjectorConfig` / `ProjectedDimensions` | function / _type_ | typed request/result/error attribution projected into the existing request context |
+| `DimensionCollision` / `SetRequestDimensionsOptions` | _type_ | explicit overwrite, preserve or error policy for dimension keys |
+| `createBoundedLogger` | function | decorate a `StitchLogger` with request context, shared sanitisation, redaction and total bounds |
+| `DEFAULT_REDACT_PATHS` | constant | baseline credential/token paths added to the sanitizer's sensitive-key policy |
+| `BoundedLoggerBounds` / `BoundedLoggerOptions` | _type_ | per-value and total record ceilings (`stringLengthByKey` raises the string bound for named keys such as `stack`) plus sink/redaction configuration |
+| `RequestEvent` | _type_ | the normalised audit event handed to the sink; opt-in HTTP cancellation rows carry `outcome: 'cancelled'` |
+| `runUnitOfWork` / `RunUnitOfWorkOptions` | function / _type_ | run work that did not arrive over a transport — an agent loop, a scheduled broadcast — inside one context, with the same completion record a request gets |
+| `RequestContextKind` | _type_ | `'request'` \| `'job'` — what the row describes, written on every event so a filter never has to read it off an absent `method` |
+| `ObservabilityConfig` | _type_ | independent request and tool sink configuration |
+| `Observability` | _type_ | `{ request?, toolCall, getStatus(), flush(bound?): Promise<boolean>, close(bound?) }` with bounded sink lifecycle |
+| `ObservabilityDrainBound` | _type_ | `timeoutMs` and/or `signal` limiting how long `flush`/`close` wait; the wait ends, outstanding writes do not |
+| `ObservabilitySinkStatus` | _type_ | immutable counters for one bounded request/tool sink |
+| `ObservabilityStatus` | _type_ | per-surface plus aggregate operational snapshot |
+| `ObservabilityDrainReport` | _type_ | final closed/drained snapshot, `durationMs` (the shared drain's age, not this call's wait), and `drained` — read from the counters, `false` only when `total.pending + total.preparing` is nonzero |
+| `ObservabilitySinkStatusSchema` / `ObservabilityStatusSchema` / `ObservabilityDrainReportSchema` | schema | runtime schemas for status/report integration boundaries |
+| `RequestEventSinkConfig` | _type_ | `write`, filter/sanitisation, `maxPending`, `onSinkError` and `onDrop` |
+| `RequestObservabilityConfig` | _type_ | request sink plus opt-in payload capture and default-off `includeCancelled` rows |
+| `SinkDropReason` | _type_ | `'capacity' \| 'closed'` |
+| `SinkError` | _type_ | isolated sink/projection failure and optional event |
+| `SinkDrop` | _type_ | rejected event, reason and current pending count |
+| `HttpRequestCompletion` | _type_ | the single framework-owned HTTP outcome, including optional neutral cancellation, projected to logging and request events |
+| `HttpRequestObserver` | _type_ | server-facing projection consumed by `HandlerConfig.observability` |
+
+### Request context
+
+| Export | Kind | Summary |
+|--------|------|---------|
+| `WrapRequestContextOptions` | _type_ | options for `wrapInRequestContext` |
+| `wrapInRequestContext` | function | run a fetch handler inside a request context — [guide](../guide/observability.md#request-context) |
+| `getRequestContext` | function | the active request context |
+| `getTraceId` | function | the active trace id — pass as `traceId` to `createServer` |
+| `getUserId` | function | the active user id, once auth has resolved it |
+| `setRequestUser` | function | attach the resolved user to the active context |
+| `setRequestEndpoint` | function | attach the resolved endpoint identity to the active context |
+| `setRequestDimensions` | function | attach custom audit dimensions to the active context |
+| `setRequestError` | function | record the error outcome on the active context |
+| `runWithRequestContext` | function | run a function inside a given context |
+| `RequestContext` | _type_ | the per-request record |
+
+### Trace context
+
+| Export | Kind | Summary |
+|--------|------|---------|
+| `resolveTraceContext` | function | the trace for a request — `traceparent` continued or fresh |
+| `resolvePropagationContext` | function | continue bounded MCP/W3C propagation metadata with optional ambient fallback |
+| `parseTraceparent` | function | parse a `traceparent` header |
+| `formatTraceparent` | function | render a `traceparent` header value |
+| `createTraceContext` | function | a fresh root trace |
+| `childSpan` | function | a child span of a parent trace |
+| `TraceContext` | _type_ | `{ traceId, spanId, parentSpanId?, tracestate?, baggage? }` |
+
+### Sanitisation
+
+| Export | Kind | Summary |
+|--------|------|---------|
+| `sanitizePayload` | function | redact secrets and cap size — [guide](../guide/observability.md#sanitisation) |
+| `redact` | function | mask secret-named keys, drop binary blobs |
+| `truncatePreview` | function | cap a value by serialised size |
+| `measureSize` | function | item count + byte size of a result |
+| `JsonValue` | _type_ | a JSON-serialisable value |
+| `SanitizeOptions` | _type_ | tuning for `redact` / `sanitizePayload` — `maxStringLengthByKey` bounds a named key wherever it sits without changing the record-wide bound |
+| `SizeMeasure` | _type_ | the result of `measureSize` |
+
+---