@arnilo/prism 0.0.2 → 0.0.4

package/docs/postgres-persistence.md

@@ -0,0 +1,141 @@
+# PostgreSQL persistence
+
+## What it does
+
+The optional `@arnilo/prism-session-store-postgres` package ships a production-oriented PostgreSQL adapter that implements:
+
+- `SessionStore` — atomic `append` / `list` / `get` / `readBranchPath`
+- `RunLedger` — durable run, event, tool-call, and usage rows
+- `ProductionPersistenceStore` — cursor-paginated `query*` reads plus generic `checkpoints` and atomic `leases` capabilities
+
+Factory:
+
+- `createPostgresPersistence(options)` (async)
+- `PostgresPersistenceOptions`
+- `PostgresPersistence.close()` (async; ends adapter-owned pools only)
+
+The adapter uses `pg@^8.22.0`, applies versioned migrations from the shared Plan 056 schema model inside a transaction guarded by `pg_advisory_xact_lock`, validates/quotes schema identifiers, and passes the full session-store and run-ledger conformance suites when `PRISM_TEST_POSTGRES_URL` is set.
+
+## When to use it
+
+Use this package when you need server-backed persistence with pooled connections and multi-writer concurrency:
+
+- hosted SaaS agents with PostgreSQL as the system of record
+- managed cloud databases (RDS, Cloud SQL, Neon, Supabase, etc.)
+- CI integration tests against a real PostgreSQL service
+
+Prefer [`@arnilo/prism-session-store-sqlite`](sqlite-persistence.md) for local CLI tools, single-writer workloads, and network-free default tests.
+
+## Inputs / request
+
+```ts
+import { Pool } from "pg";
+import { createPostgresPersistence } from "@arnilo/prism-session-store-postgres";
+```
+
+| Field | Type | Purpose |
+| --- | --- | --- |
+| `pool` | `Pool` | Existing `pg` pool (caller owns lifecycle). |
+| `connectionString` | `string` | Create an adapter-owned bounded pool when `pool` is omitted. |
+| `schema` | `string` | PostgreSQL schema for Prism tables. Defaults to `"prism"`. Validated and double-quoted. |
+| `poolMax` | `number` | Maximum pool size for adapter-owned pools. Defaults to `10`. |
+| `poolConfig` | `PoolConfig` | Additional `pg` options (TLS, idle timeout, application name, etc.). |
+
+Hosts own TLS (`ssl` in `poolConfig`), credentials, connection limits, and backup/retention enforcement.
+
+## Outputs / response / events
+
+`createPostgresPersistence()` returns one object implementing the three persistence contracts plus generic checkpoints and leases:
+
+| Method group | Behavior |
+| --- | --- |
+| `SessionStore.append` | One transaction per append: parent existence check, idempotency dedup, duplicate-id rejection, entry insert. |
+| `SessionStore.list` / `get` | Indexed reads by `session_id` and primary key. |
+| `SessionStore.readBranchPath` | Recursive ancestor query from `leafId` (or latest leaf) in root→leaf order. |
+| `RunLedger.append*` | Inserts run/event/tool/usage rows; events receive monotonic per-run `sequence` values. |
+| `ProductionPersistenceStore.query*` | Parameterized cursor pagination on indexed columns with tenant/account/user filters. |
+| `checkpoints` | Generic versioned `CheckpointStore` backed by `prism_checkpoints`; ownership, CAS/fencing checks, and bounded pagination. |
+| `leases` | Atomic `LeaseStore` backed by `prism_leases`; database-clock expiry, opaque renew/release token, monotonic takeover fence. |
+| `close()` | Ends the pool when the adapter created it from `connectionString`. |
+
+Migrations run automatically on open and are idempotent across reopen. Concurrent setup uses per-schema advisory transaction locks.
+
+## Request/response example
+
+```json
+{
+  "connectionString": "postgres://app:***@db.example.com:5432/prism",
+  "schema": "prism",
+  "poolMax": 10,
+  "poolConfig": {
+    "ssl": { "rejectUnauthorized": true }
+  }
+}
+```
+
+## Implementation example
+
+```ts
+import { Pool } from "pg";
+import { createAgentSession } from "@arnilo/prism";
+import { createPostgresPersistence } from "@arnilo/prism-session-store-postgres";
+import { runSessionStoreConformance } from "@arnilo/prism/testing/session-store-conformance";
+
+const pool = new Pool({
+  connectionString: process.env.DATABASE_URL,
+  max: 10,
+  ssl: { rejectUnauthorized: true },
+});
+
+const persistence = await createPostgresPersistence({ pool, schema: "prism" });
+
+await runSessionStoreConformance(
+  async () => createPostgresPersistence({ pool, schema: "prism" }),
+  { exerciseReadBranchPath: true, exerciseReopen: true },
+);
+
+const session = createAgentSession({
+  sessionStore: persistence,
+  runLedger: persistence,
+});
+
+await session.run("hello");
+await persistence.close();
+await pool.end();
+```
+
+Live conformance and integration tests:
+
+```bash
+PRISM_TEST_POSTGRES_URL="$DATABASE_URL" npm run test:postgres --workspace @arnilo/prism-session-store-postgres
+```
+
+## Extension and configuration notes
+
+- The package is optional and workspace-local; `@arnilo/prism` core has no PostgreSQL dependency.
+- Schema names must match `^[a-zA-Z_][a-zA-Z0-9_]*$`; the adapter quotes them and never interpolates user values into identifier positions.
+- `SessionAppendOptions` idempotency rows are durable in `prism_session_append_idempotency` and survive reopen.
+- Schema version **1** (`001_init`) matches `@arnilo/prism/testing/persistence-schema` — SQLite adapters share the same model with dialect-local DDL.
+- Pass an existing `pg` `Pool` when your host already manages pooling, TLS, and credential rotation.
+
+## Security and performance notes
+
+- **Parameterized SQL only.** Session ids, idempotency keys, tenant ids, and JSON payloads are bound parameters (`$1`, `$2`, …).
+- **Identifier validation.** Configurable `schema` is validated and double-quoted; table names are fixed constants in adapter SQL.
+- **TLS and credentials.** Configure via `pg` `Pool` / `PoolConfig`; the adapter does not read environment variables unless the host passes them into `connectionString` or `poolConfig`.
+- **Redaction upstream.** Event and tool-call payloads may contain secrets; redact before ledger writes. The adapter does not scan or rewrite row contents.
+- **Bounded pool.** Adapter-owned pools default to `max: 10`. Hosts with heavy concurrency should supply their own pool sizing.
+- **Indexed operations.** Append, parent validation, idempotency dedup, branch reads, and pagination use the indexes documented in [Database persistence](database-persistence.md); normal paths avoid sequential scans.
+- **Migration locking.** `pg_advisory_xact_lock` prevents concurrent `001_init` races when multiple processes open the adapter at once.
+- **Tenant isolation.** `tenant_id` / `account_id` / `user_id` columns participate in query filters; hosts must still scope writes correctly.
+- **Benchmark target.** Indexed append + paginated branch read on a warm pool should stay under **50 ms p95** for local/CI-sized datasets (≤100k entries per session); measure with your pool size and hardware before production sizing.
+
+## Related APIs
+
+- [Database persistence](database-persistence.md): shared schema model, conditional append pattern, indexes.
+- [SQLite persistence](sqlite-persistence.md): file-backed alternative for local/single-writer hosts.
+- [Session store conformance](session-store-conformance.md): `assertSessionStoreConforms` / `runSessionStoreConformance`.
+- [Run ledger conformance](run-ledger-conformance.md): `assertRunLedgerConforms` / `runRunLedgerConformance`.
+- [Persistence, credentials, and multimodality primitives](persistence-credentials-multimodality-primitives.md): package matrix and threat model.
+- [Workflows](workflows.md): adapt `persistence.checkpoints` and pass `persistence.leases` to `createWorkflowCoordinator()` for durable multi-process workflow execution.
+- [Migration guide](migration.md): moving from JSONL/in-memory to database-backed persistence.

package/docs/provider-conformance.md

@@ -93,6 +93,22 @@ const body = JSON.parse(String(fetchInit.body));
 assertSerializedRequestCoversContent(request, body, { unsupported: ["image"] });
 ```
 
+Multimodal coverage example:
+
+```ts
+const request = {
+  model: { provider: "openai", model: "gpt-5.1", capabilities: { input: ["text", "file", "document", "audio"] } },
+  messages: [{ role: "user", content: [
+    { type: "file", mediaType: "application/pdf", name: "report.pdf", data: "..." },
+    { type: "audio", mediaType: "audio/wav", data: "..." },
+  ] }],
+};
+
+assertSerializedRequestCoversContent(request, body);
+```
+
+Pass `unsupported` only for modalities the provider deliberately omits from the wire format while still accepting the turn via another block type.
+
 Protected header-ownership example:
 
 ```ts
@@ -127,6 +143,7 @@ The helpers are a testing subpath only. Provider packages can use them with thei
 - Use fake credentials only in fixtures.
 - The helpers collect one stream into memory; keep conformance fixtures small.
 - Redaction remains the provider/runtime boundary's job. Use `assertNoSecretLeak()` with known fake secrets to catch regressions, not as a general secret scanner.
+- First-party providers read SSE streams and HTTP error bodies through bounded helpers from `@arnilo/prism/providers/transport` (`readSseEvents` / `readSseData`, `readBoundedResponseText`). Oversized remote input terminates with `ProviderTransportError` instead of unbounded buffering.
 
 ## Related APIs
 

package/docs/provider-layer.md

@@ -129,7 +129,7 @@ const agent = createAgent({ model: { provider: own.id, model: "demo" }, provider
 - Provider event helpers return plain `ProviderEvent` objects.
 - `providerError()` converts unknown errors to redacted `ErrorInfo` through `errorToErrorInfo()` and preserves safe string/number `code` fields for retry classification.
 - `createMockProvider()` returns an `AIProvider` whose `generate()` yields the scripted events in order and checks `request.signal?.aborted` before each event.
-- The agent/session runtime passes its per-run abort signal as `ProviderRequest.signal`. `ProviderRequestOptions.timeoutMs`, `maxRetries`, and `maxRetryDelayMs` are deprecated inert hints in first-party providers; use `RunOptions.signal`/host abort controllers for timeouts and `AgentConfig.retry`/`RunOptions.retry` for retry.
+- The agent/session runtime passes its per-run abort signal as `ProviderRequest.signal`. `ProviderRequestOptions.structuredOutput` requests provider-native JSON-schema output when the model declares `capabilities.structuredOutput`; unsupported models fail before fetch. `ProviderRequestOptions.timeoutMs`, `maxRetries`, and `maxRetryDelayMs` are deprecated inert hints in first-party providers; use `RunOptions.signal`/host abort controllers for timeouts and `AgentConfig.retry`/`RunOptions.retry` for retry.
 
 ## Request/response example
 

package/docs/provider-primitives.md

@@ -0,0 +1,281 @@
+# Provider primitives
+
+## What it does
+
+This page freezes the reusable provider transport, OpenAI-style serialization, structured-output capability, and observability designs for Plan 054. It inventories duplicated helpers across `@arnilo/prism` and first-party provider packages, documents trust boundaries, and selects the generic APIs that Tasks 1–6 will implement and migrate to.
+
+Implementation is **shipped** for transport and OpenAI serialization primitives (Plan 054 Task 1). **All first-party providers are migrated** to those primitives (Plan 054 Task 2); package-local `sse.ts`, `safeText`, `parseArgs`, and duplicate OpenAI serializers are removed. **Native structured output** (`StructuredOutputOptions`, provider mappers, capability gating) shipped in Task 4. Observability contracts remain design-only until Task 5.
+
+## When to use it
+
+- **Provider package authors** implementing or migrating a first-party adapter should import shared primitives from `@arnilo/prism/providers/transport` and `@arnilo/prism/providers/openai` instead of copying `sse.ts`, `safeText`, `parseArgs`, or serializers.
+- **Host apps** choose native structured output via `ProviderRequestOptions.structuredOutput` when the model declares support; otherwise they keep the artifact generate→validate→revise loop ([Structured output](structured-output.md)).
+- **Operators** enable observability through extended agent events and the optional OpenTelemetry adapter package ([Observability](observability.md)).
+
+## Inventory (2026-07-14 baseline)
+
+Static scan of root `src/providers/` and `packages/provider-*/src/` before Plan 054 implementation.
+
+### Duplicated protocol helpers (baseline → Task 2)
+
+| Helper | Baseline copies | Task 2 status |
+| --- | ---: | --- |
+| `readSseData` / SSE reader | 7 (6× `sse.ts` + inline openai-compatible) | **Migrated** — `@arnilo/prism/providers/transport`; local `sse.ts` deleted |
+| `readNeuralWattSseFrames` | 1 | **Migrated** — thin adapter over `readSseEvents` `comments` in `provider-neuralwatt` |
+| `safeText` | 9 | **Migrated** — `readBoundedResponseText` with optional `secrets` |
+| `parseArgs` | 8 | **Migrated** — `parseJsonObjectArguments` (typed error on malformed JSON) |
+| `toTool` | 7 OpenAI-style | **Migrated** — `serializeOpenAITool` where wire shape matches |
+| `toUsage` / usage mapper | 6 | **Local** — provider-specific wire field names remain per package |
+| Message serializer | 7 local | **Mixed** — `serializeOpenAIChatMessage` / `assertOpenAIChatMessage` where OpenAI Chat Completions; Anthropic, OpenRouter cache, OpenAI Responses, NeuralWatt `reasoning_content` stay local |
+
+### Retry / rate-limit metadata paths
+
+| Surface | Owner | Behavior today |
+| --- | --- | --- |
+| Runtime retry | `@arnilo/prism` `AgentConfig.retry` / `RunOptions.retry` | Classifies `ErrorInfo.code`; provider packages set numeric HTTP `code` on errors |
+| `ProviderRequestOptions.maxRetries` / `timeoutMs` | Contracts | **Deprecated / inert** in first-party providers |
+| NeuralWatt `classifyNeuralWattError` | `packages/provider-neuralwatt` | Parses `Retry-After`, `error.retry_after`, `retry_strategy`; no extra network calls |
+| Quota endpoint throttling | `packages/provider-neuralwatt/quota.ts` | Documents 1 rps limit; caller-owned cache |
+
+No generic core helper extracts `Retry-After` / `x-request-id` for all providers yet.
+
+### Structured output
+
+| Surface | Status |
+| --- | --- |
+| `ProviderRequestOptions` | `structuredOutput?: StructuredOutputOptions` |
+| `ModelCapabilities` | `structuredOutput?: boolean \| "json_schema"` |
+| Provider wire mapping | None — artifact loop is the only structured-output path |
+| Schema validation | Host-owned via artifact `validator`; no provider-native request |
+
+### Telemetry / observability events
+
+| Event / hook | Owner | Payload classification |
+| --- | --- | --- |
+| `ProviderEvent` union | Core | Text/thinking/tool/usage/done/error — no request metadata |
+| `AgentEvent` stream | Core | Turn/message/tool/retry — content redacted when `redactor` configured |
+| `neuralwatt:telemetry` | `provider-neuralwatt` | Numeric energy/cost only — **safe metadata** |
+| Middleware `provider_request` | Core | Full `ProviderRequest` — host must redact |
+| OpenTelemetry adapter | **Not shipped** | Planned optional package |
+
+## Chosen generic APIs (frozen for Task 1+)
+
+Primitives are **stdlib-only**, exposed as peer-importable subpaths on `@arnilo/prism`. Provider-specific request fields, event mapping, cache/reasoning knobs, and NeuralWatt comment telemetry stay local.
+
+### Decision table
+
+| Concern | Option A | Option B | **Chosen** | Rationale |
+| --- | --- | --- | --- | --- |
+| SSE parsing | External npm SSE library | Incremental stdlib parser | **B** | Small surface; avoids dependency |
+| Transport packaging | New `@arnilo/prism-provider-transport` package | Core subpath `@arnilo/prism/providers/transport` | **Core subpath** | Matches existing `providers/openai-compatible` pattern; one peer dep |
+| OpenAI serializers | Duplicate per package | `@arnilo/prism/providers/openai` | **Core subpath** | Proven by ≥6 providers |
+| NeuralWatt comments | Force into generic reader | `readSseEvents` yields optional `comments[]`; NeuralWatt maps locally | **Optional comments** | Keeps generic reader used by ≥2 providers without NeuralWatt-only branches in core |
+| Structured output wire shape | Vendor fields in core contracts | Provider-neutral option + local mapper | **Neutral option** | Avoids leaking `response_format` into contracts |
+| Observability | Hard OTel in core | Extended events + optional adapter package | **Events + optional pkg** | No mandatory heavyweight dep |
+
+### `@arnilo/prism/providers/transport` — **shipped**
+
+Import:
+
+```ts
+import {
+  readSseEvents,
+  readSseData,
+  readBoundedResponseText,
+  parseJsonObjectArguments,
+  ProviderTransportError,
+  DEFAULT_MAX_EVENT_BYTES,
+  DEFAULT_MAX_BUFFER_BYTES,
+  DEFAULT_MAX_RESPONSE_BODY_BYTES,
+} from "@arnilo/prism/providers/transport";
+```
+
+```ts
+export interface BoundedStreamLimits {
+  /** Max bytes per completed SSE event (all data: lines + field names). Default: 262_144 (256 KiB). */
+  readonly maxEventBytes?: number;
+  /** Max bytes retained for an incomplete event/buffer. Default: 524_288 (512 KiB). */
+  readonly maxBufferBytes?: number;
+  /** Max bytes read from non-streaming HTTP bodies (errors). Default: 65_536 (64 KiB). */
+  readonly maxResponseBodyBytes?: number;
+}
+
+export interface SseEvent {
+  readonly id?: string;
+  readonly event?: string;
+  /** Joined multiline `data:` payload for one SSE event. */
+  readonly data: string;
+  /** Raw `:` comment lines (without leading colon), when present. */
+  readonly comments?: readonly string[];
+}
+
+export class ProviderTransportError extends Error {
+  readonly code: "sse_buffer_overflow" | "sse_event_overflow" | "response_body_overflow" | "aborted";
+  readonly limitBytes?: number;
+}
+
+/** Incremental O(bytes) SSE parser. Supports LF/CRLF, comment lines, multiline data:, final partial flush, abort. */
+export async function* readSseEvents(
+  body: ReadableStream<Uint8Array>,
+  options?: BoundedStreamLimits & { signal?: AbortSignal },
+): AsyncGenerator<SseEvent>;
+
+/** Read response body text with a hard byte ceiling; releases the reader. */
+export async function readBoundedResponseText(
+  response: Response,
+  options?: BoundedStreamLimits & { secrets?: readonly (string | undefined)[] },
+): Promise<string>;
+
+/** Parse tool-call arguments JSON to a plain object; throws typed error on invalid/non-object input. */
+export function parseJsonObjectArguments(
+  text: string,
+  options?: { toolName?: string; maxBytes?: number },
+): JsonObject;
+```
+
+**Performance:** Single pass over chunks; retained memory is `O(min(buffer, maxBufferBytes))`, not `O(stream)`. No full-stream accumulation.
+
+### `@arnilo/prism/providers/openai` — **shipped**
+
+Import:
+
+```ts
+import {
+  serializeOpenAITool,
+  serializeOpenAIChatMessage,
+  mapOpenAIChatUsage,
+  assertOpenAIChatMessage,
+} from "@arnilo/prism/providers/openai";
+```
+
+```ts
+/** OpenAI Chat Completions tool schema. */
+export function serializeOpenAITool(tool: ToolDefinition): JsonObject;
+
+/** OpenAI Chat Completions message wire shape with capability guards. */
+export function serializeOpenAIChatMessage(
+  message: Message,
+  capabilities?: ModelCapabilities,
+): JsonObject;
+
+/** Map OpenAI `usage` object to Prism `Usage` (incl. cache fields). */
+export function mapOpenAIChatUsage(usage: unknown): Usage | undefined;
+
+/** Fail fast with indexed, content-free diagnostics (no payload stringify). */
+export function assertOpenAIChatMessage(message: unknown, path: string): asserts message is Message;
+```
+
+`src/providers/openai-compatible.ts` becomes a thin adapter over these helpers in Task 2.
+
+### Structured output capability (Task 4 — **shipped**)
+
+```ts
+// Added to src/contracts.ts — provider-neutral host option
+export interface StructuredOutputOptions {
+  readonly name: string;
+  readonly schema: JsonObject;
+  readonly strict?: boolean;
+}
+
+// ProviderRequestOptions
+readonly structuredOutput?: StructuredOutputOptions;
+
+// ModelCapabilities
+readonly structuredOutput?: boolean | "json_schema";
+```
+
+| Provider family | Native support (planned) | Wire mapping owner |
+| --- | --- | --- |
+| OpenAI / OpenAI-compatible / OpenRouter / OpenCode Go (OpenAI chat) | `response_format: { type: "json_schema", json_schema: { name, schema, strict } }` | Respective provider package |
+| OpenAI Responses API | `text.format` / JSON schema fields per API version | `provider-openai` |
+| Anthropic (OpenCode Go) | Documented fallback only unless API gains parity | `provider-opencode-go` |
+| Z.AI / Kimi / NeuralWatt | Capability-gated per package docs | Local mapper or clear unsupported error |
+| Unsupported | — | Host selects artifact loop explicitly |
+
+**Fallback rule:** Unsupported providers **do not** silently repair. Runtime returns a clear error unless the host configured the artifact loop ([Structured output](structured-output.md)).
+
+### Observability capability (Task 5 contract)
+
+Core extends existing seams — no second event bus.
+
+```ts
+export interface ProviderTurnMetadata {
+  readonly providerId: string;
+  readonly model: ModelConfig;
+  readonly requestId?: string;
+  readonly latencyMs?: number;
+  readonly attempt?: number;
+  readonly httpStatus?: number;
+  readonly rateLimitRemaining?: number;
+  readonly rateLimitResetMs?: number;
+}
+
+// New AgentEvent variants (metadata only — no prompts, tool args, or credentials):
+// - provider_turn_started { sessionId, runId, turn, metadata }
+// - provider_turn_finished { sessionId, runId, turn, metadata, usage?, error? }
+```
+
+Optional package `@arnilo/prism-observability-opentelemetry` subscribes via middleware + agent events. **Default:** content redacted/absent; high-cardinality IDs are span attributes, not metric labels. NeuralWatt `neuralwatt:telemetry` events remain package-local; the adapter may forward numeric energy/cost fields.
+
+## Migration conformance fixtures
+
+Every migrated provider must pass this shared matrix (implemented in Task 1 tests, run per provider in Task 2/6):
+
+| # | Fixture | Assert |
+| ---: | --- | --- |
+| 1 | UTF-8 chunk split mid-codepoint | Event data reconstructs valid Unicode |
+| 2 | CRLF and LF event delimiters | Same logical events |
+| 3 | Multiline `data:` field | Joined payload parses as one JSON value |
+| 4 | SSE comment lines (`:`) | Ignored by default reader; NeuralWatt reader surfaces comments |
+| 5 | Partial final buffer without trailing blank line | Flushed on stream end |
+| 6 | `AbortSignal` during read | `ProviderTransportError` code `aborted`; reader released |
+| 7 | Event exceeds `maxEventBytes` | `sse_event_overflow`; stream terminated |
+| 8 | Incomplete buffer exceeds `maxBufferBytes` | `sse_buffer_overflow` |
+| 9 | Error body exceeds `maxResponseBodyBytes` | `response_body_overflow`; no unbounded growth |
+| 10 | Malformed tool arguments | Typed parse error with `toolName` context |
+| 11 | Malformed message at index *n* | `assertOpenAIChatMessage` path in error; no content echo |
+| 12 | Secret in error body | Redacted when `secrets` passed to `readBoundedResponseText` |
+| 13 | Caller `authorization` header | Provider-owned header wins (existing conformance) |
+| 14 | Stream order / tool-call reconstruction | Unchanged provider conformance suite |
+
+## Security and performance notes
+
+### Trust boundaries
+
+| Boundary | Rule |
+| --- | --- |
+| Credentials | Resolved in provider package; provider-owned `authorization` overrides caller headers |
+| Redaction order | Secrets redacted **before** error strings, ledger rows, events, and observability metadata |
+| Transport errors | Include limit kind and byte ceiling — never full bodies, tokens, or prompts |
+| Structured-output schemas | JSON-safe objects only; reject `__proto__` / `prototype` / `constructor` keys; size cap enforced |
+| OAuth (Task 3) | Device/authorization/access/refresh tokens redacted from all token-endpoint failures |
+| Observability | Metadata-only by default; prompt/content/tool payloads opt-in and redacted |
+
+### Performance defaults
+
+| Limit | Default | Override |
+| --- | ---: | --- |
+| `maxEventBytes` | 256 KiB | Per-request `BoundedStreamLimits` |
+| `maxBufferBytes` | 512 KiB | Per-request |
+| `maxResponseBodyBytes` | 64 KiB | Per-request |
+| Observability overhead (Task 5 target) | <5% vs disabled | Excludes exporter I/O |
+
+## Related APIs
+
+- [Provider layer](provider-layer.md): registry, mock provider, event helpers
+- [Provider conformance](provider-conformance.md): stream order, abort, header ownership
+- [OpenAI-compatible provider](providers/openai-compatible.md): reference adapter subpath
+- [Structured output](structured-output.md): artifact loop fallback
+- [Provider request policies](provider-request-policies.md): cache and request hooks
+- [Review coverage (2026-07-14)](review-coverage-2026-07-14.md): finding → plan traceability
+
+## Task ownership map
+
+| Finding / capability | Plan 054 task | Primitive / doc |
+| --- | --- | --- |
+| R-008 Unbounded SSE/error bodies | 1, 2 | `readSseEvents`, `readBoundedResponseText` |
+| R-009 OAuth device polling | 3 | `packages/provider-openai/src/oauth.ts` |
+| R-010 Duplicated helpers | 1, 2 | This page + subpaths |
+| C-002 Native structured output | 4 | `StructuredOutputOptions` |
+| C-004 Shared resilient transport | 1, 2 | `providers/transport` |
+| C-008 Observability hooks | 5 | `ProviderTurnMetadata`, optional OTel package |

package/docs/providers/kimi.md

@@ -111,6 +111,7 @@ await kernel.load([
 
 ## Security and performance notes
 
+- SSE streams and HTTP error bodies use bounded `@arnilo/prism/providers/transport` helpers (`readSseData`, `readBoundedResponseText`).
 - No network calls during import, setup, build, or default tests.
 - No automatic environment, file, keychain, or shell credential lookup.
 - Kimi credentials are resolved per request from caller-supplied values or resolvers

package/docs/providers/neuralwatt.md

@@ -356,6 +356,7 @@ const decision = classifyNeuralWattError({ status: 429, headers: { "retry-after"
 
 ## Security and performance notes
 
+- SSE streams, HTTP error bodies, and quota/model-discovery failures use bounded `@arnilo/prism/providers/transport` helpers (`readSseEvents`, `readBoundedResponseText`). NeuralWatt `: energy` / `: cost` comment frames are surfaced via `readSseEvents` `comments` and mapped locally.
 - No network calls during import, setup, build, default tests, or generation beyond
   the explicit Chat Completions request. `listNeuralWattModels()` is opt-in and
   makes one `GET /v1/models` call per invocation; `getNeuralWattQuota()` is opt-in

package/docs/providers/openai-compatible.md

@@ -116,8 +116,9 @@ const provider = createOpenAICompatibleProvider({
 - Resolved API keys are used for the HTTP `Authorization` header and passed to error redaction; they are not stored in registries or events.
 - Redaction only removes known values supplied to the helper. Avoid logging raw provider requests/responses.
 - `fetch` receives the request `AbortSignal`.
+- SSE and HTTP error bodies are read through bounded `@arnilo/prism/providers/transport` helpers (`readSseData`, `readBoundedResponseText`) with configurable byte ceilings.
 - Tests should use injected `fetch` and never make real network calls.
-- Tool-call arguments are accumulated as streamed text, parsed as JSON only when the final tool call is emitted, and default to `{}` for empty argument text.
+- Tool-call arguments are accumulated as streamed text, parsed with `parseJsonObjectArguments` when the final tool call is emitted; empty argument text yields `{}`, malformed JSON yields an `error` event.
 
 ## Related APIs
 

package/docs/providers/openai.md

@@ -108,6 +108,10 @@ const challenge = computeS256Challenge(verifier);
   run/model through `RunOptions` and `ModelConfig.compat`.
 - OAuth browser/device-code flows run only when the caller explicitly invokes the
   OAuth provider.
+- Device-code login polls the token endpoint with server-directed `interval` and
+  `expires_in`, honors RFC 8628 `authorization_pending` / `slow_down`, and stops
+  on terminal errors or expiry. Pass `signal` on `OAuthLoginCallbacks` to abort
+  polling promptly.
 
 ### Cache behavior
 
@@ -132,11 +136,14 @@ const challenge = computeS256Challenge(verifier);
 
 ## Security and performance notes
 
+- SSE streams and HTTP error bodies use bounded `@arnilo/prism/providers/transport` helpers (`readSseData`, `readBoundedResponseText`).
 - No network calls during import, setup, build, or default tests.
 - No automatic environment, file, keychain, or shell credential lookup; Prism never
   reads `process.env` on its own.
 - API keys/access tokens are resolved per request from caller-supplied values or
-  resolvers; OAuth errors redact known token values (`[REDACTED]`).
+  resolvers; OAuth errors redact known token values (`[REDACTED]`) including
+  authorization codes, PKCE verifiers, device codes, user codes, and
+  access/refresh tokens echoed in token-endpoint failures.
 - The PKCE verifier is exchanged at the token endpoint, never sent on the authorize
   URL.
 - Live tests stay opt-in behind `PRISM_LIVE_PROVIDER_TESTS=1` plus fake-safe

package/docs/providers/opencode-go.md

@@ -110,6 +110,7 @@ await kernel.load([
 
 ## Security and performance notes
 
+- SSE streams and HTTP error bodies use bounded `@arnilo/prism/providers/transport` helpers (`readSseData`, `readBoundedResponseText`).
 - No network calls during import, setup, build, or default tests.
 - No automatic environment, file, keychain, or shell credential lookup.
 - API keys are resolved per request from caller-supplied values or resolvers and

package/docs/providers/openrouter.md

@@ -110,6 +110,7 @@ await kernel.load([
 
 ## Security and performance notes
 
+- SSE streams and HTTP error bodies use bounded `@arnilo/prism/providers/transport` helpers (`readSseData`, `readBoundedResponseText`).
 - No catalog fetch during setup; no automatic environment, file, keychain, or shell
   credential lookup.
 - API keys are resolved per request from caller-supplied values or resolvers and

package/docs/providers/zai.md

@@ -104,6 +104,7 @@ await kernel.load([
 
 ## Security and performance notes
 
+- SSE streams and HTTP error bodies use bounded `@arnilo/prism/providers/transport` helpers (`readSseData`, `readBoundedResponseText`).
 - No network calls during import, setup, build, or default tests.
 - No automatic environment, file, keychain, or shell credential lookup.
 - API keys are resolved per request from caller-supplied values or resolvers and

package/docs/public-contracts.md

@@ -15,7 +15,7 @@ Current contract groups:
 - Extensions/middleware: `ExtensionLifecycleEventName`, `ExtensionEvent`, `Extension`, `ExtensionAPI`, `MiddlewareHookName`, `Middleware`, `MiddlewareNext`, `MiddlewareRegistry`
 - Configuration/manifests: `ConfigProvider`, `ConfigLayer`, `ConfigLoadContext`, `PrismManifest`, `ManifestContributionDeclaration`, `ManifestResourceDeclaration`, `ManifestContributionKind`
 - Stores/resources/settings/credentials/compaction/retry/cache helpers: `SessionEntry`, `SessionStore`, `StoreFactory`, `Resource`, `ResourceLoader`, `ResourceLoadContext`, `SettingsProvider`, `CredentialRequest`, `Credential`, `CredentialResolver`, `CompactionStrategy`, `CompactionContext`, `CompactionResult`, `CompactionOptions`, `CompactionMiddlewarePayload`, `CompactionEntryData`, `DefaultCompactionStrategyOptions`, `RetryPolicy`, `RetryContext`, `RetryDecision`, `RetryOptions`, `RetryMiddlewarePayload`, `DefaultRetryPolicyOptions`, `CacheUsageReport`, `sanitizeCacheKey`, `mapCacheRetention`, `applyCacheControl`, `cacheHitRate`, `cacheSavings`, `cacheUsageReport`
-- Production persistence (adapter-facing): `ProductionPersistenceStore`, `PersistencePage`, `PersistenceQuery`, `OwnershipScope`, `SessionRecord`, `SessionQuery`, `BranchRecord`, `BranchQuery`, `SessionEntryQuery`, `RunRecord`, `RunQuery`, `AgentEventRecord`, `AgentEventQuery`, `ToolCallRecord`, `ToolCallQuery`, `UsageRecord`, `UsageQuery`, `AgentDefinitionRecord`, `AgentDefinitionQuery`, `RetentionPolicy`, `RetentionPolicyQuery`, `MigrationRecord`, `MigrationQuery`
+- Production persistence (adapter-facing): `ProductionPersistenceStore`, `CheckpointStore`, `CheckpointKey`, `CheckpointSaveInput`, `CheckpointRecord`, `CheckpointQuery`, `LeaseStore`, `LeaseKey`, `LeaseAcquireInput`, `LeaseClaimInput`, `LeaseRecord`, `PersistencePage`, `PersistenceQuery`, `OwnershipScope`, `SessionRecord`, `SessionQuery`, `BranchRecord`, `BranchQuery`, `SessionEntryQuery`, `RunRecord`, `RunQuery`, `AgentEventRecord`, `AgentEventQuery`, `ToolCallRecord`, `ToolCallQuery`, `UsageRecord`, `UsageQuery`, `AgentDefinitionRecord`, `AgentDefinitionQuery`, `RetentionPolicy`, `RetentionPolicyQuery`, `MigrationRecord`, `MigrationQuery`
 
 ## When to use it
 
@@ -49,6 +49,8 @@ import type {
   BranchRecord,
   CommandDefinition,
   CacheUsageReport,
+  CheckpointStore,
+  LeaseStore,
   CompactionStrategy,
   ConfigLayer,
   ConfigProvider,
@@ -140,7 +142,10 @@ Important request shapes:
 | `SystemPromptContribution` | Explicit caller-selected prompt layer with source, mode, text, and metadata. |
 | `ConfigLayer` | Named JSON config layer consumed by `mergeConfigLayers()`. |
 | `PrismManifest` | Data-only package manifest with config defaults, contribution declarations, and resource declarations. |
-| `ProductionPersistenceStore` | Adapter-facing interface for durable, paginated, multi-tenant storage of sessions, branches, entries, runs, events, tool calls, usage, agent definitions, retention policies, and migrations. No SQL/ORM/host file storage/network dependency. |
+| `ProductionPersistenceStore` | Adapter-facing interface for durable, paginated, multi-tenant storage plus optional `checkpoints?: CheckpointStore` and `leases?: LeaseStore`. No SQL/ORM/host file storage/network dependency. |
+| `CheckpointStore` | Generic versioned checkpoint capability: save/load/bounded-list/delete by namespace and key, with ownership, exact-version CAS, and lease fencing. `createMemoryCheckpointStore()` is the reference implementation. |
+| `LeaseStore` | Atomic acquire/renew/release/get by namespace and key, with opaque claim tokens, expiry, ownership scope, and monotonically increasing takeover fences. `createMemoryLeaseStore()` is the reference implementation. |
+| `EventMultiplexer<T>` | Generic bounded fan-in from async sources. `createEventMultiplexer()` owns queue limits, overflow policy, abort, source teardown, and close behavior. |
 | `PersistencePage<T>` | Cursor-paginated result page: `items`, optional `nextCursor`, optional `total`. |
 | `PersistenceQuery` | Common pagination controls: `cursor?`, `limit?`, `order?: "asc" \| "desc"`. |
 | `OwnershipScope` | Multi-tenant scope: `tenantId?`, `accountId?`, `userId?`. Included in records and queries. |
@@ -446,5 +451,7 @@ void credentials;
 - [Provider conformance](provider-conformance.md): testing subpath for network-free provider adapter checks.
 - [Credentials and redaction](credentials-and-redaction.md): helpers for resolving host-owned credentials, explicit resolver order, OAuth refresh, env-object lookup, and redacting known secret values.
 - [OpenAI-compatible provider](providers/openai-compatible.md): optional provider adapter implementing `AIProvider`.
+- `@arnilo/prism/providers/transport`: bounded SSE/event parsing, bounded HTTP error-body reads, and JSON-object tool-argument parsing for provider packages.
+- `@arnilo/prism/providers/openai`: OpenAI Chat Completions message/tool serialization, usage mapping, and indexed message validation helpers.
 
 Phase 10 public helpers include `createStaticSettingsProvider`, `createChainedSettingsProvider`, `createMemoryCredentialStore`, `createChainedCredentialResolver`, `createStaticTrustPolicy`, `assertTrusted`, `createStaticPermissionPolicy`, `assertPermission`, and `createSecretRedactor`. Phase 11 auth/request/prompt helpers include `createExplicitCredentialResolver`, `createEnvCredentialResolver`, `refreshOAuthCredential`, `createProviderRequestPolicyChain`, `createSessionCachePolicy`, `mergeProviderRequestOptions`, `composeSystemPrompt`, and `mergeSystemPromptConfig`; they do not read env vars, persist OAuth tokens, create cache stores, discover prompt files, or load packages unless the host supplies that behavior. `@arnilo/prism/testing/provider-conformance` exports network-free provider assertion helpers. Node subpaths `@arnilo/prism/node/settings` and `@arnilo/prism/node/trust` are explicit filesystem/path helpers.