@arnilo/prism 0.0.3 → 0.0.4

package/docs/performance.md

@@ -7,6 +7,7 @@ This page states Prism runtime limits that keep slow consumers and long sessions
 Current surfaces:
 
 - `SubscribeOptions` for bounded live `AgentEvent` subscriber queues.
+- Bounded provider transport primitives (`readSseEvents`, `readBoundedResponseText`) used by every first-party provider package — see [Provider primitives](provider-primitives.md).
 - `SessionStore.readBranchPath(query)` for branch reads that avoid full-session scans.
 - `ProductionPersistenceStore` cursor queries for entries, events, runs, tool calls, and usage.
 - JSONL and memory stores documented as development/local adapters, not production multi-writer stores.
@@ -112,10 +113,48 @@ const store = {
 
 - Overflow events contain only counts and policy, never message text, tool arguments, prompts, provider payloads, or credentials.
 - Runtime event payloads can be large (`Message`, content deltas, tool results, summaries, artifact metadata). Size queues by events and keep payload size in mind.
+- `toolConcurrency` on the single-shot loop bounds in-flight tool dispatches per provider turn to `min(toolConcurrency, calls.length)`. Independent slow tools can overlap; transcript appends remain ordered. Default `1` preserves sequential behavior.
+- `read` image bounds: default `maxImageBytes` is 10 MB (`DEFAULT_MAX_IMAGE_BYTES`). Oversize images are rejected by `stat` before read when possible; hosts may supply `transformImage` for resize/re-encode without adding image-processing deps to the base package.
 - Live subscriber queues are bounded by default. Durable replay belongs to host storage.
 - `SessionStore.list(sessionId)` is a full-session read. It is fine for memory/JSONL development stores, but production adapters should use `readBranchPath` for provider context and branch views.
 - The JSONL store rereads/parses the file for validation/list/get and serializes appends only within one process. It has no cross-process lock, pagination, migrations, tenant isolation, or retention.
 - Recommended database indexes: session id, run id, parent id, branch leaf id, timestamps, tenant/account/user, event type, entry kind, `(run_id, sequence)` for event timelines, and `(run_id, recorded_at, id)` for usage. Allocate event `sequence` per run for stable timeline pagination.
+- Provider SSE parsing defaults: 256 KiB per completed event, 512 KiB incomplete buffer, 64 KiB error response bodies. Override per call via `BoundedStreamLimits` on `@arnilo/prism/providers/transport`.
+
+### Provider-phase benchmark snapshot (2026-07-14)
+
+Node v24.18.0, Linux x86_64, AMD Ryzen 9 PRO 7940HS; local synthetic streams, no network or exporter I/O:
+
+| Case | Result |
+| --- | --- |
+| `readSseEvents`, 16 MiB total as 4 KiB events | 380 MiB/s, +1.7 MiB end-of-run heap delta |
+| 100 provider deltas with 1 ms transport delay, no telemetry | 107.97 ms median |
+| Same run, disabled adapter attached | 107.25 ms median (-0.67%, noise) |
+| Same run, enabled no-op exporter | 107.22 ms median (-0.70%, noise) |
+
+Nine measured runs per telemetry mode after warm-up; table reports median. A zero-I/O burst of 5,000 deltas measured 1.06 ms without telemetry and 2.09 ms with the adapter: about 1 ms absolute adapter cost, but a large percentage against an unrealistically tiny baseline. No span is created for message deltas. Add subscriber-side event filtering only if measured high-frequency in-memory streams make that ceiling material.
+
+Configured overflow behavior is enforced by `src/__tests__/provider-transport.test.ts` for event, incomplete-buffer, response-body, argument, and abort limits.
+
+### 0.0.4 release audit snapshot (2026-07-14)
+
+Node v24.18.0 on the same Linux x86_64 / Ryzen 9 PRO 7940HS host. Results are medians of 7-9 warm runs unless the row describes file/database appends. Synthetic operations use local memory/files only. These numbers are release ceilings and comparison points, not cross-machine guarantees.
+
+| Surface | Workload | Result | 0.0.4 release threshold |
+| --- | --- | --- | --- |
+| Run ledger | One mock run, 500 text deltas, 510 total records | 1.19 ms with in-memory ledger vs 0.17 ms without; +1.02 ms absolute; event append max concurrency 1 | < 10 ms with zero-I/O adapter; event appends remain serialized |
+| JSONL store | 500 sequential label appends, including fail-closed reread/validation | 141.10 ms; 3,544 appends/s | < 500 ms; development/single-process only |
+| JSON Schema compile cache | 5,000 validations through one warm adapter | 4.97 ms; 0.99 µs/validation | < 25 µs/validation |
+| JSON Schema cold compile | 100 new adapters + first validation | 249.89 ms; 2.50 ms/compile | Warm cache must remain at least 20x faster than cold compile |
+| Parallel tools | Six independent 20 ms calls | concurrency 1: 121.12 ms; concurrency 2: 60.92 ms; 1.99x speedup | concurrency 2 < 75% of sequential; configured worker cap remains enforced |
+| SQLite session store | 1,000 sequential transactional label appends | 31.84 ms; 31,405 appends/s | < 250 ms on local SSD/tmp storage |
+| Secret redaction | 10,000 shallow objects containing one known secret | 4.79 ms; 2.09 million objects/s | < 25 ms |
+| Credential KDF | Default scrypt + AES-256-GCM encryption | 48.09 ms median | 20-250 ms; security floor stays `N >= 16,384` |
+| Workflow runner | Existing bounded 1,000-node DAG fixture | 27.68 ms in aggregate release gate | < 1 s; no rescan failure |
+
+Provider SSE remained at the frozen 380 MiB/s / +1.7 MiB heap snapshot. Media and MCP retain 10 MB defaults and finite timeout/total-byte guards; their malicious-input and oversize fixtures pass. PostgreSQL latency remains environment-dependent and is gated by transactional conformance in CI rather than a hardware-specific wall-clock assertion.
+
+The ledger percentage overhead is intentionally not a threshold: its no-ledger baseline is below 1 ms, making the percentage unstable while absolute added latency remains about 1 ms. JSONL's append path is intentionally O(n²) across repeated appends because it rereads for corruption/conflict checks; move production or high-volume workloads to SQLite/PostgreSQL rather than weakening validation.
 
 ## Related APIs
 
@@ -124,4 +163,4 @@ const store = {
 - [Session stores](session-stores.md): `SessionStore.readBranchPath` and dev-vs-production branch reads.
 - [Database persistence](database-persistence.md): cursor queries, reference schema, indexes, and event sequence guidance.
 - [Runs and usage ledger](runs-and-usage.md): durable event, tool-call, and usage persistence.
-- [Node JSONL session store](node-jsonl-session-store.md): development-only JSONL limits.
+- [Provider primitives](provider-primitives.md): bounded SSE/error-body limits for first-party providers.

package/docs/persistence-credentials-multimodality-primitives.md

@@ -0,0 +1,303 @@
+# Persistence, credentials, and multimodality primitives
+
+## What it does
+
+This page freezes the Plan 056 Task 0 inventory for production persistence, credential storage, and multimodal content. It maps existing `@arnilo/prism` contracts and conformance helpers, documents JSONL/security boundaries, records capability gaps **C-005**, **C-010**, and **C-011**, and pins package dependency choices for Tasks 1–7.
+
+Implementation is **shipped and phase-verified** (Tasks 0–7). Optional packages cover SQLite/PostgreSQL persistence and Node credential storage; core remains dependency-free.
+
+## When to use it
+
+- **Adapter authors** implementing SQLite/PostgreSQL `SessionStore` + `RunLedger` should start here, then follow [Database persistence](database-persistence.md) and [Session store conformance](session-store-conformance.md).
+- **Host apps** wiring CLI/desktop credential persistence should use the credential seams and package matrix here before choosing `@arnilo/prism-credentials-node` backends.
+- **Provider and core authors** extending multimodal input should use the content/resource/capability designs here instead of embedding provider upload IDs in core contracts.
+- **Security reviewers** use the threat model and conformance matrix on this page as the acceptance baseline for Plan 056 Tasks 1–7.
+
+## Inventory (2026-07-14 baseline)
+
+Static review of `src/contracts.ts`, `src/session-stores.ts`, `src/credentials.ts`, `src/input.ts`, `src/agents.ts`, `src/redaction.ts`, `src/node/session-store-jsonl.ts`, `src/testing/session-store-conformance.ts`, `examples/external-app-db-backed.ts`, and docs under `docs/session-stores*.md`, `docs/database-persistence.md`, `docs/runs-and-usage.md`, `docs/credentials-and-redaction.md`, `docs/settings-auth-trust-security.md`, `docs/input-and-prompt-assembly.md`, `docs/resource-loading.md`, `docs/model-registry.md`, `docs/provider-conformance.md`.
+
+### Session store and branch invariants (shipped)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `SessionStore` | `src/contracts.ts` | `append`, `list`, optional `get`, optional `readBranchPath` |
+| `SessionAppendOptions` | `src/contracts.ts` | `expectedParentId` existence validation; opaque `idempotencyKey` for same-position retry dedup |
+| `SessionAppendConflictError` | `src/contracts.ts` | Stable `code: "session_append_conflict"`; `isSessionAppendConflict()` |
+| Branch helpers | `src/session-stores.ts` | `getSessionBranchEntries`, `listSessionBranches`, `rebuildSessionContext`; reader overload follows `nextCursor` (max 64 pages) |
+| In-memory store | `src/session-stores.ts` | `createMemorySessionStore()` — O(1) dup/idempotency/parent maps; not durable |
+| JSONL store | `src/node/session-store-jsonl.ts` | Single-process dev adapter; per-line quarantine; in-memory idempotency only |
+| Conformance | `src/testing/session-store-conformance.ts` | `assertSessionStoreConforms(store, { exerciseReadBranchPath? })` |
+| Reference adapter | `examples/external-app-db-backed.ts` | In-memory tables implementing `SessionStore` + `RunLedger` + `ProductionPersistenceStore` |
+
+**Frozen branch semantics:** `expectedParentId` is parent **existence** validation, not tip-CAS. Two children of the same parent are allowed (fork/branch). Production adapters may add stricter tip compare-and-swap and populate `currentLeafId` in conflicts.
+
+**Frozen idempotency semantics:** Dedup key is `(session_id, expected_parent_id, idempotency_key)`. A run-level key may appear on multiple linear appends as the leaf advances; only exact same-position retries collapse.
+
+### Run ledger (shipped)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `RunLedger` | `src/contracts.ts` | `appendRun`, `appendEvent`, `appendToolCall`, `appendUsage` |
+| Runtime wiring | `src/agents.ts` | Active ledger from `RunOptions.runLedger ?? AgentConfig.runLedger`; ownership/idempotency from run options |
+| Serialization | `src/agents.ts` | `ledgerChain` serializes event ledger appends (R-004 backpressure fix) |
+| Redaction | `src/redaction.ts` | `redactRunLedgerRecord()` before every ledger write when `SecretRedactor` active |
+| Conformance helper | `src/testing/session-store-conformance.ts` | `assertSessionStoreConforms`, `runSessionStoreConformance` |
+| Run ledger conformance | `src/testing/run-ledger-conformance.ts` | `assertRunLedgerConforms`, `runRunLedgerConformance` |
+| Shared schema model | `src/testing/persistence-schema.ts` | `createPersistenceSchemaModel`, `createPersistenceMigrationContract`, pagination/tenant fixtures |
+
+### Production persistence contracts (shipped)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `ProductionPersistenceStore` | `src/contracts.ts` | Cursor `query*` methods for sessions, branches, entries, runs, events, tool calls, usage, agent definitions, retention, migrations; optional `readBranchPath` |
+| `OwnershipScope` | `src/contracts.ts` | `tenantId`, `accountId`, `userId` on records and queries — enforcement host-owned |
+| Reference schema | `docs/database-persistence.md` | Relational table/index reference, conditional append transaction pattern, NoSQL mapping notes |
+| Type compile test | `src/__tests__/persistence-contracts.types.test.ts` | Host adapter shapes compile without DB deps |
+
+**Core boundary:** No ORM, SQL driver, connection pool, migration runner, or filesystem persistence in `@arnilo/prism`.
+
+### Credential and OAuth seams (shipped — storage gap)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `CredentialResolver` / `CredentialRequest` | `src/contracts.ts` | Host-implemented resolve at request edge |
+| `resolveCredentialValue` | `src/credentials.ts` | String, callback, or resolver |
+| `createExplicitCredentialResolver` | `src/credentials.ts` | Named source order (runtime → stored → env → fallback) |
+| `createEnvCredentialResolver` | `src/credentials.ts` | Caller-supplied env object only; never reads `process.env` |
+| `createMemoryCredentialStore` | `src/credentials.ts` | In-memory resolver + `set`/`delete`; not durable |
+| `createChainedCredentialResolver` | `src/credentials.ts` | First match wins |
+| `refreshOAuthCredential` | `src/credentials.ts` | Calls `OAuthProvider.refresh`; optional `OAuthCredentialStore.set` |
+| `OAuthCredentialStore` | `src/contracts.ts` | `set(provider, credentials)` only — no `get`/`delete` in core contract |
+| `OAuthProvider` / `OAuthCredentials` | `src/contracts.ts` | `login`, optional `refresh`, optional `getCredential` |
+| Device-code OAuth | `packages/provider-openai` | Bounded polling; abort via `OAuthLoginCallbacks.signal` |
+| Redaction | `src/redaction.ts` | Exact known-secret replacement; not secret detection |
+
+**Gaps (C-011):** No encrypted file store, no system keychain adapter, no versioned credential envelope, no `OAuthCredentialStore` `get`/`delete`/`list` in core (Task 4 package may extend store interface locally while integrating `refreshOAuthCredential`).
+
+### Content, resources, and model capabilities (shipped — modality gap)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `ContentBlock` union | `src/contracts.ts` | `text`, `image`, `audio`, `file`, `document`, `thinking`, `tool_call_delta`, `tool_call`, `tool_result` |
+| `AudioContent` / `FileContent` / `DocumentContent` | `src/content.ts` | MIME, name, `data`/`url`/`resourceUri`, optional transcript/metadata |
+| `ImageContent` | `src/contracts.ts` | `mimeType?`, `data?`, `url?`, `resourceUri?`, `name?` |
+| `InputAttachment` | `src/input.ts` | `text`, inline `content` blocks, or `uri` + `ResourceLoader` → text user message |
+| `ResourceLoader` / `Resource` | `src/contracts.ts` | Host-owned `load(uri)`; optional `list`; `data`/`text`/`mediaType` |
+| Resource helpers | `src/resources.ts` | `loadTextResource`, `loadJsonResource`, `loadManifestResource`, `loadBinaryResource` — decode/bounds only |
+| Media helpers | `src/content.ts` | `resolveMediaContentBlock`, `assertSsrfAllowedUrl`, `assertMediaBlocksWithinBounds`, `UnsupportedModalityError` |
+| `ModelCapabilities.input` | `src/contracts.ts` | Known tags: `text`, `image`, `audio`, `file`, `document`; `MODEL_INPUT_CAPABILITIES` export |
+| Provider image mapping | first-party packages | OpenAI Responses, OpenRouter, OpenCode Go (Anthropic route), Kimi, NeuralWatt map `image` when capability allows |
+| Provider audio/file/document mapping | first-party packages | OpenAI Responses maps `audio`/`file`/`document`; Anthropic routes map PDF `document`/`file`; others reject undeclared media |
+| Provider conformance | `src/testing/provider-conformance.ts` | `assertSerializedRequestCoversContent` with per-provider `unsupported` list |
+
+### JSONL and dev-store security boundaries (shipped)
+
+| Boundary | Rule |
+| --- | --- |
+| Cross-process writes | JSONL has no lock; two processes on one file can race |
+| Durable idempotency | JSONL idempotency is in-process only |
+| Corrupt lines | Quarantined per line; do not poison whole file (R-005) |
+| Schema/kind | Unknown kinds and future `schemaVersion` fail closed |
+| Secrets in file | Host responsibility; runtime redacts before `append` when redactor configured |
+| Production use | JSONL documented as dev-only; database adapters required for multi-writer |
+
+## Package dependency and support matrix (pinned Task 0)
+
+Prism `engines.node` is `>=20`. Optional packages stay package-local; core adds no new runtime dependencies.
+
+| Package (planned) | Driver / backend | Pinned version | Node support | Rationale |
+| --- | --- | --- | --- | --- |
+| `@arnilo/prism-session-store-sqlite` | `better-sqlite3` | `^12.11.1` | 20.x–26.x per upstream engines | Synchronous API, WAL/busy_timeout, mature Node 20 baseline; `node:sqlite` rejected — requires Node ≥22.5 and is still experimental vs declared `>=20` baseline |
+| `@arnilo/prism-session-store-postgres` | `pg` | `^8.22.0` | ≥16 (satisfies 20+) | Standard pool + parameterized queries; TLS/credentials host-owned |
+| `@arnilo/prism-credentials-node` encrypted file | Node `crypto` (AES-256-GCM + scrypt) | built-in | 20+ | No extra deps for AEAD/KDF; atomic rename writes |
+| `@arnilo/prism-credentials-node` keychain | `@napi-rs/keyring` | `^1.3.0` | ≥10 (satisfies 20+) | Cross-platform, actively maintained (2026); `keytar@7.9.0` rejected — last release 2022, heavier native rebuild friction |
+
+**Rejected options:**
+
+| Option | Why rejected |
+| --- | --- |
+| Database/ORM in core | Violates dependency-light core; hosts choose SQL dialect |
+| `node:sqlite` at current baseline | Incompatible with Node 20 floor unless release plan raises engine |
+| Generic SQL layer shared by SQLite + Postgres | Dialect/driver differences outweigh shared SQL abstraction |
+| Core global credential store | Violates host ownership |
+| Keychain silent fallback to plaintext file | Fail closed; explicit backend selection only |
+| Provider upload IDs in `ContentBlock` | Provider-local mapping over generic references (Task 5–6) |
+
+## ADR decision table (frozen for Tasks 1–7)
+
+| Concern | Option A | Option B | **Chosen** | Rationale |
+| --- | --- | --- | --- | --- |
+| Persistence location | Core built-in DB | Optional packages over contracts | **Optional packages** | Matches Plan 053 JSONL boundary and `ProductionPersistenceStore` extension point |
+| Schema/migrations | Core DDL generator | Shared fixture model + dialect-local SQL | **Shared fixtures + local SQL** | Two adapters without ORM |
+| Run ledger conformance | Per-package tests only | Shared conformance module (Task 1) | **Shared module** | Parity with session-store conformance |
+| Credential persistence | Core global store | `@arnilo/prism-credentials-node` | **Optional package** | Host selects file vs keychain |
+| KDF | PBKDF2 default | scrypt with documented minimums | **scrypt** (configurable N/r/p) | Node built-in; calibrate in Task 4 tests |
+| Multimodal content | Provider-specific options only | Generic `ContentBlock` + capability tags | **Generic blocks + capabilities** | Portable input; provider maps/uploads locally |
+| URL/file sources | Loader reads anything | Bounded loader policy + trust integration | **Bounded + trust** | Reuse `createPathTrustPolicy` patterns; SSRF deny-by-default for URLs |
+| Unsupported modality | Silent drop | Capability check before network | **Reject before provider call** | `UnsupportedModalityError` or equivalent host-visible error |
+
+## Planned generic APIs (Tasks 1–6)
+
+### Task 1 — Shared schema/conformance primitives
+
+```ts
+// Extend @arnilo/prism/testing exports (names illustrative)
+export async function assertSessionStoreConforms(store, options?): Promise<void>;
+export async function assertRunLedgerConforms(factory, options?): Promise<void>;
+export interface PersistenceSchemaModel { /* versioned tables, tenant columns, idempotency side table */ }
+```
+
+### Task 2–3 — Database adapters
+
+```ts
+const store: SessionStore = createPostgresSessionStore({ pool, schema: "prism" });
+const persistence = createSqlitePersistence({ filename: "./prism.db" });
+await persistence.close();
+```
+
+Transaction boundary: **one transaction per `SessionStore.append`** covering idempotency insert, parent check, entry insert, optional branch leaf update.
+
+### Task 4 — Credential package
+
+```ts
+const store = createEncryptedCredentialStore({ path, getPassphrase });
+const keychain = createKeychainCredentialStore({ service: "prism" });
+const resolver = createStoredCredentialResolver(store);
+await refreshOAuthCredential({ provider, credentials, store });
+```
+
+Versioned envelope: `{ version, kdf, cipher, salt, nonce, ciphertext }` with authenticated encryption; file mode `0600` on Unix; wrong passphrase fails closed.
+
+### Task 5 — Content contracts
+
+```ts
+{ type: "file", mediaType: "application/pdf", name: "report.pdf", data: bytes }
+{ type: "audio", mediaType: "audio/wav", data: bytes, durationMs?: number }
+```
+
+`ResourceLoader` resolves `bytes`/`url`/`resource:` references under configurable global and per-item limits. `ModelCapabilities.input` gains `audio`, `file`, `document` tags when models support them.
+
+### Task 6 — Provider mapping
+
+```ts
+if (!model.capabilities?.input?.includes(block.type)) {
+  throw new UnsupportedModalityError(block.type);
+}
+```
+
+Upload-backed providers manage create/use/delete lifecycle in package-local code; conformance asserts capabilities match behavior.
+
+## Conformance and threat-model matrix
+
+Tasks 1–7 must pass this matrix (unit/integration tests + existing `assertSessionStoreConforms` / provider conformance extensions):
+
+| # | Scenario | Expected behavior |
+| ---: | --- | --- |
+| 1 | Process restart after successful append | Entry durable; idempotency row survives |
+| 2 | Concurrent append same parent | One succeeds; other conflicts or serializes per adapter policy |
+| 3 | Exact idempotency retry | `SessionAppendConflictError` with `idempotencyDuplicate: true`; no duplicate row |
+| 4 | Distinct linear appends same run key | Both entries persist |
+| 5 | Tenant A query with tenant B id | Empty or denied; never cross-tenant rows |
+| 6 | SQL injection in session id / idempotency key | Parameterized queries only; no string interpolation of values |
+| 7 | Wrong encryption passphrase | Decrypt fails closed; no partial plaintext |
+| 8 | Tampered ciphertext / truncated file | AEAD verification fails |
+| 9 | Key rotation / rewrap | Old credentials readable during migration window; new writes use new key |
+| 10 | Keychain unavailable / timeout | Explicit locked/unavailable error; no silent plaintext fallback |
+| 11 | URL fetch to RFC1918 / metadata IP | Denied by SSRF policy before fetch |
+| 12 | Local path outside trust root | `createPathTrustPolicy` denies before read |
+| 13 | MIME spoof (extension vs magic) | Reject or re-label per policy; no trust extension alone |
+| 14 | Media bomb (zip/gzip/pdf) | Byte/time ceilings; streaming handles closed |
+| 15 | Unsupported provider modality | Error before provider HTTP; never silent drop |
+| 16 | Provider upload lifecycle | Temp remote IDs cleaned up on run end/abort within retention bound |
+| 17 | Ledger + store redaction | No raw secrets in durable rows after `redactRunLedgerRecord` / `redactSessionEntry` |
+| 18 | PostgreSQL migration race | Advisory lock or equivalent; one winner applies migrations |
+
+### Threat model summary
+
+| Threat | Owner | Mitigation |
+| --- | --- | --- |
+| SQL injection | Adapter package | Parameterized statements; validated/quoted identifiers for schema names |
+| Tenant isolation bypass | Host + adapter | `tenantId` in unique/FK boundaries; integration tests per tenant |
+| World-readable credential file | Credential package | `0600` perms, umask guidance, no plaintext temp files |
+| Weak KDF parameters | Credential package | Documented minimum scrypt work factor; reject weak config |
+| Keychain trust / spoof UI | Host OS | Document service/name namespacing; no auto-trust |
+| SSRF via content URL | Host loader policy | Deny private/link-local/metadata ranges; allow-list option |
+| Path traversal via file block | Host trust policy | Realpath containment before read |
+| Media decompression bomb | Core limits + loader | Byte caps, stat-first reject, timeouts |
+| MIME spoofing | Loader validation | Magic-byte sniff + declared type cross-check |
+| Secret retention in DB/JSONL | Host + runtime | Redact before append; audit scans in Task 7 |
+| JSONL multi-writer corruption | Host deployment | Document dev-only boundary; use DB adapter in production |
+| Unsupported modality data loss | Provider layer | Capability-gated reject before network |
+
+## Performance notes
+
+| Concern | Target / guidance |
+| --- | --- |
+| Branch read | `readBranchPath` single ancestor query; avoid `list(sessionId)` for production context |
+| Pagination | Cursor on `(timestamp, id)` or `(run_id, sequence)`; no offset scans on long sessions |
+| `SessionStore.append` | One entry per transaction; O(1) indexed parent/idempotency checks |
+| Connection pool (Postgres) | Host-sized; default max ~10 per process documented in package README |
+| SQLite WAL | Enable WAL; busy timeout ≥ 5s documented; single-writer still recommended |
+| Ledger writes | Serialized per session run; batching inside adapter allowed if order preserved |
+| scrypt KDF | Default N=2^15, r=8, p=1 (adjust in docs after Task 4 calibration); cap passphrase attempts |
+| Keychain calls | Timeout/abort where backend supports; do not block run loop unbounded |
+| Global media budget | Default 32 MiB total per request assembly (Task 5); per-item default 10 MiB (align with `DEFAULT_MAX_IMAGE_BYTES`) |
+| Audio duration ceiling | Default 5 minutes decoded (Task 5) |
+| Fetch timeout | Default 30s per URL resource (Task 5) |
+| Upload cache | Provider-local; bounded entries per run; cleanup on `done`/`error`/`abort` |
+
+## Implementation example
+
+```ts
+import {
+  createExplicitCredentialResolver,
+  createMemoryCredentialStore,
+  createMemorySessionStore,
+} from "@arnilo/prism";
+import { assertSessionStoreConforms } from "@arnilo/prism/testing/session-store-conformance";
+
+// Today: memory + JSONL dev stores only.
+const store = createMemorySessionStore();
+
+// Task 2+:
+// const store = createSqliteSessionStore({ filename: "./prism.db" });
+
+await assertSessionStoreConforms(store, { exerciseReadBranchPath: true });
+
+const resolver = createExplicitCredentialResolver([
+  { name: "memory", resolver: createMemoryCredentialStore() },
+]);
+void resolver;
+```
+
+## Extension and configuration notes
+
+- Hosts implement `SessionStore` and optional `RunLedger` / `ProductionPersistenceStore`; Prism runtime does not open databases or keychains.
+- Credential backends are selected explicitly at app startup; no core singleton.
+- Multimodal limits and SSRF/path policy are host-configurable; defaults are finite.
+- Provider packages declare truthful `ModelCapabilities.input` and map or reject each block type.
+- Live PostgreSQL tests remain opt-in for local runs (`PRISM_TEST_POSTGRES_URL` / `npm run test:postgres`) and are enforced in CI by the `postgres-integration` job; default `npm test` / `sdk:ready` stay network-free.
+
+## Security and performance notes
+
+See **Threat model summary** and **Performance notes** above. Cross-cutting rules:
+
+- Never store `CredentialResolver`, provider instances, API keys, or unredacted secrets in session entries, ledger rows, or idempotency tables.
+- Redact with `createSecretRedactor` / `redactRunLedgerRecord` / `redactSessionEntry` before durable writes.
+- JSONL remains development-only; production multi-writer requires database adapters from Tasks 2–3.
+
+## Related APIs
+
+- [Database persistence](database-persistence.md): reference schema, indexes, conditional append pattern
+- [Session stores](session-stores.md): runtime `SessionStore` contract and branch handles
+- [Session store conformance](session-store-conformance.md): `assertSessionStoreConforms`
+- [Runs and usage ledger](runs-and-usage.md): `RunLedger` write seam
+- [Node JSONL session store](node-jsonl-session-store.md): dev-only file adapter boundaries
+- [Credentials and redaction](credentials-and-redaction.md): core resolver/OAuth helpers
+- [Security/auth/trust](settings-auth-trust-security.md): trust, permissions, memory credentials
+- [Input and prompt assembly](input-and-prompt-assembly.md): attachments and input layout
+- [Resource loading](resource-loading.md): `ResourceLoader` decode helpers
+- [Model registry](model-registry.md): `ModelCapabilities` metadata
+- [Provider conformance](provider-conformance.md): content preservation and secret leak checks
+- [Review coverage (2026-07-14)](review-coverage-2026-07-14.md): traceability for C-005, C-010, C-011

package/docs/postgres-persistence.md

@@ -0,0 +1,141 @@
+# PostgreSQL persistence
+
+## What it does
+
+The optional `@arnilo/prism-session-store-postgres` package ships a production-oriented PostgreSQL adapter that implements:
+
+- `SessionStore` — atomic `append` / `list` / `get` / `readBranchPath`
+- `RunLedger` — durable run, event, tool-call, and usage rows
+- `ProductionPersistenceStore` — cursor-paginated `query*` reads plus generic `checkpoints` and atomic `leases` capabilities
+
+Factory:
+
+- `createPostgresPersistence(options)` (async)
+- `PostgresPersistenceOptions`
+- `PostgresPersistence.close()` (async; ends adapter-owned pools only)
+
+The adapter uses `pg@^8.22.0`, applies versioned migrations from the shared Plan 056 schema model inside a transaction guarded by `pg_advisory_xact_lock`, validates/quotes schema identifiers, and passes the full session-store and run-ledger conformance suites when `PRISM_TEST_POSTGRES_URL` is set.
+
+## When to use it
+
+Use this package when you need server-backed persistence with pooled connections and multi-writer concurrency:
+
+- hosted SaaS agents with PostgreSQL as the system of record
+- managed cloud databases (RDS, Cloud SQL, Neon, Supabase, etc.)
+- CI integration tests against a real PostgreSQL service
+
+Prefer [`@arnilo/prism-session-store-sqlite`](sqlite-persistence.md) for local CLI tools, single-writer workloads, and network-free default tests.
+
+## Inputs / request
+
+```ts
+import { Pool } from "pg";
+import { createPostgresPersistence } from "@arnilo/prism-session-store-postgres";
+```
+
+| Field | Type | Purpose |
+| --- | --- | --- |
+| `pool` | `Pool` | Existing `pg` pool (caller owns lifecycle). |
+| `connectionString` | `string` | Create an adapter-owned bounded pool when `pool` is omitted. |
+| `schema` | `string` | PostgreSQL schema for Prism tables. Defaults to `"prism"`. Validated and double-quoted. |
+| `poolMax` | `number` | Maximum pool size for adapter-owned pools. Defaults to `10`. |
+| `poolConfig` | `PoolConfig` | Additional `pg` options (TLS, idle timeout, application name, etc.). |
+
+Hosts own TLS (`ssl` in `poolConfig`), credentials, connection limits, and backup/retention enforcement.
+
+## Outputs / response / events
+
+`createPostgresPersistence()` returns one object implementing the three persistence contracts plus generic checkpoints and leases:
+
+| Method group | Behavior |
+| --- | --- |
+| `SessionStore.append` | One transaction per append: parent existence check, idempotency dedup, duplicate-id rejection, entry insert. |
+| `SessionStore.list` / `get` | Indexed reads by `session_id` and primary key. |
+| `SessionStore.readBranchPath` | Recursive ancestor query from `leafId` (or latest leaf) in root→leaf order. |
+| `RunLedger.append*` | Inserts run/event/tool/usage rows; events receive monotonic per-run `sequence` values. |
+| `ProductionPersistenceStore.query*` | Parameterized cursor pagination on indexed columns with tenant/account/user filters. |
+| `checkpoints` | Generic versioned `CheckpointStore` backed by `prism_checkpoints`; ownership, CAS/fencing checks, and bounded pagination. |
+| `leases` | Atomic `LeaseStore` backed by `prism_leases`; database-clock expiry, opaque renew/release token, monotonic takeover fence. |
+| `close()` | Ends the pool when the adapter created it from `connectionString`. |
+
+Migrations run automatically on open and are idempotent across reopen. Concurrent setup uses per-schema advisory transaction locks.
+
+## Request/response example
+
+```json
+{
+  "connectionString": "postgres://app:***@db.example.com:5432/prism",
+  "schema": "prism",
+  "poolMax": 10,
+  "poolConfig": {
+    "ssl": { "rejectUnauthorized": true }
+  }
+}
+```
+
+## Implementation example
+
+```ts
+import { Pool } from "pg";
+import { createAgentSession } from "@arnilo/prism";
+import { createPostgresPersistence } from "@arnilo/prism-session-store-postgres";
+import { runSessionStoreConformance } from "@arnilo/prism/testing/session-store-conformance";
+
+const pool = new Pool({
+  connectionString: process.env.DATABASE_URL,
+  max: 10,
+  ssl: { rejectUnauthorized: true },
+});
+
+const persistence = await createPostgresPersistence({ pool, schema: "prism" });
+
+await runSessionStoreConformance(
+  async () => createPostgresPersistence({ pool, schema: "prism" }),
+  { exerciseReadBranchPath: true, exerciseReopen: true },
+);
+
+const session = createAgentSession({
+  sessionStore: persistence,
+  runLedger: persistence,
+});
+
+await session.run("hello");
+await persistence.close();
+await pool.end();
+```
+
+Live conformance and integration tests:
+
+```bash
+PRISM_TEST_POSTGRES_URL="$DATABASE_URL" npm run test:postgres --workspace @arnilo/prism-session-store-postgres
+```
+
+## Extension and configuration notes
+
+- The package is optional and workspace-local; `@arnilo/prism` core has no PostgreSQL dependency.
+- Schema names must match `^[a-zA-Z_][a-zA-Z0-9_]*$`; the adapter quotes them and never interpolates user values into identifier positions.
+- `SessionAppendOptions` idempotency rows are durable in `prism_session_append_idempotency` and survive reopen.
+- Schema version **1** (`001_init`) matches `@arnilo/prism/testing/persistence-schema` — SQLite adapters share the same model with dialect-local DDL.
+- Pass an existing `pg` `Pool` when your host already manages pooling, TLS, and credential rotation.
+
+## Security and performance notes
+
+- **Parameterized SQL only.** Session ids, idempotency keys, tenant ids, and JSON payloads are bound parameters (`$1`, `$2`, …).
+- **Identifier validation.** Configurable `schema` is validated and double-quoted; table names are fixed constants in adapter SQL.
+- **TLS and credentials.** Configure via `pg` `Pool` / `PoolConfig`; the adapter does not read environment variables unless the host passes them into `connectionString` or `poolConfig`.
+- **Redaction upstream.** Event and tool-call payloads may contain secrets; redact before ledger writes. The adapter does not scan or rewrite row contents.
+- **Bounded pool.** Adapter-owned pools default to `max: 10`. Hosts with heavy concurrency should supply their own pool sizing.
+- **Indexed operations.** Append, parent validation, idempotency dedup, branch reads, and pagination use the indexes documented in [Database persistence](database-persistence.md); normal paths avoid sequential scans.
+- **Migration locking.** `pg_advisory_xact_lock` prevents concurrent `001_init` races when multiple processes open the adapter at once.
+- **Tenant isolation.** `tenant_id` / `account_id` / `user_id` columns participate in query filters; hosts must still scope writes correctly.
+- **Benchmark target.** Indexed append + paginated branch read on a warm pool should stay under **50 ms p95** for local/CI-sized datasets (≤100k entries per session); measure with your pool size and hardware before production sizing.
+
+## Related APIs
+
+- [Database persistence](database-persistence.md): shared schema model, conditional append pattern, indexes.
+- [SQLite persistence](sqlite-persistence.md): file-backed alternative for local/single-writer hosts.
+- [Session store conformance](session-store-conformance.md): `assertSessionStoreConforms` / `runSessionStoreConformance`.
+- [Run ledger conformance](run-ledger-conformance.md): `assertRunLedgerConforms` / `runRunLedgerConformance`.
+- [Persistence, credentials, and multimodality primitives](persistence-credentials-multimodality-primitives.md): package matrix and threat model.
+- [Workflows](workflows.md): adapt `persistence.checkpoints` and pass `persistence.leases` to `createWorkflowCoordinator()` for durable multi-process workflow execution.
+- [Migration guide](migration.md): moving from JSONL/in-memory to database-backed persistence.

package/docs/provider-conformance.md

@@ -93,6 +93,22 @@ const body = JSON.parse(String(fetchInit.body));
 assertSerializedRequestCoversContent(request, body, { unsupported: ["image"] });
 ```
 
+Multimodal coverage example:
+
+```ts
+const request = {
+  model: { provider: "openai", model: "gpt-5.1", capabilities: { input: ["text", "file", "document", "audio"] } },
+  messages: [{ role: "user", content: [
+    { type: "file", mediaType: "application/pdf", name: "report.pdf", data: "..." },
+    { type: "audio", mediaType: "audio/wav", data: "..." },
+  ] }],
+};
+
+assertSerializedRequestCoversContent(request, body);
+```
+
+Pass `unsupported` only for modalities the provider deliberately omits from the wire format while still accepting the turn via another block type.
+
 Protected header-ownership example:
 
 ```ts
@@ -127,6 +143,7 @@ The helpers are a testing subpath only. Provider packages can use them with thei
 - Use fake credentials only in fixtures.
 - The helpers collect one stream into memory; keep conformance fixtures small.
 - Redaction remains the provider/runtime boundary's job. Use `assertNoSecretLeak()` with known fake secrets to catch regressions, not as a general secret scanner.
+- First-party providers read SSE streams and HTTP error bodies through bounded helpers from `@arnilo/prism/providers/transport` (`readSseEvents` / `readSseData`, `readBoundedResponseText`). Oversized remote input terminates with `ProviderTransportError` instead of unbounded buffering.
 
 ## Related APIs
 

package/docs/provider-layer.md

@@ -129,7 +129,7 @@ const agent = createAgent({ model: { provider: own.id, model: "demo" }, provider
 - Provider event helpers return plain `ProviderEvent` objects.
 - `providerError()` converts unknown errors to redacted `ErrorInfo` through `errorToErrorInfo()` and preserves safe string/number `code` fields for retry classification.
 - `createMockProvider()` returns an `AIProvider` whose `generate()` yields the scripted events in order and checks `request.signal?.aborted` before each event.
-- The agent/session runtime passes its per-run abort signal as `ProviderRequest.signal`. `ProviderRequestOptions.timeoutMs`, `maxRetries`, and `maxRetryDelayMs` are deprecated inert hints in first-party providers; use `RunOptions.signal`/host abort controllers for timeouts and `AgentConfig.retry`/`RunOptions.retry` for retry.
+- The agent/session runtime passes its per-run abort signal as `ProviderRequest.signal`. `ProviderRequestOptions.structuredOutput` requests provider-native JSON-schema output when the model declares `capabilities.structuredOutput`; unsupported models fail before fetch. `ProviderRequestOptions.timeoutMs`, `maxRetries`, and `maxRetryDelayMs` are deprecated inert hints in first-party providers; use `RunOptions.signal`/host abort controllers for timeouts and `AgentConfig.retry`/`RunOptions.retry` for retry.
 
 ## Request/response example