@arnilo/prism 0.0.2 → 0.0.4

package/docs/observability.md

@@ -0,0 +1,163 @@
+# Observability
+
+## What it does
+
+Prism exposes provider and tool timing through stable, metadata-only `AgentEvent` variants. Hosts subscribe via `session.subscribe()` or persist events through `RunLedger`. Core helpers build `ProviderTurnMetadata` and classify HTTP failures without echoing prompts, tool arguments, or credentials.
+
+Optional package `@arnilo/prism-observability-opentelemetry` maps those events to OpenTelemetry spans and low-cardinality metrics. OpenTelemetry is **not** a dependency of `@arnilo/prism`.
+
+APIs:
+
+- `ProviderTurnMetadata`, `ToolExecutionMetadata` on `AgentEvent`
+- `createProviderTurnMetadata()`, `readProviderHttpStatus()` in `@arnilo/prism`
+- `createOpenTelemetryInstrumentation()`, `wrapOpenTelemetryApi()`, `createInMemoryTelemetry()` in `@arnilo/prism-observability-opentelemetry`
+
+## When to use it
+
+Use agent events when you need run-scoped latency, retry attempt numbers, token/cache usage, tool duration, or error classification in-process or through your own exporter.
+
+Use the OpenTelemetry adapter when you already run the OpenTelemetry SDK and want spans/metrics without forking the runtime.
+
+Do not parse raw provider SSE for timing — provider packages normalize stream events; the session emits `provider_turn_*` once per `generate()` attempt.
+
+## Inputs / request
+
+Core metadata helpers:
+
+```ts
+import { createProviderTurnMetadata, readProviderHttpStatus } from "@arnilo/prism";
+
+const metadata = createProviderTurnMetadata(request, providerId, { attempt: 2, latencyMs: 120 });
+const httpStatus = readProviderHttpStatus(errorInfo);
+```
+
+New agent event variants (metadata only):
+
+| Variant | When | Key fields |
+| --- | --- | --- |
+| `provider_turn_started` | Before each provider `generate()` attempt | `turn`, `metadata: ProviderTurnMetadata` |
+| `provider_turn_finished` | After success or failure of that attempt | `metadata` (includes `latencyMs`, optional `httpStatus`), `usage?`, `error?` |
+
+`ToolExecutionMetadata` on terminal tool events:
+
+| Field | Meaning |
+| --- | --- |
+| `durationMs` | Wall time from dispatch start to finish/block/error |
+| `status` | `finished` \| `error` \| `blocked` |
+
+OpenTelemetry adapter:
+
+```ts
+import { trace, metrics } from "@opentelemetry/api";
+import { createOpenTelemetryInstrumentation, wrapOpenTelemetryApi } from "@arnilo/prism-observability-opentelemetry";
+
+const { tracer, meter } = wrapOpenTelemetryApi(trace.getTracer("app"), metrics.getMeter("app"));
+const telemetry = createOpenTelemetryInstrumentation({ tracer, meter, onExporterError: console.error });
+
+const detach = telemetry.attachSession(session);
+// or: for await (const event of session.subscribe()) telemetry.handleAgentEvent(event);
+```
+
+Set `enabled: false` or omit `tracer`/`meter` for a no-op adapter.
+
+## Outputs / response / events
+
+Provider turn metadata fields:
+
+| Field | Source |
+| --- | --- |
+| `providerId` | Active provider id |
+| `model` | `ProviderRequest.model` |
+| `requestId` | `request.metadata.requestId` or `request.options.sessionId` |
+| `attempt` | Retry attempt (1-based) |
+| `latencyMs` | Set on `provider_turn_finished` |
+| `httpStatus` | Numeric `ErrorInfo.code` when present |
+| `rateLimitRemaining` / `rateLimitResetMs` | Reserved for provider adapters (optional) |
+
+OpenTelemetry mapping (when enabled):
+
+| Agent event | Span | Metric labels |
+| --- | --- | --- |
+| `agent_started` / `agent_finished` | `prism.agent.run` | — |
+| `provider_turn_*` | `prism.provider.turn` | `provider_id`, `outcome` on duration histogram |
+| `tool_execution_*` (terminal) | `prism.tool.execute` when started | `status` on duration histogram |
+| `provider_turn_finished` / `agent_finished` usage | span attributes | `prism.provider.tokens` counter (`kind`: input/output/cache_*) |
+
+High-cardinality identifiers (`sessionId`, `runId`, `requestId`, `toolCallId`) are **span attributes only**, never metric labels.
+
+## Request/response example
+
+```json
+{
+  "type": "provider_turn_finished",
+  "sessionId": "sess_01J...",
+  "runId": "run_01J...",
+  "turn": 1,
+  "metadata": {
+    "providerId": "openai",
+    "model": { "provider": "openai", "model": "gpt-4.1" },
+    "requestId": "sess_01J...",
+    "attempt": 2,
+    "latencyMs": 842,
+    "httpStatus": 503
+  },
+  "error": { "message": "upstream unavailable", "code": 503 }
+}
+```
+
+```json
+{
+  "type": "tool_execution_finished",
+  "sessionId": "sess_01J...",
+  "runId": "run_01J...",
+  "result": { "toolCallId": "call_1", "name": "echo" },
+  "metadata": { "durationMs": 12, "status": "finished" }
+}
+```
+
+## Implementation example
+
+```ts
+import { createAgent, createMockProvider, providerDone, providerTextDelta } from "@arnilo/prism";
+import { createInMemoryTelemetry, createOpenTelemetryInstrumentation } from "@arnilo/prism-observability-opentelemetry";
+
+const memory = createInMemoryTelemetry();
+const telemetry = createOpenTelemetryInstrumentation({ tracer: memory.tracer, meter: memory.meter });
+
+const session = createAgent({
+  model: { provider: "mock", model: "demo" },
+  provider: createMockProvider([providerTextDelta("hi"), providerDone()]),
+}).createSession();
+
+const detach = telemetry.attachSession(session);
+await session.run("hello");
+detach();
+
+console.log(memory.spans.map((span) => span.name));
+```
+
+## Extension and configuration notes
+
+- Events flow through `redactAgentEvent` before subscribers and ledger writes — configure `createSecretRedactor` on the agent/run.
+- `retry_scheduled` still signals backoff; each retry attempt emits its own `provider_turn_*` pair with `metadata.attempt`.
+- NeuralWatt `neuralwatt:telemetry` provider events remain package-local; hosts may forward numeric cost/energy into custom metrics.
+- `@arnilo/prism-observability-opentelemetry` is optional and included through `@arnilo/prism-sdk` and `@arnilo/prism-all`; instrumentation remains disabled until a host configures it.
+- Exporter failures are isolated: instrumentation catches tracer/meter errors and invokes `onExporterError` without affecting the run.
+- Disabled instrumentation performs no per-delta span work (`enabled: false` or missing tracer/meter).
+
+## Security and performance notes
+
+- Default events are metadata-only — no prompts, streamed deltas, tool arguments, or credentials.
+- Opt-in content in other event types (`message_delta`, tool `result`) is still subject to `redactAgentEvent`.
+- Metric labels stay low-cardinality (`provider_id`, `outcome`, `status`, token `kind`); never use `sessionId`/`runId` as labels.
+- Target overhead when enabled is under 5% excluding exporter I/O; disabled hooks allocate no spans.
+- Provider transport limits and redaction order are documented in [Provider primitives](provider-primitives.md).
+
+## Related APIs
+
+- [Agent events](agent-events.md): full `AgentEvent` union and subscriber semantics.
+- [Runs and usage ledger](runs-and-usage.md): durable `AgentEventRecord` persistence.
+- [Middleware hooks](middleware-hooks.md): transform boundaries alongside event subscribers.
+- [Provider primitives](provider-primitives.md): frozen observability contract for Plan 054.
+- [Credentials and redaction](credentials-and-redaction.md): secret redaction before events and ledger rows.
+- [Workflows](workflows.md): package-local `WorkflowEvent` stream that can wrap redacted `AgentEvent`s from agent nodes.

package/docs/performance.md

@@ -7,6 +7,7 @@ This page states Prism runtime limits that keep slow consumers and long sessions
 Current surfaces:
 
 - `SubscribeOptions` for bounded live `AgentEvent` subscriber queues.
+- Bounded provider transport primitives (`readSseEvents`, `readBoundedResponseText`) used by every first-party provider package — see [Provider primitives](provider-primitives.md).
 - `SessionStore.readBranchPath(query)` for branch reads that avoid full-session scans.
 - `ProductionPersistenceStore` cursor queries for entries, events, runs, tool calls, and usage.
 - JSONL and memory stores documented as development/local adapters, not production multi-writer stores.
@@ -112,10 +113,48 @@ const store = {
 
 - Overflow events contain only counts and policy, never message text, tool arguments, prompts, provider payloads, or credentials.
 - Runtime event payloads can be large (`Message`, content deltas, tool results, summaries, artifact metadata). Size queues by events and keep payload size in mind.
+- `toolConcurrency` on the single-shot loop bounds in-flight tool dispatches per provider turn to `min(toolConcurrency, calls.length)`. Independent slow tools can overlap; transcript appends remain ordered. Default `1` preserves sequential behavior.
+- `read` image bounds: default `maxImageBytes` is 10 MB (`DEFAULT_MAX_IMAGE_BYTES`). Oversize images are rejected by `stat` before read when possible; hosts may supply `transformImage` for resize/re-encode without adding image-processing deps to the base package.
 - Live subscriber queues are bounded by default. Durable replay belongs to host storage.
 - `SessionStore.list(sessionId)` is a full-session read. It is fine for memory/JSONL development stores, but production adapters should use `readBranchPath` for provider context and branch views.
 - The JSONL store rereads/parses the file for validation/list/get and serializes appends only within one process. It has no cross-process lock, pagination, migrations, tenant isolation, or retention.
 - Recommended database indexes: session id, run id, parent id, branch leaf id, timestamps, tenant/account/user, event type, entry kind, `(run_id, sequence)` for event timelines, and `(run_id, recorded_at, id)` for usage. Allocate event `sequence` per run for stable timeline pagination.
+- Provider SSE parsing defaults: 256 KiB per completed event, 512 KiB incomplete buffer, 64 KiB error response bodies. Override per call via `BoundedStreamLimits` on `@arnilo/prism/providers/transport`.
+
+### Provider-phase benchmark snapshot (2026-07-14)
+
+Node v24.18.0, Linux x86_64, AMD Ryzen 9 PRO 7940HS; local synthetic streams, no network or exporter I/O:
+
+| Case | Result |
+| --- | --- |
+| `readSseEvents`, 16 MiB total as 4 KiB events | 380 MiB/s, +1.7 MiB end-of-run heap delta |
+| 100 provider deltas with 1 ms transport delay, no telemetry | 107.97 ms median |
+| Same run, disabled adapter attached | 107.25 ms median (-0.67%, noise) |
+| Same run, enabled no-op exporter | 107.22 ms median (-0.70%, noise) |
+
+Nine measured runs per telemetry mode after warm-up; table reports median. A zero-I/O burst of 5,000 deltas measured 1.06 ms without telemetry and 2.09 ms with the adapter: about 1 ms absolute adapter cost, but a large percentage against an unrealistically tiny baseline. No span is created for message deltas. Add subscriber-side event filtering only if measured high-frequency in-memory streams make that ceiling material.
+
+Configured overflow behavior is enforced by `src/__tests__/provider-transport.test.ts` for event, incomplete-buffer, response-body, argument, and abort limits.
+
+### 0.0.4 release audit snapshot (2026-07-14)
+
+Node v24.18.0 on the same Linux x86_64 / Ryzen 9 PRO 7940HS host. Results are medians of 7-9 warm runs unless the row describes file/database appends. Synthetic operations use local memory/files only. These numbers are release ceilings and comparison points, not cross-machine guarantees.
+
+| Surface | Workload | Result | 0.0.4 release threshold |
+| --- | --- | --- | --- |
+| Run ledger | One mock run, 500 text deltas, 510 total records | 1.19 ms with in-memory ledger vs 0.17 ms without; +1.02 ms absolute; event append max concurrency 1 | < 10 ms with zero-I/O adapter; event appends remain serialized |
+| JSONL store | 500 sequential label appends, including fail-closed reread/validation | 141.10 ms; 3,544 appends/s | < 500 ms; development/single-process only |
+| JSON Schema compile cache | 5,000 validations through one warm adapter | 4.97 ms; 0.99 µs/validation | < 25 µs/validation |
+| JSON Schema cold compile | 100 new adapters + first validation | 249.89 ms; 2.50 ms/compile | Warm cache must remain at least 20x faster than cold compile |
+| Parallel tools | Six independent 20 ms calls | concurrency 1: 121.12 ms; concurrency 2: 60.92 ms; 1.99x speedup | concurrency 2 < 75% of sequential; configured worker cap remains enforced |
+| SQLite session store | 1,000 sequential transactional label appends | 31.84 ms; 31,405 appends/s | < 250 ms on local SSD/tmp storage |
+| Secret redaction | 10,000 shallow objects containing one known secret | 4.79 ms; 2.09 million objects/s | < 25 ms |
+| Credential KDF | Default scrypt + AES-256-GCM encryption | 48.09 ms median | 20-250 ms; security floor stays `N >= 16,384` |
+| Workflow runner | Existing bounded 1,000-node DAG fixture | 27.68 ms in aggregate release gate | < 1 s; no rescan failure |
+
+Provider SSE remained at the frozen 380 MiB/s / +1.7 MiB heap snapshot. Media and MCP retain 10 MB defaults and finite timeout/total-byte guards; their malicious-input and oversize fixtures pass. PostgreSQL latency remains environment-dependent and is gated by transactional conformance in CI rather than a hardware-specific wall-clock assertion.
+
+The ledger percentage overhead is intentionally not a threshold: its no-ledger baseline is below 1 ms, making the percentage unstable while absolute added latency remains about 1 ms. JSONL's append path is intentionally O(n²) across repeated appends because it rereads for corruption/conflict checks; move production or high-volume workloads to SQLite/PostgreSQL rather than weakening validation.
 
 ## Related APIs
 
@@ -124,4 +163,4 @@ const store = {
 - [Session stores](session-stores.md): `SessionStore.readBranchPath` and dev-vs-production branch reads.
 - [Database persistence](database-persistence.md): cursor queries, reference schema, indexes, and event sequence guidance.
 - [Runs and usage ledger](runs-and-usage.md): durable event, tool-call, and usage persistence.
-- [Node JSONL session store](node-jsonl-session-store.md): development-only JSONL limits.
+- [Provider primitives](provider-primitives.md): bounded SSE/error-body limits for first-party providers.

package/docs/persistence-credentials-multimodality-primitives.md

@@ -0,0 +1,303 @@
+# Persistence, credentials, and multimodality primitives
+
+## What it does
+
+This page freezes the Plan 056 Task 0 inventory for production persistence, credential storage, and multimodal content. It maps existing `@arnilo/prism` contracts and conformance helpers, documents JSONL/security boundaries, records capability gaps **C-005**, **C-010**, and **C-011**, and pins package dependency choices for Tasks 1–7.
+
+Implementation is **shipped and phase-verified** (Tasks 0–7). Optional packages cover SQLite/PostgreSQL persistence and Node credential storage; core remains dependency-free.
+
+## When to use it
+
+- **Adapter authors** implementing SQLite/PostgreSQL `SessionStore` + `RunLedger` should start here, then follow [Database persistence](database-persistence.md) and [Session store conformance](session-store-conformance.md).
+- **Host apps** wiring CLI/desktop credential persistence should use the credential seams and package matrix here before choosing `@arnilo/prism-credentials-node` backends.
+- **Provider and core authors** extending multimodal input should use the content/resource/capability designs here instead of embedding provider upload IDs in core contracts.
+- **Security reviewers** use the threat model and conformance matrix on this page as the acceptance baseline for Plan 056 Tasks 1–7.
+
+## Inventory (2026-07-14 baseline)
+
+Static review of `src/contracts.ts`, `src/session-stores.ts`, `src/credentials.ts`, `src/input.ts`, `src/agents.ts`, `src/redaction.ts`, `src/node/session-store-jsonl.ts`, `src/testing/session-store-conformance.ts`, `examples/external-app-db-backed.ts`, and docs under `docs/session-stores*.md`, `docs/database-persistence.md`, `docs/runs-and-usage.md`, `docs/credentials-and-redaction.md`, `docs/settings-auth-trust-security.md`, `docs/input-and-prompt-assembly.md`, `docs/resource-loading.md`, `docs/model-registry.md`, `docs/provider-conformance.md`.
+
+### Session store and branch invariants (shipped)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `SessionStore` | `src/contracts.ts` | `append`, `list`, optional `get`, optional `readBranchPath` |
+| `SessionAppendOptions` | `src/contracts.ts` | `expectedParentId` existence validation; opaque `idempotencyKey` for same-position retry dedup |
+| `SessionAppendConflictError` | `src/contracts.ts` | Stable `code: "session_append_conflict"`; `isSessionAppendConflict()` |
+| Branch helpers | `src/session-stores.ts` | `getSessionBranchEntries`, `listSessionBranches`, `rebuildSessionContext`; reader overload follows `nextCursor` (max 64 pages) |
+| In-memory store | `src/session-stores.ts` | `createMemorySessionStore()` — O(1) dup/idempotency/parent maps; not durable |
+| JSONL store | `src/node/session-store-jsonl.ts` | Single-process dev adapter; per-line quarantine; in-memory idempotency only |
+| Conformance | `src/testing/session-store-conformance.ts` | `assertSessionStoreConforms(store, { exerciseReadBranchPath? })` |
+| Reference adapter | `examples/external-app-db-backed.ts` | In-memory tables implementing `SessionStore` + `RunLedger` + `ProductionPersistenceStore` |
+
+**Frozen branch semantics:** `expectedParentId` is parent **existence** validation, not tip-CAS. Two children of the same parent are allowed (fork/branch). Production adapters may add stricter tip compare-and-swap and populate `currentLeafId` in conflicts.
+
+**Frozen idempotency semantics:** Dedup key is `(session_id, expected_parent_id, idempotency_key)`. A run-level key may appear on multiple linear appends as the leaf advances; only exact same-position retries collapse.
+
+### Run ledger (shipped)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `RunLedger` | `src/contracts.ts` | `appendRun`, `appendEvent`, `appendToolCall`, `appendUsage` |
+| Runtime wiring | `src/agents.ts` | Active ledger from `RunOptions.runLedger ?? AgentConfig.runLedger`; ownership/idempotency from run options |
+| Serialization | `src/agents.ts` | `ledgerChain` serializes event ledger appends (R-004 backpressure fix) |
+| Redaction | `src/redaction.ts` | `redactRunLedgerRecord()` before every ledger write when `SecretRedactor` active |
+| Conformance helper | `src/testing/session-store-conformance.ts` | `assertSessionStoreConforms`, `runSessionStoreConformance` |
+| Run ledger conformance | `src/testing/run-ledger-conformance.ts` | `assertRunLedgerConforms`, `runRunLedgerConformance` |
+| Shared schema model | `src/testing/persistence-schema.ts` | `createPersistenceSchemaModel`, `createPersistenceMigrationContract`, pagination/tenant fixtures |
+
+### Production persistence contracts (shipped)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `ProductionPersistenceStore` | `src/contracts.ts` | Cursor `query*` methods for sessions, branches, entries, runs, events, tool calls, usage, agent definitions, retention, migrations; optional `readBranchPath` |
+| `OwnershipScope` | `src/contracts.ts` | `tenantId`, `accountId`, `userId` on records and queries — enforcement host-owned |
+| Reference schema | `docs/database-persistence.md` | Relational table/index reference, conditional append transaction pattern, NoSQL mapping notes |
+| Type compile test | `src/__tests__/persistence-contracts.types.test.ts` | Host adapter shapes compile without DB deps |
+
+**Core boundary:** No ORM, SQL driver, connection pool, migration runner, or filesystem persistence in `@arnilo/prism`.
+
+### Credential and OAuth seams (shipped — storage gap)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `CredentialResolver` / `CredentialRequest` | `src/contracts.ts` | Host-implemented resolve at request edge |
+| `resolveCredentialValue` | `src/credentials.ts` | String, callback, or resolver |
+| `createExplicitCredentialResolver` | `src/credentials.ts` | Named source order (runtime → stored → env → fallback) |
+| `createEnvCredentialResolver` | `src/credentials.ts` | Caller-supplied env object only; never reads `process.env` |
+| `createMemoryCredentialStore` | `src/credentials.ts` | In-memory resolver + `set`/`delete`; not durable |
+| `createChainedCredentialResolver` | `src/credentials.ts` | First match wins |
+| `refreshOAuthCredential` | `src/credentials.ts` | Calls `OAuthProvider.refresh`; optional `OAuthCredentialStore.set` |
+| `OAuthCredentialStore` | `src/contracts.ts` | `set(provider, credentials)` only — no `get`/`delete` in core contract |
+| `OAuthProvider` / `OAuthCredentials` | `src/contracts.ts` | `login`, optional `refresh`, optional `getCredential` |
+| Device-code OAuth | `packages/provider-openai` | Bounded polling; abort via `OAuthLoginCallbacks.signal` |
+| Redaction | `src/redaction.ts` | Exact known-secret replacement; not secret detection |
+
+**Gaps (C-011):** No encrypted file store, no system keychain adapter, no versioned credential envelope, no `OAuthCredentialStore` `get`/`delete`/`list` in core (Task 4 package may extend store interface locally while integrating `refreshOAuthCredential`).
+
+### Content, resources, and model capabilities (shipped — modality gap)
+
+| Surface | Location | Behavior today |
+| --- | --- | --- |
+| `ContentBlock` union | `src/contracts.ts` | `text`, `image`, `audio`, `file`, `document`, `thinking`, `tool_call_delta`, `tool_call`, `tool_result` |
+| `AudioContent` / `FileContent` / `DocumentContent` | `src/content.ts` | MIME, name, `data`/`url`/`resourceUri`, optional transcript/metadata |
+| `ImageContent` | `src/contracts.ts` | `mimeType?`, `data?`, `url?`, `resourceUri?`, `name?` |
+| `InputAttachment` | `src/input.ts` | `text`, inline `content` blocks, or `uri` + `ResourceLoader` → text user message |
+| `ResourceLoader` / `Resource` | `src/contracts.ts` | Host-owned `load(uri)`; optional `list`; `data`/`text`/`mediaType` |
+| Resource helpers | `src/resources.ts` | `loadTextResource`, `loadJsonResource`, `loadManifestResource`, `loadBinaryResource` — decode/bounds only |
+| Media helpers | `src/content.ts` | `resolveMediaContentBlock`, `assertSsrfAllowedUrl`, `assertMediaBlocksWithinBounds`, `UnsupportedModalityError` |
+| `ModelCapabilities.input` | `src/contracts.ts` | Known tags: `text`, `image`, `audio`, `file`, `document`; `MODEL_INPUT_CAPABILITIES` export |
+| Provider image mapping | first-party packages | OpenAI Responses, OpenRouter, OpenCode Go (Anthropic route), Kimi, NeuralWatt map `image` when capability allows |
+| Provider audio/file/document mapping | first-party packages | OpenAI Responses maps `audio`/`file`/`document`; Anthropic routes map PDF `document`/`file`; others reject undeclared media |
+| Provider conformance | `src/testing/provider-conformance.ts` | `assertSerializedRequestCoversContent` with per-provider `unsupported` list |
+
+### JSONL and dev-store security boundaries (shipped)
+
+| Boundary | Rule |
+| --- | --- |
+| Cross-process writes | JSONL has no lock; two processes on one file can race |
+| Durable idempotency | JSONL idempotency is in-process only |
+| Corrupt lines | Quarantined per line; do not poison whole file (R-005) |
+| Schema/kind | Unknown kinds and future `schemaVersion` fail closed |
+| Secrets in file | Host responsibility; runtime redacts before `append` when redactor configured |
+| Production use | JSONL documented as dev-only; database adapters required for multi-writer |
+
+## Package dependency and support matrix (pinned Task 0)
+
+Prism `engines.node` is `>=20`. Optional packages stay package-local; core adds no new runtime dependencies.
+
+| Package (planned) | Driver / backend | Pinned version | Node support | Rationale |
+| --- | --- | --- | --- | --- |
+| `@arnilo/prism-session-store-sqlite` | `better-sqlite3` | `^12.11.1` | 20.x–26.x per upstream engines | Synchronous API, WAL/busy_timeout, mature Node 20 baseline; `node:sqlite` rejected — requires Node ≥22.5 and is still experimental vs declared `>=20` baseline |
+| `@arnilo/prism-session-store-postgres` | `pg` | `^8.22.0` | ≥16 (satisfies 20+) | Standard pool + parameterized queries; TLS/credentials host-owned |
+| `@arnilo/prism-credentials-node` encrypted file | Node `crypto` (AES-256-GCM + scrypt) | built-in | 20+ | No extra deps for AEAD/KDF; atomic rename writes |
+| `@arnilo/prism-credentials-node` keychain | `@napi-rs/keyring` | `^1.3.0` | ≥10 (satisfies 20+) | Cross-platform, actively maintained (2026); `keytar@7.9.0` rejected — last release 2022, heavier native rebuild friction |
+
+**Rejected options:**
+
+| Option | Why rejected |
+| --- | --- |
+| Database/ORM in core | Violates dependency-light core; hosts choose SQL dialect |
+| `node:sqlite` at current baseline | Incompatible with Node 20 floor unless release plan raises engine |
+| Generic SQL layer shared by SQLite + Postgres | Dialect/driver differences outweigh shared SQL abstraction |
+| Core global credential store | Violates host ownership |
+| Keychain silent fallback to plaintext file | Fail closed; explicit backend selection only |
+| Provider upload IDs in `ContentBlock` | Provider-local mapping over generic references (Task 5–6) |
+
+## ADR decision table (frozen for Tasks 1–7)
+
+| Concern | Option A | Option B | **Chosen** | Rationale |
+| --- | --- | --- | --- | --- |
+| Persistence location | Core built-in DB | Optional packages over contracts | **Optional packages** | Matches Plan 053 JSONL boundary and `ProductionPersistenceStore` extension point |
+| Schema/migrations | Core DDL generator | Shared fixture model + dialect-local SQL | **Shared fixtures + local SQL** | Two adapters without ORM |
+| Run ledger conformance | Per-package tests only | Shared conformance module (Task 1) | **Shared module** | Parity with session-store conformance |
+| Credential persistence | Core global store | `@arnilo/prism-credentials-node` | **Optional package** | Host selects file vs keychain |
+| KDF | PBKDF2 default | scrypt with documented minimums | **scrypt** (configurable N/r/p) | Node built-in; calibrate in Task 4 tests |
+| Multimodal content | Provider-specific options only | Generic `ContentBlock` + capability tags | **Generic blocks + capabilities** | Portable input; provider maps/uploads locally |
+| URL/file sources | Loader reads anything | Bounded loader policy + trust integration | **Bounded + trust** | Reuse `createPathTrustPolicy` patterns; SSRF deny-by-default for URLs |
+| Unsupported modality | Silent drop | Capability check before network | **Reject before provider call** | `UnsupportedModalityError` or equivalent host-visible error |
+
+## Planned generic APIs (Tasks 1–6)
+
+### Task 1 — Shared schema/conformance primitives
+
+```ts
+// Extend @arnilo/prism/testing exports (names illustrative)
+export async function assertSessionStoreConforms(store, options?): Promise<void>;
+export async function assertRunLedgerConforms(factory, options?): Promise<void>;
+export interface PersistenceSchemaModel { /* versioned tables, tenant columns, idempotency side table */ }
+```
+
+### Task 2–3 — Database adapters
+
+```ts
+const store: SessionStore = createPostgresSessionStore({ pool, schema: "prism" });
+const persistence = createSqlitePersistence({ filename: "./prism.db" });
+await persistence.close();
+```
+
+Transaction boundary: **one transaction per `SessionStore.append`** covering idempotency insert, parent check, entry insert, optional branch leaf update.
+
+### Task 4 — Credential package
+
+```ts
+const store = createEncryptedCredentialStore({ path, getPassphrase });
+const keychain = createKeychainCredentialStore({ service: "prism" });
+const resolver = createStoredCredentialResolver(store);
+await refreshOAuthCredential({ provider, credentials, store });
+```
+
+Versioned envelope: `{ version, kdf, cipher, salt, nonce, ciphertext }` with authenticated encryption; file mode `0600` on Unix; wrong passphrase fails closed.
+
+### Task 5 — Content contracts
+
+```ts
+{ type: "file", mediaType: "application/pdf", name: "report.pdf", data: bytes }
+{ type: "audio", mediaType: "audio/wav", data: bytes, durationMs?: number }
+```
+
+`ResourceLoader` resolves `bytes`/`url`/`resource:` references under configurable global and per-item limits. `ModelCapabilities.input` gains `audio`, `file`, `document` tags when models support them.
+
+### Task 6 — Provider mapping
+
+```ts
+if (!model.capabilities?.input?.includes(block.type)) {
+  throw new UnsupportedModalityError(block.type);
+}
+```
+
+Upload-backed providers manage create/use/delete lifecycle in package-local code; conformance asserts capabilities match behavior.
+
+## Conformance and threat-model matrix
+
+Tasks 1–7 must pass this matrix (unit/integration tests + existing `assertSessionStoreConforms` / provider conformance extensions):
+
+| # | Scenario | Expected behavior |
+| ---: | --- | --- |
+| 1 | Process restart after successful append | Entry durable; idempotency row survives |
+| 2 | Concurrent append same parent | One succeeds; other conflicts or serializes per adapter policy |
+| 3 | Exact idempotency retry | `SessionAppendConflictError` with `idempotencyDuplicate: true`; no duplicate row |
+| 4 | Distinct linear appends same run key | Both entries persist |
+| 5 | Tenant A query with tenant B id | Empty or denied; never cross-tenant rows |
+| 6 | SQL injection in session id / idempotency key | Parameterized queries only; no string interpolation of values |
+| 7 | Wrong encryption passphrase | Decrypt fails closed; no partial plaintext |
+| 8 | Tampered ciphertext / truncated file | AEAD verification fails |
+| 9 | Key rotation / rewrap | Old credentials readable during migration window; new writes use new key |
+| 10 | Keychain unavailable / timeout | Explicit locked/unavailable error; no silent plaintext fallback |
+| 11 | URL fetch to RFC1918 / metadata IP | Denied by SSRF policy before fetch |
+| 12 | Local path outside trust root | `createPathTrustPolicy` denies before read |
+| 13 | MIME spoof (extension vs magic) | Reject or re-label per policy; no trust extension alone |
+| 14 | Media bomb (zip/gzip/pdf) | Byte/time ceilings; streaming handles closed |
+| 15 | Unsupported provider modality | Error before provider HTTP; never silent drop |
+| 16 | Provider upload lifecycle | Temp remote IDs cleaned up on run end/abort within retention bound |
+| 17 | Ledger + store redaction | No raw secrets in durable rows after `redactRunLedgerRecord` / `redactSessionEntry` |
+| 18 | PostgreSQL migration race | Advisory lock or equivalent; one winner applies migrations |
+
+### Threat model summary
+
+| Threat | Owner | Mitigation |
+| --- | --- | --- |
+| SQL injection | Adapter package | Parameterized statements; validated/quoted identifiers for schema names |
+| Tenant isolation bypass | Host + adapter | `tenantId` in unique/FK boundaries; integration tests per tenant |
+| World-readable credential file | Credential package | `0600` perms, umask guidance, no plaintext temp files |
+| Weak KDF parameters | Credential package | Documented minimum scrypt work factor; reject weak config |
+| Keychain trust / spoof UI | Host OS | Document service/name namespacing; no auto-trust |
+| SSRF via content URL | Host loader policy | Deny private/link-local/metadata ranges; allow-list option |
+| Path traversal via file block | Host trust policy | Realpath containment before read |
+| Media decompression bomb | Core limits + loader | Byte caps, stat-first reject, timeouts |
+| MIME spoofing | Loader validation | Magic-byte sniff + declared type cross-check |
+| Secret retention in DB/JSONL | Host + runtime | Redact before append; audit scans in Task 7 |
+| JSONL multi-writer corruption | Host deployment | Document dev-only boundary; use DB adapter in production |
+| Unsupported modality data loss | Provider layer | Capability-gated reject before network |
+
+## Performance notes
+
+| Concern | Target / guidance |
+| --- | --- |
+| Branch read | `readBranchPath` single ancestor query; avoid `list(sessionId)` for production context |
+| Pagination | Cursor on `(timestamp, id)` or `(run_id, sequence)`; no offset scans on long sessions |
+| `SessionStore.append` | One entry per transaction; O(1) indexed parent/idempotency checks |
+| Connection pool (Postgres) | Host-sized; default max ~10 per process documented in package README |
+| SQLite WAL | Enable WAL; busy timeout ≥ 5s documented; single-writer still recommended |
+| Ledger writes | Serialized per session run; batching inside adapter allowed if order preserved |
+| scrypt KDF | Default N=2^15, r=8, p=1 (adjust in docs after Task 4 calibration); cap passphrase attempts |
+| Keychain calls | Timeout/abort where backend supports; do not block run loop unbounded |
+| Global media budget | Default 32 MiB total per request assembly (Task 5); per-item default 10 MiB (align with `DEFAULT_MAX_IMAGE_BYTES`) |
+| Audio duration ceiling | Default 5 minutes decoded (Task 5) |
+| Fetch timeout | Default 30s per URL resource (Task 5) |
+| Upload cache | Provider-local; bounded entries per run; cleanup on `done`/`error`/`abort` |
+
+## Implementation example
+
+```ts
+import {
+  createExplicitCredentialResolver,
+  createMemoryCredentialStore,
+  createMemorySessionStore,
+} from "@arnilo/prism";
+import { assertSessionStoreConforms } from "@arnilo/prism/testing/session-store-conformance";
+
+// Today: memory + JSONL dev stores only.
+const store = createMemorySessionStore();
+
+// Task 2+:
+// const store = createSqliteSessionStore({ filename: "./prism.db" });
+
+await assertSessionStoreConforms(store, { exerciseReadBranchPath: true });
+
+const resolver = createExplicitCredentialResolver([
+  { name: "memory", resolver: createMemoryCredentialStore() },
+]);
+void resolver;
+```
+
+## Extension and configuration notes
+
+- Hosts implement `SessionStore` and optional `RunLedger` / `ProductionPersistenceStore`; Prism runtime does not open databases or keychains.
+- Credential backends are selected explicitly at app startup; no core singleton.
+- Multimodal limits and SSRF/path policy are host-configurable; defaults are finite.
+- Provider packages declare truthful `ModelCapabilities.input` and map or reject each block type.
+- Live PostgreSQL tests remain opt-in for local runs (`PRISM_TEST_POSTGRES_URL` / `npm run test:postgres`) and are enforced in CI by the `postgres-integration` job; default `npm test` / `sdk:ready` stay network-free.
+
+## Security and performance notes
+
+See **Threat model summary** and **Performance notes** above. Cross-cutting rules:
+
+- Never store `CredentialResolver`, provider instances, API keys, or unredacted secrets in session entries, ledger rows, or idempotency tables.
+- Redact with `createSecretRedactor` / `redactRunLedgerRecord` / `redactSessionEntry` before durable writes.
+- JSONL remains development-only; production multi-writer requires database adapters from Tasks 2–3.
+
+## Related APIs
+
+- [Database persistence](database-persistence.md): reference schema, indexes, conditional append pattern
+- [Session stores](session-stores.md): runtime `SessionStore` contract and branch handles
+- [Session store conformance](session-store-conformance.md): `assertSessionStoreConforms`
+- [Runs and usage ledger](runs-and-usage.md): `RunLedger` write seam
+- [Node JSONL session store](node-jsonl-session-store.md): dev-only file adapter boundaries
+- [Credentials and redaction](credentials-and-redaction.md): core resolver/OAuth helpers
+- [Security/auth/trust](settings-auth-trust-security.md): trust, permissions, memory credentials
+- [Input and prompt assembly](input-and-prompt-assembly.md): attachments and input layout
+- [Resource loading](resource-loading.md): `ResourceLoader` decode helpers
+- [Model registry](model-registry.md): `ModelCapabilities` metadata
+- [Provider conformance](provider-conformance.md): content preservation and secret leak checks
+- [Review coverage (2026-07-14)](review-coverage-2026-07-14.md): traceability for C-005, C-010, C-011