@arnilo/prism 0.0.4 → 0.0.6

package/docs/review-coverage-2026-07-15.md

@@ -0,0 +1,193 @@
+# Review coverage — 2026-07-15
+
+This page freezes Prism 0.0.5 scope at commit `f5128a816ae204c52f3e2f089de71c99bd5de6d4` after the Prism/Mastra review. It maps every confirmed finding and accepted capability to one owning roadmap phase, package/public surface, focused test owner, and documentation owner.
+
+Status: **scope frozen; Phases 1-14 implementation complete; publication handoff pending**. Phase 0 changes no public runtime behavior.
+
+## Release decision
+
+- Target release is **0.0.5**. Do not publish the currently versioned but unpublished 0.0.4 package graph.
+- npm currently reports `@arnilo/prism@0.0.3`; Phase 14 has retargeted every repository package and lock entry to 0.0.5. Publication remains pending the clean signed-tag workflow.
+- Core remains a Node.js 20-compatible, zero-runtime-dependency harness.
+- New capabilities are opt-in and package-owned. No server, database, credential store, telemetry exporter, memory worker, schedule, remote agent, or privileged tool activates on install.
+- Any row below that changes a trust boundary remains a release blocker until its owning phase and regression checks pass.
+
+## Frozen confirmed findings
+
+Each ID appears once. “Planned” means owned, not fixed.
+
+| ID | Confirmed finding | Disposition and single owner | Package / public surface | Focused test owner | Documentation owner | Status |
+| --- | --- | --- | --- | --- | --- | --- |
+| S-001 | Bracket-normalization lets IPv6 loopback, link-local, ULA, and IPv4-mapped private literals bypass media SSRF checks; DNS resolution/rebinding behavior is not sufficient for a private-network denial claim. | Phase 1 | Core `src/content.ts`; `SsrfPolicy` and media fetch path | `src/__tests__/content.test.ts` with injected resolver/requester matrix | `multimodal-content.md`, `host-security.md` | completed 2026-07-15 |
+| S-002 | `createReadOnlyTools()` does not apply the aggregate `executionPolicy`. | Phase 1 | `@arnilo/prism-coding-agent` read-only aggregator | coding-agent execution-policy tests | `coding-agent-tools.md` | completed 2026-07-15 |
+| S-003 | Approval caching defaults to a process-lifetime map instead of `none`; `run` and `session` scopes have no identity in the cache key. | Phase 1 | `@arnilo/prism-coding-security`; approval policy and execution identity metadata | coding-security approval tests | `coding-security.md`, `host-security.md` | completed 2026-07-15 |
+| C-001 | Sandbox adapter returns only an exit code and cannot forward stdout/stderr to coding-agent `onData`. | Phase 2 | coding-security sandbox adapter / coding-agent bash operations bridge | sandbox and shell integration tests | `coding-security.md`, `coding-agent-tools.md` | completed 2026-07-15 |
+| C-002 | OpenTelemetry agent spans end only on `agent_finished`; provider failure can leave `prism.agent.run` active. | Phase 2 | `@arnilo/prism-observability-opentelemetry` instrumentation lifecycle | in-memory telemetry failed/aborted-run tests | `observability.md` | completed 2026-07-15 |
+| C-003 | `prism.provider.tokens` is incremented for provider-turn and agent-total events, double-counting one usage source. | Phase 2 | OTel token metric names/scopes | in-memory metric assertions | `observability.md`, `runs-and-usage.md` | completed 2026-07-15 |
+| C-004 | Usage rows do not distinguish provider-turn values from run totals; the loop returns the latest turn rather than an explicit aggregate. | Phase 2 | core `Usage`, `UsageRecord`, runtime accumulator, persistence adapters | multi-turn run-ledger conformance and runtime tests | `runs-and-usage.md`, persistence docs | completed 2026-07-15 |
+| C-005 | Providers resolve media one block at a time, so request-wide 32-item/32-MiB bounds are not accumulated before provider/upload side effects. | Phase 2 | core media request resolver and provider media serializers | content/provider-media/provider package tests | `multimodal-content.md` | completed 2026-07-15 |
+| A-001 | Workflow coordinator concurrency test was timing-sensitive under load. | Phase 0 | test only; production workflow API unchanged | `packages/workflows/src/__tests__/coordinator.test.ts` repeated five times plus full suite | this page, `performance.md` | completed at frozen HEAD `f5128a8` |
+| A-002 | `AgentConfig.extensions`, `settings`, and `credentials` are accepted but explicitly ignored by agent/session runtime. | Phase 3 | core `AgentConfig` and host composition docs | compile fixtures and runtime behavior tests | `agent-session-runtime.md`, `customization.md`, `migration.md` | completed 2026-07-15 |
+| A-003 | `AgentSession.run()`/`prompt()` return `Promise<void>` and integrated streaming requires a separately started subscriber. | Phase 3 | core `AgentSession`, new run-result/stream contract | agent/session, CLI/RPC, workflow tests | agent/session, events, CLI/RPC, workflow docs | completed 2026-07-15 |
+| A-004 | Direct source/phase-text boundary tests make refactors fail without behavioral changes. | Phase 14 | test architecture; replace implementation assertions when touched, retain manifest/docs/absence boundary checks | docs/export/behavior/pack tests replacing implementation-text assertions | testing/release coverage docs | completed for touched tests; broad historical conversion deferred |
+| A-005 | `src/contracts.ts`, `src/agents.ts`, and `packages/workflows/src/run.ts` are conflict hotspots; docs/plans are larger than production source. | Phase 14 | maintainability gate; split only where completed work proves a cohesive domain | typecheck, public exports, behavior suites, docs links | review coverage and release docs | reviewed; no safe release-time cohesive split, deferred with measurements |
+| R-001 | 0.0.4 must not be published after the review; all package/version/tag/provenance checks must target 0.0.5. | Phase 14 | 30-package graph, lockfile, release workflow/script | release, packaging, install, Node 20/24, registry checks | `release-and-install.md`, changelogs, migration docs | completed 2026-07-16; publication handoff pending |
+
+## Accepted capability scope
+
+These are the twelve accepted Mastra-comparison recommendations. Each has one implementation owner; adjacent phases may consume the result but do not own a duplicate implementation.
+
+| ID | Capability | Single owner | Reused Prism primitives | Minimum missing surface | Test owner | Documentation owner |
+| --- | --- | --- | --- | --- | --- | --- |
+| F-001 | Direct run result and integrated stream | Phase 3 / core | `AgentSession`, loop, `AgentEvent`, bounded subscriber | `AgentRunResult`; race-free `session.stream()` wrapper over one execution | core agent/session suite | agent/session and events docs |
+| F-002 | Minimal scorers, sampling, datasets, batch experiments | Phase 4 / `@arnilo/prism-evals` (completed 2026-07-15) | F-001 result, run/trace IDs, package-local evaluation store | package-local scorer/dataset/experiment records and runner | eval package tests | `evaluations.md` |
+| F-003 | Minimal project scaffold | Phase 5 / existing root CLI (completed 2026-07-15) | CLI parser, examples, provider packages, pack smoke | deterministic `prism init`; tiny templates | CLI temp-project pack/install/typecheck/test | CLI, README, release/install docs |
+| F-004 | AI SDK model interoperability | Phase 6 / `@arnilo/prism-provider-ai-sdk` (completed 2026-07-15) | `AIProvider`, provider events, transport/content/structured-output conformance | one supported AI SDK language-model adapter | provider conformance with fake AI SDK model | provider package and AI SDK page |
+| F-005 | Working memory and semantic recall | Phase 7 / `@arnilo/prism-memory` (completed 2026-07-15) | context providers, middleware, ownership, Postgres package | package-local `Embedder`, `VectorStore`, working-memory store; one pgvector adapter | memory conformance and PostgreSQL opt-in suite | `working-and-semantic-memory.md` |
+| F-006 | Durable human approval and suspend/resume | Phase 8 / workflows plus coding-security bridge (completed 2026-07-15) | checkpoints, leases, fencing, workflow resume/cancel, execution policy | suspended state, validated payload, exact-once resume cursor | workflow restart/race/authorization tests | workflow and coding-security docs |
+| F-007 | Small text/Markdown RAG | Phase 9 / `@arnilo/prism-rag` (completed 2026-07-16) | Phase 7 embed/vector contracts, resource/content bounds, context provider | deterministic chunk/index/retrieve/citation helpers | RAG package tests | `rag.md` |
+| F-008 | Web-standard agent/workflow handler and MCP server | Phase 10 / `@arnilo/prism-server` plus existing MCP package (completed 2026-07-16) | F-001 result/stream, workflow commands, MCP SDK, abort/redaction | `Request -> Response` handler; explicit MCP server registration | server and MCP in-memory tests | `server.md`, `mcp-tools.md` |
+| F-009 | Durable schedules and reconnectable background runs | Phase 11 / workflows (completed 2026-07-16) | coordinator, checkpoints, leases, active-run registry | schedule records/claims and one-time/interval/host-calculated next run | workflow multi-coordinator tests | workflow/persistence/server docs |
+| F-010 | Workflow composition, state, and replay | Phase 11 / workflows (completed 2026-07-16) | DAG runner, node adapters, checkpoints, lineage/events | workflow node, bounded typed state, replay lineage/cursor | workflow nested/replay tests | workflow docs |
+| F-011 | Trace/run feedback linked to evaluations | Phase 12 / persistence plus OTel/evals (completed 2026-07-16) | run/trace IDs, cursor queries, redaction, OTel | bounded feedback record/query and safe OTel projection | persistence conformance and OTel tests | observability/evaluation docs |
+| F-012 | Supervisor delegation and A2A | Phase 13 / `@arnilo/prism-supervisor` (completed 2026-07-16) | F-001, memory scope, permission/abort/budget, web handler | bounded child delegation and current A2A protocol mapping/signing | supervisor isolation and A2A protocol tests | `supervisors.md`, `a2a.md` |
+
+## Existing primitive inventory
+
+| Domain | Existing reusable primitive | What it already covers | Frozen gap/decision |
+| --- | --- | --- | --- |
+| Agent execution | `Agent`, `AgentSession`, `AgentLoopStrategy`, `singleShotLoop`, generate/validate/revise loop | provider/tool turns, retry, compaction, abort, stores, middleware, ledger | Add only F-001 result/stream in Phase 3; do not add a second engine or central application object. |
+| Events and fan-in | `AgentEvent`, `createEventMultiplexer<T>()`, bounded `subscribe()` queues | normalized live events, ordered source fan-in, overflow policies | Reuse for server, workflow, and supervisor streams. Durable replay remains storage-owned. |
+| Provider boundary | `AIProvider.generate()`, `ProviderEvent`, request policies, bounded SSE/body/argument transport, provider/media/openai helpers | first-party model streams, tool fragments, structured output, abort, metadata | AI SDK is an optional adapter only; no new provider abstraction. |
+| Input/context | `InputBuilder`, `PromptBuilder`, `ContextProvider`, instruction injectors, resource loader, ten middleware hook names | explicit context/prompt assembly and host contributions | Memory and RAG inject through context/middleware; no memory-specific core hook. |
+| Tools | `ToolDefinition`, validator, registry/filter, permission policy, `ExecutionPolicy`, bounded parallel dispatch | schema validation, allow/deny, ownership metadata, ordered tool transcript | Fix policy propagation/identity; durable approval extends workflow checkpoints rather than tool loop internals. |
+| Redaction | `SecretRedactor`, message/event/request/session/ledger redactors | active-path cycle handling, object/Map key redaction, metadata-safe errors | Reuse at every new storage/remote boundary; no second redaction system. |
+| Session/run persistence | `SessionStore`, `RunLedger`, `ProductionPersistenceStore`, cursor pages, ownership scope | sessions, branches, entries, runs, events, tool calls, usage, definitions, retention, migrations | Add narrowly typed records only when package-local stores cannot preserve cross-package run linkage. Vector search stays outside `SessionStore`. |
+| Durable coordination | `CheckpointStore`, `LeaseStore`, fencing tokens; memory/SQLite/PostgreSQL implementations | versioned CAS state, atomic claims, expiry/takeover | Reuse for suspend, schedules, background runs, and replay. No new queue/lock engine. |
+| Workflows | bounded DAG with agent/function/tool/conditional/fan-out/join nodes; checkpoint/resume/cancel/coordinator/RPC | deterministic local and multi-process orchestration | Extended for suspension, schedules/background runs, composition/state/replay without a durable-agent engine. |
+| MCP | client bridge over SDK stdio and Streamable HTTP; list cache, timeout, abort, bounded results | consuming remote MCP tools | Add explicit server exposure in existing package; expose nothing by default. |
+| CLI/RPC | `prism` CLI, LF-delimited RPC, `CommandDefinition`, workflow commands | host-controlled run and durable workflow operations | Add stdlib-only `init`; server remains optional package-owned. |
+| Testing | mock provider plus provider/session/ledger/compaction/tool/extension/persistence conformance | network-free adapter and behavior checks | Add package-specific conformance only for genuinely reusable memory/eval/server seams. Avoid new source-text boundary tests. |
+| Packaging | root plus 29 workspaces; 23 capability packages and six profile bundles | independent installation, dependency-ordered release, pack/import smoke | `prism-all` reaches all 30 packages; provider umbrella reaches all seven adapters; focused profiles stay unchanged. Core keeps zero runtime dependencies. |
+
+## Current package and export inventory
+
+The publishable graph is root plus 28 workspaces (29 packages). Every package remains explicit; no package discovery exists.
+
+| Export group | Current public surface |
+| --- | --- |
+| Root runtime | `@arnilo/prism` |
+| Root provider helpers | `@arnilo/prism/providers/openai-compatible`, `/transport`, `/openai`, `/media` |
+| Root testing helpers | `/testing/provider-conformance`, `/session-store-conformance`, `/compaction-conformance`, `/tool-conformance`, `/extension-conformance`, `/persistence-schema`, `/run-ledger-conformance` |
+| Root Node helpers | `/node/config`, `/settings`, `/trust`, `/session-store-jsonl`, `/contribution-discovery`, `/instruction-injectors`, `/system-prompts`, `/agent-definitions` |
+| Provider packages | `prism-provider-openai`, `-opencode-go`, `-openrouter`, `-zai`, `-kimi`, `-neuralwatt` |
+| Compaction packages | `prism-compaction-llm`, `prism-compaction-observational-memory` |
+| Optional feature packages | `prism-observability-opentelemetry`, `prism-tool-validator-json-schema`, `prism-mcp`, `prism-coding-agent`, `prism-coding-security`, `prism-session-store-sqlite`, `prism-session-store-postgres`, `prism-credentials-node`, `prism-workflows`, `prism-evals`, `prism-provider-ai-sdk`, `prism-memory`, `prism-rag`, `prism-server`, `prism-supervisor` |
+| Profile bundles | `prism-all`, `prism-base`, `prism-code`, `prism-compaction`, `prism-providers`, `prism-sdk` |
+
+Core middleware exposes exactly ten built-in hook names: `provider_request`, `input_assembly`, `prompt_build`, `context`, `tool_call`, `tool_result`, `retry`, `compaction`, `session_start`, and `session_shutdown`. Custom string hooks remain extension-owned. New phases reuse these hooks or package APIs unless the owning phase documents a generic cross-package gap.
+
+## Package and primitive decisions by phase
+
+| Phase | Owner | Existing primitive consumed | Why it is insufficient | Minimum generic addition allowed |
+| --- | --- | --- | --- | --- |
+| 1 | core, coding-agent, coding-security | content policy, execution policy, approval policy | hostname normalization/DNS pinning and cache identity are incomplete | normalized/pinned network seam and execution run/session identity only |
+| 2 | core, coding-security, OTel, providers | bash callback, runtime events, usage rows, media resolver | callbacks/terminal states/accounting/request aggregation are incomplete | sandbox output callback; usage scope/aggregate; request-level media resolver |
+| 3 | core | session run, loop, events | no direct terminal result or integrated stream | `AgentRunResult` and one stream wrapper |
+| 4 | optional eval package | Phase 3 result, IDs, persistence | no scorer/dataset record or bounded runner | package-local contracts; generic persistence addition only for durable cross-package linkage |
+| 5 | root CLI | CLI and compile-checked examples | `prism init` + `templates/init` | templates and command only; no new library primitive |
+| 6 | optional AI SDK provider | `AIProvider` and provider conformance | AI SDK models do not implement Prism's interface | adapter only; no core change |
+| 7 | optional memory plus PostgreSQL adapter | context, ownership, middleware | no embedding/vector/working-memory contract | package-owned contracts and one production adapter |
+| 8 | workflows/coding-security | checkpoint/resume/lease/policy | completed: suspended/denied state, validated/redacted resume, expected-version CAS, tool policy recheck | workflow status/checkpoint extension; no generic scheduler |
+| 9 | optional RAG | Phase 7 and context/resource limits | completed: bounded text/Markdown chunk/index/filter/retrieve/citation/context helpers | package-only helpers |
+| 10 | optional server and MCP | Web APIs, result/stream, workflow commands, MCP SDK | completed: bounded authorized agent/workflow Web handler plus explicit MCP tool/command registration and SDK Web transport | package-owned handler/router and MCP registrations |
+| 11 | workflows/persistence | coordinator/checkpoint/lease/DAG | completed: ownership-scoped one-time/interval/calculated schedules, deterministic background enqueue, nested runner composition, bounded validated state/history, immutable replay lineage and fresh approval checks | package records over existing generic checkpoint/lease primitives; no migration or second worker engine |
+| 12 | persistence/evals/OTel | run/trace IDs and cursor queries | completed: immutable exact-owned feedback append/query/delete, ID-only evaluation linkage, migration-003 SQLite/PostgreSQL stores, safe span projection and fixed-label counters | one bounded typed record/store plus optional OTel handlers; no vendor exporter |
+| 13 | optional supervisor/A2A | session result, memory scope, policy, event multiplexer, Request/Response, WebCrypto | completed: explicit bounded child delegation, narrowing permissions/hooks, derived scopes, A2A 1.0 cards/ES256/JSON-RPC/SSE/exact-origin client | one zero-dependency optional package; text-only protocol subset |
+| 14 | release/docs/tests | deterministic release and pack checks | completed: graph/docs/changelogs retargeted to 0.0.5; packed cross-capability and release matrix evidence recorded | no release-time source split; all umbrella inclusion preserves independent direct installs and inert activation |
+
+## Threat-boundary matrix
+
+| Boundary | Untrusted input / asset | Primary risks | Required 0.0.5 controls | Owner |
+| --- | --- | --- | --- | --- |
+| Media network fetch | URL, hostname, DNS answer, response bytes/MIME | private-network access, rebinding, redirects, decompression/size abuse | normalize literals, classify all private ranges, resolve and pin public address or require allow-list/injected hardened loader, reject redirects, timeout/abort, per-item and whole-request bounds before provider side effects | Phases 1-2 |
+| Tool execution | model-produced name/arguments/path/command | shell execution, traversal/symlink escape, policy bypass, output exhaustion | schema/filter/permission/execution policy on every aggregator, realpath containment, approval, sandbox, abort, bounded/redacted output | Phases 1-2 |
+| Approval cache/resume | action and human decision | cross-run/session/tenant decision reuse, stale approval, duplicate side effect | default no cache, identity-bound keys, durable owner check, version/fencing/idempotency, policy recheck on resume | Phases 1 and 8 |
+| Remote HTTP/MCP | request body, identity headers, selected capability, stream consumer | unauthorized capability use, DoS, data leak, orphaned runs | expose nothing by default, host auth callback, owner checks, body/event/concurrency/time bounds, disconnect abort, redacted errors | Phase 10 |
+| Working/semantic memory and RAG | stored user text, embeddings, retrieved documents/metadata | cross-tenant recall, prompt injection, retention leak, unbounded context | mandatory tenant/resource/thread scope, bounded top-K/context, redaction, inert context, host retention/deletion | Phases 7 and 9 |
+| Workflow suspend/schedule/replay | resume payload, checkpoint, schedule, source run | forged resume, stale worker, duplicate fire/side effect, approval replay | schema validation, authorization, CAS/fencing, idempotency, immutable lineage, bounded state/history | Phases 8 and 11 |
+| Feedback/evaluation | comments, tags, scorer output, dataset items | PII/secret persistence, tenant leak, metric-cardinality explosion | ownership, redaction, limits, retention, no free text in metric labels, scorer isolation | Phases 4 and 12 |
+| Supervisor delegation | child prompt/result, selected child/tool, delegated memory | privilege amplification, recursive/cyclic delegation, budget exhaustion, memory/credential leak | implemented: explicit allow-list, AND-only permission narrowing, unique resources, depth/concurrency/token/time/tool limits, cancellation propagation | Phase 13 complete |
+| A2A | remote card, signature, task/message stream | impersonation, replay, malicious/oversized remote content, SSRF/credential forwarding | implemented: protocol-1.0 validation, host auth, ES256 signature/expiry verification, exact HTTPS origin allow-list/redirect rejection, bounds/timeouts/abort, untrusted output mapping | Phase 13 complete |
+
+## Capabilities already present — do not reimplement
+
+| Existing capability | Evidence | 0.0.5 rule |
+| --- | --- | --- |
+| Revision ownership/redaction fix and valid multi-round tool transcript | `src/agent-loops.ts`, `src/redaction.ts`, regression tests | Preserve; Phase 3 reuses loop output. |
+| Object and `Map` key redaction, active-path cycle handling | redaction suite | Use the same redactor everywhere. |
+| Bounded provider SSE, error body, argument parsing, retry, native structured output | provider primitives and conformance | Adapters reuse these helpers/limits. |
+| Polling OpenAI device OAuth | provider-openai OAuth tests | No new credential flow in core. |
+| JSON Schema tool validation, bounded parallel calls, exclusive tools | core/tool-validator and loop tests | Preserve shared dispatch. |
+| MCP client bridge | `@arnilo/prism-mcp` | Phase 10 adds server direction only. |
+| SQLite/PostgreSQL production persistence, checkpoints, leases | adapter conformance/live CI | Extend only required records/migrations. |
+| Encrypted file/keychain credentials | `@arnilo/prism-credentials-node` | Hosts keep explicit credential ownership. |
+| Audio/file/document blocks and MIME/size checks | core/provider media tests | Fix SSRF/request aggregation; do not add a second media model. |
+| OpenTelemetry agent/provider/tool instrumentation | optional OTel package | Correct lifecycle/accounting; no vendor matrix. |
+| Bounded DAG workflows and distributed coordinator | workflows package | Extend the same engine. |
+| Deterministic resumable package release | `scripts/release.mjs`, release workflow/tests | Retarget to 0.0.5 only after all gates. |
+
+## Explicit exclusions
+
+0.0.5 does not include Studio/editor/cloud services, browser automation, voice providers, chat-channel adapters, framework-specific HTTP adapters, auth-provider packages, deployment-provider packages, vendor observability exporters, a vector-store matrix, advanced document parsers/GraphRAG, an interactive TUI, automatic discovery/activation, or a mandatory central application object.
+
+A cron-expression parser is also excluded: Phase 11 provides one-time timestamps, fixed intervals, and a host-supplied next-run calculator. Add a cron adapter only from a concrete requirement.
+
+## Frozen baseline summary
+
+Detailed commands and measurements are in [Performance limits](performance.md#005-phase-0-baseline-2026-07-15).
+
+| Area | Frozen value |
+| --- | --- |
+| Runtime/toolchain | Node 24.18.0 measurement host; npm 11.16.0; supported runtime remains Node >=20 |
+| Full network-free tests | 1,475 total; 1,450 pass; 25 explicit live skips; 0 fail; 25.750 s |
+| `sdk:ready` | pass; 54.341 s |
+| Publishable graph | 24 packages at Phase 0 freeze; 25 after Phase 4 (evals); 26 after Phase 6 (AI SDK); 27 after Phase 7 (memory); 28 after Phase 9 (RAG); 29 after Phase 10 (server); 30 after Phase 13 (supervisor/A2A). Phase 14 follow-up puts all 30 behind `prism-all` and all seven adapters behind `prism-providers`. |
+| Tarballs | 542,993 packed bytes / 2,084,900 unpacked bytes aggregate; root 346.0 kB / 1.3 MB |
+| Installed workspace | 72 MiB `node_modules` |
+| Source/tests | 189 production TypeScript files / 26,828 lines; 144 test files / 23,535 lines |
+| Docs/plans/examples | 70 docs / 12,662 lines; 58 plans / 24,270 lines; 39 examples / 3,134 lines |
+| Current generator | `prism init`; default sources ~3.3 KB / 8 files; default clean install ~27.5 MB vs Mastra 439 MB |
+| Mastra comparator | default scaffold measured 439 MB `node_modules`, 300 MB build output, 427 installed packages |
+
+## Phase 0 verification
+
+Executed from repository root at frozen HEAD:
+
+```bash
+npm test
+npm run sdk:ready
+for i in 1 2 3 4 5; do node --test packages/workflows/dist/__tests__/coordinator.test.js; done
+node /tmp/prism-phase0-bench.mjs
+```
+
+Results:
+
+- Pre-change baseline and post-documentation full checks passed with zero failures. Final `sdk:ready` completed in 57.764 s with all 24 dry-run packs; the frozen pre-change value remains 54.341 s.
+- Coordinator suite passed five consecutive runs after the pre-freeze test correction.
+- Traceability validation found 14 unique confirmed-finding IDs, 12 unique capability IDs, all roadmap phases 0-14, and no missing or duplicate owner.
+- The benchmark script was temporary because these measurements are dated evidence, not CI wall-clock assertions.
+
+## Documentation reviewed
+
+- Local: `docs/index.md`, `public-contracts.md`, `agent-session-runtime.md`, `agent-loops.md`, `runs-and-usage.md`, `workflows.md`, `workflow-orchestration-primitives.md`, `host-security.md`, `release-and-install.md`, `performance.md`, and `review-coverage-2026-07-14.md`.
+- Source: core contracts/runtime/middleware/input/tools/content/redaction/persistence/checkpoint/lease/provider primitives, workflow package, optional package manifests/exports, and all seven testing conformance helpers.
+- Comparison: Mastra repository commit `2745031d1d4a4978f037092da371428c32e2842a` and current docs/source reviewed on 2026-07-15 for agents, memory, RAG, workflows, evals, observability, server, MCP, scheduling, supervisors, and A2A.
+
+## Related pages
+
+- [Roadmap](../roadmap.md): executable Phase 0-14 plan and release gate.
+- [Review coverage — 2026-07-14](review-coverage-2026-07-14.md): evidence for capabilities already shipped before this review.
+- [Performance limits](performance.md): dated baseline details and existing runtime limits.
+- [Host security](host-security.md): current host responsibilities pending the Phase 1-2 corrections.
+- [Release and install](release-and-install.md): current package graph and deterministic publication process.

package/docs/review-coverage-2026-07-17-provider-validation.md

@@ -0,0 +1,192 @@
+# Review coverage — 2026-07-17 provider validation
+
+Working evidence page for Plan 067. Freezes the 2026-07-14 P0–P2 re-verification map, first-party provider validation owners, official-doc priority URLs, Pi secondary references, cache/thinking/discovery surfaces, credential notes, and use-case model-binding inventory.
+
+**Evidence frozen:** 2026-07-17 (offline inventory; no live provider calls).
+**Priority rule:** official provider documentation wins; Pi (`badlogic/pi-mono`) is secondary when official docs are silent or ambiguous.
+
+Related: [2026-07-14 coverage](review-coverage-2026-07-14.md) (0.0.4), [2026-07-15 coverage](review-coverage-2026-07-15.md) (0.0.5), [provider caching](provider-caching.md), [provider packages](provider-packages.md).
+
+## Status legend
+
+| Status | Meaning |
+| --- | --- |
+| `verify` | Prior plan marked fixed; Plan 067 must re-verify with regression tests. |
+| `gap` | Known missing or incorrect behavior/docs relative to official sources. |
+| `fixed` | Closed in this plan (updated as tasks complete). |
+| `by-design` | Intentional absence (document; do not “fix” into a catalog). |
+
+## P0–P2 finding → Plan 067 owner matrix
+
+Source: `code-reviews/2026-07-14.md`. Prior Plans 053 / 054 / 058 marked these implemented for 0.0.4; this plan re-verifies rather than skipping.
+
+| Review ID | Priority | Finding | 0.0.4 owner | Plan 067 task | Current status | Credential / redaction notes |
+| --- | --- | --- | --- | --- | --- | --- |
+| R-001 | P0 | Revision request duplicated + corrupted by redaction | 053-1 | 1 | fixed | Secret canaries must not appear in repair requests, events, or redacted graphs. Re-verified 2026-07-17: `pendingHistory` + active-path redaction; revision+redactor suite asserts one repair and no `[Circular]`. |
+| R-002 | P1 | Multi-round tool transcript chronologically invalid | 053-2 | 1 | fixed | N/A. Re-verified 2026-07-17: two rounds × two calls keep `user → assistant → tool → tool → …` order in history and assembled request. |
+| R-003 | P1 | Redactor leaks secrets in object/Map keys | 053-1 | 1 | fixed | Object/Map string keys redact; collisions use deterministic `__N` suffixes. |
+| R-004 | P1 | Event-ledger writes lack backpressure | 053-3 | 1 | fixed | Ledger appends serialized (concurrency 1), order preserved, append failures reject run completion. |
+| R-008 | P1 | Unbounded SSE / error bodies; multiline `data:` | 054-1/2 | 2 | fixed | Bounded readers; error text redacted. |
+| R-009 | P1 | OpenAI device-code OAuth does not poll | 054-3 | 2 | fixed | Redact device/user/access/refresh codes from OAuth errors. |
+| R-010 | P2 | Duplicated provider protocol utilities | 054-1/2 | 2 | fixed | Shared transport/primitives remain authoritative. |
+| R-005 | P2 | JSONL append silent on corrupt lines | 053-4 | 2 | fixed | Dev-only store; fail closed on corrupt lines. |
+| R-011 | P2 | Coding-agent image read unbounded / resize no-op | 055-5 | 2 | fixed | Enforce `maxImageBytes`; deprecate `autoResizeImages` honestly. |
+| R-006 | P2 | Optional config ENOENT detected by message text | 053-5 | 2 | fixed | Typed `code === "ENOENT"`. |
+| R-012 | P2 | Release tag/version mismatch; no provenance | 058-7/9 | 2 | fixed | Tag/version gate + `--provenance`; no secrets in artifacts. |
+
+Bug-report fixes A–D remain covered by R-001 / R-003 / R-007 (malformed message shape was Plan 053; re-check under Task 1 if touched).
+
+## Provider package validation matrix
+
+| Package | Plan 067 task | Official cache | Prism `cache.kind` | Thinking / reasoning (official → Prism) | Discovery endpoint | Static catalog (bootstrap only) | Pi secondary ref | Credential surface | Status |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| `@arnilo/prism-provider-openai` | 6 | `prompt_cache_key`; older models `prompt_cache_retention` (`24h` / `in_memory`); GPT-5.6+ `prompt_cache_options` / breakpoints tracked (host may pass via compat/extra; discovery sets `longRetention: false`) | `openai_key` (+ `longRetention`, `maxKeyLength`) | Official: Responses top-level `reasoning.effort` (+ `summary`). Prism: merges `model.compat.reasoning` + `options.compat.reasoning` (request wins); `applyThinkingLevel(..., "openai_reasoning")` | Official: `GET /models` | Featured `openAIModels` / `openAICodexModels` + **`listOpenAIModels`**; factory accepts `models?` / `codexModels?` | `packages/ai/src/api/openai-responses.ts`, `openai-responses-shared.ts` | API key; Codex OAuth device-code (`oauth.ts`) — poll + redact codes | **fixed** 2026-07-17: Responses P0s + discovery + reasoning + models override |
+| `@arnilo/prism-provider-kimi` | 7 | Anthropic-style `cache_control` on `/messages` when supported; OpenAI Moonshot route none | default implicit; opt-in `cache_control` | Official K2.x: `thinking.type` (`enabled`/`disabled`/`keep`); K2.7-code always enabled; K3: top-level `reasoning_effort`. Prism: `kimiThinking`/`kimiReasoningEffort`/`kimiPreserveThinking` (request wins); Coding Anthropic + Moonshot Chat Completions both callable | Official: `GET https://api.moonshot.ai/v1/models` (also `.cn`); Coding has no public list API | Featured Coding (`kimi-for-coding`, `kimi-for-coding-highspeed`, `k3`) + Moonshot (`kimi-k2.7-code`, `kimi-k3`) + **`listKimiModels`** | `kimi-coding*.ts`, `moonshotai*.ts`, `api/anthropic-messages.ts` | API key (`KIMI` / Moonshot, not interchangeable); redacted from errors | **fixed** 2026-07-17: discovery + Moonshot provider + thinking + official Coding ids |
+| `@arnilo/prism-provider-zai` | 8 | Implicit (no client `cache_control` / `prompt_cache_key`); official `usage.prompt_tokens_details.cached_tokens` | `implicit` | Official: `thinking` (`{type, clear_thinking?}`), `reasoning_effort` (GLM-5.2+), `tool_stream` (GLM-4.6+). Prism: `zaiThinking`/`zaiReasoningEffort`/`zaiToolStream`/`zaiClearThinking`/`zaiPreserveThinking` (request wins); Preserved Thinking replays `reasoning_content`. Obsolete `thinkingFormat`/`developerRoleFallback` docs removed | No first-class docs.z.ai list page; OpenAI-compatible `GET {baseUrl}/models` best-effort + curated featured set from Chat Completions enum / overview | Featured `zaiModels` (`glm-5.2`…`glm-4.5`) + **`listZaiModels`**; default base `https://api.z.ai/api/paas/v4` | `zai.ts`, `zai.models.ts` (Pi secondary ids only) | API key; redacted from errors / discovery | **fixed** 2026-07-17: docs drift closed; catalog + discovery + clear_thinking/preserve |
+| `@arnilo/prism-provider-openrouter` | 9 | Official prompt caching via `cache_control` + sticky `session_id` routing; top-level automatic when no breakpoints | `cache_control` (legacy `compat.openRouterCache`) | Official: `reasoning: { effort | max_tokens }`. Prism: `resolveOpenRouterReasoning` merge + `preserveThinking` replay as body `reasoning` | Official: `GET https://openrouter.ai/api/v1/models` | **App-controlled** `models:` + optional **`listOpenRouterModels`** (no bundled mega-catalog) | `openrouter.models.ts` (do not vendor), `api/openai-completions.ts` | API key; sanitize session/cache ids | **fixed** 2026-07-17: discovery + reasoning merge/preserve + automatic top-level cache_control |
+| `@arnilo/prism-provider-opencode-go` | 10 | Anthropic route: selected `cache_control`; OpenAI route: none; `x-opencode-session` from cache/session key | route-specific (`cache_control` on Anthropic; `implicit` on OpenAI) | Dual-route: Anthropic thinking blocks + OpenAI `reasoning_content`; upstream `thinking`/`reasoning_effort`/`reasoning` passthrough (request wins); `preserveThinking` default for reasoning models | Official `GET https://opencode.ai/zen/go/v1/models` (sparse) | Featured official Go ids (Grok/GLM/Kimi/MiMo/MiniMax/Qwen/DeepSeek) + **`listOpenCodeGoModels`**; default base `https://opencode.ai/zen/go/v1` | `opencode-go.ts`, `opencode-go.models.ts` (Pi secondary ids/limits only) | API key; redacted from errors / discovery | **fixed** 2026-07-18: catalog + discovery + base URL + thinking preserve |
+| `@arnilo/prism-provider-neuralwatt` | 11 | Implicit vLLM prefix caching; `prompt_tokens_details.cached_tokens` | `implicit` | Official: `reasoning_effort` when `capabilities.reasoning_effort` (GLM-5.2 default `max`); `thinking_token_budget`; `chat_template_kwargs` (`preserve_thinking`/`clear_thinking`/`enable_thinking`). Prism: `thinking.ts` resolves owned fields + `stripNeuralWattOwnedCompat`; `applyThinkingLevel(..., "reasoning_effort")`; Preserved Thinking replays `reasoning_content` | Official: `GET https://api.neuralwatt.com/v1/models` (auth optional for public models) | Featured `neuralWattModels` (official aliases incl. `gemma-4-31b`; no guessed pricing) + **`listNeuralWattModels()`** | **No Pi NeuralWatt provider** — official docs only | Optional API key on discovery; quota helper; redact secrets in error bodies | **fixed** 2026-07-18: catalog refresh + kwargs routing + owned-compat strip |
+| `@arnilo/prism-provider-ai-sdk` | 12 | Host model owns request caching; adapter maps usage only | host-owned / N/A | Host `LanguageModelV4` owns reasoning; Prism maps stream `reasoning` parts | **None by design** (host supplies model) | None | N/A (AI SDK official spec > Pi) | No package credentials; host model may hold secrets | **fixed** 2026-07-18: host-owned catalog/cache/reasoning validated; usage mapping + docs |
+
+## Frozen official evidence sources (priority)
+
+| Provider | Frozen URLs (official) | Notes frozen 2026-07-17 |
+| --- | --- | --- |
+| OpenAI | [List models](https://developers.openai.com/api/reference/resources/models/methods/list); [Prompt caching](https://developers.openai.com/api/docs/guides/prompt-caching); [Reasoning](https://developers.openai.com/api/docs/guides/reasoning); [Responses create](https://developers.openai.com/api/reference/resources/responses/methods/create/); [Models guide](https://developers.openai.com/api/docs/models) | `GET /models`. Caching: `prompt_cache_key`; pre-5.6 `prompt_cache_retention`; 5.6+ `prompt_cache_options` / breakpoints. Reasoning: `reasoning.effort`. |
+| Kimi / Moonshot | [List models](https://platform.kimi.ai/docs/api/list-models); [API overview](https://platform.kimi.ai/docs/api/overview); [Model parameter reference](https://platform.kimi.ai/docs/api/models-overview); [Thinking mode](https://platform.kimi.ai/docs/guide/use-kimi-k2-thinking-model); [Thinking effort](https://platform.kimi.ai/docs/guide/use-thinking-effort) | `GET /v1/models` on `api.moonshot.ai` / `.cn`. K2.x `thinking`; K3 `reasoning_effort: "max"`. Anthropic `/messages` compat remains under-documented — record empirical gaps in Task 7. |
+| Z.AI | [Deep thinking](https://docs.z.ai/guides/capabilities/thinking); [Thinking mode](https://docs.z.ai/guides/capabilities/thinking-mode); [Tool streaming](https://docs.z.ai/guides/capabilities/stream-tool); [Context caching](https://docs.z.ai/guides/capabilities/cache); [Chat completion](https://docs.z.ai/api-reference/llm/chat-completion); [Migrate to GLM-5.2](https://docs.z.ai/guides/overview/migrate-to-glm-new); [Overview](https://docs.z.ai/guides/overview/overview) | Code + `docs/providers/zai.md` match official `thinking` / `reasoning_effort` / `tool_stream` / `clear_thinking`. Historical mismatch (`thinkingFormat`) closed in Task 8. |
+| OpenRouter | [Get models](https://openrouter.ai/docs/api/api-reference/models/get-models); [Prompt caching](https://openrouter.ai/docs/guides/best-practices/prompt-caching); [Reasoning tokens](https://openrouter.ai/docs/guides/best-practices/reasoning-tokens) | `GET /api/v1/models`. Cache via `cache_control` + sticky routing. Reasoning via `reasoning` object. |
+| OpenCode Go | [Go](https://opencode.ai/docs/go/); [Providers](https://opencode.ai/docs/providers/) | Dual OpenAI + Anthropic compatible APIs. Official `GET /zen/go/v1/models`. Featured: Grok 4.5, GLM-5.2/5.1, Kimi K3/K2.7 Code/K2.6, MiMo, MiniMax, Qwen3.7/3.6, DeepSeek V4. Default base `https://opencode.ai/zen/go/v1`. |
+| NeuralWatt | [Models](https://portal.neuralwatt.com/docs/api/models); [Chat completions](https://portal.neuralwatt.com/docs/api/chat-completions); [API overview](https://portal.neuralwatt.com/docs/api/overview); [Quickstart](https://portal.neuralwatt.com/docs/quickstart) | `GET /v1/models` returns pricing/capabilities/limits metadata. Prefix caching automatic; `reasoning_effort` when capability flagged. |
+| AI SDK | [Custom provider / LanguageModelV4](https://ai-sdk.dev/providers/community-providers/custom-providers); AI SDK usage types (`inputTokens` / cache details) | Adapter validates `specificationVersion`; maps cache read/write from usage. No Prism catalog. |
+
+## Frozen Pi secondary references
+
+Repo: `https://github.com/badlogic/pi-mono` (`packages/ai`).
+
+| Area | Path / note |
+| --- | --- |
+| OpenAI Responses | `packages/ai/src/providers/openai-responses.ts` (also Codex / Azure variants) |
+| OpenAI Completions | `packages/ai/src/providers/openai-completions.ts` / `api/openai-completions.ts` |
+| Anthropic Messages | `packages/ai/src/providers/anthropic-messages.ts` / `api/anthropic-messages.ts` |
+| Generated catalogs | `packages/ai/src/providers/*.models.ts` via `scripts/generate-models.ts` — **not** copied as Prism’s sole strategy |
+| Provider registry docs | Pi mintlify “LLM Providers” / README provider list |
+
+Use Pi only to fill gaps or cross-check wire shapes after official docs.
+
+## Shared discovery pattern (Task 3 decision — frozen)
+
+**Status (2026-07-17 Task 3):** pattern documented; no new core list-models primitive. Shared reuse is limited to existing transport/credential helpers.
+
+### Inventory
+
+| Primitive | Location | Role |
+| --- | --- | --- |
+| `listNeuralWattModels({ apiKey?, fetch?, baseUrl?, signal?, headers? })` | `packages/provider-neuralwatt/src/models.ts` | **Template** — only shipped `list*Models` today |
+| `mapNeuralWattModel(entry)` | same | Provider-specific entry → `ModelConfig` |
+| `neuralWattModels` featured aliases | same | Offline bootstrap; no guessed pricing |
+| `readBoundedResponseText` | `@arnilo/prism/providers/transport` | Bounded error-body read + optional secret redaction |
+| `resolveCredentialValue` / `redactSecrets` | `@arnilo/prism` | Auth resolution + error redaction |
+| Package `models?: readonly ModelConfig[]` | Kimi/Z.AI/OpenRouter/OpenCode Go/NeuralWatt/**OpenAI** factories | Host override of registered catalog |
+| OpenAI factory `models?` / `listOpenAIModels` | **done** Task 6 | Fixed |
+| OpenRouter / OpenCode Go `list*Models` | OpenRouter **`listOpenRouterModels` done** Task 9; OpenCode Go **`listOpenCodeGoModels` done** Task 10; Z.AI **`listZaiModels` done** Task 8; Kimi **`listKimiModels` done** Task 7 | Per-package work |
+| AI SDK catalog | N/A | Host-owned `LanguageModelV4` — no discovery export |
+
+### Decisions
+
+- Template options + return: NeuralWatt `listNeuralWattModels(...) → ModelConfig[]`.
+- **Never** call discovery from `create*ProviderPackage()` / extension setup.
+- Static catalogs = offline bootstrap / featured aliases only.
+- OpenRouter stays app-registration-first; **`listOpenRouterModels`** (done Task 9) feeds `models:` only.
+- AI SDK: no discovery export.
+- Prefer **package-local** helpers. Do **not** add a core model-discovery registry or OpenAI-compatible mega-mapper in Task 3. Extract a shared HTTP/list helper later only if ≥2 packages share identical parsing (unlikely: OpenRouter/NeuralWatt/OpenAI response shapes and cache/cost mapping diverge).
+- Discovery may populate `ModelConfig.cache` / `ModelConfig.cost` from live metadata when officially documented; static catalogs must not invent those fields.
+- Docs: [Provider packages — Caller-gated model discovery](provider-packages.md#caller-gated-model-discovery), [Provider caching — Discovery and live cache/cost metadata](provider-caching.md#discovery-and-live-cache-cost-metadata), [Provider conformance — Model discovery checklist](provider-conformance.md#model-discovery-checklist).
+
+## Shared thinking / per-turn override (Task 4 decision — frozen)
+
+| Layer | Contract |
+| --- | --- |
+| Model default | `ModelConfig.compat` (+ `capabilities.reasoning` where declared) |
+| Per-turn override | `ProviderRequestOptions.compat` via existing `mergeProviderRequestOptions` (request wins) |
+| Shared helpers | Core `applyThinkingLevel` / `thinkingCompatFor` / `thinkingFamilyForModel` → official compat fields; **not** a second options tree |
+| Families | `openai_reasoning` (`reasoning.effort`), `reasoning_effort`, `thinking_type` (`thinking.type`), `noop` (host-owned) — only shapes shared by ≥2 packages (or explicit no-op) |
+| Use-case wiring | LLM compaction + OM workers map `thinkingLevel` into `compat` via helpers (no longer inert `extra.thinkingLevel`) |
+| Docs | [Thinking and reasoning](thinking-and-reasoning.md) |
+
+**Decision (2026-07-17):** Implement thin core helpers; keep unique knobs (NeuralWatt budgets/kwargs, Kimi keep/all, Z.AI `tool_stream`) package-local. Core must not name forbidden provider literals (`openrouter`/`zai`/`kimi`/…). Hosts pick a family explicitly when inference is ambiguous. Per-provider first-class field hardening remains Tasks 6–12.
+
+## Use-case model binding inventory (Task 5 — done 2026-07-17)
+
+| Site | Current binding | Session fallback today? | Thinking path today |
+| --- | --- | --- | --- |
+| `AgentSession.run` / `RunOptions.model` | Per-run override; writes `model_change` | Explicit override of session model | Host `providerOptions` / `applyThinkingLevel` |
+| Observational memory workers | `resolveUseCaseModel({ configured: workerModel, sessionModel })` | **Yes** — host passes `sessionModel`; `requireExplicitModel` restores skip | `thinkingLevel` → `compat` via `applyThinkingLevel` |
+| LLM compaction | `resolveUseCaseModel({ configured: summaryModel, sessionModel: model })` | **Yes** — `model` is the fallback slot | `thinkingLevel` → `compat` via `applyThinkingLevel` |
+| Supervisor children | Child `Agent` owns its own `model` | Independent | Child config |
+| Declarative agents | `resolveAgentDefinition` model string / `ModelConfig` | Definition-scoped | Definition / run options |
+| Evaluations | `runOptions` including model | Eval-owned | Via run options |
+| Workflows / RPC / CLI | Pass-through `runOptions.model` | Caller-owned | Via run options |
+| Structured output | Reuses session/run model | Yes | Same as run |
+| Memory / RAG embedders | Host `Embedder` — separate from chat LLM | N/A (not chat) | N/A |
+
+**Decision (2026-07-17):** Core exports `UseCaseModelBinding`, `resolveUseCaseModel`, `resolveUseCaseModelBinding`, `useCaseCredentialProviderId`. Docs: [use-case-model-selection.md](use-case-model-selection.md). OM behavior change: session fallback when `sessionModel` supplied and worker model omitted; `requireExplicitModel` preserves historical `missing_model` skip.
+
+Desired Plan 067 outcome: every use-case accepts `{ provider, model, thinking }` with **explicit session-model fallback** when no use-case default is set — **done** for OM + LLM compaction; other sites documented as already-separate.
+
+## Known doc / code mismatches (frozen)
+
+| Item | Docs claim | Code / official | Owner task |
+| --- | --- | --- | --- |
+| Z.AI thinking | Was: `thinkingFormat: "zai"`, `developerRoleFallback` in `docs/providers/zai.md` | **fixed** Task 8 — docs+code match official `thinking` / `reasoning_effort` / `tool_stream` / `clear_thinking` | 8 (done) |
+| OpenAI reasoning | Was capabilities-only + opaque compat spread | Task 6 merges `model`/`options` `compat.reasoning` into body `reasoning`; Task 4 helper writes `compat.reasoning.effort` | 4 (done) / 6 (done) |
+| Kimi Moonshot | Was metadata-only (`provider: "moonshot"` without provider) | **fixed** Task 7: `createMoonshotProvider` registered when `includeMoonshotModels`; official Coding ids + `listKimiModels` | 7 (done) |
+| OpenCode Go catalog | Featured official Go open models + `listOpenCodeGoModels` | **done** Task 10 (removed stale `gpt-5.1-go` / `claude-sonnet-4.5-go`) | 10 (done) |
+| Z.AI catalog | Was: `glm-4.7` / `glm-4.5` only | **fixed** Task 8 — featured GLM-5.2…4.5 + `listZaiModels` | 8 (done) |
+| OM/LLM `thinkingLevel` + session fallback | Was inert `extra.thinkingLevel`; OM skipped without workerModel | Task 4 maps into `compat`; Task 5 session fallback + `requireExplicitModel` | 4 (done) / 5 (done) |
+
+## Credential and redaction canaries (per package)
+
+| Package | Secrets in flight | Redaction canary expectation |
+| --- | --- | --- |
+| openai | API key; OAuth device/user/access/refresh | Absent from errors, events, requests, discovery failures |
+| kimi | API key | Absent from errors / cache keys |
+| zai | API key | Absent from errors / model metadata |
+| openrouter | API key; session/cache ids | Sanitize length; never treat cache key as secret storage |
+| opencode-go | API key | Absent from errors |
+| neuralwatt | Optional API key; quota responses | Discovery/quota error bodies redacted |
+| ai-sdk | Host-owned | Adapter must not echo host secrets in Prism errors |
+
+No secrets are committed in this matrix.
+
+## Plan 067 task map (evidence owners)
+
+| Task | Owns |
+| --- | --- |
+| 0 | This page + evidence freeze — **done** |
+| 1 | R-001–R-004 re-verify — **fixed** 2026-07-17 |
+| 2 | R-005–R-006, R-008–R-012 re-verify — **fixed** 2026-07-17 |
+| 3 | Shared discovery pattern — **done** 2026-07-17 (package-local NeuralWatt template; no core list helper) |
+| 4 | Shared thinking / per-turn surface — **done** 2026-07-17 (core helpers + OM/LLM compat wiring; see thinking-and-reasoning.md) |
+| 5 | Use-case model selection + session fallback — **done** 2026-07-17 (`resolveUseCaseModel`, OM session fallback, use-case-model-selection.md) |
+| 6 | OpenAI validate/harden — **done** 2026-07-17 (Responses P0s, `listOpenAIModels`, `models?`, reasoning merge) |
+| 7 | Kimi validate/harden — **done** 2026-07-17 (`listKimiModels`, Moonshot provider, thinking, official Coding ids) |
+| 8 | Z.AI validate/harden — **done** 2026-07-17 (`listZaiModels`, GLM-5.x catalog, docs drift closed, clear_thinking/preserve) |
+| 9 | OpenRouter validate/harden — **done** 2026-07-17 (`listOpenRouterModels`, reasoning merge/preserve, automatic top-level `cache_control`) |
+| 10 | OpenCode Go validate/harden — **done** 2026-07-18 (`listOpenCodeGoModels`, official Go catalog, `zen/go/v1` base, thinking preserve) |
+| 11 | NeuralWatt validate/harden — **done** 2026-07-18 (featured catalog refresh, kwargs routing, owned-compat strip, applyThinkingLevel) |
+| 12 | AI SDK validate — **done** 2026-07-18 (host-owned catalog/cache/reasoning; usage mapping + docs) |
+| 13 | Cross-provider conformance + final verification — **done** 2026-07-18 (`sdk:ready`; 1,089 core tests + workspace suites + pack dry-runs) |
+
+Exact task titles live in `plans/067-provider-doc-validation-caching-discovery-and-review-hardening.md`.
+
+## Verification for this page
+
+- Final verification completed 2026-07-18 without live provider HTTP.
+- Lists all seven first-party provider packages; every provider row is `fixed` or intentional by-design behavior.
+- Lists every 2026-07-14 P0–P2 id (R-001–R-006, R-008–R-012); all are `fixed`.
+- Distinguishes official-doc priority vs Pi secondary.
+- `phase12-boundaries.test.ts` verifies all six HTTP package discovery exports and setup zero-fetch; AI SDK no-catalog behavior has its own adapter contract test.
+- `provider_validation_final_contract_covers_all_adapters_and_binding_sites` verifies provider docs, cache kinds, thinking rows, use-case sites, matrix statuses, and index navigation.
+- `npm run sdk:ready` passes: typecheck/build, 1,089 core tests, all workspace suites, packaging/provenance guards, and publish-graph pack dry-runs.
+- Linked from `docs/index.md` under Release and install.

package/docs/runs-and-usage.md

@@ -2,7 +2,7 @@
 
 ## What it does
 
-`RunLedger` is the host-implemented, write-only seam Prism uses to durably persist run metadata, agent events, tool calls, and usage during a `session.run()`. The runtime calls the adapter as each record becomes available; the adapter decides how to write it (SQL insert, NoSQL put, JSONL append, time-series batch, etc.).
+`RunLedger` is the host-implemented, write-only seam Prism uses to durably persist run metadata, agent events, tool calls, and usage during a `session.run()`. `RunFeedbackStore` is the separate post-run seam for immutable ratings, comments, tags, and evaluation links. The runtime calls the adapter as each record becomes available; the adapter decides how to write it (SQL insert, NoSQL put, JSONL append, time-series batch, etc.).
 
 APIs:
 
@@ -13,6 +13,7 @@ APIs:
 - `ToolCallRecord` / `ToolCallStatus`
 - `UsageRecord`
 - `redactRunLedgerRecord()`
+- `RunFeedbackRecord` / `RunFeedbackStore` / `createMemoryRunFeedbackStore()`
 
 ## When to use it
 
@@ -37,7 +38,7 @@ Set the ledger and optional ownership scope/idempotency key on the agent or the
 | `appendRun` | `RunRecord` | After run starts (`running`) and again at finish (`succeeded`/`failed`/`aborted`). |
 | `appendEvent` | `AgentEventRecord` | After every emitted `AgentEvent`, after redaction. |
 | `appendToolCall` | `ToolCallRecord` | For each tool-call `started`, `progress`, `finished`, `error`, and `blocked` transition. |
-| `appendUsage` | `UsageRecord` | For each provider `usage` event and for the final loop usage. |
+| `appendUsage` | `UsageRecord` | Once per terminal provider turn (`scope: "provider_turn"`) and once for the O(turns) aggregate (`scope: "run_total"`). |
 
 All methods may be sync or async (`void | Promise<void>`). The runtime awaits them at safe boundaries, so a slow adapter blocks the run.
 
@@ -92,9 +93,40 @@ The adapter receives these record shapes:
 | --- | --- |
 | `id` | Unique ledger row id. |
 | `runId` / `sessionId` / `entryId` | Correlation ids. |
+| `scope` | `provider_turn` for billable source rows; `run_total` for the aggregate. Never sum both scopes. |
+| `turn` / `attempt` | Provider-turn attribution; absent on `run_total`. |
 | `usage` | `Usage` shape: input/output/total/cache tokens, cost, currency. |
 | `recordedAt` | ISO timestamp. |
 
+## Run/trace feedback
+
+`RunFeedbackStore.append()` accepts an immutable record only when `resolveRun` finds the same `runId` under the exact `{ tenantId, accountId?, userId? }` scope. A tenant plus account or user is mandatory. Records contain `sessionId`, optional `traceId`, finite `rating` in `[-1, 1]`, comment, tags, scorer IDs, evaluation IDs, timestamp, creator, and metadata. Correction appends a new ID; records are never updated in place. `delete()` is the explicit privacy/retention operation.
+
+```ts
+import { createMemoryRunFeedbackStore } from "@arnilo/prism";
+
+const feedback = createMemoryRunFeedbackStore({
+  resolveRun: ({ runId }) => runId === result.runId
+    ? { runId, sessionId: result.sessionId, tenantId: "t1", userId: "u1" }
+    : false,
+  redactor,
+});
+await feedback.append({
+  id: "fb_1",
+  runId: result.runId,
+  rating: 1,
+  comment: "Useful and cited",
+  tags: ["reviewed"],
+  evaluationIds: ["eval_1"],
+  tenantId: "t1",
+  userId: "u1",
+});
+const page = await feedback.query({ runId: result.runId, tenantId: "t1", userId: "u1", limit: 50 });
+await feedback.delete({ id: "fb_1", tenantId: "t1", userId: "u1" });
+```
+
+Default/hard bounds: comment 4/16 KiB, tags 16/64, scorer/evaluation IDs 16/64 each, metadata 16/64 KiB, query page 100/500; tags are 64 characters and identifiers 128. The store redacts comment/tags/metadata after run ownership validation and before persistence. IDs are linked, not scorer payloads. `ProductionPersistenceStore.feedback?` exposes this capability; first-party SQLite/PostgreSQL adapters implement it in schema migration `003_run_feedback` and reject missing/cross-owned runs.
+
 ## Status transitions
 
 ```
@@ -189,7 +221,9 @@ await session.run("Hello", {
 });
 
 console.log(runs.at(-1)?.status); // succeeded
-console.log(cacheUsageReport(usageRows.at(-1)?.usage));
+const billable = usageRows.filter((row) => row.scope === "provider_turn");
+const aggregate = usageRows.find((row) => row.scope === "run_total");
+console.log(cacheUsageReport(aggregate?.usage));
 // { cacheReadTokens: 0, cacheWriteTokens: 0, ... } when provider usage is present
 ```
 
@@ -207,7 +241,8 @@ console.log(cacheUsageReport(usageRows.at(-1)?.usage));
 - `AgentConfig.ownership` is the default ownership scope; `RunOptions.ownership` overrides it per run.
 - `AgentConfig.idempotencyKey` is the default idempotency key; `RunOptions.idempotencyKey` overrides it per run.
 - The runtime resolves `model` and `provider` from `AgentConfig`/`RunOptions`/`AgentDefinition` before writing the start `RunRecord`.
-- Adapters should treat appends as ordered within a `runId`: event and tool-call rows preserve emission order because the runtime drains pending appends before writing the final `RunRecord`.
+- Adapters should treat appends as ordered within a `runId`: event and tool-call rows preserve emission order because the runtime serializes event ledger appends through one promise chain (concurrency 1), drains pending appends before writing the final `RunRecord`, and propagates append failures by rejecting run completion.
+- Billing queries must filter `scope = "provider_turn"`; presentation queries normally read the single `run_total`. `UsageQuery.scope`, `turn`, and `attempt` are explicit filters.
 - Adapters that need upsert semantics can use `RunRecord.id` (== `runId`) as the stable key.
 - Use `cacheUsageReport(record.usage, model)` for cache diagnostics from normalized usage. It works when a provider reports `cacheReadTokens` without `cacheWriteTokens`; missing write tokens are reported as `0`, and unavailable hit rate/savings stay `undefined`.
 - **Provider-specific telemetry is package-owned.** Core `Usage` carries token counts and `cost`/`currency`; it has no energy or detailed cost-breakdown fields. Providers that surface extra telemetry (e.g. `@arnilo/prism-provider-neuralwatt` exposes `neuralWattEventsWithTelemetry()`, `parseNeuralWattComment()`, and `mapNeuralWattTelemetry()` for `: energy`/`: cost` SSE comments and non-streaming top-level fields) keep that data in package-specific helpers/types. Telemetry never enters `RunLedger` usage rows unless the host explicitly copies it in; it carries usage/cost numbers only — never prompts, API keys, or headers. Account-level quota is likewise package-owned: `@arnilo/prism-provider-neuralwatt` exports an explicit `getNeuralWattQuota()` helper that the host calls on demand (never during generation); NeuralWatt rate-limits that endpoint to 1 request per second per customer, so the caller owns throttling.
@@ -219,9 +254,11 @@ console.log(cacheUsageReport(usageRows.at(-1)?.usage));
 - **Redaction.** The runtime calls `redactRunLedgerRecord()` and `redactAgentEvent()` with the active `SecretRedactor` before handing records to the adapter. `AgentEventRecord.redacted` and `ToolCallRecord.redacted` are set to `true` when a redactor is configured. Hosts should still redact before writing to durable storage if they perform additional transformations.
 - **Message content stays in `SessionStore`.** `AgentEventRecord.event` may contain `message_delta` / `message_finished` payloads; these are redacted but still belong conceptually to the session store. Do not use the ledger as the source of truth for messages.
 - **Cache diagnostics stay numeric.** `cacheUsageReport()` derives reports from `Usage` numbers and optional `ModelConfig.cost`; do not add prompt text, cache keys, headers, credentials, or provider payloads to usage rows.
+- **No double billing.** Sum `provider_turn` rows or read `run_total`; never sum both. Run totals add every turn/attempt in O(turns), derive missing per-turn totals from input/output tokens, and omit aggregate cost when reported currencies conflict.
 - **Synchronous adapters block the run.** An adapter that performs network or heavy DB writes inline will slow down the agent loop. For high-throughput hosts, buffer or batch inside the adapter and return quickly; the runtime awaits the returned promise. If batching, preserve per-run order before acknowledging a batch: `appendEvent` rows should be pageable by `(runId, sequence)`, run rows by `(sessionId, startedAt, id)`, and usage rows by `(runId, recordedAt, id)`.
 - **Idempotency is host-owned.** The runtime writes the key into `RunRecord.idempotencyKey`; enforcing unique keys and deduplicating retries is the host adapter's responsibility.
-- **Tenant isolation.** `OwnershipScope` fields are copied from the active ownership scope, but the runtime does not enforce tenant isolation. Host adapters must apply their own access controls when querying persisted ledger rows.
+- **Tenant isolation.** `OwnershipScope` fields are copied from the active ownership scope, but the runtime does not enforce tenant isolation for ledger rows. Feedback is stricter: append/query/delete require tenant plus account/user, and first-party stores compare the exact scope to the linked run.
+- **Feedback privacy.** Comments/tags/metadata can contain PII. Configure a feedback redactor, apply retention, and call owned `delete()` for erasure. Never copy comments or tag values into metric labels.
 
 ## Related APIs