@arnilo/prism 0.0.1 → 0.0.3

package/docs/providers/openrouter.md

@@ -83,12 +83,41 @@ await kernel.load([
 - Hosts choose base URL, attribution, credential source, and `fetch` impl.
 - Package contributes models and an `api_key` auth method.
 
+### Cache and session behavior
+
+- `session_id` (request body) and the `X-Session-Id` header are derived from
+  `ProviderRequestOptions.cacheKey` (falling back to `sessionId`) and sanitized
+  + clamped to 256 characters via the shared `sanitizeCacheKey()` helper.
+  Session ids route requests and identify conversations; never credentials or
+  raw prompts.
+- Anthropic-style `cache_control: { type: "ephemeral" }` markers are applied only
+  to the Prism `PromptCacheBreakpoint` locations the caller selects via
+  `ProviderRequestOptions.cache.breakpoints` (resolved with the shared
+  `applyCacheControl()` helper), and only on the last content block of each
+  selected message — not to every content block of every message. With no
+  breakpoints, no markers are emitted and the provider relies on implicit prefix
+  caching where available.
+- Caching is enabled unless disabled (`cacheRetention: "none"` /
+  `cache.mode: "off"`) and the model opts in via `ModelConfig.cache.kind`
+  (`"cache_control"`) or the legacy `compat.openRouterCache: true` flag.
+- `cacheRetention: "long"` (or `cache.retention: "long"`) emits
+  `cache_control: { type: "ephemeral", ttl: "1h" }` markers when the model allows
+  long retention (`ModelConfig.cache.longRetention !== false`); otherwise the
+  default 5-minute ephemeral window applies.
+- Usage accounting is preserved: OpenRouter `prompt_tokens_details.cached_tokens`
+  maps to `Usage.cacheReadTokens` and `prompt_tokens_details.cache_write_tokens`
+  maps to `Usage.cacheWriteTokens`.
+
 ## Security and performance notes
 
 - No catalog fetch during setup; no automatic environment, file, keychain, or shell
   credential lookup.
 - API keys are resolved per request from caller-supplied values or resolvers and
   redacted from errors.
+- Caller-supplied `ProviderRequest.options.headers` can add non-owned headers,
+  but OpenRouter-owned headers are applied last: `Authorization`,
+  `Content-Type`, `X-Session-Id`, `HTTP-Referer`, and `X-Title` cannot be
+  overridden by caller headers.
 - Attribution headers are sent only when `appUrl`/`appTitle` are supplied — no
   hidden app identity.
 - Live tests stay opt-in behind `PRISM_LIVE_PROVIDER_TESTS=1` plus fake-safe

package/docs/providers/zai.md

@@ -88,12 +88,29 @@ await kernel.load([
   fallback).
 - Package contributes models via the extension `api` and an `api_key` auth method.
 
+### Cache behavior
+
+- Z.AI GLM models use **implicit context caching**: the server caches prompt
+  prefixes automatically based on request content, with no explicit request-side
+  cache payload. Catalog models declare `cache: { kind: "implicit" }`.
+- The provider sends no `cache_control`, `prompt_cache_key`, `prompt_cache_retention`,
+  or other explicit cache-control fields regardless of `ProviderRequestOptions.cache`
+  / `cacheKey` / `cacheRetention` settings — those options have no effect on the
+  Z.AI request body. Hosts relying on cache hits should keep their stable prompt
+  prefix byte-stable and stable inputs unchanged.
+- Usage accounting is preserved: `prompt_tokens_details.cached_tokens` maps to
+  `Usage.cacheReadTokens` and `prompt_tokens_details.cache_write_tokens` maps to
+  `Usage.cacheWriteTokens` when the server reports them.
+
 ## Security and performance notes
 
 - No network calls during import, setup, build, or default tests.
 - No automatic environment, file, keychain, or shell credential lookup.
 - API keys are resolved per request from caller-supplied values or resolvers and
   redacted from errors.
+- Caller-supplied `ProviderRequest.options.headers` can add non-owned headers,
+  but provider-owned headers (`content-type`, `authorization`) are applied last
+  and cannot be overridden by caller headers.
 - Live tests stay opt-in behind `PRISM_LIVE_PROVIDER_TESTS=1` plus fake-safe
   provider-specific env names; default tests are network-free.
 

package/docs/public-contracts.md

@@ -7,14 +7,15 @@ The root `@arnilo/prism` export provides TypeScript contracts for host-owned age
 Current contract groups:
 
 - JSON/data: `JsonPrimitive`, `JsonValue`, `JsonObject`, `ErrorInfo`
-- Content/messages: `ContentBlock`, `TextContent`, `ImageContent`, `ThinkingContent`, `ToolCallContent`, `ToolResultContent`, `Message`
-- Providers/models/auth: `ModelConfig`, `ModelCapabilities`, `ModelLimits`, `ModelCost`, `Usage`, `CacheRetention`, `ProviderRequestOptions`, `ProviderRequest`, `ProviderEvent`, `AIProvider`, `ProviderPackage`, `ProviderPackageAPI`, `ProviderPackageDocs`, `AuthMethod`, `ApiKeyAuthMethod`, `OAuthAuthMethod`, `CustomAuthMethod`, `OAuthLoginCallbacks`, `OAuthCredentials`, `OAuthProvider`, `CredentialResolverSource`, `OAuthCredentialStore`, `ProviderRequestPolicy`, `ProviderRequestPolicyContext`, `ProviderRequestPolicyResult`, `SystemPromptContribution`, `SystemPromptMode`, `SystemPromptSource`, `SystemPromptConfig`
-- Agents/sessions: `AgentConfig`, `AgentDefinition`, `Agent`, `AgentSessionConfig`, `AgentSessionForkOptions`, `AgentSessionCloneOptions`, `AgentSession`, `RunOptions`, `AgentEvent`
+- Content/messages: `ContentBlock`, `TextContent`, `ImageContent`, `ThinkingContent`, `ToolCallDeltaContent`, `ToolCallContent`, `ToolResultContent`, `Message`
+- Providers/models/auth: `ModelConfig`, `ModelCapabilities`, `ModelLimits`, `ModelCost`, `ModelCacheCapabilities`, `PromptCacheKind`, `Usage`, `CacheRetention`, `PromptCacheMode`, `PromptCacheBreakpoint`, `PromptCacheHints`, `ProviderRequestOptions`, `ProviderRequest`, `ProviderEvent`, `AIProvider`, `ProviderPackage`, `ProviderPackageAPI`, `ProviderPackageDocs`, `AuthMethod`, `ApiKeyAuthMethod`, `OAuthAuthMethod`, `CustomAuthMethod`, `OAuthLoginCallbacks`, `OAuthCredentials`, `OAuthProvider`, `CredentialResolverSource`, `OAuthCredentialStore`, `ProviderRequestPolicy`, `ProviderRequestPolicyContext`, `ProviderRequestPolicyResult`, `SystemPromptContribution`, `SystemPromptMode`, `SystemPromptSource`, `SystemPromptConfig`
+- Agents/sessions: `AgentConfig`, `AgentDefinition`, `Agent`, `AgentSessionConfig`, `AgentSessionForkOptions`, `AgentSessionCloneOptions`, `AgentSession`, `SubscribeOptions`, `SubscriberOverflowPolicy`, `RunOptions`, `AgentEvent`
 - Tools/commands: `ToolDefinition`, `ToolRegistry`, `ToolExecutionContext`, `ToolResult`, `CommandDefinition`, `CommandExecutionContext`, `CommandResult`
 - Input/prompt/context/skills: `InputBuilder`, `InputBuildContext`, `AgentInput`, `DefaultInputBuilder`, `DefaultInputBuildContext`, `InputAttachment`, `PromptInstruction`, `PromptBuilder`, `PromptBuildRequest`, `ContextBlock`, `ContextProvider`, `ContextResolutionContext`, `Skill`, `SkillRegistry`
 - Extensions/middleware: `ExtensionLifecycleEventName`, `ExtensionEvent`, `Extension`, `ExtensionAPI`, `MiddlewareHookName`, `Middleware`, `MiddlewareNext`, `MiddlewareRegistry`
 - Configuration/manifests: `ConfigProvider`, `ConfigLayer`, `ConfigLoadContext`, `PrismManifest`, `ManifestContributionDeclaration`, `ManifestResourceDeclaration`, `ManifestContributionKind`
-- Stores/resources/settings/credentials/compaction/retry: `SessionEntry`, `SessionStore`, `StoreFactory`, `Resource`, `ResourceLoader`, `ResourceLoadContext`, `SettingsProvider`, `CredentialRequest`, `Credential`, `CredentialResolver`, `CompactionStrategy`, `CompactionContext`, `CompactionResult`, `CompactionOptions`, `CompactionMiddlewarePayload`, `CompactionEntryData`, `DefaultCompactionStrategyOptions`, `RetryPolicy`, `RetryContext`, `RetryDecision`, `RetryOptions`, `RetryMiddlewarePayload`, `DefaultRetryPolicyOptions`
+- Stores/resources/settings/credentials/compaction/retry/cache helpers: `SessionEntry`, `SessionStore`, `StoreFactory`, `Resource`, `ResourceLoader`, `ResourceLoadContext`, `SettingsProvider`, `CredentialRequest`, `Credential`, `CredentialResolver`, `CompactionStrategy`, `CompactionContext`, `CompactionResult`, `CompactionOptions`, `CompactionMiddlewarePayload`, `CompactionEntryData`, `DefaultCompactionStrategyOptions`, `RetryPolicy`, `RetryContext`, `RetryDecision`, `RetryOptions`, `RetryMiddlewarePayload`, `DefaultRetryPolicyOptions`, `CacheUsageReport`, `sanitizeCacheKey`, `mapCacheRetention`, `applyCacheControl`, `cacheHitRate`, `cacheSavings`, `cacheUsageReport`
+- Production persistence (adapter-facing): `ProductionPersistenceStore`, `PersistencePage`, `PersistenceQuery`, `OwnershipScope`, `SessionRecord`, `SessionQuery`, `BranchRecord`, `BranchQuery`, `SessionEntryQuery`, `RunRecord`, `RunQuery`, `AgentEventRecord`, `AgentEventQuery`, `ToolCallRecord`, `ToolCallQuery`, `UsageRecord`, `UsageQuery`, `AgentDefinitionRecord`, `AgentDefinitionQuery`, `RetentionPolicy`, `RetentionPolicyQuery`, `MigrationRecord`, `MigrationQuery`
 
 ## When to use it
 
@@ -30,16 +31,41 @@ Public contracts are imported from the root package:
 import type {
   AgentConfig,
   AgentDefinition,
+  AgentDefinitionRecord,
+  AgentDefinitionQuery,
+  AgentEventQuery,
+  AgentEventRecord,
+  AgentLoopOptions,
+  AgentLoopStrategy,
   AIProvider,
+  ArtifactContext,
+  ArtifactParseResult,
+  ArtifactParser,
+  ArtifactRepairer,
+  ArtifactValidation,
+  ArtifactValidator,
   AuthMethod,
+  BranchQuery,
+  BranchRecord,
   CommandDefinition,
+  CacheUsageReport,
   CompactionStrategy,
   ConfigLayer,
   ConfigProvider,
   ContextProvider,
   CredentialResolver,
+  MigrationQuery,
+  MigrationRecord,
   OAuthProvider,
+  OwnershipScope,
+  PersistencePage,
+  PersistenceQuery,
+  ProductionPersistenceStore,
   ProviderRequestOptions,
+  ModelCacheCapabilities,
+  PromptCacheBreakpoint,
+  PromptCacheHints,
+  PromptCacheKind,
   DefaultInputBuildContext,
   Extension,
   InputBuilder,
@@ -53,29 +79,48 @@ import type {
   PromptTemplateOptions,
   ProviderPackage,
   ProviderRequestPolicy,
+  ProviderTurnResult,
   ResourceLoader,
+  RetentionPolicy,
+  RetentionPolicyQuery,
+  RunQuery,
+  RunRecord,
+  SessionEntryQuery,
+  SessionQuery,
+  SessionRecord,
+  SubscribeOptions,
+  SubscriberOverflowPolicy,
   SettingsProvider,
   Skill,
   StoreFactory,
   SystemPromptContribution,
   SystemPromptConfig,
+  ToolCallQuery,
+  ToolCallRecord,
   ToolDefinition,
+  UsageQuery,
+  UsageRecord,
 } from "@arnilo/prism";
+
+import { applyCacheControl, cacheHitRate, cacheSavings, cacheUsageReport, mapCacheRetention, sanitizeCacheKey } from "@arnilo/prism";
 ```
 
 Important request shapes:
 
 | Contract | Purpose |
 | --- | --- |
-| `ModelConfig` | Provider/model id plus optional display name, capabilities, limits, cost/cache pricing, opaque compat JSON, parameters, and metadata. |
+| `ModelConfig` | Provider/model id plus optional display name, capabilities, limits, cost/cache pricing, `cache` capability metadata, opaque compat JSON, parameters, and metadata. |
+| `ModelCacheCapabilities` / `PromptCacheKind` | Generic model cache support metadata (`implicit`, `openai_key`, `cache_control`, `provider_specific`, `none`). See [Model registry](model-registry.md). |
+| `PromptCacheHints` / `PromptCacheBreakpoint` | Structured provider cache intent and reusable prompt anchors. See [Provider caching](provider-caching.md). |
 | `ProviderPackage` | Inert provider package definition with docs metadata and explicit `setup(api)` registration. |
 | `ProviderRequest` | Normalized provider input: `model`, `messages`, optional `tools`, `context`, generic `options`, `metadata`, and `signal`. |
-| `ProviderRequestOptions` | Generic provider adapter hints: session/cache identifiers, cache retention, headers, timeout/retry hints, compat, and opaque `extra`. |
+| `ProviderRequestOptions` | Generic provider adapter hints: session id, legacy `cacheKey`/`cacheRetention`, structured `cache?: PromptCacheHints`, headers, compat, and opaque `extra`; `timeoutMs`, `maxRetries`, and `maxRetryDelayMs` are deprecated inert hints in first-party providers. |
 | `ProviderRequestPolicy` | Ordered pre-provider hook that can patch the request and return exact secrets for provider-error redaction. |
 | `ToolRegistry` | Host active tool registry shape: `register()`, `get()`, `resolve()`, and `list()`. |
 | `ToolExecutionContext` | Host tool execution context: session/run ids, tool call id, optional abort signal, metadata, and progress callback. |
 | `ContextResolutionContext` | Context provider input: messages plus optional session/run ids, metadata, and signal. |
-| `DefaultInputBuildContext` | Optional default input assembly context: instructions, history, summaries, attachments, explicit resources, tool results, middleware, ids, metadata, and signal. |
+| `InputAssemblyLayout` | Default input layout selector: `"legacy"` (default) or opt-in `"cache_aware"`. |
+| `DefaultInputBuildContext` | Optional default input assembly context: input layout, instructions, history, summaries, attachments, explicit resources, tool results, middleware, ids, metadata, and signal. |
 | `ResolveContextOptions` | Ordered context resolution input: selected providers, messages, ids, metadata, signal, and optional middleware. |
 | `AssembleProviderInputOptions` | Provider input assembly input: model, input, optional builders, selected context providers/skills, active tools, metadata, and signal. |
 | `PromptTemplateOptions` | Missing-variable behavior for tiny `renderPromptTemplate()` substitutions. |
@@ -83,10 +128,33 @@ Important request shapes:
 | `CredentialRequest` | Credential lookup request: credential `name`, optional provider id, and metadata. |
 | `OAuthProvider` | Host/package OAuth callbacks for login, optional refresh, and conversion to a `Credential`. |
 | `AgentSessionConfig` | Session creation input: optional id, agent, store, leaf id, and metadata. |
-| `RunOptions` | Per-run overrides: optional abort signal, model, max tool rounds, provider options/request policies, system prompt layers, compaction, retry, and metadata. |
+| `RunOptions` | Per-run overrides: optional abort signal, model, input layout, max tool rounds, provider options/request policies, system prompt layers, compaction, retry, metadata, skill selection, validate, redactor, and loop. |
+| `SubscribeOptions` / `SubscriberOverflowPolicy` | Live `AgentEvent` subscriber queue limit and overflow policy: `maxQueuedEvents`, `overflow: "close" \| "drop_oldest" \| "drop_newest"`. |
+| `AgentConfig.loop` / `RunOptions.loop` | Replaceable per-run control loop: `singleShotLoop` default, `generate-validate-revise` options, or a custom `AgentLoopStrategy`. `RunOptions.loop` wins. See [Agent loops](agent-loops.md). |
+| `AgentLoopStrategy` | `{ name; run(ctx: LoopContext): Promise<Usage \| undefined> }` — orchestrates shared runtime primitives via `LoopContext`. |
+| `LoopContext` | Loop-facing surface: run ids, signal, live `history`, `input`/`inputMessages`/`maxToolRounds`, and bound `assemble`/`generate`/`dispatchToolCall`/`appendMessage`/`emit` primitives. |
+| `ProviderTurnResult` | The result of `LoopContext.generate()`: `content`, `calls`, optional `messageId`, `started`, `usage`. |
+| `ArtifactValidation` | `{ ok; errors?: readonly { path?; message }[]; metadata? }` — host validator result. |
+| `ArtifactContext` | `{ sessionId, runId, turn, signal, metadata }` — passed to artifact callbacks. |
+| `ArtifactParser<T>` / `ArtifactValidator<T>` / `ArtifactRepairer<T>` | Host-supplied callbacks for `generate-validate-revise`; `T` is host-defined, Prism never instantiates it. |
 | `SystemPromptContribution` | Explicit caller-selected prompt layer with source, mode, text, and metadata. |
 | `ConfigLayer` | Named JSON config layer consumed by `mergeConfigLayers()`. |
 | `PrismManifest` | Data-only package manifest with config defaults, contribution declarations, and resource declarations. |
+| `ProductionPersistenceStore` | Adapter-facing interface for durable, paginated, multi-tenant storage of sessions, branches, entries, runs, events, tool calls, usage, agent definitions, retention policies, and migrations. No SQL/ORM/host file storage/network dependency. |
+| `PersistencePage<T>` | Cursor-paginated result page: `items`, optional `nextCursor`, optional `total`. |
+| `PersistenceQuery` | Common pagination controls: `cursor?`, `limit?`, `order?: "asc" \| "desc"`. |
+| `OwnershipScope` | Multi-tenant scope: `tenantId?`, `accountId?`, `userId?`. Included in records and queries. |
+| `SessionRecord` / `SessionQuery` | Stored session and query filters (parent, agent definition, retention policy, timestamps, ownership). |
+| `BranchRecord` / `BranchQuery` | Branch handle/leaf pointer and query filters (session, name, parent branch, leaf presence). |
+| `SessionEntryQuery` | Paginated entry filters: `sessionId`, `runId`, `parentId`, `leafId`, `kind`, timestamp range, ownership. |
+| `RunRecord` / `RunQuery` | Stored run and filters: session, branch, status, timestamps, ownership. |
+| `AgentEventRecord` / `AgentEventQuery` | Event ledger row with `redacted` flag and filters by type, session, run, entry, timestamp, ownership. |
+| `ToolCallRecord` / `ToolCallQuery` | Tool-call row with `redacted` flag and filters by name, status, session, run, entry, timestamps, ownership. |
+| `UsageRecord` / `UsageQuery` | Usage row and filters: session, run, entry, recorded-at range, ownership. |
+| `CacheUsageReport` | Numeric cache diagnostics from normalized `Usage`: read/write tokens, hit rate, estimated savings, and optional currency. |
+| `AgentDefinitionRecord` / `AgentDefinitionQuery` | Versioned agent-definition snapshot and filters. Does not store credentials or provider instances. |
+| `RetentionPolicy` / `RetentionPolicyQuery` | Retention policy and filters: age, entry count, byte limits, archive store, applied kinds. |
+| `MigrationRecord` / `MigrationQuery` | Applied migration record and filters. |
 
 ## Outputs / response / events
 
@@ -95,7 +163,7 @@ Important output/event shapes:
 | Contract | Output |
 | --- | --- |
 | `ProviderEvent` | Provider stream events: message start, content delta, tool-call delta, final tool call, usage, done, or error. |
-| `AgentEvent` | Session/runtime events: agent/turn/message/tool/queue/compaction/retry/error events, including tool started/progress/finished/error/blocked. |
+| `AgentEvent` | Session/runtime events: agent/turn/message/tool/queue/subscriber-overflow/compaction/retry/error events, including tool started/progress/finished/error/blocked. |
 | `ToolResult` | Host tool output with optional content, value, error, and metadata. |
 | `ContextBlock` | Context text or content blocks with optional title, priority, and metadata. |
 | `SessionEntry` | Branch-aware store entry for messages, events, summaries, metadata, model changes, labels, custom data, or compaction markers. |
@@ -134,6 +202,7 @@ import type {
   AIProvider,
   AssembleProviderInputOptions,
   CommandDefinition,
+  CacheUsageReport,
   CompactionStrategy,
   ConfigLayer,
   ConfigProvider,
@@ -338,9 +407,10 @@ void credentials;
 - Contracts are host-owned and package-friendly. External packages can implement `AIProvider`, `ToolDefinition`, `CommandDefinition`, `AgentDefinition`, `InputBuilder`, `PromptBuilder`, `Middleware`, `ContextProvider`, `Skill`, `Extension`, config providers, data-only manifests, compaction strategies, store factories, resource loaders, settings providers, and credential resolvers.
 - `ExtensionAPI` is implemented by the extension kernel. It exposes explicit registries, ordered middleware registration, ordered event subscription/emission, and registration methods for Phase 2 contribution categories.
 - `AgentConfig.provider` can hold a direct provider instance for simple host wiring. Hosts that need config-driven selection should use `ModelConfig.provider` with explicit `createProviderRegistry()` / `createModelRegistry()` objects; Prism does not create a hidden global provider registry.
-- `SettingsProvider` and `CredentialResolver` are explicit dependencies. Prism must not hide global settings or credentials behind these contracts, and `CredentialResolver` should be passed only to the edge that needs a credential.
+- `SettingsProvider` and `CredentialResolver` are explicit dependencies. Prism must not hide global settings or credentials behind these contracts, and `CredentialResolver` should be passed only to the edge that needs a credential. `AgentConfig.settings` and `AgentConfig.credentials` are host-owned metadata; the session runtime does not call `settings.get()` or `credentials.resolve()`.
+- `AgentConfig.extensions` is host-owned metadata; the session runtime does not load extensions or call `Extension.setup()`. Use `createExtensionKernel().load(...)` before creating an agent, then pass selected contributions into `AgentConfig`.
 - `PrismManifest` is data-only. It can describe contribution modules/resources and config defaults, but parsing it does not import modules, execute package code, or mutate registries.
-- Resource helper functions decode resources from a caller-provided `ResourceLoader`; Prism does not include filesystem, network, package, or URI router loaders.
+- Resource helper functions decode resources from a caller-provided `ResourceLoader`; Prism does not include host file storage, network, package, or URI router loaders.
 - `createDefaultInputBuilder()` is a small default implementation of `InputBuilder`. It is replaceable and only loads explicit URI resources through a caller-provided `ResourceLoader`.
 - `resolveContextProviders()`, `createDefaultPromptBuilder()`, `assembleProviderInput()`, `createSkillRegistry()`, `resolveActiveSkills()`, and `renderPromptTemplate()` are replaceable Phase 5 helpers. They do not execute tools, evaluate template code, or grant tool permissions.
 - `createAgent()` and `createAgentSession()` implement the session runtime. They use explicit providers only; no hidden provider registry is created. Store-backed sessions use explicit `SessionStore` values and branch methods on `AgentSession`. `AgentSession.compact()` and `AgentConfig`/`RunOptions.compaction` provide manual and opt-in auto-compaction. `AgentConfig`/`RunOptions.retry` provide bounded provider-turn retry before observable output.
@@ -365,7 +435,12 @@ void credentials;
 - [Contribution registries](contribution-registries.md): explicit registries for contribution contracts.
 - [Tools](tools.md): active tool registry and exact allow/deny filtering built on `ToolDefinition`.
 - [Agent/session runtime](agent-session-runtime.md): `createAgent()` / `createAgentSession()` runtime, `AgentSession.compact()`, and auto-compaction config built on these contracts.
-- [Session stores and branching](session-stores-and-branching.md): branch-aware `SessionEntry` helpers and context rebuild.
+- [Agent loops](agent-loops.md): `singleShotLoop` default, `generateValidateReviseLoop`, `resolveLoop`, and the `Artifact*`/`AgentLoop*`/`LoopContext` contracts.
+- [Agent events](agent-events.md): the `AgentEvent` union including `artifact_*` variants and event ordering.
+- [Structured output](structured-output.md): the `ArtifactParser<T>`/`ArtifactValidator<T>`/`ArtifactRepairer<T>` seam — the only typed-output path from a loop.
+- [Session stores](session-stores.md): `SessionStore` contract, branch-aware `SessionEntry` helpers, context rebuild, and store responsibilities.
+- [Database persistence](database-persistence.md): production persistence contracts, paginated query shapes, reference schema, indexes, retention, migrations, and NoSQL mapping.
+- [Session stores and branching](session-stores-and-branching.md): detailed branch semantics and helper reference (compatibility page).
 - [Compaction and retry policies](compaction-and-retry.md): default compaction strategy, default retry policy, runtime compaction/retry options, middleware payloads, and compaction entry data.
 - [Provider layer](provider-layer.md): runtime registries, provider event helpers, and mock provider built on these contracts.
 - [Provider conformance](provider-conformance.md): testing subpath for network-free provider adapter checks.

package/docs/release-and-install.md

@@ -2,23 +2,24 @@
 
 ## What it does
 
-Prism is published as one core package plus seven first-party workspace packages and three umbrella convenience packages. This page describes how they are packed, what each tarball contains, how to install them, the required `@arnilo/prism` peer dependency, the release workflow, and the offline test budget.
+Prism is published as one core package plus nine first-party workspace packages and three umbrella convenience packages. This page describes how they are packed, what each tarball contains, how to install them, the required `@arnilo/prism` peer dependency, the release workflow, and the offline test budget.
 
 Core package:
 
 - `@arnilo/prism` — the runtime, contracts, registries, streaming events, CLI, and the `/docs` hub. `files`: `dist` (with `!dist/__tests__` and `!dist/**/*.map` negations), `docs`, `CHANGELOG.md`. `bin`: `prism` -> `./dist/cli.js`. `sideEffects`: `["dist/cli.js"]`.
 
-First-party workspace packages (each `peerDependencies: { "@arnilo/prism": "0.0.1" }`, non-optional; `sideEffects: false`):
+First-party workspace packages (each `peerDependencies: { "@arnilo/prism": "0.0.3" }`, non-optional; `sideEffects: false`):
 
-- `@arnilo/prism-provider-openai`, `@arnilo/prism-provider-openrouter`, `@arnilo/prism-provider-kimi`, `@arnilo/prism-provider-zai`, `@arnilo/prism-provider-opencode-go` — provider adapters.
+- `@arnilo/prism-provider-openai`, `@arnilo/prism-provider-openrouter`, `@arnilo/prism-provider-kimi`, `@arnilo/prism-provider-zai`, `@arnilo/prism-provider-opencode-go`, `@arnilo/prism-provider-neuralwatt` — provider adapters.
 - `@arnilo/prism-compaction-llm` — optional LLM-backed compaction strategy.
 - `@arnilo/prism-compaction-observational-memory` — optional source-backed observational memory.
+- `@arnilo/prism-coding-agent` — optional host shell/filesystem coding tools (`shell`, `read`, `write`, `edit`). **Not included in `@arnilo/prism-all`** because these tools perform real host operations and must be opted into explicitly.
 
 Umbrella packages (pure manifests, no code, no `dist`; ship only `README.md`; use hard `dependencies` to transitively install their family):
 
-- `@arnilo/prism-providers` — depends on all 5 `@arnilo/prism-provider-*` packages.
+- `@arnilo/prism-providers` — depends on all 6 `@arnilo/prism-provider-*` packages.
 - `@arnilo/prism-compaction` — depends on both `@arnilo/prism-compaction-*` packages.
-- `@arnilo/prism-all` — depends on `@arnilo/prism` + `@arnilo/prism-providers` + `@arnilo/prism-compaction` (the full kit in one install).
+- `@arnilo/prism-all` — depends on `@arnilo/prism` + `@arnilo/prism-providers` + `@arnilo/prism-compaction` (the full runtime/provider/compaction kit in one install). Does **not** include `@arnilo/prism-coding-agent`; install that separately.
 
 Each code package's `files` array is `["dist", "!dist/__tests__", "!dist/**/*.map", "README.md", "CHANGELOG.md"]`; `README.md`, `LICENSE`, and `CHANGELOG.md` ship in every code-package tarball, the core tarball also ships the `docs/` directory, and umbrella tarballs ship only `README.md` + `package.json`.
 
@@ -35,24 +36,34 @@ Consumers install the core package for the runtime and add first-party packages
 | Install core only | `npm install @arnilo/prism` |
 | Install core + all providers | `npm install @arnilo/prism @arnilo/prism-providers` |
 | Install core + compaction | `npm install @arnilo/prism @arnilo/prism-compaction` |
+| Install core + coding tools (opt-in) | `npm install @arnilo/prism @arnilo/prism-coding-agent` |
 | Install everything (core + providers + compaction) | `npm install @arnilo/prism-all` |
 | Install core + a single provider | `npm install @arnilo/prism @arnilo/prism-provider-openai` |
 | Build everything (core + workspaces) | `npm run build` |
 | Run the default (network-free) test suite | `npm test` |
 | Dry-run pack core + every package | `npm run pack:dry-run` |
 | Local mirror of the release verify gate | `npm run release:dry-run` |
+| Full SDK readiness gate (typecheck + offline tests + pack) | `npm run sdk:ready` |
 
 Public core import specifiers (from the root `exports` map):
 
 | Specifier | Resolves to |
 | --- | --- |
-| `@arnilo/prism` | `dist/index.js` / `dist/index.d.ts` |
+| `@arnilo/prism` | `dist/index.{js,d.ts}` |
 | `@arnilo/prism/providers/openai-compatible` | `dist/providers/openai-compatible.{js,d.ts}` |
 | `@arnilo/prism/testing/provider-conformance` | `dist/testing/provider-conformance.{js,d.ts}` |
+| `@arnilo/prism/testing/session-store-conformance` | `dist/testing/session-store-conformance.{js,d.ts}` |
+| `@arnilo/prism/testing/compaction-conformance` | `dist/testing/compaction-conformance.{js,d.ts}` |
+| `@arnilo/prism/testing/tool-conformance` | `dist/testing/tool-conformance.{js,d.ts}` |
+| `@arnilo/prism/testing/extension-conformance` | `dist/testing/extension-conformance.{js,d.ts}` |
 | `@arnilo/prism/node/config` | `dist/node/config.{js,d.ts}` |
 | `@arnilo/prism/node/settings` | `dist/node/settings.{js,d.ts}` |
 | `@arnilo/prism/node/trust` | `dist/node/trust.{js,d.ts}` |
 | `@arnilo/prism/node/session-store-jsonl` | `dist/node/session-store-jsonl.{js,d.ts}` |
+| `@arnilo/prism/node/contribution-discovery` | `dist/node/contribution-discovery.{js,d.ts}` |
+| `@arnilo/prism/node/instruction-injectors` | `dist/node/instruction-injectors.{js,d.ts}` |
+| `@arnilo/prism/node/system-prompts` | `dist/node/system-project-prompts.{js,d.ts}` |
+| `@arnilo/prism/node/agent-definitions` | `dist/node/agent-definitions.{js,d.ts}` |
 
 ## Outputs / response / events
 
@@ -62,7 +73,7 @@ A packed tarball contains only public compiled output and release files:
 - `README.md`, `LICENSE`, `CHANGELOG.md` in every package.
 - The core tarball additionally ships the full `docs/` directory (the docs hub).
 - `dist/cli.js` and the `bin` link in core.
-- **Tarball filenames.** npm strips the `@scope/` prefix, so the core package `@arnilo/prism` produces a tarball named `arnilo-prism-0.0.1.tgz`; first-party packages produce `arnilo-prism-provider-<name>-0.0.1.tgz` / `arnilo-prism-compaction-<name>-0.0.1.tgz`; umbrella packages produce `arnilo-prism-providers-0.0.1.tgz` / `arnilo-prism-compaction-0.0.1.tgz` / `arnilo-prism-all-0.0.1.tgz`. The CLI bin name `prism` is unaffected by the package name (`npx prism` still works; npm allows the bin field to differ from the package name).
+- **Tarball filenames.** npm strips the `@scope/` prefix, so the core package `@arnilo/prism` produces a tarball named `arnilo-prism-0.0.3.tgz`; first-party packages produce `arnilo-prism-provider-<name>-0.0.3.tgz` / `arnilo-prism-compaction-<name>-0.0.3.tgz` / `arnilo-prism-coding-agent-0.0.3.tgz`; umbrella packages produce `arnilo-prism-providers-0.0.3.tgz` / `arnilo-prism-compaction-0.0.3.tgz` / `arnilo-prism-all-0.0.3.tgz`. The CLI bin name `prism` is unaffected by the package name (`npx prism` still works; npm allows the bin field to differ from the package name).
 
 Excluded from every tarball by `files` negation:
 
@@ -79,9 +90,9 @@ Excluded from every tarball by `files` negation:
   "name": "host-app",
   "type": "module",
   "dependencies": {
-    "@arnilo/prism": "0.0.1",
-    "@arnilo/prism-provider-openai": "0.0.1",
-    "@arnilo/prism-compaction-observational-memory": "0.0.1"
+    "@arnilo/prism": "0.0.3",
+    "@arnilo/prism-provider-openai": "0.0.3",
+    "@arnilo/prism-compaction-observational-memory": "0.0.3"
   }
 }
 ```
@@ -91,48 +102,89 @@ Installing the provider/compaction packages without `@arnilo/prism` present prod
 ```text
 npm error code ERESOLVE
 npm error Could not resolve dependency:
-npm error peer @arnilo/prism@"0.0.1" from @arnilo/prism-provider-openai@0.0.1
+npm error peer @arnilo/prism@"0.0.3" from @arnilo/prism-provider-openai@0.0.3
 ```
 
 ## Implementation example
 
 ```ts
-// Core runtime
-import { createAgent, createAgentSession } from "@arnilo/prism";
-// OpenAI-compatible provider subpath
+import { createAgent, createAgentSession, type ModelConfig } from "@arnilo/prism";
 import { createOpenAICompatibleProvider } from "@arnilo/prism/providers/openai-compatible";
-// Node filesystem config loader
 import { loadConfigFile } from "@arnilo/prism/node/config";
 
-const agent = createAgent({ provider: createOpenAICompatibleProvider({ /* ... */ }) });
-const session = createAgentSession(agent, { /* session store, etc. */ });
+const config = await loadConfigFile("./prism.config.json");
+const model: ModelConfig = { provider: "openai-compatible", model: "gpt-4.1-mini" };
+const provider = createOpenAICompatibleProvider({
+  id: "openai-compatible",
+  baseUrl: String(config.providers?.openai?.baseUrl ?? "https://api.openai.com/v1"),
+  apiKey: () => process.env.OPENAI_API_KEY,
+});
+const agent = createAgent({ model, provider });
+const session = createAgentSession({ agent });
 ```
 
-Local release dry-run mirrors the GitHub Actions `verify` job (build + tests + packaging/install-smoke guards + pack dry-run):
+Local release dry-run mirrors the GitHub Actions `verify` job and delegates to the SDK readiness gate:
 
 ```bash
 npm run release:dry-run
 ```
 
+For SDK readiness, run the same one-command gate directly. It composes existing scripts only: examples/workspace typecheck, build, network-free core tests (docs/export/package/install smoke included), workspace tests, and pack dry-run.
+
+```bash
+npm run sdk:ready
+```
+
+Optional live smoke tests stay separate from SDK readiness because they require credentials and network access:
+
+```bash
+PRISM_LIVE_PROVIDER_TESTS=1 npm run test --workspaces --if-present
+```
+
 ## Extension and configuration notes
 
-- **Required `@arnilo/prism` peer.** Every first-party package declares `peerDependencies: { "@arnilo/prism": "0.0.1" }` with no `peerDependenciesMeta` (non-optional). The range stays pinned to `0.0.1` for the 0.x series and will widen to `^1.0.0` at the 1.x stable release. Inside the workspace each package also declares `"@arnilo/prism": "file:../.."` in `devDependencies` so `npm install` resolves the peer locally; that devDependency is stripped from consumer installs and is not a runtime dependency.
-- **Public access.** All 11 manifests (8 code packages + 3 umbrellas) declare `"publishConfig": { "access": "public" }` so a manual `npm publish` of a scoped `@arnilo/prism-*` package defaults to public rather than `restricted` (paid). The `release.yml` flags (`npm publish --access public` + `npm publish --workspaces --access public`) are belt-and-suspenders backups.
+- **Required `@arnilo/prism` peer.** Every first-party package declares `peerDependencies: { "@arnilo/prism": "0.0.3" }` with no `peerDependenciesMeta` (non-optional). The range stays pinned to `0.0.3` for the 0.x series and will widen to `^1.0.0` at the 1.x stable release. Inside the workspace each package also declares `"@arnilo/prism": "file:../.."` in `devDependencies` so `npm install` resolves the peer locally; that devDependency is stripped from consumer installs and is not a runtime dependency.
+- **Public access.** All 13 manifests (10 code packages + 3 umbrellas) declare `"publishConfig": { "access": "public" }` so a manual `npm publish` of a scoped `@arnilo/prism-*` package defaults to public rather than `restricted` (paid). The `release.yml` flags (`npm publish --access public` + `npm publish --workspaces --access public`) are belt-and-suspenders backups.
 - **Map retention knob.** Source maps are emitted locally but stripped from tarballs by `!dist/**/*.map`. Removing that `files` negation ships maps in releases (larger tarballs, better consumer stack traces).
-- **Release workflow.** `.github/workflows/release.yml` has two jobs. `verify` runs on push (main/master, `v*` tags) and pull requests: `npm ci`, `npm test` (builds core + workspaces first, then runs the packaging and install-smoke guards), and `npm run pack:dry-run`. `publish` runs only on `refs/tags/v*` after `verify` succeeds: `npm run build`, then `npm publish --access public` for core (first, because packages require the `@arnilo/prism` peer on the registry) and `npm publish --workspaces --access public`. With no `NPM_TOKEN` secret it runs `--dry-run` instead of a real publish. `permissions.id-token: write` is set so `--provenance` can be added later without re-architecting permissions.
+- **Release workflow.** `.github/workflows/release.yml` has three jobs. `verify` runs the full SDK readiness gate on Node 24: `npm ci`, then `npm run sdk:ready` (`npm run typecheck`, network-free `npm test`, and `npm run pack:dry-run`). `node20-compat` runs on Node 20: `npm ci`, `npm run build`, then imports every public root `exports` default target from `dist/`. This proves the published package basics under the declared `engines.node >=20` without running docs examples, which require Node >=22.6 native TypeScript stripping. `publish` runs only on `refs/tags/v*` after both `verify` and `node20-compat` succeed: `npm run build`, then `npm publish --access public` for core (first, because packages require the `@arnilo/prism` peer on the registry) and `npm publish --workspaces --access public`. With no `NPM_TOKEN` secret it runs `--dry-run` instead of a real publish. `permissions.id-token: write` is set so `--provenance` can be added later without re-architecting permissions. Local `npm run release:dry-run` delegates to the same `npm run sdk:ready` gate.
 - **Adding a package.** New workspace packages are picked up automatically by `npm run build --workspaces`, `npm test --workspaces`, `npm run pack:dry-run`, the packaging guard (`src/__tests__/packaging.test.ts`), and the install-smoke test (`src/__tests__/install-smoke.test.ts`) via the workspace glob; add the package to both tests' config arrays for explicit per-package assertions.
 
 ## Security and performance notes
 
 - **No secrets or fixtures in tarballs.** Tests, fixtures, `src/`, `plans/`, `.agents/`, `roadmap.md`, and `tsconfig` files are excluded. The `docs avoid real-looking secret examples` docs check and the packaging guard's deny list prevent secret-bearing fixtures from shipping.
-- **Live tests stay opt-in.** The default `npm test` is network-free by construction and never sets these vars. Three opt-in gate vars exist, each gating a different set of placeholder live smoke tests; none is set by default, in CI, or during release verification. Every gated live test is currently an empty placeholder awaiting provider-specific/worker checks in a later phase.
-  - `PRISM_LIVE_PROVIDER_TESTS=1` — gates the five provider packages' `src/__tests__/live.test.ts` (`@arnilo/prism-provider-openai`, `provider-opencode-go`, `provider-openrouter`, `provider-zai`, `provider-kimi`).
-  - `PRISM_LIVE_COMPACTION_TESTS=1` — gates `@arnilo/prism-compaction-llm`'s live summary-provider smoke test.
-  - `PRISM_LIVE_OBSERVATIONAL_MEMORY_TESTS=1` — gates `@arnilo/prism-compaction-observational-memory`'s live worker/provider checks.
-  - These guards use fake-safe names and carry no real credentials; the gated bodies intentionally do not read `OPENAI_API_KEY` or any provider key — they are placeholders. Itemize any provider-specific key env here only when a future phase adds a real live check that reads it.
+- **Live tests stay opt-in.** The default `npm test` is network-free by construction and never sets these vars. Three opt-in gate vars exist, each gating a different set of live smoke tests; none is set by default, in CI, or during release verification. Provider live tests are real smoke tests (text generation, tool-call loop, abort, no-secret-leak) that also require a provider-specific API key; compaction live tests remain empty placeholders awaiting provider-specific checks.
+  - `PRISM_LIVE_PROVIDER_TESTS=1` — gates the six provider packages' `src/__tests__/live.test.ts` (`@arnilo/prism-provider-openai`, `provider-opencode-go`, `provider-openrouter`, `provider-zai`, `provider-kimi`, `provider-neuralwatt`). Each provider live test also requires its own API key env var and skips safely when it is missing:
+    - `OPENAI_API_KEY` for `@arnilo/prism-provider-openai`
+    - `OPENROUTER_API_KEY` for `@arnilo/prism-provider-openrouter`
+    - `KIMI_API_KEY` for `@arnilo/prism-provider-kimi`
+    - `ZAI_API_KEY` for `@arnilo/prism-provider-zai`
+    - `NEURALWATT_API_KEY` for `@arnilo/prism-provider-neuralwatt`
+    - `OPENCODE_API_KEY` for `@arnilo/prism-provider-opencode-go`
+  - `PRISM_LIVE_COMPACTION_TESTS=1` — gates `@arnilo/prism-compaction-llm`'s live summary-provider smoke test (placeholder).
+  - `PRISM_LIVE_OBSERVATIONAL_MEMORY_TESTS=1` — gates `@arnilo/prism-compaction-observational-memory`'s live worker/provider checks (placeholder).
+  - Provider live tests read the API key from the env only when both gates are set; the key is used as a bearer token and never logged. `assertNoSecretLeak` verifies the key value does not appear in any streamed event. The compaction placeholders still carry no real credentials.
   - Enforced by `network-free-guard.test.ts` (default suite stays network-free) and by source-scanning meta-tests that assert each `live.test.ts` keeps its `skip:` guard.
 - **Install smoke is offline.** The install-smoke test packs core + every package into a temp dir and installs the tarballs with `--offline --no-audit --no-fund` into a fresh temp project; zero registry fetches happen because Prism has no runtime dependencies.
-- **Offline test budget.** The default `npm test` (no `PRISM_LIVE_PROVIDER_TESTS`) is pinned at **< 30s on Node 20** with a measured baseline of ~22s (build ~12.5s + tests ~9.5s, tests parallelized). The `npm test` CI step has `timeout-minutes: 3` as a hang backstop. If a future change pushes the CI median over 30s, raise it here and in `roadmap.md` Phase 17 with rationale rather than silently regressing.
+- **Offline test budget.** The default `npm test` (no `PRISM_LIVE_PROVIDER_TESTS`) is pinned at **< 60s on Node 20** with a measured local baseline of ~45s (build ~18s + network-free tests/workspace tests/packaging smoke ~27s). The full CI `sdk:ready` gate runs on Node 24 because docs tests execute `examples/*.ts` via native TypeScript stripping. `npm run sdk:ready` also runs typecheck and pack dry-run, so it is allowed to exceed the `npm test` budget while remaining network-free. The CI `sdk:ready` step has `timeout-minutes: 5` as a hang backstop; the separate Node 20 compatibility step has `timeout-minutes: 3`. The budget was raised from 30s after the default suite grew to include every first-party package, offline install smoke, packaging guards, docs examples, and workspace tests; optimize before raising it again.
+
+## Release checklist
+
+Every release gate maps to an exact enforcement test or command, so the checklist is executable rather than manual. Run `npm run sdk:ready` for the full local SDK readiness gate: `npm run typecheck`, network-free `npm test`, and `npm run pack:dry-run`. `npm run release:dry-run` is an alias for the same gate. The GitHub Actions `verify` job runs `npm ci` and `npm run sdk:ready` on Node 24; `node20-compat` runs `npm ci`, `npm run build`, and public export imports on Node 20.
+
+| Gate | Enforcement |
+| --- | --- |
+| Docs coverage for persistence/runtime/migration surfaces | `docs.test.ts` enrolls every API page in `apiPages` (heading + index-link + bare-specifier + secret-scan checks); dedicated section assertions pin `database-persistence.md`, `runs-and-usage.md`, `session-stores-and-branching.md`, `migration.md`, `agent-definitions.md`, `performance.md`, and the Phase 41 `external_app_example_*` / `phase41_external_app_surfaces_*` gates. |
+| Package exports/subpaths resolve to built output | `public-export-contract.test.ts` asserts every `exports`/`main`/`types`/`bin` target resolves to a built file under `dist/` with a sibling `.d.ts`, and no target escapes `dist/` (no `src/` or `examples/` leak). CI `node20-compat` also imports every public root `exports` default target on Node 20. |
+| Public-API drift | `public-export-contract.test.ts` `phase39_public_protocol_exports_and_types_do_not_drift` pins the runtime protocol (`providerToolCallDelta`, `ToolCallDeltaContent`), the `/testing/provider-conformance` subpath shape, and the observational-memory runtime `.d.ts` surface. |
+| Root SDK export surface freeze | `public-export-contract.test.ts` `root export surface is frozen` snapshots every value and type export of `src/index.ts` (107 value + 69 type) so any add/remove is a deliberate test update; `every frozen value export resolves at runtime` rebuilds `dist/index.js` and asserts each value export is present (catches build drift), and `every frozen type export appears in the built type declarations` asserts each type export is in `dist/index.d.ts`. |
+| Examples compile and are listed | `npm run typecheck` runs `tsc -p examples --noEmit`; `docs.test.ts` `examples_files_exist_and_index_links_examples` and Phase 48 release gates check every example file exists and the cache-aware + NeuralWatt examples are listed in `examples/README.md`. |
+| Examples run to completion with no secret leakage | `docs.test.ts` `examples_demos_run_to_completion_and_emit_no_secret` runs each demo (Node strips TypeScript types natively) with exit-0 and real-secret scans; `external_app_example_*` pins the DB-backed adapter reference exercising the `RunLedger`, branch-handle checkout, fork, and prior-run resume. |
+| Tarball excludes built tests, source maps, and source | `packaging.test.ts` deny list rejects `dist/__tests__/`, `*.map`, `src/`, `plans/`, and internal files per package; confirms `README.md`/`LICENSE`/`CHANGELOG.md` ship, the core tarball ships `docs/` + `dist/cli.js`, every `exports` target is present as compiled output, and the Phase 48 NeuralWatt release gate pins `@arnilo/prism-provider-neuralwatt` `dist/index.js` + `dist/index.d.ts` plus umbrella membership. |
+| NeuralWatt package/docs/examples release gate | `packaging.test.ts` pins `@arnilo/prism-provider-neuralwatt` package exports/type declarations and `@arnilo/prism-providers`/`@arnilo/prism-all` membership; `docs.test.ts` asserts `docs/index.md` links `providers/neuralwatt.md` and `provider-caching.md`, and that `examples/cache-aware-prompt-assembly.ts` plus `examples/neuralwatt-agent-run.ts` exist and are listed. |
+| Network-free + offline test budget | `network-free-guard.test.ts` keeps the default suite network-free; budget pinned `< 60s` (measured baseline above). Install-smoke is offline (`--offline --no-audit --no-fund`, zero registry fetches). |
+| Core security invariants reaffirmed | Runtime/docs tests hold the trust boundary: **no built-in app tools** (hosts register tools; the core ships only the mock provider and contract helpers), **no hidden provider/credential globals** (providers/credentials are host-owned `AgentConfig` fields, resolved via explicit `providerSource`/`CredentialResolver`), **no auto package discovery** (provider/tool/skill packages are opt-in and individually installed; contribution discovery is realpath-contained and emits inert envelopes the host registers), and **no secret persistence in core** (redaction applies before any `RunLedger`/`SessionStore` append; the ledger gate asserts each message event is written exactly once and redacted). |
+
+A change that adds a public persistence/runtime surface, a new package, or a new example must extend the matching row's enforcement (add the page to `apiPages`, the package to the `packages` array, or the example to the demos list) so the checklist stays self-maintaining.
 
 ## Related APIs