@arnilo/prism 0.0.1 → 0.0.2

package/docs/extension-authoring.md

@@ -0,0 +1,193 @@
+# Extension authoring guide
+
+## What it does
+
+This guide shows third-party package authors how to publish a Prism extension package without taking over a host app. An extension exports an `Extension` object with a `setup(api)` function. During explicit host loading, `setup()` registers inert contributions into host-owned registries: providers, models, auth descriptors, tools, context providers, skills, commands, input/prompt builders, compaction strategies, retry policies, store/resource/settings/credential hooks, provider request policies, system prompt contributions, and instruction injectors.
+
+Extensions do not start agents, execute tools, read credentials, scan files, or call providers by themselves. The host app loads the extension, inspects/filters contributions, then chooses which entries become active runtime config.
+
+## When to use it
+
+Use an extension package when you want reusable Prism capabilities that many host apps can opt into:
+
+- provider/model metadata and provider-package registration
+- reusable tools, context providers, skills, commands, input builders, and prompt builders
+- compaction/retry strategies and middleware hooks
+- data-only manifests/resources that hosts can inspect before importing code
+
+Do not use an extension to hide host policy. The host still owns trust, permissions, credentials, provider selection, active tool registries, skill activation, storage, UI, and sandboxing. Prism does not auto-discover or sandbox extension packages.
+
+## Inputs / request
+
+Package authors export a public `Extension` value:
+
+```ts
+import type { Extension } from "@arnilo/prism";
+
+export const extension: Extension = {
+  name: "acme-prism-extension",
+  setup(api) {
+    api.registerSkill({ name: "acme.brief", instructions: "Answer briefly." });
+  },
+};
+```
+
+Host apps load it explicitly:
+
+```ts
+import { createExtensionKernel } from "@arnilo/prism";
+import { extension } from "acme-prism-extension";
+
+import { createContributionRegistries } from "@arnilo/prism";
+
+const registries = createContributionRegistries({ duplicate: "error" });
+const kernel = createExtensionKernel({ registries, secrets: [apiKey] });
+await kernel.load([extension]);
+```
+
+Common `ExtensionAPI` registration calls:
+
+| Call | Registers | Activation is host-owned |
+| --- | --- | --- |
+| `registerProviderPackage()` | `ProviderPackage` setup metadata | host calls package setup / selects provider |
+| `registerProvider()` / `registerModel()` | provider/model records | host resolves provider/model for an agent/run |
+| `registerAuthMethod()` | credential descriptor | host resolves actual credentials |
+| `registerTool()` | `ToolDefinition` | host copies selected tools into an active `ToolRegistry` |
+| `registerContextProvider()` | `ContextProvider` | host passes selected providers to agent/input assembly |
+| `registerSkill()` | `Skill` | host selects skills via config or `RunOptions.activeSkills` |
+| `registerInputBuilder()` / `registerPromptBuilder()` | replaceable builders | host passes selected builders to `createAgent()` / assembly |
+| `registerCompactionStrategy()` / `registerRetryPolicy()` | strategies | host selects them in agent/run config |
+| `registerCommand()` / `registerAgent()` | command/agent definitions | host exposes/runs selected entries |
+| `registerProviderRequestPolicy()` / `registerSystemPromptContribution()` | provider/prompt policies | host includes selected policy/layer in runtime config |
+| `registerInstructionInjector()` | inert instruction injector | host passes selected injectors to agent/run config |
+| `use(hook, middleware)` | middleware hook | host passes the kernel middleware registry to runtime config |
+
+## Outputs / response / events
+
+`kernel.load([extension])` returns after `setup(api)` completes. The host can then inspect `kernel.registries.*.list()` or resolve named entries. Contributions stay inert until the host wires them into runtime config.
+
+Extension errors follow the kernel policy:
+
+- default `errorPolicy: "event"` emits an `extension_error` event with known secrets redacted
+- `errorPolicy: "throw"` rejects/throws so the host can fail fast
+
+The event bus and middleware registry are ordered and explicit. No hidden global extension kernel is created.
+
+## Request/response example
+
+```json
+{
+  "loaded": ["acme-prism-extension"],
+  "contributed": {
+    "tools": ["acme.echo"],
+    "skills": ["acme.brief"],
+    "contextProviders": ["acme.project"]
+  },
+  "active": {
+    "tools": ["acme.echo"],
+    "skills": ["acme.brief"]
+  }
+}
+```
+
+The `contributed` set is what the extension registered. The `active` set is what the host chose to pass into the runtime.
+
+## Implementation example
+
+```ts
+import {
+  createAgent,
+  createExtensionKernel,
+  createMockProvider,
+  createContributionRegistries,
+  createSkillRegistry,
+  createToolRegistry,
+  providerDone,
+  type Extension,
+} from "@arnilo/prism";
+
+export const extension: Extension = {
+  name: "acme-prism-extension",
+  setup(api) {
+    api.registerModel({ provider: "mock", model: "demo" });
+    api.registerAuthMethod({ provider: "mock", kind: "api_key", credentialName: "ACME_API_KEY" });
+    api.registerTool({
+      name: "acme.echo",
+      description: "Echo a JSON object.",
+      execute(args, ctx) {
+        return { toolCallId: ctx.toolCallId, name: "acme.echo", value: args };
+      },
+    });
+    api.registerContextProvider({
+      name: "acme.project",
+      resolve: () => [{ title: "Project", content: "Use Acme conventions." }],
+    });
+    api.registerSkill({
+      name: "acme.brief",
+      instructions: "Answer in one short paragraph.",
+      toolNames: ["acme.echo"],
+    });
+    api.registerPromptBuilder({ name: "acme.prompt", build: async (request) => request.messages });
+    api.registerCompactionStrategy({ name: "acme.compact", compact: async () => ({ summary: "summary" }) });
+    api.registerRetryPolicy({ name: "acme.retry", decide: () => ({ retry: false }) });
+    api.registerCommand({ name: "acme.status", execute: () => ({ ok: true }) });
+    api.use("provider_request", (request) => request);
+  },
+};
+
+const registries = createContributionRegistries({ duplicate: "error" });
+const kernel = createExtensionKernel({ registries, errorPolicy: "throw" });
+await kernel.load([extension]);
+
+// Host activation: select contributions explicitly.
+const tool = kernel.registries.tools.resolve("acme.echo");
+const skill = kernel.registries.skills.resolve("acme.brief");
+const provider = createMockProvider([providerDone()]);
+
+const agent = createAgent({
+  model: { provider: "mock", model: "demo" },
+  provider,
+  tools: createToolRegistry([tool]),
+  skills: createSkillRegistry([skill]),
+  context: kernel.registries.contextProviders.list(),
+  promptBuilder: kernel.registries.promptBuilders.resolve("acme.prompt"),
+  middleware: kernel.middleware,
+});
+
+await agent.createSession().run("Use the Acme extension.", { activeSkills: ["acme.brief"] });
+```
+
+## Extension and configuration notes
+
+- Export a stable named `Extension` value. Avoid side effects at module top level; keep registration inside `setup(api)`.
+- Prefix contribution names (`acme.echo`, `acme.brief`) to avoid collisions. Hosts loading third-party packages should use `duplicate: "error"`.
+- A data-only `prism` manifest can describe contributions/resources before the host imports executable package code. Manifest parsing never executes modules.
+- `registerTool()` contributes a definition only. It does not grant permission, add allow-list entries, or execute the tool.
+- `registerSkill()` contributes instructions only. Referenced `toolNames` are checked against host-active tools when the skill is activated.
+- `registerAuthMethod()` and `registerCredentialResolver()` must not contain resolved credential values. Use descriptors/resolvers; the host resolves secrets at the provider/request edge.
+- Middleware from `api.use()` runs only when the host passes `kernel.middleware` into runtime configuration.
+- Provider packages, provider request policies, system prompt contributions, instruction injectors, builders, strategies, commands, store factories, resource loaders, settings providers, and credential resolvers are all inert until host code selects or invokes them.
+
+## Security and performance notes
+
+- Prism does not sandbox extension code. Hosts should load only trusted packages or run untrusted packages in their own sandbox/process before calling Prism APIs.
+- Prism does not auto-discover extensions. Filesystem discovery is a separate opt-in scanner that reads `SKILL.md`/`manifest.json` text and still does not activate contributions.
+- Use host trust and permission policies to deny extension setup (`extension:<name>:setup`), resource loads, and tool execution before side effects.
+- Pass known secret values to `createExtensionKernel({ secrets })` so setup/listener errors are redacted. Redaction is exact known-secret replacement, not general secret detection.
+- Never put API keys, OAuth tokens, provider clients, credential resolver outputs, headers, or raw secrets in manifests, registry metadata, extension events, prompts, sessions, ledgers, or idempotency keys.
+- Extension loading performs only the code in `setup(api)` and registry/middleware/event operations. Prism adds no background workers, watchers, network calls, provider calls, filesystem scans, or tool execution.
+- Keep `setup(api)` bounded and deterministic. Long-running initialization, remote auth flows, migrations, and approval UI belong in the host app.
+
+## Related APIs
+
+- [Extension kernel and event bus](extensions.md): low-level `ExtensionAPI`, registries, events, middleware, and error policy.
+- [Contribution registries](contribution-registries.md): inert registry bundle populated by extensions.
+- [Configuration and manifests](configuration-and-manifests.md): data-only package manifests and contribution declarations.
+- [Contribution discovery (workspace)](contribution-discovery.md): opt-in filesystem scanner; no import or activation.
+- [Provider packages](provider-packages.md): package-level provider/model/auth/request-policy contributions.
+- [Tools](tools.md): host-owned active tool registry, filtering, dispatch, and permission checks.
+- [Context and skills](context-and-skills.md): host selection and `toolNames` fail-closed skill activation.
+- [Input and prompt assembly](input-and-prompt-assembly.md): selecting contributed builders/context/providers.
+- [Instruction injection](instruction-injection.md): inert injectors that grant no capabilities.
+- [Settings/auth/trust](settings-auth-trust-security.md): trust, permission, credentials, no sandbox, and redaction boundaries.
+- [Extension conformance](extension-conformance.md): test extension setup, inertness, and error redaction/rethrow behavior.

package/docs/extension-conformance.md

@@ -0,0 +1,80 @@
+# Extension conformance
+
+## What it does
+
+Extension conformance helpers are dependency-free assertions for `Extension` adapter tests. They exercise setup execution, inert contribution registration, and setup-error handling under the kernel's error policy, without network or credentials.
+
+Exported from `@arnilo/prism/testing/extension-conformance`:
+
+- `assertExtensionConforms(extension, options?)`
+- `ExtensionConformanceOptions`
+
+## When to use it
+
+Use this helper when authoring an `Extension` package. It asserts:
+
+- `setup` runs on load
+- registered contributions land in the inert contribution registries (no side effects until host code resolves and invokes them)
+- under the default `errorPolicy: "event"`, a failing setup emits a redacted `extension_error` event (when `secrets` is supplied)
+- under `expectThrow: true`, a failing setup rethrows to the caller
+
+## Inputs / request
+
+```ts
+import { assertExtensionConforms } from "@arnilo/prism/testing/extension-conformance";
+import type { Extension } from "@arnilo/prism";
+
+const extension: Extension = {
+  name: "demo",
+  setup(api) { api.registerSkill({ name: "brief", instructions: "Be brief." }); },
+};
+
+const kernel = await assertExtensionConforms(extension, { secrets: ["token-123"] });
+```
+
+`ExtensionConformanceOptions`:
+- `secrets?: readonly string[]` — secrets that must be redacted in the `extension_error` event (default policy only)
+- `expectThrow?: boolean` — assert a failing setup rethrows under `errorPolicy: "throw"`
+
+## Outputs / response / events
+
+Returns `Promise<ExtensionKernel>` so the caller can inspect registered contributions; throws a plain `Error` on the first violation. No runner.
+
+## Request/response example
+
+```ts
+import { assertExtensionConforms } from "@arnilo/prism/testing/extension-conformance";
+
+const kernel = await assertExtensionConforms(myExtension, { secrets: ["api-key"] });
+// throws if a failing setup's error event leaks one of the supplied secrets.
+```
+
+## Implementation example
+
+```ts
+import { assertExtensionConforms } from "@arnilo/prism/testing/extension-conformance";
+
+const kernel = await assertExtensionConforms({
+  name: "demo",
+  setup(api) { api.registerTool({ name: "ping", execute: () => ({ value: "pong" }) }); },
+});
+```
+
+## Extension and configuration notes
+
+- The helper builds a fresh `ExtensionKernel` for each call; it does not reuse host kernel state.
+- Contributions are inert by construction — the kernel stores envelopes and never invokes provider/tool/skill capabilities until host code resolves and calls them.
+- Under `expectThrow`, the helper asserts the kernel rethrows a failing setup rather than isolating it.
+
+## Security and performance notes
+
+- No credentials, no network required; pass fake secret strings.
+- Redaction is exact-match (mirroring `createSecretRedactor`); the helper does not detect arbitrary secret patterns.
+- The kernel does not sandbox tool/extension execution — hosts retain responsibility for trust and permission gating (see [Settings, auth, trust, security](settings-auth-trust-security.md)).
+
+## Related APIs
+
+- [Extensions](extensions.md)
+- [Contribution registries](contribution-registries.md)
+- [Settings, auth, trust, security](settings-auth-trust-security.md)
+- [Provider conformance](provider-conformance.md)

package/docs/extensions.md

@@ -101,11 +101,13 @@ await kernel.middleware.run("provider_request", { metadata: {} });
 ## Extension and configuration notes
 
 - Extension loading is explicit. Prism does not discover packages, read manifests, or load filesystem config in the kernel.
+- `AgentConfig.extensions` is host-owned metadata for compatibility; `createAgent()` and `session.run()` do not load it or call `Extension.setup()`. Load extensions with `createExtensionKernel().load(...)`, then pass selected contributions (`tools`, `context`, `skills`, middleware, etc.) into `createAgent()`.
 - Setup order is the order provided by the host.
 - The kernel writes only to explicit registries returned by `createContributionRegistries()` or provided by the host.
 - `api.registerTool()` contributes an inert `ToolDefinition` to `registries.tools`; it does not add the tool to an active tool registry, allow list, or dispatch loop.
 - `api.registerInputBuilder()`, `api.registerPromptBuilder()`, and `api.registerContextProvider()` contribute inert builders/providers; they do not replace defaults or run until the host passes selected entries to Phase 5 helpers.
 - `api.registerSkill()` contributes an inert `Skill` to `registries.skills`; it does not disclose instructions, activate referenced tools, or grant permissions until the host selects it.
+- `api.registerInstructionInjector()` (Phase 30) contributes an inert `InstructionInjector` to `registries.instructionInjectors`; it grants no tools, skills, or permissions and is only applied when the host selects it via `AgentConfig.instructionInjectors`/`RunOptions.instructionInjectors`. See [Instruction injection](instruction-injection.md).
 - `api.registerProviderPackage()`, `api.registerAuthMethod()`, `api.registerProviderRequestPolicy()`, and `api.registerSystemPromptContribution()` contribute inert provider-package data; they do not load packages, resolve credentials, mutate provider payloads, or change prompts until selected by a host/runtime helper that documents that behavior.
 - `api.registerAgent()` contributes an inert `AgentDefinition`; its `create()` can call `createAgent()`, but the runtime is not started until host code resolves the definition and creates/runs a session.
 - The kernel registers middleware only into the explicit registry returned by `createMiddlewareRegistry()` or provided by the host.
@@ -115,6 +117,7 @@ await kernel.middleware.run("provider_request", { metadata: {} });
 ## Security and performance notes
 
 - No hidden global extension kernel, provider registry, credential resolver, settings provider, store, or resource loader is created.
+- `AgentConfig.extensions` does not auto-execute, so constructing or running an agent cannot unexpectedly run extension code.
 - Error events use `ErrorInfo` and redact only known secret values passed in `secrets`.
 - Do not put resolved credential values in extension events, registry metadata, docs, logs, prompts, or session stores.
 - Event and middleware dispatch are ordered and dependency-free. They use no timers, background workers, filesystem discovery, network calls, provider calls, or tool execution.
@@ -122,10 +125,13 @@ await kernel.middleware.run("provider_request", { metadata: {} });
 
 ## Related APIs
 
+- [Extension authoring guide](extension-authoring.md): package-author checklist for inert contributions, host activation, trust, permissions, no sandbox, and redaction.
 - [Middleware hooks](middleware-hooks.md): ordered hook registry populated by `ExtensionAPI.use()`.
 - [Provider packages](provider-packages.md): provider package and model metadata registration through `ExtensionAPI`.
 - [Contribution registries](contribution-registries.md): registry bundle populated by `ExtensionAPI`.
+- [Contribution discovery (workspace)](contribution-discovery.md): filesystem-driven complement to extension registration — opt-in scan without `import()` or activation.
 - [Tools](tools.md): host activation, filtering, and dispatch for contributed tool definitions.
+- [Instruction injection](instruction-injection.md): package injectors that layer instructions and context blocks for `first_turn`/`every_turn`/`on_input` without granting tools.
 - [Input and prompt assembly](input-and-prompt-assembly.md): host selection for contributed input/prompt builders.
 - [System prompts](system-prompts.md): host selection for contributed system prompt layers.
 - [Context and skills](context-and-skills.md): host selection and tool checks for contributed context providers and skills.

package/docs/host-security.md

@@ -0,0 +1,141 @@
+# Host security guide
+
+## What it does
+
+This guide is a fail-closed checklist for apps embedding Prism. It maps host security responsibilities to existing Prism APIs for credentials, settings, redaction, trust roots, permission policies, session and ledger persistence, extension loading, and tool validation.
+
+Prism supplies seams and checks. The host owns policy decisions, secret sources, durable storage, approval UI, sandboxing, and which contributed capabilities become active.
+
+## When to use it
+
+Use this guide before exposing an agent to users, third-party extensions, durable storage, or real provider credentials.
+
+Do not use it as a replacement for product threat modeling, process/container sandboxing, secret management, database access control, or provider-side policy. Prism does not sandbox tools/extensions and does not detect arbitrary secrets.
+
+## Inputs / request
+
+Start from explicit host inputs. Do not let runtime code discover security state implicitly.
+
+| Security input | Host-owned source | Prism API / page |
+| --- | --- | --- |
+| Settings | app config object or caller-named files | `createStaticSettingsProvider`, `loadSettingsFiles()` |
+| Credentials | runtime override, memory store, vault/env object | `createExplicitCredentialResolver`, `createEnvCredentialResolver`, `resolveCredentialValue()` |
+| Redaction values | exact known credential strings | `createSecretRedactor`, `redactSecrets()` |
+| Trust roots | app-selected directories/resources | `createPathTrustPolicy`, `assertTrusted()` |
+| Permission decisions | allow/deny rules or approval UI result | `createStaticPermissionPolicy`, `assertPermission()` |
+| Tool allow-list | active tools for this agent/session/run | `createToolRegistry`, `filterTools()`, `dispatchToolCall()` |
+| Tool argument rules | host validator | `AgentConfig.validator`, `RunOptions.validate`, `ToolValidator` |
+| Durable history | host database adapter | `SessionStore`, `assertSessionStoreConforms()` |
+| Durable audit | host ledger adapter | `RunLedger`, `redactRunLedgerRecord()` |
+| Extensions | explicit package imports only | `createExtensionKernel`, `ExtensionAPI` |
+
+## Outputs / response / events
+
+Security controls fail closed before side effects when wired at the guarded edge:
+
+- trust denial blocks resource/extension reads before load/use
+- permission denial blocks extension setup, resource loading, and tool execution
+- unknown or denied tools emit `tool_execution_blocked`
+- validator failures emit `tool_execution_blocked` with `validation_failed`
+- configured redactors scrub provider requests, agent events, session entries, ledger records, tool errors, extension errors, and injector context
+
+These checks are explicit function calls during load, assembly, dispatch, append, or run handling. Prism adds no background watchers, filesystem scanners, network probes, credential polling, or automatic extension discovery.
+
+## Request/response example
+
+```json
+{
+  "credentialSource": "caller-owned env object",
+  "trustedRoots": ["/workspace/app"],
+  "allowedActions": ["tool:notes/read:execute", "extension:acme:setup"],
+  "activeTools": ["notes/read"],
+  "toolValidation": "host ToolValidator",
+  "persistence": "redacted SessionStore + RunLedger"
+}
+```
+
+The JSON above is an app security plan, not a Prism config format. Hosts translate each field into the explicit APIs listed in this guide.
+
+## Implementation example
+
+```ts
+import {
+  createEnvCredentialResolver,
+  createSecretRedactor,
+  createStaticPermissionPolicy,
+  createToolRegistry,
+  filterTools,
+  resolveCredentialValue,
+  type ToolDefinition,
+  type ToolValidator,
+} from "@arnilo/prism";
+import { createPathTrustPolicy } from "@arnilo/prism/node/trust";
+
+const workspaceRoot = "/workspace/app";
+const env = { DEMO_API_KEY: "fake-demo-key" }; // docs-only placeholder
+const credentials = createEnvCredentialResolver(env, { demo: "DEMO_API_KEY" });
+const apiKey = await resolveCredentialValue(credentials, { provider: "demo", name: "apiKey" });
+
+const redactor = createSecretRedactor([apiKey]);
+const permission = createStaticPermissionPolicy({
+  allow: ["tool:notes/read:execute", "extension:acme:setup"],
+});
+const trust = createPathTrustPolicy({ trustedRoots: [workspaceRoot] });
+
+const readNotes: ToolDefinition = {
+  name: "notes/read",
+  parameters: { type: "object", properties: { id: { type: "string" } } },
+  execute(args, context) {
+    return { toolCallId: context.toolCallId, name: "notes/read", value: { id: args.id } };
+  },
+};
+
+const validate: ToolValidator = (_tool, args) =>
+  typeof args.id === "string" && args.id.length <= 100
+    ? undefined
+    : "id must be a short string";
+
+const tools = createToolRegistry(filterTools([readNotes], { allow: ["notes/read"] }), { duplicate: "error" });
+
+void { apiKey, redactor, permission, trust, tools, validate };
+```
+
+Wire those values where they matter: provider adapters receive the resolved credential, agents/runs receive `redactor`, tool dispatch receives `permission` and `validate`, resource/extension loaders receive `trust` and `permission`, and durable adapters receive already-redacted entries/records.
+
+## Extension and configuration notes
+
+- Keep security state explicit. `AgentConfig.settings` and `AgentConfig.credentials` are host-owned metadata; `createAgent()` and `session.run()` do not automatically call `settings.get()` or `credentials.resolve()`.
+- Resolve credentials at the provider/request edge, as late as possible. Do not put resolved credentials in configs, manifests, registries, prompts, messages, events, session entries, run ledgers, idempotency keys, cache keys, or logs.
+- Use `createExplicitCredentialResolver()` to document source order such as runtime override → stored credential → caller-supplied env object → fallback.
+- Use `createEnvCredentialResolver()` only with an object the host passes in. Prism does not read `process.env` for credentials.
+- Use `createPathTrustPolicy()` for workspace/resource roots and fail closed on symlink escapes.
+- Use `createContributionRegistries({ duplicate: "error" })` and prefixed names for third-party packages to prevent silent shadowing.
+- Extension contributions are inert until selected. Loading an extension package runs its `setup(api)` code, so hosts should load only trusted packages or isolate untrusted code outside Prism.
+- Skills and instruction injectors grant no tools, permissions, validators, or resource access. Host-active tools and permission policies still decide execution.
+- For production persistence, implement a database-backed `SessionStore`/`RunLedger`, run `assertSessionStoreConforms()` against the store, and follow the database schema guidance. Do not ship provider instances, credential resolvers, or secrets into durable rows.
+
+## Security and performance notes
+
+- Fail closed: unknown providers, unknown tools, denied tools, invalid tool arguments, missing skill tool dependencies, trust failures, permission failures, append conflicts, and validator failures should stop the unsafe action.
+- Prism does not sandbox host tools, extensions, provider adapters, credential resolvers, or custom middleware. Use OS/container/process isolation when code is untrusted.
+- Redaction is exact known-secret replacement only. It is not arbitrary secret detection, entropy scanning, or DLP.
+- Known secrets must be passed into redactors before data is emitted or persisted. Redact again in host adapters if they transform records after Prism redaction.
+- Tool `parameters` metadata is not validation. Add a `ToolValidator` or validate inside the tool before side effects.
+- Permission checks happen before tool validation and before `tool.execute()`. Middleware cannot grant permission by renaming a tool.
+- Session stores and ledgers receive redacted values when a redactor is active, but durable storage remains host-owned. Enforce tenant/account/user ownership and retention in the database layer.
+- Provider-owned auth/content/session/cache/security headers win over caller headers in adapters that merge headers.
+- Security checks are bounded explicit calls on the active path. Prism adds no hidden global middleware, background workers, watchers, network calls, or filesystem scans.
+
+## Related APIs
+
+- [Settings, auth, trust, and security controls](settings-auth-trust-security.md): low-level helpers and boundary hardening table.
+- [Credentials and redaction](credentials-and-redaction.md): credential resolver order, caller-supplied env objects, OAuth refresh, exact redaction, and no persistent secret store.
+- [Tools](tools.md): active tool registry, allow/deny filters, permission order, validator order, blocked events, and no sandbox.
+- [Extension authoring guide](extension-authoring.md): inert contribution package boundary and extension loading security notes.
+- [Extension kernel and event bus](extensions.md): `createExtensionKernel`, setup error redaction/rethrow, and permission-gated extension setup.
+- [Contribution discovery](contribution-discovery.md): opt-in realpath-contained scanner that imports nothing and activates nothing.
+- [Instruction injection](instruction-injection.md): redacted injector context and no capability grants.
+- [Session stores](session-stores.md): durable session store contract and secret/persistence boundaries.
+- [Runs and usage ledger](runs-and-usage.md): redacted run/event/tool/usage ledger records.
+- [Database persistence](database-persistence.md): production schema, ownership, indexes, retention, and adapter readiness checklist.
+- [Provider caching](provider-caching.md): cache keys and provider-owned header safety rules.

package/docs/index.md

@@ -6,50 +6,71 @@ Prism is a TypeScript/Node.js agent harness. Host apps and extension packages ow
 - [Public contracts](public-contracts.md): type shapes for messages, content, agents, sessions, providers, tools, context, skills, extensions, stores, resources, settings, credentials, and events.
 
 ## Agent/session runtime
-- [Agent/session runtime](agent-session-runtime.md): create agents and sessions, run prompts, and subscribe to normalized session events.
+- [Agent/session runtime](agent-session-runtime.md): create agents and sessions, run prompts, subscribe to normalized events, and see which `AgentConfig` fields are runtime-consumed vs host-owned metadata. Covers tool-call loop transcript shape and prior-reasoning preservation across turns.
+- [Agent definitions](agent-definitions.md): resolve declarative `AgentDefinition` values via `resolveAgentDefinition`, and turn app-config `<configRoot>/agents/<name>/AGENT.md` bundles into runnable agents via `discoverAgentBundles` / `resolveAgentBundle` (explicit tool/skill activation by name, fail-closed omitted capabilities, migration-only `activateAllCapabilities`, strict duplicate scope checks, configurable prompt layers, no auto-discovery).
+- [Agent loops](agent-loops.md): replaceable per-run control loops — `singleShotLoop` default and `generate-validate-revise` with host-supplied `validator`/`parser`/`repairer` callbacks.
+- [Agent events](agent-events.md): the `AgentEvent` stream — agent/turn/message (including live `tool_call_delta` fragments), tool execution, queue/subscriber overflow, compaction/retry, artifact validation/refinement, and error variants, redacted via `redactAgentEvent`.
+- [Runs and usage ledger](runs-and-usage.md): `RunLedger` adapter for durable run, event, tool-call, usage persistence, cache diagnostics, ownership/idempotency, and redaction guidance.
+- [Performance limits](performance.md): bounded live subscriber queues, branch-read pagination expectations, JSONL/dev-store limits, and production sizing assumptions.
+- [Structured output](structured-output.md): the `Artifact*` seam (parser/validator/repairer, host-defined `T`) — the only typed-output path from a loop, with a Synapta-style schema→`ArtifactValidation` mapping example and an end-to-end third-party integration walkthrough.
 
 ## Compaction/session memory
 - [Compaction and retry policies](compaction-and-retry.md): summarize branch history and retry transient provider failures with host-replaceable policies.
-- [LLM compaction package](compaction-llm.md): optional provider-backed compaction strategy package.
-- [Observational memory compaction package](compaction-observational-memory.md): optional source-backed memory, fast compaction, recall tool, and status/view command package.
-- [Session stores and branching](session-stores-and-branching.md): store session entries, rebuild branch context, and navigate branch leaves.
-- [Node JSONL session store](node-jsonl-session-store.md): persist session entries to caller-named JSONL files in Node hosts.
+- [LLM compaction package](compaction-llm.md): optional provider-backed compaction strategy package with max-output budgets mapped through `model.parameters.maxTokens` to provider wire fields.
+- [Observational memory compaction package](compaction-observational-memory.md): optional source-backed memory, owned runtime append callback, provider-valid worker transcripts, fast compaction, recall tool, and status/view command package.
+- [Session stores](session-stores.md): `SessionStore` contract, `SessionAppendOptions`, `SessionAppendConflictError`, branch handles, `readBranchPath`, and dev-vs-production branch reads — start here for session persistence.
+- [Session stores and branching](session-stores-and-branching.md): detailed branch semantics and helper reference (kept for compatibility; links back to the canonical atomic append / branch-handle sections).
+- [Database persistence](database-persistence.md): production persistence contracts, conditional append transaction pattern, idempotency indexes, `readBranchPath`, reference relational schema, retention, migrations, and NoSQL mapping.
+- [Migration guide](migration.md): the two cross-cutting app migrations in one place — in-memory/JSONL → database-backed `ProductionPersistenceStore` persistence (+ `RunLedger`) and permissive capability defaults → Phase 38 explicit `tools`/`skills` activation, with before/after shapes and links to the detailed pages.
+- [Node JSONL session store](node-jsonl-session-store.md): development-only JSONL file adapter for single-process Node hosts; no cross-process safety.
 
 ## Provider and model connection
-- [Provider layer](provider-layer.md): register and resolve host-owned providers/models, create provider events, use generic provider request options, and test with the mock provider.
-- [Provider packages](provider-packages.md): define explicit provider packages, model metadata, auth descriptors, and request/cache policies without package discovery or provider-specific core behavior.
-  - Phase 12 package workspaces: [`@arnilo/prism-provider-openai`](providers/openai.md), [`@arnilo/prism-provider-opencode-go`](providers/opencode-go.md), [`@arnilo/prism-provider-openrouter`](providers/openrouter.md), [`@arnilo/prism-provider-zai`](providers/zai.md), and [`@arnilo/prism-provider-kimi`](providers/kimi.md).
+- [Provider layer](provider-layer.md): register and resolve host-owned providers/models, choose replace-or-error duplicate policy, create provider events, stream/reconstruct tool-call deltas, use generic provider request options, and test with the mock provider; deprecated provider-level timeout/retry hints point to runtime abort/retry.
+- [Model registry](model-registry.md): register and resolve `ModelConfig` records with capabilities, limits, cost, cache support metadata, compat data, and duplicate policy.
+- [Provider caching](provider-caching.md): use `PromptCacheHints`, `PromptCacheBreakpoint`, `ModelCacheCapabilities`, cache-aware stable-prefix guidance, and shared cache diagnostics helpers; includes a per-provider explicit/implicit cache matrix for OpenAI, OpenRouter, OpenCode Go, Z.AI, Kimi, and NeuralWatt; cache hints are best-effort and cache keys are never secrets.
+- [Provider request policies](provider-request-policies.md): chain `ProviderRequestPolicy` hooks, use `createSessionCachePolicy`, and merge legacy/structured cache options safely.
+- [Provider packages](provider-packages.md): define explicit provider packages, model metadata, auth descriptors, request/cache policies, and provider-owned header precedence without package discovery or provider-specific core behavior; includes a first-party cache behavior summary.
+  - Phase 12 package workspaces: [`@arnilo/prism-provider-openai`](providers/openai.md), [`@arnilo/prism-provider-opencode-go`](providers/opencode-go.md), [`@arnilo/prism-provider-openrouter`](providers/openrouter.md), [`@arnilo/prism-provider-zai`](providers/zai.md), [`@arnilo/prism-provider-kimi`](providers/kimi.md), and [`@arnilo/prism-provider-neuralwatt`](providers/neuralwatt.md) with implicit vLLM prefix caching, reasoning controls (`reasoning_effort`/`thinking_token_budget`/`enable_thinking`/`preserve_thinking`/`clear_thinking`), reasoning preservation, OpenAI-style tool-call loop, quota, telemetry, and retry classification helpers.
 - [OpenAI-compatible provider](providers/openai-compatible.md): optional provider subpath using native or injected `fetch` for Chat Completions streaming.
 
 ## Input, prompt, and context assembly
-- [Input and prompt assembly](input-and-prompt-assembly.md): render tiny prompt templates and turn common host input, history, attachments, explicit resources, summaries, and tool results into messages with replaceable builders and provider-input assembly.
-- [System prompts](system-prompts.md): compose explicit package/app/user/run system prompt layers without filesystem discovery or hidden globals.
-- [Context and skills](context-and-skills.md): resolve ordered context providers and keep context/skill selection host-owned.
+- [SDK customization guide](customization.md): map provider resolution, middleware, context, builders, injectors, loops, compaction, retry, stores, and skills to explicit host-wired APIs.
+- [Input and prompt assembly](input-and-prompt-assembly.md): render tiny prompt templates and turn common host input, history, attachments, explicit resources, summaries, and tool results into messages with replaceable builders, provider-input assembly, legacy default order, and opt-in cache-aware ordering.
+- [System prompts](system-prompts.md): compose explicit user/package/app/run system prompt layers, auto-load the standard `AGENTS.md` (workspace) / `SYSTEM.md` prompt files via the Node `loadSystemPromptFiles` loader (trust-gated for `AGENTS.md`), and append `SYSTEM.md` → per-agent `AGENT.md` body → repo `AGENTS.md` layers from a discovered agent bundle via `resolveAgentBundle`.
+- [Instruction injection](instruction-injection.md): register package injectors that layer redacted instructions/context blocks without granting tools, permissions, or resource escapes.
+- [Context and skills](context-and-skills.md): resolve ordered context providers and keep context/skill selection host-owned; omitted declarative skills stay inactive by default, `toolNames` fail closed before provider turns, and strict skill registries prevent silent shadowing.
 
 ## Tools
-- [Tools](tools.md): register host-owned active tools, apply exact allow/deny filtering, and dispatch tool calls.
+- [Tools](tools.md): register host-owned active tools with replace-or-error duplicate policy, apply exact allow/deny filtering, and dispatch tool calls.
 
 ## Extensions/plugins
-- [Contribution registries](contribution-registries.md): explicit host-owned registries for extension/package contributions without hidden globals.
+- [Contribution discovery (workspace)](contribution-discovery.md): opt-in, realpath-contained directory scanner turning `SKILL.md`/`manifest.json` into inert `DiscoveredContribution` envelopes the host registers — no `import()`, no auto-activate, no provider scanning. (Per-agent `AGENT.md` bundles live under an app-controlled `configRoot`; see [Agent definitions](agent-definitions.md).)
+- [Contribution registries](contribution-registries.md): explicit host-owned registries for extension/package contributions without hidden globals, with `duplicate: "error"` strict mode for provider/model/tool/skill shadowing prevention.
 - [Extension kernel and event bus](extensions.md): load host-provided extensions in order, register contributions, emit lifecycle events, and isolate extension errors.
+- [Extension authoring guide](extension-authoring.md): publish third-party extension packages that register inert contributions and show host-owned activation, trust, permissions, redaction, and no-sandbox boundaries.
 - [Middleware hooks](middleware-hooks.md): ordered hook registry for provider, input, context, tool, retry, compaction, and session lifecycle boundaries.
 
 ## Configuration/manifests
-- [Configuration and manifests](configuration-and-manifests.md): merge in-memory JSON config layers and validate data-only package manifests.
+- [Configuration and manifests](configuration-and-manifests.md): merge in-memory JSON config layers and validate data-only package manifests with prototype-pollution key rejection.
 - [Node filesystem config loader](node-filesystem-config.md): explicitly read caller-named JSON config files in Node hosts.
 - [Resource loading](resource-loading.md): decode text, JSON, and manifest resources through caller-provided loaders.
 
 ## CLI/RPC
-- [CLI/RPC](cli-rpc.md): Run print/json modes and LF-delimited RPC over the public AgentSession runtime.
+- [CLI/RPC](cli-rpc.md): Run print/json modes and LF-delimited RPC over the public AgentSession runtime, including branch-handle results, fixed `forkSession`, and `checkout`.
 
 ## Security and credentials
-- [Security/auth/trust](settings-auth-trust-security.md): settings providers, credential helpers, trust/permission policies, and redaction controls.
-- [Credentials and redaction](credentials-and-redaction.md): compose explicit credential resolver order, use caller-supplied env objects/OAuth refresh helpers, and redact known secret values.
+- [Host security guide](host-security.md): fail-closed checklist for credentials, settings, redaction, trust roots, permission policies, persistence, extension loading, and tool validation.
+- [Security/auth/trust](settings-auth-trust-security.md): settings providers, credential helpers, trust/permission policies, redaction controls, host-owned `AgentConfig.settings`/`credentials`, and security-boundary hardening summary.
+- [Credentials and redaction](credentials-and-redaction.md): compose explicit credential resolver order, use caller-supplied env objects/OAuth refresh helpers, avoid eager `AgentConfig.credentials` resolution, and redact known secret values.
 
 ## Testing and examples
 - [Provider layer](provider-layer.md): use `createMockProvider()` and provider event helpers for deterministic tests without timers, credentials, or network.
-- [Provider conformance](provider-conformance.md): run network-free provider adapter assertions from `@arnilo/prism/testing/provider-conformance`.
-- `examples/`: compile-checked typed examples and runnable mock demos (SDK basics, provider registration, auth, tools, stores/branching, compaction, observational-memory recall, CLI, RPC).
+- [Provider conformance](provider-conformance.md): run network-free provider adapter assertions (stream order, abort, tool-call reconstruction, cache usage, content coverage, protected header ownership, secret leak) from `@arnilo/prism/testing/provider-conformance`.
+- [Session store conformance](session-store-conformance.md): assert any `SessionStore` adapter satisfies append/idempotency/conflict/branch invariants from `@arnilo/prism/testing/session-store-conformance`.
+- [Compaction conformance](compaction-conformance.md): assert any `CompactionStrategy` returns a non-empty redacted summary and observes abort from `@arnilo/prism/testing/compaction-conformance`.
+- [Tool conformance](tool-conformance.md): assert the tool-dispatch blocked-reason matrix (unknown/denied/invalid/permission/validator) and success path from `@arnilo/prism/testing/tool-conformance`.
+- [Extension conformance](extension-conformance.md): assert an `Extension` setup runs, contributions stay inert, and setup errors are redacted or rethrown from `@arnilo/prism/testing/extension-conformance`.
+- `examples/`: compile-checked typed examples and runnable mock demos (SDK basics, provider registration, auth, tools, cache-aware prompt assembly, NeuralWatt agent run, stores/branching, compaction, observational-memory recall, structured-output/artifact-loop, CLI, RPC).
 
 ## Release and install
 - [Release and install](release-and-install.md): package layout, install specifiers, required `@arnilo/prism` peer, tarball contents and exclusions, the map-retention knob, the release workflow, and the offline test budget.