@arnilo/prism 0.0.13 → 0.0.15

package/docs/rag.md

@@ -2,11 +2,11 @@
 
 ## What it does
 
-`@arnilo/prism-rag` is an optional package for deterministic plain-text/Markdown chunking, bounded embedding/vector indexing, filtered semantic retrieval, stable citations, and explicit `ContextProvider` injection. It reuses `Embedder` and `VectorStore` from `@arnilo/prism-memory`; Prism core input assembly is unchanged.
+`@arnilo/prism-rag` is an optional package for deterministic text/Markdown chunking, bounded embedding/vector indexing, atomic scoped source replacement/deletion, focused text/Markdown/HTML/PDF parsing, bounded reranking, ingestion status, attributable citations, content-trust metadata, and explicit `ContextProvider` injection. It reuses `Embedder` and `VectorStore` from `@arnilo/prism-memory`; Prism core input assembly is unchanged.
 
 ## When to use it
 
-Use it when a host already owns trusted document text and needs small retrieval primitives without a document framework. Do not use it for PDF/HTML/LaTeX parsing, semantic chunking, metadata extraction agents, reranker pipelines, GraphRAG, crawling, URL fetching, or filesystem discovery.
+Use it when a host needs bounded replacement of one owned source, focused parsing after a host-authorized resource or host-selected web fetch, or a host-selected reranker over a finite candidate set. Do not use it for LaTeX parsing, semantic chunking, metadata extraction agents, a hosted reranker implementation, GraphRAG, crawling, URL fetching outside `@arnilo/prism-web-tools`, or filesystem discovery.
 
 ## Inputs / request
 
@@ -20,6 +20,16 @@ Chunking:
 | `size` / `overlap` | Character ceiling and repeated context |
 | `metadata` | JSON metadata copied to every chunk |
 
+Document lifecycle:
+
+| API/field | Meaning |
+| --- | --- |
+| `replaceSource({ sourceId, chunks, store, scope, ... })` | Atomically replaces one source after all bounded embedding succeeds; the store must implement scoped `getBySource()` and `transaction()`. |
+| `deleteSource({ sourceId, store, scope })` | Deletes only matching IDs under exact tenant/resource/corpus scope. |
+| `replaceDocument({ uri, loader, parser, store, scope, ... })` | Loads through a host seam, parses, chunks, and atomically replaces. `sourceId` is required unless loader supplies one. |
+| `DocumentLoader` / `Parser` | Small host-replaceable seams. Root and `@arnilo/prism-rag/loaders` / `@arnilo/prism-rag/parsers` export reference adapters. |
+| `textParser` / `markdownParser` / `htmlParser` / `pdfParser` | UTF-8 text, Markdown, script/style-stripping HTML, and uncompressed-text PDF parsers. |
+
 Index/retrieve:
 
 | Field | Required | Meaning |
@@ -29,18 +39,24 @@ Index/retrieve:
 | `chunks` | indexing | `RagChunk[]` from package chunkers or compatible host parser |
 | `topK` / `queryCandidates` | retrieval | Returned result count and bounded pre-filter candidates |
 | `filter` | no | Shallow JSON metadata equality filter |
-| `redactor` / `secrets` | no | Redact before embedding, persistence, and injection |
-| `signal` | no | Abort embedding, vector operations, and batch progression |
+| `reranker` | no | Host-owned `Reranker` receives redacted bounded `RagHit[]` and must return the same IDs once each, in preferred order. |
+| `maxRerankBytes` / `maxRerankMs` / `rerankConcurrency` | no | Reranker caps; defaults/hard limits are 64/256 KiB, 2/10 s, and 2/8 active calls per reranker object. |
+| `statusStore` | no | `IngestionStatusStore` records per-source pending/indexed/failed/partial byte/chunk progress; use `listIngestionStatus()` for capped exact-scope pages. |
+| `redactor` / `secrets` | no | Redact before embedding, persistence, reranking, and injection |
+| `signal` | no | Abort embedding, vector operations, reranking, and batch progression |
 
 ## Outputs / response / events
 
 - `chunkText()` / `chunkMarkdown()` return frozen `RagChunk[]` with `sourceId`, zero-based index, offsets, and stable IDs such as `guide#0001`.
 - `indexChunks()` returns `{ indexed, sourceIds }` after bounded batch upserts.
-- `retrieveContext()` returns `{ query, text, hits, citations, truncated }`. Rendered text uses `[citation-id] text` blocks.
+- `replaceSource()` / `deleteSource()` return `{ sourceId, deleted, indexed }`.
+- `replaceDocument()` carries loader parser metadata into chunk metadata; the web loader preserves web-tools citation ID and `untrusted: true`.
+- `retrieveContext()` returns `{ query, trust, text, hits, citations, truncated }`. Every hit/citation carries `{ provenance: { sourceId, chunkId, citationId, provider, retrieval: "vector", retrievedAt }, trust: { untrusted: true, inert: true, injectionCapable: true } }`; `retrievalRank` preserves pre-rerank order. Rendered text uses `[citation-id] text` blocks.
+- `createMemoryIngestionStatusStore()` is a bounded in-memory reference adapter. `listIngestionStatus({ store, scope, limit, cursor })` returns capped status pages; hosts supply durable stores when status must survive process restart.
 - `createRagContextProvider()` returns one ordinary context provider. Empty queries/results contribute no block.
 - No events, tools, permissions, provider calls, loaders, or network requests are added.
 
-Default hard ceilings include 1,000/16,384 chunk characters, 100/4,096 overlap, 1,048,576/8,388,608 document characters, 2,048/8,192 chunks, 32/128 embed batch, top-K 5/32, candidates 20/128, result 64/512 KiB, and context 2,000/8,000 estimated tokens.
+Default/hard ceilings include 1,000/16,384 chunk characters, 100/4,096 overlap, 1,048,576/8,388,608 document bytes/chars, 30 s parsing, 256 PDF pages, 2,048/8,192 chunks, 32/128 embed batch, top-K 5/32, candidates 20/128, result 64/512 KiB, context 2,000/8,000 estimated tokens, reranker input 64/256 KiB, reranker wall time 2/10 s, reranker active calls 2/8, and status pages 50/200.
 
 ## Request/response example
 
@@ -51,7 +67,8 @@ Default hard ceilings include 1,000/16,384 chunk characters, 100/4,096 overlap,
   "topK": 1,
   "result": {
     "text": "[security-guide#0001] Recheck policy before side effects.",
-    "citations": [{ "id": "security-guide#0001", "sourceId": "security-guide" }]
+    "trust": { "untrusted": true, "inert": true, "injectionCapable": true },
+    "citations": [{ "id": "security-guide#0001", "sourceId": "security-guide", "provenance": { "provider": "host", "retrieval": "vector" } }]
   }
 }
 ```
@@ -61,7 +78,7 @@ Default hard ceilings include 1,000/16,384 chunk characters, 100/4,096 overlap,
 ```ts
 import { createAgent, createMockProvider, providerDone, providerTextDelta } from "@arnilo/prism";
 import { createHashEmbedder, createMemoryVectorStore } from "@arnilo/prism-memory";
-import { chunkMarkdown, createRagContextProvider, indexChunks, retrieveContext } from "@arnilo/prism-rag";
+import { chunkMarkdown, createMemoryIngestionStatusStore, createRagContextProvider, indexChunks, listIngestionStatus, retrieveContext } from "@arnilo/prism-rag";
 
 const embedder = createHashEmbedder(); // deterministic demo/test helper, not production semantic quality
 const store = createMemoryVectorStore();
@@ -70,7 +87,9 @@ const chunks = chunkMarkdown("# Approval\n\nRecheck current policy before side e
   sourceId: "security-guide",
   metadata: { category: "security" },
 });
-await indexChunks({ chunks, embedder, store, scope });
+const statusStore = createMemoryIngestionStatusStore();
+await indexChunks({ chunks, embedder, store, scope, statusStore });
+// For a replaceable source use `replaceSource`; it keeps previous chunks until embedding succeeds.
 
 const found = await retrieveContext("approval policy", {
   embedder,
@@ -78,7 +97,9 @@ const found = await retrieveContext("approval policy", {
   scope,
   topK: 4,
   filter: { category: "security" },
+  reranker: { rerank: async ({ hits }) => [...hits].sort((a, b) => b.score - a.score) },
 });
+console.log(await listIngestionStatus({ store: statusStore, scope }));
 
 const agent = createAgent({
   model: { provider: "mock", model: "demo" },
@@ -92,9 +113,13 @@ console.log(found.text, await agent.createSession().run("How do approvals work?"
 
 - Supply any Phase 7-conforming embedder/vector store, including the in-memory reference or PostgreSQL/pgvector adapter.
 - Metadata filtering is package-local after a bounded candidate query so existing vector contracts/adapters remain unchanged. Increase `queryCandidates` only when selective filters measurably need it.
+- `Reranker` is a host seam, not a provider integration. Return each redacted candidate ID exactly once; Prism retains canonical hit/provenance/trust fields and exposes `retrievalRank` for diagnostics. Add a hosted reranker only when a host owns its credentials, quota, and retry policy.
+- `IngestionStatusStore` is optional observability storage. It is keyed by exact scope and source ID; use `listIngestionStatus()` rather than an unbounded corpus scan. The reference memory store is process-local; implement the same capped scope behavior for durable status.
 - `createRagContextProvider()` derives its query from latest user text by default; pass a fixed string or callback for host-controlled query generation.
-- Load source text separately with a host-owned `ResourceLoader`. The RAG package intentionally accepts text, not URLs or filesystem paths.
-- Package is available directly or through `@arnilo/prism-all`; installation does not create an embedder, vector store, or context provider.
+- `createResourceDocumentLoader({ loader })` calls one host-owned `ResourceLoader`; it scans nothing and performs no filesystem or network I/O itself. Pass the host's permission/trust context to that loader.
+- `createWebFetchDocumentLoader({ fetcher })` accepts an already-configured `@arnilo/prism-web-tools` fetch adapter. It never opens a socket, rejects file/local/private/IP-literal URLs, and carries normalized citation/trust metadata forward. The fetch adapter still owns DNS/SSRF policy.
+- `pdfParser` is deliberately limited to bounded, uncompressed PDF text. Provide a host parser through `Parser` for compressed, scanned, or complex PDFs; do not silently index partial text.
+- Package is available directly or through `@arnilo/prism-all`; installation does not create an embedder, vector store, loader, parser, or context provider.
 
 ## Security and performance notes
 
@@ -102,7 +127,11 @@ console.log(found.text, await agent.createSession().run("How do approvals work?"
 - Source IDs become citation/storage IDs and must be stable non-secret identifiers. Text and user metadata can be redacted before external embedding and persistence.
 - Retrieved documents are untrusted inert context. Prompt-injection text cannot activate tools, skills, credentials, permissions, or extensions.
 - Remote sources must pass existing resource/media trust, SSRF, MIME, and byte policies before their decoded text reaches this package.
-- Indexing is bounded per batch and checks abort between embed/upsert operations. A failure can leave completed batches persisted; retry is idempotent for the same stable source/chunk IDs.
+- `replaceSource()` stages every bounded embedding before opening the store transaction. It requires a source-aware transactional store and fails closed rather than pretending generic upserts are atomic. `createMemoryVectorStore()` supplies the reference `getBySource()` / transaction capability; durable stores must implement equivalent exact-scope behavior.
+- `deleteSource()` rechecks every returned record's tenant/resource/corpus and source metadata before delete. Same source IDs in another corpus remain untouched.
+- Parsers enforce byte/page/time caps, abort before and after parsing, decode UTF-8 strictly, and strip HTML script/style content. Parsed and retrieved text remains untrusted inert context; it never gains tool authority.
+- Rerankers receive redacted input under byte/time/concurrency caps. Timeout, abort, unknown/duplicate/missing IDs, oversized input, and reranker failures fail closed; returned objects cannot overwrite Prism provenance/trust fields.
+- Ingestion failure errors are redacted before status storage. Status reads reject foreign scope entries and page-limit violations; status itself creates no permission or tool authority.
 - Filtering scans at most `queryCandidates` hits; rendering stops at top-K, UTF-8 result bytes, or estimated context-token ceiling.
 
 ## Related APIs

package/docs/release-and-install.md

@@ -2,13 +2,13 @@
 
 ## What it does
 
-Prism is published as one core package, thirty-four first-party capability packages, and six pure-manifest family/profile packages (**41** publishable manifests total). This page describes how they are packed, what each tarball contains, how to install them, the required `@arnilo/prism` peer dependency, the release workflow, and the offline test budget.
+Prism is published as one core package, thirty-six first-party capability packages, and six pure-manifest family/profile packages (**43** publishable manifests total). This page describes how they are packed, what each tarball contains, how to install them, the required `@arnilo/prism` peer dependency, the release workflow, and the offline test budget.
 
 Core package:
 
 - `@arnilo/prism` — the runtime, contracts, registries, streaming events, CLI (including `prism init`), and the `/docs` hub. `files`: `dist` (with `!dist/__tests__` and `!dist/**/*.map` negations), `docs`, `templates`, `CHANGELOG.md`. `bin`: `prism` -> `dist/cli.js`. `sideEffects`: `["dist/cli.js"]`.
 
-First-party workspace packages (each has non-optional `@arnilo/prism@0.0.13` peer and `sideEffects: false`; RAG also peers on memory, and server also peers on workflows):
+First-party workspace packages (each has non-optional `@arnilo/prism@0.0.15` peer and `sideEffects: false`; RAG also peers on memory, and server also peers on workflows):
 
 - `@arnilo/prism-provider-anthropic`, `@arnilo/prism-provider-google`, `@arnilo/prism-provider-openai`, `@arnilo/prism-provider-openrouter`, `@arnilo/prism-provider-kimi`, `@arnilo/prism-provider-zai`, `@arnilo/prism-provider-opencode-go`, `@arnilo/prism-provider-neuralwatt` — provider adapters.
 - `@arnilo/prism-provider-azure`, `@arnilo/prism-provider-bedrock`, `@arnilo/prism-provider-vertex` — optional enterprise-cloud adapters (Entra/IAM/ADC; separate from consumer Anthropic/Google).
@@ -36,11 +36,11 @@ First-party workspace packages (each has non-optional `@arnilo/prism@0.0.13` pee
 
 ### 0.0.12 AG-UI package boundary
 
-`@arnilo/prism-ag-ui` is a publishable optional code package with root AG-UI exports and stable `./acp` sibling, peer `@arnilo/prism@0.0.12`, pinned `@ag-ui/core@0.0.57` / `@agentclientprotocol/sdk@1.3.0`, and no import-time network/listener/run. It is included by `@arnilo/prism-all` only—not `@arnilo/prism-code` or `@arnilo/prism-sdk`—so coding and SDK profiles stay free of UI protocol dependencies.
+`@arnilo/prism-ag-ui` is a publishable optional code package with root AG-UI exports and stable `./acp` sibling, peer `@arnilo/prism@0.0.15`, pinned `@ag-ui/core@0.0.57` / `@agentclientprotocol/sdk@1.3.0`, and no import-time network/listener/run. It is included by `@arnilo/prism-all` only—not `@arnilo/prism-code` or `@arnilo/prism-sdk`—so coding and SDK profiles stay free of UI protocol dependencies.
 
 Family/profile packages (pure manifests, no code or `dist`; ship `README.md` and `CHANGELOG.md`; use exact hard `dependencies`):
 
-- `@arnilo/prism-providers` — all nine `@arnilo/prism-provider-*` packages: eight HTTP adapters plus AI SDK interoperability.
+- `@arnilo/prism-providers` — all eleven `@arnilo/prism-provider-*` packages: ten HTTP adapters plus AI SDK interoperability.
 - `@arnilo/prism-compaction` — both `@arnilo/prism-compaction-*` packages.
 - `@arnilo/prism-base` — core + compaction family + JSON Schema validator; excludes providers, MCP, native credentials/storage, and coding tools.
 - `@arnilo/prism-code` — base + coding-agent + coding-security + MCP; providers and persistence remain explicit choices.
@@ -77,9 +77,9 @@ Consumers install the core package for the runtime and add first-party packages
 | Run the default (network-free) test suite | `npm test` |
 | Dry-run pack core + every package | `npm run pack:dry-run` |
 | Local mirror of the release verify gate | `npm run release:dry-run` |
-| Validate clean tag/version/ranges and reject registry collisions | `npm run release:check -- --version 0.0.12` |
-| Preview deterministic publish order | `npm run release:publish -- --version 0.0.12 --dry-run --allow-dirty --allow-untagged` |
-| Resume interrupted tagged publication | `npm run release:publish -- --version 0.0.12 --resume --report release-artifacts/publish-report.json` |
+| Validate clean tag/version/ranges and reject registry collisions | `npm run release:check -- --version 0.0.15` |
+| Preview deterministic publish order | `npm run release:publish -- --version 0.0.15 --dry-run --allow-dirty --allow-untagged` |
+| Resume interrupted tagged publication | `npm run release:publish -- --version 0.0.15 --resume --report release-artifacts/publish-report.json` |
 | Full SDK readiness gate (typecheck + offline tests + pack) | `npm run sdk:ready` |
 
 Public core import specifiers (from the root `exports` map):
@@ -116,7 +116,7 @@ A packed tarball contains only public compiled output and release files:
 - Code packages ship `README.md`, `LICENSE`, and `CHANGELOG.md`; family/profile packages ship `README.md` and `CHANGELOG.md`.
 - The core tarball additionally ships the full `docs/` directory (the docs hub) and `templates/init/` used by `prism init`.
 - `dist/cli.js` and the `bin` link in core.
-- **Tarball filenames.** npm strips the `@scope/` prefix, so the core package `@arnilo/prism` produces a tarball named `arnilo-prism-0.0.12.tgz`; first-party packages produce `arnilo-prism-provider-<name>-0.0.12.tgz` / `arnilo-prism-compaction-<name>-0.0.12.tgz` / `arnilo-prism-coding-agent-0.0.12.tgz`; family/profile packages produce `arnilo-prism-{providers,compaction,base,code,sdk,all}-0.0.12.tgz`. The CLI bin name `prism` is unaffected by the package name (`npx prism` still works; npm allows the bin field to differ from the package name).
+- **Tarball filenames.** npm strips the `@scope/` prefix, so the core package `@arnilo/prism` produces a tarball named `arnilo-prism-0.0.15.tgz`; first-party packages produce `arnilo-prism-provider-<name>-0.0.15.tgz` / `arnilo-prism-compaction-<name>-0.0.15.tgz` / `arnilo-prism-coding-agent-0.0.15.tgz`; family/profile packages produce `arnilo-prism-{providers,compaction,base,code,sdk,all}-0.0.15.tgz`. The CLI bin name `prism` is unaffected by the package name (`npx prism` still works; npm allows the bin field to differ from the package name).
 
 Excluded from every tarball by `files` negation:
 
@@ -135,9 +135,9 @@ Excluded from every tarball by `files` negation:
   "name": "host-app",
   "type": "module",
   "dependencies": {
-    "@arnilo/prism": "0.0.12",
-    "@arnilo/prism-provider-openai": "0.0.12",
-    "@arnilo/prism-compaction-observational-memory": "0.0.12"
+    "@arnilo/prism": "0.0.15",
+    "@arnilo/prism-provider-openai": "0.0.15",
+    "@arnilo/prism-compaction-observational-memory": "0.0.15"
   }
 }
 ```
@@ -147,7 +147,7 @@ Installing the provider/compaction packages without `@arnilo/prism` present prod
 ```text
 npm error code ERESOLVE
 npm error Could not resolve dependency:
-npm error peer @arnilo/prism@"0.0.12" from @arnilo/prism-provider-openai@0.0.12
+npm error peer @arnilo/prism@"0.0.15" from @arnilo/prism-provider-openai@0.0.15
 ```
 
 ## Implementation example
@@ -180,11 +180,11 @@ For SDK readiness, run the same one-command gate directly. It composes existing
 npm run sdk:ready
 ```
 
-Release publication derives all 34 packages from the workspace once, validates exact `0.0.12` manifest/lockfile/internal ranges, then uses deterministic dependency order. `release:check` requires a clean commit tagged `v0.0.12` and rejects any existing registry version. `release:publish --resume` skips only registry versions whose internal dependency fingerprint matches the local manifest; conflicting versions fail closed. Each attempted package is written immediately to the JSON report, so a failed job can rerun safely. `--dry-run` still performs registry availability checks and invokes `npm publish --dry-run` with explicit public access, provenance, and `latest` tag.
+Release publication derives all **43** manifests from the workspace once, validates exact `0.0.15` manifest/lockfile/internal ranges, then uses deterministic dependency order. `release:check` requires a clean commit tagged `v0.0.15` and rejects any existing registry version. `release:publish --resume` skips only registry versions whose internal dependency fingerprint matches the local manifest; conflicting versions fail closed. Each attempted package is written immediately to the JSON report, so a failed job can rerun safely. `--dry-run` performs registry availability checks and invokes `npm publish --dry-run` with explicit public access, provenance, and `latest` tag, but does not publish.
 
 ```bash
-npm run release:check -- --version 0.0.12
-npm run release:publish -- --version 0.0.12 --dry-run --allow-dirty --allow-untagged
+npm run release:check -- --version 0.0.15
+npm run release:publish -- --version 0.0.15 --dry-run --allow-dirty --allow-untagged
 ```
 
 `--allow-dirty` and `--allow-untagged` exist only for local preview; real publication and CI never pass them. npm registry calls occur only in these release preflight/publication commands, never build/test/package discovery.
@@ -195,6 +195,124 @@ Optional live smoke tests stay separate from SDK readiness because they require
 PRISM_LIVE_PROVIDER_TESTS=1 npm run test --workspaces --if-present
 ```
 
+### 0.0.15 protected live-canary matrix
+
+Default `npm test`, `npm run sdk:ready`, and `benchmark-0.0.15` are network-free. Run live rows only from a protected scheduled/release environment (or an explicitly authorized operator workstation); never place credentials in fixtures, benchmark JSON, pull-request jobs, or package scripts. Use least-privilege keys, one bounded request, and retain only redacted aggregate status. A blank **checked-in gate** means Prism deliberately has no generic credential fixture: host owns that provider/account compatibility probe.
+
+| Surface | Gate and credential | Checked-in/protected command | Canary scope |
+| --- | --- | --- | --- |
+| OpenAI Responses baseline | `PRISM_LIVE_PROVIDER_TESTS=1` + `OPENAI_API_KEY` | `npm test -w @arnilo/prism-provider-openai` | Bounded text/tool/abort smoke; key never enters events. |
+| OpenAI hosted tools + Realtime | `OPENAI_API_KEY`; protected release harness additionally supplies host-owned safety identifier and hosted-tool entitlement | No generic fixture; record result with the release evidence | Provider-hosted `web_search`/similar execution and Realtime audio/interruption need account-specific availability, so fake transport coverage remains default gate. |
+| AI SDK adapter | Host-selected AI SDK v4 model factory plus its provider credential | No generic fixture; run host integration in protected release environment | Exact `@ai-sdk/provider@4.0.3` mapping/version check; Prism does not own upstream model credentials. |
+| Kimi / Moonshot | `PRISM_LIVE_PROVIDER_TESTS=1` + `KIMI_API_KEY` | `npm test -w @arnilo/prism-provider-kimi` | Coding route; Moonshot entitlement is account-specific. |
+| Z.AI | `PRISM_LIVE_PROVIDER_TESTS=1` + `ZAI_API_KEY` | `npm test -w @arnilo/prism-provider-zai` | GLM stream/tool/reasoning smoke. |
+| OpenRouter | `PRISM_LIVE_PROVIDER_TESTS=1` + `OPENROUTER_API_KEY` | `npm test -w @arnilo/prism-provider-openrouter` | Routed stream/model metadata smoke; host chooses permitted route. |
+| OpenCode Go | `PRISM_LIVE_PROVIDER_TESTS=1` + `OPENCODE_API_KEY` | `npm test -w @arnilo/prism-provider-opencode-go` | OpenAI/Anthropic route selection smoke. |
+| Alibaba DashScope | Alibaba least-privilege API key | No generic fixture; host compatibility probe in protected release environment | Region/preset/catalog entitlement varies; offline serializer and catalog tests remain default gate. |
+| Ollama Cloud/local | Cloud API key or host-local authenticated endpoint | No generic fixture; host compatibility probe in protected release environment | Cloud account and local daemon/model availability are host-owned; no daemon starts during Prism tests. |
+| NeuralWatt | `PRISM_LIVE_PROVIDER_TESTS=1` + `NEURALWATT_API_KEY` | `npm test -w @arnilo/prism-provider-neuralwatt` | Stream/retry/quota telemetry smoke. |
+| Anthropic | `PRISM_LIVE_PROVIDER_TESTS=1` + `ANTHROPIC_API_KEY` | `npm test -w @arnilo/prism-provider-anthropic` | Restricted one-turn provider smoke. |
+| Google | `PRISM_LIVE_PROVIDER_TESTS=1` + `GOOGLE_API_KEY` or `GEMINI_API_KEY` | `npm test -w @arnilo/prism-provider-google` | Restricted one-turn provider smoke. |
+| Memory PostgreSQL/pgvector | `PRISM_TEST_POSTGRES_URL` with `vector` extension | `npm run test:postgres -w @arnilo/prism-memory` | Shared memory conformance, export/rebuild pagination, and finite-vector boundary. |
+
+The scheduled/manual `live-canaries` workflow uses protected environment `live-canaries`; release validation uses its protected release environment. Neither workflow receives a broad workspace key. A successful offline benchmark is never evidence that a live row ran; each protected invocation must record its enabled matrix rows and skipped/missing prerequisites.
+
+### 0.0.15 publish handoff
+
+**Decision: GO after protected operator prerequisites below.** Phase 10 closes provider, memory, and RAG ecosystem parity without changing the Task 0 package freeze: the exact graph remains **43 publishable manifests**. It adds OpenAI hosted-tool attribution, bounded Responses continuation and Realtime; exact AI SDK V4 mapping; bounded RAG source lifecycle/document adapters/reranking/provenance/trust/status; and memory export/rebuild production conformance. No Studio, Office, remote-browser vendor, additional vector-store, Slack/Teams, voice/desktop-control, internal-auth, or queue package ships. Protected CI, signed tag, npm authentication, OIDC attestation, and protected live-canary evidence remain operator/workflow prerequisites; no package is published by this handoff.
+
+```bash
+git diff --check
+npm ci
+npm run sdk:ready
+node scripts/benchmark-0.0.15.mjs
+node --test scripts/benchmark-0.0.15.test.mjs
+npm audit --audit-level=high
+npm run release:check -- --version 0.0.15 --allow-dirty --allow-untagged --report /tmp/prism-0.0.15-preflight.json
+npm run release:publish -- --version 0.0.15 --dry-run --allow-dirty --allow-untagged --report /tmp/prism-0.0.15-dry-run.json
+git tag -s v0.0.15 -m "Prism 0.0.15"
+git verify-tag v0.0.15
+git push origin v0.0.15
+```
+
+The dry-run checks every registry collision and executes npm's non-publishing tarball validation for each dependency-ordered manifest. The protected tag workflow alone publishes through `npm run release:publish -- --version "${GITHUB_REF_NAME#v}" --resume --report release-artifacts/publish-report.json`; re-run a failed job for the same tag. `npm audit signatures --json --include-attestations` and artifact checksums remain post-publish checks.
+
+#### Rollback limitations
+
+npm publication is immutable: partial publication is a resume case, and a confirmed defect requires deprecation plus a fixed version rather than rollback.
+
+The 0.0.15 package set is unchanged from the canonical **43-package** list below; `release:check` derives it from the workspace and rejects missing, private, version-skewed, or internally mismatched manifests.
+
+### 0.0.14 publish handoff
+
+**Decision: GO after operator prerequisites below.** Phase 9 personal/work-agent conversations, memory consent/lifecycle, durable artifact review + authorized delivery, AG-UI co-work events, scoped M365/GWS OAuth connectors, browser verified-state checkpoints, a deny-by-default device adapter contract, and two new optional provider packages (`@arnilo/prism-provider-alibaba`, `@arnilo/prism-provider-ollama`). The exact 0.0.14 graph has **43 manifests** (41 → 43; only the two provider packages are new, enrolled via `@arnilo/prism-providers`). `@arnilo/prism-code` and `@arnilo/prism-sdk` stay lean; browser/ag-ui/work-tools remain optional. no Office package, Slack/Teams channel package, voice/desktop-control vendor package, internal auth DB, or Redis/SQS queue adapter ships. Clean protected-branch CI, signed commit/tag, npm authentication, OIDC attestation, protected live canaries, and actual publication remain operator/workflow prerequisites.
+
+```bash
+git diff --check
+npm ci
+npm run sdk:ready
+node scripts/benchmark-0.0.14.mjs
+node --test scripts/benchmark-0.0.14.test.mjs
+npm run release:check -- --version 0.0.14 --allow-untagged --report /tmp/prism-0.0.14-preflight.json
+git tag -s v0.0.14 -m "Prism 0.0.14"
+git verify-tag v0.0.14
+git push origin v0.0.14
+```
+
+The tag workflow publishes only through `npm run release:publish -- --version "${GITHUB_REF_NAME#v}" --resume --report release-artifacts/publish-report.json`. Re-run failed jobs for the same tag; registry state is the resumable journal. `npm audit signatures --json --include-attestations` and artifact checksums remain post-publish operator checks.
+
+#### Rollback limitations
+
+npm publication is immutable: partial publication is a resume case, and confirmed defects require deprecation plus a fixed version rather than rollback.
+
+Package set (43):
+
+```text
+@arnilo/prism
+@arnilo/prism-ag-ui
+@arnilo/prism-browser
+@arnilo/prism-coding-agent
+@arnilo/prism-coding-security
+@arnilo/prism-compaction-llm
+@arnilo/prism-compaction-observational-memory
+@arnilo/prism-credentials-node
+@arnilo/prism-evals
+@arnilo/prism-mcp
+@arnilo/prism-memory
+@arnilo/prism-model-router
+@arnilo/prism-observability-opentelemetry
+@arnilo/prism-policy
+@arnilo/prism-all
+@arnilo/prism-base
+@arnilo/prism-code
+@arnilo/prism-compaction
+@arnilo/prism-providers
+@arnilo/prism-sdk
+@arnilo/prism-provider-ai-sdk
+@arnilo/prism-provider-alibaba
+@arnilo/prism-provider-anthropic
+@arnilo/prism-provider-azure
+@arnilo/prism-provider-bedrock
+@arnilo/prism-provider-google
+@arnilo/prism-provider-kimi
+@arnilo/prism-provider-neuralwatt
+@arnilo/prism-provider-ollama
+@arnilo/prism-provider-openai
+@arnilo/prism-provider-opencode-go
+@arnilo/prism-provider-openrouter
+@arnilo/prism-provider-vertex
+@arnilo/prism-provider-zai
+@arnilo/prism-rag
+@arnilo/prism-server
+@arnilo/prism-session-store-postgres
+@arnilo/prism-session-store-sqlite
+@arnilo/prism-supervisor
+@arnilo/prism-tool-validator-json-schema
+@arnilo/prism-web-tools
+@arnilo/prism-work-tools
+@arnilo/prism-workflows
+```
+
 ### 0.0.13 publish handoff
 
 **Decision: GO after operator prerequisites below.** Phase 8 enterprise identity, policy/audit, model governance, Azure/Bedrock/Vertex providers, server deployment seams, persistence schema v5 lifecycle hooks, and M365/GWS work connectors. The exact 0.0.13 graph has **41 manifests**. Phase 8 optional packages (`@arnilo/prism-policy`, `@arnilo/prism-model-router`, enterprise providers, `@arnilo/prism-work-tools`) enroll in `@arnilo/prism-all` only; `@arnilo/prism-code` and `@arnilo/prism-sdk` stay lean. no Office package, 0.0.14 conversation/artifact services, internal auth DB, or Redis/SQS queue adapter ships. Clean protected-branch CI, signed commit/tag, npm authentication, OIDC attestation, protected live canaries, and actual publication remain operator/workflow prerequisites.
@@ -603,8 +721,8 @@ npm publication is not transactional and published versions are immutable. Parti
 
 ## Extension and configuration notes
 
-- **Required `@arnilo/prism` peer.** Every first-party code package declares a non-optional `@arnilo/prism@0.0.13` peer (`peerDependenciesMeta` must not mark `@arnilo/prism` optional; other peers such as `playwright-core` may be optional). The range stays pinned to `0.0.13` for the current 0.x release and will widen to `^1.0.0` at the 1.x stable release. Inside the workspace each package also declares `"@arnilo/prism": "file:../.."` in `devDependencies` so `npm install` resolves the peer locally; that devDependency is stripped from consumer installs and is not a runtime dependency.
-- **Public access.** All 41 manifests (35 code packages + 6 family/profile packages) declare `"publishConfig": { "access": "public" }`; the publisher also passes `--access public` explicitly because scoped packages otherwise default to restricted on first publish.
+- **Required `@arnilo/prism` peer.** Every first-party code package declares a non-optional `@arnilo/prism@0.0.15` peer (`peerDependenciesMeta` must not mark `@arnilo/prism` optional; other peers such as `playwright-core` may be optional). The range stays pinned to `0.0.15` for the current 0.x release and will widen to `^1.0.0` at the 1.x stable release. Inside the workspace each package also declares `"@arnilo/prism": "file:../.."` in `devDependencies` so `npm install` resolves the peer locally; that devDependency is stripped from consumer installs and is not a runtime dependency.
+- **Public access.** All 43 manifests (37 code packages + 6 family/profile packages) declare `"publishConfig": { "access": "public" }`; the publisher also passes `--access public` explicitly because scoped packages otherwise default to restricted on first publish.
 - **Map retention knob.** Source maps are emitted locally but stripped from tarballs by `!dist/**/*.map`. Removing that `files` negation ships maps in releases (larger tarballs, better consumer stack traces).
 - **Release workflow.** `.github/workflows/release.yml` has six jobs. `verify` runs network-free SDK readiness on Node 24; `node20-compat` builds/imports every public root `exports` default target on Node 20 for declared `engines.node >=20` (docs examples need Node >=22.6 native TypeScript stripping); `postgres-integration` uses `pgvector/pgvector:pg16`; `supply-chain` runs high-severity audit, SPDX/license policy, and tracked-source secret scanning; and tag-only `codeql-release` runs SAST. Tag-only `publish` needs all five gates, preserves clean exact-tag/version/topological publication, and alone receives `NPM_TOKEN`, `id-token: write`, and `attestations: write`. Before npm publish it packs all current tarballs, generates checksums plus SPDX, scans unpacked public artifacts, creates GitHub attestations for tarballs and SBOM, then retains artifacts for 30 days. Registry state remains the resumable journal. Local `npm run release:dry-run` remains network-free SDK readiness; local PostgreSQL coverage is `PRISM_TEST_POSTGRES_URL=... npm run test:postgres`.
 - **Adding a package.** New workspace packages are picked up automatically by `npm run build --workspaces`, `npm test --workspaces`, `npm run pack:dry-run`, the packaging guard (`src/__tests__/packaging.test.ts`), and the install-smoke test (`src/__tests__/install-smoke.test.ts`) via the workspace glob; add the package to both tests' config arrays for explicit per-package assertions.

package/docs/resource-loading.md

@@ -87,6 +87,8 @@ console.log(bytes.byteLength, manifest.name, prompt);
 - Helpers do not choose a loader by URI scheme. Hosts can use contribution registries or their own routing when they need that.
 - Helpers do not execute loaded text or imported modules. Package activation remains a host decision.
 - `loadManifestResource()` only validates manifest data; it does not register manifest contributions.
+- `@arnilo/prism-rag` `createResourceDocumentLoader({ loader, context? })` is the RAG bridge for an already-authorized artifact. It calls the supplied `ResourceLoader` once for a caller-selected URI, preserves text/binary media type, and adds no URI routing, local-file discovery, or network fallback. Pair it with a bounded RAG `Parser`; `replaceDocument()` then chunks and atomically replaces one exact RAG source.
+- For public web documents, use `createWebFetchDocumentLoader({ fetcher })` with a host-configured `@arnilo/prism-web-tools` fetch adapter instead of adding web I/O to a `ResourceLoader`. It reuses normalized citation/trust data; the web adapter retains DNS/SSRF policy ownership.
 
 ## Security and performance notes
 
@@ -96,6 +98,7 @@ console.log(bytes.byteLength, manifest.name, prompt);
 - Helpers call `loader.load()` once per helper call and do not cache, scan, list, watch, poll, or discover packages.
 - JSON parsing fails closed for invalid JSON or non-object JSON.
 - Do not put resolved credential values, tokens, headers, or executable code in loaded config, manifests, prompts, skills, or metadata.
+- A RAG resource loader is not permission escalation: pass the same host-owned trust/permission context used for any resource load. HTML/PDF parser output and web content are untrusted inert text; compressed/scanned PDFs require a host parser rather than partial fallback.
 
 ## MCP resources