@arnilo/prism 0.11.1 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -0
- package/README.md +28 -27
- package/dist/cli-dev.js +1 -1
- package/dist/cli-init.js +4 -4
- package/dist/cli-provider-add.js +2 -2
- package/dist/field-evidence.d.ts +53 -0
- package/dist/field-evidence.js +143 -0
- package/dist/index.d.ts +3 -1
- package/dist/index.js +2 -1
- package/dist/run-bundle.js +2 -2
- package/docs/acp-agent.md +2 -2
- package/docs/ag-ui.md +1 -1
- package/docs/attention-compiler.md +1 -1
- package/docs/cli-rpc.md +2 -2
- package/docs/coding-tools.md +3 -3
- package/docs/compaction-observational-memory.md +13 -3
- package/docs/computer-use-linux.md +2 -2
- package/docs/core.md +9 -10
- package/docs/dev-inspector.md +1 -1
- package/docs/disaster-recovery.md +1 -1
- package/docs/document-extraction.md +140 -0
- package/docs/evaluations.md +1 -1
- package/docs/execution-timeline.md +30 -0
- package/docs/guardrails.md +53 -0
- package/docs/history/130-cyclic-workflows-primitive-review.md +293 -0
- package/docs/history/132-document-ingestion-primitive-review.md +124 -0
- package/docs/history/README.md +2 -0
- package/docs/history/migrate-to-0.12.0.md +57 -0
- package/docs/history/retire-node-runtime.md +59 -0
- package/docs/hooks.md +1 -1
- package/docs/host-compositions.md +2 -2
- package/docs/host-security.md +1 -1
- package/docs/index.md +29 -25
- package/docs/live-testing.md +1 -1
- package/docs/mcp-tools.md +2 -2
- package/docs/messaging-channels.md +1 -1
- package/docs/migrate-to-0.7.md +1 -1
- package/docs/migrate-to-0.8.md +2 -2
- package/docs/migrate-to-0.9.md +1 -1
- package/docs/migration.md +2 -2
- package/docs/model-registry.md +1 -1
- package/docs/model-routing.md +21 -0
- package/docs/multi-agent-patterns.md +77 -4
- package/docs/obscura.md +1 -1
- package/docs/openapi-tools.md +1 -1
- package/docs/operations.md +2 -2
- package/docs/peer-dependencies.md +18 -17
- package/docs/performance.md +32 -32
- package/docs/prompt-registry.md +1 -1
- package/docs/provider-packages.md +22 -22
- package/docs/providers/decisions.md +134 -0
- package/docs/providers/laya.md +3 -0
- package/docs/providers/typesafe.md +5 -2
- package/docs/rag.md +17 -5
- package/docs/release-and-install.md +118 -114
- package/docs/run-bundle.md +4 -0
- package/docs/runs-and-usage.md +1 -1
- package/docs/signal-channel.md +1 -1
- package/docs/sqlite-persistence.md +5 -5
- package/docs/synapta-integration-review.md +238 -0
- package/docs/telegram-channel.md +1 -1
- package/docs/testing.md +33 -14
- package/docs/wiki.md +12 -11
- package/docs/work-sandbox.md +2 -0
- package/docs/work-tools.md +3 -3
- package/docs/workflows.md +213 -11
- package/package.json +22 -21
- package/templates/business-worker/README.md.tmpl +3 -3
- package/templates/business-worker/package.json.tmpl +4 -5
- package/templates/business-worker/src/tests/agent.test.ts.tmpl +1 -1
- package/templates/business-worker/tsconfig.json.tmpl +2 -1
- package/templates/deep-research/README.md.tmpl +4 -4
- package/templates/deep-research/package.json.tmpl +4 -5
- package/templates/deep-research/src/tests/research.test.ts.tmpl +1 -1
- package/templates/deep-research/tsconfig.json.tmpl +2 -1
- package/templates/init/README.md.tmpl +3 -3
- package/templates/init/optional/evals-example.ts.tmpl +1 -1
- package/templates/init/optional/workflows-example.ts.tmpl +1 -1
- package/templates/init/package.json.tmpl +4 -5
- package/templates/init/src/tests/agent.test.ts.tmpl +1 -1
- package/templates/init/tsconfig.json.tmpl +2 -1
- package/templates/personal-assistant/README.md.tmpl +3 -3
- package/templates/personal-assistant/package.json.tmpl +4 -5
- package/templates/personal-assistant/src/tests/agent.test.ts.tmpl +1 -1
- package/templates/personal-assistant/tsconfig.json.tmpl +2 -1
- package/templates/provider/README.md.tmpl +2 -2
- package/templates/provider/package.json.tmpl +3 -4
- package/templates/provider/src/tests/provider.test.ts.tmpl +1 -1
- package/templates/provider/tsconfig.json.tmpl +1 -1
package/docs/performance.md
CHANGED
|
@@ -10,7 +10,7 @@ This page states Prism runtime limits that keep slow consumers and long sessions
|
|
|
10
10
|
|
|
11
11
|
`scripts/benchmark-0.1.0.mjs` composes the six phase benchmark scripts
|
|
12
12
|
(0.0.23–0.0.28) into one 0.1.0 capacity envelope; the merged evidence is
|
|
13
|
-
checked in as `scripts/benchmark-0.1.0.json` and re-gated on every `
|
|
13
|
+
checked in as `scripts/benchmark-0.1.0.json` and re-gated on every `bun run test`
|
|
14
14
|
by `scripts/benchmark-0.1.0.test.mjs` against the Task 0 freeze-manifest
|
|
15
15
|
capacity contract (`scripts/phase12-freeze-manifest.json`): a row that drifts
|
|
16
16
|
above its frozen p95 ceiling, a startup import above 250 ms, or a root pack
|
|
@@ -24,16 +24,16 @@ Measured on Node v24.18.0 / Linux x64 (local hardware; values are environment
|
|
|
24
24
|
evidence, not universal SLOs). Regenerate with:
|
|
25
25
|
|
|
26
26
|
```bash
|
|
27
|
-
|
|
28
|
-
PRISM_TEST_POSTGRES_URL="postgresql://…"
|
|
27
|
+
bun scripts/benchmark-0.1.0.mjs --out scripts/benchmark-0.1.0.json
|
|
28
|
+
PRISM_TEST_POSTGRES_URL="postgresql://…" bun scripts/benchmark-0.1.0.mjs --out scripts/benchmark-0.1.0.json # adds protected legs
|
|
29
29
|
```
|
|
30
30
|
|
|
31
31
|
## Multi-agent runtime concurrency (phase 35)
|
|
32
32
|
|
|
33
|
-
`
|
|
33
|
+
`bun scripts/benchmark.mjs --scenario multi-agent-runtime` is network-free (mock providers, in-process memory stores, no credentials). It measures concurrent independent sessions (1/4/16/32), supervisor fan-out and saturation (32 attempted delegates vs `maxActiveChildren`), parallel workflow fan-out maps (8×20 ms items at concurrency 2, ≥1.75× vs sequential), parallel workflow agent nodes, in-run tool concurrency, and an abort storm. Each result row carries p50/p95, throughput, heap delta, queued/dropped events, peak active provider calls, completions, and abort settle. Ceilings live in `scripts/budgets.json#multiAgentRuntime` (sanity bounds, machine-dependent). Exhaustive 59-manifest classification and recorded numbers: [`docs/_evidence/phase35-ai-runtime-package-matrix.md`](./_evidence/phase35-ai-runtime-package-matrix.md). Schema/safety/invariants: `scripts/benchmark-multi-agent.test.mjs`. Fan-out row: 8×20 ms items at concurrency 2, ≥1.75× vs sequential, peak workers ≤ 2. Supervisor saturation: 32 attempted delegates vs `maxActiveChildren` 4, overflow rejected, `activeAfter` 0.
|
|
34
34
|
|
|
35
35
|
```bash
|
|
36
|
-
|
|
36
|
+
bun scripts/benchmark.mjs --scenario multi-agent-runtime --out /tmp/prism-multi-agent.json
|
|
37
37
|
```
|
|
38
38
|
|
|
39
39
|
Recorded 2026-08-27, Node v24.19.0 / Linux x64, 5 warmups + 20 waves, 8 ms mock delay. 32 independent sessions p95 10.1 ms (vs 9.0 ms at n=1); supervisor cap-4 fan-out p95 9.4 ms; workflow 4 agent nodes at concurrency 2 p95 17.9 ms; 8 tools at concurrency 4 p95 17.4 ms; abort storm settled in 5.5 ms with zero leftover provider calls. Dropped events: 0 on every row. Task 6 three-run median p95 (2026-08-28, same fixture): sessions 9.9/9.9/10.6/12.0, supervisorFanOut 10.4, supervisorSaturation 10.3, workflowFanOut 84.5 (1.87×, peak workers 2), workflowAgentNodes 19.7, toolConcurrency 19.1, abortStorm 3.3 — all under `scripts/budgets.json#multiAgentRuntime` ceilings. Protected PostgreSQL (`PRISM_TEST_POSTGRES_URL`) skipped on this host; `release:gate` blocked until durable evidence exists. Memory-store router 16/32-worker reservations do not oversubscribe.
|
|
@@ -50,7 +50,7 @@ allocate an encoded buffer per provider event.
|
|
|
50
50
|
Run with the existing network-free fixture:
|
|
51
51
|
|
|
52
52
|
```bash
|
|
53
|
-
|
|
53
|
+
bun scripts/benchmark.mjs --scenario multi-agent-runtime
|
|
54
54
|
```
|
|
55
55
|
|
|
56
56
|
Recorded 2026-08-28 on Node v24.19.0 / Linux x64, 5 warmups + 20 measured waves.
|
|
@@ -67,30 +67,30 @@ newest-history preservation remain covered by the root context-budget tests.
|
|
|
67
67
|
|
|
68
68
|
## Tool progressive disclosure (plan 041)
|
|
69
69
|
|
|
70
|
-
`
|
|
70
|
+
`bun scripts/benchmark.mjs --scenario tool-search` is network-free (mock assembly, in-memory, no credentials). It builds a 128-tool fixture registry and assembles the provider input once per mode through `assembleProviderInput`: `toolsDisclosure "all"` (default, full tool set) vs `"search"` (top-k 16 plus the generated `search_tools` tool), then asserts provider-request tool-definition bytes shrink ≥ 60% and the index+score pass stays well under a turn. Frozen caps live in `scripts/budgets.json#toolSearch` (reduction floor 0.6, index+score ceiling 50 ms, disclosed-count ceiling 33 — sanity bounds, machine-dependent). Schema/caps/network-free gating in `bun run test`: `scripts/benchmark-tool-search.test.mjs`.
|
|
71
71
|
|
|
72
72
|
```bash
|
|
73
|
-
|
|
73
|
+
bun scripts/benchmark.mjs --scenario tool-search --out /tmp/prism-tool-search.json
|
|
74
74
|
```
|
|
75
75
|
|
|
76
76
|
Recorded 2026-08-30, Node v24.19.0 / Linux x64: tool bytes 31,923 → 4,329 (**86.4% reduction**, floor 60%), index+score 1.9–2.5 ms across three runs, disclosed 17 tools (top-k 16 + `search_tools`). Tool-accuracy fixtures (mock provider picking by name among 64/128 distractors, scripted scanner reading only the disclosed list) show search mode at full-exposure pick accuracy in both sizes — the conformance floor `search ≥ all` holds (`src/__tests__/tool-search.test.ts`).
|
|
77
77
|
|
|
78
78
|
## Workflow loop refinement (plan 045)
|
|
79
79
|
|
|
80
|
-
`
|
|
80
|
+
`bun scripts/benchmark.mjs --scenario workflow-loop` is network-free: five serial `loopNode` iterations each run one refinement through a mock provider and an in-memory checkpoint adapter. The frozen budget in `scripts/budgets.json#workflowLoop` allows 50 ms p95 per node execution, or 250 ms across all five iterations. The scenario also checks five provider calls, five finished iteration records, peak provider concurrency of one, and zero active work after completion.
|
|
81
81
|
|
|
82
82
|
Recorded 2026-08-31 on Node v24.19.0 / Linux x64: 5 warmups + 20 measured runs, p50 **2.356 ms**, p95 **6.443 ms** (**1.289 ms/iteration**), 352.88 runs/s. `maxNodes` remains the declared-node count; `maxIterations` is the independent runtime budget and stays hard-capped at 64. These timings are local evidence, not portable SLOs.
|
|
83
83
|
|
|
84
84
|
```bash
|
|
85
|
-
|
|
85
|
+
bun scripts/benchmark.mjs --scenario workflow-loop --out /tmp/prism-workflow-loop.json
|
|
86
86
|
```
|
|
87
87
|
|
|
88
88
|
## Secret redaction (plan 070)
|
|
89
89
|
|
|
90
|
-
`
|
|
90
|
+
`bun scripts/benchmark.mjs --scenario redaction` is network-free (in-memory, no credentials). It builds one transcript-scale string (≥ 1 MiB) and one small entry-shaped object, both carrying 16 realistic secret-shaped needles, and measures `redactSecrets` against a local copy of the ordered `needles.reduce(split/join)` loop it replaced for large strings: the two must stay byte-identical and no needle may survive either path. Caps live in `scripts/budgets.json#redaction` (same-process speedup floor 5, transcript p95 ceiling 250 ms, small-entry p95 ceiling 25 ms — sanity bounds, machine-dependent). Schema/caps/network-free gating in `bun run test`: `scripts/benchmark-redaction.test.mjs`.
|
|
91
91
|
|
|
92
92
|
```bash
|
|
93
|
-
|
|
93
|
+
bun scripts/benchmark.mjs --scenario redaction --out /tmp/prism-redaction.json
|
|
94
94
|
```
|
|
95
95
|
|
|
96
96
|
Recorded 2026-09-11, Node v24.19.0 / Linux x64: 1 MiB transcript × 16 needles **9.39 ms → 0.69 ms p50 (13.7×, floor 5×)**, small entry 0.15 ms p50 / 0.22 ms p95. Strings below 16 KB keep the ordered loop (`src/redaction.ts`): the equivalence check that guards the single scan costs more than the passes it saves below ~4 KB.
|
|
@@ -116,14 +116,14 @@ import fails above `startupImportMsCeiling` (250 ms) on a quiet machine — see
|
|
|
116
116
|
startup-gate note below for how the in-chain check stays load-tolerant; root
|
|
117
117
|
packed bytes and
|
|
118
118
|
file count fail above baseline × 1.05. Labels: **network-free** = runs in
|
|
119
|
-
`
|
|
119
|
+
`bun run test` evidence, no network; **protected** = requires live PostgreSQL.
|
|
120
120
|
|
|
121
|
-
**Startup gate (plan 071 Task 3).** The in-`
|
|
121
|
+
**Startup gate (plan 071 Task 3).** The in-`bun run test` check in
|
|
122
122
|
`scripts/budget-gate.test.mjs` asserts a *machine-relative ratio* rather than a raw
|
|
123
123
|
millisecond bound: cold-process `import('./dist/index.js')` wall time (trimmed mean
|
|
124
124
|
of five spawns, min/max dropped) over the median empty `node -e ''` process start
|
|
125
125
|
measured in the same run. External CPU load inflates both numbers, so the ratio held
|
|
126
|
-
at 3.3 idle, 3.5–4.1 with a full `
|
|
126
|
+
at 3.3 idle, 3.5–4.1 with a full `bun run test` running concurrently, and up to 7.2 with
|
|
127
127
|
40 competing processes, while the absolute import alone moved 60 ms → 258 ms — the
|
|
128
128
|
same contention that failed the old fixed 250 ms check (273 ms at plan 070 Task 11,
|
|
129
129
|
1104.8 ms at Task 13, both at load average ~25). Off load
|
|
@@ -194,7 +194,7 @@ Task 3 packed-install journeys and Task 4 restart-recovery evidence (see
|
|
|
194
194
|
|
|
195
195
|
## Release 0.0.28 enterprise auth, policy, MCP OAuth, API, and artifact adapters
|
|
196
196
|
|
|
197
|
-
`node scripts/benchmark-0.0.28.mjs` is network-free (in-process fake JWKS/OPA/API fetches plus loopback fixture servers for the authorization server, Prism MCP server, and S3-compatible object store). Checked `scripts/benchmark-0.0.28.json` (Node v24.18.0/Linux x64): 20 warmups, 100 measured ops per seam.
|
|
197
|
+
`node scripts/benchmark-0.0.28.mjs` is network-free (in-process fake JWKS/OPA/API fetches plus loopback fixture servers for the authorization server, Prism MCP server, and S3-compatible object store). Checked `scripts/benchmark-0.0.28.json` (Node v24.18.0/Linux x64): 20 warmups, 100 measured ops per seam. (Node-era command)
|
|
198
198
|
|
|
199
199
|
| Scenario | Recorded p95 ms | Ceiling |
|
|
200
200
|
| --- | ---: | ---: |
|
|
@@ -211,7 +211,7 @@ Conformance: `scripts/phase11-conformance.test.mjs` (5 network-free cases: compo
|
|
|
211
211
|
|
|
212
212
|
## Release 0.0.25 durable loops and human-in-the-loop
|
|
213
213
|
|
|
214
|
-
`node scripts/benchmark-0.0.25.mjs` is network-free (in-memory checkpoint store). Checked `scripts/benchmark-0.0.25.json` (Node v24.18.0/Linux x64): 20 warmups, 100 measured ops, 32 pending decisions, ~250 KiB snapshot, 64 A2UI ops/message.
|
|
214
|
+
`node scripts/benchmark-0.0.25.mjs` is network-free (in-memory checkpoint store). Checked `scripts/benchmark-0.0.25.json` (Node v24.18.0/Linux x64): 20 warmups, 100 measured ops, 32 pending decisions, ~250 KiB snapshot, 64 A2UI ops/message. (Node-era command)
|
|
215
215
|
|
|
216
216
|
| Scenario | Recorded p95 ms | Ceiling |
|
|
217
217
|
| --- | ---: | ---: |
|
|
@@ -224,7 +224,7 @@ Conformance: `scripts/phase8-conformance.test.mjs` (8 network-free cases). Value
|
|
|
224
224
|
|
|
225
225
|
## Release 0.0.26 coding intelligence, processes, forge, and egress
|
|
226
226
|
|
|
227
|
-
`node scripts/benchmark-0.0.26.mjs` is network-free (fake LSP/forge/proxy, synthetic 100k-file repo, real process spill). Checked `scripts/benchmark-0.0.26.json` (Node v24.18.0/Linux x64): 5 warmups, 20 measured ops, 100k enumeration files, 1 GiB process spill, 1,000 LSP diagnostics, 100 forge pages × 100 items, 64 MiB proxy download.
|
|
227
|
+
`node scripts/benchmark-0.0.26.mjs` is network-free (fake LSP/forge/proxy, synthetic 100k-file repo, real process spill). Checked `scripts/benchmark-0.0.26.json` (Node v24.18.0/Linux x64): 5 warmups, 20 measured ops, 100k enumeration files, 1 GiB process spill, 1,000 LSP diagnostics, 100 forge pages × 100 items, 64 MiB proxy download. (Node-era command)
|
|
228
228
|
|
|
229
229
|
| Scenario | Recorded p95 ms | Ceiling |
|
|
230
230
|
| --- | ---: | ---: |
|
|
@@ -240,7 +240,7 @@ Conformance: `scripts/phase9-conformance.test.mjs` (8 network-free cases: compos
|
|
|
240
240
|
|
|
241
241
|
## Release 0.0.24 distributed events and tool effects
|
|
242
242
|
|
|
243
|
-
`node scripts/benchmark-0.0.24.mjs` is an explicit protected PostgreSQL benchmark behind `PRISM_TEST_POSTGRES_URL`. Checked `scripts/benchmark-0.0.24.json` (Node v24.18.0/Linux x64, PostgreSQL 16.14): 10 tenants × 10 principals × 1,000 events/owner, 16 producers/subscribers, 100 warmups, 1,000 measured ops, 10,000-event sustained replay, 100-row cleanup.
|
|
243
|
+
`node scripts/benchmark-0.0.24.mjs` is an explicit protected PostgreSQL benchmark behind `PRISM_TEST_POSTGRES_URL`. Checked `scripts/benchmark-0.0.24.json` (Node v24.18.0/Linux x64, PostgreSQL 16.14): 10 tenants × 10 principals × 1,000 events/owner, 16 producers/subscribers, 100 warmups, 1,000 measured ops, 10,000-event sustained replay, 100-row cleanup. (Node-era command)
|
|
244
244
|
|
|
245
245
|
| Scenario | Recorded p95 ms | Ceiling |
|
|
246
246
|
| --- | ---: | ---: |
|
|
@@ -252,7 +252,7 @@ Sustained replay delivered 160,000 subscriber-events at 101.34 events/s. Five `E
|
|
|
252
252
|
|
|
253
253
|
## Release 0.0.23 enterprise PostgreSQL evidence
|
|
254
254
|
|
|
255
|
-
`node scripts/benchmark-0.0.23.mjs` is an explicit protected PostgreSQL benchmark, not part of `npm test` or `sdk:ready`. It requires `PRISM_TEST_POSTGRES_URL`, creates/drops an isolated schema, and checks frozen p95 ceilings from `scripts/budgets.json`. The checked `scripts/benchmark-0.0.23.json` evidence was recorded on Node v24.18.0/Linux x64 with `postgres:16-alpine`: 10 tenants × 10 principals × 1,000 policy/evaluation rows, 10,000 router keys, 16 pool clients, 100 warmups, 1,000 measured operations, and 100-row cleanup batches.
|
|
255
|
+
`node scripts/benchmark-0.0.23.mjs` is an explicit protected PostgreSQL benchmark, not part of `npm test` or `sdk:ready`. It requires `PRISM_TEST_POSTGRES_URL`, creates/drops an isolated schema, and checks frozen p95 ceilings from `scripts/budgets.json`. The checked `scripts/benchmark-0.0.23.json` evidence was recorded on Node v24.18.0/Linux x64 with `postgres:16-alpine`: 10 tenants × 10 principals × 1,000 policy/evaluation rows, 10,000 router keys, 16 pool clients, 100 warmups, 1,000 measured operations, and 100-row cleanup batches. (Node-era command)
|
|
256
256
|
|
|
257
257
|
| Scenario | Recorded p95 ms | Ceiling |
|
|
258
258
|
| --- | ---: | ---: |
|
|
@@ -268,8 +268,8 @@ The same run accepted 1,000 rate claims, accumulated 16,000 budget tokens, grant
|
|
|
268
268
|
|
|
269
269
|
Release 0.0.16 is a simplification/readiness release: it added no performance-affecting code, so the six network-free scenario medians are held at the 0.0.15 baseline and the win is a smaller published artifact. Budgets live in `scripts/budgets.json` (measured baselines + tolerance) and are enforced two ways:
|
|
270
270
|
|
|
271
|
-
- **Fast gate (every `
|
|
272
|
-
- **Release evidence runner** — `node scripts/benchmark-0.0.16.mjs` re-measures root pack + startup, spawns `benchmark-0.0.15.mjs` for the six scenario medians (reused unchanged), compares every value to `budgets.json` (throughput floor / latency ceiling at ±25%), prints the evidence report below, and exits non-zero on any regression. *(0.1.3, plan 015 Task 1: the per-version runners were consolidated into the parameterized runner `scripts/benchmark.mjs --scenario <name>`; the 0.0.16 evidence below is the historical record, budgets.json medians unchanged.)*
|
|
271
|
+
- **Fast gate (every `bun run test`)** — `scripts/budget-gate.test.mjs` re-packs the root tarball (`npm pack --dry-run --json`) and fails if packed bytes, unpacked bytes, or file count exceed baseline + 5%, and fails if cold-process `import('./dist/index.js')` exceeds the 250 ms sanity ceiling. Negative fixtures prove an inflated/regressed value fails.
|
|
272
|
+
- **Release evidence runner** — `node scripts/benchmark-0.0.16.mjs` re-measures root pack + startup, spawns `benchmark-0.0.15.mjs` for the six scenario medians (reused unchanged), compares every value to `budgets.json` (throughput floor / latency ceiling at ±25%), prints the evidence report below, and exits non-zero on any regression. *(0.1.3, plan 015 Task 1: the per-version runners were consolidated into the parameterized runner `scripts/benchmark.mjs --scenario <name>`; the 0.0.16 evidence below is the historical record, budgets.json medians unchanged.)* (Node-era command)
|
|
273
273
|
|
|
274
274
|
**Artifact diet (the 0.0.16 finding).** The Task 1 tarball deny list dropped the historical `docs/review-coverage-*.md` (11 files, 283,022 bytes) from the root package: the root tarball went from **659,478 packed / 2,310,686 unpacked / 281 files** (0.0.15) to a budgeted **≈575,680 packed / 2,043,402 unpacked / 270 files**. The per-release `scripts/benchmark-0.0.*.mjs` history never shipped in artifacts (root `files` is `dist`/`docs`/`templates`/`CHANGELOG.md` only — zero `scripts/` entries packed), so no archive move was needed; `benchmark-0.0.16.mjs` consolidates the current evidence behind one budget-gating runner.
|
|
275
275
|
|
|
@@ -284,7 +284,7 @@ Release 0.0.16 is a simplification/readiness release: it added no performance-af
|
|
|
284
284
|
| Startup `import('./dist/index.js')` | ~38 ms | ceiling 250 ms |
|
|
285
285
|
| Six scenario medians (below) | 0.0.15 baseline | ±25% |
|
|
286
286
|
|
|
287
|
-
**0.0.16 measured evidence** (`node scripts/benchmark-0.0.16.mjs`, 100 iterations each, network-free, 0 backpressure / 0 resource-limit signals; all 22 budget checks passed):
|
|
287
|
+
**0.0.16 measured evidence** (`node scripts/benchmark-0.0.16.mjs`, 100 iterations each, network-free, 0 backpressure / 0 resource-limit signals; all 22 budget checks passed): (Node-era command)
|
|
288
288
|
|
|
289
289
|
| Scenario | throughput/s | p50 ms | p95 ms |
|
|
290
290
|
| --- | --- | --- | --- |
|
|
@@ -299,7 +299,7 @@ Root startup measured ≈37.7 ms (ceiling 250 ms). Timing is machine-dependent,
|
|
|
299
299
|
|
|
300
300
|
## Release 0.0.15 provider, RAG, and memory evidence
|
|
301
301
|
|
|
302
|
-
Run `node scripts/benchmark-0.0.15.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.15.test.mjs`. Default mode is network-free: fake Responses SSE/WebSocket transports, a fake AI SDK v4 model, zero-fetch provider-package registration, hash embeddings, in-memory RAG replacement/reranking/retrieval/status, and in-memory memory retention/export/rebuild.
|
|
302
|
+
Run `node scripts/benchmark-0.0.15.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.15.test.mjs`. Default mode is network-free: fake Responses SSE/WebSocket transports, a fake AI SDK v4 model, zero-fetch provider-package registration, hash embeddings, in-memory RAG replacement/reranking/retrieval/status, and in-memory memory retention/export/rebuild. (Node-era command)
|
|
303
303
|
|
|
304
304
|
Scenarios: `openai-hosted-continuation`, `openai-realtime-envelope`, `ai-sdk-v4-stream-mapping`, `provider-package-metadata`, `rag-parse-replace-rerank-retrieve`, and `memory-retention-export-rebuild`.
|
|
305
305
|
|
|
@@ -337,7 +337,7 @@ These values are dated local comparison evidence, not portable thresholds.
|
|
|
337
337
|
|
|
338
338
|
Reconnect is one ownership-scoped redacted durable page plus an optional bounded live subscriber. It is at-least-once at a page boundary, never a polling loop or terminal-run rerun. ACP uses the same event/byte/queue caps. Coding compaction reuses LLM summary/reserve/error/file-operation bounds (16,384 / 131,072 summary and reserve tokens; 1 / 8 KiB summary errors) and makes no additional provider call.
|
|
339
339
|
|
|
340
|
-
Run `node scripts/benchmark-0.0.12.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.12.test.mjs`. Default mode is network-free and reports mapper/handler/replay throughput and p50/p95, peak emitted queue rows, event bytes, heap, and coding-preparation overhead. Bounds and hostile-input fixtures—not these host-local timings—are release gates.
|
|
340
|
+
Run `node scripts/benchmark-0.0.12.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.12.test.mjs`. Default mode is network-free and reports mapper/handler/replay throughput and p50/p95, peak emitted queue rows, event bytes, heap, and coding-preparation overhead. Bounds and hostile-input fixtures—not these host-local timings—are release gates. (Node-era command)
|
|
341
341
|
|
|
342
342
|
2026-07-22 baseline: Node v24.18.0, Linux x64, 100 iterations/scenario, network=false, credentials=false.
|
|
343
343
|
|
|
@@ -365,15 +365,15 @@ Finite caps (defaults / hard) — full matrix in [Phase 6 evidence](_evidence/re
|
|
|
365
365
|
| Context omission rows | 256 / 1024 |
|
|
366
366
|
| Pending steers | 8 messages / 64 KiB |
|
|
367
367
|
|
|
368
|
-
Run `node scripts/benchmark-0.0.11.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.11.test.mjs`. Default mode is network-free: memory-linear `searchSessions` (label + query) plus assembler `contextBudget` eviction/fit. Emits environment, scenario mode, throughput, p50/p95 latency, heap, disk bytes, process counts, zero external cost, backpressure, and resource-limit signals. Search never default-scans an unbounded store; budget fails closed on mandatory prefix overflow; steer overflow fails closed. These are evidence fields, not CI timing gates.
|
|
368
|
+
Run `node scripts/benchmark-0.0.11.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.11.test.mjs`. Default mode is network-free: memory-linear `searchSessions` (label + query) plus assembler `contextBudget` eviction/fit. Emits environment, scenario mode, throughput, p50/p95 latency, heap, disk bytes, process counts, zero external cost, backpressure, and resource-limit signals. Search never default-scans an unbounded store; budget fails closed on mandatory prefix overflow; steer overflow fails closed. These are evidence fields, not CI timing gates. (Node-era command)
|
|
369
369
|
|
|
370
370
|
## Release 0.0.10 reproducible workspace-mode evidence
|
|
371
371
|
|
|
372
|
-
Run `node scripts/benchmark-0.0.10.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.10.test.mjs`. Default mode is network-free: host-composition write/read/list plus sandbox-fake composition write/read/list/search (in-memory `DisposableSandbox`). Emits environment, scenario mode, throughput, p50/p95 latency, heap, disk bytes, process counts, zero external cost, backpressure, and resource-limit signals. Optional `PRISM_BENCH_DOCKER=1` (with `PRISM_TEST_DOCKER_*`) appends real local Docker composition rows. Unified workspace mode reuses existing sandbox/repo hard caps and adds no unbounded host↔container sync. These are evidence fields, not CI timing gates.
|
|
372
|
+
Run `node scripts/benchmark-0.0.10.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.10.test.mjs`. Default mode is network-free: host-composition write/read/list plus sandbox-fake composition write/read/list/search (in-memory `DisposableSandbox`). Emits environment, scenario mode, throughput, p50/p95 latency, heap, disk bytes, process counts, zero external cost, backpressure, and resource-limit signals. Optional `PRISM_BENCH_DOCKER=1` (with `PRISM_TEST_DOCKER_*`) appends real local Docker composition rows. Unified workspace mode reuses existing sandbox/repo hard caps and adds no unbounded host↔container sync. These are evidence fields, not CI timing gates. (Node-era command)
|
|
373
373
|
|
|
374
374
|
## Release 0.0.9 reproducible coding/browser evidence
|
|
375
375
|
|
|
376
|
-
Run `node scripts/benchmark-0.0.9.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.9.test.mjs`. Default mode is network-free fake/in-process only and emits environment, scenario mode, throughput, p50/p95 latency, heap, disk bytes, process counts, zero external cost, backpressure, and resource-limit signals for repository list/search, Git status, and browser open/snapshot/action/close. Optional `PRISM_BENCH_DOCKER=1` (with `PRISM_TEST_DOCKER_*`) and `PRISM_BENCH_PLAYWRIGHT=1` append real local Docker / protected Playwright rows. These are evidence fields, not CI timing gates.
|
|
376
|
+
Run `node scripts/benchmark-0.0.9.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.9.test.mjs`. Default mode is network-free fake/in-process only and emits environment, scenario mode, throughput, p50/p95 latency, heap, disk bytes, process counts, zero external cost, backpressure, and resource-limit signals for repository list/search, Git status, and browser open/snapshot/action/close. Optional `PRISM_BENCH_DOCKER=1` (with `PRISM_TEST_DOCKER_*`) and `PRISM_BENCH_PLAYWRIGHT=1` append real local Docker / protected Playwright rows. These are evidence fields, not CI timing gates. (Node-era command)
|
|
377
377
|
|
|
378
378
|
2026-07-21 baseline: Node v24.18.0, Linux x64, 100 iterations/scenario, network=false, credentials=false, docker=false, playwright=false.
|
|
379
379
|
|
|
@@ -388,7 +388,7 @@ Rows exercise shipped repository/Git helpers and fake Playwright APIs only. Real
|
|
|
388
388
|
|
|
389
389
|
## Release 0.0.8 reproducible synthetic evidence
|
|
390
390
|
|
|
391
|
-
Run `node scripts/benchmark-0.0.8.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000. Script uses no network/credentials and emits environment, throughput, p50/p95 latency, heap, synthetic disk bytes, zero external cost, and backpressure signals. These are evidence fields, not CI timing gates.
|
|
391
|
+
Run `node scripts/benchmark-0.0.8.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000. Script uses no network/credentials and emits environment, throughput, p50/p95 latency, heap, synthetic disk bytes, zero external cost, and backpressure signals. These are evidence fields, not CI timing gates. (Node-era command)
|
|
392
392
|
|
|
393
393
|
2026-07-20 baseline: Node v24.18.0, Linux x64, 1,000 operations/scenario.
|
|
394
394
|
|
|
@@ -419,7 +419,7 @@ Durable coding plan/checkpoint defaults/hard caps: plan Markdown 256 KiB/1 MiB;
|
|
|
419
419
|
|
|
420
420
|
Browser automation defaults/hard caps from the `browser` subpath: pages 4/16; actions 100/256; queued actions 16/64; snapshot refs 2,000/10,000; depth 30/100; snapshot bytes 256 KiB/2 MiB; navigation 30 s/120 s; action 10 s/60 s; wait 30 s/120 s; run wall 20 min/30 min; popups 4/16; dialogs 16/64; listeners 64/256; action input 64 KiB/256 KiB; close grace 5 s/30 s; network requests 1,000/10,000 with 10/32 redirects per request and 8/32 WebSockets; screenshots 16/64 with 16/64 megapixels and 10 MiB/32 MiB encoded; uploads 8/32 files, 16 MiB/64 MiB each, 64 MiB/256 MiB aggregate; downloads 8/32 files, 32 MiB/256 MiB each, 64 MiB/512 MiB aggregate. Caps charge before context/page/action/queue/snapshot/network/artifact retention. Host supplies Playwright and egress proxy attestation; package import launches nothing.
|
|
421
421
|
|
|
422
|
-
0.0.14 co-work defaults/hard caps (frozen in [Phase 9 evidence](_evidence/review-coverage-2026-07-25-phase-9.md)): conversation thread list pages 50/200, active branches per thread 16/64, replay/export page 100/500 events; artifact revisions per artifact 32/128, artifacts per thread 64/256, metadata record 8/64 KiB, preview 16/64 KiB, citations 32/128 (2/8 KiB each), delivery-link TTL 5 min/24 h, delivery token 4/16 KiB, compare exactly 2 revisions; memory retention batch 500/5000; proactive capability TTL 24 h/31 d, capability token record 16 KiB; browser checkpoint URL 8 KiB/16 KiB, domain-state hash 256 B/1 KiB, host-data ref 2 KiB/8 KiB, 16/64 checkpoints per run; device stream chunk 1 MiB/8 MiB, concurrent device sessions per identity 1/4 (device wall/turns/tool calls consume shared `RunLimits`). All caps charge before persist/emit and fail closed on overflow. Benchmark placeholder: `node scripts/benchmark-0.0.14.mjs` (release Task 12) reports conversation replay, memory injection/consent, artifact revision/delivery, AG-UI co-work mapping, and connector refresh overhead against these budgets.
|
|
422
|
+
0.0.14 co-work defaults/hard caps (frozen in [Phase 9 evidence](_evidence/review-coverage-2026-07-25-phase-9.md)): conversation thread list pages 50/200, active branches per thread 16/64, replay/export page 100/500 events; artifact revisions per artifact 32/128, artifacts per thread 64/256, metadata record 8/64 KiB, preview 16/64 KiB, citations 32/128 (2/8 KiB each), delivery-link TTL 5 min/24 h, delivery token 4/16 KiB, compare exactly 2 revisions; memory retention batch 500/5000; proactive capability TTL 24 h/31 d, capability token record 16 KiB; browser checkpoint URL 8 KiB/16 KiB, domain-state hash 256 B/1 KiB, host-data ref 2 KiB/8 KiB, 16/64 checkpoints per run; device stream chunk 1 MiB/8 MiB, concurrent device sessions per identity 1/4 (device wall/turns/tool calls consume shared `RunLimits`). All caps charge before persist/emit and fail closed on overflow. Benchmark placeholder: `node scripts/benchmark-0.0.14.mjs` (release Task 12) reports conversation replay, memory injection/consent, artifact revision/delivery, AG-UI co-work mapping, and connector refresh overhead against these budgets. (Node-era command)
|
|
423
423
|
|
|
424
424
|
Current surfaces:
|
|
425
425
|
|
|
@@ -797,13 +797,13 @@ Optional health/drain/rate-limit/replay/deployment-lease helpers on `@arnilo/pri
|
|
|
797
797
|
|
|
798
798
|
### 0.0.13 Phase 8 identity, policy, router, and work connectors (2026-07-24)
|
|
799
799
|
|
|
800
|
-
Enterprise governance and connector caps (defaults / hard). Timings: `node scripts/benchmark-0.0.13.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.13.test.mjs`. Default mode is network-free and reports identity/policy/router/work-connector/deployment throughput and p50/p95 with frozen budget refs in the report JSON. Bounds and hostile-input fixtures—not these host-local timings—are release gates.
|
|
800
|
+
Enterprise governance and connector caps (defaults / hard). Timings: `node scripts/benchmark-0.0.13.mjs`; `PRISM_BENCH_ITERATIONS` accepts 10–100,000 (default 100). Schema/bounds test: `node --test scripts/benchmark-0.0.13.test.mjs`. Default mode is network-free and reports identity/policy/router/work-connector/deployment throughput and p50/p95 with frozen budget refs in the report JSON. Bounds and hostile-input fixtures—not these host-local timings—are release gates. (Node-era command)
|
|
801
801
|
|
|
802
802
|
Offline behavior tests (identity propagation, policy export, router deny paths, fake CLI argv) are release gates; live tenant canaries remain operator-gated.
|
|
803
803
|
|
|
804
804
|
### 0.3.x Phase 39 Obscura browser-engine envelopes (2026-08-29)
|
|
805
805
|
|
|
806
|
-
`obscura` binary-backed legs, network-free, driven by a deterministic fake CLI: `
|
|
806
|
+
`obscura` binary-backed legs, network-free, driven by a deterministic fake CLI: `bun scripts/benchmark-obscura.mjs` (3 runs, medians vs reviewed ceilings; artifact `scripts/benchmark-obscura.json`). Startup leg probes SIG-0 liveness after spawn — a real host waits on its readiness endpoint inside the same bound.
|
|
807
807
|
|
|
808
808
|
| Leg | Median (3 runs) | Ceiling | Notes |
|
|
809
809
|
| --- | --- | --- | --- |
|
package/docs/prompt-registry.md
CHANGED
|
@@ -67,7 +67,7 @@ const postgres = await createPostgresPromptStore({
|
|
|
67
67
|
});
|
|
68
68
|
```
|
|
69
69
|
|
|
70
|
-
SQLite uses `
|
|
70
|
+
SQLite uses the Bun runtime's `bun:sqlite`; PostgreSQL uses a caller-supplied or adapter-owned `pg` pool. Both adapters use the package-owned `prism_prompts` and `prism_prompt_labels` tables, exact ownership predicates, bound values, and an indexed label lookup. Startup applies checked `001_init` migration history and refuses checksum drift. SQLite exposes `applySqlitePromptMigrations` for managed setup tests; PostgreSQL migration setup is guarded by `pg_advisory_xact_lock`.
|
|
71
71
|
|
|
72
72
|
## Eval-gated promotion
|
|
73
73
|
|
|
@@ -25,28 +25,28 @@ Do not use provider packages as a package manager, credential store, env loader,
|
|
|
25
25
|
|
|
26
26
|
| adapter package | version |
|
|
27
27
|
| --- | --- |
|
|
28
|
-
| `@arnilo/prism-providers/ai-sdk` | 0.
|
|
29
|
-
| `@arnilo/prism-providers/alibaba` | 0.
|
|
30
|
-
| `@arnilo/prism-providers/anthropic` | 0.
|
|
31
|
-
| `@arnilo/prism-providers/azure` | 0.
|
|
32
|
-
| `@arnilo/prism-providers/bedrock` | 0.
|
|
33
|
-
| `@arnilo/prism-providers/clinepass` | 0.
|
|
34
|
-
| `@arnilo/prism-providers/commandcode` | 0.
|
|
35
|
-
| `@arnilo/prism-providers/deepseek` | 0.
|
|
36
|
-
| `@arnilo/prism-providers/google` | 0.
|
|
37
|
-
| `@arnilo/prism-providers/hyper` | 0.
|
|
38
|
-
| `@arnilo/prism-providers/kimi` | 0.
|
|
39
|
-
| `@arnilo/prism-providers/laya` | 0.
|
|
40
|
-
| `@arnilo/prism-providers/model-discovery` | 0.
|
|
41
|
-
| `@arnilo/prism-providers/neuralwatt` | 0.
|
|
42
|
-
| `@arnilo/prism-providers/ollama` | 0.
|
|
43
|
-
| `@arnilo/prism-providers/openai` | 0.
|
|
44
|
-
| `@arnilo/prism-providers/opencode-go` | 0.
|
|
45
|
-
| `@arnilo/prism-providers/openrouter` | 0.
|
|
46
|
-
| `@arnilo/prism-providers/typesafe` | 0.
|
|
47
|
-
| `@arnilo/prism-providers/vertex` | 0.
|
|
48
|
-
| `@arnilo/prism-providers/xai` | 0.
|
|
49
|
-
| `@arnilo/prism-providers/zai` | 0.
|
|
28
|
+
| `@arnilo/prism-providers/ai-sdk` | 0.12.0 |
|
|
29
|
+
| `@arnilo/prism-providers/alibaba` | 0.12.0 |
|
|
30
|
+
| `@arnilo/prism-providers/anthropic` | 0.12.0 |
|
|
31
|
+
| `@arnilo/prism-providers/azure` | 0.12.0 |
|
|
32
|
+
| `@arnilo/prism-providers/bedrock` | 0.12.0 |
|
|
33
|
+
| `@arnilo/prism-providers/clinepass` | 0.12.0 |
|
|
34
|
+
| `@arnilo/prism-providers/commandcode` | 0.12.0 |
|
|
35
|
+
| `@arnilo/prism-providers/deepseek` | 0.12.0 |
|
|
36
|
+
| `@arnilo/prism-providers/google` | 0.12.0 |
|
|
37
|
+
| `@arnilo/prism-providers/hyper` | 0.12.0 |
|
|
38
|
+
| `@arnilo/prism-providers/kimi` | 0.12.0 |
|
|
39
|
+
| `@arnilo/prism-providers/laya` | 0.12.0 |
|
|
40
|
+
| `@arnilo/prism-providers/model-discovery` | 0.12.0 |
|
|
41
|
+
| `@arnilo/prism-providers/neuralwatt` | 0.12.0 |
|
|
42
|
+
| `@arnilo/prism-providers/ollama` | 0.12.0 |
|
|
43
|
+
| `@arnilo/prism-providers/openai` | 0.12.0 |
|
|
44
|
+
| `@arnilo/prism-providers/opencode-go` | 0.12.0 |
|
|
45
|
+
| `@arnilo/prism-providers/openrouter` | 0.12.0 |
|
|
46
|
+
| `@arnilo/prism-providers/typesafe` | 0.12.0 |
|
|
47
|
+
| `@arnilo/prism-providers/vertex` | 0.12.0 |
|
|
48
|
+
| `@arnilo/prism-providers/xai` | 0.12.0 |
|
|
49
|
+
| `@arnilo/prism-providers/zai` | 0.12.0 |
|
|
50
50
|
<!-- generated:package-truth:providers end -->
|
|
51
51
|
|
|
52
52
|
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
# System One decision calls
|
|
2
|
+
|
|
3
|
+
## What it does
|
|
4
|
+
|
|
5
|
+
`@arnilo/prism-providers/decisions` is the host-facing typed decision call over the shared `POST /v1/systemone` wire client used by both [TypeSafe Jev](typesafe.md) and [Laya](laya.md). One call returns the raw `answers` (noul probability, choice probabilities and confidence, score with legend and distribution), the checkpoint that actually answered, mapped usage, and wall-clock timing. Nothing is coerced: an answer is never converted to a boolean, rounded to an integer, replaced by a synthetic `0.5`, or flattened to a schema value. The subpath re-exports the question/answer types, the typed error family, the pre-fetch limits, and the schema compiler (`compileSystemOneQuestions`, `compileSystemOneState`) for hosts that build requests from JSON Schema or Prism messages.
|
|
6
|
+
|
|
7
|
+
## When to use it
|
|
8
|
+
|
|
9
|
+
Use it when a host needs calibrated decision output for confidence gating, shadow classification, calibration runs, or custom telemetry, and wants the raw probabilities plus the responding checkpoint. Use the `typesafe` or `laya` `AIProvider` adapter instead when the call is a normal structured-output request and only schema-valid JSON text is needed. The call is host-invoked; it never registers a provider, model, tool, or automatic agent hook, and a decision answer is never effect authority.
|
|
10
|
+
|
|
11
|
+
## Inputs / request
|
|
12
|
+
|
|
13
|
+
`askSystemOneDecisions(body, options, signal?)` takes the wire body plus client options.
|
|
14
|
+
|
|
15
|
+
| Body field | Type | Purpose |
|
|
16
|
+
| --- | --- | --- |
|
|
17
|
+
| `model` | `string` | Sent verbatim. Advisory: the answering checkpoint can differ and is reported in the result. |
|
|
18
|
+
| `state` | `string`, JSON object, or array | Conversation content only. Questions must never be written into `state` — the model judges that text instead of answering the question. |
|
|
19
|
+
| `questions` | `Record<string, SystemOneQuestion>` | Keyed by question id; 1–256 questions. `noul` takes optional `criteria.true`/`criteria.false` strings, `choice` an object of 2–255 option → description (`string` or `null`), `score` an array of 2–10 level descriptions. Every question needs non-empty `instructions`. |
|
|
20
|
+
|
|
21
|
+
| Option | Type | Purpose |
|
|
22
|
+
| --- | --- | --- |
|
|
23
|
+
| `provider` | `string` | Label used in errors and credential resolution, e.g. `"TypeSafe Jev"` or `"Laya"`. |
|
|
24
|
+
| `baseUrl` | `string` | Base URL without the path; `/v1/systemone` is appended and trailing slashes are trimmed. |
|
|
25
|
+
| `apiKey` | `CredentialValueSource` | Optional Bearer credential, resolved once per call under `provider`. Absent means no `Authorization` header (anonymous `laya-serve`). |
|
|
26
|
+
| `fetch` | `typeof fetch` | Fetch implementation for tests and hosts. |
|
|
27
|
+
| `maxRetries`, `baseDelayMs`, `maxDelayMs`, `jitter`, `random` | numbers | Forwarded to the shared bounded retry policy; retries after the first attempt default to 2, with jittered backoff honoring `Retry-After`. |
|
|
28
|
+
| `timeoutMs` | `number` | Whole-call deadline in milliseconds, retries included. Must be a positive finite number. |
|
|
29
|
+
| `maxStateBytes` | `number` | Serialized `state` byte ceiling. Default `DEFAULT_MAX_SYSTEMONE_STATE_BYTES` (262,144). |
|
|
30
|
+
| `signal` | `AbortSignal` | Third argument. Composes with `timeoutMs`: whichever aborts first ends the call. |
|
|
31
|
+
|
|
32
|
+
Pre-fetch gates run before credential resolution or fetch: question count and criteria bounds mirror the schema compiler, `state` must serialize under `maxStateBytes`, `instructions` must be non-empty, and `model`/`timeoutMs`/`maxStateBytes` must be valid.
|
|
33
|
+
|
|
34
|
+
## Outputs / response / events
|
|
35
|
+
|
|
36
|
+
`SystemOneDecisionResult`:
|
|
37
|
+
|
|
38
|
+
| Field | Type | Meaning |
|
|
39
|
+
| --- | --- | --- |
|
|
40
|
+
| `model` | `string` | Checkpoint that actually answered. |
|
|
41
|
+
| `answers` | `Record<string, SystemOneAnswer>` | Raw answers by question id: `noul` probability, `choice` with `choice`, `probabilities`, and `confidence`, `score` with `score`, optional `legend`, `probabilities`, and `confidence`. |
|
|
42
|
+
| `usage` | `Usage?` | Input tokens mapped as `inputTokens`/`totalTokens`; `outputTokens` stays 0. Omitted when the API reports none. |
|
|
43
|
+
| `timingMs` | `number` | Wall-clock milliseconds for the whole call, retries included. |
|
|
44
|
+
|
|
45
|
+
There is one HTTP round trip per attempt and no streaming, polling, or extra discovery request.
|
|
46
|
+
|
|
47
|
+
Failures are typed and never become a decision:
|
|
48
|
+
|
|
49
|
+
| Error | When | `status` |
|
|
50
|
+
| --- | --- | --- |
|
|
51
|
+
| `SystemOneError` | Local bound violation (no request sent), or any other non-retryable HTTP status | `0` for local violations |
|
|
52
|
+
| `SystemOneAuthError` | `401`; never retried | `401` |
|
|
53
|
+
| `SystemOneInvalidRequestError` | `422`; message carries the API field detail | `422` |
|
|
54
|
+
| `SystemOneRetryExhaustedError` | `429`/`5xx` survived every bounded retry | last status |
|
|
55
|
+
| `SystemOneAbortedError` | Caller signal or `timeoutMs` deadline ended the call | `0` |
|
|
56
|
+
|
|
57
|
+
Other transport failures (for example a DNS error) propagate unchanged. HTTP error messages are built through the redacting error builder, so an echoed body and the API key never reach logs unredacted.
|
|
58
|
+
|
|
59
|
+
## Request/response example
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{
|
|
63
|
+
"model": "jev-latest",
|
|
64
|
+
"state": "rm -rf ./build",
|
|
65
|
+
"questions": {
|
|
66
|
+
"route": {
|
|
67
|
+
"type": "choice",
|
|
68
|
+
"instructions": "How should this be handled?",
|
|
69
|
+
"criteria": { "run": "run", "ask": "ask", "reject": "reject" }
|
|
70
|
+
},
|
|
71
|
+
"irreversible": { "type": "noul", "instructions": "Would running this destroy data?" }
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
```json
|
|
77
|
+
{
|
|
78
|
+
"model": "jev-1.13.0",
|
|
79
|
+
"answers": {
|
|
80
|
+
"route": { "type": "choice", "choice": "ask", "probabilities": { "run": 0.12, "ask": 0.71, "reject": 0.17 }, "confidence": 0.71 },
|
|
81
|
+
"irreversible": { "type": "noul", "noul": 0.91 }
|
|
82
|
+
},
|
|
83
|
+
"usage": { "input_tokens": 96, "output_tokens": 0 }
|
|
84
|
+
}
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
The result keeps `0.91` and every probability; it does not return `true`, `"ask"`, or a synthesized default.
|
|
88
|
+
|
|
89
|
+
## Implementation example
|
|
90
|
+
|
|
91
|
+
```ts
|
|
92
|
+
import { askSystemOneDecisions } from "@arnilo/prism-providers/decisions";
|
|
93
|
+
|
|
94
|
+
const result = await askSystemOneDecisions(
|
|
95
|
+
{
|
|
96
|
+
model: "jev-latest",
|
|
97
|
+
state: conversationText,
|
|
98
|
+
questions: {
|
|
99
|
+
route: { type: "choice", instructions: "How should this be handled?", criteria: { run: "run", ask: "ask", reject: "reject" } },
|
|
100
|
+
irreversible: { type: "noul", instructions: "Would running this destroy data?" },
|
|
101
|
+
},
|
|
102
|
+
},
|
|
103
|
+
{ provider: "TypeSafe Jev", baseUrl: "https://api.typesafe.ai", apiKey: process.env.TYPESAFE_API_KEY, timeoutMs: 2_000 },
|
|
104
|
+
signal,
|
|
105
|
+
);
|
|
106
|
+
|
|
107
|
+
if (result.answers.route.type === "choice" && (result.answers.route.confidence ?? 0) < 0.6) {
|
|
108
|
+
return escalateToHostPolicy(result); // a decision never authorizes an effect by itself
|
|
109
|
+
}
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
For Laya, pass `provider: "Laya"` and the `laya-serve` base URL (or omit `apiKey` for an anonymous local server). For compiled schemas, call `compileSystemOneQuestions(schema)` and send the returned questions.
|
|
113
|
+
|
|
114
|
+
## Extension and configuration notes
|
|
115
|
+
|
|
116
|
+
- Engine-specific calibration controls are not forwarded: the supported wire contract is verified for `model`, `state`, `questions`, and Bearer auth only. Temperature/floor refits belong in host configuration until the wire documents them.
|
|
117
|
+
- `maxRetries`, backoff, jitter, and `random` are the shared client's knobs, not a second retry loop.
|
|
118
|
+
- The wrapper is re-exported from `@arnilo/prism-providers/typesafe` and `@arnilo/prism-providers/laya` for hosts that already import the provider package.
|
|
119
|
+
- `compileSystemOneState` converts Prism messages to wire state (single user text message → string, else `[{ role, text }]` with non-text parts dropped); questions stay out of it by construction.
|
|
120
|
+
|
|
121
|
+
## Security and performance notes
|
|
122
|
+
|
|
123
|
+
- **Egress.** `state` text and compiled questions leave the process for the configured `baseUrl`; TypeSafe's hosted default or a self-hosted `laya-serve` deployment. Do not put secrets in `state`, and do not write questions into it.
|
|
124
|
+
- **Credentials.** One resolution per call under `provider`; the same value serves transport and redaction and is never logged.
|
|
125
|
+
- **Cost and latency.** One round trip per attempt; questions are free in parallel, state tokens are the cost. `timingMs` includes retries; `timeoutMs` bounds them altogether.
|
|
126
|
+
- **Fail closed.** Local bounds and deadline validation fail before any fetch. Abort, deadline, auth, invalid-request, and exhausted-retry outcomes are distinguishable from a measured probability; no failure fabricates an answer.
|
|
127
|
+
- **Authority.** Decision answers are advisory. Authorization, capability grants, money paths, and commit rechecks stay host-side and deterministic.
|
|
128
|
+
|
|
129
|
+
## Related APIs
|
|
130
|
+
|
|
131
|
+
- [TypeSafe Jev](typesafe.md): hosted adapter and schema mapping.
|
|
132
|
+
- [Laya](laya.md): self-hosted adapter over the same wire.
|
|
133
|
+
- [Structured output](../structured-output.md): `options.structuredOutput` contract the adapters require.
|
|
134
|
+
- [Credentials and redaction](../credentials-and-redaction.md): `CredentialValueSource` and error redaction.
|
package/docs/providers/laya.md
CHANGED
|
@@ -6,6 +6,8 @@
|
|
|
6
6
|
|
|
7
7
|
The package registers provider `laya`, checkpoints `laya`, `laya-multilingual`, and `laya-typed-decisions`, and an optional `api_key` auth method. The server Router picks the checkpoint. The `model` field is advisory and is not a client-side force.
|
|
8
8
|
|
|
9
|
+
For raw probabilities, confidence, and the actual responding checkpoint instead of rendered JSON, use the shared [`decisions`](decisions.md) subpath (re-exported from this package); the adapter deliberately does not copy those values onto events.
|
|
10
|
+
|
|
9
11
|
## When to use it
|
|
10
12
|
|
|
11
13
|
Use it when a host runs `laya-serve` locally or on another machine and wants the same structured decision contract as Jev without a hosted account. Do not use it as a general chat model or a tool-calling agent.
|
|
@@ -107,6 +109,7 @@ The default `http://localhost:8000` is plaintext loopback only. A non-loopback `
|
|
|
107
109
|
|
|
108
110
|
## Related APIs
|
|
109
111
|
|
|
112
|
+
- [System One decisions](decisions.md): raw typed decision calls over the same wire, with probabilities, confidence, actual model, usage, and timing.
|
|
110
113
|
- [TypeSafe Jev](typesafe.md): hosted twin. Schema mapping and event sequence are defined there.
|
|
111
114
|
- [Structured output](../structured-output.md): `options.structuredOutput` contract this adapter requires.
|
|
112
115
|
- [Provider packages](../provider-packages.md): registration and auth-method shape.
|
|
@@ -6,6 +6,8 @@
|
|
|
6
6
|
|
|
7
7
|
The package registers provider `typesafe`, models `jev-latest` and `jev-preview`, and an `api_key` auth method through `createExtensionKernel().load([...])`. Versioned pins (`jev-1.13.0`) go through `defineTypeSafeModel`.
|
|
8
8
|
|
|
9
|
+
For raw probabilities, confidence, and the actual responding checkpoint instead of rendered JSON, use the shared [`decisions`](decisions.md) subpath (re-exported from this package); the adapter deliberately does not copy those values onto events.
|
|
10
|
+
|
|
9
11
|
## When to use it
|
|
10
12
|
|
|
11
13
|
Use it when a host wants a hosted yes/no, pick-one, or rubric decision and already has a JSON object schema. Do not use it as a general chat model, a tool-calling agent, or a streaming text model.
|
|
@@ -75,7 +77,7 @@ Curated models declare `capabilities: { input: ["text"], output: ["text"], tools
|
|
|
75
77
|
| `done` | `stopReason: "end_turn"`, same usage when present. |
|
|
76
78
|
| `error` | Terminal event for gate failures, HTTP failures, and render failures. `401` is not retried. `422` is not retried and the message includes the API field detail. `429` and `5xx` (including `529`) retry, honoring `Retry-After`. |
|
|
77
79
|
|
|
78
|
-
Booleans come from noul vs `boolean_threshold`. Enums come from the choice string (whole-number options stay numbers). Rubric integers are `Math.round` of the score, half rounds up, clamped to the rubric. Nested objects are reassembled from dotted ids. Confidence, probabilities, and score distributions are not copied onto events; the text must stay schema-valid.
|
|
80
|
+
Booleans come from noul vs `boolean_threshold`. Enums come from the choice string (whole-number options stay numbers). Rubric integers are `Math.round` of the score, half rounds up, clamped to the rubric. Nested objects are reassembled from dotted ids. Confidence, probabilities, and score distributions are not copied onto events; the text must stay schema-valid. Success bodies are validated before output: malformed answer types, missing/non-finite numeric values, probabilities outside `0..1`, and invalid token counts produce an error rather than a coerced decision. Finite rubric scores still round and clamp as described above.
|
|
79
81
|
|
|
80
82
|
## Request/response example
|
|
81
83
|
|
|
@@ -133,12 +135,13 @@ A request must carry `options.structuredOutput`. The answer arrives as one text
|
|
|
133
135
|
## Security and performance notes
|
|
134
136
|
|
|
135
137
|
- **Egress.** State text and compiled questions leave the process for TypeSafe's hosted API (`https://api.typesafe.ai` unless `baseUrl` overrides it). Do not put secrets in the state.
|
|
136
|
-
- **Credentials.** The bearer token is resolved
|
|
138
|
+
- **Credentials.** The bearer token is resolved once per request under the provider id; that same value is used for transport and error redaction. It is never logged.
|
|
137
139
|
- **Cost and latency.** One `POST /v1/systemone` round trip per structured-output request. Questions are free in parallel; state tokens are the cost. Output tokens are not billed (`output: 0`). There is no streaming and no second hop for model discovery.
|
|
138
140
|
- **Fail closed.** Missing schema, tools, an out-of-range threshold, and unsupported fields fail before fetch.
|
|
139
141
|
|
|
140
142
|
## Related APIs
|
|
141
143
|
|
|
144
|
+
- [System One decisions](decisions.md): raw typed decision calls over the same wire, with probabilities, confidence, actual model, usage, and timing.
|
|
142
145
|
- [Laya](laya.md): same wire and schema mapping against a self-hosted `laya-serve`.
|
|
143
146
|
- [Structured output](../structured-output.md): `options.structuredOutput` contract this adapter requires.
|
|
144
147
|
- [Provider packages](../provider-packages.md): registration and auth-method shape.
|
package/docs/rag.md
CHANGED
|
@@ -34,7 +34,7 @@ Document lifecycle:
|
|
|
34
34
|
| `resolveReranker(config)` | Declarative reranker config (`kind: "local" \| "tei" \| "openai-compatible" \| "voyage" \| "fake" \| "none"`) → `Reranker`. |
|
|
35
35
|
| `createGoogleDriveConnector({ tokenProvider, resolveAccess })` | Drive `files.list` + `changes.list` connector. Host maps permissions; watch payloads are not authorization. |
|
|
36
36
|
| `DocumentLoader` / `Parser` | Small host-replaceable seams. `@arnilo/prism-memory/rag/loaders` and `/rag/parsers` export reference adapters. |
|
|
37
|
-
| `textParser` / `markdownParser` / `htmlParser` / `pdfParser` | UTF-8 text, Markdown, script/style-stripping HTML, and uncompressed-text PDF parsers. |
|
|
37
|
+
| `textParser` / `markdownParser` / `htmlParser` / `pdfParser` | UTF-8 text, Markdown, script/style-stripping HTML, and uncompressed-text PDF parsers. A host parser from `createDocumentIngest` ([Document extraction](document-extraction.md)) can replace `pdfParser`; it must throw on failed OCR so `replaceDocument` does not index. Do not raise `HARD_MAX_PARSE_MS_CAP` (30 s). |
|
|
38
38
|
|
|
39
39
|
Index/retrieve:
|
|
40
40
|
|
|
@@ -112,7 +112,7 @@ const result = await propagator.propagate("doc:erp-lead");
|
|
|
112
112
|
- `propagate(sourceId)` expands the source through `_lineage.sourceIds` (`collectInvalidationIds`, depth 8) into a closed id set, tombstones **all** of it with the propagator's resolved reason (`forgotten` by default; `legal_hold` stamps `hold: true`) inside one store transaction, then runs every registered handler with `{ sourceId, ids, scope, reason, signal }`. Handlers return how many artifacts they removed (reported per `kind` in `layers`). The context `reason` is the single source of truth: a handler's own `reason` option is only the fallback for a hand-built context, so a `legal_hold` propagation cannot land in a handler's tombstones as `forgotten`.
|
|
113
113
|
- Tombstones, not deletions, for derived rows: rows stay for explainability (`recall({ explain: true })` reports the invalidation), and lineage links never dangle. Handlers own physical removal (chunk rows, files, ledger entries).
|
|
114
114
|
- Retrieval is belt-and-suspenders: `retrieveContext()` reads per-scope invalidations before assembly and drops any candidate whose record id, `_lineage.sourceIds`, or `_rag.sourceId` is tombstoned — so a delete that lands after the query legs read rows still returns zero hits. The split matters for direct store users: the store's own SQL predicate filters by record id and `_lineage` edge, while a source's *own* chunk rows are covered by the `_rag.sourceId` rule at the retrieval boundary (or removed physically by the `rag` handler) — a raw `store.query()` is not a recall path.
|
|
115
|
-
- `HARD_PROPAGATION_EDGES` (4,096) is the one-pass privileged ceiling; over it the whole delete rejects (fail-closed), never a half-tombstoned document. Each store `invalidate` call carries at most `HARD_INVALIDATION_BATCH` (64) entries. On a durable store that shape holds: PostgreSQL/pgvector tombstones 1,001 rows (1,000 derived chunk rows + the source root) in **one transaction and 22 statements** (16 of them `HARD_INVALIDATION_BATCH`-sized `INSERT`s), measured at **29–155 ms** across runs on an AMD Ryzen 9 PRO 7940HS against `pgvector/pgvector:pg16` (more under parallel load) — the durable counterpart of the in-memory suite's 1k-under-2s check, and evidence rather than a gate. Re-run it with `PRISM_TEST_POSTGRES_URL=…
|
|
115
|
+
- `HARD_PROPAGATION_EDGES` (4,096) is the one-pass privileged ceiling; over it the whole delete rejects (fail-closed), never a half-tombstoned document. Each store `invalidate` call carries at most `HARD_INVALIDATION_BATCH` (64) entries. On a durable store that shape holds: PostgreSQL/pgvector tombstones 1,001 rows (1,000 derived chunk rows + the source root) in **one transaction and 22 statements** (16 of them `HARD_INVALIDATION_BATCH`-sized `INSERT`s), measured at **29–155 ms** across runs on an AMD Ryzen 9 PRO 7940HS against `pgvector/pgvector:pg16` (more under parallel load) — the durable counterpart of the in-memory suite's 1k-under-2s check, and evidence rather than a gate. Re-run it with `PRISM_TEST_POSTGRES_URL=… bun run test:postgres` (`packages/memory/src/__tests__/postgres-propagation.integration.test.ts`); the leg also proves the store's own SQL predicate hides the tombstoned rows, not only the in-app guard, and that a denied propagation opens no transaction at all.
|
|
116
116
|
- Deletion is privileged: `authorization` is required, tenant-checked, and enforced through the store's existing `checkSourceAccess` ACL when the store declares `authorization: "acl"` (missing grant → `MemoryScopeError` before anything is written). Retrieval paths never construct a propagator.
|
|
117
117
|
- Observational memory registers its own leg: `createObservationalMemoryDropHandler({ session, appendEntry })` (from `@arnilo/prism-memory/compaction/observational-memory`) folds the session ledger once per propagation and writes one `om.observations.dropped` entry for the observations that rest on a tombstoned record id; see [observational memory](compaction-observational-memory.md).
|
|
118
118
|
|
|
@@ -152,6 +152,18 @@ const blocks = buildObservationalMemoryContextBlocks(entries, { invalidatedIds:
|
|
|
152
152
|
- One `listInvalidatedIds` read per projection build (one scope read, `corrected` entries stay), and the recipe adds no work beyond the propagator: the same `layers` result already answers per-leg counts, so nothing is re-read to report it.
|
|
153
153
|
- The fabric leg is the one that cannot be left out: a note names its document by `metadata.path`, so no `_lineage` edge exists to walk and a deleted path would otherwise keep being served. `createFabricRepointHandler()` tombstones the notes recorded against the deleted id in the same pass (plan 102 Task 11), and the same handler follows a `repointSource()` move — see the re-point section below.
|
|
154
154
|
|
|
155
|
+
## Revocation through derived context
|
|
156
|
+
|
|
157
|
+
Grant rechecks and tombstones cover retrieval; derived context (observational memory, wiki pages, fabric notes) needs the propagation pass above. `examples/revocation-propagation.ts` runs the whole chain offline on in-memory stores:
|
|
158
|
+
|
|
159
|
+
1. **Access loss for one principal** (`setSourceAccess` with the grant narrowed) excludes the source for that principal, still serves it to another, keeps the rows, and writes no tombstone.
|
|
160
|
+
2. **Deletion** (`createDeletionPropagator` + `createRagDeletionHandler`) tombstones the source, deletes its chunk rows, and excludes it for every principal — including one that still held a grant.
|
|
161
|
+
3. **The observational-memory leg runs in two observable steps:** `listInvalidatedIds(store, scope)` feeds `invalidatedIds` into `buildObservationalMemoryContextBlocks()` so a revoked observation is withheld before the drop entry lands, and `createObservationalMemoryDropHandler()` later writes the `om.observations.dropped` entry; both paths render the same memory.
|
|
162
|
+
4. **A revoke landing while a reranker runs** is withheld by the post-rerank gate and reported through `onAccessDenied` (`no_grant`), so a mid-flight grant change cannot leak.
|
|
163
|
+
5. **A thrown grant check** fails closed as `check_failed` with the store error redacted and capped at 256 chars; the query still completes with the remaining hits.
|
|
164
|
+
|
|
165
|
+
The example also keeps the disclosure boundary honest: a report generated before the revoke did contain the authorized text, and revocation prevents subsequent exposure — it does not claim to erase prior external disclosure.
|
|
166
|
+
|
|
155
167
|
## Grant recheck and re-pointing
|
|
156
168
|
|
|
157
169
|
Retrieval never trusts a grant snapshot. `retrieveContext()` re-asks the store for **each distinct source** it is about to inject, on both sides of the reranker:
|
|
@@ -260,10 +272,10 @@ const result = await retrieveContext("How do approvals work?", { embedder, store
|
|
|
260
272
|
```
|
|
261
273
|
|
|
262
274
|
- `resolveReranker({ kind: "local" })` is the zero-config path. The model runtime is a host seam exactly like `Embedder`: `createLocalReranker({ model?, runtime?, onLoad?, cacheDir?, dtype?, device?, allowRemoteModels? })`. Pass `runtime: { load(model) → { id, score({ query, documents, signal }) } }` to inject a runtime the host already owns (transformers.js, onnxruntime-node, llama.cpp). With no `runtime`, the built-in loader resolves `@huggingface/transformers` at first use — the package declares no inference dependency (no new dependency name in any manifest) and nothing resolves it at build/install time.
|
|
263
|
-
- Sizing trade-off: the download is one-time and host-cached, and per-query latency is CPU-bound and grows with candidates × tokens, so keep `topK`/`queryCandidates` near what recall actually needs — the reranker reorders what retrieval returned, it cannot recover a chunk the candidate pool never returned. Measured on one corpus (24 queries / 96 chunks: one answering chunk + three mention-only chunks per query, k=5, `Xenova/bge-reranker-base` q8 on x86 CPU, vector-only) with two embedders: the deterministic lexical `createHashEmbedder` baseline gives recall@5 **0.21 → 0.79** and recall@20 **0.63** at the package default 20-candidate pool, while the semantic `Xenova/all-MiniLM-L6-v2` q8/cpu (384 dims) gives **0.79 → 0.79** and recall@20 **1.00** — the semantic baseline starts at the lexical reranked number, the reranker's lift is 0.000 on this corpus, and the pool is not the binding constraint. Read the row for your own embedder: with a lexical/deterministic embedder the pool bound is what to raise first; with a semantic one the reranker is ordering quality only and the residual misses need better retrieval. Top-50 median was 95–289 ms across runs. Corpus, misses, pool-bound numbers, latency, cache state, and the semantic side-by-side live in [`docs/_evidence/phase111-reranker-semantic-recall.md`](_evidence/phase111-reranker-semantic-recall.md), with the lexical control in [`docs/_evidence/phase102-local-rerank-latency.md`](_evidence/phase102-local-rerank-latency.md); both are regenerated by `PRISM_TEST_LOCAL_RERANK=1
|
|
275
|
+
- Sizing trade-off: the download is one-time and host-cached, and per-query latency is CPU-bound and grows with candidates × tokens, so keep `topK`/`queryCandidates` near what recall actually needs — the reranker reorders what retrieval returned, it cannot recover a chunk the candidate pool never returned. Measured on one corpus (24 queries / 96 chunks: one answering chunk + three mention-only chunks per query, k=5, `Xenova/bge-reranker-base` q8 on x86 CPU, vector-only) with two embedders: the deterministic lexical `createHashEmbedder` baseline gives recall@5 **0.21 → 0.79** and recall@20 **0.63** at the package default 20-candidate pool, while the semantic `Xenova/all-MiniLM-L6-v2` q8/cpu (384 dims) gives **0.79 → 0.79** and recall@20 **1.00** — the semantic baseline starts at the lexical reranked number, the reranker's lift is 0.000 on this corpus, and the pool is not the binding constraint. Read the row for your own embedder: with a lexical/deterministic embedder the pool bound is what to raise first; with a semantic one the reranker is ordering quality only and the residual misses need better retrieval. Top-50 median was 95–289 ms across runs. Corpus, misses, pool-bound numbers, latency, cache state, and the semantic side-by-side live in [`docs/_evidence/phase111-reranker-semantic-recall.md`](_evidence/phase111-reranker-semantic-recall.md), with the lexical control in [`docs/_evidence/phase102-local-rerank-latency.md`](_evidence/phase102-local-rerank-latency.md); both are regenerated by `PRISM_TEST_LOCAL_RERANK=1 bun run test:live`. Treat the numbers as one data point on one machine, not a ceiling: dtype, device, and the embedder move them. The non-CPU (fp16/GPU) leg is host-provisioned — re-measure before expecting the CPU numbers to hold. The package guarantees the plumbing (one lazy load, one batched score call per rerank), not the model's speed. The hosted/TEI adapters stay for scale (higher throughput, no local RAM, no download).
|
|
264
276
|
- Host defaults: `dtype: "q8"` with `device: "cpu"` on x86 — fp32 weights are roughly 4× the download for no measurable ranking gain in this size class, and fp16/GPU is worth opting into only when the host already provisions it. Weights are cached per host: pass one `cacheDir` (e.g. `~/.cache/prism/models`) and the runtime lays out one subdirectory per model id, so a second model or a second process reuses the same files — point local embedders running through the same runtime at that directory too. With `cacheDir` omitted the runtime's own default cache applies (inside the installed package). On a cache miss the model is downloaded once into that directory and later runs stay on disk: add `allowRemoteModels: false` on an offline host to fail instead of reaching the model registry, which is exactly what the live leg's second pass proves.
|
|
265
277
|
- Cheap by construction: the model loads lazily once per reranker instance, `score` is called once per rerank with every candidate (never one call per document), and `onLoad({ model, loadMs })` is the only opt-in observability — no document text is ever logged. Zero network after load; the built-in loader only touches the model registry at load time, and `allowRemoteModels: false` pins it to local files.
|
|
266
|
-
- Failure is loud: a missing runtime, an unreachable model, or a runtime that returns no per-document scores throws a redacted `RagValidationError` naming the model and the install path (`
|
|
278
|
+
- Failure is loud: a missing runtime, an unreachable model, or a runtime that returns no per-document scores throws a redacted `RagValidationError` naming the model and the install path (`bun add @huggingface/transformers` or pass `{ runtime }`). There is deliberately **no** silent lexical fallback.
|
|
267
279
|
- `rerankHits` is unchanged and still owns the caps and the trust boundary: local scores reorder the same `RagHit` references (provenance/trust untouched), byte/ms/concurrency limits apply, and abort/timeout/malformed-score cases fail closed.
|
|
268
280
|
|
|
269
281
|
## Implementation example
|
|
@@ -385,7 +397,7 @@ Reranker adapters are probed against operator-deployed endpoints — each leg sk
|
|
|
385
397
|
|
|
386
398
|
```bash
|
|
387
399
|
PRISM_TEST_TEI_RERANKER_URL=http://tei.svc:8080 \
|
|
388
|
-
|
|
400
|
+
bun test packages/memory/dist/rag/__tests__/live.test.js
|
|
389
401
|
```
|
|
390
402
|
|
|
391
403
|
| Env var | Purpose |
|