@treeseed/dsh-chapters 0.1.23 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (104) hide show
  1. package/README.md +2 -1
  2. package/cordis.patch.yml +54 -1
  3. package/docs/architecture.md +1 -466
  4. package/docs/backlog.md +234 -4
  5. package/docs/contract.md +127 -1
  6. package/docs/development.md +248 -12
  7. package/docs/host-compaction-seam.md +108 -0
  8. package/docs/knowledge-repo.md +532 -16
  9. package/docs/project-state.md +323 -0
  10. package/docs/verify.md +217 -0
  11. package/lib/archive.d.ts +49 -1
  12. package/lib/archive.js +56 -2
  13. package/lib/archive.js.map +1 -1
  14. package/lib/arrival.js +24 -10
  15. package/lib/arrival.js.map +1 -1
  16. package/lib/attach.d.ts +46 -0
  17. package/lib/attach.js +120 -0
  18. package/lib/attach.js.map +1 -0
  19. package/lib/auto-save.d.ts +145 -0
  20. package/lib/auto-save.js +318 -0
  21. package/lib/auto-save.js.map +1 -0
  22. package/lib/client.js +413 -11
  23. package/lib/client.js.map +1 -1
  24. package/lib/commands.d.ts +11 -1
  25. package/lib/commands.js +138 -7
  26. package/lib/commands.js.map +1 -1
  27. package/lib/compose.js +25 -6
  28. package/lib/compose.js.map +1 -1
  29. package/lib/continue-core.d.ts +24 -1
  30. package/lib/continue-core.js +41 -9
  31. package/lib/continue-core.js.map +1 -1
  32. package/lib/diag.d.ts +47 -0
  33. package/lib/diag.js +141 -0
  34. package/lib/diag.js.map +1 -0
  35. package/lib/engine-core.d.ts +51 -1
  36. package/lib/engine-core.js +70 -7
  37. package/lib/engine-core.js.map +1 -1
  38. package/lib/engine.d.ts +93 -0
  39. package/lib/engine.js +477 -36
  40. package/lib/engine.js.map +1 -1
  41. package/lib/enrich-queue.js +28 -2
  42. package/lib/enrich-queue.js.map +1 -1
  43. package/lib/enrich-wire.d.ts +92 -2
  44. package/lib/enrich-wire.js +135 -17
  45. package/lib/enrich-wire.js.map +1 -1
  46. package/lib/enrich.d.ts +27 -2
  47. package/lib/enrich.js +114 -18
  48. package/lib/enrich.js.map +1 -1
  49. package/lib/index.d.ts +59 -0
  50. package/lib/index.js +215 -2
  51. package/lib/index.js.map +1 -1
  52. package/lib/indexing.d.ts +7 -0
  53. package/lib/indexing.js.map +1 -1
  54. package/lib/injections.d.ts +1 -0
  55. package/lib/injections.js +38 -0
  56. package/lib/injections.js.map +1 -1
  57. package/lib/last-resort.d.ts +24 -0
  58. package/lib/last-resort.js +27 -0
  59. package/lib/last-resort.js.map +1 -0
  60. package/lib/prompt.d.ts +130 -0
  61. package/lib/prompt.js +255 -0
  62. package/lib/prompt.js.map +1 -0
  63. package/lib/registry.d.ts +39 -1
  64. package/lib/registry.js +76 -9
  65. package/lib/registry.js.map +1 -1
  66. package/lib/render.d.ts +31 -3
  67. package/lib/render.js +82 -3
  68. package/lib/render.js.map +1 -1
  69. package/lib/repair-run.d.ts +49 -0
  70. package/lib/repair-run.js +423 -0
  71. package/lib/repair-run.js.map +1 -0
  72. package/lib/repair.d.ts +200 -0
  73. package/lib/repair.js +354 -0
  74. package/lib/repair.js.map +1 -0
  75. package/lib/rules-commands.d.ts +22 -0
  76. package/lib/rules-commands.js +177 -20
  77. package/lib/rules-commands.js.map +1 -1
  78. package/lib/rules.d.ts +184 -7
  79. package/lib/rules.js +312 -27
  80. package/lib/rules.js.map +1 -1
  81. package/lib/save-core.d.ts +72 -0
  82. package/lib/save-core.js +220 -0
  83. package/lib/save-core.js.map +1 -0
  84. package/lib/session-rules.d.ts +222 -0
  85. package/lib/session-rules.js +454 -0
  86. package/lib/session-rules.js.map +1 -0
  87. package/lib/store.d.ts +22 -1
  88. package/lib/store.js +94 -17
  89. package/lib/store.js.map +1 -1
  90. package/lib/sync.js +13 -1
  91. package/lib/sync.js.map +1 -1
  92. package/lib/tools.d.ts +14 -0
  93. package/lib/tools.js +341 -91
  94. package/lib/tools.js.map +1 -1
  95. package/lib/types.d.ts +31 -1
  96. package/lib/types.js +12 -0
  97. package/lib/types.js.map +1 -1
  98. package/lib/version.d.ts +9 -0
  99. package/lib/version.js +10 -0
  100. package/lib/version.js.map +1 -0
  101. package/lib/warnings.d.ts +10 -0
  102. package/lib/warnings.js +18 -0
  103. package/lib/warnings.js.map +1 -0
  104. package/package.json +9 -3
@@ -9,7 +9,7 @@ this machine, not assumed.
9
9
 
10
10
  | Thing | Value |
11
11
  |---|---|
12
- | Host serving this session | global npm install, `/usr/lib/node_modules/@deepseek-ai/dsh/lib/bin.js`, **0.1.5-rc.1**, listening on `127.0.0.1:3080` |
12
+ | Host serving this session | the GLOBAL npm install — but the PREFIX IS NOT PART OF THE FACT. Resolve it: `readlink -f $(command -v dsh)`, then read that package's `version`. On this box today: `/home/adrian/node_modules/@deepseek-ai/dsh`, **0.2.0-rc.2**, listening on `127.0.0.1:3080`. This row used to assert `/usr/lib/...` and `0.1.5-rc.1` as if they were the machine; a drift gate that hardcoded that path then skipped forever while the package sat in the repo's own `node_modules` (#54). |
13
13
  | Profile root | `~/.dsh` (`DSH_HOME` default) — `profiles/`, `sessions/`, `storages/`, `.credentials.yaml` |
14
14
  | Sandbox effect | `~/.dsh` is **read-only** from the agent sandbox: `dsh web --help` died with `EROFS: read-only file system, open '/home/adrian/.dsh/profiles/web/cordis.yml'` |
15
15
  | Dev fork | `~/Projects/deepseek-harness` — git `fb2c4b9e69` (2026-09-10), **0.1.5-rc.2**, pnpm workspace, `node_modules` installed |
@@ -24,6 +24,14 @@ Two consequences worth stating plainly:
24
24
  snapshot. Line numbers in `docs/contract.md` come from the `examples/` copy. Treat any of the three as
25
25
  truth only for the question you asked it; resolve real behaviour against the version you are actually
26
26
  booting.
27
+ 3. **Check the binary you are measuring, not just the code you changed.** #54 lost hours to
28
+ `.dshdev-local/profiles/web/node_modules/@treeseed/dsh-chapters` having become a COPIED tree
29
+ dated six days earlier: every browser journey, probe and re-distill attempt was running a build
30
+ predating the feature under test, and drew confident conclusions about the HOST — including a
31
+ seed-admission refusal the current code does not produce (`tests/e2e/boot.ts` refuses to spawn
32
+ on a copied tree now, and names the re-link command). The general habit: where a built artifact
33
+ is loaded by path, link it or assert its version — an old binary passes every test you write
34
+ against it and reports them as facts.
27
35
 
28
36
  ## The Isolation Mechanism: `DSH_HOME`
29
37
 
@@ -108,6 +116,49 @@ later session may re-reference them for the *why* behind an implementation. Two
108
116
  contradiction with the shipped code, update or supersede it in place via git history — never rewrite
109
117
  silently.
110
118
 
119
+ ### Is the corpus anchored to the host we pin? (`tests/fixtures/model-tape/CORPUS_HOST`)
120
+
121
+ Tape matching is normalized request-PREFIX matching, and the prefixes carry the host's
122
+ system-prompt composition. So moving `dev/dsh-host-lock`, editing the preset persona, or changing a
123
+ tool description invalidates the whole corpus — and a replay failure then describes an *earlier*
124
+ change, not the one under review. `scripts/corpus-anchor.mjs` states the provenance in one file and
125
+ answers the question in exactly one place; two consumers use it:
126
+
127
+ - `tests/e2e/boot.ts` **refuses replay by name** when `host` ≠ the pinned version, quoting the
128
+ recorded `reason` and the tracking issue. `E2E_MODEL=record` is deliberately exempt — re-distilling
129
+ is the way out, and a gate that blocked it would strand itself.
130
+ - `.github/workflows/ci.yml`'s `host-capability` job sets `corpus-anchored`, the `e2e-replay` job
131
+ requires it, and the run log gets a step summary plus a `::warning::` naming the issue. Skipped,
132
+ visibly — the same discipline as the preset-row refusal it sits beside: never silently green,
133
+ never permanently red.
134
+
135
+ The gate lifts with no code change: re-distill the eight projects
136
+ (`scripts/distill-tapes.sh`), write the new `host =` line, done. Current state: pinned
137
+ `0.2.0-rc.2`, corpus distilled `0.1.5-rc.1`, tracked by
138
+ [issue #56](https://github.com/treeseed-ai/dsh-chapters/issues/56) (with the door-selector blocker
139
+ [#57](https://github.com/treeseed-ai/dsh-chapters/issues/57) inside it). Rows in
140
+ `tests/unit/corpus-anchor.test.ts` assert both directions — matching lifts, missing/garbled
141
+ provenance refuses, and the committed repo state is reported honestly as NOT anchored.
142
+
143
+ ### Reading a distill while it runs (#54)
144
+
145
+ `scripts/distill-tapes.sh` writes each leg's FULL playwright output to `var/distill/<proj>-<hhmmss>.log`
146
+ (never `tail`, which buffers until the leg ends) and prints a 60-second heartbeat: tape-entry count
147
+ plus the last line of `var/e2e-upstream.log`. That journal is the proxy's own record of every upstream
148
+ attempt — `START / FIRSTBYTE / STALLED / DONE / DEAD`, with elapsed ms and byte counts — and the proxy
149
+ now bounds the wait at `E2E_UPSTREAM_TIMEOUT_MS` (default 600000, 10 minutes: generous for a cold
150
+ prefill at the stress regime, short enough that a hung connection ends the run with an answer instead
151
+ of silently consuming the harness's own 3600000 ms profile timeout).
152
+
153
+ So the question "is this turn slow, or is the connection dead?" is a lookup:
154
+
155
+ | journal shape | meaning |
156
+ |---|---|
157
+ | `START` then `FIRSTBYTE` | answered; `DONE elapsed=… bytes=…` says how long and how much |
158
+ | `START`, `STALLED elapsed=…ms bytes=0`, then `DEAD` | upstream never responded — a hang, not a slow model |
159
+ | `START`, `FIRSTBYTE`, then `STALLED … bytes=N` growing | real streaming progress; leave it alone |
160
+ | no journal line at all | the request never reached the proxy (boot/port/profile problem) |
161
+
111
162
  ### Three test layers (and why the third exists)
112
163
 
113
164
  | Layer | Command | Runs | Covers |
@@ -180,7 +231,7 @@ Re-record discipline (learned the hard way 2026-09-22/23):
180
231
  sweep files rather than by lowering the bar.
181
232
  - `npm run coverage` — the same deterministic suite under node's native
182
233
  `--experimental-test-coverage` (a quick second opinion; no gate).
183
- - `npm run coverage:e2e` — replays the eight browser segments with
234
+ - `npm run coverage:e2e` — replays the **nine** browser segments with
184
235
  `E2E_COVERAGE=1`, which makes every boot write NODE_V8_COVERAGE profiles
185
236
  under `var/e2e-cov/<port>` (flushed on the boot's clean exit), then runs
186
237
  `scripts/e2e-coverage.mjs` → **c8 with real source maps** (tsconfig
@@ -188,10 +239,24 @@ Re-record discipline (learned the hard way 2026-09-22/23):
188
239
  per-module Stmts/Branch/Funcs. Read it as "covered by SOME boot of the
189
240
  suite" (a union), and mind the script header's c8 gotcha: include/exclude
190
241
  filters apply BEFORE remapping, so `--include=src/**` would silently
191
- zero every row. First full ledger: 80% stmts / 62% branches overall;
192
- tools.ts 72/49, engine.ts 81/59, index.ts 86/55 — plus the honest plane
193
- split (src/treedx reads low HERE because the browser suite rides git;
194
- the deterministic + live ledgers own that transport).
242
+ zero every row.
243
+ **Ledger, re-measured 2026-10-23 on the PINNED hermetic host (`97c7c79`, strict replay
244
+ rc 0 / zero misses, all nine segments): 78.26% stmts / 59.01% branch / 71.38% funcs.** Per module:
245
+ engine.ts 87.93/55.06, index.ts 85.78/52.43, rules.ts 88.83/58.87,
246
+ store.ts 91.64/79.59, continue-core.ts 94.05/66.66, arrival.ts 94.87/60.60,
247
+ sync.ts 83.95/59.89, tools.ts 64.06/49.66.
248
+ **This is a slip, stated as one:** the 2026-09-25 ledger read 80/62 overall
249
+ (tools 72/49, engine 81/59, index 86/55). Statements and branches both went
250
+ DOWN while engine's statement coverage rose — because the re-anchored legs
251
+ are shorter, not weaker: `heavy` replays in 4 tape entries (2 compaction
252
+ triples, and it now answers) where the old corpus carried 16–22, and a
253
+ shorter conversation walks fewer engine arms. Nothing here is gated — the
254
+ gated ledger is `coverage:ci` at 85/85/85/85 on the deterministic plane —
255
+ so the honest reading is: *e2e coverage tracks how much conversation the
256
+ corpus replays, and a faster-converging corpus covers less of the engine by
257
+ accident.* Recovering the number means adding breadth deliberately (more
258
+ distinct journeys, or longer ones with their own claim), never padding a
259
+ leg to fatten a percentage.
195
260
  Both ledgers together are the coverage claim; neither alone is honest.
196
261
 
197
262
  ### Release status (measured 2026-09-25)
@@ -211,7 +276,17 @@ environment `Production`. Prerelease versions publish under a matching
211
276
  dist-tag so 'latest' stays stable. `0.1.1` is the first OIDC + provenance
212
277
  ship; `0.1.0` itself carries no attestation.
213
278
 
214
- ### Hosted CI (`.github/workflows/`, wired + dress-rehearsed 2026-09-23)
279
+ ### Hosted CI
280
+ **The rig writes into its own port's store, never the project's (#61 item 8).** Each boot pins
281
+ `artifactStoreRoot` to `var/e2e-home-<port>/chapters` through the profile patch
282
+ (`tests/e2e/boot.ts::applyRigPluginPatch`), and CI brackets the replay job with
283
+ `scripts/assert-store-untouched.mjs snapshot` / `verify` — a listing comparison (names, sizes, mtimes)
284
+ that fails the job if the production `.dsh-chapters/` changed, including the pristine-runner case where
285
+ the store did not exist and a run CREATED one. Before this, every acceptance run left its chapters in the
286
+ repo's real store: 42 test trees and 29 of 47 live registry rows with e2e-shaped ids, which #60's auditor
287
+ then reported as 441 `test-litter` findings. Contamination like that does not just waste disk; it teaches
288
+ people to ignore a report that is telling them the truth about something else.
289
+ (`.github/workflows/`, wired + dress-rehearsed 2026-09-23)
215
290
 
216
291
  - **`ci.yml`** — two jobs. `test`: `npm ci` (all deps public on
217
292
  registry.npmjs.org — **no install secret**), typecheck, build,
@@ -220,14 +295,63 @@ ship; `0.1.0` itself carries no attestation.
220
295
  failing-test lines and c8's threshold verdicts as `::error::`, because on the
221
296
  0.1.6 tag a red gate left nothing behind but "exit code 1" while every local
222
297
  reproduction of the same SHA was green; the summary step now runs
223
- `if: always()`). `e2e-replay`: installs the pinned
224
- host (`npm i -g @deepseek-ai/dsh@0.1.5-rc.1` — the SAME version as the
225
- gate line in `scripts/ci-replay.sh`, which fails loudly on mismatch) plus
226
- chromium's apt libs, then `scripts/ci-replay.sh` — eight browser segments
227
- served entirely from the committed tapes. **No LLM, no key, no GPU:** in
298
+ `if: always()`). `e2e-replay`: materializes the pinned host from
299
+ `dev/dsh-host-lock` (`npm ci`, hermetic — `npm i -g` rewrites the plugin tree and the
300
+ host's caret ranges float), plus chromium's apt libs, then `scripts/ci-replay.sh`, which
301
+ READS the version out of that lock rather than repeating it (#54 removed the second
302
+ literal; three copies of a pin is how it goes stale in three places) — eight browser
303
+ segments served entirely from the committed tapes. **No LLM, no key, no GPU:** in
228
304
  replay mode `localModelUp()` short-circuits to true (specs RUN, they do not
229
305
  skip) and a tape miss is a loud proxy 503, so a scenario that legitimately
230
306
  changed fails the job instead of silently passing.
307
+ ### Distilling tapes WITHOUT the GPU box (#54)
308
+
309
+ `E2E_MODEL=record` drives the real model once per scenario to make a tape. Until #54 that
310
+ meant the local vLLM server had to be up. It no longer does, because the tape proxy
311
+ translates on the way out instead of asking the fixture to change:
312
+
313
+ ```bash
314
+ E2E_MODEL=record E2E_TAPE=rules E2E_PORT=41735 E2E_THRESHOLD=0.9 \
315
+ E2E_UPSTREAM=https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode \
316
+ E2E_UPSTREAM_MODEL=qwen3.8-flash PLAYWRIGHT_BROWSERS_PATH=$PWD/var/ms-playwright \
317
+ E2E_UPSTREAM_KEY="$(node -e "import('./scripts/llm.mjs').then(m=>process.stdout.write(m.apiKey()||''))")" \
318
+ npx playwright test -c tests/e2e/playwright.config.ts --project=rules
319
+ ```
320
+
321
+ - `E2E_UPSTREAM` is the endpoint WITHOUT the `/v1` suffix (the proxy appends the request
322
+ path, which already carries it).
323
+ - `E2E_UPSTREAM_KEY` is the credential the PROXY presents. The profile keeps `apiKeyEnv:
324
+ LOCAL_API_KEY` and the boot still injects its `e2e-tape-proxy-dummy` when none is
325
+ configured — a real secret never enters a fixture profile.
326
+ - `E2E_UPSTREAM_MODEL` rewrites the requested id, so the profile can keep pinning
327
+ `qwen3.8-flash-next` (the 32K stress-regime name) while a hosted endpoint is addressed by
328
+ the id it serves. **The tape records the profile's name**, because a tape is a fixture of
329
+ this rig, not of whatever service distilled it.
330
+ - A remote `E2E_UPSTREAM` in record mode with no key refuses at boot in one line, instead of
331
+ discovering it 401 times across eight projects.
332
+ - `scripts/llm.mjs` is the same resolution the dev scripts use (local server preferred, hosted
333
+ fallback, credential resolved from env → key file → profile export → harness store, value
334
+ never printed).
335
+
336
+ Three 0.2.x host drifts the first boots found, all rig-side, all now handled — each is the
337
+ kind that produces a wrong conclusion instead of an error:
338
+
339
+ 1. **the durable log is `session.v4.jsonl.zstd`.** Five places hardcoded `session.v3…`, so
340
+ every log-backed assertion decoded nothing and reported "the archive never landed".
341
+ `sessionLogFileFor()` resolves the highest `session.v<N>.jsonl.zstd`; a row scans
342
+ `tests/e2e/` for a hand-built versioned path (comment lines excluded — prose may name the
343
+ formats it explains).
344
+ 2. **the host opens a Preview Notice, and it is a focus trap** — the 2026-09-18 saga with a
345
+ new member. Dismissal now targets `[role=dialog]`-shaped elements and presses the control
346
+ that acknowledges *that* dialog; matching "any div mentioning a notice" matched the app
347
+ wrapper and pressed an unrelated 'Close'.
348
+ 3. **provider config must ride `cordis.patch.yml`.** A profile-dir `settings.yaml` is read by
349
+ nothing on 0.2.x, so `dev/settings.yaml`'s `llm-pi-ai` block was invisible: the host had no
350
+ `local` provider and the browser opened on "Add an API key to get started" with the
351
+ composer never reachable. `dev/profile-cordis.patch.yml` now defines the provider, and
352
+ `tests/e2e/boot.ts` guarantees the block in the copied home before rewriting the window
353
+ pins and the proxy `baseURL`.
354
+
231
355
  - **`release.yml`** — bare-semver tag (`0.1.0`, matching the Production
232
356
  environment's `*.*.*` tag protection) or manual dispatch → the full
233
357
  gated deterministic suite as publish blocker → `npm publish
@@ -574,3 +698,115 @@ Three traps, all paid for:
574
698
 
575
699
  Prune after: `var/host-versions/`, the throwaway homes and `var/coverage` are all regenerable in minutes and
576
700
  together cost ~2 GB — this repository's contract is the *measurements written down*, not their scratch.
701
+
702
+ ### The rig's regime has ONE table (`tests/e2e/rig-pins.ts`) — added 2026-10-23 (#56)
703
+
704
+ Three copies had silently diverged: `scripts/distill-tapes.sh`'s `THRESH`/`EXTRA` tables, the inline
705
+ env in `npm run test:e2e`, and `boot.ts`'s defaults — under a comment claiming distill "mirrors
706
+ `npm run test:e2e`'s pins exactly". Measured result: replay booted `suite`/`rules`/`fork` with **no
707
+ threshold** while their tapes had been recorded at 0.75/0.9/0.75. The realm differed, replay's requests
708
+ carried `[dsh:artifact …]` stubs the tapes contain none of, and the chain failed as tape MISSes that
709
+ look exactly like prompt drift.
710
+
711
+ `RIG_PINS` is now the single source, with two consumers: `playwright.config.ts` derives each project's
712
+ `env` through `rigEnv(proj)` (so workers see it), and `globalSetup` resolves the selected project from
713
+ `fullConfig.argv` and calls `applyRigEnv()` **before booting**. The second half is not optional —
714
+ Playwright applies project `env` to workers, and the object handed to `globalSetup` exposes an
715
+ UNFILTERED `projects` list whose entries carry no `env` (measured: 11 projects for a
716
+ `--project=heavy` run). "Just put it in the config" looked fixed and was not: the boot kept using
717
+ defaults. An explicit command-line value still wins, so re-tuning one leg never needs an edit; an
718
+ unknown project derives nothing instead of inventing defaults.
719
+
720
+ Rows: `tests/unit/e2e-pins-single-authority.test.ts` (table contents, `rigEnv` derivation, argv parsing
721
+ in both `--project=x` and `--project x` forms, apply-not-clobber, unknown-project, both consumers
722
+ reading the table, a literal `env: { 'E2E_TAPE' … }` block failing, and the runners carrying no copy).
723
+ `npm run test:e2e`'s per-leg header now prints the EFFECTIVE regime read from the table, so a tape's
724
+ provenance is auditable from the runner log alone.
725
+
726
+ ### Provenance is generated (`scripts/write-provenance.mjs`), never hand-authored (#56)
727
+
728
+ `PROVENANCE.md` lives inside each project directory — and the runner `rm -rf`s that directory before
729
+ recording. The re-anchor commit therefore deleted `plans/PROVENANCE.md` and `prompt/PROVENANCE.md`,
730
+ which are precisely the records stating those legs were distilled from the SCRIPTED stub rather than
731
+ a live model. A claim about what a tape means must not be destroyed by the act of re-making the tape:
732
+ the file is generated from the pin table plus the recorded verdict, and
733
+ `tests/unit/tape-provenance.test.ts` fails if any project lacks it or if the runner stops
734
+ regenerating it.
735
+
736
+ ### The pool mirror must be per-boot too, or notices are not reproducible (#85)
737
+
738
+ `artifactStoreRoot` has been pinned per port since #61, and I believed that made a boot's state
739
+ test-local. It did not: the compaction/continuation notice composes its `## CORE RULES` block from
740
+ `path.join(cwd, DEFAULT_CLONE_DIR)` (`src/engine.ts`) — a **fixed path in whatever project the rig is
741
+ running in**. In this repo that directory holds real development state, so a leg's notice content
742
+ depended on rules that had nothing to do with the leg: measured 2026-10-23, `heavy`'s recorded notice
743
+ carried `[dev2/001] (security) Never log raw tokens…` while the replayed notice carried none, the two
744
+ texts agreeing to character 346 and diverging there. Same build, same pins, same tapes — a different
745
+ repo.
746
+
747
+ The realm config key `knowledgeMirrorDir` (default `.dsh-knowledge`, unchanged for real installs) now
748
+ feeds that read, and `boot.ts` pins it to `var/e2e-home-<port>/knowledge`. The rule generalizes: any
749
+ path a boot reads to build model-visible text must be per-port or per-session, because a tape is a
750
+ claim about ONE world. `tests/unit/e2e-store-root.test.ts` keeps the pins honest, and
751
+ `tests/unit/preset-declaration.test.ts` refuses a config key that drifts out of the preset row.
752
+
753
+ ### Don't push while an acceptance run is the thing you're waiting on (#89, measured against myself)
754
+
755
+ `.github/workflows/ci.yml` sets `concurrency: ci-<ref>` with `cancel-in-progress: true`. Twice on
756
+ 2026-10-23 I pushed a small commit (a docs line, a corrected message) while the ~25-minute acceptance
757
+ chain was mid-flight, and GitHub cancelled the run that held the answer — then the new run restarted
758
+ from zero. The commit was never the problem; the wait was. Batch the push: while a long gate is
759
+ running, either finish what you're doing locally first or park the commit, and push once. Same rule in
760
+ the other direction: don't judge a fix by the run it cancelled — re-read which head each verdict belongs
761
+ to before believing a green.
762
+
763
+ ### Throwaway homes live outside the repository — `E2E_HOME_ROOT` (#89)
764
+
765
+ `e2eHomeRoot()` defaults to `os.tmpdir()/dsh-e2e`, and CI sets `E2E_HOME_ROOT=/tmp/dsh-e2e` explicitly
766
+ so the diagnostics it uploads are the ones it booted. Not tidiness: Node resolves `@deepseek-ai/*`
767
+ bundles by walking UP from the profile directory, so a home under `var/` reaches this repo's
768
+ `node_modules` (0.1.5-rc.2 by lockfile) and then the developer's `~/node_modules`. On the machine that
769
+ recorded the first re-anchored corpus, that second hop is a **linked dev build of the harness whose
770
+ version string equals the pin** — so the boot mounted tool guidance (goals, ralph, subagents, web
771
+ search, glob/grep, job output) the published tarball does not have.
772
+
773
+ Measured by moving nothing but the home's location:
774
+
775
+ | home root | system header composed by the SAME pinned binary | `suite` strict replay |
776
+ |---|---|---|
777
+ | `var/e2e-home-41731` | 7,276 chars | 4 passed, zero misses |
778
+ | `/tmp/dsh-e2e/e2e-home-41731` | 2,783 chars (CI's shape) | `commonPrefix=1`, `divergeAt=msg1` |
779
+
780
+ That is the whole "green on my laptop, impossible on CI" class in two rows of one table, and why
781
+ `tests/unit/e2e-host-binary.test.ts` now asserts hermeticity structurally: the home root must be
782
+ outside `ROOT`, and no ancestor of the home may contain `@deepseek-ai` packages — if one ever does, the
783
+ row names the path instead of agreeing silently. Pinning the *binary* (`resolveHostBinary`) was not
784
+ enough; the ancestor chain is part of the identity, and AGENTS' warning that these bundles' version
785
+ strings are not identity turns out to apply one directory level up from any lockfile.
786
+
787
+ Consequence, stated plainly: changing this changes what every boot composes, so the entire corpus was
788
+ re-distilled afterward. A rule that doesn't cost a re-record is probably not a reproducibility rule.
789
+
790
+ ## One build per corpus (#90, measured 2026-10-24)
791
+
792
+ A tape records what the **plugin** asked, not just what the model answered. So a corpus is only coherent
793
+ if every leg in it was distilled against one `lib/`. This was violated all day by well-meaning
794
+ incrementalism — re-recording whichever leg a fix touched — and the cost was invisible until a replay
795
+ missed on a *post-compaction* surface: a 4-message request with `commonPrefix=0` against a tape that held
796
+ the 6-message pre-compaction form. Record ran on a build whose last-resort deferral always abstained
797
+ (`resolveConfig` defaults `maxTokens` to `headroomTokens` = 65,536, so the coexistence ceiling was
798
+ underivable); replay ran on a build where it is derivable (11,880) and can fire. Both behaviors are
799
+ correct. The corpus spanning them is not.
800
+
801
+ Rules that follow, in the order they bite:
802
+
803
+ 1. **Change `src/` → rebuild → re-distill ALL nine legs.** A partial re-record is a mixed corpus, and a
804
+ mixed corpus fails at whichever step the builds disagreed — usually inside compaction, where message
805
+ counts are the thing being matched.
806
+ 2. **`PROVENANCE.md` per leg is necessary, not sufficient.** It records the host and regime; it does not
807
+ say "this leg predates three plugin commits". When in doubt about build drift, re-record everything.
808
+ 3. **Test-only changes (`tests/e2e/**`) need no re-record** — normalization folds apply at load to both
809
+ sides. Product-behavior changes do. The distinction is exactly AGENTS' "re-record when PRODUCT prompt
810
+ bytes change", generalized: anything that changes what the plugin *sends*.
811
+ 4. If a green replay is ever obtained on a corpus whose legs have different build stamps, treat it as
812
+ unexplained rather than as evidence.
@@ -115,3 +115,111 @@ durable log length **never decreased** (shadowing, not deletion — r12 proved t
115
115
  (c) a compacted session can still answer a question whose answer lives only in a chapter it chooses to
116
116
  `read`; (d) the summary call cost is **zero** tokens, since our `summarize()` is deterministic —
117
117
  proved with the r12 stub (no `usage` field), must re-prove with the real TOC.
118
+
119
+ ## Measured on 0.2.0-rc.2 (2026-10-23, #78) — what the seam actually does now
120
+
121
+ Every line below came from an artifact in this repo's own run: the diagnostics sink
122
+ (`realm:engine:compact`), the tape proxy journal (`reqBytes=`, `OVERFLOW refused`), or a decoded
123
+ `session.v4.jsonl.zstd`. None of it is inferred from the 0.1.x notes above, which is the point —
124
+ those notes are what made the failure legible as "the hook is never called", and that was wrong.
125
+
126
+ 1. **The host does call our override.** Pre-step, with `trigger="pressure"`:
127
+ `compactIfNeeded entered (trigger="pressure", session=session-…)` — three entries in one probe
128
+ session, seven compactions in one `heavy` session. The claim "never invoked" was an inference
129
+ from absent `compaction/*` events, and the instrument replaced it with a fact in one run.
130
+ 2. **The base decision path** (`dsh-compaction-basic/lib/index.js`) is: `routedTarget(session)` →
131
+ `undefined` ⇒ return `null` immediately; else `tokenMeter.measure(session)`, then
132
+ `llm.resolveModelInfo(provider, model)` for `contextWindow`, then
133
+ `resolveCompactSpec(policy, window, reservedCompletionTokens(agent, defaultMaxTokens))`, then
134
+ `measurement.totalTokens < spec.thresholdTokens` ⇒ `null`; else `selectCompactableRange(...)`,
135
+ which can itself return `null` — that is the documented "no safe range" case, and it is the case
136
+ `heavy` reaches.
137
+ 3. **Two failure shapes for the same field symptom.** A decline can be a `null` return or a THROW
138
+ (`selectCompactableRange` threw on a harness session whose surface it could not read). The
139
+ engine now reports on both paths, because "the endpoint said 400" was the only user-visible
140
+ trace of a state the product knew about.
141
+ 4. **A single oversized `tool/result` is not relievable by pressure.** Measured: one 120,650-byte
142
+ result (~30K tokens) in a 32,000 window at ratio 0.5 — six compactions of *older* content, then
143
+ the hard `400 … exceeds the context window`. The knob that prevents it is the **arrival floor**
144
+ (`toolResultArtifactTokens`), and `heavy` deliberately sets it to 1,000,000 so pressure — not
145
+ arrival — is what gets exercised. The engine does NOT override that floor: AGENTS calls it "a
146
+ behavior switch, not a knob", and an engine quietly reinterpreting an operator's switch is the
147
+ same species of dishonesty as a pin that is collected and never applied. What it does now is name
148
+ the switch, the offending node size, and the count of nodes above the trigger, in the sink.
149
+ 5. **Bytes ↔ tokens, calibrated.** 78,140 serialized request bytes measured 27,306 / 27,952 /
150
+ 28,928 tokens by the host meter — **≈2.8 B/token**, not the conventional 4. The rig's window
151
+ enforcement uses whole-body `/3` (deliberately conservative: early refusal still exercises the
152
+ regime; late refusal exercises nothing) and the rows assert the estimator against the two field
153
+ values that matter, including the 120,129-byte / 35,428-token request that `4/` let through.
154
+ 6. **The regime must be identical in record and replay.** Enforcement applies in both modes now:
155
+ a distill whose request was refused, relieved, and re-asked recorded a sequence *shaped by* that
156
+ refusal, so a replay without it diverges and looks like prompt drift.
157
+ 7. **Realm config lives on the preset row, not the host row.** `thresholdRatio`,
158
+ `toolResultArtifactTokens`, `artifactStoreRoot`, `auto`, `elicitedPlot` are read from the
159
+ `chapters-engine` row inside `preset-chapters` (`cordis.patch.yml`); the host plugin row's
160
+ `compactionThresholdRatio` / `installChaptersPreset` are inert since 0.1.18. This is why an
161
+ apparently-configured rig ran every leg on realm defaults while its own header claimed 0.5.
162
+
163
+ ## Relief must not shadow the question (#87, measured 2026-10-23)
164
+
165
+ The seam's real failure mode is not a crash — it is a **surface with no request in it**. Field
166
+ evidence from one `heavy` session (299 events, decoded `session.v4.jsonl.zstd`):
167
+
168
+ ```
169
+ 13 compaction summaries, 108 distinct shadowed seqs
170
+ STILL-HERE instruction seqs: [109,116,138,154,172,188,216,236,254,268,291]
171
+ shadowed away by relief: [109,116,138,154,172,188,216,236,254,268] ← 10 of 11
172
+ final assistant text: ""
173
+ ```
174
+
175
+ Why: `retainRatio` reserves a *token fraction* as a verbatim tail. When an oversized `tool/result`
176
+ sits in that tail it eats the whole budget, and range selection shadows the newest `user/message`
177
+ anyway. Our summary is deterministic and plot-shaped, so it carries no request. The user sees an agent
178
+ that stopped responding, with nothing in the sink — a `#61a` violation twice over (undetected, and
179
+ therefore undelivered).
180
+
181
+ **The rule we ship.** We author the summary text, so the newest human turn travels *into* it — verbatim
182
+ (`carryOpenRequest`, `src/engine.ts`). Nothing recorded is rewritten: this honors invariant 1, appends
183
+ to our own output, and the input array is never mutated. Details that are the point:
184
+
185
+ - **Injected scaffolding is skipped.** `<system-reminder>`, the host's checkpoint preamble,
186
+ `Time sampled …`, artifact stubs: re-injecting machine state as if it were a request would be a
187
+ different lie, so the predicate that decides "is this a human turn" lives in ONE place — the same
188
+ one-predicate rule that bit us with #59's title chooser.
189
+ - **A cap is stated, never silent.** `openRequestCarryMaxChars` (realm config, default 4000) bounds the
190
+ block, and when it bites the text says `carried in part: N characters total, M included — the whole
191
+ turn stays in the durable log`. Never truncate to fit is a plugin rule, and it applies to our own
192
+ summaries.
193
+ - **Nothing is claimed when there is nothing to carry.** No human turn in the region ⇒ no OPEN REQUEST
194
+ block, no empty promise of one.
195
+
196
+ Rows: `tests/unit/engine-open-request.test.ts` (verbatim carry; injections excluded; stated truncation;
197
+ empty region), written before the wiring was trusted. **Process lapse, recorded honestly:** the mandated
198
+ cycle is Issue → Docs → Tests → Code, and here I wrote the implementation before this section. The
199
+ issue (#87) and the tests did come first, and the docs landed before the commit — but the order slipped,
200
+ and a rule that is only followed when convenient is the kind of drift this file exists to catch.
201
+
202
+ ## The last-resort deferral (#78 option A, decided 2026-10-23)
203
+
204
+ **The rule.** A single `tool/result` that cannot coexist with the mandatory verbatim tail makes pressure
205
+ relief *impossible*, not merely hard: `selectCompactableRange` finds no safe range, the engine declines,
206
+ the endpoint refuses the request, and the session is left spinning (#86) or answering something other
207
+ than what was asked (#78's original symptom). Survivability is therefore an invariant, not a tuning
208
+ outcome: such a node is artifact-deferred **as a last resort**, with a delivered warning naming the seq,
209
+ the node's tokens, and the ceiling it failed.
210
+
211
+ **Three bounds, because the obvious implementation is the wrong one.**
212
+ 1. **After a decline, never before.** The trigger is the measured fact that relief ran, found no safe
213
+ range, and the surface is over the trigger. Deferring big nodes eagerly makes compaction unnecessary
214
+ and turns `oversized-turn.spec.ts` into a tautology — the trap AGENTS names in the rig rules — so an
215
+ anti-vacuity row asserts the ceiling does NOT fire when relief is possible.
216
+ 2. **Derived, not invented.** The ceiling is `windowTokens − retainBudget`, both from the base's own
217
+ config (`resolveCompactSpec` inputs), so no new magic constant appears. The one new key is the
218
+ operator-visible boolean switch `lastResortArtifact` (default on), which `preset-declaration.test.ts`
219
+ must see declared or the loader strips it.
220
+ 3. **The archive keeps every byte.** The deferral writes the artifact and leaves a citation, so the worst
221
+ case is a context-economy miss and `chapters_artifact` still answers for it — which is the only thing
222
+ that makes overriding a configured economy knob defensible at all.
223
+
224
+ Rows: `tests/unit/last-resort.test.ts` (the ceiling math, `null` on incomplete inputs, and the anti-vacuity ordering row) plus the engine lifecycle suite; below the ceiling, untouched;
225
+ switch off, untouched even after a decline; and the warning carries the numbers.