@fractalsynapse/dsh-chapters 0.0.0-stage → 0.2.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (195) hide show
  1. package/LICENSE +14 -0
  2. package/README.md +76 -2
  3. package/cordis.patch.yml +363 -0
  4. package/docs/architecture.md +692 -0
  5. package/docs/autorun-guide.md +248 -0
  6. package/docs/backlog.md +2022 -0
  7. package/docs/contract.md +938 -0
  8. package/docs/development.md +990 -0
  9. package/docs/host-compaction-seam.md +225 -0
  10. package/docs/host-protected-head-ask.md +80 -0
  11. package/docs/knowledge-repo.md +1332 -0
  12. package/docs/plans-carriage.md +571 -0
  13. package/docs/project-state.md +642 -0
  14. package/docs/provider.md +98 -0
  15. package/docs/superpowers/plans/2026-10-17-chapters-save.md +198 -0
  16. package/docs/superpowers/plans/2026-11-02-chapter-removal.md +151 -0
  17. package/docs/superpowers/specs/2026-11-02-chapter-removal-design.md +115 -0
  18. package/docs/verify.md +561 -0
  19. package/lib/archive.d.ts +161 -0
  20. package/lib/archive.js +229 -0
  21. package/lib/archive.js.map +1 -0
  22. package/lib/arrival.d.ts +23 -0
  23. package/lib/arrival.js +125 -0
  24. package/lib/arrival.js.map +1 -0
  25. package/lib/artifact-query.d.ts +62 -0
  26. package/lib/artifact-query.js +187 -0
  27. package/lib/artifact-query.js.map +1 -0
  28. package/lib/attach.d.ts +46 -0
  29. package/lib/attach.js +120 -0
  30. package/lib/attach.js.map +1 -0
  31. package/lib/auto-save.d.ts +145 -0
  32. package/lib/auto-save.js +334 -0
  33. package/lib/auto-save.js.map +1 -0
  34. package/lib/aux-message.d.ts +17 -0
  35. package/lib/aux-message.js +20 -0
  36. package/lib/aux-message.js.map +1 -0
  37. package/lib/client.js +1419 -0
  38. package/lib/client.js.map +1 -0
  39. package/lib/codegraph.d.ts +76 -0
  40. package/lib/codegraph.js +190 -0
  41. package/lib/codegraph.js.map +1 -0
  42. package/lib/commands.d.ts +52 -0
  43. package/lib/commands.js +946 -0
  44. package/lib/commands.js.map +1 -0
  45. package/lib/compose.d.ts +30 -0
  46. package/lib/compose.js +185 -0
  47. package/lib/compose.js.map +1 -0
  48. package/lib/continue-core.d.ts +245 -0
  49. package/lib/continue-core.js +405 -0
  50. package/lib/continue-core.js.map +1 -0
  51. package/lib/diag.d.ts +55 -0
  52. package/lib/diag.js +186 -0
  53. package/lib/diag.js.map +1 -0
  54. package/lib/engine-core.d.ts +301 -0
  55. package/lib/engine-core.js +531 -0
  56. package/lib/engine-core.js.map +1 -0
  57. package/lib/engine-note.d.ts +35 -0
  58. package/lib/engine-note.js +32 -0
  59. package/lib/engine-note.js.map +1 -0
  60. package/lib/engine.d.ts +329 -0
  61. package/lib/engine.js +1179 -0
  62. package/lib/engine.js.map +1 -0
  63. package/lib/enrich-queue.d.ts +66 -0
  64. package/lib/enrich-queue.js +126 -0
  65. package/lib/enrich-queue.js.map +1 -0
  66. package/lib/enrich-store.d.ts +31 -0
  67. package/lib/enrich-store.js +103 -0
  68. package/lib/enrich-store.js.map +1 -0
  69. package/lib/enrich-wire.d.ts +163 -0
  70. package/lib/enrich-wire.js +353 -0
  71. package/lib/enrich-wire.js.map +1 -0
  72. package/lib/enrich.d.ts +102 -0
  73. package/lib/enrich.js +267 -0
  74. package/lib/enrich.js.map +1 -0
  75. package/lib/gitops.d.ts +107 -0
  76. package/lib/gitops.js +539 -0
  77. package/lib/gitops.js.map +1 -0
  78. package/lib/index.d.ts +241 -0
  79. package/lib/index.js +1162 -0
  80. package/lib/index.js.map +1 -0
  81. package/lib/indexing.d.ts +98 -0
  82. package/lib/indexing.js +223 -0
  83. package/lib/indexing.js.map +1 -0
  84. package/lib/injections.d.ts +90 -0
  85. package/lib/injections.js +165 -0
  86. package/lib/injections.js.map +1 -0
  87. package/lib/last-resort.d.ts +24 -0
  88. package/lib/last-resort.js +27 -0
  89. package/lib/last-resort.js.map +1 -0
  90. package/lib/log-shape.d.ts +38 -0
  91. package/lib/log-shape.js +93 -0
  92. package/lib/log-shape.js.map +1 -0
  93. package/lib/models-command.d.ts +74 -0
  94. package/lib/models-command.js +100 -0
  95. package/lib/models-command.js.map +1 -0
  96. package/lib/notice.d.ts +74 -0
  97. package/lib/notice.js +92 -0
  98. package/lib/notice.js.map +1 -0
  99. package/lib/plans.d.ts +154 -0
  100. package/lib/plans.js +315 -0
  101. package/lib/plans.js.map +1 -0
  102. package/lib/preset.d.ts +186 -0
  103. package/lib/preset.js +212 -0
  104. package/lib/preset.js.map +1 -0
  105. package/lib/prompt-keys.d.ts +7 -0
  106. package/lib/prompt-keys.js +8 -0
  107. package/lib/prompt-keys.js.map +1 -0
  108. package/lib/prompt-loop.d.ts +73 -0
  109. package/lib/prompt-loop.js +147 -0
  110. package/lib/prompt-loop.js.map +1 -0
  111. package/lib/prompt-push.d.ts +47 -0
  112. package/lib/prompt-push.js +115 -0
  113. package/lib/prompt-push.js.map +1 -0
  114. package/lib/prompt-runner.d.ts +48 -0
  115. package/lib/prompt-runner.js +90 -0
  116. package/lib/prompt-runner.js.map +1 -0
  117. package/lib/prompt-state.d.ts +142 -0
  118. package/lib/prompt-state.js +101 -0
  119. package/lib/prompt-state.js.map +1 -0
  120. package/lib/prompt.d.ts +270 -0
  121. package/lib/prompt.js +408 -0
  122. package/lib/prompt.js.map +1 -0
  123. package/lib/provider.d.ts +44 -0
  124. package/lib/provider.js +66 -0
  125. package/lib/provider.js.map +1 -0
  126. package/lib/redact.d.ts +24 -0
  127. package/lib/redact.js +85 -0
  128. package/lib/redact.js.map +1 -0
  129. package/lib/registry.d.ts +187 -0
  130. package/lib/registry.js +235 -0
  131. package/lib/registry.js.map +1 -0
  132. package/lib/removal.d.ts +159 -0
  133. package/lib/removal.js +556 -0
  134. package/lib/removal.js.map +1 -0
  135. package/lib/render.d.ts +97 -0
  136. package/lib/render.js +530 -0
  137. package/lib/render.js.map +1 -0
  138. package/lib/repair-run.d.ts +76 -0
  139. package/lib/repair-run.js +506 -0
  140. package/lib/repair-run.js.map +1 -0
  141. package/lib/repair.d.ts +208 -0
  142. package/lib/repair.js +357 -0
  143. package/lib/repair.js.map +1 -0
  144. package/lib/repo.d.ts +78 -0
  145. package/lib/repo.js +184 -0
  146. package/lib/repo.js.map +1 -0
  147. package/lib/rules-commands.d.ts +46 -0
  148. package/lib/rules-commands.js +379 -0
  149. package/lib/rules-commands.js.map +1 -0
  150. package/lib/rules.d.ts +267 -0
  151. package/lib/rules.js +478 -0
  152. package/lib/rules.js.map +1 -0
  153. package/lib/save-core.d.ts +72 -0
  154. package/lib/save-core.js +220 -0
  155. package/lib/save-core.js.map +1 -0
  156. package/lib/search.d.ts +112 -0
  157. package/lib/search.js +282 -0
  158. package/lib/search.js.map +1 -0
  159. package/lib/session-rules.d.ts +222 -0
  160. package/lib/session-rules.js +454 -0
  161. package/lib/session-rules.js.map +1 -0
  162. package/lib/signature.d.ts +45 -0
  163. package/lib/signature.js +199 -0
  164. package/lib/signature.js.map +1 -0
  165. package/lib/store.d.ts +156 -0
  166. package/lib/store.js +368 -0
  167. package/lib/store.js.map +1 -0
  168. package/lib/sync.d.ts +220 -0
  169. package/lib/sync.js +810 -0
  170. package/lib/sync.js.map +1 -0
  171. package/lib/tools.d.ts +113 -0
  172. package/lib/tools.js +1090 -0
  173. package/lib/tools.js.map +1 -0
  174. package/lib/treedx/client.d.ts +57 -0
  175. package/lib/treedx/client.js +82 -0
  176. package/lib/treedx/client.js.map +1 -0
  177. package/lib/treedx/provider.d.ts +39 -0
  178. package/lib/treedx/provider.js +479 -0
  179. package/lib/treedx/provider.js.map +1 -0
  180. package/lib/treedx/state.d.ts +27 -0
  181. package/lib/treedx/state.js +65 -0
  182. package/lib/treedx/state.js.map +1 -0
  183. package/lib/types.d.ts +210 -0
  184. package/lib/types.js +31 -0
  185. package/lib/types.js.map +1 -0
  186. package/lib/version.d.ts +9 -0
  187. package/lib/version.js +10 -0
  188. package/lib/version.js.map +1 -0
  189. package/lib/vocabulary.d.ts +41 -0
  190. package/lib/vocabulary.js +118 -0
  191. package/lib/vocabulary.js.map +1 -0
  192. package/lib/warnings.d.ts +10 -0
  193. package/lib/warnings.js +18 -0
  194. package/lib/warnings.js.map +1 -0
  195. package/package.json +151 -4
@@ -0,0 +1,990 @@
1
+ # Development and Test Loop
2
+
3
+ The problem is real: **this repository is edited from inside a running DeepSeek Harness, and a plugin
4
+ only takes effect after that host restarts.** Restarting it would kill the session doing the work. So
5
+ the loop has to be built so that it never touches the host it runs in. Everything below was verified on
6
+ this machine, not assumed.
7
+
8
+ ## Environment Facts (checked, not inferred)
9
+
10
+ | Thing | Value |
11
+ |---|---|
12
+ | Host serving this session | the GLOBAL npm install — but the PREFIX IS NOT PART OF THE FACT. Resolve it: `readlink -f $(command -v dsh)`, then read that package's `version`. On this box today: `/home/adrian/node_modules/@deepseek-ai/dsh`, **0.2.0-rc.2**, listening on `127.0.0.1:3080`. This row used to assert `/usr/lib/...` and `0.1.5-rc.1` as if they were the machine; a drift gate that hardcoded that path then skipped forever while the package sat in the repo's own `node_modules` (#54). |
13
+ | Profile root | `~/.dsh` (`DSH_HOME` default) — `profiles/`, `sessions/`, `storages/`, `.credentials.yaml` |
14
+ | Sandbox effect | `~/.dsh` is **read-only** from the agent sandbox: `dsh web --help` died with `EROFS: read-only file system, open '/home/adrian/.dsh/profiles/web/cordis.yml'` |
15
+ | Dev fork | `~/Projects/deepseek-harness` — git `fb2c4b9e69` (2026-09-10), **0.1.5-rc.2**, pnpm workspace, `node_modules` installed |
16
+ | `/app/checkout` | **does not exist on this machine.** It appeared in my *session context* as a presumed harness checkout, not in this repository's docs — I checked, and no file here references it. The real fork is the row above |
17
+
18
+ Two consequences worth stating plainly:
19
+
20
+ 1. **The sandbox is an ally, not an obstacle.** Because `~/.dsh` is read-only to an agent shell, the
21
+ accidental self-destruct path — installing into the live profile and restarting the host running the
22
+ work — is *structurally unavailable*. The dev loop is the only loop available.
23
+ 2. **The fork is newer than the host** (rc.2 vs rc.1), and `examples/deepseek-harness` is a third
24
+ snapshot. Line numbers in `docs/contract.md` come from the `examples/` copy. Treat any of the three as
25
+ truth only for the question you asked it; resolve real behaviour against the version you are actually
26
+ booting.
27
+ 3. **Check the binary you are measuring, not just the code you changed.** #54 lost hours to
28
+ `.dshdev-local/profiles/web/node_modules/@fractalsynapse/dsh-chapters` having become a COPIED tree
29
+ dated six days earlier: every browser journey, probe and re-distill attempt was running a build
30
+ predating the feature under test, and drew confident conclusions about the HOST — including a
31
+ seed-admission refusal the current code does not produce (`tests/e2e/boot.ts` refuses to spawn
32
+ on a copied tree now, and names the re-link command). The general habit: where a built artifact
33
+ is loaded by path, link it or assert its version — an old binary passes every test you write
34
+ against it and reports them as facts.
35
+
36
+ ## The Isolation Mechanism: `DSH_HOME`
37
+
38
+ `dsh --help` documents profiles as "the profile under `$DSH_HOME/profiles` to boot", and `DSH_HOME`
39
+ relocates the entire root — profiles, sessions, storages, credentials. Verified:
40
+
41
+ ```bash
42
+ mkdir -p .dshdev
43
+ DSH_HOME=$PWD/.dshdev dsh web --port 0 --no-open
44
+ # → dsh web: http://127.0.0.1:32869/?token=…
45
+ ```
46
+
47
+ A **fresh, fully isolated instance** booted: it created its own `profiles/web` under `.dshdev`, chose a
48
+ free port, minted its own token, and left `~/.dsh` and the `:3080` host untouched. `--port 0` means the OS
49
+ picks, so nothing conflicts. This is the whole answer: we get a disposable harness.
50
+
51
+ `.dshdev/` is gitignored. Never point `DSH_HOME` at `~/.dsh`, and never run `dsh plugin --profile web add`
52
+ without it — that is the command that mutates the profile this session is living in.
53
+
54
+ ### The dev profile for the real target: `scripts/bootstrap-dev-profile.sh`
55
+
56
+ Everything so far measured on cloud models; the plugin's target is the user's **Local
57
+ qwen3.8-flash-next** at the **64K window the user actually runs** (32K was tried and
58
+ rejected as too small for real work — the 27-tool header alone is ~13–15K; the user
59
+ confirmed the model is configured for 64K). `dev/settings.yaml` +
60
+ `dev/profile-cordis.patch.yml` mirror the live `~/.dsh` Local-provider block exactly —
61
+ same model, window (64K, just under the server's real `n_ctx: 65536`), and timeouts — and
62
+ make **Chapters the default preset** and **Local the default model**, so nothing there is
63
+ "enabled by remembering". The r27 gold run proves the engine's pressure math at this exact
64
+ regime (threshold 57,600; a 93K-token session compacts before its first request).
65
+ Bootstrap is write-if-missing for your tuned settings; `--force` re-templates. Build + link +
66
+ settings + credential-refs copy in one shot:
67
+
68
+ ```bash
69
+ scripts/bootstrap-dev-profile.sh # -> .dshdev-local (browser-safe: no probe)
70
+ scripts/bootstrap-dev-profile.sh --with-probe # scripted rounds only (probe self-exits)
71
+ ```
72
+
73
+ ### The `dev/install/` platform (three development modes)
74
+
75
+ `dev/install/` provides a unified installer and containerized platform that replaces ad-hoc bootstrap
76
+ for new developers. Three modes, same plugin set (dsh-chapters + dsh-codegraph + dsh-tinyfish-search),
77
+ same Cordis configuration mirror:
78
+
79
+ | Mode | Command | When to use |
80
+ |------|---------|-------------|
81
+ | **Centralized** | `./dev/install/install.sh && dsh web` | Day-to-day dev on your main machine; modifies your live web profile additively |
82
+ | **Compartmentalized** | `DSH_HOME=/tmp/dsh-dev ./dev/install/install.sh` | Isolated testing without touching live sessions; throwaway or persistent alternate home |
83
+ | **Containerized** | `cd dev/install && docker compose up -d` | Full stack: vLLM (Qwen3-4B-AWQ, inference + systemone), DSH Web UI, Playwright test runner |
84
+
85
+ The containerized stack is self-contained — no external API keys required for core operation. The vLLM
86
+ service serves two model names (`qwen3-4b-instruct` for inference, `qwen3-4b-systemone` for decisions)
87
+ from the same weights, cached in a Docker volume. The test runner mounts the repo source read-only so
88
+ edits are reflected without rebuild. Environment variables are read from the parent shell — no `.env`
89
+ file needed. See `dev/install/README.md` for full documentation and `.dsh/skills/local-llm/SKILL.md`
90
+ for the agent skill covering endpoint usage.
91
+
92
+ The legacy `scripts/bootstrap-dev-profile.sh` still works for the probe-based isolation loop described
93
+ below; `dev/install/` is the recommended path for new setups.
94
+
95
+ ### CodeGraph — structural code search
96
+
97
+ All three dev modes install `@jiangzhenguo/dsh-codegraph`, which builds an AST-level index of every
98
+ symbol and call edge in the project (`.codegraph/codegraph.db`). The agent uses `codegraph_explore`
99
+ instead of grep/glob/read to find symbols, understand flows, and trace callers/callees — including
100
+ dynamic-dispatch hops that text search misses. Configured with `surface: core` (status/init/sync/explore)
101
+ and `frontload: true` (structural context injected into prompts). After editing source, run
102
+ `codegraph_sync` so subsequent queries reflect the new code. The index is gitignored and persists
103
+ across sessions.
104
+
105
+ ### The `docs/superpowers/` plan directory
106
+
107
+ The upstream `writing-plans` / `brainstorming` skills (`.dsh/skills/`, from obra/superpowers — see
108
+ dev/install README § Skills) default their artifacts to `docs/superpowers/plans/YYYY-MM-DD-<feature>.md`
109
+ and `docs/superpowers/specs/…`. Those files are **tracked**: they are per-feature design records, and a
110
+ later session may re-reference them for the *why* behind an implementation. Two rules keep them honest:
111
+
112
+ 1. A plan is stamped with a `**Status:**` header when its work lands (implemented → commit refs +
113
+ pointers to the shipped code/tests/site docs), so nobody executes a spent to-do list.
114
+ 2. The durable cross-session record remains the chapters store and `docs/` proper; a superpowers plan
115
+ is a working artifact of one pre-implementation session, not a second index. If it rots into
116
+ contradiction with the shipped code, update or supersede it in place via git history — never rewrite
117
+ silently.
118
+
119
+ ### Is the corpus anchored to the host we pin? (`tests/fixtures/model-tape/CORPUS_HOST`)
120
+
121
+ Tape matching is normalized request-PREFIX matching, and the prefixes carry the host's
122
+ system-prompt composition. So moving `dev/dsh-host-lock`, editing the preset persona, or changing a
123
+ tool description invalidates the whole corpus — and a replay failure then describes an *earlier*
124
+ change, not the one under review. `scripts/corpus-anchor.mjs` states the provenance in one file and
125
+ answers the question in exactly one place; two consumers use it:
126
+
127
+ - `tests/e2e/boot.ts` **refuses replay by name** when `host` ≠ the pinned version, quoting the
128
+ recorded `reason` and the tracking issue. `E2E_MODEL=record` is deliberately exempt — re-distilling
129
+ is the way out, and a gate that blocked it would strand itself.
130
+ - `.github/workflows/ci.yml`'s `host-capability` job sets `corpus-anchored`, the `e2e-replay` job
131
+ requires it, and the run log gets a step summary plus a `::warning::` naming the issue. Skipped,
132
+ visibly — the same discipline as the preset-row refusal it sits beside: never silently green,
133
+ never permanently red.
134
+
135
+ The gate lifts with no code change: re-distill the eight projects
136
+ (`scripts/distill-tapes.sh`), write the new `host =` line, done. Current state: pinned
137
+ `0.2.0-rc.2`, corpus distilled `0.1.5-rc.1`, tracked by
138
+ [issue #56](https://github.com/fractalsynapse/dsh-chapters/issues/56) (with the door-selector blocker
139
+ [#57](https://github.com/fractalsynapse/dsh-chapters/issues/57) inside it). Rows in
140
+ `tests/unit/corpus-anchor.test.ts` assert both directions — matching lifts, missing/garbled
141
+ provenance refuses, and the committed repo state is reported honestly as NOT anchored.
142
+
143
+ ### Reading a distill while it runs (#54)
144
+
145
+ `scripts/distill-tapes.sh` writes each leg's FULL playwright output to `var/distill/<proj>-<hhmmss>.log`
146
+ (never `tail`, which buffers until the leg ends) and prints a 60-second heartbeat: tape-entry count
147
+ plus the last line of `var/e2e-upstream.log`. That journal is the proxy's own record of every upstream
148
+ attempt — `START / FIRSTBYTE / STALLED / DONE / DEAD`, with elapsed ms and byte counts — and the proxy
149
+ now bounds the wait at `E2E_UPSTREAM_TIMEOUT_MS` (default 600000, 10 minutes: generous for a cold
150
+ prefill at the stress regime, short enough that a hung connection ends the run with an answer instead
151
+ of silently consuming the harness's own 3600000 ms profile timeout).
152
+
153
+ So the question "is this turn slow, or is the connection dead?" is a lookup:
154
+
155
+ | journal shape | meaning |
156
+ |---|---|
157
+ | `START` then `FIRSTBYTE` | answered; `DONE elapsed=… bytes=…` says how long and how much |
158
+ | `START`, `STALLED elapsed=…ms bytes=0`, then `DEAD` | upstream never responded — a hang, not a slow model |
159
+ | `START`, `FIRSTBYTE`, then `STALLED … bytes=N` growing | real streaming progress; leave it alone |
160
+ | no journal line at all | the request never reached the proxy (boot/port/profile problem) |
161
+
162
+ ### Three test layers (and why the third exists)
163
+
164
+ | Layer | Command | Runs | Covers |
165
+ |---|---|---|---|
166
+ | unit | `npm test` (`tests/unit`) | node --test, no build, no network | pure core: registry, render, archive, notice, engine-core, tools, commands |
167
+ | integration | (today: probe rounds — see spikes/probe/README pattern) | boots real hosts in scratch homes | host wiring, realm mounts, provider economics |
168
+ | e2e (tape — the default loop) | `npm run test:e2e:replay` | same boots with `E2E_MODEL=replay`; model turns answered from `tests/fixtures/model-tape/<project>` via the record/replay proxy (`tests/e2e/model-proxy.ts`) — seconds, deterministic |
169
+ | e2e (distillation — occasional) | `npm run test:e2e:record` | `E2E_MODEL=record`: forwards every exchange to the real Local model and appends the tape. Run once per scenario; wipe `var/model-tape` first for a clean corpus; tapes are isolated per project (eight: suite/heavy/arrival/fanout/enrich/rules/fork/plans); re-record when scenarios, prompts, or thresholds change. The GPU model is the DISTILLER, never the test oracle (user directive 2026-09-19) |
170
+ | e2e | `npm run test:e2e` (`tests/e2e`, EIGHT projects: suite/heavy/arrival/fanout/enrich/rules/fork/plans) | Playwright + workspace-cached chromium vs a throwaway `var/e2e-home-<port>` rebuilt per boot (port-keyed so no boot can stomp another's state) (isolated from live agents — see FINDINGS 2026-09-19) | the browser: identity, fork button + switch, knowledge loop end-to-end, the oversized turn (mid-turn compactions), the subagent fan-out, arrival-time artifacting (low-floor project) |
171
+
172
+ The e2e layer exists because client-plane facts were being GUESSED (services, methods, tooltips,
173
+ switching) and every guess cost the user a browser round-trip. Playwright answers those in seconds
174
+ and its suite is the regression net for all future UI surface. Each project boots through
175
+ `boot.ts` (probe bundle removed first — it would otherwise kill the test server mid-suite; per-
176
+ project env pins: suite threshold 0.75, heavy 0.5 + arrival floor off, arrival floor 500 at the
177
+ production 0.9; homes are port-keyed). `discovery.spec.ts` keeps a health assertion on client-entry
178
+ activation, `fork-button.spec.ts` proves the click end-to-end on durable + visible planes, and the
179
+ heavy specs pin their claims to DURABLE facts (own-log turn/end, registry chapters, coverage) —
180
+ never to model verbosity. ACCEPTANCE DEFAULTS TO THE TAPE (`test:e2e:replay`, model turns served
181
+ from `tests/fixtures/model-tape/<project>` in seconds — explore: 14.7 s replayed vs 1.7 min live); the live
182
+ chain remains the distiller (`test:e2e:record`) and the ultimate witness. AGENTS-BLIND
183
+ masking (user decision 2026-09-20, reaffirmed 2026-10-01 with no exceptions): the tapes
184
+ stand in for ANOTHER project — a workspace docs edit (AGENTS.md, skill catalogs) NEVER
185
+ requires a re-record. The delimiter mask strips injected doc blocks from matching on both
186
+ sides, and a compaction summary that paraphrases doc prose (observed 2026-09-23 in
187
+ `heavy`/`fanout`) rides the tape as recorded model output — replay-side requests never
188
+ regenerate it, so committed tapes stay green across doc edits. Never gate a doc edit on
189
+ a distillation.
190
+
191
+ Re-record discipline (learned the hard way 2026-09-22/23):
192
+ - **A tape records the CONTEXT it was distilled in.** A spec replayed in a different
193
+ predecessor set (or a different boot port before HOSTPORT-BLIND landed) sees different
194
+ injected presence and misses. Re-record the whole project, not a single spec, when specs
195
+ share a boot — and never run two GPU-consuming suites concurrently (RR2=137 + timeout
196
+ cascades: a fanout that passes alone failed at 1.1 h while a heal chain shared the server).
197
+ **Home freshness is part of the context** (measured 2026-09-29): the host injects
198
+ skill-catalog-changed and runtime-context snapshot notices on the FIRST boot against a
199
+ fresh/absent throwaway home. Distilling against a stale home produced a tape without those
200
+ messages that a pristine replay then demanded — structural message-count divergence, which
201
+ no content mask can bridge. Discipline: `rm -rf var/e2e-home-<port>` before RECORD and
202
+ before the verifying REPLAY, so both sides saw the same boot world. Hosted CI is inherently
203
+ pristine: a locally-distilled tape that passes only on an accumulated home is a red flag,
204
+ not a pass.
205
+ - **Ports are parameterized end to end**: the model proxy derives `E2E_PROXY_PORT = boot port
206
+ + 1000` (a hard-coded 41799 turned one killed run's orphan into EADDRINUSE for every later
207
+ boot), and `boot.ts` fails FAST with a named-port message when the boot port is already
208
+ owned (an orphan's silence cost 20+ minutes of opaque hangs). Kill leftovers:
209
+ `pkill -f "dsh web --port 417"`.
210
+ - **The tape's matching is normalized on BOTH sides** — legacy stored prefixes are re-run
211
+ through `substituteAll` at load, so a new volatile rule (HOSTPORT-BLIND, `e2e-home-\d+`)
212
+ repairs older tapes without a re-record. Prefer that route over distilling again.
213
+ - **When a browser scenario fails, suspect its own tooling before the product.** The
214
+ parked-for-eras `rules` e2e was a spec reading zstd-compressed directories as flat files
215
+ (fixed 2026-09-23 with the session.ts `zstd -dc` route + dynamic rule-id capture); the
216
+ enrich e2e exposed two REAL product bugs (the frontmatter-anchor convention mismatch and
217
+ the missing host-plane route) that every deterministic layer had missed because each side
218
+ only ever tested against its own convention. The e2e is the integration witness — keep it.
219
+
220
+ ### Reading a dead boot (#165, #166)
221
+
222
+ When a rig boot never announces its URL, `tests/e2e/boot.ts` throws `bootFailureMessage()` from
223
+ `tests/e2e/boot-failure.ts`, and the message carries: the measured wait, the port, the name of the knob that raises the
224
+ window for diagnosis (`E2E_BOOT_TIMEOUT_MS` — readiness to serve, never a product claim), and the **child's own tail**.
225
+ That tail is the whole value: on 2026-10-28 the single line `[chapters-prompt] turn/end door armed` distinguished "the
226
+ child mounted and stalled before listening" from "the child crashed", and separated the real cause from four wrong
227
+ theories I had already published.
228
+
229
+ Three shapes must not be blurred, and are pinned by `tests/unit/boot-failure-message.test.ts`: a tail with no lines reads
230
+ `(empty)`; a log that could not be read reads `unreadable or never written` (absence is not emptiness); and output is
231
+ bounded at 1,200 characters with any removal marked `…[redacted]`, so suppression is never mistaken for silence. The
232
+ redaction is not cosmetic: the child prints its own boot URL **carrying a session token**, and the old verbatim slice
233
+ copied it into the Playwright error, the per-leg log and CI annotations. Read `var/e2e-server-<port>.log` and
234
+ `var/e2e-boot-<port>.json` as sensitive for the same reason — I `cat`-ed one during tonight's diagnosis.
235
+
236
+ **One boot per journey still applies** (site §24, #129): a chain failure after leg one is an environment symptom, not a
237
+ tape symptom. Ruled out on this machine by check rather than hope: dangling tool links, orphan servers, port squatting,
238
+ `ulimit`, inotify instances, free memory, lock files. If a lone leg passes and a chain does not, stop and read the boot
239
+ message before touching a test.
240
+
241
+ ### What `coverage:e2e` promises, and what it cannot deliver
242
+
243
+ `coverage:e2e` runs the replay chain under `E2E_COVERAGE=1` (a V8 profile per boot, aggregated by
244
+ `scripts/e2e-coverage.mjs`). **It is a coverage report, not an acceptance gate, and it cannot carry the browser timing
245
+ guarantees acceptance has.** Measured 2026-10-28 on one tree with load recorded (3.16–4.30 average on 20 CPUs, two other
246
+ live harnesses on this host, ~68 GB free):
247
+
248
+ | attempt | legs that failed | signature |
249
+ |---|---|---|
250
+ | 1 | `suite` | `button[aria-label="Fork with chapters"]:visible` — `Timeout: 120000ms` |
251
+ | 2 | `prompt` | `locator.click` — `Test timeout of 300000ms exceeded` |
252
+ | 3 | eight of nine | mixed |
253
+ | 4 | `fanout` | `new-session click never set dsh.sessions.current` |
254
+
255
+ The controlled A/B fixes the cause: `LEG=suite MODE=replay` → **`4 passed`**; the same command with `E2E_COVERAGE=1` →
256
+ the same 120 s door timeout. So it is V8 instrumentation cost on the server process, landing on whichever leg happens to
257
+ be slow — not a stale corpus or a broken product. Profile volume is not it (19 MB / 4 files for a failing boot), which is
258
+ why narrowing what gets instrumented is the only lever, and that changes what the percentage means.
259
+
260
+ **Rules that follow.** Two ceilings are involved (120 s `toBeVisible`, 300 s test timeout) and **neither is to be raised**:
261
+ widening a wait to satisfy an instrumentation mode is `AGENTS.md`'s "never widen a test to pass". Read the e2e number as
262
+ a report; the blocking coverage verdict is CI's deterministic gate (statements/branches/functions against 85, in the merge
263
+ job). And when quoting an e2e figure, quote the tree and the machine conditions with it — that is how 77.94%, 77.97%,
264
+ 78.05% and 78.34% came to be cited interchangeably when each belongs to a different corpus. See #158, and site §24 for the
265
+ user-facing version.
266
+
267
+ ### The coverage ledgers (added 2026-09-23 by the coverage audit; gated same day)
268
+
269
+ - `npm run coverage:ci` — **the gate CI enforces.** `c8 --check-coverage
270
+ --statements 85 --branches 85 --functions 85 --lines 85` over the whole
271
+ deterministic suite (unit + integration in one process graph; live TreeDX
272
+ rows self-skip when the container is down, so it passes on a runner with no
273
+ TreeDX and no LLM). Below any threshold it exits non-zero; the report lands
274
+ in `var/coverage/` and the summary is posted to the CI job page. Current
275
+ ledger: **97.98% stmts / 85.61% branch / 93.62% funcs** across 450 tests.
276
+ The branch number is the binding one (defensive `??`/ternary arms dominate
277
+ the miss count); it is deliberately kept above 85 by the `*-branches.test.ts`
278
+ sweep files rather than by lowering the bar.
279
+ - `npm run coverage` — the same deterministic suite under node's native
280
+ `--experimental-test-coverage` (a quick second opinion; no gate).
281
+ - `npm run coverage:e2e` — replays the **nine** browser segments with
282
+ `E2E_COVERAGE=1`, which makes every boot write NODE_V8_COVERAGE profiles
283
+ under `var/e2e-cov/<port>` (flushed on the boot's clean exit), then runs
284
+ `scripts/e2e-coverage.mjs` → **c8 with real source maps** (tsconfig
285
+ `sourceMap: true`; lib/*.js maps back to src/*.ts) for istanbul-grade
286
+ per-module Stmts/Branch/Funcs. Read it as "covered by SOME boot of the
287
+ suite" (a union), and mind the script header's c8 gotcha: include/exclude
288
+ filters apply BEFORE remapping, so `--include=src/**` would silently
289
+ zero every row.
290
+ **Ledger, re-measured 2026-10-23 on the PINNED hermetic host (`97c7c79`, strict replay
291
+ rc 0 / zero misses, all nine segments): 78.26% stmts / 59.01% branch / 71.38% funcs.** Per module:
292
+ engine.ts 87.93/55.06, index.ts 85.78/52.43, rules.ts 88.83/58.87,
293
+ store.ts 91.64/79.59, continue-core.ts 94.05/66.66, arrival.ts 94.87/60.60,
294
+ sync.ts 83.95/59.89, tools.ts 64.06/49.66.
295
+ **This is a slip, stated as one:** the 2026-09-25 ledger read 80/62 overall
296
+ (tools 72/49, engine 81/59, index 86/55). Statements and branches both went
297
+ DOWN while engine's statement coverage rose — because the re-anchored legs
298
+ are shorter, not weaker: `heavy` replays in 4 tape entries (2 compaction
299
+ triples, and it now answers) where the old corpus carried 16–22, and a
300
+ shorter conversation walks fewer engine arms. Nothing here is gated — the
301
+ gated ledger is `coverage:ci` at 85/85/85/85 on the deterministic plane —
302
+ so the honest reading is: *e2e coverage tracks how much conversation the
303
+ corpus replays, and a faster-converging corpus covers less of the engine by
304
+ accident.* Recovering the number means adding breadth deliberately (more
305
+ distinct journeys, or longer ones with their own claim), never padding a
306
+ leg to fatten a percentage.
307
+ Both ledgers together are the coverage claim; neither alone is honest.
308
+
309
+ ### Release status (measured 2026-09-25)
310
+
311
+ `@fractalsynapse/dsh-chapters@0.1.0` is LIVE on npmjs.org — published manually from
312
+ the CI-verified tag (`npm publish --access public --provenance=false`; the
313
+ registry shasum matches the CI tree byte-for-byte, install-smoke green).
314
+ Manual because the Production environment's NPM_TOKEN cannot CREATE a
315
+ package it is not scoped to (E404 = granular-token scope limitation, not a
316
+ secret problem) — and that whole class of auth is now gone: **CI publishes
317
+ run on npm TRUSTED PUBLISHING (OIDC)**. release.yml carries no token;
318
+ setup-node gets `registry-url` but never a `token:` (any `_authToken` in an
319
+ npmrc overrides the OIDC path), `id-token: write` is the credential, and the
320
+ npm side must register this package's Trusted Publisher as: repo
321
+ `fractalsynapse/dsh-chapters`, branch `main`, workflow `release.yml`,
322
+ environment `Production`. Prerelease versions publish under a matching
323
+ dist-tag so 'latest' stays stable. `0.1.1` is the first OIDC + provenance
324
+ ship; `0.1.0` itself carries no attestation.
325
+
326
+ ### Hosted CI
327
+ **The rig writes into its own port's store, never the project's (#61 item 8).** Each boot pins
328
+ `artifactStoreRoot` to `var/e2e-home-<port>/chapters` through the profile patch
329
+ (`tests/e2e/boot.ts::applyRigPluginPatch`), and CI brackets the replay job with
330
+ `scripts/assert-store-untouched.mjs snapshot` / `verify` — a listing comparison (names, sizes, mtimes)
331
+ that fails the job if the production `.dsh-chapters/` changed, including the pristine-runner case where
332
+ the store did not exist and a run CREATED one. Before this, every acceptance run left its chapters in the
333
+ repo's real store: 42 test trees and 29 of 47 live registry rows with e2e-shaped ids, which #60's auditor
334
+ then reported as 441 `test-litter` findings. Contamination like that does not just waste disk; it teaches
335
+ people to ignore a report that is telling them the truth about something else.
336
+ (`.github/workflows/`, wired + dress-rehearsed 2026-09-23)
337
+
338
+ - **`ci.yml`** — two jobs. `test`: `npm ci` (all deps public on
339
+ registry.npmjs.org — **no install secret**), typecheck, build,
340
+ `bash scripts/ci-gate.sh` (the 85/85/85/85 gate — since 0.1.6 the gate
341
+ ANNOTATES its own failure: metric lines, the pass/fail tally, up to 24
342
+ failing-test lines and c8's threshold verdicts as `::error::`, because on the
343
+ 0.1.6 tag a red gate left nothing behind but "exit code 1" while every local
344
+ reproduction of the same SHA was green; the summary step now runs
345
+ `if: always()`). `e2e-replay`: materializes the pinned host from
346
+ `dev/dsh-host-lock` (`npm ci`, hermetic — `npm i -g` rewrites the plugin tree and the
347
+ host's caret ranges float), plus chromium's apt libs, then `scripts/ci-replay.sh`, which
348
+ READS the version out of that lock rather than repeating it (#54 removed the second
349
+ literal; three copies of a pin is how it goes stale in three places) — eight browser
350
+ segments served entirely from the committed tapes. **No LLM, no key, no GPU:** in
351
+ replay mode `localModelUp()` short-circuits to true (specs RUN, they do not
352
+ skip) and a tape miss is a loud proxy 503, so a scenario that legitimately
353
+ changed fails the job instead of silently passing.
354
+ ### Distilling tapes WITHOUT the GPU box (#54)
355
+
356
+ `E2E_MODEL=record` drives the real model once per scenario to make a tape. Until #54 that
357
+ meant the local vLLM server had to be up. It no longer does, because the tape proxy
358
+ translates on the way out instead of asking the fixture to change:
359
+
360
+ ```bash
361
+ E2E_MODEL=record E2E_TAPE=rules E2E_PORT=41735 E2E_THRESHOLD=0.9 \
362
+ E2E_UPSTREAM=https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode \
363
+ E2E_UPSTREAM_MODEL=qwen3.8-flash PLAYWRIGHT_BROWSERS_PATH=$PWD/var/ms-playwright \
364
+ E2E_UPSTREAM_KEY="$(node -e "import('./scripts/llm.mjs').then(m=>process.stdout.write(m.apiKey()||''))")" \
365
+ npx playwright test -c tests/e2e/playwright.config.ts --project=rules
366
+ ```
367
+
368
+ - `E2E_UPSTREAM` is the endpoint WITHOUT the `/v1` suffix (the proxy appends the request
369
+ path, which already carries it).
370
+ - `E2E_UPSTREAM_KEY` is the credential the PROXY presents. The profile keeps `apiKeyEnv:
371
+ LOCAL_API_KEY` and the boot still injects its `e2e-tape-proxy-dummy` when none is
372
+ configured — a real secret never enters a fixture profile.
373
+ - `E2E_UPSTREAM_MODEL` rewrites the requested id, so the profile can keep pinning
374
+ `qwen3.8-flash-next` (the 32K stress-regime name) while a hosted endpoint is addressed by
375
+ the id it serves. **The tape records the profile's name**, because a tape is a fixture of
376
+ this rig, not of whatever service distilled it.
377
+ - A remote `E2E_UPSTREAM` in record mode with no key refuses at boot in one line, instead of
378
+ discovering it 401 times across eight projects.
379
+ - `scripts/llm.mjs` is the same resolution the dev scripts use (local server preferred, hosted
380
+ fallback, credential resolved from env → key file → profile export → harness store, value
381
+ never printed).
382
+
383
+ Three 0.2.x host drifts the first boots found, all rig-side, all now handled — each is the
384
+ kind that produces a wrong conclusion instead of an error:
385
+
386
+ 1. **the durable log is `session.v4.jsonl.zstd`.** Five places hardcoded `session.v3…`, so
387
+ every log-backed assertion decoded nothing and reported "the archive never landed".
388
+ `sessionLogFileFor()` resolves the highest `session.v<N>.jsonl.zstd`; a row scans
389
+ `tests/e2e/` for a hand-built versioned path (comment lines excluded — prose may name the
390
+ formats it explains).
391
+ 2. **the host opens a Preview Notice, and it is a focus trap** — the 2026-09-18 saga with a
392
+ new member. Dismissal now targets `[role=dialog]`-shaped elements and presses the control
393
+ that acknowledges *that* dialog; matching "any div mentioning a notice" matched the app
394
+ wrapper and pressed an unrelated 'Close'.
395
+ 3. **provider config must ride `cordis.patch.yml`.** A profile-dir `settings.yaml` is read by
396
+ nothing on 0.2.x, so `dev/settings.yaml`'s `llm-pi-ai` block was invisible: the host had no
397
+ `local` provider and the browser opened on "Add an API key to get started" with the
398
+ composer never reachable. `dev/profile-cordis.patch.yml` now defines the provider, and
399
+ `tests/e2e/boot.ts` guarantees the block in the copied home before rewriting the window
400
+ pins and the proxy `baseURL`.
401
+
402
+ - **`release.yml`** — bare-semver tag (`0.1.0`, matching the Production
403
+ environment's `*.*.*` tag protection) or manual dispatch → the full
404
+ gated deterministic suite as publish blocker → `npm publish
405
+ --provenance`. Two token facts learned the hard way (measured on the
406
+ first ship): the token must be written into the file npm actually reads
407
+ (setup-node exports `NPM_CONFIG_USERCONFIG` — `~/.npmrc` is ignored),
408
+ and the FIRST publish of a new package name needs a granular token with
409
+ 'Read and publish' over ALL packages (a package-scoped token 404s — it
410
+ cannot be scoped to a package that does not exist yet; narrow it to
411
+ `dsh-chapters` after the first ship).
412
+ --provenance`. Auth path B is OIDC trusted publishing (no long-lived
413
+ secret); path A uses `NPM_TOKEN` if configured.
414
+ - The dress rehearsal on a pristine machine caught three real blockers, all
415
+ fixed and pinned: (1) `bootstrap-dev-profile.sh` refused the comments-only
416
+ `cordis.patch.yml` scaffold that `dsh plugin add` lays down on a fresh home
417
+ — the guard now only protects patches with real entries; (2) a credential-
418
+ less home boots but every session dies composing the local provider —
419
+ tape-mode boots now inject a dummy `LOCAL_API_KEY` (the proxy ignores
420
+ auth; a real env key always wins); (3) **a fresh home has no registered
421
+ workspace**, so the UI's New-session draft never commits — every browser
422
+ spec dies at `newSessionWithTurn` (this had been masked for weeks by
423
+ accumulated state in the dev home — the exact class of drift CI exists to
424
+ catch). The bootstrap now seeds `storages/workspace.json` (exact host
425
+ schema: ISO dates, `updatedAt`, registered path = the checkout) when absent.
426
+
427
+ ### Safe default guard
428
+
429
+ Prefix every harness command with `DSH_HOME=$PWD/.dshdev` — or do not type it at all:
430
+ **`scripts/dsh-scratch.sh` is that wrapper** (`--home .dshdev2` for a parallel profile; it refuses any
431
+ resolved home under the real `~/.dsh` and execs `dsh` with `DSH_HOME` exported).
432
+
433
+ This stopped being advice the hard way: during Stage 0 a bare `dsh plugin --profile web add ...` (no
434
+ `DSH_HOME`) targeted the LIVE profile — the sandbox's read-only `~/.dsh` turned it into a harmless
435
+ `pnpm failed` and the live `package.json` mtime proved nothing was touched, but "the sandbox protects us"
436
+ is not a guard rail. Any future command form that can reach `~/.dsh` must route through the wrapper.
437
+
438
+ ## Test Layers
439
+
440
+ Adopted from how the reference plugins actually iterate (they mostly *avoid* restarting the host):
441
+ `dsh-session-fork` has ~8,000 lines across 28 `bun test` files and **every test file header says "no
442
+ cordis"** — logic takes an injected `Ports` interface and only `makePorts(ctx)` touches the kernel. Copy
443
+ that shape, and fix its one gap (§L1).
444
+
445
+ ### L0 — Pure logic, no cordis, no harness
446
+
447
+ Ranges validation, the chapter renderer, TOC assembly, budget math, slug/number allocation. Fast, CI-able,
448
+ no DSH at all. Structure for it from day one:
449
+
450
+ ```ts
451
+ // src/continue.ts — no cordis import anywhere in this module
452
+ export interface ContinuePorts {
453
+ readSession(id: string): Promise<SourceLog | null>
454
+ createContinuation(opts: { sessionId: string; seed: readonly SessionEvent[]; cwd?: string }): Promise<void>
455
+ reserveNumbers(key: string, count: number): Promise<number[]>
456
+ putRegistry(state: SessionState): Promise<void>
457
+ }
458
+ export async function runContinue(ports: ContinuePorts, args: ContinueArgs): Promise<ContinueResult>
459
+ ```
460
+
461
+ `src/index.ts` is then the **only** cordis-facing file — a thin adapter, deliberately untested.
462
+
463
+ ### L1 — A *stateful* seed fake (where the reference plugin fell short)
464
+
465
+ `dsh-session-fork`'s `createChildFromSeed` fake records `cut` as a number and validates nothing: it proves
466
+ *which* port calls happened in *what* order, never what the kernel does with a seed. Our seed is one
467
+ synthetic event, so a ~30-line fake can close that hole cheaply — **materialize** the seed and assert:
468
+
469
+ - seqs contiguous from `0`, our notice at `seq = 0`
470
+ - `surfaceOp: 'append'`
471
+ - `inheritedEventCount` equals `seed.length`
472
+ - the message source we emit is inside the kernel's accepted vocabulary
473
+
474
+ That last bullet is `dsh-session-fork`'s most expensive lesson: a retired message source **bricked 66
475
+ stored logs** which then refused to load, and they now pin the rule in a `format-watch` test because the
476
+ validator isn't published. Our TOC is a synthetic `user/message` — pin the vocabulary test *first*, not
477
+ after a chain of corrupted session files. Their `replaySurface()` idea (re-fold the durable log, assert it
478
+ equals the live surface) is the right invariant to steal wholesale.
479
+
480
+ ### L2 — Boot-time probe in an isolated profile → **this is how Phase 0 runs**
481
+
482
+ No LLM, no tokens, no HTTP driving, no restart of anything:
483
+
484
+ 1. Write the probe as a minimal plugin whose `apply()` runs the experiment during boot and writes JSON
485
+ facts to a file in the workspace.
486
+ 2. Install into the **scratch** profile. `dsh plugin add` runs pnpm and may reach for the network; if it
487
+ does, bypass it by symlinking the package into `.dshdev/profiles/web/node_modules/` and adding its id
488
+ via a `--patch` overlay (`dsh --patch <file>` applies an extra patch layer at boot).
489
+ 3. `DSH_HOME=$PWD/.dshdev dsh web --port 0 --no-open`, let it boot, read the JSON.
490
+
491
+ Phase 0 questions it answers: is an unseeded/one-event-seeded `agents.create` durable, listed, and
492
+ resumable; does `Session.create` accept the seed; does workspace attach work; does the child's header match.
493
+
494
+ ### L3 — Headless one-shot with a real model (only when a measurement needs it)
495
+
496
+ `dsh --profile headless "…"` boots a profile, answers one task, prints, and exits — scriptable. The
497
+ headless bundle emits JSON with `usage.cacheReadTokens`, which is the only way to get the cache numbers
498
+ G1–G3 in `docs/verify.md` without a browser.
499
+
500
+ Costs real model tokens and needs credentials: `$DSH_HOME/.credentials.yaml` is *inside* the relocated
501
+ root, so a scratch home starts unauthenticated. **Copying the user's credentials into a scratch root is
502
+ their call — ask before doing it, and ask before spending tokens on a measured run.** Prefer L0/L1 for
503
+ correctness and reserve L3 for the few numeric assertions.
504
+
505
+ ### L4 — Upstream drift pins
506
+
507
+ The habit that lets you iterate against a moving host without booting it: text-pin what you depend on in
508
+ the *installed* bundles and fail loudly on drift.
509
+
510
+ - `parity`-style: run the real published package against any vendored copy of its logic.
511
+ - `upstream-watch`-style: assert literal call sites still look the way our assumption says (e.g. that
512
+ `session.fork` still seeds with `events.slice(0, cut)` — the fact invariant 2 rests on).
513
+ - `vendor`-style: pin `file:line` + SHA markers in a header comment and assert they still exist.
514
+
515
+ On failure, the procedure is *re-read upstream and re-anchor*, never "relax the assertion".
516
+
517
+ ## Speed Notes — What Made This Fast
518
+
519
+ Learned while running eleven probe boots. Each is a concrete loop improvement, not a preference.
520
+
521
+ **One env var is the whole isolation story.** `DSH_HOME=$PWD/.dshdev` cost nothing to adopt and removed
522
+ the restart problem entirely. Every harness command in this repo should carry it — wrap them in
523
+ `scripts/` or a `Makefile` so the bare form never appears in a doc or a shell history.
524
+
525
+ **Failures before the provider are free.** Round 10's every turn errored on `{{model}}` assembly and
526
+ spent **zero** tokens. A probe that fails early costs a boot, not a bill — which means it is cheap to
527
+ probe aggressively and wrong. Prefer a probe that might error over reading five more source files.
528
+
529
+ **Plain `.js` probes beat a build step.** The spike needed no TypeScript, no `tsc`, no `lib/` output:
530
+ write `lib/roundN.js`, point `package.json` `main` at it, boot, read JSON. Ten seconds of edit-build-test
531
+ with no compile errors to fight. Keep the *real* plugin in TypeScript; keep the *spike* in JS.
532
+
533
+ **Reading the host's own session logs is the fastest source of truth.** `~/.dsh/sessions/<projectKey>/<id>/session.v3.jsonl.zstd`
534
+ shows exactly what a real `user/message` looks like, which `source.kind` values are in play, and how large a
535
+ request header actually is — all without a model call. Two gotchas that each cost a cycle:
536
+
537
+ - `zstd -dc <path>` **fails** on these paths: they begin `--home-…` and are parsed as flags. Use
538
+ `zstd -dc -- <path>`.
539
+ - Paths are bucketed by `projectKey(cwd)` (`session-persistence-jsonl/src/format.ts:224`), so a project
540
+ directory is encoded with `-` separators, e.g. `--home-adrian-Projects-dsh-chapter-fork--`.
541
+
542
+ **`Function.prototype.toString` is useless on the bundled host** — service methods report
543
+ `function () { [native code] }`. Do not plan to discover signatures by introspection; grep the
544
+ `examples/deepseek-harness` checkout instead. Three signatures cost one wasted boot each when guessed:
545
+ `listSessions(signal?)` is **positional**, `agents.resume({ resumeSessionId })` is an **options object**,
546
+ and `create` needs `agentOptions` plus a preset.
547
+
548
+ **TypeScript tests run with zero install and zero build — use this before reaching for vitest.**
549
+ `node --test tests/unit/*.test.ts` executes `.ts` directly on Node 24 via type stripping, so the pure core
550
+ (`src/render.ts`, `src/types.ts`) is covered by 15 tests in ~135 ms with **no `node_modules` at all**. That
551
+ keeps the L0/L1 layer free: no dependency resolution, no bundler, no config, and it works inside the agent
552
+ sandbox where an install may not. `npm test` is wired to it.
553
+
554
+ Reserve the vitest/bun decision for when kernel-facing mocks need more than Node's runner gives, and note
555
+ that `npm run typecheck` **does** need `typescript` + `@types/node` installed — a separate step from
556
+ testing, deliberately not taken yet.
557
+
558
+ **Keep probe turns tiny.** `Reply with exactly: ALPHA. No explanation.` finishes in one step and a few
559
+ tokens, so a measurement that needs five real turns still costs pocket change.
560
+
561
+ **Instrument the metric, not just the code.** The cache numbers only became readable once the formula was
562
+ pinned: `totalPrompt = inputTokens + cacheReadTokens`, `hit = cacheReadTokens / totalPrompt`. Dividing by
563
+ the uncached delta instead produces values like 1721% and quietly wrong conclusions — which is exactly what
564
+ happened in these notes before it was caught.
565
+
566
+ ## Implementation Status (kept current)
567
+
568
+ | Layer | State |
569
+ |---|---|
570
+ | `src/types.ts`, `src/render.ts` — pure chapter renderer, ranges, fence safety, artifact deferral | **done**, 15 tests |
571
+ | `src/archive.ts` — reserve→write→verify ordering, dedup, idempotent retry, coverage, tamper detection | **done**, 11 tests |
572
+ | `src/notice.ts` + `test/notice.test.ts` — TOC-notice builder, kernel-rejection reproduction, vocabulary pins + drift guard (Phase 0b check 5) | **done**, 10 tests |
573
+ | `src/registry.ts` + `src/store.ts` — pure numbering/ancestry/plan state; zod domain spec, storage-domain + node-fs adapters | **done**, 13 + 7 tests; durability witness passed |
574
+ | `src/engine-core.ts` + `src/engine.ts` — ChaptersCompactionEngine | **done & boot-proven** (r18/r19: realm subpath row mounts; real cascade finalizes; catch-path fix — FINDINGS § Phase 1) |
575
+ | preset delivery | **0.1.18: the preset IS a declaration row** — `preset-chapters` in `cordis.patch.yml`, mounted by `@fractalsynapse/dsh-chapters/preset` (`src/preset.ts`), which registers what the host can load and drops what it cannot. The `presets/chapters/` + copy-on-boot mechanism is deleted with the 0.2.0 floor. Verify.md P24 (rows + three-host boots) |
576
+ | `npm test` (unit + integration) | **188 passing, <2 s** — unit imports no harness services; integration runs the sync loop against a fake git driver AND a real git smart-HTTP server (git-http-backend on loopback; the git binary is test-only scaffolding, the product stays pure-JS) |
577
+ | 20-check verify.md pass | **r26/26b**: 18 mechanical on `.dshdev2` (refusals with numbers, tamper marks, idempotent retry, cross-boot registry + resume); human rows per verify.md evidence map |
578
+ | dsh-session-fork coexistence | **moot at this version** (r26a pair-boot): the example fails to boot the installed host ALONE (unguarded `webServer` read in the installed dsh-client-connection) — assumed-neutral until a version-matched pair-boot |
579
+ | `scripts/dsh-scratch.sh` | **done** — refuses any DSH_HOME under the live `~/.dsh`; the near-miss is in FINDINGS |
580
+ | `chapters_segment` / `chapters_continue` / `chapters_fork` tools (`src/tools.ts` → `continue-core.ts`) | **done & boot-proven** (r22 9/9 full loop incl. child read-back; r24 fork siblings reserve nothing; r25 durable title; r26/26b checklist pass) |
581
+ | E3: header cache across a compaction | **measured, per-machine (r23 cloud, r28 local)**: cloud held 7,424 cached across the replacement (refill 13,658 → ~7,000); local llama.cpp reuses NOTHING across a head replacement (0 cached) but steady-state turns cost only the new content. Never quote one machine's number for the other |
582
+ | Fork + Continue buttons | **shipped** (client bundle + slots; `remote.commands` trigger — `connection.rpc.handle` from a plugin is broken on this host, r26a; the buttons ride the command path instead). Both deterministic doors are rows over ONE client runner (`src/client/doors.ts` + the `DOOR_ACTIONS` table): `/chapters-continue` shipped as a composer command only, so its child was created correctly with no visible affordance |
583
+ | Long-context mechanics (architecture amendments) | **built + verified**: arrival-time artifacting (`src/arrival.ts`, `chapters_artifact`; e2e artifact-arrival green — 373KB book never prefilled, cache intact) and plot carriage (persona PLOT: + elicited fallback); e2e = 7 specs / 2 projects on a throwaway per-boot home (188 unit/integration) |
584
+ | Knowledge layer P1 (record §13) | **complete**: project identity, redaction, live turn signatures, topic-sequential composition (fork proven live, r36; engine wired, r29), sharded index + incremental manifest, `chapters_search`, `/chapters-link`+`/chapters-status`+notice v2 (Project line, `(N msgs)`), the sync loop with §5.3 local-only degradation + diverged-rebuild, both schedulers and all §5.1 triggers wired. 9/9 live (r35j) + the two-machine exit criterion PROVEN as a browser journey (e2e suite green 2026-09-18) (composer→real remote→machine B searches) |
585
+ | Topic mapping (composer) | **live-verified at the fork (r36)**: same-topic turns merge into one chapter on real signatures; topic changes split; pressure fragments stay legacy (measured, documented §4.2/4.3 + FINDINGS r28-36) — engine composition (r29) waits for deep selections |
586
+ | `npm run typecheck` / `npm run build` | **working** — typescript 5.9 + @types/node + host packages as devDeps; `.ts`-import convention kept via `rewriteRelativeImportExtensions` |
587
+
588
+ Two bugs the pure tests caught that would have been expensive later, which is the argument for this order:
589
+
590
+ 1. `coverage()` started its cursor at the first chapter, so a chapter set that skipped the **opening** of a
591
+ conversation reported no gap — silent history loss, exactly the failure the plugin exists to prevent.
592
+ 2. Substring assertions against `/chapters/` matched the store root `.dsh-chapters/` first. Worth knowing
593
+ as a general hazard: **the default store root name contains the word `chapters`**, so any path matching
594
+ in real code must anchor on the separator.
595
+
596
+ ## Unavoidably Manual
597
+
598
+ Say it rather than pretend the suite covers it:
599
+
600
+ | Needs a human | Why |
601
+ |---|---|
602
+ | Sidebar listing and visual resume of a continuation | no client-side harness reaches the real module loader contract |
603
+ | Cache/prefill economics across process generations | inherently multi-process; a restart recomposes every header |
604
+ | TOC readability and whether the model actually reloads chapters | judgement about prose and behaviour |
605
+ | The Phase 0 verdict itself, read from probe output | L2 produces the facts; a human confirms the interpretation |
606
+
607
+ ## The TreeDX dev loop (knowledge transport provider #2)
608
+
609
+ The TreeDX transport (`src/treedx/`, contract in `docs/provider.md`) is developed against an
610
+ **in-process stub service** — `tests/integration/treedx-stub-server.ts` — which is part of the
611
+ normal `npm test` loop (real HTTP on loopback, no docker, ~400 ms). The stub's semantics are
612
+ the documented ones (bearer 401, one writable lease per branch, fail-closed moved-base commit,
613
+ UTF-8 1 MiB File cap); when a real service disagrees, the live test below catches it and the
614
+ stub changes to match — never the other way.
615
+
616
+ The real service, for the exit-criterion run:
617
+
618
+ ```bash
619
+ # the HOST user drives docker (the agent sandbox has the docker group stripped; its
620
+ # HTTP view of the booted service is fine — that is where live tests execute from)
621
+ scripts/treedx-local.sh up # builds examples/treedx (dev target); FIRST BOOT compiles the Rust NIF — minutes
622
+ scripts/treedx-local.sh token # dev-token (30d), writes var/treedx-dev.env (gitignored, 0600)
623
+ scripts/treedx-local.sh smoke # the six-step transport walk via curl
624
+ node --test tests/integration/treedx-live.test.ts # the §13 exit criterion over the real API
625
+ scripts/treedx-local.sh down # stop; the data volume persists (down -v wipes)
626
+ ```
627
+
628
+ `treedx-live.test.ts` skips ONLY when unconfigured (no env, no `var/treedx-dev.env`):
629
+ configured-but-unreachable FAILS loudly — the skip flag is decided at load time, and a
630
+ test that silently declines to witness after being told to run is broken tooling (a real
631
+ bug class caught 2026-09-21, by the human, by running it). Proven live 2026-09-21:
632
+ 6/6 live tests green, full suite 286/286 with zero skips.
633
+ `dev/treedx.compose.yaml` exists because the cloned repo's own `compose.yaml` is bound to the
634
+ Treeseed platform network + connected auth and will not boot standalone (measured; recorded in
635
+ `spikes/treedx/FINDINGS.md` §Environment). The cloned checkout under `examples/treedx/` is
636
+ gitignored reference material — the live loop works from any path via `TREEDX_LOCAL_*` overrides.
637
+
638
+ ## Command Reference
639
+
640
+ ```bash
641
+ # disposable instance — prefer the wrapper (DSH_HOME is its problem, not yours):
642
+ # NOTE: .dshdev mounts dsh-chapters-probe, whose rounds process.exit when done — fine for
643
+ # scripted one-shot boots, WRONG for an interactive test (the server dies seconds after the
644
+ # URL prints). Manual testing belongs to .dshdev2 (plugin only; probe removed 2026-09-17).
645
+ scripts/dsh-scratch.sh web --port 0 --no-open # .dshdev (probe home)
646
+ scripts/dsh-scratch.sh --home .dshdev2 web --port 0 --no-open # clean home for the real plugin
647
+
648
+ # manual form, if the wrapper is unavailable:
649
+ mkdir -p .dshdev
650
+ DSH_HOME=$PWD/.dshdev dsh web --port 0 --no-open
651
+
652
+ # inspect the composed bundle tree of a profile without booting it
653
+ DSH_HOME=$PWD/.dshdev dsh --profile web --dump-config | head
654
+
655
+ # add our plugin to the SCRATCH profile only
656
+ scripts/dsh-scratch.sh --home .dshdev2 plugin --profile web add "link:$(pwd)"
657
+
658
+ # one-shot headless run (needs credentials in the scratch root; costs tokens)
659
+ DSH_HOME=$PWD/.dshdev dsh --profile headless "list your tools"
660
+
661
+ # unit tests
662
+ bun test # or vitest — decide in Phase 0, see docs/architecture.md § Open Questions
663
+ ```
664
+
665
+ **Never** run `dsh plugin --profile web add` or `dsh web` without `DSH_HOME=` set to the scratch root:
666
+ that is the profile and port this session is running on.
667
+
668
+ ## Correction to Earlier Claims in This Repo
669
+
670
+ Two things I asserted before checking, now that they are checkable:
671
+
672
+ 1. **"`dsh web` runs in a tmux pane on the dev fork at `/app/checkout`"** — wrong for this machine. There
673
+ is no `tmux` binary and no `/app/checkout`; the harness serving this session is the global npm install
674
+ on `:3080`. The dev fork is `~/Projects/deepseek-harness`.
675
+ 2. **An isolated instance's port.** I first reported `34159` from a garbled read; the actual boot printed
676
+ `32869`. The curl to `34159` failing was the tell — it was my number, not the server's.
677
+
678
+ Both are worth recording rather than quietly fixing, since the whole point of this file is that its
679
+ commands were *verified*.
680
+
681
+ ## Suggested Additions to the Other Docs
682
+
683
+ `docs/contract.md`'s file:line citations are from the `examples/` snapshot (rc.1-era), not the rc.2 fork —
684
+ re-anchor them against whichever version a test actually boots.
685
+
686
+ ### Running the TS tests on this box (node 22, no native type stripping)
687
+
688
+ Measured 2026-10-16: `npm test` (`node --test tests/unit/*.test.ts`) needs node ≥ 23 for type
689
+ stripping, and this machine's `/usr/bin/node` is **v22.22.1 compiled WITHOUT TypeScript support**
690
+ (`ERR_NO_TYPESCRIPT`, not merely a missing flag). The package's `engines` floor is node 24, and CI
691
+ installs it; locally, run the same files through `tsx`:
692
+
693
+ ```bash
694
+ npm install --no-save --no-audit --no-fund tsx unrun # both, in ONE command
695
+ node --import tsx --test tests/unit/*.test.ts tests/integration/*.test.ts
696
+ npx c8 --check-coverage --statements 85 --branches 85 --functions 85 --lines 85 \
697
+ node --import tsx --test tests/unit/*.test.ts tests/integration/*.test.ts
698
+ ```
699
+
700
+ Two traps measured while finding that:
701
+
702
+ - **`npm install --no-save A` then `--no-save B` removes A.** npm prunes what is not in the manifest,
703
+ so unsaved helpers must be installed in a single command (`unrun` disappeared the moment `tsx` was
704
+ added, and `npm run build` — which bundles the client through `tsdown` — failed with
705
+ `Cannot find module "unrun"`).
706
+ - `tsconfig.test.json` (committed) transpiles `src` + `tests` to `var/tsbuild` with `--noCheck` for
707
+ the rare case you want emitted JS instead of a loader. It is NOT how the suite is meant to run;
708
+ tests import `.ts` sources directly and are not typechecked by design (`npm run typecheck` covers
709
+ `src/` only).
710
+
711
+ Never fix a missing test runner by editing the tests. The suite is the contract; the runner is local.
712
+
713
+ ### Verifying against a host version that is not the installed one
714
+
715
+ A preset/delivery change must be proven on EVERY host release the package claims to serve, and the
716
+ installed host is one data point. Recipe measured 2026-10-16 (this is how 0.1.17 was verified on
717
+ 0.2.0-rc.2, 0.2.1-alpha.4 and the pinned 0.1.5-rc.1 in one sitting):
718
+
719
+ ```bash
720
+ # 1. a second host, installed as its own tree — never over the running one
721
+ mkdir -p var/host-versions/a4 && cd var/host-versions/a4
722
+ echo '{"name":"a4-host","private":true,"dependencies":{"@deepseek-ai/dsh":"0.2.1-alpha.4"}}' > package.json
723
+ npm install --no-audit --no-fund # ~1 min, 616 packages
724
+ # 2. a THROWAWAY home for it (profiles/web/package.json listing the bundles + this plugin),
725
+ # then boot THAT bin with DSH_HOME pointed at the new home:
726
+ DSH_HOME=$PWD/var/a4-home DSH_CHAPTERS_WITNESS=$PWD/var/a4-home/w.json \
727
+ node var/host-versions/a4/node_modules/@deepseek-ai/dsh/lib/bin.js web --port 0 --no-open
728
+ ```
729
+
730
+ The witness JSON is the answer — `presetRegister`, `presetDroppedRows`, `presetRoster` and
731
+ `presetInventoryRows`/`presetUnusableRows` say what a log line only hints at, and `presetUnusableRows: []`
732
+ is the assertion that the composition actually ACTIVATES rather than merely being declared.
733
+
734
+ Three traps, all paid for:
735
+
736
+ - **Never symlink a host tree into an npm-managed profile's `node_modules`.** `npm install` treats what it
737
+ does not recognize as extraneous and **deletes it through the link** — it ate a 500-package host closure
738
+ this way ("the closure is gone, `ls` returns nothing"). Copy instead (`cp -r`), or point the boot's own
739
+ installation anchor at the other tree and install ONLY the plugin into the profile.
740
+ - **`npm pack` + install-from-tarball, not `file:`-linked sources.** The artifact users get is the packed
741
+ `files` list; a linked working tree hides a missing `files` entry (e.g. an asset the preset needs).
742
+ - Boot with `timeout 45 …` and read the exit code: **124 means it stayed up** (healthy), `1` means the
743
+ plugin tree refused to load. A crashed boot and a slow boot both print to the same log; the code is the
744
+ difference between "no preset" and "no harness".
745
+
746
+ Prune after: `var/host-versions/`, the throwaway homes and `var/coverage` are all regenerable in minutes and
747
+ together cost ~2 GB — this repository's contract is the *measurements written down*, not their scratch.
748
+
749
+ ### The rig's regime has ONE table (`tests/e2e/rig-pins.ts`) — added 2026-10-23 (#56)
750
+
751
+ Three copies had silently diverged: `scripts/distill-tapes.sh`'s `THRESH`/`EXTRA` tables, the inline
752
+ env in `npm run test:e2e`, and `boot.ts`'s defaults — under a comment claiming distill "mirrors
753
+ `npm run test:e2e`'s pins exactly". Measured result: replay booted `suite`/`rules`/`fork` with **no
754
+ threshold** while their tapes had been recorded at 0.75/0.9/0.75. The realm differed, replay's requests
755
+ carried `[dsh:artifact …]` stubs the tapes contain none of, and the chain failed as tape MISSes that
756
+ look exactly like prompt drift.
757
+
758
+ `RIG_PINS` is now the single source, with two consumers: `playwright.config.ts` derives each project's
759
+ `env` through `rigEnv(proj)` (so workers see it), and `globalSetup` resolves the selected project from
760
+ `fullConfig.argv` and calls `applyRigEnv()` **before booting**. The second half is not optional —
761
+ Playwright applies project `env` to workers, and the object handed to `globalSetup` exposes an
762
+ UNFILTERED `projects` list whose entries carry no `env` (measured: 11 projects for a
763
+ `--project=heavy` run). "Just put it in the config" looked fixed and was not: the boot kept using
764
+ defaults. An explicit command-line value still wins, so re-tuning one leg never needs an edit; an
765
+ unknown project derives nothing instead of inventing defaults.
766
+
767
+ Rows: `tests/unit/e2e-pins-single-authority.test.ts` (table contents, `rigEnv` derivation, argv parsing
768
+ in both `--project=x` and `--project x` forms, apply-not-clobber, unknown-project, both consumers
769
+ reading the table, a literal `env: { 'E2E_TAPE' … }` block failing, and the runners carrying no copy).
770
+ `npm run test:e2e`'s per-leg header now prints the EFFECTIVE regime read from the table, so a tape's
771
+ provenance is auditable from the runner log alone.
772
+
773
+ ### Provenance is generated (`scripts/write-provenance.mjs`), never hand-authored (#56)
774
+
775
+ `PROVENANCE.md` lives inside each project directory — and the runner `rm -rf`s that directory before
776
+ recording. The re-anchor commit therefore deleted `plans/PROVENANCE.md` and `prompt/PROVENANCE.md`,
777
+ which are precisely the records stating those legs were distilled from the SCRIPTED stub rather than
778
+ a live model. A claim about what a tape means must not be destroyed by the act of re-making the tape:
779
+ the file is generated from the pin table plus the recorded verdict, and
780
+ `tests/unit/tape-provenance.test.ts` fails if any project lacks it or if the runner stops
781
+ regenerating it.
782
+
783
+ ### The pool mirror must be per-boot too, or notices are not reproducible (#85)
784
+
785
+ `artifactStoreRoot` has been pinned per port since #61, and I believed that made a boot's state
786
+ test-local. It did not: the compaction/continuation notice composes its `## CORE RULES` block from
787
+ `path.join(cwd, DEFAULT_CLONE_DIR)` (`src/engine.ts`) — a **fixed path in whatever project the rig is
788
+ running in**. In this repo that directory holds real development state, so a leg's notice content
789
+ depended on rules that had nothing to do with the leg: measured 2026-10-23, `heavy`'s recorded notice
790
+ carried `[dev2/001] (security) Never log raw tokens…` while the replayed notice carried none, the two
791
+ texts agreeing to character 346 and diverging there. Same build, same pins, same tapes — a different
792
+ repo.
793
+
794
+ The realm config key `knowledgeMirrorDir` (default `.dsh-knowledge`, unchanged for real installs) now
795
+ feeds that read, and `boot.ts` pins it to `var/e2e-home-<port>/knowledge`. The rule generalizes: any
796
+ path a boot reads to build model-visible text must be per-port or per-session, because a tape is a
797
+ claim about ONE world. `tests/unit/e2e-store-root.test.ts` keeps the pins honest, and
798
+ `tests/unit/preset-declaration.test.ts` refuses a config key that drifts out of the preset row.
799
+
800
+ ### Don't push while an acceptance run is the thing you're waiting on (#89, measured against myself)
801
+
802
+ `.github/workflows/ci.yml` sets `concurrency: ci-<ref>` with `cancel-in-progress: true`. Twice on
803
+ 2026-10-23 I pushed a small commit (a docs line, a corrected message) while the ~25-minute acceptance
804
+ chain was mid-flight, and GitHub cancelled the run that held the answer — then the new run restarted
805
+ from zero. The commit was never the problem; the wait was. Batch the push: while a long gate is
806
+ running, either finish what you're doing locally first or park the commit, and push once. Same rule in
807
+ the other direction: don't judge a fix by the run it cancelled — re-read which head each verdict belongs
808
+ to before believing a green.
809
+
810
+ ### Throwaway homes live outside the repository — `E2E_HOME_ROOT` (#89)
811
+
812
+ `e2eHomeRoot()` defaults to `os.tmpdir()/dsh-e2e`, and CI sets `E2E_HOME_ROOT=/tmp/dsh-e2e` explicitly
813
+ so the diagnostics it uploads are the ones it booted. Not tidiness: Node resolves `@deepseek-ai/*`
814
+ bundles by walking UP from the profile directory, so a home under `var/` reaches this repo's
815
+ `node_modules` (0.1.5-rc.2 by lockfile) and then the developer's `~/node_modules`. On the machine that
816
+ recorded the first re-anchored corpus, that second hop is a **linked dev build of the harness whose
817
+ version string equals the pin** — so the boot mounted tool guidance (goals, ralph, subagents, web
818
+ search, glob/grep, job output) the published tarball does not have.
819
+
820
+ Measured by moving nothing but the home's location:
821
+
822
+ | home root | system header composed by the SAME pinned binary | `suite` strict replay |
823
+ |---|---|---|
824
+ | `var/e2e-home-41731` | 7,276 chars | 4 passed, zero misses |
825
+ | `/tmp/dsh-e2e/e2e-home-41731` | 2,783 chars (CI's shape) | `commonPrefix=1`, `divergeAt=msg1` |
826
+
827
+ That is the whole "green on my laptop, impossible on CI" class in two rows of one table, and why
828
+ `tests/unit/e2e-host-binary.test.ts` now asserts hermeticity structurally: the home root must be
829
+ outside `ROOT`, and no ancestor of the home may contain `@deepseek-ai` packages — if one ever does, the
830
+ row names the path instead of agreeing silently. Pinning the *binary* (`resolveHostBinary`) was not
831
+ enough; the ancestor chain is part of the identity, and AGENTS' warning that these bundles' version
832
+ strings are not identity turns out to apply one directory level up from any lockfile.
833
+
834
+ Consequence, stated plainly: changing this changes what every boot composes, so the entire corpus was
835
+ re-distilled afterward. A rule that doesn't cost a re-record is probably not a reproducibility rule.
836
+
837
+ ## One build per corpus (#90, measured 2026-10-24)
838
+
839
+ A tape records what the **plugin** asked, not just what the model answered. So a corpus is only coherent
840
+ if every leg in it was distilled against one `lib/`. This was violated all day by well-meaning
841
+ incrementalism — re-recording whichever leg a fix touched — and the cost was invisible until a replay
842
+ missed on a *post-compaction* surface: a 4-message request with `commonPrefix=0` against a tape that held
843
+ the 6-message pre-compaction form. Record ran on a build whose last-resort deferral always abstained
844
+ (`resolveConfig` defaults `maxTokens` to `headroomTokens` = 65,536, so the coexistence ceiling was
845
+ underivable); replay ran on a build where it is derivable (11,880) and can fire. Both behaviors are
846
+ correct. The corpus spanning them is not.
847
+
848
+ Rules that follow, in the order they bite:
849
+
850
+ 1. **Two questions, two answers — `npm run corpus:check` (#156).** A rebuild changes the *bundle*; that alone does **not**
851
+ make the corpus stale, it makes the *stamp* stale: the check prints an `ADVISORY` and the replay still runs, then
852
+ `npm run corpus:stamp` re-records `BUILD_STAMP.json` (`lib` + `promptAnchor`) once the legs are green. What does
853
+ invalidate a corpus is the **prompt-byte anchor** moving — tool and command `description`s, dock `label`s, the preset
854
+ persona, and the realm keys whose values reach the model (`thresholdRatio`, `retain*`, `headroom*`, artifact/defer
855
+ tokens, …). Then `verdictFor` refuses with `REFUSED: model-facing bytes moved …` and **all** affected projects must be
856
+ re-distilled before anything is trusted, because a partial re-record yields a mixed corpus and matching happens across
857
+ the whole set. The rule in `AGENTS.md` — "re-record when PRODUCT prompt bytes change" — is now a computed quantity
858
+ rather than a judgement call; logging keys, store paths and `maxBytes` are deliberately outside the anchor, and a
859
+ manifest file that contributes no tokens fails its own row, so a silently-dead extraction rule cannot masquerade as
860
+ protection.
861
+ 2. **`PROVENANCE.md` per leg is necessary, not sufficient.** It records the host and regime; it cannot
862
+ say "this leg predates three plugin commits". The stamp is what says that, and #156's split is what stops the stamp from
863
+ crying wolf on every rebuild.
864
+ 3. **Test-only changes (`tests/e2e/**`) need no re-record** — normalization folds apply at load to both
865
+ sides. Product-behavior changes do. The distinction is exactly AGENTS' "re-record when PRODUCT prompt
866
+ bytes change", generalized: anything that changes what the plugin *sends*.
867
+ 4. If a green replay is ever obtained on a corpus whose legs have different build stamps, treat it as
868
+ unexplained rather than as evidence.
869
+
870
+ ## Harness-native acceptance (`E2E_PROVIDER=harness`)
871
+
872
+ The rig used to have exactly one model route: a tracked profile block pinning `local` at
873
+ `http://localhost:8080/v1`, because a GitHub runner has no credentials and can start a llama.cpp server. On
874
+ this machine that made acceptance a second-class citizen — the model was almost never up, live legs
875
+ `test.skip`-ed themselves (which reads like a pass), and re-distilling meant standing up an inference server
876
+ before you could look at UI.
877
+
878
+ `E2E_PROVIDER=harness` takes the route from **the profile this host is actually running** — read from
879
+ `$DSH_HOME/profiles/web/cordis.patch.yml` (override with `E2E_AMBIENT_PROFILE`) — and the credential from, in
880
+ order:
881
+
882
+ 1. **the parent environment**, `QWEN_TOKEN_PLAN_API_KEY` — the same way this harness instance got it. When you
883
+ run a leg from your own shell, this is the only step.
884
+ 2. `.dshdev-local/e2e.env`, untracked and gitignored, parsed **without executing it** (`readEnvFile` — values
885
+ are data, never a shell).
886
+ 3. otherwise the boot **refuses**, naming the remedy. No fallback to `local`, no silent skip, no guessed route.
887
+
888
+ What it deliberately keeps: the **pinned 32K stress window**. `harnessProviderPatch` rewrites
889
+ `contextWindow` rather than inheriting the endpoint's 196K, because the whole pressure/cache journey exists to
890
+ observe `context-overflow` — a hosted window would swallow a 33K prompt and the leg would measure a boot where
891
+ nothing is wrong.
892
+
893
+ Commands (all of them print one `[provider] …` line naming provider, model, window and where the key came
894
+ from, with the credential's length but never its value):
895
+
896
+ | command | mode | what it's for |
897
+ |---|---|---|
898
+ | `npm run e2e:harness:live` | live | a real look at the UI on the model the product runs against |
899
+ | `npm run e2e:harness:heal` | replay + fallback | tape hits replay, **misses go live and are appended** — a self-healing pass instead of a full re-record (the switch existed in code but was never surfaced) |
900
+ | `npm run corpus:heal -- --project=<leg>` | replay + fallback | one leg, when you know which one drifted |
901
+ | `npm run corpus:record -- --project=<leg>` | record | re-distill one leg after a deliberate product-prompt change |
902
+
903
+ Why this is more than convenience: the `#90 REASONING-BLIND` entry in the tape's blind list masks
904
+ `reasoning_content` shape differences. Tapes distilled through a different provider than the product uses will
905
+ always need masks; distilling through the product's own route removes a class of divergence instead of hiding
906
+ it. Stated as strong inference from the measurement, to be confirmed the first time a harness-native
907
+ re-distill stops needing that mask.
908
+
909
+ Note on what an agent shell can see: the harness **withholds** `QWEN_TOKEN_PLAN_API_KEY` from tool processes
910
+ (measured — the harness's own process env has it, the agent's child shell does not). So a distill or live run
911
+ from inside an agent session needs the `.dshdev-local/e2e.env` path, or the operator runs it. Reading the key
912
+ out of the parent process's environment is technically possible for the same user and is deliberately **not**
913
+ done: the withholding looks intentional, and reaching around it to move a credential into a tool shell would be
914
+ exactly the kind of quiet workaround this project documents itself out of.
915
+
916
+ ## The provider identity is part of the tape key
917
+
918
+ A tape is matched on the **request**, and the request's first message is the composed system header — which
919
+ states the provider and model. So a corpus distilled with one provider row cannot replay under another, and the
920
+ failure does not look like configuration: it looks like a corrupt tape, because matching dies at
921
+ `commonPrefix=0` and every leg then reports its own downstream symptom (no assistant message, no fork door, no
922
+ plan card, a post-compaction turn that never answers).
923
+
924
+ Measured the hard way on 2026-10-27: re-distilling the corpus with `E2E_PROVIDER=harness` while hosted CI
925
+ replays in the tracked-local shape produced `MISS tape=suite … tied=[every entry]` on CI and a green chain
926
+ locally — the same command, two different headers. The fix is placement, not normalization:
927
+
928
+ - **the host profile stays on the fixture provider** (`local/qwen3.8-flash-next`) in record and replay, so the
929
+ header is identical on every machine and in CI;
930
+ - **the real endpoint lives in the tape proxy** — `E2E_UPSTREAM` derived from the installed provider module and
931
+ `E2E_UPSTREAM_MODEL` translating the fixture name — so the model answering is the maintainer's while the
932
+ bytes being recorded are the rig's;
933
+ - `E2E_HARNESS_PROFILE=1` restores the profile rewrite for **live** runs, where there is no tape to stay
934
+ identical with and the point is genuinely to exercise the ambient provider.
935
+
936
+ Two consequences worth holding onto: the `reasoning`/`retryPolicy` fields of the synthesized row change product
937
+ behavior (`maxRetries: 0` versus the host default), so "same model, different row" is not a neutral edit; and
938
+ a re-distill must run the leg to **green** — a tape distilled by a failing journey is an incomplete capture, and
939
+ replaying it back as a miss blames the artifact for a run that never finished.
940
+
941
+ ## Two rig rules from the 2026-10-27 mount chase
942
+
943
+ **A boot must resolve what its profile declares, before the host starts.** The booted profile's preset names 28
944
+ `@deepseek-ai/*` plugin packages, and none of them is a dependency of this repository. On a development host they
945
+ resolve out of `~/node_modules`; on a CI runner they resolve nowhere, `pruneUnresolvableRows()` drops them
946
+ without complaint, and the session composes a 7-tool surface where a laptop gets 34. Because the system header
947
+ is message 0 of every request — the thing tapes match on — that asymmetry makes a locally recorded corpus
948
+ unreplayable in CI, which is what every `commonPrefix=0` on this branch actually was. `linkPinnedToolPackages()`
949
+ now symlinks the pinned install's packages into `node_modules/@deepseek-ai` **before** `spawn`, skipping anything
950
+ the repo already provides (never replace a hoisted dependency with a symlink and risk the build under test), and
951
+ `var/e2e-tool-links-<port>.log` records `rows/linked/skippedAlreadyPresent/missingFromPin` for every boot. The
952
+ ordering is not ceremonial: linking after the URL line was measured to change nothing, because the host has
953
+ already mounted by then.
954
+
955
+ **Normalize, THEN diff.** Comparing two machines' RAW header dumps showed an 18-character gap — exactly the
956
+ length of the two checkout paths — and produced a duplicate blind for a fold that already existed
957
+ (`<DSH-CHECKOUT>`, `model-proxy.ts:159`). The #89 ordering row caught the collision the moment it shipped; the
958
+ procedure error was mine and is the one to remember: run the text through `substituteAll` before concluding a
959
+ fold is missing. One predicate per semantic decision, and a fold that already exists is not a missing fold.
960
+
961
+ ## A recorded journey must not depend on when the host compacts
962
+
963
+ **Symptom that gives it away:** the acceptance run misses a *different set of legs* each time, while message 0 keeps
964
+ matching. That is not an unblinded identity token and not a matcher bug. It means the two machines ran the same
965
+ journey and both compacted — at different moments. A checkpoint rewrites the surface, so the same assistant turn
966
+ lands at message index 6 here and index 2 there, and the surface is the tape key.
967
+
968
+ **The rule:** a leg that is NOT testing the machinery pins `noAuxModel`. It is a rig-side alias, translated at
969
+ profile-write time into the realm keys the engine really reads — `auto: false` + `elicitedPlot: false`,
970
+ `tests/e2e/rig-preset-pin.ts:79`, declared at `src/engine.ts:287` — so the request sequence in the tape is the whole
971
+ story regardless of how large a runner's header happens to be.
972
+
973
+ **The exceptions are measured, not preferred:** `heavy`, `suite`, `arrival` and `fanout` stay unpinned, because
974
+ pinning them deletes the behavior under test — a pinned `suite` never renders its fork door, a pinned `arrival`
975
+ never gets its stub pair, a pinned `fanout` loses parent synthesis, and `heavy`'s journey *is* the pressure one. Same
976
+ family as setting the arrival floor below the chunk size and calling the passing row a result. The drift row
977
+ (`only the pressure journeys leave model-involving machinery enabled`, `tests/unit/rig-pins.test.ts`) names offenders
978
+ in its failure text and asserts those four stay unpinned.
979
+
980
+ **And the method trap I walked into, recorded so it costs the next reader nothing:** I searched `src/` for the string
981
+ `noAuxModel`, found none, and concluded the switch was decorative — then shipped a "fix" folding a check into
982
+ `overTrigger`, a local that is computed and dropped, plus a guard test asserting the plugin must contain the alias's
983
+ literal name. Absence of a NAME in a source directory is not absence of an EFFECT: read what the rig writes into the
984
+ profile it boots, not what it calls the field on the way in.
985
+
986
+ **Why the transcript has to travel with the artifact:** this took three runs to see because the workspace-only
987
+ artifact never contained the booted home's session logs. `scripts/chain-replay.sh` now copies each leg's
988
+ `session.v4.jsonl.zstd` files and registry into `var/e2e-session-logs/<leg>-<port>` (six files, under 4 MB each) so
989
+ the runner's own history sits next to the tape diffs and the misses. When a miss claims "the conversation had
990
+ already taken a different shape", the answer is in that transcript — read it, don't theorize about it.