@treeseed/dsh-chapters 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (119) hide show
  1. package/LICENSE +14 -0
  2. package/README.md +644 -0
  3. package/cordis.patch.yml +88 -0
  4. package/docs/architecture.md +460 -0
  5. package/docs/contract.md +375 -0
  6. package/docs/development.md +437 -0
  7. package/docs/host-compaction-seam.md +117 -0
  8. package/docs/knowledge-repo.md +730 -0
  9. package/docs/provider.md +98 -0
  10. package/docs/verify.md +200 -0
  11. package/lib/archive.d.ts +88 -0
  12. package/lib/archive.js +114 -0
  13. package/lib/archive.js.map +1 -0
  14. package/lib/arrival.d.ts +23 -0
  15. package/lib/arrival.js +86 -0
  16. package/lib/arrival.js.map +1 -0
  17. package/lib/artifact-query.d.ts +53 -0
  18. package/lib/artifact-query.js +130 -0
  19. package/lib/artifact-query.js.map +1 -0
  20. package/lib/client.js +234 -0
  21. package/lib/client.js.map +1 -0
  22. package/lib/commands.d.ts +28 -0
  23. package/lib/commands.js +305 -0
  24. package/lib/commands.js.map +1 -0
  25. package/lib/compose.d.ts +30 -0
  26. package/lib/compose.js +131 -0
  27. package/lib/compose.js.map +1 -0
  28. package/lib/continue-core.d.ts +153 -0
  29. package/lib/continue-core.js +252 -0
  30. package/lib/continue-core.js.map +1 -0
  31. package/lib/engine-core.d.ts +219 -0
  32. package/lib/engine-core.js +405 -0
  33. package/lib/engine-core.js.map +1 -0
  34. package/lib/engine.d.ts +211 -0
  35. package/lib/engine.js +571 -0
  36. package/lib/engine.js.map +1 -0
  37. package/lib/enrich-queue.d.ts +66 -0
  38. package/lib/enrich-queue.js +100 -0
  39. package/lib/enrich-queue.js.map +1 -0
  40. package/lib/enrich-store.d.ts +31 -0
  41. package/lib/enrich-store.js +103 -0
  42. package/lib/enrich-store.js.map +1 -0
  43. package/lib/enrich-wire.d.ts +73 -0
  44. package/lib/enrich-wire.js +209 -0
  45. package/lib/enrich-wire.js.map +1 -0
  46. package/lib/enrich.d.ts +73 -0
  47. package/lib/enrich.js +150 -0
  48. package/lib/enrich.js.map +1 -0
  49. package/lib/gitops.d.ts +107 -0
  50. package/lib/gitops.js +539 -0
  51. package/lib/gitops.js.map +1 -0
  52. package/lib/index.d.ts +81 -0
  53. package/lib/index.js +305 -0
  54. package/lib/index.js.map +1 -0
  55. package/lib/indexing.d.ts +85 -0
  56. package/lib/indexing.js +212 -0
  57. package/lib/indexing.js.map +1 -0
  58. package/lib/injections.d.ts +66 -0
  59. package/lib/injections.js +83 -0
  60. package/lib/injections.js.map +1 -0
  61. package/lib/notice.d.ts +60 -0
  62. package/lib/notice.js +84 -0
  63. package/lib/notice.js.map +1 -0
  64. package/lib/provider.d.ts +44 -0
  65. package/lib/provider.js +66 -0
  66. package/lib/provider.js.map +1 -0
  67. package/lib/redact.d.ts +24 -0
  68. package/lib/redact.js +85 -0
  69. package/lib/redact.js.map +1 -0
  70. package/lib/registry.d.ts +120 -0
  71. package/lib/registry.js +130 -0
  72. package/lib/registry.js.map +1 -0
  73. package/lib/render.d.ts +40 -0
  74. package/lib/render.js +280 -0
  75. package/lib/render.js.map +1 -0
  76. package/lib/repo.d.ts +78 -0
  77. package/lib/repo.js +184 -0
  78. package/lib/repo.js.map +1 -0
  79. package/lib/rules-commands.d.ts +24 -0
  80. package/lib/rules-commands.js +138 -0
  81. package/lib/rules-commands.js.map +1 -0
  82. package/lib/rules.d.ts +90 -0
  83. package/lib/rules.js +185 -0
  84. package/lib/rules.js.map +1 -0
  85. package/lib/search.d.ts +48 -0
  86. package/lib/search.js +152 -0
  87. package/lib/search.js.map +1 -0
  88. package/lib/signature.d.ts +28 -0
  89. package/lib/signature.js +149 -0
  90. package/lib/signature.js.map +1 -0
  91. package/lib/store.d.ts +130 -0
  92. package/lib/store.js +235 -0
  93. package/lib/store.js.map +1 -0
  94. package/lib/sync.d.ts +183 -0
  95. package/lib/sync.js +719 -0
  96. package/lib/sync.js.map +1 -0
  97. package/lib/tools.d.ts +84 -0
  98. package/lib/tools.js +581 -0
  99. package/lib/tools.js.map +1 -0
  100. package/lib/treedx/client.d.ts +57 -0
  101. package/lib/treedx/client.js +82 -0
  102. package/lib/treedx/client.js.map +1 -0
  103. package/lib/treedx/provider.d.ts +39 -0
  104. package/lib/treedx/provider.js +479 -0
  105. package/lib/treedx/provider.js.map +1 -0
  106. package/lib/treedx/state.d.ts +27 -0
  107. package/lib/treedx/state.js +65 -0
  108. package/lib/treedx/state.js.map +1 -0
  109. package/lib/types.d.ts +100 -0
  110. package/lib/types.js +19 -0
  111. package/lib/types.js.map +1 -0
  112. package/lib/vocabulary.d.ts +41 -0
  113. package/lib/vocabulary.js +118 -0
  114. package/lib/vocabulary.js.map +1 -0
  115. package/package.json +132 -0
  116. package/presets/chapters/agent.cordis.yml +272 -0
  117. package/presets/chapters/preset.yml +3 -0
  118. package/presets/probe-roots/agent.cordis.yml +11 -0
  119. package/presets/probe-roots/preset.yml +3 -0
@@ -0,0 +1,88 @@
1
+ # dsh-chapters bundle patch — ACTIVE: package.json declares this file as the
2
+ # plugin's composition patch, applied when a profile adds the plugin. The YAML
3
+ # below the comment block is functional; everything in this comment is the
4
+ # complete configuration reference (kept in sync with src/index.ts and
5
+ # src/engine.ts — verified 2026-09-20 against both schemas).
6
+ # Scratch-profile install: DSH_HOME=$PWD/.dshdev dsh plugin --profile web add "link:$(pwd)"
7
+ #
8
+ # Two planes, two surfaces:
9
+ # HOST PLANE — the `config:` block below (defaults shown commented; set any
10
+ # under your profile's plugin entry).
11
+ # REALM PLANE — the compaction ENGINE row in the preset
12
+ # (presets/chapters/agent.cordis.yml); shared names behave per
13
+ # plane. Design record: docs/knowledge-repo.md §12.
14
+ #
15
+ # config:
16
+ # # ---- identity & install --------------------------------------------
17
+ # installChaptersPreset: true # copy presets/chapters into the home's
18
+ # # .agent-presets at boot; a user-edited
19
+ # # preset is never clobbered (write-once)
20
+ # fallbackPreset: 'chapters' # preset mounted for children when the
21
+ # # caller's own preset cannot be resolved
22
+ # harnessId: '<hostname>' # per-machine identity: git commit author,
23
+ # # token scoping, edits/<harness>/ curation
24
+ # # partition, rule-approval authorship
25
+ #
26
+ # # ---- stores ---------------------------------------------------------
27
+ # artifactStoreRoot: '.dsh-chapters' # workspace store: chapters, artifacts,
28
+ # # rules; must be reachable by `read`
29
+ #
30
+ # # ---- chapters & composition ----------------------------------------
31
+ # chapterTokenTarget: 8000 # soft per-chapter size target
32
+ # chapterLimit: 8000 # hard split threshold
33
+ # mergeThreshold: 0.3 # τ for merging turns into one chapter at
34
+ # # archive time. Composition is archive-time
35
+ # # only: tune for future archives.
36
+ #
37
+ # # ---- context economy -------------------------------------------------
38
+ # toolResultDeferFloorTokens: 200 # archive-time inline floor
39
+ # continuationBudgetRatio: 0.25 # notice+note allowance as share of
40
+ # # the window REMAINING AFTER the
41
+ # # header — never of the whole window.
42
+ # # Over budget = refuse with numbers.
43
+ # searchDefaultMaxTokens: 400 # chapters_search pack size
44
+ #
45
+ # # ---- knowledge sync --------------------------------------------------
46
+ # knowledgeRemote: '' # repo URL; '' = link via /chapters-link
47
+ # # (view | <path> | <url> <token>)
48
+ # projectKeyOverride: '' # wins over the remote-derived project key
49
+ # syncDebounceMs: 30000 # coalesce window for post-archive pushes
50
+ #
51
+ # # ---- P2 enrichment ---------------------------------------------------
52
+ # # The host plane is the COMMAND SURFACE (manual drains only); the realm
53
+ # # engine row owns the auto triggers — enrichmentTrigger exists on the row.
54
+ # enrichmentEnabled: true # kill switch: false = pure no-op everywhere,
55
+ # # corpus stays fully working (deterministic)
56
+ # enrichmentModel: '' # 'provider/model'; '' = conversation route.
57
+ # # /chapters-enrich model <x|clear> overrides
58
+ # enrichmentIdleMs: 60000 # idle trigger silence window (row value)
59
+ # enrichmentBatchCap: 5 # chapters per drain
60
+ #
61
+ # # ---- P2 vocabulary -----------------------------------------------------
62
+ # vocabApply: false # SHADOW by default: candidates are reported
63
+ # # in every sync status, nothing written;
64
+ # # true = git-visible topic-alias curation
65
+ # vocabCoMin: 3 # co-occurrences across chapters to trust
66
+ # vocabOverlapMin: 0.5 # label token-set Jaccard threshold
67
+ #
68
+ # # ---- P3 rules ----------------------------------------------------------
69
+ # coreRulesBudgetTokens: 1200 # cap on the notice's CORE RULES block;
70
+ # # overflow refuses with per-rule numbers —
71
+ # # the notice NEVER clips
72
+ # rulesCoreBonus: 0.15 # rank bonus for rules effective-core on
73
+ # # THIS machine (approval = per-machine
74
+ # # curation fact, record §15)
75
+ #
76
+ # Realm-row-only keys (in the preset's engine row, not this patch):
77
+ # thresholdRatio, retainRatio, retainTokens, maxTokens, compactionRetries,
78
+ # maxOverflowRetries, auto, summarizationProvider/summarizationModel (legacy
79
+ # override), enrichmentTrigger (afterPush|idle|both|manual),
80
+ # elicitedPlot (true — one bounded call to mint a missing PLOT note),
81
+ # toolResultArtifactTokens (8000 — arrival-stub floor; keep ABOVE your chunk
82
+ # size: a floor below it stubs chunks and compaction silently stops being
83
+ # needed, measured), plus shared names above honored per plane.
84
+ # ---------------------------------------------------------------------------
85
+
86
+ - insert:
87
+ - id: '@treeseed/dsh-chapters'
88
+ name: '@treeseed/dsh-chapters'
@@ -0,0 +1,460 @@
1
+ # Architecture and Design Decisions
2
+
3
+ Decisions with their reasoning, so they can be revisited knowingly rather than rediscovered.
4
+ API mechanics and file:line citations live in `docs/contract.md`; this file is the *why*.
5
+
6
+ ## Naming
7
+
8
+ **Decided: `dsh-chapters`** — package `dsh-chapters`, tools `chapters_segment` and `chapters_continue`,
9
+ storage domain `dsh_chapters`, store root `.dsh-chapters/`. Recorded with its reasoning because the
10
+ reasoning constrains future renames, not because the question is open.
11
+
12
+ Why the two most available-sounding words were rejected: DSH's "fork" means *copy the parent's history
13
+ into the child*, which is the one thing this plugin must not do. And "compact"/"summarize" implies the
14
+ lossy in-place editing this replaces.
15
+
16
+ | Candidate | Tools | Why taken or left |
17
+ |---|---|---|
18
+ | **`dsh-chapters`** ✓ | `chapters_segment`, `chapters_continue` | Names the durable artifact, makes no mechanism claim, greppable, and `continue` is exactly what happens: the conversation continues in a new session. Short enough for a plugin list. |
19
+ | `dsh-chapter-continue` | `segment_chapters`, `continue_with_chapters` | Verb-forward and self-describing to a model scanning a tool list; clunkier as a package. |
20
+ | `dsh-tome` | `tome_index`, `tome_continue` | Memorable and literally apt — a tome is a volume of chapters with a table of contents, which *is* the mechanic. But opaque to anyone who hasn't read the README. |
21
+
22
+ Also rejected: `dsh-archive` (sounds read-only, and the plugin's job is to keep working).
23
+
24
+ **One leftover: the repository directory.** The checkout is still `~/Projects/dsh-chapter-fork`. The
25
+ folder name is cosmetic — installs use `link:$(pwd)`, and the bundle id comes from `package.json` and
26
+ `cordis.patch.yml`, both of which say `dsh-chapters`. Rename it out-of-band with `mv` rather than from
27
+ inside a session whose working directory *is* that folder, which would strand the tree. Keeping the
28
+ identifiers in sync is what actually matters, and they are in sync.
29
+
30
+ ## File Format: Markdown with Frontmatter, Not XML
31
+
32
+ Asked and answered: chapters stay **Markdown**, with structure in YAML frontmatter.
33
+
34
+ The reasoning against XML, strongest first:
35
+
36
+ 1. **The consumer is a model reading text, not a parser.** Every markup token is a context token, in a
37
+ plugin whose entire purpose is bounding context. `<turn role="assistant" seq="184">` costs several
38
+ times what `**Assistant:**` does, repeated across a whole chapter.
39
+ 2. **Structure already has two better homes.** The session log *is* the lossless machine-readable
40
+ record, keyed by seq, and the registry stores ranges and hashes. Duplicating turn structure into the
41
+ readable artifact creates a third source of truth that can silently drift from the other two.
42
+ 3. **`.md` gets rendered; `.xml` gets downloaded.** The harness's own markdown component renders `.md`
43
+ inline in the web UI, and the `read` tool, editors, and git diffs are all Markdown-shaped. XML files
44
+ become write-only artifacts nobody looks at.
45
+ 4. **Markdown is the established convention for exactly this shape.** DSH's skill loader parses
46
+ YAML-frontmatter-plus-Markdown-body files — the same idiom, already understood by the tooling.
47
+
48
+ What we *do* take from the XML instinct, because it was a real concern:
49
+
50
+ - **Frontmatter carries the parseable structure** — `chapter`, `session`, `root`, `seqRange`, `sha256`,
51
+ `tokens`, `artifacts`. Queryable with a YAML parser, invisible to the reading experience.
52
+ - **There is one index, not three.** The TOC lists the chapters for reading; the storage domain already
53
+ holds the machine-readable enumeration. A separate `MANIFEST.md` was proposed and **rejected as
54
+ redundant** — a regenerated duplicate that can drift, whose growth becomes a context cost when read
55
+ whole, and the sole exception to append-only. Dropping it removes all three problems at once.
56
+ - **Fence collision is a renderer bug, not a format problem.** Conversation text containing ```` ``` ````
57
+ must be escaped by choosing a longer fence per block. This is the actual failure mode an XML format
58
+ would have prevented, and it is solved in ~10 lines of the renderer. Get it wrong and chapters
59
+ silently truncate at an embedded fence.
60
+
61
+ **The one case that would reopen this:** if a future feature needs to *rewrite* chapter structure
62
+ programmatically — re-segmenting an existing chapter by turn boundaries without a session log — then a
63
+ losslessly round-trippable format earns its tokens. Rendering is currently one-way from log to file, so
64
+ it doesn't. Revisit if chapter *editing* ever enters scope.
65
+
66
+ ## Chapter Rendering and the Artifact Store
67
+
68
+ The model supplies ranges; the plugin renders. Within a range:
69
+
70
+ | Event content | Handling |
71
+ |---|---|
72
+ | user / assistant text | inline, verbatim, attributed |
73
+ | tool **invocation** (name + arguments) | always inline — this is the input side worth re-reading |
74
+ | tool **result** judged **not worth keeping** | deferred to `artifacts/`, replaced by a reference line |
75
+ | tool **result** judged worth keeping, ≤ floor | inline, verbatim, no model input needed |
76
+ | tool **result** judged worth keeping, > floor | inline by model override **and** still written to `artifacts/` |
77
+ | images / attachments | deferred to `artifacts/` by path (Markdown cannot carry them) |
78
+ | surface ops, projections, lifecycle events | omitted from the body; their seq ranges are recorded in frontmatter |
79
+
80
+ ### Who decides: the model, not a threshold
81
+
82
+ The keep/defer call is **semantic**, so it belongs to the agent that lived the conversation. "This diff
83
+ matters; this npm install log does not" is not inferable from size — a 4-line compiler error is load-bearing
84
+ and a 400-line `read` of a config file may be pure noise. A size threshold gets this backwards exactly often
85
+ enough to matter. *(Amended 2026-09-19 below: one band — results too large to ever be reasoning context —
86
+ is artifacted at arrival without judgment; this section's rule governs everything that can genuinely fit.)*
87
+
88
+ Division of labour:
89
+
90
+ - **Plugin enumerates.** It has the log, so `chapters_segment` returns, per chapter, the candidate tool
91
+ results with `seq`, tool name, and size. The model never guesses at byte counts.
92
+ - **Default is defer** for anything above `toolResultDeferFloorTokens`; trivial results stay inline without
93
+ being itemized, so the narrative reads naturally and the exception list stays short.
94
+ - **Model submits exceptions only**: `toolResultOverrides: [{seq, inline}]`. Usually empty or one entry, so
95
+ the judgment costs almost no output tokens — which is why this is affordable where "the model writes the
96
+ chapter bodies" was not.
97
+ - **Invocations and arguments always stay inline.** Plugin rule, not a choice; it is the "what was
98
+ attempted" half of the record and the thing a successor most needs.
99
+ - **The artifact is written either way.** Inlining is a convenience copy *alongside* its reference. So a
100
+ wrong call is a context-economy miss, never data loss — that asymmetry is what makes delegating this
101
+ judgment to the model safe.
102
+
103
+ The floor is a formatting device, not the policy, and its value is a Phase 1 calibration guess. Oversized
104
+ output is caught by the continuation budget and the chapter split signal, both of which **report or refuse
105
+ rather than clip**.
106
+
107
+ Reference line format — stable, greppable, and `read`-able:
108
+
109
+ ```
110
+ ↳ tool: bash `npm run migrate -- --dry-run`
111
+ result: 41.2 KB, sha256 8f3a1c… → .dsh-chapters/<root>/artifacts/8f/8f3a1c9d….txt
112
+ ```
113
+
114
+ Artifacts are **content-addressed** at `artifacts/<first-2-hex>/<sha256>.txt`, which means identical
115
+ results across an entire subtree — the same 200-line file read ten times — are stored once. That
116
+ deduplication is a large part of why this design stays cheap over long chains.
117
+
118
+ ### This changes the losslessness claim, and the claim must change with it
119
+
120
+ Chapters are no longer self-contained. The honest formulation is:
121
+
122
+ > **Every byte of the conversation remains retrievable** — inline in its chapter, or by an exact path
123
+ > from it. Nothing is paraphrased, summarized, or dropped.
124
+
125
+ Do **not** write "the chapter contains the full history" in a TOC preamble or a README. The distinction
126
+ matters operationally too: following a reference costs the model a turn and real tokens, so the default
127
+ rendering must keep the 99% case (who said what, and what was attempted) inline. That is precisely the
128
+ trade the user's judgment called for: long dumps are the least valuable thing to re-read, and the most
129
+ expensive to carry.
130
+
131
+ If a chapter still exceeds `chapterTokenTarget` after deferral, that is the *split* signal — not a
132
+ license to truncate. Truncation is refused by invariant 4 and by "never truncate to fit."
133
+
134
+ ### Amendment (user-directed, 2026-09-19, APPROVED, **landed**): arrival-time artifacting for the impossible band
135
+
136
+ > Shipped with phases 2-4 of the artifacting plan: `src/arrival.ts` writes the stub, `chapters_artifact`
137
+ > is in `src/tools.ts`. Verified by unit rows plus browser journey **E7** in
138
+ > [verify.md](verify.md) — one `read` of a 373 KB book yields exactly one arrival stub at the tail, zero
139
+ > compaction events at the production 0.9 threshold, the blob fingerprint absent from every `request/*`
140
+ > event, and `cacheReadTokens` monotonic across the turn (the prefix was never rewritten).
141
+
142
+ The section above lets the model decide what to defer **at archive time** — a judgment about material
143
+ that fit in the window to begin with. This amendment carves off the band where there is no judgment to
144
+ make: a tool result above a hard threshold (config key `toolResultArtifactTokens`, default 8000,
145
+ floored at 256) **never enters the surface at all**. A 300-page book does not fit a 32K desk; loading it
146
+ to "see what happens" is not deference to the model's semantics, it is an emergency waiting to happen.
147
+
148
+ Mechanics:
149
+
150
+ - The result is written to the artifact store (content-addressed, same path scheme as archival
151
+ deferral — dedup applies) and the surface receives a **reference stub** instead: tool, size, sha,
152
+ path, plus the query instruction below. A few hundred tokens, appended at the tail — prefix cache
153
+ untouched. (This is distinct from the pruner's mid-history rewrite, whose re-prefill cliff on
154
+ llama.cpp we measured r28; arrival-time interception never rewrites the past.)
155
+ - A new read-only tool, **`chapters_artifact`**, queries artifacts *inside* their file: `search`
156
+ (terms/regex → matching blocks with line numbers, bounded by a maxTokens pack), `read`
157
+ (offset/limit — the model can still pull an exact section when its judgment says so), and `toc`
158
+ (for structured documents: markdown headings parsed to a line-numbered contents list — a 300-page
159
+ paper arrives with its own table of contents in hand).
160
+ - Consequence on the failure inventory: the **retention paradox** (a newest node too large to file,
161
+ too new to keep ⇒ bounded refusal) becomes structurally unreachable for tool results — the monster
162
+ is never on the desk to be caught in retention. This is the "handle all abortions" directive
163
+ realized at the gate rather than at the rescue.
164
+ - External validation: this is the Recursive Language Model result (Zhang, Kraska & Khattab,
165
+ arXiv:2512.24601) — arbitrarily long material as *external environment* the agent inspects
166
+ programmatically. Their mechanism is a Python REPL over the prompt; ours is a file + query tool
167
+ + parsed TOC, which delivers the same decomposition without code execution or a sandbox. Their
168
+ reported compaction-baseline gap (median 26%) is the quality argument; the SRLM follow-up
169
+ (arXiv:2603.15653) supplies the policy guard: recursion that splits what already fits **hurts** —
170
+ hence the hard band boundary, not a general split-everything stance, and bounded pulls remain the
171
+ model's choice.
172
+
173
+ The archive-time model-judgment machinery above is unchanged for results below the floor.
174
+
175
+ **Verification (2026-09-19):** `tests/e2e/artifact-arrival.spec.ts` (green 9.3m and
176
+ 4.1m runs) — a 373,861-byte book fetched in one read produced exactly the arrival
177
+ prune+stub pair, ZERO blob occurrences in any `request/*` event (never prefilled),
178
+ 5–15 `chapters_artifact` calls recovering facts from inside it, and a cache table
179
+ (FINDINGS) showing `cacheReadTokens` monotone across the stub boundary — the tail
180
+ rewrite costs the prefix nothing, unlike the pruner's mid-history cliff. Row config
181
+ `toolResultArtifactTokens` is pinned per e2e project (500 for arrival, default
182
+ 8000 elsewhere).
183
+
184
+ ### Amendment (user-directed, 2026-09-19, APPROVED, **landed**): the plot note across in-place compaction
185
+
186
+ > Shipped: the forward path is the preset persona suffix (the agent is told to end substantive turns
187
+ > with a `PLOT:` line) and the engine copies the latest such note verbatim into the replacement above
188
+ > the index cards; the elicited fallback is one bounded call gated by `elicitedPlot` (default `true`) —
189
+ > tail-heavy excerpt capped at ~8K chars in, `maxTokens: 220`, note clamped to 900 chars, any failure
190
+ > degrading to `null` (no note, never a crash). Both halves pinned in unit; the forward path proven live
191
+ > in verify **E8** (a suite-B run carried `PLOT:` blocks through seven compactions). The 60-word figure
192
+ > below supersedes the ~120 originally proposed.
193
+
194
+ Heavy mid-turn tidying measurably erodes *plan* continuity (FINDINGS, oversized-turn runs: model
195
+ finished the reading but abandoned the wrap-up). The TOC preserves what was **done**; this adds what
196
+ is **going on**, authored while the model still holds the thread:
197
+
198
+ - **Forward-maintained (preferred):** preset discipline — at each turn's end the agent leaves a
199
+ compact *plot note* (objective, current hypothesis, next step; ~40–80 words, no tools). Compaction
200
+ copies the latest note verbatim into the replacement, above the index cards. Cost: a little output
201
+ per turn; zero extra prefill at compaction, which is when cost bites.
202
+ - **Elicited (fallback, bounded):** if no plot note exists at compaction time, one auxiliary
203
+ completion over the **TOC + retained tail only** (tail-heavy excerpt, ~8K chars in, ≤60 words out,
204
+ `maxTokens: 220`; key `elicitedPlot`, default `true`) asks the model to state the plan. This is
205
+ deliberately not a re-read of the archived span — it costs a fraction of what a naive summary call
206
+ would, and only when the free path failed.
207
+ - Invariant check: the model authors *state*, never archive text (invariant 4 intact); the note rides
208
+ the replacement event append-only; every byte of the conversation still lives in chapters.
209
+
210
+ ## Budgets and the Reserve Margin
211
+
212
+ Two distinct ceilings. Conflating them caused the original design's silent-failure mode.
213
+
214
+ **Continuation budget** — the *newly added* context of a child must be small and predictable, as a share
215
+ of the space left after the fixed header:
216
+
217
+ ```
218
+ tocTokens + handoffNoteTokens ≤ continuationBudgetRatio × (window − systemPromptTokens)
219
+ ```
220
+
221
+ **Chapter target** — per-file size, governing rendering and splitting. Not a hard error.
222
+
223
+ Deriving the budget from the **remaining** window rather than the whole window is not a nitpick. On a 32K
224
+ model with a ~9K system prompt and tool schemas, a 25%-of-window rule gives an 8K ceiling that the TOC and
225
+ note may blow through before any work happens, and the plugin would refuse every continuation on exactly
226
+ the small-context hardware it targets. The header is a fixed cost to subtract, not a competitor for the
227
+ index. Measure it once per generation and cache it.
228
+
229
+ The budget is enforced as a **preflight**: measure, then refuse with concrete numbers
230
+ (`TOC 4,120 + note 2,800 = 6,920 > budget 5,750 (0.25 × (32,000 − 9,000))`) *before* creating any
231
+ session. Never truncate to fit — a clipped handoff note is the lossy behaviour this plugin exists to
232
+ avoid, and a silent clip is exactly the bug that takes a month to notice.
233
+
234
+ There is deliberately **no cap on the handoff note.** The budget check subsumes it: a model that dumps the
235
+ whole conversation into the note fails preflight and is told by how much. The note should be state of play
236
+ — decisions held, the next step, anything in flight — and the tool description should say so rather than a
237
+ numeric limit doing the talking.
238
+
239
+ This margin is also what automatic triggering needs, which is why the phases were reordered: pressure
240
+ measurement, the preflight, and a *suggestion* at `turn/end` all land in Phase 2; only autonomous
241
+ action stays late.
242
+
243
+ ### The native branch button: cosmetic hide (user-approved 2026-09-17)
244
+
245
+ The message-row branch glyph is host chrome, not a slot entry. The chosen lever is
246
+ **display:none on its accessible label, scoped to action-row buttons** (en+zh spellings);
247
+ the sidebar fork entry stays. If a harness update rewords the label, the native button
248
+ reappears beside ours — accepted deliberately: a visible, temporary duplicate beats
249
+ behavioral monkey-patching of the client's fork call (the dsh-session-fork route, rejected
250
+ for its version-drift surface). Do not "upgrade" the hide into an intercept without asking.
251
+
252
+ ## Ancestry: A DAG We Flatten For The Model
253
+
254
+ History becomes a tree the moment someone continues an older ancestor twice. The worry that agents
255
+ won't traverse a DAG is correct, and the answer is that **they never should have to.**
256
+
257
+ Three rules:
258
+
259
+ 1. **The plugin walks; the model reads a list.** At continuation time the plugin walks the ancestor path
260
+ in the registry and emits a flat, chronologically ordered, numbered TOC. Reading order *is* causal
261
+ order. Depth 10 or mid-branch, the model sees a numbered index and a `read` path — no graph, no
262
+ inference, no traversal.
263
+ 2. **Back-links are printed, not implied.** The TOC footer names the root session and the parent session,
264
+ so the trace is visible without being reconstructed.
265
+ 3. **The TOC is the complete index for the current line of work.** Every chapter on the ancestor path is
266
+ listed with its summary, so ordinary follow-up needs no traversal at all.
267
+
268
+ Cross-**branch** recall — "what did the other attempt try?" — is the only case the linear TOC cannot serve,
269
+ and it is rare. It is answered by the registry (which holds the whole tree) behind a future read-only
270
+ command, never by a per-fork index file. Carrying it in every session's prefix to serve an occasional
271
+ question would tax all work for one.
272
+
273
+ Two structural consequences:
274
+
275
+ - **Chapters are keyed by creating session**, not by a linear chain id. Siblings branching from one
276
+ ancestor get different, correct TOCs and cannot collide on numbering.
277
+ - **Append-only has no exceptions.** Nothing derived is written back into the store; the registry is the
278
+ only mutable state, and it is a domain table designed for it.
279
+
280
+ ## Storage
281
+
282
+ Two stores because they have opposite requirements.
283
+
284
+ **Chapters and artifacts: workspace files.** The `read` tool must reach them, and it runs under the
285
+ session's file sandbox (`workspace-write` permits only the workspace tree). The original
286
+ `~/.dsh/chapters` design fails asymmetrically: writes to `$HOME` may succeed while `read` cannot reach
287
+ the path, so every TOC bullet is silently dead and the plugin looks like it worked. `ctx.fs` is also
288
+ **optional**, so its absence must be a clean refusal.
289
+
290
+ Layout:
291
+
292
+ ```
293
+ <store-root>/ # default .dsh-chapters/
294
+ <rootSession>/
295
+ chapters/001-project-setup.md # append-only, never renumbered
296
+ artifacts/8f/8f3a1c9d….txt # content-addressed, deduplicated
297
+ ```
298
+
299
+ Add `<store-root>/` to `.gitignore` on first write and say so — polluting a user's tree silently is its
300
+ own small bug.
301
+
302
+ **Registry: a DSH storage domain.** Ancestry, numbering, hashes, ranges, bullet lists. Sanctioned
303
+ durability, survives restart, and gives the append-only counter a real allocation home instead of a
304
+ directory scan racing a restart.
305
+
306
+ Trade-off to keep honest: a workspace-relative root makes a chain **cwd-scoped**. Crossing workspaces
307
+ means an absolute configured root, at which point `read` reachability must be re-verified — do not assume
308
+ an absolute path inside one workspace stays reachable in another.
309
+
310
+ ## Atomicity
311
+
312
+ Writing files, allocating numbers, creating a session, and committing registry state is not
313
+ transactional. Order them so the worst failure is an orphan, never a lie:
314
+
315
+ 1. **Reserve** numbering, chapter ids, and slug assignments in the registry *before* writing files.
316
+ 2. Render and write chapter files and artifacts; verify each path resolves.
317
+ 3. Create the child session with the TOC notice.
318
+ 5. **Only then** commit the registry entry linking parent → child — this is the commit point that makes
319
+ the continuation visible.
320
+
321
+ Retry must be **idempotent**: key the attempt on `sourceSession + archiveCeiling`, so a second call reuses
322
+ reserved numbers and existing files rather than duplicating chapters. Failure before step 5 leaves
323
+ unreferenced files, which is recoverable garbage; failure after step 4 without step 5 leaves a child
324
+ citing chapters that don't exist, which is not. Hence the ordering.
325
+
326
+ Concurrency is real, not theoretical: parallel branch work is a first-class pattern in this ecosystem, so
327
+ two agents can continue from one ancestor at the same time. A directory-scan counter races; registry
328
+ allocation does not.
329
+
330
+ ## Integrity
331
+
332
+ Chapter files are plain Markdown in a writable tree whose content is replayed into a child as an
333
+ authoritative `user/message` — the model's own claimed memory. Anything that can write those files,
334
+ including a tool executed inside the workspace, injects text into every session downstream. That is a
335
+ durable prompt-injection surface, and worse than the ordinary kind because it arrives wearing the
336
+ guise of the agent's own past.
337
+
338
+ Mitigation, in the order that pays:
339
+
340
+ 1. **Hash every chapter body and artifact at write time** (`sha256` in frontmatter *and* registry).
341
+ 2. **Verify before citing** at continuation time; a mismatch gets an explicit `⚠ modified since
342
+ archived` marker on its TOC bullet rather than silent trust.
343
+ 3. **Record the authoring session** per chapter, so tampering can be correlated with a turn.
344
+
345
+ Not solved: a determined in-workspace writer can update the registry hash too. The honest claim is
346
+ *accidental drift and single-shot injection are detected*, not *the archive is tamper-proof*. Say it
347
+ that way.
348
+
349
+ ## Bounds — What Still Grows
350
+
351
+ The head is bounded; the index and the disk are not, and neither doc should imply otherwise.
352
+
353
+ - **The flattened TOC must stay an ancestor-path list, not a whole-tree dump.** Listing every chapter of
354
+ every branch would make "flatten the DAG" a new way to exhaust context — the § Ancestry rule (walk the
355
+ ancestor path only; siblings stay behind a future read-only command) is what bounds the index; folding
356
+ below handles depth itself.
357
+ *(This bullet reconstructs a line corrupted in the file's first commit — the fragment `DAG" becomes a
358
+ new way to exhaust context` survived without its opening. The intent was restated, not recalled; see
359
+ spikes/probe/FINDINGS.md § Doc corrections. Delete this parenthetical once re-reviewed.)*
360
+ - **TOC length grows with depth.** At ~30 tokens per bullet and 3 chapters per continuation, a depth-10
361
+ chain costs ~900 tokens — fine. Depth 100 costs ~9K, which is not. **Bullet folding** belongs in
362
+ Phase 4: when ancestor bullets exceed `tocBulletBudgetTokens`, fold the oldest group into one summary
363
+ bullet citing a fold chapter. Folding happens only at continuation boundaries.
364
+ - **Reload is a feedback loop.** A model that `read`s three 8K chapters immediately re-triggers pressure
365
+ and may continue again. State the remaining reload headroom in the TOC preamble so the model spends it
366
+ deliberately, and have the pressure detector count reloaded content like anything else.
367
+ - **Retention.** Append-only with no GC is unbounded disk growth across abandoned chains. A retention
368
+ policy, or at minimum a documented manual prune command, is required before anyone runs this for
369
+ months. Content-addressed artifacts make pruning safe to reason about — refcount them.
370
+
371
+ ## Open Questions
372
+
373
+ Carried into Phase 0 deliberately, not by omission:
374
+
375
+ 1. **Is an unseeded `agents.create` durable, sidebar-listed, and resumable?** The premise of everything.
376
+ If not, the design needs a different carrier and should not be funded as written.
377
+ 2. **Does creating a session schedule a turn?** If so, the child burns its first turn acknowledging its
378
+ own TOC. Fix by instructing the notice to stay silent until addressed, or by creating without
379
+ running.
380
+ 3. **Coexistence with `dsh-session-fork`.** It already maintains session lineage and a branch registry,
381
+ and patches client `sessions.fork` globally. Depend on it as a peer, vendor against the same pinned
382
+ version, or declare mutual exclusion — all defensible; picking silently is not.
383
+ 4. **How do tool results that reference deleted files render?** A `read` of a file later removed is
384
+ still verbatim text, which is correct — but a `bash` output naming a path that no longer exists is
385
+ misleading without context. Probably a renderer annotation, not a format change.
386
+ 5. **Test strategy.** The verification checklist is manual, but "the child is smaller than the parent"
387
+ and "ranges validate" are exactly what an automated regression suite should own. Both reference
388
+ plugins have `tests/`; we should decide the harness (vitest, like `dsh-compact`) before Phase 1.
389
+
390
+ ## References
391
+
392
+ `examples/` is gitignored; read the **one** file that answers the question. Do not read all four.
393
+
394
+ | Concern | File |
395
+ |---|---|
396
+ | Workspace attach, preset compose, resume kernel | `dsh-session-fork/src/vendor/fork.ts` — header lists upstream sources + pinned commit |
397
+ | Why `agents.create` is the only durable route; notice event | `dsh-session-fork/src/branch.ts:8-14,154` |
398
+ | Storage-domain registry | `dsh-session-fork/src/store.ts:36-43` |
399
+ | `defineTool`, `jsonOutput`, `exec` caller | `dsh-session-fork/src/tools.ts:110-121,200` |
400
+ | `inject`, registration | `dsh-session-fork/src/index.ts:161,786` |
401
+ | Real fork handler — confirm what we avoid | `deepseek-harness/packages/api/session-controller/src/commands.ts:185-282` (`seed` at `:260`) |
402
+ | `CreateAgentOptions`, optional `seed` | `deepseek-harness/packages/core/agent/src/index.ts:95,388` |
403
+ | `SessionHeader` | `deepseek-harness/packages/core/session/src/types.ts:93-122` |
404
+ | `defineTool` signature | `deepseek-harness/packages/core/tools/src/schema.ts:545` |
405
+ | `ctx.llm.stream()` prefix reuse | `deepseek-harness/packages/compaction/compaction-basic/src/index.ts:225,290` |
406
+ | `ctx.fs` is optional | `deepseek-harness/packages/context/agent-instructions/src/index.ts:6` |
407
+ | `Schema<Config>` idiom | `dsh-compact/src/index.ts:15,26-37` |
408
+ | Packaging (`dsh.bundle.patch`) | `dsh-session-fork/package.json`, `cordis.patch.yml` |
409
+
410
+ **Do not port from:** `dsh-session-fork/src/vendor/compact.ts`, `squash*.ts`, `merge-region.ts` (span
411
+ rewriting — forbidden); `dsh-session-fork/src/prompt.ts` (system-prompt contribution — invariant 3);
412
+ `dsh-compact`'s mutation strategy (in-place by design). `dsh-plugin-dev-kb` is a minimal plugin shell
413
+ only; skip it otherwise.
414
+
415
+ ## Phases
416
+
417
+ > **Superseded in part.** `docs/host-compaction-seam.md` found that the harness already ships automatic
418
+ > pressure compaction (`thresholdRatio`, default 0.8), provider-confirmed overflow recovery, per-model
419
+ > policies, and a single sanctioned `summarize()` subclass hook on `BasicCompactionEngine`. Phases 1-3 below
420
+ > predate that discovery: `pressure.ts`, `compact.ts`, custom surface mutation, and our own compact command
421
+ > are all **dropped**. The MVP becomes a subclassed engine that returns a deterministic TOC over archived
422
+ > chapters, plus the archive, registry, and fork tool. **Phase 0b (rounds 12-17) verified that revised MVP
423
+ > end-to-end** and settled the mount: our engine rides a preset realm (`isolate: { compaction: true }`
424
+ > group, our row replacing `compaction-basic`), continuations compose via
425
+ > `setup: agentCtx => agentPresets.mount(agentCtx, presetId)`, and `chapters_continue` remains the
426
+ > *branching* mechanism — automatic relief is the engine's. Read
427
+ > [docs/host-compaction-seam.md](host-compaction-seam.md) and FINDINGS § Phase 0b before implementing.
428
+
429
+ Reordered so the reserve-margin and pressure machinery — which automatic triggering needs — is built
430
+ early rather than last.
431
+
432
+ **Phase 0 — Prove the premise.** Skeleton, one trivial tool with an `output` schema, then: unseeded
433
+ create → durable? listed? resumable? header-matching? Single-event seed at `seq = 0` accepted by
434
+ continuity validation? Child first-request tokens materially below the parent's? Settle naming, the test
435
+ harness, and the `dsh-session-fork` coexistence question here.
436
+ *Gate: if the child is not durable and listed, stop and redesign.*
437
+
438
+ **Phase 1 — Render, cite, and bound.** `chapters_continue` with validated ranges; bodies rendered from
439
+ the log; artifact deferral and content-addressed dedup; registry with ancestry, numbering, hashes; flat
440
+ ancestor-path TOC with handoff note and back-links; the continuation **preflight** with
441
+ concrete refusal numbers; atomic ordering and idempotent retry. Manual trigger, agent supplies ranges.
442
+
443
+ **Phase 2 — Segmentation as a function, plus pressure detection.** `chapters_segment` returning validated
444
+ ranges, implemented as a callable usable **with or without an agent turn**; `turn/end` pressure check
445
+ that **surfaces a suggestion** (no autonomous action). Both the UI button and Phase 3 depend on this
446
+ seam, which is why it is here rather than near the end.
447
+
448
+ **Phase 3 — Automatic continuation.** Fires on the Phase 2 detector at `turn/end` only, through the same
449
+ Phase 1 preflight. Never mid-step, never `agent/pre-step`.
450
+
451
+ **Phase 4 — Bound the index.** Chapter splitting at continuation boundaries; bullet folding past
452
+ `tocBulletBudgetTokens`; reload budget in the TOC preamble; artifact refcounting and a retention/prune
453
+ command.
454
+
455
+ **Phase 5 — UI trigger.** Title-bar button sharing Phase 2's segmentation. Deliberately last: it needs a
456
+ client bundle, `tsdown.client.config.ts`, and the `slots`/`locale`/`connection` client injects — the
457
+ heaviest packaging in the project, for ergonomics rather than correctness.
458
+
459
+ Permanently out of scope: in-place span compaction, prefix rewriting, seeding from parent events,
460
+ system-prompt contributions, and truncating anything to fit.