@try-works/dsh-memory 0.1.18 → 0.1.20

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. package/LICENSE +21 -21
  2. package/README.md +585 -585
  3. package/STATUS.md +287 -287
  4. package/cordis.patch.yml +69 -69
  5. package/package.json +1 -1
  6. package/scripts/check-package.mjs +76 -76
  7. package/src/SYSTEM.template.md +311 -311
  8. package/src/capture.js +242 -242
  9. package/src/client/index.js +886 -886
  10. package/src/config.js +57 -57
  11. package/src/consolidate-run.js +132 -132
  12. package/src/consolidate.js +330 -330
  13. package/src/context.js +88 -88
  14. package/src/decisions.js +106 -106
  15. package/src/defaults.js +93 -93
  16. package/src/digest.js +255 -255
  17. package/src/entry.js +179 -179
  18. package/src/eval.js +72 -72
  19. package/src/files.js +60 -60
  20. package/src/fixture.js +56 -56
  21. package/src/forget.js +71 -71
  22. package/src/freshness.js +65 -65
  23. package/src/fts.js +166 -166
  24. package/src/git.js +53 -53
  25. package/src/history.js +226 -226
  26. package/src/host.js +866 -857
  27. package/src/index.js +115 -94
  28. package/src/inject.js +108 -108
  29. package/src/layers.js +178 -178
  30. package/src/ledger.js +19 -19
  31. package/src/map.js +184 -184
  32. package/src/models.js +136 -136
  33. package/src/queue.js +602 -602
  34. package/src/rank.js +31 -31
  35. package/src/recover.js +91 -91
  36. package/src/refresh.js +80 -80
  37. package/src/remember.js +58 -58
  38. package/src/rerank.js +90 -90
  39. package/src/search.js +75 -75
  40. package/src/select.js +57 -57
  41. package/src/session.js +60 -60
  42. package/src/state.js +240 -240
  43. package/src/store.js +128 -128
  44. package/src/summarize.js +195 -195
  45. package/src/system-doc.js +171 -171
  46. package/src/tasks.js +617 -617
  47. package/src/telemetry.js +122 -122
  48. package/src/tools.js +828 -818
  49. package/src/working.js +288 -288
  50. package/src/workspace-key.js +68 -68
  51. package/src/workspace.js +87 -87
package/STATUS.md CHANGED
@@ -1,287 +1,287 @@
1
- # dsh-memory — STATUS
2
-
3
- A tracker and planner: what exists, what is verified working, what is written but not
4
- loaded, and what is left. Kept as a record of MEASURED state, not intention.
5
-
6
- **Last measured: 2026-10-06** (347 tests passing, 41 src files, 50 test files).
7
-
8
- ## How to read this
9
-
10
- Four states, and the difference between them is the whole point of the document:
11
-
12
- | mark | meaning |
13
- |---|---|
14
- | **LIVE** | verified working in the running host, by observing an effect it produced |
15
- | **WRITTEN** | implemented and tested on disk; the running host has not loaded it |
16
- | **NOT VERIFIED** | believed to work; no observation confirms it. Treat as unknown |
17
- | **TODO** | not done |
18
-
19
- This distinction exists because of two failures in this project. A retry loop was reported
20
- done and was logically correct but could never run, starved behind a backlog in a bounded
21
- batch. And session content search was reported enabled on the strength of a config flag; the
22
- database was created and the index was empty. In both cases a green result masked a dead
23
- feature. Both are recorded below as they actually are.
24
-
25
- ## What dsh-memory is
26
-
27
- A DSH plugin storing durable agent memory as layered Markdown under a configurable root, with
28
- a global layer and per-workspace layers, committable to a git origin. Markdown is the record;
29
- indexes and state are derived and gitignored.
30
-
31
- | layer | scope | contents |
32
- |---|---|---|
33
- | `semantic` | global + workspace | facts, preferences, standing rules |
34
- | `episodic` | global + workspace | what happened, session by session |
35
- | `procedural` | global + workspace | how to do things here |
36
- | `learning` | global + workspace | lessons, usually from bugs |
37
- | `decision` | global + workspace | settled choices with their reasons |
38
- | `codebase` | workspace only | what each module does |
39
- | `history` | workspace only | commits, deduped by content |
40
- | `state.md` | workspace only | the repository now, replaced per section, freshness-marked |
41
- | `working.md` | workspace only | the agent now: task, activity, next steps |
42
-
43
- Live store: **1200 entries** — decision 264, learning 280, semantic 278, codebase 173,
44
- procedural 103, episodic 62, history 40.
45
- ## Built and LIVE
46
-
47
- Observed in the running host. Each row names the effect that was seen, not a claim.
48
-
49
- | capability | evidence |
50
- |---|---|
51
- | Queue is the only state store | `state/retired/sessions.json`; `state/queue.jsonl` holds 170 entries |
52
- | Capture writes transitions | queue states move with every session; `summarized` 19 → 23 since boot |
53
- | Consolidation writes transitions | `consolidate` events fire; digests reach `added` |
54
- | Retry scheduling | one entry is `failed` with a reason; `due()` returns it as `retry` |
55
- | Stalled-stage watchdog | `reapStalled` runs before each batch; 0 stranded mid-stage |
56
- | Explicit abandonment | states drain rather than accumulate; `ABANDONED` reported by `problems()` |
57
- | `memory_status` reports the queue | per-state counts, parked reasons, grouped attention with ages |
58
- | Attention is GROUPED | 2 rows where the first version produced 79, 78 of them identical |
59
- | Injection selects by RELEVANCE | the context card reads "Relevant learnings" and lists this session's work, where it previously always listed UI notes |
60
- | History ingest | `history-check` events, 5 since the instrumented host started |
61
- | Injection metric counts injections | `events.inject` advances by 1 per injection; the old character total is preserved under `legacy.inject` |
62
- | `memory_recover` is wired | calling it no longer answers "unavailable in this context"; it reaches output validation |
63
- | Session logs are searchable in principle | zstd, not encrypted; 368 logs, 740MB compressed, ~2.96GB plaintext |
64
-
65
- ## WRITTEN but NOT LOADED
66
-
67
- Implemented and tested on disk. The running host predates them, so none of this is in effect.
68
-
69
- | change | why it matters | how it was verified on disk |
70
- |---|---|---|
71
- | `due()` priority ordering | retry → abandon → work → recover, so a bounded batch reaches a due retry | measured: the retry sat at position 101 of 111 and is now position 1 |
72
- | `hostExtras(host)` as one source | `index.js` and the wiring test shared a copied list that drifted | the test now derives extras from the host object |
73
- | `memory_recover` report omits its `entries` field rather than blanking it | the old line added `entries: undefined`, which is NOT valid JSON, so the harness rejected the whole result | the new guard round-trips every tool output; reintroducing it fails with `value.entries is undefined` |
74
-
75
- How to tell loaded from not: ask the host. `memory_recover` used to reply *"unavailable in this
76
- context"* and now replies *"returned invalid output: value is not lossless JSON"*. The change of
77
- ERROR is the proof the first fix is loaded and the tool now executes — a different failure, one
78
- layer further along. The report fix in the last row is what remains pending.
79
- ### The tasks layer (new since the last restart)
80
-
81
- A fifth layer, `tasks`, global only: `global/tasks/` holds one JSON file per DSH scheduled task
82
- plus a regenerated `index.md`. It backs up the schedule store, which is otherwise the only copy of
83
- your automated tasks and is in no git repository. The host DELETES a task when its owning session
84
- stops, so a task can vanish with nothing to observe; the backup runs on the sync cadence and a
85
- restore re-arms what is missing at the first session after boot.
86
-
87
- Verified on disk: 396 tests pass, and `@try-works/dsh-memory@0.1.16` is published and installed
88
- byte-identical.
89
-
90
- - `backupTasks` against `E:\dsh-memory`: two cron tasks captured, a second pass `written=0`.
91
- - The layer is at `global/tasks/`. It first shipped at `<root>/tasks`; every test used a
92
- self-consistent temp root and could not see the difference (`learning-0213`).
93
- - **The emptied-store drill, on a COPY of the real `schedule.json`:** emptied, backed up, restored
94
- through the schedule plugin's own `createCronScheduleRecord`. Both tasks came back with their
95
- expressions, zones and prompts (3597 and 2678 chars). The SECOND boot reported `add=0 present=2` —
96
- which is the property that matters, since `create` mints a fresh id every time and id-based
97
- de-duplication would duplicate every task on every boot.
98
- - The record -> request -> record round trip holds against the plugin's own builder; only the id
99
- differs.
100
- - No task file leaks into memory search, checked by building the index over a store holding the layer.
101
- - `SYSTEM.md` names the layer, its path, `memory_tasks`, and all eight states.
102
-
103
- **NOT verified, and the log proves it.** `E:\dsh-memory\logs\memory.log` holds 3976 entries and
104
- ZERO `tasks-backup` or `tasks-restore` ones, while the running host is 3h24m old and predates every
105
- install from 0.1.10 on. The plugin is alive — it is injecting, capturing and consolidating — it has
106
- simply never executed this code. The cadence and the boot restore have only ever run inside the
107
- wiring tests, where the schedule service is a stub and the store is temporary. **Calling the module
108
- directly is not the same claim as the plugin calling it.**
109
-
110
- Seven defects were found by asking what the REAL path does, and not one by an existing test: the
111
- empty-string config fallback, the wrong directory, a fixture-coincidence assertion, `orphaned` with
112
- no producer, `skipped` never written while unknown-kind tasks were dropped from the backup, an
113
- exits table read by nothing and wrong twice, and a manual restore that returned silently while
114
- reporting success.
115
-
116
- Known limitation, by design: the host deletes a task when its owning session stops, so a restored
117
- task lives as long as the session that adopted it. The boot restore is what re-arms them.
118
-
119
- ## NOT VERIFIED — treat these as unknown
120
-
121
- | item | what is actually known | what would settle it |
122
- |---|---|---|
123
- | Session CONTENT search | The index database exists at `~/.dsh/storages/session-search.sqlite` and was opened, so `openAt: first-search` took effect. It holds **0 sessions, 0 documents, `global_generation` 0**. Every sidebar result therefore comes from a browser-side TITLE match. | A **content-only** hit: a query matching text inside a session that appears in no title or workspace name. |
124
- | Cause of the empty index | Reconciliation runs `_observeStable -> persistence.list -> readColdSessionLog` and indexes nothing. Candidates, untraced: persistence binding unavailable, cold reads failing silently, live sessions skipped. | Tracing the reconcile path with a real store. |
125
- | Sidebar behaviour with a populated index | The service uses FTS5 MATCH and the client merges content hits with local title matches. The merge code is real; it has never had content to merge. | The same content-only hit above. |
126
-
127
- ### Where the sidebar search actually comes from
128
-
129
- Recorded because it was got wrong once. `client/ui-workspace/src/client/tree.ts`
130
- `deriveSearchResults` has two sources:
131
-
132
- ```js
133
- // LOCAL - browser-side substring match, computed from data already loaded
134
- sessionTitle(summary).toLowerCase().includes(q)
135
- || labelOf(summary).toLowerCase().includes(q)
136
-
137
- // CONTENT - server-side FTS hits, empty today
138
- for (const item of content.items) include(summary)
139
- ```
140
-
141
- So title search works and content search is wired but starved. The distinction matters: turning
142
- the flag on did not change what the user sees.
143
-
144
- ## TODO — ordered
145
-
146
- ### Blocked on a restart
147
-
148
- - [x] **Restart, then check the tasks layer — DONE 2026-10-09.** The restart confirmed it. See the
149
- tasks-layer section above for the two log entries and the counts. It writes to ANOTHER plugin's storage through
150
- that plugin's service, so it deserves a real observation rather than a passing suite. After
151
- the restart, ask for `memory_tasks` and expect a report with `live: 2`, `backedUp: 2` and
152
- `pending.present: 2`. Then check `logs/memory.log` for a `tasks-backup` entry and a
153
- `tasks-restore` entry: the first should say `refreshed: 2`, the second `present: 2, added: 0`.
154
- An `added` above zero would mean de-duplication failed and it duplicated a task.
155
- - [ ] **Restart again.** The previous restart worked: `memory_recover` went from "unavailable in
156
- this context" to executing. It now fails one layer further on, at output validation, which
157
- is fixed on disk. Confirm by calling `memory_recover({})` and getting a report, not an error.
158
-
159
- ### Recovery (needs the restart)
160
-
161
- - [ ] `memory_recover({})` — report only, no cost.
162
- - [ ] `memory_recover({ limit: 1 })` — **prove it end to end on one** before spending on 76.
163
- - [ ] Then the rest in batches of ~10. Cost is roughly 76-150 model calls plus consolidation,
164
- against **52 MB** of logs. All 76 are under the 8 MB approval threshold, so nothing will
165
- stop an accidental `limit: 76` — the one-candidate proof is the guard, not the threshold.
166
- ### Open work, not blocked
167
-
168
- - [ ] **Held backfills: 28.** `state/backfill-hold.json`. Released one as a gate test and it
169
- produced 2 entries against a 1-3 target, where the live rate was 7-10 — so the conservative
170
- bar works. Release in batches and watch the rate after each; ~7 per digest means the bar is
171
- not reaching the prompt and the batch should stop.
172
- - [ ] **79 held records, reason `unclassified`.** Legacy records whose summary was empty and
173
- whose cause was never recorded, so a conclusion and a defect are indistinguishable. Parked
174
- deliberately: retrying them unasked would be a backfill nobody requested. An operator
175
- decision, not a bug.
176
- - [ ] **Session content search.** See NOT VERIFIED. Two routes: trace why reconciliation indexes
177
- nothing, or accept title search and close it. Do not claim it works until a content-only
178
- hit is observed.
179
- - [ ] **Agent-facing session tools** (`session_search` and friends). NOT a config change. In a
180
- Web profile the tool plane belongs to a preset, and a patch REPLACES a preset row config
181
- wholesale — so adding one tool means restating the whole ~20-row `config.plugins` list and
182
- re-syncing by hand whenever the shipped preset changes. Decide this only if the capability
183
- is actually missed.
184
- - [ ] **Pre-`inject`-fix metric distortion is cleared**; `legacy.inject` records the old value
185
- and what it meant. Nothing to do, noted so the number is not misread again.
186
-
187
- ## Known bugs, fixed, and how each was found
188
-
189
- Kept because the pattern matters more than the list: every one of these was found by asking
190
- "does this actually work?" rather than by a test going red.
191
-
192
- | bug | how it surfaced |
193
- |---|---|
194
- | Retry loop starved | reading the live due list: the retry sat at position 101 of 111 |
195
- | `memory_recover` unwired | calling the tool; it threw "unavailable in this context" |
196
- | Wiring test could not catch it | it copied the extras list AND only checked registration |
197
- | Session content search believed working | reading the index with `node:sqlite`; 0 rows |
198
- | `inject` metric counted characters | 218,590 "injections" in a store with 101 |
199
- | 78 orphans misclassified as parked | the migration checked the digest only when unsettled |
200
- | `orphaned` treated as retryable | it would have burned 78 model calls on missing material |
201
- | A gate that proved nothing | it asserted guidance present only when a backfill is in the batch |
202
- | `memory_recover` returned invalid JSON | the harness rejected it; the tests called `execute()` directly and never checked the shape |
203
- | A test edit silently discarded | my own script re-read and overwrote the file before writing |
204
- ## Published
205
-
206
- **`@try-works/dsh-memory@0.1.16`** on npm (0.1.0 was the first release; the Memory tab and the
207
- tasks layer each added several). The unscoped name `dsh-memory` was already taken by an
208
- unrelated DeepSeek Harness memory plugin, so the package publishes under the existing `@try-works`
209
- scope.
210
-
211
- What did NOT change, deliberately: the Cordis plugin name (`export const name = 'dsh-memory'`) and
212
- the bundle row id (`id: dsh-memory`). Config is addressed by ROW ID, so an existing profile's
213
- `memoryRoot` row survives the rename untouched. Only the package name and the module specifier in
214
- `cordis.patch.yml` moved.
215
-
216
- Verified after publishing: a clean install in a temp directory resolves 0.1.0, ships 49 files and
217
- 42 src modules with no test directory, and the installed entry exports `name`, `inject`, `apply`,
218
- `Config` and `hostExtras`.
219
-
220
- Note for a first publish of any new name: npm creates a `0.0.0-stage` placeholder while the real
221
- version is staged, so an immediate registry check 404s. It took about three minutes to appear.
222
-
223
- ## Layout
224
-
225
- ```
226
- src/
227
- index.js plugin entry: Config schema, apply(), hostExtras(host)
228
- host.js hooks: session/event, agent/pre-step, session/disposed, idle ticker
229
- queue.js THE state store: states, transitions, retry, watchdog, migration
230
- capture.js session -> digest -> summary; reads and writes queue states
231
- summarize.js the summary call and its reply parsing
232
- digest.js projection: strips injected context, extracts candidates
233
- consolidate.js prompt construction, the line protocol, backfill guidance
234
- consolidate-run.js the consolidation pass
235
- recover.js orphan recovery: plan (dry run) and rebuild-one
236
- entry.js entry formatting, id allocation, parsing
237
- layers.js layered files, topic files, indexes
238
- remember.js the canonical write path
239
- forget.js tombstones
240
- refresh.js freshness re-examination
241
- inject.js the per-turn context card
242
- select.js relevance selection with a recency fallback
243
- fts.js FTS5 index: word + trigram
244
- search.js query -> ranked hits with citations
245
- rank.js rank blending
246
- ledger.js the derived entry ledger
247
- state.js state.md: replace-per-section, freshness
248
- working.js working.md: the agent file
249
- tools.js the memory_* tool surface
250
- telemetry.js metrics and the run log
251
- defaults.js config defaults and validation
252
- models.js the model ladder
253
- client/ the Settings page (browser half)
254
- ```
255
-
256
- Conventions: plain ESM, no build step, `node --test`. The only dependency is `node:sqlite`,
257
- which is built into Node.
258
-
259
- ## Documents
260
-
261
- | file | what it is |
262
- |---|---|
263
- | **STATUS.md** | this file: what works, what does not, what is left |
264
- | `README.md` | the plugin as a product: install, configure, tools |
265
- | `PLAN-queue-wiring.md` | the completed nine-phase plan that made the queue the state store |
266
- | `dsh-memory-v2-plan.md` | the layered-store design |
267
- | `old-memory-plan.md` | the retired prototype. Superseded; kept for archaeology only |
268
- | `BRIEF-sandbox-acl-fix.md` | an unrelated Windows sandbox note |
269
-
270
- ## The discipline this project settled on
271
-
272
- Written down because it is the thing that actually caught the bugs:
273
-
274
- 1. **Reintroduce the bug to prove the guard.** A test that has never failed has not been shown
275
- to test anything. Every guard added late in this project was verified by breaking the thing
276
- it protects.
277
- 2. **Verify at the level the user experiences it.** A config flag taking effect is not the
278
- feature working. The database existed and the index was empty.
279
- 3. **Prefer a state to a flag.** A reader should never have to derive what happened from a
280
- timestamp plus a counter.
281
- 4. **Fail closed on cost.** An unknown size asks for approval; `unavailable` from the approval
282
- service means no.
283
- 5. **A bounded consumer turns ordering into policy.** Whatever sorts last may never run.
284
- 6. **Say what is measured and label what is inferred.** The difference is in the tables above.
285
- 7. **A changed error is evidence of progress.** "Unavailable" becoming "invalid output" means the
286
- first fix landed and the failure moved one layer deeper. Reading the message rather than the
287
- success flag is what located three of the bugs above.
1
+ # dsh-memory — STATUS
2
+
3
+ A tracker and planner: what exists, what is verified working, what is written but not
4
+ loaded, and what is left. Kept as a record of MEASURED state, not intention.
5
+
6
+ **Last measured: 2026-10-06** (347 tests passing, 41 src files, 50 test files).
7
+
8
+ ## How to read this
9
+
10
+ Four states, and the difference between them is the whole point of the document:
11
+
12
+ | mark | meaning |
13
+ |---|---|
14
+ | **LIVE** | verified working in the running host, by observing an effect it produced |
15
+ | **WRITTEN** | implemented and tested on disk; the running host has not loaded it |
16
+ | **NOT VERIFIED** | believed to work; no observation confirms it. Treat as unknown |
17
+ | **TODO** | not done |
18
+
19
+ This distinction exists because of two failures in this project. A retry loop was reported
20
+ done and was logically correct but could never run, starved behind a backlog in a bounded
21
+ batch. And session content search was reported enabled on the strength of a config flag; the
22
+ database was created and the index was empty. In both cases a green result masked a dead
23
+ feature. Both are recorded below as they actually are.
24
+
25
+ ## What dsh-memory is
26
+
27
+ A DSH plugin storing durable agent memory as layered Markdown under a configurable root, with
28
+ a global layer and per-workspace layers, committable to a git origin. Markdown is the record;
29
+ indexes and state are derived and gitignored.
30
+
31
+ | layer | scope | contents |
32
+ |---|---|---|
33
+ | `semantic` | global + workspace | facts, preferences, standing rules |
34
+ | `episodic` | global + workspace | what happened, session by session |
35
+ | `procedural` | global + workspace | how to do things here |
36
+ | `learning` | global + workspace | lessons, usually from bugs |
37
+ | `decision` | global + workspace | settled choices with their reasons |
38
+ | `codebase` | workspace only | what each module does |
39
+ | `history` | workspace only | commits, deduped by content |
40
+ | `state.md` | workspace only | the repository now, replaced per section, freshness-marked |
41
+ | `working.md` | workspace only | the agent now: task, activity, next steps |
42
+
43
+ Live store: **1200 entries** — decision 264, learning 280, semantic 278, codebase 173,
44
+ procedural 103, episodic 62, history 40.
45
+ ## Built and LIVE
46
+
47
+ Observed in the running host. Each row names the effect that was seen, not a claim.
48
+
49
+ | capability | evidence |
50
+ |---|---|
51
+ | Queue is the only state store | `state/retired/sessions.json`; `state/queue.jsonl` holds 170 entries |
52
+ | Capture writes transitions | queue states move with every session; `summarized` 19 → 23 since boot |
53
+ | Consolidation writes transitions | `consolidate` events fire; digests reach `added` |
54
+ | Retry scheduling | one entry is `failed` with a reason; `due()` returns it as `retry` |
55
+ | Stalled-stage watchdog | `reapStalled` runs before each batch; 0 stranded mid-stage |
56
+ | Explicit abandonment | states drain rather than accumulate; `ABANDONED` reported by `problems()` |
57
+ | `memory_status` reports the queue | per-state counts, parked reasons, grouped attention with ages |
58
+ | Attention is GROUPED | 2 rows where the first version produced 79, 78 of them identical |
59
+ | Injection selects by RELEVANCE | the context card reads "Relevant learnings" and lists this session's work, where it previously always listed UI notes |
60
+ | History ingest | `history-check` events, 5 since the instrumented host started |
61
+ | Injection metric counts injections | `events.inject` advances by 1 per injection; the old character total is preserved under `legacy.inject` |
62
+ | `memory_recover` is wired | calling it no longer answers "unavailable in this context"; it reaches output validation |
63
+ | Session logs are searchable in principle | zstd, not encrypted; 368 logs, 740MB compressed, ~2.96GB plaintext |
64
+
65
+ ## WRITTEN but NOT LOADED
66
+
67
+ Implemented and tested on disk. The running host predates them, so none of this is in effect.
68
+
69
+ | change | why it matters | how it was verified on disk |
70
+ |---|---|---|
71
+ | `due()` priority ordering | retry → abandon → work → recover, so a bounded batch reaches a due retry | measured: the retry sat at position 101 of 111 and is now position 1 |
72
+ | `hostExtras(host)` as one source | `index.js` and the wiring test shared a copied list that drifted | the test now derives extras from the host object |
73
+ | `memory_recover` report omits its `entries` field rather than blanking it | the old line added `entries: undefined`, which is NOT valid JSON, so the harness rejected the whole result | the new guard round-trips every tool output; reintroducing it fails with `value.entries is undefined` |
74
+
75
+ How to tell loaded from not: ask the host. `memory_recover` used to reply *"unavailable in this
76
+ context"* and now replies *"returned invalid output: value is not lossless JSON"*. The change of
77
+ ERROR is the proof the first fix is loaded and the tool now executes — a different failure, one
78
+ layer further along. The report fix in the last row is what remains pending.
79
+ ### The tasks layer (new since the last restart)
80
+
81
+ A fifth layer, `tasks`, global only: `global/tasks/` holds one JSON file per DSH scheduled task
82
+ plus a regenerated `index.md`. It backs up the schedule store, which is otherwise the only copy of
83
+ your automated tasks and is in no git repository. The host DELETES a task when its owning session
84
+ stops, so a task can vanish with nothing to observe; the backup runs on the sync cadence and a
85
+ restore re-arms what is missing at the first session after boot.
86
+
87
+ Verified on disk: 396 tests pass, and `@try-works/dsh-memory@0.1.16` is published and installed
88
+ byte-identical.
89
+
90
+ - `backupTasks` against `E:\dsh-memory`: two cron tasks captured, a second pass `written=0`.
91
+ - The layer is at `global/tasks/`. It first shipped at `<root>/tasks`; every test used a
92
+ self-consistent temp root and could not see the difference (`learning-0213`).
93
+ - **The emptied-store drill, on a COPY of the real `schedule.json`:** emptied, backed up, restored
94
+ through the schedule plugin's own `createCronScheduleRecord`. Both tasks came back with their
95
+ expressions, zones and prompts (3597 and 2678 chars). The SECOND boot reported `add=0 present=2` —
96
+ which is the property that matters, since `create` mints a fresh id every time and id-based
97
+ de-duplication would duplicate every task on every boot.
98
+ - The record -> request -> record round trip holds against the plugin's own builder; only the id
99
+ differs.
100
+ - No task file leaks into memory search, checked by building the index over a store holding the layer.
101
+ - `SYSTEM.md` names the layer, its path, `memory_tasks`, and all eight states.
102
+
103
+ **NOT verified, and the log proves it.** `E:\dsh-memory\logs\memory.log` holds 3976 entries and
104
+ ZERO `tasks-backup` or `tasks-restore` ones, while the running host is 3h24m old and predates every
105
+ install from 0.1.10 on. The plugin is alive — it is injecting, capturing and consolidating — it has
106
+ simply never executed this code. The cadence and the boot restore have only ever run inside the
107
+ wiring tests, where the schedule service is a stub and the store is temporary. **Calling the module
108
+ directly is not the same claim as the plugin calling it.**
109
+
110
+ Seven defects were found by asking what the REAL path does, and not one by an existing test: the
111
+ empty-string config fallback, the wrong directory, a fixture-coincidence assertion, `orphaned` with
112
+ no producer, `skipped` never written while unknown-kind tasks were dropped from the backup, an
113
+ exits table read by nothing and wrong twice, and a manual restore that returned silently while
114
+ reporting success.
115
+
116
+ Known limitation, by design: the host deletes a task when its owning session stops, so a restored
117
+ task lives as long as the session that adopted it. The boot restore is what re-arms them.
118
+
119
+ ## NOT VERIFIED — treat these as unknown
120
+
121
+ | item | what is actually known | what would settle it |
122
+ |---|---|---|
123
+ | Session CONTENT search | The index database exists at `~/.dsh/storages/session-search.sqlite` and was opened, so `openAt: first-search` took effect. It holds **0 sessions, 0 documents, `global_generation` 0**. Every sidebar result therefore comes from a browser-side TITLE match. | A **content-only** hit: a query matching text inside a session that appears in no title or workspace name. |
124
+ | Cause of the empty index | Reconciliation runs `_observeStable -> persistence.list -> readColdSessionLog` and indexes nothing. Candidates, untraced: persistence binding unavailable, cold reads failing silently, live sessions skipped. | Tracing the reconcile path with a real store. |
125
+ | Sidebar behaviour with a populated index | The service uses FTS5 MATCH and the client merges content hits with local title matches. The merge code is real; it has never had content to merge. | The same content-only hit above. |
126
+
127
+ ### Where the sidebar search actually comes from
128
+
129
+ Recorded because it was got wrong once. `client/ui-workspace/src/client/tree.ts`
130
+ `deriveSearchResults` has two sources:
131
+
132
+ ```js
133
+ // LOCAL - browser-side substring match, computed from data already loaded
134
+ sessionTitle(summary).toLowerCase().includes(q)
135
+ || labelOf(summary).toLowerCase().includes(q)
136
+
137
+ // CONTENT - server-side FTS hits, empty today
138
+ for (const item of content.items) include(summary)
139
+ ```
140
+
141
+ So title search works and content search is wired but starved. The distinction matters: turning
142
+ the flag on did not change what the user sees.
143
+
144
+ ## TODO — ordered
145
+
146
+ ### Blocked on a restart
147
+
148
+ - [x] **Restart, then check the tasks layer — DONE 2026-10-09.** The restart confirmed it. See the
149
+ tasks-layer section above for the two log entries and the counts. It writes to ANOTHER plugin's storage through
150
+ that plugin's service, so it deserves a real observation rather than a passing suite. After
151
+ the restart, ask for `memory_tasks` and expect a report with `live: 2`, `backedUp: 2` and
152
+ `pending.present: 2`. Then check `logs/memory.log` for a `tasks-backup` entry and a
153
+ `tasks-restore` entry: the first should say `refreshed: 2`, the second `present: 2, added: 0`.
154
+ An `added` above zero would mean de-duplication failed and it duplicated a task.
155
+ - [ ] **Restart again.** The previous restart worked: `memory_recover` went from "unavailable in
156
+ this context" to executing. It now fails one layer further on, at output validation, which
157
+ is fixed on disk. Confirm by calling `memory_recover({})` and getting a report, not an error.
158
+
159
+ ### Recovery (needs the restart)
160
+
161
+ - [ ] `memory_recover({})` — report only, no cost.
162
+ - [ ] `memory_recover({ limit: 1 })` — **prove it end to end on one** before spending on 76.
163
+ - [ ] Then the rest in batches of ~10. Cost is roughly 76-150 model calls plus consolidation,
164
+ against **52 MB** of logs. All 76 are under the 8 MB approval threshold, so nothing will
165
+ stop an accidental `limit: 76` — the one-candidate proof is the guard, not the threshold.
166
+ ### Open work, not blocked
167
+
168
+ - [ ] **Held backfills: 28.** `state/backfill-hold.json`. Released one as a gate test and it
169
+ produced 2 entries against a 1-3 target, where the live rate was 7-10 — so the conservative
170
+ bar works. Release in batches and watch the rate after each; ~7 per digest means the bar is
171
+ not reaching the prompt and the batch should stop.
172
+ - [ ] **79 held records, reason `unclassified`.** Legacy records whose summary was empty and
173
+ whose cause was never recorded, so a conclusion and a defect are indistinguishable. Parked
174
+ deliberately: retrying them unasked would be a backfill nobody requested. An operator
175
+ decision, not a bug.
176
+ - [ ] **Session content search.** See NOT VERIFIED. Two routes: trace why reconciliation indexes
177
+ nothing, or accept title search and close it. Do not claim it works until a content-only
178
+ hit is observed.
179
+ - [ ] **Agent-facing session tools** (`session_search` and friends). NOT a config change. In a
180
+ Web profile the tool plane belongs to a preset, and a patch REPLACES a preset row config
181
+ wholesale — so adding one tool means restating the whole ~20-row `config.plugins` list and
182
+ re-syncing by hand whenever the shipped preset changes. Decide this only if the capability
183
+ is actually missed.
184
+ - [ ] **Pre-`inject`-fix metric distortion is cleared**; `legacy.inject` records the old value
185
+ and what it meant. Nothing to do, noted so the number is not misread again.
186
+
187
+ ## Known bugs, fixed, and how each was found
188
+
189
+ Kept because the pattern matters more than the list: every one of these was found by asking
190
+ "does this actually work?" rather than by a test going red.
191
+
192
+ | bug | how it surfaced |
193
+ |---|---|
194
+ | Retry loop starved | reading the live due list: the retry sat at position 101 of 111 |
195
+ | `memory_recover` unwired | calling the tool; it threw "unavailable in this context" |
196
+ | Wiring test could not catch it | it copied the extras list AND only checked registration |
197
+ | Session content search believed working | reading the index with `node:sqlite`; 0 rows |
198
+ | `inject` metric counted characters | 218,590 "injections" in a store with 101 |
199
+ | 78 orphans misclassified as parked | the migration checked the digest only when unsettled |
200
+ | `orphaned` treated as retryable | it would have burned 78 model calls on missing material |
201
+ | A gate that proved nothing | it asserted guidance present only when a backfill is in the batch |
202
+ | `memory_recover` returned invalid JSON | the harness rejected it; the tests called `execute()` directly and never checked the shape |
203
+ | A test edit silently discarded | my own script re-read and overwrote the file before writing |
204
+ ## Published
205
+
206
+ **`@try-works/dsh-memory@0.1.16`** on npm (0.1.0 was the first release; the Memory tab and the
207
+ tasks layer each added several). The unscoped name `dsh-memory` was already taken by an
208
+ unrelated DeepSeek Harness memory plugin, so the package publishes under the existing `@try-works`
209
+ scope.
210
+
211
+ What did NOT change, deliberately: the Cordis plugin name (`export const name = 'dsh-memory'`) and
212
+ the bundle row id (`id: dsh-memory`). Config is addressed by ROW ID, so an existing profile's
213
+ `memoryRoot` row survives the rename untouched. Only the package name and the module specifier in
214
+ `cordis.patch.yml` moved.
215
+
216
+ Verified after publishing: a clean install in a temp directory resolves 0.1.0, ships 49 files and
217
+ 42 src modules with no test directory, and the installed entry exports `name`, `inject`, `apply`,
218
+ `Config` and `hostExtras`.
219
+
220
+ Note for a first publish of any new name: npm creates a `0.0.0-stage` placeholder while the real
221
+ version is staged, so an immediate registry check 404s. It took about three minutes to appear.
222
+
223
+ ## Layout
224
+
225
+ ```
226
+ src/
227
+ index.js plugin entry: Config schema, apply(), hostExtras(host)
228
+ host.js hooks: session/event, agent/pre-step, session/disposed, idle ticker
229
+ queue.js THE state store: states, transitions, retry, watchdog, migration
230
+ capture.js session -> digest -> summary; reads and writes queue states
231
+ summarize.js the summary call and its reply parsing
232
+ digest.js projection: strips injected context, extracts candidates
233
+ consolidate.js prompt construction, the line protocol, backfill guidance
234
+ consolidate-run.js the consolidation pass
235
+ recover.js orphan recovery: plan (dry run) and rebuild-one
236
+ entry.js entry formatting, id allocation, parsing
237
+ layers.js layered files, topic files, indexes
238
+ remember.js the canonical write path
239
+ forget.js tombstones
240
+ refresh.js freshness re-examination
241
+ inject.js the per-turn context card
242
+ select.js relevance selection with a recency fallback
243
+ fts.js FTS5 index: word + trigram
244
+ search.js query -> ranked hits with citations
245
+ rank.js rank blending
246
+ ledger.js the derived entry ledger
247
+ state.js state.md: replace-per-section, freshness
248
+ working.js working.md: the agent file
249
+ tools.js the memory_* tool surface
250
+ telemetry.js metrics and the run log
251
+ defaults.js config defaults and validation
252
+ models.js the model ladder
253
+ client/ the Settings page (browser half)
254
+ ```
255
+
256
+ Conventions: plain ESM, no build step, `node --test`. The only dependency is `node:sqlite`,
257
+ which is built into Node.
258
+
259
+ ## Documents
260
+
261
+ | file | what it is |
262
+ |---|---|
263
+ | **STATUS.md** | this file: what works, what does not, what is left |
264
+ | `README.md` | the plugin as a product: install, configure, tools |
265
+ | `PLAN-queue-wiring.md` | the completed nine-phase plan that made the queue the state store |
266
+ | `dsh-memory-v2-plan.md` | the layered-store design |
267
+ | `old-memory-plan.md` | the retired prototype. Superseded; kept for archaeology only |
268
+ | `BRIEF-sandbox-acl-fix.md` | an unrelated Windows sandbox note |
269
+
270
+ ## The discipline this project settled on
271
+
272
+ Written down because it is the thing that actually caught the bugs:
273
+
274
+ 1. **Reintroduce the bug to prove the guard.** A test that has never failed has not been shown
275
+ to test anything. Every guard added late in this project was verified by breaking the thing
276
+ it protects.
277
+ 2. **Verify at the level the user experiences it.** A config flag taking effect is not the
278
+ feature working. The database existed and the index was empty.
279
+ 3. **Prefer a state to a flag.** A reader should never have to derive what happened from a
280
+ timestamp plus a counter.
281
+ 4. **Fail closed on cost.** An unknown size asks for approval; `unavailable` from the approval
282
+ service means no.
283
+ 5. **A bounded consumer turns ordering into policy.** Whatever sorts last may never run.
284
+ 6. **Say what is measured and label what is inferred.** The difference is in the tables above.
285
+ 7. **A changed error is evidence of progress.** "Unavailable" becoming "invalid output" means the
286
+ first fix landed and the failure moved one layer deeper. Reading the message rather than the
287
+ success flag is what located three of the bugs above.