@try-works/dsh-memory 0.1.19 → 0.1.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +585 -585
- package/STATUS.md +287 -287
- package/cordis.patch.yml +69 -69
- package/package.json +1 -1
- package/scripts/check-package.mjs +76 -76
- package/src/SYSTEM.template.md +311 -311
- package/src/capture.js +242 -242
- package/src/client/index.js +886 -886
- package/src/config.js +57 -57
- package/src/consolidate-run.js +132 -132
- package/src/consolidate.js +330 -330
- package/src/context.js +88 -88
- package/src/decisions.js +106 -106
- package/src/defaults.js +93 -93
- package/src/digest.js +255 -255
- package/src/entry.js +179 -179
- package/src/eval.js +72 -72
- package/src/files.js +60 -60
- package/src/fixture.js +56 -56
- package/src/forget.js +71 -71
- package/src/freshness.js +65 -65
- package/src/fts.js +166 -166
- package/src/git.js +53 -53
- package/src/history.js +226 -226
- package/src/host.js +866 -866
- package/src/index.js +115 -94
- package/src/inject.js +108 -108
- package/src/layers.js +178 -178
- package/src/ledger.js +19 -19
- package/src/map.js +184 -184
- package/src/models.js +136 -136
- package/src/queue.js +602 -602
- package/src/rank.js +31 -31
- package/src/recover.js +91 -91
- package/src/refresh.js +80 -80
- package/src/remember.js +58 -58
- package/src/rerank.js +90 -90
- package/src/search.js +75 -75
- package/src/select.js +57 -57
- package/src/session.js +60 -60
- package/src/state.js +240 -240
- package/src/store.js +128 -128
- package/src/summarize.js +195 -195
- package/src/system-doc.js +171 -171
- package/src/tasks.js +617 -617
- package/src/telemetry.js +122 -122
- package/src/tools.js +828 -818
- package/src/working.js +288 -288
- package/src/workspace-key.js +68 -68
- package/src/workspace.js +87 -87
package/STATUS.md
CHANGED
|
@@ -1,287 +1,287 @@
|
|
|
1
|
-
# dsh-memory — STATUS
|
|
2
|
-
|
|
3
|
-
A tracker and planner: what exists, what is verified working, what is written but not
|
|
4
|
-
loaded, and what is left. Kept as a record of MEASURED state, not intention.
|
|
5
|
-
|
|
6
|
-
**Last measured: 2026-10-06** (347 tests passing, 41 src files, 50 test files).
|
|
7
|
-
|
|
8
|
-
## How to read this
|
|
9
|
-
|
|
10
|
-
Four states, and the difference between them is the whole point of the document:
|
|
11
|
-
|
|
12
|
-
| mark | meaning |
|
|
13
|
-
|---|---|
|
|
14
|
-
| **LIVE** | verified working in the running host, by observing an effect it produced |
|
|
15
|
-
| **WRITTEN** | implemented and tested on disk; the running host has not loaded it |
|
|
16
|
-
| **NOT VERIFIED** | believed to work; no observation confirms it. Treat as unknown |
|
|
17
|
-
| **TODO** | not done |
|
|
18
|
-
|
|
19
|
-
This distinction exists because of two failures in this project. A retry loop was reported
|
|
20
|
-
done and was logically correct but could never run, starved behind a backlog in a bounded
|
|
21
|
-
batch. And session content search was reported enabled on the strength of a config flag; the
|
|
22
|
-
database was created and the index was empty. In both cases a green result masked a dead
|
|
23
|
-
feature. Both are recorded below as they actually are.
|
|
24
|
-
|
|
25
|
-
## What dsh-memory is
|
|
26
|
-
|
|
27
|
-
A DSH plugin storing durable agent memory as layered Markdown under a configurable root, with
|
|
28
|
-
a global layer and per-workspace layers, committable to a git origin. Markdown is the record;
|
|
29
|
-
indexes and state are derived and gitignored.
|
|
30
|
-
|
|
31
|
-
| layer | scope | contents |
|
|
32
|
-
|---|---|---|
|
|
33
|
-
| `semantic` | global + workspace | facts, preferences, standing rules |
|
|
34
|
-
| `episodic` | global + workspace | what happened, session by session |
|
|
35
|
-
| `procedural` | global + workspace | how to do things here |
|
|
36
|
-
| `learning` | global + workspace | lessons, usually from bugs |
|
|
37
|
-
| `decision` | global + workspace | settled choices with their reasons |
|
|
38
|
-
| `codebase` | workspace only | what each module does |
|
|
39
|
-
| `history` | workspace only | commits, deduped by content |
|
|
40
|
-
| `state.md` | workspace only | the repository now, replaced per section, freshness-marked |
|
|
41
|
-
| `working.md` | workspace only | the agent now: task, activity, next steps |
|
|
42
|
-
|
|
43
|
-
Live store: **1200 entries** — decision 264, learning 280, semantic 278, codebase 173,
|
|
44
|
-
procedural 103, episodic 62, history 40.
|
|
45
|
-
## Built and LIVE
|
|
46
|
-
|
|
47
|
-
Observed in the running host. Each row names the effect that was seen, not a claim.
|
|
48
|
-
|
|
49
|
-
| capability | evidence |
|
|
50
|
-
|---|---|
|
|
51
|
-
| Queue is the only state store | `state/retired/sessions.json`; `state/queue.jsonl` holds 170 entries |
|
|
52
|
-
| Capture writes transitions | queue states move with every session; `summarized` 19 → 23 since boot |
|
|
53
|
-
| Consolidation writes transitions | `consolidate` events fire; digests reach `added` |
|
|
54
|
-
| Retry scheduling | one entry is `failed` with a reason; `due()` returns it as `retry` |
|
|
55
|
-
| Stalled-stage watchdog | `reapStalled` runs before each batch; 0 stranded mid-stage |
|
|
56
|
-
| Explicit abandonment | states drain rather than accumulate; `ABANDONED` reported by `problems()` |
|
|
57
|
-
| `memory_status` reports the queue | per-state counts, parked reasons, grouped attention with ages |
|
|
58
|
-
| Attention is GROUPED | 2 rows where the first version produced 79, 78 of them identical |
|
|
59
|
-
| Injection selects by RELEVANCE | the context card reads "Relevant learnings" and lists this session's work, where it previously always listed UI notes |
|
|
60
|
-
| History ingest | `history-check` events, 5 since the instrumented host started |
|
|
61
|
-
| Injection metric counts injections | `events.inject` advances by 1 per injection; the old character total is preserved under `legacy.inject` |
|
|
62
|
-
| `memory_recover` is wired | calling it no longer answers "unavailable in this context"; it reaches output validation |
|
|
63
|
-
| Session logs are searchable in principle | zstd, not encrypted; 368 logs, 740MB compressed, ~2.96GB plaintext |
|
|
64
|
-
|
|
65
|
-
## WRITTEN but NOT LOADED
|
|
66
|
-
|
|
67
|
-
Implemented and tested on disk. The running host predates them, so none of this is in effect.
|
|
68
|
-
|
|
69
|
-
| change | why it matters | how it was verified on disk |
|
|
70
|
-
|---|---|---|
|
|
71
|
-
| `due()` priority ordering | retry → abandon → work → recover, so a bounded batch reaches a due retry | measured: the retry sat at position 101 of 111 and is now position 1 |
|
|
72
|
-
| `hostExtras(host)` as one source | `index.js` and the wiring test shared a copied list that drifted | the test now derives extras from the host object |
|
|
73
|
-
| `memory_recover` report omits its `entries` field rather than blanking it | the old line added `entries: undefined`, which is NOT valid JSON, so the harness rejected the whole result | the new guard round-trips every tool output; reintroducing it fails with `value.entries is undefined` |
|
|
74
|
-
|
|
75
|
-
How to tell loaded from not: ask the host. `memory_recover` used to reply *"unavailable in this
|
|
76
|
-
context"* and now replies *"returned invalid output: value is not lossless JSON"*. The change of
|
|
77
|
-
ERROR is the proof the first fix is loaded and the tool now executes — a different failure, one
|
|
78
|
-
layer further along. The report fix in the last row is what remains pending.
|
|
79
|
-
### The tasks layer (new since the last restart)
|
|
80
|
-
|
|
81
|
-
A fifth layer, `tasks`, global only: `global/tasks/` holds one JSON file per DSH scheduled task
|
|
82
|
-
plus a regenerated `index.md`. It backs up the schedule store, which is otherwise the only copy of
|
|
83
|
-
your automated tasks and is in no git repository. The host DELETES a task when its owning session
|
|
84
|
-
stops, so a task can vanish with nothing to observe; the backup runs on the sync cadence and a
|
|
85
|
-
restore re-arms what is missing at the first session after boot.
|
|
86
|
-
|
|
87
|
-
Verified on disk: 396 tests pass, and `@try-works/dsh-memory@0.1.16` is published and installed
|
|
88
|
-
byte-identical.
|
|
89
|
-
|
|
90
|
-
- `backupTasks` against `E:\dsh-memory`: two cron tasks captured, a second pass `written=0`.
|
|
91
|
-
- The layer is at `global/tasks/`. It first shipped at `<root>/tasks`; every test used a
|
|
92
|
-
self-consistent temp root and could not see the difference (`learning-0213`).
|
|
93
|
-
- **The emptied-store drill, on a COPY of the real `schedule.json`:** emptied, backed up, restored
|
|
94
|
-
through the schedule plugin's own `createCronScheduleRecord`. Both tasks came back with their
|
|
95
|
-
expressions, zones and prompts (3597 and 2678 chars). The SECOND boot reported `add=0 present=2` —
|
|
96
|
-
which is the property that matters, since `create` mints a fresh id every time and id-based
|
|
97
|
-
de-duplication would duplicate every task on every boot.
|
|
98
|
-
- The record -> request -> record round trip holds against the plugin's own builder; only the id
|
|
99
|
-
differs.
|
|
100
|
-
- No task file leaks into memory search, checked by building the index over a store holding the layer.
|
|
101
|
-
- `SYSTEM.md` names the layer, its path, `memory_tasks`, and all eight states.
|
|
102
|
-
|
|
103
|
-
**NOT verified, and the log proves it.** `E:\dsh-memory\logs\memory.log` holds 3976 entries and
|
|
104
|
-
ZERO `tasks-backup` or `tasks-restore` ones, while the running host is 3h24m old and predates every
|
|
105
|
-
install from 0.1.10 on. The plugin is alive — it is injecting, capturing and consolidating — it has
|
|
106
|
-
simply never executed this code. The cadence and the boot restore have only ever run inside the
|
|
107
|
-
wiring tests, where the schedule service is a stub and the store is temporary. **Calling the module
|
|
108
|
-
directly is not the same claim as the plugin calling it.**
|
|
109
|
-
|
|
110
|
-
Seven defects were found by asking what the REAL path does, and not one by an existing test: the
|
|
111
|
-
empty-string config fallback, the wrong directory, a fixture-coincidence assertion, `orphaned` with
|
|
112
|
-
no producer, `skipped` never written while unknown-kind tasks were dropped from the backup, an
|
|
113
|
-
exits table read by nothing and wrong twice, and a manual restore that returned silently while
|
|
114
|
-
reporting success.
|
|
115
|
-
|
|
116
|
-
Known limitation, by design: the host deletes a task when its owning session stops, so a restored
|
|
117
|
-
task lives as long as the session that adopted it. The boot restore is what re-arms them.
|
|
118
|
-
|
|
119
|
-
## NOT VERIFIED — treat these as unknown
|
|
120
|
-
|
|
121
|
-
| item | what is actually known | what would settle it |
|
|
122
|
-
|---|---|---|
|
|
123
|
-
| Session CONTENT search | The index database exists at `~/.dsh/storages/session-search.sqlite` and was opened, so `openAt: first-search` took effect. It holds **0 sessions, 0 documents, `global_generation` 0**. Every sidebar result therefore comes from a browser-side TITLE match. | A **content-only** hit: a query matching text inside a session that appears in no title or workspace name. |
|
|
124
|
-
| Cause of the empty index | Reconciliation runs `_observeStable -> persistence.list -> readColdSessionLog` and indexes nothing. Candidates, untraced: persistence binding unavailable, cold reads failing silently, live sessions skipped. | Tracing the reconcile path with a real store. |
|
|
125
|
-
| Sidebar behaviour with a populated index | The service uses FTS5 MATCH and the client merges content hits with local title matches. The merge code is real; it has never had content to merge. | The same content-only hit above. |
|
|
126
|
-
|
|
127
|
-
### Where the sidebar search actually comes from
|
|
128
|
-
|
|
129
|
-
Recorded because it was got wrong once. `client/ui-workspace/src/client/tree.ts`
|
|
130
|
-
`deriveSearchResults` has two sources:
|
|
131
|
-
|
|
132
|
-
```js
|
|
133
|
-
// LOCAL - browser-side substring match, computed from data already loaded
|
|
134
|
-
sessionTitle(summary).toLowerCase().includes(q)
|
|
135
|
-
|| labelOf(summary).toLowerCase().includes(q)
|
|
136
|
-
|
|
137
|
-
// CONTENT - server-side FTS hits, empty today
|
|
138
|
-
for (const item of content.items) include(summary)
|
|
139
|
-
```
|
|
140
|
-
|
|
141
|
-
So title search works and content search is wired but starved. The distinction matters: turning
|
|
142
|
-
the flag on did not change what the user sees.
|
|
143
|
-
|
|
144
|
-
## TODO — ordered
|
|
145
|
-
|
|
146
|
-
### Blocked on a restart
|
|
147
|
-
|
|
148
|
-
- [x] **Restart, then check the tasks layer — DONE 2026-10-09.** The restart confirmed it. See the
|
|
149
|
-
tasks-layer section above for the two log entries and the counts. It writes to ANOTHER plugin's storage through
|
|
150
|
-
that plugin's service, so it deserves a real observation rather than a passing suite. After
|
|
151
|
-
the restart, ask for `memory_tasks` and expect a report with `live: 2`, `backedUp: 2` and
|
|
152
|
-
`pending.present: 2`. Then check `logs/memory.log` for a `tasks-backup` entry and a
|
|
153
|
-
`tasks-restore` entry: the first should say `refreshed: 2`, the second `present: 2, added: 0`.
|
|
154
|
-
An `added` above zero would mean de-duplication failed and it duplicated a task.
|
|
155
|
-
- [ ] **Restart again.** The previous restart worked: `memory_recover` went from "unavailable in
|
|
156
|
-
this context" to executing. It now fails one layer further on, at output validation, which
|
|
157
|
-
is fixed on disk. Confirm by calling `memory_recover({})` and getting a report, not an error.
|
|
158
|
-
|
|
159
|
-
### Recovery (needs the restart)
|
|
160
|
-
|
|
161
|
-
- [ ] `memory_recover({})` — report only, no cost.
|
|
162
|
-
- [ ] `memory_recover({ limit: 1 })` — **prove it end to end on one** before spending on 76.
|
|
163
|
-
- [ ] Then the rest in batches of ~10. Cost is roughly 76-150 model calls plus consolidation,
|
|
164
|
-
against **52 MB** of logs. All 76 are under the 8 MB approval threshold, so nothing will
|
|
165
|
-
stop an accidental `limit: 76` — the one-candidate proof is the guard, not the threshold.
|
|
166
|
-
### Open work, not blocked
|
|
167
|
-
|
|
168
|
-
- [ ] **Held backfills: 28.** `state/backfill-hold.json`. Released one as a gate test and it
|
|
169
|
-
produced 2 entries against a 1-3 target, where the live rate was 7-10 — so the conservative
|
|
170
|
-
bar works. Release in batches and watch the rate after each; ~7 per digest means the bar is
|
|
171
|
-
not reaching the prompt and the batch should stop.
|
|
172
|
-
- [ ] **79 held records, reason `unclassified`.** Legacy records whose summary was empty and
|
|
173
|
-
whose cause was never recorded, so a conclusion and a defect are indistinguishable. Parked
|
|
174
|
-
deliberately: retrying them unasked would be a backfill nobody requested. An operator
|
|
175
|
-
decision, not a bug.
|
|
176
|
-
- [ ] **Session content search.** See NOT VERIFIED. Two routes: trace why reconciliation indexes
|
|
177
|
-
nothing, or accept title search and close it. Do not claim it works until a content-only
|
|
178
|
-
hit is observed.
|
|
179
|
-
- [ ] **Agent-facing session tools** (`session_search` and friends). NOT a config change. In a
|
|
180
|
-
Web profile the tool plane belongs to a preset, and a patch REPLACES a preset row config
|
|
181
|
-
wholesale — so adding one tool means restating the whole ~20-row `config.plugins` list and
|
|
182
|
-
re-syncing by hand whenever the shipped preset changes. Decide this only if the capability
|
|
183
|
-
is actually missed.
|
|
184
|
-
- [ ] **Pre-`inject`-fix metric distortion is cleared**; `legacy.inject` records the old value
|
|
185
|
-
and what it meant. Nothing to do, noted so the number is not misread again.
|
|
186
|
-
|
|
187
|
-
## Known bugs, fixed, and how each was found
|
|
188
|
-
|
|
189
|
-
Kept because the pattern matters more than the list: every one of these was found by asking
|
|
190
|
-
"does this actually work?" rather than by a test going red.
|
|
191
|
-
|
|
192
|
-
| bug | how it surfaced |
|
|
193
|
-
|---|---|
|
|
194
|
-
| Retry loop starved | reading the live due list: the retry sat at position 101 of 111 |
|
|
195
|
-
| `memory_recover` unwired | calling the tool; it threw "unavailable in this context" |
|
|
196
|
-
| Wiring test could not catch it | it copied the extras list AND only checked registration |
|
|
197
|
-
| Session content search believed working | reading the index with `node:sqlite`; 0 rows |
|
|
198
|
-
| `inject` metric counted characters | 218,590 "injections" in a store with 101 |
|
|
199
|
-
| 78 orphans misclassified as parked | the migration checked the digest only when unsettled |
|
|
200
|
-
| `orphaned` treated as retryable | it would have burned 78 model calls on missing material |
|
|
201
|
-
| A gate that proved nothing | it asserted guidance present only when a backfill is in the batch |
|
|
202
|
-
| `memory_recover` returned invalid JSON | the harness rejected it; the tests called `execute()` directly and never checked the shape |
|
|
203
|
-
| A test edit silently discarded | my own script re-read and overwrote the file before writing |
|
|
204
|
-
## Published
|
|
205
|
-
|
|
206
|
-
**`@try-works/dsh-memory@0.1.16`** on npm (0.1.0 was the first release; the Memory tab and the
|
|
207
|
-
tasks layer each added several). The unscoped name `dsh-memory` was already taken by an
|
|
208
|
-
unrelated DeepSeek Harness memory plugin, so the package publishes under the existing `@try-works`
|
|
209
|
-
scope.
|
|
210
|
-
|
|
211
|
-
What did NOT change, deliberately: the Cordis plugin name (`export const name = 'dsh-memory'`) and
|
|
212
|
-
the bundle row id (`id: dsh-memory`). Config is addressed by ROW ID, so an existing profile's
|
|
213
|
-
`memoryRoot` row survives the rename untouched. Only the package name and the module specifier in
|
|
214
|
-
`cordis.patch.yml` moved.
|
|
215
|
-
|
|
216
|
-
Verified after publishing: a clean install in a temp directory resolves 0.1.0, ships 49 files and
|
|
217
|
-
42 src modules with no test directory, and the installed entry exports `name`, `inject`, `apply`,
|
|
218
|
-
`Config` and `hostExtras`.
|
|
219
|
-
|
|
220
|
-
Note for a first publish of any new name: npm creates a `0.0.0-stage` placeholder while the real
|
|
221
|
-
version is staged, so an immediate registry check 404s. It took about three minutes to appear.
|
|
222
|
-
|
|
223
|
-
## Layout
|
|
224
|
-
|
|
225
|
-
```
|
|
226
|
-
src/
|
|
227
|
-
index.js plugin entry: Config schema, apply(), hostExtras(host)
|
|
228
|
-
host.js hooks: session/event, agent/pre-step, session/disposed, idle ticker
|
|
229
|
-
queue.js THE state store: states, transitions, retry, watchdog, migration
|
|
230
|
-
capture.js session -> digest -> summary; reads and writes queue states
|
|
231
|
-
summarize.js the summary call and its reply parsing
|
|
232
|
-
digest.js projection: strips injected context, extracts candidates
|
|
233
|
-
consolidate.js prompt construction, the line protocol, backfill guidance
|
|
234
|
-
consolidate-run.js the consolidation pass
|
|
235
|
-
recover.js orphan recovery: plan (dry run) and rebuild-one
|
|
236
|
-
entry.js entry formatting, id allocation, parsing
|
|
237
|
-
layers.js layered files, topic files, indexes
|
|
238
|
-
remember.js the canonical write path
|
|
239
|
-
forget.js tombstones
|
|
240
|
-
refresh.js freshness re-examination
|
|
241
|
-
inject.js the per-turn context card
|
|
242
|
-
select.js relevance selection with a recency fallback
|
|
243
|
-
fts.js FTS5 index: word + trigram
|
|
244
|
-
search.js query -> ranked hits with citations
|
|
245
|
-
rank.js rank blending
|
|
246
|
-
ledger.js the derived entry ledger
|
|
247
|
-
state.js state.md: replace-per-section, freshness
|
|
248
|
-
working.js working.md: the agent file
|
|
249
|
-
tools.js the memory_* tool surface
|
|
250
|
-
telemetry.js metrics and the run log
|
|
251
|
-
defaults.js config defaults and validation
|
|
252
|
-
models.js the model ladder
|
|
253
|
-
client/ the Settings page (browser half)
|
|
254
|
-
```
|
|
255
|
-
|
|
256
|
-
Conventions: plain ESM, no build step, `node --test`. The only dependency is `node:sqlite`,
|
|
257
|
-
which is built into Node.
|
|
258
|
-
|
|
259
|
-
## Documents
|
|
260
|
-
|
|
261
|
-
| file | what it is |
|
|
262
|
-
|---|---|
|
|
263
|
-
| **STATUS.md** | this file: what works, what does not, what is left |
|
|
264
|
-
| `README.md` | the plugin as a product: install, configure, tools |
|
|
265
|
-
| `PLAN-queue-wiring.md` | the completed nine-phase plan that made the queue the state store |
|
|
266
|
-
| `dsh-memory-v2-plan.md` | the layered-store design |
|
|
267
|
-
| `old-memory-plan.md` | the retired prototype. Superseded; kept for archaeology only |
|
|
268
|
-
| `BRIEF-sandbox-acl-fix.md` | an unrelated Windows sandbox note |
|
|
269
|
-
|
|
270
|
-
## The discipline this project settled on
|
|
271
|
-
|
|
272
|
-
Written down because it is the thing that actually caught the bugs:
|
|
273
|
-
|
|
274
|
-
1. **Reintroduce the bug to prove the guard.** A test that has never failed has not been shown
|
|
275
|
-
to test anything. Every guard added late in this project was verified by breaking the thing
|
|
276
|
-
it protects.
|
|
277
|
-
2. **Verify at the level the user experiences it.** A config flag taking effect is not the
|
|
278
|
-
feature working. The database existed and the index was empty.
|
|
279
|
-
3. **Prefer a state to a flag.** A reader should never have to derive what happened from a
|
|
280
|
-
timestamp plus a counter.
|
|
281
|
-
4. **Fail closed on cost.** An unknown size asks for approval; `unavailable` from the approval
|
|
282
|
-
service means no.
|
|
283
|
-
5. **A bounded consumer turns ordering into policy.** Whatever sorts last may never run.
|
|
284
|
-
6. **Say what is measured and label what is inferred.** The difference is in the tables above.
|
|
285
|
-
7. **A changed error is evidence of progress.** "Unavailable" becoming "invalid output" means the
|
|
286
|
-
first fix landed and the failure moved one layer deeper. Reading the message rather than the
|
|
287
|
-
success flag is what located three of the bugs above.
|
|
1
|
+
# dsh-memory — STATUS
|
|
2
|
+
|
|
3
|
+
A tracker and planner: what exists, what is verified working, what is written but not
|
|
4
|
+
loaded, and what is left. Kept as a record of MEASURED state, not intention.
|
|
5
|
+
|
|
6
|
+
**Last measured: 2026-10-06** (347 tests passing, 41 src files, 50 test files).
|
|
7
|
+
|
|
8
|
+
## How to read this
|
|
9
|
+
|
|
10
|
+
Four states, and the difference between them is the whole point of the document:
|
|
11
|
+
|
|
12
|
+
| mark | meaning |
|
|
13
|
+
|---|---|
|
|
14
|
+
| **LIVE** | verified working in the running host, by observing an effect it produced |
|
|
15
|
+
| **WRITTEN** | implemented and tested on disk; the running host has not loaded it |
|
|
16
|
+
| **NOT VERIFIED** | believed to work; no observation confirms it. Treat as unknown |
|
|
17
|
+
| **TODO** | not done |
|
|
18
|
+
|
|
19
|
+
This distinction exists because of two failures in this project. A retry loop was reported
|
|
20
|
+
done and was logically correct but could never run, starved behind a backlog in a bounded
|
|
21
|
+
batch. And session content search was reported enabled on the strength of a config flag; the
|
|
22
|
+
database was created and the index was empty. In both cases a green result masked a dead
|
|
23
|
+
feature. Both are recorded below as they actually are.
|
|
24
|
+
|
|
25
|
+
## What dsh-memory is
|
|
26
|
+
|
|
27
|
+
A DSH plugin storing durable agent memory as layered Markdown under a configurable root, with
|
|
28
|
+
a global layer and per-workspace layers, committable to a git origin. Markdown is the record;
|
|
29
|
+
indexes and state are derived and gitignored.
|
|
30
|
+
|
|
31
|
+
| layer | scope | contents |
|
|
32
|
+
|---|---|---|
|
|
33
|
+
| `semantic` | global + workspace | facts, preferences, standing rules |
|
|
34
|
+
| `episodic` | global + workspace | what happened, session by session |
|
|
35
|
+
| `procedural` | global + workspace | how to do things here |
|
|
36
|
+
| `learning` | global + workspace | lessons, usually from bugs |
|
|
37
|
+
| `decision` | global + workspace | settled choices with their reasons |
|
|
38
|
+
| `codebase` | workspace only | what each module does |
|
|
39
|
+
| `history` | workspace only | commits, deduped by content |
|
|
40
|
+
| `state.md` | workspace only | the repository now, replaced per section, freshness-marked |
|
|
41
|
+
| `working.md` | workspace only | the agent now: task, activity, next steps |
|
|
42
|
+
|
|
43
|
+
Live store: **1200 entries** — decision 264, learning 280, semantic 278, codebase 173,
|
|
44
|
+
procedural 103, episodic 62, history 40.
|
|
45
|
+
## Built and LIVE
|
|
46
|
+
|
|
47
|
+
Observed in the running host. Each row names the effect that was seen, not a claim.
|
|
48
|
+
|
|
49
|
+
| capability | evidence |
|
|
50
|
+
|---|---|
|
|
51
|
+
| Queue is the only state store | `state/retired/sessions.json`; `state/queue.jsonl` holds 170 entries |
|
|
52
|
+
| Capture writes transitions | queue states move with every session; `summarized` 19 → 23 since boot |
|
|
53
|
+
| Consolidation writes transitions | `consolidate` events fire; digests reach `added` |
|
|
54
|
+
| Retry scheduling | one entry is `failed` with a reason; `due()` returns it as `retry` |
|
|
55
|
+
| Stalled-stage watchdog | `reapStalled` runs before each batch; 0 stranded mid-stage |
|
|
56
|
+
| Explicit abandonment | states drain rather than accumulate; `ABANDONED` reported by `problems()` |
|
|
57
|
+
| `memory_status` reports the queue | per-state counts, parked reasons, grouped attention with ages |
|
|
58
|
+
| Attention is GROUPED | 2 rows where the first version produced 79, 78 of them identical |
|
|
59
|
+
| Injection selects by RELEVANCE | the context card reads "Relevant learnings" and lists this session's work, where it previously always listed UI notes |
|
|
60
|
+
| History ingest | `history-check` events, 5 since the instrumented host started |
|
|
61
|
+
| Injection metric counts injections | `events.inject` advances by 1 per injection; the old character total is preserved under `legacy.inject` |
|
|
62
|
+
| `memory_recover` is wired | calling it no longer answers "unavailable in this context"; it reaches output validation |
|
|
63
|
+
| Session logs are searchable in principle | zstd, not encrypted; 368 logs, 740MB compressed, ~2.96GB plaintext |
|
|
64
|
+
|
|
65
|
+
## WRITTEN but NOT LOADED
|
|
66
|
+
|
|
67
|
+
Implemented and tested on disk. The running host predates them, so none of this is in effect.
|
|
68
|
+
|
|
69
|
+
| change | why it matters | how it was verified on disk |
|
|
70
|
+
|---|---|---|
|
|
71
|
+
| `due()` priority ordering | retry → abandon → work → recover, so a bounded batch reaches a due retry | measured: the retry sat at position 101 of 111 and is now position 1 |
|
|
72
|
+
| `hostExtras(host)` as one source | `index.js` and the wiring test shared a copied list that drifted | the test now derives extras from the host object |
|
|
73
|
+
| `memory_recover` report omits its `entries` field rather than blanking it | the old line added `entries: undefined`, which is NOT valid JSON, so the harness rejected the whole result | the new guard round-trips every tool output; reintroducing it fails with `value.entries is undefined` |
|
|
74
|
+
|
|
75
|
+
How to tell loaded from not: ask the host. `memory_recover` used to reply *"unavailable in this
|
|
76
|
+
context"* and now replies *"returned invalid output: value is not lossless JSON"*. The change of
|
|
77
|
+
ERROR is the proof the first fix is loaded and the tool now executes — a different failure, one
|
|
78
|
+
layer further along. The report fix in the last row is what remains pending.
|
|
79
|
+
### The tasks layer (new since the last restart)
|
|
80
|
+
|
|
81
|
+
A fifth layer, `tasks`, global only: `global/tasks/` holds one JSON file per DSH scheduled task
|
|
82
|
+
plus a regenerated `index.md`. It backs up the schedule store, which is otherwise the only copy of
|
|
83
|
+
your automated tasks and is in no git repository. The host DELETES a task when its owning session
|
|
84
|
+
stops, so a task can vanish with nothing to observe; the backup runs on the sync cadence and a
|
|
85
|
+
restore re-arms what is missing at the first session after boot.
|
|
86
|
+
|
|
87
|
+
Verified on disk: 396 tests pass, and `@try-works/dsh-memory@0.1.16` is published and installed
|
|
88
|
+
byte-identical.
|
|
89
|
+
|
|
90
|
+
- `backupTasks` against `E:\dsh-memory`: two cron tasks captured, a second pass `written=0`.
|
|
91
|
+
- The layer is at `global/tasks/`. It first shipped at `<root>/tasks`; every test used a
|
|
92
|
+
self-consistent temp root and could not see the difference (`learning-0213`).
|
|
93
|
+
- **The emptied-store drill, on a COPY of the real `schedule.json`:** emptied, backed up, restored
|
|
94
|
+
through the schedule plugin's own `createCronScheduleRecord`. Both tasks came back with their
|
|
95
|
+
expressions, zones and prompts (3597 and 2678 chars). The SECOND boot reported `add=0 present=2` —
|
|
96
|
+
which is the property that matters, since `create` mints a fresh id every time and id-based
|
|
97
|
+
de-duplication would duplicate every task on every boot.
|
|
98
|
+
- The record -> request -> record round trip holds against the plugin's own builder; only the id
|
|
99
|
+
differs.
|
|
100
|
+
- No task file leaks into memory search, checked by building the index over a store holding the layer.
|
|
101
|
+
- `SYSTEM.md` names the layer, its path, `memory_tasks`, and all eight states.
|
|
102
|
+
|
|
103
|
+
**NOT verified, and the log proves it.** `E:\dsh-memory\logs\memory.log` holds 3976 entries and
|
|
104
|
+
ZERO `tasks-backup` or `tasks-restore` ones, while the running host is 3h24m old and predates every
|
|
105
|
+
install from 0.1.10 on. The plugin is alive — it is injecting, capturing and consolidating — it has
|
|
106
|
+
simply never executed this code. The cadence and the boot restore have only ever run inside the
|
|
107
|
+
wiring tests, where the schedule service is a stub and the store is temporary. **Calling the module
|
|
108
|
+
directly is not the same claim as the plugin calling it.**
|
|
109
|
+
|
|
110
|
+
Seven defects were found by asking what the REAL path does, and not one by an existing test: the
|
|
111
|
+
empty-string config fallback, the wrong directory, a fixture-coincidence assertion, `orphaned` with
|
|
112
|
+
no producer, `skipped` never written while unknown-kind tasks were dropped from the backup, an
|
|
113
|
+
exits table read by nothing and wrong twice, and a manual restore that returned silently while
|
|
114
|
+
reporting success.
|
|
115
|
+
|
|
116
|
+
Known limitation, by design: the host deletes a task when its owning session stops, so a restored
|
|
117
|
+
task lives as long as the session that adopted it. The boot restore is what re-arms them.
|
|
118
|
+
|
|
119
|
+
## NOT VERIFIED — treat these as unknown
|
|
120
|
+
|
|
121
|
+
| item | what is actually known | what would settle it |
|
|
122
|
+
|---|---|---|
|
|
123
|
+
| Session CONTENT search | The index database exists at `~/.dsh/storages/session-search.sqlite` and was opened, so `openAt: first-search` took effect. It holds **0 sessions, 0 documents, `global_generation` 0**. Every sidebar result therefore comes from a browser-side TITLE match. | A **content-only** hit: a query matching text inside a session that appears in no title or workspace name. |
|
|
124
|
+
| Cause of the empty index | Reconciliation runs `_observeStable -> persistence.list -> readColdSessionLog` and indexes nothing. Candidates, untraced: persistence binding unavailable, cold reads failing silently, live sessions skipped. | Tracing the reconcile path with a real store. |
|
|
125
|
+
| Sidebar behaviour with a populated index | The service uses FTS5 MATCH and the client merges content hits with local title matches. The merge code is real; it has never had content to merge. | The same content-only hit above. |
|
|
126
|
+
|
|
127
|
+
### Where the sidebar search actually comes from
|
|
128
|
+
|
|
129
|
+
Recorded because it was got wrong once. `client/ui-workspace/src/client/tree.ts`
|
|
130
|
+
`deriveSearchResults` has two sources:
|
|
131
|
+
|
|
132
|
+
```js
|
|
133
|
+
// LOCAL - browser-side substring match, computed from data already loaded
|
|
134
|
+
sessionTitle(summary).toLowerCase().includes(q)
|
|
135
|
+
|| labelOf(summary).toLowerCase().includes(q)
|
|
136
|
+
|
|
137
|
+
// CONTENT - server-side FTS hits, empty today
|
|
138
|
+
for (const item of content.items) include(summary)
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
So title search works and content search is wired but starved. The distinction matters: turning
|
|
142
|
+
the flag on did not change what the user sees.
|
|
143
|
+
|
|
144
|
+
## TODO — ordered
|
|
145
|
+
|
|
146
|
+
### Blocked on a restart
|
|
147
|
+
|
|
148
|
+
- [x] **Restart, then check the tasks layer — DONE 2026-10-09.** The restart confirmed it. See the
|
|
149
|
+
tasks-layer section above for the two log entries and the counts. It writes to ANOTHER plugin's storage through
|
|
150
|
+
that plugin's service, so it deserves a real observation rather than a passing suite. After
|
|
151
|
+
the restart, ask for `memory_tasks` and expect a report with `live: 2`, `backedUp: 2` and
|
|
152
|
+
`pending.present: 2`. Then check `logs/memory.log` for a `tasks-backup` entry and a
|
|
153
|
+
`tasks-restore` entry: the first should say `refreshed: 2`, the second `present: 2, added: 0`.
|
|
154
|
+
An `added` above zero would mean de-duplication failed and it duplicated a task.
|
|
155
|
+
- [ ] **Restart again.** The previous restart worked: `memory_recover` went from "unavailable in
|
|
156
|
+
this context" to executing. It now fails one layer further on, at output validation, which
|
|
157
|
+
is fixed on disk. Confirm by calling `memory_recover({})` and getting a report, not an error.
|
|
158
|
+
|
|
159
|
+
### Recovery (needs the restart)
|
|
160
|
+
|
|
161
|
+
- [ ] `memory_recover({})` — report only, no cost.
|
|
162
|
+
- [ ] `memory_recover({ limit: 1 })` — **prove it end to end on one** before spending on 76.
|
|
163
|
+
- [ ] Then the rest in batches of ~10. Cost is roughly 76-150 model calls plus consolidation,
|
|
164
|
+
against **52 MB** of logs. All 76 are under the 8 MB approval threshold, so nothing will
|
|
165
|
+
stop an accidental `limit: 76` — the one-candidate proof is the guard, not the threshold.
|
|
166
|
+
### Open work, not blocked
|
|
167
|
+
|
|
168
|
+
- [ ] **Held backfills: 28.** `state/backfill-hold.json`. Released one as a gate test and it
|
|
169
|
+
produced 2 entries against a 1-3 target, where the live rate was 7-10 — so the conservative
|
|
170
|
+
bar works. Release in batches and watch the rate after each; ~7 per digest means the bar is
|
|
171
|
+
not reaching the prompt and the batch should stop.
|
|
172
|
+
- [ ] **79 held records, reason `unclassified`.** Legacy records whose summary was empty and
|
|
173
|
+
whose cause was never recorded, so a conclusion and a defect are indistinguishable. Parked
|
|
174
|
+
deliberately: retrying them unasked would be a backfill nobody requested. An operator
|
|
175
|
+
decision, not a bug.
|
|
176
|
+
- [ ] **Session content search.** See NOT VERIFIED. Two routes: trace why reconciliation indexes
|
|
177
|
+
nothing, or accept title search and close it. Do not claim it works until a content-only
|
|
178
|
+
hit is observed.
|
|
179
|
+
- [ ] **Agent-facing session tools** (`session_search` and friends). NOT a config change. In a
|
|
180
|
+
Web profile the tool plane belongs to a preset, and a patch REPLACES a preset row config
|
|
181
|
+
wholesale — so adding one tool means restating the whole ~20-row `config.plugins` list and
|
|
182
|
+
re-syncing by hand whenever the shipped preset changes. Decide this only if the capability
|
|
183
|
+
is actually missed.
|
|
184
|
+
- [ ] **Pre-`inject`-fix metric distortion is cleared**; `legacy.inject` records the old value
|
|
185
|
+
and what it meant. Nothing to do, noted so the number is not misread again.
|
|
186
|
+
|
|
187
|
+
## Known bugs, fixed, and how each was found
|
|
188
|
+
|
|
189
|
+
Kept because the pattern matters more than the list: every one of these was found by asking
|
|
190
|
+
"does this actually work?" rather than by a test going red.
|
|
191
|
+
|
|
192
|
+
| bug | how it surfaced |
|
|
193
|
+
|---|---|
|
|
194
|
+
| Retry loop starved | reading the live due list: the retry sat at position 101 of 111 |
|
|
195
|
+
| `memory_recover` unwired | calling the tool; it threw "unavailable in this context" |
|
|
196
|
+
| Wiring test could not catch it | it copied the extras list AND only checked registration |
|
|
197
|
+
| Session content search believed working | reading the index with `node:sqlite`; 0 rows |
|
|
198
|
+
| `inject` metric counted characters | 218,590 "injections" in a store with 101 |
|
|
199
|
+
| 78 orphans misclassified as parked | the migration checked the digest only when unsettled |
|
|
200
|
+
| `orphaned` treated as retryable | it would have burned 78 model calls on missing material |
|
|
201
|
+
| A gate that proved nothing | it asserted guidance present only when a backfill is in the batch |
|
|
202
|
+
| `memory_recover` returned invalid JSON | the harness rejected it; the tests called `execute()` directly and never checked the shape |
|
|
203
|
+
| A test edit silently discarded | my own script re-read and overwrote the file before writing |
|
|
204
|
+
## Published
|
|
205
|
+
|
|
206
|
+
**`@try-works/dsh-memory@0.1.16`** on npm (0.1.0 was the first release; the Memory tab and the
|
|
207
|
+
tasks layer each added several). The unscoped name `dsh-memory` was already taken by an
|
|
208
|
+
unrelated DeepSeek Harness memory plugin, so the package publishes under the existing `@try-works`
|
|
209
|
+
scope.
|
|
210
|
+
|
|
211
|
+
What did NOT change, deliberately: the Cordis plugin name (`export const name = 'dsh-memory'`) and
|
|
212
|
+
the bundle row id (`id: dsh-memory`). Config is addressed by ROW ID, so an existing profile's
|
|
213
|
+
`memoryRoot` row survives the rename untouched. Only the package name and the module specifier in
|
|
214
|
+
`cordis.patch.yml` moved.
|
|
215
|
+
|
|
216
|
+
Verified after publishing: a clean install in a temp directory resolves 0.1.0, ships 49 files and
|
|
217
|
+
42 src modules with no test directory, and the installed entry exports `name`, `inject`, `apply`,
|
|
218
|
+
`Config` and `hostExtras`.
|
|
219
|
+
|
|
220
|
+
Note for a first publish of any new name: npm creates a `0.0.0-stage` placeholder while the real
|
|
221
|
+
version is staged, so an immediate registry check 404s. It took about three minutes to appear.
|
|
222
|
+
|
|
223
|
+
## Layout
|
|
224
|
+
|
|
225
|
+
```
|
|
226
|
+
src/
|
|
227
|
+
index.js plugin entry: Config schema, apply(), hostExtras(host)
|
|
228
|
+
host.js hooks: session/event, agent/pre-step, session/disposed, idle ticker
|
|
229
|
+
queue.js THE state store: states, transitions, retry, watchdog, migration
|
|
230
|
+
capture.js session -> digest -> summary; reads and writes queue states
|
|
231
|
+
summarize.js the summary call and its reply parsing
|
|
232
|
+
digest.js projection: strips injected context, extracts candidates
|
|
233
|
+
consolidate.js prompt construction, the line protocol, backfill guidance
|
|
234
|
+
consolidate-run.js the consolidation pass
|
|
235
|
+
recover.js orphan recovery: plan (dry run) and rebuild-one
|
|
236
|
+
entry.js entry formatting, id allocation, parsing
|
|
237
|
+
layers.js layered files, topic files, indexes
|
|
238
|
+
remember.js the canonical write path
|
|
239
|
+
forget.js tombstones
|
|
240
|
+
refresh.js freshness re-examination
|
|
241
|
+
inject.js the per-turn context card
|
|
242
|
+
select.js relevance selection with a recency fallback
|
|
243
|
+
fts.js FTS5 index: word + trigram
|
|
244
|
+
search.js query -> ranked hits with citations
|
|
245
|
+
rank.js rank blending
|
|
246
|
+
ledger.js the derived entry ledger
|
|
247
|
+
state.js state.md: replace-per-section, freshness
|
|
248
|
+
working.js working.md: the agent file
|
|
249
|
+
tools.js the memory_* tool surface
|
|
250
|
+
telemetry.js metrics and the run log
|
|
251
|
+
defaults.js config defaults and validation
|
|
252
|
+
models.js the model ladder
|
|
253
|
+
client/ the Settings page (browser half)
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
Conventions: plain ESM, no build step, `node --test`. The only dependency is `node:sqlite`,
|
|
257
|
+
which is built into Node.
|
|
258
|
+
|
|
259
|
+
## Documents
|
|
260
|
+
|
|
261
|
+
| file | what it is |
|
|
262
|
+
|---|---|
|
|
263
|
+
| **STATUS.md** | this file: what works, what does not, what is left |
|
|
264
|
+
| `README.md` | the plugin as a product: install, configure, tools |
|
|
265
|
+
| `PLAN-queue-wiring.md` | the completed nine-phase plan that made the queue the state store |
|
|
266
|
+
| `dsh-memory-v2-plan.md` | the layered-store design |
|
|
267
|
+
| `old-memory-plan.md` | the retired prototype. Superseded; kept for archaeology only |
|
|
268
|
+
| `BRIEF-sandbox-acl-fix.md` | an unrelated Windows sandbox note |
|
|
269
|
+
|
|
270
|
+
## The discipline this project settled on
|
|
271
|
+
|
|
272
|
+
Written down because it is the thing that actually caught the bugs:
|
|
273
|
+
|
|
274
|
+
1. **Reintroduce the bug to prove the guard.** A test that has never failed has not been shown
|
|
275
|
+
to test anything. Every guard added late in this project was verified by breaking the thing
|
|
276
|
+
it protects.
|
|
277
|
+
2. **Verify at the level the user experiences it.** A config flag taking effect is not the
|
|
278
|
+
feature working. The database existed and the index was empty.
|
|
279
|
+
3. **Prefer a state to a flag.** A reader should never have to derive what happened from a
|
|
280
|
+
timestamp plus a counter.
|
|
281
|
+
4. **Fail closed on cost.** An unknown size asks for approval; `unavailable` from the approval
|
|
282
|
+
service means no.
|
|
283
|
+
5. **A bounded consumer turns ordering into policy.** Whatever sorts last may never run.
|
|
284
|
+
6. **Say what is measured and label what is inferred.** The difference is in the tables above.
|
|
285
|
+
7. **A changed error is evidence of progress.** "Unavailable" becoming "invalid output" means the
|
|
286
|
+
first fix landed and the failure moved one layer deeper. Reading the message rather than the
|
|
287
|
+
success flag is what located three of the bugs above.
|