dsh-hypatia-auto-memory 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +454 -0
- package/cordis.patch.yml +20 -0
- package/lib/client.js +1081 -0
- package/lib/client.js.map +1 -0
- package/package.json +90 -0
- package/skills/hypatia/SKILL.md +585 -0
- package/skills/hypatia-dream/SKILL.md +246 -0
- package/skills/hypatia-dream/evals/evals.json +92 -0
- package/skills/hypatia-memory/SKILL.md +175 -0
- package/src/auto-approve.js +133 -0
- package/src/cascade.js +190 -0
- package/src/client/SettingsCard.module.css +420 -0
- package/src/client/SettingsCard.tsx +700 -0
- package/src/client/card-form.ts +450 -0
- package/src/client/consolidation-models.ts +69 -0
- package/src/client/css-modules.d.ts +4 -0
- package/src/client/index.tsx +106 -0
- package/src/client/locales.ts +111 -0
- package/src/client/shelves.ts +88 -0
- package/src/client/slot-contract.ts +17 -0
- package/src/collector.js +596 -0
- package/src/config.js +225 -0
- package/src/consolidator.js +592 -0
- package/src/content-policy.js +301 -0
- package/src/housekeeping.js +197 -0
- package/src/hypatia-cli.js +294 -0
- package/src/hypatia-client.js +72 -0
- package/src/hypatia-mcp.js +477 -0
- package/src/index.js +418 -0
- package/src/persisted-session.js +99 -0
- package/src/progress.js +56 -0
- package/src/queue.js +466 -0
- package/src/recall.js +148 -0
- package/src/shelf.js +180 -0
- package/src/skills.js +166 -0
- package/src/state.js +87 -0
- package/src/status.js +53 -0
- package/src/writer.js +320 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Ronald White
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,454 @@
|
|
|
1
|
+
# dsh-hypatia-auto-memory
|
|
2
|
+
|
|
3
|
+
Event-driven [Hypatia](https://github.com/tkliuxing/hypatia) memory for
|
|
4
|
+
[DSH](https://github.com/deepseek-ai/deepseek-harness): automatic conversation
|
|
5
|
+
logging and background consolidation — span summaries, a log₁₆(n) archive
|
|
6
|
+
cascade, and adjudicated work-unit extraction — all on a **dedicated model
|
|
7
|
+
route**, with zero model work in the main session.
|
|
8
|
+
|
|
9
|
+
Writing is automatic; **reading is the agent's job**, through the bundled
|
|
10
|
+
`hypatia-memory` skill. That split is deliberate — see *Recall* below.
|
|
11
|
+
|
|
12
|
+
**This plugin replaces `dsh-hypatia`.** That plugin left the writing to the
|
|
13
|
+
agent, which in practice did not happen: in the deployment this one was written
|
|
14
|
+
for, the model loaded the protocol and still issued a `hypatia` command in one
|
|
15
|
+
recorded session out of fifteen — the one where it was asked to directly. Here
|
|
16
|
+
the protocol is driven from **DSH native events**, and the plugin talks to
|
|
17
|
+
hypatia over its own `hypatia mcp` process on the subprocess service (the CLI,
|
|
18
|
+
with argv arrays, as fallback) — no shell, no tool call, no sandbox
|
|
19
|
+
escalation, no approval prompt. What is still worth having from `dsh-hypatia`
|
|
20
|
+
ships here: auto-approval for the agent's own bash `hypatia` calls, and its
|
|
21
|
+
`hypatia` / `hypatia-dream` skills. Install one or the other, not both.
|
|
22
|
+
|
|
23
|
+
## Install
|
|
24
|
+
|
|
25
|
+
Prerequisite: `hypatia` CLI on PATH (or add its basename to `binaries`). A
|
|
26
|
+
build with the `mcp` subcommand (hypatia #20) is used over MCP; an older one
|
|
27
|
+
works through the CLI instead, with one warning at the first call.
|
|
28
|
+
|
|
29
|
+
```bash
|
|
30
|
+
# from a local checkout (development)
|
|
31
|
+
dsh plugin --profile web add link:/path/to/dsh-hypatia-auto-memory
|
|
32
|
+
|
|
33
|
+
# registry install once published
|
|
34
|
+
dsh plugin --profile web add dsh-hypatia-auto-memory
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Repeat for other profiles (desktop, dsh-tui). Restart the profile after install.
|
|
38
|
+
|
|
39
|
+
The browser settings card is pre-built in `lib/client.js` and shipped with the
|
|
40
|
+
package. If you edit `src/client/*`, run `npm install && npm run build` in this
|
|
41
|
+
directory to regenerate it; while `pnpm run dev:web` is active in the DSH
|
|
42
|
+
checkout, client-plugin changes reload without a full page refresh.
|
|
43
|
+
|
|
44
|
+
**Replacing `dsh-hypatia`:** remove it from the profile and restart.
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
dsh plugin --profile web remove dsh-hypatia
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Both register the same skill names and the registry keeps whichever plugin got
|
|
51
|
+
there first. This one refuses to shadow another provider (it warns instead), so
|
|
52
|
+
leaving `dsh-hypatia` installed hands the agent exactly the agent-driven
|
|
53
|
+
protocol whose failure this plugin exists to fix. If it has to stay for some
|
|
54
|
+
other reason, `skills: false` on its bundle row is the minimum.
|
|
55
|
+
|
|
56
|
+
A **skill on disk** of the same name — `~/.agents/skills`, `~/.dsh/skills`, a
|
|
57
|
+
custom directory, or a project's `.dsh/skills` / `.agents/skills` — also wins,
|
|
58
|
+
in every agent session. DSH's skill registry is layered per scope: the nearest
|
|
59
|
+
layer's same-name entry wins outright, and rank (project > plugin > user)
|
|
60
|
+
only breaks ties inside one layer. Agent presets (`st`, `standard`, …) mount
|
|
61
|
+
their own filesystem skill provider in the preset's layer, nearer to the agent
|
|
62
|
+
than this plugin's global registration. That includes the canonical
|
|
63
|
+
`hypatia-memory` which `hypatia skill install --agent codex` writes to
|
|
64
|
+
`~/.agents/skills`: it is the agent-driven protocol and needs host hooks DSH
|
|
65
|
+
does not have. Delete such a copy or move it out of the skills directory;
|
|
66
|
+
renaming its directory in place is not enough, because DSH names a skill by its
|
|
67
|
+
frontmatter `name`. The startup warning names the directory. The skill center shows the global view, so it can list this plugin
|
|
68
|
+
as a skill's provider while sessions load the disk copy.
|
|
69
|
+
|
|
70
|
+
## How it works
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
session/event (emit)
|
|
74
|
+
├─ turn/end ──▶ durable queue (storageDomain tasks)
|
|
75
|
+
│ ├─ log-message ──▶ hypatia msg-<sid>-<index>
|
|
76
|
+
│ ├─ consolidate ──▶ ctx.llm.stream ──▶ sum-* (summary 1) + wu-*
|
|
77
|
+
│ └─ cascade ──▶ $not-summaried ──▶ sum<N>-* (summary N)
|
|
78
|
+
└─ session/title | compaction/summary ──▶ session-node ──▶ session-<sid> + belongTo
|
|
79
|
+
session/disposed ──▶ final flush + consolidation, thresholds waived (a session
|
|
80
|
+
closed while DSH keeps running; a restart is covered by the
|
|
81
|
+
startup consolidation backfill instead)
|
|
82
|
+
agent/session-start ──▶ rules/taboos inject()
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
- **Transport** is a private `hypatia mcp` process: JSON-RPC over stdio on the
|
|
86
|
+
subprocess service, one request at a time, which is how that server handles
|
|
87
|
+
them anyway. It is not a `dsh-mcp-client` entry, so the model never sees
|
|
88
|
+
these tools and the plugin's calls meet no tool policy, hook or approval.
|
|
89
|
+
Over the CLI, every call was a process that opened the shelf; here a burst of
|
|
90
|
+
writes shares one. Content travels as JSON rather than one argv element, so
|
|
91
|
+
it is no longer cut at 96 KiB to fit a command line. Tags and scopes are
|
|
92
|
+
split on commas as the CLI splits them, so an entry is stored the same over
|
|
93
|
+
either transport.
|
|
94
|
+
|
|
95
|
+
One process means one call at a time, where the CLI ran as many as
|
|
96
|
+
`queue.concurrency` allowed. A call waits behind the one in flight — at
|
|
97
|
+
worst a `similar` reloading the embedding model, or a write paying overdue
|
|
98
|
+
embedding debt — and that includes the rules/taboos preload at session
|
|
99
|
+
start.
|
|
100
|
+
|
|
101
|
+
The process starts on the first call and is closed after 60 s without one.
|
|
102
|
+
`hypatia mcp` keeps the embedding model's allocations after a semantic call
|
|
103
|
+
(hundreds of MB), and reads the shelf registry once, at start. For the same
|
|
104
|
+
reason a `shelf '<name>' is not connected` error restarts it, so a shelf
|
|
105
|
+
connected meanwhile is found on the retry, and the shelf listing for the
|
|
106
|
+
settings card always comes from `hypatia list`. A call that overruns its
|
|
107
|
+
30 s deadline kills the process, as the CLI path kills a command; the next
|
|
108
|
+
call starts another.
|
|
109
|
+
|
|
110
|
+
`transport: cli` restores one CLI command per call. A binary that serves no
|
|
111
|
+
usable MCP — no `mcp` subcommand, a tool this plugin calls missing, or an
|
|
112
|
+
error in answer to the handshake — makes the plugin use the CLI until the
|
|
113
|
+
profile reloads. Any other failure to start
|
|
114
|
+
fails only that call, and the next one tries MCP again: a binary that cannot
|
|
115
|
+
be started at all would fail the CLI too, and a start that timed out or
|
|
116
|
+
crashed may not do so twice.
|
|
117
|
+
|
|
118
|
+
- **Collector** records human `user/message` and `assistant/message`, redacts
|
|
119
|
+
secrets, rewrites relative dates against the time the message was sent, caps
|
|
120
|
+
both user and assistant text, and enqueues idempotent log tasks.
|
|
121
|
+
|
|
122
|
+
Each assistant message carries a ledger of its own step's tool calls in the
|
|
123
|
+
shape the hypatia-memory protocol asks for — what was called, how long it
|
|
124
|
+
took, whether it worked, and one line of error on failure — **never the
|
|
125
|
+
output** (`1. \`read\` ×5 — ✅ 1.2s total`). Tool outputs had been 84% of all
|
|
126
|
+
stored message bytes, mostly file paths and contents, and made unrelated
|
|
127
|
+
keyword searches match chat logs. Tool-call blocks leave no `[tool-call]`
|
|
128
|
+
placeholder in the content; a step that only called tools says
|
|
129
|
+
`(tool calls only)`. Plugin-sourced
|
|
130
|
+
messages are skipped — no feedback loops.
|
|
131
|
+
|
|
132
|
+
Spans are cut at `turn/end`, and — for a turn that outlasts `flushWindowMs` —
|
|
133
|
+
at `step/end`, never at an arbitrary moment. DSH appends `assistant/message`
|
|
134
|
+
*before* running the tools it requested and `step/end` only after they have
|
|
135
|
+
all run, so a span always holds whole steps and no entry is written without
|
|
136
|
+
its own tool results. Tool results that DSH compaction re-appends after a
|
|
137
|
+
`compaction/prune` are copies of old results and are not counted again.
|
|
138
|
+
|
|
139
|
+
Entries are named by a **dense message ordinal**, not a session-log seq: DSH
|
|
140
|
+
gives every streamed token delta its own seq, so seq numbering produced a
|
|
141
|
+
sparse keyspace that broke `$not-summaried`'s FIFO batching. The ordinal is
|
|
142
|
+
recounted from the immutable log prefix, so a replay reproduces the same names.
|
|
143
|
+
|
|
144
|
+
- **Queue** is durable before scheduling: per-session ordering, `queue.concurrency`
|
|
145
|
+
cross-session limit, retry with backoff, and progress-watermark backfill on
|
|
146
|
+
boot. A task that absorbs new work while running is re-scheduled rather than
|
|
147
|
+
dropped. Writers are get-before-create AND re-assert their edges, so a crash
|
|
148
|
+
between an entry and its statements is repaired by the replay.
|
|
149
|
+
|
|
150
|
+
- **Consolidator** runs at `turn/end` when the thresholds are met, and again
|
|
151
|
+
unconditionally at `session/disposed` — a task shorter than `checkEveryTurns`
|
|
152
|
+
turns would otherwise be logged and never turned into knowledge. `turn/end`
|
|
153
|
+
carries a reason: a user interrupt is treated as a task boundary and lowers
|
|
154
|
+
the token floor, while a blocked, failed or truncated turn is not a place to
|
|
155
|
+
cut a memory at all. Its transcript is redacted and date-absolutized exactly
|
|
156
|
+
like the log path, and an over-budget span consolidates its **oldest** entries
|
|
157
|
+
and advances the watermark only that far, so the remainder is not silently
|
|
158
|
+
marked done.
|
|
159
|
+
|
|
160
|
+
- **Cascade** archives sixteen unarchived tier-N summaries into one tier-(N+1)
|
|
161
|
+
entry, using hypatia's own `$not-summaried` (which anti-joins on
|
|
162
|
+
`statement.tail` and orders `created_at ASC`, giving FIFO batching for free).
|
|
163
|
+
This is the protocol's log₁₆(n) archive: each tier compresses only material a
|
|
164
|
+
previous tier already distilled.
|
|
165
|
+
|
|
166
|
+
- **Work units** are adjudicated, not guessed. Candidates come from `similar`
|
|
167
|
+
(the only search that reports a distance), filtered by a distance ceiling and
|
|
168
|
+
by an exclusion list for the operational layer, then a small model call
|
|
169
|
+
decides `duplicate | refines | extends | supersedes | contradicts | unrelated`.
|
|
170
|
+
A contradiction keeps **both** entries and records `supersedes` — a memory
|
|
171
|
+
system must not quietly forget what it once believed.
|
|
172
|
+
|
|
173
|
+
- **Housekeeping** runs once per startup over the progress table. It resets a
|
|
174
|
+
row whose session has no `msg-*` entry left in the shelf — a wiped or
|
|
175
|
+
replaced shelf would otherwise leave that session unlogged for good — and
|
|
176
|
+
removes the row and tasks of a session DSH no longer has. Resetting costs a
|
|
177
|
+
re-log and a re-consolidation, so both passes act only on positive evidence:
|
|
178
|
+
a failed shelf query or an empty session listing changes nothing.
|
|
179
|
+
|
|
180
|
+
A third pass consolidates what a restart interrupted. The session-end trigger
|
|
181
|
+
does not survive one: DSH runs its close path at shutdown (it appends
|
|
182
|
+
`session/end-seed`), but nothing queued there becomes durable before the
|
|
183
|
+
process is gone — measured on a live restart, the storage file was not written
|
|
184
|
+
and the session came back with `lastConsolidatedSeq: 0`. So a row whose logged
|
|
185
|
+
tail was never consolidated is queued at the next startup, gated by the same
|
|
186
|
+
`consolidation.minNewTokens` floor and read from the row rather than by
|
|
187
|
+
loading the session. Without that gate every restart would spend one model
|
|
188
|
+
call per session carrying any tail at all.
|
|
189
|
+
|
|
190
|
+
Those tasks then read their events from **storage**, not only from the live
|
|
191
|
+
store. `session/created` — the signal that wakes a deferred task — fires only
|
|
192
|
+
where an agent actually runs, so a finished session may never emit it again,
|
|
193
|
+
and a subagent session (which the sidebar does not even list) almost certainly
|
|
194
|
+
will not: one sat `deferred` across a restart and stayed deferred while its
|
|
195
|
+
transcript was open on screen. `sessionPersistence.load()` returns the same
|
|
196
|
+
log without publishing anything, and the executors use only `snapshotEvents`,
|
|
197
|
+
`inheritedEventCount`, `seq` and `header`, so a tail nothing will reopen still
|
|
198
|
+
becomes knowledge. A composition without `sessionPersistence` behaves as
|
|
199
|
+
before: the task waits.
|
|
200
|
+
|
|
201
|
+
- **Recall** preloads project/global rules and taboos at session start. Nothing
|
|
202
|
+
else is pushed, except one line naming the shelf when it is not `default`, so
|
|
203
|
+
the agent's own `hypatia` calls look where the plugin writes. Retrieval is the agent's job through the bundled skill, which
|
|
204
|
+
is what [`memory-nolinear.md`](https://github.com/tkliuxing/hypatia/blob/main/docs/memory-nolinear.md)
|
|
205
|
+
prescribes for agents that hold context and can call tools; the previous
|
|
206
|
+
per-turn injector also vetoed DSH's own runtime context section by returning
|
|
207
|
+
from `agent/pre-step` without calling `next()`.
|
|
208
|
+
|
|
209
|
+
- **Auto-approve** answers the approval request for the agent's own bash
|
|
210
|
+
`hypatia` calls, and only those: the executable word must be a trusted
|
|
211
|
+
basename (after `KEY=value` prefixes) and the command must carry no pipe,
|
|
212
|
+
redirect, chain or command substitution *outside quotes* — a JSE argument full
|
|
213
|
+
of `|` and `>` inside quotes still qualifies. Everything else goes to the
|
|
214
|
+
human. The plugin's own writes never take this path.
|
|
215
|
+
|
|
216
|
+
Scope, measured rather than assumed: in DSH a bash call raises an approval
|
|
217
|
+
request **only** when the sandbox denies it and the model retries with
|
|
218
|
+
`sandbox_permissions`. Retrieval (`query`, `search`, `list`, `knowledge-get`)
|
|
219
|
+
runs inside a `workspace-write` sandbox and never asks, with or without this
|
|
220
|
+
plugin. What this answers is the hypatia **write** — `~/.hypatia` is outside
|
|
221
|
+
the workspace, so `knowledge-create` and friends are denied, escalated, and
|
|
222
|
+
would otherwise stop on a prompt every time the user says "remember this".
|
|
223
|
+
|
|
224
|
+
- **Skills** are bundled: `hypatia-memory` in this plugin's variant (the
|
|
225
|
+
automatic layer writes, the agent retrieves), plus byte-identical copies of
|
|
226
|
+
hypatia's `hypatia` CLI reference and `hypatia-dream`, carried because
|
|
227
|
+
removing `dsh-hypatia` would otherwise take them with it. Another plugin's
|
|
228
|
+
registration of the same name is left alone and reported, since DSH keeps the
|
|
229
|
+
first runtime registration. A copy on disk is registered over but still wins
|
|
230
|
+
in agent sessions — presets load disk skills in a nearer layer — so it is
|
|
231
|
+
reported with its directory.
|
|
232
|
+
|
|
233
|
+
## Configuration
|
|
234
|
+
|
|
235
|
+
Settings namespace `hypatia-auto-memory` (edit `settings.yaml` or Web
|
|
236
|
+
settings; all fields optional, defaults shown):
|
|
237
|
+
|
|
238
|
+
```yaml
|
|
239
|
+
hypatia-auto-memory:
|
|
240
|
+
enabled: true
|
|
241
|
+
binaries: [hypatia]
|
|
242
|
+
transport: mcp # or cli; a binary without `mcp` falls back to cli
|
|
243
|
+
shelf: default # where every entry goes; read at startup (see below)
|
|
244
|
+
autoApprove: true # approve the AGENT's plain `hypatia …` bash calls
|
|
245
|
+
collector:
|
|
246
|
+
enabled: true
|
|
247
|
+
maxAssistantChars: 8000 # per-message cap before truncation marker
|
|
248
|
+
maxUserChars: 32000 # same, for what the user typed or pasted
|
|
249
|
+
toolLedger: true # collapse repeated tool calls
|
|
250
|
+
consolidation:
|
|
251
|
+
enabled: true
|
|
252
|
+
# Each attempt, including a retry, rotates to the next selected route.
|
|
253
|
+
models: [] # select one or more { provider, model } routes
|
|
254
|
+
maxInputTokens: 16000 # transcript cap (chars/4 estimate)
|
|
255
|
+
maxOutputTokens: 2000
|
|
256
|
+
timeoutMs: 120000
|
|
257
|
+
checkEveryTurns: 5 # minimum turns between accepted tasks
|
|
258
|
+
minNewTokens: 3000 # unconsolidated tokens required to trigger
|
|
259
|
+
maxWorkUnitsPerRun: 3
|
|
260
|
+
adjudicate: true # judge how a new work unit relates to nearby ones
|
|
261
|
+
dedupMaxDistance: 0.45 # cosine ceiling on candidates worth judging
|
|
262
|
+
dedupCandidates: 5
|
|
263
|
+
cascade:
|
|
264
|
+
enabled: true
|
|
265
|
+
batchSize: 16 # entries per tier before archiving one tier up
|
|
266
|
+
queue:
|
|
267
|
+
concurrency: 1 # parallel sessions
|
|
268
|
+
maxAttempts: 3
|
|
269
|
+
retryDelayMs: 5000
|
|
270
|
+
# NOT a batching knob. Spans are cut at turn/end; for a turn still running
|
|
271
|
+
# this long, the next step/end flushes what is complete so far.
|
|
272
|
+
flushWindowMs: 120000
|
|
273
|
+
recall:
|
|
274
|
+
enabled: true
|
|
275
|
+
preloadRulesTaboos: true
|
|
276
|
+
housekeeping:
|
|
277
|
+
reconcileOnStartup: true # reset rows whose session has no msg-* left in the shelf
|
|
278
|
+
pruneVanishedSessions: true # drop rows and tasks of sessions DSH no longer has
|
|
279
|
+
backfillConsolidation: true # consolidate a logged tail the in-session trigger never reached
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
The cordis config block on the bundle row only carries skill packaging:
|
|
283
|
+
|
|
284
|
+
```yaml
|
|
285
|
+
- id: hypatia-auto-memory
|
|
286
|
+
config:
|
|
287
|
+
skills: true # register hypatia-memory + hypatia + hypatia-dream
|
|
288
|
+
skillsDir: /abs/path # override the packaged skills/ directory
|
|
289
|
+
```
|
|
290
|
+
|
|
291
|
+
`enabled` and `autoApprove` are read when the plugin starts, so changing either
|
|
292
|
+
needs a profile reload; every other switch applies immediately.
|
|
293
|
+
|
|
294
|
+
### Choosing the shelf
|
|
295
|
+
|
|
296
|
+
`shelf` names the hypatia shelf every entry is written to and every lookup
|
|
297
|
+
reads — logging, consolidation, the cascade, the rules/taboos preload, and the
|
|
298
|
+
startup housekeeping. The settings card offers it as a dropdown of what
|
|
299
|
+
`hypatia list` reports (refreshed about once a minute, and after every settings
|
|
300
|
+
change), marking shelves that are registered but not connected.
|
|
301
|
+
|
|
302
|
+
- **Takes effect after a profile reload**, like `enabled`. A running queue keeps
|
|
303
|
+
writing where it started; the log says so when the setting changes.
|
|
304
|
+
- **Progress is kept per shelf.** Watermarks and queued tasks are stored per
|
|
305
|
+
shelf, so a newly chosen shelf starts every session from zero, and switching
|
|
306
|
+
back resumes exactly where that shelf left off — nothing is re-logged or
|
|
307
|
+
re-consolidated into a shelf that already has it. The cost is on the new
|
|
308
|
+
shelf: every live session is logged from its start there — and every other
|
|
309
|
+
session the next time it is active — and consolidated again, one model call
|
|
310
|
+
per span. Work queued for the other shelf waits, untouched, until it is
|
|
311
|
+
chosen again, and housekeeping (pruning vanished sessions, failed tasks) only
|
|
312
|
+
touches the current shelf's rows. `default` keeps the rows
|
|
313
|
+
written before this setting existed.
|
|
314
|
+
- **The agent follows.** When the shelf is not `default`, the session seed tells
|
|
315
|
+
the agent to pass `--shelf <name>` to its own `hypatia` commands; the bundled
|
|
316
|
+
`hypatia-memory` skill says the same.
|
|
317
|
+
- **A missing shelf is logged at startup**, not refused: writes to it fail and
|
|
318
|
+
retry like any other hypatia failure until it is connected
|
|
319
|
+
(`hypatia connect <dir> --name <name>`). Over MCP, that failure restarts the
|
|
320
|
+
server, so the retry after a `connect` reaches the shelf.
|
|
321
|
+
|
|
322
|
+
The listing reaches the browser as a second, read-only settings namespace,
|
|
323
|
+
`hypatia-auto-memory-shelves`, which the Host re-registers whenever the listing
|
|
324
|
+
changes. No card claims it, so it renders nowhere; nothing ever writes to it.
|
|
325
|
+
|
|
326
|
+
## Operations checklist
|
|
327
|
+
|
|
328
|
+
1. After a turn ends, entries appear within seconds:
|
|
329
|
+
`hypatia knowledge-get msg-<sessionId>-<n>` (`n` counts messages, from 0)
|
|
330
|
+
2. Watermarks live in the state domain; failed tasks stay in the `tasks`
|
|
331
|
+
table with their last error for inspection.
|
|
332
|
+
3. Restart mid-write is safe: messages already stored are skipped
|
|
333
|
+
(get-before-create); uncovered ranges are re-enqueued from watermarks.
|
|
334
|
+
4. To check the route warning: consolidation with no selected `models`
|
|
335
|
+
logs a one-time warning and otherwise stays silent.
|
|
336
|
+
5. Uninstall: `dsh plugin --profile web remove dsh-hypatia-auto-memory` —
|
|
337
|
+
hypatia entries themselves are left in `~/.hypatia/`.
|
|
338
|
+
|
|
339
|
+
## Failure modes
|
|
340
|
+
|
|
341
|
+
| Symptom | Cause / handling |
|
|
342
|
+
|---|---|
|
|
343
|
+
| No entries after chatting | `enabled: false`, missing `hypatia` binary, or the collect fiber PENDING (needs `sessions`, `storageDomain`, `subprocess`, `settings` from the base composition) — check profile logs |
|
|
344
|
+
| Nothing logged for a whole run after a restart | The storage domain refused to open because a stored record failed its schema. The storage service validates on read, not on write, so a bad write only surfaces at the next startup — and one bad row fails the whole domain. Look for `startup failed` / `does not match its schema` in the profile log. Task rows missing `error` are now defaulted; for any other bad row, stop DSH, remove it from `~/.dsh/storages/hypatia_auto_memory.json`, and restart |
|
|
345
|
+
| Entries appear only after a turn finishes | By design: spans are cut at `turn/end`, or at the first `step/end` once a turn has run longer than `flushWindowMs`, so an entry never lacks its own tool results |
|
|
346
|
+
| Logging works, no summaries | `consolidation.models` is empty or invalid — one warning at first trigger |
|
|
347
|
+
| Summaries but no `sum2-*` | Fewer than `cascade.batchSize` unarchived tier-1 summaries in that project yet |
|
|
348
|
+
| Work units have no relationships | No embedding model on the shelf (`similar` fails), or every candidate was beyond `dedupMaxDistance` |
|
|
349
|
+
| Task in `deferred` state | Neither the live store nor persistence could supply its session. Not an error: it spends no attempts and runs as soon as one of them can. Normally storage answers immediately — the state persists only when `sessionPersistence` is absent from the composition, or its read failed (look for `could not be read` in the log) |
|
|
350
|
+
| Task in `failed` state | A hypatia or model error persisted after `maxAttempts`. Failed `log-message` records are pruned at the next startup, since the watermark re-derives their range; other kinds are kept for inspection — delete one to let the next trigger re-create it |
|
|
351
|
+
| Watermark says logged, but the shelf has no entries | The shelf was reset or switched after logging. Handled at startup: `housekeeping.reconcileOnStartup` resets any row whose session has no `msg-*` left, and that session is re-logged — and re-consolidated — from the start on its next activity. A shelf query that fails leaves the row untouched. A session whose messages were all deleted on purpose is indistinguishable and is logged again; turn the switch off if that matters |
|
|
352
|
+
| Warning `hypatia mcp unavailable: …; using the hypatia CLI until the profile reloads` | The binary predates `hypatia mcp` (the message quotes its `unrecognized subcommand`), lacks a tool the plugin calls, or answered the handshake with an error. Everything keeps working over the CLI; upgrade hypatia and reload the profile to use MCP |
|
|
353
|
+
| Every call fails with `hypatia mcp exited …` or `timed out` | The binary starts but its MCP server does not come up — a wrapper script in `binaries` that changes clap's exit code or wording is not recognised as an old binary. Set `transport: cli` |
|
|
354
|
+
| Every write fails right after changing `shelf` | The shelf is not registered or not connected — startup logs `shelf "<name>" is not registered`. Connect it (`hypatia connect <dir> --name <name>`) and reload the profile, or pick another |
|
|
355
|
+
| The agent searches `default` while the plugin writes elsewhere | A disk copy of `hypatia-memory` (see below) replaced the bundled skill, or recall is disabled, so nothing told the agent which shelf to use |
|
|
356
|
+
| Duplicate `msg-*` after weird manual edits | Delete the entry in hypatia and lower `lastLoggedSeq` for that session in the state domain — backfill recreates it once |
|
|
357
|
+
| The agent's own `hypatia` write still asks for approval | `autoApprove: false` (needs a profile reload to change), the command pipes/redirects/chains outside quotes, or its first word is not one of `binaries` — only plain calls are answered, by design. Reads never reach approval at all, so nothing to fix there |
|
|
358
|
+
| The agent got a memory protocol that tells it to log messages by hand | Another plugin registered `hypatia-memory` first (`dsh-hypatia` still in the profile), or a `hypatia-memory` exists on disk — typically `~/.agents/skills/hypatia-memory`, written by `hypatia skill install --agent codex`. Agent presets load disk skills in a layer nearer than plugins, so it wins in sessions even though the skill center may list this plugin. Remove the copy the startup warning names |
|
|
359
|
+
| Startup warnings never appear in the terminal | `dsh web` mounts no log exporter, so plugin log lines of any level go nowhere; the console exporter's default threshold would also drop warnings (warn is level 2, above info's 1). Mount a logger such as `dsh-logbook` or `dsh-boot-doctor` temporarily to read them |
|
|
360
|
+
| Project scope looks wrong | Scope = basename of the session cwd's git top level (`git rev-parse --show-toplevel`), or of the cwd itself when git finds no work tree or cannot answer (not installed, a repo it refuses as unsafe). A `rev-parse` slower than 3 s leaves that one session on the cwd's own name. Two same-named checkouts share a scope by design; a linked worktree is scoped by its own directory, not the main checkout's; git resolves symlinks, so a checkout opened through a link named differently from its target gets the target's name. A name hypatia would rewrite is normalized first: a session at `/` is scoped `/`, commas become `_`, surrounding whitespace goes. Entries written before these fixes stay where they were stored: a subdirectory session's under that directory's name (`repo/src` wrote under `src` — the git root was never read), a session at `/` with no scope, `a,b` under both `a` and `b`, `foo,` under `foo` and global. Padded names were stored trimmed, which is what the query now asks for. Nothing migrates them; `knowledge-update --scopes` moves one entry and keeps its `created_at` |
|
|
361
|
+
|
|
362
|
+
## Known limitations
|
|
363
|
+
|
|
364
|
+
Deliberate, and worth knowing before you rely on them:
|
|
365
|
+
|
|
366
|
+
- **No topic-shift detection.**
|
|
367
|
+
[`memory-nolinear.md`](https://github.com/tkliuxing/hypatia/blob/main/docs/memory-nolinear.md)
|
|
368
|
+
ranks topic switches as the strongest session-splitting signal, above task
|
|
369
|
+
boundaries and time gaps. Detecting one needs a model call every turn, which contradicts this
|
|
370
|
+
plugin's central trade-off (zero model work in the main session), so it is not
|
|
371
|
+
implemented. Task boundaries (`turn/end` reasons) and session close are
|
|
372
|
+
honoured; a session covering three unrelated topics is summarised as one.
|
|
373
|
+
- **Summary names are mechanical.** The protocol asks for descriptive names
|
|
374
|
+
extracted from content. A model-authored title cannot be the key — a retry may
|
|
375
|
+
word it differently and silently fork the entry — so keys stay derivable
|
|
376
|
+
(`sum-<session>-<from>-<to>`, `sum<N>-<digest>`) and the descriptive title is
|
|
377
|
+
the first heading in the body.
|
|
378
|
+
- **No `session-<id>` node without a host summary.** The protocol forbids
|
|
379
|
+
inventing one, and DSH emits no session-summary event of its own; the node is
|
|
380
|
+
built from a `session/title` or `compaction/summary` when one appears.
|
|
381
|
+
- **The plugin does not use `knowledge-update` yet.** hypatia gained it in #20
|
|
382
|
+
(it keeps `created_at`, discards the old vector and re-embeds on the next
|
|
383
|
+
flush), but older binaries lack it, so the plugin still never edits an entry
|
|
384
|
+
in place: a second session title does not replace the first, and a work unit
|
|
385
|
+
is never rewritten.
|
|
386
|
+
- **Duplicate `knowledge-create` is detected by error text.** hypatia has no
|
|
387
|
+
upsert for knowledge entries, so `hypatia-cli.js` recognises
|
|
388
|
+
`UNIQUE constraint failed:` / `duplicate key` to make a replay idempotent.
|
|
389
|
+
`statement-create` has been idempotent on its own since hypatia #20 (a repeat
|
|
390
|
+
exits 0 with `Statement already exists`); on an older binary the same
|
|
391
|
+
error-text path covers it. The matching is confined to one function, and
|
|
392
|
+
`test/integration` accepts either statement behaviour.
|
|
393
|
+
- **Adjudication and dedup need an embedding model.** On a shelf without one,
|
|
394
|
+
`similar` fails outright and work units are stored with no relationship —
|
|
395
|
+
never dropped.
|
|
396
|
+
- **Boot backfill reads live sessions.** A persisted-but-not-loaded session's
|
|
397
|
+
gap is picked up when it next loads.
|
|
398
|
+
- **A cascade crash mid-batch shifts a grouping boundary.** Members already
|
|
399
|
+
linked leave the tier's unarchived set, so the retry archives a different
|
|
400
|
+
batch. Everything is still archived exactly once; the grouping is just not the
|
|
401
|
+
one the first attempt intended.
|
|
402
|
+
- **Two of the three bundled skills are vendored copies.** `hypatia` and
|
|
403
|
+
`hypatia-dream` are byte-identical copies of the [hypatia
|
|
404
|
+
repository](https://github.com/tkliuxing/hypatia)'s `skills/`, because a
|
|
405
|
+
published package cannot reach outside itself. Nothing refreshes them: when
|
|
406
|
+
the originals change upstream, copy them in by hand.
|
|
407
|
+
- **`enabled: false` at the top level is read at plugin startup**; toggling it
|
|
408
|
+
live requires a profile reload, while the per-feature switches apply
|
|
409
|
+
immediately.
|
|
410
|
+
|
|
411
|
+
## Directory structure
|
|
412
|
+
|
|
413
|
+
```
|
|
414
|
+
dsh-hypatia-auto-memory/
|
|
415
|
+
├── package.json # bundle + client manifest, deps
|
|
416
|
+
├── cordis.patch.yml # bundle layer inserting id=hypatia-auto-memory
|
|
417
|
+
├── tsconfig.json # browser TS/TSX type-check config
|
|
418
|
+
├── tsconfig.build.json # declaration emit for client bundle
|
|
419
|
+
├── tsdown.config.ts # browser CJS factory build
|
|
420
|
+
├── src/
|
|
421
|
+
│ ├── index.js # fiber composition (collect + optional children)
|
|
422
|
+
│ ├── config.js # settings namespace, defaults, live updates
|
|
423
|
+
│ ├── shelf.js # per-shelf table views, `hypatia list`, shelf inventory
|
|
424
|
+
│ ├── state.js # storageDomain spec + progress helpers
|
|
425
|
+
│ ├── collector.js # session/event filtering, ledger, backfill
|
|
426
|
+
│ ├── content-policy.js # redaction, dates, caps, slugs (pure)
|
|
427
|
+
│ ├── queue.js # durable per-session queue with retry/dispose
|
|
428
|
+
│ ├── hypatia-client.js # transport switch: MCP, CLI fallback
|
|
429
|
+
│ ├── hypatia-mcp.js # private `hypatia mcp` connection + result mapping
|
|
430
|
+
│ ├── hypatia-cli.js # argv-only subprocess wrapper
|
|
431
|
+
│ ├── writer.js # idempotent get-before-create writes
|
|
432
|
+
│ ├── consolidator.js # thresholds, prompt, llm.stream, validation
|
|
433
|
+
│ ├── cascade.js # log₁₆(n) hierarchical summary archive
|
|
434
|
+
│ ├── recall.js # rules/taboos preload at session start
|
|
435
|
+
│ ├── auto-approve.js # approves the agent's own plain bash hypatia calls
|
|
436
|
+
│ ├── skills.js # bundled skill registration (never shadows another provider)
|
|
437
|
+
│ ├── status.js # counters + structured logging
|
|
438
|
+
│ └── client/ # browser settings card
|
|
439
|
+
│ ├── index.tsx # client plugin entry + slot registration
|
|
440
|
+
│ ├── SettingsCard.tsx
|
|
441
|
+
│ ├── shelves.ts # shelf dropdown choices from the inventory namespace
|
|
442
|
+
│ └── slot-contract.ts
|
|
443
|
+
├── lib/
|
|
444
|
+
│ └── client.js # built browser factory (commit this)
|
|
445
|
+
├── skills/
|
|
446
|
+
│ ├── hypatia-memory/ # this plugin's variant (retrieval is the agent's)
|
|
447
|
+
│ ├── hypatia/ # vendored from hypatia's skills/
|
|
448
|
+
│ └── hypatia-dream/ # vendored from hypatia's skills/
|
|
449
|
+
├── scripts/
|
|
450
|
+
│ └── it-shelf.sh # throwaway shelf for manual CLI probing
|
|
451
|
+
├── test/ # node:test units (npm test)
|
|
452
|
+
│ └── integration/ # contracts against a real hypatia (npm run test:integration)
|
|
453
|
+
└── README.md
|
|
454
|
+
```
|
package/cordis.patch.yml
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
# dsh-hypatia-auto-memory bundle patch: inserts the plugin row into the
|
|
2
|
+
# profile roster over dsh-base. Installed with
|
|
3
|
+
# dsh plugin --profile <name> add dsh-hypatia-auto-memory (registry)
|
|
4
|
+
# dsh plugin --profile <name> add link:/path/to/dsh-hypatia-auto-memory
|
|
5
|
+
#
|
|
6
|
+
# Behavior is configured through the `hypatia-auto-memory` settings namespace
|
|
7
|
+
# (settings.yaml / Web settings), NOT through this config block. The only
|
|
8
|
+
# cordis-level knobs are the bundled skill:
|
|
9
|
+
#
|
|
10
|
+
# - id: hypatia-auto-memory
|
|
11
|
+
# config:
|
|
12
|
+
# skills: true # register hypatia-memory + hypatia + hypatia-dream
|
|
13
|
+
# skillsDir: /abs/path # override the packaged skills/ directory
|
|
14
|
+
#
|
|
15
|
+
# This plugin replaces dsh-hypatia: it carries that plugin's skills and its
|
|
16
|
+
# auto-approval for the agent's own bash `hypatia` calls. Remove dsh-hypatia
|
|
17
|
+
# from the profile rather than running both.
|
|
18
|
+
- insert:
|
|
19
|
+
- id: hypatia-auto-memory
|
|
20
|
+
name: dsh-hypatia-auto-memory
|