dsh-hypatia-auto-memory 2.3.0 → 2.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  > 中文:[README.zh-CN.md](./README.zh-CN.md)
4
4
 
5
- Event-driven [Hypatia](https://github.com/tkliuxing/hypatia) memory for
5
+ Event-driven [Hypatia](https://github.com/MarchLiu/hypatia) memory for
6
6
  [DSH](https://github.com/deepseek-ai/deepseek-harness): automatic conversation
7
7
  logging and background consolidation — span summaries, a log₁₆(n) archive
8
8
  cascade, and adjudicated work-unit extraction — all on a **dedicated model
@@ -225,6 +225,18 @@ agent/created ──▶ rules/taboos inject()
225
225
  do have text/image content. In either mode **tool outputs themselves are never
226
226
  stored** in the log layer.
227
227
 
228
+ - **A message the user took back is not remembered.** `dsh-rewind-plugin`
229
+ (`/rewind`) and the two `plugin:rollback` plugins hide messages from the model
230
+ by appending a surface replacement; the log is append-only, so the messages
231
+ themselves stay in the session. This plugin reads that record: a hidden message
232
+ is not written, and one an earlier flush already wrote is deleted from the
233
+ shelf. The sources are a whitelist of producers verified to mean "the user
234
+ discarded this branch" — DSH's own compaction is not on it, because what it
235
+ folds is history the user keeps, and neither is any editor plugin. Two limits:
236
+ a summary written before the rewind still carries the withdrawn text
237
+ (summaries are never rewritten), and a rewind that lands after the session's
238
+ last flush is not seen at all. See `docs/rewind-memory-consistency.md`.
239
+
228
240
  - **The log layer is not embedded.** `msg-*` and `session-*` entries and the
229
241
  `belongTo` / `summary` edges are written `embed: false` (hypatia #26): stored,
230
242
  full-text indexed and reachable by JSE, but given no vector. The protocol has
@@ -380,7 +392,7 @@ agent/created ──▶ rules/taboos inject()
380
392
  - **Recall** preloads project/global rules and taboos at session start. Nothing
381
393
  else is pushed, except one line naming the shelf when it is not `default`, so
382
394
  the agent's own `hypatia` calls look where the plugin writes. Retrieval is the agent's job through the bundled skill, which
383
- is what [`memory-nolinear.md`](https://github.com/tkliuxing/hypatia/blob/main/docs/memory-nolinear.md)
395
+ is what [`memory-nolinear.md`](https://github.com/MarchLiu/hypatia/blob/main/docs/memory-nolinear.md)
384
396
  prescribes for agents that hold context and can call tools; the previous
385
397
  per-turn injector also vetoed DSH's own runtime context section by returning
386
398
  from `agent/pre-step` without calling `next()`.
@@ -668,7 +680,7 @@ now only projects plugin Config forms.)
668
680
  Deliberate, and worth knowing before you rely on them:
669
681
 
670
682
  - **No topic-shift detection.**
671
- [`memory-nolinear.md`](https://github.com/tkliuxing/hypatia/blob/main/docs/memory-nolinear.md)
683
+ [`memory-nolinear.md`](https://github.com/MarchLiu/hypatia/blob/main/docs/memory-nolinear.md)
672
684
  ranks topic switches as the strongest session-splitting signal, above task
673
685
  boundaries and time gaps. Detecting one needs a model call every turn, which contradicts this
674
686
  plugin's central trade-off (zero model work in the main session), so it is not
@@ -711,7 +723,7 @@ Deliberate, and worth knowing before you rely on them:
711
723
  one the first attempt intended.
712
724
  - **Two of the three bundled skills are vendored copies.** `hypatia` and
713
725
  `hypatia-dream` are copied from the [hypatia
714
- repository](https://github.com/tkliuxing/hypatia)'s `skills/`, because a
726
+ repository](https://github.com/MarchLiu/hypatia)'s `skills/`, because a
715
727
  published package cannot reach outside itself. Nothing refreshes them: when
716
728
  the originals change upstream, copy them in by hand. `hypatia` is
717
729
  byte-identical. `hypatia-dream` carries two local patches, which a refresh
package/README.zh-CN.md CHANGED
@@ -3,7 +3,7 @@
3
3
  > 本文件是 [README.md](./README.md) 的中文版。若两者不一致,以英文版为准。
4
4
 
5
5
  面向 [DSH](https://github.com/deepseek-ai/deepseek-harness) 的事件驱动
6
- [Hypatia](https://github.com/tkliuxing/hypatia) 记忆:自动记录对话、后台巩固
6
+ [Hypatia](https://github.com/MarchLiu/hypatia) 记忆:自动记录对话、后台巩固
7
7
  (span 摘要、log₁₆(n) 归档级联、带裁决的工作单元抽取)——全部走**专用模型路由**,
8
8
  主会话零模型工作。
9
9
 
@@ -180,6 +180,15 @@ agent/created ──▶ rules/taboos inject()
180
180
  文本/图片内容的助手消息记录一份精简台账(`1. \`read\` ×5 — ✅ 1.2s total`)。无论
181
181
  哪种模式,**工具输出本身永远不会存入**日志层。
182
182
 
183
+ - **被用户撤回的消息不会记入记忆。** `dsh-rewind-plugin`(`/rewind`)与两个
184
+ `plugin:rollback` 插件都靠追加一条 surface 替换,把消息从模型视野里藏起来;而日志
185
+ 是 append-only 的,消息本身仍留在会话里。本插件读这条记录:被藏起来的消息不写,
186
+ 早先的 flush 已经写出的则从 shelf 上删掉。来源是一份白名单,只收语义确实是"用户丢
187
+ 弃了这个分支"的生产者——DSH 自己的压缩不在其中(它折叠的是用户保留的历史),编辑
188
+ 类插件也不在其中。两个边界:回退之前已经写出的摘要仍带着被撤回的原文(摘要从不重
189
+ 写),以及落在会话最后一次 flush 之后的回退完全看不到。见
190
+ `docs/rewind-memory-consistency.md`。
191
+
183
192
  - **日志层不嵌入。** `msg-*` 和 `session-*` 条目以及 `belongTo` / `summary` 边都以
184
193
  `embed: false` 写入(hypatia #26):已存储、可全文检索、可被 JSE 找到,但不生成
185
194
  向量。协议设计了沿 `summary` 边从摘要精确下钻的召回方式,而不是按语义命中原始
@@ -292,7 +301,7 @@ agent/created ──▶ rules/taboos inject()
292
301
  - **Recall** 在会话启动时预载项目/全局的 rules 和 taboos。除此之外什么都不推送,
293
302
  除了一行在 shelf 不是 `default` 时指明 shelf 名字,好让 Agent 自己的 `hypatia`
294
303
  调用去插件写入的地方找。检索由 Agent 通过随包技能完成,这正是
295
- [`memory-nolinear.md`](https://github.com/tkliuxing/hypatia/blob/main/docs/memory-nolinear.md)
304
+ [`memory-nolinear.md`](https://github.com/MarchLiu/hypatia/blob/main/docs/memory-nolinear.md)
296
305
  为那些持有上下文、能调用工具的 Agent 所开出的方案;此前的每轮注入器还会因为从
297
306
  `agent/pre-step` 返回而不调用 `next()`,否决掉 DSH 自己的运行时上下文段。
298
307
 
@@ -525,7 +534,7 @@ shelf 清单本来就不是配置,而设置域现在只投影插件的 Config
525
534
  刻意为之,依赖它们之前值得了解:
526
535
 
527
536
  - **没有话题切换检测。**
528
- [`memory-nolinear.md`](https://github.com/tkliuxing/hypatia/blob/main/docs/memory-nolinear.md)
537
+ [`memory-nolinear.md`](https://github.com/MarchLiu/hypatia/blob/main/docs/memory-nolinear.md)
529
538
  把话题切换列为最强的会话切分信号,排在任务边界和时间间隔之上。检测它需要每个 turn
530
539
  一次模型调用,这与本插件的核心取舍(主会话零模型工作)相悖,所以未实现。任务边界
531
540
  (`turn/end` 原因)和会话关闭是被尊重的;一个覆盖三个无关话题的会话会被总结成一条。
@@ -553,7 +562,7 @@ shelf 清单本来就不是配置,而设置域现在只投影插件的 Config
553
562
  - **级联在批次中途崩溃会移动分组边界。** 已连接的成员离开该层的未归档集合,所以
554
563
  重试会归档另一批。每样东西仍恰好归档一次;只是分组不再是第一次尝试想要的。
555
564
  - **三个随包技能里有两个是 vendored 拷贝。** `hypatia` 和 `hypatia-dream` 从
556
- [hypatia 仓库](https://github.com/tkliuxing/hypatia) 的 `skills/` 拷贝而来,因为
565
+ [hypatia 仓库](https://github.com/MarchLiu/hypatia) 的 `skills/` 拷贝而来,因为
557
566
  已发布的包够不到自身之外。没有任何东西刷新它们:上游原件变了就手工拷进来。
558
567
  `hypatia` 逐字节相同。`hypatia-dream` 带两处本地补丁,刷新时必须重新打:
559
568
  - **Shelf。** 请求里没给 shelf 时,它用会话种子指明的那个,只有种子没指明时才用
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-hypatia-auto-memory",
3
- "version": "2.3.0",
3
+ "version": "2.4.0",
4
4
  "description": "Event-driven Hypatia memory for DSH: automatic conversation logging via session events, queued background consolidation (span summaries + work-unit extraction) with a dedicated model route, and recall injection — zero per-call model work in the main session.",
5
5
  "icon": "./icon.svg",
6
6
  "type": "module",
package/src/collector.js CHANGED
@@ -136,6 +136,158 @@ export function countLoggableMessages(events) {
136
136
  return count
137
137
  }
138
138
 
139
+ /**
140
+ * Event sources whose surface replacement means "the user took this branch
141
+ * back", as opposed to the producers that fold history without discarding it.
142
+ *
143
+ * Only a `replace` event carrying one of these `data.source.kind` values is a
144
+ * withdrawal (docs/rewind-memory-consistency.md §2.2–§2.4). The distinction is
145
+ * load-bearing, because every other producer's replacement is real history:
146
+ *
147
+ * - `compact-checkpoint` — DSH's own compaction, and `dsh-clear-mind`'s wrapper
148
+ * around the same seam: the folded region is summarised for the model, never
149
+ * discarded by the user.
150
+ * - `system-prompt` — the runtime-context snapshot (`system/message`) replaced
151
+ * by its own next version; observed on a real log as seq 395/396.
152
+ * - `context-editor` — an edit or condensation, whose own restore action brings
153
+ * the messages back.
154
+ *
155
+ * The list stays a whitelist on purpose. A denylist of "known non-withdrawal
156
+ * producers" fails open: the next plugin that uses `replace` for an edit would
157
+ * have its history deleted as if the user had taken it back. Members are added
158
+ * only after reading the producer's code and confirming the semantics, and each
159
+ * one gets a test in test/collector-withdrawn.test.mjs.
160
+ *
161
+ * Audited 2026-10-14: `dsh-rewind-plugin` (`dsh-rewind`) and the two rollback
162
+ * plugins that share `plugin:rollback` (`@domitor-syh/dsh-rollback`,
163
+ * `@nianchu/dsh-rollback`).
164
+ *
165
+ * The kind alone is not enough — a member also has to cite **exactly** the nodes
166
+ * it shadowed, see {@link withdrawnRanges}. `plugin:rollback` is the generic
167
+ * `plugin:<name>` spelling, so any future plugin calling itself `rollback`
168
+ * matches it too. Accepted knowingly: a rename shows up as a test failure,
169
+ * whereas a narrower string would silently stop matching the plugin it was
170
+ * written for.
171
+ */
172
+ export const WITHDRAWAL_SOURCE_KINDS = Object.freeze(['dsh-rewind', 'plugin:rollback'])
173
+
174
+ /**
175
+ * The withdrawn ranges a log slice contains, one entry per withdrawal marker.
176
+ *
177
+ * What a marker hid comes from its `sourceEventSeqs`, and the host only requires
178
+ * that list to **include** every shadowed node: citing extra, earlier seqs is
179
+ * legal (`assertSourceEventReferences` checks for `missing`, not equality).
180
+ * Everything cited here is deleted, so this reads exactly as far as the producer
181
+ * can be trusted — which is a per-member property, verified by reading each
182
+ * member of {@link WITHDRAWAL_SOURCE_KINDS}: `dsh-rewind-plugin` passes
183
+ * `[...plan.shadowedSeqs]` (its own `surface.slice(targetIndex)`), and the
184
+ * rollback plugins pass `seqsForStorage(shadowed)` — that same set, reordered
185
+ * for three or more seqs so the store's range encoder cannot compress it, which
186
+ * is why the **set**, not the order, is what this keeps.
187
+ *
188
+ * A new member needs that check too (docs/rewind-memory-consistency.md §2.4): a
189
+ * producer that also cited, say, the anchor it rewound to would have that message
190
+ * deleted here.
191
+ *
192
+ * @param {readonly any[]} events - a log slice that may contain markers. Callers
193
+ * must read to `session.seq`, not merely to their own task range: a marker's
194
+ * seq is greater than every seq it shadows, so it can sit past the range being
195
+ * logged (docs/rewind-memory-consistency.md §3 fact 5).
196
+ * @returns {Array<{from: number, to: number, at: number, kind: string, seqs: Set<number>}>}
197
+ */
198
+ export function withdrawnRanges(events) {
199
+ const ranges = []
200
+ for (const event of events ?? []) {
201
+ const op = event?.surfaceOp
202
+ if (op?.op !== 'replace') continue
203
+ // Only a user message can carry these markers. This also rules out the
204
+ // runtime-context snapshot (`system/message`) and the tool-result pruner
205
+ // (`tool/result`), both of which replace a single node with itself.
206
+ if (event.type !== 'user/message') continue
207
+ const kind = event.data?.source?.kind
208
+ if (typeof kind !== 'string' || WITHDRAWAL_SOURCE_KINDS.includes(kind) === false) continue
209
+ const cited = Array.isArray(event.sourceEventSeqs) ? event.sourceEventSeqs : []
210
+ const seqs = new Set(cited.filter((seq) => Number.isInteger(seq)))
211
+ // The host refuses an empty list on a replacement, so a marker citing nothing
212
+ // is one this build cannot read. Reading `[startSeq, endSeq]` instead would
213
+ // be a guess, and wrong in the case that matters most: those two numbers
214
+ // denote SURFACE order, so a slice containing a compaction checkpoint can
215
+ // have `startSeq > endSeq`.
216
+ if (seqs.size === 0) continue
217
+ ranges.push({ from: op.startSeq, to: op.endSeq, at: event.seq, kind, seqs })
218
+ }
219
+ return ranges
220
+ }
221
+
222
+ /** Every shadowed seq of every marker in the slice, as one lookup set. */
223
+ export function withdrawnSeqs(events) {
224
+ const seqs = new Set()
225
+ for (const range of withdrawnRanges(events)) {
226
+ for (const seq of range.seqs) seqs.add(seq)
227
+ }
228
+ return seqs
229
+ }
230
+
231
+ /**
232
+ * The dense entry ordinal each of `seqs` was named by — `msg-<session>-<N>`.
233
+ *
234
+ * Same count as `countLoggableMessages`, over the same prefix, so a name derived
235
+ * here cannot drift from the name the span wrote: a withdrawal that arrives
236
+ * after its messages were logged has to retract them by name, and the only
237
+ * durable record of that name is the log itself.
238
+ *
239
+ * @param {readonly any[]} events - events covering the counting prefix, i.e. from
240
+ * the inherited prefix through the shadowed seqs.
241
+ * @param {Set<number>} seqs - shadowed seqs, as `withdrawnSeqs` returns them.
242
+ * @param {number} [inherited] - size of the prefix a fork inherited, which
243
+ * contributes no ordinals of its own.
244
+ * @returns {Map<number, number>} seq -> ordinal. A shadowed seq that is not a
245
+ * loggable message is absent: it never had an entry.
246
+ */
247
+ export function withdrawnOrdinals(events, seqs, inherited = 0) {
248
+ const ordinals = new Map()
249
+ if (seqs === undefined || seqs.size === 0) return ordinals
250
+ let index = 0
251
+ for (const event of events ?? []) {
252
+ if (event.seq < inherited || isLoggableMessage(event) === false) continue
253
+ if (seqs.has(event.seq)) ordinals.set(event.seq, index)
254
+ index += 1
255
+ }
256
+ return ordinals
257
+ }
258
+
259
+ /**
260
+ * What one log task must do about the withdrawals it can see — the whole
261
+ * decision, as a pure function of the log, so the executor only carries it out.
262
+ *
263
+ * A withdrawal has two faces, and they are told apart by position rather than by
264
+ * state: a shadowed message *inside* the range being written must not be written
265
+ * at all, while one *before* it was written by an earlier span and has to be
266
+ * retracted by name. Which of the two it is, is `seq < fromSeq`.
267
+ *
268
+ * @param {{log: readonly any[], prefix: readonly any[], fromSeq: number, inherited?: number}} input
269
+ * `log` is the marker window: from the start of the range being written to the
270
+ * end of the session log, because a marker's seq is greater than every seq it
271
+ * shadows and a window cut at the task's own end can miss one. `prefix` is
272
+ * `[inherited, fromSeq)`, where the messages an earlier span already wrote are
273
+ * counted — the same prefix `baseIndex` counts, so a retraction names the entry
274
+ * the span named.
275
+ * @returns {{seqs: Set<number>, retract: Map<number, number>}} `seqs` are the
276
+ * shadowed seqs in the window (in-span members are matched against the items
277
+ * about to be written); `retract` maps each shadowed seq *before* `fromSeq` to
278
+ * the dense entry ordinal it was named by. Shadowed seqs that are not loggable
279
+ * messages appear in neither: they never had an entry.
280
+ */
281
+ export function withdrawalWork({ log, prefix, fromSeq, inherited = 0 }) {
282
+ const seqs = withdrawnSeqs(log)
283
+ if (seqs.size === 0) return { seqs, retract: new Map() }
284
+ const before = new Set([...seqs].filter((seq) => seq < fromSeq))
285
+ return {
286
+ seqs,
287
+ retract: before.size === 0 ? new Map() : withdrawnOrdinals(prefix, before, inherited),
288
+ }
289
+ }
290
+
139
291
  /**
140
292
  * @param {{
141
293
  * ctx: import('@deepseek-ai/cordis').Context,
@@ -15,7 +15,7 @@
15
15
  */
16
16
 
17
17
  import { absolutizeDates, blocksToText, eventDate, redactSecrets } from './content-policy.js'
18
- import { countLoggableMessages, isCompactionReplacement, isLoggableMessage } from './collector.js'
18
+ import { countLoggableMessages, isCompactionReplacement, isLoggableMessage, withdrawnSeqs } from './collector.js'
19
19
  import { messageName, summaryName } from './writer.js'
20
20
  import { EMPTY_PROGRESS, advanceProgress } from './progress.js'
21
21
  import { TaskDeferredError } from './queue.js'
@@ -251,13 +251,15 @@ function consolidationInstruction(maxWorkUnits) {
251
251
  * @param {{maxInputTokens: number}} config
252
252
  * @param {Date} now - base for relative dates in an event that carries no
253
253
  * timestamp; otherwise each line is resolved against its own event's time.
254
- * @param {{before?: any}} [context] - the event just ahead of the span, so a
255
- * leading compaction replacement is recognised (it restates an old result).
254
+ * @param {{before?: any, withdrawn?: Set<number>}} [context] - the event just
255
+ * ahead of the span, so a leading compaction replacement is recognised (it
256
+ * restates an old result); and the seqs a withdrawal marker took out of the
257
+ * model's view, which must not reach the summariser at all.
256
258
  * @returns {{text: string, lastSeq: number, complete: boolean}} `lastSeq` is the
257
259
  * seq of the last event included; `complete` is false when the budget forced
258
260
  * later events out of this run.
259
261
  */
260
- export function buildTranscript(events, config, now = new Date(), { before } = {}) {
262
+ export function buildTranscript(events, config, now = new Date(), { before, withdrawn } = {}) {
261
263
  const bySeq = new Map(events.map((event) => [event.seq, event]))
262
264
  // Tool results cite their call by id; the readable name lives on the call.
263
265
  const toolNames = new Map()
@@ -272,6 +274,12 @@ export function buildTranscript(events, config, now = new Date(), { before } = {
272
274
  /** @type {Array<{seq: number, line: string}>} */
273
275
  const entries = []
274
276
  for (const event of events) {
277
+ // A withdrawn message never reaches the summariser. Retracting it is only
278
+ // half the job: a summary written from it would carry the branch the user
279
+ // took back into the summary layer, where no later delete can reach it,
280
+ // because summaries are immutable and named by span
281
+ // (docs/rewind-memory-consistency.md §4.4).
282
+ if (withdrawn?.has(event.seq) === true) continue
275
283
  if (event.type === 'user/message') {
276
284
  if (event.data?.source?.kind !== 'user') continue
277
285
  const text = clean(blocksToText(event.data?.content ?? []), event)
@@ -696,7 +704,13 @@ export function createConsolidator({ queue, progress, sessions, llm, cli, writer
696
704
  const events = session.snapshotEvents(fromSeq, task.toSeq)
697
705
  const now = new Date()
698
706
  const before = fromSeq > 0 ? session.snapshotEvents(fromSeq - 1, fromSeq)[0] : undefined
699
- const { text: transcript, lastSeq, complete } = buildTranscript(events, consolidation, now, { before })
707
+ // Read the markers to `session.seq` rather than to `task.toSeq`: a marker's
708
+ // seq is greater than every seq it shadows, so this span's upper bound — and
709
+ // here that is `min(session.seq, lastLoggedSeq)` — can cut one off. Same
710
+ // visibility problem as the log executor (docs/rewind-memory-consistency.md
711
+ // §3 fact 5).
712
+ const withdrawn = withdrawnSeqs(session.snapshotEvents(fromSeq, session.seq ?? 0))
713
+ const { text: transcript, lastSeq, complete } = buildTranscript(events, consolidation, now, { before, withdrawn })
700
714
  if (transcript.trim() === '') {
701
715
  await advanceProgress(progress, task.sessionId, () => ({ lastConsolidatedSeq: task.toSeq }))
702
716
  return
@@ -829,7 +843,13 @@ export function createConsolidator({ queue, progress, sessions, llm, cli, writer
829
843
  const items = []
830
844
  for (const event of events) {
831
845
  if (!isLoggableMessage(event)) continue
832
- if (event.seq < coveredToSeq) items.push(messageName(task.sessionId, itemIndex))
846
+ // A withdrawn message has no entry to cite, and `items` is by contract the
847
+ // list of entries the span actually produced (writer.js `writeSummary`), so
848
+ // the ordinal is still counted but the name is not asserted. Counting it
849
+ // keeps the remaining ordinals aligned with the log.
850
+ if (event.seq < coveredToSeq && withdrawn.has(event.seq) === false) {
851
+ items.push(messageName(task.sessionId, itemIndex))
852
+ }
833
853
  itemIndex += 1
834
854
  }
835
855
  await writer.writeSummary({
@@ -183,13 +183,19 @@ export function createHypatiaCli(ctx, config) {
183
183
  * about `--no-embed` — they gained it together, but the answer is only worth
184
184
  * having if it is true of each command it gates.
185
185
  *
186
- * @returns {Promise<{noEmbed: boolean, similarFilters: boolean}>}
186
+ * @returns {Promise<{noEmbed: boolean, similarFilters: boolean, deleteKnowledge: boolean}>}
187
187
  */
188
188
  async function probeFeatures() {
189
- const [knowledge, statement, similar] = await Promise.all([
189
+ const [knowledge, statement, similar, remove] = await Promise.all([
190
190
  run(['knowledge-create', '--help']),
191
191
  run(['statement-create', '--help']),
192
192
  run(['similar', '--help']),
193
+ // Asked like the others, answered differently: a binary from before
194
+ // `knowledge-delete` existed (hypatia #22) prints a clap usage error and
195
+ // exits 2, and that is an answer ("cannot delete"), not a broken binary —
196
+ // the three required probes above still decide whether the client works at
197
+ // all. A withdrawal is therefore skipped rather than retracted there.
198
+ run(['knowledge-delete', '--help']),
193
199
  ])
194
200
  for (const result of [knowledge, statement, similar]) {
195
201
  if (result.exitCode !== 0) {
@@ -203,6 +209,7 @@ export function createHypatiaCli(ctx, config) {
203
209
  return {
204
210
  noEmbed: knowledge.stdout.includes('--no-embed') && statement.stdout.includes('--no-embed'),
205
211
  similarFilters: similar.stdout.includes('--exclude-tags'),
212
+ deleteKnowledge: remove.exitCode === 0,
206
213
  }
207
214
  }
208
215
 
@@ -216,7 +223,7 @@ export function createHypatiaCli(ctx, config) {
216
223
  // with its own error; the answer here only decides which flags that
217
224
  // call carries, and the conservative answer is none.
218
225
  featuresProbe = undefined
219
- return { noEmbed: false, similarFilters: false }
226
+ return { noEmbed: false, similarFilters: false, deleteKnowledge: false }
220
227
  })
221
228
  }
222
229
  return featuresProbe
@@ -281,6 +288,31 @@ export function createHypatiaCli(ctx, config) {
281
288
  return runCreate(argv)
282
289
  },
283
290
 
291
+ /**
292
+ * Remove one knowledge entry.
293
+ *
294
+ * Deleting a name that is already gone exits 1 with `Error: not found:
295
+ * knowledge '<name>'` — a normal outcome for a caller whose retraction may
296
+ * be replayed, so it is reported as `false` instead of thrown. Statements
297
+ * mentioning the name are deliberately kept by hypatia
298
+ * (`wu-knowledge-delete-retains-statements-7487bf50`), which is why a
299
+ * withdrawal leaves `belongTo` / `summary` edges behind and nothing here
300
+ * tries to remove them.
301
+ *
302
+ * @returns {Promise<boolean>} true when this call removed an entry.
303
+ */
304
+ async knowledgeDelete(name, shelf = shelfOf()) {
305
+ const result = await run(['knowledge-delete', name, '--shelf', shelf])
306
+ if (result.exitCode === 0) return true
307
+ const stderr = result.stderr.trim()
308
+ if (/^Error: not found:/.test(stderr)) return false
309
+ throw new HypatiaCliError(`hypatia exited ${result.exitCode}: knowledge-delete ${name}`, {
310
+ code: 'NONZERO_EXIT',
311
+ stderr: stderr.slice(0, 500),
312
+ exitCode: result.exitCode,
313
+ })
314
+ },
315
+
284
316
  /**
285
317
  * `embed: false` as on `knowledgeCreate`. A statement has no update
286
318
  * command, so the choice is made once, here.
@@ -61,7 +61,7 @@ export function createHypatiaClient(ctx, config, { status, connection }) {
61
61
  if (!announced) {
62
62
  announced = true
63
63
  const yes = (flag) => (flag ? 'yes' : 'no')
64
- status.info?.(`hypatia over ${usesMcp() ? 'mcp' : 'cli'}: no-embed ${yes(found.noEmbed)}, similar filters ${yes(found.similarFilters)}`)
64
+ status.info?.(`hypatia over ${usesMcp() ? 'mcp' : 'cli'}: no-embed ${yes(found.noEmbed)}, similar filters ${yes(found.similarFilters)}, delete ${yes(found.deleteKnowledge)}`)
65
65
  }
66
66
  return found
67
67
  }
@@ -71,6 +71,7 @@ export function createHypatiaClient(ctx, config, { status, connection }) {
71
71
  features,
72
72
  knowledgeGet: (...args) => via('knowledgeGet', args),
73
73
  knowledgeCreate: (...args) => via('knowledgeCreate', args),
74
+ knowledgeDelete: (...args) => via('knowledgeDelete', args),
74
75
  statementCreate: (...args) => via('statementCreate', args),
75
76
  search: (...args) => via('search', args),
76
77
  similar: (...args) => via('similar', args),
@@ -58,7 +58,7 @@ export const REQUIRED_TOOLS = Object.freeze([
58
58
  * The names mirror hypatia-cli.js's `features()`.
59
59
  *
60
60
  * @param {Array<{name?: string, inputSchema?: {properties?: object}}>} tools
61
- * @returns {{noEmbed: boolean, similarFilters: boolean}}
61
+ * @returns {{noEmbed: boolean, similarFilters: boolean, deleteKnowledge: boolean}}
62
62
  */
63
63
  export function featuresOf(tools) {
64
64
  const accepts = (name, argument) => Object.hasOwn(
@@ -68,6 +68,11 @@ export function featuresOf(tools) {
68
68
  return {
69
69
  noEmbed: accepts('knowledge_create', 'embed') && accepts('statement_create', 'embed'),
70
70
  similarFilters: accepts('similar', 'exclude_tags'),
71
+ // Presence, not schema: `knowledge_delete` takes nothing this client does not
72
+ // already send. It is a capability rather than a REQUIRED_TOOL — a server
73
+ // that cannot delete is still a server worth writing memory to, and
74
+ // rejecting its whole MCP over it would push every call back to the CLI.
75
+ deleteKnowledge: tools.some((tool) => tool?.name === 'knowledge_delete'),
71
76
  }
72
77
  }
73
78
 
@@ -499,6 +504,23 @@ export function createHypatiaMcp(connection, config) {
499
504
  return (await create('knowledge_create', lists(args, entry))) !== undefined
500
505
  },
501
506
 
507
+ /**
508
+ * Remove one knowledge entry. A name that is already gone arrives as a tool
509
+ * error (`not found: knowledge '<name>'`, classified `NOT_FOUND`), which is
510
+ * a normal outcome for a replayable retraction rather than a failure.
511
+ *
512
+ * @returns {Promise<boolean>} true when this call removed an entry.
513
+ */
514
+ async knowledgeDelete(name, shelf = shelfOf()) {
515
+ try {
516
+ await invoke('knowledge_delete', { name, shelf })
517
+ return true
518
+ } catch (error) {
519
+ if (error instanceof HypatiaCliError && error.code === 'NOT_FOUND') return false
520
+ throw error
521
+ }
522
+ },
523
+
502
524
  /**
503
525
  * `embed: false` as on `knowledgeCreate`.
504
526
  * @param {{data?: string, scopes?: string[], embed?: boolean, shelf?: string}} [entry]
package/src/index.js CHANGED
@@ -37,7 +37,9 @@ import { createHypatiaClient } from './hypatia-client.js'
37
37
  import { createHypatiaCli } from './hypatia-cli.js'
38
38
  import { createWriter } from './writer.js'
39
39
  import { TaskDeferredError, createQueue } from './queue.js'
40
- import { countLoggableMessages, createCollector, formatSpan } from './collector.js'
40
+ import {
41
+ countLoggableMessages, createCollector, formatSpan, withdrawalWork, withdrawnOrdinals, withdrawnSeqs,
42
+ } from './collector.js'
41
43
  import { createCascade } from './cascade.js'
42
44
  import { createConsolidator, PLUGIN_NAME } from './consolidator.js'
43
45
  import {
@@ -371,6 +373,18 @@ function applyCollect(ctx, configHandle, status) {
371
373
  // is replay-stable by construction and cannot drift out of sync.
372
374
  const inherited = session.inheritedEventCount ?? 0
373
375
  const baseIndex = countLoggableMessages(session.snapshotEvents(inherited, task.fromSeq))
376
+ // The marker window ends at `session.seq`, never at the task's own end: a
377
+ // marker's seq is greater than every seq it shadows, so it can sit past the
378
+ // range being logged — which is exactly the case where what it hid was
379
+ // written by an earlier run (docs/rewind-memory-consistency.md §3 fact 5).
380
+ // Everything else about the decision is collector.js's, where it is a pure
381
+ // function over the log and therefore testable without a session.
382
+ const { seqs: withdrawn, retract } = withdrawalWork({
383
+ log: session.snapshotEvents(task.fromSeq, session.seq ?? 0),
384
+ prefix: session.snapshotEvents(inherited, task.fromSeq),
385
+ fromSeq: task.fromSeq,
386
+ inherited,
387
+ })
374
388
  const formatted = formatSpan(events, {
375
389
  now: new Date(),
376
390
  maxAssistantChars: config.collector.maxAssistantChars,
@@ -379,14 +393,28 @@ function applyCollect(ctx, configHandle, status) {
379
393
  baseIndex,
380
394
  before: task.fromSeq > 0 ? session.snapshotEvents(task.fromSeq - 1, task.fromSeq)[0] : undefined,
381
395
  })
396
+ // A message the user took back is skipped, and one an earlier run wrote
397
+ // before the marker landed is retracted below. Anchored on the log range,
398
+ // never on "does the entry exist": a replayed span sees the same marker
399
+ // again, which is what keeps a retracted message from being written back.
400
+ const skipped = new Set()
382
401
  for (const item of formatted) {
402
+ const isWithdrawn = withdrawn.has(item.seq)
403
+ if (isWithdrawn) skipped.add(item.index)
383
404
  await writer.writeMessage({
384
405
  sessionId: task.sessionId,
385
406
  index: item.index,
386
407
  markdown: item.markdown,
387
408
  project: task.project,
409
+ withdrawn: isWithdrawn,
388
410
  })
389
411
  }
412
+ // Retracting is idempotent, and the attempts stop once the watermark passes
413
+ // the marker's own seq, after which it leaves every later window too.
414
+ for (const index of retract.values()) {
415
+ skipped.add(index)
416
+ await writer.retractMessage({ sessionId: task.sessionId, index })
417
+ }
390
418
  const linkTo = baseIndex + formatted.length
391
419
  // `belongTo` edges only exist once the session node does. Most sessions
392
420
  // never produce one (it needs a host-supplied summary), so this stays a
@@ -397,8 +425,11 @@ function applyCollect(ctx, configHandle, status) {
397
425
  sessionId: task.sessionId,
398
426
  markdown: '',
399
427
  project: task.project,
428
+ // The cursor counts every loggable message, withdrawn ones included, so
429
+ // the ordinals stay aligned with the log; only the edges are skipped.
400
430
  linkFrom: before.lastBelongToIndex,
401
431
  linkTo,
432
+ skip: skipped,
402
433
  })
403
434
  }
404
435
  // Awaited: the task must not be reported done — and deleted — before the
@@ -432,12 +463,21 @@ function applyCollect(ctx, configHandle, status) {
432
463
  const inherited = session.inheritedEventCount ?? 0
433
464
  const current = state.progress.get(task.sessionId) ?? EMPTY_PROGRESS
434
465
  const linkTo = countLoggableMessages(session.snapshotEvents(inherited, current.lastLoggedSeq))
466
+ // This node links every message logged so far, so it owes the same skip set
467
+ // the log executor used — without it this node's own range would settle
468
+ // edges to messages that have no entry to link from. The markers are read
469
+ // from the whole log (the node is written once per session), and a marker
470
+ // anywhere in it can only hide earlier seqs: one slice, two passes.
471
+ const log = session.snapshotEvents(inherited, session.seq ?? 0)
472
+ const withdrawn = withdrawnSeqs(log)
473
+ const skip = new Set(withdrawnOrdinals(log, withdrawn, inherited).values())
435
474
  await writer.writeSessionNode({
436
475
  sessionId: task.sessionId,
437
476
  markdown,
438
477
  project: task.project,
439
478
  linkFrom: current.lastBelongToIndex,
440
479
  linkTo,
480
+ skip,
441
481
  })
442
482
  await advanceProgress(state.progress, task.sessionId, (progressRow) => ({
443
483
  hasSessionNode: 1,
package/src/writer.js CHANGED
@@ -116,6 +116,52 @@ export function summaryName(sessionId, fromSeq, toSeq) {
116
116
  * relationship rather than a guessed one.
117
117
  */
118
118
  export function createWriter(cli, { status, adjudicate }) {
119
+ /**
120
+ * Whether this shelf can delete at all, warned about once.
121
+ *
122
+ * `knowledgeDelete` is a capability, not part of the baseline: a hypatia from
123
+ * before `knowledge-delete` existed (hypatia #22), or an MCP server without
124
+ * the tool, answers `features()` false. A withdrawal is then skipped instead
125
+ * of retracted — the message is still never written again, because the
126
+ * decision is anchored on the log range rather than on state this plugin
127
+ * keeps, so the only cost is one entry left for the user to remove by hand.
128
+ */
129
+ let warnedNoDelete = false
130
+ async function canRetract() {
131
+ const { deleteKnowledge } = await cli.features()
132
+ if (deleteKnowledge === false && warnedNoDelete === false) {
133
+ warnedNoDelete = true
134
+ status.warn('withdrawn messages cannot be retracted: this hypatia has no knowledge-delete, so they are skipped instead of removed')
135
+ }
136
+ return deleteKnowledge !== false
137
+ }
138
+
139
+ /**
140
+ * Take back one message the user withdrew.
141
+ *
142
+ * Get before delete, because the entry usually does not exist yet: a
143
+ * withdrawal is normally seen by the very span that would have written it.
144
+ * The get is also what makes a replay safe — `knowledge-delete` on a missing
145
+ * name is an error on the CLI and `NOT_FOUND` over MCP.
146
+ *
147
+ * Statements that mention the name are kept by hypatia on purpose
148
+ * (`wu-knowledge-delete-retains-statements-7487bf50`), so a withdrawal can
149
+ * leave a `belongTo` or `summary` edge pointing at an entry that is gone.
150
+ * That is the recorded, accepted graph semantic; nothing here removes them.
151
+ *
152
+ * @param {{sessionId: string, index: number}} entry
153
+ * @returns {Promise<{name: string, retracted: boolean}>}
154
+ */
155
+ async function retractMessage({ sessionId, index }) {
156
+ const name = messageName(sessionId, index)
157
+ const existing = await cli.knowledgeGet(name)
158
+ if (existing.found === false) return { name, retracted: false }
159
+ if ((await canRetract()) === false) return { name, retracted: false }
160
+ const removed = await cli.knowledgeDelete(name)
161
+ if (removed) status.count('retracted')
162
+ return { name, retracted: removed === true }
163
+ }
164
+
119
165
  /**
120
166
  * Log one conversation message. Duplicates (same name already stored, e.g.
121
167
  * from a replayed task) are counted and skipped — the watermark must still
@@ -128,9 +174,14 @@ export function createWriter(cli, { status, adjudicate }) {
128
174
  * stored and full-text searchable; only the vector is not made. On a binary
129
175
  * without `--no-embed` the flag is dropped by the transport.
130
176
  *
131
- * @param {{sessionId: string, index: number, markdown: string, project: string}} entry
177
+ * @param {{sessionId: string, index: number, markdown: string, project: string,
178
+ * withdrawn?: boolean}} entry - `withdrawn` marks a message a
179
+ * withdrawal marker took out of the model's view: it is not written, and one
180
+ * an earlier run already wrote is removed. The executor decides this from
181
+ * the log (`withdrawnSeqs` in collector.js); here it is only carried out.
132
182
  */
133
- async function writeMessage({ sessionId, index, markdown, project }) {
183
+ async function writeMessage({ sessionId, index, markdown, project, withdrawn = false }) {
184
+ if (withdrawn) return { ...(await retractMessage({ sessionId, index })), written: false }
134
185
  const name = messageName(sessionId, index)
135
186
  const existing = await cli.knowledgeGet(name)
136
187
  if (existing.found) {
@@ -201,18 +252,23 @@ export function createWriter(cli, { status, adjudicate }) {
201
252
  * without one is simply skipped rather than given an invented summary
202
253
  * (`skills/hypatia-memory/SKILL.md`, "do not fabricate session summaries").
203
254
  *
204
- * hypatia has no `knowledge-update`, so a second title cannot replace the
205
- * first; the original stands and the caller is told nothing was written.
255
+ * This writer never rewrites a session node: the plugin does not use
256
+ * `knowledge-update` on the log layer, so a second title cannot replace the
257
+ * first — the original stands and the caller is told nothing was written.
206
258
  *
207
259
  * Node and `belongTo` edges are written `embed: false`, as the messages are:
208
260
  * the node is bookkeeping for the traversal, and its title is what
209
261
  * `session/title` already gave DSH.
210
262
  *
211
263
  * @param {{sessionId: string, markdown: string, project: string,
212
- * linkFrom: number, linkTo: number}} entry - `[linkFrom, linkTo)` is
213
- * the half-open range of message ordinals still needing a `belongTo` edge.
264
+ * linkFrom: number, linkTo: number, skip?: Set<number>}} entry -
265
+ * `[linkFrom, linkTo)` is the half-open range of message ordinals still
266
+ * needing a `belongTo` edge. `skip` holds ordinals that must NOT be linked:
267
+ * a message withdrawn from the model's view has no entry to link from, while
268
+ * the cursor still advances past it, so the caller keeps counting every
269
+ * loggable message (docs/rewind-memory-consistency.md §4.5).
214
270
  */
215
- async function writeSessionNode({ sessionId, markdown, project, linkFrom = 0, linkTo = 0 }) {
271
+ async function writeSessionNode({ sessionId, markdown, project, linkFrom = 0, linkTo = 0, skip }) {
216
272
  const name = sessionNodeName(sessionId)
217
273
  const existing = await cli.knowledgeGet(name)
218
274
  if (existing.found) {
@@ -227,10 +283,13 @@ export function createWriter(cli, { status, adjudicate }) {
227
283
  await cli.knowledgeCreate(name, { data: markdown, tags: ['session'], scopes: [project], embed: false })
228
284
  status.count('written')
229
285
  }
286
+ let links = 0
230
287
  for (let index = linkFrom; index < linkTo; index += 1) {
288
+ if (skip?.has(index) === true) continue
231
289
  await cli.statementCreate(messageName(sessionId, index), 'belongTo', name, { scopes: [project], embed: false })
290
+ links += 1
232
291
  }
233
- return { name, written: existing.found === false, links: Math.max(0, linkTo - linkFrom) }
292
+ return { name, written: existing.found === false, links }
234
293
  }
235
294
 
236
295
  /**
@@ -349,5 +408,5 @@ export function createWriter(cli, { status, adjudicate }) {
349
408
  return { name, written: existing.found === false, verdict, nearest }
350
409
  }
351
410
 
352
- return { writeMessage, writeSummary, writeSessionNode, writeWorkUnit, findCandidates, messageName, summaryName }
411
+ return { writeMessage, retractMessage, writeSummary, writeSessionNode, writeWorkUnit, findCandidates, messageName, summaryName }
353
412
  }