claude-mem-lite 3.82.0 → 3.84.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +14 -6
- package/README.zh-CN.md +8 -4
- package/hook-context.mjs +3 -8
- package/hook-episode.mjs +66 -8
- package/hook-llm.mjs +52 -19
- package/hook-shared.mjs +36 -4
- package/hook.mjs +306 -28
- package/lib/citation-tracker.mjs +196 -5
- package/lib/cite-back-hint.mjs +15 -8
- package/lib/cooldown-path.mjs +42 -0
- package/lib/edge-attribution.mjs +4 -7
- package/lib/events-injection.mjs +10 -3
- package/lib/hook-telemetry.mjs +20 -4
- package/lib/private-strip.mjs +37 -2
- package/npm-shrinkwrap.json +2 -2
- package/package.json +2 -1
- package/project-utils.mjs +20 -1
- package/scripts/post-tool-use.sh +31 -1
- package/scripts/pre-tool-recall.js +53 -8
- package/source-files.mjs +3 -0
- package/utils.mjs +29 -9
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
"plugins": [
|
|
11
11
|
{
|
|
12
12
|
"name": "claude-mem-lite",
|
|
13
|
-
"version": "3.
|
|
13
|
+
"version": "3.84.0",
|
|
14
14
|
"source": "./",
|
|
15
15
|
"description": "Persistent long-term memory for Claude Code via MCP — captures coding decisions, bugfixes, and context across sessions. Hybrid FTS5 + TF-IDF search with episode batching. Single SQLite DB, no external services. A lighter, lower-cost alternative to claude-mem (episode batching + a smaller model; cost savings are an internal estimate, not a measured benchmark)."
|
|
16
16
|
}
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "claude-mem-lite",
|
|
3
|
-
"version": "3.
|
|
3
|
+
"version": "3.84.0",
|
|
4
4
|
"description": "Persistent long-term memory for Claude Code via MCP — captures coding decisions, bugfixes, and context across sessions. Hybrid FTS5 + TF-IDF search with episode batching. Single SQLite DB, no external services. A lighter, lower-cost alternative to claude-mem (episode batching + a smaller model; cost savings are an internal estimate, not a measured benchmark).",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "sdsrss"
|
package/README.md
CHANGED
|
@@ -163,7 +163,9 @@ Plugin mode manages its own hooks/runtime. On session start it only **checks and
|
|
|
163
163
|
|
|
164
164
|
> **The plugin install is complete on its own** — hooks, MCP tools, and the bundled slash commands (`/mem`, `/lesson`, `/bug`, `/adopt`) all run from the plugin with no second step. The slash commands invoke the bundled CLI by an absolute path resolved from the plugin directory (`${CLAUDE_PLUGIN_ROOT}/cli.mjs <cmd>`), so they work without anything on your `PATH`. A global `claude-mem-lite` **shell** command (for running queries yourself in a terminal) is **optional** — `npm i -g claude-mem-lite` — and is a *separate* npm install: the plugin's auto-update does **not** refresh it, so re-run `npm i -g claude-mem-lite@latest` if you want that shell command kept in sync. You do **not** need it for the plugin to be fully functional.
|
|
165
165
|
|
|
166
|
-
> **Auto-adopt
|
|
166
|
+
> **Auto-adopt writes into your project, on every SessionStart (v3.13+).** The plugin adds a slug-scoped **managed block** to your project's own **`<cwd>/CLAUDE.md`** — a file that is normally committed to git — plus a `<cwd>/.claude/plugin_claude_mem_lite.md` detail file. The block is a system-authority pointer that boosts Claude's proactive use of `mem_recall` / `mem_save`. Everything outside the block is preserved verbatim, and it coexists with other plugins' blocks in the same file ([details](#invited-memory-v232)). This happens on **every** SessionStart, not just the first: the sync is idempotent and re-applies the block if it is edited away, and refreshes it when the shipped template changes. It applies regardless of install path (npm, npx, `/plugin`, manual), so **no manual `/adopt` is needed**.
|
|
167
|
+
>
|
|
168
|
+
> Opt out per project with `claude-mem-lite adopt --disable` (`--enable` to re-arm), globally with `export MEM_NO_AUTO_ADOPT=1`, or freeze an already-adopted block against template refreshes with `CLAUDE_MEM_NO_TEMPLATE_REFRESH=1`. `claude-mem-lite unadopt` removes the block and the detail file. Manual `/adopt` remains available for re-applying after edits and for the `--all` batch path.
|
|
167
169
|
|
|
168
170
|
### Method 2: npx (one-liner)
|
|
169
171
|
|
|
@@ -355,8 +357,11 @@ Slash commands `/adopt` and `/unadopt` wrap the same CLI.
|
|
|
355
357
|
ever rewritten, and duplicate / CRLF-orphaned copies are collapsed to one.
|
|
356
358
|
Unlike the legacy `MEMORY.md` scheme there is no line-budget gate — `CLAUDE.md`
|
|
357
359
|
has no truncation cap.
|
|
358
|
-
- **Auto-adopt
|
|
359
|
-
|
|
360
|
+
- **Auto-adopt runs on EVERY SessionStart, for any install path (v2.82.1+;
|
|
361
|
+
target moved from the memdir to `<cwd>/CLAUDE.md` in v3.13).** The sync is
|
|
362
|
+
idempotent — it re-applies the managed block if it was edited away and
|
|
363
|
+
refreshes it when the shipped template changes (freeze with
|
|
364
|
+
`CLAUDE_MEM_NO_TEMPLATE_REFRESH=1`). Per-project opt-out: `claude-mem-lite adopt --disable`
|
|
360
365
|
(writes a durable `<memdir>/.mem-no-auto-adopt` sentinel that survives marker
|
|
361
366
|
deletion / plugin reinstalls). Global opt-out: `MEM_NO_AUTO_ADOPT=1`.
|
|
362
367
|
Pre-v2.82.1 the `CLAUDE_PLUGIN_ROOT` gate left auto-adopt unreachable for
|
|
@@ -783,8 +788,8 @@ claude-mem-lite.
|
|
|
783
788
|
| `OPENROUTER_MODEL` | Overrides the OpenRouter model slug for **all** background calls (e.g. `openai/gpt-4o-mini`, `qwen/qwen-2.5-72b-instruct`). When unset, the `CLAUDE_MEM_MODEL` tier maps to `anthropic/claude-haiku-4.5` (haiku) or `anthropic/claude-sonnet-4.5` (sonnet). | _(tier default)_ |
|
|
784
789
|
| `CLAUDE_MEM_DEBUG` | Enable debug logging (`1` to enable). | _(disabled)_ |
|
|
785
790
|
| `MEM_QUIET_HOOKS` | Low-noise hooks. `1` drops the `File Lessons` / `Key Context` sections from SessionStart injection, the lesson suffix from `[mem] Related memories`, and the `WHEN TO USE` / `Decision rules` blocks from MCP server instructions. IDs and the `Recent` table still surface so `mem_get(ids=[…])` remains reachable. Intended for users running the invited-memory adopt path or who otherwise want minimal auto-injection. **Since v2.82.0 this env no longer gates auto-adopt — use `MEM_NO_AUTO_ADOPT=1` for that.** | _(disabled)_ |
|
|
786
|
-
| `MEM_NO_AUTO_ADOPT` | Global opt-out for auto-adopt (v2.82.0+). `1` prevents the
|
|
787
|
-
| `MEM_NO_ADOPT_HINT` | Silences the one-line "Invited-memory 未启用:`claude-mem-lite adopt`…" hint that SessionStart appends when the current project hasn't been adopted. Since v2.82.1 auto-adopt
|
|
791
|
+
| `MEM_NO_AUTO_ADOPT` | Global opt-out for auto-adopt (v2.82.0+). `1` prevents the per-SessionStart auto-write of the `CLAUDE.md` managed block across **all** projects. For per-project opt-out use `claude-mem-lite adopt --disable` instead (writes a durable `<memdir>/.mem-no-auto-adopt` sentinel that survives marker deletion). | _(disabled)_ |
|
|
792
|
+
| `MEM_NO_ADOPT_HINT` | Silences the one-line "Invited-memory 未启用:`claude-mem-lite adopt`…" hint that SessionStart appends when the current project hasn't been adopted. Since v2.82.1 auto-adopt runs on every SessionStart for any install path, so this hint typically surfaces only when you've explicitly opted out (`MEM_NO_AUTO_ADOPT=1` or `claude-mem-lite adopt --disable`). | _(disabled)_ |
|
|
788
793
|
|
|
789
794
|
### What gets injected into your context
|
|
790
795
|
|
|
@@ -827,7 +832,8 @@ benchmark and A/B harness are calibrated against — changing them invalidates t
|
|
|
827
832
|
| `CLAUDE_MEM_AUTO_DEEP` | `0` disables automatic deep-search escalation (one Haiku call rewriting a weak query into keyword/concept/HyDE variants). Explicit `deep: true` still works. | _(auto)_ |
|
|
828
833
|
| `CLAUDE_MEM_AUTO_DEEP_CLI` | `0` disables the same auto-escalation on the CLI path only. | _(auto)_ |
|
|
829
834
|
| `CLAUDE_MEM_VECTORS` | `1` re-enables the persisted TF-IDF vector arm (off by default; also needs a vector rebuild via `maintain`). | _(off)_ |
|
|
830
|
-
| `CLAUDE_MEM_SCOPE_FILTER` | `1` stops environment-scoped observations from firing on file-triggered recall. They stay reachable via search. | _(off)_ |
|
|
835
|
+
| `CLAUDE_MEM_SCOPE_FILTER` | `1` stops environment-scoped observations from firing on file-triggered recall. They stay reachable via search. **Leave it off**: on the face it gates, `environment` is not the low-relevance class its premise assumes — it cites at least as well as `project` (47.5% vs 44.3%, intervals overlapping), and an earlier measurement left 173 recall groups empty with it on. | _(off)_ |
|
|
836
|
+
| `CLAUDE_MEM_READS_CARRY` | An episode flush collects `reads-<project>.txt` only when it will actually save an observation, so a flush that records nothing no longer discards the Read paths it swept up (42.2% of the paths a flush consumed, measured over 1122 transcripts). `0` restores the pre-v3.83.0 behaviour. | _(on)_ |
|
|
831
837
|
|
|
832
838
|
### Citation tracking and feedback
|
|
833
839
|
|
|
@@ -840,6 +846,8 @@ benchmark and A/B harness are calibrated against — changing them invalidates t
|
|
|
840
846
|
| `CLAUDE_MEM_CITE_NUDGE_THRESHOLD` | Cite-rate below which the nudge fires. | `0.6` |
|
|
841
847
|
| `CLAUDE_MEM_CITE_NUDGE_MIN_INJECTED` | Minimum injection volume before the ratio gate is judged at all. | `5` |
|
|
842
848
|
| `CLAUDE_MEM_CITE_NUDGE_SILENCE_AFTER` | Consecutive low-cite sessions before the nudge goes quiet; `0` = never silence. | `3` |
|
|
849
|
+
| `CLAUDE_MEM_CITATION_RELEVANCE_GATE` | Stop credits an `access_count` to a memory the session cited only when something made that memory relevant to the session — it was injected, or you typed its `#NN` yourself. `off` restores the pre-v3.84.0 behaviour of crediting every `#NN` the assistant wrote, which over-counts sessions that discuss memories in prose (release notes, audit reports): measured on real transcripts, 267 of 859 credited (id, session) pairs — 31.1% — were mentions nothing had put in front of the model. Superseded citations are redirected to their keeper on both settings. | _(on)_ |
|
|
850
|
+
| `CLAUDE_MEM_SUBAGENT_DECAY` | The `subagent` injection face feeds the decay loop: memories handed to a dispatched agent enter the denominator, and the citation that agent makes in its own transcript counts as the numerator. `0` returns the face to metered-but-never-decaying (v3.77–v3.82). | _(on)_ |
|
|
843
851
|
| `CLAUDE_MEM_METRICS` | `1` records feature-injection counters surfaced by `claude-mem-lite stats`. | _(off)_ |
|
|
844
852
|
|
|
845
853
|
### Background work
|
package/README.zh-CN.md
CHANGED
|
@@ -152,7 +152,9 @@ node install.mjs install
|
|
|
152
152
|
1. **安装依赖** -- `npm install --omit=dev`(编译原生 `better-sqlite3`)
|
|
153
153
|
2. **注册 MCP 服务器** -- `mem-lite` 服务器,包含 20 个工具(9 个核心通过 `tools/list` 暴露 + 11 个隐藏但可调;完整表见 Usage 段)。v2.78 前服务器名为通用的 `mem`,现已改名为 `mem-lite` 避免与用户其它 `.mcp.json` 冲突;工具名(`mem_search`/`mem_recall` 等)保持不变。
|
|
154
154
|
|
|
155
|
-
>
|
|
155
|
+
> **自动 adopt 会写进你的项目,且每次 SessionStart 都跑(v3.13+)。** 插件向**项目自己的 `<cwd>/CLAUDE.md`**(通常是会进 git 的文件)写入一个 slug 限定的**托管块**,外加 `<cwd>/.claude/plugin_claude_mem_lite.md` 详情文件。该块是一条提升 Claude 主动调用 `mem_recall` / `mem_save` 的 system-authority 指针;块以外的内容逐字保留,也能与其它插件的块共存于同一文件。这是**每次** SessionStart 都做的幂等同步,不只是第一次——块被删掉会重新写回,出货模板变了会刷新。**任何安装路径都生效**(npm、npx、`/plugin`、手动),**无需再手动跑 `/adopt`**。
|
|
156
|
+
>
|
|
157
|
+
> 关闭方式:项目级 `claude-mem-lite adopt --disable`(重新启用用 `--enable`);全局 `export MEM_NO_AUTO_ADOPT=1`;只冻结模板刷新用 `CLAUDE_MEM_NO_TEMPLATE_REFRESH=1`。`claude-mem-lite unadopt` 可移除托管块与详情文件。手动 `/adopt` 仍保留用于编辑后重写或 `--all` 批量场景。
|
|
156
158
|
3. **配置钩子** -- `PostToolUse`、`PreToolUse`、`SessionStart`、`Stop`、`UserPromptSubmit` 生命周期钩子
|
|
157
159
|
4. **创建数据目录** -- `~/.claude-mem-lite/`(隐藏目录),存放数据库、运行时和托管资源文件
|
|
158
160
|
5. **自动迁移** -- 自动检测 `~/.claude-mem/`(原版 claude-mem)或 `~/claude-mem-lite/`(v0.5 前的非隐藏目录),将数据库和运行时文件迁移到 `~/.claude-mem-lite/`,原目录保持不变
|
|
@@ -297,7 +299,9 @@ Slash 命令 `/adopt` 和 `/unadopt` 是上述 CLI 的包装。
|
|
|
297
299
|
- Hash 守护:你手动改了 sentinel 段 → 下一次 adopt 报 `UserEditedError`,
|
|
298
300
|
除非显式 `--force`。
|
|
299
301
|
- 预算门:MEMORY.md 已 >180 行时拒绝新增(避开 Claude Code 200 行截断)。
|
|
300
|
-
-
|
|
302
|
+
- **任何安装路径每次 SessionStart 都自动 adopt(v2.82.1+;v3.13 起写入目标由
|
|
303
|
+
memdir 改为 `<cwd>/CLAUDE.md`)。** 同步是幂等的——托管块被删会写回,出货模板
|
|
304
|
+
变化会刷新(用 `CLAUDE_MEM_NO_TEMPLATE_REFRESH=1` 冻结)。项目级关闭:
|
|
301
305
|
`claude-mem-lite adopt --disable`(写 `<memdir>/.mem-no-auto-adopt` 哨兵,
|
|
302
306
|
存活于 marker 删除 / 插件重装)。全局关闭:`export MEM_NO_AUTO_ADOPT=1`。
|
|
303
307
|
v2.82.1 前因 `CLAUDE_PLUGIN_ROOT` gate 与 `install.mjs` 写出的 hook 命令
|
|
@@ -614,8 +618,8 @@ npm run benchmark:gate # CI 门控:指标回退超过 5% 容差时失败
|
|
|
614
618
|
| `OPENROUTER_MODEL` | 覆盖**所有**后台调用的 OpenRouter 模型 slug(如 `openai/gpt-4o-mini`、`qwen/qwen-2.5-72b-instruct`)。未设时按 `CLAUDE_MEM_MODEL` 分层映射到 `anthropic/claude-haiku-4.5`(haiku)或 `anthropic/claude-sonnet-4.5`(sonnet)。 | _(分层默认)_ |
|
|
615
619
|
| `CLAUDE_MEM_DEBUG` | 启用调试日志(设为 `1` 启用)。 | _(禁用)_ |
|
|
616
620
|
| `MEM_QUIET_HOOKS` | 低噪声 hook。设为 `1` 时,SessionStart 注入去掉 `File Lessons` / `Key Context` 两节,`[mem] Related memories` 去掉 lesson 后缀,MCP server instructions 去掉 `WHEN TO USE` / `Decision rules` 两段。ID 与 `Recent` 表仍保留,`mem_get(ids=[…])` 可继续展开细节。适用于启用了 invited-memory adopt 流程或偏好最小化自动注入的用户。**v2.82.0 起此 env 不再阻挡 auto-adopt——如需关闭 auto-adopt 用 `MEM_NO_AUTO_ADOPT=1`。** | _(禁用)_ |
|
|
617
|
-
| `MEM_NO_AUTO_ADOPT` | auto-adopt 全局关闭开关(v2.82.0+)。设为 `1`
|
|
618
|
-
| `MEM_NO_ADOPT_HINT` | 静音当前项目未 adopt 时 SessionStart 追加的那一行 "Invited-memory 未启用…" 提示。v2.82.1
|
|
621
|
+
| `MEM_NO_AUTO_ADOPT` | auto-adopt 全局关闭开关(v2.82.0+)。设为 `1` 阻止每次 SessionStart 在**所有**项目自动写入 `CLAUDE.md` 托管块。项目级关闭走 `claude-mem-lite adopt --disable`(写 `<memdir>/.mem-no-auto-adopt` 哨兵,存活于 marker 删除)。 | _(禁用)_ |
|
|
622
|
+
| `MEM_NO_ADOPT_HINT` | 静音当前项目未 adopt 时 SessionStart 追加的那一行 "Invited-memory 未启用…" 提示。v2.82.1 起任何安装路径每次 SessionStart 都自动 adopt,所以该提示一般只在你显式 opt out(`MEM_NO_AUTO_ADOPT=1` 或 `claude-mem-lite adopt --disable`)的项目才会出现。 | _(禁用)_ |
|
|
619
623
|
|
|
620
624
|
## 许可证
|
|
621
625
|
|
package/hook-context.mjs
CHANGED
|
@@ -16,19 +16,14 @@ import { STALE_SESSION_MS, FALLBACK_OBS_WINDOW_MS, RUNTIME_DIR, effectiveQuiet,
|
|
|
16
16
|
import { extractUnfinishedSummary } from './hook-handoff.mjs';
|
|
17
17
|
import { recentInjectableEvents, renderInjectableEvent } from './lib/events-injection.mjs';
|
|
18
18
|
import { liveObsFilterSql } from './lib/inject-search-core.mjs';
|
|
19
|
+
// The canonical one (v3.84.0): this file carried a byte-identical private copy, which is
|
|
20
|
+
// the same one-home rule this release enforced for the cooldown path and the dashboard.
|
|
21
|
+
import { inferProjectDir } from './project-utils.mjs';
|
|
19
22
|
// Single source for the type-quality weights (audit 2026-08-22 P2-10) — this table used
|
|
20
23
|
// to be hand-copied here and in hook-memory.mjs, kept equal only by comment convention.
|
|
21
24
|
import { TYPE_QUALITY, TYPE_QUALITY_DEFAULT } from './scoring-sql.mjs';
|
|
22
25
|
|
|
23
26
|
import { DAY_MS } from './lib/time-constants.mjs';
|
|
24
|
-
/**
|
|
25
|
-
* Infer the project directory from environment variables or cwd.
|
|
26
|
-
* @returns {string} Absolute path to the project directory
|
|
27
|
-
*/
|
|
28
|
-
function inferProjectDir() {
|
|
29
|
-
return process.env.CLAUDE_PROJECT_DIR || process.env.PWD || process.cwd();
|
|
30
|
-
}
|
|
31
|
-
|
|
32
27
|
/**
|
|
33
28
|
* Compute adaptive recall time windows based on project activity velocity.
|
|
34
29
|
* High activity -> shorter windows (recent data more relevant).
|
package/hook-episode.mjs
CHANGED
|
@@ -314,17 +314,75 @@ const RESEARCH_ENTRY_THRESHOLD = 8;
|
|
|
314
314
|
* 2026-08 20/ 193 10.4% >=8: 0
|
|
315
315
|
* lifetime reaching >=8: 81
|
|
316
316
|
*
|
|
317
|
-
* For three consecutive months this field fed the threshold at a real rate.
|
|
318
|
-
* 2026-05
|
|
319
|
-
*
|
|
320
|
-
*
|
|
321
|
-
* boundary
|
|
317
|
+
* For three consecutive months this field fed the threshold at a real rate. The break is
|
|
318
|
+
* sharp: 2026-05-08 reads 11%, 05-09 onward reads 0.
|
|
319
|
+
*
|
|
320
|
+
* "THE REACHABLE INPUT IS THE EPISODE BOUNDARY" WAS WRONG — D#174 investigated it and the
|
|
321
|
+
* boundary never moved. One query falsifies the whole family of boundary explanations:
|
|
322
|
+
* measure the SIBLING column the same producer writes. `files_modified` is non-empty on
|
|
323
|
+
* 85-98% of rows every month from 2026-02 through 2026-08, averaging 2.0-2.6 paths, and it
|
|
324
|
+
* does not so much as dip across the break — while `files_read` goes 60% -> 1%. Episodes
|
|
325
|
+
* still carry ~2 edits each; they just stopped carrying reads. A smaller boundary would have
|
|
326
|
+
* taken both columns down together. Consistent with that, nothing in this repo changed at the
|
|
327
|
+
* break: zero commits on 05-08/05-09, scripts/post-tool-use.sh byte-identical from 03-29 to
|
|
328
|
+
* 05-24, hooks/hooks.json byte-identical from 04-22 to 05-10, and EPISODE_BUFFER_SIZE /
|
|
329
|
+
* EPISODE_TIME_GAP_MS / isRelatedToEpisode untouched since 2026-02-11.
|
|
330
|
+
*
|
|
331
|
+
* WHAT ACTUALLY SET THE RATE (probed, not read): a flush consumed reads-<project>.txt
|
|
332
|
+
* unconditionally but only PERSISTED it when the episode was significant
|
|
333
|
+
* (flushEpisodeGroup saves on `isSignificant`, and unlinks the flush file otherwise). So a
|
|
334
|
+
* buffered-but-insignificant flush — a successful `npm test` on its own, say — swallowed every
|
|
335
|
+
* Read accumulated since the previous flush and wrote none of them anywhere. Measured in a
|
|
336
|
+
* sandbox: seed 2 Reads, fire one such flush, and 0 observations are saved, the reads-file is
|
|
337
|
+
* gone, and the NEXT (edit-bearing, significant) observation carries `files_read=[]`.
|
|
338
|
+
*
|
|
339
|
+
* PAST TENSE SINCE v3.83.0: D#178 is FIXED. `flushEpisodeWithDb` now decides significance
|
|
340
|
+
* before it touches the file, and an insignificant flush leaves it in place for the next
|
|
341
|
+
* saving one (`CLAUDE_MEM_READS_CARRY=0` restores the old order). Two numbers in the
|
|
342
|
+
* paragraph above were also wrong and are corrected here rather than left to be re-quoted:
|
|
343
|
+
* the significant share is ~59%, not "~4-8%" — the `episode_significance` meter reads 40.7%
|
|
344
|
+
* INsignificant over n=938 across three active days — and the 92-96% figure D#178 was filed
|
|
345
|
+
* on came from the same slip. What the loss actually was, replayed over 1122 real
|
|
346
|
+
* transcripts through this file's own batcher (`benchmark/episode-flush-replay.mjs`):
|
|
347
|
+
* 42.2% of the Read paths a flush consumed destroyed, 72.7% of significant flushes
|
|
348
|
+
* carrying none. Same measurement pass as CHANGELOG v3.83.0, CLAUDE.md and README — quoting
|
|
349
|
+
* a second pass here would be the stitched-across-runs error one file at a time.
|
|
350
|
+
*
|
|
351
|
+
* The D#171 conclusion below is UNAFFECTED and that is worth stating explicitly, because
|
|
352
|
+
* the fix moves the quantity its arithmetic used. Post-fix the carried distinct set runs
|
|
353
|
+
* median 1, p95 6, max 21 per delivering flush — still nowhere near rule 4's threshold of
|
|
354
|
+
* 8 on a per-EPISODE basis, and rule 4 does not read this field anyway.
|
|
355
|
+
*
|
|
356
|
+
* Note the first version of that probe used `echo hello` as its "insignificant" entry.
|
|
357
|
+
* detectBashSignificance drops it, so the episode had zero entries, flushEpisode
|
|
358
|
+
* early-returned at `entries.length === 0`, the reads were never touched — and the probe
|
|
359
|
+
* confidently reported the opposite conclusion. An insignificant entry must be asserted into
|
|
360
|
+
* the buffer before it proves anything.
|
|
322
361
|
*
|
|
323
362
|
* D#171 closed as won't-fix-as-specified: the repair it named does not work at the current
|
|
324
363
|
* cadence, and re-pointing the rule would move the dormancy to a field nobody suspects.
|
|
325
|
-
*
|
|
326
|
-
*
|
|
327
|
-
*
|
|
364
|
+
* That closure stands, and D#174 no longer offers a reason to reopen it — the rule's own
|
|
365
|
+
* input (`readCount`, which counts Read/Grep ENTRIES, and Read never reaches Node) is a
|
|
366
|
+
* different quantity from `filesRead` and is untouched by any of the above.
|
|
367
|
+
*
|
|
368
|
+
* EXACTLY ONE claim above is pinned by a test, and deliberately so (D#175). Every number
|
|
369
|
+
* here is a corpus measured at a timestamp — a test over those would be a snapshot that
|
|
370
|
+
* rots and gets edited into greenness. The per-FLUSH claim is different in kind: it is a
|
|
371
|
+
* property of code (the reads-file is renamed aside, then the copy is unlinked), and if
|
|
372
|
+
* someone later makes reads accumulate across flushes, every "out of reach at ~1 Read per
|
|
373
|
+
* episode" sentence above silently becomes false. That is the one this closure rests on,
|
|
374
|
+
* so `tests/feature-sweep-hooks.test.mjs` → "the reads-file is consumed, not accumulated
|
|
375
|
+
* (D#175)" drives two real flushes through the subprocess and asserts the second one starts
|
|
376
|
+
* empty. Rename-becomes-copy and the dropped unlink are separate mutations caught by
|
|
377
|
+
* separate assertions there — one does not cover the other.
|
|
378
|
+
*
|
|
379
|
+
* THAT ALARM DID NOT FIRE FOR D#178, and the reason is worth keeping. Both of its flushes
|
|
380
|
+
* are SIGNIFICANT (each buffers a `.sql` Write), so both take the collect branch under the
|
|
381
|
+
* new order too — the v3.83.0 change walked straight underneath a guard installed one
|
|
382
|
+
* commit earlier to catch exactly "reads accumulate across flushes". Its sibling cases in
|
|
383
|
+
* the same file now cover the insignificant arm, in both flag positions and in the
|
|
384
|
+
* multi-session shape; a per-flush guard whose fixture only ever exercises one arm of the
|
|
385
|
+
* branch it guards is covering the arm nobody was going to change.
|
|
328
386
|
*
|
|
329
387
|
* @param {object} episode
|
|
330
388
|
* @returns {{significant: boolean, rule: 1|2|3|4|null, readCount: number,
|
package/hook-llm.mjs
CHANGED
|
@@ -25,6 +25,7 @@ import { episodeHasSignificantContent } from './hook-episode.mjs';
|
|
|
25
25
|
import { OBS_TYPE_SET } from './lib/obs-types.mjs';
|
|
26
26
|
|
|
27
27
|
import { DAY_MS } from './lib/time-constants.mjs';
|
|
28
|
+
import { liveObsFilterSql } from './lib/inject-search-core.mjs';
|
|
28
29
|
// T9: memdir-incompatible types live in the `events` table, not `observations`.
|
|
29
30
|
// Set lookup is O(1) — authoritative source is lib/activity.mjs::EVENT_TYPES.
|
|
30
31
|
const EVENT_TYPE_SET = new Set(EVENT_TYPES);
|
|
@@ -351,7 +352,18 @@ export function persistHaikuSummary(db, summary, ctx) {
|
|
|
351
352
|
|
|
352
353
|
if (ctx.preSavedObsId) {
|
|
353
354
|
const id = db.transaction(() => {
|
|
354
|
-
|
|
355
|
+
// Same live-row guard as the in-place upgrade (FLOW-7). If auto-dedup superseded
|
|
356
|
+
// or compressed the pre-saved row while this worker was in flight, that row is no
|
|
357
|
+
// longer ours to hard-delete — a keeper may have absorbed it, and children can
|
|
358
|
+
// point at it through compressed_into. Leave it to the maintenance path, which
|
|
359
|
+
// recovers children before deleting; the event still gets written either way.
|
|
360
|
+
const removed = db.prepare(
|
|
361
|
+
`DELETE FROM observations WHERE id = ? AND ${liveObsFilterSql('')}`,
|
|
362
|
+
).run(ctx.preSavedObsId).changes;
|
|
363
|
+
if (removed === 0) {
|
|
364
|
+
debugLog('DEBUG', 'llm-episode',
|
|
365
|
+
`upgrade-delete: pre-saved obs #${ctx.preSavedObsId} no longer live — left in place`);
|
|
366
|
+
}
|
|
355
367
|
return insertEvent();
|
|
356
368
|
})();
|
|
357
369
|
return { table: 'events', id };
|
|
@@ -1014,12 +1026,18 @@ ${actionList}`;
|
|
|
1014
1026
|
lesson_learned: obs.lessonLearned || null,
|
|
1015
1027
|
search_aliases: obs.searchAliases || null,
|
|
1016
1028
|
});
|
|
1017
|
-
|
|
1029
|
+
// The live-row guard (FLOW-7, 2026-08-29 audit). This worker is 2-5s behind the
|
|
1030
|
+
// foreground pre-save, and auto-dedup can supersede or compress that row inside
|
|
1031
|
+
// the window. An unguarded `WHERE id = ?` then writes the whole enrichment onto a
|
|
1032
|
+
// tombstone: `changes` is 1, nothing looks wrong, and the row it landed on is
|
|
1033
|
+
// excluded from every read face by liveObsFilterSql. Same clause as those read
|
|
1034
|
+
// faces, so "what the update may touch" and "what a query may return" cannot drift.
|
|
1035
|
+
const upgraded = db.prepare(`
|
|
1018
1036
|
UPDATE observations SET type=?, title=?, subtitle=?,
|
|
1019
1037
|
narrative=COALESCE(NULLIF(?, ''), narrative), concepts=?, facts=?,
|
|
1020
1038
|
text=?, importance=?, files_read=?, minhash_sig=?, lesson_learned=?, search_aliases=?,
|
|
1021
1039
|
scope=COALESCE(?, scope)
|
|
1022
|
-
WHERE id = ?
|
|
1040
|
+
WHERE id = ? AND ${liveObsFilterSql('')}
|
|
1023
1041
|
`).run(
|
|
1024
1042
|
obs.type, safe.title, safe.subtitle,
|
|
1025
1043
|
safe.narrative,
|
|
@@ -1031,23 +1049,38 @@ ${actionList}`;
|
|
|
1031
1049
|
safe.search_aliases,
|
|
1032
1050
|
normalizeScope(obs.scope),
|
|
1033
1051
|
episode.savedId
|
|
1034
|
-
);
|
|
1035
|
-
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1040
|
-
|
|
1041
|
-
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1046
|
-
|
|
1047
|
-
|
|
1052
|
+
).changes;
|
|
1053
|
+
|
|
1054
|
+
if (upgraded === 0) {
|
|
1055
|
+
// The pre-saved row is gone or tombstoned. Dropping the enrichment here is the
|
|
1056
|
+
// silent-loss shape this repository keeps paying for, so save it as a fresh row
|
|
1057
|
+
// and let the normal dedup path decide whether it merges into the keeper.
|
|
1058
|
+
debugLog('DEBUG', 'llm-episode',
|
|
1059
|
+
`pre-saved obs #${episode.savedId} no longer live — saving enrichment as a fresh row`);
|
|
1060
|
+
const result = persistHaikuSummary(db, obsToSummary(obs), {
|
|
1061
|
+
project: episode.project,
|
|
1062
|
+
session_id: episode.sessionId,
|
|
1063
|
+
});
|
|
1064
|
+
savedId = result.id;
|
|
1065
|
+
savedTable = result.table;
|
|
1066
|
+
} else {
|
|
1067
|
+
savedId = episode.savedId;
|
|
1068
|
+
savedTable = 'observations';
|
|
1069
|
+
debugLog('DEBUG', 'llm-episode', `upgraded pre-saved obs #${savedId}`);
|
|
1070
|
+
|
|
1071
|
+
// Update TF-IDF vector with enriched content
|
|
1072
|
+
try {
|
|
1073
|
+
const vocab = getVocabulary(db);
|
|
1074
|
+
if (vocab) {
|
|
1075
|
+
const vecText = vecTextForRow({ title: obs.title, narrative: obs.narrative, concepts: conceptsText, lesson_learned: safe.lesson_learned, search_aliases: safe.search_aliases });
|
|
1076
|
+
const vec = computeVector(vecText, vocab);
|
|
1077
|
+
if (vec) {
|
|
1078
|
+
db.prepare('INSERT OR REPLACE INTO observation_vectors (observation_id, vector, vocab_version, created_at_epoch) VALUES (?, ?, ?, ?)')
|
|
1079
|
+
.run(savedId, Buffer.from(vec.buffer), vocab.version, Date.now());
|
|
1080
|
+
}
|
|
1048
1081
|
}
|
|
1049
|
-
}
|
|
1050
|
-
}
|
|
1082
|
+
} catch (e) { debugCatch(e, 'handleLLMEpisode-vector'); }
|
|
1083
|
+
}
|
|
1051
1084
|
}
|
|
1052
1085
|
} else {
|
|
1053
1086
|
// Clean insert (no pre-save) — dispatcher routes by type.
|
package/hook-shared.mjs
CHANGED
|
@@ -30,6 +30,16 @@ export const EPISODE_TIME_GAP_MS = 5 * 60 * 1000; // 5 min
|
|
|
30
30
|
export const SESSION_EXPIRY_MS = 12 * 60 * 60 * 1000; // 12h
|
|
31
31
|
export const STALE_SESSION_MS = 24 * 60 * 60 * 1000; // 24h
|
|
32
32
|
export const STALE_LOCK_MS = 30000; // 30s
|
|
33
|
+
|
|
34
|
+
// The background-maintenance mutex, defined HERE next to the sweeper policy it has to
|
|
35
|
+
// escape. cleanStaleLockFiles() below unlinks any `*.lock` older than STALE_LOCK_MS
|
|
36
|
+
// WITHOUT checking whether the holder is alive — right for the episode lock's millisecond
|
|
37
|
+
// critical section, fatal for a maintenance pass that runs for seconds to minutes. The
|
|
38
|
+
// name therefore ends in `.proclock`, and `tests/auto-maintain-proc-lock.test.mjs` asserts
|
|
39
|
+
// that against THIS constant rather than a re-typed copy: the first version of that test
|
|
40
|
+
// built its own path from a literal, so renaming the lock left it green with the hazard
|
|
41
|
+
// back. proc-lock's own staleness policy (age OR provably-dead pid) is the correct one.
|
|
42
|
+
export const AUTO_MAINTAIN_LOCK = 'auto-maintain.proclock';
|
|
33
43
|
export const DEDUP_WINDOW_MS = 5 * 60 * 1000; // 5 min (title dedup)
|
|
34
44
|
export const RELATED_OBS_WINDOW_MS = 7 * DAY_MS; // 7 days
|
|
35
45
|
export const FALLBACK_OBS_WINDOW_MS = RELATED_OBS_WINDOW_MS; // same window
|
|
@@ -108,13 +118,35 @@ export function sweepOrphanEpisodeFiles(runtimeDir, { ageMs = ORPHAN_EPISODE_AGE
|
|
|
108
118
|
const readsCutoff = now - readsAgeMs;
|
|
109
119
|
let count = 0;
|
|
110
120
|
for (const f of entries) {
|
|
111
|
-
//
|
|
112
|
-
//
|
|
113
|
-
|
|
121
|
+
// Crash residue: this runtime dir writes four families of temp name, each the middle
|
|
122
|
+
// of a rename-or-unlink pair that leaks if the process dies between the two steps.
|
|
123
|
+
// The predicate covered only `.claim-`, whose comment states the reason it exists —
|
|
124
|
+
// and the other three are that same window (audit FLOW-3):
|
|
125
|
+
// .claim- handleStop's lock-contended fallback (hook.mjs)
|
|
126
|
+
// .collect- the reads-file rename a flush performs (hook.mjs)
|
|
127
|
+
// .trim- the reads-file truncation (hook.mjs)
|
|
128
|
+
// .tmp- every atomic write (hook-episode.mjs, atomicWrite)
|
|
129
|
+
// Neither of the old clauses could reach them: `reads-<p>.txt.collect-<ts>` does not
|
|
130
|
+
// end in `.txt`, and `ep-<p>.json.tmp-<pid>` does not start with `ep-flush-`.
|
|
131
|
+
//
|
|
132
|
+
// Anchored to the END of the name, and the reason is not the one first written here.
|
|
133
|
+
// The original note claimed it protected `reads-x.tmp-y.txt` from the short clock; it
|
|
134
|
+
// does not — that name ends in `.txt`, so `isReads` picks the 24h cutoff either way,
|
|
135
|
+
// and dropping the anchor killed no test (caught by a pre-tag reviewer). What the
|
|
136
|
+
// anchor actually protects is the LIVE episode buffer of a project whose sanitized
|
|
137
|
+
// name contains the token: `ep-x.tmp-y.json` matches an unanchored pattern, and would
|
|
138
|
+
// then be swept as residue one hour into a session that is still using it.
|
|
139
|
+
const isCrashResidue = /\.(claim|collect|trim|tmp)-[^.]*$/.test(f);
|
|
140
|
+
const isEpisode = f.startsWith('ep-flush-') || f.startsWith('pending-');
|
|
114
141
|
const isReads = f.startsWith('reads-') && f.endsWith('.txt');
|
|
115
|
-
if (!isEpisode && !isReads) continue;
|
|
142
|
+
if (!isCrashResidue && !isEpisode && !isReads) continue;
|
|
116
143
|
const full = join(runtimeDir, f);
|
|
117
144
|
try {
|
|
145
|
+
// Residue takes the short cutoff and a live tracker takes the 24h one, with no
|
|
146
|
+
// tie-break needed: residue always APPENDS its suffix, so it never ends in `.txt`
|
|
147
|
+
// and `isReads` is already false for it. (A `&& !isCrashResidue` tie-break was
|
|
148
|
+
// written here first and no mutation could kill it — it was guarding a state the
|
|
149
|
+
// two predicates cannot both be in.)
|
|
118
150
|
if (statSync(full).mtimeMs < (isReads ? readsCutoff : cutoff)) {
|
|
119
151
|
unlinkSync(full);
|
|
120
152
|
count++;
|