open-memex 0.9.0-alpha.2 → 0.9.0-alpha.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/AGENTS.md CHANGED
@@ -35,7 +35,7 @@ npm run typecheck # tsc --noEmit — the only
35
35
  npm run cli -- where | list | search "q" | add ... | forget <id> | reindex
36
36
  npm run cli -- useful <id...> # D77-L2: explicit useful signal + cumulative counters
37
37
  npm run cli -- <command> --help # per-command help (AI assistants discover flags this way)
38
- npm run cli -- sync-status # last sync time/kind + outbox drafts + repo review states + uncommitted files
38
+ npm run cli -- sync-status # last sync time/kind + outbox drafts + repo review states + uncommitted files + D77 reuse report
39
39
  npm run cli -- inventory [--format text|json|html] [--scope both] # what is remembered, in the open (D68); personal-bearing output refused inside a git worktree without --allow-personal
40
40
  npm run cli -- audit [--scope project|personal|both] # memory health: near-dup pairs, stale actives, broken chains, personal-in-repo, expired working-state notes (read-only)
41
41
  npm run cli -- wstate [--clear] # show/clear the session working-state note (D89 §3.6; the file it prints is editable directly)
@@ -126,7 +126,7 @@ Layout: `memories/<scope_key>/<id>.md` (YAML frontmatter + body) + `index.db` (S
126
126
 
127
127
  **Markdown files are the source of truth; `index.db` is rebuildable.** `syncScope()` (in `src/store/sync.ts`) reconciles the two by mtime on plugin load and before every CLI read. When adding new frontmatter fields, update `Frontmatter` in `src/store/markdown.ts`, the `memories` table in `src/store/db.ts`, and `upsertFromFile` in `src/store/sync.ts` together, and bump the reindex path.
128
128
 
129
- D77-L2 counters are the one deliberate exception: `memory_usage` (in `index.db`, owned by `src/store/usage.ts`) holds `retrieved_count` / `useful_self` / `useful_team` keyed by memory id, is created with `CREATE TABLE IF NOT EXISTS` outside the versioned schema, and is NOT dropped by `rebuildIndexSchema` — accumulated telemetry has no Markdown source to rebuild from. They never go into frontmatter and never leave the machine. Only `memory_useful` (explicit), a human approve at review, and a supersede raise the useful side; passive retrieval (search/list/injection, opt-in via `record: true`) raises only `retrieved_count` — never "fix" this by counting exposure as useful.
129
+ D77-L2 counters are the one deliberate exception: `memory_usage` (in `index.db`, owned by `src/store/usage.ts`) holds `retrieved_count` / `useful_self` / `useful_team` keyed by memory id, is created with `CREATE TABLE IF NOT EXISTS` outside the versioned schema, and is NOT dropped by `rebuildIndexSchema` — accumulated telemetry has no Markdown source to rebuild from. They never go into frontmatter and never leave the machine. Only `memory_useful` (explicit), a human approve at review, and a supersede raise the useful side; passive retrieval (search/list/injection, opt-in via `record: true`) raises only `retrieved_count` — never "fix" this by counting exposure as useful. D77-L3's report (`loadReuseReport` in usage.ts, pure rendering in `src/reuse-report.ts`) rides `memory_status` / `cli sync-status`: per-scope rates with the useful split shown separately (self vs team, per D77), top reused, zombies (retrieved, never useful; retracted/archived excluded from both lists), and a lifecycle tag on superseded/deprecated entries so historical reuse never reads as current guidance.
130
130
 
131
131
  ## Scope keys
132
132
 
package/CHANGELOG.md CHANGED
@@ -9,9 +9,13 @@ the `@alpha` tag are development snapshots and are not listed individually.
9
9
 
10
10
  ## [Unreleased]
11
11
 
12
+ ## [0.9.0-alpha.3] - 2026-11-04
13
+
14
+ - D77-L3: the reuse report — D77's third and final layer. `memory_status` (MCP) and `open-memex sync-status` now end with a **Reuse (D77)** block built from the `memory_usage` counters: per-scope `retrieved` / `useful` rates (personal + this project) with the **useful split shown separately** (`self` vs `team`, as D77 requires — the combined total drives the rate), the **top reused** memories, and **zombies** — retrieved at least once, never confirmed useful, i.e. the decay/prune candidates. Superseded/deprecated entries stay in the lists (their reuse was real) but carry a lifecycle tag like `[superseded]`, so historical data never reads to the agent as current guidance (a supersede itself ticks the old memory). The agent-facing tool result caps each list (cut disclosed, pointing at the CLI); the CLI prints them uncapped. Retracted/archived memories are excluded from both lists but still count toward the scope totals; an empty store gets an honest "no data yet" line instead of a silent section. Rendering is pure (`src/reuse-report.ts`, smoke-tested without SQLite); collection is one join query in `loadReuseReport`. The counters themselves stay local-only SQLite telemetry (D77).
15
+
12
16
  ## [0.9.0-alpha.2] - 2026-11-03
13
17
 
14
- - D77-L2: measure usefulness by reuse (the D77 counters, layer 2). Each memory now carries local counters in a `memory_usage` table beside the rebuildable index (D77: counters live in SQLite, never in frontmatter, never off the machine): `retrieved_count` — incremented when a memory is actually returned by `memory_search`, `memory_list`, the CLI `search`/`list`, or the session-start injection — and the useful split, `useful_self` (personal memories and unpublished project drafts) vs `useful_team` (approved/published project memories). Only explicit signals raise the useful side: the new **`memory_useful`** tool (thirteenth MCP tool, seventh on the opencode plugin; also `open-memex useful <id...>`), a human approve at review (`promote` to approved), and a supersede. Passive exposure never counts as useful — the tool description, MCP handshake, READMEs, skill, and AGENTS.md all state the restraint so an agent that reads only one still behaves. `memory_usage` is created with `CREATE TABLE IF NOT EXISTS` outside the versioned schema and survives `rebuildIndexSchema` (ids come from the files); hard forget prunes the row, branch-switch removals do not. `memory_useful` returns counts plus `reuse_rate` (useful / retrieved, `n/a` when never retrieved). The D77-L3 reuse report (top reused, zero-reuse zombies, per-scope rates) remains open.
18
+ - D77-L2: measure usefulness by reuse (the D77 counters, layer 2). Each memory now carries local counters in a `memory_usage` table beside the rebuildable index (D77: counters live in SQLite, never in frontmatter, never off the machine): `retrieved_count` — incremented when a memory is actually returned by `memory_search`, `memory_list`, the CLI `search`/`list`, or the session-start injection — and the useful split, `useful_self` (personal memories and unpublished project drafts) vs `useful_team` (approved/published project memories). Only explicit signals raise the useful side: the new **`memory_useful`** tool (thirteenth MCP tool, seventh on the opencode plugin; also `open-memex useful <id...>`), a human approve at review (`promote` to approved, or the `--local-approve` propose shortcut), and a supersede (recorded even when the index row is momentarily missing — the Markdown file is the source of truth). Passive exposure never counts as useful — the tool description, MCP handshake, READMEs, skill, and AGENTS.md all state the restraint so an agent that reads only one still behaves. `memory_usage` is created with `CREATE TABLE IF NOT EXISTS` outside the versioned schema and survives `rebuildIndexSchema` (ids come from the files); hard forget prunes the row, branch-switch removals do not. `memory_useful` returns counts plus `reuse_rate` (useful / retrieved, `n/a` when never retrieved). The D77-L3 reuse report (top reused, zero-reuse zombies, per-scope rates) remains open.
15
19
 
16
20
  ## [0.9.0-alpha.1] - 2026-11-03
17
21
 
package/README.md CHANGED
@@ -457,7 +457,7 @@ they only matter once you share memories through Git.
457
457
  | ● `memory_forget` | Delete a memory by id (`soft=true` hides it instead — retracted, one-way) |
458
458
  | ● `memory_workingstate` | Read/write/clear the per-project working-state note (§3.6, D89) — resumes an interrupted task; scratch state, never a memory |
459
459
  | ● `memory_useful` | Mark a memory as useful (D77) — explicit confirmation only; exposure is counted separately as `retrieved_count` |
460
- | `memory_status` | Show the sync queue: outbox drafts, repo review states, uncommitted files |
460
+ | `memory_status` | Show the sync queue (outbox drafts, repo review states, uncommitted files) plus the D77 reuse report |
461
461
  | `memory_submit` | Move named drafts into the repo memory dir (local branch + commit) |
462
462
  | `memory_propose` | Copy personal memories into the project scope as review candidates |
463
463
  | `memory_promote` | Advance `proposed → approved → published` (or reject / resubmit) |
@@ -478,8 +478,12 @@ confirms it paid off, a human approval at review, or a supersede. Passive
478
478
  exposure never counts as useful — a memory that merely showed up in results
479
479
  has shown nothing yet. Same-side CLI surface: `open-memex useful <id...>`.
480
480
  Counters live in the local SQLite index only (never in the Markdown files,
481
- never off this machine); the reuse report (top reused, zero-reuse zombies,
482
- per-scope rates) rides the D77-L3 follow-up.
481
+ never off this machine). The report rides `memory_status` (and
482
+ `open-memex sync-status`): per-scope retrieved/useful rates with the self/team
483
+ split shown separately, the most-reused memories, and the "zombies" —
484
+ retrieved but never confirmed useful, i.e. the decay/prune candidates
485
+ (D77-L3). Superseded/deprecated entries stay visible as reuse history but
486
+ carry a lifecycle tag, never presented as current guidance.
483
487
 
484
488
  ## Memory types
485
489
 
package/README.zh-CN.md CHANGED
@@ -297,7 +297,7 @@ MCP 服务器暴露十三个工具;opencode 原生插件暴露其中七个核
297
297
  | ● `memory_forget` | 按 id 删除一条记忆(`soft=true` 改为隐藏——retracted,单向) |
298
298
  | ● `memory_workingstate` | 读/写/清除每项目的工作状态笔记(§3.6,D89)——让被中断的任务得以续接;只是草稿状态,永远不会成为记忆 |
299
299
  | ● `memory_useful` | 确认一条记忆有用(D77)——只认显式确认;被动的搜索/列表出现只计入 `retrieved_count`,绝不算有用 |
300
- | `memory_status` | 显示同步队列:outbox 草稿、仓库评审状态、未提交文件 |
300
+ | `memory_status` | 显示同步队列(outbox 草稿、仓库评审状态、未提交文件),并附带 D77 复用报告 |
301
301
  | `memory_submit` | 把指定的草稿移进仓库记忆目录(本地分支 + 提交) |
302
302
  | `memory_propose` | 把个人记忆复制到项目作用域,作为评审候选 |
303
303
  | `memory_promote` | 推进 `proposed → approved → published`(或拒绝 / 重新提交) |
@@ -306,7 +306,7 @@ MCP 服务器暴露十三个工具;opencode 原生插件暴露其中七个核
306
306
 
307
307
  所以 MCP 接入的编辑器永远是全套;opencode 插件主攻捕获与回想,工作流类的事走 CLI 或 MCP 接入的编辑器。
308
308
 
309
- **有用性靠复用来衡量,不靠存储量(D77)。** 每条记忆带本地计数器:`retrieved_count`——它真正被递交给助手的次数(搜索结果、清单、会话开始注入);有用计数则拆成 `useful_self`(个人记忆、未评审的项目草稿)和 `useful_team`(approved/published 的项目记忆)。只有显式信号会抬升有用侧:`memory_useful`(人确认它帮上忙时)、评审中的人为 approve、或一次 supersede。被动曝光永远不算有用——记忆只是出现在结果里,还不能说明任何事。同侧的人用入口:`open-memex useful <id...>`。计数器只存在本地 SQLite 索引里(不进 Markdown 文件、不出这台机器);复用报告(复用最多、零复用的"僵尸"、各作用域复用率)由 D77-L3 后续跟进。
309
+ **有用性靠复用来衡量,不靠存储量(D77)。** 每条记忆带本地计数器:`retrieved_count`——它真正被递交给助手的次数(搜索结果、清单、会话开始注入);有用计数则拆成 `useful_self`(个人记忆、未评审的项目草稿)和 `useful_team`(approved/published 的项目记忆)。只有显式信号会抬升有用侧:`memory_useful`(人确认它帮上忙时)、评审中的人为 approve、或一次 supersede。被动曝光永远不算有用——记忆只是出现在结果里,还不能说明任何事。同侧的人用入口:`open-memex useful <id...>`。计数器只存在本地 SQLite 索引里(不进 Markdown 文件、不出这台机器)。复用报告挂在 `memory_status`(以及 `open-memex sync-status`)上:各作用域的 retrieved/useful 比率(self/team 分开显示)、复用最多的记忆,以及"僵尸"——被检索过却从未被确认有用、即衰减/清理候选项(D77-L3)。superseded/deprecated 的记忆仍作为复用历史出现,但带生命周期标记,不会被当成当前指引。
310
310
 
311
311
  ## 记忆类型
312
312
 
package/dist/.build-stamp CHANGED
@@ -1 +1 @@
1
- a8f61bc31b972df3523d0cebe5e6664a9adc28108162385725a004ee6b116656
1
+ 507da63044e3c491b1ec5b7f077ccfbc492c986775ce97b77f172e5395fccca3
package/dist/cli.js CHANGED
@@ -10,7 +10,8 @@ import { proposeMemories, promoteMemory, listConflicts, resolveConflict, formatR
10
10
  import { getSyncStatus, formatSyncStatus, submitMemories } from "./submit.js";
11
11
  import { getPrStatus, formatPrStatus, applyPrStatus } from "./github.js";
12
12
  import { search, list, countList, hitStateLabel } from "./retrieve/search.js";
13
- import { recordUseful, getUsage, reuseRate, usefulCount, usefulKindFor, deleteUsage } from "./store/usage.js";
13
+ import { recordUseful, getUsage, reuseRate, usefulCount, usefulKindFor, deleteUsage, loadReuseReport } from "./store/usage.js";
14
+ import { formatReuseReport } from "./reuse-report.js";
14
15
  import { DEFAULT_EXPANSIONS_WEIGHT } from "./retrieve/weights.js";
15
16
  import { formatInventoryLine, truncationNote } from "./retrieve/display.js";
16
17
  import { writeMemoryFile, readMemoryFile, ulid, msToRfc3339, normalizeAliases, } from "./store/markdown.js";
@@ -206,7 +207,9 @@ Examples:
206
207
  open-memex resolve 01ABC`,
207
208
  "sync-status": `Show the project memory sync pipeline: when the index last synced
208
209
  and what triggered it, drafts waiting in the outbox (appdata), memories in the
209
- repo awaiting review or published, and repo files not yet committed.
210
+ repo awaiting review or published, and repo files not yet committed. The
211
+ D77 reuse report follows: per-scope retrieved/useful rates, most-reused
212
+ memories, and "zombies" (retrieved but never confirmed useful).
210
213
 
211
214
  Usage: open-memex sync-status
212
215
 
@@ -1400,6 +1403,16 @@ async function main() {
1400
1403
  syncScope(PERSONAL_SCOPE.key, "cli");
1401
1404
  try {
1402
1405
  console.log(formatSyncStatus(getSyncStatus(), { cap: Number.POSITIVE_INFINITY }));
1406
+ // D77-L3: same reuse block as the memory_status tool, uncapped (a
1407
+ // human asked; the CLI is where the full list lives).
1408
+ const reuse = loadReuseReport([
1409
+ { scopeKey: project.key, label: "this project" },
1410
+ { scopeKey: PERSONAL_SCOPE.key, label: "personal" },
1411
+ ], { topLimit: Number.POSITIVE_INFINITY, zombieLimit: Number.POSITIVE_INFINITY });
1412
+ console.log("");
1413
+ for (const line of formatReuseReport(reuse, { entryCap: Number.POSITIVE_INFINITY })) {
1414
+ console.log(line);
1415
+ }
1403
1416
  }
1404
1417
  catch (e) {
1405
1418
  console.error(`sync-status failed: ${e.message}`);
package/dist/init.js CHANGED
@@ -107,7 +107,9 @@ yet. Syncing them into the repo for review is an explicit, user-approved step:
107
107
  - When the user says "sync memory" (or "同步记忆"), run the sync
108
108
  flow above: call \`memory_status\`, summarize the outbox drafts, and ask which
109
109
  ones to sync. ALWAYS use the \`memory_status\` tool for this — never browse
110
- the memory data directory directly.
110
+ the memory data directory directly. That tool also carries the D77 reuse
111
+ report (per-scope rates, most-reused memories, and "zombies"): surface it
112
+ when the user asks what memory is actually being reused.
111
113
  - When the user approves, call \`memory_submit\` with the approved ids. It
112
114
  copies the drafts into the current project's \`.ai/open-memex/\` directory as
113
115
  \`proposed\` and commits locally on the CURRENT branch. It NEVER creates a
package/dist/mcp.js CHANGED
@@ -67,7 +67,9 @@ them by these full names.
67
67
  - When the user says "sync memory" (or "同步记忆"), call memory_status,
68
68
  summarize the outbox drafts (one line each), and ask which ones to sync.
69
69
  ALWAYS use the memory_status tool for this — never browse the memory data
70
- directory directly.
70
+ directory directly. memory_status also carries the D77 reuse report
71
+ (per-scope rates, most-reused, and never-used-but-retrieved "zombies") —
72
+ surface it when the user asks what memory is actually being reused.
71
73
  - After memory_submit, ask ONE follow-up: "want me to create a branch + push +
72
74
  open the PR, or will you handle it yourself?" NEVER create branches, push, or
73
75
  open PRs without the user's explicit approval. A "yes, you do it" covers the
@@ -0,0 +1,64 @@
1
+ /**
2
+ * D77-L3: the reuse report — pure types + rendering, no SQLite import (the
3
+ * data comes from `loadReuseReport` in store/usage.ts). Split like
4
+ * weights.ts so smoke-pure can test the formatting without a database.
5
+ *
6
+ * Report rules (D77):
7
+ * - per-scope reuse rate = useful / retrieved over tracked memories
8
+ * (counters exist per memory; scopes are personal vs the current project);
9
+ * - the useful side is shown split into self vs team — D77 requires reporting
10
+ * them separately so the team-knowledge claim stays honest (the combined
11
+ * total still drives ranking and the rate);
12
+ * - "top reused" = highest useful, then highest exposure;
13
+ * - "zombies" = retrieved at least once, never confirmed useful — the
14
+ * decay/prune candidates the D77 text calls out;
15
+ * - retracted/archived memories are excluded from the two lists (their
16
+ * counters still count toward the scope totals — history is history);
17
+ * - superseded/deprecated entries stay in the lists (their reuse is real)
18
+ * but carry a lifecycle tag, so historical data never reads as current
19
+ * guidance — supersede itself ticks the OLD memory's useful counter;
20
+ * - lists are capped and the cut is disclosed (D68 honesty rule).
21
+ */
22
+ function rateText(rate) {
23
+ return rate === null ? "n/a" : `${Math.round(rate * 100)}%`;
24
+ }
25
+ /**
26
+ * Render the report as plain lines. Always emits the "Reuse (D77" header, so
27
+ * the section exists (with an honest empty-state line) even before any
28
+ * retrieval has been recorded — discoverability over silence.
29
+ */
30
+ export function formatReuseReport(report, opts = {}) {
31
+ const cap = opts.entryCap ?? 5;
32
+ const lines = ["Reuse (D77 — local counters, never leave this machine):"];
33
+ if (report.tracked === 0) {
34
+ lines.push(" no data yet — search/list/session-start injection record exposure; memory_useful records usefulness.");
35
+ return lines;
36
+ }
37
+ for (const s of report.scopes) {
38
+ if (s.tracked === 0)
39
+ continue;
40
+ lines.push(` ${s.label}: retrieved ${s.retrieved}, useful ${s.useful} (self ${s.usefulSelf}, team ${s.usefulTeam}) — ${rateText(s.rate)} over ${s.tracked} memor${s.tracked === 1 ? "y" : "ies"}`);
41
+ }
42
+ // Only disambiguate each entry's scope when both scopes have data — the
43
+ // single-scope case would just repeat the same tag on every line.
44
+ const multi = report.scopes.filter((s) => s.tracked > 0).length > 1;
45
+ const tag = (e) => (multi ? `[${e.scopeLabel}/${e.type}]` : `[${e.type}]`);
46
+ // Lifecycle tag: superseded/deprecated entries are historical reuse, not
47
+ // current guidance (a supersede ticks the old memory, so it can rank high).
48
+ const life = (e) => (e.status && e.status !== "active" ? ` [${e.status}]` : "");
49
+ lines.push(` top reused: ${report.topTotal}`);
50
+ for (const e of report.top.slice(0, cap)) {
51
+ lines.push(` · ${e.id} ${tag(e)}${life(e)} retrieved=${e.retrieved} useful=${e.useful} (self ${e.usefulSelf}, team ${e.usefulTeam}) — ${e.snippet}`);
52
+ }
53
+ if (report.topTotal > cap) {
54
+ lines.push(` … and ${report.topTotal - cap} more (full list: open-memex sync-status)`);
55
+ }
56
+ lines.push(` zombies (retrieved, never confirmed useful): ${report.zombieTotal}`);
57
+ for (const e of report.zombies.slice(0, cap)) {
58
+ lines.push(` · ${e.id} ${tag(e)}${life(e)} retrieved=${e.retrieved} — ${e.snippet}`);
59
+ }
60
+ if (report.zombieTotal > cap) {
61
+ lines.push(` … and ${report.zombieTotal - cap} more (full list: open-memex sync-status)`);
62
+ }
63
+ return lines;
64
+ }
package/dist/review.js CHANGED
@@ -109,6 +109,13 @@ function copyPersonalToProject(sourceId, srcFm, srcBody, opts = {}) {
109
109
  };
110
110
  const { filePath } = writeMemoryFile(fm, srcBody);
111
111
  upsertFromFile({ fm, body: srcBody, filePath, mtimeMs: Date.now() });
112
+ // D77-L2: a local approval is still a human approval at review, so it must
113
+ // record the same useful_team tick the promote path records. It bypasses
114
+ // promoteMemory by design (solo-dev shortcut) and submit preserves the
115
+ // approved state, so no later step would otherwise count it — and the
116
+ // later `approved → published` promote stays non-counting for both paths.
117
+ if (targetState === "approved")
118
+ recordUseful(fm.id, "team", "promote");
112
119
  return { id: fm.id, filePath, reviewState: fm.review_state };
113
120
  }
114
121
  /**
@@ -35,23 +35,23 @@ export function usefulKindFor(scope, reviewState) {
35
35
  return "team";
36
36
  return "self";
37
37
  }
38
- const INSERT_RETRIEVED_SQL = `
39
- INSERT INTO memory_usage (id, retrieved_count, useful_self, useful_team, last_retrieved_at, last_useful_at, updated_at)
40
- VALUES (?, 1, 0, 0, ?, NULL, ?)
41
- ON CONFLICT(id) DO UPDATE SET
42
- retrieved_count = memory_usage.retrieved_count + 1,
43
- last_retrieved_at = excluded.last_retrieved_at,
44
- updated_at = excluded.updated_at
38
+ const INSERT_RETRIEVED_SQL = `
39
+ INSERT INTO memory_usage (id, retrieved_count, useful_self, useful_team, last_retrieved_at, last_useful_at, updated_at)
40
+ VALUES (?, 1, 0, 0, ?, NULL, ?)
41
+ ON CONFLICT(id) DO UPDATE SET
42
+ retrieved_count = memory_usage.retrieved_count + 1,
43
+ last_retrieved_at = excluded.last_retrieved_at,
44
+ updated_at = excluded.updated_at
45
45
  `;
46
46
  function insertUsefulSql(self) {
47
47
  const col = self ? "useful_self" : "useful_team";
48
- return `
49
- INSERT INTO memory_usage (id, retrieved_count, useful_self, useful_team, last_retrieved_at, last_useful_at, updated_at)
50
- VALUES (?, 0, ${self ? 1 : 0}, ${self ? 0 : 1}, '', ?, ?)
51
- ON CONFLICT(id) DO UPDATE SET
52
- ${col} = memory_usage.${col} + 1,
53
- last_useful_at = excluded.last_useful_at,
54
- updated_at = excluded.updated_at
48
+ return `
49
+ INSERT INTO memory_usage (id, retrieved_count, useful_self, useful_team, last_retrieved_at, last_useful_at, updated_at)
50
+ VALUES (?, 0, ${self ? 1 : 0}, ${self ? 0 : 1}, '', ?, ?)
51
+ ON CONFLICT(id) DO UPDATE SET
52
+ ${col} = memory_usage.${col} + 1,
53
+ last_useful_at = excluded.last_useful_at,
54
+ updated_at = excluded.updated_at
55
55
  `;
56
56
  }
57
57
  function rfc(ms) {
@@ -82,23 +82,27 @@ export function recordRetrieved(ids, atMs = Date.now()) {
82
82
  }
83
83
  export const VIA_VALUES = ["tool", "supersede", "promote"];
84
84
  /**
85
- * Count one explicit useful signal. Returns the updated row, or null when
86
- * the id is not in the index (callers surface that as not-found — a useful
87
- * tick for a memory nobody has must not create phantom telemetry).
85
+ * Count one explicit useful signal. The upsert is a single statement, so it
86
+ * is atomic on both drivers — there is deliberately no separate existence
87
+ * check: a memory can be valid on disk while its index row is momentarily
88
+ * missing (schema rebuild, or `findMemoryFile`'s file-scan fallback), and a
89
+ * guard keyed on `memories` would silently drop a real supersede/approve
90
+ * signal. The explicit surfaces (`memory_useful`, CLI `useful`) do their own
91
+ * not-found check before calling, so an agent's typo id never lands here.
92
+ * A concurrent hard forget can still leave an orphaned row; `listUsage`
93
+ * inner-joins `memories`, so orphans never surface in reports.
88
94
  */
89
95
  export function recordUseful(id, kind, _via, atMs = Date.now()) {
90
96
  const d = db();
91
- const mem = d.prepare(`SELECT id FROM memories WHERE id = ?`).get(id);
92
- if (!mem)
93
- return null;
94
97
  const at = rfc(atMs);
95
98
  d.prepare(insertUsefulSql(kind === "self")).run(id, at, at);
99
+ // getUsage cannot miss the row just upserted.
96
100
  return getUsage(id);
97
101
  }
98
102
  export function getUsage(id) {
99
103
  const row = db()
100
- .prepare(`SELECT id, retrieved_count, useful_self, useful_team,
101
- last_retrieved_at, last_useful_at, updated_at
104
+ .prepare(`SELECT id, retrieved_count, useful_self, useful_team,
105
+ last_retrieved_at, last_useful_at, updated_at
102
106
  FROM memory_usage WHERE id = ?`)
103
107
  .get(id);
104
108
  if (!row)
@@ -122,12 +126,12 @@ export function reuseRate(u) {
122
126
  /** D77-L3's selection helpers ride the same table scan for free. */
123
127
  export function listUsage(scopeKey) {
124
128
  const sql = scopeKey
125
- ? `SELECT u.id, u.retrieved_count, u.useful_self, u.useful_team,
126
- u.last_retrieved_at, u.last_useful_at, u.updated_at
127
- FROM memory_usage u JOIN memories m ON m.id = u.id
129
+ ? `SELECT u.id, u.retrieved_count, u.useful_self, u.useful_team,
130
+ u.last_retrieved_at, u.last_useful_at, u.updated_at
131
+ FROM memory_usage u JOIN memories m ON m.id = u.id
128
132
  WHERE m.scope_key = ?`
129
- : `SELECT u.id, u.retrieved_count, u.useful_self, u.useful_team,
130
- u.last_retrieved_at, u.last_useful_at, u.updated_at
133
+ : `SELECT u.id, u.retrieved_count, u.useful_self, u.useful_team,
134
+ u.last_retrieved_at, u.last_useful_at, u.updated_at
131
135
  FROM memory_usage u`;
132
136
  const args = scopeKey ? [scopeKey] : [];
133
137
  const rows = db().prepare(sql).all(...args);
@@ -138,6 +142,117 @@ export function listUsage(scopeKey) {
138
142
  }
139
143
  return rows;
140
144
  }
145
+ function oneLine(s, max = 120) {
146
+ const t = s.replace(/\s+/g, " ").trim();
147
+ return t.length > max ? t.slice(0, max - 1) + "…" : t;
148
+ }
149
+ /**
150
+ * D77-L3: assemble the reuse report for the given scopes (personal and the
151
+ * current project on every surface). One join query, computed in JS — the
152
+ * store is local and the row count is a corpus, not a warehouse.
153
+ *
154
+ * Scope totals include every row with telemetry (retracted/archived too:
155
+ * their exposure was real while it lasted) and keep the useful split —
156
+ * D77 requires self vs team reported separately. The two actionable lists —
157
+ * top reused and zombies — exclude retracted/archived, because neither
158
+ * pruning nor celebrating a withdrawn memory helps; superseded/deprecated
159
+ * rows stay (their reuse is real) and carry their status so the render can
160
+ * mark them as history rather than current guidance.
161
+ */
162
+ export function loadReuseReport(scopes, opts = {}) {
163
+ const topLimit = opts.topLimit ?? 5;
164
+ const zombieLimit = opts.zombieLimit ?? 5;
165
+ const labels = new Map(scopes.map((s) => [s.scopeKey, s.label]));
166
+ const empty = {
167
+ scopes: scopes.map((s) => ({
168
+ label: s.label,
169
+ scopeKey: s.scopeKey,
170
+ tracked: 0,
171
+ retrieved: 0,
172
+ useful: 0,
173
+ usefulSelf: 0,
174
+ usefulTeam: 0,
175
+ rate: null,
176
+ })),
177
+ top: [],
178
+ topTotal: 0,
179
+ zombies: [],
180
+ zombieTotal: 0,
181
+ tracked: 0,
182
+ };
183
+ if (scopes.length === 0)
184
+ return empty;
185
+ const keys = scopes.map((s) => s.scopeKey);
186
+ const rows = db()
187
+ .prepare(`SELECT u.id, u.retrieved_count, u.useful_self, u.useful_team,
188
+ m.scope_key, m.type, m.review_state, m.status,
189
+ substr(m.content, 1, 200) AS content
190
+ FROM memory_usage u JOIN memories m ON m.id = u.id
191
+ WHERE m.scope_key IN (${keys.map(() => "?").join(",")})`)
192
+ .all(...keys);
193
+ const byScope = new Map();
194
+ for (const s of empty.scopes)
195
+ byScope.set(s.scopeKey, s);
196
+ const tracked = [];
197
+ for (const r of rows) {
198
+ const retrieved = Number(r.retrieved_count) || 0;
199
+ const usefulSelf = Number(r.useful_self) || 0;
200
+ const usefulTeam = Number(r.useful_team) || 0;
201
+ const useful = usefulSelf + usefulTeam;
202
+ const stat = byScope.get(r.scope_key);
203
+ if (stat) {
204
+ stat.tracked++;
205
+ stat.retrieved += retrieved;
206
+ stat.useful += useful;
207
+ stat.usefulSelf += usefulSelf;
208
+ stat.usefulTeam += usefulTeam;
209
+ }
210
+ tracked.push({
211
+ id: r.id,
212
+ scopeKey: r.scope_key,
213
+ scopeLabel: labels.get(r.scope_key) ?? r.scope_key,
214
+ type: r.type,
215
+ reviewState: r.review_state ?? "draft",
216
+ status: r.status,
217
+ retrieved,
218
+ useful,
219
+ usefulSelf,
220
+ usefulTeam,
221
+ snippet: oneLine(r.content ?? ""),
222
+ });
223
+ }
224
+ for (const stat of byScope.values()) {
225
+ stat.rate = stat.retrieved > 0 ? stat.useful / stat.retrieved : null;
226
+ }
227
+ const actionable = tracked.filter((e) => e.status !== "retracted" && e.status !== "archived");
228
+ const top = actionable
229
+ .filter((e) => e.useful > 0)
230
+ .sort((a, b) => b.useful - a.useful || b.retrieved - a.retrieved || a.id.localeCompare(b.id));
231
+ const zombies = actionable
232
+ .filter((e) => e.retrieved > 0 && e.useful === 0)
233
+ .sort((a, b) => b.retrieved - a.retrieved || a.id.localeCompare(b.id));
234
+ const strip = (e) => ({
235
+ id: e.id,
236
+ scopeKey: e.scopeKey,
237
+ scopeLabel: e.scopeLabel,
238
+ type: e.type,
239
+ reviewState: e.reviewState,
240
+ status: e.status,
241
+ retrieved: e.retrieved,
242
+ useful: e.useful,
243
+ usefulSelf: e.usefulSelf,
244
+ usefulTeam: e.usefulTeam,
245
+ snippet: e.snippet,
246
+ });
247
+ return {
248
+ scopes: empty.scopes,
249
+ top: top.slice(0, topLimit).map(strip),
250
+ topTotal: top.length,
251
+ zombies: zombies.slice(0, zombieLimit).map(strip),
252
+ zombieTotal: zombies.length,
253
+ tracked: tracked.length,
254
+ };
255
+ }
141
256
  /**
142
257
  * Drop one memory's telemetry. ONLY the hard-forget path calls this (the
143
258
  * memory is gone for good); plain index removal — which fires on a branch
package/dist/tools/ops.js CHANGED
@@ -15,7 +15,8 @@ import { writeMemoryFile, readMemoryFile, ulid, msToRfc3339, MEMORY_TYPE_TAXONOM
15
15
  import { upsertFromFile, deleteFromIndex } from "../store/sync.js";
16
16
  import { findDuplicates, findTurnEcho, supersede, setStatus } from "../store/lifecycle.js";
17
17
  import { TURN_ECHO_WINDOW_MS } from "../capture/handoff.js";
18
- import { getUsage, recordUseful, reuseRate, usefulKindFor, usefulCount, deleteUsage, } from "../store/usage.js";
18
+ import { getUsage, recordUseful, reuseRate, usefulKindFor, usefulCount, deleteUsage, loadReuseReport, } from "../store/usage.js";
19
+ import { formatReuseReport } from "../reuse-report.js";
19
20
  import { db } from "../store/db.js";
20
21
  import { redact } from "../redact.js";
21
22
  import { getSyncStatus, formatSyncStatus, submitMemories, outboxDraftCount } from "../submit.js";
@@ -45,7 +46,7 @@ export const TOOL_DESCRIPTIONS = {
45
46
  memory_list: "List memories in a scope, newest first, as a numbered inventory. Each line keeps the raw fields — [type] id=… created=… source=… — say them back to the user in plain words (source=user → 'you told me this'; inference → 'I inferred this, check me'; keyword → 'caught from your own wording'), never read the raw id aloud unless they ask. When the user asks 'what do you remember about me?', use scope=both and present the inventory conversationally. The user may point at an entry by its number ('delete #3'): numbers are only valid for the listing you just produced — re-run memory_list, read the candidate back in full, and get a confirmation before calling memory_forget (deletion is permanent). include=all is the audit view (superseded versions, retracted, archived also shown). `type` filters to one memory type; `limit` caps how many entries come back.",
46
47
  memory_supersede: "Replace an existing memory with a newer version. The old memory is kept as history (status: superseded) and retrieval returns the new one. Use when a saved fact becomes outdated and should be replaced rather than duplicated. Only active memories can be superseded; the new memory inherits type, tags, and aliases unless you pass replacements. An unknown id is reported back as a failure; nothing is written.",
47
48
  memory_forget: "Delete a memory by id. Use when the user asks to forget something. To replace an outdated fact with a corrected one instead of deleting it, use memory_supersede. Deletion is permanent: before deleting, restate the memory's content to the user and get a confirmation. With soft=true the memory is hidden instead (retracted: out of lists and search, file kept, one-way).",
48
- memory_status: "Show the project memory sync pipeline: drafts waiting in the outbox (appdata), memories in the repo awaiting review or published, and any repo files not yet committed. Call this at session start, when the server reports drafts waiting for review, or when the user says 'sync memory' (or '同步记忆'); then ask the user which drafts to sync. For the memories themselves use memory_list; for PR review state use memory_pr_status. (MCP server and the `open-memex sync-status` CLI; the opencode native plugin does not expose this tool.)",
49
+ memory_status: "Show the project memory sync pipeline: drafts waiting in the outbox (appdata), memories in the repo awaiting review or published, and any repo files not yet committed — plus the D77 reuse report: per-scope retrieved/useful rates, the most-reused memories, and \"zombies\" (retrieved but never confirmed useful). Call this at session start, when the server reports drafts waiting for review, when the user says 'sync memory' (or '同步记忆'), or when they ask what memory is pulling its weight. Then ask the user which drafts to sync. For the memories themselves use memory_list; for PR review state use memory_pr_status. (MCP server and the `open-memex sync-status` CLI; the opencode native plugin does not expose this tool.)",
49
50
  memory_submit: "Move outbox drafts into the repo memory dir for review: copies the drafts in as proposed (or keeps a local approval), commits locally on the current branch, and moves the outbox originals out. Never creates a branch on its own — pass branch= only with the user's explicit approval for the full chain. Prints the push and PR commands — those need the user's explicit approval and are never run automatically. (MCP server and the `open-memex submit` CLI; the opencode native plugin does not expose this tool.) Unknown ids or a failed local commit abort the submit with the reason; nothing is published automatically.",
50
51
  memory_propose: "Copy personal memories into the project outbox as review drafts. The personal originals stay put. Use this when personal knowledge should enter a project's review flow; memory_submit then moves the drafts into the repo, and memory_promote advances drafts already there. (MCP server and the `open-memex propose` CLI; the opencode native plugin does not expose this tool.)",
51
52
  memory_promote: "Advance a project memory one step up the review ladder (proposed → approved → published), or reject it with a note. Rejected memories are never deleted — they can be revised and resubmitted. One step per call; an unknown id or a memory already published errors. Pass `by` to record the reviewer when it should differ from the default (e.g. approving on the strength of someone else's review). For brand-new outbox drafts, memory_submit is the entry point. (MCP server and the `open-memex promote` CLI; the opencode native plugin does not expose this tool.)",
@@ -493,14 +494,7 @@ export async function markMemoryUseful(args) {
493
494
  };
494
495
  }
495
496
  const kind = args.kind ?? usefulKindFor(row.scope, row.review_state);
496
- const after = recordUseful(args.id, kind, "tool");
497
- // recordUseful re-checks existence; both branches above agree on not-found.
498
- if (!after) {
499
- return {
500
- title: "memory: useful not found",
501
- output: `No memory with id ${args.id} in the index.`,
502
- };
503
- }
497
+ recordUseful(args.id, kind, "tool");
504
498
  const u = getUsage(args.id);
505
499
  const rate = reuseRate(u);
506
500
  const useful = usefulCount(u);
@@ -514,9 +508,16 @@ export async function statusMemories() {
514
508
  const st = getSyncStatus();
515
509
  const n = st.outbox.length +
516
510
  st.inRepo.filter((e) => e.reviewState === "proposed" || e.reviewState === "approved").length;
511
+ // D77-L3: the reuse report rides the status surface (the D77 text calls
512
+ // memory_status out by name). Personal + this project, capped for an agent
513
+ // result; the CLI prints the same block uncapped.
514
+ const reuse = loadReuseReport([
515
+ { scopeKey: st.scopeKey, label: "this project" },
516
+ { scopeKey: PERSONAL_SCOPE.key, label: "personal" },
517
+ ]);
517
518
  return {
518
519
  title: `memory: sync status (${n} pending)`,
519
- output: formatSyncStatus(st),
520
+ output: [formatSyncStatus(st), "", ...formatReuseReport(reuse)].join("\n"),
520
521
  };
521
522
  }
522
523
  export async function submitMemoriesOp(args) {
package/docs/V2-DESIGN.md CHANGED
@@ -580,7 +580,9 @@ requirement: personal data never touches third-party services). Benchmarks to tr
580
580
  - **Phase 4 — Future, signal-gated.** Cloud `RemoteProvider` customization only on: multi-private-repo
581
581
  sharing needs, fine-grained ACL, audit/compliance mandates · optional API-backed exporters/providers
582
582
  for enterprise knowledge systems.
583
- - **Near-term backlog — committed direction, not yet built (2026-10-07).** Stone's rule: anything said as "later" is written down here. D77-L2 ✅ built 2026-11-03 (0.9.0-alpha.2): per-memory `retrieved_count`/`useful_count` (`useful_self` vs `useful_team`) + `memory_useful` tool, counters in a `memory_usage` table beside the rebuildable index. D77-L3: `memory_status` reuse report (top reused, zero-reuse, per-scope reuse rate) feeding the Spark Tank pilot metrics. D77: external bench — LongMemEval-S run 2026-10-07 (`scripts/bench-longmemeval.ts`, 470 questions, recall_any@k: R@5 97.0% / MRR 0.909 per-question, R@5 40.6% on a pooled 22k corpus; ahead of agentmemory’s BM25-only, level with their BM25+vector). MemoryAgentBench still open as a second external data point. D79 follow-up SUPERSEDED → built as **D80** (2026-10-07): index-time synonym/translation expansion shipped; 41-query fixture recall@5 0.854 → 0.976, MRR 0.791 → 0.859. Embedding-model configurability (Stone 2026-10-07): the multilingual model is his personal need (EN↔ZH), not everyone’s — the model must be user-configurable. Timing: (1) NOW in the bench only — a `--model` flag to measure the multilingual tax (multilingual vs English-only on English data), which decides the DEFAULT. (2) AT SHIP TIME — `embedding.model` in `MyOMemoryConfig` + `SETTABLE_KEYS`, model-agnostic storage from day one (dim + model id stored with vectors, cache dir), download UX and `doctor` checks. Not before the gate passes: building config UX for a layer we haven’t decided to ship is premature. If multilingual tax ≈ 0, default multilingual (covers everyone); else default English-only with a docs note.D78: curate+submit — pre-submit lint, then confidence-tier auto-curation (auto-apply / batch-confirm / explicit human). D79 follow-ups: feedback-weighted ranking once D77 counters land; embeddings gate decision against the published baseline; reranker evaluation. Cross-lingual embedding rerank (Stone 2026-10-16, from obsidian-mind comparison): ablation showed multilingual vectors beat BM25 by +12pp R@5 on the cross-lingual slice — the one gap lexical can't cover. Design as opt-in rerank for EN↔ZH queries only, not a default path; local stays BM25-first per the retrieval split above. TDQS nit (separate PR): tighten search/list `type` to `z.enum`. Retrieval eval: grow the fixture (project-scope tier cases) as retrieval logic evolves. Retrieval split — local vs share server (Stone 2026-10-07): the two environments emphasize different mechanisms. Local (personal machine): BM25 + curated index-time expansion as the primary — deterministic, zero model download (D7), fast on CPU, index bloat bounded by the small corpus; embeddings opt-in only. Share server (team/org, enterprise): embeddings + ANN + reranker as the primary — server-grade compute with models resident, corpus too large and multi-author for any curated map; the curated map stays as a precision layer plus org glossary. The server additionally aggregates D77 useful_count across users into feedback-weighted ranking (already in D79 follow-ups) — a signal a single machine can never have. Bottom-up loop: curated pairs that many users converge on locally become org-glossary candidates. Economics flip: N users each downloading a model vs one model amortized on the server. OptMem lessons (Stone 2026-10-16, from kafkaup's OptMem post): three friction-reduction ideas worth stealing from the minimalist end of the spectrum. (1) **Token cost as spec** — measure and publish open-memex's prompt/token overhead in the README, the way OptMem advertises its "426-token prompt"; acquisition copy, not just engineering. (2) **`wake` convention** — document a session-start ritual (one `memory_recall` call at the beginning of every session), mirroring OptMem's `memo wake`; the MCP tools exist but the habit isn't specified. (3) **Capture length nudge** — a soft length cap or conciseness hint at capture time, inspired by OptMem's 280-byte hard cap; forces conciseness at write time, cheap to implement, aligned with the distill-to-AGENTS.md direction (D40). Three-tier memory zones (Stone 2026-10-18, from Westenberg "I Deleted My Second Brain" + his own reflection): the second brain should have zones like cache vs database, not "keep everything" vs "delete everything". **Hot** (active zone, cache-like: small, curated, always in context — D81 aliases, WHAT-level) · **Warm** (compost/incubation: not active but not cold; periodic review, rotted items deleted — this is "forget on purpose" as engineering) · **Cold** (full archive, searchable, not resident). The hard part is the **promotion/demotion policy** — what moves between zones, when, by what rule (the cache-replacement problem for knowledge; "useful" ≠ "recently used"). Capture restraint is a separate gate: not everything deserves to enter even cold storage. Spec the policy before building the zones. Auto-capture design brief (Stone 2026-10-19, from ai-memory comparison): open-memex wants auto-capture but under the restraint philosophy (not ai-memory's capture-everything). Five design questions: (1) **End-session trigger** — what hooks does opencode actually expose? (check before designing; ai-memory built finalize-session fallback for harnesses without SessionEnd); candidates: native hook / idle timeout / explicit command / context-near-full. (2) **Distillation** — rules first (what happened: git diff, tool calls, decision sentences; zero LLM), interpretation layer (what it means) left to human/LLM. (3) **Save-value triage** — three tiers: clearly valuable → save; clearly noise → drop; ambiguous → ask human. Boundary principle: **rather over-ask than over-save** (polluted retrieval costs more than a missed memory). (4) **Routing** — four exits: project memory / personal memory / AGENTS.md proposal (distill-agents exists) / discard; keep v1 routing rules simple. (5) **Human interaction** — one batch confirmation at session end ("save these 3? [y/n/edit]"), never per-item popups. Relations: D73 keyword capture is turn-level (no conflict); `wake` convention is session-start (this is session-end — the other bookend); three-tier zones govern where auto-captured items land. Cross-agent study (Stone 2026-10-19): open-memex has cross-agent on the roadmap — study how ai-memory does it (handoff claim-once semantics, lifecycle hooks per harness, VSCode support via MCP-only) before designing ours; borrow the semantics, not the push-injection behavior (discoverability > injection). Injection cost discipline (Stone 2026-10-19, from Harness² paper arXiv:2610.14224): "ALWAYS-ON PROSE IS THE EXPENSIVE OPTION" — every always-on word competes with the agent's output budget. Current session-start injection (8 project + 5 personal) is the expensive path; direction: move "important but not needed every turn" memories to on-demand (tool/skill-triggered loading), keep only truly every-turn items resident. Tolerable now, better not to tolerate. Component cost ladder for reference: SKILL (default, zero-cost until invoked) > SCRIPT > SUBAGENT > PLUGIN ("prose can be skipped, a hook cannot") > ALWAYS-ON TEXT (last resort). Read-less compression (Stone 2026-10-19, from caveman https://github.com/JuliusBrussee/caveman): the execution layer of injection cost discipline — compress what the agent READS before it reaches context, not just what we inject. Caveman's measured numbers: 33.2% fewer input tokens per session (CSV 98.9%, logs 98.5%, YAML 99.1%, JSON 98.5% per-file; HTML has no compressor yet). Direction for open-memex: a pre-retrieval compression pass on tool outputs and retrieved passages (structured data → compact form), originals stay on disk and pullable on demand. Two design principles borrowed: (1) never compress the user's own prompt/input (Adobe CAVEWOMAN finding: cavemanning user prompts makes answers worse); (2) leave code/tool calls/payloads verbatim — compress the wrapper, not the payload. Caveats: caveman's own rules cost ~1,160 input tokens fixed; per-request billing (Copilot-style) nullifies the saving — compression pays off on per-token billing only. Knowledge upkeep as a first-class direction (Stone 2026-10-23; external validation: Hedy Zhang's FDE field retrospective on enterprise AI knowledge bases, X article 2026-10-22 — enterprise knowledge bases die on unowned upkeep, not on retrieval: stale answers served silently, conflicts with no owner, updates nobody owns after delivery). The mechanisms already exist (supersede chains, lifecycle states, `audit` health check, curator convention); the direction is driving the human cost of upkeep toward zero: (1) staleness signals — age, contradiction with newer captures, D77 zero-reuse counters — feeding a standing review queue instead of waiting for someone to remember to run `audit`; (2) a conflict inbox — semantic conflicts and broken chains as a persistent todo; (3) the endgame is D78 confidence-tier auto-curation (auto-apply the safe tier, batch-confirm the rest). Success metric: the base stays trustworthy on minutes-per-week of human attention, not a curator shift. README pitches the shipped half of this as core concept 4 (Knowledge stays current) plus a "When facts change" comparison row. Session working state note (D89, 2026-11-02) → BUILT, shipped 2026-11-03 as D90: 12th tool `memory_workingstate` on both hosts, session-start surfacing, `audit` expiry report, `wstate` CLI; opencode write path is instruction + tool (plugin API has no Stop hook).
583
+ - **Near-term backlog — committed direction, not yet built (2026-10-07).** Stone's rule: anything said as "later" is written down here. D77-L2 ✅ built 2026-11-03 (0.9.0-alpha.2): per-memory `retrieved_count`/`useful_count` (`useful_self` vs `useful_team`) + `memory_useful` tool, counters in a `memory_usage` table beside the rebuildable index. D77-L3 ✅ built 2026-11-04 (0.9.0-alpha.3): `memory_status`/`sync-status` reuse report — per-scope rates, top reused, zero-reuse "zombies". D77: external bench — LongMemEval-S run 2026-10-07 (`scripts/bench-longmemeval.ts`, 470 questions, recall_any@k: R@5 97.0% / MRR 0.909 per-question, R@5 40.6% on a pooled 22k corpus; ahead of agentmemory’s BM25-only, level with their BM25+vector). MemoryAgentBench still open as a second external data point. D79 follow-up SUPERSEDED → built as **D80** (2026-10-07): index-time synonym/translation expansion shipped; 41-query fixture recall@5 0.854 → 0.976, MRR 0.791 → 0.859. Embedding-model configurability (Stone 2026-10-07): the multilingual model is his personal need (EN↔ZH), not everyone’s — the model must be user-configurable. Timing: (1) NOW in the bench only — a `--model` flag to measure the multilingual tax (multilingual vs English-only on English data), which decides the DEFAULT. (2) AT SHIP TIME — `embedding.model` in `MyOMemoryConfig` + `SETTABLE_KEYS`, model-agnostic storage from day one (dim + model id stored with vectors, cache dir), download UX and `doctor` checks. Not before the gate passes: building config UX for a layer we haven’t decided to ship is premature. If multilingual tax ≈ 0, default multilingual (covers everyone); else default English-only with a docs note.D78: curate+submit — pre-submit lint, then confidence-tier auto-curation (auto-apply / batch-confirm / explicit human). D79 follow-ups: feedback-weighted ranking once D77 counters land; embeddings gate decision against the published baseline; reranker evaluation. Cross-lingual embedding rerank (Stone 2026-10-16, from obsidian-mind comparison): ablation showed multilingual vectors beat BM25 by +12pp R@5 on the cross-lingual slice — the one gap lexical can't cover. Design as opt-in rerank for EN↔ZH queries only, not a default path; local stays BM25-first per the retrieval split above. TDQS nit (separate PR): tighten search/list `type` to `z.enum`. Retrieval eval: grow the fixture (project-scope tier cases) as retrieval logic evolves. Retrieval split — local vs share server (Stone 2026-10-07): the two environments emphasize different mechanisms. Local (personal machine): BM25 + curated index-time expansion as the primary — deterministic, zero model download (D7), fast on CPU, index bloat bounded by the small corpus; embeddings opt-in only. Share server (team/org, enterprise): embeddings + ANN + reranker as the primary — server-grade compute with models resident, corpus too large and multi-author for any curated map; the curated map stays as a precision layer plus org glossary. The server additionally aggregates D77 useful_count across users into feedback-weighted ranking (already in D79 follow-ups) — a signal a single machine can never have. Bottom-up loop: curated pairs that many users converge on locally become org-glossary candidates. Economics flip: N users each downloading a model vs one model amortized on the server. OptMem lessons (Stone 2026-10-16, from kafkaup's OptMem post): three friction-reduction ideas worth stealing from the minimalist end of the spectrum. (1) **Token cost as spec** — measure and publish open-memex's prompt/token overhead in the README, the way OptMem advertises its "426-token prompt"; acquisition copy, not just engineering. (2) **`wake` convention** — document a session-start ritual (one `memory_recall` call at the beginning of every session), mirroring OptMem's `memo wake`; the MCP tools exist but the habit isn't specified. (3) **Capture length nudge** — a soft length cap or conciseness hint at capture time, inspired by OptMem's 280-byte hard cap; forces conciseness at write time, cheap to implement, aligned with the distill-to-AGENTS.md direction (D40). Three-tier memory zones (Stone 2026-10-18, from Westenberg "I Deleted My Second Brain" + his own reflection): the second brain should have zones like cache vs database, not "keep everything" vs "delete everything". **Hot** (active zone, cache-like: small, curated, always in context — D81 aliases, WHAT-level) · **Warm** (compost/incubation: not active but not cold; periodic review, rotted items deleted — this is "forget on purpose" as engineering) · **Cold** (full archive, searchable, not resident). The hard part is the **promotion/demotion policy** — what moves between zones, when, by what rule (the cache-replacement problem for knowledge; "useful" ≠ "recently used"). Capture restraint is a separate gate: not everything deserves to enter even cold storage. Spec the policy before building the zones. Auto-capture design brief (Stone 2026-10-19, from ai-memory comparison): open-memex wants auto-capture but under the restraint philosophy (not ai-memory's capture-everything). Five design questions: (1) **End-session trigger** — what hooks does opencode actually expose? (check before designing; ai-memory built finalize-session fallback for harnesses without SessionEnd); candidates: native hook / idle timeout / explicit command / context-near-full. (2) **Distillation** — rules first (what happened: git diff, tool calls, decision sentences; zero LLM), interpretation layer (what it means) left to human/LLM. (3) **Save-value triage** — three tiers: clearly valuable → save; clearly noise → drop; ambiguous → ask human. Boundary principle: **rather over-ask than over-save** (polluted retrieval costs more than a missed memory). (4) **Routing** — four exits: project memory / personal memory / AGENTS.md proposal (distill-agents exists) / discard; keep v1 routing rules simple. (5) **Human interaction** — one batch confirmation at session end ("save these 3? [y/n/edit]"), never per-item popups. Relations: D73 keyword capture is turn-level (no conflict); `wake` convention is session-start (this is session-end — the other bookend); three-tier zones govern where auto-captured items land. Cross-agent study (Stone 2026-10-19): open-memex has cross-agent on the roadmap — study how ai-memory does it (handoff claim-once semantics, lifecycle hooks per harness, VSCode support via MCP-only) before designing ours; borrow the semantics, not the push-injection behavior (discoverability > injection). Injection cost discipline (Stone 2026-10-19, from Harness² paper arXiv:2610.14224): "ALWAYS-ON PROSE IS THE EXPENSIVE OPTION" — every always-on word competes with the agent's output budget. Current session-start injection (8 project + 5 personal) is the expensive path; direction: move "important but not needed every turn" memories to on-demand (tool/skill-triggered loading), keep only truly every-turn items resident. Tolerable now, better not to tolerate. Component cost ladder for reference: SKILL (default, zero-cost until invoked) > SCRIPT > SUBAGENT > PLUGIN ("prose can be skipped, a hook cannot") > ALWAYS-ON TEXT (last resort). Read-less compression (Stone 2026-10-19, from caveman https://github.com/JuliusBrussee/caveman): the execution layer of injection cost discipline — compress what the agent READS before it reaches context, not just what we inject. Caveman's measured numbers: 33.2% fewer input tokens per session (CSV 98.9%, logs 98.5%, YAML 99.1%, JSON 98.5% per-file; HTML has no compressor yet). Direction for open-memex: a pre-retrieval compression pass on tool outputs and retrieved passages (structured data → compact form), originals stay on disk and pullable on demand. Two design principles borrowed: (1) never compress the user's own prompt/input (Adobe CAVEWOMAN finding: cavemanning user prompts makes answers worse); (2) leave code/tool calls/payloads verbatim — compress the wrapper, not the payload. Caveats: caveman's own rules cost ~1,160 input tokens fixed; per-request billing (Copilot-style) nullifies the saving — compression pays off on per-token billing only. Knowledge upkeep as a first-class direction (Stone 2026-10-23; external validation: Hedy Zhang's FDE field retrospective on enterprise AI knowledge bases, X article 2026-10-22 — enterprise knowledge bases die on unowned upkeep, not on retrieval: stale answers served silently, conflicts with no owner, updates nobody owns after delivery). The mechanisms already exist (supersede chains, lifecycle states, `audit` health check, curator convention); the direction is driving the human cost of upkeep toward zero: (1) staleness signals — age, contradiction with newer captures, D77 zero-reuse counters — feeding a standing review queue instead of waiting for someone to remember to run `audit`; (2) a conflict inbox — semantic conflicts and broken chains as a persistent todo; (3) the endgame is D78 confidence-tier auto-curation (auto-apply the safe tier, batch-confirm the rest). Success metric: the base stays trustworthy on minutes-per-week of human attention, not a curator shift. README pitches the shipped half of this as core concept 4 (Knowledge stays current) plus a "When facts change" comparison row. Session working state note (D89, 2026-11-02) → BUILT, shipped 2026-11-03 as D90: 12th tool `memory_workingstate` on both hosts, session-start surfacing, `audit` expiry report, `wstate` CLI; opencode write path is instruction + tool (plugin API has no Stop hook).
584
+
585
+ - **Build order after D77-L3 (2026-11-04, Stone approved).** Once PR #74 (D77-L3 reuse report) merges, the queue is: (1) §20 / D88 deterministic half — `graph build` (structural edges only), compiled outline injected at session start, `graph neighbours` / `graph path` CLI, `graph lint` (orphans / stale / missing pages); no LLM, no ranking change. Fixes the live pain (knowing what is in memory without searching) and produces the lint input D78 consumes. (2) D79 feedback-weighted ranking — D77-L2 counters landed 2026-11-03, so this is newly unblocked; fixture + baseline first (measurement before mechanism). (3) D78 curate+submit — the upkeep endgame; consumes §20 lint output. (4) Embeddings gate decision (bench `--model` measures the multilingual tax) + opt-in cross-lingual rerank for EN↔ZH queries only; local stays BM25-first. (5) OptMem small items — token cost in README, `wake` session-start ritual, capture length nudge. (6) Three-tier memory zones policy spec (Hot/Warm/Cold promotion/demotion rules) before building; §20's compiled outline already covers part of Hot. (7) Auto-capture design — answer the five open questions first (notably which session-end trigger opencode exposes). Then measurement/candidates: MemoryAgentBench second external data point; Phase 2C native plugins (candidate, not committed); D9 LAN reference server likely dropped; D2/D11 iron-rule gate is a known safety gap awaiting a build/drop decision. Non-code track: Phase 2B 1–2 colleague pilot (needs people, not code). Dependency chain: §20 lint → D78; D77-L2 counters → D79; gate decision → rerank. Tension to respect: §20's resident outline must stay compact per the injection-cost discipline (2026-10-19) — "important but not every turn" stays on-demand.
584
586
 
585
587
  - **Phase 5 — Candidates recorded 2026-10-04 (not committed).** From the competitor scan (D75): an offline FTS5 retrieval eval with a published recall number (gates any semantic-layer decision) [promoted to next build, D77] · per-client trust levels on the review ladder · cold-start Markdown import into outbox drafts · `doctor` check that each editor's MCP registration points at the intended data root (mis-rooted servers are a real failure mode elsewhere). Sequencing and scope TBD per candidate.
586
588
  - **Phase 5 — Candidates recorded 2026-10-11 (not committed).** From the XERJ review (xerj-org/xerj, Apache-2.0; founder Ivan N requested open-memex D81 for the XERJ Corpus Hub — agreed 2026-10-11, attribution + repo link): schema-first querying (`xerj_map` — agent fetches the field map before writing query DSL; candidate: a schema/describe helper ahead of `memory_search` to cut dead queries) · token-budgeted MCP results (their `token_budget.rs` — truncate to a token budget, not a fixed row limit) · hybrid retrieval as the scale answer (BM25 + vector kNN with fused ranking, e.g. RRF — the reference pattern for the eventual embedding layer; our lexical ceiling is already measured at R@5 40.6% pooled) · NL-to-validated-query planning (`xerj_plan` — natural language in, validated query out; candidate: a query lint/validate step before search executes) · evidence-carrying memory links (typed links with evidence + confidence, soft-retire never hard-delete — same spirit as our review-state ladder; prior art for a future memory-graph direction) · honest isolation docs (state what a namespace guarantees and does NOT — query-level vs trust boundary; same spirit as our Limitations section) · cost-accounted evals (report tokens + dollars per approach, as in their case studies — the format for proving "memory makes agents cheaper"). Sequencing and scope TBD per candidate.
@@ -1110,7 +1110,7 @@
1110
1110
  *Status (2026-11-02): 已实现 — CHANGELOG 0.7.3 "Tool-definition quality pass (Glama TDQS 4.0/5 on v0.7.2): D76".*
1111
1111
 
1112
1112
  - **D77 — measure usefulness by reuse, not storage (2026-10-07).** The Spark Tank submission commits the metric: "measured by reuse — fewer repeated questions, less context-prep time, approved memories reused by non-authors — not by memory count." Three layers, in build order: (1) **Offline retrieval eval.** A synthetic fixture (memories + queries with gold ids) run against the FTS5 baseline; publish recall@k/MRR in the README. This also answers the D75 "retrieval evaluation gate" open question — the number gates any future semantic-layer decision (reference points: agentmemory's published LongMemEval-S R@5 95.2%; lotargo/memory_plugin's BM25-only vs hybrid table). Later, run the MemoryAgentBench harness for directly comparable numbers (Knowl 90 / Mem0 18 / MIRIX 14 / Zep 7). (2) **Online reuse counters.** Per memory: `retrieved_count` (returned by search/list/first-turn injection) and `useful_count`, split into `useful_self` (author's own reuse — the personal-utility story) and `useful_team` (reuse of published project memories across sessions — the Spark Tank "reused by non-authors" story; no identity system needed: published state plus session diversity is the proxy). Both count as useful; reporting them separately keeps the team-knowledge claim honest. Explicit confirmation: a `memory_useful` tool call, human approve at review; behavioral: supersede/promote referencing it. Passive surfacing must never refresh usefulness — the opencode-memory lesson: unguarded exposure creates a self-reinforcing retrieval loop. Derived: `reuse_rate = useful / retrieved`; "zombie memories" (retrieved but never confirmed useful) surface in `memory_status` as decay/prune candidates. Counters live in SQLite, not frontmatter; local-only. (3) **Status surfaces.** `memory_status` gains a reuse report: top reused, zero-reuse list, per-scope reuse rate — feeding the Spark Tank pilot metrics. Moat note: competitors' "useful" is agent self-reported; our review ladder gives a human-grounded signal — approved/published means a human already said it is worth sharing. **Priority: this is the next build after 0.7.3** — fixture eval first (1–2 days, publishable number), then counters + `memory_useful`, then the status report, then the external bench.
1113
- *Status (2026-11-03): 部分实现 — 第 1 层 eval 已发;第 2 层已实现(0.9.0-alpha.2:`memory_usage` 表(独立于可重建索引,跨 rebuild 存活)+ `retrieved_count`/`useful_self`/`useful_team` + `memory_useful` 工具与 `open-memex useful` CLI;分类代理:personal 或未评审项目草稿→self,approved/published 项目记忆→team;supersede 与评审 approve 各自动计一次 useful;曝光只走 search/list/injection 的显式 `record` 开关);第 3 层(reuse 报告)未实现.*
1113
+ *Status (2026-11-04): 已实现 — 三层全建。第 1 层 eval 已发;第 2 层(0.9.0-alpha.2):`memory_usage` 表(独立于可重建索引,跨 rebuild 存活)+ `retrieved_count`/`useful_self`/`useful_team` + `memory_useful` 工具与 `open-memex useful` CLI,分类代理:personal 或未评审项目草稿→self,approved/published 项目记忆→team,supersede 与评审 approve(含 `--local-approve`)各自动计一次 useful,曝光只走 search/list/injection 的显式 `record` 开关;第 3 层(0.9.0-alpha.3):`memory_status` / `sync-status` 复用报告(各作用域 retrieved/useful 比率且 self/team 分开显示、top reused、zombie=被检索却从未确认有用;superseded/deprecated 带生命周期标记,避免历史复用被读成当前指引;retracted/archived 不计入两列表,总数仍计)。仍未跑:MemoryAgentBench 第二个外部数据点(属第 1 层延伸,不在三层机制内)。*
1114
1114
 
1115
1115
  - **D78 — pre-submit curation: human-on-the-loop, not in-the-loop (2026-10-07).** Fragmented captures must be organized before submit, but the submitter's patience is the scarce resource — every manual gate lowers the submit rate, so curation defaults to automatic and the human handles exceptions. Confidence tiers: **auto-apply** (exact duplicates, trivial fragments, inferred type/tags, title normalization); **batch-confirm** (semantic merges, splits — one summary screen, one tap to accept or undo, never per-item approval); **explicit human decision** (dropping substantive content, and submit itself — the D-hard boundary "project scope never auto-submits" stands: automation covers organizing, sharing stays human). The safety net is the supersede chain: every auto-merge preserves provenance and is undoable, so pre-approval is replaced by reversibility (undo-send, not confirm-dialog). Trigger: no new step to remember — `submit` becomes curate+submit in one command, with the curation summary attached to the PR description; session wrap-up is the alternative hook (batch-silent, per the D61 checkpoint posture). Closed loop: human undos feed back into confidence thresholds, consuming the D77 `useful_count` signals. This revises the earlier review-ladder instinct of per-item approval: the human moves from in-the-loop to on-the-loop.
1116
1116
  *Status (2026-11-02): 未实现 — 纯设计条目:confidence 三档(auto-apply / batch-confirm / explicit human)、`submit` 变 curate+submit、undo-send 替代 pre-approval 均无代码.*