open-memex 0.6.0 → 0.6.1-alpha.3

package/docs/CONCEPTS.md

@@ -0,0 +1,181 @@
+# Concepts — how open-memex works
+
+[中文版](./CONCEPTS.zh-CN.md)
+
+> This document explains open-memex's **mental model**: where your memories
+> live, how they flow, and who can see them. It is not a how-to — for
+> installation and commands, start with the [README](../README.md). Read this
+> when you want to predict what open-memex will do before it does it.
+
+## In one sentence
+
+open-memex is a local-first memory layer: **Markdown is the source of truth;
+SQLite is just a rebuildable index.** Lose the index and you rebuild it.
+Lose the Markdown and it's really gone.
+
+## 1. Two scopes: personal vs project
+
+Every memory belongs to a scope, which decides **how far it may travel**:
+
+| | personal | project |
+|---|---|---|
+| Holds | Facts about you: preferences, habits, cross-project info | Facts about the project: stack, conventions, decisions, gotchas |
+| Lives | Only on your machine (see §2) | Local + eligible for team sharing (see §2, §5) |
+| Rule | **Never leaves this machine** | Can be promoted to team knowledge |
+
+On the opencode native plugin, keyword capture routes automatically:
+"remember for me… / 记住我… / 我觉得… / 我喜欢…" (about me) → personal;
+"我们决定… / 帮我们记住…" (about us) and everything else → project.
+Everywhere else, the scope is whatever `memory_add` / `--scope` says.
+
+## 2. Two homes: appdata vs the repo directory
+
+| | appdata (the desk) | repo `.ai/open-memex/` (the shelf) |
+|---|---|---|
+| Location | Windows `%APPDATA%/open-memex`, macOS/Linux `~/.local/share/open-memex` | Project root, default `.ai/open-memex/` (configurable via `memoryDir`) |
+| Holds | `memories/personal/` personal memories; project-scope **draft outbox**; `index.db` local index | Submitted project memories (`proposed` and up) |
+| In git? | No | Yes — git is its courier |
+| The index? | `index.db` is rebuildable, **never committed** | No index stored; rebuilt from Markdown on demand |
+
+The private notebook (personal) never goes on the shelf. That's an iron rule, not a setting.
+
+Project memories start life as **drafts in the appdata outbox** — git-invisible,
+branch-independent. Only drafts you explicitly name move to the shelf, via
+`open-memex submit` (§5). One stage, one home: after a successful submit the
+outbox original is gone; before it, the repo knows nothing.
+
+## 3. First-turn injection: which 8 project + 5 personal memories get chosen
+
+Before the agent's first turn, open-memex injects an `[OPEN-MEMEX]` context block so it
+doesn't need a search call just to get oriented. Defaults: **8 project + 5 personal**
+memories (`maxProjectMemories` / `maxProfileItems`, tunable).
+
+Selection has exactly one criterion: **most recently updated first**
+(`ORDER BY updated_at DESC`), minus retracted/archived items, with superseded
+versions resolved away. Not "the N most important" — "the N most recently touched".
+What you just updated is most likely relevant to what you're doing now.
+
+Token budget: each item is squeezed to one line (≤240 chars, ~60 tokens), so 8 items
+cost ≈ 500 tokens worst case. Beyond that, diminishing returns — the long tail is what
+`memory_search` is for; stuffing everything into the first turn buries the signal.
+
+Raise it: `open-memex config set maxProjectMemories 12`
+
+## 4. Later turns: when memory gets searched again
+
+First-turn injection happens once. After that there is **no automatic re-search** —
+MCP is request/response; the server cannot push. Whether `memory_search` gets called
+is entirely the model's judgment, guided by two things:
+
+- The injected block's footer: "Use the `memory_search` tool to look up more."
+- The instructions written by `init`: "before asking the user about something
+  they may have told you before, call `memory_search` first — try a few keyword
+  variants before giving up."
+
+So: when the conversation touches past decisions, preferences, or conventions, the
+model *should* search first — but nothing enforces it. This is an honest boundary of
+the current design: on the opencode native plugin, first-turn injection is built in,
+but later-turn recall still depends on the agent choosing to search. Over MCP there
+is no hard session hook at all — guidance only.
+
+## 5. How a memory becomes team knowledge
+
+A personal observation becomes team knowledge by exactly one road —
+**explicit promotion, never automatic sync**:
+
+```
+personal idea ──propose──▶ outbox draft ──submit──▶ proposed ──┬──promote──▶ approved ──promote──▶ published/shared
+                                                              │                        │
+                                                           --reject                 --reject
+                                                              │                        │  (approval withdrawn
+                                                              ▼                        │   before merge)
+                                                           rejected ──resubmit──▶ proposed
+```
+
+Keep the four jobs straight — most confusion comes from mixing them up:
+
+- **propose crosses the boundary** (personal → project outbox; the only step
+  that copies across). It *copies* one or more personal memories into the
+  outbox as review candidates, each with a new id and a `derived_from` pointer
+  back. The personal original always stays. Solo? `--local-approve` lets you
+  self-approve.
+- **submit moves drafts into the repo** (outbox → `.ai/open-memex/`,
+  `draft → proposed`). It makes a **local** commit **on your current branch** —
+  it never creates a branch on its own, never pushes. All-or-nothing: same id
+  with different content aborts for a human to decide.
+- **promote only flips the status label** on a file already in
+  `.ai/open-memex/` (`proposed → approved → published`, or `--reject` with a
+  note — also allowed from `approved`, withdrawing approval before merge). It
+  never moves files between directories. Every transition is appended to the
+  memory's `review_history` (who / when / from → to / why), so the trail
+  survives the PR.
+- **git does the transport** (push, PR, merge).
+
+Review itself needs no special UI. A memory PR contains **only memory files**,
+reviewed and audited separately from code PRs; reviewers check "is this true?
+is it safe to share? any secrets?" — things a code PR's CI never checks.
+"Request changes" needs no command: while the PR is open, edit the file, commit,
+push; the state stays `proposed` and the PR is the review mechanism.
+
+Two helpers close the loop between GitHub and the local index:
+
+- `open-memex pr-status [--apply]` maps the branch PR's state onto each
+  in-repo memory (merged → `published`, approved → `approved`, changes
+  requested → a suggestion only, never an auto-reject). Report by default;
+  `--apply` performs the transitions locally.
+- `open-memex resolve [id]` lists conflicted memory files, or field-level
+  3-way-merges one. Semantic conflicts are **reported, never auto-resolved**.
+
+**A rejection never deletes anything.** The file stays on your branch; what
+happens next is the human's call:
+
+1. **Accept**: close the PR and delete the branch — the file goes with it;
+2. **Revise and resubmit**: edit, `promote --resubmit`, commit, push — review
+   continues on the same PR;
+3. **Keep as a record**: leave it visible with a `[rejected]` tag and your
+   note, so the team can see what was considered and why not.
+
+Finally: `list` / `search` show the review state (`[draft]` / `[proposed]` /
+`[approved]` / `[published]` / `[rejected]`) next to project memories, and
+retrieval ranks reviewed knowledge (`approved` / `published`) above unreviewed
+outbox drafts — drafts stay findable, but never pose as vetted.
+
+## 6. Sync: git is the courier, not the brain
+
+- **Write**: `memory_add` (project scope) → writes the **appdata outbox** and
+  updates the local index. **Never touches the repo, never auto-commits,
+  never auto-pushes.**
+- **Submit** (explicit, your call): `open-memex submit <id...>` → local commit
+  into `.ai/open-memex/`; push/PR commands are printed for you (or done by your
+  agent on your Yes).
+- **Pull**: `open-memex pull` (always explicit, never automatic) → git fetch +
+  fast-forward → scans `.ai/open-memex/*.md` → merges into the local `index.db`
+  by file mtime. Retrieval always goes through SQLite, never walks git.
+- **personal scope**: never syncs (§1 iron rule).
+- **Projects without git**: keep working; project scope degrades to
+  path-keyed local memories with a clear notice. Nothing breaks.
+
+## 7. Security model
+
+- **Memory is data, not instructions.** Memories come back into the agent's
+  context as text to *use*, never as commands to obey — including memories
+  that arrive via sync or were written by other people. There is no automated
+  screening of shared memory; the real boundary is the human review in §5
+  ("is this true? is it safe? any secrets?") plus this data/instruction
+  separation. Read shared memories the way you'd read a wiki page a stranger
+  edited.
+- **Provenance travels with the file.** Each memory records its `source`,
+  and review-track memories carry `proposed_by` / `approved_by` plus the
+  append-only `review_history` — all inside the Markdown file itself, so the
+  trail survives branches, PRs, and exports.
+- **Secrets are stopped at write time.** `<private>…</private>` sections are
+  stripped before saving; recognized API keys and tokens are masked in place
+  (first 4 characters kept, so you can tell which key it was) and the write
+  proceeds. Preview with `open-memex capture --dry-run`.
+- **personal never enters git** (§1) — not by `submit`, not by `export`
+  defaults, not ever.
+
+---
+
+*Design record: `docs/V2-DESIGN.md` (append-only decision log). Per-scope
+details: `docs/SCOPES.md`. Reviewer convention: `docs/CURATOR.md`.*

package/docs/CONCEPTS.zh-CN.md

@@ -0,0 +1,163 @@
+# 概念 —— open-memex 是怎么工作的
+
+[English](./CONCEPTS.md)
+
+> 本文档解释 open-memex 的**心智模型**：你的记忆住在哪里、如何流动、谁能看见。
+> 这不是操作手册——安装与命令请看 [README](../README.zh-CN.md)。当你想在
+> open-memex 动手之前先预判它会怎么做时，读这份。
+
+## 一句话
+
+open-memex 是一个本地优先的记忆层：**Markdown 是事实来源；
+SQLite 只是可重建的索引。**丢了索引可以重建。丢了 Markdown 就真的没了。
+
+## 1. 两种作用域：personal vs project
+
+每条记忆属于一个作用域，它决定**这条记忆最远能到哪**：
+
+| | personal | project |
+|---|---|---|
+| 装什么 | 关于你的事实：偏好、习惯、跨项目信息 | 关于项目的事实：技术栈、约定、决策、坑 |
+| 住在哪 | 只在你的机器上（见 §2） | 本地 + 可以进入团队共享（见 §2、§5） |
+| 铁则 | **永不离开这台机器** | 可以被提升为团队知识 |
+
+在 opencode 原生插件上，关键词捕获会自动分流：
+"记住我… / 替我记… / 我觉得… / 我喜欢…"（关于我）→ personal；
+"我们决定… / 帮我们记住…"（关于我们）和其余一切 → project。
+其他接入方式下，作用域由 `memory_add` / `--scope` 明确给出。
+
+## 2. 两个家：appdata vs 仓库目录
+
+| | appdata（书桌） | 仓库 `.ai/open-memex/`（书架） |
+|---|---|---|
+| 位置 | Windows `%APPDATA%/open-memex`，macOS/Linux `~/.local/share/open-memex` | 项目根目录，默认 `.ai/open-memex/`（可用 `memoryDir` 配置） |
+| 装什么 | `memories/personal/` 个人记忆；project 作用域的 **draft 发件箱**；`index.db` 本地索引 | 已提交的 project 记忆（`proposed` 及以上） |
+| 进 git？ | 不进 | 进——git 是它的快递员 |
+| 索引？ | `index.db` 可重建，**永不提交** | 不存索引；按需从 Markdown 重建 |
+
+私人笔记本（personal）永远不上书架。这是铁则，不是设置项。
+
+Project 记忆出生在 **appdata 发件箱里当 draft**——git 看不见、与分支无关。
+只有你点名的 draft 才会通过 `open-memex submit`（§5）上书架。一个阶段一个家：
+submit 成功后发件箱原件消失；成功之前，仓库什么都不知道。
+
+## 3. 首轮注入：8 条 project + 5 条 personal 是怎么选出来的
+
+在助手第一轮回复前，open-memex 会注入一个 `[OPEN-MEMEX]` 上下文块，
+让它不必先调一次搜索就能进入状态。默认 **8 条 project + 5 条 personal**
+（`maxProjectMemories` / `maxProfileItems`，可调）。
+
+选择标准只有一个：**按最近更新优先**
+（`ORDER BY updated_at DESC`），去掉已撤回/归档项，旧版本被
+supersede 掉的也不算一份。不是"最重要的 N 条"，是"最近碰过的 N 条"。
+你刚更新过的东西，最可能与你眼下在做的事有关。
+
+Token 账：每条压成一行（≤240 字符，约 60 tokens），8 条最坏约 500 tokens。
+再多收益递减——长尾是 `memory_search` 的活；把所有东西都塞进首轮只会淹没信号。
+
+调大：`open-memex config set maxProjectMemories 12`
+
+## 4. 后续轮次：什么时候会再次搜索记忆
+
+首轮注入只发生一次。之后**没有自动重搜**——
+MCP 是请求/响应模式，服务端没法主动推。调不调 `memory_search`
+完全取决于模型自己的判断，靠两处引导：
+
+- 注入块的页脚："Use the `memory_search` tool to look up more."
+- `init` 写入的编辑器规则："before asking the user about something
+  they may have told you before, call `memory_search` first — try a few keyword
+  variants before giving up."
+
+所以：当对话碰到过往决策、偏好或约定时，模型*应该*先搜——
+但没有任何机制强制它。这是当前设计的真实边界：在 opencode 原生插件上
+首轮注入是内建的，但后续轮次的回想仍取决于助手是否主动搜索；
+走 MCP 时没有硬会话钩子，只有引导。
+
+## 5. 一条记忆如何成为团队知识
+
+个人观察变成团队知识只有一条路——
+**显式提升，永不自动同步**：
+
+```
+personal idea ──propose──▶ outbox draft ──submit──▶ proposed ──┬──promote──▶ approved ──promote──▶ published/shared
+                                                              │                        │
+                                                           --reject                 --reject
+                                                              │                        │  (merge 前撤回批准)
+                                                              ▼                        │
+                                                           rejected ──resubmit──▶ proposed
+```
+
+把四个动作的分工理清——大多数困惑都来自混淆它们：
+
+- **propose 跨界**（personal → project 发件箱；唯一一步跨作用域拷贝）。
+  它把一条或多条个人记忆*拷贝*进发件箱当评审候选，各自生成新 id、
+  `derived_from` 指回原件。原件永远保留。单人开发？`--local-approve` 自批。
+- **submit 把 draft 搬进仓库**（发件箱 → `.ai/open-memex/`，
+  `draft → proposed`）。它在你的**当前分支**上做**本地** commit——
+  绝不自作主张建分支、绝不 push。全有全无：同 id 不同内容会中止，交人裁决。
+- **promote 只翻状态标签**，文件已在 `.ai/open-memex/` 里
+  （`proposed → approved → published`，或 `--reject` 附说明——
+  也可从 `approved` 驳回，即 merge 前撤回批准）。它从不在目录间搬文件。
+  每次流转都追加进该记忆的 `review_history`（谁/何时/从→到/为什么），
+  轨迹熬得过 PR 关掉。
+- **git 负责运输**（push、PR、merge）。
+
+评审本身不需要专用 UI。一个记忆 PR **只含记忆文件**，
+与代码 PR 分开评审审计；审稿人看的是"这是真的吗？能共享吗？有密钥吗？"——
+这是代码 PR 的 CI 永远不查的事。"Request changes" 不需要命令：
+PR 开着时改文件、commit、push 即可，状态保持 `proposed`，PR 就是评审机制。
+
+两个助手把 GitHub 和本地索引的圈闭上：
+
+- `open-memex pr-status [--apply]` 把分支 PR 的状态映射回每条仓库内记忆
+  （已合并 → `published`，已批准 → `approved`，要求修改 → 仅建议、绝不自动驳回）。
+  默认只报告；`--apply` 在本地执行这些流转。
+- `open-memex resolve [id]` 列出冲突的记忆文件，或对某条做字段级三路合并。
+  语义冲突**只报告、绝不自动解决**。
+
+**驳回永远不删任何东西。**文件留在你的分支上，接下来由人决定：
+
+1. **接受**：关 PR 删分支——文件随之而去；
+2. **修改重提**：改文件、`promote --resubmit`、commit、push——同一 PR 继续评审；
+3. **留作记录**：留着，带 `[rejected]` 标签和你的说明，
+   让团队看见考虑过什么、为什么没过。
+
+最后：`list` / `search` 会在 project 记忆旁标出评审状态
+（`[draft]` / `[proposed]` / `[approved]` / `[published]` / `[rejected]`），
+并且检索排序让已评审知识（`approved` / `published`）压过未评审的发件箱 draft——
+draft 找得到，但绝不冒充已审。
+
+## 6. 同步：git 是快递员，不是大脑
+
+- **写**：`memory_add`（project 作用域）→ 写**appdata 发件箱**
+  并更新本地索引。**永不碰仓库、永不自动 commit、永不自动 push。**
+- **提交**（显式、你说了算）：`open-memex submit <id...>` → 本地 commit
+  入 `.ai/open-memex/`；push/PR 命令打出来给你看（或你的助手在你点头后代劳）。
+- **拉**：`open-memex pull`（永远显式、永不自动）→ git fetch +
+  fast-forward → 扫 `.ai/open-memex/*.md` → 按文件 mtime 合入本地 `index.db`。
+  检索永远走 SQLite，不翻 git。
+- **personal 作用域**：永不同步（§1 铁则）。
+- **无 git 的项目**：照常工作；project 作用域退化为按路径索引的本地记忆，
+  并给出明确提示。什么都不会坏。
+
+## 7. 安全模型
+
+- **记忆是数据，不是命令。**记忆以文本形式回到助手的上下文里，
+  是*用的*，不是*听的*——包括通过同步拉来的、或别人写的记忆。
+  共享记忆没有自动化筛查；真正的边界是 §5 的人工评审
+  （"这是真的吗？能共享吗？有密钥吗？"）加上这条数据/指令分离。
+  读共享记忆，要像读陌生人编辑过的 wiki 页面。
+- **出处随文件旅行。**每条记忆记着 `source`，
+  评审轨迹里的记忆还带 `proposed_by` / `approved_by`
+  和只追加的 `review_history`——全部写在 Markdown 文件自身里，
+  所以轨迹熬得过分支、PR 和导出。
+- **密钥在写入时拦下。**`<private>…</private>` 段在保存前剥掉；
+  识别出的 API 密钥/token 就地打码（保留前 4 位，方便认出是哪把）
+  然后继续写。用 `open-memex capture --dry-run` 预览。
+- **personal 永不进 git**（§1）——`submit` 不行、`export` 默认不行、
+  任何方式都不行。
+
+---
+
+*设计记录：`docs/V2-DESIGN.md`（只追加的决策日志）。按作用域的细节：
+`docs/SCOPES.md`。审稿人约定：`docs/CURATOR.md`。*

package/docs/V2-DESIGN.md

@@ -1134,11 +1134,91 @@ requirement: personal data never touches third-party services). Benchmarks to tr
   tells agents to prefer MCP tools when available. *Lesson: don't ship two
   overlapping integrations for the same editor — the agent pays the
   duplication cost in chatter. Reported 2026-10-01.*
+- **D56** — question-shaped searches and host parity, said out loud
+  (0.6.1-alpha.1). Two-persona review (Stone 2026-10-03) converged on small,
+  honest fixes rather than new machinery. (1) Retrieval already OR-ed terms
+  and ranked with bm25, but fed whole questions ("how do we configure…",
+  "中文记忆怎么检索") straight into FTS — question words matched everything
+  and diluted ranking. Query construction now filters a small EN/CJK
+  function-word set, dedupes, and caps terms at 24 per side, with a
+  never-empty fallback (query.ts; partially answers the retrieval-robustness
+  open question — layers 2 agent-side expansion / 3 capture-time aliasing
+  remain open, embeddings stay reserved). (2) The plugin(5)/MCP(11) tool
+  asymmetry is now stated in the six MCP-only tool descriptions themselves,
+  with the CLI equivalent named, so an agent looking for `memory_status`
+  inside opencode finds the answer instead of assuming breakage (the
+  host-parity open question's "document it" option; true parity undecided).
+  (3) `init`'s closing lines gain a one-line privacy statement at the moment
+  of maximum doubt: memories live only on this machine, personal ones never
+  leave it, nothing is uploaded.
+
+- **D57** — explainable search, a memory audit, and the concurrency posture
+  written down (0.6.1-alpha.2; addresses most of the retrieval-explainability
+  / memory-health open question and establishes the concurrent-sessions
+  behavior it asked for). (1) `open-memex search --explain` prints the
+  constructed FTS expression, per-hit bm25 score, and how many matches
+  lifecycle hid (superseded vs retracted/archived), so a miss is diagnosable
+  — bad query, stale memory, or chain-hidden — without reading the index.
+  (2) New `open-memex audit` (read-only; doctor checks the environment, this
+  checks the memories): near-duplicate active pairs (same ≥ 0.8 Jaccard as
+  write-time dedup), actives untouched for 90+ days, broken supersede chains,
+  personal files found inside the in-repo memory dir (iron-rule breach), and
+  index/file drift. Reports only; fixes stay with `supersede` / `forget` by
+  hand. (3) Concurrency, measured: every process opens its own connection to
+  one WAL index; memory files are per-memory ULIDs, so concurrent writes to
+  *different* memories never collide on disk. SQLite was the gap — no
+  `busy_timeout` was set, so better-sqlite3 silently tolerated 5s while
+  bun:sqlite (opencode plugin) timed out at 0 and failed instantly under any
+  overlapping write. Now unified at `PRAGMA busy_timeout = 5000` in `db()`.
+  Past the timeout the CLI prints one plain line ("store is busy … nothing
+  was lost") instead of a stack trace — and nothing *is* lost: the memory
+  file is written before the index update, so the next sync reconciles it
+  (verified end-to-end: a write blocked 8s still lands once the lock frees).
+  Memory file writes are tmp+rename, so a concurrent reader never sees a
+  torn file. Same-memory concurrent edits remain last-writer-wins (accepted:
+  different processes editing the same memory simultaneously is a human
+  conflict, not a storage one). `doctor` reports the live locking settings.
+
+- **D58** — the opencode plugin must never runtime-import the host SDK
+  (0.6.1-alpha.3). Field report: after a global `npm i -g open-memex`, the
+  plugin was configured in `~/.config/opencode/opencode.jsonc` but silently
+  inactive — no memory tools, no error, nothing in opencode's log. Bisected
+  with the real host (opencode 1.18.34) and a minimal plugin: a bare
+  `file://` entry loads fine; adding `import { tool } from
+  "@opencode-ai/plugin/tool"` makes opencode skip the plugin without a
+  trace. That import only ever resolved from a source checkout (where
+  `npm install` provides the devDependency); from a global install's
+  location nothing provides the module and the host does not resolve it
+  for file:// plugins. The helper is a runtime identity (`return input`),
+  so the fix is a local same-signature stand-in in `tools/memory.ts` —
+  types still come from the SDK (type-only imports erase at runtime), and
+  the plugin now depends only on real runtime dependencies. Verified in
+  the real host against a global-install-shaped tree: all five tools
+  register. Rule going forward: plugin code imports host SDK types only;
+  runtime imports must resolve from the package's own dependencies.
 
 ## Open Questions
 
 _All resolved — see D10 (rename), D11 (type/role split), D12 (explicit pull)._
 
+- **Checkpoint proactivity for capture (open, 2026-10-02):** D53 retired checkpoint *polling* — the server pushes draft counts instead. But agent proactivity at task checkpoints genuinely helps knowledge capture (§3.5 distillation), so the open question is the degree: proactive enough to catch distillable moments, without nagging the user or burning tokens at every checkpoint. Candidate directions (server-pushed hints in tool results, client-side heuristics, or a middle setting) to be evaluated in a later version; default posture TBD.
+
+- **Retrieval robustness: phrasing, synonyms, extra words (open, 2026-10-03):** keyword search matches word forms, not meaning — the asker phrases it differently, adds a word, or uses a synonym and the memory is missed. The current FTS5 setup (porter stemming + pre-tokenized CJK bigrams) already covers morphology and substrings; it cannot bridge semantics. Layered candidate directions, cheapest first: (1) query construction — never feed a raw sentence to FTS; extract content terms, drop stopwords, OR them and let bm25 rank, so extra words hurt ranking instead of killing the match; (2) agent-side query expansion as an explicit protocol in the skill/instructions (decompose the question into concepts, generate 2–3 synonym variants per concept, run several searches before giving up — the LLM is the synonym dictionary, no embedding model needed); (3) capture-time aliasing — when a memory is written, the agent attaches keyword/synonym aliases so future phrasings can hit it. Embeddings are the design's reserved optional capability: real semantic matching, but they require a design decision and a local model to distribute, so they stay deferred unless (1)–(3) prove insufficient. Note (3) pairs naturally with the proactive-capture question below: the same capture pass that proposes a memory can attach its aliases.
+
+- **Proactive capture: agent judgment of what is worth remembering (open, 2026-10-03):** companion to the checkpoint-proactivity question above — that one asks *when* to prompt; this one asks *what qualifies*. Agent hosts with good long-term memory (e.g. Muse) get their results from explicit memorability criteria plus checkpoint consolidation, not from silent background magic; open-memex can adopt both halves. Candidate criteria to teach via the skill/instructions: decisions **and their rationale**, preferences, conventions, gotchas, and abandoned paths ("we tried X, it failed because…") are worth proposing; one-off task details and anything re-derivable from the code are not. Two postures to evaluate: propose-at-checkpoint (agent asks "worth saving?", user vetoes, then `memory_add`) versus auto-draft (agent writes outbox drafts unasked and surfaces them in a batch at the next checkpoint — drafts are git-invisible and deletable, so the cost of a wrong guess is low). Hard boundaries either way: project scope never auto-submits, and no cross-session background consolidation is possible — the plugin lives inside sessions, so consolidation happens at in-session checkpoints and wrap-up passes only. This question also decides how much aliasing (retrieval question above) the capture pass should generate.
+
+- **Non-programmer users: PMs, BAs, and entry points (open, 2026-10-03):** the current experience assumes a programmer's toolchain — terminal, npm install, editor plugin, git branches, PR review. Project managers and BAs have none of that, and their most likely entry points are not the VS Code chat window: at most an agentic CLI (opencode, Claude Code), more probably Teams chat, MS Copilot, or another chat surface calling the same MCP server. The mechanics must hide behind two confirmations ("remember this?" / "share with the team?") — scope/promote/supersede/outbox vocabulary is a programmer dialect they should never have to learn. Three roles to design for, in increasing order of build cost: (1) **sources** — their meetings and decisions are the highest-value project memory, captured by someone else's agent with attribution in `source`; needs nothing new. (2) **curators** — "is this true? is it safe to share?" is business-truth review, which BAs are often better placed to do than programmers; GitHub's web PR review is the only existing no-CLI door, and CURATOR.md would need a plain-language path for reviewers who have never touched git. (3) **direct users** — capture sources shift from chat commands to meeting transcripts, email, and tickets; and without git there is no courier: a shared-folder transport is mechanically easy (memories are Markdown files) but has no PR-equivalent review step, so a substitute review flow has to be designed. Also weigh: stakeholder/customer facts carry PII and retention weight that code conventions never do, so "is it safe to share" becomes a compliance question, not a hygiene one. If a company ever gives every employee one unified agent entry point (a work buddy / AI team mate), most of this audience split dissolves — the question is what open-memex should assume until then: heterogeneous entry points, one memory service behind them.
+
+- **Retrospective distillation: mining host transcripts after the fact (open, 2026-10-03):** open-memex captures only in-session (keyword triggers, explicit `memory_add`) and keeps **no transcript** — so anything not captured during the conversation is gone once it ends. Hosts with durable memory (e.g. Muse) avoid this loss with two layers open-memex skipped: full transcript retention plus background distillation. The gap is architectural, not a timing detail: the raw material for a second chance does not exist on our side. But the hosts themselves already retain it — opencode persists sessions and messages in its local store; Claude Code keeps JSONL transcripts under `~/.claude/projects/`; VS Code Copilot's chat state is locked in workspace storage (readable, fragile, not a first target). Candidate direction: a "replay distillation" flow that exports a past session's transcript and has an agent mine it in a dedicated review session, with every extracted candidate landing as an **outbox draft** for human review — never a direct save. Boundaries that keep this inside the existing design: open-memex builds no transcript store of its own (borrow the host's), mining judgment is the agent's LLM work (we provide the export and the queue, not an extractor), fully local, mined drafts carry their provenance (`source` marks them as distilled, from which session). Open sub-questions: host adapters per tool (opencode store format, Claude Code JSONL), how the user picks sessions to mine (by date? by project? "everything since last sync"?), and whether the proactive-capture criteria (open question above) should be the mining rubric verbatim — likely yes.
+
+- **Retrieval explainability and memory health (open, 2026-10-03):** from a heavy-user review. Search is currently a black box: when a memory is missed, there is no way to tell whether the query was bad, the memory is stale, it was ranked down by review state, or it is hiding behind a supersede chain. Candidate additions: a debug/explain view for `search` (matched terms, bm25 score, ranking effects, supersede exclusions), and a memory-side audit next to `doctor` (which today checks the environment, not the memories): near-duplicate candidates, memories untouched for N days still occupying injection slots, broken supersede chains, orphan scopes. Also undocumented and untested territory the same review surfaced: **concurrent sessions** — several agents on one machine writing the same store (simultaneous `memory_add`, reindex races, submit vs pull). File-locking/WAL behavior needs to be established, documented, and ideally guarded by a doctor check before a user finds out via a corrupt index.
+
+- **Host parity: plugin vs MCP tool surface (open, 2026-10-03):** the opencode plugin exposes 5 memory tools; the MCP server exposes 11 (`memory_status`, submit/propose/promote/resolve/pr_status missing on the plugin side). A user switching hosts mid-week hits missing tools with no explanation and reasonably assumes breakage. Options: bring the plugin to parity, or document the asymmetry as intentional (CLI covers the rest) in the tool descriptions themselves so the absence reads as design rather than defect. Needs a decision; silence is the current — worst — option.
+
+- **Novice experience: first success and plain language (open, 2026-10-03):** from an AI-novice review. The install ends with instructions, not an experience: candidate fix is a 60-second magic moment — init offers to save one sample memory and immediately has the user's agent answer a question using it, so the user *sees* recall work before being asked to trust it. Same screen should state the privacy model in plain words (personal never leaves this machine, no cloud, detected secrets are masked — all already true, all currently buried in docs), and the first time a suspected secret is masked, the capture path should say so. Underneath: a plain-language layer over the state machine ("saved on this computer" / "shared with the team") with outbox/submit/promote vocabulary becoming the advanced view — this is the same vocabulary wall the PM/BA question above hits from the other side.
+
+- **Memory visibility and correctability (open, 2026-10-03):** also from the novice review, and arguably the trust foundation for every other feature. A non-CLI user currently cannot answer "what does my assistant remember about me?" without reading Markdown files. Candidate directions: a human-readable inventory (local HTML export or equivalent) listing memories in plain language with provenance shown the way a person asks for it ("saved 3 days ago while you were discussing deploys in project X") and one-step forget/undo from the same view. Provenance fields already exist in frontmatter; this is presentation, not schema. Without it, proactive capture (above) will read to novices as surveillance rather than help — so this question probably gates the auto-draft posture decision in the capture question.
+
 ---
 
 *End of draft v2.*

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "open-memex",
-  "version": "0.6.0",
+  "version": "0.6.1-alpha.3",
   "description": "Local-first memory layer and protocol for AI coding agents. Markdown source of truth, SQLite FTS5 index, zero cloud.",
   "type": "module",
   "license": "Apache-2.0",

package/scripts/smoke-pure.ts

@@ -7,6 +7,7 @@ import { detectKeywords } from "../src/capture/keywords.ts";
 import { DEFAULT_CONFIG } from "../src/config.ts";
 import { resolveProjectScope, resolveCwdScope, PERSONAL_SCOPE } from "../src/scope.ts";
 import { cjkIndexText, cjkQueryExpr, hasCjk } from "../src/retrieve/cjk.ts";
+import { toFtsQuery } from "../src/retrieve/query.ts";
 import { contentHash, similarity, NEAR_DUP_THRESHOLD } from "../src/store/lifecycle.ts";
 import { userMcpConfigPath, mergeServerEntry, detectInstalledClients, mergePluginEntry, opencodeGlobalConfigPath, printManualEntryHint, parseJsonConfig, removeServerEntry, removePluginEntry, removeInstructionsSection, LEGACY_PACKAGE_NAME, shouldInstallSkill } from "../src/init.ts";
 
@@ -243,6 +244,41 @@ ok(
 ok("query single char", cjkQueryExpr("猫") === '"猫"', cjkQueryExpr("猫"));
 ok("query no cjk", cjkQueryExpr("hello") === "");
 
+console.log("== query construction ==");
+const fq1 = toFtsQuery("how do we configure the SSO login");
+ok(
+  "stopwords dropped",
+  !fq1.includes('"how"*') && !fq1.includes('"we"*') && !fq1.includes('"do"*'),
+  fq1,
+);
+ok(
+  "content terms kept",
+  fq1.includes('"configure"*') && fq1.includes('"sso"*') && fq1.includes('"login"*'),
+  fq1,
+);
+const fq2 = toFtsQuery("中文记忆怎么检索");
+ok(
+  "question bigrams dropped",
+  fq2.includes('"检索"') && !fq2.includes('"怎么"'),
+  fq2,
+);
+ok(
+  "filter never empties",
+  toFtsQuery("what is it") !== "" && toFtsQuery("怎么") !== "",
+  `${toFtsQuery("what is it")} / ${toFtsQuery("怎么")}`,
+);
+ok(
+  "tokens deduped",
+  toFtsQuery("deploy deploy deploy") === '"deploy"*',
+  toFtsQuery("deploy deploy deploy"),
+);
+const fq5 = toFtsQuery("auth 登录 sso 检索");
+ok(
+  "mixed query keeps both sides",
+  fq5.includes('"auth"*') && fq5.includes("{cjk}:"),
+  fq5,
+);
+
 console.log("== v2 times ==");
 ok("epoch 0 → rfc3339", msToRfc3339(0) === "1970-01-01T00:00:00Z", msToRfc3339(0));
 ok("rfc3339 round-trip", timeToMs(msToRfc3339(1758854400000)) === 1758854400000);

package/src/cli.ts

@@ -50,12 +50,14 @@ Examples:
 
   search: `Search memories by keyword (BM25 full-text), best matches first.
 
-Usage: open-memex search "query" [--scope project|personal|both] [--type T] [--limit N]
+Usage: open-memex search "query" [--scope project|personal|both] [--type T] [--limit N] [--explain]
 
 Flags:
-  --scope   project (default), personal, or both
-  --type    filter by memory type
-  --limit   max results
+  --scope    project (default), personal, or both
+  --type     filter by memory type
+  --limit    max results
+  --explain  show the FTS expression, per-hit score, and how many matches
+             lifecycle hid (superseded / retracted / archived)
 
 Example:
   open-memex search "deploy checklist" --scope both`,
@@ -337,6 +339,19 @@ Usage: open-memex doctor
 
 Example:
   open-memex doctor`,
+
+  audit: `Memory health audit (read-only, reports never mutates): near-duplicate
+pairs among active memories, stale actives untouched for 90+ days, broken
+supersede chains, personal files found inside the repo memory dir, and
+index/file drift.
+
+Usage: open-memex audit [--scope project|personal|both]
+
+Flags:
+  --scope   project (default), personal, or both
+
+Example:
+  open-memex audit --scope both`,
 };
 
 function usage(exitCode = 1): never {
@@ -345,7 +360,7 @@ function usage(exitCode = 1): never {
 Usage:
   open-memex where
   open-memex list [--scope project|personal] [--type T] [--limit N]
-  open-memex search "query" [--scope project|personal|both] [--type T] [--limit N]
+  open-memex search "query" [--scope project|personal|both] [--type T] [--limit N] [--explain]
   open-memex add "content" [--scope project|personal] [--type T] [--tag t1,t2]
   open-memex supersede <id> "new content" [--type T] [--tag t1,t2]
   open-memex status <id> active|deprecated|retracted|archived
@@ -362,6 +377,7 @@ Usage:
   open-memex submit <id...> [--branch <name>] [--base <branch>]
   open-memex pr-status [--apply]
   open-memex reindex
+  open-memex audit [--scope project|personal|both]
   open-memex scopes
   open-memex migrate [--from <key>] [--to <key>]
                                           [--dry-run] [--on-conflict newer|overwrite|skip]
@@ -698,6 +714,39 @@ async function main() {
     return;
   }
 
+  if (cmd === "audit") {
+    const flags = parseFlags(rest);
+    syncScope(project.key, "cli");
+    syncScope(PERSONAL_SCOPE.key, "cli");
+    const keys =
+      flags.scope === "personal"
+        ? [PERSONAL_SCOPE.key]
+        : flags.scope === "both"
+          ? [project.key, PERSONAL_SCOPE.key]
+          : [project.key];
+    const { runAudit, STALE_DAYS } = await import("./store/audit.ts");
+    const r = runAudit(keys);
+    console.log(`audit: ${r.totalMemories} memories (${keys.length === 2 ? "project+personal" : keys[0] === PERSONAL_SCOPE.key ? "personal" : "project"})`);
+    console.log(`  near-duplicate active pairs: ${r.duplicatePairs.length}`);
+    for (const p of r.duplicatePairs.slice(0, 10)) {
+      console.log(`    ${p.a} ~ ${p.b} (similarity ${p.score.toFixed(2)}) — consider supersede`);
+    }
+    console.log(`  stale actives (>= ${STALE_DAYS} days untouched): ${r.stale.length}`);
+    for (const s of r.stale.slice(0, 10)) {
+      console.log(`    ${s.id} — ${s.ageDays} days`);
+    }
+    console.log(`  broken supersede chains: ${r.brokenChains.length}`);
+    for (const b of r.brokenChains) {
+      console.log(`    ${b.id} -> missing ${b.missingTarget}`);
+    }
+    console.log(`  personal files inside repo memory dir: ${r.personalInRepo.length}`);
+    for (const f of r.personalInRepo) console.log(`    ${f}`);
+    console.log(
+      `  index drift: ${r.drift.filesNotIndexed} files not indexed, ${r.drift.rowsMissingFiles} rows missing files`,
+    );
+    return;
+  }
+
   // `config` prints the effective configuration (defaults + file). No DB needed.
   // `config set <key> <value>` persists a setting to the config file.
   if (cmd === "config") {
@@ -831,11 +880,22 @@ async function main() {
         : flags.scope === "project"
           ? [project.key]
           : [project.key, PERSONAL_SCOPE.key];
+    const explain = flags.explain === "true";
+    const stats = explain
+      ? { ftsQuery: "", candidates: 0, hiddenSuperseded: 0, hiddenExcluded: 0 }
+      : undefined;
     const hits = search(query, {
       scopeKeys: keys,
       limit: flags.limit ? Number(flags.limit) : undefined,
       type: flags.type,
+      stats,
     });
+    if (explain && stats) {
+      console.log(`fts: ${stats.ftsQuery || "(empty query)"}`);
+      console.log(
+        `candidates: ${stats.candidates}, hidden by lifecycle: ${stats.hiddenSuperseded} superseded, ${stats.hiddenExcluded} retracted/archived`,
+      );
+    }
     if (hits.length === 0) {
       console.log("(no matches)");
       return;
@@ -844,7 +904,8 @@ async function main() {
       const tag = h.scope_key === PERSONAL_SCOPE.key ? "personal" : "project";
       const dep = h.status === "deprecated" ? " [deprecated]" : "";
       const rev = hitStateLabel(h);
-      console.log(`[${tag}/${h.type}]${dep}${rev} ${h.id}  ${h.snippet.replace(/\s+/g, " ").trim()}`);
+      const sc = explain ? ` score=${h.score.toFixed(3)}` : "";
+      console.log(`[${tag}/${h.type}]${dep}${rev}${sc} ${h.id}  ${h.snippet.replace(/\s+/g, " ").trim()}`);
     }
     return;
   }
@@ -1361,7 +1422,17 @@ function positionalArgs(argv: string[]): string[] {
 
 main()
   .catch((err) => {
-    console.error(err);
+    // D57: lock contention past the busy timeout should read as a transient
+    // condition, not a stack trace. The memory file is written before the
+    // index update, so a locked-out write still lands on the next sync.
+    const code = (err as { code?: string })?.code;
+    if (code === "SQLITE_BUSY" || code === "SQLITE_LOCKED") {
+      console.error(
+        "open-memex: the memory store is busy (another open-memex process is writing). Wait a moment and retry — nothing was lost.",
+      );
+    } else {
+      console.error(err);
+    }
     process.exit(1);
   })
   .finally(() => {