open-memex 0.10.0-alpha.1 → 0.10.0-alpha.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +4 -1
- package/CHANGELOG.md +3 -1
- package/README.md +24 -8
- package/README.zh-CN.md +13 -5
- package/dist/.build-stamp +1 -1
- package/dist/cli.js +88 -12
- package/dist/curate.js +177 -0
- package/dist/graph/lint.js +11 -1
- package/dist/mcp.js +3 -0
- package/dist/retrieve/feedback.js +2 -2
- package/dist/store/curate.js +225 -0
- package/dist/submit.js +208 -78
- package/dist/tools/ops.js +24 -2
- package/docs/V2-DESIGN.md +2 -2
- package/docs/design/decisions.md +7 -4
- package/package.json +66 -66
- package/scripts/curate-eval.ts +376 -0
- package/scripts/smoke-pure.ts +77 -4
- package/src/cli.ts +86 -13
- package/src/curate.ts +236 -0
- package/src/graph/lint.ts +10 -2
- package/src/mcp.ts +3 -0
- package/src/retrieve/feedback.ts +128 -128
- package/src/store/curate.ts +270 -0
- package/src/submit.ts +264 -103
- package/src/tools/ops.ts +21 -2
package/AGENTS.md
CHANGED
|
@@ -45,7 +45,8 @@ npm run cli -- push # push current bran
|
|
|
45
45
|
npm run cli -- export [--scope project|personal|both] [--all] [-o <file>] # portable .tar.gz bundle (markdown + manifest); private excluded unless --all
|
|
46
46
|
npm run cli -- import <bundle.tar.gz> [--dry-run] # restore: personal → personal dir, project → outbox re-keyed; conflicts reported, never overwritten
|
|
47
47
|
npm run cli -- distill-agents [--type t1,t2] [-o <file>] # propose an AGENTS.md snippet from project memories (assist only; you merge by hand)
|
|
48
|
-
npm run cli -- submit <id...> [--branch <name>] [--base <branch>] # drafts → .ai/open-memex/ (current branch + local commit; never auto-branches)
|
|
48
|
+
npm run cli -- submit <id...> [--branch <name>] [--base <branch>] [--no-curate] # lint + deterministic curation, then drafts → .ai/open-memex/ (current branch + local commit; never auto-branches)
|
|
49
|
+
npm run cli -- curate undo <id> # reverse the latest auto-curation on one memory
|
|
49
50
|
npm run cli -- propose <id...> --to project [--local-approve] # copy personal → project outbox (batch OK)
|
|
50
51
|
npm run cli -- promote <id> [--reject] [--resubmit] [--note "..."] # proposed → approved → published (audit trail appended)
|
|
51
52
|
npm run cli -- pr-status [--apply] # map branch PR's GitHub state onto review_state (report; apply = local only)
|
|
@@ -168,6 +169,8 @@ AGENTS.md hygiene footer, the MCP handshake, and the tool descriptions.
|
|
|
168
169
|
- Chain integrity is self-healing: on read, a missing counterpart is auto-completed with a warning; a dangling pointer warns but is never fabricated. Don't "fix" chains by editing frontmatter directly — let the read path do it.
|
|
169
170
|
- Frontmatter is `schema_version: 2`. The SQLite index schema is versioned separately and rebuilds automatically on version change — never hand-edit `index.db`.
|
|
170
171
|
|
|
172
|
+
**Pre-submit curation (D78/D93) — deterministic only, every action undoable.** `submit` reports graph lint (report-only, never blocks) and auto-applies exactly three things: exact duplicates (`contentHash`-identical normalized body) merge into the oldest free active canonical via the supersede chain, empty/whitespace-only bodies archive, and a ragged first non-empty line is whitespace-normalized. Near-duplicates stay report-only; semantic merge/split and inferred type/tags are deliberately out. Planner is pure (`src/curate.ts`), writes go through `src/store/curate.ts` using `atomicWriteTextSync`/`setStatus` and append a `curation:` receipt to `review_history`; `open-memex curate undo <id>` reverses the latest merge/title action, `open-memex status <id> active` restores an archived empty. Never choose a chain-managed record as canonical, never merge into a non-active record, and never submit a non-active record — curated-out files stay in the outbox as history, not as proposed memories. Curation failure is reported and skipped, never fatal to the human's submit; a failure before the local commit rolls applied curation back. Keep the CLI, MCP `memory_submit` output, READMEs, and the fixture (`scripts/curate-eval.ts`) saying the same thing.
|
|
173
|
+
|
|
171
174
|
Keyword capture fires from `chat.message` on the user's message parts (`UserMessage`). Patterns live in `src/capture/keywords.ts` / config `keywordPatterns`; regex group 1 is the memory body. Scope routing: personal patterns (`cfg.keywordPersonalPatterns`) force the personal scope — the rule is 我 → personal (记住我/替我记/帮我记/我觉得/我喜欢/remember for me/help me remember), 我们 → current scope (我们认为/我们决定/帮我们记住); personal patterns run first and claim their line so a generic trigger can't double-fire (a rejected personal match keeps the claim — see D67).
|
|
172
175
|
|
|
173
176
|
D73 — one user statement, one memory. The hook and the agent are two writers for one turn, and only the hook knows the sentence is stored. Both hooks therefore take the session id (`chat.message` on v1, `prompt` on v2), `captureFromText` records each capture in `PluginState.captures`, and the system/context hook pushes the hand-off note (`captureHandoff` → `formatHandoffBlock`, `src/capture/handoff.ts`) once per capture. `memory_add` additionally refuses a write overlapping a `source: keyword`/`user` memory created in the same scope within `TURN_ECHO_WINDOW_MS` (`findTurnEcho`, threshold `TURN_ECHO_THRESHOLD` = 0.28) and returns the stored id plus the supersede route. Keep the note, the refusal text, `TOOL_DESCRIPTIONS.memory_add`, the skill, and this paragraph saying the same thing — an agent that reads only one of them still has to behave. Never lower `TURN_ECHO_THRESHOLD` to catch more: the measured reason it is high is that an agent's gloss (0.211) and a genuinely new fact from the same sentence (0.214) are lexically indistinguishable.
|
package/CHANGELOG.md
CHANGED
|
@@ -9,7 +9,9 @@ the `@alpha` tag are development snapshots and are not listed individually.
|
|
|
9
9
|
|
|
10
10
|
## [Unreleased]
|
|
11
11
|
|
|
12
|
-
-
|
|
12
|
+
- D78 pre-submit curation, first slice: `submit` now runs a report-only graph lint and a deterministic auto-apply tier before copying drafts into the repo. Lint reports orphans, dangling edges, and near-stale pairs but never blocks; exact duplicates are merged into the oldest free active copy through the existing supersede chain, empty / whitespace-only bodies are archived (not deleted), and a first non-empty line with ragged whitespace is normalized without touching the rest of the body. Near-duplicates stay report-only - semantic merge/split and inferred type/tags need a human. Every applied action prints a receipt and is reversible: `open-memex curate undo <id>` for merges and title fixes, `open-memex status <id> active` for empty-body archives (the marker lives in the file's `review_history`, so undo works across branches and machines). `submit --no-curate` skips the apply tier but still reports lint; it can now appear before or after the ids. Curated-out outbox files remain in appdata as history and are no longer counted as pending drafts (`sync-status` reports them separately); submit refuses non-active records instead of proposing history. `graph lint` no longer calls an intentional supersede-chain duplicate stale. Checked-in fixture (`scripts/curate-eval.ts`): 9 lint findings and 3 deterministic candidates before curation, 6 findings after (orphans 7→4, dangling 1, stale 1), `--no-curate` applies 0, and all 3 undo checks pass. Planner is pure (`src/curate.ts`), writes live in `src/store/curate.ts`; `smoke-pure` and `test-full` cover the new paths.
|
|
13
|
+
|
|
14
|
+
- D79 feedback-weighted ranking, first slice: `search()` can now reorder near-tied BM25 hits by the D77 reuse signal. A memory with `retrieved_count >= 3` and a higher `reuse_rate` moves ahead of equally relevant hits inside the same 25%-score band; the BM25 score is never modified, review tiers are never crossed, and hits with too little telemetry keep their exact BM25 slot (no cold-start penalty). Off by default (`feedbackWeighting: false`, a flat key settable with `config set`), because the counters start empty and the loop is only worth trusting once explicit useful signals exist. Guardrails unchanged: only `memory_useful`, a human approve at review, or a supersede raise usefulness — passive exposure only increments `retrieved_count`. `search --explain` reports whether weighting ran and lists every promoted hit with its reuse rate and move size. Fixture grows to 60 memories / 47 queries with a 3-query reuse slice; measured baseline (off) R@1 0.660 / R@5 0.979 / MRR 0.804 → weighted R@1 0.702 / R@5 0.979 / MRR 0.825 (feedback slice MRR 0.500 → 0.833). Pure band logic in `src/retrieve/feedback.ts`, smoke-tested without SQLite.
|
|
13
15
|
|
|
14
16
|
## [0.9.0] - 2026-11-06
|
|
15
17
|
|
package/README.md
CHANGED
|
@@ -465,7 +465,7 @@ they only matter once you share memories through Git.
|
|
|
465
465
|
| ● `memory_workingstate` | Read/write/clear the per-project working-state note (§3.6, D89) — resumes an interrupted task; scratch state, never a memory |
|
|
466
466
|
| ● `memory_useful` | Mark a memory as useful (D77) — explicit confirmation only; exposure is counted separately as `retrieved_count` |
|
|
467
467
|
| `memory_status` | Show the sync queue (outbox drafts, repo review states, uncommitted files) plus the D77 reuse report |
|
|
468
|
-
| `memory_submit` |
|
|
468
|
+
| `memory_submit` | Curate deterministic fixes, then move named drafts into the repo memory dir (local branch + commit) |
|
|
469
469
|
| `memory_propose` | Copy personal memories into the project scope as review candidates |
|
|
470
470
|
| `memory_promote` | Advance `proposed → approved → published` (or reject / resubmit) |
|
|
471
471
|
| `memory_resolve` | List conflicted memory files / 3-way-merge one of them |
|
|
@@ -531,9 +531,14 @@ capture → outbox (draft, local) → submit → repo (.ai/open-memex/) → PR r
|
|
|
531
531
|
git); `open-memex sync-status` — or just saying "sync memory" in chat —
|
|
532
532
|
shows what's pending.
|
|
533
533
|
3. **Submit** — you name the memories; they move into `<repo>/.ai/open-memex/`
|
|
534
|
-
with a local commit on your current branch.
|
|
535
|
-
|
|
536
|
-
|
|
534
|
+
with a local commit on your current branch. Before copying, `submit` prints
|
|
535
|
+
a report-only graph lint and auto-applies only deterministic cleanup: exact
|
|
536
|
+
duplicates are merged through the supersede chain, empty bodies are
|
|
537
|
+
archived, and ragged first-line titles are normalized. Near-duplicates stay
|
|
538
|
+
report-only; semantic merge/split needs a human. Every change prints its
|
|
539
|
+
undo command, and `--no-curate` skips the cleanup while keeping the lint
|
|
540
|
+
report. open-memex never pushes on its own; it prints the push + PR
|
|
541
|
+
commands, and an agent holding your explicit yes can carry them out.
|
|
537
542
|
4. **PR review** — memories are plain Markdown; reviewers approve, request
|
|
538
543
|
changes, or reject through the normal branch/PR process.
|
|
539
544
|
5. **Recall** — published memories are injected at session start and searchable
|
|
@@ -583,7 +588,7 @@ not a real-world claim.
|
|
|
583
588
|
Optional feedback weighting (D79, first slice, **off by default**) breaks
|
|
584
589
|
near-ties among BM25 hits using the D77 reuse counters: a memory with at least
|
|
585
590
|
3 recorded retrievals and a higher `reuse_rate` moves ahead inside the same
|
|
586
|
-
|
|
591
|
+
25%-score band. It never edits the BM25 score, never crosses a review tier,
|
|
587
592
|
never reorders hits with too little telemetry (new memories keep their exact
|
|
588
593
|
BM25 slot), and usefulness still comes only from explicit signals — passive
|
|
589
594
|
retrieval exposure never counts. Enable with
|
|
@@ -798,11 +803,15 @@ open-memex sync-status
|
|
|
798
803
|
# (draft / proposed / approved / published / rejected),
|
|
799
804
|
# and any uncommitted repo memory files.
|
|
800
805
|
|
|
801
|
-
open-memex submit <id...> [--branch <name>] [--base <branch>]
|
|
806
|
+
open-memex submit <id...> [--branch <name>] [--base <branch>] [--no-curate]
|
|
802
807
|
# move your named drafts into .ai/open-memex/ as "proposed":
|
|
803
808
|
# copies, flips review_state, local git commit ON THE CURRENT BRANCH.
|
|
804
809
|
# Never creates a branch on its own — branch creation is your call
|
|
805
810
|
# (or the agent's, only with your explicit approval for the full chain).
|
|
811
|
+
# Before copying: report-only graph lint + deterministic curation
|
|
812
|
+
# (exact-duplicate merge, empty-body archive, first-line title cleanup).
|
|
813
|
+
# --no-curate skips the cleanup but still prints lint. Near-duplicates are
|
|
814
|
+
# reported, never merged. Every applied action prints its undo command.
|
|
806
815
|
# All-or-nothing; conflicts (same id, different content) abort cleanly.
|
|
807
816
|
# Prints the push + gh pr commands; an agent holding your Yes carries
|
|
808
817
|
# through push/PR itself. --branch <name> creates the branch first
|
|
@@ -810,6 +819,12 @@ open-memex submit <id...> [--branch <name>] [--base <branch>]
|
|
|
810
819
|
# PRs stack onto your working branch); --base redirects it to main or
|
|
811
820
|
# wherever you review.
|
|
812
821
|
|
|
822
|
+
open-memex curate undo <id>
|
|
823
|
+
# reverse the latest automatic curation on one memory (merge or title fix).
|
|
824
|
+
# The receipt lives in the file's review_history, so this works across
|
|
825
|
+
# branches and machines; an empty-body archive is restored with
|
|
826
|
+
# `open-memex status <id> active`.
|
|
827
|
+
|
|
813
828
|
open-memex pr-status [--apply]
|
|
814
829
|
# read the branch's GitHub PR and map its state onto each in-repo memory:
|
|
815
830
|
# merged PR → published, PR approval → approved (approved_by = reviewer),
|
|
@@ -932,8 +947,9 @@ open-memex graph lint [--json] # orphans, dangling edges, near-stale pai
|
|
|
932
947
|
```
|
|
933
948
|
|
|
934
949
|
`lint` reports only — nothing is auto-resolved; a human decides. The same report
|
|
935
|
-
is the
|
|
936
|
-
the MCP surface stays at
|
|
950
|
+
is the input for D78's pre-submit curation (D93): `submit` prints it before the
|
|
951
|
+
deterministic cleanup. The graph interface is CLI-only; the MCP surface stays at
|
|
952
|
+
its thirteen memory tools.
|
|
937
953
|
|
|
938
954
|
## MCP server
|
|
939
955
|
|
package/README.zh-CN.md
CHANGED
|
@@ -302,7 +302,7 @@ MCP 服务器暴露十三个工具;opencode 原生插件暴露其中七个核
|
|
|
302
302
|
| ● `memory_workingstate` | 读/写/清除每项目的工作状态笔记(§3.6,D89)——让被中断的任务得以续接;只是草稿状态,永远不会成为记忆 |
|
|
303
303
|
| ● `memory_useful` | 确认一条记忆有用(D77)——只认显式确认;被动的搜索/列表出现只计入 `retrieved_count`,绝不算有用 |
|
|
304
304
|
| `memory_status` | 显示同步队列(outbox 草稿、仓库评审状态、未提交文件),并附带 D77 复用报告 |
|
|
305
|
-
| `memory_submit` |
|
|
305
|
+
| `memory_submit` | 先做确定性策展,再把指定草稿移进仓库记忆目录(本地分支 + 提交) |
|
|
306
306
|
| `memory_propose` | 把个人记忆复制到项目作用域,作为评审候选 |
|
|
307
307
|
| `memory_promote` | 推进 `proposed → approved → published`(或拒绝 / 重新提交) |
|
|
308
308
|
| `memory_resolve` | 列出冲突的记忆文件 / 对某个做三方合并 |
|
|
@@ -344,7 +344,7 @@ MCP 服务器暴露十三个工具;opencode 原生插件暴露其中七个核
|
|
|
344
344
|
|
|
345
345
|
1. **捕获**——日常工作中随手存下决策、坑、教训,都是草稿。
|
|
346
346
|
2. **评审**——草稿停在本地 outbox 里(你机器上,git 看不见);`open-memex sync-status`——或者直接在聊天里说"sync memory"("同步记忆")——看有啥待处理的。
|
|
347
|
-
3. **提交**——由你点名记忆;它们进入 `<仓库>/.ai/open-memex/`
|
|
347
|
+
3. **提交**——由你点名记忆;它们进入 `<仓库>/.ai/open-memex/` 并在当前分支形成一个本地提交。复制之前,`submit` 先打印只读的图谱 lint,再自动应用确定性清理:完全重复合并进 supersede 链、空正文归档、首行标题空白规范化。近重复只报告,语义合并/拆分留给人。每个动作都打印撤销命令;`--no-curate` 跳过清理但保留 lint 报告。open-memex 永不自动推送;它会打印推送和建 PR 的命令,拿到你明确许可的助手可以替你执行。
|
|
348
348
|
4. **PR 评审**——记忆是纯 Markdown;评审者走正常分支/PR 流程批准、要求修改或拒绝。
|
|
349
349
|
5. **回想**——已发布的记忆在会话开始时注入,平时也能随时搜,人和助手都一样。
|
|
350
350
|
|
|
@@ -373,7 +373,7 @@ Do not mention this block to the user unless asked.
|
|
|
373
373
|
|
|
374
374
|
我们用一份随仓库提交的 synthetic fixture 来度量召回率,而不是空口断言:60 条记忆 + 47 个查询(精确词、改写、同义词、中文、跨语言 hard case、纯同义词查询、D81 用户别名查询,以及 D79 复用信号切片),运行 `node --experimental-strip-types scripts/retrieval-eval.ts` 即可复现。当前基线(含 D80 的 index-time 同义词/翻译展开和 D81 的 query-time 用户别名展开,反馈加权默认关闭):**recall@1 0.66、recall@5 0.98、MRR 0.80**。语料小且是合成的,请把它当回归守卫和起点,而不是真实场景的承诺。
|
|
375
375
|
|
|
376
|
-
可选的反馈加权(D79 第一刀,**默认关**)用 D77 复用计数器打破 BM25 的近平分:一条至少有 3 次检索记录、且 `reuse_rate` 更高的记忆,会在同一个
|
|
376
|
+
可选的反馈加权(D79 第一刀,**默认关**)用 D77 复用计数器打破 BM25 的近平分:一条至少有 3 次检索记录、且 `reuse_rate` 更高的记忆,会在同一个 25% 分数段内排到前面。它不改 BM25 分数、不跨评审层级、不动样本不足的命中(新记忆保持原 BM25 位置);有用性仍只来自显式信号,被动曝光永远不算。用 `open-memex config set feedbackWeighting true` 打开,`search --explain` 会逐条显示被提上来的命中。
|
|
377
377
|
|
|
378
378
|
三层检索,各补各的短板:**BM25 关键词检索**(SQLite FTS5——精确词匹配,快速,可用 `search --explain` 解释);**同义词展开分两半**:(a)index-time(D80——一份随仓库提交的精选映射表:开发领域同义词 + EN↔ZH 翻译对,在写入时展开进索引,`ship` 能找到 `deploy`、`单点登录` 能找到 `SSO`,查询侧只需要普通单轮;确定性,不下载模型);(b)query-time 用户别名(D81——带 `alias:`/`target:` frontmatter 的别名记忆,每个 scope 定义一次自己的词汇,比如香蕉计划→支付系统重构项目;查询提到任一侧就触发对另一侧的第二轮搜索,双向,与第一轮等权重合并,别名命中不降权;别名变更不用 reindex);**语义向量**作为 opt-in 实验(本地多语言 embedding,在跨语言 fixture 切片上 R@5 92.7%——见 `docs/retrieval-ablation-study.md` §4.7;未经你明确允许绝不下载 embedding 模型)。
|
|
379
379
|
|
|
@@ -508,15 +508,23 @@ open-memex sync-status
|
|
|
508
508
|
# (draft / proposed / approved / published / rejected)、
|
|
509
509
|
# 以及任何未提交的仓库记忆文件。
|
|
510
510
|
|
|
511
|
-
open-memex submit <id...> [--branch <name>] [--base <branch>]
|
|
511
|
+
open-memex submit <id...> [--branch <name>] [--base <branch>] [--no-curate]
|
|
512
512
|
# 把你点名的草稿移进 .ai/open-memex/,标为 "proposed":
|
|
513
513
|
# 复制、翻转 review_state、在当前分支本地 git 提交。
|
|
514
514
|
# 永不自己建分支——建分支是你(或拿到完整链路明确许可的助手)的事。
|
|
515
|
+
# 复制前:只读图谱 lint + 确定性策展(完全重复合并、空正文归档、
|
|
516
|
+
# 首行标题清理)。--no-curate 跳过清理但仍打印 lint;近重复只报告,
|
|
517
|
+
# 永不自动合并。每个已应用动作都会打印撤销命令。
|
|
515
518
|
# 全有全无;冲突(同 id 不同内容)干净地中止。
|
|
516
519
|
# 打印 push + gh pr 命令;持有你 Yes 的助手可以一并执行 push/PR。
|
|
517
520
|
# --branch <name> 先建分支(助手全链路路径)。PR base 默认是当前分支
|
|
518
521
|
# (记忆 PR 叠在你的工作分支上);--base 可改指到 main 或你的评审分支。
|
|
519
522
|
|
|
523
|
+
open-memex curate undo <id>
|
|
524
|
+
# 撤销某条记忆上最近一次自动策展(合并或标题修正)。
|
|
525
|
+
# 凭据写在文件 review_history 里,所以跨分支/跨机器都能撤销;
|
|
526
|
+
# 空正文归档用 `open-memex status <id> active` 恢复。
|
|
527
|
+
|
|
520
528
|
open-memex pr-status [--apply]
|
|
521
529
|
# 读分支的 GitHub PR,把状态映射到每条仓库内记忆:
|
|
522
530
|
# PR 合并 → published,PR 批准 → approved(approved_by = 评审者),
|
|
@@ -605,7 +613,7 @@ open-memex graph path <a> <b> # 最短跳链,带凭据
|
|
|
605
613
|
open-memex graph lint [--json] # 孤立节点、悬空边、近陈旧对
|
|
606
614
|
```
|
|
607
615
|
|
|
608
|
-
`lint` 只报告——不自动处理,由人决定。同一份报告就是 D78
|
|
616
|
+
`lint` 只报告——不自动处理,由人决定。同一份报告就是 D78 提交前策展(D93)的输入:`submit` 在确定性清理前先打印它。图谱接口仅限 CLI;MCP 表面保持十三个记忆工具不变。
|
|
609
617
|
|
|
610
618
|
## MCP 服务器
|
|
611
619
|
|
package/dist/.build-stamp
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
2375384d33d2c269139ad6efd211a1ee8b5ddf3c529db58766dd7634947c17eb
|
package/dist/cli.js
CHANGED
|
@@ -6,7 +6,7 @@ import { syncScope, upsertFromFile, deleteFromIndex } from "./store/sync.js";
|
|
|
6
6
|
import { migrateScope, scopeHasFiles } from "./store/migrate.js";
|
|
7
7
|
import { migrateV2 } from "./store/v2migrate.js";
|
|
8
8
|
import { findDuplicates, supersede, setStatus } from "./store/lifecycle.js";
|
|
9
|
-
import { proposeMemories, promoteMemory, listConflicts, resolveConflict, formatReviewHistory } from "./review.js";
|
|
9
|
+
import { proposeMemories, promoteMemory, listConflicts, resolveConflict, formatReviewHistory, currentAuthor } from "./review.js";
|
|
10
10
|
import { getSyncStatus, formatSyncStatus, submitMemories } from "./submit.js";
|
|
11
11
|
import { getPrStatus, formatPrStatus, applyPrStatus } from "./github.js";
|
|
12
12
|
import { search, list, countList, hitStateLabel } from "./retrieve/search.js";
|
|
@@ -22,6 +22,8 @@ import { readWorkingState, clearWorkingState, workingStatePath } from "./working
|
|
|
22
22
|
import { refreshGraph, refreshGraphQuiet, ensureGraph, loadGraphEdges, loadGraphNodes, loadAllMemoryIds } from "./store/graph.js";
|
|
23
23
|
import { graphNeighbours, shortestPath } from "./graph/queries.js";
|
|
24
24
|
import { lintGraph } from "./graph/lint.js";
|
|
25
|
+
import { formatCurationAction } from "./curate.js";
|
|
26
|
+
import { undoCuration } from "./store/curate.js";
|
|
25
27
|
import { resolveMcpCommand } from "./init.js";
|
|
26
28
|
import fs from "node:fs";
|
|
27
29
|
import path from "node:path";
|
|
@@ -290,15 +292,32 @@ the repo memory dir as proposed (a local-approved copy keeps its approval),
|
|
|
290
292
|
commits locally on the CURRENT branch, and moves the outbox originals out.
|
|
291
293
|
Never creates a branch on its own — branch creation is your call.
|
|
292
294
|
|
|
293
|
-
Usage: open-memex submit <id...> [--branch <name>] [--base <branch>]
|
|
295
|
+
Usage: open-memex submit <id...> [--branch <name>] [--base <branch>] [--no-curate]
|
|
296
|
+
|
|
297
|
+
D78: submit reports graph lint (report-only, never blocks) and auto-applies
|
|
298
|
+
only deterministic curation: exact duplicate merges, empty-body archives, and
|
|
299
|
+
first-line title whitespace normalization. Every change is listed with its
|
|
300
|
+
undo command. Near-duplicates stay report-only - a human decides.
|
|
294
301
|
|
|
295
302
|
Flags:
|
|
296
|
-
--branch
|
|
297
|
-
|
|
298
|
-
--base
|
|
303
|
+
--branch create this branch and submit onto it (only with your explicit
|
|
304
|
+
approval for the full chain); default: stay on current branch
|
|
305
|
+
--base PR base override (default: the branch you're on)
|
|
306
|
+
--no-curate skip the deterministic auto-apply tier; lint still reports
|
|
299
307
|
|
|
300
308
|
Example:
|
|
301
|
-
open-memex submit 01ABC 01DEF
|
|
309
|
+
open-memex submit 01ABC 01DEF
|
|
310
|
+
open-memex submit 01ABC --no-curate`,
|
|
311
|
+
curate: `Undo one deterministic curation action applied by submit.
|
|
312
|
+
|
|
313
|
+
Every action is recorded in the memory file's review_history, so undo works
|
|
314
|
+
across branches and machines. The latest curation on that id is undone; run it
|
|
315
|
+
again to undo an older one (where the latest is itself an undo, it refuses).
|
|
316
|
+
|
|
317
|
+
Usage: open-memex curate undo <id>
|
|
318
|
+
|
|
319
|
+
Example:
|
|
320
|
+
open-memex curate undo 01ABC`,
|
|
302
321
|
"pr-status": `Map the current branch's GitHub PR state back onto review_state:
|
|
303
322
|
merged → published, approval → approved (approved_by = the reviewer),
|
|
304
323
|
changes-requested → suggestion only (never auto-rejects).
|
|
@@ -485,7 +504,8 @@ Usage:
|
|
|
485
504
|
open-memex export [--scope project|personal|both] [--type T] [--tag t] [--all] [-o <file>]
|
|
486
505
|
open-memex import <bundle.tar.gz> [--dry-run]
|
|
487
506
|
open-memex distill-agents [--scope project|personal] [--type t1,t2] [--limit N] [-o <file>]
|
|
488
|
-
open-memex submit <id...> [--branch <name>] [--base <branch>]
|
|
507
|
+
open-memex submit <id...> [--branch <name>] [--base <branch>] [--no-curate]
|
|
508
|
+
open-memex curate undo <id>
|
|
489
509
|
open-memex pr-status [--apply]
|
|
490
510
|
open-memex reindex
|
|
491
511
|
open-memex audit [--scope project|personal|both]
|
|
@@ -565,7 +585,7 @@ function setConfigPath(key, value) {
|
|
|
565
585
|
node[parts[parts.length - 1]] = value;
|
|
566
586
|
return { [top]: cur };
|
|
567
587
|
}
|
|
568
|
-
function parseFlags(argv) {
|
|
588
|
+
function parseFlags(argv, booleans = new Set()) {
|
|
569
589
|
const out = {};
|
|
570
590
|
for (let i = 0; i < argv.length; i++) {
|
|
571
591
|
const a = argv[i];
|
|
@@ -578,6 +598,10 @@ function parseFlags(argv) {
|
|
|
578
598
|
out[raw.slice(0, eq)] = raw.slice(eq + 1);
|
|
579
599
|
continue;
|
|
580
600
|
}
|
|
601
|
+
if (booleans.has(raw)) {
|
|
602
|
+
out[raw] = "true";
|
|
603
|
+
continue;
|
|
604
|
+
}
|
|
581
605
|
const val = argv[i + 1];
|
|
582
606
|
if (val !== undefined && !val.startsWith("--")) {
|
|
583
607
|
out[raw] = val;
|
|
@@ -1411,11 +1435,14 @@ async function main() {
|
|
|
1411
1435
|
return;
|
|
1412
1436
|
}
|
|
1413
1437
|
/** Positional args with flags (and their values) skipped. */
|
|
1414
|
-
function positionalArgs(argv) {
|
|
1438
|
+
function positionalArgs(argv, booleans = new Set()) {
|
|
1415
1439
|
const out = [];
|
|
1416
1440
|
for (let i = 0; i < argv.length; i++) {
|
|
1417
1441
|
const a = argv[i];
|
|
1418
1442
|
if (a.startsWith("--")) {
|
|
1443
|
+
const raw = a.slice(2);
|
|
1444
|
+
if (raw.includes("=") || booleans.has(raw))
|
|
1445
|
+
continue;
|
|
1419
1446
|
const val = argv[i + 1];
|
|
1420
1447
|
if (val !== undefined && !val.startsWith("--"))
|
|
1421
1448
|
i++; // skip the flag's value
|
|
@@ -1761,13 +1788,42 @@ async function main() {
|
|
|
1761
1788
|
return;
|
|
1762
1789
|
}
|
|
1763
1790
|
if (cmd === "submit") {
|
|
1764
|
-
const
|
|
1765
|
-
const
|
|
1791
|
+
const booleans = new Set(["no-curate"]);
|
|
1792
|
+
const flags = parseFlags(rest, booleans);
|
|
1793
|
+
const ids = positionalArgs(rest, booleans);
|
|
1766
1794
|
if (ids.length === 0)
|
|
1767
1795
|
usage();
|
|
1768
1796
|
syncScope(project.key, "submit");
|
|
1769
1797
|
try {
|
|
1770
|
-
const r = submitMemories(ids, {
|
|
1798
|
+
const r = submitMemories(ids, {
|
|
1799
|
+
branch: flags.branch,
|
|
1800
|
+
base: flags.base,
|
|
1801
|
+
noCurate: flags["no-curate"] === "true",
|
|
1802
|
+
});
|
|
1803
|
+
const c = r.curation;
|
|
1804
|
+
console.log(`curation: ${c.lintFindings.length} lint finding(s) - orphans ${c.lintCounts.orphan}, dangling ${c.lintCounts["dangling-edge"]}, stale ${c.lintCounts.stale}. Reports only - a human decides; nothing is auto-resolved.`);
|
|
1805
|
+
for (const f of c.lintFindings.slice(0, 5)) {
|
|
1806
|
+
console.log(` [${f.kind}] ${f.id}${f.other ? ` ${f.other}` : ""}: ${f.detail}`);
|
|
1807
|
+
}
|
|
1808
|
+
if (c.lintFindings.length > 5) {
|
|
1809
|
+
console.log(` ... and ${c.lintFindings.length - 5} more (full list: open-memex graph lint)`);
|
|
1810
|
+
}
|
|
1811
|
+
if (c.disabled) {
|
|
1812
|
+
console.log("curation auto-apply skipped (--no-curate); no files were changed by curation.");
|
|
1813
|
+
}
|
|
1814
|
+
else if (c.applied.length === 0) {
|
|
1815
|
+
console.log("curation auto-applied: 0 (nothing deterministic to fix)");
|
|
1816
|
+
}
|
|
1817
|
+
else {
|
|
1818
|
+
console.log(`curation auto-applied: ${c.applied.length}`);
|
|
1819
|
+
for (const a of c.applied)
|
|
1820
|
+
console.log(` ${formatCurationAction(a.action)}`);
|
|
1821
|
+
}
|
|
1822
|
+
for (const s of c.skipped) {
|
|
1823
|
+
console.log(` curation skipped ${s.id}: ${s.reason}`);
|
|
1824
|
+
}
|
|
1825
|
+
for (const w of c.warnings)
|
|
1826
|
+
console.log(` curation note: ${w}`);
|
|
1771
1827
|
for (const s of r.submitted) {
|
|
1772
1828
|
console.log(`submitted ${s.id} → ${path.relative(process.cwd(), s.filePath)} [${s.reviewState}]`);
|
|
1773
1829
|
}
|
|
@@ -1790,6 +1846,26 @@ async function main() {
|
|
|
1790
1846
|
}
|
|
1791
1847
|
return;
|
|
1792
1848
|
}
|
|
1849
|
+
if (cmd === "curate") {
|
|
1850
|
+
const sub = rest[0];
|
|
1851
|
+
const id = rest[1];
|
|
1852
|
+
if (sub !== "undo" || !id)
|
|
1853
|
+
usage();
|
|
1854
|
+
syncScope(project.key, "cli");
|
|
1855
|
+
try {
|
|
1856
|
+
const r = undoCuration(id, { by: currentAuthor() });
|
|
1857
|
+
refreshGraphQuiet(project.key, cfg);
|
|
1858
|
+
console.log(`curation undone: ${r.kind} ${id}`);
|
|
1859
|
+
for (const p of r.filePaths) {
|
|
1860
|
+
console.log(` ${path.relative(process.cwd(), p)}`);
|
|
1861
|
+
}
|
|
1862
|
+
}
|
|
1863
|
+
catch (e) {
|
|
1864
|
+
console.error(`curate undo failed: ${e.message}`);
|
|
1865
|
+
process.exit(2);
|
|
1866
|
+
}
|
|
1867
|
+
return;
|
|
1868
|
+
}
|
|
1793
1869
|
if (cmd === "scopes") {
|
|
1794
1870
|
const { memories } = paths();
|
|
1795
1871
|
if (!fs.existsSync(memories)) {
|
package/dist/curate.js
ADDED
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* D78: deterministic curation for the submit path.
|
|
3
|
+
*
|
|
4
|
+
* This module is PURE: it decides what can be auto-applied and renders the
|
|
5
|
+
* receipts. File writes live in src/store/curate.ts; `submit` owns the
|
|
6
|
+
* ordering (validate -> branch/conflict preflight -> curate -> copy -> commit).
|
|
7
|
+
*
|
|
8
|
+
* The only auto tier is deterministic:
|
|
9
|
+
* - exact duplicates (contentHash: whitespace-normalized body identical)
|
|
10
|
+
* - empty / whitespace-only bodies
|
|
11
|
+
* - title normalization (the first non-empty body line is the de-facto title)
|
|
12
|
+
*
|
|
13
|
+
* Near-duplicates stay REPORT-ONLY. Semantic merge/split needs a human; D78's
|
|
14
|
+
* first slice must never guess.
|
|
15
|
+
*/
|
|
16
|
+
import { contentHash, similarity, NEAR_DUP_THRESHOLD, } from "./store/lifecycle.js";
|
|
17
|
+
/** Trim + collapse internal whitespace. Never strips Markdown markers. */
|
|
18
|
+
export function normalizeTitleLine(line) {
|
|
19
|
+
return line.trim().replace(/\s+/g, " ");
|
|
20
|
+
}
|
|
21
|
+
export function firstNonEmptyLineIndex(body) {
|
|
22
|
+
const lines = body.split("\n");
|
|
23
|
+
for (let i = 0; i < lines.length; i++) {
|
|
24
|
+
if (lines[i].trim().length > 0)
|
|
25
|
+
return i;
|
|
26
|
+
}
|
|
27
|
+
return -1;
|
|
28
|
+
}
|
|
29
|
+
export function firstNonEmptyLine(body) {
|
|
30
|
+
const i = firstNonEmptyLineIndex(body);
|
|
31
|
+
return i < 0 ? "" : body.split("\n")[i];
|
|
32
|
+
}
|
|
33
|
+
/**
|
|
34
|
+
* Rewrite only the first non-empty line. Line count and every other byte are
|
|
35
|
+
* preserved, so the body remains the same memory - just a stable title.
|
|
36
|
+
*/
|
|
37
|
+
export function normalizeTitle(body) {
|
|
38
|
+
const lines = body.split("\n");
|
|
39
|
+
const i = firstNonEmptyLineIndex(body);
|
|
40
|
+
if (i < 0)
|
|
41
|
+
return null;
|
|
42
|
+
const before = lines[i];
|
|
43
|
+
const after = normalizeTitleLine(before);
|
|
44
|
+
if (before === after)
|
|
45
|
+
return null;
|
|
46
|
+
lines[i] = after;
|
|
47
|
+
return { body: lines.join("\n"), before, after };
|
|
48
|
+
}
|
|
49
|
+
function undoLine(kind, id) {
|
|
50
|
+
if (kind === "archive-empty")
|
|
51
|
+
return `open-memex status ${id} active`;
|
|
52
|
+
return `open-memex curate undo ${id}`;
|
|
53
|
+
}
|
|
54
|
+
/**
|
|
55
|
+
* Build the deterministic plan. Input order is irrelevant: output order is
|
|
56
|
+
* archive-empty, merge-duplicate, normalize-title, then id ascending inside
|
|
57
|
+
* each group, so two runs over the same store render the same summary.
|
|
58
|
+
*/
|
|
59
|
+
export function planCuration(records) {
|
|
60
|
+
const actions = [];
|
|
61
|
+
const skipped = [];
|
|
62
|
+
const active = records.filter((r) => r.status === "active");
|
|
63
|
+
const nonEmpty = active.filter((r) => r.body.trim().length > 0);
|
|
64
|
+
// 1. Empty fragments are archived, never deleted: the file is the source of
|
|
65
|
+
// truth and `status ... active` is the undo path.
|
|
66
|
+
for (const r of active) {
|
|
67
|
+
if (r.body.trim().length === 0) {
|
|
68
|
+
actions.push({
|
|
69
|
+
kind: "archive-empty",
|
|
70
|
+
id: r.id,
|
|
71
|
+
undo: undoLine("archive-empty", r.id),
|
|
72
|
+
reason: "empty body - archived, not deleted (status is reversible)",
|
|
73
|
+
});
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
// 2. Exact duplicates. Oldest createdAt wins; ties break on id so the
|
|
77
|
+
// canonical is stable across machines and runs.
|
|
78
|
+
const groups = new Map();
|
|
79
|
+
for (const r of nonEmpty) {
|
|
80
|
+
const key = contentHash(r.body);
|
|
81
|
+
const list = groups.get(key) ?? [];
|
|
82
|
+
list.push(r);
|
|
83
|
+
groups.set(key, list);
|
|
84
|
+
}
|
|
85
|
+
const merged = new Set();
|
|
86
|
+
for (const list of groups.values()) {
|
|
87
|
+
if (list.length < 2)
|
|
88
|
+
continue;
|
|
89
|
+
// A canonical must be a free active record. A copy already in a chain is
|
|
90
|
+
// not a merge target - its history is already owned by a human decision.
|
|
91
|
+
const eligible = list.filter((r) => !r.supersedes && !r.supersededBy);
|
|
92
|
+
eligible.sort((a, b) => a.createdAt.localeCompare(b.createdAt) || a.id.localeCompare(b.id));
|
|
93
|
+
const canonical = eligible[0];
|
|
94
|
+
if (!canonical) {
|
|
95
|
+
for (const r of list) {
|
|
96
|
+
skipped.push({
|
|
97
|
+
id: r.id,
|
|
98
|
+
reason: "every copy is already in a supersede chain - a human must decide which version is canonical",
|
|
99
|
+
});
|
|
100
|
+
}
|
|
101
|
+
continue;
|
|
102
|
+
}
|
|
103
|
+
for (const dup of list) {
|
|
104
|
+
if (dup.id === canonical.id)
|
|
105
|
+
continue;
|
|
106
|
+
if (dup.supersedes || dup.supersededBy) {
|
|
107
|
+
skipped.push({
|
|
108
|
+
id: dup.id,
|
|
109
|
+
reason: "already in a supersede chain - a human must decide which version is canonical",
|
|
110
|
+
});
|
|
111
|
+
continue;
|
|
112
|
+
}
|
|
113
|
+
actions.push({
|
|
114
|
+
kind: "merge-duplicate",
|
|
115
|
+
id: dup.id,
|
|
116
|
+
canonicalId: canonical.id,
|
|
117
|
+
undo: undoLine("merge-duplicate", dup.id),
|
|
118
|
+
reason: "normalized body is identical to " + canonical.id + "; supersede chain preserves the duplicate file",
|
|
119
|
+
});
|
|
120
|
+
merged.add(dup.id);
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
// 3. Title normalization. A duplicate being merged is history now, so don't
|
|
124
|
+
// report a title action for it; the canonical survives and can be fixed.
|
|
125
|
+
for (const r of nonEmpty) {
|
|
126
|
+
if (merged.has(r.id))
|
|
127
|
+
continue;
|
|
128
|
+
const n = normalizeTitle(r.body);
|
|
129
|
+
if (!n)
|
|
130
|
+
continue;
|
|
131
|
+
actions.push({
|
|
132
|
+
kind: "normalize-title",
|
|
133
|
+
id: r.id,
|
|
134
|
+
before: n.before,
|
|
135
|
+
after: n.after,
|
|
136
|
+
undo: undoLine("normalize-title", r.id),
|
|
137
|
+
reason: "first line is the de-facto title; whitespace normalized, no content changed",
|
|
138
|
+
});
|
|
139
|
+
}
|
|
140
|
+
// Near-duplicates are a report, never an action (D78 first slice).
|
|
141
|
+
const nearDuplicates = [];
|
|
142
|
+
for (let i = 0; i < nonEmpty.length; i++) {
|
|
143
|
+
for (let j = i + 1; j < nonEmpty.length; j++) {
|
|
144
|
+
const a = nonEmpty[i];
|
|
145
|
+
const b = nonEmpty[j];
|
|
146
|
+
if (contentHash(a.body) === contentHash(b.body))
|
|
147
|
+
continue;
|
|
148
|
+
const score = similarity(a.body, b.body);
|
|
149
|
+
if (score < NEAR_DUP_THRESHOLD)
|
|
150
|
+
continue;
|
|
151
|
+
nearDuplicates.push({
|
|
152
|
+
a: a.id < b.id ? a.id : b.id,
|
|
153
|
+
b: a.id < b.id ? b.id : a.id,
|
|
154
|
+
score,
|
|
155
|
+
});
|
|
156
|
+
}
|
|
157
|
+
}
|
|
158
|
+
nearDuplicates.sort((x, y) => x.a.localeCompare(y.a) || x.b.localeCompare(y.b) || y.score - x.score);
|
|
159
|
+
const order = {
|
|
160
|
+
"archive-empty": 0,
|
|
161
|
+
"merge-duplicate": 1,
|
|
162
|
+
"normalize-title": 2,
|
|
163
|
+
};
|
|
164
|
+
actions.sort((a, b) => order[a.kind] - order[b.kind] || a.id.localeCompare(b.id));
|
|
165
|
+
skipped.sort((a, b) => a.id.localeCompare(b.id));
|
|
166
|
+
return { actions, nearDuplicates, skipped };
|
|
167
|
+
}
|
|
168
|
+
/** One human-readable receipt line for an action that was applied. */
|
|
169
|
+
export function formatCurationAction(a) {
|
|
170
|
+
if (a.kind === "merge-duplicate") {
|
|
171
|
+
return `merged ${a.id} into ${a.canonicalId} (exact duplicate; undo: ${a.undo})`;
|
|
172
|
+
}
|
|
173
|
+
if (a.kind === "archive-empty") {
|
|
174
|
+
return `archived ${a.id} (empty body; undo: ${a.undo})`;
|
|
175
|
+
}
|
|
176
|
+
return `normalized ${a.id} title ${JSON.stringify(a.before)} -> ${JSON.stringify(a.after)} (undo: ${a.undo})`;
|
|
177
|
+
}
|
package/dist/graph/lint.js
CHANGED
|
@@ -65,12 +65,22 @@ export function lintGraph(nodes, edges, foreignIds = []) {
|
|
|
65
65
|
});
|
|
66
66
|
}
|
|
67
67
|
// Stale: active content that nearly duplicates a superseded/deprecated
|
|
68
|
-
// memory
|
|
68
|
+
// memory - the newer copy and the old chain never met. D78 merges exact
|
|
69
|
+
// duplicates INTO the chain, so an intentional chain link is not stale.
|
|
70
|
+
const chained = new Set();
|
|
71
|
+
for (const e of edges) {
|
|
72
|
+
if (e.type !== "supersedes")
|
|
73
|
+
continue;
|
|
74
|
+
chained.add(`${e.from}|${e.to}`);
|
|
75
|
+
chained.add(`${e.to}|${e.from}`);
|
|
76
|
+
}
|
|
69
77
|
const retired = nodes.filter((n) => n.status === "superseded" || n.status === "deprecated");
|
|
70
78
|
for (const n of nodes) {
|
|
71
79
|
if (n.status !== "active")
|
|
72
80
|
continue;
|
|
73
81
|
for (const r of retired) {
|
|
82
|
+
if (chained.has(`${n.id}|${r.id}`))
|
|
83
|
+
continue;
|
|
74
84
|
const score = similarity(n.body, r.body);
|
|
75
85
|
if (score < NEAR_DUP_THRESHOLD)
|
|
76
86
|
continue;
|
package/dist/mcp.js
CHANGED
|
@@ -74,6 +74,9 @@ them by these full names.
|
|
|
74
74
|
open the PR, or will you handle it yourself?" NEVER create branches, push, or
|
|
75
75
|
open PRs without the user's explicit approval. A "yes, you do it" covers the
|
|
76
76
|
whole chain — do NOT re-ask at each step.
|
|
77
|
+
- memory_submit prints its deterministic curation receipts (duplicate merges,
|
|
78
|
+
empty-body archives, title normalization) with undo commands. Relay them; do
|
|
79
|
+
not hide an automatic change to the user's memories.
|
|
77
80
|
- BE PROACTIVE about facts the user states directly: when the user shares a
|
|
78
81
|
decision, preference, project convention, or fix-and-cause worth remembering
|
|
79
82
|
across sessions, call memory_add without being asked. Keep each memory to one
|
|
@@ -25,10 +25,10 @@
|
|
|
25
25
|
export const FEEDBACK_MIN_RETRIEVED = 3;
|
|
26
26
|
/**
|
|
27
27
|
* A hit is in the same band as the band head while its score is at least
|
|
28
|
-
* this fraction of the head's. 0.
|
|
28
|
+
* this fraction of the head's. 0.75 = within 25%, i.e. a near-tie, not a
|
|
29
29
|
* relevance gap. Measured on the D79 fixture (see docs/design/decisions.md).
|
|
30
30
|
*/
|
|
31
|
-
export const DEFAULT_FEEDBACK_BAND_RATIO = 0.
|
|
31
|
+
export const DEFAULT_FEEDBACK_BAND_RATIO = 0.75;
|
|
32
32
|
/**
|
|
33
33
|
* Reorder one review tier. Callers pass a tier's hits in BM25 order. Within
|
|
34
34
|
* each near-tied band, only contiguous runs of eligible hits are re-sorted by
|