@fonlan/dsh-gitmemo 0.4.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (9) hide show
  1. package/README.md +96 -27
  2. package/README.zh.md +33 -16
  3. package/lib/index.d.ts +108 -0
  4. package/lib/index.js +69 -8
  5. package/lib/mem.d.ts +37 -1
  6. package/lib/mem.js +74 -10
  7. package/lib/systemone.d.ts +38 -1
  8. package/lib/systemone.js +85 -9
  9. package/package.json +5 -1
package/README.md CHANGED
@@ -16,7 +16,7 @@ dependency, and no manual memory commands are ever needed.
16
16
  - **Git-only** — no runtime dependency beyond the `git` CLI
17
17
  - **Token-efficient** — reuses prior conclusions via `mem_search`; subagents carry neither the workflow rules nor the tool schemas
18
18
  - **Immutable entries** — every write creates a new file; corrections use `mem_replace` (one commit deletes the old file and adds the new one), withdrawal uses `mem_delete`
19
- - **Knowledge freshness** — `mem_replace` is not just for user corrections: work that overrules a stored conclusion actively replaces it; `mem_search` scores hits as distinct matched keywords + a mild recency bonus (≤ +0.5, linearly decaying to 0 over ~180 days), so conflicting stale conclusions lose to their replacements
19
+ - **Knowledge freshness** — `mem_replace` is not just for user corrections: work that overrules a stored conclusion actively replaces it; `mem_search` adds a mild recency bonus (≤ +0.5, linearly decaying to 0 over ~180 days) on top of the match score, so conflicting stale conclusions lose to their replacements
20
20
  - **Topic pages (MOC)** — a `kind: "topic"` entry aggregates a topic's "current truth": the summary states the currently-valid conclusions and the page evolves via `mem_replace` (kind is inherited), not per-task rewrites
21
21
  - **Wiki links between entries** — content references other entries as `[[<commit-hash>]]`; `mem_read` with `expand: true` resolves those links one hop and automatically follows the `GitMemo-Replaces` chain to a replaced target's active version (dangling links come back with an error)
22
22
  - **Structured search** — commit messages carry `GitMemo-*` trailers (keywords, digest, search-text projections); search is `git log --grep --fixed-strings` over commit messages only — entry bodies are never scanned
@@ -28,7 +28,7 @@ dependency, and no manual memory commands are ever needed.
28
28
 
29
29
  | Piece | Description |
30
30
  | --- | --- |
31
- | `mem_search` | Search memories: `keywords` (array of 1–12, 中英文同义词), `skip` + `snapshot` (stable pagination). Returns up to 20 scored hits with `summary` / `keywords` / `kind` / `matched_keywords` (score = matched keywords + recency bonus) |
31
+ | `mem_search` | Search memories: `keywords` (array of 1–15, 中英文同义词), `skip` + `snapshot` (stable pagination). Returns up to 20 scored hits with `summary` / `keywords` / `kind` / `matched_keywords`. Score = the summed rarity of the matched keywords within the page (default) or the matched count (`searchScoring: count`), plus a recency bonus |
32
32
  | `mem_read` | Read one memory entry by create/replace commit hash (full markdown; historical hashes stay readable); returns `kind`; optional `expand: true` resolves `[[hash]]` links one hop |
33
33
  | `mem_write` | Store a task outcome: `title` + `summary` + `keywords` (2–12) + `content` (engine generates front matter), optional `kind` (`"task"` default / `"topic"` aggregated page), `related_branches` / `related_paths`. One ADD commit per immutable file |
34
34
  | `mem_delete` | Withdraw an obsolete conclusion (requires `commit_hash` + `reason`) |
@@ -63,6 +63,7 @@ The bundle patch ships with sensible defaults; override them in the profile's
63
63
  - id: dsh-gitmemo
64
64
  config:
65
65
  searchLimit: 20 # max hits per mem_search call (page size)
66
+ searchScoring: weighted # weighted (default) | count — see "Search scoring"
66
67
  lockTimeoutMs: 30000 # cross-process lock wait timeout
67
68
  projectRoot: null # optional explicit project root (defaults to the session cwd)
68
69
  systemOne: # optional System-one recall gate, see below
@@ -70,15 +71,65 @@ The bundle patch ships with sensible defaults; override them in the profile's
70
71
  endpoint: https://api.typesafe.ai/v1/systemone
71
72
  model: jev-latest
72
73
  mode: noul # noul | score
73
- threshold: 0.5 # noul mode: keep a candidate when P(yes) >= this
74
+ policy: rerank # rerank (default) | filter — see "System-one Recall Gate"
75
+ threshold: 0.5 # filter mode: drop a candidate when P(yes) < this
76
+ maxDropFraction: 0.25 # filter mode: never drop more than this share of a page
74
77
  ```
75
78
 
79
+ ## Search Scoring
80
+
81
+ `mem_search` matches entries with a fixed-string OR grep over the commit-message
82
+ projections — the title, the summary, and the entry's keywords (bodies are never
83
+ scanned). Two modes decide the *order* of the matched entries; both match exactly
84
+ the same entries:
85
+
86
+ | `searchScoring` | Score | Trade-off |
87
+ | --- | --- | --- |
88
+ | `weighted` (default) | `Σ` rarity of each matched keyword within the recall page, plus the recency bonus | A distinctive match outranks several common ones. One match can beat two, which the `count` mode never allows. |
89
+ | `count` | number of matched keywords, plus the recency bonus | The previous default arithmetic, kept for exact backwards compatibility. |
90
+
91
+ Rarity is the BM25-style weight `log(1 + (n − df + 0.5) / (df + 0.5))` over the
92
+ candidate page (`n` candidates, `df` matching the keyword), so it costs no extra
93
+ git call. The recency bonus stays ≤ +0.5, so it only reorders near-equal scores.
94
+
95
+ **Why this is the default.** On LongMemEval `_S` (500 instances, session
96
+ granularity) the engine reached `recall_all@50 = 0.998` while `ndcg_any@5` was
97
+ only 0.686: evidence was essentially always reachable but ranked badly, because
98
+ a plain match count cannot distinguish a rare keyword from a ubiquitous one.
99
+ Weighting the same matches lifted `recall_all@5` 0.738 → 0.755 and `ndcg@5`
100
+ 0.687 → 0.700 on identical ingested data, and `recall_all@50` was unchanged
101
+ (0.998) — matching is untouched by construction.
102
+ `evals/longmemeval/README.md` has the full run and the ablation across scoring,
103
+ granularity and ingestion policy.
104
+
76
105
  ## System-one Recall Gate (optional)
77
106
 
78
107
  A `mem_search` page can be handed to a **System-one model** (TypeSafe Jev by
79
108
  default) for one fast judgement, so memories that are **completely unrelated**
80
- to the current task are dropped before they are injected. Only genuinely
81
- reusable entries survive.
109
+ to the current task do not get injected.
110
+
111
+ **How the judgement is used — `policy`:**
112
+
113
+ | `policy` | Effect | Context | Measured accuracy |
114
+ | --- | --- | --- | --- |
115
+ | `rerank` (default) | keeps every judged memory and orders the page by the judge's value | unchanged | **0.8020** |
116
+ | `filter` | drops memories below `threshold`, bounded by `maxDropFraction` | **−80 %** | **0.3800** |
117
+
118
+ Those two figures are LongMemEval `_S` LLM-as-Judge over all 500 questions, with
119
+ the same judge, the same ingested memories and the same per-candidate values —
120
+ only the use made of them differs. The reason is in the judge's own signal: its
121
+ AUC separating evidence from non-evidence is ~0.71–0.75, so **no threshold is
122
+ safe**, while the same value is a perfectly good ordering feature. A hard drop on
123
+ a 0.75-AUC signal took 343 gold memories out of the prompt across 254 of 500
124
+ instances. Full study: `evals/longmemeval/README.md`.
125
+
126
+ `filter` remains available because dropping memories is the point of a recall
127
+ gate when context budget is the binding constraint — and it is now **bounded**:
128
+ `maxDropFraction` (default 0.25) restores the best-scoring rejects whenever a
129
+ threshold would drop more than that share of a page. An offline sweep held
130
+ `recall_all@5` at 0.8875 with a 20 % cap against 0.6875 uncapped. A memory
131
+ dropped is gone; a memory kept only costs context, so the errors are not
132
+ symmetric and the bound is deliberately tight.
82
133
 
83
134
  - **Inert by default.** The gate runs only when a usable credential is
84
135
  configured (`apiKey`, the credential named by `apiKeyEnv`, or the environment
@@ -87,9 +138,15 @@ reusable entries survive.
87
138
  - **Fails open.** A timeout, a non-2xx response, an unparseable body — any
88
139
  failure keeps every candidate and reports why in `gated.degraded` +
89
140
  `gated.reason`. **A broken endpoint can never hide a memory.**
90
- - **Never empties a page.** If the model rejects the whole page, at least
91
- `minKeep` (default 1) best-ranked candidates are retained, so "the gate removed
92
- it" and "there was nothing there" stay distinguishable. The tool's `gated` line
141
+ - **Never empties a page.** Under `filter`, at least `minKeep` (default 1)
142
+ best-ranked candidates are retained when the whole page is rejected, and
143
+ `maxDropFraction` (default 0.25) caps how much of a judged page may be removed
144
+ at all — so "the gate removed it" and "there was nothing there" stay
145
+ distinguishable, and a mis-calibrated threshold cannot gut a page. Under
146
+ `rerank` nothing is removed by construction.
147
+ - **Ordering is reported either way.** The decision carries `ordered` (candidate
148
+ hashes best-first by the judge's value) and each verdict's value, so a caller
149
+ can log or re-rank from the same judgement without a second call. The tool's `gated` line
93
150
  says so out loud — such a page carries a `NOTE(no candidate scored above the
94
151
  threshold…)` marker, so a retained floor is never mistaken for a candidate that
95
152
  genuinely passed. `judged=` counts only the candidates actually sent to the
@@ -114,6 +171,8 @@ reusable entries survive.
114
171
  | `mode` | `noul` | `noul` (yes/no probability) or `score` (graded levels) |
115
172
  | `threshold` | `0.5` | `noul` mode: keep when P(yes) ≥ this |
116
173
  | `scoreMin` | `2` | `score` mode cutoff. Note Jev's `score` is `Σ(level_index × probability)`, ranging 0…levels−1 (5 levels → 0…4), **not 0–1** |
174
+ | `policy` | `rerank` | `rerank` keeps every judged memory and orders the page by the judge's value; `filter` drops below `threshold` |
175
+ | `maxDropFraction` | `0.25` | `filter` only: never drop more than this share of the judged page, lowest-scoring first |
117
176
  | `minKeep` | `1` | Candidates retained when the whole page is rejected |
118
177
  | `maxCandidates` | `20` | Candidates judged per request; the remainder is **never judged and never dropped** |
119
178
  | `maxTaskChars` | `2000` | Task-text truncation length |
@@ -134,23 +193,6 @@ what writes it (the switch alone is a draft). An absent `enabled` reads as on, s
134
193
  an unconfigured section shows the gate as enabled — which is what the host
135
194
  default does.
136
195
 
137
- ### Known trade-offs
138
-
139
- - **English-first.** Jev's primary training language is English; CJK works but
140
- the vendor documents lower accuracy. The gate's question wording is English;
141
- only the candidate text and task text may be Chinese.
142
- - **Context ceiling.** 64k tokens per request, of which `state` + the longest
143
- single question may use 32k. `maxCandidates` and `maxTaskChars` exist to hold
144
- that line.
145
- - **Latency on the critical path.** Recall runs before every repo task, and the
146
- gate adds one network round trip (~0.4s observed), so `timeoutMs` defaults to
147
- 8s and every failure path fails open.
148
- - **Data egress.** Memory summaries leave the machine. TypeSafe currently offers
149
- **no** zero data retention; point `endpoint` at a self-hosted implementation
150
- (e.g. MIT-licensed `jeff`) or a local endpoint to avoid egress entirely.
151
- - **Billing.** Priced per input token ($0.042 per million input tokens, output
152
- free) — not per call.
153
-
154
196
  ## Memory Location & Format
155
197
 
156
198
  The `.mem` repository lives at the **project root** of the calling session's workspace
@@ -182,7 +224,7 @@ git -C .mem show <commit-hash>
182
224
 
183
225
  ## Agent Workflow (always-on rules, root agents)
184
226
 
185
- 1. **Before work — search.** For repo-related tasks, extract 1–12 中英文关键词 → `mem_search`.
227
+ 1. **Before work — search.** For repo-related tasks, extract 1–15 中英文关键词 → `mem_search`.
186
228
  Pure chat and general Q&A need no search.
187
229
  2. **Preselect results.** Based on `title` / `summary` / `keywords` / `score` /
188
230
  `matched_keywords`, `mem_read` at most 5 most relevant memories (`kind: "topic"` entries
@@ -215,6 +257,30 @@ npm run build # tsc → lib/
215
257
  npm test # build + engine/plugin/migration unit tests (node:test)
216
258
  ```
217
259
 
260
+ ## LongMemEval Evaluation
261
+
262
+ The engine is benchmarked against [LongMemEval](https://github.com/xiaowu0162/LongMemEval) two
263
+ ways, both runnable from this checkout. See [`evals/longmemeval/README.md`](evals/longmemeval/README.md)
264
+ for the retrieval metrics and [`evals/omnimemeval/README.md`](evals/omnimemeval/README.md) for the
265
+ end-to-end QA pipeline.
266
+
267
+ ```bash
268
+ # official retrieval metrics (recall/NDCG, no LLM needed)
269
+ npm run eval:lme:oracle # wiring check
270
+ npm run eval:lme:s # all 500 instances of the _S split
271
+
272
+ # end-to-end QA accuracy through OmniMemEval's 6-step pipeline
273
+ npm run eval:omni:setup # clone + adapter + venv + env file
274
+ cd .omnimemeval && ./scripts/run_lme_eval.sh --lib gitmemo \
275
+ --env ~/.dsh-gitmemo-lme/.env.gitmemo --streaming 1
276
+ ```
277
+
278
+ Headline result on `_S` (500/500 instances, session granularity, zero failures): `recall_all@5`
279
+ **0.736** against **0.870** for a plain BM25 over full session text, but `recall_all@50` **0.998** —
280
+ the loss is ranking precision, not coverage. Holding everything else fixed and letting an LLM
281
+ choose what to write raises `recall_all@5` to **0.842**, which is the point of the benchmark:
282
+ for a keyword-grep memory, what you commit to memory dominates the retrieval algorithm.
283
+
218
284
  ## Layout
219
285
 
220
286
  ```
@@ -227,7 +293,10 @@ dsh-gitmemo/
227
293
  │ ├── migrate.ts # legacy migration (dry-run/apply, backup refs, CAS swap)
228
294
  │ └── cli.ts # `dsh-gitmemo migrate` CLI entry
229
295
  ├── lib/ # built output (committed; used by file:/git installs)
230
- └── test/mem.test.mjs # engine + plugin + migration unit tests
296
+ ├── test/mem.test.mjs # engine + plugin + migration unit tests
297
+ └── evals/
298
+ ├── longmemeval/ # retrieval harness: official recall/NDCG, no LLM required
299
+ └── omnimemeval/ # gitmemo as an OmniMemEval memory backend (QA accuracy)
231
300
  ```
232
301
 
233
302
  ## License
package/README.zh.md CHANGED
@@ -15,7 +15,7 @@
15
15
  - **仅依赖 Git** —— 除 `git` CLI 外无任何运行时依赖
16
16
  - **省 token** —— 通过 `mem_search` 复用既有结论；子代理既不携带工作流规则也不携带工具 schema
17
17
  - **条目不可变** —— 每次写入都创建新文件；更正用 `mem_replace`（一个提交同时删除旧文件、新增新文件），作废用 `mem_delete`
18
- - **知识保鲜** —— `mem_replace` 不只用于用户更正：本次工作覆盖旧结论时同样主动替换；`mem_search` 评分为「命中关键词数 + 温和的 recency 加成（≤ +0.5，180 天线性衰减到 0）」，新旧结论冲突时自动偏向新条目
18
+ - **知识保鲜** —— `mem_replace` 不只用于用户更正：本次工作覆盖旧结论时同样主动替换；`mem_search` 在匹配分之上叠加温和的 recency 加成（≤ +0.5，180 天线性衰减到 0），新旧结论冲突时自动偏向新条目
19
19
  - **主题聚合页（MOC）** —— `kind: "topic"` 条目聚合一个主题的「当前真相」：summary 概括当下有效结论，靠 `mem_replace` 演进（kind 自动继承），不按任务重复新增
20
20
  - **条目间 Wiki 链接** —— content 里用 `[[<commit-hash>]]` 引用其他条目；`mem_read` 传 `expand: true` 一跳展开链接目标，并自动跟随 `GitMemo-Replaces` 链到被替换条目的最新版本（悬空链接带 error 返回）
21
21
  - **结构化搜索** —— commit message 携带 `GitMemo-*` trailers（keywords、digest、search-text 投影）；搜索只对 commit message 执行 `git log --grep --fixed-strings`，绝不扫描条目正文
@@ -27,7 +27,7 @@
27
27
 
28
28
  | 内容 | 说明 |
29
29
  | --- | --- |
30
- | `mem_search` | 搜索记忆：`keywords`（1–12 个关键词数组，建议中英文同义词）、`skip` + `snapshot`（稳定分页）。每次最多返回 20 条带 `summary` / `keywords` / `kind` / `matched_keywords` 的评分结果（score = 命中关键词数 + recency 加成） |
30
+ | `mem_search` | 搜索记忆：`keywords`（1–15 个关键词数组，建议中英文同义词）、`skip` + `snapshot`（稳定分页）。每次最多返回 20 条带 `summary` / `keywords` / `kind` / `matched_keywords` 的评分结果。score = 本页内命中关键词的稀有度之和（默认）或命中条数（`searchScoring: count`），再加 recency 加成 |
31
31
  | `mem_read` | 按创建/替换提交哈希读取一条记忆的完整 markdown（历史哈希仍可读）；返回 `kind`；可选 `expand: true` 一跳展开条目内的 `[[hash]]` 链接 |
32
32
  | `mem_write` | 存储任务结论：`title` + `summary` + `keywords`（2–12）+ `content`（front matter 由引擎生成），可选 `kind`（`"task"` 缺省 / `"topic"` 聚合页）、`related_branches` / `related_paths`。每个不可变文件对应一个 ADD 提交 |
33
33
  | `mem_delete` | 作废无替代结论（需要 `commit_hash` + `reason`） |
@@ -61,6 +61,7 @@ bundle patch 自带合理默认值，可在 profile 的 `cordis.patch.yml` 中
61
61
  - id: dsh-gitmemo
62
62
  config:
63
63
  searchLimit: 20 # 每次 mem_search 返回的最大条数（每页大小）
64
+ searchScoring: weighted # weighted（默认）| count —— 见「检索评分」
64
65
  lockTimeoutMs: 30000 # 跨进程锁等待超时
65
66
  projectRoot: null # 可选：显式项目根目录（默认取会话工作目录）
66
67
  systemOne: # 可选：System-one 召回门控，见下节
@@ -68,11 +69,37 @@ bundle patch 自带合理默认值，可在 profile 的 `cordis.patch.yml` 中
68
69
  endpoint: https://api.typesafe.ai/v1/systemone
69
70
  model: jev-latest
70
71
  mode: noul # noul | score
71
- threshold: 0.5 # noul 模式：概率 ≥ 该值才保留
72
+ policy: rerank # rerank（默认）| filter —— 见「System-one 召回门控」
73
+ threshold: 0.5 # filter 模式：概率 < 该值则丢弃
74
+ maxDropFraction: 0.25 # filter 模式：单页最多丢弃此比例
72
75
  ```
73
76
 
77
+ ## 检索评分
78
+
79
+ `mem_search` 用固定字符串 OR grep 在提交消息投影上匹配——即标题、摘要与条目的关键词（正文从不扫描）。两种模式决定**顺序**，匹配到的条目集合完全相同：
80
+
81
+ | `searchScoring` | 分值 | 取舍 |
82
+ | --- | --- | --- |
83
+ | `weighted`（默认） | 本页内各命中关键词稀有度之和，再加 recency 加成 | 有区分度的命中能压过若干常见命中；一次命中可以胜过两次，这是 `count` 模式不允许的 |
84
+ | `count` | 命中关键词条数，再加 recency 加成 | 此前的默认算法，为完全向后兼容保留 |
85
+
86
+ 稀有度用的是候选页上的 BM25 式权重 `log(1 + (n − df + 0.5) / (df + 0.5))`（`n` 为候选数，`df` 为命中该关键词的候选数），因此不增加额外 git 调用。recency 加成仍 ≤ +0.5，只用于重排近乎同分的条目。
87
+
88
+ **为什么把它设为默认。** 在 LongMemEval `_S`（500 个实例、会话粒度）上，引擎的 `recall_all@50` 达到 0.998，而 `ndcg_any@5` 只有 0.686：证据基本都能召回，但排不上来——因为纯命中计数分不清罕见词与随处可见的词。在完全相同的入库数据上只改加权，`recall_all@5` 从 0.738 升到 0.755、`ndcg@5` 从 0.687 升到 0.700，而 `recall_all@50` 仍是 0.998（匹配逻辑在构造上未改动）。完整实验与「打分 / 粒度 / 写入策略」三组消融见 `evals/longmemeval/README.md`。
89
+
74
90
  ## System-one 召回门控（可选）
75
91
 
92
+ **判定结果怎么用 —— `policy`：**
93
+
94
+ | `policy` | 作用 | 上下文 | 实测准确率 |
95
+ | --- | --- | --- | --- |
96
+ | `rerank`（默认） | 保留全部被判定的记忆，只按判定分值重排本页 | 不变 | **0.8020** |
97
+ | `filter` | 丢弃低于 `threshold` 的记忆，并由 `maxDropFraction` 兜住上限 | **−80 %** | **0.3800** |
98
+
99
+ 以上是 LongMemEval `_S` 全部 500 题、同一个 judge、同一批已入库记忆、同一组候选分值下的 LLM-as-Judge 结果——**唯一的差别就是怎么用这些分值**。原因在信号本身：judge 区分证据与非证据的 AUC 只有约 0.71–0.75，因此**没有任何阈值是安全的**，但同一个分值作为排序特征是可靠的。在 0.75 AUC 的信号上做硬删，把 343 条 gold 记忆挡在了提示之外，涉及 500 个实例中的 254 个。
100
+
101
+ `filter` 仍然保留：当上下文预算才是硬约束时，「丢弃」正是召回门控的意义所在。而它现在**有上限**了：`maxDropFraction`（默认 0.25）会在阈值想丢掉超过该比例时，把得分最高的被拒候选恢复回来。离线扫描显示 20% 上限能把 `recall_all@5` 稳在 0.8875，而不设上限只有 0.6875。丢掉的记忆就没了，留下的只是多花点上下文——两类错误并不对称，所以上限刻意收得紧。
102
+
76
103
  `mem_search` 的候选页可以交给一个 **System-one 模型**（默认 [TypeSafe Jev](https://docs.typesafe.ai)）
77
104
  做一次快速判断，把与当前任务**完全无关**的记忆从注入内容里剔除，只留下真正可复用的条目。
78
105
 
@@ -103,6 +130,8 @@ bundle patch 自带合理默认值，可在 profile 的 `cordis.patch.yml` 中
103
130
  | `threshold` | `0.5` | `noul` 模式：概率 ≥ 该值保留 |
104
131
  | `scoreMin` | `2` | `score` 模式阈值。注意 Jev 的 `score` 是 `Σ(层级序号 × 概率)`，范围是 0…层级数−1（默认 5 层 → 0…4），**不是 0–1** |
105
132
  | `minKeep` | `1` | 整页被拒时至少保留几条 |
133
+ | `policy` | `rerank` | `rerank` 保留全部判定过的记忆并按分值重排；`filter` 丢弃低于 `threshold` 的候选 |
134
+ | `maxDropFraction` | `0.25` | 仅 `filter`：单页最多丢弃此比例，从最低分开始丢 |
106
135
  | `maxCandidates` | `20` | 单次请求最多判定多少条；超出部分**不判定、不剔除** |
107
136
  | `maxTaskChars` | `2000` | 任务文本截断长度 |
108
137
  | `timeoutMs` | `8000` | 请求超时 |
@@ -116,18 +145,6 @@ bundle patch 自带合理默认值，可在 profile 的 `cordis.patch.yml` 中
116
145
  开关暂存的是 `systemOne.enabled`：关闭后召回行为与门控存在前逐字节一致；需要点**保存**才会写入
117
146
  （只拨开关是草稿）。未配置 `enabled` 时按开启处理，与该字段的宿主默认值一致。
118
147
 
119
- ### 已知取舍
120
-
121
- - **英文优先。** Jev 的主要训练语言是英文，CJK 文本可用但官方标注准确率较低。门控的提问措辞固定为
122
- 英文，只有候选正文与任务文本可能是中文。
123
- - **上下文上限。** 单次请求 64k tokens，其中 `state` + 最长问题合计 32k。`maxCandidates` 与
124
- `maxTaskChars` 就是为守住这条线而存在的。
125
- - **延迟在关键路径上。** 召回发生在每次仓库任务之前，门控会给 `mem_search` 增加一次网络往返
126
- （实测约 0.4s 量级），因此 `timeoutMs` 默认 8s 且失败即放行。
127
- - **数据外发。** 记忆摘要会离开本机。TypeSafe 目前**不提供** zero data retention；若要避免外发，
128
- 可把 `endpoint` 指向自建实现（如 MIT 许可的 `jeff`）或本地端点。
129
- - **计费口径。** 按输入 token 计费（$0.042 / 百万输入 token，输出免费），不是按次计费。
130
-
131
148
  ## 记忆存放位置与格式
132
149
 
133
150
  `.mem` 仓库位于调用会话工作区的**项目根目录**（`git rev-parse --show-toplevel`，找不到时退回
@@ -156,7 +173,7 @@ git -C .mem show <commit-hash>
156
173
 
157
174
  ## Agent 工作流（常驻规则，仅根 Agent）
158
175
 
159
- 1. **开工前 —— 搜索。** 仓库相关任务开始前提取 1–12 个中英文关键词 → `mem_search`。纯闲聊和通用问答无需搜索。
176
+ 1. **开工前 —— 搜索。** 仓库相关任务开始前提取 1–15 个中英文关键词 → `mem_search`。纯闲聊和通用问答无需搜索。
160
177
  2. **结果预筛。** 根据 `title` / `summary` / `keywords` / `score` / `matched_keywords` 最多
161
178
  `mem_read` 5 条最相关记忆（`kind: "topic"` 的条目是该主题的聚合入口，优先读）；用 `skip` + 返回的
162
179
  `snapshot` 翻页。
package/lib/index.d.ts CHANGED
@@ -21,6 +21,7 @@
21
21
  * @module dsh-gitmemo
22
22
  */
23
23
  import z from "@deepseek-ai/schemastery";
24
+ import { type SearchScoring } from "./mem.js";
24
25
  /** Cordis plugin name. */
25
26
  declare const name = "dsh-gitmemo";
26
27
  /** Host services this plugin needs. */
@@ -31,6 +32,21 @@ declare const Config: z<Schemastery.ObjectS<NoInfer<{
31
32
  memDirName: z<string, string, "defined">;
32
33
  /** Max search hits per mem_search call (page size). Default 20. */
33
34
  searchLimit: z<number, number, "defined">;
35
+ /**
36
+ * How a matched entry is scored (search order only — matching itself is
37
+ * identical in both modes).
38
+ *
39
+ * - `weighted` (default): sum of per-keyword rarity weights within the
40
+ * recall page, so a distinctive match outranks several common ones.
41
+ * - `count`: the previous default behaviour (one point per matched keyword plus the
42
+ * recency bonus), kept for exact backwards compatibility.
43
+ *
44
+ * Measured motivation: on LongMemEval `_S` the engine reached
45
+ * recall_all@50 = 0.998 while ndcg@5 was only 0.686 — evidence was reachable
46
+ * but ranked poorly, because a saturated-free match count cannot tell a rare
47
+ * keyword from a ubiquitous one.
48
+ */
49
+ searchScoring: z<"count" | "weighted", "count" | "weighted", "defined">;
34
50
  /** @deprecated — `.mem` always stays on main; kept for one version, no effect. */
35
51
  branchAlign: z<boolean, boolean, "defined">;
36
52
  /** @deprecated — session-start recent seeding was removed; kept for one version, no effect. */
@@ -74,6 +90,25 @@ declare const Config: z<Schemastery.ObjectS<NoInfer<{
74
90
  scoreMin: z<number, number, "volatile-defined">;
75
91
  /** Retain at least this many candidates when the model rejects an entire page. */
76
92
  minKeep: z<number, number, "volatile-defined">;
93
+ /**
94
+ * What happens to a judged candidate.
95
+ *
96
+ * `rerank` (default) keeps every candidate and lets the judge's value order
97
+ * the page; `filter` drops those below the threshold, bounded by
98
+ * `maxDropFraction`. Measured end-to-end on LongMemEval `_S` with the same
99
+ * judge and the same per-candidate values: filtering at the default
100
+ * threshold scored 0.3800, ordering scored 0.8020. The judge's AUC is
101
+ * ~0.71-0.75, so no threshold is safe, but the ordering is useful. See
102
+ * evals/longmemeval/README.md.
103
+ */
104
+ policy: z<"filter" | "rerank", "filter" | "rerank", "volatile-defined">;
105
+ /**
106
+ * `filter` policy only: never drop more than this share of the judged page,
107
+ * lowest-scoring first. Caps the damage a mis-calibrated threshold can do —
108
+ * a memory dropped is gone, a memory kept only costs context. The offline
109
+ * sweep held recall_all@5 at 0.8875 with a 20 % cap against 0.6875 uncapped.
110
+ */
111
+ maxDropFraction: z<number, number, "volatile-defined">;
77
112
  /** Never judge more than this many candidates per request. */
78
113
  maxCandidates: z<number, number, "volatile-defined">;
79
114
  /** Truncate the task text to this many characters before sending it. */
@@ -101,6 +136,25 @@ declare const Config: z<Schemastery.ObjectS<NoInfer<{
101
136
  scoreMin: z<number, number, "volatile-defined">;
102
137
  /** Retain at least this many candidates when the model rejects an entire page. */
103
138
  minKeep: z<number, number, "volatile-defined">;
139
+ /**
140
+ * What happens to a judged candidate.
141
+ *
142
+ * `rerank` (default) keeps every candidate and lets the judge's value order
143
+ * the page; `filter` drops those below the threshold, bounded by
144
+ * `maxDropFraction`. Measured end-to-end on LongMemEval `_S` with the same
145
+ * judge and the same per-candidate values: filtering at the default
146
+ * threshold scored 0.3800, ordering scored 0.8020. The judge's AUC is
147
+ * ~0.71-0.75, so no threshold is safe, but the ordering is useful. See
148
+ * evals/longmemeval/README.md.
149
+ */
150
+ policy: z<"filter" | "rerank", "filter" | "rerank", "volatile-defined">;
151
+ /**
152
+ * `filter` policy only: never drop more than this share of the judged page,
153
+ * lowest-scoring first. Caps the damage a mis-calibrated threshold can do —
154
+ * a memory dropped is gone, a memory kept only costs context. The offline
155
+ * sweep held recall_all@5 at 0.8875 with a 20 % cap against 0.6875 uncapped.
156
+ */
157
+ maxDropFraction: z<number, number, "volatile-defined">;
104
158
  /** Never judge more than this many candidates per request. */
105
159
  maxCandidates: z<number, number, "volatile-defined">;
106
160
  /** Truncate the task text to this many characters before sending it. */
@@ -113,6 +167,21 @@ declare const Config: z<Schemastery.ObjectS<NoInfer<{
113
167
  memDirName: z<string, string, "defined">;
114
168
  /** Max search hits per mem_search call (page size). Default 20. */
115
169
  searchLimit: z<number, number, "defined">;
170
+ /**
171
+ * How a matched entry is scored (search order only — matching itself is
172
+ * identical in both modes).
173
+ *
174
+ * - `weighted` (default): sum of per-keyword rarity weights within the
175
+ * recall page, so a distinctive match outranks several common ones.
176
+ * - `count`: the previous default behaviour (one point per matched keyword plus the
177
+ * recency bonus), kept for exact backwards compatibility.
178
+ *
179
+ * Measured motivation: on LongMemEval `_S` the engine reached
180
+ * recall_all@50 = 0.998 while ndcg@5 was only 0.686 — evidence was reachable
181
+ * but ranked poorly, because a saturated-free match count cannot tell a rare
182
+ * keyword from a ubiquitous one.
183
+ */
184
+ searchScoring: z<"count" | "weighted", "count" | "weighted", "defined">;
116
185
  /** @deprecated — `.mem` always stays on main; kept for one version, no effect. */
117
186
  branchAlign: z<boolean, boolean, "defined">;
118
187
  /** @deprecated — session-start recent seeding was removed; kept for one version, no effect. */
@@ -156,6 +225,25 @@ declare const Config: z<Schemastery.ObjectS<NoInfer<{
156
225
  scoreMin: z<number, number, "volatile-defined">;
157
226
  /** Retain at least this many candidates when the model rejects an entire page. */
158
227
  minKeep: z<number, number, "volatile-defined">;
228
+ /**
229
+ * What happens to a judged candidate.
230
+ *
231
+ * `rerank` (default) keeps every candidate and lets the judge's value order
232
+ * the page; `filter` drops those below the threshold, bounded by
233
+ * `maxDropFraction`. Measured end-to-end on LongMemEval `_S` with the same
234
+ * judge and the same per-candidate values: filtering at the default
235
+ * threshold scored 0.3800, ordering scored 0.8020. The judge's AUC is
236
+ * ~0.71-0.75, so no threshold is safe, but the ordering is useful. See
237
+ * evals/longmemeval/README.md.
238
+ */
239
+ policy: z<"filter" | "rerank", "filter" | "rerank", "volatile-defined">;
240
+ /**
241
+ * `filter` policy only: never drop more than this share of the judged page,
242
+ * lowest-scoring first. Caps the damage a mis-calibrated threshold can do —
243
+ * a memory dropped is gone, a memory kept only costs context. The offline
244
+ * sweep held recall_all@5 at 0.8875 with a 20 % cap against 0.6875 uncapped.
245
+ */
246
+ maxDropFraction: z<number, number, "volatile-defined">;
159
247
  /** Never judge more than this many candidates per request. */
160
248
  maxCandidates: z<number, number, "volatile-defined">;
161
249
  /** Truncate the task text to this many characters before sending it. */
@@ -183,6 +271,25 @@ declare const Config: z<Schemastery.ObjectS<NoInfer<{
183
271
  scoreMin: z<number, number, "volatile-defined">;
184
272
  /** Retain at least this many candidates when the model rejects an entire page. */
185
273
  minKeep: z<number, number, "volatile-defined">;
274
+ /**
275
+ * What happens to a judged candidate.
276
+ *
277
+ * `rerank` (default) keeps every candidate and lets the judge's value order
278
+ * the page; `filter` drops those below the threshold, bounded by
279
+ * `maxDropFraction`. Measured end-to-end on LongMemEval `_S` with the same
280
+ * judge and the same per-candidate values: filtering at the default
281
+ * threshold scored 0.3800, ordering scored 0.8020. The judge's AUC is
282
+ * ~0.71-0.75, so no threshold is safe, but the ordering is useful. See
283
+ * evals/longmemeval/README.md.
284
+ */
285
+ policy: z<"filter" | "rerank", "filter" | "rerank", "volatile-defined">;
286
+ /**
287
+ * `filter` policy only: never drop more than this share of the judged page,
288
+ * lowest-scoring first. Caps the damage a mis-calibrated threshold can do —
289
+ * a memory dropped is gone, a memory kept only costs context. The offline
290
+ * sweep held recall_all@5 at 0.8875 with a 20 % cap against 0.6875 uncapped.
291
+ */
292
+ maxDropFraction: z<number, number, "volatile-defined">;
186
293
  /** Never judge more than this many candidates per request. */
187
294
  maxCandidates: z<number, number, "volatile-defined">;
188
295
  /** Truncate the task text to this many characters before sending it. */
@@ -231,6 +338,7 @@ declare function apply(ctx: {
231
338
  }, config?: Partial<{
232
339
  memDirName?: string;
233
340
  searchLimit?: number;
341
+ searchScoring?: SearchScoring;
234
342
  branchAlign?: boolean;
235
343
  recentContextLimit?: number;
236
344
  lockTimeoutMs?: number;
package/lib/index.js CHANGED
@@ -34,6 +34,21 @@ const Config = z.object({
34
34
  memDirName: z.string().default(".mem"),
35
35
  /** Max search hits per mem_search call (page size). Default 20. */
36
36
  searchLimit: z.number().default(20),
37
+ /**
38
+ * How a matched entry is scored (search order only — matching itself is
39
+ * identical in both modes).
40
+ *
41
+ * - `weighted` (default): sum of per-keyword rarity weights within the
42
+ * recall page, so a distinctive match outranks several common ones.
43
+ * - `count`: the previous default behaviour (one point per matched keyword plus the
44
+ * recency bonus), kept for exact backwards compatibility.
45
+ *
46
+ * Measured motivation: on LongMemEval `_S` the engine reached
47
+ * recall_all@50 = 0.998 while ndcg@5 was only 0.686 — evidence was reachable
48
+ * but ranked poorly, because a saturated-free match count cannot tell a rare
49
+ * keyword from a ubiquitous one.
50
+ */
51
+ searchScoring: z.union(["count", "weighted"]).default("weighted"),
37
52
  /** @deprecated — `.mem` always stays on main; kept for one version, no effect. */
38
53
  branchAlign: z.boolean().default(true),
39
54
  /** @deprecated — session-start recent seeding was removed; kept for one version, no effect. */
@@ -77,6 +92,25 @@ const Config = z.object({
77
92
  scoreMin: z.number().default(2).volatile(),
78
93
  /** Retain at least this many candidates when the model rejects an entire page. */
79
94
  minKeep: z.number().step(1).min(0).default(1).volatile(),
95
+ /**
96
+ * What happens to a judged candidate.
97
+ *
98
+ * `rerank` (default) keeps every candidate and lets the judge's value order
99
+ * the page; `filter` drops those below the threshold, bounded by
100
+ * `maxDropFraction`. Measured end-to-end on LongMemEval `_S` with the same
101
+ * judge and the same per-candidate values: filtering at the default
102
+ * threshold scored 0.3800, ordering scored 0.8020. The judge's AUC is
103
+ * ~0.71-0.75, so no threshold is safe, but the ordering is useful. See
104
+ * evals/longmemeval/README.md.
105
+ */
106
+ policy: z.union(["rerank", "filter"]).default("rerank").volatile(),
107
+ /**
108
+ * `filter` policy only: never drop more than this share of the judged page,
109
+ * lowest-scoring first. Caps the damage a mis-calibrated threshold can do —
110
+ * a memory dropped is gone, a memory kept only costs context. The offline
111
+ * sweep held recall_all@5 at 0.8875 with a 20 % cap against 0.6875 uncapped.
112
+ */
113
+ maxDropFraction: z.number().min(0).max(1).default(0.25).volatile(),
80
114
  /** Never judge more than this many candidates per request. */
81
115
  maxCandidates: z.number().step(1).min(1).default(20).volatile(),
82
116
  /** Truncate the task text to this many characters before sending it. */
@@ -120,6 +154,8 @@ function readSystemOne(config) {
120
154
  threshold: liveValue(section.threshold, 0.5),
121
155
  scoreMin: liveValue(section.scoreMin, 2),
122
156
  minKeep: liveValue(section.minKeep, 1),
157
+ policy: liveValue(section.policy, "rerank") === "filter" ? "filter" : "rerank",
158
+ maxDropFraction: liveValue(section.maxDropFraction, 0.25),
123
159
  maxCandidates: liveValue(section.maxCandidates, 20),
124
160
  maxTaskChars: liveValue(section.maxTaskChars, 2000),
125
161
  timeoutMs: liveValue(section.timeoutMs, 8000),
@@ -205,7 +241,8 @@ async function engineFor(exec, config) {
205
241
  const root = config.projectRoot ?? (await resolveProjectRoot(cwd));
206
242
  const engineConfig = {
207
243
  searchLimit: config.searchLimit,
208
- lockTimeoutMs: config.lockTimeoutMs
244
+ lockTimeoutMs: config.lockTimeoutMs,
245
+ searchScoring: config.searchScoring
209
246
  };
210
247
  return new GitMemo(root, engineConfig);
211
248
  }
@@ -231,7 +268,9 @@ const SEARCH_HIT = {
231
268
  score: {
232
269
  type: "number",
233
270
  required: true,
234
- description: "Relevance: distinct matched keywords + a mild recency bonus (≤ +0.5, decaying to 0 by ~180 days)."
271
+ description: "Relevance. In the default `weighted` scoring: the summed rarity of the matched keywords within this page " +
272
+ "(a distinctive match outweighs several common ones) plus a mild recency bonus (≤ +0.5, decaying to 0 by ~180 days). " +
273
+ "With `searchScoring: count`: distinct matched keywords + the same recency bonus."
235
274
  },
236
275
  matched_keywords: {
237
276
  type: "array",
@@ -264,10 +303,17 @@ const LINK_RESOLUTION = {
264
303
  error: { type: "string", description: "Present when the link target could not be resolved (dangling link)." }
265
304
  }
266
305
  };
306
+ /** Stored-entry keywords (mem_write / mem_replace): 2-12, mirrors MAX_WRITE_KEYWORDS. */
267
307
  const KEYWORDS_ARRAY = {
268
308
  type: "array",
269
309
  items: { type: "string" },
270
- description: "1-12 keywords (include 中英文 synonyms when useful)."
310
+ description: "2-12 keywords (include 中英文 synonyms when useful)."
311
+ };
312
+ /** Query keywords (mem_search): 1-15, mirrors MAX_SEARCH_KEYWORDS. */
313
+ const SEARCH_KEYWORDS_ARRAY = {
314
+ type: "array",
315
+ items: { type: "string" },
316
+ description: "1-15 keywords (include 中英文 synonyms when useful)."
271
317
  };
272
318
  const RELATED_BRANCHES = {
273
319
  type: "array",
@@ -342,9 +388,9 @@ function formatSearchResults(value) {
342
388
  function registerMemTools(ctx, config) {
343
389
  ctx.tools.register(defineTool({
344
390
  name: "mem_search",
345
- description: "Search the gitmemo long-term memory (.mem git repo) for past task outcomes. Call BEFORE starting repo-related work: extract 1-12 中英文关键词 from the user request (include synonyms in both languages when useful) and run mem_search; pure chat and general Q&A need no search. Returns a snapshot, total, next_skip and up to 20 scored hits with summary/keywords/kind/matched_keywords — score = distinct matched keywords + a mild recency bonus, so newer entries rank slightly higher and superseded conclusions lose to their replacements. Select at most 5 most relevant hits and mem_read them (prefer kind:\"topic\" hits as topic-page entry points). Paginate by passing skip and the returned snapshot back unchanged; a stale snapshot is rejected explicitly. An optional System-one gate may narrow the page: when the output carries a `gated` block, the entries it removed are listed as 8-char `dropped_hashes` — mem_read one of those if you suspect a relevant memory was filtered out (they stay fully readable). A `NOTE(...)` on the `gated` line means every candidate was judged irrelevant and only a `minKeep` floor was retained, not that one candidate passed.",
391
+ description: "Search the gitmemo long-term memory (.mem git repo) for past task outcomes. Call BEFORE starting repo-related work: extract 1-15 中英文关键词 from the user request (include synonyms in both languages when useful) and run mem_search; pure chat and general Q&A need no search. Returns a snapshot, total, next_skip and up to 20 scored hits with summary/keywords/kind/matched_keywords — score = distinct matched keywords + a mild recency bonus, so newer entries rank slightly higher and superseded conclusions lose to their replacements. Select at most 5 most relevant hits and mem_read them (prefer kind:\"topic\" hits as topic-page entry points). Paginate by passing skip and the returned snapshot back unchanged; a stale snapshot is rejected explicitly. An optional System-one gate may narrow the page: when the output carries a `gated` block, the entries it removed are listed as 8-char `dropped_hashes` — mem_read one of those if you suspect a relevant memory was filtered out (they stay fully readable). A `NOTE(...)` on the `gated` line means every candidate was judged irrelevant and only a `minKeep` floor was retained, not that one candidate passed.",
346
392
  parameters: {
347
- keywords: { ...KEYWORDS_ARRAY, required: true },
393
+ keywords: { ...SEARCH_KEYWORDS_ARRAY, required: true },
348
394
  skip: {
349
395
  type: "integer",
350
396
  description: "Pagination offset (default 0)."
@@ -454,8 +500,10 @@ function registerMemTools(ctx, config) {
454
500
  " candidates=" + candidates.length +
455
501
  " judged=" + decision.judged +
456
502
  (decision.untouched === 0 ? "" : " untouched=" + decision.untouched) +
503
+ " policy=" + decision.policy +
457
504
  " kept=" + decision.kept.length +
458
505
  " dropped=" + decision.dropped.length +
506
+ (decision.bounded ? " bounded=yes" : "") +
459
507
  (decision.reason === undefined ? "" : " reason=" + decision.reason) +
460
508
  (decision.usage?.inputTokens === undefined ? "" : " input_tokens=" + decision.usage.inputTokens) +
461
509
  " verdicts=" +
@@ -463,10 +511,20 @@ function registerMemTools(ctx, config) {
463
511
  .map((verdict) => verdict.hash.slice(0, 8) + ":" + (verdict.value ?? "na") + ":" + (verdict.keep ? "keep" : "drop"))
464
512
  .join(","));
465
513
  const kept = new Set(decision.kept);
514
+ // `rerank` reorders the page by the judge's value and drops nothing;
515
+ // `filter` keeps the shipped behaviour, already bounded by
516
+ // `maxDropFraction` inside `decide`. Both preserve page order for
517
+ // everything the gate left alone.
518
+ const surviving = result.results.filter((hit) => kept.has(hit.hash));
519
+ const reranked = decision.policy === "rerank"
520
+ ? [...surviving].sort((a, b) => decision.ordered.indexOf(a.hash) - decision.ordered.indexOf(b.hash))
521
+ : surviving;
466
522
  return {
467
523
  ...result,
468
- results: result.results.filter((hit) => kept.has(hit.hash)),
524
+ results: reranked,
469
525
  gated: {
526
+ policy: decision.policy,
527
+ ...(decision.bounded ? { bounded: true } : {}),
470
528
  mode: decision.mode,
471
529
  model: decision.model,
472
530
  candidates: candidates.length,
@@ -746,10 +804,10 @@ function registerPromptSection(ctx) {
746
804
  order: 150,
747
805
  text: [
748
806
  "This deployment provides gitmemo long-term memory: a local .mem git repository at the project root stores past task outcomes as immutable markdown entries; git is the only dependency. Use the dedicated tools only (mem_search / mem_read / mem_write / mem_delete / mem_replace — all auto-initialize, and legacy-format .mem repos migrate automatically on first use); never shell out to git or read .mem files directly.",
749
- "- BEFORE WORK — search: 仓库相关任务开始前提取 1–12 个中英文关键词调用 mem_search（含中英文同义词）；纯闲聊和通用问答无需搜索。",
807
+ "- BEFORE WORK — search: 仓库相关任务开始前提取 1–15 个中英文关键词调用 mem_search（含中英文同义词；搜索关键词上限高于写入上限，详见 KEYWORD SELECTION）；纯闲聊和通用问答无需搜索。",
750
808
  "- RESULT PRESELECT — 根据 title、summary、keywords、score 和 matched keywords 选择最多 5 条执行 mem_read，复用相关结论；kind 为 topic 的条目是该主题的聚合入口，优先读；无相关结果时用返回的 snapshot + skip 翻页。",
751
809
  "- END-OF-SESSION CHECKPOINT — the ONLY write path: 会话结束时回顾全程，仅当结论有价值、可复用或用户明确要求记住时 mem_write；纯问答、未完成任务、非仓库工作、纯操作类 git 动作（仅 commit/push）不写；避免语义重复。",
752
- "- KEYWORD SELECTION — mem_write/mem_replace 的 keywords 应选择与任务相关但未出现在 title/summary 中的词（中英文同义词皆可）；title/summary 里已有的词本身就能被检索到，重复它们不会提高召回率。",
810
+ "- KEYWORD SELECTION — mem_write/mem_replace 的 keywords 应选择与任务相关但未出现在 title/summary 中的词（中英文同义词皆可，2–12 个）；mem_search 的查询词上限更高（1–15 个），多给近义词/中英对照可提升召回。title/summary 里已有的词本身就能被检索到，重复它们不会提高召回率。",
753
811
  "- KEEP FRESH — 结论保鲜是写路径的一部分，不必等用户指出：用户更正旧结论时，有替代结论用 mem_replace（禁止先 delete 再 write），作废且无替代用带 reason 的 mem_delete；本次会话的工作覆盖/推翻了某条已存结论（被后续实现、重构取代）时，同样主动 mem_replace 或 mem_delete 对应旧条目；检索结果中新旧结论冲突时，优先采信 score 更高/日期更新的条目（score 已含温和的 recency 加成）。",
754
812
  "- TOPIC PAGE — 同一主题已积累多条记忆、或需要一个「当前真相」入口时，用 mem_write 写一条 kind:\"topic\" 聚合条目：summary 概括当下仍然有效的结论，content 用 [[<commit-hash>]] 链接各证据条目并各配一句话状态；topic 条目靠 mem_replace 演进（kind 自动继承），不是按任务新增。",
755
813
  "- WIKI LINKS — content 中引用其他记忆条目一律写 [[<commit-hash>]]（用 mem_search/mem_read 返回的 hash）；mem_read 传 expand:true 会把链接一跳展开到各条目的最新版本（悬空链接以 error 返回），读 topic 条目时建议展开。",
@@ -769,6 +827,7 @@ async function apply(ctx, config = {}) {
769
827
  };
770
828
  const resolved = {
771
829
  searchLimit: config.searchLimit ?? 20,
830
+ searchScoring: config.searchScoring ?? "weighted",
772
831
  lockTimeoutMs: config.lockTimeoutMs ?? 30000,
773
832
  projectRoot: config.projectRoot,
774
833
  log,
@@ -793,6 +852,8 @@ async function apply(ctx, config = {}) {
793
852
  threshold: section.threshold,
794
853
  scoreMin: section.scoreMin,
795
854
  minKeep: section.minKeep,
855
+ policy: section.policy,
856
+ maxDropFraction: section.maxDropFraction,
796
857
  maxCandidates: section.maxCandidates,
797
858
  maxTaskChars: section.maxTaskChars,
798
859
  timeoutMs: section.timeoutMs,
package/lib/mem.d.ts CHANGED
@@ -10,6 +10,15 @@ export declare const LOCK_FILE_NAME = ".mem.gitmemo.lock";
10
10
  export declare const MIGRATION_JOURNAL_NAME = ".mem.gitmemo-migration.json";
11
11
  /** Max entry content size. */
12
12
  export declare const MAX_CONTENT_BYTES: number;
13
+ /** Upper bound on stored entry keywords (mem_write / mem_replace). */
14
+ export declare const MAX_WRITE_KEYWORDS = 12;
15
+ /**
16
+ * Upper bound on query keywords for one mem_search call. Deliberately higher
17
+ * than the write bound: recall is a fixed-string OR over the query terms, so a
18
+ * wider (still human-curated) query only widens recall, while every matched
19
+ * term still contributes to the score and the write side keeps entries tight.
20
+ */
21
+ export declare const MAX_SEARCH_KEYWORDS = 15;
13
22
  /** Error type thrown by every gitmemo operation. */
14
23
  export declare class GitMemoError extends Error {
15
24
  }
@@ -21,6 +30,17 @@ export interface GitMemoConfig {
21
30
  gitTimeoutMs?: number;
22
31
  /** Cross-process lock wait timeout in milliseconds. Default 30000. */
23
32
  lockTimeoutMs?: number;
33
+ /**
34
+ * How `mem_search` scores a matched entry.
35
+ *
36
+ * - `weighted` (default): sum of per-keyword rarity weights over the recall
37
+ * page. A distinctive match outranks several common ones.
38
+ * - `count`: the previous default behaviour — one point per matched keyword plus the
39
+ * recency bonus. Kept for exact backwards compatibility.
40
+ *
41
+ * Both modes match the same entries; only the ordering differs.
42
+ */
43
+ searchScoring?: SearchScoring;
24
44
  /** @deprecated parsed for compatibility only — the memory dir is fixed at `<projectRoot>/.mem`. */
25
45
  memDirName?: string;
26
46
  /** @deprecated parsed for compatibility only — `.mem` always stays on main. */
@@ -28,6 +48,10 @@ export interface GitMemoConfig {
28
48
  /** @deprecated parsed for compatibility only — session-start recent seeding was removed. */
29
49
  recentContextLimit?: number;
30
50
  }
51
+ /** Search scoring mode; see {@link GitMemoConfig.searchScoring}. */
52
+ export type SearchScoring = "count" | "weighted";
53
+ /** Accepted `searchScoring` values, for config validation and tool schemas. */
54
+ export declare const SEARCH_SCORING_MODES: readonly SearchScoring[];
31
55
  /** One scored search hit. */
32
56
  export interface SearchHit {
33
57
  hash: string;
@@ -258,6 +282,18 @@ export declare function buildEntryMarkdown(input: ValidatedWriteInput, ctx: Code
258
282
  * matched keyword, so freshness only reorders equal-match entries.
259
283
  */
260
284
  export declare function recencyBonus(committerTime: number, nowMs?: number): number;
285
+ /**
286
+ * Rarity weight of a keyword matched by `df` of `n` candidates, in the
287
+ * BM25-style form already used across this project. A term matched by every
288
+ * candidate is worth almost nothing; a term matched by one candidate out of
289
+ * many is worth the most.
290
+ *
291
+ * Used by `searchScoring: "weighted"` so a distinctive match can outrank a
292
+ * pile of common ones. The weight is computed over the *recall page*, which
293
+ * costs no extra git call (the candidate commit messages are already in hand)
294
+ * and is the pool the ordering actually has to get right.
295
+ */
296
+ export declare function keywordRarity(df: number, n: number): number;
261
297
  /** ADD commit message (plan 4.1). */
262
298
  export declare function buildAddMessage(title: string, summary: string, keywords: string[], digest: string, legacy?: boolean, kind?: "task" | "topic"): string;
263
299
  /** DELETE commit message (plan 4.2). */
@@ -349,7 +385,7 @@ export declare class GitMemo {
349
385
  * messages only, active-entry filtering, field-level matching and scoring,
350
386
  * snapshot-stable pagination, in-process LRU.
351
387
  *
352
- * @param keywords - 1-12 keywords (legacy callers may pass a comma-separated string).
388
+ * @param keywords - 1-15 keywords (legacy callers may pass a comma-separated string).
353
389
  * @param options.skip - pagination offset.
354
390
  * @param options.snapshot - pass back the snapshot from the first page.
355
391
  */
package/lib/mem.js CHANGED
@@ -34,9 +34,20 @@ export const LOCK_FILE_NAME = ".mem.gitmemo.lock";
34
34
  export const MIGRATION_JOURNAL_NAME = ".mem.gitmemo-migration.json";
35
35
  /** Max entry content size. */
36
36
  export const MAX_CONTENT_BYTES = 1024 * 1024;
37
+ /** Upper bound on stored entry keywords (mem_write / mem_replace). */
38
+ export const MAX_WRITE_KEYWORDS = 12;
39
+ /**
40
+ * Upper bound on query keywords for one mem_search call. Deliberately higher
41
+ * than the write bound: recall is a fixed-string OR over the query terms, so a
42
+ * wider (still human-curated) query only widens recall, while every matched
43
+ * term still contributes to the score and the write side keeps entries tight.
44
+ */
45
+ export const MAX_SEARCH_KEYWORDS = 15;
37
46
  /** Error type thrown by every gitmemo operation. */
38
47
  export class GitMemoError extends Error {
39
48
  }
49
+ /** Accepted `searchScoring` values, for config validation and tool schemas. */
50
+ export const SEARCH_SCORING_MODES = ["count", "weighted"];
40
51
  const UNSAFE_CONTROL_RE = /[\u0000-\u0008\u000B\u000C\u000E-\u001F\u007F]/;
41
52
  const TRAILER_INJECT_RE = /^\s*GitMemo-[A-Za-z0-9-]+:/im;
42
53
  const COMMIT_RE = /^[0-9a-f]{40}$/;
@@ -256,8 +267,8 @@ export function validateWriteInput(input, legacy = false) {
256
267
  throw new GitMemoError("gitmemo: keywords must be an array");
257
268
  const rawKeywords = input.keywords.map((k) => (typeof k === "string" ? k.trim() : "")).filter((k) => k.length > 0);
258
269
  const minKeywords = legacy ? 0 : 2;
259
- if (rawKeywords.length < minKeywords || rawKeywords.length > 12) {
260
- throw new GitMemoError(`gitmemo: keywords must contain ${minKeywords === 0 ? "0-12" : "2-12"} entries, got ${rawKeywords.length}`);
270
+ if (rawKeywords.length < minKeywords || rawKeywords.length > MAX_WRITE_KEYWORDS) {
271
+ throw new GitMemoError(`gitmemo: keywords must contain ${minKeywords === 0 ? "0" : "2"}-${MAX_WRITE_KEYWORDS} entries, got ${rawKeywords.length}`);
261
272
  }
262
273
  const keywords = [];
263
274
  const seen = new Set();
@@ -271,7 +282,7 @@ export function validateWriteInput(input, legacy = false) {
271
282
  keywords.push(kw);
272
283
  }
273
284
  if (keywords.length < minKeywords) {
274
- throw new GitMemoError(`gitmemo: keywords must contain ${minKeywords === 0 ? "0-12" : "2-12"} distinct entries after normalization, got ${keywords.length}`);
285
+ throw new GitMemoError(`gitmemo: keywords must contain ${minKeywords === 0 ? "0" : "2"}-${MAX_WRITE_KEYWORDS} distinct entries after normalization, got ${keywords.length}`);
275
286
  }
276
287
  let related_branches = [];
277
288
  if (input.related_branches !== undefined) {
@@ -495,6 +506,23 @@ export function recencyBonus(committerTime, nowMs = Date.now()) {
495
506
  return 0;
496
507
  return Math.round(((0.5 * (180 - ageDays)) / 150) * 100) / 100;
497
508
  }
509
+ /**
510
+ * Rarity weight of a keyword matched by `df` of `n` candidates, in the
511
+ * BM25-style form already used across this project. A term matched by every
512
+ * candidate is worth almost nothing; a term matched by one candidate out of
513
+ * many is worth the most.
514
+ *
515
+ * Used by `searchScoring: "weighted"` so a distinctive match can outrank a
516
+ * pile of common ones. The weight is computed over the *recall page*, which
517
+ * costs no extra git call (the candidate commit messages are already in hand)
518
+ * and is the pool the ordering actually has to get right.
519
+ */
520
+ export function keywordRarity(df, n) {
521
+ if (n <= 0)
522
+ return 0;
523
+ const present = Math.min(Math.max(df, 0), n);
524
+ return Math.log(1 + (n - present + 0.5) / (present + 0.5));
525
+ }
498
526
  /** Commit trailer emitted only for topic pages (absence means "task"). */
499
527
  const KIND_TRAILER_TOPIC = "GitMemo-Kind: topic";
500
528
  /** ADD commit message (plan 4.1). */
@@ -1248,10 +1276,15 @@ export class GitMemo {
1248
1276
  }
1249
1277
  return value;
1250
1278
  };
1279
+ const scoring = config.searchScoring ?? "weighted";
1280
+ if (!SEARCH_SCORING_MODES.includes(scoring)) {
1281
+ throw new GitMemoError(`gitmemo: searchScoring must be one of ${SEARCH_SCORING_MODES.join(" | ")}, got ${JSON.stringify(config.searchScoring)}`);
1282
+ }
1251
1283
  this.config = {
1252
1284
  searchLimit: positiveInt(config.searchLimit, 20, "searchLimit"),
1253
1285
  gitTimeoutMs: positiveInt(config.gitTimeoutMs, 60000, "gitTimeoutMs"),
1254
- lockTimeoutMs: positiveInt(config.lockTimeoutMs, 30000, "lockTimeoutMs")
1286
+ lockTimeoutMs: positiveInt(config.lockTimeoutMs, 30000, "lockTimeoutMs"),
1287
+ searchScoring: scoring
1255
1288
  };
1256
1289
  this.memDir = join(this.root, ".mem");
1257
1290
  }
@@ -1299,7 +1332,7 @@ export class GitMemo {
1299
1332
  * messages only, active-entry filtering, field-level matching and scoring,
1300
1333
  * snapshot-stable pagination, in-process LRU.
1301
1334
  *
1302
- * @param keywords - 1-12 keywords (legacy callers may pass a comma-separated string).
1335
+ * @param keywords - 1-15 keywords (legacy callers may pass a comma-separated string).
1303
1336
  * @param options.skip - pagination offset.
1304
1337
  * @param options.snapshot - pass back the snapshot from the first page.
1305
1338
  */
@@ -1312,7 +1345,7 @@ export class GitMemo {
1312
1345
  .map((k) => k.trim())
1313
1346
  .filter((k) => k.length > 0)
1314
1347
  .map((k) => normalizeKeyword(k));
1315
- adapterWarning = "gitmemo: deprecation — mem_search.keywords as a comma-separated string; pass an array of 1-12 keywords";
1348
+ adapterWarning = `gitmemo: deprecation — mem_search.keywords as a comma-separated string; pass an array of 1-${MAX_SEARCH_KEYWORDS} keywords`;
1316
1349
  }
1317
1350
  else {
1318
1351
  if (!Array.isArray(keywords))
@@ -1324,8 +1357,8 @@ export class GitMemo {
1324
1357
  adapterWarning = (adapterWarning ? adapterWarning + " " : "") +
1325
1358
  "gitmemo: deprecation — mem_search.mode is gone; recall is always OR";
1326
1359
  }
1327
- if (normalized.length < 1 || normalized.length > 12) {
1328
- throw new GitMemoError("gitmemo: search requires 1-12 non-empty keywords");
1360
+ if (normalized.length < 1 || normalized.length > MAX_SEARCH_KEYWORDS) {
1361
+ throw new GitMemoError(`gitmemo: search requires 1-${MAX_SEARCH_KEYWORDS} non-empty keywords`);
1329
1362
  }
1330
1363
  const skip = options.skip ?? 0;
1331
1364
  if (!Number.isInteger(skip) || skip < 0)
@@ -1348,7 +1381,11 @@ export class GitMemo {
1348
1381
  : undefined;
1349
1382
  return { ...resolved, blockedNote: note };
1350
1383
  });
1351
- const cacheKey = this.memDir + "\x00" + snapshot + "\x00" + normalized.join("\x00");
1384
+ // The scoring mode is part of the key: two handles on the SAME repo with
1385
+ // different `searchScoring` must not share a cached ranking. (Omitting it
1386
+ // made an A/B of the two modes return identical results — the second
1387
+ // handle simply read the first one's cached computation.)
1388
+ const cacheKey = this.memDir + "\x00" + snapshot + "\x00" + this.config.searchScoring + "\x00" + normalized.join("\x00");
1352
1389
  let computation = searchCache.get(cacheKey);
1353
1390
  if (computation === undefined) {
1354
1391
  computation = await this.computeSearch(snapshot, normalized, legacy);
@@ -1383,6 +1420,15 @@ export class GitMemo {
1383
1420
  const diagnostics = [...(legacySnapshot?.diagnostics ?? [])];
1384
1421
  const seenFiles = new Set();
1385
1422
  const nowMs = Date.now();
1423
+ // Two passes: collect the matching candidates first, then (in weighted
1424
+ // mode) score each match by how rare its keyword is *within this recall
1425
+ // page*. Saturation-free counting treats a distinctive term like
1426
+ // "biscuit" exactly like a ubiquitous one, which is what the LongMemEval
1427
+ // measurements pinned as the engine's ranking defect: evidence was almost
1428
+ // always reachable (recall_all@50 = 0.998) but ranked poorly
1429
+ // (ndcg_any@5 = 0.686). Rarity re-weights the same matches without
1430
+ // changing which entries match at all, so recall is untouched by design.
1431
+ const scored = [];
1386
1432
  for (const record of candidates) {
1387
1433
  if (record.added.length !== 1) {
1388
1434
  if (record.added.length > 1)
@@ -1409,6 +1455,24 @@ export class GitMemo {
1409
1455
  }
1410
1456
  if (matched.length === 0)
1411
1457
  continue;
1458
+ scored.push({ record, matched });
1459
+ }
1460
+ let weights;
1461
+ if (this.config.searchScoring === "weighted" && scored.length > 0) {
1462
+ weights = new Map();
1463
+ for (const kw of kws) {
1464
+ let df = 0;
1465
+ for (const entry of scored) {
1466
+ if (entry.matched.includes(kw))
1467
+ df += 1;
1468
+ }
1469
+ weights.set(kw, keywordRarity(df, scored.length));
1470
+ }
1471
+ }
1472
+ for (const { record, matched } of scored) {
1473
+ const base = weights === undefined
1474
+ ? matched.length
1475
+ : matched.reduce((sum, kw) => sum + (weights.get(kw) ?? 0), 0);
1412
1476
  hits.push({
1413
1477
  hash: record.hash,
1414
1478
  title: record.message.subject,
@@ -1416,7 +1480,7 @@ export class GitMemo {
1416
1480
  summary: record.message.summary,
1417
1481
  keywords: record.message.keywords,
1418
1482
  kind: record.message.kind,
1419
- score: Math.round((matched.length + recencyBonus(record.committerTime, nowMs)) * 100) / 100,
1483
+ score: Math.round((base + recencyBonus(record.committerTime, nowMs)) * 100) / 100,
1420
1484
  matched_keywords: matched
1421
1485
  });
1422
1486
  }
package/lib/systemone.d.ts CHANGED
@@ -54,6 +54,15 @@ export interface GateOptions {
54
54
  scoreMin: number;
55
55
  /** Retain at least this many candidates when the model rejects the whole page. */
56
56
  minKeep: number;
57
+ /** What the caller does with the values; see {@link GatePolicy}. */
58
+ policy?: GatePolicy;
59
+ /**
60
+ * `filter` policy only: never drop more than this fraction of the judged page,
61
+ * lowest-scoring first. Bounds the damage a mis-calibrated threshold can do —
62
+ * an offline sweep kept recall_all@5 at 0.8875 with a 20 % cap, against 0.6875
63
+ * uncapped. Default 0.25.
64
+ */
65
+ maxDropFraction?: number;
57
66
  /** Never send more than this many candidates in one request. */
58
67
  maxCandidates: number;
59
68
  /** Truncate the task text to this many characters. */
@@ -96,6 +105,16 @@ export interface GateDecision {
96
105
  dropped: string[];
97
106
  /** Per-candidate verdicts, in candidate order. */
98
107
  verdicts: GateVerdict[];
108
+ /** Policy actually applied; see {@link GatePolicy}. */
109
+ policy: GatePolicy;
110
+ /** True when `maxDropFraction` restored candidates a threshold would have removed. */
111
+ bounded: boolean;
112
+ /**
113
+ * Candidate hashes ordered by the judge's value, best first (nulls last and
114
+ * kept stable). This is what the `rerank` policy applies; in `filter` mode it
115
+ * is still reported so a caller can log or inspect the judge's ordering.
116
+ */
117
+ ordered: string[];
99
118
  /** Question type actually used. */
100
119
  mode: SystemOneMode;
101
120
  /** Model identifier the request named. */
@@ -142,6 +161,23 @@ export declare const NOUL_CRITERIA: {
142
161
  };
143
162
  /** The judgement asked of the model in `score` mode. */
144
163
  export declare const SCORE_INSTRUCTION = "How relevant is the `candidate` memory record to the `task`? Rate how reusable its conclusions, decisions, constraints, or findings are for this exact task.";
164
+ /**
165
+ * What the caller does with the judge's values.
166
+ *
167
+ * - `rerank` (default): keep every judged candidate and let the values reorder
168
+ * the page. Cannot lose a memory by construction.
169
+ * - `filter`: drop candidates below the threshold, then apply `maxDropFraction`.
170
+ *
171
+ * Measured on LongMemEval `_S` (500 questions, identical ingested memories): at
172
+ * threshold 0.5 the filter held 343 gold memories out of the prompt across 254
173
+ * instances and took LLM-as-Judge from 0.7840 to 0.3800, while the same values
174
+ * used as a reranker scored 0.8020 on the same context. The judge's AUC is only
175
+ * ~0.71-0.75, so no threshold can make a hard drop safe; the same signal is a
176
+ * fine ordering feature. See evals/longmemeval/README.md.
177
+ */
178
+ export type GatePolicy = "rerank" | "filter";
179
+ /** Accepted `policy` values, for config validation and tool schemas. */
180
+ export declare const GATE_POLICIES: readonly GatePolicy[];
145
181
  /**
146
182
  * Build the System-one request body: the task is the shared `state`, and each
147
183
  * candidate travels inside its own question's structured `instructions`, so
@@ -192,11 +228,12 @@ export declare function parseUsage(payload: unknown): GateUsage | undefined;
192
228
  * @param options - resolved gate settings.
193
229
  * @returns kept/dropped hash lists and per-candidate verdicts.
194
230
  */
195
- export declare function decide(candidates: readonly GateCandidate[], values: ReadonlyArray<number | undefined>, options: Pick<GateOptions, "mode" | "threshold" | "scoreMin" | "minKeep">): {
231
+ export declare function decide(candidates: readonly GateCandidate[], values: ReadonlyArray<number | undefined>, options: Pick<GateOptions, "mode" | "threshold" | "scoreMin" | "minKeep" | "policy" | "maxDropFraction">): {
196
232
  kept: string[];
197
233
  dropped: string[];
198
234
  verdicts: GateVerdict[];
199
235
  floored: boolean;
236
+ bounded: boolean;
200
237
  };
201
238
  /**
202
239
  * Evaluate the gate for one search page.
package/lib/systemone.js CHANGED
@@ -52,6 +52,8 @@ export const NOUL_CRITERIA = {
52
52
  };
53
53
  /** The judgement asked of the model in `score` mode. */
54
54
  export const SCORE_INSTRUCTION = "How relevant is the `candidate` memory record to the `task`? Rate how reusable its conclusions, decisions, constraints, or findings are for this exact task.";
55
+ /** Accepted `policy` values, for config validation and tool schemas. */
56
+ export const GATE_POLICIES = ["rerank", "filter"];
55
57
  /** Trim a candidate summary before it is sent to the model. */
56
58
  function clamp(text, limit) {
57
59
  const collapsed = text.replace(/\s+/g, " ").trim();
@@ -153,7 +155,29 @@ export function parseUsage(payload) {
153
155
  * @returns kept/dropped hash lists and per-candidate verdicts.
154
156
  */
155
157
  export function decide(candidates, values, options) {
158
+ // `decide` is the low-level primitive: with no explicit policy it applies the
159
+ // literal threshold semantics, and with no explicit bound it applies none. The
160
+ // product policy (rerank by default, 25 % drop cap) lives in the config layer,
161
+ // which passes both in explicitly — the same split as `minKeep`.
162
+ const policy = options.policy ?? "filter";
156
163
  const cutoff = options.mode === "noul" ? options.threshold : options.scoreMin;
164
+ if (policy === "rerank") {
165
+ // Reordering cannot lose a memory: report every judged candidate as kept and
166
+ // let `ordered` carry the ranking. The threshold verdict is still visible in
167
+ // each verdict's `value`.
168
+ const all = candidates.map((candidate, index) => ({
169
+ hash: candidate.hash,
170
+ value: values[index] ?? null,
171
+ keep: true
172
+ }));
173
+ return {
174
+ kept: candidates.map((c) => c.hash),
175
+ dropped: [],
176
+ verdicts: all,
177
+ floored: false,
178
+ bounded: false
179
+ };
180
+ }
157
181
  const verdicts = candidates.map((candidate, index) => {
158
182
  const value = values[index];
159
183
  if (value === undefined)
@@ -179,12 +203,40 @@ export function decide(candidates, values, options) {
179
203
  kept = verdicts.filter((v) => v.keep).map((v) => v.hash);
180
204
  floored = true;
181
205
  }
206
+ // Bound the damage: a threshold that rejects most of a page is far more
207
+ // likely to be mis-calibrated than the page is to be mostly irrelevant, and
208
+ // the cost of the two errors is not symmetric — a dropped memory is gone,
209
+ // a kept one only costs context. Restore the best-scoring rejects until the
210
+ // drop share is within `maxDropFraction`.
211
+ let bounded = false;
212
+ const maxDropFraction = options.maxDropFraction ?? 1;
213
+ if (candidates.length > 0 && maxDropFraction < 1) {
214
+ const allowedDrops = Math.max(0, Math.floor(candidates.length * Math.max(0, maxDropFraction)));
215
+ const dropCount = candidates.length - kept.length;
216
+ if (dropCount > allowedDrops) {
217
+ const restoreCount = dropCount - allowedDrops;
218
+ const rejects = verdicts
219
+ .map((verdict, index) => ({ verdict, index }))
220
+ .filter((entry) => !entry.verdict.keep)
221
+ .sort((a, b) => {
222
+ const av = a.verdict.value ?? Number.NEGATIVE_INFINITY;
223
+ const bv = b.verdict.value ?? Number.NEGATIVE_INFINITY;
224
+ return bv - av || a.index - b.index;
225
+ })
226
+ .slice(0, restoreCount);
227
+ for (const entry of rejects)
228
+ entry.verdict.keep = true;
229
+ kept = verdicts.filter((v) => v.keep).map((v) => v.hash);
230
+ bounded = true;
231
+ }
232
+ }
182
233
  const keptSet = new Set(kept);
183
234
  return {
184
235
  kept,
185
236
  dropped: candidates.map((c) => c.hash).filter((hash) => !keptSet.has(hash)),
186
237
  verdicts,
187
- floored
238
+ floored,
239
+ bounded
188
240
  };
189
241
  }
190
242
  /** Combine a caller signal with a timeout, keeping both cancellation sources. */
@@ -206,12 +258,15 @@ function withTimeout(signal, timeoutMs) {
206
258
  */
207
259
  export async function evaluateGate(task, candidates, options, signal) {
208
260
  const mode = options.mode;
261
+ const policy = options.policy ?? "rerank";
209
262
  const base = {
210
263
  mode,
211
- model: options.model
264
+ model: options.model,
265
+ policy,
266
+ bounded: false
212
267
  };
213
268
  if (candidates.length === 0) {
214
- return { ...base, kept: [], dropped: [], verdicts: [], judged: 0, untouched: 0, degraded: false };
269
+ return { ...base, kept: [], dropped: [], verdicts: [], ordered: [], judged: 0, untouched: 0, degraded: false };
215
270
  }
216
271
  const judged = candidates.slice(0, Math.max(1, options.maxCandidates));
217
272
  const untouched = candidates.slice(judged.length);
@@ -220,6 +275,7 @@ export async function evaluateGate(task, candidates, options, signal) {
220
275
  kept: candidates.map((c) => c.hash),
221
276
  dropped: [],
222
277
  verdicts: candidates.map((c) => ({ hash: c.hash, value: null, keep: true })),
278
+ ordered: candidates.map((c) => c.hash),
223
279
  judged: judged.length,
224
280
  untouched: untouched.length,
225
281
  degraded: true,
@@ -260,24 +316,44 @@ export async function evaluateGate(task, candidates, options, signal) {
260
316
  return keepAll("system-one request failed: " + detail);
261
317
  }
262
318
  const values = parseAnswers(payload, mode, judged.length);
263
- const decision = decide(judged, values, options);
319
+ const decision = decide(judged, values, { ...options, policy });
264
320
  // Candidates past `maxCandidates` were never evaluated, so they are never
265
321
  // removed: the gate may only drop what it actually judged. Leaving them out
266
322
  // of the kept set would silently truncate every page larger than the cap.
267
323
  const keptSet = new Set([...decision.kept, ...untouched.map((c) => c.hash)]);
324
+ const verdicts = [
325
+ ...decision.verdicts,
326
+ ...untouched.map((c) => ({ hash: c.hash, value: null, keep: true }))
327
+ ];
328
+ // Best value first; nulls (unjudged / unreadable) keep page order at the end.
329
+ const ordered = candidates
330
+ .map((candidate, index) => ({ candidate, index, value: verdicts[index]?.value ?? null }))
331
+ .sort((a, b) => {
332
+ if (a.value === b.value)
333
+ return a.index - b.index;
334
+ if (a.value === null)
335
+ return 1;
336
+ if (b.value === null)
337
+ return -1;
338
+ return b.value - a.value;
339
+ })
340
+ .map((entry) => entry.candidate.hash);
268
341
  const usage = parseUsage(payload);
269
342
  return {
270
343
  ...base,
344
+ ordered,
345
+ bounded: decision.bounded,
271
346
  kept: [...candidates.map((c) => c.hash).filter((hash) => keptSet.has(hash))],
272
347
  dropped: decision.dropped,
273
- verdicts: [
274
- ...decision.verdicts,
275
- ...untouched.map((c) => ({ hash: c.hash, value: null, keep: true }))
276
- ],
348
+ verdicts,
277
349
  judged: judged.length,
278
350
  untouched: untouched.length,
279
351
  degraded: false,
280
352
  ...(usage === undefined ? {} : { usage }),
281
- ...(decision.floored ? { reason: "no candidate scored above the threshold; retained top " + decision.kept.length } : {})
353
+ ...(decision.floored
354
+ ? { reason: "no candidate scored above the threshold; retained top " + decision.kept.length }
355
+ : decision.bounded
356
+ ? { reason: "threshold would have dropped more than maxDropFraction; restored the best-scoring rejects" }
357
+ : {})
282
358
  };
283
359
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@fonlan/dsh-gitmemo",
3
- "version": "0.4.1",
3
+ "version": "0.5.0",
4
4
  "description": "Git-backed long-term memory for DeepSeek Harness: a local .mem git repo (single main branch, immutable entries, structured commit messages) with mem_search / mem_read / mem_write / mem_delete / mem_replace tools and fully automatic legacy migration (no CLI needed; the CLI remains for explicit control), auto-managed by the root agent with zero manual steps.",
5
5
  "author": "fonlan",
6
6
  "publishConfig": {
@@ -48,6 +48,10 @@
48
48
  "bundle": "tsdown",
49
49
  "watch": "tsdown --watch",
50
50
  "test": "npm run build && node --test",
51
+ "eval:lme": "node evals/longmemeval/run_retrieval.mjs",
52
+ "eval:lme:oracle": "node evals/longmemeval/run_retrieval.mjs --split oracle",
53
+ "eval:lme:s": "node --max-old-space-size=8192 evals/longmemeval/run_retrieval.mjs --split s --concurrency 8",
54
+ "eval:omni:setup": "node evals/omnimemeval/setup.mjs",
51
55
  "prepublishOnly": "npm run build",
52
56
  "release:patch": "npm version patch && git push && git push --tags",
53
57
  "release:minor": "npm version minor && git push && git push --tags",