dsh-limit-break 1.3.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +37 -0
- package/MAINTAIN.md +12 -2
- package/README.md +14 -18
- package/lib/index.js +89 -303
- package/package.json +51 -51
- package/tools/recall-bench.mjs +72 -0
- package/tools/prune-selftest.mjs +0 -70
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,43 @@ All notable changes to `dsh-limit-break`. Format follows
|
|
|
5
5
|
|
|
6
6
|
## Unreleased
|
|
7
7
|
|
|
8
|
+
## 1.5.0 — 2026-10-22
|
|
9
|
+
|
|
10
|
+
### Changed
|
|
11
|
+
- **`memory_recall` looks where you point it.** The scorer used raw substring counts over five
|
|
12
|
+
lowercased blobs, which let common words outvote the one term that identifies a fact ("the",
|
|
13
|
+
"how", "long"). It now drops English stopwords, stems (`slides`/`slide`, `rendering`/`render`),
|
|
14
|
+
matches whole words with a prefix fallback (`ray` → `raytracer`), splits dotted ids into segments
|
|
15
|
+
(`bench.gpu_monte_carlo` → `bench`/`gpu`/`monte`/`carlo`), weights each query token by idf
|
|
16
|
+
against the live store, and adds a bonus for a whole-phrase hit in the statement.
|
|
17
|
+
- **`tools/recall-bench.mjs`** (new): 20 hand-labelled questions → hit@1, hit@3, MRR, misses
|
|
18
|
+
printed. Same 20-fact store, before and after: **hit@1 0.75 → 0.95**, MRR **0.867 → 0.975**,
|
|
19
|
+
hit@3 1.0 → 1.0. The remaining miss ("monte carlo pi benchmark on the CPU") is ambiguous
|
|
20
|
+
between `bench.cpu_suite` and `bench.gpu_monte_carlo` and ranks second.
|
|
21
|
+
|
|
22
|
+
### Notes
|
|
23
|
+
- No tool added, removed or renamed: the four-tool ABI is untouched (see 1.4.0). `tokenize`,
|
|
24
|
+
`scoreFact` and `rankFacts` are newly exported from `lib/index.js` only so the benchmark measures
|
|
25
|
+
the shipped scorer instead of a copy of it.
|
|
26
|
+
- Store format unchanged; 1.2.x–1.4.0 stores load as-is.
|
|
27
|
+
|
|
28
|
+
## 1.4.0 — 2026-10-22
|
|
29
|
+
|
|
30
|
+
### Removed
|
|
31
|
+
- **`memory_prune` is gone.** Added in 1.3.0 and removed 18 hours later, because adding a tool
|
|
32
|
+
to a running plugin **bricks existing sessions**: DSH projects the new tool as a
|
|
33
|
+
`tool_addition` block when an old session resumes, and the API rejects the whole turn
|
|
34
|
+
(`messages.196: tool_addition blocks can only be in system messages`). Two sessions died
|
|
35
|
+
that way. The pruner logic was sound; the delivery was not. A plugin's tool set is part of
|
|
36
|
+
the session ABI, not an implementation detail.
|
|
37
|
+
- `tools/prune-selftest.mjs` removed with the tool; `writeJsonAtomic` was only used by the
|
|
38
|
+
pruner and went with it.
|
|
39
|
+
|
|
40
|
+
### Rules adopted
|
|
41
|
+
- The four tools are **frozen for the 1.x line**. No new tool, no removal: both change the
|
|
42
|
+
tool set that old sessions were created under.
|
|
43
|
+
- Nothing is lost by removing the pruner: the store schema is unchanged, `facts.archive.json`
|
|
44
|
+
stays valid, and 1.2.x / 1.3.0 stores load as-is.
|
|
8
45
|
## 1.3.0 — 2026-10-22
|
|
9
46
|
|
|
10
47
|
### Added
|
package/MAINTAIN.md
CHANGED
|
@@ -108,7 +108,16 @@ git add -A && git commit -m "..." && git push
|
|
|
108
108
|
|
|
109
109
|
---
|
|
110
110
|
|
|
111
|
-
## Backlog(按 2026-10-
|
|
111
|
+
## Backlog(按 2026-10-22 20:10 的真实状态)
|
|
112
|
+
|
|
113
|
+
- [x] **1.5.0 召回排序改进**(2026-10-22):`memory_recall` 由“子串计数”改为 stopword 过滤 +
|
|
114
|
+
轻量词干 + 整词/前缀匹配 + dotted id 分段 + idf 加权 + 短语加分;新增
|
|
115
|
+
`tools/recall-bench.mjs`(20 题),实测 hit@1 **0.75 → 0.95**、MRR **0.867 → 0.975**、hit@3 1.0。
|
|
116
|
+
四工具 ABI 未变,1.4.0 的“工具集冻结”规则仍然成立。
|
|
117
|
+
- [ ] 基准里剩下的 miss(“monte carlo pi benchmark on the CPU”)是 `bench.cpu_suite` 与
|
|
118
|
+
`bench.gpu_monte_carlo` 的真实歧义;再往上做要引入上下文/近义词表,性价比低,当作已知上限。
|
|
119
|
+
- [ ] desktop profile 可能钉在旧版本——每次发布后用
|
|
120
|
+
`dsh.cmd plugin --profile desktop add dsh-limit-break@<version>` 消漂移。
|
|
112
121
|
|
|
113
122
|
- [x] **1.2.0 已上线**(2026-10-20T12:02:52Z):npm `latest` = `1.2.0`,GitHub `origin/main` =
|
|
114
123
|
`fd844a4`,`verify` PASS,`test_plugin.mjs` 6/6 PASS。用户 20:04 完成了 OTP 发布和推送。
|
|
@@ -120,7 +129,8 @@ git add -A && git commit -m "..." && git push
|
|
|
120
129
|
- [ ] npm 上还挂着占位包 `0.0.0-stage`(`stub: true`)。`npm deprecate` 也要 OTP(本轮 EOTP 失败);
|
|
121
130
|
用户一条命令即可:`npm deprecate dsh-limit-break@0.0.0-stage "placeholder - use 1.2.0"`。
|
|
122
131
|
- [ ] `1.0.0` 是无元数据的旧包体(author/repository 都不在)。同样一条 `npm deprecate` 可清掉噪音(需 OTP)。
|
|
123
|
-
- [
|
|
132
|
+
- [x] `facts.json` 的 `env.gpu` 已改写成“GPU compute throughput is recorded separately in
|
|
133
|
+
bench.gpu_monte_carlo and bench.render”(2026-10-21 的回写),陈旧描述不再存在。
|
|
124
134
|
- [x] 插件工具对**已经打开的会话**不可见:已在 README「Known limits」第 1 条写明
|
|
125
135
|
(tool list 是 per-session 快照,插件侧修不了)。事实 `harness.plugin-tool-visibility` 有实测。
|
|
126
136
|
- [ ] `resilient_fetch` 目前按"主机名别名"降级,可以继续加:`--resolve` 到已知 IP、
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# dsh-limit-break
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Four small tools for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) that came out of a
|
|
4
4
|
self-benchmark: two things an agent is normally told it cannot do, made cheap enough to just do.
|
|
5
5
|
|
|
6
6
|
| Tool | What it does |
|
|
@@ -9,32 +9,30 @@ self-benchmark: two things an agent is normally told it cannot do, made cheap en
|
|
|
9
9
|
| `memory_save` | Write or overwrite one fact: `{ id, statement, value?, unit?, details?, tags?, source? }`. |
|
|
10
10
|
| `resilient_fetch` | Fetch with a four-rung ladder and a report of which rung answered. |
|
|
11
11
|
| `memory_stats` | Where the store is, how many facts, how big. |
|
|
12
|
-
| `memory_prune` | Finds redundant facts (session state, exact duplicates) and archives them. Dry run unless `apply: true`; nothing is ever deleted. |
|
|
13
12
|
|
|
14
13
|
Zero npm dependencies. No import-time side effects. Everything fails soft — a broken store or a dead
|
|
15
14
|
network returns an error object, never an exception that could take the host down.
|
|
16
15
|
|
|
16
|
+
**Recall quality.** `memory_recall` is measured, not vibed: `tools/recall-bench.mjs` runs 20
|
|
17
|
+
hand-labelled questions against the live store and prints hit@1 / hit@3 / MRR. At 1.5.0 the ranking
|
|
18
|
+
skips English stopwords, stems, matches whole words (with a prefix fallback, so `ray` finds
|
|
19
|
+
`raytracer`), splits dotted ids into segments, weights each query token by inverse document
|
|
20
|
+
frequency, and rewards a whole-phrase hit: **hit@1 0.75 → 0.95, MRR 0.867 → 0.975** on the same 20
|
|
21
|
+
questions and the same 20 facts. The one remaining miss is a genuinely ambiguous question that
|
|
22
|
+
matches two benchmark facts.
|
|
23
|
+
|
|
17
24
|
**Runtime requirement: `curl`.** Rung 1 of the fetch ladder is plain `fetch` and needs nothing; rungs
|
|
18
25
|
2–4 (IPv6 → IPv4 → sibling hostname) shell out to `curl`, so it has to be on `PATH`. Windows 10+,
|
|
19
26
|
macOS and mainstream Linux all ship it. Without it the ladder degrades to rung 1 and says so in the
|
|
20
27
|
report instead of pretending it climbed.
|
|
21
28
|
|
|
22
|
-
## Why these
|
|
29
|
+
## Why these four
|
|
23
30
|
|
|
24
31
|
**Memory.** A session starts from zero. A store on disk plus a read at the start is enough to beat
|
|
25
32
|
that. Measured with two brand-new agents, zero context, the same five questions: **5/5 with the
|
|
26
33
|
store, 0/5 without** — and the control did not hallucinate, it answered "I don't have that" five
|
|
27
34
|
times.
|
|
28
35
|
|
|
29
|
-
**Redundancy.** A store's slow failure mode is drift: two facts restating one measurement, or a
|
|
30
|
-
fact that is really session state, until a recall returns two slightly different answers. Cleaning
|
|
31
|
-
it by hand works once; `memory_prune` makes it repeatable. The two safe rules — *session state* and
|
|
32
|
-
*exact duplicates* — are deterministic, reported as a plan first, and applied only on
|
|
33
|
-
`apply: true`, with the pre-prune store copied to `facts.json.bak-<timestamp>` and the dropped
|
|
34
|
-
facts moved to `facts.archive.json`. Nothing is deleted. Soft suspicions (facts citing the same
|
|
35
|
-
measurements, facts that contradict each other on one topic) come back as read-only warnings,
|
|
36
|
-
because those need a human decision.
|
|
37
|
-
|
|
38
36
|
**Network.** The reflex "this host is unreachable" is wrong more often than it is right. Measured on
|
|
39
37
|
one network, same edge IP, 75 seconds apart: `api.wikimedia.org` → 200 in 0.43 s,
|
|
40
38
|
`en.wikipedia.org` → connection reset in 1.21 s, `api.wikimedia.org` again → 200 in 0.39 s. The
|
|
@@ -79,9 +77,6 @@ package the profile does not declare is silently ignored.
|
|
|
79
77
|
Plain JSON on purpose: anything that can read a file can use it, with or without this plugin.
|
|
80
78
|
Writes are tmp-file + rename.
|
|
81
79
|
|
|
82
|
-
`memory_prune` keeps the same contract: a plan first, a `facts.json.bak-<timestamp>` copy before
|
|
83
|
-
any write, and the removed facts kept in `facts.archive.json` next to the store.
|
|
84
|
-
|
|
85
80
|
## Disable / remove
|
|
86
81
|
|
|
87
82
|
* Hot: set `disabled: true` on the entry in `cordis.patch.yml` — Cordis recomposes in about a second.
|
|
@@ -100,9 +95,10 @@ Measured, not guessed — each line is something a session actually hit.
|
|
|
100
95
|
machine is the intended use.
|
|
101
96
|
* **The fetch ladder is hostname-level.** It does not defeat IP-level blocks, TLS interception,
|
|
102
97
|
or a proxy that terminates the connection; it does not skip certificate verification.
|
|
103
|
-
* **`memory_recall` ranks by keyword overlap only** (id/topic/tags/statement), no
|
|
104
|
-
|
|
105
|
-
|
|
98
|
+
* **`memory_recall` ranks by keyword overlap only** (id/topic/tags/statement/details), no
|
|
99
|
+
embeddings: stopword-filtered, crudely stemmed word matching weighted by idf, so it is good at
|
|
100
|
+
"which fact did I mean" and bad at paraphrase. `tools/recall-bench.mjs` states the current
|
|
101
|
+
numbers (hit@1 0.95 on 20 questions); a query that shares no word with a fact still misses it.
|
|
106
102
|
* **`resilient_fetch` reads at most 2 MiB per response** and returns at most 20 000 characters;
|
|
107
103
|
bigger payloads are truncated, not streamed.
|
|
108
104
|
|
package/lib/index.js
CHANGED
|
@@ -182,31 +182,105 @@ async function saveStore(dir, store) {
|
|
|
182
182
|
return { file, bytes: Buffer.byteLength(body) }
|
|
183
183
|
}
|
|
184
184
|
|
|
185
|
+
// Ranking inputs. The old scorer used raw substring counts, which let a common word
|
|
186
|
+
// ("the", "how", "long") outvote the one rare word that identifies the fact. Words are
|
|
187
|
+
// matched against word sets instead, suffixed words share a stem, and each query token is
|
|
188
|
+
// weighted by how rare it is in the store (idf), so a distinctive term wins a tie.
|
|
189
|
+
const STOPWORDS = new Set([
|
|
190
|
+
'a', 'about', 'all', 'also', 'an', 'and', 'any', 'are', 'as', 'at', 'be', 'been', 'but', 'by',
|
|
191
|
+
'can', 'could', 'did', 'do', 'does', 'each', 'few', 'for', 'from', 'had', 'has', 'have', 'here',
|
|
192
|
+
'how', 'if', 'in', 'into', 'is', 'it', 'its', 'just', 'many', 'me', 'more', 'most', 'much', 'my',
|
|
193
|
+
'no', 'not', 'of', 'on', 'once', 'only', 'or', 'other', 'our', 'out', 'over', 'should', 'some',
|
|
194
|
+
'such', 'than', 'that', 'the', 'their', 'then', 'there', 'these', 'they', 'this', 'those', 'to',
|
|
195
|
+
'too', 'under', 'up', 'very', 'was', 'we', 'were', 'what', 'when', 'where', 'which', 'who', 'why',
|
|
196
|
+
'will', 'with', 'would', 'you', 'your'
|
|
197
|
+
])
|
|
198
|
+
|
|
199
|
+
function stem(word) {
|
|
200
|
+
let w = word
|
|
201
|
+
if (w.length > 5 && w.endsWith('ies')) return `${w.slice(0, -3)}y`
|
|
202
|
+
if (w.length > 4 && w.endsWith('es')) w = w.slice(0, -2)
|
|
203
|
+
else if (w.length > 3 && w.endsWith('s') && !w.endsWith('ss')) w = w.slice(0, -1)
|
|
204
|
+
if (w.length > 5 && w.endsWith('ing')) w = w.slice(0, -3)
|
|
205
|
+
else if (w.length > 4 && w.endsWith('ed')) w = w.slice(0, -2)
|
|
206
|
+
if (w.length > 5 && w.endsWith('ly')) w = w.slice(0, -2)
|
|
207
|
+
return w
|
|
208
|
+
}
|
|
209
|
+
|
|
185
210
|
function tokenize(query) {
|
|
186
211
|
return String(query || '')
|
|
212
|
+
.toLowerCase()
|
|
213
|
+
// "memory_prune" should match the id segment "memory_prune" as well as "memory"/"prune".
|
|
214
|
+
.replace(/_+/g, ' ')
|
|
215
|
+
.split(/[^\p{L}\p{N}_.+-]+/u)
|
|
216
|
+
.filter((part) => part.length > 1 && !STOPWORDS.has(part))
|
|
217
|
+
.map(stem)
|
|
218
|
+
}
|
|
219
|
+
|
|
220
|
+
function words(text) {
|
|
221
|
+
return String(text || '')
|
|
187
222
|
.toLowerCase()
|
|
188
223
|
.split(/[^\p{L}\p{N}_.+-]+/u)
|
|
189
|
-
.filter(
|
|
224
|
+
.filter(Boolean)
|
|
225
|
+
.map(stem)
|
|
226
|
+
}
|
|
227
|
+
|
|
228
|
+
function wordSet(text) {
|
|
229
|
+
return new Set(words(text))
|
|
230
|
+
}
|
|
231
|
+
|
|
232
|
+
/** 1 for the exact word, 0.5 when the query token is a prefix of a stored word ("ray" ~ "raytracer"). */
|
|
233
|
+
function wordMatch(set, token) {
|
|
234
|
+
if (set.has(token)) return 1
|
|
235
|
+
for (const word of set) {
|
|
236
|
+
if (word.length > token.length && word.startsWith(token)) return 0.5
|
|
237
|
+
}
|
|
238
|
+
return 0
|
|
190
239
|
}
|
|
191
240
|
|
|
192
|
-
|
|
241
|
+
const FIELD_WEIGHTS = { id: 7, topic: 2, tags: 3, statement: 3, details: 1 }
|
|
242
|
+
|
|
243
|
+
function scoreFact(fact, tokens, idf) {
|
|
193
244
|
if (!tokens.length) return 0
|
|
194
|
-
|
|
195
|
-
const
|
|
196
|
-
const topic =
|
|
197
|
-
const tags = (Array.isArray(fact.tags) ? fact.tags : []).join(' ')
|
|
198
|
-
const
|
|
245
|
+
// Ids are dotted paths ("bench.gpu_monte_carlo"); split them so a token can match a segment.
|
|
246
|
+
const id = new Set(String(fact.id || '').toLowerCase().split(/[^\p{L}\p{N}]+/u).filter(Boolean).map(stem))
|
|
247
|
+
const topic = wordSet(fact.topic)
|
|
248
|
+
const tags = wordSet((Array.isArray(fact.tags) ? fact.tags : []).join(' '))
|
|
249
|
+
const statement = wordSet(fact.statement)
|
|
250
|
+
const details = wordSet(fact.details ? JSON.stringify(fact.details) : '')
|
|
199
251
|
let score = 0
|
|
200
252
|
for (const token of tokens) {
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
253
|
+
const boost = idf instanceof Map ? idf.get(token) || 1 : 1
|
|
254
|
+
score += boost * (
|
|
255
|
+
FIELD_WEIGHTS.id * wordMatch(id, token) +
|
|
256
|
+
FIELD_WEIGHTS.topic * wordMatch(topic, token) +
|
|
257
|
+
FIELD_WEIGHTS.tags * wordMatch(tags, token) +
|
|
258
|
+
FIELD_WEIGHTS.statement * wordMatch(statement, token) +
|
|
259
|
+
FIELD_WEIGHTS.details * wordMatch(details, token)
|
|
260
|
+
)
|
|
206
261
|
}
|
|
262
|
+
const phrase = tokens.join(' ')
|
|
263
|
+
if (tokens.length > 1 && String(fact.statement || '').toLowerCase().includes(phrase)) score += 6
|
|
207
264
|
return score
|
|
208
265
|
}
|
|
209
266
|
|
|
267
|
+
/** Scored ordering used by memory_recall; exported so the recall benchmark can measure it. */
|
|
268
|
+
function rankFacts(facts, query) {
|
|
269
|
+
const tokens = tokenize(query)
|
|
270
|
+
const living = (facts || []).filter(Boolean)
|
|
271
|
+
if (!tokens.length) return living.map((fact) => ({ fact, score: 0 }))
|
|
272
|
+
const idf = new Map()
|
|
273
|
+
for (const token of tokens) {
|
|
274
|
+
let df = 0
|
|
275
|
+
for (const fact of living) if (scoreFact(fact, [token]) > 0) df += 1
|
|
276
|
+
idf.set(token, +(1 + Math.log((living.length + 1) / (df + 1))).toFixed(4))
|
|
277
|
+
}
|
|
278
|
+
return living
|
|
279
|
+
.map((fact) => ({ fact, score: scoreFact(fact, tokens, idf) }))
|
|
280
|
+
.filter((row) => row.score > 0)
|
|
281
|
+
.sort((a, b) => b.score - a.score)
|
|
282
|
+
}
|
|
283
|
+
|
|
210
284
|
function shapeFact(fact) {
|
|
211
285
|
return {
|
|
212
286
|
id: fact.id,
|
|
@@ -221,246 +295,6 @@ function shapeFact(fact) {
|
|
|
221
295
|
}
|
|
222
296
|
}
|
|
223
297
|
|
|
224
|
-
/* ------------------------------------------------------------------ prune */
|
|
225
|
-
|
|
226
|
-
/**
|
|
227
|
-
* Redundancy is the slow failure mode of a memory store: two facts that restate one
|
|
228
|
-
* measurement, a fact that is really session state, a stale claim superseded by a better
|
|
229
|
-
* one. They still answer recall fine — until they disagree with each other.
|
|
230
|
-
*
|
|
231
|
-
* The rules below are deliberately deterministic: a plan you can read, never a guess. Only
|
|
232
|
-
* two of them are safe to act on without a human, and even then nothing is deleted — the
|
|
233
|
-
* facts move to `facts.archive.json` and the store is backed up first.
|
|
234
|
-
*/
|
|
235
|
-
const SESSION_STATE = /(in this session|this conversation|current session|the approval policy (?:changed|is|was)\b)/i
|
|
236
|
-
const ANTONYM_PAIRS = [
|
|
237
|
-
['allowed', 'denied'],
|
|
238
|
-
['succeeds', 'fails'],
|
|
239
|
-
['succeeded', 'failed'],
|
|
240
|
-
['worked', 'failed'],
|
|
241
|
-
['passes', 'fails']
|
|
242
|
-
]
|
|
243
|
-
const STOPWORDS = new Set([
|
|
244
|
-
'that', 'this', 'these', 'those', 'their', 'there', 'which', 'while', 'where', 'when', 'with',
|
|
245
|
-
'without', 'about', 'after', 'before', 'would', 'could', 'should', 'being', 'under', 'over',
|
|
246
|
-
'from', 'into', 'than', 'then', 'also', 'been', 'have', 'having', 'will', 'were', 'what'
|
|
247
|
-
])
|
|
248
|
-
|
|
249
|
-
function normalizeStatement(text) {
|
|
250
|
-
return String(text || '')
|
|
251
|
-
.toLowerCase()
|
|
252
|
-
.replace(/[\s\u3000]+/g, ' ')
|
|
253
|
-
.replace(/[.,;:!?'"`()[\]{}]+/g, '')
|
|
254
|
-
.trim()
|
|
255
|
-
}
|
|
256
|
-
|
|
257
|
-
/** Years and other 4-digit calendar noise are not measurements. */
|
|
258
|
-
function isCalendarNoise(token) {
|
|
259
|
-
return /^\d{4}$/.test(token) && Number(token) >= 1990 && Number(token) <= 2100
|
|
260
|
-
}
|
|
261
|
-
|
|
262
|
-
function numbersIn(fact) {
|
|
263
|
-
const hay = [fact.statement, fact.value, fact.details ? JSON.stringify(fact.details) : ''].join(' ')
|
|
264
|
-
const found = new Set()
|
|
265
|
-
for (const match of String(hay).matchAll(/-?\d+(?:\.\d+)?(?:[eE][+-]?\d+)?/g)) {
|
|
266
|
-
const canonical = match[0].replace(/^[-+]/, '')
|
|
267
|
-
if (isCalendarNoise(canonical)) continue
|
|
268
|
-
if (canonical.replace(/[^0-9]/g, '').length >= 3) found.add(canonical)
|
|
269
|
-
}
|
|
270
|
-
return found
|
|
271
|
-
}
|
|
272
|
-
|
|
273
|
-
function keywordsIn(fact) {
|
|
274
|
-
const words = String(fact.statement || '')
|
|
275
|
-
.toLowerCase()
|
|
276
|
-
.split(/[^\p{L}\p{N}_]+/u)
|
|
277
|
-
.filter((word) => word.length >= 4 && !STOPWORDS.has(word) && !/^\d{4}$/.test(word))
|
|
278
|
-
return new Set(words)
|
|
279
|
-
}
|
|
280
|
-
|
|
281
|
-
/** Ordered by how much a fact carries, so a merge keeps the informative one. */
|
|
282
|
-
function richness(fact) { let score = 0
|
|
283
|
-
if (fact.value !== undefined && fact.value !== null) score += 2
|
|
284
|
-
if (fact.details) score += 2
|
|
285
|
-
if (fact.source) score += 1
|
|
286
|
-
if (Array.isArray(fact.tags) && fact.tags.length) score += 1
|
|
287
|
-
score += String(fact.statement || '').length / 1000
|
|
288
|
-
if (fact.timestamp) score += (Date.parse(fact.timestamp) || 0) / 1e15
|
|
289
|
-
return score
|
|
290
|
-
}
|
|
291
|
-
|
|
292
|
-
/**
|
|
293
|
-
* A contradiction must share a *subject*, not just a topic tag: two facts about the network
|
|
294
|
-
* that both mention "host" are not enemies. This only fires on a real antonym pair, either
|
|
295
|
-
* from the list above or from the general un-prefix shape (reachable / unreachable).
|
|
296
|
-
*/
|
|
297
|
-
function antonymConflict(a, b) {
|
|
298
|
-
const ta = String(a.statement || '').toLowerCase()
|
|
299
|
-
const tb = String(b.statement || '').toLowerCase()
|
|
300
|
-
for (const [x, y] of ANTONYM_PAIRS) {
|
|
301
|
-
const rx = new RegExp(`\\b${x}\\b`)
|
|
302
|
-
const ry = new RegExp(`\\b${y}\\b`)
|
|
303
|
-
if (rx.test(ta) && ry.test(tb)) return [x, y]
|
|
304
|
-
if (ry.test(ta) && rx.test(tb)) return [y, x]
|
|
305
|
-
}
|
|
306
|
-
for (const [left, right] of [[ta, tb], [tb, ta]]) {
|
|
307
|
-
for (const raw of left.match(/\bun(\p{L}{4,})\b/gu) || []) {
|
|
308
|
-
const stem = raw.slice(2)
|
|
309
|
-
if (new RegExp(`\\b${stem}\\b`).test(right)) return [stem, raw]
|
|
310
|
-
}
|
|
311
|
-
}
|
|
312
|
-
return null
|
|
313
|
-
}
|
|
314
|
-
|
|
315
|
-
function planPrune(facts) {
|
|
316
|
-
const living = facts.filter(Boolean)
|
|
317
|
-
const candidates = []
|
|
318
|
-
const warnings = []
|
|
319
|
-
const flagged = new Set()
|
|
320
|
-
|
|
321
|
-
const flag = (id, entry) => {
|
|
322
|
-
if (!id || flagged.has(id)) return
|
|
323
|
-
flagged.add(id)
|
|
324
|
-
candidates.push(entry)
|
|
325
|
-
}
|
|
326
|
-
|
|
327
|
-
for (const fact of living) {
|
|
328
|
-
if (SESSION_STATE.test(String(fact.statement || ''))) {
|
|
329
|
-
flag(fact.id, {
|
|
330
|
-
id: fact.id,
|
|
331
|
-
rule: 'session-state',
|
|
332
|
-
statement: fact.statement,
|
|
333
|
-
reason: 'reads as conversational/session state, which a durable store must not hold'
|
|
334
|
-
})
|
|
335
|
-
}
|
|
336
|
-
}
|
|
337
|
-
|
|
338
|
-
const byStatement = new Map()
|
|
339
|
-
for (const fact of living) {
|
|
340
|
-
const key = normalizeStatement(fact.statement)
|
|
341
|
-
if (!key) continue
|
|
342
|
-
if (!byStatement.has(key)) byStatement.set(key, [])
|
|
343
|
-
byStatement.get(key).push(fact)
|
|
344
|
-
}
|
|
345
|
-
for (const group of byStatement.values()) {
|
|
346
|
-
if (group.length < 2) continue
|
|
347
|
-
const [keep, ...rest] = [...group].sort((a, b) => richness(b) - richness(a))
|
|
348
|
-
for (const duplicate of rest) {
|
|
349
|
-
flag(duplicate.id, {
|
|
350
|
-
id: duplicate.id,
|
|
351
|
-
rule: 'exact-duplicate',
|
|
352
|
-
keep: keep.id,
|
|
353
|
-
statement: duplicate.statement,
|
|
354
|
-
reason: `statement is identical to ${keep.id}`
|
|
355
|
-
})
|
|
356
|
-
}
|
|
357
|
-
}
|
|
358
|
-
|
|
359
|
-
for (let i = 0; i < living.length; i += 1) {
|
|
360
|
-
for (let j = i + 1; j < living.length; j += 1) {
|
|
361
|
-
const a = living[i]
|
|
362
|
-
const b = living[j]
|
|
363
|
-
if (a.id === b.id || flagged.has(a.id) || flagged.has(b.id)) continue
|
|
364
|
-
const shared = [...numbersIn(a)].filter((n) => numbersIn(b).has(n))
|
|
365
|
-
const overlap = [...keywordsIn(a)].filter((k) => keywordsIn(b).has(k))
|
|
366
|
-
// Two facts citing the same tiny number can be unrelated (a year, a version); require
|
|
367
|
-
// either several numbers or one number plus overlapping prose.
|
|
368
|
-
if (shared.length >= 2 || (shared.length >= 1 && overlap.length >= 2)) {
|
|
369
|
-
warnings.push({
|
|
370
|
-
rule: 'shared-numbers',
|
|
371
|
-
ids: [a.id, b.id],
|
|
372
|
-
shared,
|
|
373
|
-
reason: 'both facts cite the same measurements: one is probably a subset of the other'
|
|
374
|
-
})
|
|
375
|
-
}
|
|
376
|
-
const sameTopic = (a.topic || '') && a.topic === b.topic
|
|
377
|
-
if (sameTopic && overlap.length >= 1) {
|
|
378
|
-
const conflict = antonymConflict(a, b)
|
|
379
|
-
if (conflict) {
|
|
380
|
-
warnings.push({
|
|
381
|
-
rule: 'contradiction',
|
|
382
|
-
ids: [a.id, b.id],
|
|
383
|
-
terms: conflict,
|
|
384
|
-
shared: overlap.slice(0, 4),
|
|
385
|
-
reason: 'same topic, opposite claims: one of them is stale by definition'
|
|
386
|
-
})
|
|
387
|
-
}
|
|
388
|
-
}
|
|
389
|
-
}
|
|
390
|
-
}
|
|
391
|
-
return { candidates, warnings }
|
|
392
|
-
}
|
|
393
|
-
|
|
394
|
-
/** Dry run by default. Nothing leaves the store without a backup and an archive copy. */
|
|
395
|
-
async function pruneStore(dir, options) {
|
|
396
|
-
const apply = Boolean(options && options.apply)
|
|
397
|
-
const store = await loadStore(dir)
|
|
398
|
-
const archiveFile = join(dir, 'facts.archive.json')
|
|
399
|
-
if (!store.existed) {
|
|
400
|
-
return { ok: false, code: 'memory.empty', store: store.file, total: 0, archived: 0, candidates: [], warnings: [] }
|
|
401
|
-
}
|
|
402
|
-
const living = store.facts.filter(Boolean)
|
|
403
|
-
const plan = planPrune(living)
|
|
404
|
-
const total = living.length
|
|
405
|
-
const kept = total - plan.candidates.length
|
|
406
|
-
|
|
407
|
-
if (!apply || plan.candidates.length === 0) {
|
|
408
|
-
return {
|
|
409
|
-
ok: true,
|
|
410
|
-
mode: apply ? 'applied' : 'plan',
|
|
411
|
-
store: store.file,
|
|
412
|
-
archiveFile,
|
|
413
|
-
total,
|
|
414
|
-
kept,
|
|
415
|
-
archived: plan.candidates.length,
|
|
416
|
-
candidates: plan.candidates,
|
|
417
|
-
warnings: plan.warnings,
|
|
418
|
-
changed: false
|
|
419
|
-
}
|
|
420
|
-
}
|
|
421
|
-
|
|
422
|
-
const entries = []
|
|
423
|
-
try {
|
|
424
|
-
const existing = JSON.parse(await readFile(archiveFile, 'utf8'))
|
|
425
|
-
if (Array.isArray(existing)) entries.push(...existing)
|
|
426
|
-
} catch {
|
|
427
|
-
entries.length = 0
|
|
428
|
-
}
|
|
429
|
-
const byId = new Map(living.map((fact) => [fact.id, fact]))
|
|
430
|
-
const archivedAt = new Date().toISOString()
|
|
431
|
-
for (const candidate of plan.candidates) {
|
|
432
|
-
entries.push({ fact: byId.get(candidate.id), archivedAt, rule: candidate.rule, reason: candidate.reason })
|
|
433
|
-
}
|
|
434
|
-
|
|
435
|
-
const backupFile = `${store.file}.bak-${archivedAt.replace(/[:.]/g, '-')}`
|
|
436
|
-
await writeFile(backupFile, await readFile(store.file, 'utf8'), 'utf8')
|
|
437
|
-
await writeJsonAtomic(archiveFile, entries)
|
|
438
|
-
const dropped = new Set(plan.candidates.map((candidate) => candidate.id))
|
|
439
|
-
store.facts = living.filter((fact) => !dropped.has(fact.id))
|
|
440
|
-
await saveStore(dir, store)
|
|
441
|
-
|
|
442
|
-
return {
|
|
443
|
-
ok: true,
|
|
444
|
-
mode: 'applied',
|
|
445
|
-
store: store.file,
|
|
446
|
-
archiveFile,
|
|
447
|
-
backupFile,
|
|
448
|
-
total,
|
|
449
|
-
kept: store.facts.length,
|
|
450
|
-
archived: plan.candidates.length,
|
|
451
|
-
candidates: plan.candidates,
|
|
452
|
-
warnings: plan.warnings,
|
|
453
|
-
changed: true
|
|
454
|
-
}
|
|
455
|
-
}
|
|
456
|
-
|
|
457
|
-
async function writeJsonAtomic(file, value) {
|
|
458
|
-
await mkdir(dirname(file), { recursive: true })
|
|
459
|
-
const tmp = `${file}.${process.pid}.tmp`
|
|
460
|
-
await writeFile(tmp, JSON.stringify(value, null, 2), 'utf8')
|
|
461
|
-
await rename(tmp, file)
|
|
462
|
-
}
|
|
463
|
-
|
|
464
298
|
/* ------------------------------------------------------------------ fetch */
|
|
465
299
|
|
|
466
300
|
/**
|
|
@@ -666,11 +500,7 @@ export function apply(ctx, config) {
|
|
|
666
500
|
const tokens = tokenize(input.query)
|
|
667
501
|
let picked = store.facts.filter(Boolean)
|
|
668
502
|
if (!input.all && tokens.length) {
|
|
669
|
-
picked = picked
|
|
670
|
-
.map((fact) => ({ fact, score: scoreFact(fact, tokens) }))
|
|
671
|
-
.filter((row) => row.score > 0)
|
|
672
|
-
.sort((a, b) => b.score - a.score)
|
|
673
|
-
.map((row) => row.fact)
|
|
503
|
+
picked = rankFacts(picked, input.query).map((row) => row.fact)
|
|
674
504
|
}
|
|
675
505
|
// Number(input.limit) || 8 turned 0/NaN into 8 but let Infinity through to slice()
|
|
676
506
|
// (which returns everything) and gave no upper bound below the cap.
|
|
@@ -870,54 +700,10 @@ export function apply(ctx, config) {
|
|
|
870
700
|
'limit-break: memory_stats tool'
|
|
871
701
|
)
|
|
872
702
|
|
|
873
|
-
/* ---- memory_prune ------------------------------------------------- */
|
|
874
|
-
effect(
|
|
875
|
-
() =>
|
|
876
|
-
ctx.tools.register({
|
|
877
|
-
name: 'memory_prune',
|
|
878
|
-
description:
|
|
879
|
-
'Find redundant facts in the durable memory store and archive them. Never deletes: the plan is ' +
|
|
880
|
-
'returned first (apply defaults to false), and applying writes facts.archive.json plus a ' +
|
|
881
|
-
'facts.json.bak-<timestamp> backup. Candidates are deterministic (session state, exact duplicates); ' +
|
|
882
|
-
'softer suspicions (shared measurements, contradictory claims) come back as warnings only.',
|
|
883
|
-
parameters: {
|
|
884
|
-
type: 'object',
|
|
885
|
-
properties: {
|
|
886
|
-
apply: {
|
|
887
|
-
type: 'boolean',
|
|
888
|
-
description: 'Archive the safe candidates. Defaults to false: report the plan and change nothing.'
|
|
889
|
-
}
|
|
890
|
-
},
|
|
891
|
-
additionalProperties: false
|
|
892
|
-
},
|
|
893
|
-
output: {
|
|
894
|
-
schema: { type: 'object' },
|
|
895
|
-
render: (_args, value) => {
|
|
896
|
-
if (!value || !value.ok) return [{ type: 'text', text: 'Memory store is empty or unreadable.' }]
|
|
897
|
-
const head = value.mode === 'applied'
|
|
898
|
-
? `Archived ${value.archived} of ${value.total} facts -> ${value.archiveFile}`
|
|
899
|
-
: `Plan: ${value.archived} of ${value.total} facts are safe to archive (nothing changed yet)`
|
|
900
|
-
const lines = value.candidates.map((c) => `- ${c.id} [${c.rule}]${c.keep ? ` keep ${c.keep}` : ''}`)
|
|
901
|
-
const warns = value.warnings.map((w) => `- ~ ${w.ids.join(' vs ')} [${w.rule}]`)
|
|
902
|
-
return [{ type: 'text', text: [head, ...lines, ...warns].join('\n') }]
|
|
903
|
-
}
|
|
904
|
-
},
|
|
905
|
-
presentCall: (args) => ({
|
|
906
|
-
card: 'generic',
|
|
907
|
-
title: 'Memory prune',
|
|
908
|
-
kind: args && args.apply ? 'write' : 'read'
|
|
909
|
-
}),
|
|
910
|
-
async execute(args) {
|
|
911
|
-
return pruneStore(storeDir, { apply: Boolean(args && args.apply) })
|
|
912
|
-
}
|
|
913
|
-
}),
|
|
914
|
-
'limit-break: memory_prune tool'
|
|
915
|
-
)
|
|
916
|
-
|
|
917
703
|
diag(config, 'mounted', {
|
|
918
|
-
tools: ['memory_recall', 'memory_save', 'resilient_fetch', 'memory_stats'
|
|
704
|
+
tools: ['memory_recall', 'memory_save', 'resilient_fetch', 'memory_stats']
|
|
919
705
|
})
|
|
920
706
|
}
|
|
921
707
|
|
|
922
708
|
export default { name, inject, apply }
|
|
923
|
-
export { resolveStoreDir, loadStore, saveStore, alternatesFor, clip,
|
|
709
|
+
export { resolveStoreDir, loadStore, saveStore, alternatesFor, clip, tokenize, scoreFact, rankFacts }
|
package/package.json
CHANGED
|
@@ -1,51 +1,51 @@
|
|
|
1
|
-
{
|
|
2
|
-
"name": "dsh-limit-break",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "Durable cross-session memory
|
|
5
|
-
"type": "module",
|
|
6
|
-
"main": "lib/index.js",
|
|
7
|
-
"exports": {
|
|
8
|
-
".": "./lib/index.js",
|
|
9
|
-
"./package.json": "./package.json"
|
|
10
|
-
},
|
|
11
|
-
"files": [
|
|
12
|
-
"lib/",
|
|
13
|
-
"cordis.patch.yml",
|
|
14
|
-
"README.md",
|
|
15
|
-
"LICENSE",
|
|
16
|
-
"tools/",
|
|
17
|
-
"MAINTAIN.md",
|
|
18
|
-
"CHANGELOG.md"
|
|
19
|
-
],
|
|
20
|
-
"keywords": [
|
|
21
|
-
"deepseek",
|
|
22
|
-
"deepseek-harness",
|
|
23
|
-
"dsh",
|
|
24
|
-
"dsh-plugin",
|
|
25
|
-
"memory",
|
|
26
|
-
"long-term-memory",
|
|
27
|
-
"fetch",
|
|
28
|
-
"network"
|
|
29
|
-
],
|
|
30
|
-
"engines": {
|
|
31
|
-
"node": ">=22.6"
|
|
32
|
-
},
|
|
33
|
-
"license": "MIT",
|
|
34
|
-
"author": {
|
|
35
|
-
"name": "mapingyue",
|
|
36
|
-
"email": "2067984403@qq.com"
|
|
37
|
-
},
|
|
38
|
-
"repository": {
|
|
39
|
-
"type": "git",
|
|
40
|
-
"url": "git+https://github.com/Ma354/dsh-limit-break.git"
|
|
41
|
-
},
|
|
42
|
-
"homepage": "https://github.com/Ma354/dsh-limit-break#readme",
|
|
43
|
-
"bugs": {
|
|
44
|
-
"url": "https://github.com/Ma354/dsh-limit-break/issues"
|
|
45
|
-
},
|
|
46
|
-
"dsh": {
|
|
47
|
-
"bundle": {
|
|
48
|
-
"patch": "./cordis.patch.yml"
|
|
49
|
-
}
|
|
50
|
-
}
|
|
51
|
-
}
|
|
1
|
+
{
|
|
2
|
+
"name": "dsh-limit-break",
|
|
3
|
+
"version": "1.5.0",
|
|
4
|
+
"description": "Durable cross-session memory and a fetch ladder that survives hostname-keyed blocking — four small tools for DeepSeek Harness.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "lib/index.js",
|
|
7
|
+
"exports": {
|
|
8
|
+
".": "./lib/index.js",
|
|
9
|
+
"./package.json": "./package.json"
|
|
10
|
+
},
|
|
11
|
+
"files": [
|
|
12
|
+
"lib/",
|
|
13
|
+
"cordis.patch.yml",
|
|
14
|
+
"README.md",
|
|
15
|
+
"LICENSE",
|
|
16
|
+
"tools/",
|
|
17
|
+
"MAINTAIN.md",
|
|
18
|
+
"CHANGELOG.md"
|
|
19
|
+
],
|
|
20
|
+
"keywords": [
|
|
21
|
+
"deepseek",
|
|
22
|
+
"deepseek-harness",
|
|
23
|
+
"dsh",
|
|
24
|
+
"dsh-plugin",
|
|
25
|
+
"memory",
|
|
26
|
+
"long-term-memory",
|
|
27
|
+
"fetch",
|
|
28
|
+
"network"
|
|
29
|
+
],
|
|
30
|
+
"engines": {
|
|
31
|
+
"node": ">=22.6"
|
|
32
|
+
},
|
|
33
|
+
"license": "MIT",
|
|
34
|
+
"author": {
|
|
35
|
+
"name": "mapingyue",
|
|
36
|
+
"email": "2067984403@qq.com"
|
|
37
|
+
},
|
|
38
|
+
"repository": {
|
|
39
|
+
"type": "git",
|
|
40
|
+
"url": "git+https://github.com/Ma354/dsh-limit-break.git"
|
|
41
|
+
},
|
|
42
|
+
"homepage": "https://github.com/Ma354/dsh-limit-break#readme",
|
|
43
|
+
"bugs": {
|
|
44
|
+
"url": "https://github.com/Ma354/dsh-limit-break/issues"
|
|
45
|
+
},
|
|
46
|
+
"dsh": {
|
|
47
|
+
"bundle": {
|
|
48
|
+
"patch": "./cordis.patch.yml"
|
|
49
|
+
}
|
|
50
|
+
}
|
|
51
|
+
}
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Recall benchmark: how often does memory_recall's ranking put the intended fact first?
|
|
3
|
+
*
|
|
4
|
+
* It reads the live store (read-only) and runs a fixed query set whose expected ids were
|
|
5
|
+
* written by hand from the fact statements. Hit@1 is the number that matters: an agent asks
|
|
6
|
+
* one question and reads the top answer. Run: node tools/recall-bench.mjs [--json]
|
|
7
|
+
*/
|
|
8
|
+
import { readFile } from 'node:fs/promises'
|
|
9
|
+
import { join } from 'node:path'
|
|
10
|
+
import { homedir } from 'node:os'
|
|
11
|
+
import { rankFacts } from '../lib/index.js'
|
|
12
|
+
|
|
13
|
+
const STORE = join(homedir(), '.dsh', 'memory', 'facts.json')
|
|
14
|
+
|
|
15
|
+
/** query -> expected fact id. Phrased the way a later session actually asks. */
|
|
16
|
+
const CASES = [
|
|
17
|
+
['which CPU is in this machine', 'env.cpu'],
|
|
18
|
+
['how many cores does the processor have', 'env.cpu'],
|
|
19
|
+
['how much VRAM does the GPU have', 'env.gpu'],
|
|
20
|
+
['what python version is bundled', 'env.python'],
|
|
21
|
+
['where is cupy installed', 'gpu.cupy_installed'],
|
|
22
|
+
['what is the DSH home directory', 'env.dsh_home'],
|
|
23
|
+
['which node and npm versions are installed', 'env.node_npm'],
|
|
24
|
+
['monte carlo pi benchmark on the CPU', 'bench.cpu_suite'],
|
|
25
|
+
['how long did the ray tracer take at 1080p', 'bench.render'],
|
|
26
|
+
['GPU monte carlo speedup over CPU', 'bench.gpu_monte_carlo'],
|
|
27
|
+
['how long did the batch of four pipelines run', 'batch.summary'],
|
|
28
|
+
['why is wikipedia blocked by SNI', 'net.wikipedia_access'],
|
|
29
|
+
['is github push failing', 'network.github-git'],
|
|
30
|
+
['deck rating for physical world execution', 'deck.ratings'],
|
|
31
|
+
['how many slides in the round 2 deck', 'deck.round2'],
|
|
32
|
+
['which tools does the memory plugin provide', 'tool.limit_break_plugin'],
|
|
33
|
+
['weekly maintenance objective for the plugin', 'task.limit-break.maintenance'],
|
|
34
|
+
['what is the latest npm release', 'release.dsh-limit-break'],
|
|
35
|
+
['why was memory_prune removed', 'tool.memory_prune'],
|
|
36
|
+
['what renders the film with numpy', 'film.wotaiyang']
|
|
37
|
+
]
|
|
38
|
+
|
|
39
|
+
const raw = JSON.parse(await readFile(STORE, 'utf8'))
|
|
40
|
+
const facts = (raw.facts || []).filter(Boolean)
|
|
41
|
+
|
|
42
|
+
let hit1 = 0
|
|
43
|
+
let hit3 = 0
|
|
44
|
+
let rr = 0
|
|
45
|
+
const misses = []
|
|
46
|
+
for (const [query, want] of CASES) {
|
|
47
|
+
const ranked = rankFacts(facts, query).map((row) => row.fact.id)
|
|
48
|
+
const at = ranked.indexOf(want)
|
|
49
|
+
if (at === 0) hit1 += 1
|
|
50
|
+
if (at >= 0 && at < 3) hit3 += 1
|
|
51
|
+
if (at >= 0) rr += 1 / (at + 1)
|
|
52
|
+
if (at !== 0) misses.push({ query, want, at: at < 0 ? 'miss' : at + 1, top: ranked.slice(0, 3) })
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
const report = {
|
|
56
|
+
store: STORE,
|
|
57
|
+
facts: facts.length,
|
|
58
|
+
cases: CASES.length,
|
|
59
|
+
hit_at_1: +(hit1 / CASES.length).toFixed(3),
|
|
60
|
+
hit_at_3: +(hit3 / CASES.length).toFixed(3),
|
|
61
|
+
mrr: +(rr / CASES.length).toFixed(3),
|
|
62
|
+
misses
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
if (process.argv.includes('--json')) {
|
|
66
|
+
console.log(JSON.stringify(report, null, 2))
|
|
67
|
+
} else {
|
|
68
|
+
console.log(`facts=${report.facts} cases=${report.cases} hit@1=${report.hit_at_1} hit@3=${report.hit_at_3} mrr=${report.mrr}`)
|
|
69
|
+
for (const miss of report.misses) {
|
|
70
|
+
console.log(` miss: "${miss.query}" wanted ${miss.want} got rank ${miss.at} -> ${miss.top.join(', ')}`)
|
|
71
|
+
}
|
|
72
|
+
}
|
package/tools/prune-selftest.mjs
DELETED
|
@@ -1,70 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Self-test for memory_prune. Run: node tools/prune-selftest.mjs
|
|
3
|
-
*
|
|
4
|
-
* It builds a store with one of each redundancy shape, checks that the dry run changes
|
|
5
|
-
* nothing, applies, and checks that a second run is a no-op. Exit code 1 on any mismatch.
|
|
6
|
-
*/
|
|
7
|
-
import assert from 'node:assert/strict'
|
|
8
|
-
import { mkdtemp, readFile, writeFile, readdir } from 'node:fs/promises'
|
|
9
|
-
import { createHash } from 'node:crypto'
|
|
10
|
-
import { tmpdir } from 'node:os'
|
|
11
|
-
import { join } from 'node:path'
|
|
12
|
-
import { pruneStore } from '../lib/index.js'
|
|
13
|
-
|
|
14
|
-
const dir = await mkdtemp(join(tmpdir(), 'limit-break-prune-'))
|
|
15
|
-
const facts = [
|
|
16
|
-
{ id: 'bench.latency_a', statement: 'Wikipedia via SNI api.wikimedia.org answers in 0.43 s', value: 0.43, unit: 's', topic: 'network' },
|
|
17
|
-
{ id: 'bench.latency_b', statement: 'Wikipedia via SNI api.wikimedia.org answered in 0.43 s', value: 0.43, unit: 's', topic: 'network' },
|
|
18
|
-
{ id: 'env.approval', statement: 'The approval policy changed in this session to never', topic: 'env' },
|
|
19
|
-
{ id: 'dup.a', statement: 'GIL means threads do not help CPU-bound Python', tags: ['python'] },
|
|
20
|
-
{ id: 'dup.b', statement: 'GIL means threads do not help CPU-bound Python', value: 1, details: { who: 'dynamodb' }, source: 'bench' },
|
|
21
|
-
{ id: 'net.pos', statement: 'Wikipedia is reachable from this host via SNI api.wikimedia.org', topic: 'network' },
|
|
22
|
-
{ id: 'net.neg', statement: 'Wikipedia is unreachable from this host', topic: 'network' },
|
|
23
|
-
{ id: 'stable', statement: 'crypto dyn32_b pow-32 measures 0.051 ms/sign', value: 0.051, unit: 'ms/sign', topic: 'crypto' }
|
|
24
|
-
]
|
|
25
|
-
|
|
26
|
-
await writeFile(
|
|
27
|
-
join(dir, 'facts.json'),
|
|
28
|
-
JSON.stringify({ schema_version: 1, kind: 'agent-memory-store', facts }, null, 2),
|
|
29
|
-
'utf8'
|
|
30
|
-
)
|
|
31
|
-
const storeFile = join(dir, 'facts.json')
|
|
32
|
-
const hash = async () => createHash('sha256').update(await readFile(storeFile)).digest('hex')
|
|
33
|
-
|
|
34
|
-
const before = await hash()
|
|
35
|
-
const plan = await pruneStore(dir, {})
|
|
36
|
-
assert.equal(plan.ok, true)
|
|
37
|
-
assert.equal(plan.mode, 'plan')
|
|
38
|
-
assert.equal(plan.changed, false)
|
|
39
|
-
assert.deepEqual(plan.candidates.map((c) => c.id).sort(), ['dup.a', 'env.approval'])
|
|
40
|
-
assert.equal(plan.warnings.length >= 2, true)
|
|
41
|
-
assert.equal(plan.warnings.some((w) => w.rule === 'shared-numbers'), true)
|
|
42
|
-
assert.equal(plan.warnings.some((w) => w.rule === 'contradiction'), true)
|
|
43
|
-
assert.equal(await hash(), before, 'dry run must not touch facts.json')
|
|
44
|
-
|
|
45
|
-
const applied = await pruneStore(dir, { apply: true })
|
|
46
|
-
assert.equal(applied.mode, 'applied')
|
|
47
|
-
assert.equal(applied.changed, true)
|
|
48
|
-
assert.equal(applied.archived, 2)
|
|
49
|
-
assert.equal(applied.kept, 6)
|
|
50
|
-
assert.equal(applied.backupFile.includes('facts.json.bak-'), true)
|
|
51
|
-
|
|
52
|
-
const kept = JSON.parse(await readFile(storeFile, 'utf8'))
|
|
53
|
-
assert.equal(kept.facts.length, 6)
|
|
54
|
-
assert.equal(kept.facts.some((f) => f.id === 'env.approval' || f.id === 'dup.a'), false)
|
|
55
|
-
assert.equal(kept.facts.some((f) => f.id === 'dup.b'), true, 'the richer duplicate must survive')
|
|
56
|
-
|
|
57
|
-
const archive = JSON.parse(await readFile(applied.archiveFile, 'utf8'))
|
|
58
|
-
assert.equal(archive.length, 2)
|
|
59
|
-
assert.equal(archive.every((e) => e.archivedAt && e.rule && e.fact), true)
|
|
60
|
-
|
|
61
|
-
const again = await pruneStore(dir, { apply: true })
|
|
62
|
-
assert.equal(again.archived, 0)
|
|
63
|
-
assert.equal(again.changed, false)
|
|
64
|
-
const archiveAgain = JSON.parse(await readFile(applied.archiveFile, 'utf8'))
|
|
65
|
-
assert.equal(archiveAgain.length, 2, 'second run must not grow the archive')
|
|
66
|
-
|
|
67
|
-
const leftovers = (await readdir(dir)).filter((name) => name.endsWith('.tmp'))
|
|
68
|
-
assert.deepEqual(leftovers, [], 'no temp files left behind')
|
|
69
|
-
|
|
70
|
-
console.log('prune selftest: ok')
|