dsh-limit-break 1.4.0 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,47 @@ All notable changes to `dsh-limit-break`. Format follows
5
5
 
6
6
  ## Unreleased
7
7
 
8
+ ## 1.6.0 — 2026-10-29
9
+
10
+ ### Fixed
11
+ - **`memory_save` no longer loses writes when two sessions save at the same time.** Saving was
12
+ load → mutate → write-temp → rename with nothing serialising it, so concurrent writers each
13
+ wrote a store missing the other's fact, and on Windows the losing `rename` often failed outright
14
+ with `EPERM`. Saves now take a lock file created with `O_EXCL` around the whole read-modify-write,
15
+ released in a `finally`; a lock left behind by a dead process is stolen after 15 s, and a writer
16
+ that cannot get in within 8 s returns `{ ok: false, code: 'memory.locked' }` instead of throwing.
17
+ A failed `stat` is never read as "stale" — that is exactly how two writers end up inside at once.
18
+ - **`tools/concurrency-selftest.mjs`** (new): spawns N worker processes that each save M facts,
19
+ reads the store back, and counts what survived; it also checks stale-lock recovery.
20
+ 8 workers × 6 saves × 3 rounds on the same machine: **16–17 of 48 saves lost** and **up to 25 of
21
+ 48 saves failed with `EPERM`** before; **0 lost, 0 failed** after (also clean at 12 × 5 × 4 = 240
22
+ saves, and a 60 s-old lock is recovered with 6/6 saves kept).
23
+
24
+ ### Notes
25
+ - No tool added, removed or renamed: the four-tool ABI is untouched. Store format unchanged, and
26
+ single-writer behaviour is identical (the lock is uncontended: one `open` + one `rm` per save).
27
+ - Read paths (`memory_recall`, `memory_stats`) do not take the lock.
28
+
29
+ ## 1.5.0 — 2026-10-22
30
+
31
+ ### Changed
32
+ - **`memory_recall` looks where you point it.** The scorer used raw substring counts over five
33
+ lowercased blobs, which let common words outvote the one term that identifies a fact ("the",
34
+ "how", "long"). It now drops English stopwords, stems (`slides`/`slide`, `rendering`/`render`),
35
+ matches whole words with a prefix fallback (`ray` → `raytracer`), splits dotted ids into segments
36
+ (`bench.gpu_monte_carlo` → `bench`/`gpu`/`monte`/`carlo`), weights each query token by idf
37
+ against the live store, and adds a bonus for a whole-phrase hit in the statement.
38
+ - **`tools/recall-bench.mjs`** (new): 20 hand-labelled questions → hit@1, hit@3, MRR, misses
39
+ printed. Same 20-fact store, before and after: **hit@1 0.75 → 0.95**, MRR **0.867 → 0.975**,
40
+ hit@3 1.0 → 1.0. The remaining miss ("monte carlo pi benchmark on the CPU") is ambiguous
41
+ between `bench.cpu_suite` and `bench.gpu_monte_carlo` and ranks second.
42
+
43
+ ### Notes
44
+ - No tool added, removed or renamed: the four-tool ABI is untouched (see 1.4.0). `tokenize`,
45
+ `scoreFact` and `rankFacts` are newly exported from `lib/index.js` only so the benchmark measures
46
+ the shipped scorer instead of a copy of it.
47
+ - Store format unchanged; 1.2.x–1.4.0 stores load as-is.
48
+
8
49
  ## 1.4.0 — 2026-10-22
9
50
 
10
51
  ### Removed
package/MAINTAIN.md CHANGED
@@ -108,21 +108,39 @@ git add -A && git commit -m "..." && git push
108
108
 
109
109
  ---
110
110
 
111
- ## Backlog(按 2026-10-20 20:06 的真实状态)
111
+ ## Backlog(按 2026-10-22 20:10 的真实状态)
112
+
113
+ - [x] **1.5.0 已上线**(2026-10-22T12:20:22Z):npm `latest` = `1.5.0`(用户 OTP 发布),GitHub
114
+ `origin/main` = `cd5f409`(github.com:443 被掐,改用 api.github.com + GraphQL 推送),
115
+ desktop profile 已从 1.2.1 升到 **1.5.0**,`verify` PASS。四工具 ABI 未变,老会话不会 brick。
116
+ - [ ] 基准里剩下的 miss(“monte carlo pi benchmark on the CPU”)是 `bench.cpu_suite` 与
117
+ `bench.gpu_monte_carlo` 的真实歧义;再往上做要引入上下文/近义词表,性价比低,当作已知上限。
118
+ - [x] desktop profile 版本漂移已消(1.2.1 → 1.5.0)。以后每次发布都跟一条
119
+ `dsh.cmd plugin --profile desktop add dsh-limit-break@<version>`。
112
120
 
113
121
  - [x] **1.2.0 已上线**(2026-10-20T12:02:52Z):npm `latest` = `1.2.0`,GitHub `origin/main` =
114
122
  `fd844a4`,`verify` PASS,`test_plugin.mjs` 6/6 PASS。用户 20:04 完成了 OTP 发布和推送。
115
123
  `env.node_npm` 与 `release.dsh-limit-break` 两条事实已回写。
116
124
  - [x] 旧定时任务 `schedule-803cdbdc`(会话 5133bee1)已由用户在面板删除;`storages/schedule.json`
117
125
  里只剩 `schedule-0b6c60ea`(hello world)和新的 `schedule-b3675d28`。无需再处理。
118
- - [ ] 市场条目 PR `awesome-dsh-plugin/awesome-dsh-plugin#8296` 仍是 open;合并前 `status` 会一直报
119
- `market listing not live yet`。找合并节奏,不要重提。
126
+ - [x] 市场条目**已合并上线**(2026-10-22 之后):`awesome-dsh-plugin/awesome-dsh-plugin#8296`
127
+ merged,条目文件 `data/plugins/Ma354__dsh-limit-break.yml`(category: memory);
128
+ 2026-10-27 01:06 的 `#9282`(加 storefront icon,`assets/icon-180.png`)也已 merged。
129
+ `maintain.py status` 现在 `listed_in_market: true`、`problems: []`。不要再提新 PR。
130
+ - [ ] 本地仓库落后远端:本地 `94eafb7`,远端 `b84161a`(用户 10-25 推了 icon 等);
131
+ `github.com:443` 被掐,`git fetch` 连试 3 次全失败。等网络恢复后跑
132
+ `git fetch origin && git reset --hard origin/main`。在此之前文档改动走
133
+ `..\_maintain\ghpush.mjs`(GraphQL,按远端 head 建提交,内容全量替换,不受分叉影响)。
120
134
  - [ ] npm 上还挂着占位包 `0.0.0-stage`(`stub: true`)。`npm deprecate` 也要 OTP(本轮 EOTP 失败);
121
135
  用户一条命令即可:`npm deprecate dsh-limit-break@0.0.0-stage "placeholder - use 1.2.0"`。
122
136
  - [ ] `1.0.0` 是无元数据的旧包体(author/repository 都不在)。同样一条 `npm deprecate` 可清掉噪音(需 OTP)。
123
- - [ ] `facts.json` 里 `env.gpu` 仍写着"no GPU compute benchmark was run"——已过时(GPU 实测 54.3×)。
137
+ - [x] `facts.json` 的 `env.gpu` 已改写成“GPU compute throughput is recorded separately in
138
+ bench.gpu_monte_carlo and bench.render”(2026-10-21 的回写),陈旧描述不再存在。
124
139
  - [x] 插件工具对**已经打开的会话**不可见:已在 README「Known limits」第 1 条写明
125
140
  (tool list 是 per-session 快照,插件侧修不了)。事实 `harness.plugin-tool-visibility` 有实测。
126
141
  - [ ] `resilient_fetch` 目前按"主机名别名"降级,可以继续加:`--resolve` 到已知 IP、
127
142
  IPv6 优先、以及把成功路径写回记忆库(现在是只读阶梯)。
128
- - [ ] 没有任何自动化测试覆盖 `memory_save` 的并发写。单进程够用,多进程未验证。
143
+ - [x] **`memory_save` 的并发写已有自动化覆盖**(1.6.0,2026-10-29):`tools/concurrency-selftest.mjs`
144
+ 起 N 个独立进程同时写同一个 store,再读回数存活条数,另测“陈旧锁被接管”。同机 8 进程 × 6 次 × 3 轮:
145
+ 修复前丢 16–17/48、且最多 25/48 直接 `EPERM`(rename 输家);加 `O_EXCL` 锁后 0 丢 0 失败,
146
+ 12 × 5 × 4 = 240 次同样全存活。改了 `memory_save` 的写入路径就跑一遍它(约 20 s)。
package/README.md CHANGED
@@ -13,6 +13,14 @@ self-benchmark: two things an agent is normally told it cannot do, made cheap en
13
13
  Zero npm dependencies. No import-time side effects. Everything fails soft — a broken store or a dead
14
14
  network returns an error object, never an exception that could take the host down.
15
15
 
16
+ **Recall quality.** `memory_recall` is measured, not vibed: `tools/recall-bench.mjs` runs 20
17
+ hand-labelled questions against the live store and prints hit@1 / hit@3 / MRR. At 1.5.0 the ranking
18
+ skips English stopwords, stems, matches whole words (with a prefix fallback, so `ray` finds
19
+ `raytracer`), splits dotted ids into segments, weights each query token by inverse document
20
+ frequency, and rewards a whole-phrase hit: **hit@1 0.75 → 0.95, MRR 0.867 → 0.975** on the same 20
21
+ questions and the same 20 facts. The one remaining miss is a genuinely ambiguous question that
22
+ matches two benchmark facts.
23
+
16
24
  **Runtime requirement: `curl`.** Rung 1 of the fetch ladder is plain `fetch` and needs nothing; rungs
17
25
  2–4 (IPv6 → IPv4 → sibling hostname) shell out to `curl`, so it has to be on `PATH`. Windows 10+,
18
26
  macOS and mainstream Linux all ship it. Without it the ladder degrades to rung 1 and says so in the
@@ -67,7 +75,10 @@ package the profile does not declare is silently ignored.
67
75
  ```
68
76
 
69
77
  Plain JSON on purpose: anything that can read a file can use it, with or without this plugin.
70
- Writes are tmp-file + rename.
78
+ Writes are tmp-file + rename, and a save takes a `.facts.lock` (created `O_EXCL`, stolen after
79
+ 15 s if its writer died) around the whole read-modify-write, so two sessions saving at the same
80
+ moment both keep their facts. `tools/concurrency-selftest.mjs` states the numbers: 16–17 of 48
81
+ saves lost before the lock, 0 after.
71
82
 
72
83
  ## Disable / remove
73
84
 
@@ -82,14 +93,16 @@ Measured, not guessed — each line is something a session actually hit.
82
93
  * **Tools are visible only to sessions opened after the mount.** A session created before the
83
94
  plugin mounted keeps its tool snapshot; new tools do not appear until a new session. Not fixable
84
95
  from the plugin side (the tool list is materialised per session).
85
- * **One writer per store.** Writes are tmp-file + rename, which is crash-safe, but two processes
86
- writing `facts.json` concurrently still lose one update — there is no lock. One agent per
87
- machine is the intended use.
96
+ * **Concurrent saves serialise, and wait at most 8 s.** A second session saving at the same moment
97
+ waits for the lock instead of losing its fact; if the holder is wedged, the save returns
98
+ `{ ok: false, code: "memory.locked" }` rather than hanging. Read paths take no lock, so a save
99
+ still cannot stop a reader from seeing the previous snapshot.
88
100
  * **The fetch ladder is hostname-level.** It does not defeat IP-level blocks, TLS interception,
89
101
  or a proxy that terminates the connection; it does not skip certificate verification.
90
- * **`memory_recall` ranks by keyword overlap only** (id/topic/tags/statement), no embeddings, no
91
- stemming above the tokenisation in `tokenize()`. A query that shares no token with a fact will
92
- miss it.
102
+ * **`memory_recall` ranks by keyword overlap only** (id/topic/tags/statement/details), no
103
+ embeddings: stopword-filtered, crudely stemmed word matching weighted by idf, so it is good at
104
+ "which fact did I mean" and bad at paraphrase. `tools/recall-bench.mjs` states the current
105
+ numbers (hit@1 0.95 on 20 questions); a query that shares no word with a fact still misses it.
93
106
  * **`resilient_fetch` reads at most 2 MiB per response** and returns at most 20 000 characters;
94
107
  bigger payloads are truncated, not streamed.
95
108
 
package/lib/index.js CHANGED
@@ -18,7 +18,7 @@
18
18
 
19
19
  import { execFile } from 'node:child_process'
20
20
  import { appendFileSync, mkdirSync, readFileSync, statSync, writeFileSync } from 'node:fs'
21
- import { mkdir, readFile, rename, stat, writeFile } from 'node:fs/promises'
21
+ import { mkdir, open, readFile, rename, rm, stat, writeFile } from 'node:fs/promises'
22
22
  import { homedir } from 'node:os'
23
23
  import { dirname, isAbsolute, join, resolve } from 'node:path'
24
24
  import { fileURLToPath } from 'node:url'
@@ -182,31 +182,156 @@ async function saveStore(dir, store) {
182
182
  return { file, bytes: Buffer.byteLength(body) }
183
183
  }
184
184
 
185
+ // memory_save is load -> mutate -> save, and every DSH window is its own process, so two sessions
186
+ // that save at the same moment both write a store that is missing the other's fact (and on Windows
187
+ // the loser's rename can fail outright with EPERM). A lock file created with O_EXCL is enough:
188
+ // writes are tiny and rare, and a crashed writer is recovered by stealing a lock older than
189
+ // LOCK_STALE_MS. A failed stat is treated as "still held", never as "stale" — stealing on a
190
+ // transient EPERM is exactly how two writers end up in the critical section at once.
191
+ const LOCK_STALE_MS = 15000
192
+ const LOCK_WAIT_MS = 8000
193
+
194
+ async function withStoreLock(dir, fn) {
195
+ const lock = join(dir, '.facts.lock')
196
+ await mkdir(dir, { recursive: true })
197
+ const started = Date.now()
198
+ for (;;) {
199
+ let handle = null
200
+ try {
201
+ handle = await open(lock, 'wx')
202
+ await handle.writeFile(`${process.pid} ${new Date().toISOString()}\n`)
203
+ await handle.close()
204
+ break
205
+ } catch (error) {
206
+ if (handle) await handle.close().catch(() => {})
207
+ // On Windows a create that races a delete comes back as EPERM, not EEXIST, so both mean
208
+ // "somebody else is inside the critical section, or just left it".
209
+ if (!error || (error.code !== 'EEXIST' && error.code !== 'EPERM')) throw error
210
+ const info = await stat(lock).catch((problem) => problem)
211
+ if (!info || info.code) {
212
+ // The lock vanished or is pending deletion: that is a released lock, retry promptly.
213
+ await new Promise((resolve) => setTimeout(resolve, 5))
214
+ continue
215
+ }
216
+ const age = typeof info.mtimeMs === 'number' ? Date.now() - info.mtimeMs : 0
217
+ if (age > LOCK_STALE_MS) {
218
+ await rm(lock, { force: true }).catch(() => {})
219
+ continue
220
+ }
221
+ if (Date.now() - started > LOCK_WAIT_MS) {
222
+ const err = new Error(`memory store is locked by another process (${lock}); gave up after ${LOCK_WAIT_MS} ms`)
223
+ err.code = 'memory.locked'
224
+ throw err
225
+ }
226
+ await new Promise((resolve) => setTimeout(resolve, 10 + Math.random() * 30))
227
+ }
228
+ }
229
+ try {
230
+ return await fn()
231
+ } finally {
232
+ await rm(lock, { force: true }).catch(() => {})
233
+ }
234
+ }
235
+
236
+ // Ranking inputs. The old scorer used raw substring counts, which let a common word
237
+ // ("the", "how", "long") outvote the one rare word that identifies the fact. Words are
238
+ // matched against word sets instead, suffixed words share a stem, and each query token is
239
+ // weighted by how rare it is in the store (idf), so a distinctive term wins a tie.
240
+ const STOPWORDS = new Set([
241
+ 'a', 'about', 'all', 'also', 'an', 'and', 'any', 'are', 'as', 'at', 'be', 'been', 'but', 'by',
242
+ 'can', 'could', 'did', 'do', 'does', 'each', 'few', 'for', 'from', 'had', 'has', 'have', 'here',
243
+ 'how', 'if', 'in', 'into', 'is', 'it', 'its', 'just', 'many', 'me', 'more', 'most', 'much', 'my',
244
+ 'no', 'not', 'of', 'on', 'once', 'only', 'or', 'other', 'our', 'out', 'over', 'should', 'some',
245
+ 'such', 'than', 'that', 'the', 'their', 'then', 'there', 'these', 'they', 'this', 'those', 'to',
246
+ 'too', 'under', 'up', 'very', 'was', 'we', 'were', 'what', 'when', 'where', 'which', 'who', 'why',
247
+ 'will', 'with', 'would', 'you', 'your'
248
+ ])
249
+
250
+ function stem(word) {
251
+ let w = word
252
+ if (w.length > 5 && w.endsWith('ies')) return `${w.slice(0, -3)}y`
253
+ if (w.length > 4 && w.endsWith('es')) w = w.slice(0, -2)
254
+ else if (w.length > 3 && w.endsWith('s') && !w.endsWith('ss')) w = w.slice(0, -1)
255
+ if (w.length > 5 && w.endsWith('ing')) w = w.slice(0, -3)
256
+ else if (w.length > 4 && w.endsWith('ed')) w = w.slice(0, -2)
257
+ if (w.length > 5 && w.endsWith('ly')) w = w.slice(0, -2)
258
+ return w
259
+ }
260
+
185
261
  function tokenize(query) {
186
262
  return String(query || '')
187
263
  .toLowerCase()
264
+ // "memory_prune" should match the id segment "memory_prune" as well as "memory"/"prune".
265
+ .replace(/_+/g, ' ')
188
266
  .split(/[^\p{L}\p{N}_.+-]+/u)
189
- .filter((part) => part.length > 1)
267
+ .filter((part) => part.length > 1 && !STOPWORDS.has(part))
268
+ .map(stem)
190
269
  }
191
270
 
192
- function scoreFact(fact, tokens) {
271
+ function words(text) {
272
+ return String(text || '')
273
+ .toLowerCase()
274
+ .split(/[^\p{L}\p{N}_.+-]+/u)
275
+ .filter(Boolean)
276
+ .map(stem)
277
+ }
278
+
279
+ function wordSet(text) {
280
+ return new Set(words(text))
281
+ }
282
+
283
+ /** 1 for the exact word, 0.5 when the query token is a prefix of a stored word ("ray" ~ "raytracer"). */
284
+ function wordMatch(set, token) {
285
+ if (set.has(token)) return 1
286
+ for (const word of set) {
287
+ if (word.length > token.length && word.startsWith(token)) return 0.5
288
+ }
289
+ return 0
290
+ }
291
+
292
+ const FIELD_WEIGHTS = { id: 7, topic: 2, tags: 3, statement: 3, details: 1 }
293
+
294
+ function scoreFact(fact, tokens, idf) {
193
295
  if (!tokens.length) return 0
194
- const id = String(fact.id || '').toLowerCase()
195
- const statement = String(fact.statement || '').toLowerCase()
196
- const topic = String(fact.topic || '').toLowerCase()
197
- const tags = (Array.isArray(fact.tags) ? fact.tags : []).join(' ').toLowerCase()
198
- const details = fact.details ? JSON.stringify(fact.details).toLowerCase() : ''
296
+ // Ids are dotted paths ("bench.gpu_monte_carlo"); split them so a token can match a segment.
297
+ const id = new Set(String(fact.id || '').toLowerCase().split(/[^\p{L}\p{N}]+/u).filter(Boolean).map(stem))
298
+ const topic = wordSet(fact.topic)
299
+ const tags = wordSet((Array.isArray(fact.tags) ? fact.tags : []).join(' '))
300
+ const statement = wordSet(fact.statement)
301
+ const details = wordSet(fact.details ? JSON.stringify(fact.details) : '')
199
302
  let score = 0
200
303
  for (const token of tokens) {
201
- if (id.includes(token)) score += 5
202
- if (topic.includes(token)) score += 2
203
- if (tags.includes(token)) score += 2
204
- if (statement.includes(token)) score += 3
205
- if (details.includes(token)) score += 1
304
+ const boost = idf instanceof Map ? idf.get(token) || 1 : 1
305
+ score += boost * (
306
+ FIELD_WEIGHTS.id * wordMatch(id, token) +
307
+ FIELD_WEIGHTS.topic * wordMatch(topic, token) +
308
+ FIELD_WEIGHTS.tags * wordMatch(tags, token) +
309
+ FIELD_WEIGHTS.statement * wordMatch(statement, token) +
310
+ FIELD_WEIGHTS.details * wordMatch(details, token)
311
+ )
206
312
  }
313
+ const phrase = tokens.join(' ')
314
+ if (tokens.length > 1 && String(fact.statement || '').toLowerCase().includes(phrase)) score += 6
207
315
  return score
208
316
  }
209
317
 
318
+ /** Scored ordering used by memory_recall; exported so the recall benchmark can measure it. */
319
+ function rankFacts(facts, query) {
320
+ const tokens = tokenize(query)
321
+ const living = (facts || []).filter(Boolean)
322
+ if (!tokens.length) return living.map((fact) => ({ fact, score: 0 }))
323
+ const idf = new Map()
324
+ for (const token of tokens) {
325
+ let df = 0
326
+ for (const fact of living) if (scoreFact(fact, [token]) > 0) df += 1
327
+ idf.set(token, +(1 + Math.log((living.length + 1) / (df + 1))).toFixed(4))
328
+ }
329
+ return living
330
+ .map((fact) => ({ fact, score: scoreFact(fact, tokens, idf) }))
331
+ .filter((row) => row.score > 0)
332
+ .sort((a, b) => b.score - a.score)
333
+ }
334
+
210
335
  function shapeFact(fact) {
211
336
  return {
212
337
  id: fact.id,
@@ -426,11 +551,7 @@ export function apply(ctx, config) {
426
551
  const tokens = tokenize(input.query)
427
552
  let picked = store.facts.filter(Boolean)
428
553
  if (!input.all && tokens.length) {
429
- picked = picked
430
- .map((fact) => ({ fact, score: scoreFact(fact, tokens) }))
431
- .filter((row) => row.score > 0)
432
- .sort((a, b) => b.score - a.score)
433
- .map((row) => row.fact)
554
+ picked = rankFacts(picked, input.query).map((row) => row.fact)
434
555
  }
435
556
  // Number(input.limit) || 8 turned 0/NaN into 8 but let Infinity through to slice()
436
557
  // (which returns everything) and gave no upper bound below the cap.
@@ -495,25 +616,34 @@ export function apply(ctx, config) {
495
616
  if (id.length > MAX_ID_CHARS || statement.length > MAX_TEXT) {
496
617
  return { ok: false, code: 'memory.invalid', message: `id must be <= ${MAX_ID_CHARS} chars and statement <= ${MAX_TEXT} chars.` }
497
618
  }
498
- const store = await loadStore(storeDir)
499
- const now = new Date().toISOString()
500
- const record = {
501
- id,
502
- topic: String(id).includes('.') ? String(id).split('.')[0] : 'general',
503
- statement,
504
- value: input.value === undefined ? null : input.value,
505
- unit: input.unit === undefined ? null : input.unit,
506
- details: input.details && typeof input.details === 'object' ? input.details : null,
507
- source: input.source ? String(input.source) : null,
508
- timestamp: now,
509
- tags: Array.isArray(input.tags) ? input.tags.map(String) : []
619
+ try {
620
+ return await withStoreLock(storeDir, async () => {
621
+ const store = await loadStore(storeDir)
622
+ const now = new Date().toISOString()
623
+ const record = {
624
+ id,
625
+ topic: String(id).includes('.') ? String(id).split('.')[0] : 'general',
626
+ statement,
627
+ value: input.value === undefined ? null : input.value,
628
+ unit: input.unit === undefined ? null : input.unit,
629
+ details: input.details && typeof input.details === 'object' ? input.details : null,
630
+ source: input.source ? String(input.source) : null,
631
+ timestamp: now,
632
+ tags: Array.isArray(input.tags) ? input.tags.map(String) : []
633
+ }
634
+ const at = store.facts.findIndex((f) => f && String(f.id) === id)
635
+ const action = at >= 0 ? 'updated' : 'created'
636
+ if (at >= 0) store.facts[at] = record
637
+ else store.facts.push(record)
638
+ const written = await saveStore(storeDir, store)
639
+ return { ok: true, id, action, total: store.facts.length, bytes: written.bytes, file: written.file }
640
+ })
641
+ } catch (error) {
642
+ if (error && error.code === 'memory.locked') {
643
+ return { ok: false, code: 'memory.locked', message: error.message }
644
+ }
645
+ throw error
510
646
  }
511
- const at = store.facts.findIndex((f) => f && String(f.id) === id)
512
- const action = at >= 0 ? 'updated' : 'created'
513
- if (at >= 0) store.facts[at] = record
514
- else store.facts.push(record)
515
- const written = await saveStore(storeDir, store)
516
- return { ok: true, id, action, total: store.facts.length, bytes: written.bytes, file: written.file }
517
647
  }
518
648
  }),
519
649
  'limit-break: memory_save tool'
@@ -636,4 +766,4 @@ export function apply(ctx, config) {
636
766
  }
637
767
 
638
768
  export default { name, inject, apply }
639
- export { resolveStoreDir, loadStore, saveStore, alternatesFor, clip }
769
+ export { resolveStoreDir, loadStore, saveStore, alternatesFor, clip, tokenize, scoreFact, rankFacts }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-limit-break",
3
- "version": "1.4.0",
3
+ "version": "1.6.0",
4
4
  "description": "Durable cross-session memory and a fetch ladder that survives hostname-keyed blocking — four small tools for DeepSeek Harness.",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",
@@ -0,0 +1,112 @@
1
+ /**
2
+ * Concurrency probe: do simultaneous memory_save calls lose facts?
3
+ *
4
+ * memory_save is load -> mutate -> save (tmp file + rename). Without a lock, two processes that
5
+ * load the same store each write their own version back and the later rename wins, so the other
6
+ * process' fact disappears. On Windows the losing writer can even fail the rename outright
7
+ * (EPERM: the target is open in the other process).
8
+ *
9
+ * This test makes the damage visible in plain numbers: N worker processes each attempt M saves,
10
+ * then the store is read back and the surviving ids are counted.
11
+ *
12
+ * Run: node tools/concurrency-selftest.mjs [workers] [saves-per-worker] [rounds]
13
+ */
14
+ import { execFile } from 'node:child_process'
15
+ import { mkdir, mkdtemp, readdir, readFile, rm, utimes, writeFile } from 'node:fs/promises'
16
+ import { tmpdir } from 'node:os'
17
+ import { join } from 'node:path'
18
+ import { fileURLToPath } from 'node:url'
19
+ import { promisify } from 'node:util'
20
+
21
+ const exec = promisify(execFile)
22
+ const LIB = fileURLToPath(new URL('../lib/index.js', import.meta.url))
23
+ const workers = Number(process.argv[2] || 8)
24
+ const perWorker = Number(process.argv[3] || 6)
25
+ const rounds = Number(process.argv[4] || 3)
26
+
27
+ const worker = `
28
+ import(process.env.LIB).then(async (m) => {
29
+ const tools = new Map()
30
+ const ctx = {
31
+ effect(fn) { fn() },
32
+ tools: { register(d) { tools.set(d.name, d); return () => tools.delete(d.name) } }
33
+ }
34
+ m.apply(ctx, { storeDir: process.env.STORE_DIR })
35
+ const save = tools.get('memory_save')
36
+ const w = Number(process.env.WORKER)
37
+ let ok = 0
38
+ const errors = []
39
+ for (let i = 0; i < ${perWorker}; i += 1) {
40
+ try {
41
+ const res = await save.execute({ id: 'race.w' + w + '_' + i, statement: 'concurrent write w' + w + ' i' + i + ' from a separate process' })
42
+ if (res && res.ok) ok += 1
43
+ else errors.push((res && res.code ? res.code : 'not-ok') + ' ' + JSON.stringify(res))
44
+ } catch (e) {
45
+ const detail = [e && e.code, e && e.syscall, e && e.path, e && e.message].filter(Boolean).join(' | ')
46
+ errors.push(detail || String(e))
47
+ }
48
+ }
49
+ console.log('RESULT ' + JSON.stringify({ worker: w, ok, errors }))
50
+ process.exit(0)
51
+ }).catch((e) => { console.error(e); process.exit(1) })
52
+ `
53
+
54
+ const results = []
55
+ for (let round = 1; round <= rounds; round += 1) {
56
+ const dir = await mkdtemp(join(tmpdir(), 'limit-break-race-'))
57
+ const storeDir = join(dir, 'memory')
58
+ const replies = await Promise.all(
59
+ Array.from({ length: workers }, (_, w) =>
60
+ exec(process.execPath, ['--input-type=module', '-e', worker], {
61
+ env: { ...process.env, LIB: `file:///${LIB.replaceAll('\\', '/')}`, STORE_DIR: storeDir, WORKER: String(w) },
62
+ timeout: 120000
63
+ }).then((r) => r.stdout)
64
+ )
65
+ )
66
+ const reports = replies.flatMap((out) =>
67
+ out.split('\n').filter((line) => line.startsWith('RESULT ')).map((line) => JSON.parse(line.slice(7)))
68
+ )
69
+ const succeeded = reports.reduce((sum, r) => sum + r.ok, 0)
70
+ const errorCodes = reports.flatMap((r) => r.errors)
71
+ const store = JSON.parse(await readFile(join(storeDir, 'facts.json'), 'utf8'))
72
+ const kept = store.facts.filter((f) => f && /^race\.w/.test(String(f.id))).length
73
+ const leftover = (await readdir(storeDir)).filter((name) => name !== 'facts.json' && name !== 'plugin-load.log')
74
+ results.push({ round, attempted: workers * perWorker, succeeded, kept, lost: succeeded - kept, failed: workers * perWorker - succeeded, errorCodes, leftover })
75
+ await rm(dir, { recursive: true, force: true })
76
+ }
77
+
78
+ for (const r of results) {
79
+ const codes = r.errorCodes.length ? ` (errors: ${[...new Set(r.errorCodes)].join(',')})` : ''
80
+ console.log(`round ${r.round}: attempted ${r.attempted}, save-ok ${r.succeeded}, kept ${r.kept}, lost ${r.lost}${codes}${r.leftover.length ? ` leftover: ${r.leftover.join(',')}` : ''}`)
81
+ }
82
+ // A writer that dies while holding the lock must not wedge the store forever: a lock older than
83
+ // the stale window is stolen, and the save goes through.
84
+ const staleDir = await mkdtemp(join(tmpdir(), 'limit-break-stale-'))
85
+ const staleStore = join(staleDir, 'memory')
86
+ await mkdir(staleStore, { recursive: true })
87
+ await writeFile(join(staleStore, '.facts.lock'), '999999 2020-01-01T00:00:00.000Z\n')
88
+ const past = new Date(Date.now() - 60000)
89
+ await utimes(join(staleStore, '.facts.lock'), past, past)
90
+ const staleOut = await exec(process.execPath, ['--input-type=module', '-e', worker], {
91
+ env: { ...process.env, LIB: `file:///${LIB.replaceAll('\\', '/')}`, STORE_DIR: staleStore, WORKER: '99' },
92
+ timeout: 120000
93
+ })
94
+ const stale = JSON.parse(staleOut.stdout.split('\n').filter((line) => line.startsWith('RESULT ')).map((line) => line.slice(7))[0])
95
+ const staleStoreJson = JSON.parse(await readFile(join(staleStore, 'facts.json'), 'utf8'))
96
+ const staleKept = staleStoreJson.facts.filter((f) => f && /^race\.w99/.test(String(f.id))).length
97
+ const staleLeftover = (await readdir(staleStore)).filter((name) => name !== 'facts.json' && name !== 'plugin-load.log')
98
+ await rm(staleDir, { recursive: true, force: true })
99
+ console.log(`stale lock: save-ok ${stale.ok}/${perWorker}, kept ${staleKept}, leftover ${staleLeftover.length ? staleLeftover.join(',') : 'none'}`)
100
+
101
+ const lost = results.map((r) => r.lost)
102
+ const failed = results.map((r) => r.failed)
103
+ const worstLost = Math.max(...lost)
104
+ const avgLost = +(lost.reduce((a, b) => a + b, 0) / lost.length).toFixed(1)
105
+ const worstFailed = Math.max(...failed)
106
+ console.log(`${workers} workers x ${perWorker} saves x ${rounds} rounds -> lost facts: worst ${worstLost}, average ${avgLost}; failed saves: worst ${worstFailed}`)
107
+ const staleBad = stale.ok !== perWorker || staleKept !== perWorker || staleLeftover.length > 0
108
+ if (worstLost > 0 || worstFailed > 0 || staleBad) {
109
+ console.log('FAIL: concurrent saves lose facts or fail outright')
110
+ process.exit(1)
111
+ }
112
+ console.log('PASS: every concurrent save survived and none failed')
@@ -0,0 +1,72 @@
1
+ /**
2
+ * Recall benchmark: how often does memory_recall's ranking put the intended fact first?
3
+ *
4
+ * It reads the live store (read-only) and runs a fixed query set whose expected ids were
5
+ * written by hand from the fact statements. Hit@1 is the number that matters: an agent asks
6
+ * one question and reads the top answer. Run: node tools/recall-bench.mjs [--json]
7
+ */
8
+ import { readFile } from 'node:fs/promises'
9
+ import { join } from 'node:path'
10
+ import { homedir } from 'node:os'
11
+ import { rankFacts } from '../lib/index.js'
12
+
13
+ const STORE = join(homedir(), '.dsh', 'memory', 'facts.json')
14
+
15
+ /** query -> expected fact id. Phrased the way a later session actually asks. */
16
+ const CASES = [
17
+ ['which CPU is in this machine', 'env.cpu'],
18
+ ['how many cores does the processor have', 'env.cpu'],
19
+ ['how much VRAM does the GPU have', 'env.gpu'],
20
+ ['what python version is bundled', 'env.python'],
21
+ ['where is cupy installed', 'gpu.cupy_installed'],
22
+ ['what is the DSH home directory', 'env.dsh_home'],
23
+ ['which node and npm versions are installed', 'env.node_npm'],
24
+ ['monte carlo pi benchmark on the CPU', 'bench.cpu_suite'],
25
+ ['how long did the ray tracer take at 1080p', 'bench.render'],
26
+ ['GPU monte carlo speedup over CPU', 'bench.gpu_monte_carlo'],
27
+ ['how long did the batch of four pipelines run', 'batch.summary'],
28
+ ['why is wikipedia blocked by SNI', 'net.wikipedia_access'],
29
+ ['is github push failing', 'network.github-git'],
30
+ ['deck rating for physical world execution', 'deck.ratings'],
31
+ ['how many slides in the round 2 deck', 'deck.round2'],
32
+ ['which tools does the memory plugin provide', 'tool.limit_break_plugin'],
33
+ ['weekly maintenance objective for the plugin', 'task.limit-break.maintenance'],
34
+ ['what is the latest npm release', 'release.dsh-limit-break'],
35
+ ['why was memory_prune removed', 'tool.memory_prune'],
36
+ ['what renders the film with numpy', 'film.wotaiyang']
37
+ ]
38
+
39
+ const raw = JSON.parse(await readFile(STORE, 'utf8'))
40
+ const facts = (raw.facts || []).filter(Boolean)
41
+
42
+ let hit1 = 0
43
+ let hit3 = 0
44
+ let rr = 0
45
+ const misses = []
46
+ for (const [query, want] of CASES) {
47
+ const ranked = rankFacts(facts, query).map((row) => row.fact.id)
48
+ const at = ranked.indexOf(want)
49
+ if (at === 0) hit1 += 1
50
+ if (at >= 0 && at < 3) hit3 += 1
51
+ if (at >= 0) rr += 1 / (at + 1)
52
+ if (at !== 0) misses.push({ query, want, at: at < 0 ? 'miss' : at + 1, top: ranked.slice(0, 3) })
53
+ }
54
+
55
+ const report = {
56
+ store: STORE,
57
+ facts: facts.length,
58
+ cases: CASES.length,
59
+ hit_at_1: +(hit1 / CASES.length).toFixed(3),
60
+ hit_at_3: +(hit3 / CASES.length).toFixed(3),
61
+ mrr: +(rr / CASES.length).toFixed(3),
62
+ misses
63
+ }
64
+
65
+ if (process.argv.includes('--json')) {
66
+ console.log(JSON.stringify(report, null, 2))
67
+ } else {
68
+ console.log(`facts=${report.facts} cases=${report.cases} hit@1=${report.hit_at_1} hit@3=${report.hit_at_3} mrr=${report.mrr}`)
69
+ for (const miss of report.misses) {
70
+ console.log(` miss: "${miss.query}" wanted ${miss.want} got rank ${miss.at} -> ${miss.top.join(', ')}`)
71
+ }
72
+ }