dsh-limit-break 1.6.0 → 1.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -5,6 +5,29 @@ All notable changes to `dsh-limit-break`. Format follows
5
5
 
6
6
  ## Unreleased
7
7
 
8
+ ## 1.7.0 — 2026-11-05
9
+
10
+ ### Changed
11
+ - **`resilient_fetch` remembers which rung worked instead of rediscovering it on every call.** A
12
+ non-direct success is written back as `network.route.<host>` (14-day TTL, written under the same
13
+ lock as `memory_save`) and the next call to that host starts at that rung; if a plain fetch starts
14
+ working again the route is deleted, so an unblocked host heals itself. Measured on
15
+ `en.wikipedia.org/wiki/Monte_Carlo_method` with the default 12 s per-rung timeout: cold **12.11 s**
16
+ with attempts `direct>alternate-host`, warm **1.08 s / 1.14 s** with a single `alternate-host`
17
+ attempt — **11.2× faster**; the tool now also reports `millis`, `route`
18
+ (`learned|reused|dropped|none`) and `cached_via`.
19
+ - **`tools/fetch-route-bench.mjs`** (new): cold-vs-warm measurement for one blocked host plus a
20
+ self-heal check (a seeded stale route must be dropped once direct works). 6/6 checks PASS.
21
+ - Route facts are ordinary facts: visible to `memory_recall`/`memory_stats`, removable with any
22
+ store edit. Only fallback successes and self-heal events write, so a working network writes nothing.
23
+
24
+ ### Notes
25
+ - No tool added, removed or renamed; store format unchanged. The tool description text now mentions
26
+ the remembered route, which an already-open session will not see until its tool snapshot refreshes.
27
+ - `maintain.py status` still prints `market listing not live yet (PR #8296 closed)` while the same
28
+ call reports `listed_in_market: true`: the check treats a merged-and-closed PR as not live. Left
29
+ as-is, documented in MAINTAIN.md so the next round does not read it as a regression.
30
+
8
31
  ## 1.6.0 — 2026-10-29
9
32
 
10
33
  ### Fixed
package/MAINTAIN.md CHANGED
@@ -138,8 +138,14 @@ git add -A && git commit -m "..." && git push
138
138
  bench.gpu_monte_carlo and bench.render”(2026-10-21 的回写),陈旧描述不再存在。
139
139
  - [x] 插件工具对**已经打开的会话**不可见:已在 README「Known limits」第 1 条写明
140
140
  (tool list 是 per-session 快照,插件侧修不了)。事实 `harness.plugin-tool-visibility` 有实测。
141
- - [ ] `resilient_fetch` 目前按"主机名别名"降级,可以继续加:`--resolve` 到已知 IP、
142
- IPv6 优先、以及把成功路径写回记忆库(现在是只读阶梯)。
141
+ - [x] **`resilient_fetch` 会把成功路径写回记忆库**(1.7.0,2026-11-05):非 direct 成功写成
142
+ `network.route.<host>`(14 天 TTL,走 `memory_save` 同一把锁),下次同主机直接从该级起步;
143
+ direct 恢复后删掉路由(自愈)。实测 `en.wikipedia.org/wiki/Monte_Carlo_method`、12 s 超时:
144
+ cold 12.11 s(`direct>alternate-host`)→ warm 1.08 s / 1.14 s(单次 `alternate-host`),11.2×。
145
+ 跑 `node tools/fetch-route-bench.mjs <url> <timeoutMs>` 复测(约 15 s,6 项检查)。
146
+ 仍可继续加:`--resolve` 到已知 IP、IPv6 优先(backlog)。
147
+ - [ ] `maintain.py status` 的 `problems` 里 `market listing not live yet (PR #8296 closed)` 是误报:
148
+ 同一份输出里 `listed_in_market: true`。原因未查,先记为已知噪音,别当成回归。
143
149
  - [x] **`memory_save` 的并发写已有自动化覆盖**(1.6.0,2026-10-29):`tools/concurrency-selftest.mjs`
144
150
  起 N 个独立进程同时写同一个 store,再读回数存活条数,另测“陈旧锁被接管”。同机 8 进程 × 6 次 × 3 轮:
145
151
  修复前丢 16–17/48、且最多 25/48 直接 `EPERM`(rename 输家);加 `O_EXCL` 锁后 0 丢 0 失败,
package/README.md CHANGED
@@ -43,6 +43,12 @@ IPv6 → IPv4 → a **sibling hostname of the same service** whose certificate s
43
43
  It does **not** disable certificate verification and does not route through a proxy that would
44
44
  impersonate the origin. Those are MITM, not workarounds.
45
45
 
46
+ **The ladder learns.** A fallback rung that worked is written back as `network.route.<host>` and the
47
+ next call to that host starts there; if a plain fetch starts working again the route is deleted (host
48
+ self-heals). Measured on `en.wikipedia.org/wiki/Monte_Carlo_method`, 12 s per-rung timeout: cold
49
+ **12.1 s** (rung 1 burned the timeout, rung 4 answered) → warm **1.08 s / 1.14 s**, one attempt
50
+ instead of two — **11.2× faster**. Numbers come from `tools/fetch-route-bench.mjs`.
51
+
46
52
  ## Install
47
53
 
48
54
  ```sh
@@ -99,6 +105,9 @@ Measured, not guessed — each line is something a session actually hit.
99
105
  still cannot stop a reader from seeing the previous snapshot.
100
106
  * **The fetch ladder is hostname-level.** It does not defeat IP-level blocks, TLS interception,
101
107
  or a proxy that terminates the connection; it does not skip certificate verification.
108
+ * **A remembered route is a bet with a 14-day expiry.** The ladder trusts `network.route.<host>` for
109
+ two weeks and drops it as soon as a direct request works; if the winning rung starts failing the
110
+ call still walks the rest of the ladder, it just pays that one rung first.
102
111
  * **`memory_recall` ranks by keyword overlap only** (id/topic/tags/statement/details), no
103
112
  embeddings: stopword-filtered, crudely stemmed word matching weighted by idf, so it is good at
104
113
  "which fact did I mean" and bad at paraphrase. `tools/recall-bench.mjs` states the current
package/lib/index.js CHANGED
@@ -233,6 +233,64 @@ async function withStoreLock(dir, fn) {
233
233
  }
234
234
  }
235
235
 
236
+ // A failed fetch tells you which rung of the ladder works for that host *today*; throwing that
237
+ // away means paying the same dead rungs on every call. A non-direct success is remembered per
238
+ // host (network.route.<host>) for two weeks and the ladder starts at that rung next time. When
239
+ // direct starts working again the route is dropped, so a host that unblocks heals itself.
240
+ const ROUTE_TTL_MS = 14 * 24 * 60 * 60 * 1000
241
+
242
+ function routeId(host) {
243
+ return `network.route.${String(host).toLowerCase().replace(/[^a-z0-9.-]+/g, '-')}`
244
+ }
245
+
246
+ async function readRoute(dir, host) {
247
+ const store = await loadStore(dir)
248
+ const fact = store.facts.find((f) => f && String(f.id) === routeId(host))
249
+ if (!fact) return null
250
+ const details = fact.details && typeof fact.details === 'object' ? fact.details : {}
251
+ const observed = Date.parse(String(details.observedAt || fact.timestamp || ''))
252
+ if (!Number.isFinite(observed) || Date.now() - observed > ROUTE_TTL_MS) return null
253
+ const via = typeof details.via === 'string' ? details.via : ''
254
+ if (!via || via === 'direct') return null
255
+ return { via, url: typeof details.url === 'string' ? details.url : '', observedAt: new Date(observed).toISOString() }
256
+ }
257
+
258
+ async function writeRoute(dir, host, info) {
259
+ const now = new Date().toISOString()
260
+ const id = routeId(host)
261
+ return withStoreLock(dir, async () => {
262
+ const store = await loadStore(dir)
263
+ const record = {
264
+ id,
265
+ topic: 'network',
266
+ statement: `${host} answers through the ${info.via} rung of the fetch ladder (${info.url}); a direct request to it fails today, so the ladder should start at ${info.via}.`,
267
+ value: null,
268
+ unit: null,
269
+ details: { host, via: info.via, url: info.url, observedAt: now, millis: info.millis || null },
270
+ source: 'resilient_fetch',
271
+ timestamp: now,
272
+ tags: ['network', 'fetch-route']
273
+ }
274
+ const at = store.facts.findIndex((f) => f && String(f.id) === id)
275
+ if (at >= 0) store.facts[at] = record
276
+ else store.facts.push(record)
277
+ await saveStore(dir, store)
278
+ return record
279
+ })
280
+ }
281
+
282
+ async function dropRoute(dir, host) {
283
+ const id = routeId(host)
284
+ return withStoreLock(dir, async () => {
285
+ const store = await loadStore(dir)
286
+ const before = store.facts.length
287
+ store.facts = store.facts.filter((f) => !(f && String(f.id) === id))
288
+ if (store.facts.length === before) return false
289
+ await saveStore(dir, store)
290
+ return true
291
+ })
292
+ }
293
+
236
294
  // Ranking inputs. The old scorer used raw substring counts, which let a common word
237
295
  // ("the", "how", "long") outvote the one rare word that identifies the fact. Words are
238
296
  // matched against word sets instead, suffixed words share a stem, and each query token is
@@ -655,7 +713,7 @@ export function apply(ctx, config) {
655
713
  ctx.tools.register({
656
714
  name: 'resilient_fetch',
657
715
  description:
658
- 'Fetch a URL through a 4-step ladder for hosts whose block is keyed on the TLS hostname (SNI) rather than the IP: plain fetch -> IPv6 -> IPv4 -> a same-service alternate hostname. Use when web_fetch or a plain request fails with a timeout, a connection reset, or "resolves to a non-public IP", and you still need the content.',
716
+ 'Fetch a URL through a 4-step ladder for hosts whose block is keyed on the TLS hostname (SNI) rather than the IP: plain fetch -> IPv6 -> IPv4 -> a same-service alternate hostname. A host that only answered on a later rung is remembered in the memory store for two weeks and the next call starts at that rung. Use when web_fetch or a plain request fails with a timeout, a connection reset, or "resolves to a non-public IP", and you still need the content.',
659
717
  parameters: {
660
718
  type: 'object',
661
719
  properties: {
@@ -669,7 +727,7 @@ export function apply(ctx, config) {
669
727
  schema: { type: 'object' },
670
728
  render: (_args, value) =>
671
729
  value && value.ok
672
- ? [{ type: 'text', text: `HTTP ${value.status} via ${value.via} (${value.bytes} bytes)\n${String(value.text || '').slice(0, 400)}` }]
730
+ ? [{ type: 'text', text: `HTTP ${value.status} via ${value.via} in ${(Number(value.millis) / 1000).toFixed(1)}s [route ${value.route}] (${value.bytes} bytes)\n${String(value.text || '').slice(0, 400)}` }]
673
731
  : [{ type: 'text', text: `All attempts failed: ${(value && value.attempts ? value.attempts : []).map((a) => `${a.via}=${a.error || a.status}`).join(' | ')}` }]
674
732
  },
675
733
  presentCall: (args) => ({
@@ -682,11 +740,23 @@ export function apply(ctx, config) {
682
740
  const input = args && typeof args === 'object' ? args : {}
683
741
  const url = typeof input.url === 'string' ? input.url.trim() : ''
684
742
  if (!/^https?:\/\//i.test(url)) return { ok: false, code: 'fetch.invalid', attempts: [], message: 'url must start with http:// or https://' }
743
+ let host = ''
744
+ try {
745
+ host = new URL(url).hostname
746
+ } catch {
747
+ return { ok: false, code: 'fetch.invalid', attempts: [], message: 'url is not a valid absolute URL' }
748
+ }
685
749
  const per = Math.max(2000, Number(input.timeoutMs) || timeoutMs)
686
- const targets = [{ url, via: 'direct' }, ...alternatesFor(url).map((u) => ({ url: u, via: 'alternate-host' }))]
750
+ const cached = await readRoute(storeDir, host).catch(() => null)
751
+ let targets = [{ url, via: 'direct' }, ...alternatesFor(url).map((u) => ({ url: u, via: 'alternate-host' }))]
687
752
  if (shellFallback) {
688
753
  targets.push({ url, via: 'curl-ipv6' }, { url, via: 'curl-ipv4' })
689
754
  }
755
+ if (cached) {
756
+ const preferred = targets.filter((target) => target.via === cached.via)
757
+ if (preferred.length) targets = [...preferred, ...targets.filter((target) => target.via !== cached.via)]
758
+ }
759
+ const startedAt = Date.now()
690
760
  const attempts = []
691
761
  for (const target of targets) {
692
762
  try {
@@ -697,6 +767,18 @@ export function apply(ctx, config) {
697
767
  attempts.push({ via: target.via, url: target.url, status: result.status })
698
768
  if (result.ok && result.text) {
699
769
  const clipped = clip(result.text)
770
+ const millis = Date.now() - startedAt
771
+ let route = cached ? 'reused' : 'none'
772
+ try {
773
+ if (target.via === 'direct') {
774
+ if (cached) route = (await dropRoute(storeDir, host)) ? 'dropped' : 'none'
775
+ } else {
776
+ await writeRoute(storeDir, host, { via: target.via, url: target.url, millis })
777
+ route = cached && cached.via === target.via ? 'reused' : 'learned'
778
+ }
779
+ } catch {
780
+ route = 'unwritten'
781
+ }
700
782
  return {
701
783
  ok: true,
702
784
  status: result.status,
@@ -706,14 +788,17 @@ export function apply(ctx, config) {
706
788
  bytes: clipped.bytes,
707
789
  truncated: clipped.truncated,
708
790
  text: clipped.text,
709
- attempts
791
+ attempts,
792
+ millis,
793
+ route,
794
+ cached_via: cached ? cached.via : null
710
795
  }
711
796
  }
712
797
  } catch (error) {
713
798
  attempts.push({ via: target.via, url: target.url, error: String((error && error.message) || error).slice(0, 300) })
714
799
  }
715
800
  }
716
- return { ok: false, code: 'fetch.failed', requested: url, attempts }
801
+ return { ok: false, code: 'fetch.failed', requested: url, cached_via: cached ? cached.via : null, attempts }
717
802
  }
718
803
  }),
719
804
  'limit-break: resilient_fetch tool'
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-limit-break",
3
- "version": "1.6.0",
3
+ "version": "1.7.0",
4
4
  "description": "Durable cross-session memory and a fetch ladder that survives hostname-keyed blocking — four small tools for DeepSeek Harness.",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",
@@ -0,0 +1,98 @@
1
+ /**
2
+ * Route-cache probe: does remembering which rung of the fetch ladder works actually pay?
3
+ *
4
+ * `resilient_fetch` walks direct -> alternates -> curl IPv6 -> curl IPv4. On a host whose direct
5
+ * request is blocked (SNI-keyed), every call used to rediscover that by burning a timeout on rung 1
6
+ * before rung 4 answered. Since 1.7.0 a non-direct success is stored as `network.route.<host>` and
7
+ * the next call starts at that rung.
8
+ *
9
+ * This measures one blocked host: cold call (cache miss) vs warm calls (cache hit), printing
10
+ * attempts and seconds, then checks the store holds the route fact.
11
+ *
12
+ * Run: node tools/fetch-route-bench.mjs [url] [timeoutMs]
13
+ */
14
+ import { mkdtemp, readFile } from 'node:fs/promises'
15
+ import { tmpdir } from 'node:os'
16
+ import { join } from 'node:path'
17
+ import { fileURLToPath, pathToFileURL } from 'node:url'
18
+
19
+ const LIB = fileURLToPath(new URL('../lib/index.js', import.meta.url))
20
+ const url = process.argv[2] || 'https://en.wikipedia.org/wiki/Monte_Carlo_method'
21
+ const timeoutMs = Number(process.argv[3] || 6000)
22
+ const host = new URL(url).hostname
23
+
24
+ const { apply } = await import(pathToFileURL(LIB).href)
25
+ const mod = await import(pathToFileURL(LIB).href)
26
+ const tools = new Map()
27
+ const ctx = {
28
+ effect(fn) {
29
+ fn()
30
+ },
31
+ tools: {
32
+ register(tool) {
33
+ tools.set(tool.name, tool)
34
+ return () => tools.delete(tool.name)
35
+ }
36
+ }
37
+ }
38
+ const storeDir = join(await mkdtemp(join(tmpdir(), 'limit-break-route-')), 'memory')
39
+ apply(ctx, { storeDir })
40
+ const fetchTool = tools.get('resilient_fetch')
41
+
42
+ const call = async (label) => {
43
+ const started = Date.now()
44
+ const result = await fetchTool.execute({ url, timeoutMs })
45
+ const seconds = +((Date.now() - started) / 1000).toFixed(2)
46
+ console.log(
47
+ `${label}: ok=${result.ok} via=${result.via || '-'} route=${result.route || '-'} attempts=${(result.attempts || []).map((a) => a.via).join('>')} seconds=${seconds}`
48
+ )
49
+ return { result, seconds }
50
+ }
51
+
52
+ const cold = await call('cold')
53
+ if (!cold.result.ok || cold.result.via === 'direct') {
54
+ console.log('SKIP: direct fetch to this host works today, so there is no failed rung to remember')
55
+ process.exit(0)
56
+ }
57
+ const warm1 = await call('warm1')
58
+ const warm2 = await call('warm2')
59
+
60
+ const store = JSON.parse(await readFile(join(storeDir, 'facts.json'), 'utf8'))
61
+ const fact = store.facts.find((row) => row && row.id === `network.route.${host.toLowerCase()}`)
62
+ console.log(`route fact: ${fact ? JSON.stringify(fact.details) : 'MISSING'}`)
63
+
64
+ // Self-heal: if a host has a remembered route but a direct request now works, the stale route
65
+ // must be dropped instead of being trusted forever.
66
+ const healUrl = 'https://api.wikimedia.org/core/v1/wikipedia/en/page/Monte_Carlo_method'
67
+ const healHost = new URL(healUrl).hostname
68
+ const seeded = await mod.loadStore(storeDir)
69
+ seeded.facts.push({
70
+ id: `network.route.${healHost}`,
71
+ topic: 'network',
72
+ statement: 'seeded stale route for the self-heal check',
73
+ details: { host: healHost, via: 'alternate-host', url: healUrl, observedAt: new Date().toISOString(), millis: null },
74
+ source: 'bench',
75
+ timestamp: new Date().toISOString(),
76
+ tags: ['network']
77
+ })
78
+ await mod.saveStore(storeDir, seeded)
79
+ const heal = await fetchTool.execute({ url: healUrl, timeoutMs })
80
+ const afterHeal = await mod.loadStore(storeDir)
81
+ const healDropped = !afterHeal.facts.some((row) => row && row.id === `network.route.${healHost}`)
82
+ console.log(`self-heal: ok=${heal.ok} via=${heal.via} route=${heal.route} factDropped=${healDropped}`)
83
+
84
+ const checks = [
85
+ ['cold used a fallback rung', cold.result.via !== 'direct' && cold.result.attempts.length > 1],
86
+ ['warm call starts at the remembered rung', warm1.result.attempts.length === 1 && warm1.result.attempts[0].via === cold.result.via],
87
+ ['warm call is at least 2x faster', warm1.seconds * 2 <= cold.seconds],
88
+ ['warm call is stable', warm2.result.via === cold.result.via && warm2.result.route === 'reused'],
89
+ ['route fact written', Boolean(fact) && fact.details && fact.details.via === cold.result.via],
90
+ ['stale route dropped once direct works', heal.ok && heal.via === 'direct' && heal.route === 'dropped' && healDropped]
91
+ ]
92
+ let failed = 0
93
+ for (const [name, ok] of checks) {
94
+ console.log(`${ok ? 'PASS' : 'FAIL'} ${name}`)
95
+ if (!ok) failed += 1
96
+ }
97
+ console.log(`cold ${cold.seconds}s -> warm ${warm1.seconds}s / ${warm2.seconds}s (${(cold.seconds / Math.max(warm1.seconds, 0.01)).toFixed(1)}x faster)`)
98
+ process.exit(failed ? 1 : 0)