dsh-fast-mode 1.5.29 → 1.5.31

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## 1.5.31（`turns.py` 顺手审另一条提醒：压缩后重述规则的那条，会不会无理由地多出来）
+
+- 插件只主动发两条提醒：入场步的「少写散文」（每轮恰好一次）和压缩后的「规则重申」（每次压缩后一次，因为规则可能被裁掉）。`bench/turns.py` 原来只审第一条，第二条只当噪音忽略——如果它的标志位泄漏，就会在每个合格轮里白送一段规则文本。
+- 新增读数：有多少轮带过重述、其中有多少轮的重述条数**超过**该轮的压缩次数 + 1（+1 是允许的：压缩发生在上一轮末尾时，重申落在下一轮第一步）。近 72 小时真实会话：**624 轮**带重述，**0 轮**没有压缩可指。
+- 同时把「首次提醒落在第几步」的直方图、`--explain-doubles`（列出每个重复提醒的轮、步号、seq 与压缩点）固化进脚本，方便下次换版本时重跑对照。
+- 只动 `bench/turns.py` 与文档，`src/` 无改动。
+
+## 1.5.30（把 1.5.28 的修复拿真实流量做 A/B：同一份会话，同一串事件，两个版本）
+
+- **新增 `bench/replay.mjs`**：把一份留存会话的事件流原样喂给插件自己的钩子（`session/event` + `agent/pre-step`），数出「这一版会发几条提醒、落在第几步」。会话 id 是合成的，`metrics`/`benchmarkFile` 关掉，不碰 `~/.dsh` 状态。
+- **量化修复效果**：轮内压缩时宿主用**同一个 turn 号**重入该轮，而这次重入不落盘——所以真实会话记录里看不到它，只看得到「同一轮两条提醒」。`--simulate-reentry` 在每次 `compaction/start` 后按当前 turn 号补发一条 `turn/start`，把这个假设显式化：
+  - 1.5.27 的接线（`noteTurn` 不知道 turn 号）：1529 个合格轮里 **940 轮**会被发两次以上；
+  - 1.5.28 的接线：同样 1529 轮 **0 轮**；
+  - 单会话对照更直观：`session-1ee3a310` 111 个合格轮，旧版 82 轮重复（最长的 121 号轮在第 4/13/38/65/102 步各发一次），新版 0 轮。
+- **真实流量侧读数不变**：`bench/turns.py` 近 72 小时 651 个合格轮里 120 轮真收到两条、其中 110 轮两次之间夹着压缩，与重入假设一致。
+- **没有行为变更**：只加脚本、改文档与版本号；`src/` 与 1.5.29 逐字相同。
+
 ## 1.5.29（1.5.28 的补丁：把 `bench/turns.py` 真的放进包里）
 
 - `package.json` 的 `files` 是一份白名单，1.5.28 新增的 `bench/turns.py` 没进去：`npm pack dsh-fast-mode@1.5.28` 里没有这个脚本，而 `CHANGELOG` 和 `bench/README.md` 都在教读者运行它。1.5.29 只改白名单 + 版本号，**代码与 1.5.28 逐字相同**。

package/bench/README.md

@@ -136,6 +136,20 @@ node bench/verify-arm-reminders.mjs --arms                # 臂 x 场景的提
 
 采集方式：跑这一格时把臂补丁另存为 `<tag>.arm.yml` 并加 `debug: true` + `debugFile: <runs>/<tag>.debug`，跑完从调试日志里按 `step=N ... turnSteps=M ... directive=kind` 抽行，然后删掉日志与补丁。`debug` 只在步边界写一行已经格式化好的文本，不参与任何调度判断、不额外发请求；日志写在 run 目录之外，所以 `leak_scan`/session id 统计不受影响。老 result 行没有这些字段，读取方一律用 `.get()`，缺字段等于「没有证据」，不是「零提醒」。
 
+### 同一份会话，两个版本各会发几条提醒
+
+`bench/replay.mjs` 把留存会话的事件流喂回插件自己的钩子（`session/event` + `agent/pre-step`），输出每一轮的步数、压缩次数、提醒条数与落点。会话 id 是合成的，`metrics`/`benchmarkFile` 关掉，不写任何 `~/.dsh` 状态。
+
+```bash
+node bench/replay.mjs ~/.dsh/sessions/<slug>/session-<uuid>/session.v4.jsonl.zstd
+node bench/replay.mjs <transcript> --json
+node bench/replay.mjs <transcript> --simulate-reentry   # 每次压缩后按当前 turn 号重入
+```
+
+`turns.py` 同时审两条提醒：入场步的「少写散文」（每轮恰好一次）与压缩后的「规则重申」（每次压缩后一次）。后者允许比该轮压缩次数多一条（压缩发生在上一轮末尾时重申落在下一轮第一步），超过就是标志位泄漏；近 72 小时真实会话是 624 轮带重申、0 轮无压缩可指。`--explain-doubles` 会逐条列出重复提醒所在的轮、步号、seq 与压缩点。
+
+`--simulate-reentry` 补的是会话记录里**不存在**的那条事件：宿主压缩后重入同一轮时用同一个 turn 号再发一次 `turn/start`，而这次重入不落盘。加上它以后，1.5.27 与 1.5.28 的差别就能在同一份数据上量出来（1529 个合格轮：940 → 0）。
+
 ### 提醒是否每轮重新上膛、且每轮只发一次
 
 `bench/turns.py` 读留存会话，按 `turn/start` 切轮，输出：轮数/会话、合格轮（默认 ≥ 4 步）、**没收到提醒**的合格轮、**收到两次以上**的轮、多轮会话里后续轮的重新上膛率。只有 `user/message` / `agent/inbox/spliced` / `system/message` 里的文案算投递，模型在自己正文里复述的不算。

package/bench/replay.mjs

@@ -0,0 +1,154 @@
+#!/usr/bin/env node
+/**
+ * Replay one retained transcript through the plugin's own hooks.
+ *
+ * The reminder is delivered at a step boundary, so a transcript could only ever
+ * count *delivered* reminders, never the ones the plugin would have sent under a
+ * different version. This replays the recorded event stream into `apply()` with a
+ * stand-in context, counts what the scheduler decides to emit, and prints one
+ * line per turn: steps, reminders, and the step each reminder landed on.
+ *
+ * That is how the 1.5.28 latch fix is checked against real traffic instead of
+ * synthetic test events: same transcript, same hook order, two versions of
+ * `noteTurn`, two counts.
+ *
+ * Usage:
+ *   node bench/replay.mjs ~/.dsh/sessions/<slug>/session-<uuid>/session.v4.jsonl.zstd
+ *   node bench/replay.mjs <transcript> --min-steps 4 --json
+ *   node bench/replay.mjs <transcript> --simulate-reentry
+ *
+ * `--simulate-reentry` adds the event the transcript cannot show: after each
+ * `compaction/start` it re-fires `turn/start` with the same turn number, which is
+ * the hypothesis for the doubled reminders (the harness re-enters a turn it
+ * compacted mid-flight, and that re-entry is not persisted). With the event
+ * added, the pre-1.5.28 latch resets and the same turn is reminded twice; with
+ * the turn guard it is not.
+ *
+ * The session id is synthetic, so the run never touches `~/.dsh` state; metrics
+ * and benchmark output are switched off in the config passed to `apply()`.
+ */
+import { execFileSync } from 'node:child_process'
+import { readFileSync } from 'node:fs'
+
+import { apply } from '../index.js'
+
+const NEEDLE = 'prose is now the cost'
+const CONFIG = { enabled: true, gear: 'auto', outputDiscipline: 'late', metrics: false, benchmarkFile: '' }
+
+/** Minimal stand-in for the Cordis context: handlers plus a silent logger. */
+function fakeContext() {
+  const handlers = new Map()
+  return {
+    handlers,
+    on(event, handler) {
+      handlers.set(event, handler)
+      return () => handlers.delete(event)
+    },
+    inject: () => undefined,
+    get: () => undefined,
+    logger: { info: () => {} },
+    fire(event, ...args) {
+      const handler = handlers.get(event)
+      if (!handler) return undefined
+      return handler(...args)
+    },
+  }
+}
+
+/** Decide the way the harness does: enter, and keep whatever the plugin added. */
+const preStep = (ctx, agentId, step) => {
+  const decision = {
+    kind: 'enter',
+    messages: [{ id: 'sys', role: 'system', content: [] }, { id: `last-${step}`, role: 'assistant', content: [] }],
+  }
+  return ctx.fire('agent/pre-step', { agent: { id: agentId }, messages: [] }, async () => decision)
+}
+
+const events = path => {
+  const raw = path.endsWith('.zstd')
+    ? execFileSync('zstd', ['-dc', path], { maxBuffer: 1 << 30 })
+    : readFileSync(path)
+  const out = []
+  for (const line of raw.toString('utf8').split('\n')) {
+    if (!line.trim()) continue
+    try {
+      out.push(JSON.parse(line))
+    } catch {
+      // a torn tail line is not fatal: the replay is a census, not a forensic read
+    }
+  }
+  return out
+}
+
+const reminderText = notice => {
+  const text = notice?.content?.[0]?.text ?? ''
+  return text.includes(NEEDLE)
+}
+
+async function main() {
+  const argv = process.argv.slice(2)
+  const path = argv.find(arg => !arg.startsWith('--'))
+  if (!path) throw new Error('usage: node bench/replay.mjs <transcript.jsonl.zstd> [--min-steps N] [--json]')
+  const minSteps = Number(argv[argv.indexOf('--min-steps') + 1]) || 4
+  const asJson = argv.includes('--json')
+  const reentry = argv.includes('--simulate-reentry')
+
+  const ctx = fakeContext()
+  apply(ctx, CONFIG)
+  const session = { id: 'replay' }
+  const turns = []
+  let cur = { steps: 0, reminders: [], compactions: 0, turnNo: 0 }
+  for (const event of events(path)) {
+    const type = event.type
+    const data = event.data ?? {}
+    if (type === 'turn/start') {
+      if (cur.steps || cur.reminders.length) turns.push(cur)
+      cur = { steps: 0, reminders: [], compactions: 0, turnNo: data.turn ?? turns.length + 1 }
+      await ctx.fire('session/event', session, event)
+    } else if (type === 'step/start') {
+      cur.steps += 1
+      await ctx.fire('session/event', session, event)
+      const decision = await preStep(ctx, session.id, cur.steps)
+      const added = (decision?.messages ?? []).filter(message => message?.source?.form === 'notice' && reminderText(message))
+      if (added.length) cur.reminders.push(cur.steps)
+    } else if (type === 'compaction/start') {
+      cur.compactions += 1
+      await ctx.fire('session/event', session, event)
+      if (reentry) {
+        // Same turn number, delivered the way the harness delivers it: the event
+        // exists in the hook stream but never reaches the transcript.
+        await ctx.fire('session/event', session, { type: 'turn/start', turn: cur.turnNo, time: event.time })
+      }
+    } else if (type === 'compaction/end') {
+      await ctx.fire('session/event', session, event)
+    } else if (type === 'assistant/message' || type === 'tool/call' || type === 'tool/result') {
+      await ctx.fire('session/event', session, event)
+    }
+  }
+  if (cur.steps || cur.reminders.length) turns.push(cur)
+
+  const rows = turns.map((turn, index) => ({
+    turn: index + 1,
+    steps: turn.steps,
+    compactions: turn.compactions,
+    reminders: turn.reminders.length,
+    at: turn.reminders,
+  }))
+  if (asJson) {
+    console.log(JSON.stringify({ path, minSteps, simulateReentry: reentry, turns: rows }, null, 2))
+    return
+  }
+  const qualifying = rows.filter(row => row.steps >= minSteps)
+  const doubled = qualifying.filter(row => row.reminders > 1)
+  console.log(`${path}`)
+  for (const row of qualifying) {
+    console.log(`  turn ${row.turn}: steps ${row.steps} compactions ${row.compactions} `
+      + `reminders ${row.reminders}${row.at.length ? ` at ${row.at.join(',')}` : ''}`)
+  }
+  console.log(`qualifying turns (>= ${minSteps} steps) ${qualifying.length}  doubled ${doubled.length}`)
+}
+
+main().catch(error => {
+  console.error(error.message)
+  process.exit(1)
+})

package/bench/turns.py

@@ -33,6 +33,11 @@ import time
 
 SESS = os.path.expanduser('~/.dsh/sessions')
 NEEDLE = 'this turn is long enough that prose'
+# The other notice the plugin can deliver, the one that restates the rules after a
+# compaction. It is *supposed* to repeat, once per compaction whose rules were
+# evicted; the audit question is only whether it ever fires with no compaction to
+# point at, which would mean a stale flag.
+REASSERT = 'the context was just compacted'
 # A delivered directive is a message part; the model quoting it back inside its
 # own prose is an echo, so assistant-authored text never counts as delivery.
 PLUGIN_ROLES = {'user/message', 'agent/inbox/spliced', 'system/message'}
@@ -70,10 +75,11 @@ def scan(path):
     started = 0
     turns = []
     cur = {'steps': 0, 'reminders': 0, 'reminderSeqs': [], 'compactions': [],
-           'reminderSteps': []}
+           'reminderSteps': [], 'reasserts': 0}
     for line in p.stdout.decode('utf-8', 'replace').splitlines():
         if ('turn/start' not in line and 'step/start' not in line
-                and 'Auto schedule' not in line and 'compaction/start' not in line):
+                and 'Auto schedule' not in line and 'compaction/start' not in line
+                and 'context was just compacted' not in line):
             continue
         try:
             o = json.loads(line)
@@ -87,11 +93,13 @@ def scan(path):
             if cur['steps'] or cur['reminders']:
                 turns.append(cur)
             cur = {'steps': 0, 'reminders': 0, 'reminderSeqs': [], 'compactions': [],
-           'reminderSteps': []}
+                   'reminderSteps': [], 'reasserts': 0}
         elif kind == 'step/start':
             cur['steps'] += 1
         elif kind == 'compaction/start':
             cur['compactions'].append(o.get('seq'))
+        elif kind in PLUGIN_ROLES and REASSERT in text_of(data):
+            cur['reasserts'] += 1
         elif kind in PLUGIN_ROLES and NEEDLE in text_of(data):
             cur['reminders'] += 1
             cur['reminderSeqs'].append(o.get('seq'))
@@ -121,6 +129,7 @@ def main():
     later_qualifying = later_reminded = 0
     per_session_turns = collections.Counter()
     entry_steps = collections.Counter()
+    reasserted = reassert_orphan = 0
     offenders = []
     cutoff = (time.time() - a.since_hours * 3600) * 1000 if a.since_hours else 0
     skipped_old = 0
@@ -151,6 +160,12 @@ def main():
                     later_reminded += 1 if t['reminders'] else 0
             if t['reminders']:
                 entry_steps[t['reminderSteps'][0]] += 1
+            if t['reasserts']:
+                reasserted += 1
+                # One reassert belongs to a compaction from the previous turn: the
+                # flag is set, the turn ends, the next step of the next turn speaks.
+                if t['reasserts'] > len(t['compactions']) + 1:
+                    reassert_orphan += 1
             if t['reminders'] > 1:
                 doubled += 1
                 seqs = t['reminderSeqs']
@@ -178,6 +193,8 @@ def main():
         'doubled': doubled,
         'doubled_with_compaction_between': doubled_compacted,
         'entry_steps': dict(sorted(entry_steps.items())),
+        'turns_with_compaction_reassert': reasserted,
+        'reasserts_without_compaction': reassert_orphan,
         'later_qualifying_turns': later_qualifying,
         'later_reminded': later_reminded,
         'skipped_old': skipped_old,
@@ -195,6 +212,8 @@ def main():
           f"(with a compaction between: {doubled_compacted})")
     if entry_steps:
         print(f"first reminder landed at step: {dict(sorted(entry_steps.items()))}")
+    print(f"compaction re-assert: {reasserted} turns carried one, "
+          f"{reassert_orphan} had more of them than compactions to point at")
     if a.explain_doubles and offenders:
         print('doubled turns:')
         for o in offenders[:20]:

package/index.js

@@ -97,7 +97,7 @@ const TAG = '[fast-mode]'
  * Shipped identity of this build. Traced into the debug log so a stale install is
  * visible at a glance, and pinned by a test against `package.json`.
  */
-export const VERSION = '1.5.29'
+export const VERSION = '1.5.31'
 const GLOBAL_AGENT = '__global__'
 
 /**

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-fast-mode",
-  "version": "1.5.29",
+  "version": "1.5.31",
   "description": "DeepSeek Harness fast mode: same thinking effort, fewer round trips. An adaptive parallel tool fan-out, output discipline that never trims the answer, and a cancellable stall guard. Never changes reasoning effort, the model, or the request body. Provider-agnostic, one dependency (@deepseek-ai/schemastery).",
   "repository": {
     "type": "git",
@@ -45,6 +45,7 @@
     "bench/patchshare.py",
     "bench/ablate.py",
     "bench/turns.py",
+    "bench/replay.mjs",
     "docs/",
     "LICENSE"
   ],