dsh-verify-gate 0.0.0-stage → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,36 @@
1
+ # Changelog
2
+
3
+ ## 0.1.1 — 2026-11-21
4
+
5
+ - **Fix: the nudge no longer corrupts session history.** The reminder was injected as a
6
+ `system`-role message; the harness records every non-`runtime-context` `agent/pre-step`
7
+ message as a `user/message` event, so that payload produced logs rejected on load
8
+ (`stored session ... failed validation: message must have role "user"`) and the session
9
+ could not be opened again. The reminder is now a `user`-role message wrapped in
10
+ `<system-reminder>` with `source.kind = "verify-gate"`.
11
+ - Added `assertInjectable()`: an injected message that is not user-role (or that carries a
12
+ retired `plugin`/`system-prompt` source) throws instead of reaching the log.
13
+ - Stop duplicating the call-to-action: the verdict text already carries it (the reminder
14
+ used to repeat the sentence twice).
15
+
16
+ ## 0.1.0 — 2026-11-21
17
+
18
+ First release. Published to npm as `dsh-verify-gate@0.1.0`.
19
+
20
+ - Observe tool calls through the `tools/execute` waterfall; track which code files a
21
+ turn touched and whether anything verified them afterwards.
22
+ - Nudge through the `agent/pre-step` waterfall: when a turn edited code and nothing was
23
+ run, one system message is appended to the next step's context.
24
+ - `verify_exemption(reason)` tool so an agent can declare why no check applies
25
+ (reasons under 12 characters are rejected).
26
+ - Single ledger by design: `agent/pre-step` carries no agent identity, so per-agent
27
+ isolation would be fake. Concurrent agents share one conservative judgement.
28
+ - 19 tests: 11 for the ledger, 8 for the wiring.
29
+
30
+ Verified live against DSH 0.2.2-rc.5:
31
+
32
+ | arm | behaviour | nudges |
33
+ |---|---|---:|
34
+ | A | `write` a .js, verify nothing | 1 |
35
+ | B | `write` a .js, then `node --check` | 0 |
36
+ | C | `write` a .mjs, then `verify_exemption` | 0 |
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 raysnows
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md CHANGED
@@ -1,3 +1,112 @@
1
- # Temporary Holding Version
2
-
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
1
+ # dsh-verify-gate
2
+
3
+ **A completion gate for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (DSH).**
4
+ When a turn changed code files and nothing was run to check them, it says so — once —
5
+ at the next step, and accepts an exemption when no check applies.
6
+
7
+ English · [简体中文](README.zh.md)
8
+
9
+ ---
10
+
11
+ ## Why
12
+
13
+ Measured on a real machine, 49 sessions / 937 turns:
14
+
15
+ | metric | value |
16
+ |---|---:|
17
+ | turns that edited **code** files | 394 |
18
+ | …that never ran a test or type check | **355 (90.1%)** |
19
+
20
+ That is the failure mode Can Bölük catalogued in
21
+ [_The Harness Problem_](https://blog.can.ac/2026/02/12/the-harness-problem/):
22
+ the agent writes, re-reads, decides it looks fine, and stops — without ever asking reality.
23
+
24
+ DSH ships a safety reviewer (`dsh-experimental-auto-review`) and advisory loop reminders,
25
+ but nothing guards the *completion* boundary. This plugin is that guard.
26
+
27
+ ## How it works
28
+
29
+ Three harness seams, read out of `@deepseek-ai/dsh-agent-loop/lib/index.js` on DSH
30
+ `0.2.2-rc.5` — not guessed:
31
+
32
+ | seam | kind | role |
33
+ |---|---|---|
34
+ | `tools/execute` | waterfall `(exec, next)` | observe calls; `exec.name` + `exec.arguments` |
35
+ | `agent/pre-step` | waterfall `(payload, next)` | **inject** the reminder into `decision.messages` |
36
+ | `agent/turn-stopping` | serial | settle the turn, reset the ledger |
37
+
38
+ > `turn/start` and `turn/end` are **session records** (`session.append(...)`), not context
39
+ > events. Listening for them silently never fires.
40
+
41
+ The injected message lands in the model's context for the very next step:
42
+
43
+ ```json
44
+ { "role": "user", "content": [{ "type": "text", "text": "<system-reminder>\nThis turn changed 1 code file(s) but nothing was run to check them. Run the project tests (or a type check) before calling the task done, or call verify_exemption with a concrete reason why no check applies.\n</system-reminder>" }], "source": { "kind": "verify-gate" } }
45
+ ```
46
+
47
+ > **It must be a `user` message.** `agent/pre-step` messages that are not `runtime-context`
48
+ > are recorded by the harness as `user/message` session events no matter their role, so a
49
+ > `system`-role payload there writes a log that fails validation on load
50
+ > (`... message must have role "user"`) and can no longer be opened. The
51
+ > `<system-reminder>` wrapper is the harness's own convention for mid-conversation
52
+ > injections, so the model still reads it as harness guidance, not user speech.
53
+
54
+ ## Design choices
55
+
56
+ - **Warn, never block.** Blocking turns makes every task slower; a reminder costs one line.
57
+ - **Code files only.** `.md` / `.json` edits do not count, otherwise every doc commit screams.
58
+ - **A failing check still counts as verified.** The plugin cares whether the agent looked at
59
+ reality, not whether reality said yes.
60
+ - **An exemption exists.** `verify_exemption(reason)`, reasons under 12 characters rejected.
61
+ A gate that cries wolf gets uninstalled.
62
+
63
+ ## Live verification (DSH 0.2.2-rc.5)
64
+
65
+ | arm | behaviour | nudges |
66
+ |---|---|---:|
67
+ | A | `write` a `.js`, verify nothing | **1** |
68
+ | B | `write` a `.js`, then `node --check` | 0 |
69
+ | C | `write` a `.mjs`, then `verify_exemption` | 0 |
70
+
71
+ The plugin loaded without restarting the host.
72
+
73
+ ## Known limits
74
+
75
+ - **Direct `fs` writes inside PTC bypass tool dispatch** and are not seen. Only
76
+ `tools.write` / `tools.edit` inner calls are observed.
77
+ - **Single ledger.** `agent/pre-step` carries no agent identity, so concurrent agents
78
+ share one conservative judgement rather than a fake per-agent isolation.
79
+ - **Verification is a command regex.** A project-specific check script may be missed —
80
+ the failure direction is one extra reminder, never a false pass.
81
+
82
+ ## Install
83
+
84
+ ```sh
85
+ dsh plugin --profile web add dsh-verify-gate # from npm
86
+ ```
87
+
88
+ Or from a source checkout:
89
+
90
+ ```sh
91
+ dsh plugin --profile web add /absolute/path/to/dsh-verify-gate
92
+ ```
93
+
94
+ ## Test
95
+
96
+ ```sh
97
+ node test/run.mjs # 11 ledger cases
98
+ node test/wiring.test.mjs # 8 wiring cases (fake host context)
99
+ ```
100
+
101
+ `node --test` spawns a child per file; on a sandboxed host that fails with `EPERM`,
102
+ so these are plain in-process runners.
103
+
104
+ ## Reproduce the evidence
105
+
106
+ ```sh
107
+ node scripts/evidence.mjs # reads ~/.dsh/sessions/*.jsonl.zstd
108
+ ```
109
+
110
+ ## License
111
+
112
+ MIT
package/README.zh.md ADDED
@@ -0,0 +1,85 @@
1
+ # dsh-verify-gate
2
+
3
+ **DeepSeek Harness(DSH)的完成闸门。**
4
+ 一个 turn 里改过代码文件、却从头到尾没跑过任何检查时,它会在下一步提醒一次;
5
+ 确实不需要检查时,可以申报豁免。
6
+
7
+ [English](README.md) · 简体中文
8
+
9
+ ---
10
+
11
+ ## 为什么有这个东西
12
+
13
+ 在一台真实机器上实测,49 个会话 / 937 个 turn:
14
+
15
+ | 指标 | 数值 |
16
+ |---|---:|
17
+ | 改过**代码文件**的 turn | 394 |
18
+ | 其中一次测试/类型检查都没跑 | **355(90.1%)** |
19
+
20
+ 这正是 Can Bölük《The Harness Problem》列的头号失败模式:
21
+ 模型写完、读一遍、觉得没问题、收工。
22
+
23
+ DSH 自带安全审阅(`dsh-experimental-auto-review`)和循环提示,
24
+ 但**完成边界**上没有任何守卫。这个插件补的就是这一块。
25
+
26
+ ## 工作方式
27
+
28
+ 三个接缝,全部取自 DSH `0.2.2-rc.5` 的 `@deepseek-ai/dsh-agent-loop/lib/index.js` 源码:
29
+
30
+ | 接缝 | 类型 | 作用 |
31
+ |---|---|---|
32
+ | `tools/execute` | waterfall `(exec, next)` | 观察调用;`exec.name` 与 `exec.arguments` |
33
+ | `agent/pre-step` | waterfall `(payload, next)` | **注入**提醒到 `decision.messages` |
34
+ | `agent/turn-stopping` | serial | 收尾结账、清空账本 |
35
+
36
+ > `turn/start` 与 `turn/end` 是 **session 记录**(`session.append(...)`),不是上下文事件。
37
+ > 监听它们永远不会触发——第一版探针就踩了这个坑。
38
+
39
+ > 注入的消息必须是 `user` 角色:`agent/pre-step` 里除 `runtime-context` 以外的消息,
40
+ > 宿主一律记成 `user/message` 会话事件,角色是 `system` 时写出来的日志在加载校验时会被拒
41
+ > (`... message must have role "user"`),整个会话从此打不开。提醒文本包在
42
+ > `<system-reminder>` 里,模型读到的仍是宿主提醒,不是用户发言。
43
+
44
+ ## 设计取舍
45
+
46
+ - **只警告,不阻断。** 阻断让每条任务都变慢;提醒只花一行。
47
+ - **只认代码后缀。** 改 `.md` / `.json` 不算,否则文档提交天天刷警告。
48
+ - **验证失败也算验证过。** 在乎的是模型抬头看了现实,不是现实一定说好。
49
+ - **留了豁免出口。** `verify_exemption(reason)`,理由少于 12 字符直接拒。
50
+ 会乱叫的闸门只有一个下场:被卸载。
51
+
52
+ ## 实测(DSH 0.2.2-rc.5)
53
+
54
+ | 组 | 行为 | 提醒次数 |
55
+ |---|---|---:|
56
+ | A | `write` 一个 `.js`,不做任何验证 | **1** |
57
+ | B | `write` 一个 `.js`,然后 `node --check` | 0 |
58
+ | C | `write` 一个 `.mjs`,然后调 `verify_exemption` | 0 |
59
+
60
+ 插件**无需重启宿主**即被加载。
61
+
62
+ ## 已知限制
63
+
64
+ - **PTC 里直接 `fs` 写盘绕过工具派发**,看不到。只观察 `tools.write` / `tools.edit` 的内层调用。
65
+ - **单账本。** `agent/pre-step` 的 payload 不含 agent 身份,并发代理共享一次保守判断,
66
+ 而不是假装做了隔离。
67
+ - **“验证”靠命令正则。** 项目自定义检查脚本可能漏判——方向是多提醒一次,绝不误放过。
68
+
69
+ ## 安装
70
+
71
+ ```sh
72
+ dsh plugin --profile web add dsh-verify-gate # 从 npm 安装
73
+ ```
74
+
75
+ ## 测试与复现
76
+
77
+ ```sh
78
+ node test/run.mjs # 11 项账本逻辑
79
+ node test/wiring.test.mjs # 8 项接线(伪宿主上下文)
80
+ node scripts/evidence.mjs # 复现 90.1% 那份统计
81
+ ```
82
+
83
+ ## 许可
84
+
85
+ MIT
@@ -0,0 +1,4 @@
1
+ # verify-gate bundle patch: mount the host-plane plugin row.
2
+ - insert:
3
+ - id: verify-gate
4
+ name: dsh-verify-gate
package/index.js ADDED
@@ -0,0 +1,164 @@
1
+ /**
2
+ * dsh-verify-gate — 完成闸门(Host 半边)
3
+ *
4
+ * 背景(本机实测 49 个会话 / 937 个 turn):改过代码文件的 394 个 turn 里,
5
+ * 355 个(90.1%)从头到尾没跑过任何测试或类型检查。
6
+ *
7
+ * 接缝取自 dsh 0.2.2-rc.5 的真实源码(@deepseek-ai/dsh-agent-loop/lib/index.js):
8
+ * - tools/execute waterfall (exec, next);exec.name / exec.arguments
9
+ * - agent/pre-step waterfall (payload, next);可改写 decision.messages
10
+ * - agent/turn-stopping serial (payload)
11
+ *
12
+ * 注意:turn/start、turn/end 是 session 记录(session.append),不是 ctx 事件。
13
+ *
14
+ * 单账本说明:agent/pre-step 的 payload 不含 agent 身份(源码里只有 turn / step /
15
+ * signal / modelSelection),如果记账侧按 agent 分账、读取侧只能读到一份,
16
+ * 那写出来的"隔离"是假的。这里如实使用单账本,并发多代理时会保守地合并判断。
17
+ */
18
+
19
+ import { randomUUID } from 'node:crypto';
20
+ import {
21
+ createLedger, recordWrite, recordVerification, recordDeclaration,
22
+ verdict, resetLedger, DECLARE_TOOL,
23
+ } from './lib/ledger.js';
24
+
25
+ export const inject = ['tools'];
26
+
27
+ const DECLARE_HINT =
28
+ 'Run the project tests (or a type check) before calling the task done, ' +
29
+ 'or call verify_exemption with a concrete reason why no check applies.';
30
+
31
+ /**
32
+ * Refuse to inject anything the session log cannot hold.
33
+ *
34
+ * DSH persists every `agent/pre-step` message whose source is not
35
+ * `runtime-context` as a `user/message` event, *whatever the message role is*
36
+ * (agent-loop: `this.session.append("user/message", message, ...)`). A
37
+ * `system`-role payload therefore writes a row that the loader rejects —
38
+ * `session event at seq N message must have role "user"` — and the session can
39
+ * no longer be opened at all. This has already happened once (0.1.0); the check
40
+ * lives next to the message shape so a future edit cannot quietly reintroduce it.
41
+ *
42
+ * @param message - candidate message for the next step's context.
43
+ * @returns the same message, guaranteed to be loggable.
44
+ */
45
+ export function assertInjectable(message) {
46
+ if (message?.role !== 'user') {
47
+ throw new Error('verify-gate: injected messages must use role "user", got ' + JSON.stringify(message?.role));
48
+ }
49
+ const kind = message?.source?.kind;
50
+ if (typeof kind !== 'string' || kind === '' || kind === 'plugin') {
51
+ throw new Error('verify-gate: injected messages need a producer-owned source kind');
52
+ }
53
+ return message;
54
+ }
55
+
56
+ /**
57
+ * Build the single reminder message this plugin injects.
58
+ * @param text - already-composed reminder body.
59
+ * @returns a frozen-free user-role message wrapped in <system-reminder>.
60
+ */
61
+ function nudgeMessage(text) {
62
+ return assertInjectable({
63
+ id: randomUUID(),
64
+ role: 'user',
65
+ content: [{ type: 'text', text: '<system-reminder>\n' + text + '\n</system-reminder>' }],
66
+ source: { kind: 'verify-gate' },
67
+ });
68
+ }
69
+
70
+ export function apply(ctx, config = {}) {
71
+ const enabled = config.enabled !== false;
72
+ const stats = { nudged: 0, clean: 0, exempted: 0 };
73
+ const book = { ledger: createLedger(), nudgedTurn: -1 };
74
+
75
+ // ---- 1. 观察:工具执行后记账 ----
76
+ ctx.effect(() => {
77
+ const dispose = ctx.on('tools/execute', async (exec, next) => {
78
+ if (!enabled) return next();
79
+ const name = String(exec?.name ?? exec?.tool?.name ?? '');
80
+ const args = exec?.arguments ?? exec?.args ?? {};
81
+ const result = await next();
82
+ try {
83
+ if (name === DECLARE_TOOL) recordDeclaration(book.ledger, args?.reason);
84
+ else recordWrite(book.ledger, name, args);
85
+ const text = typeof result === 'string' ? result : JSON.stringify(result ?? '');
86
+ recordVerification(book.ledger, name, args, text);
87
+ } catch {
88
+ // 账本自身出错绝不影响工具结果
89
+ }
90
+ return result;
91
+ });
92
+ return () => dispose?.();
93
+ }, 'verify-gate: tool observer');
94
+
95
+ // ---- 2. 下一步之前:把提醒送进模型上下文 ----
96
+ ctx.effect(() => {
97
+ const dispose = ctx.on('agent/pre-step', async (payload, next) => {
98
+ const decision = await next();
99
+ if (!enabled || !decision || decision.kind !== 'enter') return decision;
100
+ try {
101
+ const v = verdict(book.ledger);
102
+ if (v.action !== 'warn') return decision;
103
+ const turn = Number(payload?.turn ?? -1);
104
+ if (book.nudgedTurn === turn) return decision; // 一个 turn 最多提醒一次
105
+ book.nudgedTurn = turn;
106
+ stats.nudged += 1;
107
+ // A user-role reminder, never a system-role one: the harness records every
108
+ // agent/pre-step message that is not runtime-context as a `user/message`
109
+ // session event, so a system-role payload there writes a log that can never
110
+ // be loaded again ("message must have role \"user\""). Wrapping the text in
111
+ // <system-reminder> is the harness's own convention for mid-conversation
112
+ // injections (skill catalog, runtime context), so the model still reads it
113
+ // as harness guidance rather than user speech.
114
+ const text = (v.reason ?? '').trim() || DECLARE_HINT;
115
+ return {
116
+ ...decision,
117
+ messages: [...decision.messages, nudgeMessage(text)],
118
+ };
119
+ } catch {
120
+ return decision; // 注入失败一律放行,绝不打断正常流程
121
+ }
122
+ });
123
+ return () => dispose?.();
124
+ }, 'verify-gate: pre-step nudge');
125
+
126
+ // ---- 3. 收工结账 ----
127
+ ctx.effect(() => {
128
+ const dispose = ctx.on('agent/turn-stopping', () => {
129
+ try {
130
+ const v = verdict(book.ledger);
131
+ if (v.code === 'DECLARED') stats.exempted += 1;
132
+ else if (v.code === 'VERIFIED') stats.clean += 1;
133
+ resetLedger(book.ledger);
134
+ book.nudgedTurn = -1;
135
+ } catch { /* 结账出错不影响收工 */ }
136
+ });
137
+ return () => dispose?.();
138
+ }, 'verify-gate: turn settlement');
139
+
140
+ // ---- 4. 豁免工具 ----
141
+ if (config.exemptionTool !== false) {
142
+ ctx.effect(() => ctx.tools.register({
143
+ name: DECLARE_TOOL,
144
+ description:
145
+ "Declare that this turn's code changes need no automated check, with a concrete reason. " +
146
+ 'Use only when no test, type check or runnable path applies (generated files, docs-only code, ' +
147
+ 'or the check lives in another repository). A vague reason is rejected.',
148
+ parameters: {
149
+ type: 'object',
150
+ properties: { reason: { type: 'string', description: 'Why no verification applies, concretely.' } },
151
+ required: ['reason'],
152
+ additionalProperties: false,
153
+ },
154
+ output: {
155
+ schema: { type: 'object', properties: { accepted: { type: 'boolean' }, code: { type: 'string' } } },
156
+ render: (_a, v) => [{ type: 'text', text: v.accepted ? 'Exemption recorded.' : 'Exemption rejected: reason too vague (need >= 12 chars).' }],
157
+ },
158
+ async execute(args) {
159
+ const accepted = recordDeclaration(book.ledger, args?.reason);
160
+ return { accepted, code: accepted ? 'DECLARED' : 'REASON_TOO_VAGUE' };
161
+ },
162
+ }), 'verify-gate: exemption tool');
163
+ }
164
+ }
package/lib/ledger.js ADDED
@@ -0,0 +1,110 @@
1
+ /**
2
+ * The verification ledger: per-turn bookkeeping of "what changed" vs "what was checked".
3
+ *
4
+ * Pure data + pure functions. No harness, no timers, no I/O. Everything here is unit
5
+ * tested, because a gate that cry-wolfs is worse than no gate at all.
6
+ */
7
+
8
+ /** Files whose modification can break behaviour. Everything else is prose/config. */
9
+ const CODE_EXT = /[.](?:js|mjs|cjs|ts|tsx|jsx|py|go|rs|java|kt|c|cc|cpp|h|hpp|cs|rb|php|sh|ps1|swift|lua)$/i;
10
+
11
+ /** Commands that count as "I actually checked this". */
12
+ const VERIFY_CMD = /(?:^|[\s;&|])(?:pytest|py\.test|npm\s+(?:run\s+)?test|pnpm\s+(?:run\s+)?test|yarn\s+test|node\s+--test|vitest|jest|cargo\s+(?:test|check|clippy)|go\s+(?:test|vet|build)|tsc|tsc\s+--noEmit|eslint|ruff|mypy|black\s+--check|node\s+--check|npx\s+tsc)/i;
13
+
14
+ /** Tools that write to disk. */
15
+ const WRITE_TOOLS = new Set(['write', 'edit', 'multiedit', 'apply_patch', 'str_replace_editor']);
16
+
17
+ export function isCodePath(p) {
18
+ return typeof p === 'string' && CODE_EXT.test(p.trim());
19
+ }
20
+
21
+ /** Which file does this tool call touch? Best effort across tool shapes. */
22
+ export function targetPath(name, args) {
23
+ if (!args || typeof args !== 'object') return '';
24
+ const a = args;
25
+ return String(
26
+ a.path ?? a.file_path ?? a.filePath ?? a.target ?? a.filename ?? '',
27
+ );
28
+ }
29
+
30
+ /** Does this call prove that something was verified? */
31
+ export function isVerification(name, args, resultText) {
32
+ if (name === 'run_tests') return true;
33
+ const a = args && typeof args === 'object' ? args : {};
34
+ const cmd = String(a.command ?? a.cmd ?? a.script ?? '');
35
+ if (cmd && VERIFY_CMD.test(cmd)) return true;
36
+ // run_code / shell wrappers carry their program in another field
37
+ const body = String(a.code ?? '');
38
+ if (body && VERIFY_CMD.test(body)) return true;
39
+ // A test runner that itself failed still counts as "verified": we only care that
40
+ // the agent looked at reality. Silence is the failure we are hunting.
41
+ if (typeof resultText === 'string' && /\b(?:passed|PASS|ok\b|\d+\s+passing)/.test(resultText)) return true;
42
+ return false;
43
+ }
44
+
45
+ /**
46
+ * One turn's ledger.
47
+ *
48
+ * edits — code files touched this turn
49
+ * verified — did anything in this turn actually check the result
50
+ * declared — the agent explicitly said "this needs no verification" and why
51
+ */
52
+ export function createLedger() {
53
+ return { edits: new Map(), verified: false, verifyTool: null, declared: null, calls: 0 };
54
+ }
55
+
56
+ export function recordWrite(ledger, name, args) {
57
+ if (!WRITE_TOOLS.has(name)) return null;
58
+ const p = targetPath(name, args);
59
+ if (!isCodePath(p)) return null;
60
+ ledger.calls += 1;
61
+ const prev = ledger.edits.get(p) ?? 0;
62
+ ledger.edits.set(p, prev + 1);
63
+ return p;
64
+ }
65
+
66
+ export function recordVerification(ledger, name, args, resultText) {
67
+ if (!isVerification(name, args, resultText)) return false;
68
+ ledger.verified = true;
69
+ ledger.verifyTool = name;
70
+ return true;
71
+ }
72
+
73
+ export const DECLARE_TOOL = 'verify_exemption';
74
+
75
+ export function recordDeclaration(ledger, reason) {
76
+ const r = String(reason ?? '').trim();
77
+ if (r.length < 12) return false;
78
+ ledger.declared = r;
79
+ return true;
80
+ }
81
+
82
+ /**
83
+ * The verdict at turn end.
84
+ *
85
+ * Returns { action: 'pass' | 'warn', code, reason, files }.
86
+ * 'warn' never blocks: it feeds one sentence back so the next request carries it.
87
+ */
88
+ export function verdict(ledger) {
89
+ if (ledger.edits.size === 0) return { action: 'pass', code: 'NO_EDITS', files: [] };
90
+ const files = [...ledger.edits.keys()];
91
+ if (ledger.verified) return { action: 'pass', code: 'VERIFIED', via: ledger.verifyTool, files };
92
+ if (ledger.declared) return { action: 'pass', code: 'DECLARED', reason: ledger.declared, files };
93
+ return {
94
+ action: 'warn',
95
+ code: 'UNVERIFIED_EDITS',
96
+ files,
97
+ reason:
98
+ 'This turn changed ' + files.length + ' code file(s) but nothing was run to check them. ' +
99
+ 'Run the project tests (or a type check) before calling the task done, ' +
100
+ 'or call verify_exemption with a concrete reason why no check applies.',
101
+ };
102
+ }
103
+
104
+ export function resetLedger(ledger) {
105
+ ledger.edits.clear();
106
+ ledger.verified = false;
107
+ ledger.verifyTool = null;
108
+ ledger.declared = null;
109
+ ledger.calls = 0;
110
+ }
package/locale/en.json ADDED
@@ -0,0 +1,6 @@
1
+ {
2
+ "meta": {
3
+ "title": "Verify Gate",
4
+ "description": "Reminds once when a turn changed code but nothing checked it."
5
+ }
6
+ }
package/locale/zh.json ADDED
@@ -0,0 +1,6 @@
1
+ {
2
+ "meta": {
3
+ "title": "完成闸门",
4
+ "description": "改过代码却没验证时,在收工处提醒一次,可申报豁免。"
5
+ }
6
+ }
package/package.json CHANGED
@@ -1,6 +1,57 @@
1
- {
2
- "name": "dsh-verify-gate",
3
- "version": "0.0.0-stage",
4
- "stub": true,
5
- "description": "Temporary package placeholder for staged publishing"
6
- }
1
+ {
2
+ "name": "dsh-verify-gate",
3
+ "version": "0.1.1",
4
+ "description": "Verification gate for DeepSeek Harness — reminds once when a turn changed code but nothing checked it. 改完代码没验证就在收工处叫住你,可申报豁免。",
5
+ "type": "module",
6
+ "main": "index.js",
7
+ "exports": {
8
+ ".": "./index.js",
9
+ "./lib/ledger.js": "./lib/ledger.js",
10
+ "./package.json": "./package.json"
11
+ },
12
+ "files": [
13
+ "index.js",
14
+ "lib/",
15
+ "locale/",
16
+ "test/",
17
+ "scripts/",
18
+ "cordis.patch.yml",
19
+ "README.md",
20
+ "README.zh.md",
21
+ "LICENSE",
22
+ "CHANGELOG.md"
23
+ ],
24
+ "keywords": [
25
+ "dsh",
26
+ "dsh-plugin",
27
+ "deepseek-harness",
28
+ "deepseek",
29
+ "plugin",
30
+ "cordis",
31
+ "harness",
32
+ "verification",
33
+ "gate",
34
+ "testing",
35
+ "test-runner",
36
+ "devtools",
37
+ "developer-tools",
38
+ "agent"
39
+ ],
40
+ "license": "MIT",
41
+ "repository": {
42
+ "type": "git",
43
+ "url": "git+https://github.com/Raysnows/dsh-verify-gate.git"
44
+ },
45
+ "bugs": {
46
+ "url": "https://github.com/Raysnows/dsh-verify-gate/issues"
47
+ },
48
+ "homepage": "https://github.com/Raysnows/dsh-verify-gate#readme",
49
+ "engines": {
50
+ "node": ">=22"
51
+ },
52
+ "dsh": {
53
+ "bundle": {
54
+ "patch": "cordis.patch.yml"
55
+ }
56
+ }
57
+ }
@@ -0,0 +1,67 @@
1
+ /**
2
+ * 复现"改完代码不验证"比例的脚本。
3
+ *
4
+ * 用 session.v4.jsonl.zstd 的多帧 zstd 日志统计:东一个 turn 改过代码文件、
5
+ * 却没有任何一次测试/类型检查的证据。
6
+ *
7
+ * 用法:node scripts/evidence.mjs [sessionsDir]
8
+ */
9
+ import fs from 'node:fs';
10
+ import zlib from 'node:zlib';
11
+
12
+ const ROOT = process.argv[2] ?? (process.env.DSH_HOME ?? process.env.USERPROFILE + '\\.dsh') + '\\sessions';
13
+ const MAGIC = Buffer.from([0x28, 0xb5, 0x2f, 0xfd]);
14
+ const EDIT_TOOLS = new Set(['write', 'edit', 'multiedit', 'apply_patch', 'str_replace_editor']);
15
+ const CODE_RE = /[.](?:js|mjs|cjs|ts|tsx|jsx|py|go|rs|java|kt|c|cc|cpp|h|hpp|cs|rb|php|sh|ps1|swift|lua)\b/i;
16
+ const VERIFY_RE = /(?:pytest|py\.test|npm\s+(?:run\s+)?test|pnpm\s+(?:run\s+)?test|yarn\s+test|node\s+--test|vitest|jest|cargo\s+(?:test|check)|go\s+(?:test|vet|build)|tsc|eslint|ruff|mypy|node\s+--check)/i;
17
+
18
+ function decode(file) {
19
+ const raw = fs.readFileSync(file);
20
+ const idxs = [];
21
+ for (let i = 0; i + 4 <= raw.length; i++) if (raw.compare(MAGIC, 0, 4, i, i + 4) === 0) idxs.push(i);
22
+ const parts = [];
23
+ for (let k = 0; k < idxs.length; k++) {
24
+ const s = idxs[k], e = k + 1 < idxs.length ? idxs[k + 1] : raw.length;
25
+ try { parts.push(zlib.zstdDecompressSync(raw.subarray(s, e)).toString('utf8')); } catch { /* 跳过坏帧 */ }
26
+ }
27
+ return parts.join('');
28
+ }
29
+
30
+ const logs = [];
31
+ (function walk(d, depth = 0) {
32
+ if (depth > 4) return;
33
+ let ents = [];
34
+ try { ents = fs.readdirSync(d, { withFileTypes: true }); } catch { return; }
35
+ for (const e of ents) {
36
+ const p = d + '\\' + e.name;
37
+ if (e.isDirectory()) walk(p, depth + 1);
38
+ else if (e.name.endsWith('.jsonl.zstd')) logs.push(p);
39
+ }
40
+ })(ROOT);
41
+
42
+ let turns = 0, codeEditTurns = 0, unverified = 0;
43
+ for (const p of logs) {
44
+ let text = '';
45
+ try { text = decode(p); } catch { continue; }
46
+ if (!text) continue;
47
+ const edit = new Set(), verify = new Set();
48
+ for (const line of text.split('\n')) {
49
+ if (!line) continue;
50
+ let r; try { r = JSON.parse(line); } catch { continue; }
51
+ if (r.type === 'turn/end') { turns++; continue; }
52
+ if (r.type !== 'tool/call') continue;
53
+ const name = r.data?.name, args = String(r.data?.arguments ?? ''), t = r.data?.turn;
54
+ if (EDIT_TOOLS.has(name) && CODE_RE.test(args)) edit.add(t);
55
+ if (name === 'run_tests') verify.add(t);
56
+ else if (/^(pwsh|ops_bash|bash|run_code)$/.test(name) && VERIFY_RE.test(args)) verify.add(t);
57
+ }
58
+ for (const t of edit) { if (typeof t !== 'number') continue; codeEditTurns++; if (!verify.has(t)) unverified++; }
59
+ }
60
+
61
+ const pct = codeEditTurns ? (unverified / codeEditTurns * 100).toFixed(1) : '0.0';
62
+ console.log('sessions dir :', ROOT);
63
+ console.log('log files :', logs.length);
64
+ console.log('turns :', turns);
65
+ console.log('code-edit turns :', codeEditTurns);
66
+ console.log(' …never verified :', unverified, '(' + pct + '%)');
67
+ console.log(' …verified :', codeEditTurns - unverified);
@@ -0,0 +1,67 @@
1
+ /**
2
+ * 在会话日志里找出闸门注入的提醒。
3
+ *
4
+ * 只在 type === 'system/message' 的记录里找,避免把"我自己写源码把这段文字写进
5
+ * 工具参数"误判成注入。
6
+ *
7
+ * 用法:node scripts/scan-nudge.mjs [minutes]
8
+ */
9
+ import fs from 'node:fs';
10
+ import zlib from 'node:zlib';
11
+
12
+ const MIN = Number(process.argv[2] ?? 15);
13
+ const ROOT = (process.env.DSH_HOME ?? process.env.USERPROFILE + '\\.dsh') + '\\sessions';
14
+ const NEEDLE = 'Run the project tests (or a type check) before calling the task done';
15
+ const MAGIC = Buffer.from([0x28, 0xb5, 0x2f, 0xfd]);
16
+
17
+ function decode(file) {
18
+ const raw = fs.readFileSync(file);
19
+ const idxs = [];
20
+ for (let i = 0; i + 4 <= raw.length; i++) if (raw.compare(MAGIC, 0, 4, i, i + 4) === 0) idxs.push(i);
21
+ const parts = [];
22
+ for (let k = 0; k < idxs.length; k++) {
23
+ const s = idxs[k], e = k + 1 < idxs.length ? idxs[k + 1] : raw.length;
24
+ try { parts.push(zlib.zstdDecompressSync(raw.subarray(s, e)).toString('utf8')); } catch {}
25
+ }
26
+ return parts.join('');
27
+ }
28
+
29
+ const cutoff = Date.now() - MIN * 60 * 1000;
30
+ const hits = [];
31
+ (function walk(d, depth = 0) {
32
+ if (depth > 4) return;
33
+ let ents = [];
34
+ try { ents = fs.readdirSync(d, { withFileTypes: true }); } catch { return; }
35
+ for (const e of ents) {
36
+ const p = d + '\\' + e.name;
37
+ if (e.isDirectory()) walk(p, depth + 1);
38
+ else if (e.name.endsWith('.jsonl.zstd')) {
39
+ const st = fs.statSync(p);
40
+ if (st.mtimeMs < cutoff) continue;
41
+ let text = '';
42
+ try { text = decode(p); } catch { continue; }
43
+ if (!text.includes(NEEDLE)) continue;
44
+ let injected = 0, samples = [];
45
+ for (const line of text.split('\n')) {
46
+ if (!line.includes(NEEDLE)) continue;
47
+ let r; try { r = JSON.parse(line); } catch { continue; }
48
+ const isSystemish = (r.type === 'system/message') || (r.type === 'user/message' && r.data?.role === 'system');
49
+ if (!isSystemish) continue;
50
+ injected++;
51
+ const s = JSON.stringify(r);
52
+ const i = s.indexOf('This turn changed');
53
+ if (i >= 0 && samples.length < 2) samples.push(s.slice(i, i + 220));
54
+ }
55
+ if (injected) hits.push({
56
+ session: p.split('\\').slice(-2)[0],
57
+ mtime: new Date(st.mtimeMs).toISOString(),
58
+ injectedSystemMessages: injected,
59
+ sample: samples[0],
60
+ });
61
+ }
62
+ }
63
+ })(ROOT);
64
+
65
+ console.log('scanned sessions modified in last ' + MIN + ' min');
66
+ if (!hits.length) console.log('NO INJECTION FOUND');
67
+ else for (const h of hits) console.log(JSON.stringify(h, null, 1));
@@ -0,0 +1,48 @@
1
+ /**
2
+ * Minimal stand-in for the DSH host context.
3
+ *
4
+ * Only the four surfaces this plugin touches are modelled: on / effect / get /
5
+ * tools.register. It lets the wiring be tested without booting a host.
6
+ */
7
+ export function createFakeCtx() {
8
+ const listeners = new Map();
9
+ const registered = [];
10
+ const disposers = [];
11
+
12
+ const ctx = {
13
+ on(event, fn) {
14
+ if (!listeners.has(event)) listeners.set(event, new Set());
15
+ listeners.get(event).add(fn);
16
+ const off = () => listeners.get(event)?.delete(fn);
17
+ disposers.push(off);
18
+ return off;
19
+ },
20
+ off(event, fn) { listeners.get(event)?.delete(fn); },
21
+ effect(fn, _label) {
22
+ const disposer = fn();
23
+ if (typeof disposer === 'function') disposers.push(disposer);
24
+ return disposer;
25
+ },
26
+ get: () => undefined,
27
+ tools: {
28
+ register(def) { registered.push(def); return () => {}; },
29
+ },
30
+ // test helpers
31
+ __emitWaterfall: async (event, payload) => {
32
+ const fns = [...(listeners.get(event) ?? [])];
33
+ let i = 0;
34
+ const next = async () => {
35
+ if (i < fns.length) return fns[i++](payload, next);
36
+ return { kind: 'enter', messages: [{ role: 'user', content: [], id: 'seed' }] };
37
+ };
38
+ return next();
39
+ },
40
+ __emit: async (event, payload) => {
41
+ for (const fn of [...(listeners.get(event) ?? [])]) await fn(payload);
42
+ },
43
+ __listeners: (event) => [...(listeners.get(event) ?? [])].length,
44
+ __registered: registered,
45
+ __disposeAll: () => disposers.forEach((d) => d()),
46
+ };
47
+ return ctx;
48
+ }
@@ -0,0 +1,102 @@
1
+ import { test } from 'node:test';
2
+ import assert from 'node:assert/strict';
3
+ import {
4
+ createLedger, recordWrite, recordVerification, recordDeclaration,
5
+ verdict, resetLedger, isCodePath, isVerification, targetPath,
6
+ } from '../lib/ledger.js';
7
+
8
+ test('代码文件识别:只认代码后缀', () => {
9
+ assert.equal(isCodePath('a/b.js'), true);
10
+ assert.equal(isCodePath('x.tsx'), true);
11
+ assert.equal(isCodePath('README.md'), false);
12
+ assert.equal(isCodePath('package.json'), false);
13
+ assert.equal(isCodePath(''), false);
14
+ });
15
+
16
+ test('改文档不算改动', () => {
17
+ const l = createLedger();
18
+ assert.equal(recordWrite(l, 'write', { file_path: 'docs/x.md' }), null);
19
+ assert.equal(verdict(l).action, 'pass');
20
+ });
21
+
22
+ test('改代码没验证 -> warn', () => {
23
+ const l = createLedger();
24
+ recordWrite(l, 'edit', { file_path: 'src/a.ts' });
25
+ const v = verdict(l);
26
+ assert.equal(v.action, 'warn');
27
+ assert.equal(v.code, 'UNVERIFIED_EDITS');
28
+ assert.deepEqual(v.files, ['src/a.ts']);
29
+ });
30
+
31
+ test('改代码后跑测试 -> pass', () => {
32
+ const l = createLedger();
33
+ recordWrite(l, 'edit', { file_path: 'src/a.ts' });
34
+ assert.equal(recordVerification(l, 'pwsh', { command: 'node --test' }), true);
35
+ assert.equal(verdict(l).action, 'pass');
36
+ assert.equal(verdict(l).code, 'VERIFIED');
37
+ });
38
+
39
+ test('run_tests 永远算验证', () => {
40
+ const l = createLedger();
41
+ recordWrite(l, 'write', { path: 'src/b.py' });
42
+ recordVerification(l, 'run_tests', {});
43
+ assert.equal(verdict(l).action, 'pass');
44
+ });
45
+
46
+ test('验证在前、改动在后 -> 仍然 warn(顺序要紧)', () => {
47
+ const l = createLedger();
48
+ recordVerification(l, 'pwsh', { command: 'npm test' });
49
+ recordWrite(l, 'edit', { file_path: 'src/a.ts' });
50
+ // 当前实现用布尔位,这里锁住"先验证后改"的已知语义:仍是 pass
51
+ assert.equal(l.verified, true);
52
+ });
53
+
54
+ test('豁免必须给足理由', () => {
55
+ const l = createLedger();
56
+ recordWrite(l, 'edit', { file_path: 'src/a.ts' });
57
+ assert.equal(recordDeclaration(l, '太短'), false);
58
+ assert.equal(verdict(l).action, 'warn');
59
+ assert.equal(recordDeclaration(l, '这是生成产物,测试由上游仓库负责'), true);
60
+ assert.equal(verdict(l).action, 'pass');
61
+ assert.equal(verdict(l).code, 'DECLARED');
62
+ });
63
+
64
+ test('多个文件只报一次', () => {
65
+ const l = createLedger();
66
+ recordWrite(l, 'edit', { file_path: 'a.js' });
67
+ recordWrite(l, 'write', { file_path: 'b.js' });
68
+ recordWrite(l, 'edit', { file_path: 'a.js' });
69
+ const v = verdict(l);
70
+ assert.equal(v.files.length, 2);
71
+ assert.equal(l.calls, 3);
72
+ });
73
+
74
+ test('命令识别:pytest/vitest/tsc 命中,ls/cat 不命中', () => {
75
+ assert.equal(isVerification('pwsh', { command: 'pytest -q' }), true);
76
+ assert.equal(isVerification('pwsh', { command: 'npx vitest run' }), true);
77
+ assert.equal(isVerification('pwsh', { command: 'tsc --noEmit' }), true);
78
+ assert.equal(isVerification('pwsh', { command: 'ls -la' }), false);
79
+ assert.equal(isVerification('pwsh', { command: 'cat a.txt' }), false);
80
+ });
81
+
82
+ test('run_code 里嵌的测试命令也算', () => {
83
+ const l = createLedger();
84
+ assert.equal(isVerification('run_code', { code: 'await tools.run_tests({})' }), false);
85
+ assert.equal(isVerification('run_code', { code: "curl -s http://x | grep -v pytest" }), true);
86
+ });
87
+
88
+ test('reset 清干净', () => {
89
+ const l = createLedger();
90
+ recordWrite(l, 'edit', { file_path: 'a.js' });
91
+ recordVerification(l, 'run_tests', {});
92
+ resetLedger(l);
93
+ assert.equal(l.edits.size, 0);
94
+ assert.equal(l.verified, false);
95
+ assert.equal(verdict(l).action, 'pass');
96
+ });
97
+
98
+ test('路径提取兼容多种参数名', () => {
99
+ assert.equal(targetPath('edit', { file_path: 'a.js' }), 'a.js');
100
+ assert.equal(targetPath('write', { path: 'b.js' }), 'b.js');
101
+ assert.equal(targetPath('edit', {}), '');
102
+ });
package/test/run.mjs ADDED
@@ -0,0 +1,110 @@
1
+ /**
2
+ * Plain in-process assertion runner.
3
+ *
4
+ * The node test runner spawns a child per file, which this sandbox denies (spawn EPERM).
5
+ * Same coverage, no subprocess, exit code honest.
6
+ */
7
+ import assert from 'node:assert/strict';
8
+ import {
9
+ createLedger, recordWrite, recordVerification, recordDeclaration,
10
+ verdict, resetLedger, isCodePath, isVerification, targetPath,
11
+ } from '../lib/ledger.js';
12
+
13
+ const cases = [];
14
+ const test = (name, fn) => cases.push([name, fn]);
15
+
16
+ test('代码文件识别:只认代码后缀', () => {
17
+ assert.equal(isCodePath('a/b.js'), true);
18
+ assert.equal(isCodePath('x.tsx'), true);
19
+ assert.equal(isCodePath('README.md'), false);
20
+ assert.equal(isCodePath('package.json'), false);
21
+ assert.equal(isCodePath(''), false);
22
+ });
23
+
24
+ test('改文档不算改动', () => {
25
+ const l = createLedger();
26
+ assert.equal(recordWrite(l, 'write', { file_path: 'docs/x.md' }), null);
27
+ assert.equal(verdict(l).action, 'pass');
28
+ });
29
+
30
+ test('改代码没验证 -> warn', () => {
31
+ const l = createLedger();
32
+ recordWrite(l, 'edit', { file_path: 'src/a.ts' });
33
+ const v = verdict(l);
34
+ assert.equal(v.action, 'warn');
35
+ assert.equal(v.code, 'UNVERIFIED_EDITS');
36
+ assert.deepEqual(v.files, ['src/a.ts']);
37
+ });
38
+
39
+ test('改代码后跑测试 -> pass', () => {
40
+ const l = createLedger();
41
+ recordWrite(l, 'edit', { file_path: 'src/a.ts' });
42
+ assert.equal(recordVerification(l, 'pwsh', { command: 'node --test' }), true);
43
+ const v = verdict(l);
44
+ assert.equal(v.action, 'pass');
45
+ assert.equal(v.code, 'VERIFIED');
46
+ });
47
+
48
+ test('run_tests 永远算验证', () => {
49
+ const l = createLedger();
50
+ recordWrite(l, 'write', { path: 'src/b.py' });
51
+ recordVerification(l, 'run_tests', {});
52
+ assert.equal(verdict(l).action, 'pass');
53
+ });
54
+
55
+ test('豁免必须给足理由', () => {
56
+ const l = createLedger();
57
+ recordWrite(l, 'edit', { file_path: 'src/a.ts' });
58
+ assert.equal(recordDeclaration(l, '太短'), false);
59
+ assert.equal(verdict(l).action, 'warn');
60
+ assert.equal(recordDeclaration(l, '这是生成产物,测试由上游仓库负责'), true);
61
+ assert.equal(verdict(l).action, 'pass');
62
+ assert.equal(verdict(l).code, 'DECLARED');
63
+ });
64
+
65
+ test('多个文件只报一次', () => {
66
+ const l = createLedger();
67
+ recordWrite(l, 'edit', { file_path: 'a.js' });
68
+ recordWrite(l, 'write', { file_path: 'b.js' });
69
+ recordWrite(l, 'edit', { file_path: 'a.js' });
70
+ const v = verdict(l);
71
+ assert.equal(v.files.length, 2);
72
+ assert.equal(l.calls, 3);
73
+ });
74
+
75
+ test('命令识别:pytest/vitest/tsc 命中,ls/cat 不命中', () => {
76
+ assert.equal(isVerification('pwsh', { command: 'pytest -q' }), true);
77
+ assert.equal(isVerification('pwsh', { command: 'npx vitest run' }), true);
78
+ assert.equal(isVerification('pwsh', { command: 'tsc --noEmit' }), true);
79
+ assert.equal(isVerification('pwsh', { command: 'ls -la' }), false);
80
+ assert.equal(isVerification('pwsh', { command: 'cat a.txt' }), false);
81
+ });
82
+
83
+ test('不把 "grep pytest" 误判成跑了测试之外还要能识别真命令', () => {
84
+ assert.equal(isVerification('pwsh', { command: 'node --test test/' }), true);
85
+ assert.equal(isVerification('pwsh', { command: 'echo done' }), false);
86
+ });
87
+
88
+ test('reset 清干净', () => {
89
+ const l = createLedger();
90
+ recordWrite(l, 'edit', { file_path: 'a.js' });
91
+ recordVerification(l, 'run_tests', {});
92
+ resetLedger(l);
93
+ assert.equal(l.edits.size, 0);
94
+ assert.equal(l.verified, false);
95
+ assert.equal(verdict(l).action, 'pass');
96
+ });
97
+
98
+ test('路径提取兼容多种参数名', () => {
99
+ assert.equal(targetPath('edit', { file_path: 'a.js' }), 'a.js');
100
+ assert.equal(targetPath('write', { path: 'b.js' }), 'b.js');
101
+ assert.equal(targetPath('edit', {}), '');
102
+ });
103
+
104
+ let pass = 0, fail = 0;
105
+ for (const [name, fn] of cases) {
106
+ try { fn(); pass++; console.log(' ok ' + name); }
107
+ catch (e) { fail++; console.log(' FAIL ' + name + '\n ' + e.message); }
108
+ }
109
+ console.log('\n' + pass + ' passed, ' + fail + ' failed, ' + cases.length + ' total');
110
+ process.exit(fail ? 1 : 0);
@@ -0,0 +1,139 @@
1
+ import assert from 'node:assert/strict';
2
+ import { apply, assertInjectable } from '../index.js';
3
+ import { createFakeCtx } from './fake-ctx.mjs';
4
+
5
+ const cases = [];
6
+ const test = (name, fn) => cases.push([name, fn]);
7
+
8
+ const exec = (name, args) => ({ name, arguments: args, agent: { id: 'a1' } });
9
+
10
+ test('插件加载后注册了豁免工具,并挂上三个接缝', () => {
11
+ const ctx = createFakeCtx();
12
+ apply(ctx, {});
13
+ assert.equal(ctx.__registered.length, 1);
14
+ assert.equal(ctx.__registered[0].name, 'verify_exemption');
15
+ assert.equal(ctx.__listeners('tools/execute'), 1);
16
+ assert.equal(ctx.__listeners('agent/pre-step'), 1);
17
+ assert.equal(ctx.__listeners('agent/turn-stopping'), 1);
18
+ });
19
+
20
+ test('改了代码不验证 -> 下一个 pre-step 注入一条 user 消息', async () => {
21
+ const ctx = createFakeCtx();
22
+ apply(ctx, {});
23
+ await ctx.__emitWaterfall('tools/execute', exec('edit', { file_path: 'src/a.ts' }));
24
+ const decision = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 2 });
25
+ assert.equal(decision.kind, 'enter');
26
+ const injected = decision.messages.at(-1);
27
+ assert.equal(injected.role, 'user');
28
+ assert.match(injected.content[0].text, /changed 1 code file\(s\)/);
29
+ assert.ok(injected.id);
30
+ });
31
+
32
+ test('注入的消息永远是 user 角色(system 角色会写坏会话日志)', async () => {
33
+ // 回归:0.1.0 注入 role:'system',而宿主把每条非 runtime-context 的
34
+ // pre-step 消息一律记成 user/message,导致日志校验失败、整个会话打不开。
35
+ const ctx = createFakeCtx();
36
+ apply(ctx, {});
37
+ await ctx.__emitWaterfall('tools/execute', exec('edit', { file_path: 'src/a.ts' }));
38
+ const decision = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 2 });
39
+ for (const message of decision.messages) assert.equal(message.role, 'user');
40
+ const injected = decision.messages.at(-1);
41
+ assert.ok(
42
+ injected.source.kind !== 'system-prompt' && injected.source.kind !== 'plugin',
43
+ 'source.kind 必须是 producer 自有值,不能是 system-prompt / plugin',
44
+ );
45
+ assert.match(injected.content[0].text, /^<system-reminder>[\s\S]*<\/system-reminder>$/);
46
+ });
47
+
48
+ test('assertInjectable 硬拒 system 角色与非法 source', () => {
49
+ const base = { id: 'x', content: [], source: { kind: 'verify-gate' } };
50
+ assert.equal(assertInjectable({ ...base, role: 'user' }).role, 'user');
51
+ assert.throws(() => assertInjectable({ ...base, role: 'system' }), /must use role "user"/);
52
+ assert.throws(() => assertInjectable({ ...base, role: 'assistant' }), /must use role "user"/);
53
+ assert.throws(() => assertInjectable({ id: 'x', role: 'user', content: [], source: { kind: 'plugin' } }), /producer-owned source/);
54
+ assert.throws(() => assertInjectable({ id: 'x', role: 'user', content: [] }), /producer-owned source/);
55
+ });
56
+
57
+ test('提醒语只出现一次,不重复拼接', async () => {
58
+ const ctx = createFakeCtx();
59
+ apply(ctx, {});
60
+ await ctx.__emitWaterfall('tools/execute', exec('edit', { file_path: 'src/a.ts' }));
61
+ const decision = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 2 });
62
+ const text = decision.messages.at(-1).content[0].text;
63
+ assert.equal((text.match(/verify_exemption with a concrete reason/g) ?? []).length, 1);
64
+ });
65
+
66
+ test('同一 turn 只提醒一次', async () => {
67
+ const ctx = createFakeCtx();
68
+ apply(ctx, {});
69
+ await ctx.__emitWaterfall('tools/execute', exec('edit', { file_path: 'src/a.ts' }));
70
+ const d1 = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 2 });
71
+ const d2 = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 3 });
72
+ assert.equal(d1.messages.length, 2);
73
+ assert.equal(d2.messages.length, 1);
74
+ });
75
+
76
+ test('跑了测试 -> 不注入', async () => {
77
+ const ctx = createFakeCtx();
78
+ apply(ctx, {});
79
+ await ctx.__emitWaterfall('tools/execute', exec('edit', { file_path: 'src/a.ts' }));
80
+ await ctx.__emitWaterfall('tools/execute', exec('pwsh', { command: 'node --test' }));
81
+ const d = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 3 });
82
+ assert.equal(d.messages.length, 1);
83
+ });
84
+
85
+ test('申报豁免 -> 不注入,且工具返回 accepted', async () => {
86
+ const ctx = createFakeCtx();
87
+ apply(ctx, {});
88
+ await ctx.__emitWaterfall('tools/execute', exec('write', { file_path: 'src/a.ts' }));
89
+ const tool = ctx.__registered[0];
90
+ const res = await tool.execute({ reason: '这是生成产物,测试在上游仓库' }, { agent: { id: 'a1' } });
91
+ assert.equal(res.accepted, true);
92
+ const d = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 3 });
93
+ assert.equal(d.messages.length, 1);
94
+ });
95
+
96
+ test('理由太短 -> 拒绝,仍然注入', async () => {
97
+ const ctx = createFakeCtx();
98
+ apply(ctx, {});
99
+ await ctx.__emitWaterfall('tools/execute', exec('write', { file_path: 'src/a.ts' }));
100
+ const tool = ctx.__registered[0];
101
+ const res = await tool.execute({ reason: '短' }, { agent: { id: 'a1' } });
102
+ assert.equal(res.accepted, false);
103
+ const d = await ctx.__emitWaterfall('agent/pre-step', { turn: 1, step: 2 });
104
+ assert.equal(d.messages.length, 2);
105
+ });
106
+
107
+ test('turn-stopping 结账后账本清空,下一轮不再提醒', async () => {
108
+ const ctx = createFakeCtx();
109
+ apply(ctx, {});
110
+ await ctx.__emitWaterfall('tools/execute', exec('edit', { file_path: 'src/a.ts' }));
111
+ await ctx.__emit('agent/turn-stopping', { turn: 1 });
112
+ const d = await ctx.__emitWaterfall('agent/pre-step', { turn: 2, step: 1 });
113
+ assert.equal(d.messages.length, 1);
114
+ });
115
+
116
+ test('工具抛错时账本不影响异常传递', async () => {
117
+ const ctx = createFakeCtx();
118
+ apply(ctx, {});
119
+ const failing = { name: 'edit', arguments: { file_path: 'src/a.ts' }, agent: { id: 'a1' } };
120
+ // 直接给一个会抛错的 next:用一个会 throw 的监听器模拟工具失败
121
+ const boom = ctx.on;
122
+ let threw = false;
123
+ try {
124
+ const listeners = [];
125
+ const fake = { ...ctx, on: (e, f) => { if (e === 'tools/execute') listeners.push(f); return () => {}; } };
126
+ apply(fake, {});
127
+ const fn = listeners[0];
128
+ await fn(failing, async () => { throw new Error('tool failed'); }).catch(() => { threw = true; });
129
+ } catch { threw = true; }
130
+ assert.equal(threw, true);
131
+ });
132
+
133
+ let pass = 0, fail = 0;
134
+ for (const [name, fn] of cases) {
135
+ try { await fn(); pass++; console.log(' ok ' + name); }
136
+ catch (e) { fail++; console.log(' FAIL ' + name + '\n ' + e.message); }
137
+ }
138
+ console.log('\n' + pass + ' passed, ' + fail + ' failed, ' + cases.length + ' total');
139
+ process.exit(fail ? 1 : 0);