uni-harness 0.2.1 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,209 @@
1
+ # uni-harness
2
+
3
+ 🇺🇸 [English](README.md) | 🇰🇷 [한국어](README.ko.md) | 🇨🇳 **简体中文** | 🇯🇵 [日本語](README.ja.md)
4
+
5
+ ![license](https://img.shields.io/badge/license-MIT-blue)
6
+ ![node](https://img.shields.io/badge/node-%E2%89%A516-brightgreen)
7
+ ![runtime](https://img.shields.io/badge/runtime-bash%20%2B%20python3%20stdlib-lightgrey)
8
+ ![for](https://img.shields.io/badge/for-Claude%20Code-d97757)
9
+
10
+ 一个面向 Claude Code 的智能体挽具(harness)套件。它为编码智能体包上
11
+ 自动验证(传感器)、破坏性命令拦截(守卫)、基于检查点的会话恢复、
12
+ 带绊线告警的全量工具调用日志,以及把每次失败都转化为永久结构的
13
+ 棘轮(ratchet)工作流。
14
+
15
+ > 公式:**Agent = Model + Harness。** 模型带来推理能力;
16
+ > 其余的一切 — 规则、传感器、循环上限、记忆、可观测性 —
17
+ > 由这个套件提供。
18
+
19
+ ## 安装
20
+
21
+ ```bash
22
+ npx uni-harness init # 在项目根目录执行(或:init <路径>)
23
+ ```
24
+
25
+ 然后在项目中打开 Claude Code — Claude 会发现挽具尚未配置,并
26
+ **主动提议运行 `/harness-init`**(也可以自己运行)。它会扫描仓库、
27
+ 检测构建/测试/lint 命令、**通过实际运行来验证它们**,并在你批准后
28
+ 填入 `CLAUDE.md` 和 `.harness/commands.env`。在此步骤完成前,
29
+ 验证传感器保持待机。
30
+
31
+ 其他安装器命令:
32
+
33
+ ```bash
34
+ npx uni-harness update # 刷新套件机件(绝不触碰你的文件)
35
+ npx uni-harness doctor # 诊断安装状态
36
+ npx uni-harness uninstall --yes # 移除套件机件,保留你的文件
37
+ ```
38
+
39
+ 依赖要求:bash、python3(仅标准库 — 无需任何包)、安装器本身需要
40
+ node ≥16。
41
+
42
+ **对进行中的项目同样安全。** `init` 绝不覆盖你拥有的任何东西:
43
+ 已有的 `CLAUDE.md` 会被保留(运行 `/harness-init` 后它会以追加方式
44
+ 提议补充挽具章节),已有 `settings.json` 中的钩子和权限会被保留
45
+ (套件钩子以合并方式加入),你的 `.gitignore` 只会被追加而不是替换。
46
+ `update` 只刷新未被修改的套件文件 — 你自定义过的一律跳过
47
+ (会列出,可用 `--force` 覆盖)。
48
+
49
+ ## 内含组件
50
+
51
+ | 文件 | 作用 |
52
+ |---|---|
53
+ | `CLAUDE.md` | 项目命令、规则、反模式、工作循环与检查点协议 |
54
+ | `AGENTS.md` | 规则的跨工具镜像 — Codex、Cursor、Aider、Gemini CLI 等 20+ 工具都会读取 |
55
+ | `.claude/settings.json` | 钩子注册 |
56
+ | `.claude/hooks/sensor-post-edit.sh` | 每次代码编辑立即运行 lint,将失败反馈回去 |
57
+ | `.claude/hooks/stop-gate.sh` | 回合结束时批量运行测试;失败状态下阻止收工(每会话 3 次上限,超过则强制升级上报);执行可选的轨迹规则(`.harness/trace.rules`:"回合结束前必须发生过 X") |
58
+ | `.claude/hooks/guard-pre-bash.sh` | 在执行前拦截破坏性命令与验证绕过(`--no-verify`);当要提交从未通过测试的编辑时先向用户确认 |
59
+ | `.claude/hooks/guard-pre-edit.sh` | 反作弊关卡:削弱测试的编辑(skip/only 标记、删除测试、掏空断言)以及配置/挽具文件的更改都需要用户批准 |
60
+ | `.claude/hooks/pre-compact.sh` | 在上下文压缩前对检查点做快照;恢复时的检查点刷新指令由 `session-start.sh` 负责 |
61
+ | `.claude/hooks/session-start.sh` | 会话启动/恢复/压缩时重新注入进行中的检查点;失败堆积时提示 `/ratchet` |
62
+ | `.claude/hooks/session-end.sh` | 每个会话写入一行台账(时长、调用/失败/编辑数、收工拦截次数、测试状态)→ `sessions.jsonl` |
63
+ | `.claude/hooks/observe-log.sh` | 以 JSONL 记录每次工具调用 + 绊线(同一失败 3 次、调用激增) |
64
+ | `harness/harness_report.py` | 基于日志的健康记分卡 |
65
+ | `harness/tests/` | 钩子与安装器的自测 |
66
+ | `bin/cli.js` | 安装器(init / update / doctor / uninstall) |
67
+
68
+ 技能:
69
+
70
+ | 技能 | 作用 |
71
+ |---|---|
72
+ | `/harness-init` | 扫描仓库 → 填写 PROJECT 章节 & commands.env(安装时一次) |
73
+ | `/checkpoint` | 将状态保存到 `.harness/state/`(plan.md / decisions.jsonl / progress.json) |
74
+ | `/ratchet [失误]` | 复现 → 归类 → 提议规则/传感器/权限 → 验证(无参数:诊断日志)。每个提议附带可证伪的预测(`predictions.jsonl`) |
75
+ | `/guide-audit` | 审计 CLAUDE.md 规则 — 保留 / 删除 / 转为传感器(每月);将过往预测与日志比对验证,没起作用的规则凭证据删除 |
76
+ | `/distill [成功]` | 成功侧的棘轮:从日志中挖掘已完成的任务,提议为可复用的项目技能 |
77
+
78
+ ## 推荐权限配置(可选)
79
+
80
+ 套件不强加权限策略。若用于无人值守或高自治场景,可考虑在项目的
81
+ `.claude/settings.json` 中加入类似配置 — 尤其是阻止智能体修改
82
+ 自身挽具的那些条目:
83
+
84
+ ```json
85
+ {
86
+ "permissions": {
87
+ "ask": [
88
+ "Edit(.claude/**)", "Write(.claude/**)",
89
+ "Edit(.harness/**)", "Write(.harness/**)",
90
+ "Edit(.env*)", "Edit(*.config.*)", "Edit(.github/**)"
91
+ ],
92
+ "deny": [
93
+ "Read(.env)", "Read(.env.*)", "Read(**/secrets/**)",
94
+ "Read(**/*.pem)", "Bash(sudo*)"
95
+ ]
96
+ }
97
+ }
98
+ ```
99
+
100
+ ## 运营例程
101
+
102
+ - **每天 / 每个任务:** 正常干活即可。传感器和守卫是自动的。
103
+ 长任务用 `/checkpoint` 保存状态。
104
+ - **发现失误时:** 不要只在对话里修掉 — 运行 `/ratchet [发生了什么]`
105
+ 把它结构化。同一审查意见出现 3 次 → 变成规则;同一规则被违反
106
+ 3 次 → 升级为守卫/权限。
107
+ - **每周:** 用 `python3 harness/harness_report.py` 查看记分卡;
108
+ 出现重复失败就运行 `/ratchet`。
109
+ - **一次有意义的成功之后:** 若流程还会重复,用 `/distill` 把它
110
+ 变成项目技能。
111
+ - **每月:** 用 `/guide-audit` 审计 CLAUDE.md — 删除传感器已自动
112
+ 强制执行的规则,合并互相矛盾的规则,并将每条规则的预测与日志
113
+ 比对(同一失败仍在发生的规则,凭数字而非感觉删除或升级)。
114
+
115
+ ## 自定义
116
+
117
+ - 危险命令模式:`guard-pre-bash.sh` 中的 `DENY_PATTERNS`
118
+ - 绊线阈值:`observe-log.sh`(默认:同一失败 3 次、300 次调用)
119
+ - 传感器文件扩展名:`sensor-post-edit.sh` 中的 case 语句
120
+ - 收工拦截上限:`stop-gate.sh`(默认:每会话 3 次)
121
+ - 修改了钩子就在 `harness/tests/test_hooks.sh` 加一个用例并运行
122
+
123
+ ## 什么时候不该用
124
+
125
+ 对一次性提问、探索性头脑风暴、无法验证的创意工作来说它是杀鸡用牛刀。
126
+ 它的价值体现在:反复运行的工作、失败有真实代价的工作、无人值守运行
127
+ 的工作、以及必须跨会话保持状态的工作。
128
+
129
+ ## 设计背景
130
+
131
+ 本套件是两份材料的可运行实现。完整分析在 [`docs/`](docs/) 中 —
132
+ 它们不随 npm 包分发(运行中的智能体不需要理论),但解释了这里的
133
+ 每一个设计决策。
134
+
135
+ ### 📄 EnvHarness: Awakening Static Worlds for Agent Learning
136
+
137
+ > **[arXiv:2608.19880](https://arxiv.org/abs/2608.19880)** · Chengsong Huang, Zifeng Wang, Rujun Han, Chen-Yu Lee 等 (2026)
138
+ > 深度解读:[docs/analysis-01-envharness.md](docs/analysis-01-envharness.md)
139
+
140
+ **论文主张。** 正如给*智能体*装上挽具(工具、记忆、技能)可以在不动
141
+ 权重的情况下扩展它,你也可以给*环境*装上挽具,在不动其代码的情况下
142
+ 定制学习信号:`Static Env + EnvHarness = Customized Env`。环境只在
143
+ 接口层通过三个可组合的组件被变换 — **Stage**(重塑初始状态)、
144
+ **Contract**(过滤动作、变换观测、包装转移)、**Chain**(组合环境)—
145
+ 因此原始的真值验证器始终得到保留,这正是它相对 LLM 生成环境的核心
146
+ 优势。一个名为 **EnvRigger** 的自动化循环驱动设计:*观察*失败轨迹 →
147
+ *诊断*根因 → *编写*变换 → 对原始失败进行*验证*。在五个基准上,
148
+ 包装后的环境提升了智能体表现(如 ALFWorld 分布外 +9.0 分、SWE-bench
149
+ Verified +2.7 且步数减少 9.8%)— 分布外增益最大,是迁移而非记忆的
150
+ 证据。
151
+
152
+ **本套件从中借鉴的:**
153
+
154
+ | 论文概念 | 此处的实现 |
155
+ |---|---|
156
+ | EnvRigger 循环(观察 → 诊断 → 编写 → 验证) | `/ratchet` 技能 — 自动接受被替换为**用户批准** |
157
+ | Contract:动作过滤 | `guard-pre-bash.sh`(破坏性命令执行前拦截) |
158
+ | Contract:结构化反馈 | `sensor-post-edit.sh` / `stop-gate.sh`(验证结果反馈) |
159
+ | Stage:备妥的初始状态 | 会话启动时的检查点恢复 |
160
+ | **保留原始验证器** | 不变量:挽具包装你的测试套件,但绝不修改或绕过它 |
161
+
162
+ ### 📘 挽具工程:6 层生产实战手册
163
+
164
+ > 公开实践者材料的综合 — [Mitchell Hashimoto](https://mitchellh.com/)
165
+ > 的棘轮方法论、[OpenAI Codex 实战报告](https://openai.com/index/harness-engineering)、
166
+ > [Martin Fowler](https://martinfowler.com/) 的向导与传感器分类法,以及
167
+ > Anthropic / LangChain / Cursor 的资料。
168
+ > 深度解读:[docs/analysis-02-harness-engineering.md](docs/analysis-02-harness-engineering.md)
169
+
170
+ **手册主张。** 提示词工程(模型*说*什么)和上下文工程(模型*看到*
171
+ 什么)都被挽具工程所涵盖:模型*能做*什么、什么能在失败中幸存、什么被
172
+ 允许、什么算完成。**Agent = Model + Harness** 这一主张有自报但一致的
173
+ 证据支撑 — 同一个模型仅靠更换挽具在 GAIA 上从 30.91% 跃升至 74.55%,
174
+ 固定模型仅靠挽具优化在 Terminal Bench 上从第 30 名爬到第 5 名。
175
+ 架构分为六层,与本套件一一对应:
176
+
177
+ | # | 层 | 原则 | 在本套件中 |
178
+ |---|---|---|---|
179
+ | 1 | **向导** | 每一行 = 对一次过往失败的永久预防 | `CLAUDE.md` |
180
+ | 2 | **传感器** | 外部确定性检查,绝不自我评判 | `sensor-post-edit.sh`、`stop-gate.sh` |
181
+ | 3 | **智能体循环** | 每项预算都有上限,耗尽即升级上报 | 工作循环协议 + 绊线 + 3 次收工拦截上限 |
182
+ | 4 | **记忆** | 文件系统即记忆;必须通过恢复测试 | `/checkpoint` + `session-start.sh` |
183
+ | 5 | **权限** | 模型无法限制它自己 | `guard-pre-bash.sh` + 推荐权限 |
184
+ | 6 | **可观测性** | 记录一切,对漂移告警 | `observe-log.sh` + `harness_report.py` |
185
+
186
+ 运营规则也来自于此 — Hashimoto 的**棘轮原则**("智能体每犯一次错,
187
+ 就工程化一个让该错误不可能重演的方案")以及 Cursor 的晋升阶梯:
188
+
189
+ ```mermaid
190
+ flowchart LR
191
+ F[观察到失败] --> R["/ratchet: 复现 + 诊断"]
192
+ R --> P{最强的层}
193
+ P -->|缺的是上下文| G["向导规则 (CLAUDE.md)"]
194
+ P -->|检查可以捕获| S[传感器 / lint 规则]
195
+ P -->|应当彻底不可能| H[守卫 / 权限]
196
+ G & S & H --> V[针对原始失败验证]
197
+ V --> N[同一失败无法重演]
198
+ ```
199
+
200
+ 同一审查意见出现 **3 次** → 变成规则。同一规则被违反 **3 次** →
201
+ 变成关卡。而成熟的信号是规则增速的*下降* — 只会堆积的挽具就是债务,
202
+ 这正是 `/guide-audit` 存在的意义。
203
+
204
+ ---
205
+
206
+ 想扩展套件本身?从
207
+ [docs/AGENT_BRIEFING.md](docs/AGENT_BRIEFING.md) 开始 — 那里有
208
+ 不变量(不得削弱验证、未经批准不得自动应用、最小基础设施)和经过
209
+ 审核的待办清单。
package/bin/cli.js CHANGED
@@ -229,7 +229,7 @@ function init(target, force) {
229
229
  }
230
230
 
231
231
  let claudeMdExisted = false;
232
- for (const rel of ['CLAUDE.md', '.harness/commands.env']) {
232
+ for (const rel of ['CLAUDE.md', 'AGENTS.md', '.harness/commands.env']) {
233
233
  const dst = path.join(target, rel);
234
234
  if (fs.existsSync(dst)) {
235
235
  log(' = ' + rel + ' (existing file kept)');
@@ -304,6 +304,9 @@ function update(target, force) {
304
304
  writeManifest(target, files);
305
305
  ensureRuntimeDirs(target);
306
306
  migrateLegacyState(target);
307
+ // hooks added in newer kit versions need registering too — mergeSettings
308
+ // is append-only and idempotent, so this never disturbs user settings
309
+ mergeSettings(target);
307
310
  log(`\n${changed} file(s) updated (v${manifest.kitVersion} -> v${KIT_VERSION}). ` +
308
311
  'CLAUDE.md / commands.env / settings.json were not touched.');
309
312
 
@@ -360,7 +363,7 @@ function doctor(target) {
360
363
  const body = fs.readFileSync(sp, 'utf8');
361
364
  JSON.parse(body);
362
365
  check(true, '.claude/settings.json parses');
363
- for (const h of ['guard-pre-bash.sh', 'sensor-post-edit.sh', 'stop-gate.sh', 'observe-log.sh', 'session-start.sh']) {
366
+ for (const h of ['guard-pre-bash.sh', 'guard-pre-edit.sh', 'sensor-post-edit.sh', 'stop-gate.sh', 'observe-log.sh', 'session-start.sh', 'pre-compact.sh', 'session-end.sh']) {
364
367
  if (!body.includes(h)) check(false, `hook registered: ${h}`, 'not in settings.json');
365
368
  }
366
369
  } catch { check(false, '.claude/settings.json', 'missing or failed to parse'); }
@@ -1,7 +1,9 @@
1
1
  #!/usr/bin/env python3
2
2
  """Harness health scorecard.
3
3
 
4
- Reads .harness/logs/tool_calls.jsonl and prints health metrics.
4
+ Reads .harness/logs/tool_calls.jsonl (raw tool calls) and, if present,
5
+ .harness/logs/sessions.jsonl (the per-session ledger written by
6
+ session-end.sh) and prints health metrics.
5
7
 
6
8
  Usage:
7
9
  python3 harness/harness_report.py [--days 7]
@@ -79,6 +81,19 @@ def main() -> int:
79
81
  print(f" [{n}x] {tool}: {err}")
80
82
  print("\n -> For failures repeated 2+ times, consider /ratchet to turn them into rules.")
81
83
 
84
+ ledger = load(root / ".harness" / "logs" / "sessions.jsonl", since)
85
+ if ledger:
86
+ clean = sum(1 for e in ledger if e.get("tests_clean"))
87
+ blocked = sum(e.get("stop_blocks") or 0 for e in ledger)
88
+ durations = sorted(e.get("duration_s") or 0 for e in ledger)
89
+ print("\n─ Session ledger ─")
90
+ print(f" Ended sessions: {len(ledger)}")
91
+ print(f" Tests clean at end: {clean}/{len(ledger)}")
92
+ print(f" Stop-gate blocks: {blocked}")
93
+ print(f" Duration (median): {durations[len(durations) // 2]}s")
94
+ if clean < len(ledger):
95
+ print(" -> Sessions ending with untested edits are quality-drift risk; check why.")
96
+
82
97
  print("\n─ How to read this ─")
83
98
  print(" · Is the failure rate trending down week over week?")
84
99
  print(" · The same failure signature next week means the ratchet isn't working")
@@ -44,6 +44,55 @@ OUT=$(echo '{"tool_name":"Bash","tool_input":{"command":"git push --force-with-l
44
44
  check "allows force-with-lease" "EMPTY" "$OUT"
45
45
  OUT=$(echo '{"tool_name":"Bash","tool_input":{"command":"git status"}}' | bash "$HOOKS/guard-pre-bash.sh")
46
46
  check "passes safe commands" "EMPTY" "$OUT"
47
+ # state-aware commit gate: pending_test marker + configured TEST_CMD -> ask
48
+ printf 'LINT_CMD=""\nTEST_CMD="true"\n' > "$TMP/.harness/commands.env"
49
+ echo "a.py" > "$TMP/.harness/logs/pending_test.s1"
50
+ OUT=$(echo '{"tool_name":"Bash","session_id":"s1","tool_input":{"command":"git commit -m wip"}}' | bash "$HOOKS/guard-pre-bash.sh")
51
+ check "asks on commit with untested edits" '"permissionDecision": "ask"' "$OUT"
52
+ OUT=$(echo '{"tool_name":"Bash","session_id":"s2","tool_input":{"command":"git commit -m ok"}}' | bash "$HOOKS/guard-pre-bash.sh")
53
+ check "allows commit in a session without pending edits" "EMPTY" "$OUT"
54
+ printf 'LINT_CMD=""\nTEST_CMD=""\n' > "$TMP/.harness/commands.env"
55
+ OUT=$(echo '{"tool_name":"Bash","session_id":"s1","tool_input":{"command":"git commit -m x"}}' | bash "$HOOKS/guard-pre-bash.sh")
56
+ check "commit gate silent when TEST_CMD unconfigured" "EMPTY" "$OUT"
57
+ rm -f "$TMP/.harness/logs/pending_test.s1"
58
+
59
+ # ── guard-pre-edit.sh ────────────────────────────────────────────
60
+ echo "guard-pre-edit.sh"
61
+ fresh_project
62
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"'"$TMP"'/.claude/settings.json","old_string":"a","new_string":"b"}}' | bash "$HOOKS/guard-pre-edit.sh")
63
+ check "asks on harness machinery edit" '"permissionDecision": "ask"' "$OUT"
64
+ OUT=$(echo '{"tool_name":"Write","tool_input":{"file_path":"'"$TMP"'/.harness/state/progress.json","content":"{}"}}' | bash "$HOOKS/guard-pre-edit.sh")
65
+ check "allows checkpoint writes" "EMPTY" "$OUT"
66
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"'"$TMP"'/src/app.py","old_string":"a","new_string":"b"}}' | bash "$HOOKS/guard-pre-edit.sh")
67
+ check "allows normal source edits" "EMPTY" "$OUT"
68
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"'"$TMP"'/.env","old_string":"a","new_string":"b"}}' | bash "$HOOKS/guard-pre-edit.sh")
69
+ check "asks on env file edit" '"permissionDecision": "ask"' "$OUT"
70
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"'"$TMP"'/vite.config.ts","old_string":"a","new_string":"b"}}' | bash "$HOOKS/guard-pre-edit.sh")
71
+ check "asks on tool config edit" '"permissionDecision": "ask"' "$OUT"
72
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"'"$TMP"'/tests/test_app.py","old_string":"def test_a():","new_string":"@pytest.mark.skip\ndef test_a():"}}' | bash "$HOOKS/guard-pre-edit.sh")
73
+ check "asks on added skip marker" "skip" "$OUT"
74
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"'"$TMP"'/src/api.test.ts","old_string":"it(\"works\", () => {","new_string":"it.skip(\"works\", () => {"}}' | bash "$HOOKS/guard-pre-edit.sh")
75
+ check "asks on it.skip in js tests" '"permissionDecision": "ask"' "$OUT"
76
+ mkdir -p "$TMP/tests"
77
+ printf 'def test_a():\n pass\ndef test_b():\n pass\n' > "$TMP/tests/test_app.py"
78
+ OUT=$(echo '{"tool_name":"Write","tool_input":{"file_path":"'"$TMP"'/tests/test_app.py","content":"def test_a():\n pass\n"}}' | bash "$HOOKS/guard-pre-edit.sh")
79
+ check "asks on removed test function" "removes 1 test function" "$OUT"
80
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"'"$TMP"'/tests/test_app.py","old_string":"def test_a():","new_string":"def test_a():\ndef test_c():"}}' | bash "$HOOKS/guard-pre-edit.sh")
81
+ check "allows adding tests" "EMPTY" "$OUT"
82
+ OUT=$(echo '{"tool_name":"Write","tool_input":{"file_path":"'"$TMP"'/tests/test_new.py","content":"def test_x():\n pass\n"}}' | bash "$HOOKS/guard-pre-edit.sh")
83
+ check "allows brand-new test file" "EMPTY" "$OUT"
84
+ OUT=$(echo '{"tool_name":"Edit","tool_input":{"file_path":"/somewhere/else/test_x.py","old_string":"def test_a():","new_string":""}}' | bash "$HOOKS/guard-pre-edit.sh")
85
+ check "ignores files outside the project" "EMPTY" "$OUT"
86
+
87
+ # ── pre-compact.sh ───────────────────────────────────────────────
88
+ echo "pre-compact.sh"
89
+ fresh_project
90
+ mkdir -p "$TMP/.harness/state"
91
+ echo '{"task_id":"t1"}' > "$TMP/.harness/state/progress.json"
92
+ OUT=$(echo '{"hook_event_name":"PreCompact","session_id":"s1","trigger":"auto"}' | bash "$HOOKS/pre-compact.sh")
93
+ check "emits no JSON (PreCompact schema has no context channel)" "EMPTY" "$OUT"
94
+ check "snapshots checkpoint" "yes" "$([ -f "$TMP/.harness/state/.pre-compact-backup/progress.json" ] && echo yes)"
95
+ check "logs the compaction event" "PreCompact" "$(cat "$TMP/.harness/logs/tool_calls.jsonl")"
47
96
 
48
97
  # ── observe-log.sh ───────────────────────────────────────────────
49
98
  echo "observe-log.sh"
@@ -53,11 +102,16 @@ LINES=$(wc -l < "$TMP/.harness/logs/tool_calls.jsonl" | tr -d ' ')
53
102
  check "logs one call" "1" "$LINES"
54
103
  echo '{"hook_event_name":"PostToolUse","session_id":"s1","tool_name":"Edit","tool_input":{"file_path":"a.py"},"agent_id":"x1","agent_type":"code-reviewer"}' | bash "$HOOKS/observe-log.sh" > /dev/null
55
104
  check "records subagent fields" "code-reviewer" "$(tail -1 "$TMP/.harness/logs/tool_calls.jsonl")"
56
- OUT=""
57
- for _ in 1 2 3; do
58
- OUT=$(echo '{"hook_event_name":"PostToolUseFailure","session_id":"s1","tool_name":"Bash","tool_input":{"command":"pytest"},"tool_response":"E: same error"}' | bash "$HOOKS/observe-log.sh")
59
- done
60
- check "tripwire on 3 identical failures" "The same tool failed" "$OUT"
105
+ # same error 3x — line numbers differ (normalization) and a success is
106
+ # interleaved (no longer requires "in a row")
107
+ OUT=$(echo '{"hook_event_name":"PostToolUseFailure","session_id":"s1","tool_name":"Bash","tool_input":{"command":"pytest"},"tool_response":"E: boom at line 12"}' | bash "$HOOKS/observe-log.sh")
108
+ check "no tripwire on first failure" "EMPTY" "$OUT"
109
+ echo '{"hook_event_name":"PostToolUse","session_id":"s1","tool_name":"Read","tool_input":{"file_path":"a.py"}}' | bash "$HOOKS/observe-log.sh" > /dev/null
110
+ OUT=$(echo '{"hook_event_name":"PostToolUseFailure","session_id":"s1","tool_name":"Bash","tool_input":{"command":"pytest"},"tool_response":"E: boom at line 34"}' | bash "$HOOKS/observe-log.sh")
111
+ OUT=$(echo '{"hook_event_name":"PostToolUseFailure","session_id":"s1","tool_name":"Bash","tool_input":{"command":"pytest"},"tool_response":"E: boom at line 56"}' | bash "$HOOKS/observe-log.sh")
112
+ check "tripwire on 3 normalized repeats" "escalation packet" "$OUT"
113
+ OUT=$(echo '{"hook_event_name":"PostToolUseFailure","session_id":"s1","tool_name":"Bash","tool_input":{"command":"npm test"},"tool_response":"E: different failure"}' | bash "$HOOKS/observe-log.sh")
114
+ check "no tripwire on a different error" "EMPTY" "$OUT"
61
115
 
62
116
  # ── sensor-post-edit.sh ──────────────────────────────────────────
63
117
  echo "sensor-post-edit.sh"
@@ -96,38 +150,90 @@ OUT=$(echo '{"session_id":"s1"}' | bash "$HOOKS/stop-gate.sh")
96
150
  check "silent when tests pass" "EMPTY" "$OUT"
97
151
  [ -f "$TMP/.harness/logs/pending_test.s1" ] && MARKER_LEFT="yes" || MARKER_LEFT=""
98
152
  check "clears marker on pass" "EMPTY" "$MARKER_LEFT"
153
+ # trace rules: require-before-stop blocks once, then lets the stop pass
154
+ fresh_project
155
+ printf 'LINT_CMD=""\nTEST_CMD="true"\n' > "$TMP/.harness/commands.env"
156
+ printf '# comment line\nrequire-before-stop npm.run.build Run the build before ending the turn.\n' > "$TMP/.harness/trace.rules"
157
+ echo "a.py" > "$TMP/.harness/logs/pending_test.s1"
158
+ echo '{"ts":"2026-01-01T00:00:00","session_id":"s1","event":"PostToolUse","tool":"Edit","summary":"a.py"}' > "$TMP/.harness/logs/tool_calls.jsonl"
159
+ OUT=$(echo '{"session_id":"s1"}' | bash "$HOOKS/stop-gate.sh")
160
+ check "trace rule blocks on missing required step" "trace rule" "$OUT"
161
+ OUT=$(echo '{"session_id":"s1"}' | bash "$HOOKS/stop-gate.sh")
162
+ check "trace rule blocks only once per session" "EMPTY" "$OUT"
163
+ echo "a.py" > "$TMP/.harness/logs/pending_test.s2"
164
+ echo '{"ts":"2026-01-01T00:01:00","session_id":"s2","event":"PostToolUse","tool":"Bash","summary":"npm run build"}' >> "$TMP/.harness/logs/tool_calls.jsonl"
165
+ OUT=$(echo '{"session_id":"s2"}' | bash "$HOOKS/stop-gate.sh")
166
+ check "trace rule satisfied by matching log entry" "EMPTY" "$OUT"
167
+ rm -f "$TMP/.harness/trace.rules"
168
+
169
+ # ── session-end.sh ───────────────────────────────────────────────
170
+ echo "session-end.sh"
171
+ fresh_project
172
+ OUT=$(echo '{"hook_event_name":"SessionEnd","session_id":"s9","reason":"exit"}' | bash "$HOOKS/session-end.sh")
173
+ check "silent with no session activity" "EMPTY" "$([ -f "$TMP/.harness/logs/sessions.jsonl" ] && echo present)"
174
+ cat > "$TMP/.harness/logs/tool_calls.jsonl" <<'EOF'
175
+ {"ts":"2026-01-01T10:00:00","session_id":"s9","event":"PostToolUse","tool":"Bash","summary":"ls"}
176
+ {"ts":"2026-01-01T10:05:00","session_id":"s9","event":"PostToolUse","tool":"Edit","summary":"a.py"}
177
+ {"ts":"2026-01-01T10:06:00","session_id":"s9","event":"PostToolUseFailure","tool":"Bash","error":"boom"}
178
+ {"ts":"2026-01-01T10:07:00","session_id":"other","event":"PostToolUse","tool":"Bash","summary":"ls"}
179
+ EOF
180
+ echo "a.py" > "$TMP/.harness/logs/pending_test.s9"
181
+ echo "2" > "$TMP/.harness/logs/stop_blocks.s9"
182
+ OUT=$(echo '{"hook_event_name":"SessionEnd","session_id":"s9","reason":"exit"}' | bash "$HOOKS/session-end.sh")
183
+ ROW=$(cat "$TMP/.harness/logs/sessions.jsonl" 2>/dev/null)
184
+ check "writes one ledger row" '"session_id": "s9"' "$ROW"
185
+ check "counts calls for this session only" '"calls": 3' "$ROW"
186
+ check "counts failures and edits" '"failures": 1' "$ROW"
187
+ check "records duration" '"duration_s": 360' "$ROW"
188
+ check "records unclean test state" '"tests_clean": false' "$ROW"
189
+ check "records stop blocks" '"stop_blocks": 2' "$ROW"
190
+ [ -f "$TMP/.harness/logs/pending_test.s9" ] && LEFT="yes" || LEFT=""
191
+ check "cleans session marker files" "EMPTY" "$LEFT"
99
192
 
100
193
  # ── session-start.sh ─────────────────────────────────────────────
101
194
  echo "session-start.sh"
102
195
  fresh_project
103
- OUT=$(bash "$HOOKS/session-start.sh")
196
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
104
197
  check "silent without checkpoint" "EMPTY" "$OUT"
105
198
  mkdir -p "$TMP/.harness/state"
106
199
  echo '{"task_id":"t1","status":"in_progress","next_step":"do X"}' > "$TMP/.harness/state/progress.json"
107
- OUT=$(bash "$HOOKS/session-start.sh")
200
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
108
201
  check "injects in-progress checkpoint" ".harness/state" "$OUT"
109
202
  echo '{"task_id":"t1","status":"completed"}' > "$TMP/.harness/state/progress.json"
110
- OUT=$(bash "$HOOKS/session-start.sh")
203
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
111
204
  check "skips completed checkpoint" "EMPTY" "$OUT"
112
205
  rm "$TMP/.harness/state/progress.json"
113
206
  # legacy layout (pre-v0.2): checkpoint at the project root still read
114
207
  echo '{"task_id":"t1","status":"in_progress","next_step":"do X"}' > "$TMP/progress.json"
115
- OUT=$(bash "$HOOKS/session-start.sh")
208
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
116
209
  check "reads legacy root checkpoint" "progress.json" "$OUT"
117
210
  rm "$TMP/progress.json"
211
+ # CLAUDE.md still pointing checkpoints at the root -> propose-a-diff nudge
212
+ printf '# P\n- write the plan to plan.md\n' > "$TMP/CLAUDE.md"
213
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
214
+ check "nudges CLAUDE.md path migration" "user's approval" "$OUT"
215
+ printf '# P\n- write the plan to .harness/state/plan.md\n' > "$TMP/CLAUDE.md"
216
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
217
+ check "silent when CLAUDE.md paths are current" "EMPTY" "$OUT"
218
+ rm "$TMP/CLAUDE.md"
118
219
  NOW=$(python3 -c 'import datetime; print(datetime.datetime.now().isoformat(timespec="seconds"))')
119
220
  for _ in 1 2 3; do
120
221
  echo "{\"ts\":\"$NOW\",\"session_id\":\"old\",\"event\":\"PostToolUseFailure\",\"tool\":\"Bash\",\"error\":\"x\"}" >> "$TMP/.harness/logs/tool_calls.jsonl"
121
222
  done
122
- OUT=$(bash "$HOOKS/session-start.sh")
223
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
123
224
  check "nudges /ratchet on accumulated failures" "/ratchet" "$OUT"
124
225
  fresh_project
125
226
  printf 'LINT_CMD=""\nTEST_CMD=""\n' > "$TMP/.harness/commands.env"
126
- OUT=$(bash "$HOOKS/session-start.sh")
227
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
127
228
  check "offers /harness-init when unconfigured" "/harness-init" "$OUT"
128
229
  printf 'LINT_CMD="true"\nTEST_CMD=""\n' > "$TMP/.harness/commands.env"
129
- OUT=$(bash "$HOOKS/session-start.sh")
230
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"startup"}' | bash "$HOOKS/session-start.sh")
130
231
  check "no setup nudge once configured" "EMPTY" "$OUT"
232
+ # resuming right after a compaction -> checkpoint-refresh instruction
233
+ # (pre-compact.sh cannot inject context; this is the supported channel)
234
+ mkdir -p "$TMP/.harness/state/.pre-compact-backup"
235
+ OUT=$(echo '{"hook_event_name":"SessionStart","source":"compact"}' | bash "$HOOKS/session-start.sh")
236
+ check "instructs checkpoint refresh after compaction" "pre-compact-backup" "$OUT"
131
237
 
132
238
  # ── result ───────────────────────────────────────────────────────
133
239
  echo
@@ -35,6 +35,7 @@ check "installs hooks" "yes" "$([ -f "$TGT/.claude/hooks/stop-gate.sh" ] && echo
35
35
  check "hooks are executable" "yes" "$([ -x "$TGT/.claude/hooks/stop-gate.sh" ] && echo yes)"
36
36
  check "installs skills" "yes" "$([ -f "$TGT/.claude/skills/ratchet/SKILL.md" ] && echo yes)"
37
37
  check "installs CLAUDE.md" "yes" "$([ -f "$TGT/CLAUDE.md" ] && echo yes)"
38
+ check "installs AGENTS.md" "yes" "$([ -f "$TGT/AGENTS.md" ] && echo yes)"
38
39
  check "installs settings.json" "yes" "$([ -f "$TGT/.claude/settings.json" ] && echo yes)"
39
40
  check "writes manifest" "yes" "$([ -f "$TGT/.harness/kit-manifest.json" ] && echo yes)"
40
41
  check "gitignores logs" ".harness/logs/" "$(cat "$TGT/.gitignore")"
@@ -50,6 +51,7 @@ check "does not ship test_installer.sh" "EMPTY" "$([ -f "$TGT/harness/tests/test
50
51
  echo "init (brownfield)"
51
52
  BF=$(mktemp -d)
52
53
  echo "MY PROJECT RULES" > "$BF/CLAUDE.md"
54
+ echo "MY OWN AGENTS FILE" > "$BF/AGENTS.md"
53
55
  echo "node_modules/" > "$BF/.gitignore"
54
56
  mkdir -p "$BF/.claude"
55
57
  cat > "$BF/.claude/settings.json" <<'EOF'
@@ -64,6 +66,7 @@ cat > "$BF/.claude/settings.json" <<'EOF'
64
66
  EOF
65
67
  OUT=$(node "$CLI" init "$BF" 2>&1)
66
68
  check "keeps existing CLAUDE.md" "MY PROJECT RULES" "$(cat "$BF/CLAUDE.md")"
69
+ check "keeps existing AGENTS.md" "MY OWN AGENTS FILE" "$(cat "$BF/AGENTS.md")"
67
70
  check "notes CLAUDE.md was kept" "existing CLAUDE.md was kept" "$OUT"
68
71
  check "keeps user permissions" '"Read"' "$(cat "$BF/.claude/settings.json")"
69
72
  check "keeps user's own hook" "my-own-hook.sh" "$(cat "$BF/.claude/settings.json")"
@@ -108,6 +111,20 @@ sed -i.bak '/.harness\/state/d' "$TGT/.gitignore" && rm -f "$TGT/.gitignore.bak"
108
111
  OUT=$(node "$CLI" update "$TGT" 2>&1)
109
112
  check "update re-adds state gitignore entry" ".harness/state/" "$(cat "$TGT/.gitignore")"
110
113
 
114
+ # update on an install from before a hook existed must register the new
115
+ # hook in settings.json (mergeSettings runs on update too, append-only)
116
+ python3 - "$TGT" <<'EOF'
117
+ import json, sys, os
118
+ sp = os.path.join(sys.argv[1], '.claude/settings.json')
119
+ s = json.load(open(sp))
120
+ s['hooks'].pop('SessionEnd', None)
121
+ s['user_marker'] = 'kept'
122
+ json.dump(s, open(sp, 'w'), indent=2)
123
+ EOF
124
+ OUT=$(node "$CLI" update "$TGT" 2>&1)
125
+ check "update registers newly shipped hooks" "session-end.sh" "$(cat "$TGT/.claude/settings.json")"
126
+ check "update merge keeps user settings" '"user_marker"' "$(cat "$TGT/.claude/settings.json")"
127
+
111
128
  # pre-v0.2 CLAUDE.md pointing checkpoints at the project root -> migration
112
129
  # note (told, never touched); the current template must NOT trigger it
113
130
  printf '# old project\n- read progress.json first\n' > "$TGT/CLAUDE.md"
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "uni-harness",
3
- "version": "0.2.1",
3
+ "version": "0.4.0",
4
4
  "description": "Agent harness kit for Claude Code — installs verification sensors, destructive-command guards, checkpoint recovery, observability logs, and ratchet skills into your project",
5
5
  "bin": {
6
6
  "uni-harness": "bin/cli.js"
@@ -8,6 +8,7 @@
8
8
  "files": [
9
9
  "bin",
10
10
  "CLAUDE.md",
11
+ "AGENTS.md",
11
12
  ".claude",
12
13
  ".harness/commands.env",
13
14
  "harness",