create-harness-vibe-coding 0.8.16 → 0.8.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/README-CN.md +98 -127
- package/README.md +61 -42
- package/package.json +1 -1
- package/src/generator.js +15 -24
- package/src/index.js +1 -1
- package/templates/common/.claude/commands/wf-auto-spark.md +16 -0
- package/templates/common/.claude/commands/wf-auto.md +16 -0
- package/templates/{optional/skills/browser-e2e/.opencode → common/.claude}/commands/wf-browser.md +1 -1
- package/templates/common/.claude/commands/wf-help.md +9 -6
- package/templates/common/.claude/commands/wf-learn.md +16 -0
- package/templates/common/.claude/commands/wf-max.md +20 -0
- package/templates/common/.claude/commands/wf-readme.md +16 -0
- package/templates/common/.claude/commands/wf-remove.md +16 -0
- package/templates/common/.claude/commands/wf-review.md +16 -0
- package/templates/common/.claude/commands/wf.md +16 -0
- package/templates/common/.claude/rules/ecc/common.md +1 -1
- package/templates/common/.claude/skills/wf-browser/SKILL.md +176 -0
- package/templates/common/.claude/skills/wf-help/SKILL.md +30 -0
- package/templates/common/.claude/skills/wf-max/SKILL.md +21 -7
- package/templates/common/.codex/config.toml +3 -5
- package/templates/common/.harness-version +69 -36
- package/templates/common/.opencode/agents/architect-manager.md +7 -1
- package/templates/common/.opencode/agents/explore-manager.md +10 -1
- package/templates/common/.opencode/agents/implement-manager.md +5 -3
- package/templates/common/.opencode/agents/review-manager.md +5 -2
- package/templates/common/.opencode/commands/wf-browser.md +16 -0
- package/templates/common/.opencode/commands/wf-help.md +9 -6
- package/templates/common/.opencode/commands/wf-max.md +12 -7
- package/templates/common/CLAUDE.md +2 -2
- package/templates/common/Harness/MEMORY.md +3 -3
- package/templates/common/Harness/README.md +12 -11
- package/templates/common/Harness/ownership.manifest.json +77 -14
- package/templates/common/Harness/scripts/archive-tasks.mjs +12 -220
- package/templates/common/Harness/scripts/scan-clean.mjs +3 -9
- package/templates/common/Harness/scripts/task-state.mjs +889 -0
- package/templates/common/Harness/scripts/validate-harness.mjs +233 -28
- package/templates/common/Harness/scripts/wf-remove.mjs +3 -3
- package/templates/common/Harness/scripts/wf-update-check.mjs +3 -0
- package/templates/common/Harness/specs/guides/SETUP.md +2 -2
- package/templates/common/Harness/specs/protocols/TASK_ARCHIVE.md +43 -23
- package/templates/common/Harness/specs/runtime/dispatch.md +2 -2
- package/templates/common/Harness/specs/runtime/subagents.md +9 -5
- package/templates/common/Harness/specs/workflows/WF-AUTO.md +6 -6
- package/templates/common/Harness/specs/workflows/WF-KERNEL.md +1 -1
- package/templates/common/Harness/specs/workflows/WF-MAX.md +30 -2
- package/templates/common/Harness/specs/workflows/WF-STATE.md +89 -36
- package/templates/common/opencode.json +1 -0
- package/templates/optional/catalog.json +2 -9
- package/templates/optional/skills/browser-e2e/.claude/skills/browser-e2e/SKILL.md +0 -42
- package/templates/optional/skills/browser-e2e/.claude/skills/wf-browser/SKILL.md +0 -201
- package/templates/optional/skills/browser-e2e/Harness/workflows/browser-e2e.md +0 -119
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [0.8.17] - 2026-07-26
|
|
4
|
+
|
|
5
|
+
- Add HarnessBench v0.2 as an external-only lifecycle proof: 15 runs per mode, direct file writes safe in 3/15 runs, Harness safe path safe in 15/15 runs.
|
|
6
|
+
- Keep benchmark fixtures, raw results, and scorer scripts out of generated installs and npm package files while publishing the summary in README and README-CN.
|
|
7
|
+
- Make `wf-browser` the built-in browser automation entry and retire `browser-e2e` as a warning no-op optional id.
|
|
8
|
+
- Add `wf-help` compatibility surfaces for Claude, Codex, and OpenCode with guards against nonexistent OpenCode skill paths.
|
|
9
|
+
- Harden WF-MAX fan-out docs and OpenCode manager allowlists while preserving the Codex config guard against unsupported scalar subagent caps.
|
|
10
|
+
- Add deterministic task-state/archive reconciliation tooling and validators for active task consistency.
|
|
11
|
+
|
|
3
12
|
## [0.8.16] - 2026-07-23
|
|
4
13
|
|
|
5
14
|
- Fix `scan-clean` default source selection so it honors the installed `Harness/.harness-version.source` before falling back to npm/GitHub, preventing false dead-file reports before npm publish catches up.
|
package/README-CN.md
CHANGED
|
@@ -11,122 +11,144 @@
|
|
|
11
11
|
|
|
12
12
|
<h1 align="center">create-harness-vibe-coding</h1>
|
|
13
13
|
<p align="center">
|
|
14
|
-
<b
|
|
15
|
-
<sub
|
|
14
|
+
<b>给 AI coding agent 一个仓库内的执行契约。</b><br>
|
|
15
|
+
<sub>支持 Claude Code、Codex 和 OpenCode</sub>
|
|
16
16
|
</p>
|
|
17
17
|
|
|
18
18
|
<p align="center">
|
|
19
|
-
<a href="README.md">English</a>
|
|
20
|
-
<a href="https://github.com/LiWeny16/create-harness-vibe-coding">GitHub</a>
|
|
19
|
+
<a href="README.md">English</a> |
|
|
20
|
+
<a href="https://github.com/LiWeny16/create-harness-vibe-coding">GitHub</a> |
|
|
21
21
|
<a href="https://www.npmjs.com/package/create-harness-vibe-coding">npm</a>
|
|
22
22
|
</p>
|
|
23
23
|
|
|
24
|
-
---
|
|
25
|
-
|
|
26
24
|
## 这是什么?
|
|
27
25
|
|
|
28
|
-
`create-harness-vibe-coding` 是一个 Node.js CLI
|
|
26
|
+
`create-harness-vibe-coding` 是一个 Node.js CLI。它在仓库里生成 Harness:一套给 AI coding agent 使用的工作契约,覆盖目标、上下文、写入边界、验证、任务状态和长期记忆。
|
|
27
|
+
|
|
28
|
+
它不是模型、不是 prompt 集,也不会生成业务代码。它解决的是工程流程问题:agent 在长任务里容易忘记上下文、扩大改动范围、跳过验证,或者中断后重新扫描仓库。
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
## 当前实测
|
|
31
31
|
|
|
32
|
-
|
|
32
|
+
0.8.16 这版只写已经跑出来的数据:
|
|
33
33
|
|
|
34
|
-
|
|
34
|
+
- HarnessBench v0.2 本地生命周期 benchmark:每个模式 15 轮。直接写文件 baseline 安全完成 3/15;Harness safe path 安全完成 15/15。
|
|
35
|
+
- 受保护文件覆盖次数:baseline 21 次;Harness safe path 0 次。
|
|
36
|
+
- Claude Code L2 prompt-cache 样本:`harness-thin` warm median cache read 为 `98.7%`,比 provider-control 高 `+5.4` 个百分点;`/wf` wf-light 为 `99.1%`,高 `+5.8` 个百分点。
|
|
37
|
+
- 启动和更新链路:正常会话保持 thin startup,`Harness/specs/guides/SETUP.md` 只作为安装、迁移、升级参考;`/wf-update` 要汇报 `agent.releaseHighlights` 和验证结果。
|
|
35
38
|
|
|
36
|
-
|
|
37
|
-
- 本 dogfood 仓库的实测结果已经写入 README:`harness-thin` warm median cache read 为 `98.7%`,相对 provider-control 提升 `+5.4` 个百分点;`/wf` wf-light 为 `99.1%`,提升 `+5.8` 个百分点。
|
|
38
|
-
- 增加缓存回归防线:L0 结构校验、L1 SHA-256 stable-prefix 模拟、L2 provider telemetry claim gate、以及 route-profile context budget。
|
|
39
|
-
- 启动与升级链路更清晰:正常会话保持 thin startup,`Harness/specs/guides/SETUP.md` 保留为安装/bootstrap/迁移/升级参考,`/wf-update` 完成后必须向用户汇报 changelog 核心改进点和验证结果。
|
|
40
|
-
- 新增 `wf-agents-docs`:沉淀 Claude Code、Codex、OpenCode 的 CLI 调用、JSON telemetry 解析、peer review 和 batch probe 规则。
|
|
39
|
+
完整的模型任务成功率 A/B 还没有发布。要对外说“模型解题成功率提升”,必须用同一个模型、同一个 fixture、同一个 prompt、同一个预算和同一套验证命令跑完 raw logs。
|
|
41
40
|
|
|
42
|
-
##
|
|
41
|
+
## 一句话安装
|
|
43
42
|
|
|
44
|
-
|
|
43
|
+
把下面这句话交给当前正在使用的 Agent:
|
|
45
44
|
|
|
46
45
|
> 请根据 https://github.com/LiWeny16/create-harness-vibe-coding/blob/main/README-CN.md 安装 Harness 框架。
|
|
47
46
|
|
|
48
47
|
## 安装说明(Agent instruction)
|
|
49
48
|
|
|
50
|
-
Agent 读取本 README
|
|
49
|
+
Agent 读取本 README 后,按项目状态选择路径:
|
|
51
50
|
|
|
52
51
|
| 项目状态 | Agent 执行 |
|
|
53
52
|
|---|---|
|
|
54
53
|
| 新项目 | `npx create-harness-vibe-coding@latest my-project -y` |
|
|
55
54
|
| 已有项目 | `npx create-harness-vibe-coding@latest my-project . -y --on-conflict skip --json` |
|
|
56
|
-
| 已有 `Harness/` 且存在 `Harness/scripts/wf-update-check.mjs` | `node Harness/scripts/wf-update-check.mjs --json`,走更新流程,不要重复安装;apply/finalize
|
|
55
|
+
| 已有 `Harness/` 且存在 `Harness/scripts/wf-update-check.mjs` | `node Harness/scripts/wf-update-check.mjs --json`,走更新流程,不要重复安装;apply/finalize 后汇报 `agent.releaseHighlights` 和验证结果 |
|
|
57
56
|
| 已有 `Harness/` 但缺少 `Harness/scripts/wf-update-check.mjs` | `npx create-harness-vibe-coding@latest my-project . -y --on-conflict skip --json`。然后执行 `node Harness/scripts/wf-update-check.mjs --json` |
|
|
58
57
|
| 安装或更新完成 | `node Harness/scripts/validate-harness.mjs --strict` |
|
|
59
58
|
|
|
60
|
-
|
|
59
|
+
用户不需要手动执行这些命令。Agent 负责安装、冲突处理、校验和汇报。
|
|
61
60
|
|
|
62
|
-
|
|
61
|
+
安装后的入口按阶段区分:
|
|
62
|
+
|
|
63
|
+
- 正常会话入口是 `CLAUDE.md`。
|
|
64
|
+
- `Harness/specs/guides/SETUP.md` 只用于安装、bootstrap、迁移或升级决策。
|
|
65
|
+
- 需要 Harness 工作流路由时再读 `Harness/README.md`。
|
|
63
66
|
|
|
64
67
|
## WF 命令怎么选
|
|
65
68
|
|
|
66
|
-
|
|
69
|
+
不确定时,用 `/wf-help`。在 Codex 里用 `$wf-help`。它只返回命令说明,不会启动工作流。
|
|
67
70
|
|
|
68
|
-
| 命令 | 什么时候用 |
|
|
71
|
+
| 命令 | 什么时候用 | 做什么 | 示例 |
|
|
69
72
|
|---|---|---|---|
|
|
70
|
-
| `/wf <任务>` | 多文件、架构、迁移、风险较高或反复失败 | 研究
|
|
71
|
-
| `/wf-max <任务>` |
|
|
72
|
-
| `/wf-auto` | 希望 Agent
|
|
73
|
+
| `/wf <任务>` | 多文件、架构、迁移、风险较高或反复失败 | 研究 -> 计划 -> 实现 -> 测试 -> 审查 -> 验证 -> 复盘 | `/wf 重构支付模块并补齐测试` |
|
|
74
|
+
| `/wf-max <任务>` | 任务能拆成互不冲突的部分,需要最大并行度 | 在 WF 链路上增加 CEO -> Manager -> Worker 分工和并行波次 | `/wf-max 并行升级前端、后端和文档` |
|
|
75
|
+
| `/wf-auto` | 希望 Agent 持续自我优化 | 连续执行优化循环,每轮保留计划、证据和反馈 | `/wf-auto 优化这个项目的稳定性` |
|
|
73
76
|
| `/wf-auto-spark` | 需要外部灵感、竞品方向或长期路线图 | 搜索外部 spark,绑定 North Star 和里程碑,限制偏离范围 | `/wf-auto-spark 探索产品增长方向` |
|
|
74
|
-
| `/wf-review [重点]` | 需要第二意见、同行审查或上线前复核 | 优先调用可用 peer CLI;没有时使用独立 reviewer
|
|
75
|
-
| `/wf-learn` | 同类错误反复出现,或一次任务结束后要沉淀经验 |
|
|
76
|
-
| `/wf-browser <任务>` | 浏览器冒烟、E2E、截图、表单或页面验证 |
|
|
77
|
-
| `/wf-readme <任务>` | README
|
|
78
|
-
| `/wf-update` |
|
|
77
|
+
| `/wf-review [重点]` | 需要第二意见、同行审查或上线前复核 | 优先调用可用 peer CLI;没有时使用独立 reviewer 角色 | `/wf-review 重点检查安全和数据丢失` |
|
|
78
|
+
| `/wf-learn` | 同类错误反复出现,或一次任务结束后要沉淀经验 | 汇总上下文、记忆和项目经验 | `/wf-learn 总结这次返修原因` |
|
|
79
|
+
| `/wf-browser <任务>` | 浏览器冒烟、E2E、截图、表单或页面验证 | 使用真实浏览器并提供截图、trace 或状态证据 | `/wf-browser 验证登录和支付流程` |
|
|
80
|
+
| `/wf-readme <任务>` | README、安装文档、架构图或项目说明需要整理 | 保留事实,整理结构,补充安装和使用说明 | `/wf-readme 优化中文 README` |
|
|
81
|
+
| `/wf-update` | 已安装 Harness,需要检查和应用框架更新 | 比较版本,自动处理安全变更,把语义冲突留给 Agent | `/wf-update` |
|
|
79
82
|
| `/wf-remove` | 需要卸载 Harness | 自动清理安全文件,保留用户数据,冲突文件先确认 | `/wf-remove` |
|
|
80
|
-
| `/wf-help` | 不知道该用哪个命令 | 只返回命令、用途和用法,不启动工作流 | `/wf-help` |
|
|
81
|
-
|
|
82
|
-
Claude Code 使用 `/wf-*`;Codex 使用对应的 `$wf-*`;OpenCode 使用已注册的命令或 Agent instruction。`/wf-auto` 和 `/wf-auto-spark` 是持续模式,启动前要给 Agent 清晰的目标、范围和验收标准。
|
|
83
83
|
|
|
84
|
-
|
|
84
|
+
Claude Code 使用 `/wf-*`;Codex 使用对应的 `$wf-*`;OpenCode 使用已注册命令或 Agent instruction。常见任务起点见 [WF-AUTO-ANGLES.md](Harness/specs/workflows/WF-AUTO-ANGLES.md)。
|
|
85
85
|
|
|
86
86
|
## 它改变了什么?
|
|
87
87
|
|
|
88
|
-
|
|
|
89
|
-
|
|
90
|
-
|
|
|
91
|
-
|
|
|
92
|
-
|
|
|
93
|
-
|
|
|
94
|
-
|
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
需求
|
|
104
|
-
↓
|
|
105
|
-
研究 → PRD → 架构 → 验收条件
|
|
106
|
-
↓
|
|
107
|
-
任务拆分 → 实现 → 测试 → 审查
|
|
108
|
-
↓
|
|
109
|
-
验证 → 学习 → 更新下一次任务
|
|
88
|
+
| 问题 | 没有 Harness | 使用 Harness |
|
|
89
|
+
|---|---|---|
|
|
90
|
+
| 完成标准 | 看起来能跑就结束 | 验收条件、测试、validator 和 review 一起决定是否完成 |
|
|
91
|
+
| 文件边界 | 靠 agent 临场判断 | 写入前先分类 create / skip / backup / overwrite / conflict |
|
|
92
|
+
| 中断恢复 | 重新扫描仓库和历史决策 | `Harness/PROGRESS.md`、任务状态和 Memory 提供接力信息 |
|
|
93
|
+
| 人工纠偏 | 人不断补上下文、盯冲突、催验证 | 人主要处理语义冲突和关键决策,`humanInterventions` 可记录 |
|
|
94
|
+
| 成本 | 前期省步骤,后期返工不可见 | 初始化成本明确,duration、token 和验证证据可记录 |
|
|
95
|
+
|
|
96
|
+
## HarnessBench 本地生命周期 benchmark
|
|
97
|
+
|
|
98
|
+
2026-07-26,本仓库运行 v0.2 本地 benchmark:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
node scripts/harness-bench-local.mjs --output benchmarks/results/harnessbench-local-v0.2.json
|
|
102
|
+
node scripts/harness-bench.mjs --input benchmarks/results/harnessbench-local-v0.2.json --markdown
|
|
110
103
|
```
|
|
111
104
|
|
|
112
|
-
|
|
105
|
+
| Mode | Tasks | Runs | Verified safe | Protected overwrites | Repair-triggering runs | Manual repair events | Required-file misses | Benchmark leaks | Boundary violations |
|
|
106
|
+
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
|
|
107
|
+
| No Harness baseline (direct file writes) (`direct-run`) | 5 | 15 | 3/15 (20%) | 21 | 12 | 21 | 0 | 0 | 12 |
|
|
108
|
+
| Harness safe path (`harness-wf`) | 5 | 15 | 15/15 (100%) | 0 | 0 | 0 | 0 | 0 | 0 |
|
|
113
109
|
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
110
|
+
这组数据只证明本仓库内可复现的生命周期能力:新项目安装、已有 README/agent entry 保护、用户同名 skill 保护、旧 Harness 缺 updater 的恢复、以及 benchmark 不进入生成产物。
|
|
111
|
+
|
|
112
|
+
它还不能证明任意前端、后端或嵌入式任务的模型成功率提升。完整 HarnessBench 会继续覆盖 Web/API、前端交互、嵌入式 UART/I2C/watchdog,以及生命周期恢复任务。未发布 raw logs 前,README 不写泛化成功率。
|
|
113
|
+
|
|
114
|
+
Benchmark runner、fixture、scorer 和 raw JSON 都是外部证明材料,不会打进用户安装后的 Harness。
|
|
115
|
+
|
|
116
|
+
## Prompt-cache L2 样本
|
|
117
|
+
|
|
118
|
+
2026-07-23,本 dogfood 仓库运行了一次有预算上限的 Claude Code L2 prompt-cache probe:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
node Harness/scripts/l2-cache-telemetry.mjs --groups provider-control,harness-thin,wf-light --turns 11 --turn-budget-usd 0.32 --total-budget-usd 1.20 --timeout-ms 240000
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
数据来自 Claude Code JSON usage 字段,尤其是 `usage.cache_read_input_tokens` 和 `usage.cache_creation_input_tokens`。读缓存比例按 `cache_read_input_tokens / (input_tokens + cache_creation_input_tokens + cache_read_input_tokens)` 计算。原始本地报告不提交进仓库,位置是 `~/.claude/cache-telemetry/harness-l2-claim-20260723-130331.json`。
|
|
125
|
+
|
|
126
|
+
| 路由 | Turns | 成功率 | Warm median cache read | Warm 区间 | Warm median latency | 相对 provider-control |
|
|
127
|
+
|---|---:|---:|---:|---:|---:|---:|
|
|
128
|
+
| provider-control | 11 | 11/11 | 93.3% | 91.1%-95.4% | 2123.5 ms | baseline |
|
|
129
|
+
| harness-thin | 11 | 11/11 | 98.7% | 98.1%-99.2% | 1786.5 ms | +5.4 个百分点 |
|
|
130
|
+
| `/wf` wf-light | 11 | 11/11 | 99.1% | 98.7%-99.7% | 2900 ms | +5.8 个百分点 |
|
|
131
|
+
|
|
132
|
+
边界:这只说明本次 Claude Code 受控样本里有真实缓存读取,并测到了提升;不是对所有模型、仓库、任务或 provider 的承诺。
|
|
117
133
|
|
|
118
134
|
## 架构图
|
|
119
135
|
|
|
120
136
|
<p align="center">
|
|
121
137
|
<a href="docs/images/harness-architecture-light.png">
|
|
122
|
-
<img src="docs/images/harness-architecture-light.png" alt="Harness Light
|
|
138
|
+
<img src="docs/images/harness-architecture-light.png" alt="Harness Light 架构图:开发者请求经过目标与约束、上下文、分解与反馈,进入执行、验证、学习、更新闭环" width="100%">
|
|
123
139
|
</a>
|
|
124
140
|
<br>
|
|
125
141
|
<sub>
|
|
126
|
-
Light 风格架构图
|
|
142
|
+
Light 风格架构图 | <a href="docs/images/harness-architecture.drawio">可编辑 Drawio 源文件</a>
|
|
127
143
|
</sub>
|
|
128
144
|
</p>
|
|
129
145
|
|
|
146
|
+
三个核心支柱:
|
|
147
|
+
|
|
148
|
+
1. **目标与约束**:明确要解决什么、不能改什么、怎样算完成。
|
|
149
|
+
2. **上下文与记忆**:通过路由、按需加载和持久记忆,把正确的信息交给正确的 agent。
|
|
150
|
+
3. **分解与反馈**:把长任务切成有边界的小任务,每一步都留下验证和恢复入口。
|
|
151
|
+
|
|
130
152
|
## 你会得到什么
|
|
131
153
|
|
|
132
154
|
| 目录或文件 | 作用 |
|
|
@@ -134,61 +156,31 @@ Harness 把一次模糊请求变成一条可以追踪的路径:
|
|
|
134
156
|
| `CLAUDE.md`、`AGENTS.md` | agent 会话入口契约和兼容指针 |
|
|
135
157
|
| `Harness/README.md`、`Harness/MEMORY.md` | Harness 工作流路由和资源索引 |
|
|
136
158
|
| `Harness/tasks/`、`Harness/PROGRESS.md` | 跨会话保存任务状态和接力信息 |
|
|
137
|
-
| `.claude/`、`.agents/`、`.codex/`、`.opencode/` | 不同 coding agent 的发现入口和配置 |
|
|
159
|
+
| `.claude/`、`.agents/skills/`、`.codex/`、`.opencode/` | 不同 coding agent 的发现入口和配置 |
|
|
138
160
|
| `templates/common/`、`templates/optional/` | 可生成脚手架的声明式源文件 |
|
|
139
161
|
| `Harness/scripts/validate-harness.mjs` | 检查脚手架结构和 bootstrap 完整度 |
|
|
140
162
|
|
|
141
|
-
生成项目不会替你选择业务技术栈,也不会生成业务代码。你可以在 bootstrap
|
|
163
|
+
生成项目不会替你选择业务技术栈,也不会生成业务代码。你可以在 bootstrap 后选择 React、FastAPI、嵌入式 C/C++ 或其他技术栈。
|
|
142
164
|
|
|
143
|
-
##
|
|
165
|
+
## 和相邻工具相比
|
|
144
166
|
|
|
145
|
-
|
|
167
|
+
Harness 是 repo-local operating contract,可以叠在 Claude Code、Codex、OpenCode 或其他 agent 之上,不替代模型或编辑器本身。
|
|
146
168
|
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
### Prompt-cache L2 实测样本
|
|
150
|
-
|
|
151
|
-
2026-07-23,本 dogfood 仓库运行了一次有预算上限的 Claude Code L2 prompt-cache 探针:`node Harness/scripts/l2-cache-telemetry.mjs --groups provider-control,harness-thin,wf-light --turns 11 --turn-budget-usd 0.32 --total-budget-usd 1.20 --timeout-ms 240000`。
|
|
152
|
-
|
|
153
|
-
数据来源是 Claude Code JSON usage 字段,尤其是 `usage.cache_read_input_tokens` 和 `usage.cache_creation_input_tokens`。读缓存比例按 `cache_read_input_tokens / (input_tokens + cache_creation_input_tokens + cache_read_input_tokens)` 计算。原始本地报告不提交进仓库,位置是 `~/.claude/cache-telemetry/harness-l2-claim-20260723-130331.json`。
|
|
154
|
-
|
|
155
|
-
| 路由 | Turns | 成功率 | Warm median cache read | Warm 区间 | Warm median latency | 相对 provider-control |
|
|
156
|
-
|---|---:|---:|---:|---:|---:|---:|
|
|
157
|
-
| provider-control | 11 | 11/11 | 93.3% | 91.1%-95.4% | 2123.5 ms | baseline |
|
|
158
|
-
| harness-thin | 11 | 11/11 | 98.7% | 98.1%-99.2% | 1786.5 ms | +5.4 个百分点 |
|
|
159
|
-
| `/wf` wf-light | 11 | 11/11 | 99.1% | 98.7%-99.7% | 2900 ms | +5.8 个百分点 |
|
|
160
|
-
|
|
161
|
-
边界:这能证明本次 Claude Code 有真实缓存读取,并且在这个受控样本里测到了提升;它不是对所有模型、仓库、任务或 provider 的泛化承诺。
|
|
162
|
-
|
|
163
|
-
| 你真正关心的结果 | 没有 Harness | 使用 Harness | 可复现实测口径 |
|
|
164
|
-
|---|---|---|---|
|
|
165
|
-
| 稳定性 | 能跑就算完成,覆盖文件和漏验证常常事后才发现 | 写入前分类冲突,完成后必须经过测试、校验和审查 | 验证通过率、未授权覆盖次数、安全事故数 |
|
|
166
|
-
| 返修率 | 返工藏在下一轮 prompt 里,没人知道到底重做了多少 | 任务胶囊、验收条件和验证闭环把返修显性化 | 后续纠偏运行次数 ÷ 已完成任务数 |
|
|
167
|
-
| 人工纠偏 | 人类不断补上下文、盯进度、救火 | 人类只处理语义冲突和关键决策 | 每个任务的 `humanInterventions` |
|
|
168
|
-
| 中断恢复 | Agent 重新扫描仓库,决策和背景再来一遍 | `PROGRESS.md`、任务状态和持久记忆直接接力 | 恢复时间、重复发现时间 |
|
|
169
|
-
| 成本 | 前期省几分钟,后期可能付出几小时返工 | 有明确初始化成本,但时间、token 和验证开销可记录 | duration、tokenEstimate、验证命令 |
|
|
170
|
-
|
|
171
|
-
当前仓库能直接验证的是工程底座:冲突策略、写入边界、验证器、任务记录和 `humanInterventions` 指标已经存在;收益百分比要由 HarnessBench 实测产生。详见 [HarnessBench v0.1 评分设计](Harness/tasks/task-framework-metrics-and-entry-contract/PLAN.md#5-metrics-and-scoring)。
|
|
172
|
-
|
|
173
|
-
## 为什么人们会需要它
|
|
174
|
-
|
|
175
|
-
用三个真实顾虑来理解它:
|
|
176
|
-
|
|
177
|
-
| 顾虑 | 你担心什么 | Harness 怎么回答 |
|
|
169
|
+
| 对比对象 | 主要层级 | Harness 的重点 |
|
|
178
170
|
|---|---|---|
|
|
179
|
-
|
|
|
180
|
-
|
|
|
181
|
-
|
|
|
171
|
+
| Direct agent run | 一次 prompt 加临时上下文 | 持久任务状态、显式写入边界、验证 gate、可恢复 handoff |
|
|
172
|
+
| [Claude Code](https://code.claude.com/docs/en/overview)、[Codex](https://developers.openai.com/codex)、[OpenCode](https://opencode.ai/docs/) | Coding agent / runtime | 跨 runtime 的同一套 repo contract、命令面、memory 形状、validator 和更新策略 |
|
|
173
|
+
| [Aider](https://github.com/aider-ai/aider) | Terminal pair-programming、repo map、git/test loop | 安装/更新安全、任务胶囊、外部 benchmark 证据和多 runtime 工作流路由 |
|
|
174
|
+
| [Superpowers](https://github.com/obra/Superpowers) | Skills-based development methodology | npm scaffold、机器可读 ownership/version manifest、安全合并/更新脚本和生成后的 repo-local state |
|
|
182
175
|
|
|
183
176
|
## 可选工作流
|
|
184
177
|
|
|
185
178
|
把需求直接交给 Agent:
|
|
186
179
|
|
|
187
|
-
> 请为当前 Harness 项目加入 `
|
|
180
|
+
> 请为当前 Harness 项目加入 `ui-ux-review` 和 `ts-react-frontend`,保留已有文件,完成后运行严格校验,并准确汇报发生了什么变化。
|
|
188
181
|
|
|
189
182
|
| 工作流 | 适合场景 |
|
|
190
183
|
|---|---|
|
|
191
|
-
| `browser-e2e` | 浏览器截图、追踪、冒烟测试 |
|
|
192
184
|
| `ui-ux-review` | 响应式、无障碍和界面打磨 |
|
|
193
185
|
| `ts-react-frontend` | TypeScript、React、Vite 项目 |
|
|
194
186
|
| `python-backend` | FastAPI、pytest 项目 |
|
|
@@ -222,11 +214,11 @@ npm run check:mirrors
|
|
|
222
214
|
|
|
223
215
|
## 所有权清单
|
|
224
216
|
|
|
225
|
-
`Harness/ownership.manifest.json`
|
|
217
|
+
`Harness/ownership.manifest.json` 是安装和更新时文件分类的机器可读事实来源,由 `node scripts/build-version.mjs` 从 `templates/common/` 和 `templates/optional/catalog.json` 生成。框架拥有的文件走安全升级;用户数据(tasks、memory、research、README、package、PROGRESS)保留;CLAUDE、AGENTS、Harness README 走合并;同名用户 agent、command、skill(无 marker)不会被覆盖。
|
|
226
218
|
|
|
227
219
|
## 发布门禁
|
|
228
220
|
|
|
229
|
-
|
|
221
|
+
每次 Harness 更新发版都必须同时保持两个更新通道可用:
|
|
230
222
|
|
|
231
223
|
- Canonical:npm `create-harness-vibe-coding@latest` 和 `https://github.com/LiWeny16/create-harness-vibe-coding`
|
|
232
224
|
- 低版本兼容镜像:`https://github.com/zingspark/create-harness-vibe-coding`
|
|
@@ -238,29 +230,8 @@ npm run check:mirrors
|
|
|
238
230
|
## 适配范围与体积
|
|
239
231
|
|
|
240
232
|
- 支持 Claude Code、Codex 和 OpenCode 的共享 Harness 工作流。
|
|
241
|
-
- Node.js
|
|
233
|
+
- Node.js >= 18。
|
|
242
234
|
- 运行时无额外依赖;CLI 依赖 `@clack/prompts` 和 `picocolors`。
|
|
243
235
|
- 生成的是工作基础设施,不是业务应用代码。
|
|
244
236
|
|
|
245
|
-
## 项目结构
|
|
246
|
-
|
|
247
|
-
```text
|
|
248
|
-
my-project/
|
|
249
|
-
├── CLAUDE.md / AGENTS.md ← agent 入口
|
|
250
|
-
├── Harness/
|
|
251
|
-
│ ├── README.md ← 文档路由器
|
|
252
|
-
│ ├── MEMORY.md ← 资源索引
|
|
253
|
-
│ ├── PROGRESS.md ← 任务追踪器
|
|
254
|
-
│ ├── settings.json ← 框架配置
|
|
255
|
-
│ ├── specs/ ← 工作流、协议、运行时与安装指南
|
|
256
|
-
│ ├── project/ ← 项目架构文档
|
|
257
|
-
│ ├── tasks/ ← 任务胶囊
|
|
258
|
-
│ ├── research/ ← PRD 与研究模板
|
|
259
|
-
│ └── scripts/ ← 校验器
|
|
260
|
-
├── .claude/ ← Claude Code 配置
|
|
261
|
-
├── .agents/skills/ ← Codex repo skills
|
|
262
|
-
├── .codex/ ← Codex 配置
|
|
263
|
-
└── .opencode/ ← OpenCode 配置
|
|
264
|
-
```
|
|
265
|
-
|
|
266
237
|
MIT © [LiWeny16](https://github.com/LiWeny16)
|
package/README.md
CHANGED
|
@@ -16,8 +16,8 @@
|
|
|
16
16
|
</p>
|
|
17
17
|
|
|
18
18
|
<p align="center">
|
|
19
|
-
<a href="README-CN.md">Chinese</a>
|
|
20
|
-
<a href="https://github.com/LiWeny16/create-harness-vibe-coding">GitHub</a>
|
|
19
|
+
<a href="README-CN.md">Chinese</a> |
|
|
20
|
+
<a href="https://github.com/LiWeny16/create-harness-vibe-coding">GitHub</a> |
|
|
21
21
|
<a href="https://www.npmjs.com/package/create-harness-vibe-coding">npm</a>
|
|
22
22
|
</p>
|
|
23
23
|
|
|
@@ -30,25 +30,24 @@ It is not a prompt pack and it does not generate business code. It gives Claude
|
|
|
30
30
|
Without a framework:
|
|
31
31
|
|
|
32
32
|
```text
|
|
33
|
-
Idea
|
|
33
|
+
Idea -> prompt -> patch -> demo -> rediscover context -> repair drift
|
|
34
34
|
```
|
|
35
35
|
|
|
36
36
|
With Harness:
|
|
37
37
|
|
|
38
38
|
```text
|
|
39
|
-
Idea
|
|
40
|
-
|
|
39
|
+
Idea -> Research -> PRD -> Architecture -> Acceptance Criteria
|
|
40
|
+
-> Build -> Test -> Review -> Verify -> Learn -> Update
|
|
41
41
|
```
|
|
42
42
|
|
|
43
|
-
##
|
|
43
|
+
## Measured status in 0.8.16
|
|
44
44
|
|
|
45
|
-
|
|
45
|
+
Current evidence in this repository:
|
|
46
46
|
|
|
47
|
-
-
|
|
48
|
-
-
|
|
49
|
-
-
|
|
50
|
-
-
|
|
51
|
-
- Cross-runtime CLI docs: `wf-agents-docs` records source-backed Claude Code, Codex, and OpenCode invocation patterns for peer review, batch probes, and cache telemetry.
|
|
47
|
+
- HarnessBench v0.2 local lifecycle proof: 15 runs per mode. Direct file writes completed safely in 3/15 runs; the Harness safe path completed safely in 15/15 runs.
|
|
48
|
+
- Protected overwrites in that proof: 21 for direct file writes, 0 for the Harness safe path.
|
|
49
|
+
- Claude Code L2 prompt-cache sample: `harness-thin` reached `98.7%` warm median cache read (`+5.4` percentage points vs provider-control); `/wf` wf-light reached `99.1%` (`+5.8` percentage points).
|
|
50
|
+
- Startup and update chain: thin startup remains the default, `Harness/specs/guides/SETUP.md` is only setup/migration reference material, and `/wf-update` reports `agent.releaseHighlights` plus validation results.
|
|
52
51
|
|
|
53
52
|
## One-line install (recommended for existing repos)
|
|
54
53
|
|
|
@@ -74,12 +73,12 @@ The user does not need to run commands manually. The agent handles installation,
|
|
|
74
73
|
|
|
75
74
|
## Which WF command should you use?
|
|
76
75
|
|
|
77
|
-
When in doubt, use `/wf-help
|
|
76
|
+
When in doubt, use `/wf-help` (or `$wf-help` in Codex). It returns the full command table. Use `/wf` for complex work and `/wf-max` when the work can be safely parallelized.
|
|
78
77
|
|
|
79
78
|
| Command | Use it when | What it does | Example |
|
|
80
79
|
|---|---|---|---|
|
|
81
|
-
| `/wf <task>` | Multi-file, architectural, risky, migration, or repeatedly failing work | Research
|
|
82
|
-
| `/wf-max <task>` | The task splits into independent work and needs maximum parallelism | Adds CEO
|
|
80
|
+
| `/wf <task>` | Multi-file, architectural, risky, migration, or repeatedly failing work | Research -> plan -> implement -> test -> review -> verify -> reflect | `/wf refactor the payment module and add tests` |
|
|
81
|
+
| `/wf-max <task>` | The task splits into independent work and needs maximum parallelism | Adds CEO -> Manager -> Worker roles and parallel waves to the full WF chain | `/wf-max upgrade frontend, backend, and docs in parallel` |
|
|
83
82
|
| `/wf-auto` | You want continuous self-directed optimization with adaptive probe selection | Runs repeated optimization cycles with plans, evidence, and feedback | `/wf-auto improve this project's stability` |
|
|
84
83
|
| `/wf-auto-spark` | You need external inspiration, competitive direction, or a long-term roadmap | Searches for sparks, anchors work to a North Star and milestones, and guards scope drift | `/wf-auto-spark explore product growth directions` |
|
|
85
84
|
| `/wf-review [focus]` | You need a second opinion, peer review, or a pre-release check | Uses a peer CLI when available, otherwise an independent reviewer role, and classifies findings by severity | `/wf-review focus on security and data loss` |
|
|
@@ -90,26 +89,46 @@ When in doubt, use `/wf-help`. It returns the full command table. Use `/wf` for
|
|
|
90
89
|
| `/wf-remove` | You need to uninstall Harness | Removes safe files, preserves user data, and asks before touching conflicts | `/wf-remove` |
|
|
91
90
|
| `/wf-help` | You do not know which command to use | Returns command usage without starting a workflow | `/wf-help` |
|
|
92
91
|
|
|
93
|
-
Claude Code uses `/wf-*`; Codex uses the matching `$wf-*`; OpenCode uses the registered command or Agent instruction. `/wf-auto` and `/wf-auto-spark` are continuous modes, so give the agent a clear goal, scope, and acceptance criteria before starting.
|
|
92
|
+
Claude Code uses `/wf-*`; Codex uses the matching `$wf-*`; OpenCode uses the registered command or Agent instruction. Browser E2E guidance is built into `wf-browser`. `/wf-auto` and `/wf-auto-spark` are continuous modes, so give the agent a clear goal, scope, and acceptance criteria before starting.
|
|
94
93
|
|
|
95
94
|
Common starting points: Web/API work starts with correctness, security, reliability, and verification; CLI/SDK work starts with contracts, compatibility, error UX, and docs; AI-agent work starts with context quality, tool safety, evaluation, and recovery; data jobs start with idempotency, failure recovery, and observability. See the full [WF-AUTO-ANGLES.md](Harness/specs/workflows/WF-AUTO-ANGLES.md) selection protocol.
|
|
96
95
|
|
|
97
96
|
Chinese README: [README-CN.md](README-CN.md)
|
|
98
97
|
|
|
99
|
-
##
|
|
98
|
+
## Measured difference
|
|
100
99
|
|
|
101
|
-
|
|
100
|
+
Harness does not make the model smarter. It reduces uncontrolled parts of agent
|
|
101
|
+
work: which files may change, how conflicts are handled, how recovery works, and
|
|
102
|
+
what evidence counts as complete.
|
|
102
103
|
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
104
|
+
The current published benchmark is a deterministic local lifecycle proof, not a
|
|
105
|
+
full LLM feature-success A/B result. It creates 5 fixture families x 3 seeds per
|
|
106
|
+
mode and checks the filesystem after each run. See
|
|
107
|
+
[HarnessBench v0.2](docs/benchmarks/HarnessBench.md).
|
|
108
|
+
|
|
109
|
+
Benchmark assets stay outside generated installs: users get the published
|
|
110
|
+
summary, not the runner, fixtures, scorer, or raw logs.
|
|
111
|
+
|
|
112
|
+
### HarnessBench local lifecycle proof
|
|
113
|
+
|
|
114
|
+
Run on 2026-07-26:
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
node scripts/harness-bench-local.mjs --output benchmarks/results/harnessbench-local-v0.2.json
|
|
118
|
+
node scripts/harness-bench.mjs --input benchmarks/results/harnessbench-local-v0.2.json --markdown
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
| Mode | Tasks | Runs | Verified safe | Protected overwrites | Repair-triggering runs | Manual repair events | Required-file misses | Benchmark leaks | Boundary violations |
|
|
122
|
+
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
|
|
123
|
+
| No Harness baseline (direct file writes) (`direct-run`) | 5 | 15 | 3/15 (20%) | 21 | 12 | 21 | 0 | 0 | 12 |
|
|
124
|
+
| Harness safe path (`harness-wf`) | 5 | 15 | 15/15 (100%) | 0 | 0 | 0 | 0 | 0 | 0 |
|
|
111
125
|
|
|
112
|
-
|
|
126
|
+
What this proves: Harness has measurable boundary value for fresh install,
|
|
127
|
+
existing-project preservation, same-name user skill protection, old-Harness
|
|
128
|
+
updater recovery, and generated-install benchmark exclusion. It does not prove
|
|
129
|
+
that any model solves arbitrary frontend, backend, or embedded feature work at a
|
|
130
|
+
higher rate. That requires the full HarnessBench LLM A/B suite with raw run
|
|
131
|
+
logs.
|
|
113
132
|
|
|
114
133
|
### Prompt-cache L2 sample
|
|
115
134
|
|
|
@@ -125,15 +144,16 @@ Source: Claude Code JSON usage fields, especially `usage.cache_read_input_tokens
|
|
|
125
144
|
|
|
126
145
|
Claim boundary: this proves real cache reads and a measured improvement in this bounded Claude Code sample. It is not a universal claim for every model, repository, task, or provider.
|
|
127
146
|
|
|
128
|
-
|
|
147
|
+
### Compared with adjacent tools
|
|
129
148
|
|
|
130
|
-
|
|
149
|
+
Harness is a repo-local operating contract. It can sit above Claude Code, Codex, OpenCode, or another agent instead of replacing them.
|
|
131
150
|
|
|
132
|
-
|
|
|
151
|
+
| Compared with | Primary layer | Harness advantage |
|
|
133
152
|
|---|---|---|
|
|
134
|
-
|
|
|
135
|
-
|
|
|
136
|
-
|
|
|
153
|
+
| Direct agent run | One prompt plus ad hoc context | Persistent task state, explicit write boundaries, verification gates, and recoverable handoff |
|
|
154
|
+
| [Claude Code](https://code.claude.com/docs/en/overview), [Codex](https://developers.openai.com/codex), [OpenCode](https://opencode.ai/docs/) | Coding agents and runtimes | Cross-runtime repo contract with the same commands, memory shape, validator, and update policy |
|
|
155
|
+
| [Aider](https://github.com/aider-ai/aider) | Terminal pair-programming, repo map, git/test loop | Harness focuses on install/update safety, task capsules, external benchmark evidence, and multi-runtime workflow routing |
|
|
156
|
+
| [Superpowers](https://github.com/obra/Superpowers) | Skills-based development methodology | Harness adds an npm scaffold, machine-readable ownership/version manifests, safe merge/update scripts, and generated repo-local state |
|
|
137
157
|
|
|
138
158
|
## Architecture
|
|
139
159
|
|
|
@@ -143,15 +163,15 @@ README structure should meet people where decisions actually happen. Harness use
|
|
|
143
163
|
</a>
|
|
144
164
|
<br>
|
|
145
165
|
<sub>
|
|
146
|
-
Light infographic
|
|
166
|
+
Light infographic - <a href="docs/images/harness-architecture.drawio">editable Drawio source</a>
|
|
147
167
|
</sub>
|
|
148
168
|
</p>
|
|
149
169
|
|
|
150
170
|
The architecture has three pillars:
|
|
151
171
|
|
|
152
|
-
1. **Goals & Constraints**
|
|
153
|
-
2. **Quality Context**
|
|
154
|
-
3. **Decomposition & Feedback**
|
|
172
|
+
1. **Goals & Constraints** - PRD -> research -> architecture -> acceptance criteria.
|
|
173
|
+
2. **Quality Context** - a router, context-loading protocol, and durable memory keep attention on the right evidence.
|
|
174
|
+
3. **Decomposition & Feedback** - `/wf` and `/wf-max` assign bounded work, then review, verify, learn, and update the next task.
|
|
155
175
|
|
|
156
176
|
## What gets scaffolded
|
|
157
177
|
|
|
@@ -182,11 +202,10 @@ If `Harness/` already exists, first check whether `Harness/scripts/wf-update-che
|
|
|
182
202
|
|
|
183
203
|
Ask your agent to add the capability you need:
|
|
184
204
|
|
|
185
|
-
> Add `
|
|
205
|
+
> Add `ui-ux-review` and `ts-react-frontend` to this Harness project, preserve existing files, run the strict validator, and report exactly what changed.
|
|
186
206
|
|
|
187
207
|
| Workflow | Use it for |
|
|
188
208
|
|---|---|
|
|
189
|
-
| `browser-e2e` | Screenshots, traces, and smoke tests |
|
|
190
209
|
| `ui-ux-review` | Responsive, accessibility, and polish review |
|
|
191
210
|
| `ts-react-frontend` | TypeScript, React, and Vite projects |
|
|
192
211
|
| `python-backend` | FastAPI and pytest projects |
|
|
@@ -239,11 +258,11 @@ Low-version installs can have updater scripts hardcoded to the legacy mirror. Do
|
|
|
239
258
|
|---|---|
|
|
240
259
|
| Runtime | None |
|
|
241
260
|
| Dependencies | 2 (`@clack/prompts`, `picocolors`) |
|
|
242
|
-
| Node.js |
|
|
261
|
+
| Node.js | >=18 |
|
|
243
262
|
| Generated application code | None until you choose a stack |
|
|
244
263
|
|
|
245
264
|
## Keywords and related concepts
|
|
246
265
|
|
|
247
|
-
AI coding agent framework
|
|
266
|
+
AI coding agent framework; agentic workflow; context engineering; long-running coding tasks; task orchestration; durable memory; safe merge; conflict handling; acceptance criteria; Claude Code; Codex; OpenCode; Node.js CLI; developer productivity
|
|
248
267
|
|
|
249
|
-
MIT
|
|
268
|
+
MIT (c) [LiWeny16](https://github.com/LiWeny16)
|
package/package.json
CHANGED