ai-dev-harness 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. package/COLLEAGUE_TRIAL_GUIDE.md +147 -0
  2. package/README.md +116 -52
  3. package/dist/agent.js +44 -31
  4. package/dist/cli.js +65 -42
  5. package/dist/config.js +79 -8
  6. package/dist/context.js +53 -21
  7. package/dist/init.js +8 -20
  8. package/dist/integration.js +22 -91
  9. package/dist/mcp.js +29 -112
  10. package/dist/runPaths.js +1 -0
  11. package/dist/summarize.js +42 -39
  12. package/dist/templates.js +39 -215
  13. package/package.json +3 -1
  14. package/scripts/read-evidence.ps1 +9 -3
  15. package/scripts/run-harness.ps1 +8 -5
  16. package/templates/default-project/AGENTS.md +9 -0
  17. package/templates/default-project/SKILLS.md +13 -0
  18. package/templates/default-project/TOOLS.md +8 -0
  19. package/templates/default-project/skills/add_feature/SKILL.md +34 -0
  20. package/templates/default-project/skills/add_feature/context.yaml +30 -0
  21. package/templates/default-project/skills/add_feature/verify.yaml +2 -0
  22. package/templates/default-project/skills/fix_bug/SKILL.md +45 -0
  23. package/templates/default-project/skills/fix_bug/context.yaml +30 -0
  24. package/templates/default-project/skills/fix_bug/verify.yaml +2 -0
  25. package/templates/default-project/skills/update_docs/SKILL.md +28 -0
  26. package/templates/default-project/skills/update_docs/context.yaml +26 -0
  27. package/templates/default-project/skills/update_docs/verify.yaml +2 -0
  28. package/templates/default-project/skills/write_tests/SKILL.md +30 -0
  29. package/templates/default-project/skills/write_tests/context.yaml +29 -0
  30. package/templates/default-project/skills/write_tests/verify.yaml +2 -0
  31. package/templates/integrations/claude/SKILL.md +79 -0
  32. package/templates/integrations/codex/AGENTS_BLOCK.md +51 -0
@@ -0,0 +1,147 @@
1
+ # AI Dev Harness Colleague Trial Guide
2
+
3
+ ## 1. Install
4
+
5
+ ```bash
6
+ npm install -g ai-dev-harness
7
+ ```
8
+
9
+ Check:
10
+
11
+ ```bash
12
+ harness --help
13
+ ```
14
+
15
+ ## 2. Initialize In A Business Repository
16
+
17
+ Run inside a Git repository:
18
+
19
+ ```bash
20
+ harness init
21
+ ```
22
+
23
+ This creates Harness config, project skills, Claude/Codex conversation integration, and helper scripts.
24
+
25
+ ## 3. Configure MCP Sources
26
+
27
+ Configure MCP in Claude/Codex so the agent can call the tools directly. Keep the same source metadata in `harness.yaml` for audit:
28
+
29
+ ```yaml
30
+ mcp:
31
+ codegraph:
32
+ type: sse
33
+ url: http://10.41.83.107:7878/mcp
34
+ headers: {}
35
+ knowledgeGraph:
36
+ type: http
37
+ url: http://10.41.83.107:9200/mcp
38
+ headers:
39
+ Authorization: Bearer ${OKF_TOKEN}
40
+ timeoutMs: 15000
41
+ ```
42
+
43
+ Set secrets outside Git:
44
+
45
+ ```powershell
46
+ $env:OKF_TOKEN="your-token"
47
+ ```
48
+
49
+ Claude/Codex should call MCP tools before triggering Harness and include important findings in the task text. Harness records MCP source configuration for audit, but does not implement MCP transport itself.
50
+
51
+ ## 4. Run A Smoke Test
52
+
53
+ ```bash
54
+ harness run --task "Smoke test only. Analyze repository context and do not modify repository files." --skill update_docs --agent claude --live
55
+ ```
56
+
57
+ With Codex:
58
+
59
+ ```bash
60
+ harness run --task "Smoke test only. Analyze repository context and do not modify repository files." --skill update_docs --agent codex --live
61
+ ```
62
+
63
+ `--live` streams Harness lifecycle logs and agent output in the current terminal.
64
+
65
+ ## 5. Use From Claude/Codex Conversation
66
+
67
+ After `harness init`, open Claude Code or Codex in the repository and say:
68
+
69
+ ```text
70
+ Use AI Dev Harness to run fix_bug: fix the download resume progress issue.
71
+ ```
72
+
73
+ The agent should:
74
+
75
+ 1. Use configured MCP tools first.
76
+ 2. Report MCP preflight findings in the current conversation.
77
+ 3. Trigger Harness through `.harness/ai-dev-harness/scripts/run-harness.ps1`.
78
+ 4. Stream Harness lifecycle logs.
79
+ 5. Read and summarize the evidence directory.
80
+
81
+ ## 6. Expected MCP Preflight Shape
82
+
83
+ ```text
84
+ ## MCP Preflight Findings
85
+ - CodeGraph tools used:
86
+ - Related files:
87
+ - Related symbols:
88
+ - Callers / dependencies:
89
+ - Impact scope:
90
+ - Knowledge Graph tools used:
91
+ - ADRs / decisions:
92
+ - Module notes:
93
+ - API notes / runbooks:
94
+ - Missing context:
95
+ ```
96
+
97
+ ## 7. Evidence Files
98
+
99
+ Each run writes `runs/<run_id>/`:
100
+
101
+ ```text
102
+ intake.json Task, skill, agent, Git state
103
+ mcp_queries.jsonl Agent-driven MCP source records
104
+ context_pack.md Context assembly protocol and local fallback context
105
+ agent_prompt.md Final prompt sent to Claude/Codex
106
+ plan.md Harness execution-stage protocol
107
+ execution.log Agent stdout/stderr
108
+ agent_contract.json Required-section contract check
109
+ git_diff.patch Code changes
110
+ verify.json Verification results
111
+ summary.md Review/PR summary
112
+ persist_suggestions.md Knowledge persistence suggestions
113
+ audit.jsonl Full audit event log
114
+ ```
115
+
116
+ Read evidence:
117
+
118
+ ```bash
119
+ powershell -ExecutionPolicy Bypass -File ".harness/ai-dev-harness/scripts/read-evidence.ps1" -Run <run_id>
120
+ ```
121
+
122
+ ## 8. Troubleshooting
123
+
124
+ If `harness` is not found, check npm global bin is in `PATH`.
125
+
126
+ If `harness run must execute inside a Git repository`, run inside a business Git repository and confirm:
127
+
128
+ ```bash
129
+ git status
130
+ ```
131
+
132
+ If Claude/Codex cannot see MCP tools, check the Claude/Codex MCP configuration first:
133
+
134
+ - Wrong MCP URL
135
+ - Missing bearer token
136
+ - Service down
137
+ - Endpoint path is not `/mcp`
138
+
139
+ Harness can still run with local repository fallback context, but the agent should report missing MCP context.
140
+
141
+ ## 9. Acceptance Checklist
142
+
143
+ - `harness init` creates config, skills, and Claude/Codex integration.
144
+ - Claude/Codex can trigger Harness from conversation.
145
+ - Agent reports MCP preflight findings.
146
+ - `agent_contract.json` shows whether required sections were produced.
147
+ - `verify.json`, `summary.md`, and `audit.jsonl` are generated.
package/README.md CHANGED
@@ -1,16 +1,30 @@
1
1
  # AI Dev Harness
2
2
 
3
- Auditable TypeScript CLI harness that wraps Codex CLI or Claude Code, retrieves context from CodeGraph and Knowledge Graph MCP services, verifies changes, and writes local run evidence.
3
+ AI Dev Harness 是一个可分发的 TypeScript CLI,用来把 Claude Code / Codex 的研发任务变成可审计的执行闭环。
4
4
 
5
- This repository is a distributable Harness package:
5
+ 它的核心定位不是替代 Agent,也不是实现 MCP Client,而是:
6
6
 
7
- - CLI: `dist/cli.js`
8
- - Claude/Codex conversation integration installer
9
- - Claude Skill package installed into `.claude/skills/ai-dev-harness/`
10
- - Agent helper scripts installed into `.harness/ai-dev-harness/scripts/`
11
- - Git-native project skills installed into `skills/`
7
+ > 让大模型知道如何组装上下文、制定计划、执行任务、验证结果,并把整个生命周期记录下来。
12
8
 
13
- ## Quick Start
9
+ ## 核心边界
10
+
11
+ - Claude / Codex 负责:理解任务、调用 MCP、调用工具、使用 Skill、修改代码。
12
+ - Harness 负责:执行闭环、过程日志、验证命令、审计证据、总结、沉淀建议。
13
+ - CodeGraph / Knowledge Graph 是 Agent 侧 MCP 工具源,不是 Harness 内部服务依赖。
14
+ - Harness 会记录 MCP source metadata,但不会记录 bearer token 原文。
15
+
16
+ ## 包内容
17
+
18
+ - CLI:`dist/cli.js`
19
+ - Claude/Codex 对话集成安装器
20
+ - Claude Skill:`.claude/skills/ai-dev-harness/`
21
+ - Agent 辅助脚本:`.harness/ai-dev-harness/scripts/`
22
+ - Git-native 项目技能:`skills/`
23
+ - 可编辑模板:`templates/`
24
+
25
+ 同事试用手册见 `COLLEAGUE_TRIAL_GUIDE.md`。
26
+
27
+ ## 快速开始
14
28
 
15
29
  ```bash
16
30
  npm install
@@ -19,13 +33,13 @@ node dist/cli.js init
19
33
  node dist/cli.js run --task "fix a small bug" --skill fix_bug --agent codex
20
34
  ```
21
35
 
22
- The CLI is intended to be installed inside a business Git repository. `init` creates `harness.yaml`, `skills/`, `AGENTS.md`, `TOOLS.md`, `SKILLS.md`, and `runs/`.
23
- `harness run` must execute inside a Git repository; if it is launched elsewhere, the harness writes intake/audit evidence and exits nonzero.
36
+ CLI 应在业务 Git 仓库内初始化。`init` 会创建 `harness.yaml`、`skills/`、`AGENTS.md`、`TOOLS.md`、`SKILLS.md` `runs/`。
24
37
 
25
- ## Commands
38
+ ## 命令
26
39
 
27
40
  ```bash
28
41
  node dist/cli.js init
42
+ node dist/cli.js init --no-integration
29
43
  node dist/cli.js run --task "add refund flow tests" --skill write_tests --agent claude
30
44
  node dist/cli.js run --task "analyze download startup flow. Do not modify repository files." --skill update_docs --agent claude --live
31
45
  node dist/cli.js verify --run <run_id_or_path>
@@ -33,50 +47,58 @@ node dist/cli.js summarize --run <run_id_or_path>
33
47
  node dist/cli.js install-integration --agent both --target <repo>
34
48
  ```
35
49
 
36
- ## Claude/Codex Conversation Trigger
50
+ ## Claude/Codex 对话触发
37
51
 
38
- Install conversation integration into a business repository:
52
+ `harness init` 默认安装对话集成。也可以手动刷新:
39
53
 
40
54
  ```bash
41
55
  node "E:\ai native\dist\cli.js" install-integration --agent both --target "C:\path\to\repo"
42
56
  ```
43
57
 
44
- This creates:
58
+ 它会创建:
45
59
 
46
- - `.claude/skills/ai-dev-harness/SKILL.md` for Claude Code.
47
- - An `AGENTS.md` Harness section for Codex.
60
+ - `.claude/skills/ai-dev-harness/SKILL.md`
61
+ - `AGENTS.md` 中的 Harness 规则块
48
62
  - `.harness/ai-dev-harness/scripts/run-harness.ps1`
49
63
  - `.harness/ai-dev-harness/scripts/read-evidence.ps1`
50
64
 
51
- Then, inside Claude Code or Codex, ask naturally:
65
+ 然后在 Claude Code Codex 中自然提问:
52
66
 
53
67
  ```text
54
68
  Use AI Dev Harness to run update_docs: analyze the download startup flow. Do not modify repository files.
55
69
  ```
56
70
 
57
- or:
71
+ 或:
58
72
 
59
73
  ```text
60
74
  Use Harness to run fix_bug: fix the download resume progress issue.
61
75
  ```
62
76
 
63
- The generated integration uses `--live`, so the current conversation terminal can see Harness stages and agent output as the run progresses.
77
+ 集成脚本默认使用 `--live`,所以当前对话终端能看到 Harness 阶段日志和 Agent 输出。
64
78
 
65
- The installed Skill prefers the helper script:
79
+ ## MCP Preflight
66
80
 
67
- ```bash
68
- powershell -ExecutionPolicy Bypass -File ".harness/ai-dev-harness/scripts/run-harness.ps1" -Task "<task>" -Skill fix_bug -Agent claude -Live
69
- ```
70
-
71
- The direct CLI command remains available as a fallback:
81
+ Harness 不直接调用 MCP。Agent 应在触发 Harness 前使用自己已配置的 MCP 工具,并按以下格式汇报:
72
82
 
73
- ```bash
74
- node "E:\ai native\dist\cli.js" run --task "<task>" --skill fix_bug --agent claude --live
83
+ ```text
84
+ ## MCP Preflight Findings
85
+ - CodeGraph tools used:
86
+ - Related files:
87
+ - Related symbols:
88
+ - Callers / dependencies:
89
+ - Impact scope:
90
+ - Knowledge Graph tools used:
91
+ - ADRs / decisions:
92
+ - Module notes:
93
+ - API notes / runbooks:
94
+ - Missing context:
75
95
  ```
76
96
 
77
- ## Configuration
97
+ 关键 MCP 发现应被带入 Harness 的任务文本,进入审计证据。
98
+
99
+ ## 配置
78
100
 
79
- Edit `harness.yaml` in the business repository:
101
+ 编辑业务仓库里的 `harness.yaml`:
80
102
 
81
103
  ```yaml
82
104
  agents:
@@ -86,9 +108,14 @@ agents:
86
108
  command: claude -p -
87
109
  mcp:
88
110
  codegraph:
111
+ type: sse
89
112
  url: http://localhost:7331/mcp
113
+ headers: {}
90
114
  knowledgeGraph:
115
+ type: http
91
116
  url: http://localhost:7332/mcp
117
+ headers:
118
+ Authorization: Bearer ${OKF_TOKEN}
92
119
  timeoutMs: 15000
93
120
  verify:
94
121
  defaultCommands:
@@ -98,34 +125,71 @@ audit:
98
125
  path: runs
99
126
  ```
100
127
 
101
- The agent command receives the generated prompt on stdin. CodeGraph and Knowledge Graph MCP are queried first; if either service is unavailable, the run is marked degraded and the harness falls back to local Git/file context.
102
- If the agent command or verification fails, the run still writes all evidence files and then exits nonzero so CI can treat the run as failed.
128
+ MCP header 支持环境变量,例如 `${OKF_TOKEN}`。不要把真实 token 提交到 Git
103
129
 
104
- ## Run Evidence
130
+ ## 运行证据
105
131
 
106
- Each `run` creates `runs/<run_id>/` with:
132
+ 每次 `run` 会创建 `runs/<run_id>/`:
107
133
 
108
- - `intake.json`
109
- - `mcp_queries.jsonl`
110
- - `context_pack.md`
111
- - `agent_prompt.md`
112
- - `plan.md`
113
- - `execution.log`
114
- - `git_diff.patch`
115
- - `verify.json`
116
- - `summary.md`
117
- - `persist_suggestions.md`
118
- - `audit.jsonl`
134
+ - `intake.json`:任务、skill、agent、Git 状态。
135
+ - `mcp_queries.jsonl`:Agent-driven MCP source metadata,不包含 token 原文。
136
+ - `context_pack.md`:上下文组装协议、MCP source 记录、本地兜底上下文。
137
+ - `agent_prompt.md`:发送给 Claude/Codex 的最终提示词。
138
+ - `plan.md`:Harness 执行阶段协议。
139
+ - `execution.log`:Agent stdout/stderr。
140
+ - `agent_contract.json`:Agent 必需章节检查结果。
141
+ - `git_diff.patch`:代码变更。
142
+ - `verify.json`:验证结果。
143
+ - `summary.md`:评审/PR 总结。
144
+ - `persist_suggestions.md`:知识沉淀建议。
145
+ - `audit.jsonl`:完整审计事件流。
119
146
 
120
- Knowledge Graph updates are not applied automatically in v1. Review `persist_suggestions.md` before writing them back.
147
+ Harness 会检查 Agent 输出是否包含以下章节:
121
148
 
122
- ## Engineering Prompt Contracts
149
+ - `Impact Analysis`
150
+ - `Plan`
151
+ - `Execution Notes`
152
+ - `Verification Notes`
153
+ - `Final Report`
123
154
 
124
- Generated skills and agent prompts use a company-grade execution contract:
155
+ 检查结果写入 `agent_contract.json`,并记录到 `audit.jsonl`。
125
156
 
126
- - Analysis Contract: restate the task, list assumptions, affected areas, risks, and missing context.
127
- - Execution Contract: make only task-relevant minimal changes; no opportunistic refactors or broad cleanup.
128
- - Verification Contract: run configured verification or record why checks could not run.
129
- - Report Contract: summarize intent or root cause, changed files, verification, residual risk, and knowledge suggestions.
157
+ ## Skill 规范
130
158
 
131
- The test suite verifies that initialized skills and run evidence include these contracts.
159
+ 内置 skill 会通过 `context.yaml` 告诉 Agent:
160
+
161
+ - 应该从 CodeGraph MCP 查询什么。
162
+ - 应该从 Knowledge Graph MCP 查询什么。
163
+ - 什么情况下必须停止,不要猜。
164
+ - 本地兜底上下文应包含哪些路径。
165
+
166
+ 示例:
167
+
168
+ ```yaml
169
+ mcp_preflight:
170
+ codegraph:
171
+ required: true
172
+ ask_for:
173
+ - related_files
174
+ - related_symbols
175
+ - callers
176
+ - dependencies
177
+ - impact_scope
178
+ knowledgeGraph:
179
+ required: true
180
+ ask_for:
181
+ - adr
182
+ - module_notes
183
+ - api_notes
184
+ - runbooks
185
+ stop_conditions:
186
+ - root cause is unclear
187
+ - impact scope cannot be determined
188
+ ```
189
+
190
+ ## 验证
191
+
192
+ ```bash
193
+ npm test
194
+ npm pack --dry-run
195
+ ```
package/dist/agent.js CHANGED
@@ -1,54 +1,67 @@
1
1
  import { appendFile, readFile, writeFile } from "node:fs/promises";
2
2
  import { git, runCommand, truncate } from "./utils.js";
3
3
  function renderPrompt(input, contextPack) {
4
- return `You are running inside an auditable AI development harness. Treat this as a company engineering task, not a casual chat.
4
+ return `你正在一个可审计的 AI 研发 Harness 中执行任务。请把它当作公司级研发任务,而不是随意聊天。
5
5
 
6
- Task:
6
+ 任务:
7
7
  ${input.task}
8
8
 
9
9
  Skill:
10
10
  ${input.skill.name}
11
11
 
12
- Execution Contract:
13
- 1. Analysis first.
14
- - Restate the task in one or two sentences.
15
- - Identify assumptions, affected files/modules, relevant CodeGraph context, relevant Knowledge Graph context, and risks.
16
- - If required context is missing or confidence is low, stop and report what is missing instead of guessing.
17
- 2. Plan before editing.
18
- - Provide a minimal step-by-step plan.
19
- - Keep the plan limited to the requested task.
20
- 3. Safe execution.
21
- - Make the smallest task-relevant change.
22
- - Do not perform opportunistic refactors, formatting-only sweeps, dependency upgrades, broad renames, or unrelated cleanup.
23
- - Preserve public interfaces and backward compatibility unless explicitly instructed otherwise.
24
- - Do not update Knowledge Graph directly.
25
- 4. Verification discipline.
26
- - Harness will run configured verification after you finish.
27
- - Only run extra commands when needed to understand or validate the task.
28
- - Report any verification you could not run and why.
29
- 5. Required final report.
30
- - Root cause or implementation intent.
31
- - Files changed.
32
- - Verification performed or deferred.
33
- - Residual risk.
34
- - Knowledge persistence suggestions.
12
+ 执行契约:
13
+ 1. 先分析。
14
+ - 用一两句话复述任务。
15
+ - 识别假设、受影响文件/模块、CodeGraph 上下文、Knowledge Graph 上下文和风险。
16
+ - 如果关键上下文缺失或信心不足,停止并说明缺什么,不要猜。
17
+ 2. 先计划再修改。
18
+ - 给出最小步骤计划。
19
+ - 计划必须限定在当前任务范围内。
20
+ 3. 安全执行。
21
+ - 只做和任务相关的最小变更。
22
+ - 不做机会主义重构、纯格式化清扫、依赖升级、大范围重命名或无关清理。
23
+ - 除非任务明确要求,否则保持公共接口和向后兼容。
24
+ - 不直接写入 Knowledge Graph
25
+ 4. 验证纪律。
26
+ - Harness 会在你完成后运行配置的验证命令。
27
+ - 只有在理解或验证任务确实需要时才额外运行命令。
28
+ - 说明无法执行的验证及原因。
29
+ 5. 必须输出最终报告。
30
+ - 根因或实现意图。
31
+ - 修改文件。
32
+ - 已执行或延后的验证。
33
+ - 残余风险。
34
+ - 知识沉淀建议。
35
35
 
36
- Output Format:
36
+ 必须使用以下输出格式,标题不要改名:
37
37
  ## Impact Analysis
38
38
  ## Plan
39
39
  ## Execution Notes
40
40
  ## Verification Notes
41
41
  ## Final Report
42
42
 
43
- Project Rules:
44
- - Respect AGENTS.md, TOOLS.md, and SKILLS.md when present.
45
- - Work inside this repository only.
46
- - Leave an auditable, reviewable result.
43
+ 项目规则:
44
+ - 遵守 AGENTS.mdTOOLS.md SKILLS.md
45
+ - 只在当前仓库内工作。
46
+ - 留下可审计、可复查的结果。
47
47
 
48
- Context pack:
48
+ Context Pack:
49
49
  ${contextPack}
50
50
  `;
51
51
  }
52
+ export function checkAgentContract(output) {
53
+ const required = ["Impact Analysis", "Plan", "Execution Notes", "Verification Notes", "Final Report"];
54
+ const requiredSections = required.map((name) => ({
55
+ name,
56
+ present: new RegExp(`^##\\s+${name}\\s*$`, "im").test(output)
57
+ }));
58
+ const missingSections = requiredSections.filter((section) => !section.present).map((section) => section.name);
59
+ return {
60
+ status: missingSections.length ? "failed" : "passed",
61
+ requiredSections,
62
+ missingSections
63
+ };
64
+ }
52
65
  async function executeAgent(input) {
53
66
  const contextPack = await readFile(input.contextPackPath, "utf8");
54
67
  const prompt = renderPrompt(input, contextPack);