@dommaker/harness 0.7.8 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +171 -822
- package/bin/harness.js +122 -85
- package/dist/cli/commands/acceptance.d.ts +26 -0
- package/dist/cli/commands/acceptance.d.ts.map +1 -0
- package/dist/cli/commands/acceptance.js +130 -0
- package/dist/cli/commands/acceptance.js.map +1 -0
- package/dist/cli/commands/check.js +3 -3
- package/dist/cli/commands/check.js.map +1 -1
- package/dist/cli/commands/contract.d.ts +24 -0
- package/dist/cli/commands/contract.d.ts.map +1 -0
- package/dist/cli/commands/contract.js +182 -0
- package/dist/cli/commands/contract.js.map +1 -0
- package/dist/cli/commands/index.d.ts +5 -3
- package/dist/cli/commands/index.d.ts.map +1 -1
- package/dist/cli/commands/index.js +15 -7
- package/dist/cli/commands/index.js.map +1 -1
- package/dist/cli/commands/init.js +9 -0
- package/dist/cli/commands/init.js.map +1 -1
- package/dist/cli/commands/performance.d.ts +26 -0
- package/dist/cli/commands/performance.d.ts.map +1 -0
- package/dist/cli/commands/performance.js +86 -0
- package/dist/cli/commands/performance.js.map +1 -0
- package/dist/cli/commands/review.d.ts +26 -0
- package/dist/cli/commands/review.d.ts.map +1 -0
- package/dist/cli/commands/review.js +137 -0
- package/dist/cli/commands/review.js.map +1 -0
- package/dist/cli/commands/security.d.ts +26 -0
- package/dist/cli/commands/security.d.ts.map +1 -0
- package/dist/cli/commands/security.js +147 -0
- package/dist/cli/commands/security.js.map +1 -0
- package/dist/cli/commands/status.d.ts.map +1 -1
- package/dist/cli/commands/status.js +4 -5
- package/dist/cli/commands/status.js.map +1 -1
- package/dist/core/constraints/checker.d.ts +24 -0
- package/dist/core/constraints/checker.d.ts.map +1 -1
- package/dist/core/constraints/checker.js +191 -12
- package/dist/core/constraints/checker.js.map +1 -1
- package/dist/core/constraints/definitions.d.ts.map +1 -1
- package/dist/core/constraints/definitions.js +71 -0
- package/dist/core/constraints/definitions.js.map +1 -1
- package/dist/core/validators/passes-gate.d.ts +26 -1
- package/dist/core/validators/passes-gate.d.ts.map +1 -1
- package/dist/core/validators/passes-gate.js +60 -0
- package/dist/core/validators/passes-gate.js.map +1 -1
- package/dist/extensions/long-running/constraints.d.ts +38 -0
- package/dist/extensions/long-running/constraints.d.ts.map +1 -0
- package/dist/extensions/long-running/constraints.js +153 -0
- package/dist/extensions/long-running/constraints.js.map +1 -0
- package/dist/extensions/long-running/index.d.ts +30 -0
- package/dist/extensions/long-running/index.d.ts.map +1 -0
- package/dist/extensions/long-running/index.js +50 -0
- package/dist/extensions/long-running/index.js.map +1 -0
- package/dist/extensions/long-running/types.d.ts +154 -0
- package/dist/extensions/long-running/types.d.ts.map +1 -0
- package/dist/extensions/long-running/types.js +9 -0
- package/dist/extensions/long-running/types.js.map +1 -0
- package/dist/presets/long-running.d.ts +33 -0
- package/dist/presets/long-running.d.ts.map +1 -0
- package/dist/presets/long-running.js +52 -0
- package/dist/presets/long-running.js.map +1 -0
- package/dist/types/constraint.d.ts +5 -1
- package/dist/types/constraint.d.ts.map +1 -1
- package/dist/types/constraint.js.map +1 -1
- package/dist/types/passes-gate.d.ts +47 -0
- package/dist/types/passes-gate.d.ts.map +1 -1
- package/dist/utils/exec.d.ts +17 -0
- package/dist/utils/exec.d.ts.map +1 -0
- package/dist/utils/exec.js +42 -0
- package/dist/utils/exec.js.map +1 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,922 +1,271 @@
|
|
|
1
1
|
# @dommaker/harness
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> AI Agent 的工程约束框架 — 让 Agent 不会乱承诺、不会跳过验证、不会简化测试。
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
`@dommaker/harness` 是一个通用的工程约束框架,帮助团队建立和强制执行代码质量标准。
|
|
8
|
-
|
|
9
|
-
### 核心功能
|
|
5
|
+
---
|
|
10
6
|
|
|
11
|
-
|
|
12
|
-
|------|------|
|
|
13
|
-
| **铁律系统** | 16 条内置约束(4 Iron Laws + 10 Guidelines + 2 Tips) |
|
|
14
|
-
| **门禁系统** | 7 种门禁(测试、审查、安全、性能、契约、检查点、验收标准) |
|
|
15
|
-
| **检查点验证** | 验证工作流步骤的结果是否符合预期 |
|
|
16
|
-
| **拦截器** | 抽象拦截框架,自动执行 enforcement |
|
|
17
|
-
| **Session 管理** | 启动检查点 + 结束状态管理 |
|
|
18
|
-
| **预设系统** | 提供 strict/standard/relaxed 三种预设 |
|
|
19
|
-
| **Execution Trace** | 轻量记录约束检查,异常检测,诊断系统 |
|
|
20
|
-
| **Performance Trace** | 性能日志收集,统计分析,异常检测(🆕) |
|
|
21
|
-
| **Failure Classification** | 错误分类,失败记录,等级映射(🆕) |
|
|
22
|
-
| **Spec 验证** | 验证架构文档、模块定义、API 定义 |
|
|
23
|
-
| **项目级自定义约束** | 扩展/覆盖内置约束,无需 fork |
|
|
24
|
-
| **CLI 工具** | 命令行工具执行检查 |
|
|
7
|
+
## 🚀 快速开始
|
|
25
8
|
|
|
26
|
-
|
|
9
|
+
### 安装
|
|
27
10
|
|
|
28
11
|
```bash
|
|
29
|
-
npm install @dommaker/harness
|
|
12
|
+
npm install -g @dommaker/harness
|
|
30
13
|
```
|
|
31
14
|
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
### 1. 初始化项目
|
|
15
|
+
### 最小工作流
|
|
35
16
|
|
|
36
17
|
```bash
|
|
37
|
-
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
这会创建:
|
|
41
|
-
- `.harness/config.yml` - 预设配置
|
|
42
|
-
- `.harness/checkpoints.yml` - 示例检查点
|
|
43
|
-
- `CAPABILITIES.md` - 功能清单模板
|
|
44
|
-
- `.git/hooks/pre-commit` - Git 钩子(可选)
|
|
45
|
-
- `.github/workflows/harness-check.yml` - CI 检查(可选)
|
|
46
|
-
|
|
47
|
-
### 2. CLI 命令
|
|
48
|
-
|
|
49
|
-
```bash
|
|
50
|
-
# 初始化项目配置
|
|
51
|
-
harness init --preset standard
|
|
52
|
-
|
|
53
|
-
# 输出配置代码片段(不创建文件)
|
|
54
|
-
harness init --print-snippets
|
|
55
|
-
|
|
56
|
-
# 检查铁律
|
|
18
|
+
# 开发前
|
|
57
19
|
harness check
|
|
58
20
|
|
|
59
|
-
#
|
|
60
|
-
harness check --list
|
|
61
|
-
|
|
62
|
-
# 查看状态(统计、异常、建议)
|
|
63
|
-
harness status
|
|
64
|
-
|
|
65
|
-
# 查看详细状态
|
|
66
|
-
harness status --detail
|
|
21
|
+
# 开发 + 写测试
|
|
67
22
|
|
|
68
|
-
#
|
|
69
|
-
harness
|
|
70
|
-
|
|
71
|
-
# 一键执行诊断+提案流程
|
|
72
|
-
harness flow
|
|
23
|
+
# 提交前
|
|
24
|
+
harness passes-gate
|
|
25
|
+
```
|
|
73
26
|
|
|
74
|
-
|
|
75
|
-
harness validate
|
|
27
|
+
---
|
|
76
28
|
|
|
77
|
-
|
|
78
|
-
harness passes-gate
|
|
29
|
+
## 📋 CLI 命令
|
|
79
30
|
|
|
80
|
-
|
|
81
|
-
harness report
|
|
31
|
+
### 日常使用
|
|
82
32
|
|
|
83
|
-
|
|
84
|
-
harness
|
|
85
|
-
harness
|
|
86
|
-
harness
|
|
33
|
+
```bash
|
|
34
|
+
harness check # 检查铁律(开发前后必用)
|
|
35
|
+
harness passes-gate # 测试门控(提交前验证)
|
|
36
|
+
harness status # 查看状态(异常、统计)
|
|
37
|
+
harness flow # 一键诊断 + 提案流程
|
|
87
38
|
```
|
|
88
39
|
|
|
89
|
-
###
|
|
40
|
+
### 项目初始化
|
|
90
41
|
|
|
91
|
-
```
|
|
92
|
-
#
|
|
93
|
-
|
|
94
|
-
harness-check:
|
|
95
|
-
runs-on: ubuntu-latest
|
|
96
|
-
steps:
|
|
97
|
-
- uses: actions/checkout@v4
|
|
98
|
-
- run: npm ci
|
|
99
|
-
- run: npx harness check
|
|
100
|
-
- run: npx harness passes-gate
|
|
42
|
+
```bash
|
|
43
|
+
harness init --preset standard # 初始化配置
|
|
44
|
+
harness init --print-snippets # 输出代码片段
|
|
101
45
|
```
|
|
102
46
|
|
|
103
|
-
###
|
|
104
|
-
|
|
105
|
-
验证架构文档、模块定义、API 定义等 Spec 文件:
|
|
47
|
+
### 门禁检查
|
|
106
48
|
|
|
107
49
|
```bash
|
|
108
|
-
#
|
|
109
|
-
harness
|
|
50
|
+
# 测试门控
|
|
51
|
+
harness passes-gate # 运行测试
|
|
52
|
+
harness passes-gate --coverage # 检查覆盖率
|
|
53
|
+
harness pg # 别名
|
|
110
54
|
|
|
111
|
-
#
|
|
112
|
-
harness
|
|
55
|
+
# 验收标准
|
|
56
|
+
harness acceptance --task-id TASK-001 # 检查指定任务
|
|
57
|
+
harness acceptance --check-all # 检查所有任务
|
|
58
|
+
harness acceptance list # 列出验收标准
|
|
59
|
+
harness acc # 别名
|
|
113
60
|
|
|
114
|
-
#
|
|
115
|
-
harness
|
|
61
|
+
# 性能门控
|
|
62
|
+
harness performance --coverage --coverage-threshold 85 # 覆盖率检查
|
|
63
|
+
harness performance --bundle --bundle-threshold 300 # 打包大小检查
|
|
64
|
+
harness perf # 别名
|
|
116
65
|
|
|
117
|
-
#
|
|
118
|
-
harness
|
|
66
|
+
# 安全门控
|
|
67
|
+
harness security # npm audit 检查
|
|
68
|
+
harness security --severity critical # 只显示 critical 级别
|
|
69
|
+
harness security audit # 详细漏洞报告
|
|
70
|
+
harness sec # 别名
|
|
119
71
|
|
|
120
|
-
#
|
|
121
|
-
harness
|
|
122
|
-
|
|
72
|
+
# API 契约
|
|
73
|
+
harness contract # OpenAPI Schema 验证
|
|
74
|
+
harness contract validate # Schema 语法验证
|
|
75
|
+
harness contract --contract-path api/openapi.yaml
|
|
123
76
|
|
|
124
|
-
|
|
77
|
+
# 代码审查
|
|
78
|
+
harness review # PR 审查状态检查
|
|
79
|
+
harness review --min-reviewers 2 # 要求 2 个审批
|
|
80
|
+
harness review status # PR 详情
|
|
81
|
+
```
|
|
125
82
|
|
|
126
|
-
|
|
83
|
+
### Spec 验证
|
|
127
84
|
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
export const ArchitectureSchema = z.object({
|
|
133
|
-
name: z.string(),
|
|
134
|
-
version: z.string(),
|
|
135
|
-
modules: z.array(z.string()),
|
|
136
|
-
});
|
|
137
|
-
|
|
138
|
-
export async function validate(content: string, filePath: string) {
|
|
139
|
-
// 解析并验证内容
|
|
140
|
-
// 返回 SpecValidationResult
|
|
141
|
-
}
|
|
85
|
+
```bash
|
|
86
|
+
harness spec # 验证所有 Spec
|
|
87
|
+
harness spec --staged # 验证暂存文件
|
|
88
|
+
harness spec list # 支持的 Spec 类型
|
|
142
89
|
```
|
|
143
90
|
|
|
144
|
-
###
|
|
91
|
+
### 检查点验证
|
|
145
92
|
|
|
146
|
-
```
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
SessionStartup,
|
|
152
|
-
CleanStateManager
|
|
153
|
-
} from '@dommaker/harness';
|
|
154
|
-
|
|
155
|
-
// 检查铁律
|
|
156
|
-
const checker = IronLawChecker.getInstance();
|
|
157
|
-
const results = await checker.checkAll(context);
|
|
93
|
+
```bash
|
|
94
|
+
harness validate # 验证检查点
|
|
95
|
+
harness validate --strict # 严格模式
|
|
96
|
+
harness validate -f checkpoint.json # 指定文件
|
|
97
|
+
```
|
|
158
98
|
|
|
159
|
-
|
|
160
|
-
const validator = new CheckpointValidator();
|
|
161
|
-
const result = await validator.validate(checkpoints, context);
|
|
99
|
+
### 报告
|
|
162
100
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
101
|
+
```bash
|
|
102
|
+
harness report # 生成 Markdown 报告
|
|
103
|
+
harness report -f json # JSON 格式
|
|
104
|
+
harness report -o report.md # 输出到文件
|
|
105
|
+
```
|
|
166
106
|
|
|
167
|
-
|
|
168
|
-
const startup = new SessionStartup(workDir, checkpoints);
|
|
169
|
-
const { success, results } = await startup.run();
|
|
107
|
+
### 诊断
|
|
170
108
|
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
109
|
+
```bash
|
|
110
|
+
harness flow # 一键诊断 + 提案
|
|
111
|
+
harness flow --auto-apply # 自动应用低风险提案
|
|
174
112
|
```
|
|
175
113
|
|
|
176
|
-
|
|
114
|
+
---
|
|
177
115
|
|
|
178
|
-
|
|
116
|
+
## 🎯 核心概念
|
|
117
|
+
|
|
118
|
+
### 三层约束体系
|
|
119
|
+
|
|
120
|
+
| 层级 | 严重性 | 说明 |
|
|
121
|
+
|------|:------:|------|
|
|
122
|
+
| **Iron Law** | 🔴 error | 绝对禁止,无例外 |
|
|
123
|
+
| **Guideline** | 🟡 warning | 推荐遵守,有例外 |
|
|
124
|
+
| **Tip** | 🔵 info | 信息性提示 |
|
|
125
|
+
|
|
126
|
+
### 7 条铁律
|
|
127
|
+
|
|
128
|
+
| ID | 规则 |
|
|
129
|
+
|---|------|
|
|
130
|
+
| `no_bypass_checkpoint` | 禁止跳过检查点 |
|
|
131
|
+
| `no_self_approval` | 禁止自评通过(必须测试)|
|
|
132
|
+
| `no_completion_without_verification` | 完成必须验证 |
|
|
133
|
+
| `no_test_simplification` | 禁止简化测试 |
|
|
134
|
+
| `incremental_progress` | 单任务单会话 |
|
|
135
|
+
| `verify_external_capability` | 外部能力先验证 |
|
|
136
|
+
| `no_implementation_without_requirement_review` | 实现后对比需求 |
|
|
137
|
+
|
|
138
|
+
### 7 种门禁(均有 CLI 命令)
|
|
139
|
+
|
|
140
|
+
| 门禁 | CLI 命令 | 用途 |
|
|
141
|
+
|------|:--------:|------|
|
|
142
|
+
| PassesGate | `passes-gate` / `pg` | 测试门控 |
|
|
143
|
+
| SpecAcceptanceGate | `acceptance` / `acc` | 验收标准检查 |
|
|
144
|
+
| PerformanceGate | `performance` / `perf` | 性能门控 |
|
|
145
|
+
| SecurityGate | `security` / `sec` | 安全检查 |
|
|
146
|
+
| ContractGate | `contract` | API 契约验证 |
|
|
147
|
+
| ReviewGate | `review` | PR 审核检查 |
|
|
148
|
+
| CheckpointValidator | `validate` | 检查点验证 |
|
|
149
|
+
| CheckpointValidator | 检查点验证 |
|
|
150
|
+
| SpecAcceptanceGate | 验收标准检查 |
|
|
179
151
|
|
|
180
|
-
|
|
181
|
-
|------|-----|------|
|
|
182
|
-
| 测试门控 | `PassesGate` | 禁止自评通过,必须通过真实测试 |
|
|
183
|
-
| 审查门禁 | `ReviewGate` | 检查 GitHub PR 审查状态 |
|
|
184
|
-
| 安全门禁 | `SecurityGate` | npm audit 安全漏洞扫描 |
|
|
185
|
-
| 性能门禁 | `PerformanceGate` | 响应时间、覆盖率、打包大小检查 |
|
|
186
|
-
| 契约门禁 | `ContractGate` | OpenAPI 契约验证 |
|
|
187
|
-
| 检查点验证 | `CheckpointValidator` | 验证工作流步骤结果 |
|
|
188
|
-
| 验收标准门禁 | `SpecAcceptanceGate` | 验证任务是否满足验收标准 |
|
|
152
|
+
---
|
|
189
153
|
|
|
190
|
-
|
|
154
|
+
## 🔧 代码集成
|
|
191
155
|
|
|
192
156
|
```typescript
|
|
193
157
|
import {
|
|
158
|
+
IronLawChecker,
|
|
194
159
|
PassesGate,
|
|
195
|
-
ReviewGate,
|
|
196
|
-
SecurityGate,
|
|
197
|
-
PerformanceGate,
|
|
198
|
-
ContractGate,
|
|
199
160
|
CheckpointValidator,
|
|
200
|
-
|
|
161
|
+
interceptOperation,
|
|
201
162
|
} from '@dommaker/harness';
|
|
202
163
|
|
|
164
|
+
// 铁律检查
|
|
165
|
+
const checker = IronLawChecker.getInstance();
|
|
166
|
+
const results = await checker.checkAll(context);
|
|
167
|
+
|
|
203
168
|
// 测试门控
|
|
204
|
-
const
|
|
205
|
-
const
|
|
206
|
-
|
|
207
|
-
// 审查门禁
|
|
208
|
-
const reviewGate = new ReviewGate({ minReviewers: 2 });
|
|
209
|
-
const reviewResult = await reviewGate.check({
|
|
210
|
-
projectId: 'my-project',
|
|
211
|
-
projectPath: '/path/to/project',
|
|
212
|
-
prNumber: 123,
|
|
213
|
-
});
|
|
214
|
-
|
|
215
|
-
// 安全门禁
|
|
216
|
-
const securityGate = new SecurityGate({ severityThreshold: 'high' });
|
|
217
|
-
const securityResult = await securityGate.scan({
|
|
218
|
-
projectId: 'my-project',
|
|
219
|
-
projectPath: '/path/to/project',
|
|
220
|
-
});
|
|
221
|
-
|
|
222
|
-
// 性能门禁(带超时)
|
|
223
|
-
const performanceGate = new PerformanceGate({
|
|
224
|
-
thresholds: {
|
|
225
|
-
maxResponseTime: 500,
|
|
226
|
-
minCoverage: 80,
|
|
227
|
-
maxBundleSize: 1024,
|
|
228
|
-
},
|
|
229
|
-
coverageTimeout: 120000, // 2分钟超时
|
|
230
|
-
});
|
|
231
|
-
const perfResult = await performanceGate.check({
|
|
232
|
-
projectId: 'my-project',
|
|
233
|
-
projectPath: '/path/to/project',
|
|
234
|
-
});
|
|
235
|
-
|
|
236
|
-
// 契约门禁
|
|
237
|
-
const contractGate = new ContractGate({ strict: true });
|
|
238
|
-
const contractResult = await contractGate.check({
|
|
239
|
-
projectId: 'my-project',
|
|
240
|
-
projectPath: '/path/to/project',
|
|
241
|
-
newContractPath: '/path/to/openapi.yaml',
|
|
242
|
-
});
|
|
169
|
+
const gate = new PassesGate({ requireEvidence: true });
|
|
170
|
+
const passed = await gate.runTests();
|
|
243
171
|
|
|
244
172
|
// 检查点验证
|
|
245
|
-
const
|
|
246
|
-
const
|
|
247
|
-
workdir: '/path/to/project',
|
|
248
|
-
});
|
|
249
|
-
```
|
|
250
|
-
|
|
251
|
-
**门禁结果**:
|
|
173
|
+
const validator = CheckpointValidator.getInstance();
|
|
174
|
+
const result = await validator.validate(checkpoints, context);
|
|
252
175
|
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
gate: string; // 门禁类型
|
|
256
|
-
passed: boolean; // 是否通过
|
|
257
|
-
message: string; // 结果消息
|
|
258
|
-
details?: { // 详细信息
|
|
259
|
-
metrics?: object; // 性能指标
|
|
260
|
-
failures?: string[]; // 失败项
|
|
261
|
-
warnings?: string[]; // 警告项
|
|
262
|
-
};
|
|
263
|
-
timestamp: string; // 时间戳
|
|
264
|
-
duration?: number; // 执行时长(毫秒)
|
|
265
|
-
}
|
|
176
|
+
// 拦截器(自动执行 enforcement)
|
|
177
|
+
await interceptOperation('task_completion_claim', context);
|
|
266
178
|
```
|
|
267
179
|
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
```typescript
|
|
271
|
-
const gate = new PerformanceGate({
|
|
272
|
-
thresholds: { minCoverage: 80 },
|
|
273
|
-
coverageTimeout: 60000, // 覆盖率测试超时(毫秒)
|
|
274
|
-
benchmarkTimeout: 30000, // 基准测试超时(毫秒)
|
|
275
|
-
});
|
|
276
|
-
|
|
277
|
-
// 动态设置超时
|
|
278
|
-
gate.setTimeouts({ coverage: 120000 });
|
|
279
|
-
```
|
|
280
|
-
|
|
281
|
-
### 8. 使用拦截器
|
|
180
|
+
---
|
|
282
181
|
|
|
283
|
-
|
|
182
|
+
## 📁 项目配置
|
|
284
183
|
|
|
285
|
-
|
|
286
|
-
import {
|
|
287
|
-
interceptor,
|
|
288
|
-
registerExecutor,
|
|
289
|
-
interceptOperation,
|
|
290
|
-
claimOperation,
|
|
291
|
-
type EnforcementExecutor,
|
|
292
|
-
type ConstraintContext,
|
|
293
|
-
} from '@dommaker/harness';
|
|
294
|
-
|
|
295
|
-
// 1. 注册执行器(使用者实现具体逻辑)
|
|
296
|
-
registerExecutor('verify-completion', {
|
|
297
|
-
description: '验证完成声明:运行测试命令',
|
|
298
|
-
supportedParams: ['command', 'timeout'],
|
|
299
|
-
async execute(context) {
|
|
300
|
-
// 使用者自定义验证逻辑
|
|
301
|
-
const result = await exec('npm test', { cwd: context.projectPath });
|
|
302
|
-
return {
|
|
303
|
-
passed: result.success,
|
|
304
|
-
evidence: result.stdout,
|
|
305
|
-
};
|
|
306
|
-
},
|
|
307
|
-
});
|
|
308
|
-
|
|
309
|
-
// 2. 在关键操作前拦截
|
|
310
|
-
try {
|
|
311
|
-
// 拦截 task_completion_claim 操作
|
|
312
|
-
// 自动查找适用的约束并执行对应的 enforcement
|
|
313
|
-
await claimOperation('task_completion_claim', {
|
|
314
|
-
operation: 'task_completion_claim',
|
|
315
|
-
projectPath: '/path/to/project',
|
|
316
|
-
sessionId: 'session-123',
|
|
317
|
-
});
|
|
318
|
-
|
|
319
|
-
// 拦截通过,可以宣布完成
|
|
320
|
-
await announceCompletion();
|
|
321
|
-
} catch (e) {
|
|
322
|
-
// 拦截失败,铁律违规
|
|
323
|
-
console.error('必须先验证才能完成');
|
|
324
|
-
}
|
|
325
|
-
|
|
326
|
-
// 3. 查询拦截结果(不抛异常)
|
|
327
|
-
const result = await interceptOperation('task_completion_claim', context);
|
|
328
|
-
if (result.passed) {
|
|
329
|
-
console.log('✅ 通过拦截');
|
|
330
|
-
} else {
|
|
331
|
-
console.log('❌ 约束违规:', result.violations);
|
|
332
|
-
}
|
|
333
|
-
```
|
|
334
|
-
|
|
335
|
-
**拦截器 vs 手动检查**:
|
|
336
|
-
|
|
337
|
-
| 方式 | 特点 |
|
|
338
|
-
|------|------|
|
|
339
|
-
| 手动检查 | `checkConstraints()` 需要手动调用 API |
|
|
340
|
-
| 拦截器 | 自动执行 enforcement,抽象框架,使用者实现逻辑 |
|
|
341
|
-
|
|
342
|
-
**内置 enforcement IDs**:
|
|
343
|
-
|
|
344
|
-
| enforcement | 说明 |
|
|
345
|
-
|-------------|------|
|
|
346
|
-
| verify-completion | 验证完成声明:运行测试 |
|
|
347
|
-
| verify-e2e | 验证端到端测试 |
|
|
348
|
-
| debug-systematic | 系统性调试检查 |
|
|
349
|
-
| reuse-first | 复用优先检查 |
|
|
350
|
-
| update-capabilities | CAPABILITIES.md 同步检查 |
|
|
351
|
-
| tdd-cycle | TDD 循环检查 |
|
|
352
|
-
| passes-gate | 测试门控 |
|
|
353
|
-
| checkpoint-required | 检查点必须通过 |
|
|
354
|
-
| check-coverage | 覆盖率检查 |
|
|
355
|
-
| require-discussion | 设计决策讨论检查 |
|
|
356
|
-
|
|
357
|
-
## 内置约束(16 条)
|
|
358
|
-
|
|
359
|
-
| ID | 规则 | 层级 | 严重性 |
|
|
360
|
-
|---|------|------|:------:|
|
|
361
|
-
| `no_bypass_checkpoint` | 禁止跳过检查点 | iron_law | 🔴 error |
|
|
362
|
-
| `no_self_approval` | 禁止自评通过 | iron_law | 🔴 error |
|
|
363
|
-
| `no_completion_without_verification` | 完成必须有验证证据 | iron_law | 🔴 error |
|
|
364
|
-
| `no_test_simplification` | 禁止擅自简化测试 | iron_law | 🔴 error |
|
|
365
|
-
| `no_fix_without_root_cause` | 修复前必须找到根因 | guideline | 🔴 error |
|
|
366
|
-
| `no_code_without_test` | 写代码前必须有测试 | guideline | 🔴 error |
|
|
367
|
-
| `no_any_type` | 禁止使用 any 类型 | guideline | 🟡 warning |
|
|
368
|
-
| `simplest_solution_first` | 优先选择最简方案 | guideline | 🟡 warning |
|
|
369
|
-
| `no_creation_without_reuse_check` | 创建前必须检查可复用 | guideline | 🟡 warning |
|
|
370
|
-
| `capability_sync` | 代码变更必须更新 CAPABILITIES.md | guideline | 🟡 warning |
|
|
371
|
-
| `no_simplification_without_approval` | 不能擅自简化逻辑 | guideline | 🟡 warning |
|
|
372
|
-
| `no_skill_without_test` | 创建技能前必须有测试 | guideline | 🟡 warning |
|
|
373
|
-
| `test_coverage_required` | 测试覆盖率必须达标 | guideline | 🟡 warning |
|
|
374
|
-
| `design_decision_requires_discussion` | 设计决策必须先讨论 | guideline | 🟡 warning |
|
|
375
|
-
| `readme_required` | 新模块必须有 README | tip | 🔵 info |
|
|
376
|
-
| `doc_required_for_public_api` | 公共 API 必须有文档 | tip | 🔵 info |
|
|
377
|
-
|
|
378
|
-
## 预设系统
|
|
379
|
-
|
|
380
|
-
| 预设 | 说明 |
|
|
381
|
-
|------|------|
|
|
382
|
-
| `strict` | 严格模式,所有检查启用 |
|
|
383
|
-
| `standard` | 标准模式,推荐使用 |
|
|
384
|
-
| `relaxed` | 宽松模式,警告不阻止 |
|
|
385
|
-
|
|
386
|
-
## 自定义约束(v0.4+)
|
|
387
|
-
|
|
388
|
-
### 扩展例外(v0.6+)
|
|
389
|
-
|
|
390
|
-
使用 `extend_exceptions` 追加例外,保留内置例外:
|
|
184
|
+
### .harness/config.yml
|
|
391
185
|
|
|
392
186
|
```yaml
|
|
393
|
-
|
|
394
|
-
custom_constraints:
|
|
395
|
-
no_fix_without_root_cause:
|
|
396
|
-
extend_exceptions:
|
|
397
|
-
- my_special_case_1
|
|
398
|
-
- my_special_case_2
|
|
399
|
-
```
|
|
400
|
-
|
|
401
|
-
结果:内置例外 + 新增例外
|
|
402
|
-
|
|
403
|
-
### 完全覆盖
|
|
187
|
+
preset: standard
|
|
404
188
|
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
```yaml
|
|
408
|
-
# .harness/custom-constraints.yml
|
|
189
|
+
# 自定义约束例外
|
|
409
190
|
custom_constraints:
|
|
410
191
|
no_fix_without_root_cause:
|
|
411
|
-
level: guideline
|
|
412
|
-
rule: Do not fix without root cause
|
|
413
|
-
message: 禁止没有根因分析就修复
|
|
414
|
-
trigger: bug_fix
|
|
415
|
-
exceptions:
|
|
416
|
-
- my_only_exception
|
|
417
|
-
```
|
|
418
|
-
|
|
419
|
-
结果:只使用自定义例外
|
|
420
|
-
|
|
421
|
-
### 混合模式
|
|
422
|
-
|
|
423
|
-
同时使用 `exceptions` 和 `extend_exceptions`:
|
|
424
|
-
|
|
425
|
-
```yaml
|
|
426
|
-
custom_constraints:
|
|
427
|
-
no_fix_without_root_cause:
|
|
428
|
-
exceptions:
|
|
429
|
-
- my_new_exception
|
|
430
192
|
extend_exceptions:
|
|
431
|
-
-
|
|
432
|
-
```
|
|
433
|
-
|
|
434
|
-
结果:内置例外 + my_new_exception + another_exception
|
|
435
|
-
|
|
436
|
-
## 项目模板
|
|
437
|
-
|
|
438
|
-
提供多种项目模板:
|
|
439
|
-
|
|
440
|
-
- `node-api` - Node.js API 项目
|
|
441
|
-
- `nextjs-app` - Next.js 应用
|
|
442
|
-
- `python-api` - Python API 项目
|
|
443
|
-
|
|
444
|
-
```bash
|
|
445
|
-
# 指定项目类型
|
|
446
|
-
harness init --type node-api
|
|
193
|
+
- my_special_case
|
|
447
194
|
```
|
|
448
195
|
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
### IronLawChecker
|
|
196
|
+
### CI 集成
|
|
452
197
|
|
|
453
|
-
```
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
|
|
198
|
+
```yaml
|
|
199
|
+
# .github/workflows/harness-check.yml
|
|
200
|
+
jobs:
|
|
201
|
+
harness-check:
|
|
202
|
+
runs-on: ubuntu-latest
|
|
203
|
+
steps:
|
|
204
|
+
- uses: actions/checkout@v4
|
|
205
|
+
- run: npm ci
|
|
206
|
+
- run: harness check
|
|
207
|
+
- run: harness passes-gate
|
|
461
208
|
```
|
|
462
209
|
|
|
463
|
-
|
|
210
|
+
---
|
|
464
211
|
|
|
465
|
-
|
|
466
|
-
class CheckpointValidator {
|
|
467
|
-
validate(checkpoints: Checkpoint[], context: CheckpointContext): Promise<CheckpointResult>;
|
|
468
|
-
}
|
|
469
|
-
```
|
|
212
|
+
## 📊 Trace & 日志
|
|
470
213
|
|
|
471
|
-
###
|
|
214
|
+
### 目录结构
|
|
472
215
|
|
|
473
|
-
```typescript
|
|
474
|
-
class PassesGate {
|
|
475
|
-
constructor(config: PassesGateConfig);
|
|
476
|
-
|
|
477
|
-
setPasses(taskId: string, value: boolean, workDir: string): Promise<PassesGateResult>;
|
|
478
|
-
runTests(): Promise<TestResult>;
|
|
479
|
-
}
|
|
480
216
|
```
|
|
481
|
-
|
|
482
|
-
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
|
|
486
|
-
constructor(config: ReviewGateConfig);
|
|
487
|
-
|
|
488
|
-
check(context: GateContext): Promise<GateResult>;
|
|
489
|
-
setMinReviewers(count: number): void;
|
|
490
|
-
}
|
|
217
|
+
.harness/
|
|
218
|
+
├── traces/
|
|
219
|
+
│ ├── execution.log # 约束检查记录(JSON Lines)
|
|
220
|
+
│ └── summary.json # 统计汇总
|
|
221
|
+
└── state.json # Harness 状态
|
|
491
222
|
```
|
|
492
223
|
|
|
493
|
-
###
|
|
494
|
-
|
|
495
|
-
```typescript
|
|
496
|
-
class SecurityGate {
|
|
497
|
-
constructor(config: SecurityGateConfig);
|
|
498
|
-
|
|
499
|
-
scan(context: GateContext): Promise<GateResult>;
|
|
500
|
-
}
|
|
501
|
-
```
|
|
224
|
+
### 查看日志
|
|
502
225
|
|
|
503
|
-
|
|
226
|
+
```bash
|
|
227
|
+
# 查看状态和统计
|
|
228
|
+
harness status
|
|
504
229
|
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
constructor(config: PerformanceGateConfig);
|
|
508
|
-
|
|
509
|
-
check(context: GateContext): Promise<GateResult>;
|
|
510
|
-
runBenchmark(context: GateContext): Promise<BenchmarkResult>;
|
|
511
|
-
setThresholds(thresholds: Partial<PerformanceThresholds>): void;
|
|
512
|
-
setTimeouts(options: { coverage?: number; benchmark?: number }): void;
|
|
513
|
-
}
|
|
514
|
-
```
|
|
230
|
+
# 查看详细统计
|
|
231
|
+
harness status --detail
|
|
515
232
|
|
|
516
|
-
|
|
233
|
+
# 只看异常
|
|
234
|
+
harness status --anomalies
|
|
517
235
|
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
constructor(config: ContractGateConfig);
|
|
521
|
-
|
|
522
|
-
check(context: GateContext): Promise<GateResult>;
|
|
523
|
-
}
|
|
236
|
+
# 分析最近 N 小时
|
|
237
|
+
harness status --hours 48
|
|
524
238
|
```
|
|
525
239
|
|
|
526
|
-
###
|
|
240
|
+
### 日志格式
|
|
527
241
|
|
|
528
|
-
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
// 创建验收门禁
|
|
534
|
-
const acceptanceGate = new SpecAcceptanceGate({
|
|
535
|
-
tasksPath: './tasks.yml',
|
|
536
|
-
checkAllTasks: false,
|
|
537
|
-
});
|
|
538
|
-
|
|
539
|
-
// 检查单个任务
|
|
540
|
-
const result = await acceptanceGate.check({
|
|
541
|
-
projectPath: '/path/to/project',
|
|
542
|
-
taskId: 'task-001',
|
|
543
|
-
tasksPath: './tasks.yml',
|
|
544
|
-
});
|
|
545
|
-
|
|
546
|
-
// 检查所有任务
|
|
547
|
-
const allResults = await acceptanceGate.check({
|
|
548
|
-
projectPath: '/path/to/project',
|
|
549
|
-
checkAllTasks: true,
|
|
550
|
-
});
|
|
551
|
-
|
|
552
|
-
// 自定义验收条件
|
|
553
|
-
const customGate = new SpecAcceptanceGate({
|
|
554
|
-
customAcceptanceCriteria: {
|
|
555
|
-
'has-tests': async (task) => {
|
|
556
|
-
// 自定义验证逻辑
|
|
557
|
-
return true;
|
|
558
|
-
},
|
|
559
|
-
},
|
|
560
|
-
});
|
|
242
|
+
**execution.log**(JSON Lines 格式):
|
|
243
|
+
```json
|
|
244
|
+
{"timestamp":"2026-04-28T15:00:00Z","constraintId":"no_bypass_checkpoint","result":"passed","duration":12}
|
|
245
|
+
{"timestamp":"2026-04-28T15:01:00Z","constraintId":"no_self_approval","result":"bypassed","reason":"hotfix"}
|
|
561
246
|
```
|
|
562
247
|
|
|
563
|
-
|
|
564
|
-
|
|
565
|
-
|
|
566
|
-
tasks:
|
|
567
|
-
- id: task-001
|
|
568
|
-
title: 实现用户登录
|
|
569
|
-
acceptance_criteria:
|
|
570
|
-
- id: ac-1
|
|
571
|
-
description: 用户可以使用邮箱登录
|
|
572
|
-
type: automated
|
|
573
|
-
required: true
|
|
574
|
-
- id: ac-2
|
|
575
|
-
description: 登录失败显示错误提示
|
|
576
|
-
type: manual
|
|
577
|
-
required: true
|
|
248
|
+
**summary.json**:
|
|
249
|
+
```json
|
|
250
|
+
{"totalChecks":150,"passRate":0.92,"bypassRate":0.08,"anomalies":2}
|
|
578
251
|
```
|
|
579
252
|
|
|
580
|
-
###
|
|
581
|
-
|
|
582
|
-
```typescript
|
|
583
|
-
class SessionStartup {
|
|
584
|
-
constructor(workDir: string, checkpoints: StartupCheckpoints);
|
|
585
|
-
|
|
586
|
-
run(): Promise<{ success: boolean; results: StartupCheckpointResult[] }>;
|
|
587
|
-
getCurrentTask(): Promise<{ task: any; index: number } | null>;
|
|
588
|
-
generateReport(results: StartupCheckpointResult[]): string;
|
|
589
|
-
}
|
|
590
|
-
```
|
|
253
|
+
### 异常检测
|
|
591
254
|
|
|
592
|
-
|
|
255
|
+
当 `harness status --anomalies` 发现异常时,运行 `harness flow` 进行诊断和优化提案。
|
|
593
256
|
|
|
594
|
-
|
|
595
|
-
class CleanStateManager {
|
|
596
|
-
constructor(config: CleanStateConfig);
|
|
597
|
-
|
|
598
|
-
onSessionEnd(workDir: string, sessionInfo: SessionInfo): Promise<CleanStateResult>;
|
|
599
|
-
}
|
|
600
|
-
```
|
|
257
|
+
---
|
|
601
258
|
|
|
602
|
-
## 开发
|
|
259
|
+
## 🔧 开发
|
|
603
260
|
|
|
604
261
|
```bash
|
|
605
|
-
# 安装依赖
|
|
606
262
|
npm install
|
|
607
|
-
|
|
608
|
-
# 构建
|
|
609
263
|
npm run build
|
|
610
|
-
|
|
611
|
-
# 测试
|
|
612
264
|
npm test
|
|
613
265
|
```
|
|
614
266
|
|
|
615
|
-
## 许可证
|
|
616
|
-
|
|
617
|
-
MIT © dommaker
|
|
618
|
-
|
|
619
|
-
---
|
|
620
|
-
|
|
621
|
-
## Execution Trace 系统(v0.3+)
|
|
622
|
-
|
|
623
|
-
### 概念
|
|
624
|
-
|
|
625
|
-
Execution Trace 是约束检查的轻量记录,用于:
|
|
626
|
-
- 统计约束触发频率
|
|
627
|
-
- 检测异常模式(高频绕过、失败率上升)
|
|
628
|
-
- 为 Agent 诊断提供数据基础
|
|
629
|
-
|
|
630
|
-
### 设计原则
|
|
631
|
-
|
|
632
|
-
| 原则 | 说明 |
|
|
633
|
-
|------|------|
|
|
634
|
-
| **零 Token 成本** | 记录和统计都不调用 LLM |
|
|
635
|
-
| **轻量记录** | 只记录核心字段,不记录代码片段 |
|
|
636
|
-
| **异步分析** | 统计每小时执行,异常检测每日执行 |
|
|
637
|
-
| **按需诊断** | Agent 仅在检测到异常时才介入 |
|
|
638
|
-
|
|
639
|
-
### Trace 数据结构
|
|
640
|
-
|
|
641
|
-
```typescript
|
|
642
|
-
interface ExecutionTrace {
|
|
643
|
-
constraintId: string; // 约束 ID
|
|
644
|
-
level: 'iron_law' | 'guideline' | 'tip';
|
|
645
|
-
timestamp: number; // Unix timestamp
|
|
646
|
-
result: 'pass' | 'fail' | 'bypassed';
|
|
647
|
-
operation?: string; // 触发条件
|
|
648
|
-
exceptionApplied?: string; // 例外类型
|
|
649
|
-
}
|
|
650
|
-
```
|
|
651
|
-
|
|
652
|
-
### 自动记录
|
|
653
|
-
|
|
654
|
-
约束检查时自动记录:
|
|
655
|
-
|
|
656
|
-
```typescript
|
|
657
|
-
import { checkConstraints } from '@dommaker/harness';
|
|
658
|
-
|
|
659
|
-
// 每次检查都会自动记录 trace
|
|
660
|
-
const result = await checkConstraints(context);
|
|
661
|
-
// Trace 已写入 .harness/traces/execution.log
|
|
662
|
-
```
|
|
663
|
-
|
|
664
|
-
### Trace 文件位置
|
|
665
|
-
|
|
666
|
-
```
|
|
667
|
-
.harness/traces/
|
|
668
|
-
├── execution.log # 当前 trace 文件
|
|
669
|
-
├── execution-2026-04-05.log # 滚动备份(超出 10MB 时)
|
|
670
|
-
├── summary.json # 统计汇总
|
|
671
|
-
```
|
|
672
|
-
|
|
673
|
-
### 统计汇总
|
|
674
|
-
|
|
675
|
-
每小时自动执行:
|
|
676
|
-
|
|
677
|
-
```typescript
|
|
678
|
-
import { TraceAnalyzer } from '@dommaker/harness';
|
|
679
|
-
|
|
680
|
-
const analyzer = new TraceAnalyzer();
|
|
681
|
-
const summaries = analyzer.runHourlySummary();
|
|
682
|
-
|
|
683
|
-
// 输出:
|
|
684
|
-
// [
|
|
685
|
-
// { constraintId: 'no_fix_without_root_cause', passRate: 0.7, bypassRate: 0.1 },
|
|
686
|
-
// { constraintId: 'no_code_without_test', passRate: 0.9, bypassRate: 0 },
|
|
687
|
-
// ]
|
|
688
|
-
```
|
|
689
|
-
|
|
690
|
-
### 异常检测
|
|
691
|
-
|
|
692
|
-
每日自动执行:
|
|
693
|
-
|
|
694
|
-
```typescript
|
|
695
|
-
const anomalies = analyzer.runDailyAnomalyCheck();
|
|
696
|
-
|
|
697
|
-
// 检测类型:
|
|
698
|
-
// - high_bypass_rate:绕过率 > 30%
|
|
699
|
-
// - rising_fail_rate:失败率上升趋势
|
|
700
|
-
// - exception_overuse:例外使用率 > 40%
|
|
701
|
-
```
|
|
702
|
-
|
|
703
|
-
### CLI 使用
|
|
704
|
-
|
|
705
|
-
```bash
|
|
706
|
-
# 查看 trace 文件统计
|
|
707
|
-
harness traces stats
|
|
708
|
-
|
|
709
|
-
# 查看最近 24 小时的约束汇总
|
|
710
|
-
harness traces summary --hours 24
|
|
711
|
-
|
|
712
|
-
# 检测异常
|
|
713
|
-
harness traces anomalies
|
|
714
|
-
|
|
715
|
-
# JSON 格式输出
|
|
716
|
-
harness traces summary --format json
|
|
717
|
-
|
|
718
|
-
# 运行诊断
|
|
719
|
-
harness diagnose run --hours 24 --save
|
|
720
|
-
|
|
721
|
-
# 查看诊断列表
|
|
722
|
-
harness diagnose list
|
|
723
|
-
|
|
724
|
-
# 生成提案
|
|
725
|
-
harness propose generate --save
|
|
726
|
-
|
|
727
|
-
# 审核提案
|
|
728
|
-
harness propose review --diagnosis <id> --accept
|
|
729
|
-
|
|
730
|
-
# 查看实施指导
|
|
731
|
-
harness propose implement --diagnosis <id>
|
|
732
|
-
```
|
|
733
|
-
|
|
734
|
-
### 成本控制
|
|
735
|
-
|
|
736
|
-
| 活动 | 频率 | Token 成本 |
|
|
737
|
-
|------|------|:----------:|
|
|
738
|
-
| Trace 记录 | 每次检查 | 0 |
|
|
739
|
-
| 统计汇总 | 每小时 | 0 |
|
|
740
|
-
| 异常检测 | 每日 | 0(未触发)~500(触发) |
|
|
741
|
-
| Agent 诊断 | 按需 | ~2000 |
|
|
742
|
-
|
|
743
|
-
**对比 Meta-Harness**:百万级 vs 5500/周,成本降低 **180 倍**。
|
|
744
|
-
|
|
745
267
|
---
|
|
746
268
|
|
|
747
|
-
##
|
|
748
|
-
|
|
749
|
-
### 概念
|
|
750
|
-
|
|
751
|
-
Performance Trace 是性能数据的轻量记录,用于:
|
|
752
|
-
- 监控关键操作耗时
|
|
753
|
-
- 检测性能异常(慢操作、超阈值)
|
|
754
|
-
- 统计 Token 使用和上下文大小
|
|
755
|
-
- 为性能优化提供数据基础
|
|
756
|
-
|
|
757
|
-
### 设计原则
|
|
758
|
-
|
|
759
|
-
| 原则 | 说明 |
|
|
760
|
-
|------|------|
|
|
761
|
-
| **零 Token 成本** | 记录和统计都不调用 LLM |
|
|
762
|
-
| **文件存储** | 追加写入,单行 JSON 格式 |
|
|
763
|
-
| **异步分析** | 统计每小时执行,异常检测每日执行 |
|
|
764
|
-
| **可配置阈值** | 按操作类型设置不同的性能阈值 |
|
|
765
|
-
|
|
766
|
-
### Trace 数据结构
|
|
767
|
-
|
|
768
|
-
```typescript
|
|
769
|
-
interface PerformanceTrace {
|
|
770
|
-
operation: string; // 操作名称
|
|
771
|
-
startTime: number; // 开始时间戳
|
|
772
|
-
duration: number; // 耗时(毫秒)
|
|
773
|
-
status: 'ok' | 'exceeded' | 'error';
|
|
774
|
-
threshold?: number; // 配置的阈值
|
|
775
|
-
metadata?: Record<string, unknown>;
|
|
776
|
-
}
|
|
777
|
-
```
|
|
778
|
-
|
|
779
|
-
### API 使用
|
|
780
|
-
|
|
781
|
-
```typescript
|
|
782
|
-
import { PerformanceCollector, PerformanceAnalyzer } from '@dommaker/harness';
|
|
783
|
-
|
|
784
|
-
// 创建收集器
|
|
785
|
-
const collector = new PerformanceCollector({
|
|
786
|
-
logFile: '.harness/logs/performance.log',
|
|
787
|
-
thresholds: {
|
|
788
|
-
extract: 1000, // 提取操作阈值 1s
|
|
789
|
-
transform: 500, // 转换操作阈值 500ms
|
|
790
|
-
invoke: 10000, // 调用操作阈值 10s
|
|
791
|
-
},
|
|
792
|
-
});
|
|
793
|
-
|
|
794
|
-
// 记录操作
|
|
795
|
-
await collector.recordOk('extract', 150, { contextSize: 5000 });
|
|
796
|
-
await collector.recordExceeded('invoke', 12000, 10000, { agent: 'skill-agent' });
|
|
797
|
-
|
|
798
|
-
// 分析统计
|
|
799
|
-
const analyzer = new PerformanceAnalyzer();
|
|
800
|
-
const summaries = analyzer.summarize(traces);
|
|
801
|
-
const anomalies = analyzer.detectAnomalies(traces);
|
|
802
|
-
|
|
803
|
-
// 生成报告
|
|
804
|
-
const report = analyzer.generateReport(summaries, anomalies);
|
|
805
|
-
```
|
|
269
|
+
## 📝 License
|
|
806
270
|
|
|
807
|
-
|
|
808
|
-
|
|
809
|
-
```
|
|
810
|
-
.harness/logs/
|
|
811
|
-
├── performance.log # 性能日志
|
|
812
|
-
├── performance-summary.json # 统计汇总
|
|
813
|
-
└── tokens.log # Token 使用日志
|
|
814
|
-
```
|
|
815
|
-
|
|
816
|
-
### 与 TraceCollector 对比
|
|
817
|
-
|
|
818
|
-
| 能力 | TraceCollector | PerformanceCollector |
|
|
819
|
-
|------|----------------|---------------------|
|
|
820
|
-
| **监控目标** | 约束检查结果 | 操作耗时 |
|
|
821
|
-
| **监控"什么"** | 对不对 | 快不快 |
|
|
822
|
-
| **异常类型** | 高绕过率、失败率上升 | 慢操作、超阈值 |
|
|
823
|
-
| **存储路径** | `.harness/logs/traces.log` | `.harness/logs/performance.log` |
|
|
824
|
-
|
|
825
|
-
---
|
|
826
|
-
|
|
827
|
-
## Failure Classification 系统(v0.4+)
|
|
828
|
-
|
|
829
|
-
### 概念
|
|
830
|
-
|
|
831
|
-
Failure Classification 提供通用的错误分类和记录能力,用于:
|
|
832
|
-
- 统一错误分类标准
|
|
833
|
-
- 分级处理失败(L1-L4)
|
|
834
|
-
- 记录失败历史供分析
|
|
835
|
-
|
|
836
|
-
### 设计原则
|
|
837
|
-
|
|
838
|
-
| 原则 | 说明 |
|
|
839
|
-
|------|------|
|
|
840
|
-
| **零业务逻辑** | 只提供类型 + 分类 + 记录能力 |
|
|
841
|
-
| **可扩展规则** | 支持自定义分类规则 |
|
|
842
|
-
| **文件存储** | 追加写入,单行 JSON 格式 |
|
|
843
|
-
|
|
844
|
-
### 错误类型
|
|
845
|
-
|
|
846
|
-
```typescript
|
|
847
|
-
enum ErrorType {
|
|
848
|
-
TEST_FAILED = 'TEST_FAILED', // 测试失败
|
|
849
|
-
GATE_FAILED = 'GATE_FAILED', // 门禁检查失败
|
|
850
|
-
DEPENDENCY_BLOCKED = 'DEPENDENCY_BLOCKED', // 依赖阻塞
|
|
851
|
-
CONTEXT_OVERFLOW = 'CONTEXT_OVERFLOW', // 上下文溢出
|
|
852
|
-
TIMEOUT = 'TIMEOUT', // 超时
|
|
853
|
-
NETWORK_ERROR = 'NETWORK_ERROR', // 网络错误
|
|
854
|
-
AGENT_ERROR = 'AGENT_ERROR', // Agent 错误
|
|
855
|
-
TOOL_ERROR = 'TOOL_ERROR', // 工具错误
|
|
856
|
-
VALIDATION_ERROR = 'VALIDATION_ERROR', // 验证错误
|
|
857
|
-
UNKNOWN = 'UNKNOWN', // 未知错误
|
|
858
|
-
}
|
|
859
|
-
```
|
|
860
|
-
|
|
861
|
-
### 失败等级
|
|
862
|
-
|
|
863
|
-
| 等级 | 说明 | 处理策略 |
|
|
864
|
-
|------|------|---------|
|
|
865
|
-
| L1 | 自动重试 | 重试、降级 |
|
|
866
|
-
| L2 | 需要干预 | 人工审核 |
|
|
867
|
-
| L3 | 严重问题 | 开会讨论 |
|
|
868
|
-
| L4 | 致命错误 | 回滚 |
|
|
869
|
-
|
|
870
|
-
### API 使用
|
|
871
|
-
|
|
872
|
-
```typescript
|
|
873
|
-
import {
|
|
874
|
-
ErrorClassifier,
|
|
875
|
-
FailureRecorder,
|
|
876
|
-
ErrorType,
|
|
877
|
-
FailureLevel,
|
|
878
|
-
classifyError,
|
|
879
|
-
} from '@dommaker/harness';
|
|
880
|
-
|
|
881
|
-
// 快速分类
|
|
882
|
-
const type = classifyError(new Error('test failed'));
|
|
883
|
-
// type === ErrorType.TEST_FAILED
|
|
884
|
-
|
|
885
|
-
// 完整分类器
|
|
886
|
-
const classifier = new ErrorClassifier({
|
|
887
|
-
rules: [
|
|
888
|
-
{
|
|
889
|
-
type: ErrorType.CUSTOM_ERROR,
|
|
890
|
-
keywords: ['custom', 'specific'],
|
|
891
|
-
level: FailureLevel.L2,
|
|
892
|
-
},
|
|
893
|
-
],
|
|
894
|
-
});
|
|
895
|
-
|
|
896
|
-
const result = classifier.classify(new Error('custom error'));
|
|
897
|
-
// result.type, result.level, result.matchedRule
|
|
898
|
-
|
|
899
|
-
// 失败记录
|
|
900
|
-
const recorder = new FailureRecorder({
|
|
901
|
-
logFile: '.harness/logs/failures.log',
|
|
902
|
-
});
|
|
903
|
-
|
|
904
|
-
await recorder.record({
|
|
905
|
-
type: ErrorType.TEST_FAILED,
|
|
906
|
-
level: FailureLevel.L1,
|
|
907
|
-
message: 'Test failed',
|
|
908
|
-
timestamp: Date.now(),
|
|
909
|
-
});
|
|
910
|
-
|
|
911
|
-
// 获取统计
|
|
912
|
-
const stats = await recorder.getStats();
|
|
913
|
-
// { total, byType, byLevel }
|
|
914
|
-
```
|
|
915
|
-
|
|
916
|
-
### 与 PerformanceCollector 对比
|
|
917
|
-
|
|
918
|
-
| 能力 | PerformanceCollector | FailureRecorder |
|
|
919
|
-
|------|---------------------|-----------------|
|
|
920
|
-
| **监控目标** | 操作耗时 | 错误分类 |
|
|
921
|
-
| **存储内容** | 性能指标 | 失败记录 |
|
|
922
|
-
| **分析维度** | 慢操作、超阈值 | 错误类型分布、失败等级 |
|
|
271
|
+
MIT © dommaker
|