@devflow-tools/benchmark 0.8.9 → 0.8.11

package/README.md

@@ -1,6 +1,6 @@
 # @devflow-tools/benchmark
 
-Benchmark engine for DevFlow — evaluate AI agent code quality on your projects.
+DevFlow 基准测试引擎 — 评估 AI Agent 在项目上的代码质量表现，支持 baseline/devflow 对比分析。
 
 ## Installation
 
@@ -13,16 +13,65 @@ npm install @devflow-tools/benchmark
 ```typescript
 import { BenchmarkEngine } from '@devflow-tools/benchmark';
 
-const engine = new BenchmarkEngine({ projectRoot: '/path/to/project' });
-const report = await engine.run();
-console.log(report.score);
+const engine = new BenchmarkEngine();
+
+// 查看内置任务
+const tasks = engine.getTasks();
+console.log(tasks.map(t => t.id)); // ['bug-1', 'bug-2', 'bug-3', 'feat-1', 'feat-2', 'review-1', 'review-2']
+
+// 添加自定义任务
+await engine.addTask({
+  id: 'custom-1',
+  type: 'bugfix',
+  desc: 'Fix state update white screen',
+  expectedFiles: 3,
+});
+
+// 自定义 runner 运行单任务
+const runner = async (task) => ({
+  taskId: task.id,
+  baseline: { tokens: 5000, time: 120000, correct: true },
+  experiment: { tokens: 1000, time: 30000, correct: true },
+  delta: { tokenReduction: 0.8, timeSaved: 0.75, accuracy: 0.15 },
+});
+const result = await engine.runOne('bug-1', runner);
+
+// 批量运行所有任务
+const report = await engine.runAll(runner);
+console.log(report.summary.avgTokenReduction); // "80%"
+
+// 场景对比（baseline vs devflow）
+const baseline = await engine.runScenario(scenario, 'baseline');
+const devflow = await engine.runScenario(scenario, 'devflow');
+const comparison = engine.compareResults(baseline, devflow);
 ```
 
+## Built-in Tasks
+
+| ID | Type | Description |
+|----|------|-------------|
+| `bug-1` | bugfix | 修复状态未更新导致的白屏 |
+| `bug-2` | bugfix | 修复 API 错误码未处理导致崩溃 |
+| `bug-3` | bugfix | 修复条件渲染逻辑反转 |
+| `feat-1` | feature | 添加用户搜索功能 |
+| `feat-2` | feature | 添加分页组件 |
+| `review-1` | review | Review 包含 N+1 查询的 PR |
+| `review-2` | review | Review 包含 XSS 风险的 PR |
+
 ## API
 
-- `BenchmarkEngine` — main class for running benchmarks
-- `.run(options?)` — execute a benchmark and return a report
-- `.reports` — list historical benchmark reports
+- `new BenchmarkEngine()` — 构造函数
+- `.getTasks()` — 获取所有任务（含内置和自定义）
+- `.addTask(task)` — 添加/覆盖任务
+- `.runOne(taskId, runner)` — 运行单个任务（需提供 TaskRunner）
+- `.runAll(runner, options?)` — 批量运行，返回 BenchmarkReport
+- `.runScenario(scenario, mode)` — 在 baseline 或 devflow 模式下运行场景
+- `.compareResults(baseline, devflow)` — 计算改进指标
+- `.registerScenario(scenario)` — 注册场景
+- `.runScenarioPair(id, baselineRunner, devflowRunner)` — 运行场景对比
+- `.runScenarioBatch(ids, baselineRunner, devflowRunner)` — 批量场景对比
+- `.getReport(runId)` — 获取历史报告
+- `.listReports()` — 列出所有报告元数据
 
 ## License
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@devflow-tools/benchmark",
-  "version": "0.8.9",
+  "version": "0.8.11",
   "type": "module",
   "main": "./dist/index.js",
   "types": "./dist/index.d.ts",
@@ -17,7 +17,7 @@
     "clean": "rm -rf dist"
   },
   "dependencies": {
-    "@devflow-tools/sdk": "^0.8.9"
+    "@devflow-tools/sdk": "^0.8.11"
   },
   "devDependencies": {
     "typescript": "^5.5.0",
@@ -26,5 +26,5 @@
   "files": [
     "dist"
   ],
-  "gitHead": "5c9a804d3c2e6e6822b516fde67d5b789dbdb5b3"
+  "gitHead": "a0a0b61377aa88ebbd380f50a5d532f63175ef45"
 }