dsh-ronge-adaptive-skill-archive 5.2.1 → 5.3.1

package/README.md

@@ -31,7 +31,7 @@
   <sub>会话右侧的「技能」抽屉：推荐 / 组队计划 / 经验 / 安全与更新，都在一个面板里。</sub>
 </p>
 
-> **v5.2.1 发布说明**：本 README 是当前发布稿。三大新能力 `plan → compose → draft`（任务拆解 / 技能组队 / 按缺口造技能）与 12 领域包为本次发布新增；每条声明的证据、实测状态与未验证项统一记录在 **[docs/claims.md](docs/claims.md)**。benchmark 数字见 **[docs/benchmark.md](docs/benchmark.md)**（2026-10-30 快照）。
+> **v5.3.1 发布说明**：本 README 是当前发布稿。三大新能力 `plan → compose → draft`（任务拆解 / 技能组队 / 按缺口造技能）与 12 领域包为本次发布新增；每条声明的证据、实测状态与未验证项统一记录在 **[docs/claims.md](docs/claims.md)**。benchmark 数字见 **[docs/benchmark.md](docs/benchmark.md)**（2026-10-30 快照）。
 
 ## 为什么需要 v5
 
@@ -59,8 +59,8 @@ v5 的驾驶舱把这条链路补全：
 npx dsh-ronge-adaptive-skill-archive
 #    想先体检：npx dsh-ronge-adaptive-skill-archive doctor
 
-# 2) 把打印出的这一行交给任意 DSH 会话里的 Agent 执行（v5.2.1 发布后可用）
-plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.2.1
+# 2) 把打印出的这一行交给任意 DSH 会话里的 Agent 执行（v5.3.1 发布后可用）
+plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.3.1
 
 # 3) 刷新页面 → 右上角「技能」→ 右侧抽屉；计划模式 / 沉浸模式切换
 ```
@@ -209,6 +209,7 @@ v5.0.0 在 DSH 里注册三个**只读 / 只出草稿**的工具（面板与 MCP
 |---|---|---|---|
 | **已测：推荐召回** | 30 条半人工任务，Top-4 里是否出现可用的已装技能 | [docs/eval.md](docs/eval.md) | Recall@4 = 0.9（27/30），运行于 v2.7.2 工作树，**引用需注明口径** |
 | **已测：库检索** | 24 条任务在 6,399 索引里的类目命中 | [docs/eval.md](docs/eval.md) | Category Hit@5 等指标见文档；Hit@5 偏松，不能证明技能好用 |
+| **v5：真实任务 A/B** | 12 条任务 fixture；pilot 2 条已跑 | [docs/benchmark-live.md](docs/benchmark-live.md) | pilot：成功率打平，plugin token 2.1–2.6×，**不是效率胜利**；修正后待重跑 |
 | **v5：规划 / 组队 / 效率** | planCoverage **0.333**、compositionValidity **1.00**、capabilityCoverage **0.717**、hintRecall **0.45**、gapDetection **1.00**（宽松口径：不假装已覆盖）、contextTokens mean **131.5** / p95 **193**（延迟为单机快照，不对外作 SLA）；基线 keyword 0.083/0.217/0.25/0.532；installLoad **0.3** | [docs/benchmark.md](docs/benchmark.md)（60 条 held-out，2026-10-30） | 指标是半人工基准，不等于真实成功率；installLoad 0.3 必须一起读 |
 
 规则：
@@ -246,8 +247,8 @@ v5.0.0 在 DSH 里注册三个**只读 / 只出草稿**的工具（面板与 MCP
 | 方式 | 命令 | 适合 |
 |---|---|---|
 | **npx 助手** | `npx dsh-ronge-adaptive-skill-archive`（体检：`npx dsh-ronge-adaptive-skill-archive doctor`） | 第一次用；不装东西，只体检 + 打印安装命令 |
-| **npm** | `plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.2.1` | v5.2.1 发布到 npm 后可用 |
-| **GitHub tag** | `plugin_manager install_bundle target=git+https://github.com/bugtakue/dsh-ronge-adaptive-skill-archive.git#v5.2.1` | 固定版本、可审源码 |
+| **npm** | `plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.3.1` | v5.3.1 发布到 npm 后可用 |
+| **GitHub tag** | `plugin_manager install_bundle target=git+https://github.com/bugtakue/dsh-ronge-adaptive-skill-archive.git#v5.3.1` | 固定版本、可审源码 |
 | **本地目录** | `plugin_manager install_bundle target="/absolute/path/to/dsh-ronge-adaptive-skill-archive"` | 开发、改代码、离线 |
 
 > v5.0.0 打 tag / 发布 npm 之前，`@5.0.0` 会 404：请先用本地目录，或安装当时最新的已发布版本。安装后 DSH 需**重启**才会加载新的 Host 模块；浏览器刷新加载 Client。
@@ -356,7 +357,7 @@ MIT。见 [LICENSE](LICENSE)。索引中的技能元数据归原作者，边界
 **Install** (in a DSH session):
 
 ```bash
-plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.2.1
+plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.3.1
 # then open any session → 「技能」 → switch Plan / Immersive
 ```
 

package/client.js

@@ -271,6 +271,8 @@ window.__ModuleLoader__.load({
       intelStages: '阶段',
       intelInstalledCount: '已安装技能',
       intelReviewGap: '缺少独立复核技能：计划未假装复核',
+      intelReviewSuggestions: '可安装的复核候选',
+      intelDraftQuality: '草稿质量',
       intelPlanOptions: '方案',
       intelPlanFast: '快速',
       intelPlanBalanced: '均衡',
@@ -565,6 +567,8 @@ window.__ModuleLoader__.load({
       intelStages: 'Stages',
       intelInstalledCount: 'Installed skills',
       intelReviewGap: 'No independent reviewer: plan does not fake a review',
+      intelReviewSuggestions: 'Reviewer candidates',
+      intelDraftQuality: 'Draft quality',
       intelPlanOptions: 'Plans',
       intelPlanFast: 'Fast',
       intelPlanBalanced: 'Balanced',
@@ -4118,7 +4122,18 @@ window.__ModuleLoader__.load({
             ),
           ),
           plan.reviewGap
-            ? h('div', { className: 'sf-intel-note' }, t('intelReviewGap'))
+            ? h(
+                'div',
+                { className: 'sf-intel-note' },
+                t('intelReviewGap'),
+                Array.isArray(plan.reviewGapSuggestions) && plan.reviewGapSuggestions.length
+                  ? h(
+                      'span',
+                      { className: 'sf-intel-suggest' },
+                      ' · ' + t('intelReviewSuggestions') + ': ' + plan.reviewGapSuggestions.join(', '),
+                    )
+                  : null,
+              )
             : null,
           intelPlans.length > 1
             ? h(
@@ -4346,6 +4361,13 @@ window.__ModuleLoader__.load({
                 )
               : null,
           ),
+          draft.quality && Number.isFinite(Number(draft.quality.score))
+            ? h(
+                'div',
+                { className: 'sf-intel-meta' },
+                h('span', { className: 'sf-badge' }, t('intelDraftQuality') + ': ' + draft.quality.score + ' (' + String(draft.quality.grade || '') + ')'),
+              )
+            : null,
           badgeRow(
             [
               h('span', { className: 'sf-intel-k', key: 'intel-safety-label' }, t('intelSafety')),

package/docs/claims.md

@@ -15,7 +15,7 @@
 | 智能评测 | `test/fixtures/intelligence-cases.json` + `scripts/benchmark.mjs` | 60 条；数字见下 |
 | 已安装推荐评测 | `docs/eval.md` + `scripts/eval-real.mjs` | 30 条，v2.7.2 工作树；需 v5 复跑 |
 | 库检索评测 | `docs/eval.md` + `scripts/eval-discover.mjs` | 24 条，v2.8.1 工作树；需 v5 复跑 |
-| 单元/集成测试 | `npm test` | **120/120 通过**（2026-10-30） |
+| 单元/集成测试 | `npm test` | **126/126 通过**（2026-10-30） |
 | Host E2E | `verify/host-e2e.mjs` | **20/20**（含 v5 端点） |
 | UI E2E | `verify/ui-e2e.mjs` | **27/27** |
 | DSH 兼容 | `docs/dsh-compatibility.md` | DSH Desktop 0.2.1-rc.3；安装/重启/迁移 |
@@ -57,7 +57,8 @@
 | C3 | **installLoad 0.3**：计划中约 30.0% 技能需先安装；planCoverage 不是成功率、不是已装可用率 | 同上 | verified（必须一起引用） |
 | C4 | hintRecall 0.45 是当前短板（AI 领域 0.00，多个领域 0.20）；latency 是机器相关单次墙钟，不作为对外 SLA | `docs/benchmark.md` | verified |
 | C5 | 已安装推荐 Recall@4 = 0.9（27/30）、discover Hit@5 0.958 等为旧版半人工基线 | `docs/eval.md` | needs-human（v5 复跑） |
-| C6 | 效率提升百分比 | `efficiencyEstimate` 只给 `method:'estimate'`；benchmark 不测真实任务 token 节省 | **unsupported**（禁止写具体百分比） |
+| C6 | 真实任务 A/B（pilot 2 条） | `docs/benchmark-live-pilot.json` + `docs/benchmark-live.md` | **pilot recorded：成功率 2/2 打平，plugin token 2.1–2.6×，不是效率胜利** |
+| C7 | 效率提升百分比 | `efficiencyEstimate` 只给 `method:'estimate'`；benchmark 不测真实任务 token 节省 | **unsupported**（禁止写具体百分比） |
 
 ### D. v5 新能力（已交付）
 
@@ -74,7 +75,7 @@
 
 | # | 事项 | 状态 |
 |---|---|---|
-| E1 | 测试数 55/57 旧冲突 | 已统一为 **120/120**（2026-10-30）；发布前以 `npm test` 为准 |
+| E1 | 测试数 55/57 旧冲突 | 已统一为 **126/126**（2026-10-30）；发布前以 `npm test` 为准 |
 | E2 | README 版本 3.0.0 | 已改为 v5.0.0；发布前与 package/tag 同步 |
 | E3 | 技能条数 vs 仓库数 | 已分开写：6,399 条技能 / 4,007 仓库 |
 | E4 | 索引 vs stars 快照日期 | 已分别标注 2026-10-28 / 2026-10-30 |

package/docs/dsh-compatibility.md

@@ -7,10 +7,10 @@
 
 ```bash
 # npm（推荐）
-plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.2.1
+plugin_manager install_bundle target=dsh-ronge-adaptive-skill-archive@5.3.1
 
 # GitHub tag（固定版本、可审源码）
-plugin_manager install_bundle target=git+https://github.com/bugtakue/dsh-ronge-adaptive-skill-archive.git#v5.2.1
+plugin_manager install_bundle target=git+https://github.com/bugtakue/dsh-ronge-adaptive-skill-archive.git#v5.3.1
 
 # 本地目录（开发）
 plugin_manager install_bundle target=/absolute/path/to/dsh-ronge-adaptive-skill-archive
@@ -38,11 +38,11 @@ plugin_manager install_bundle target=/absolute/path/to/dsh-ronge-adaptive-skill-
 
 ## 已验证
 
-- `npm test`：120/120（含 DSH 兼容性静态检查）
+- `npm test`：126/126（含 DSH 兼容性静态检查）
 - Host E2E：20/20（真实 loopback HTTP + raw.githubusercontent.com）
 - UI E2E：27/27
 - `npm run check:dsh`：manifest / factory id / slots / routes / 数据迁移 / market index 全部通过
-- npm registry 安装 v3.0.0 历史验证；v5.2.1 发布后以 registry 为准；市场索引 format 1 托管在 `https://bugtakue.github.io/dsh-ronge-adaptive-skill-archive/plugin-market.json`
+- npm registry 安装 v3.0.0 历史验证；v5.3.1 发布后以 registry 为准；市场索引 format 1 托管在 `https://bugtakue.github.io/dsh-ronge-adaptive-skill-archive/plugin-market.json`
 
 ## 已知边界
 

package/lib/intelligence/compose.js

@@ -260,7 +260,11 @@ function rankForCapability(pool, capability, context, options = {}) {
     .sort((a, b) => b.score - a.score || a.candidate.name.localeCompare(b.candidate.name))
 }
 
+const TEXT_SAFE_DOMAINS = new Set(['writing', 'marketing', 'research', 'productivity', 'ai', 'unknown'])
+
 function selectSkills({ pool, decomposition, installedCount, experience, maxSkills }) {
+  const textMode = decomposition.outputMode === 'text'
+  const selectionPool = textMode ? pool.filter((candidate) => TEXT_SAFE_DOMAINS.has(candidate.profile.primaryDomain)) : pool
   const required = uniqueCaps(decomposition.capabilities)
   const requiredDomains = uniqueCaps([...(decomposition.domains || [])])
   const context = { requiredDomains, experience, taskText: decomposition.goal }
@@ -271,7 +275,7 @@ function selectSkills({ pool, decomposition, installedCount, experience, maxSkil
   for (const capability of required) {
     if (covered.has(capability)) continue
     if (selected.length >= maxSkills) break
-    const ranked = rankForCapability(pool.filter((candidate) => candidate.installed), capability, context)
+    const ranked = rankForCapability(selectionPool.filter((candidate) => candidate.installed), capability, context)
     const best = ranked.find((entry) => !selectedNames.has(entry.candidate.name))
     if (!best) continue
     const threshold = best.candidate.installed ? 0.55 : 0.6
@@ -287,7 +291,7 @@ function selectSkills({ pool, decomposition, installedCount, experience, maxSkil
     const uncovered = required.filter((capability) => !covered.has(capability))
     for (const capability of uncovered) {
       if (selected.length >= Math.min(MIN_SKILLS, maxSkills)) break
-      const ranked = rankForCapability(pool.filter((candidate) => candidate.installed), capability, context)
+      const ranked = rankForCapability(selectionPool.filter((candidate) => candidate.installed), capability, context)
       const best = ranked.find((entry) => !selectedNames.has(entry.candidate.name) && entry.score >= 0.5)
       if (!best) continue
       selected.push(best.candidate)
@@ -309,6 +313,7 @@ function buildMissing({ required, covered, pool, decomposition }) {
     const domain = capabilityDomain(capability) || decomposition.domains[0] || 'unknown'
     const suggestions = pool
       .filter((candidate) => !candidate.installed)
+      .filter((candidate) => decomposition.outputMode !== 'text' || TEXT_SAFE_DOMAINS.has(candidate.profile.primaryDomain))
       .map((candidate) => {
         let score = 0
         if ((candidate.profile.capabilities || []).includes(capability)) score += 2
@@ -609,6 +614,27 @@ export function composeSkillPlan(input = {}) {
     pool,
     decomposition,
   })
+  if (decomposition.outputMode === 'text' && selection.selected.length === 0) {
+    return {
+      ok: true,
+      ready: true,
+      reviewGap: false,
+      reviewGapSuggestions: [],
+      pattern: 'sequential',
+      stages: [],
+      missing: [],
+      conflicts: [],
+      estimates: { efficiency: efficiencyEstimate({ stages: [], missing: [], selected: [], pattern: 'sequential', baselineSkills: installedSkills.length }), skills: 0, stages: 0 },
+      assumptions: ['纯文本交付：不需要加载技能，直接生成正文；不把 office/video 技能强塞进计划。'],
+      confidence: Math.max(0.5, Number(decomposition.confidence) || 0.5),
+      coverage: { required: selection.required, covered: [], ratio: 0, primaryIntentCovered: false },
+      selected: [],
+      primaryIntent: decomposition.primaryIntent,
+      secondaryIntents: decomposition.secondaryIntents,
+      deliverables: (decomposition.deliverables || []).map((deliverable) => deliverable.id),
+      goal: truncate(decomposition.goal || task, 120),
+    }
+  }
   const conflicts = buildConflicts({
     selected: selection.selected,
     requiredDomains: selection.requiredDomains,
@@ -673,6 +699,16 @@ export function composeSkillPlan(input = {}) {
 
   const coverageRatio = selection.required.length ? selection.covered.size / selection.required.length : 1
   const criticalMissing = missing.filter((entry) => entry.critical).length
+  const reviewGapSuggestions = reviewGap
+    ? pool
+        .filter((candidate) => !candidate.installed)
+        .filter((candidate) => {
+          const caps = candidate.profile.capabilities || []
+          return ['code-review', 'security-audit', 'fact-checking', 'layout-review', 'contract-review', 'testing'].some((capability) => caps.includes(capability))
+        })
+        .slice(0, 3)
+        .map((candidate) => candidate.name)
+    : []
   const cleanStages = trimStages(stages, maxSkills, 5)
   const estimates = {
     efficiency: efficiencyEstimate({
@@ -704,6 +740,7 @@ export function composeSkillPlan(input = {}) {
     ok: cleanStages.length > 0,
     ready: cleanStages.length > 0 && criticalMissing === 0 && missing.length === 0,
     reviewGap,
+    reviewGapSuggestions,
     pattern: ALLOWED_PATTERNS.has(pattern) ? pattern : 'sequential',
     stages: cleanStages,
     missing,

package/lib/intelligence/decompose.js

@@ -326,6 +326,16 @@ function buildAssumptions({ task, deliverables, primaryIntent, secondaryIntents,
  * Decompose a user task into an executable intent/deliverable graph.
  * @param {{task?:string, cwd?:string, context?:string, mode?:string}} input
  */
+const TEXT_OUTPUT_RE = /(正文|直接返回|不写文件|不需要文件|不用文件|仅文字|纯文本|作为文本|return (?:the )?text|no files?)/i
+const FILE_OUTPUT_RE = /(\.docx|\.pdf|\.pptx|\.xlsx|\.mp4|保存为|写到|导出为|生成文件|make a file|create a file)/i
+
+export function detectOutputMode(rawTask) {
+  const text = String(rawTask || '')
+  if (TEXT_OUTPUT_RE.test(text)) return 'text'
+  if (FILE_OUTPUT_RE.test(text)) return 'file'
+  return 'unknown'
+}
+
 export function decomposeTask(input = {}) {
   const rawTask = typeof input.task === 'string' ? input.task : String(input.task || '')
   const task = normalizeText(rawTask)
@@ -343,6 +353,7 @@ export function decomposeTask(input = {}) {
       subTasks: [],
       assumptions: ['任务为空：未做任何推断。'],
       confidence: 0,
+      outputMode: 'unknown',
       language: 'unknown',
     }
   }
@@ -432,6 +443,7 @@ export function decomposeTask(input = {}) {
     subTasks,
     assumptions,
     confidence,
+    outputMode: detectOutputMode(rawTask),
     language: detectLanguage(rawTask),
   }
 }

package/lib/intelligence/draft.js

@@ -306,6 +306,21 @@ function resolveContext(input) {
  * metadata, never writes or executes anything.
  * @param {{task?:string, plan?:object, missing?:object[], existingNames?:string[], context?:string}} input
  */
+function draftScore({ lint, safety, evalCases, content, name }) {
+  let score = 100
+  const issues = lint && Array.isArray(lint.issues) ? lint.issues : []
+  score -= issues.filter((issue) => issue && issue.severity === 'error').length * 20
+  score -= issues.filter((issue) => issue && issue.severity === 'warn').length * 5
+  if (!content || content.length < 500) score -= 15
+  if (!Array.isArray(evalCases) || evalCases.length < 3) score -= 10
+  if (safety && safety.available && safety.level === 'high') score -= 60
+  else if (safety && safety.available && safety.level === 'medium') score -= 15
+  if (!name || name.length < 6) score -= 10
+  score = Math.max(0, Math.min(100, Math.round(score)))
+  const grade = score >= 85 ? 'A' : score >= 70 ? 'B' : score >= 55 ? 'C' : 'D'
+  return { score, grade }
+}
+
 export function draftSkill(input = {}) {
   const { plan, task, decomposition } = resolveContext(input)
   const missing = Array.isArray(input.missing) && input.missing.length
@@ -430,9 +445,11 @@ export function draftSkill(input = {}) {
   })
   const safety = runSecurityScan(content)
   const similar = findSimilarNames(name, existingNames)
+  const quality = draftScore({ lint, safety, evalCases, content, name })
 
   return {
     ok: lint.ok && !(safety.available && safety.level === 'high'),
+    quality,
     name,
     description,
     whenToUse,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-ronge-adaptive-skill-archive",
-  "version": "5.2.1",
+  "version": "5.3.1",
   "description": "DSH-native skill intelligence cockpit: task decomposition, cross-domain multi-skill composition, skill authoring, local recommendations and safety scans for DeepSeek Harness.",
   "type": "module",
   "repository": {

package/scripts/benchmark-live.mjs

@@ -0,0 +1,83 @@
+#!/usr/bin/env node
+/**
+ * Live A/B protocol for task-level evidence. This script NEVER fabricates
+ * model runs: it either prints the protocol or aggregates records produced by
+ * a real DSH run.
+ *
+ * usage:
+ *   node scripts/benchmark-live.mjs --dry-run
+ *   node scripts/benchmark-live.mjs --record runs.json --out report.json
+ *
+ * run record shape:
+ *   [{ taskId, arm: "baseline"|"plugin", success: true, tokens: 12000,
+ *      toolCalls: 8, wallMs: 45000, notes: "..." }]
+ */
+import fs from 'node:fs'
+import path from 'node:path'
+import { fileURLToPath } from 'node:url'
+
+const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..')
+const TASKS_FILE = path.join(ROOT, 'test', 'fixtures', 'live-ab-tasks.json')
+const args = process.argv.slice(2)
+const flag = (name) => args.indexOf(name)
+const value = (name) => (flag(name) >= 0 ? args[flag(name) + 1] : null)
+const tasks = JSON.parse(fs.readFileSync(TASKS_FILE, 'utf8'))
+const dryRun = flag('--dry-run') >= 0
+const recordFile = value('--record')
+const outFile = value('--out')
+
+if (!dryRun && !recordFile) {
+  console.error('use --dry-run to print the protocol, or --record <file> to aggregate real run records')
+  process.exit(1)
+}
+
+function aggregate(records) {
+  const byArm = {}
+  for (const arm of ['baseline', 'plugin']) {
+    const rows = records.filter((record) => record.arm === arm)
+    const success = rows.filter((record) => record.success === true).length
+    const sum = (key) => rows.reduce((total, record) => total + (Number(record[key]) || 0), 0)
+    byArm[arm] = {
+      runs: rows.length,
+      successRate: rows.length ? Math.round((success / rows.length) * 1000) / 1000 : null,
+      meanTokens: rows.length ? Math.round(sum('tokens') / rows.length) : null,
+      meanToolCalls: rows.length ? Math.round((sum('toolCalls') / rows.length) * 10) / 10 : null,
+      meanWallMs: rows.length ? Math.round(sum('wallMs') / rows.length) : null,
+    }
+  }
+  return byArm
+}
+
+if (dryRun) {
+  const protocol = {
+    schema: 'skill-intelligence-live-ab/v1',
+    status: 'awaiting-real-runs',
+    tasks: tasks.length,
+    arms: ['baseline', 'plugin'],
+    instruction: 'Run each task twice (same model, same tools, same time budget). Arm "baseline" = no skill archive tools/panel; arm "plugin" = use skill_archive_plan then execute. Record success/tokens/toolCalls/wallMs per run.',
+    tasksDetail: tasks,
+    recordShape: { taskId: 'video-01', arm: 'baseline|plugin', success: true, tokens: 12000, toolCalls: 8, wallMs: 45000, notes: 'evidence' },
+    note: 'No numbers are produced until a real run is recorded; this script does not call a model.',
+  }
+  console.log(JSON.stringify(protocol, null, 2))
+  process.exit(0)
+}
+
+const raw = JSON.parse(fs.readFileSync(path.resolve(process.cwd(), recordFile), 'utf8'))
+if (!Array.isArray(raw)) throw new Error('record file must be a JSON array')
+const taskIds = new Set(tasks.map((task) => task.id))
+for (const record of raw) {
+  if (!record || !taskIds.has(record.taskId)) throw new Error('unknown taskId: ' + JSON.stringify(record && record.taskId))
+  if (record.arm !== 'baseline' && record.arm !== 'plugin') throw new Error('arm must be baseline or plugin')
+}
+const report = {
+  schema: 'skill-intelligence-live-ab/v1',
+  status: 'recorded',
+  generatedAt: new Date().toISOString(),
+  tasks: tasks.length,
+  records: raw.length,
+  arms: aggregate(raw),
+  caveat: 'Task-level A/B; success is human/rubric reviewed, tokens/time are runner-reported. Do not compare across models or task sets.',
+}
+if (outFile) fs.writeFileSync(path.resolve(process.cwd(), outFile), JSON.stringify(report, null, 2) + '\n')
+console.log(JSON.stringify(report, null, 2))

package/test/fixtures/live-ab-tasks.json

@@ -0,0 +1,157 @@
+[
+  {
+    "id": "video-01",
+    "domain": "video",
+    "task": "做一条 30 秒产品发布宣传片，竖屏，带字幕和配乐",
+    "deliverable": "mp4",
+    "checks": [
+      "时长 30±3 秒",
+      "竖屏 1080x1920",
+      "字幕与音频同步"
+    ]
+  },
+  {
+    "id": "video-02",
+    "domain": "video",
+    "task": "把这段培训录屏剪成 2 分钟横屏视频，加章节标题",
+    "deliverable": "mp4",
+    "checks": [
+      "横屏 1920x1080",
+      "章节标题出现",
+      "导出可播放"
+    ]
+  },
+  {
+    "id": "office-01",
+    "domain": "office",
+    "task": "把会议录音整理成 Word 纪要，含决议、行动项和负责人",
+    "deliverable": "docx",
+    "checks": [
+      "有决议/行动项/负责人",
+      "docx 可打开",
+      "无占位符"
+    ]
+  },
+  {
+    "id": "office-02",
+    "domain": "office",
+    "task": "把 Excel 销售数据做成 8 页发布 PPT，带图表和结论",
+    "deliverable": "pptx",
+    "checks": [
+      "8 页左右",
+      "至少 2 张图表",
+      "结论页存在"
+    ]
+  },
+  {
+    "id": "office-03",
+    "domain": "office",
+    "task": "检查这份 docx 合同的金额、日期和付款条款是否一致",
+    "deliverable": "docx-report",
+    "checks": [
+      "逐条列出差异",
+      "金额/日期有出处",
+      "风险项标注"
+    ]
+  },
+  {
+    "id": "code-01",
+    "domain": "coding",
+    "task": "修复 React 组件测试失败并补一个回归测试",
+    "deliverable": "code",
+    "checks": [
+      "测试通过",
+      "新增回归用例",
+      "说明根因"
+    ]
+  },
+  {
+    "id": "data-01",
+    "domain": "data",
+    "task": "分析这份 CSV 的月度趋势，输出图表和三条结论",
+    "deliverable": "report",
+    "checks": [
+      "图表可打开",
+      "结论有数字支撑",
+      "口径说明"
+    ]
+  },
+  {
+    "id": "research-01",
+    "domain": "research",
+    "task": "调研 2026 年 AI 视频工具市场，给出 5 个来源和对比",
+    "deliverable": "report",
+    "checks": [
+      "至少 5 个来源",
+      "有对比维度",
+      "引用可访问"
+    ]
+  },
+  {
+    "id": "marketing-01",
+    "domain": "marketing",
+    "task": "为新品写一版小红书种草文案，含 3 个卖点和行动号召",
+    "deliverable": "text",
+    "checks": [
+      "3 个卖点",
+      "有 CTA",
+      "符合平台语气"
+    ]
+  },
+  {
+    "id": "security-01",
+    "domain": "security",
+    "task": "审查一个第三方 SKILL.md 的安装脚本风险，重点看 shell 与凭据",
+    "deliverable": "report",
+    "checks": [
+      "列出风险级别",
+      "有复现/证据",
+      "给修复建议"
+    ]
+  },
+  {
+    "id": "plugin-01",
+    "domain": "engineering",
+    "task": "做一个 DSH 面板插件，加一个设置项并验证安装/卸载",
+    "deliverable": "code",
+    "checks": [
+      "插件可启用",
+      "设置项生效",
+      "可卸载"
+    ]
+  },
+  {
+    "id": "productivity-01",
+    "domain": "productivity",
+    "task": "把本周零散笔记整理成周报，附下周三个优先事项",
+    "deliverable": "docx",
+    "checks": [
+      "结构完整",
+      "三个优先级",
+      "无虚构事实"
+    ]
+  },
+  {
+    "id": "pilot-brief",
+    "domain": "marketing",
+    "task": "写一份技能驾驶舱 v5.3 发布简报（150–250 字），含发布日期、3 个能力点、1 个风险提示、1 句行动号召；直接返回正文",
+    "deliverable": "text",
+    "checks": [
+      "日期 2026-10-30",
+      "3 个能力点",
+      "1 个风险提示",
+      "1 句 CTA"
+    ]
+  },
+  {
+    "id": "pilot-video-plan",
+    "domain": "video",
+    "task": "为 Excel 数据分析产品做 30 秒竖屏宣传视频，给出 3 阶段执行计划，每阶段含技能、输入、输出/交接物、验收标准",
+    "deliverable": "plan",
+    "checks": [
+      "3 阶段",
+      "每阶段输入/输出",
+      "每阶段验收标准"
+    ]
+  }
+]