dsh-omni-router 2.0.0 → 2.1.0

package/README.md

@@ -71,7 +71,8 @@
 - **Evidence Engine / Write Locks**：`src/evidence-engine.mjs` 统一证据协议与存储；DAG scheduler 通过 `selectReadyBatch` 避免 write overlap 并行冲突。
 - **Stage 3/5**：Recursive Project Index、contextBudget 接入、Per-task Context、Missing capability handling、OmniBench v2 prompt generator。
 - **Stage 4（部分）**：`compileDagToWorkflow` 将 Mission DAG 编译为 DSH-native workflow outline。
-- **P1/P3 能力**：Real capability sandbox（已接入 `omni_mission_run`，按 role + capability 生成 toolFilter）、Capability Resolver v2（risk/reliability 综合打分）、OmniBench v2 scaffold（manifest schema + runner stub + prepare script + DAG-to-plan compile）。
+- **P1/P3 能力**：Real capability sandbox（已接入 `omni_mission_run`，按 role + capability 生成 toolFilter）、Capability Resolver v2（risk/reliability 综合打分）、OmniBench v2 runner（manifest schema + prepare + prompt generation + `--exec` 本地执行/结果采集 + DAG-to-plan compile）。
+- **2.1 收敛增强**：Mission Resume（`omni_mission_resume` 跨 session 继续 DAG，`runDagLoop` 支持 `onProgress` 周期保存）、Harness evidence extraction（`extractHarnessEvidence` 从 commands/tests/toolCalls/EVIDENCE_JSON 提取机器可验证证据）、context token budget（`maxContextTokens` 真正限制检索输出）、semantic failure retrieval + evidence-backed skill distillation、OmniBench v2 real runner。
 - **TaskDecision**：`createTaskDecision` 生成唯一决策对象，Policy/Runtime 统一消费，避免重复 classify 分叉。
 - **Evidence Protocol**：`src/evidence.mjs` 用结构化 command/file/test/finding 证据判定 PASS/FAIL；`isQaPass` 已支持从 QA 输出解析 JSON evidence，不再只信文本。
 - **Task Compiler**：`compileTask` 生成 objective / constraints / non-goals / acceptance / hidden assumptions / ambiguities / invariants / risk / artifacts；`compileTaskWithLLM` 可用 LLM 增强。
@@ -153,7 +154,7 @@ Then restart DSH and select **Omni Router** in a new session.
 - `/omni direct` — enter direct mode.
 - `/omni mode spec|react|balanced` — set thinking mode.
 - `/omni reroute plan|direct` — adaptively reroute current task.
-- Model tools: `omni_status`, `omni_plan`, `omni_direct`, `omni_mode`, `omni_reroute`, `omni_delegate`, `omni_memory`, `omni_benchmark`, `omni_mission_run`, `omni_visual_check`.
+- Model tools: `omni_status`, `omni_plan`, `omni_direct`, `omni_mode`, `omni_reroute`, `omni_delegate`, `omni_memory`, `omni_benchmark`, `omni_mission_run`, `omni_mission_resume`, `omni_visual_check`.
 
 ## Configuration
 

package/README.zh-CN.md

@@ -71,7 +71,8 @@
 - **Evidence Engine / Write Locks**：`src/evidence-engine.mjs` 统一证据协议与存储；DAG scheduler 通过 `selectReadyBatch` 避免 write overlap 并行冲突。
 - **Stage 3/5**：Recursive Project Index、contextBudget 接入、Per-task Context、Missing capability handling、OmniBench v2 prompt generator。
 - **Stage 4（部分）**：`compileDagToWorkflow` 将 Mission DAG 编译为 DSH-native workflow outline。
-- **P1/P3 能力**：Real capability sandbox（已接入 `omni_mission_run`，按 role + capability 生成 toolFilter）、Capability Resolver v2（risk/reliability 综合打分）、OmniBench v2 scaffold（manifest schema + runner stub + prepare script + DAG-to-plan compile）。
+- **P1/P3 能力**：Real capability sandbox（已接入 `omni_mission_run`，按 role + capability 生成 toolFilter）、Capability Resolver v2（risk/reliability 综合打分）、OmniBench v2 runner（manifest schema + prepare + prompt generation + `--exec` 本地执行/结果采集 + DAG-to-plan compile）。
+- **2.1 收敛增强**：Mission Resume（`omni_mission_resume` 跨 session 继续 DAG，`runDagLoop` 支持 `onProgress` 周期保存）、Harness evidence extraction（`extractHarnessEvidence` 从 commands/tests/toolCalls/EVIDENCE_JSON 提取机器可验证证据）、context token budget（`maxContextTokens` 真正限制检索输出）、semantic failure retrieval + evidence-backed skill distillation、OmniBench v2 real runner。
 - **TaskDecision**：`createTaskDecision` 生成唯一决策对象，Policy/Runtime 统一消费，避免重复 classify 分叉。
 - **Evidence Protocol**：`src/evidence.mjs` 用结构化 command/file/test/finding 证据判定 PASS/FAIL；`isQaPass` 已支持从 QA 输出解析 JSON evidence，不再只信文本。
 - **Task Compiler**：`compileTask` 生成 objective / constraints / non-goals / acceptance / hidden assumptions / ambiguities / invariants / risk / artifacts；`compileTaskWithLLM` 可用 LLM 增强。
@@ -153,7 +154,7 @@ node scripts/install-preset.mjs
 - `/omni direct` — 进入直接执行模式。
 - `/omni mode spec|react|balanced` — 设置思维模式。
 - `/omni reroute plan|direct` — 动态切换当前任务路由。
-- 模型工具：`omni_status` / `omni_plan` / `omni_direct` / `omni_mode` / `omni_reroute` / `omni_delegate` / `omni_memory` / `omni_benchmark` / `omni_mission_run` / `omni_visual_check`。
+- 模型工具：`omni_status` / `omni_plan` / `omni_direct` / `omni_mode` / `omni_reroute` / `omni_delegate` / `omni_memory` / `omni_benchmark` / `omni_mission_run` / `omni_mission_resume` / `omni_visual_check`。
 
 ## 配置
 

package/benchmark/omnibench-v2/README.md

@@ -1,8 +1,8 @@
-# OmniBench v2 (scaffold)
+# OmniBench v2
 
 Goal: move from 10 small fixture tasks to a reproducible multi-repo benchmark.
 
-## Requirements (from 优化5.md)
+## Requirements (from 优化5.md / 优化6.md)
 
 - 30~100 real GitHub repos
 - fixed starting commit
@@ -25,18 +25,18 @@ Goal: move from 10 small fixture tasks to a reproducible multi-repo benchmark.
   "task": "Fix intermittent login timeout under concurrent session refresh.",
   "acceptance": ["regression test added", "all tests pass"],
   "hiddenTests": ["tests/hidden/session-concurrent.test.js"],
+  "setupCommand": "npm ci",
+  "baselineCommand": "npm test",
+  "agentCommand": "node path/to/dsh-agent-runner.mjs",
+  "verifyCommand": "npm run test:hidden",
   "timeoutMs": 7200000,
   "runs": 5
 }
 ```
 
-See `manifest.example.json`.
-
-## Runner stub
-
-`node benchmark/omnibench-v2/runner-stub.mjs <manifest.json>` validates the
-manifest and prints a run plan. Actual execution will be driven by DSH
-subagents in a future integration.
+`setupCommand`, `baselineCommand`, `agentCommand`, and `verifyCommand` are
+optional. `agentCommand` receives the generated prompt as a single quoted
+argument; when present, `--exec` executes it inside the checked-out repo.
 
 ## Generate DSH prompts
 
@@ -46,7 +46,29 @@ npm run omnibench:v2:generate -- benchmark/omnibench-v2/manifest.local.example.j
 
 This writes one prompt per repo/arm/run under `benchmark/omnibench-v2/prompts/`.
 Open each prompt in a DSH Desktop session with the Omni Router preset to
-execute, then collect the results.
+execute manually, or wire `agentCommand` to your DSH CLI/runner for automation.
+
+## Run locally (automated)
+
+```bash
+# Plan + prompt generation only (default)
+node benchmark/omnibench-v2/run.mjs benchmark/omnibench-v2/manifest.local.example.json
+
+# Execute each run with the manifest's agentCommand
+node benchmark/omnibench-v2/run.mjs benchmark/omnibench-v2/manifest.local.example.json --exec
+
+# Override the agent command for every run
+node benchmark/omnibench-v2/run.mjs benchmark/omnibench-v2/manifest.local.example.json --exec --agent-command "node agent.mjs"
+```
+
+The runner clones/checks out each repo, optionally runs setup/baseline/verify
+commands, captures exit codes and output, and writes a timestamped JSON result
+file to `benchmark/omnibench-v2/results/`.
+
+## Runner stub
+
+`node benchmark/omnibench-v2/runner-stub.mjs <manifest.json>` still validates
+the manifest and prints the run plan for quick checks.
 
 ## Local / offline usage
 
@@ -57,6 +79,7 @@ existing `Omni group` fixture:
 ```bash
 npm run omnibench:v2:prepare -- benchmark/omnibench-v2/manifest.local.example.json
 npm run omnibench:v2:plan -- benchmark/omnibench-v2/manifest.local.example.json
+npm run omnibench:v2:generate -- benchmark/omnibench-v2/manifest.local.example.json
 ```
 
 `git clone` works with local paths, so no network is required. Replace `repo`

package/benchmark/omnibench-v2/run.mjs

@@ -2,20 +2,24 @@
 /**
  * OmniBench v2 runner / prompt generator.
  *
- * Generates per-repo per-arm per-run DSH prompts. Actual execution must happen
- * inside DSH Desktop; this script makes that reproducible.
+ * Generates per-repo per-arm per-run DSH prompts. With `--exec` and a
+ * configured `agentCommand` (or `--agent-command`), it also executes the run
+ * locally, runs baseline/verify commands, and writes machine-readable results.
  *
  * Usage:
- *   node benchmark/omnibench-v2/run.mjs <manifest.json>
+ *   node benchmark/omnibench-v2/run.mjs <manifest.json>            # plan + prompts
+ *   node benchmark/omnibench-v2/run.mjs <manifest.json> --exec     # execute agentCommand runs
+ *   node benchmark/omnibench-v2/run.mjs <manifest.json> --exec --agent-command "node agent.mjs"
  */
 
 import fs from 'node:fs'
 import path from 'node:path'
-import { fileURLToPath } from 'node:url'
+import { execSync } from 'node:child_process'
+import { fileURLToPath, pathToFileURL } from 'node:url'
 
 const here = path.dirname(fileURLToPath(import.meta.url))
 
-function buildPrompt(manifest, arm) {
+export function buildPrompt(manifest, arm) {
   const criteria = (manifest.acceptance || []).map((c) => `- ${c}`).join('\n')
   const task = `Task:\n${manifest.task}\n\nAcceptance criteria:\n${criteria}`
   if (arm === 'raw') {
@@ -24,10 +28,54 @@ function buildPrompt(manifest, arm) {
   return `${task}\n\nYou are using the Omni control plane. Follow engineering methodology, verify with real evidence, and do not claim completion without checks. When done, reply exactly "BENCHMARK: PASS" if you verified all criteria, otherwise "BENCHMARK: FAIL".`
 }
 
-function main() {
-  const manifestPath = process.argv[2] || path.join(here, 'manifest.local.example.json')
+export function readManifests(manifestPath) {
   const manifests = JSON.parse(fs.readFileSync(manifestPath, 'utf8'))
-  const list = Array.isArray(manifests) ? manifests : [manifests]
+  return Array.isArray(manifests) ? manifests : [manifests]
+}
+
+export function validateManifests(list) {
+  const errors = []
+  for (const m of list) {
+    if (!m.id) errors.push('missing id')
+    if (!m.repo) errors.push(`${m.id || '?'}: missing repo`)
+    if (!m.commit) errors.push(`${m.id || '?'}: missing commit`)
+    if (!m.task) errors.push(`${m.id || '?'}: missing task`)
+    if (!Array.isArray(m.acceptance) || !m.acceptance.length) errors.push(`${m.id || '?'}: missing acceptance`)
+    if (!m.runs || m.runs < 3) errors.push(`${m.id || '?'}: runs should be >= 3`)
+  }
+  return errors
+}
+
+export function ensureRepo(m) {
+  const workDir = path.resolve(process.env.OMNIBENCH_REPOS || path.join(here, 'repos'))
+  const dir = path.join(workDir, m.id)
+  fs.mkdirSync(workDir, { recursive: true })
+  if (!fs.existsSync(path.join(dir, '.git'))) {
+    console.log(`cloning ${m.repo} -> ${dir}`)
+    execSync(`git clone "${m.repo}" "${dir}"`, { stdio: 'inherit' })
+  }
+  console.log(`checkout ${m.id} @ ${m.commit}`)
+  execSync(`git -C "${dir}" checkout ${m.commit}`, { stdio: 'inherit' })
+  return dir
+}
+
+export function runCommand(command, cwd, label) {
+  if (!command) return { skipped: true }
+  const start = Date.now()
+  try {
+    const output = execSync(command, { cwd, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] })
+    return { skipped: false, exitCode: 0, output, durationMs: Date.now() - start }
+  } catch (error) {
+    return {
+      skipped: false,
+      exitCode: error.status ?? 1,
+      output: String(error.stdout || '') + String(error.stderr || ''),
+      durationMs: Date.now() - start,
+    }
+  }
+}
+
+export function generatePrompts(list) {
   const outDir = path.join(here, 'prompts')
   fs.mkdirSync(outDir, { recursive: true })
   let count = 0
@@ -40,8 +88,115 @@ function main() {
       }
     }
   }
-  console.log(`Generated ${count} DSH prompts in ${outDir}`)
-  console.log('Run each prompt inside a DSH Desktop session with the Omni Router preset, then collect results.')
+  return count
+}
+
+export function writeResults(results, resultsDir) {
+  fs.mkdirSync(resultsDir, { recursive: true })
+  const file = path.join(resultsDir, `omnibench-v2-${Date.now()}.json`)
+  fs.writeFileSync(file, JSON.stringify(results, null, 2), 'utf8')
+  return file
+}
+
+function main() {
+  const manifestPath = process.argv[2] || path.join(here, 'manifest.local.example.json')
+  const execMode = process.argv.includes('--exec')
+  const agentCommandOverride = (() => {
+    const idx = process.argv.indexOf('--agent-command')
+    return idx !== -1 ? process.argv[idx + 1] : null
+  })()
+  const resultsDir = path.resolve(process.env.OMNIBENCH_RESULTS || path.join(here, 'results'))
+
+  const list = readManifests(manifestPath)
+  const errors = validateManifests(list)
+  if (errors.length) {
+    console.error('Invalid manifest:')
+    for (const e of errors) console.error(`- ${e}`)
+    process.exit(1)
+  }
+  const promptCount = generatePrompts(list)
+
+  let totalRuns = 0
+  for (const m of list) totalRuns += (m.runs || 1) * 2
+  console.log(`Generated ${promptCount} DSH prompts in ${path.join(here, 'prompts')}`)
+  console.log(`OmniBench v2 run plan (${totalRuns} agent runs)`)
+
+  if (!execMode) {
+    for (const m of list) {
+      console.log(`- ${m.id}: ${m.repo} @ ${m.commit} (${m.language}/${m.framework}) runs=${m.runs} -> raw ${m.runs} + omni ${m.runs}`)
+    }
+    console.log('Pass --exec and configure agentCommand to execute runs locally.')
+    return
+  }
+
+  const results = []
+  for (const m of list) {
+    const repoDir = ensureRepo(m)
+    for (const arm of ['raw', 'omni']) {
+      for (let i = 1; i <= (m.runs || 3); i++) {
+        const agentCommand = agentCommandOverride || m.agentCommand
+        const promptFile = path.join(here, 'prompts', `${m.id}-${arm}-${i}.txt`)
+        const prompt = fs.readFileSync(promptFile, 'utf8')
+        console.log(`\n=== ${m.id} ${arm} run ${i} ===`)
+        if (m.setupCommand) {
+          const setup = runCommand(m.setupCommand, repoDir, 'setup')
+          console.log(`setup exit=${setup.exitCode ?? 'skip'}`)
+        }
+        const baseline = m.baselineCommand ? runCommand(m.baselineCommand, repoDir, 'baseline') : null
+        if (baseline) console.log(`baseline exit=${baseline.exitCode ?? 'skip'}`)
+
+        let agent = null
+        if (agentCommand) {
+          const command = `${agentCommand} ${JSON.stringify(prompt)}`
+          agent = runCommand(command, repoDir, 'agent')
+          console.log(`agent exit=${agent.exitCode ?? 'skip'} duration=${agent.durationMs ?? 0}ms`)
+        } else {
+          console.log(`agentCommand not configured; run prompt manually in DSH Desktop:\n  ${promptFile}`)
+        }
+
+        const verify = m.verifyCommand ? runCommand(m.verifyCommand, repoDir, 'verify') : null
+        if (verify) console.log(`verify exit=${verify.exitCode ?? 'skip'}`)
+
+        let success = null
+        if (verify && verify.skipped !== true) {
+          success = verify.exitCode === 0
+        } else if (agent && agent.skipped !== true) {
+          success = agent.exitCode === 0 && /BENCHMARK:\s*PASS/i.test(agent.output || '')
+        }
+
+        results.push({
+          id: m.id,
+          arm,
+          run: i,
+          repo: m.repo,
+          commit: m.commit,
+          task: m.task,
+          success,
+          agentExitCode: agent?.exitCode ?? null,
+          verifyExitCode: verify?.exitCode ?? null,
+          durationMs: (agent?.durationMs || 0) + (verify?.durationMs || 0),
+          baselineExitCode: baseline?.exitCode ?? null,
+          agentOutput: agent?.output?.slice(0, 4000) || '',
+          verifyOutput: verify?.output?.slice(0, 4000) || '',
+          metrics: {
+            tokens: 0,
+            cost: 0,
+            toolCalls: 0,
+          },
+          promptFile,
+          ranAt: new Date().toISOString(),
+        })
+      }
+    }
+  }
+
+  const file = writeResults(results, resultsDir)
+  const executed = results.filter((r) => r.success !== null).length
+  const passed = results.filter((r) => r.success === true).length
+  console.log(`\nWrote ${results.length} result record(s) to ${file}`)
+  console.log(`Executed ${executed}/${results.length} runs; passed ${passed}.`)
 }
 
-main()
+if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
+  main()
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-omni-router",
-  "version": "2.0.0",
+  "version": "2.1.0",
   "description": "Omni - a reliability and orchestration control plane for DeepSeek Harness, designed to turn fast models into dependable long-horizon software-engineering agents.",
   "type": "module",
   "main": "./lib/index.js",
@@ -41,7 +41,7 @@
     }
   },
   "scripts": {
-    "test": "node test/omni-router.test.mjs && node test/agent-chain.test.mjs && node test/skill-suggest.test.mjs && node test/methodology.test.mjs && node test/compat.test.mjs && node test/project-brain.test.mjs && node test/mission-planner.test.mjs && node test/agent-runtime.test.mjs && node test/judge.test.mjs && node test/memory.test.mjs && node test/memory-v3.test.mjs && node test/memory-engine.test.mjs && node test/mission-resume.test.mjs && node test/engineering-benchmark.test.mjs && node test/benchmark-runner.test.mjs && node test/benchmark-results.test.mjs && node test/continuous-eval.test.mjs && node --no-warnings test/project-brain-v2.test.mjs && node --no-warnings test/project-brain-v3.test.mjs && node test/visual-qa.test.mjs && node test/evidence.test.mjs && node test/evidence-store.test.mjs && node test/evidence-engine.test.mjs && node test/task-decision.test.mjs && node test/task-compiler.test.mjs && node test/capability-brain.test.mjs && node test/capability-manifest.test.mjs && node test/capability-sandbox.test.mjs && node test/planner-dag.test.mjs && node test/dynamic-context.test.mjs && node test/mission-dag.test.mjs && node test/failure-taxonomy.test.mjs && node test/context-expansion.test.mjs && node test/hybrid-retrieval.test.mjs && node test/dependency-graph.test.mjs && node test/ast-provider.test.mjs && node test/experience-router.test.mjs && node test/benchmark-tasks.test.mjs && node test/bundle-installer.test.mjs",
+    "test": "node test/omni-router.test.mjs && node test/agent-chain.test.mjs && node test/skill-suggest.test.mjs && node test/methodology.test.mjs && node test/compat.test.mjs && node test/project-brain.test.mjs && node test/mission-planner.test.mjs && node test/agent-runtime.test.mjs && node test/judge.test.mjs && node test/memory.test.mjs && node test/memory-v3.test.mjs && node test/memory-engine.test.mjs && node test/mission-resume.test.mjs && node test/engineering-benchmark.test.mjs && node test/benchmark-runner.test.mjs && node test/benchmark-results.test.mjs && node test/continuous-eval.test.mjs && node --no-warnings test/project-brain-v2.test.mjs && node --no-warnings test/project-brain-v3.test.mjs && node test/visual-qa.test.mjs && node test/evidence.test.mjs && node test/evidence-store.test.mjs && node test/evidence-engine.test.mjs && node test/task-decision.test.mjs && node test/task-compiler.test.mjs && node test/capability-brain.test.mjs && node test/capability-manifest.test.mjs && node test/capability-sandbox.test.mjs && node test/planner-dag.test.mjs && node test/dynamic-context.test.mjs && node test/mission-dag.test.mjs && node test/failure-taxonomy.test.mjs && node test/context-expansion.test.mjs && node test/hybrid-retrieval.test.mjs && node test/dependency-graph.test.mjs && node test/ast-provider.test.mjs && node test/experience-router.test.mjs && node test/benchmark-tasks.test.mjs && node test/bundle-installer.test.mjs && node test/omnibench-runner.test.mjs",
     "benchmark": "node benchmark/run.mjs",
     "benchmark:engineering": "node benchmark/engineering-run.mjs",
     "benchmark:compare": "node benchmark/compare.mjs",
@@ -50,6 +50,7 @@
     "omnibench:v2:plan": "node benchmark/omnibench-v2/runner-stub.mjs",
     "omnibench:v2:prepare": "node benchmark/omnibench-v2/prepare.mjs",
     "omnibench:v2:generate": "node benchmark/omnibench-v2/run.mjs",
+    "omnibench:v2:run": "node benchmark/omnibench-v2/run.mjs --exec",
     "analyze": "node benchmark/analyze.mjs",
     "llm-eval": "node benchmark/llm-eval.mjs",
     "install:preset": "node scripts/install-preset.mjs"

package/src/agent-runtime.mjs

@@ -165,6 +165,7 @@ export async function runDagLoop(dag, {
   maxRepairs = 0,
   maxSameActionRetries = 0,
   budget = {},
+  onProgress = null,
 } = {}) {
   const b = {
     maxSteps: maxSteps ?? budget.steps ?? 50,
@@ -231,6 +232,18 @@ export async function runDagLoop(dag, {
       actions.push({ taskId: task.id, observation })
     }
     step += 1
+    if (typeof onProgress === 'function') {
+      try {
+        await onProgress({
+          dag: current,
+          step,
+          actions,
+          metrics: { step, replanCount, repairCount, sameActionCount, tokenUsage, cost, toolCalls },
+        })
+      } catch {
+        // Progress callbacks are best-effort; they must never abort the mission.
+      }
+    }
   }
 
   const done = isMissionDagComplete(current)

package/src/context-expansion.mjs

@@ -54,7 +54,15 @@ export function buildProgressiveContext(taskText, entries, files = {}, options =
     const tests = [...names].filter((n) => /\.(test|spec)\./i.test(n))
     if (tests.length) parts.push('', 'Tests:', ...tests.map((t) => `- ${t}`))
   }
-  return parts.join('\n')
+  return applyContextBudget(parts.join('\n'), options)
+}
+
+function applyContextBudget(text, options = {}) {
+  const maxTokens = Number(options.maxContextTokens) || 0
+  if (maxTokens <= 0) return text
+  const maxChars = Math.max(32, maxTokens * 4)
+  if (text.length <= maxChars) return text
+  return `${text.slice(0, maxChars)}\n… (truncated by context budget ${maxTokens} tokens)`
 }
 
 export function shouldExpand(uncertainty = 0, threshold = 0.5) {

package/src/dynamic-context.mjs

@@ -33,6 +33,7 @@ export function buildDynamicContext(taskText, entries, files = {}, options = {})
       level,
       maxFiles: options.maxFiles || 8,
       maxFileChars: options.maxFileChars || 2000,
+      maxContextTokens: options.maxContextTokens || 0,
       graph: options.graph,
     }),
   }

package/src/evidence.mjs

@@ -70,6 +70,92 @@ export function evidencePass(evidence = {}) {
   return true
 }
 
+/**
+ * Extract machine-checkable evidence from a subagent/tool result.
+ *
+ * Accepts either a structured `result.evidence` object, top-level arrays on the
+ * result (`commands`, `tests`, `files`, `findings`), tool-call records, or an
+ * embedded `EVIDENCE_JSON` block in the result text. This lets the runtime
+ * prefer Harness-captured facts over agent prose whenever the Harness provides
+ * them.
+ */
+export function extractHarnessEvidence(result = {}) {
+  let evidence = createEvidence()
+  const source = (result && typeof result === 'object' && result.evidence) || result || {}
+
+  const apply = (key, fn) => {
+    for (const item of Array.isArray(source[key]) ? source[key] : []) {
+      try {
+        evidence = fn(evidence, item)
+      } catch {
+        // Skip malformed evidence entries; never fail the mission.
+      }
+    }
+  }
+
+  apply('commands', addCommandEvidence)
+  apply('tests', addTestEvidence)
+  apply('files', addFileEvidence)
+  apply('findings', addFindingEvidence)
+
+  if (Array.isArray(result?.toolCalls)) {
+    for (const call of result.toolCalls) {
+      try {
+        if (call?.command || call?.name) {
+          evidence = addCommandEvidence(evidence, {
+            command: call.command || call.name,
+            exitCode: call.exitCode ?? (call.ok === false ? 1 : 0),
+            output: call.output || call.stdout || '',
+            durationMs: call.durationMs || 0,
+          })
+        }
+      } catch {
+        // Ignore malformed tool-call evidence.
+      }
+    }
+  }
+
+  const embedded = parseEmbeddedEvidence(result?.output || result?.text || '')
+  if (embedded) {
+    for (const item of embedded.commands || []) {
+      try { evidence = addCommandEvidence(evidence, item) } catch { /* ignore */ }
+    }
+    for (const item of embedded.tests || []) {
+      try { evidence = addTestEvidence(evidence, item) } catch { /* ignore */ }
+    }
+    for (const item of embedded.files || []) {
+      try { evidence = addFileEvidence(evidence, item) } catch { /* ignore */ }
+    }
+    for (const item of embedded.findings || []) {
+      try { evidence = addFindingEvidence(evidence, item) } catch { /* ignore */ }
+    }
+  }
+
+  return evidence
+}
+
+function parseEmbeddedEvidence(text) {
+  const source = String(text || '')
+  const markers = ['EVIDENCE_JSON', 'EVIDENCE:']
+  for (const marker of markers) {
+    const start = source.indexOf(marker)
+    if (start === -1) continue
+    const jsonStart = source.indexOf('{', start)
+    if (jsonStart === -1) continue
+    const jsonEnd = source.lastIndexOf('}')
+    if (jsonEnd === -1 || jsonEnd <= jsonStart) continue
+    try {
+      const parsed = JSON.parse(source.slice(jsonStart, jsonEnd + 1))
+      if (parsed && (Array.isArray(parsed.commands) || Array.isArray(parsed.tests) || Array.isArray(parsed.files) || Array.isArray(parsed.findings))) {
+        return parsed
+      }
+    } catch {
+      // Not valid JSON; fall through to prose-based heuristics.
+    }
+  }
+  return null
+}
+
 export function summarizeEvidence(evidence = {}) {
   const lines = []
   if (evidence.commands?.length) {

package/src/memory-v3.mjs

@@ -17,10 +17,11 @@ export function createMemoryV3(base = {}) {
   }
 }
 
-function countOutcomes(trajectories = []) {
+function countOutcomes(trajectories = [], options = {}) {
   let successCount = 0
   let failureCount = 0
   for (const t of trajectories || []) {
+    if (options.requireEvidence && !hasEvidenceBacking(t)) continue
     const text = String(t?.text || t?.outcome || '').toLowerCase()
     if (/success|done|pass|完成|ok/.test(text)) successCount += 1
     else if (/fail|error|blocked|失败|错误/.test(text)) failureCount += 1
@@ -28,13 +29,28 @@ function countOutcomes(trajectories = []) {
   return { successCount, failureCount }
 }
 
+function hasEvidenceBacking(t = {}) {
+  const evidence = t.evidence || t.verification || t.verified
+  if (evidence === true) return true
+  if (evidence && typeof evidence === 'object') {
+    const e = evidence
+    return Boolean(
+      (Array.isArray(e.commands) && e.commands.length) ||
+      (Array.isArray(e.tests) && e.tests.length) ||
+      (Array.isArray(e.files) && e.files.length) ||
+      (Array.isArray(e.findings) && e.findings.length),
+    )
+  }
+  return false
+}
+
 export function distillSkill(memory, trajectories = [], options = {}) {
   const minSuccesses = Number(options.minSuccesses) || 3
-  const { successCount, failureCount } = countOutcomes(trajectories)
+  const { successCount, failureCount } = countOutcomes(trajectories, options)
   if (successCount < minSuccesses) return { memory, promoted: false, reason: `need ${minSuccesses} successes, got ${successCount}` }
 
   const successfulSteps = (trajectories || [])
-    .filter((t) => /success|done|pass|完成|ok/i.test(String(t?.text || t?.outcome || '')))
+    .filter((t) => (!options.requireEvidence || hasEvidenceBacking(t)) && /success|done|pass|完成|ok/i.test(String(t?.text || t?.outcome || '')))
     .map((t) => t?.text || t?.step || '')
     .filter(Boolean)
 
@@ -78,11 +94,27 @@ export function recordSkillOutcome(memory, skillName, success) {
   return { ...memory, learnedSkills: skills }
 }
 
-export function retrieveHistoricalFailures(memory, taskText = '') {
+export function retrieveHistoricalFailures(memory, taskText = '', options = {}) {
   const text = String(taskText || '').toLowerCase()
   const failures = memory.failures || []
   if (!text) return failures
-  return failures.filter((f) => String(f.text || f.reason || '').toLowerCase().includes(text))
+  const minScore = options.minScore === undefined ? 0.1 : Number(options.minScore) || 0
+  const tokens = tokenize(text)
+  return failures
+    .map((f) => {
+      const hay = `${f.text || ''} ${f.reason || ''} ${f.category || ''} ${f.type || ''}`.toLowerCase()
+      const hayTokens = tokenize(hay)
+      const overlap = hayTokens.filter((t) => tokens.includes(t)).length
+      const score = overlap / Math.max(1, Math.min(tokens.length, hayTokens.length))
+      return { ...f, _score: Math.round(score * 1000) / 1000 }
+    })
+    .filter((f) => f._score >= minScore || String(f.text || f.reason || '').toLowerCase().includes(text))
+    .sort((a, b) => b._score - a._score)
+    .map(({ _score, ...f }) => f)
+}
+
+function tokenize(text) {
+  return String(text || '').toLowerCase().match(/[a-z0-9_]+/g) || []
 }
 
 export function recordExecutionPolicy(memory, policy = {}) {

package/src/omni-router.mjs

@@ -30,9 +30,9 @@ import { runDagLoop } from './agent-runtime.mjs'
 import { buildVisualQaPrompt, buildVisualQaStepRequirement, callVisionApi, isFrontendTask, parseVisualQaResponse } from './visual-qa.mjs'
 import { createTaskDecision, buildPolicyFromTaskDecision } from './task-decision.mjs'
 import { compileTaskWithLLM } from './task-compiler.mjs'
-import { bindCapabilitiesToDag, createMissionDag, formatMissionDag } from './mission-dag.mjs'
+import { bindCapabilitiesToDag, createMissionDag, formatMissionDag, isMissionDagComplete } from './mission-dag.mjs'
 import { generateMissionDag, roleForTask } from './planner-dag.mjs'
-import { autoPopulateCapabilityBrain, createCapabilityBrain } from './capability-brain.mjs'
+import { autoPopulateCapabilityBrain, createCapabilityBrain, recordCapabilityOutcome } from './capability-brain.mjs'
 import { loadCapabilityManifests } from './capability-manifest.mjs'
 import { capabilityToolFilter } from './capability-sandbox.mjs'
 import { buildProgressiveContext } from './context-expansion.mjs'
@@ -42,7 +42,8 @@ import { retrieveContext } from './hybrid-retrieval.mjs'
 import { formatMemory, recordDecision, recordFailure, recordProject, recordTrajectory, summarizeMemory } from './memory.mjs'
 import { createMemoryEngine, loadMemoryEngine, saveMemoryEngine } from './memory-engine.mjs'
 import { captureEvidence, createEvidenceStore, evidenceSummary } from './evidence-store.mjs'
-import { saveMissionState } from './mission-resume.mjs'
+import { evidencePass, extractHarnessEvidence } from './evidence.mjs'
+import { listMissionStates, loadMissionState, saveMissionState } from './mission-resume.mjs'
 import { collectResults, formatResultSummary, importBenchmarkRecord, missingTaskIds, summarizeResults } from './benchmark-results.mjs'
 import { buildAstGraph, collectSourceFiles } from './ast-provider.mjs'
 
@@ -695,6 +696,7 @@ export function apply(ctx, config = {}) {
             uncertainty: Number(config.contextUncertainty) || 0.6,
             maxFiles,
             maxFileChars: 2000,
+            maxContextTokens: ctxBudget.retrievalBudget,
             graph: graphAdj,
           })
           return dynamic.context
@@ -1359,6 +1361,113 @@ export function apply(ctx, config = {}) {
     },
   })
 
+  async function createMissionExecutor({ session, agent, subagents, task, taskType, frontend, brief, capabilityBrain, evidenceRecords, resumeKey }) {
+    const cwd = session?.meta?.cwd || session?.header?.cwd
+    let brain = capabilityBrain
+
+    const saveProgress = async (snapshot = {}) => {
+      if (!cwd || !resumeKey) return
+      const dag = snapshot.dag
+      saveMissionState(cwd, resumeKey, {
+        status: dag ? (isMissionDagComplete(dag) ? 'completed' : 'active') : 'active',
+        dag,
+        evidence: evidenceRecords,
+        metrics: snapshot.metrics || {},
+        capabilityBrain: brain,
+        brief,
+        task,
+        taskType,
+        savedAt: new Date().toISOString(),
+      })
+    }
+
+    const act = async (action) => {
+      const goal = action.task?.goal || action.taskId || ''
+      const visualQa = frontend && /validate|verify|visual|ui/i.test(goal) && config.autoVisualQA !== false
+        ? `\n\n${buildVisualQaStepRequirement()}`
+        : ''
+      const briefText = `\n\nTask brief:\nObjective: ${brief.objective}\nAcceptance: ${brief.acceptanceCriteria.join('; ')}`
+      const taskContext = await getProjectContext(session, taskType, goal).catch(() => '')
+      const prompt = `Mission: ${task}\nTask: ${action.taskId} — ${goal}\n\nExecute this step. Reply with a short result and evidence.${briefText}${taskContext ? `\n\nContext:\n${taskContext}` : ''}${visualQa}`
+      const role = roleForTask(action.task || {})
+      const sandbox = capabilityToolFilter(brain, action.task?.requiredCapabilities || [], role)
+      const run = await subagents.start('spawn', {
+        label: `mission-${action.taskId}`,
+        prompt: [{ type: 'text', text: prompt }],
+        parent: agent,
+        maxDepth: 1,
+        ...(sandbox.allow.length || sandbox.deny.length ? { toolFilter: sandbox } : {}),
+      })
+      const result = await run.result
+      const output = (Array.isArray(result.output) ? result.output : [])
+        .filter((block) => block?.type === 'text' && typeof block.text === 'string')
+        .map((block) => block.text)
+        .join('')
+      try { await run.dispose() } catch { /* best-effort */ }
+
+      const harness = extractHarnessEvidence({ ...result, output })
+      const hasStructured = harness.commands.length > 0 || harness.tests.length > 0 || harness.files.length > 0 || harness.findings.length > 0
+      let record
+      if (hasStructured) {
+        record = {
+          id: `E-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 6)}`,
+          type: 'harness',
+          source: action.taskId,
+          value: output.slice(0, 2000),
+          ok: evidencePass(harness),
+          evidence: harness,
+          at: new Date().toISOString(),
+        }
+      } else {
+        const captured = captureEvidence({ entries: evidenceRecords }, {
+          type: 'agent_output',
+          source: action.taskId,
+          value: output.slice(0, 2000),
+          ok: !/FAIL|error|失败|not ok/i.test(output),
+        })
+        record = captured.record
+      }
+      evidenceRecords.push(record)
+      return {
+        output,
+        evidenceId: record.id,
+        evidence: harness,
+        hasStructuredEvidence: hasStructured,
+        tokenUsage: result?.tokenUsage ?? result?.usage?.totalTokens ?? 0,
+        cost: result?.cost ?? 0,
+        toolCalls: result?.toolCalls ?? result?.usage?.toolCalls ?? 0,
+      }
+    }
+
+    const observe = async (result, _current, action) => {
+      const goal = action?.task?.goal || ''
+      const needsVisual = frontend && /validate|verify|visual|ui/i.test(goal) && config.autoVisualQA !== false
+      if (needsVisual && !/VISUAL_QA:\s*PASS/i.test(result.output || '')) return { type: 'test_failure', reason: 'visual QA failed' }
+
+      const harness = result.evidence || {}
+      const hasStructured = result.hasStructuredEvidence === true || harness.commands?.length || harness.tests?.length || harness.files?.length || harness.findings?.length
+      if (hasStructured) {
+        const passed = evidencePass(harness)
+        if (!passed) {
+          brain = recordCapabilityOutcome(brain, action?.task?.allowedTools?.[0], false)
+          return { type: 'test_failure', reason: 'structured harness evidence failed', evidence: harness }
+        }
+        brain = recordCapabilityOutcome(brain, action?.task?.allowedTools?.[0], true)
+        return { type: 'step_done', evidence: harness }
+      }
+
+      if (/FAIL|error|失败|not ok/i.test(result.output || '')) {
+        const failure = classifyFailure({ type: 'unknown', reason: result.output || '' })
+        brain = recordCapabilityOutcome(brain, action?.task?.allowedTools?.[0], false)
+        return { type: failure.category, reason: failure.recovery, detail: String(result.output || '').slice(0, 500) }
+      }
+      brain = recordCapabilityOutcome(brain, action?.task?.allowedTools?.[0], true)
+      return { type: 'step_done' }
+    }
+
+    return { act, observe, saveProgress, getCapabilityBrain: () => brain }
+  }
+
   registerTool({
     name: 'omni_mission_run',
     description: 'Run a Mission Planner loop with real subagents: Observe → Think → Act → Replan until completed or maxSteps.',
@@ -1397,56 +1506,25 @@ export function apply(ctx, config = {}) {
       capabilityBrain = loadCapabilityManifests(capabilityBrain, config.capabilityManifests || [])
       const dag = bindCapabilitiesToDag(generateMissionDag(mission, brief, { taskType }), capabilityBrain)
       const evidenceRecords = []
+      const cwd = session?.meta?.cwd || session?.header?.cwd
+      const resumeKey = cwd ? `mission-${Date.now().toString(36)}` : null
+      const executor = await createMissionExecutor({
+        session,
+        agent,
+        subagents,
+        task,
+        taskType,
+        frontend,
+        brief,
+        capabilityBrain,
+        evidenceRecords,
+        resumeKey,
+      })
 
       const finalDag = await runDagLoop(dag, {
-        act: async (action) => {
-          const goal = action.task?.goal || action.taskId || ''
-          const visualQa = frontend && /validate|verify|visual|ui/i.test(goal) && config.autoVisualQA !== false
-            ? `\n\n${buildVisualQaStepRequirement()}`
-            : ''
-          const briefText = `\n\nTask brief:\nObjective: ${brief.objective}\nAcceptance: ${brief.acceptanceCriteria.join('; ')}`
-          const taskContext = await getProjectContext(session, taskType, goal).catch(() => '')
-          const prompt = `Mission: ${task}\nTask: ${action.taskId} — ${goal}\n\nExecute this step. Reply with a short result and evidence.${briefText}${taskContext ? `\n\nContext:\n${taskContext}` : ''}${visualQa}`
-          const role = roleForTask(action.task || {})
-          const sandbox = capabilityToolFilter(capabilityBrain, action.task?.requiredCapabilities || [], role)
-          const run = await subagents.start('spawn', {
-            label: `mission-${action.taskId}`,
-            prompt: [{ type: 'text', text: prompt }],
-            parent: agent,
-            maxDepth: 1,
-            ...(sandbox.allow.length || sandbox.deny.length ? { toolFilter: sandbox } : {}),
-          })
-          const result = await run.result
-          const output = (Array.isArray(result.output) ? result.output : [])
-            .filter((block) => block?.type === 'text' && typeof block.text === 'string')
-            .map((block) => block.text)
-            .join('')
-          try { await run.dispose() } catch { /* best-effort */ }
-          const captured = captureEvidence({ entries: evidenceRecords }, {
-            type: 'agent_output',
-            source: action.taskId,
-            value: output.slice(0, 2000),
-            ok: !/FAIL|error|失败|not ok/i.test(output),
-          })
-          evidenceRecords.push(captured.record)
-          return {
-            output,
-            evidenceId: captured.record.id,
-            tokenUsage: result?.tokenUsage ?? result?.usage?.totalTokens ?? 0,
-            cost: result?.cost ?? 0,
-            toolCalls: result?.toolCalls ?? result?.usage?.toolCalls ?? 0,
-          }
-        },
-        observe: async (result, _current, action) => {
-          const goal = action?.task?.goal || ''
-          const needsVisual = frontend && /validate|verify|visual|ui/i.test(goal) && config.autoVisualQA !== false
-          if (needsVisual && !/VISUAL_QA:\s*PASS/i.test(result.output || '')) return { type: 'test_failure', reason: 'visual QA failed' }
-          if (/FAIL|error|失败|not ok/i.test(result.output || '')) {
-            const failure = classifyFailure({ type: 'unknown', reason: result.output || '' })
-            return { type: failure.category, reason: failure.recovery, detail: String(result.output || '').slice(0, 500) }
-          }
-          return { type: 'step_done' }
-        },
+        act: executor.act,
+        observe: executor.observe,
+        onProgress: executor.saveProgress,
         maxSteps,
         maxParallel: Number(args?.maxParallel) || 1,
         maxTokens: Number(args?.maxTokens) || 0,
@@ -1456,19 +1534,21 @@ export function apply(ctx, config = {}) {
         maxReplans: Number(args?.maxReplans) || 0,
         maxWallClockMs: Number(args?.maxWallClockMs) || 0,
       })
+      capabilityBrain = executor.getCapabilityBrain()
 
       const metrics = finalDag.metrics || {}
       const evSummary = evidenceSummary({ entries: evidenceRecords })
-      const cwd = session?.meta?.cwd || session?.header?.cwd
       let resumeInfo = ''
-      if (cwd) {
-        const resumeKey = `mission-${Date.now().toString(36)}`
+      if (cwd && resumeKey) {
         const saved = saveMissionState(cwd, resumeKey, {
           status: finalDag.status,
           dag: finalDag.dag,
           evidence: evidenceRecords,
           metrics,
+          capabilityBrain,
+          brief,
           task,
+          taskType,
           savedAt: new Date().toISOString(),
         })
         resumeInfo = `\nSaved mission state: ${saved}`
@@ -1487,6 +1567,107 @@ export function apply(ctx, config = {}) {
     },
   })
 
+  registerTool({
+    name: 'omni_mission_resume',
+    description: 'Resume a saved Mission DAG from .omni/missions/<key>.json. Without a key, lists saved missions.',
+    parameters: {
+      type: 'object',
+      properties: {
+        key: { type: 'string', description: 'Saved mission state key (from omni_mission_run output)' },
+        maxSteps: { type: 'number', description: 'Max additional loop steps (default 20)' },
+        maxParallel: { type: 'number', description: 'Max tasks to run concurrently (default 1)' },
+        maxTokens: { type: 'number', description: 'Max token budget for the resumed portion (0 = unlimited)' },
+        maxCost: { type: 'number', description: 'Max cost budget for the resumed portion (0 = unlimited)' },
+        maxToolCalls: { type: 'number', description: 'Max tool calls for the resumed portion (0 = unlimited)' },
+        maxRepairs: { type: 'number', description: 'Max repairs before blocking (0 = unlimited)' },
+        maxReplans: { type: 'number', description: 'Max replans before blocking (0 = unlimited)' },
+        maxWallClockMs: { type: 'number', description: 'Max wall-clock time in ms (0 = unlimited)' },
+      },
+      required: [],
+    },
+    async execute(args) {
+      const session = currentSession()
+      const agent = session && agentFor(session)
+      const subagents = ctx.get('subagents') || ctx.subagents
+      const cwd = session?.meta?.cwd || session?.header?.cwd
+      if (!cwd) return 'No workspace cwd found.'
+      const key = String(args?.key || '').trim()
+      if (!key) {
+        const keys = listMissionStates(cwd)
+        return keys.length
+          ? `Saved missions:\n${keys.map((k) => `- ${k}`).join('\n')}\n\nPass "key" to resume one.`
+          : 'No saved missions found in .omni/missions.'
+      }
+      const saved = loadMissionState(cwd, key)
+      if (!saved) {
+        const keys = listMissionStates(cwd)
+        return `No saved mission found for key "${key}". Available: ${keys.join(', ') || '(none)'}`
+      }
+      if (saved.status === 'completed') return `Mission "${key}" is already completed.`
+      if (!session || !agent || !subagents?.start) return 'Mission resume requires an active session with subagents.'
+
+      const task = saved.task || saved.dag?.mission?.task || ''
+      const taskType = saved.taskType || classifyTaskType(task)
+      const frontend = isFrontendTask(task)
+      const brief = saved.brief || { objective: task, acceptanceCriteria: [] }
+      const evidenceRecords = Array.isArray(saved.evidence) ? saved.evidence : []
+      const toolsService = ctx.get('tools') || ctx.tools
+      let capabilityBrain = saved.capabilityBrain || autoPopulateCapabilityBrain(createCapabilityBrain(), await collectToolNames(toolsService))
+      capabilityBrain = loadCapabilityManifests(capabilityBrain, config.capabilityManifests || [])
+
+      const executor = await createMissionExecutor({
+        session,
+        agent,
+        subagents,
+        task,
+        taskType,
+        frontend,
+        brief,
+        capabilityBrain,
+        evidenceRecords,
+        resumeKey: key,
+      })
+
+      const finalDag = await runDagLoop(saved.dag, {
+        act: executor.act,
+        observe: executor.observe,
+        onProgress: executor.saveProgress,
+        maxSteps: Number(args?.maxSteps) || 20,
+        maxParallel: Number(args?.maxParallel) || 1,
+        maxTokens: Number(args?.maxTokens) || 0,
+        maxCost: Number(args?.maxCost) || 0,
+        maxToolCalls: Number(args?.maxToolCalls) || 0,
+        maxRepairs: Number(args?.maxRepairs) || 0,
+        maxReplans: Number(args?.maxReplans) || 0,
+        maxWallClockMs: Number(args?.maxWallClockMs) || 0,
+      })
+
+      const metrics = finalDag.metrics || {}
+      saveMissionState(cwd, key, {
+        status: finalDag.status,
+        dag: finalDag.dag,
+        evidence: evidenceRecords,
+        metrics,
+        capabilityBrain: executor.getCapabilityBrain(),
+        brief,
+        task,
+        taskType,
+        savedAt: new Date().toISOString(),
+      })
+      const evSummary = evidenceSummary({ entries: evidenceRecords })
+      return [
+        `Mission resume: ${finalDag.status}`,
+        `Tasks done: ${finalDag.dag.tasks.filter((t) => t.status === 'done').length}/${finalDag.dag.tasks.length}`,
+        `Additional steps: ${finalDag.actions.length}`,
+        `Replans: ${metrics.replanCount || 0}, Repairs: ${metrics.repairCount || 0}, ToolCalls: ${metrics.toolCalls || 0}, Tokens: ${metrics.tokenUsage || 0}, Cost: ${metrics.cost || 0}`,
+        `Evidence captured: ${evSummary.total} (failed=${evSummary.failed})`,
+        `Mission: ${task}`,
+        '',
+        formatMissionDag(finalDag.dag),
+      ].join('\n')
+    },
+  })
+
   registerTool({
     name: 'omni_plan',
     description: 'Force the current session into plan-first mode (enter built-in plan mode).',

package/test/agent-runtime.test.mjs

@@ -146,3 +146,20 @@ test('runDagLoop respects maxReplans', async () => {
   })
   assert.equal(result.status, 'max_replans')
 })
+
+test('runDagLoop calls onProgress after each batch', async () => {
+  const dag = createMissionDag(buildMission('实现退款', { taskType: 'feature' }))
+  const snapshots = []
+  const result = await runDagLoop(dag, {
+    act: async () => ({ ok: true }),
+    observe: async () => ({ type: 'step_done' }),
+    maxSteps: 100,
+    maxParallel: 1,
+    onProgress: async (snapshot) => {
+      snapshots.push(snapshot)
+    },
+  })
+  assert.equal(result.status, 'completed')
+  assert.ok(snapshots.length >= dag.tasks.length)
+  assert.ok(snapshots.every((s) => s.dag && Array.isArray(s.actions)))
+})

package/test/dynamic-context.test.mjs

@@ -36,3 +36,14 @@ test('buildDynamicContext expands when uncertainty is high', () => {
   assert.equal(result.level, 2)
   assert.match(result.context, /Implementations:/)
 })
+
+test('buildDynamicContext enforces maxContextTokens budget', () => {
+  const result = buildDynamicContext('fix login', entries, files, {
+    level: 4,
+    uncertainty: 0.8,
+    threshold: 0.5,
+    maxContextTokens: 10,
+  })
+  assert.ok(result.context.length <= 10 * 4 + 64)
+  assert.match(result.context, /truncated by context budget/)
+})

package/test/evidence.test.mjs

@@ -8,6 +8,7 @@ import {
   addTestEvidence,
   createEvidence,
   evidencePass,
+  extractHarnessEvidence,
   summarizeEvidence,
 } from '../src/evidence.mjs'
 
@@ -44,3 +45,37 @@ test('summarizeEvidence renders sections', () => {
   assert.match(text, /Commands/)
   assert.match(text, /Tests/)
 })
+
+test('extractHarnessEvidence normalizes structured result evidence', () => {
+  const evidence = extractHarnessEvidence({
+    commands: [{ command: 'npm test', exitCode: 0, output: 'ok' }],
+    tests: [{ command: 'npm test', exitCode: 0, total: 3, passed: 3, failed: 0 }],
+    files: [{ file: 'src/a.ts', lines: [1], beforeHash: 'a', afterHash: 'b', diffHash: 'c' }],
+    findings: [{ finding: 'nit', severity: 'low', file: 'src/a.ts', line: 2 }],
+  })
+  assert.equal(evidence.commands.length, 1)
+  assert.equal(evidence.tests.length, 1)
+  assert.equal(evidence.files.length, 1)
+  assert.equal(evidence.findings.length, 1)
+  assert.equal(evidencePass(evidence), true)
+})
+
+test('extractHarnessEvidence reads toolCalls as command evidence', () => {
+  const evidence = extractHarnessEvidence({
+    toolCalls: [
+      { name: 'npm test', exitCode: 0, output: 'all pass' },
+      { name: 'node run.js', exitCode: 1, output: 'boom' },
+    ],
+  })
+  assert.equal(evidence.commands.length, 2)
+  assert.equal(evidence.commands[0].exitCode, 0)
+  assert.equal(evidence.commands[1].exitCode, 1)
+  assert.equal(evidencePass(evidence), false)
+})
+
+test('extractHarnessEvidence parses embedded EVIDENCE_JSON block', () => {
+  const output = 'I ran the checks.\nEVIDENCE_JSON\n{"tests":[{"command":"npm test","exitCode":0,"total":2,"passed":2,"failed":0}]}'
+  const evidence = extractHarnessEvidence({ output })
+  assert.equal(evidence.tests.length, 1)
+  assert.equal(evidencePass(evidence), true)
+})

package/test/memory-v3.test.mjs

@@ -37,6 +37,38 @@ test('retrieveHistoricalFailures filters by task text', () => {
   assert.equal(hits.length, 1)
 })
 
+test('retrieveHistoricalFailures ranks by token overlap instead of exact substring only', () => {
+  const memory = createMemoryV3({
+    failures: [
+      { text: 'login timeout under concurrency', reason: 'session refresh race' },
+      { text: 'payment rounding', reason: 'float math' },
+    ],
+  })
+  const hits = retrieveHistoricalFailures(memory, 'session refresh race timeout')
+  assert.ok(hits.length >= 1)
+  assert.match(hits[0].text, /login timeout/)
+})
+
+test('distillSkill can require evidence-backed successes before promoting', () => {
+  const memory = createMemoryV3()
+  const noEvidence = distillSkill(
+    memory,
+    [{ text: 'step ok' }, { text: 'step ok' }, { text: 'step ok' }],
+    { name: 'no-evidence', minSuccesses: 3, requireEvidence: true },
+  )
+  assert.equal(noEvidence.promoted, false)
+  const withEvidence = distillSkill(
+    memory,
+    [
+      { text: 'step ok', evidence: { tests: [{ command: 'npm test', exitCode: 0, total: 1, passed: 1, failed: 0 }] } },
+      { text: 'step ok', evidence: { commands: [{ command: 'npm test', exitCode: 0 }] } },
+      { text: 'step ok', evidence: { tests: [{ command: 'npm test', exitCode: 0, total: 1, passed: 1, failed: 0 }] } },
+    ],
+    { name: 'evidence-skill', minSuccesses: 3, requireEvidence: true },
+  )
+  assert.equal(withEvidence.promoted, true)
+})
+
 test('execution policies and cross-session strategies persist in memory', () => {
   let memory = createMemoryV3()
   memory = recordExecutionPolicy(memory, { name: 'schema-change', trigger: 'migration' })

package/test/omnibench-runner.test.mjs

@@ -0,0 +1,68 @@
+import test from 'node:test'
+import assert from 'node:assert/strict'
+import fs from 'node:fs'
+import os from 'node:os'
+import path from 'node:path'
+
+import {
+  buildPrompt,
+  readManifests,
+  runCommand,
+  validateManifests,
+  writeResults,
+} from '../benchmark/omnibench-v2/run.mjs'
+
+test('buildPrompt includes task and BENCHMARK marker', () => {
+  const manifest = { task: 'Fix pagination', acceptance: ['all tests pass'] }
+  const raw = buildPrompt(manifest, 'raw')
+  assert.match(raw, /Fix pagination/)
+  assert.match(raw, /BENCHMARK: PASS/)
+  const omni = buildPrompt(manifest, 'omni')
+  assert.match(omni, /Omni control plane/)
+})
+
+test('validateManifests returns errors for invalid manifest', () => {
+  const errors = validateManifests([{ id: 'x' }])
+  assert.ok(errors.length >= 3)
+  assert.equal(validateManifests([{
+    id: 'x',
+    repo: 'r',
+    commit: 'c',
+    task: 't',
+    acceptance: ['a'],
+    runs: 3,
+  }]).length, 0)
+})
+
+test('readManifests supports single object and array', () => {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'omnibench-read-'))
+  try {
+    const file = path.join(dir, 'manifest.json')
+    fs.writeFileSync(file, JSON.stringify({ id: 'one', repo: 'r', commit: 'c', task: 't', acceptance: ['a'], runs: 3 }))
+    assert.equal(readManifests(file).length, 1)
+    fs.writeFileSync(file, JSON.stringify([{ id: 'one', repo: 'r', commit: 'c', task: 't', acceptance: ['a'], runs: 3 }]))
+    assert.equal(readManifests(file).length, 1)
+  } finally {
+    fs.rmSync(dir, { recursive: true, force: true })
+  }
+})
+
+test('runCommand captures exit code and output', () => {
+  const ok = runCommand('node -e "console.log(1)"', process.cwd())
+  assert.equal(ok.exitCode, 0)
+  assert.match(ok.output, /1/)
+  const bad = runCommand('node -e "process.exit(3)"', process.cwd())
+  assert.equal(bad.exitCode, 3)
+})
+
+test('writeResults writes JSON result file', () => {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'omnibench-write-'))
+  try {
+    const file = writeResults([{ id: 'x', success: true }], dir)
+    assert.ok(fs.existsSync(file))
+    const parsed = JSON.parse(fs.readFileSync(file, 'utf8'))
+    assert.equal(parsed[0].success, true)
+  } finally {
+    fs.rmSync(dir, { recursive: true, force: true })
+  }
+})