omnilane 0.13.0 → 0.14.0

package/CHANGELOG.md

@@ -6,6 +6,37 @@ semantic version tags.
 
 ## [Unreleased]
 
+## [0.14.0] - 2026-08-10
+
+### Added
+
+- Optional `omnilane doctor --probe V` and read-only MCP `provider_probe` perform
+  one bounded advise-mode live capability check only when explicitly requested;
+  reports include routing/timing metadata but never the provider response body.
+- Live Board retained-history search and state filters now drive a local
+  **Export visible** JSON download that excludes tokens, task/result bodies,
+  workdirs, and logs.
+- `omnilane benchmark` provides a versioned fixed-workload quality/timing report,
+  transparent user-supplied per-call cost estimates, default no-call dry-run,
+  and an explicit `--run` provider-call gate.
+- CI runs `omnilane doctor --strict --json` against an isolated, offline fixture
+  as a post-install acceptance contract.
+
+- `omnilane jobs recommend` and the read-only MCP `jobs_recommend` tool derive
+  per-lane vendor suggestions from bounded completed-job metadata. A configurable
+  minimum sample gate prevents one-off successes from outranking established
+  evidence; neither surface reads task/result bodies or changes routing.
+- `omnilane doctor --strict` keeps warning diagnostics intact but exits nonzero
+  when any warning is present, so local automation and CI can enforce a complete
+  runtime setup. JSON output now includes the applied `strict` policy and makes
+  `ok` reflect it.
+
+### Fixed
+
+- `doctor` now validates macOS `stat -f` output before falling back to GNU
+  `stat -c`, so owner-only job-store permissions are classified consistently on
+  Linux CI instead of becoming a false strict-mode warning.
+
 ## [0.13.0] - 2026-08-03
 
 ### Changed
@@ -563,7 +594,9 @@ work to the wrong model, and records the evidence behind the shipped defaults.
 - Initial shared routing table, cross-vendor dispatcher, runners, installer,
   and baseline lint fixes.
 
-[Unreleased]: https://github.com/Seraphim0916/omnilane/compare/v0.13.0...HEAD
+[Unreleased]: https://github.com/Seraphim0916/omnilane/compare/v0.14.0...HEAD
+
+[0.14.0]: https://github.com/Seraphim0916/omnilane/compare/v0.13.0...v0.14.0
 [0.13.0]: https://github.com/Seraphim0916/omnilane/compare/v0.12.0...v0.13.0
 [0.12.0]: https://github.com/Seraphim0916/omnilane/compare/v0.11.0...v0.12.0
 [0.11.0]: https://github.com/Seraphim0916/omnilane/compare/v0.10.4...v0.11.0

package/README.ja.md

@@ -516,6 +516,14 @@ scripts/dispatch.sh --dry-run hardest-coding "…"   # 解決済みプラン、
 
 ## 📜 リリース履歴
 
+## v0.14.0 の新機能
+
+- **根拠に基づくルーティング提案**：`jobs recommend` と MCP `jobs_recommend` は公開ジョブメタデータだけを読み、ルーティングを自動変更しません。
+- **任意の実動プローブ**：`doctor --probe V` と MCP `provider_probe` は明示された場合だけプロバイダーを呼び出し、応答本文を報告しません。
+- **Live Board の履歴検索・絞り込み・エクスポート**：直近 50 件を対象に、表示中の公開メタデータだけを出力します。
+- **固定品質／コストベンチマーク**：`omnilane benchmark` は既定で dry-run、実呼び出しには `--run` が必要です。
+- **厳格なインストール検証**：CI で隔離された `doctor --strict --json` を実行し、macOS／GNU `stat` の権限判定差も修正しました。
+
 ## v0.13.0 の新機能
 
 - **`long-context` を AA-LCR で順序付け** — Artificial Analysis の長文脈推論

package/README.ko.md

@@ -499,6 +499,14 @@ scripts/dispatch.sh --dry-run hardest-coding "…"   # 완전히 해석된 계
 
 ## 📜 릴리스 기록
 
+## v0.14.0 새 기능
+
+- **근거 기반 라우팅 제안**: `jobs recommend`와 MCP `jobs_recommend`는 공개 작업 메타데이터만 읽고 라우팅을 자동 변경하지 않습니다.
+- **선택형 실제 기능 탐지**: `doctor --probe V`와 MCP `provider_probe`는 명시적으로 요청할 때만 공급자를 호출하며 응답 본문을 보고하지 않습니다.
+- **Live Board 기록 검색·필터·내보내기**: 최근 50개 작업을 검색하고 현재 표시된 공개 메타데이터만 내보냅니다.
+- **고정 품질／비용 벤치마크**: `omnilane benchmark`는 기본적으로 dry-run이며 실제 호출에는 `--run`이 필요합니다.
+- **엄격한 설치 검증**: CI에서 격리된 `doctor --strict --json`을 실행하고 macOS／GNU `stat` 권한 판정 차이도 수정했습니다.
+
 ## v0.13.0 새 기능
 
 - **`long-context` 를 AA-LCR 기준으로 정렬** — Artificial Analysis 의 장문맥 추론

package/README.md

@@ -195,6 +195,10 @@ The reference is memory-only and disappears when the page closes. The board
 binds only to `127.0.0.1`, uses a random token, and is read-only. It shows
 `task.txt` and the public `out.txt`, but never raw worker or vendor logs.
 
+Search and state filters apply to the latest 50 retained jobs. **Export visible**
+downloads only the currently visible public metadata as local JSON; it excludes
+tokens, task text, result bodies, workdirs, and raw logs.
+
 The board reads in English, Japanese, Korean, Traditional Chinese and Simplified
 Chinese. It follows the browser language on first load; the switcher in the
 header overrides that and the choice is remembered locally.
@@ -255,7 +259,8 @@ adding a routing reminder. Configure the host to launch the installed CLI:
 
 The server exposes `route` plus read-only introspection: `list_lanes`,
 `explain`, `validate`, `dry_run`, `jobs_list`, `jobs_status`, `jobs_result`,
-`jobs_stats`, `jobs_audit`, and `doctor`. `route` defaults to read-only `advise` mode. Calls that select `work` must also
+`jobs_stats`, `jobs_recommend`, `jobs_audit`, `doctor`, and the explicitly opt-in
+`provider_probe`. `route` defaults to read-only `advise` mode. Calls that select `work` must also
 provide an explicit `workdir`.
 
 Node.js is the only runtime requirement (no npm packages). If you prefer
@@ -299,7 +304,8 @@ omnilane ui start                              # start/reuse the local Live UI;
 omnilane ui status                             # report whether the Live UI is running
 omnilane ui url                                # print the current authenticated local URL
 omnilane ui stop                               # stop the Live UI
-omnilane doctor [--json]                       # read-only routing and runtime health report
+omnilane doctor [--json] [--strict] [--probe V] [--probe-timeout SEC]  # live probe is opt-in
+omnilane benchmark [--json] [--run] [--vendor V] [--cost-per-call V=USD] # dry-run by default
 dispatch.sh [--background] [--dry-run] [--mode advise|work|sysops] [--workdir DIR]
             [--vendor V] [--model M] [--effort E] [--timeout SEC] [--job-timeout SEC]
             LANE "TASK"                              # "-" reads task from stdin
@@ -312,12 +318,24 @@ jobs.sh wait ID [--timeout N]                     # job exit; 124 timeout; 125 d
 jobs.sh cancel ID                                 # stop a running job: group SIGTERM, then SIGKILL
 jobs.sh rm ID                                     # delete one finished/dead job (refuses a running job)
 jobs.sh [--json] stats [--last N] [--lane L] [--vendor V]  # local success and routing aggregates
+jobs.sh [--json] recommend [--last N] [--lane L] [--min-samples N]  # evidence-gated vendor suggestion
 jobs.sh audit [--last N] [--json]                  # read-only job integrity/privacy check
 jobs.sh prune [--keep N] [--apply]                # preview by default; completed jobs only
 configure.sh                                        # interactive lane menu
 configure.sh set|get|unset|list|diff LANE [SPEC]    # script/inspect routing.local.yaml, no tty
 ```
 
+`jobs recommend` reads only validated public metadata and exit codes. It ranks
+eligible vendors by success rate, sample count, then name; the default minimum
+is three completed jobs. It never reads task/result bodies or changes routing.
+
+`doctor --probe V` makes one bounded advise-mode provider call and returns only
+availability, selected model, timing, and response byte count—not the response
+body. Without `--probe`, doctor remains offline. `benchmark` uses the fixed TSV
+suite in `benchmarks/workloads.tsv`; its default dry-run resolves every route
+without provider calls. `--run` is the explicit call gate, and cost totals are
+estimates based only on values supplied with `--cost-per-call`.
+
 **Big decisions can get a panel, not a person.** The `arbitrate` lane ships
 **disabled** — a panel costs one call per voter per round, so it is opt-in.
 Enable it with `arbitrate: vote codex,claude,grok -` in `routing.local.yaml`,
@@ -547,6 +565,24 @@ working notes, including per-benchmark caveats, live in
 
 ## 📜 Release history
 
+## What's new in v0.14.0
+
+- **Evidence-based routing recommendations** — `jobs recommend` and MCP
+  `jobs_recommend` rank vendors from completed public job metadata, enforce a
+  minimum sample gate, and never change routing automatically.
+- **Opt-in live capability probes** — `doctor --probe V` and MCP
+  `provider_probe` distinguish an installed CLI from a working provider call.
+  Default doctor remains offline; probe reports never include response bodies.
+- **Search, filter, and export Live Board history** — search and state filters
+  cover the latest 50 jobs, while **Export visible** downloads only filtered
+  public metadata.
+- **Repeatable quality/cost benchmark** — `omnilane benchmark` ships a fixed
+  workload suite, defaults to provider-free dry-run, and requires `--run` for
+  actual calls. Cost estimates use only explicit `--cost-per-call` values.
+- **Strict installation acceptance** — CI runs isolated
+  `omnilane doctor --strict --json`, catching incomplete runtime wiring without
+  contacting providers.
+
 ## What's new in v0.13.0
 
 - **`long-context` is ordered on AA-LCR** — Artificial Analysis's long-context

package/README.zh-CN.md

@@ -473,6 +473,14 @@ vendor 一律当成 `work`,而且它只能逐次明确指定,永远不是 lane
 
 ## 📜 版本历程
 
+## v0.14.0 新功能
+
+- **基于证据的路由建议**：`jobs recommend` 与 MCP `jobs_recommend` 只读取公开作业元数据，并且不会自动修改路由。
+- **可选的真实能力探测**：`doctor --probe V` 与 MCP `provider_probe` 仅在明确请求时调用供应商，报告不包含回答正文。
+- **Live Board 历史搜索、筛选与导出**：支持最近 50 条作业，并且只导出当前可见的公开元数据。
+- **固定质量／成本基准**：`omnilane benchmark` 默认只做 dry-run，`--run` 才实际调用供应商。
+- **严格安装验收**：CI 在隔离环境执行 `doctor --strict --json`，并修正 macOS／GNU `stat` 权限检测差异。
+
 ## v0.13.0 新功能
 
 - **`long-context` 改用 AA-LCR 排序**——那是 Artificial Analysis 的长脉络推理基准,

package/README.zh-TW.md

@@ -178,6 +178,9 @@ omnilane ui stop     # 正常停止
 `127.0.0.1`、用隨機 token 保護、全程唯讀。畫面只顯示 `task.txt` 與公開的
 `out.txt`，不顯示工作端或廠商原始 log。
 
+搜尋與狀態篩選會套用到最近保留的 50 筆工作歷史。按「匯出目前結果」只會把畫面上
+目前可見的公開中繼資料下載成 JSON；不含 token、任務本文、結果本文、工作目錄與原始 log。
+
 畫面提供英文、日文、韓文、繁體中文與簡體中文。首次載入依瀏覽器語言決定，可用標題列
 的切換器覆寫，選擇會記在本機。
 
@@ -228,7 +231,7 @@ omnilane ui stop     # 正常停止
 }
 ```
 
-Server 提供 `route`,以及一組唯讀查詢工具:`list_lanes`、`explain`、`validate`、`dry_run`、`jobs_list`、`jobs_status`、`jobs_result`、`jobs_stats`、`jobs_audit`、`doctor`。
+Server 提供 `route`,以及一組唯讀查詢工具:`list_lanes`、`explain`、`validate`、`dry_run`、`jobs_list`、`jobs_status`、`jobs_result`、`jobs_stats`、`jobs_recommend`、`jobs_audit`、`doctor`,以及必須明確選用的 `provider_probe`。
 `route` 預設唯讀 `advise` 模式;選 `work` 的呼叫必須同時提供明確的
 `workdir`。
 
@@ -266,7 +269,8 @@ omnilane ui start                              # 啟動或沿用本機 Live UI,
 omnilane ui status                             # 查看 Live UI 是否運作中
 omnilane ui url                                # 印出目前通過驗證的本機網址
 omnilane ui stop                               # 停止 Live UI
-omnilane doctor [--json]                       # 唯讀檢查路由與本機執行環境
+omnilane doctor [--json] [--strict] [--probe V] [--probe-timeout SEC]  # 實際探測必須明確選用
+omnilane benchmark [--json] [--run] [--vendor V] [--cost-per-call V=USD] # 預設只乾跑
 dispatch.sh [--background] [--dry-run] [--mode advise|work|sysops] [--workdir 目錄]
             [--vendor V] [--model M] [--effort E] [--timeout SEC] [--job-timeout SEC]
             通道 "任務"                              # "-" 表示從 stdin 讀任務
@@ -279,6 +283,7 @@ jobs.sh wait 工作ID [--timeout N]                  # 工作結束碼；124 逾
 jobs.sh cancel 工作ID                              # 停止執行中的工作:整組 SIGTERM,再 SIGKILL
 jobs.sh rm 工作ID                                  # 刪除單一已完成/已死工作(執行中會被拒絕)
 jobs.sh [--json] stats [--last N] [--lane L] [--vendor V]  # 本機成功率與路由彙整
+jobs.sh [--json] recommend [--last N] [--lane L] [--min-samples N]  # 有證據門檻的廠商建議
 jobs.sh audit [--last N] [--json]                  # 唯讀檢查工作完整性與隱私
 jobs.sh prune [--keep N] [--apply]                # 預設只預覽；只清理已完成工作
 omnilane mcp                                   # MCP stdio server(需 Node.js)
@@ -287,6 +292,15 @@ configure.sh                                        # 互動通道選單
 configure.sh set|get|unset|list|diff LANE [SPEC]    # 非互動編輯/檢視 routing.local.yaml
 ```
 
+`jobs recommend` 只讀取通過驗證的公開中繼資料與退出碼。候選達到最低樣本數後，
+依成功率、樣本數、廠商名稱排序；預設至少三筆已完成工作。它不讀任務／結果本文，
+也不修改路由。
+
+`doctor --probe V` 只會做一次有逾時上限的 advise 模式供應商呼叫，回傳可用性、
+選到的模型、耗時與回應位元組數，不回傳回答本文；未帶 `--probe` 時仍完全離線。
+`benchmark` 使用 `benchmarks/workloads.tsv` 的固定題組，預設只解析路由、不呼叫供應商；
+`--run` 才是實際呼叫閘門，成本總額也只依 `--cost-per-call` 明確提供的估值計算。
+
 **重大決定可以開評審團,不是問一個人。**`arbitrate` 通道**預設關閉**——
 評審團每評審每輪燒一次額度,所以做成選配。要用就在 `routing.local.yaml`
 寫 `arbitrate: vote codex,claude,grok -`,或跑設定選單,從
@@ -483,6 +497,19 @@ vendor 一律當成 `work`,而且它只能逐次明確指定,永遠不是 lane
 
 ## 📜 版本歷程
 
+## v0.14.0 新功能
+
+- **依證據產生路由建議**：`jobs recommend` 與 MCP `jobs_recommend` 只用已完成工作的
+  公開中繼資料排序供應商，設有最低樣本門檻，也不會自動修改路由。
+- **選配實際能力探測**：`doctor --probe V` 與 MCP `provider_probe` 可分辨「CLI 已安裝」
+  和「供應商呼叫真的可用」。doctor 預設仍離線，探測報告不含回答本文。
+- **Live Board 歷史搜尋、篩選與匯出**：搜尋與狀態篩選涵蓋最近 50 筆工作；
+  「匯出目前結果」只下載篩選後的公開中繼資料。
+- **可重現的品質／成本基準**：`omnilane benchmark` 內建固定題組，預設不呼叫供應商；
+  `--run` 才會實際執行，成本也只依明確提供的 `--cost-per-call` 估值計算。
+- **嚴格安裝驗收**：CI 以隔離環境執行 `omnilane doctor --strict --json`，
+  不接觸供應商也能抓出 runtime 接線不完整。
+
 ## v0.13.0 新功能
 
 - **`long-context` 改用 AA-LCR 排序**——那是 Artificial Analysis 的長脈絡推理基準,

package/VERSION

@@ -1 +1 @@
-0.13.0
+0.14.0

package/benchmarks/workloads.tsv

@@ -0,0 +1,4 @@
+# id<TAB>preferred_lane<TAB>weight<TAB>Python regex<TAB>prompt
+exact_alpha	consult	1	^ALPHA\s*$	Return exactly ALPHA and nothing else.
+arithmetic	consult	2	^42\s*$	Return only the integer result of 17 + 25.
+json_contract	hard-judgment	2	^\{"status":"pass","count":3\}\s*$	Return exactly this compact JSON object with the stated key order: {"status":"pass","count":3}

package/bin/omnilane

@@ -52,6 +52,13 @@ case "$cmd" in
     ;;
   release-audit) exec bash "$REPO/scripts/release-audit.sh" "$@" ;;
   doctor)    exec bash "$REPO/scripts/doctor.sh" "$@" ;;
+  benchmark)
+    command -v python3 >/dev/null 2>&1 || {
+      echo "omnilane: benchmark requires Python 3.9 or newer" >&2
+      exit 1
+    }
+    exec python3 "$REPO/scripts/benchmark.py" "$@"
+    ;;
   ui)
     command -v python3 >/dev/null 2>&1 || {
       echo "omnilane: ui requires Python 3.9 or newer" >&2
@@ -68,11 +75,12 @@ omnilane — one routing table, every harness
   omnilane list                         effective routing table
   omnilane route [--vendor V] [flags] LANE "TASK"
                                         dispatch or consult a model
-  omnilane jobs  [--json] COMMAND [ARGS]  list, status, result, stats, wait, or audit
+  omnilane jobs  [--json] COMMAND [ARGS]  list, status, result, stats, recommend, wait, or audit
   omnilane mcp                          start the MCP stdio server
   omnilane completion bash|zsh|fish      print a shell completion script
   omnilane release-audit [flags]          offline, read-only release gate
-  omnilane doctor [--json]              read-only local health report
+  omnilane doctor [--json] [--strict] [--probe V]  local health / opt-in live probe
+  omnilane benchmark [--json] [--run] [--vendor V] fixed quality/cost comparison
   omnilane ui    start|status|url|stop
   omnilane configure                    interactive lane menu
 EOF

package/bin/omnilane-mcp

@@ -10,6 +10,8 @@ const ROOT = path.resolve(__dirname, '..');
 const DISPATCH_SCRIPT = path.join(ROOT, 'scripts', 'dispatch.sh');
 const JOBS_SCRIPT = path.join(ROOT, 'scripts', 'jobs.sh');
 const DOCTOR_SCRIPT = path.join(ROOT, 'scripts', 'doctor.sh');
+const PROVIDER_PROBE_SCRIPT = process.env.OMNILANE_PROVIDER_PROBE_SCRIPT ||
+  path.join(ROOT, 'scripts', 'provider-probe.sh');
 const LATEST_PROTOCOL_VERSION = '2025-11-25';
 const SUPPORTED_PROTOCOL_VERSIONS = new Set([
   '2025-11-25',
@@ -285,6 +287,37 @@ const tools = [
       openWorldHint: false,
     },
   },
+  {
+    name: 'provider_probe',
+    description: 'Explicitly invoke one configured provider with a bounded fixed prompt to verify live inference. Incurs one provider call; never returns the provider response.',
+    inputSchema: {
+      type: 'object',
+      required: ['vendor'],
+      properties: {
+        vendor: {
+          type: 'string',
+          enum: ['codex', 'claude', 'grok', 'gemini', 'kimi', 'qwen', 'opencode', 'openrouter', 'deepseek', 'zai', 'mistral', 'groq', 'cerebras'],
+        },
+        timeout: {
+          type: 'integer',
+          minimum: 1,
+          maximum: 300,
+          default: 30,
+        },
+        json: {
+          type: 'boolean',
+          default: true,
+        },
+      },
+      additionalProperties: false,
+    },
+    annotations: {
+      readOnlyHint: true,
+      destructiveHint: false,
+      idempotentHint: false,
+      openWorldHint: true,
+    },
+  },
   {
     name: 'jobs_stats',
     description: 'Aggregate local background-job outcomes and lane/vendor distribution from bounded public metadata. Never reads task or result bodies.',
@@ -311,6 +344,45 @@ const tools = [
       openWorldHint: false,
     },
   },
+  {
+    name: 'jobs_recommend',
+    description: 'Recommend a vendor per lane from bounded local completed-job outcomes. Read-only, evidence-gated, and never reads task or result bodies.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        last: {
+          type: 'integer',
+          minimum: 1,
+          maximum: 10000,
+          description: 'Limit analysis to the most recent N jobs.',
+        },
+        lane: {
+          type: 'string',
+          pattern: '^[a-z][a-z0-9-]*$',
+          description: 'Optionally analyze one lane.',
+        },
+        min_samples: {
+          type: 'integer',
+          minimum: 1,
+          maximum: 1000,
+          default: 3,
+          description: 'Completed samples required before a vendor is eligible for recommendation.',
+        },
+        json: {
+          type: 'boolean',
+          default: false,
+          description: 'Return the versioned JSON envelope instead of the human report.',
+        },
+      },
+      additionalProperties: false,
+    },
+    annotations: {
+      readOnlyHint: true,
+      destructiveHint: false,
+      idempotentHint: true,
+      openWorldHint: false,
+    },
+  },
   {
     name: 'jobs_audit',
     description: 'Bounded read-only integrity and privacy scan of the local job store. Reports findings without printing task or result content.',
@@ -435,6 +507,24 @@ function validateJsonOnlyArguments(args) {
   return null;
 }
 
+function validateProviderProbeArguments(args) {
+  const error = validationError(args, ['vendor', 'timeout', 'json']);
+  if (error) return error;
+  const vendors = new Set([
+    'codex', 'claude', 'grok', 'gemini', 'kimi', 'qwen', 'opencode',
+    'openrouter', 'deepseek', 'zai', 'mistral', 'groq', 'cerebras',
+  ]);
+  if (typeof args.vendor !== 'string' || !vendors.has(args.vendor)) {
+    return 'vendor must name one probe-capable provider';
+  }
+  if (OWN(args, 'timeout') &&
+      (!Number.isInteger(args.timeout) || args.timeout < 1 || args.timeout > 300)) {
+    return 'timeout must be an integer from 1 to 300';
+  }
+  if (OWN(args, 'json') && typeof args.json !== 'boolean') return 'json must be a boolean';
+  return null;
+}
+
 function validateJobsQueryArguments(args) {
   const error = validationError(args, ['last', 'json']);
   if (error) return error;
@@ -445,6 +535,23 @@ function validateJobsQueryArguments(args) {
   return null;
 }
 
+function validateJobsRecommendArguments(args) {
+  const error = validationError(args, ['last', 'lane', 'min_samples', 'json']);
+  if (error) return error;
+  if (OWN(args, 'last') && (!Number.isInteger(args.last) || args.last < 1 || args.last > 10000)) {
+    return 'last must be an integer from 1 to 10000';
+  }
+  if (OWN(args, 'lane') && (typeof args.lane !== 'string' || !LANE_PATTERN.test(args.lane))) {
+    return 'lane must match ^[a-z][a-z0-9-]*$';
+  }
+  if (OWN(args, 'min_samples') &&
+      (!Number.isInteger(args.min_samples) || args.min_samples < 1 || args.min_samples > 1000)) {
+    return 'min_samples must be an integer from 1 to 1000';
+  }
+  if (OWN(args, 'json') && typeof args.json !== 'boolean') return 'json must be a boolean';
+  return null;
+}
+
 function validateExplainArguments(args) {
   const error = validationError(args, ['lane', 'json']);
   if (error) return error;
@@ -645,6 +752,14 @@ async function callTool(name, args) {
     const argv = args.json === true ? ['--json'] : [];
     return runToolScript('doctor', DOCTOR_SCRIPT, argv);
   }
+  if (name === 'provider_probe') {
+    const error = validateProviderProbeArguments(args);
+    if (error) return toolError(error);
+    const argv = ['--vendor', args.vendor];
+    if (OWN(args, 'timeout')) argv.push('--timeout', String(args.timeout));
+    if (args.json !== false) argv.push('--json');
+    return runToolScript('provider_probe', PROVIDER_PROBE_SCRIPT, argv);
+  }
   if (name === 'jobs_stats' || name === 'jobs_audit') {
     const error = validateJobsQueryArguments(args);
     if (error) return toolError(error);
@@ -654,6 +769,17 @@ async function callTool(name, args) {
     if (OWN(args, 'last')) argv.push('--last', String(args.last));
     return runToolScript(name, JOBS_SCRIPT, argv);
   }
+  if (name === 'jobs_recommend') {
+    const error = validateJobsRecommendArguments(args);
+    if (error) return toolError(error);
+    const argv = [];
+    if (args.json === true) argv.push('--json');
+    argv.push('recommend');
+    if (OWN(args, 'last')) argv.push('--last', String(args.last));
+    if (OWN(args, 'lane')) argv.push('--lane', args.lane);
+    if (OWN(args, 'min_samples')) argv.push('--min-samples', String(args.min_samples));
+    return runToolScript(name, JOBS_SCRIPT, argv);
+  }
   throw new RpcError(-32602, `Unknown tool: ${name}`);
 }
 

package/completions/_omnilane

@@ -53,7 +53,7 @@ _omnilane() {
   local command="${words[2]:-}" sub sub_index=3
   local -a lanes ids
   if (( CURRENT == 2 )); then
-    _values 'command' version list route dispatch jobs doctor release-audit ui configure completion help
+    _values 'command' version list route dispatch jobs doctor benchmark release-audit ui configure completion help
     return
   fi
   case "$command" in
@@ -78,7 +78,7 @@ _omnilane() {
         sub="${words[4]:-}"
       fi
       if (( CURRENT == sub_index )); then
-        _values 'job command' list status result tail retry stats wait audit prune help
+        _values 'job command' list status result tail retry stats recommend wait audit prune help
       elif (( CURRENT == sub_index + 1 )) &&
            [[ "$sub" == status || "$sub" == result || "$sub" == wait ||
               "$sub" == tail || "$sub" == retry ]]; then
@@ -92,7 +92,13 @@ _omnilane() {
         _arguments '--background[re-dispatch in the background]'
       elif [[ "$sub" == wait ]]; then
         _arguments '--timeout[maximum wait seconds]:seconds:'
-      elif [[ "$sub" == stats || "$sub" == audit ]]; then
+      elif [[ "$sub" == stats ]]; then
+        _arguments '--last[maximum jobs]:count:' '--lane[filter by lane]:lane:' \
+          '--vendor[filter by vendor]:vendor:' '--json[versioned metadata output]'
+      elif [[ "$sub" == recommend ]]; then
+        _arguments '--last[maximum jobs]:count:' '--lane[filter by lane]:lane:' \
+          '--min-samples[minimum completed samples]:count:' '--json[versioned metadata output]'
+      elif [[ "$sub" == audit ]]; then
         _arguments '--last[maximum jobs]:count:' '--json[versioned metadata output]'
       elif [[ "$sub" == prune ]]; then
         _arguments '--keep[completed jobs to retain]:count:' \
@@ -100,7 +106,18 @@ _omnilane() {
           '--apply[delete eligible jobs]'
       fi
       ;;
-    doctor) _values 'option' --json ;;
+    doctor)
+      _arguments '--json[versioned JSON output]' '--strict[fail on warnings]' \
+        '--probe[run one opt-in live provider probe]:vendor:' \
+        '--probe-timeout[probe timeout in seconds]:seconds:'
+      ;;
+    benchmark)
+      _arguments '--json[versioned JSON output]' '--run[invoke providers in advise mode]' \
+        '*--vendor[vendor to compare]:vendor:(codex claude grok gemini kimi qwen opencode openrouter deepseek zai mistral groq cerebras)' \
+        '--timeout[per-workload timeout]:seconds:' \
+        '--workloads[TSV workload file]:file:_files' \
+        '*--cost-per-call[user-supplied vendor USD estimate]:vendor=USD:'
+      ;;
     release-audit)
       _arguments '--target[release version]:version:' '--allow-dirty[permit a dirty worktree]' \
         '--require-tag[require an annotated tag at HEAD]' '--manifest[include tracked inventory]' \

package/completions/omnilane.bash

@@ -58,7 +58,7 @@ _omnilane() {
   prev="${COMP_WORDS[COMP_CWORD-1]:-}"
   command="${COMP_WORDS[1]:-}"
   if [[ "$COMP_CWORD" -eq 1 ]]; then
-    words="version list route dispatch jobs doctor release-audit ui configure completion help"
+      words="version list route dispatch jobs doctor benchmark release-audit ui configure completion help"
   else
     case "$command" in
       route|dispatch)
@@ -84,7 +84,7 @@ _omnilane() {
           sub="${COMP_WORDS[3]:-}"
         fi
         if [[ "$COMP_CWORD" -eq "$sub_index" ]]; then
-          words="list status result tail retry stats wait audit prune help"
+          words="list status result tail retry stats recommend wait audit prune help"
         elif [[ "$COMP_CWORD" -eq $((sub_index + 1)) &&
                 ( "$sub" == status || "$sub" == result || "$sub" == wait ||
                   "$sub" == tail || "$sub" == retry ) ]]; then
@@ -99,7 +99,11 @@ _omnilane() {
           words="--background"
         elif [[ "$sub" == wait ]]; then
           words="--timeout"
-        elif [[ "$sub" == stats || "$sub" == audit ]]; then
+        elif [[ "$sub" == stats ]]; then
+          words="--last --lane --vendor --json"
+        elif [[ "$sub" == recommend ]]; then
+          words="--last --lane --min-samples --json"
+        elif [[ "$sub" == audit ]]; then
           words="--last --json"
         elif [[ "$sub" == prune ]]; then
           words="--keep --older-than --apply"
@@ -107,7 +111,18 @@ _omnilane() {
           return
         fi
         ;;
-      doctor) words="--json" ;;
+      doctor) words="--json --strict --probe --probe-timeout" ;;
+      benchmark)
+        case "$prev" in
+          --vendor) words="codex claude grok gemini kimi qwen opencode openrouter deepseek zai mistral groq cerebras" ;;
+          --workloads)
+            COMPREPLY=( $(compgen -f -- "$cur") )
+            return
+            ;;
+          --timeout|--cost-per-call) return ;;
+          *) words="--json --run --vendor --timeout --workloads --cost-per-call" ;;
+        esac
+        ;;
       release-audit) words="--target --allow-dirty --require-tag --manifest --json" ;;
       ui) words="start status url stop" ;;
       completion) words="bash zsh" ;;

package/completions/omnilane.fish

@@ -10,6 +10,7 @@ complete -c omnilane -f -n __fish_use_subcommand -a dispatch      -d 'dispatch o
 complete -c omnilane -f -n __fish_use_subcommand -a jobs          -d 'inspect background jobs'
 complete -c omnilane -f -n __fish_use_subcommand -a mcp           -d 'MCP stdio server'
 complete -c omnilane -f -n __fish_use_subcommand -a doctor        -d 'read-only health report'
+complete -c omnilane -f -n __fish_use_subcommand -a benchmark     -d 'fixed quality/cost comparison'
 complete -c omnilane -f -n __fish_use_subcommand -a release-audit -d 'offline release gate'
 complete -c omnilane -f -n __fish_use_subcommand -a ui            -d 'Live Board server'
 complete -c omnilane -f -n __fish_use_subcommand -a configure     -d 'lane routing (menu or set/get/unset/list/diff)'
@@ -29,19 +30,30 @@ complete -c omnilane -n '__fish_seen_subcommand_from route dispatch' -l dry-run
 
 # jobs subcommands (only before a jobs subcommand is chosen).
 complete -c omnilane -f \
-  -n '__fish_seen_subcommand_from jobs; and not __fish_seen_subcommand_from list status result tail retry stats wait audit prune cancel rm help' \
-  -a 'list status result tail retry stats wait audit prune cancel rm help' -d 'jobs subcommand'
+  -n '__fish_seen_subcommand_from jobs; and not __fish_seen_subcommand_from list status result tail retry stats recommend wait audit prune cancel rm help' \
+  -a 'list status result tail retry stats recommend wait audit prune cancel rm help' -d 'jobs subcommand'
 complete -c omnilane -n '__fish_seen_subcommand_from jobs' -l json -d 'JSON output'
 # jobs list / stats filters.
-complete -c omnilane -n '__fish_seen_subcommand_from jobs; and __fish_seen_subcommand_from list stats' -l lane   -x                                                                                    -d 'filter by lane'
+complete -c omnilane -n '__fish_seen_subcommand_from jobs; and __fish_seen_subcommand_from list stats recommend' -l lane -x -d 'filter by lane'
 complete -c omnilane -n '__fish_seen_subcommand_from jobs; and __fish_seen_subcommand_from list stats' -l vendor -x -a 'codex claude grok gemini kimi qwen opencode openrouter deepseek zai mistral groq cerebras exec' -d 'filter by vendor'
 complete -c omnilane -n '__fish_seen_subcommand_from jobs; and __fish_seen_subcommand_from list'       -l status -x -a 'running done'                                                                    -d 'filter by status'
+complete -c omnilane -n '__fish_seen_subcommand_from jobs; and __fish_seen_subcommand_from stats recommend audit' -l last -x -d 'maximum recent jobs'
+complete -c omnilane -n '__fish_seen_subcommand_from jobs; and __fish_seen_subcommand_from recommend' -l min-samples -x -d 'minimum completed samples'
 
 # configure subcommands (non-interactive).
 complete -c omnilane -f -n '__fish_seen_subcommand_from configure; and not __fish_seen_subcommand_from set get unset list diff' -a 'set get unset list diff' -d 'configure action'
 
 # doctor / release-audit / ui / completion.
 complete -c omnilane -n '__fish_seen_subcommand_from doctor' -l json -d 'JSON output'
+complete -c omnilane -n '__fish_seen_subcommand_from doctor' -l strict -d 'fail when any warning is reported'
+complete -c omnilane -n '__fish_seen_subcommand_from doctor' -l probe -x -a 'codex claude grok gemini kimi qwen opencode openrouter deepseek zai mistral groq cerebras' -d 'run one opt-in live probe'
+complete -c omnilane -n '__fish_seen_subcommand_from doctor' -l probe-timeout -x -d 'probe timeout in seconds'
+complete -c omnilane -n '__fish_seen_subcommand_from benchmark' -l json -d 'JSON output'
+complete -c omnilane -n '__fish_seen_subcommand_from benchmark' -l run -d 'invoke providers in advise mode'
+complete -c omnilane -n '__fish_seen_subcommand_from benchmark' -l vendor -x -a 'codex claude grok gemini kimi qwen opencode openrouter deepseek zai mistral groq cerebras' -d 'vendor to compare; repeatable'
+complete -c omnilane -n '__fish_seen_subcommand_from benchmark' -l timeout -x -d 'per-workload timeout'
+complete -c omnilane -n '__fish_seen_subcommand_from benchmark' -l workloads -r -d 'TSV workload file'
+complete -c omnilane -n '__fish_seen_subcommand_from benchmark' -l cost-per-call -x -d 'user-supplied VENDOR=USD estimate'
 complete -c omnilane -n '__fish_seen_subcommand_from release-audit' -l target -x       -d 'target version'
 complete -c omnilane -n '__fish_seen_subcommand_from release-audit' -l allow-dirty     -d 'permit a dirty tree'
 complete -c omnilane -n '__fish_seen_subcommand_from release-audit' -l require-tag     -d 'require an annotated tag'

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "omnilane",
-  "version": "0.13.0",
+  "version": "0.14.0",
   "description": "One routing table, every harness — classify subtasks into lanes and dispatch each lane to the best vendor's agentic CLI (Codex, Claude, Gemini, Grok) using your existing subscription logins.",
   "bin": {
     "omnilane": "bin/omnilane"
@@ -11,6 +11,7 @@
     "!scripts/__pycache__",
     "!**/*.pyc",
     "completions/",
+    "benchmarks/",
     "ui/",
     "routing.yaml",
     "routing.local.yaml.example",