oh-my-knowledge 1.0.0-beta.2 → 1.0.0-beta.4

package/README.md

@@ -55,7 +55,7 @@ export OMK_EXECUTOR=codex
 
 Without `OMK_MODEL`, omk reads the model from `~/.codex/config.toml`. You can still pass `--executor codex --model <codex-model>` per command. Pass `--judge-models` or set `OMK_JUDGE_MODELS` only when you want a different judge.
 
-> The default 3-case pack is a low-cost workflow check, so `UNDERPOWERED` is expected. `--samples 20` selects a first-party, difficulty-stratified starter pack that meets omk's registered sample-size floor. Its provenance is `llm-generated`: use it to learn the statistical workflow, then review and replace it with real domain cases before making a release decision.
+> The default 3-case pack is a low-cost workflow check, so `UNDERPOWERED` is expected. `--samples 20` selects a first-party, difficulty-stratified starter pack that meets omk's default heuristic evidence floor; it is not an a priori power calculation. Its provenance is `llm-generated`: use it to learn the statistical workflow, then review and replace it with real domain cases before making a release decision.
 
 > The CLI notifies you when a newer version is available (at most once per 20h); set `OMK_SKIP_UPDATE_CHECK=1` to silence it permanently.
 
@@ -228,7 +228,7 @@ The full docs are published at **[oh-my-knowledge.pages.dev](https://oh-my-knowl
 - **[CLI reference](docs/reference/cli.md)** — all top-level commands with bash examples and flag tables
 - **[Migrate to the 1.0 preview](docs/guides/v1-preview-migration.md)** — install channel, storage reset, sample protocol, CLI automation, and embedded API changes since 0.54
 - **[Evaluation Core cutover](docs/guides/eval-core-cutover.md)** — `BREAKING-SCHEMA` storage, resume, Studio, Gold, managed-evidence, and evolve migration
-- **[Embed OMK in a service](docs/guides/eval-runtime.md)** — lightweight `eval-runtime` assembly for Node.js and FaaS hosts
+- **[Embed OMK in a service](docs/guides/eval-runtime.md)** — canonical `evaluate()` plus repeat-run `evaluateSeries()` for Node.js and FaaS hosts
 - **[Storage layout v2](docs/specs/storage-layout-spec.md)** — project/global domains, compatibility boundary, and Git policy
 - **[Executors](docs/reference/executors.md)** & **[artifact layout](docs/reference/artifact-layout.md)** — built-in / custom executors; how `variant` resolves to an artifact + runtime context
 - **[How-to guides](docs/guides/agent-eval.md)** — [evaluate an agent](docs/guides/agent-eval.md) (project runtime context) and [use non-Claude models](docs/guides/non-claude-models.md) (GLM / Qwen / DeepSeek / Moonshot / Ollama)

package/README.zh.md

@@ -55,7 +55,7 @@ export OMK_EXECUTOR=codex
 
 不设置 `OMK_MODEL` 时，omk 会读取 `~/.codex/config.toml` 的模型。也可以继续逐次显式传 `--executor codex --model <codex-model>`。自定义评委时再传 `--judge-models` 或设置 `OMK_JUDGE_MODELS`。
 
-> 默认 3 条用例是低成本流程检查，出现「数据不足（UNDERPOWERED）」符合预期。`--samples 20` 会选择经过难度分层的官方起步用例集，达到 omk 注册的样本量下限。其来源明确标记为 `llm-generated`：它适合学习统计流程，发布判断前仍应人工复核并替换为真实领域用例。
+> 默认 3 条用例是低成本流程检查，出现「数据不足（UNDERPOWERED）」符合预期。`--samples 20` 会选择经过难度分层的官方起步用例集，达到 omk 默认的启发式证据下限，但这不是先验功效计算。其来源明确标记为 `llm-generated`：它适合学习统计流程，发布判断前仍应人工复核并替换为真实领域用例。
 
 > 命令行有新版本时会自动提示（每 20 小时最多一次）；想永久关闭该提醒，设环境变量 `OMK_SKIP_UPDATE_CHECK=1` 即可。
 
@@ -157,7 +157,7 @@ RAG 专项评测请看 RAGAS（独立 niche，跟 omk 互补）。完整对比
 
 | 特性 | 说明 |
 |------|------|
-| **Core 发布决定** | 六种结论 + 稳定 reason code + exit code 路由；Studio 投影同一份经过认证的 Decision |
+| **Core 发布决定** | 六种判定 + 稳定 reason code + exit code 路由；Studio 投影同一份经过认证的 Decision |
 | **五层 evidence graph** | Assertion / LLM / Judge / Dimension / Composite 保持独立，coverage、成本、状态与 lineage 与分数正交 |
 | **多执行器** | 支持 Claude CLI / Claude SDK / Codex CLI / Codex SDK / DeepSeek Harness / OpenAI / Anthropic API 及自定义命令 |
 | **30+ 种断言** | 包含子串、正则、JSON Schema、ROUGE/BLEU/Levenshtein 相似度、Agent 工具调用、语义相似度、自定义函数等 |
@@ -228,7 +228,7 @@ omk-mcp
 - **[CLI 参考](docs/zh/reference/cli.md)** —— 顶层命令的 bash 示例和 flag 表
 - **[迁移到 1.0 预览版](docs/zh/guides/v1-preview-migration.md)** —— 从 `0.54` 升级时的安装渠道、存储重建、用例协议、CLI 自动化与嵌入式 API 变化
 - **[Evaluation Core 生产切换](docs/zh/guides/eval-core-cutover.md)** —— `BREAKING-SCHEMA` 存储、resume、Studio、Gold、受管证据与 evolve 迁移
-- **[在服务中嵌入 OMK](docs/zh/guides/eval-runtime.md)** —— 面向 Node.js 与 FaaS 宿主的轻量 `eval-runtime` 装配
+- **[在服务中嵌入 OMK](docs/zh/guides/eval-runtime.md)** —— 面向 Node.js 与 FaaS 宿主的 canonical `evaluate()` 与重复运行 `evaluateSeries()`
 - **[存储布局 v2](docs/zh/specs/storage-layout-spec.md)** —— 项目／全局领域、迁移兼容与 Git 策略
 - **[执行器](docs/zh/reference/executors.md)** & **[知识载体布局](docs/zh/reference/artifact-layout.md)** —— 内置 / 自定义执行器；variant 如何解析为 artifact + runtime context
 - **[操作指南](docs/zh/guides/agent-eval.md)** —— [评测 agent](docs/zh/guides/agent-eval.md)（项目 runtime context）与[使用非 Claude 模型](docs/zh/guides/non-claude-models.md)（GLM / 通义 / DeepSeek / Moonshot / Ollama）

package/dist/assets/agent-skills/omk/references/commands.md

@@ -63,7 +63,7 @@ omk doctor --json --gate
 
 ## omk eval
 
-跑评测：对一个 control vs 多个 treatment skill 做对照试验，产 verdict 报告。
+跑评测：用一个对照组与多个实验组 skill 做对照试验，生成判定报告。
 
 **用法:**
 
@@ -73,16 +73,16 @@ omk eval [flags]
 
 **Flags:**
 
-- `--batch` `boolean`:batch 模式:baseline vs 每个 skill
+- `--batch` `boolean`:batch 模式：baseline 作为对照组，逐个 skill 作为实验组
 - `--bootstrap` `boolean`:加 bootstrap CI
 - `--bootstrap-samples` `option`:bootstrap 重采样次数，默认 1000
-- `--budget-per-sample-ms` `option`:单 sample 时长上限 ms（必须 > 0，不传则无上限）
-- `--budget-per-sample-usd` `option`:单 sample 预算上限 USD（必须 > 0，不传则无上限）
+- `--budget-per-sample-ms` `option`:单用例时长上限 ms（必须 > 0，不传则无上限）
+- `--budget-per-sample-usd` `option`:单用例预算上限 USD（必须 > 0，不传则无上限）
 - `--budget-usd` `option`:总预算上限 USD（必须 > 0，不传则无上限）
 - `--concurrency` `option`:并发数，默认 1
 - `--config` `option`:eval.yaml 路径
-- `--control` `option`:control variant 表达式（仅 artifact 身份）
-- `--control-cwd` `option`:control 的 runtime context 目录
+- `--control` `option`:对照组（control）的 variant 表达式（仅 artifact 身份）
+- `--control-cwd` `option`:对照组（control）的 runtime context 目录
 - `--dry-run` `boolean`:只 plan 不实跑
 - `--effort` `option`:被测 LLM 扩展思考预算 low/medium/high/xhigh/max（默认 low；跨 effort 报告不严格可比）。
 - `--executor` `option`:执行器：claude / claude-sdk / codex / codex-sdk / anthropic-api / openai-api / 自定义命令。Codex 任务内自动用 codex；也可用 OMK_EXECUTOR 设置环境偏好。
@@ -90,7 +90,7 @@ omk eval [flags]
 - `--gold-dir` `option`:gold dataset 目录
 - `--holdout-ratio` `option`:留出比例 0-1（如 0.3）；切出 holdout 子集，对比 train/holdout 综合分检测过拟合
 - `--judge-models` `option`:评委配置，格式 executor:model[,...]，例 claude:haiku 或 codex:<model>（≥ 2 个 = ensemble）。默认跟随所选执行器；Codex 沿用被测模型。
-- `--judge-repeat` `option`:每个 dim 评 N 次
+- `--judge-repeat` `option`:每个维度由评委评价 N 次
 - `--lang` `option` (默认 `zh`):输出语言 zh|en，优先级 CLI > OMK_LANG env > zh。
 - `--layered-stats` `boolean`:输出分层统计
 - `--mcp-config` `option`:MCP 配置文件路径
@@ -99,30 +99,30 @@ omk eval [flags]
 - `--no-debias-length` `boolean`:关 length-debias（默认开）
 - `--no-diagnostic` `boolean`:关闭基于 Core 失败、缺失、排除与稳定 reason code 的诊断投影。
 - `--no-evidence` `boolean`:不把本次评测写成证据追加进受管记录(默认会为已 install 的 skill 自动写)。
-- `--no-gate` `boolean`:关 verdict gate
-- `--no-judge` `boolean`:跳过 LLM judge
+- `--no-gate` `boolean`:关闭判定门禁
+- `--no-judge` `boolean`:跳过 LLM 评委
 - `--no-serve` `boolean`:不启 report server
 - `--no-strict-baseline` `boolean`:关闭 baseline 隔离
 - `--output-dir` `option`:报告输出目录（默认项目级 .omk/eval）
-- `--repeat` `option`:每个 sample 重复跑 N 次
-- `--report-only` `boolean`:生成报告并打印 verdict，但始终 exit 0(不参与 CI gate）。
+- `--repeat` `option`:预先固定 Evaluation Series 的独立 run 数
+- `--report-only` `boolean`:生成报告并打印判定，但始终 exit 0(不参与 CI gate）。
 - `--resume` `option`:复用经过完整契约校验的 Core runId；拒绝时失败关闭
-- `--retry` `option`:失败 sample 重试次数
-- `--samples` `option`:用例路径。自动发现项目级或单 treatment 目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。
+- `--retry` `option`:单用例失败重试次数
+- `--samples` `option`:用例路径。自动发现项目级或单个实验组（treatment）目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。
 - `--skill-dir` `option`:skill 目录，默认 skills
 - `--skip-connectivity` `boolean`:跳 LLM 连通性预检
 - `--skip-doctor` `boolean`:escape hatch:跳 doctor 健康检查门禁（默认强制启用）。沙箱 mock 提供依赖时绕开 doctor 物理路径误报；garbage-in 风险自负。
 - `--strict-baseline` `boolean`:强制 baseline 隔离（default true）
-- `--threshold` `option`:verdict 阈值，默认 3.5
+- `--threshold` `option`:判定阈值，默认 3.5
 - `--timeout` `option`:单用例超时秒，默认 120
-- `--treatment` `option`:treatment variant 列表，逗号分隔（仅 artifact 身份）
-- `--treatment-cwd` `option`:treatment 的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）
+- `--treatment` `option`:实验组（treatment）的 variant 列表，逗号分隔（仅 artifact 身份）
+- `--treatment-cwd` `option`:实验组（treatment）的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）
 - `--trivial-diff` `option`:可忽略 diff 容差，0 表示不启用容差
 - `--verbose` `boolean`:详细日志
 
 **示例:**
 
-> 最简对照:baseline vs my-skill
+> 最简对照：baseline 作为对照组，my-skill 作为实验组
 
 ```bash
 omk eval --control baseline --treatment my-skill
@@ -169,6 +169,7 @@ omk eval gold compare <runId> [flags]
 - `--gold-dir` `option`:gold dataset 目录，必填
 - `--lang` `option` (默认 `zh`):输出语言 zh|en，优先级 CLI > OMK_LANG env > zh。
 - `--metric` `option`:显式选择 Core metric ID。
+- `--minimum-alpha` `option`:可选的一致性阈值；按 Krippendorff α 置信区间下界评估
 - `--reports-dir` `option`:只读取指定 Core 报告目录（默认当前项目 + 全局）
 - `--seed` `option`:bootstrap seed，可复现
 - `--target` `option`:显式选择 Core target ID。
@@ -274,7 +275,7 @@ omk init [targetDir] [flags]
 
 - `--force` `boolean`:允许覆盖目标目录中已有的 omk 脚手架文件
 - `--lang` `option` (默认 `zh`):输出语言 zh|en，优先级 CLI > OMK_LANG env > zh。
-- `--samples` `3|20` (默认 `3`):官方起步用例数量：3 条用于快速跑通，20 条用于达到注册样本量下限
+- `--samples` `3|20` (默认 `3`):官方起步用例数量：3 条用于快速跑通，20 条用于达到默认启发式证据下限
 
 **示例:**
 
@@ -290,7 +291,7 @@ omk init
 omk init my-project
 ```
 
-> 使用达到注册样本量下限的 20 条官方用例初始化
+> 使用达到默认启发式证据下限的 20 条官方用例初始化
 
 ```bash
 omk init my-project --samples 20

package/dist/cli/commands/eval/gold/compare.d.ts

@@ -10,6 +10,7 @@ export default class EvalGoldCompare extends BaseCommand {
         target: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         evaluator: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         metric: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
+        'minimum-alpha': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         'trial-index': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         'reports-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         'bootstrap-samples': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;

package/dist/cli/commands/eval/gold/compare.js

@@ -2,7 +2,7 @@ import { resolve } from 'node:path';
 import { Args, Flags } from '@oclif/core';
 import { BaseCommand } from '../../../oclif/base-command.js';
 import { LANG_FLAG, bilingual } from '../../../oclif/i18n.js';
-import { integerStringParser } from '../../../oclif/parsers.js';
+import { integerStringParser, numberStringParser } from '../../../oclif/parsers.js';
 import { CliExit } from '../../../lib/cli-exit.js';
 import { projectReportsDir, globalReportsDir } from '../../../../evidence/storage/directories.js';
 export default class EvalGoldCompare extends BaseCommand {
@@ -33,6 +33,13 @@ export default class EvalGoldCompare extends BaseCommand {
         metric: Flags.string({
             description: bilingual({ zh: '显式选择 Core metric ID。', en: 'Explicit Core metric ID.' }),
         }),
+        'minimum-alpha': Flags.string({
+            description: bilingual({
+                zh: '可选的一致性阈值；按 Krippendorff α 置信区间下界评估',
+                en: 'Optional agreement threshold, assessed against the Krippendorff alpha CI lower bound',
+            }),
+            parse: numberStringParser('--minimum-alpha', { min: -1, max: 1 }),
+        }),
         'trial-index': Flags.string({
             description: bilingual({ zh: '显式选择 trial index。', en: 'Explicit trial index.' }),
             parse: integerStringParser('--trial-index', { min: 0 }),
@@ -61,26 +68,31 @@ export default class EvalGoldCompare extends BaseCommand {
         await this.runWithCliExit(async () => {
             const runId = args.runId;
             if (!runId) {
-                console.error('Usage: omk eval gold compare <runId> --gold-dir <dir> --target <id> --evaluator <id> --metric <id>');
+                console.error(lang === 'zh'
+                    ? '用法：omk eval gold compare <runId> --gold-dir <dir> --target <id> --evaluator <id> --metric <id>'
+                    : 'Usage: omk eval gold compare <runId> --gold-dir <dir> --target <id> --evaluator <id> --metric <id>');
                 throw new CliExit(1);
             }
             const goldDir = flags['gold-dir'];
             if (!goldDir) {
-                console.error('--gold-dir is required');
+                console.error(lang === 'zh' ? '必须提供 --gold-dir。' : '--gold-dir is required.');
                 throw new CliExit(1);
             }
-            const { loadGoldDataset } = await import('../../../../eval-workflows/gold/dataset.js');
+            const { loadGoldDataset, validationIssueMessage } = await import('../../../../eval-workflows/gold/dataset.js');
             const { createNodeCoreContentStore, createNodeCoreRunArtifactStore, createOverlayCoreRunArtifactStore, } = await import('../../../../eval-workflows/artifact-store/index.js');
             const { compareGoldToCoreRun } = await import('../../../../eval-workflows/projections/gold.js');
             const { dataset, issues } = loadGoldDataset(goldDir);
             if (!dataset) {
-                console.error('Cannot load gold dataset:');
+                console.error(lang === 'zh' ? '无法加载 gold dataset：' : 'Cannot load gold dataset:');
                 for (const i of issues)
-                    console.error(`  - ${i.message}`);
+                    console.error(`  - ${validationIssueMessage(i, lang)}`);
                 throw new CliExit(1);
             }
-            for (const i of issues)
-                console.error(`warn: ${i.message}`);
+            for (const i of issues) {
+                console.error(lang === 'zh'
+                    ? `警告：${validationIssueMessage(i, lang)}`
+                    : `Warning: ${validationIssueMessage(i, lang)}`);
+            }
             const storeOf = (directory) => createNodeCoreRunArtifactStore(directory, {
                 contentResolver: createNodeCoreContentStore(resolve(directory, 'content')),
             });
@@ -90,8 +102,8 @@ export default class EvalGoldCompare extends BaseCommand {
             const source = await store.get(runId);
             if (source === undefined) {
                 console.error(lang === 'zh'
-                    ? `找不到 Core run「${runId}」；旧 evaluation report 不再支持 gold compare。`
-                    : `Core run "${runId}" was not found; legacy evaluation reports are no longer supported.`);
+                    ? `找不到 Core run「${runId}」。`
+                    : `Core run "${runId}" was not found.`);
                 throw new CliExit(1);
             }
             if (!flags.target || !flags.evaluator || !flags.metric) {
@@ -114,6 +126,9 @@ export default class EvalGoldCompare extends BaseCommand {
                     }),
                 },
                 bootstrapSamples: samples,
+                ...(flags['minimum-alpha'] === undefined ? {} : {
+                    minimumAlpha: Number(flags['minimum-alpha']),
+                }),
                 ...(Number.isFinite(seedVal) ? { bootstrapSeed: seedVal } : {}),
             });
             console.log(JSON.stringify(result, null, 2));

package/dist/cli/commands/eval/gold/validate.js

@@ -25,18 +25,22 @@ export default class EvalGoldValidate extends BaseCommand {
         await this.runWithCliExit(async () => {
             const dir = args.dir;
             if (!dir) {
-                console.error('Usage: omk eval gold validate <dir>');
+                console.error(lang === 'zh'
+                    ? '用法：omk eval gold validate <dir>'
+                    : 'Usage: omk eval gold validate <dir>');
                 throw new CliExit(1);
             }
             const { validateGoldDataset } = await import('../../../../eval-workflows/gold/cli.js');
-            const result = validateGoldDataset(dir);
+            const result = validateGoldDataset(dir, lang);
             if (result.ok) {
                 console.log(lang === 'zh'
                     ? `✓ gold dataset OK，共 ${result.sampleCount} 条标注`
                     : `✓ gold dataset OK — ${result.sampleCount} annotations`);
                 return;
             }
-            console.error(`✗ gold dataset has ${result.issues.length} issue(s):`);
+            console.error(lang === 'zh'
+                ? `✗ gold dataset 存在 ${result.issues.length} 个问题：`
+                : `✗ gold dataset has ${result.issues.length} issue(s):`);
             for (const msg of result.issues)
                 console.error(`  - ${msg}`);
             throw new CliExit(1);

package/dist/cli/commands/eval/index.js

@@ -174,13 +174,13 @@ async function runEval(_args, flags, lang) {
 }
 export default class Eval extends BaseCommand {
     static description = bilingual({
-        zh: '跑评测：对一个 control vs 多个 treatment skill 做对照试验，产 verdict 报告。',
+        zh: '跑评测：用一个对照组与多个实验组 skill 做对照试验，生成判定报告。',
         en: 'Run evaluation: control vs treatment(s) comparison, produce verdict report.',
     });
     static examples = [
         {
             description: bilingual({
-                zh: '最简对照:baseline vs my-skill',
+                zh: '最简对照：baseline 作为对照组，my-skill 作为实验组',
                 en: 'Minimal A/B: baseline vs my-skill',
             }),
             command: '<%= config.bin %> eval --control baseline --treatment my-skill',
@@ -197,23 +197,23 @@ export default class Eval extends BaseCommand {
         lang: LANG_FLAG,
         // ── 实验角色 ──
         control: Flags.string({
-            description: bilingual({ zh: 'control variant 表达式（仅 artifact 身份）', en: 'Control variant expr (artifact identity only)' }),
+            description: bilingual({ zh: '对照组（control）的 variant 表达式（仅 artifact 身份）', en: 'Control variant expr (artifact identity only)' }),
         }),
         treatment: Flags.string({
             description: bilingual({
-                zh: 'treatment variant 列表，逗号分隔（仅 artifact 身份）',
+                zh: '实验组（treatment）的 variant 列表，逗号分隔（仅 artifact 身份）',
                 en: 'Treatment variants, comma-separated (artifact identity only)',
             }),
         }),
         'control-cwd': Flags.string({
             description: bilingual({
-                zh: 'control 的 runtime context 目录',
+                zh: '对照组（control）的 runtime context 目录',
                 en: 'Runtime context dir for control',
             }),
         }),
         'treatment-cwd': Flags.string({
             description: bilingual({
-                zh: 'treatment 的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）',
+                zh: '实验组（treatment）的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）',
                 en: 'Runtime context dirs for treatments, comma-separated, index-aligned with --treatment (blank = none)',
             }),
         }),
@@ -222,7 +222,7 @@ export default class Eval extends BaseCommand {
         }),
         samples: Flags.string({
             description: bilingual({
-                zh: '用例路径。自动发现项目级或单 treatment 目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。',
+                zh: '用例路径。自动发现项目级或单个实验组（treatment）目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。',
                 en: 'Samples path. Auto-discovers eval-samples.json / eval-samples.yaml at project scope or for a single directory-skill treatment; an explicit path may be a JSON / YAML file or split directory.',
             }),
         }),
@@ -256,7 +256,7 @@ export default class Eval extends BaseCommand {
         }),
         // ── 评测 toggle ──
         'no-judge': Flags.boolean({
-            description: bilingual({ zh: '跳过 LLM judge', en: 'Skip LLM judge' }),
+            description: bilingual({ zh: '跳过 LLM 评委', en: 'Skip LLM judge' }),
         }),
         'no-cache': Flags.boolean({
             description: bilingual({ zh: '跳过 executor cache', en: 'Skip executor cache' }),
@@ -277,7 +277,7 @@ export default class Eval extends BaseCommand {
         }),
         batch: Flags.boolean({
             description: bilingual({
-                zh: 'batch 模式:baseline vs 每个 skill',
+                zh: 'batch 模式：baseline 作为对照组，逐个 skill 作为实验组',
                 en: 'Batch mode: baseline vs each skill',
             }),
         }),
@@ -300,7 +300,7 @@ export default class Eval extends BaseCommand {
             description: bilingual({ zh: '详细日志', en: 'Verbose logging' }),
         }),
         retry: Flags.string({
-            description: bilingual({ zh: '失败 sample 重试次数', en: 'Per-sample retry count' }),
+            description: bilingual({ zh: '单用例失败重试次数', en: 'Per-sample retry count' }),
             parse: integerStringParser('--retry', { min: 0 }),
         }),
         resume: Flags.string({
@@ -333,7 +333,10 @@ export default class Eval extends BaseCommand {
         }),
         // ── eval-runner extra ──
         repeat: Flags.string({
-            description: bilingual({ zh: '每个 sample 重复跑 N 次', en: 'Repeat each sample N times' }),
+            description: bilingual({
+                zh: '预先固定 Evaluation Series 的独立 run 数',
+                en: 'Predeclare the independent run count for the Evaluation Series',
+            }),
             parse: integerStringParser('--repeat', { min: 1 }),
         }),
         'holdout-ratio': Flags.string({
@@ -344,7 +347,7 @@ export default class Eval extends BaseCommand {
             parse: numberStringParser('--holdout-ratio', { min: 0, max: 1 }),
         }),
         'judge-repeat': Flags.string({
-            description: bilingual({ zh: '每个 dim 评 N 次', en: 'Judge each dim N times' }),
+            description: bilingual({ zh: '每个维度由评委评价 N 次', en: 'Judge each dim N times' }),
             parse: integerStringParser('--judge-repeat', { min: 1 }),
         }),
         bootstrap: Flags.boolean({
@@ -365,16 +368,16 @@ export default class Eval extends BaseCommand {
             parse: numberStringParser('--budget-usd', { minExclusive: 0 }),
         }),
         'budget-per-sample-usd': Flags.string({
-            description: bilingual({ zh: '单 sample 预算上限 USD（必须 > 0，不传则无上限）', en: 'Per-sample budget cap USD (must be > 0; omit for no cap)' }),
+            description: bilingual({ zh: '单用例预算上限 USD（必须 > 0，不传则无上限）', en: 'Per-sample budget cap USD (must be > 0; omit for no cap)' }),
             parse: numberStringParser('--budget-per-sample-usd', { minExclusive: 0 }),
         }),
         'budget-per-sample-ms': Flags.string({
-            description: bilingual({ zh: '单 sample 时长上限 ms（必须 > 0，不传则无上限）', en: 'Per-sample time cap ms (must be > 0; omit for no cap)' }),
+            description: bilingual({ zh: '单用例时长上限 ms（必须 > 0，不传则无上限）', en: 'Per-sample time cap ms (must be > 0; omit for no cap)' }),
             parse: integerStringParser('--budget-per-sample-ms', { min: 1 }),
         }),
         threshold: Flags.string({
             description: bilingual({
-                zh: `verdict 阈值，默认 ${DEFAULT_GATE_THRESHOLD}`,
+                zh: `判定阈值，默认 ${DEFAULT_GATE_THRESHOLD}`,
                 en: `Verdict threshold, default ${DEFAULT_GATE_THRESHOLD}`,
             }),
             parse: numberStringParser('--threshold'),
@@ -385,12 +388,12 @@ export default class Eval extends BaseCommand {
         }),
         'report-only': Flags.boolean({
             description: bilingual({
-                zh: '生成报告并打印 verdict，但始终 exit 0(不参与 CI gate）。',
+                zh: '生成报告并打印判定，但始终 exit 0(不参与 CI gate）。',
                 en: 'Produce the report and print verdict, but always exit 0 (no CI gate).',
             }),
         }),
         'no-gate': Flags.boolean({
-            description: bilingual({ zh: '关 verdict gate', en: 'Disable verdict gate' }),
+            description: bilingual({ zh: '关闭判定门禁', en: 'Disable verdict gate' }),
         }),
         'no-evidence': Flags.boolean({
             description: bilingual({

package/dist/cli/commands/init.js

@@ -77,8 +77,8 @@ export default class Init extends BaseCommand {
         },
         {
             description: bilingual({
-                zh: '使用达到注册样本量下限的 20 条官方用例初始化',
-                en: 'Initialize with the 20 first-party samples that meet the registered sample-size floor',
+                zh: '使用达到默认启发式证据下限的 20 条官方用例初始化',
+                en: 'Initialize with 20 first-party samples that meet the default heuristic evidence floor',
             }),
             command: '<%= config.bin %> init my-project --samples 20',
         },
@@ -108,8 +108,8 @@ export default class Init extends BaseCommand {
         lang: LANG_FLAG,
         samples: Flags.string({
             description: bilingual({
-                zh: '官方起步用例数量：3 条用于快速跑通，20 条用于达到注册样本量下限',
-                en: 'Number of first-party starter samples: 3 for a quick run, 20 to meet the registered sample-size floor',
+                zh: '官方起步用例数量：3 条用于快速跑通，20 条用于达到默认启发式证据下限',
+                en: 'Number of first-party starter samples: 3 for a quick run, 20 to meet the default heuristic evidence floor',
             }),
             options: [String(DEFAULT_INIT_SAMPLE_COUNT), String(FULL_INIT_SAMPLE_COUNT)],
             default: String(DEFAULT_INIT_SAMPLE_COUNT),

package/dist/cli/commands/studio.js

@@ -9,11 +9,11 @@ import { resolveObserveHealthDir, projectObserveHealthDir, globalObserveHealthDi
 import { DEFAULT_GLOBAL_OBSERVATIONS_DIR } from '../../observability/inbox/index.js';
 import { openWorkbench } from '../lib/open-workbench.js';
 // dev / browser-open 测试需要 mock `node:child_process` + `node:os`,通过 in-process
-// import 直接调用。把业务作为 module-level helper export 从 Command file 暴露,
-// 既保留 test 兼容又让产品命令树语义干净(无 legacy commands directory)。
+// import 直接调用。把业务作为 module-level helper export 从 Command file 暴露，
+// 便于测试命令行为，同时保持产品命令树语义干净。
 export async function runStudio(_args, flags, lang) {
-    // reports 读取目录：显式 --reports-dir 固定该目录；--global 钉全局；默认只聚合
-    // 当前项目 + 全局 Core run。旧 eval report index 已删除，不再扫描其它项目。
+    // reports 读取目录：显式 --reports-dir 固定该目录；--global 钉全局；默认聚合
+    // 当前项目与全局 Core run。
     const reportsDirOpt = flags['reports-dir']
         ? resolve(flags['reports-dir'])
         : flags.global

package/dist/cli/lib/i18n-dict/init.js

@@ -27,8 +27,8 @@ export const initDict = {
         en: '  2. Read the report verdict and Next line: 3 samples only prove the workflow, so UNDERPOWERED is expected; use --samples 20 in a new empty directory for the full starter set.',
     },
     'cli.init.next_step_report_full': {
-        zh: '  2. 看报告里的 verdict 和“下一步”：20 条用例达到注册样本量下限，但来源是 llm-generated；用于理解统计流程，发布前应人工复核并替换为真实领域用例。',
-        en: '  2. Read the report verdict and Next line: 20 samples meet the registered sample-size floor but are llm-generated; use them to learn the statistical workflow, then review and replace them with real domain cases before release.',
+        zh: '  2. 看报告里的 verdict 和“下一步”：20 条用例达到默认启发式证据下限，但不等于完成先验功效规划；其来源是 llm-generated，发布前应人工复核并替换为真实领域用例。',
+        en: '  2. Read the report verdict and Next line: 20 samples meet the default heuristic evidence floor, not an a priori power plan; they are llm-generated and must be reviewed or replaced with real domain cases before release.',
     },
     'cli.init.next_step_executor': {
         zh: '     executor / judge 会按运行环境选择；Codex 任务自动使用本机 Codex 配置。也可用 OMK_EXECUTOR / OMK_MODEL 固定环境偏好，详见 https://oh-my-knowledge.pages.dev/zh/reference/executors。',

package/dist/cli/lib/i18n-dict/run.js

@@ -64,8 +64,8 @@ export const runDict = {
         en: '⚠ --bootstrap-samples "{value}" is invalid (expected an integer ≥ 100), falling back to 1000\n',
     },
     'cli.run.bootstrap_samples_too_large': {
-        zh: '⚠ --bootstrap-samples {n} 较大, 可能耗时数秒。1000 是业内标准, 通常已够用。\n',
-        en: '⚠ --bootstrap-samples {n} is large and may take several seconds. 1000 is the industry standard and usually sufficient.\n',
+        zh: '⚠ --bootstrap-samples {n} 较大，可能耗时数秒。有限重采样误差会单独进入显著性证据。\n',
+        en: '⚠ --bootstrap-samples {n} is large and may take several seconds. Finite-resampling error is reported separately in significance evidence.\n',
     },
     'cli.run.dry_run_no_scores': {
         zh: 'eval dry-run：仅预览任务，不检查分数。下一步：确认任务无误后，去掉 --dry-run 运行正式评测。',

package/dist/cli/lib/run-core-evaluation.js

@@ -45,6 +45,9 @@ function requestFor(input, projectRoot) {
             judgeMembers: input.config.judgeModels.map((judge) => ({
                 executorId: judge.executor,
                 model: judge.model,
+                ...(judge.deploymentRevision === undefined
+                    ? {}
+                    : { deploymentRevision: judge.deploymentRevision }),
             })),
             presentation: {
                 projectOutputDirectoryLocator: projectReportsDir(),