oh-my-knowledge 1.0.0-beta.3 → 1.0.0-beta.5

package/README.md

@@ -22,6 +22,16 @@ Observe real-world performance, measure version differences, and determine wheth
 
 📖 **Full documentation: [oh-my-knowledge.pages.dev](https://oh-my-knowledge.pages.dev)** (searchable, English / 简体中文)
 
+## What knowledge means in OMK
+
+**Entities are the things knowledge describes; knowledge expresses their states, relationships, or actions in a specific context.**
+
+**How knowledge is expressed:**
+
+> **Time scope + scenario + conditions + entity A + relation or action + entity B (if any)**
+
+See [How OMK understands knowledge](docs/explanation/knowledge.md) for entity roles, shared relationships, and the distinction between knowledge and its carriers. Entity-based retrieval remains a design direction.
+
 ## What OMK helps you know
 
 | Decision | Command | Evidence you get |
@@ -35,6 +45,8 @@ Observe real-world performance, measure version differences, and determine wheth
 
 ## Quick start
 
+Already have a Node.js service, retriever, or Agent? Start with [Use OMK in your service](docs/guides/eval-runtime.md): run an example without model credentials, choose a scorer, connect your service, and read the results. The command-line quickstart follows below.
+
 ```bash
 npm i -g oh-my-knowledge
 omk init demo && cd demo
@@ -55,7 +67,7 @@ export OMK_EXECUTOR=codex
 
 Without `OMK_MODEL`, omk reads the model from `~/.codex/config.toml`. You can still pass `--executor codex --model <codex-model>` per command. Pass `--judge-models` or set `OMK_JUDGE_MODELS` only when you want a different judge.
 
-> The default 3-case pack is a low-cost workflow check, so `UNDERPOWERED` is expected. `--samples 20` selects a first-party, difficulty-stratified starter pack that meets omk's registered sample-size floor. Its provenance is `llm-generated`: use it to learn the statistical workflow, then review and replace it with real domain cases before making a release decision.
+> The default 3-case pack is a low-cost workflow check, so `UNDERPOWERED` is expected. `--samples 20` selects a first-party, difficulty-stratified starter pack that meets omk's default heuristic evidence floor; it is not an a priori power calculation. Its provenance is `llm-generated`: use it to learn the statistical workflow, then review and replace it with real domain cases before making a release decision.
 
 > The CLI notifies you when a newer version is available (at most once per 20h); set `OMK_SKIP_UPDATE_CHECK=1` to silence it permanently.
 
@@ -228,7 +240,7 @@ The full docs are published at **[oh-my-knowledge.pages.dev](https://oh-my-knowl
 - **[CLI reference](docs/reference/cli.md)** — all top-level commands with bash examples and flag tables
 - **[Migrate to the 1.0 preview](docs/guides/v1-preview-migration.md)** — install channel, storage reset, sample protocol, CLI automation, and embedded API changes since 0.54
 - **[Evaluation Core cutover](docs/guides/eval-core-cutover.md)** — `BREAKING-SCHEMA` storage, resume, Studio, Gold, managed-evidence, and evolve migration
-- **[Embed OMK in a service](docs/guides/eval-runtime.md)** — one `evaluate()` API for Node.js and FaaS hosts
+- **[Embed OMK in a service](docs/guides/eval-runtime.md)** — choose a scorer, connect a Node.js service, and interpret scores and failures
 - **[Storage layout v2](docs/specs/storage-layout-spec.md)** — project/global domains, compatibility boundary, and Git policy
 - **[Executors](docs/reference/executors.md)** & **[artifact layout](docs/reference/artifact-layout.md)** — built-in / custom executors; how `variant` resolves to an artifact + runtime context
 - **[How-to guides](docs/guides/agent-eval.md)** — [evaluate an agent](docs/guides/agent-eval.md) (project runtime context) and [use non-Claude models](docs/guides/non-claude-models.md) (GLM / Qwen / DeepSeek / Moonshot / Ollama)

package/README.zh.md

@@ -22,6 +22,16 @@
 
 📖 **完整文档：[oh-my-knowledge.pages.dev/zh](https://oh-my-knowledge.pages.dev/zh/)**（可搜索，可切换英文）
 
+## OMK 中的知识是什么
+
+**实体是知识所描述的事物；知识表达实体在特定上下文中的状态、关系或行为。**
+
+**知识的表达形式：**
+
+> **时间范围＋场景＋条件＋实体 A＋关系或行为＋实体 B（如有）**
+
+实体角色、多实体关联及知识与载体的区别，见[OMK 如何理解知识](docs/zh/explanation/knowledge.md)。实体关联检索仍是设计方向。
+
 ## OMK 让你知道什么
 
 | 决策问题 | 命令 | 你会得到的证据 |
@@ -35,6 +45,8 @@
 
 ## 快速开始
 
+已有 Node.js 服务、检索系统或 Agent，希望接入评分与版本对比？从[在服务中使用 OMK](docs/zh/guides/eval-runtime.md)开始：先运行无需模型凭证的示例，再选择评分方法、接入自己的服务并解读结果。下面是命令行上手流程。
+
 ```bash
 npm i -g oh-my-knowledge
 omk init demo && cd demo
@@ -55,7 +67,7 @@ export OMK_EXECUTOR=codex
 
 不设置 `OMK_MODEL` 时，omk 会读取 `~/.codex/config.toml` 的模型。也可以继续逐次显式传 `--executor codex --model <codex-model>`。自定义评委时再传 `--judge-models` 或设置 `OMK_JUDGE_MODELS`。
 
-> 默认 3 条用例是低成本流程检查，出现「数据不足（UNDERPOWERED）」符合预期。`--samples 20` 会选择经过难度分层的官方起步用例集，达到 omk 注册的样本量下限。其来源明确标记为 `llm-generated`：它适合学习统计流程，发布判断前仍应人工复核并替换为真实领域用例。
+> 默认 3 条用例是低成本流程检查，出现「数据不足（UNDERPOWERED）」符合预期。`--samples 20` 会选择经过难度分层的官方起步用例集，达到 omk 默认的启发式证据下限，但这不是先验功效计算。其来源明确标记为 `llm-generated`：它适合学习统计流程，发布判断前仍应人工复核并替换为真实领域用例。
 
 > 命令行有新版本时会自动提示（每 20 小时最多一次）；想永久关闭该提醒，设环境变量 `OMK_SKIP_UPDATE_CHECK=1` 即可。
 
@@ -228,7 +240,7 @@ omk-mcp
 - **[CLI 参考](docs/zh/reference/cli.md)** —— 顶层命令的 bash 示例和 flag 表
 - **[迁移到 1.0 预览版](docs/zh/guides/v1-preview-migration.md)** —— 从 `0.54` 升级时的安装渠道、存储重建、用例协议、CLI 自动化与嵌入式 API 变化
 - **[Evaluation Core 生产切换](docs/zh/guides/eval-core-cutover.md)** —— `BREAKING-SCHEMA` 存储、resume、Studio、Gold、受管证据与 evolve 迁移
-- **[在服务中嵌入 OMK](docs/zh/guides/eval-runtime.md)** —— 面向 Node.js 与 FaaS 宿主的单一 `evaluate()` API
+- **[在服务中嵌入 OMK](docs/zh/guides/eval-runtime.md)** —— 选择评分方法、接入 Node.js 服务、解读分数与失败原因
 - **[存储布局 v2](docs/zh/specs/storage-layout-spec.md)** —— 项目／全局领域、迁移兼容与 Git 策略
 - **[执行器](docs/zh/reference/executors.md)** & **[知识载体布局](docs/zh/reference/artifact-layout.md)** —— 内置 / 自定义执行器；variant 如何解析为 artifact + runtime context
 - **[操作指南](docs/zh/guides/agent-eval.md)** —— [评测 agent](docs/zh/guides/agent-eval.md)（项目 runtime context）与[使用非 Claude 模型](docs/zh/guides/non-claude-models.md)（GLM / 通义 / DeepSeek / Moonshot / Ollama）

package/dist/assets/agent-skills/omk/references/commands.md

@@ -104,7 +104,7 @@ omk eval [flags]
 - `--no-serve` `boolean`:不启 report server
 - `--no-strict-baseline` `boolean`:关闭 baseline 隔离
 - `--output-dir` `option`:报告输出目录（默认项目级 .omk/eval）
-- `--repeat` `option`:每个用例重复运行 N 次
+- `--repeat` `option`:预先固定 Evaluation Series 的独立 run 数
 - `--report-only` `boolean`:生成报告并打印判定，但始终 exit 0(不参与 CI gate）。
 - `--resume` `option`:复用经过完整契约校验的 Core runId；拒绝时失败关闭
 - `--retry` `option`:单用例失败重试次数
@@ -169,6 +169,7 @@ omk eval gold compare <runId> [flags]
 - `--gold-dir` `option`:gold dataset 目录，必填
 - `--lang` `option` (默认 `zh`):输出语言 zh|en，优先级 CLI > OMK_LANG env > zh。
 - `--metric` `option`:显式选择 Core metric ID。
+- `--minimum-alpha` `option`:可选的一致性阈值；按 Krippendorff α 置信区间下界评估
 - `--reports-dir` `option`:只读取指定 Core 报告目录（默认当前项目 + 全局）
 - `--seed` `option`:bootstrap seed，可复现
 - `--target` `option`:显式选择 Core target ID。
@@ -274,7 +275,7 @@ omk init [targetDir] [flags]
 
 - `--force` `boolean`:允许覆盖目标目录中已有的 omk 脚手架文件
 - `--lang` `option` (默认 `zh`):输出语言 zh|en，优先级 CLI > OMK_LANG env > zh。
-- `--samples` `3|20` (默认 `3`):官方起步用例数量：3 条用于快速跑通，20 条用于达到注册样本量下限
+- `--samples` `3|20` (默认 `3`):官方起步用例数量：3 条用于快速跑通，20 条用于达到默认启发式证据下限
 
 **示例:**
 
@@ -290,7 +291,7 @@ omk init
 omk init my-project
 ```
 
-> 使用达到注册样本量下限的 20 条官方用例初始化
+> 使用达到默认启发式证据下限的 20 条官方用例初始化
 
 ```bash
 omk init my-project --samples 20

package/dist/cli/commands/eval/gold/compare.d.ts

@@ -10,6 +10,7 @@ export default class EvalGoldCompare extends BaseCommand {
         target: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         evaluator: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         metric: import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
+        'minimum-alpha': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         'trial-index': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         'reports-dir': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;
         'bootstrap-samples': import("@oclif/core/interfaces").OptionFlag<string | undefined, import("@oclif/core/interfaces").CustomOptions>;

package/dist/cli/commands/eval/gold/compare.js

@@ -2,7 +2,7 @@ import { resolve } from 'node:path';
 import { Args, Flags } from '@oclif/core';
 import { BaseCommand } from '../../../oclif/base-command.js';
 import { LANG_FLAG, bilingual } from '../../../oclif/i18n.js';
-import { integerStringParser } from '../../../oclif/parsers.js';
+import { integerStringParser, numberStringParser } from '../../../oclif/parsers.js';
 import { CliExit } from '../../../lib/cli-exit.js';
 import { projectReportsDir, globalReportsDir } from '../../../../evidence/storage/directories.js';
 export default class EvalGoldCompare extends BaseCommand {
@@ -33,6 +33,13 @@ export default class EvalGoldCompare extends BaseCommand {
         metric: Flags.string({
             description: bilingual({ zh: '显式选择 Core metric ID。', en: 'Explicit Core metric ID.' }),
         }),
+        'minimum-alpha': Flags.string({
+            description: bilingual({
+                zh: '可选的一致性阈值；按 Krippendorff α 置信区间下界评估',
+                en: 'Optional agreement threshold, assessed against the Krippendorff alpha CI lower bound',
+            }),
+            parse: numberStringParser('--minimum-alpha', { min: -1, max: 1 }),
+        }),
         'trial-index': Flags.string({
             description: bilingual({ zh: '显式选择 trial index。', en: 'Explicit trial index.' }),
             parse: integerStringParser('--trial-index', { min: 0 }),
@@ -61,26 +68,31 @@ export default class EvalGoldCompare extends BaseCommand {
         await this.runWithCliExit(async () => {
             const runId = args.runId;
             if (!runId) {
-                console.error('Usage: omk eval gold compare <runId> --gold-dir <dir> --target <id> --evaluator <id> --metric <id>');
+                console.error(lang === 'zh'
+                    ? '用法：omk eval gold compare <runId> --gold-dir <dir> --target <id> --evaluator <id> --metric <id>'
+                    : 'Usage: omk eval gold compare <runId> --gold-dir <dir> --target <id> --evaluator <id> --metric <id>');
                 throw new CliExit(1);
             }
             const goldDir = flags['gold-dir'];
             if (!goldDir) {
-                console.error('--gold-dir is required');
+                console.error(lang === 'zh' ? '必须提供 --gold-dir。' : '--gold-dir is required.');
                 throw new CliExit(1);
             }
-            const { loadGoldDataset } = await import('../../../../eval-workflows/gold/dataset.js');
+            const { loadGoldDataset, validationIssueMessage } = await import('../../../../eval-workflows/gold/dataset.js');
             const { createNodeCoreContentStore, createNodeCoreRunArtifactStore, createOverlayCoreRunArtifactStore, } = await import('../../../../eval-workflows/artifact-store/index.js');
             const { compareGoldToCoreRun } = await import('../../../../eval-workflows/projections/gold.js');
             const { dataset, issues } = loadGoldDataset(goldDir);
             if (!dataset) {
-                console.error('Cannot load gold dataset:');
+                console.error(lang === 'zh' ? '无法加载 gold dataset：' : 'Cannot load gold dataset:');
                 for (const i of issues)
-                    console.error(`  - ${i.message}`);
+                    console.error(`  - ${validationIssueMessage(i, lang)}`);
                 throw new CliExit(1);
             }
-            for (const i of issues)
-                console.error(`warn: ${i.message}`);
+            for (const i of issues) {
+                console.error(lang === 'zh'
+                    ? `警告：${validationIssueMessage(i, lang)}`
+                    : `Warning: ${validationIssueMessage(i, lang)}`);
+            }
             const storeOf = (directory) => createNodeCoreRunArtifactStore(directory, {
                 contentResolver: createNodeCoreContentStore(resolve(directory, 'content')),
             });
@@ -90,8 +102,8 @@ export default class EvalGoldCompare extends BaseCommand {
             const source = await store.get(runId);
             if (source === undefined) {
                 console.error(lang === 'zh'
-                    ? `找不到 Core run「${runId}」；旧 evaluation report 不再支持 gold compare。`
-                    : `Core run "${runId}" was not found; legacy evaluation reports are no longer supported.`);
+                    ? `找不到 Core run「${runId}」。`
+                    : `Core run "${runId}" was not found.`);
                 throw new CliExit(1);
             }
             if (!flags.target || !flags.evaluator || !flags.metric) {
@@ -114,6 +126,9 @@ export default class EvalGoldCompare extends BaseCommand {
                     }),
                 },
                 bootstrapSamples: samples,
+                ...(flags['minimum-alpha'] === undefined ? {} : {
+                    minimumAlpha: Number(flags['minimum-alpha']),
+                }),
                 ...(Number.isFinite(seedVal) ? { bootstrapSeed: seedVal } : {}),
             });
             console.log(JSON.stringify(result, null, 2));

package/dist/cli/commands/eval/gold/validate.js

@@ -25,18 +25,22 @@ export default class EvalGoldValidate extends BaseCommand {
         await this.runWithCliExit(async () => {
             const dir = args.dir;
             if (!dir) {
-                console.error('Usage: omk eval gold validate <dir>');
+                console.error(lang === 'zh'
+                    ? '用法：omk eval gold validate <dir>'
+                    : 'Usage: omk eval gold validate <dir>');
                 throw new CliExit(1);
             }
             const { validateGoldDataset } = await import('../../../../eval-workflows/gold/cli.js');
-            const result = validateGoldDataset(dir);
+            const result = validateGoldDataset(dir, lang);
             if (result.ok) {
                 console.log(lang === 'zh'
                     ? `✓ gold dataset OK，共 ${result.sampleCount} 条标注`
                     : `✓ gold dataset OK — ${result.sampleCount} annotations`);
                 return;
             }
-            console.error(`✗ gold dataset has ${result.issues.length} issue(s):`);
+            console.error(lang === 'zh'
+                ? `✗ gold dataset 存在 ${result.issues.length} 个问题：`
+                : `✗ gold dataset has ${result.issues.length} issue(s):`);
             for (const msg of result.issues)
                 console.error(`  - ${msg}`);
             throw new CliExit(1);

package/dist/cli/commands/eval/index.js

@@ -160,6 +160,10 @@ async function runEval(_args, flags, lang) {
             process.stdout.write(`Core: ${outcome.status?.runStatus ?? 'prepared'}／${outcome.status?.evidenceStatus ?? 'n/a'}／${outcome.status?.conclusionStatus ?? 'n/a'}\n`
                 + `Gate: ${outcome.gate?.gateStatus ?? 'not-applicable'}${outcome.gate?.reasonCodes?.length ? `（${outcome.gate.reasonCodes.join(', ')}）` : ''}\n`);
         }
+        // A blocked gate exits immediately through oclif; flush a piped JSON report first.
+        await new Promise((resolve, reject) => {
+            process.stdout.write('', (error) => error ? reject(error) : resolve());
+        });
         throw new CliExit(result.exitCode);
     }
     catch (err) {
@@ -333,7 +337,10 @@ export default class Eval extends BaseCommand {
         }),
         // ── eval-runner extra ──
         repeat: Flags.string({
-            description: bilingual({ zh: '每个用例重复运行 N 次', en: 'Repeat each sample N times' }),
+            description: bilingual({
+                zh: '预先固定 Evaluation Series 的独立 run 数',
+                en: 'Predeclare the independent run count for the Evaluation Series',
+            }),
             parse: integerStringParser('--repeat', { min: 1 }),
         }),
         'holdout-ratio': Flags.string({

package/dist/cli/commands/init.js

@@ -77,8 +77,8 @@ export default class Init extends BaseCommand {
         },
         {
             description: bilingual({
-                zh: '使用达到注册样本量下限的 20 条官方用例初始化',
-                en: 'Initialize with the 20 first-party samples that meet the registered sample-size floor',
+                zh: '使用达到默认启发式证据下限的 20 条官方用例初始化',
+                en: 'Initialize with 20 first-party samples that meet the default heuristic evidence floor',
             }),
             command: '<%= config.bin %> init my-project --samples 20',
         },
@@ -108,8 +108,8 @@ export default class Init extends BaseCommand {
         lang: LANG_FLAG,
         samples: Flags.string({
             description: bilingual({
-                zh: '官方起步用例数量：3 条用于快速跑通，20 条用于达到注册样本量下限',
-                en: 'Number of first-party starter samples: 3 for a quick run, 20 to meet the registered sample-size floor',
+                zh: '官方起步用例数量：3 条用于快速跑通，20 条用于达到默认启发式证据下限',
+                en: 'Number of first-party starter samples: 3 for a quick run, 20 to meet the default heuristic evidence floor',
             }),
             options: [String(DEFAULT_INIT_SAMPLE_COUNT), String(FULL_INIT_SAMPLE_COUNT)],
             default: String(DEFAULT_INIT_SAMPLE_COUNT),

package/dist/cli/commands/studio.js

@@ -9,11 +9,11 @@ import { resolveObserveHealthDir, projectObserveHealthDir, globalObserveHealthDi
 import { DEFAULT_GLOBAL_OBSERVATIONS_DIR } from '../../observability/inbox/index.js';
 import { openWorkbench } from '../lib/open-workbench.js';
 // dev / browser-open 测试需要 mock `node:child_process` + `node:os`,通过 in-process
-// import 直接调用。把业务作为 module-level helper export 从 Command file 暴露,
-// 既保留 test 兼容又让产品命令树语义干净(无 legacy commands directory)。
+// import 直接调用。把业务作为 module-level helper export 从 Command file 暴露，
+// 便于测试命令行为，同时保持产品命令树语义干净。
 export async function runStudio(_args, flags, lang) {
-    // reports 读取目录：显式 --reports-dir 固定该目录；--global 钉全局；默认只聚合
-    // 当前项目 + 全局 Core run。旧 eval report index 已删除，不再扫描其它项目。
+    // reports 读取目录：显式 --reports-dir 固定该目录；--global 钉全局；默认聚合
+    // 当前项目与全局 Core run。
     const reportsDirOpt = flags['reports-dir']
         ? resolve(flags['reports-dir'])
         : flags.global

package/dist/cli/lib/i18n-dict/init.js

@@ -27,8 +27,8 @@ export const initDict = {
         en: '  2. Read the report verdict and Next line: 3 samples only prove the workflow, so UNDERPOWERED is expected; use --samples 20 in a new empty directory for the full starter set.',
     },
     'cli.init.next_step_report_full': {
-        zh: '  2. 看报告里的 verdict 和“下一步”：20 条用例达到注册样本量下限，但来源是 llm-generated；用于理解统计流程，发布前应人工复核并替换为真实领域用例。',
-        en: '  2. Read the report verdict and Next line: 20 samples meet the registered sample-size floor but are llm-generated; use them to learn the statistical workflow, then review and replace them with real domain cases before release.',
+        zh: '  2. 看报告里的 verdict 和“下一步”：20 条用例达到默认启发式证据下限，但不等于完成先验功效规划；其来源是 llm-generated，发布前应人工复核并替换为真实领域用例。',
+        en: '  2. Read the report verdict and Next line: 20 samples meet the default heuristic evidence floor, not an a priori power plan; they are llm-generated and must be reviewed or replaced with real domain cases before release.',
     },
     'cli.init.next_step_executor': {
         zh: '     executor / judge 会按运行环境选择；Codex 任务自动使用本机 Codex 配置。也可用 OMK_EXECUTOR / OMK_MODEL 固定环境偏好，详见 https://oh-my-knowledge.pages.dev/zh/reference/executors。',

package/dist/cli/lib/i18n-dict/run.js

@@ -64,8 +64,8 @@ export const runDict = {
         en: '⚠ --bootstrap-samples "{value}" is invalid (expected an integer ≥ 100), falling back to 1000\n',
     },
     'cli.run.bootstrap_samples_too_large': {
-        zh: '⚠ --bootstrap-samples {n} 较大, 可能耗时数秒。1000 是业内标准, 通常已够用。\n',
-        en: '⚠ --bootstrap-samples {n} is large and may take several seconds. 1000 is the industry standard and usually sufficient.\n',
+        zh: '⚠ --bootstrap-samples {n} 较大，可能耗时数秒。有限重采样误差会单独进入显著性证据。\n',
+        en: '⚠ --bootstrap-samples {n} is large and may take several seconds. Finite-resampling error is reported separately in significance evidence.\n',
     },
     'cli.run.dry_run_no_scores': {
         zh: 'eval dry-run：仅预览任务，不检查分数。下一步：确认任务无误后，去掉 --dry-run 运行正式评测。',

package/dist/cli/lib/run-core-evaluation.js

@@ -2,7 +2,9 @@ import { existsSync } from 'node:fs';
 import { join, resolve } from 'node:path';
 import { compileCliEvaluationInput, parseCliEvaluationRequest, } from '../../eval-workflows/input-compilation/index.js';
 import { createNodeCoreBatchArtifactStore, createNodeCoreContentStore, createNodeCoreRunArtifactStore, createOverlayCoreRunArtifactStore, } from '../../eval-workflows/artifact-store/index.js';
-import { createNodeCliProductionComposition, createProductionEvaluationHost, executeProductionEvaluationSeries, persistCoreArtifactSidecars, resolveNodeCliEvaluationRequest, } from '../../eval-workflows/production-host/index.js';
+import { createNodeCliProductionComposition, } from '../../eval-hosts/node/node-cli-composition.js';
+import { createProductionEvaluationHost, executeProductionEvaluationSeries, persistCoreArtifactSidecars, } from '../../eval-workflows/production-host/index.js';
+import { resolveNodeCliEvaluationRequest, } from '../../eval-hosts/node/node-cli-evaluation-resolver.js';
 import { projectCoreCliDryRun, projectCoreCliBatchOutcome, projectCoreCliRunOutcome, projectCoreCliSeriesOutcome, } from '../../eval-workflows/projections/cli.js';
 import { projectCoreManagedEvidence } from '../../eval-workflows/projections/managed.js';
 import { discoverBatchSkills } from '../../eval-workflows/inputs/skill-loader.js';
@@ -45,6 +47,9 @@ function requestFor(input, projectRoot) {
             judgeMembers: input.config.judgeModels.map((judge) => ({
                 executorId: judge.executor,
                 model: judge.model,
+                ...(judge.deploymentRevision === undefined
+                    ? {}
+                    : { deploymentRevision: judge.deploymentRevision }),
             })),
             presentation: {
                 projectOutputDirectoryLocator: projectReportsDir(),
@@ -223,10 +228,11 @@ export async function runCoreEvaluationCommand(input) {
         resourceLeaseRoot: machineLayout.resourceLeasesDir,
         environment: input.environment,
     });
-    const store = input.store ?? runStoreForOutput(outputDirectory, composition.support.contentResolver);
+    const store = input.store ?? runStoreForOutput(outputDirectory, composition.contentResolver);
     const host = {
         compiled,
-        ...composition,
+        runtime: composition.runtime,
+        schemaValidators: composition.schemaValidators,
         artifactStore: store,
     };
     if (compiled.orchestration.dryRun) {
@@ -258,8 +264,8 @@ export async function runCoreEvaluationCommand(input) {
         const memberArtifacts = await Promise.all(series.members.map(async (member) => {
             if (member.executionStatus !== 'started')
                 throw member.error;
-            await member.run.result;
             const persistence = await member.run.persistence;
+            await member.run.result;
             if (persistence.persistenceStatus !== 'stored') {
                 if (persistence.persistenceStatus === 'failed')
                     throw persistence.error;
@@ -313,8 +319,8 @@ export async function runCoreEvaluationCommand(input) {
             createdAt: new Date().toISOString(),
             progressSink: emitProgress(input.lang),
         });
-        await run.result;
         const persistence = await run.persistence;
+        await run.result;
         if (persistence.persistenceStatus !== 'stored') {
             if (persistence.persistenceStatus === 'failed')
                 throw persistence.error;