oh-my-knowledge 1.0.0-beta.1 → 1.0.0-beta.3

package/README.md

@@ -228,6 +228,7 @@ The full docs are published at **[oh-my-knowledge.pages.dev](https://oh-my-knowl
 - **[CLI reference](docs/reference/cli.md)** — all top-level commands with bash examples and flag tables
 - **[Migrate to the 1.0 preview](docs/guides/v1-preview-migration.md)** — install channel, storage reset, sample protocol, CLI automation, and embedded API changes since 0.54
 - **[Evaluation Core cutover](docs/guides/eval-core-cutover.md)** — `BREAKING-SCHEMA` storage, resume, Studio, Gold, managed-evidence, and evolve migration
+- **[Embed OMK in a service](docs/guides/eval-runtime.md)** — one `evaluate()` API for Node.js and FaaS hosts
 - **[Storage layout v2](docs/specs/storage-layout-spec.md)** — project/global domains, compatibility boundary, and Git policy
 - **[Executors](docs/reference/executors.md)** & **[artifact layout](docs/reference/artifact-layout.md)** — built-in / custom executors; how `variant` resolves to an artifact + runtime context
 - **[How-to guides](docs/guides/agent-eval.md)** — [evaluate an agent](docs/guides/agent-eval.md) (project runtime context) and [use non-Claude models](docs/guides/non-claude-models.md) (GLM / Qwen / DeepSeek / Moonshot / Ollama)

package/README.zh.md

@@ -157,7 +157,7 @@ RAG 专项评测请看 RAGAS（独立 niche，跟 omk 互补）。完整对比
 
 | 特性 | 说明 |
 |------|------|
-| **Core 发布决定** | 六种结论 + 稳定 reason code + exit code 路由；Studio 投影同一份经过认证的 Decision |
+| **Core 发布决定** | 六种判定 + 稳定 reason code + exit code 路由；Studio 投影同一份经过认证的 Decision |
 | **五层 evidence graph** | Assertion / LLM / Judge / Dimension / Composite 保持独立，coverage、成本、状态与 lineage 与分数正交 |
 | **多执行器** | 支持 Claude CLI / Claude SDK / Codex CLI / Codex SDK / DeepSeek Harness / OpenAI / Anthropic API 及自定义命令 |
 | **30+ 种断言** | 包含子串、正则、JSON Schema、ROUGE/BLEU/Levenshtein 相似度、Agent 工具调用、语义相似度、自定义函数等 |
@@ -228,6 +228,7 @@ omk-mcp
 - **[CLI 参考](docs/zh/reference/cli.md)** —— 顶层命令的 bash 示例和 flag 表
 - **[迁移到 1.0 预览版](docs/zh/guides/v1-preview-migration.md)** —— 从 `0.54` 升级时的安装渠道、存储重建、用例协议、CLI 自动化与嵌入式 API 变化
 - **[Evaluation Core 生产切换](docs/zh/guides/eval-core-cutover.md)** —— `BREAKING-SCHEMA` 存储、resume、Studio、Gold、受管证据与 evolve 迁移
+- **[在服务中嵌入 OMK](docs/zh/guides/eval-runtime.md)** —— 面向 Node.js 与 FaaS 宿主的单一 `evaluate()` API
 - **[存储布局 v2](docs/zh/specs/storage-layout-spec.md)** —— 项目／全局领域、迁移兼容与 Git 策略
 - **[执行器](docs/zh/reference/executors.md)** & **[知识载体布局](docs/zh/reference/artifact-layout.md)** —— 内置 / 自定义执行器；variant 如何解析为 artifact + runtime context
 - **[操作指南](docs/zh/guides/agent-eval.md)** —— [评测 agent](docs/zh/guides/agent-eval.md)（项目 runtime context）与[使用非 Claude 模型](docs/zh/guides/non-claude-models.md)（GLM / 通义 / DeepSeek / Moonshot / Ollama）

package/dist/assets/agent-skills/omk/references/commands.md

@@ -63,7 +63,7 @@ omk doctor --json --gate
 
 ## omk eval
 
-跑评测：对一个 control vs 多个 treatment skill 做对照试验，产 verdict 报告。
+跑评测：用一个对照组与多个实验组 skill 做对照试验，生成判定报告。
 
 **用法:**
 
@@ -73,16 +73,16 @@ omk eval [flags]
 
 **Flags:**
 
-- `--batch` `boolean`:batch 模式:baseline vs 每个 skill
+- `--batch` `boolean`:batch 模式：baseline 作为对照组，逐个 skill 作为实验组
 - `--bootstrap` `boolean`:加 bootstrap CI
 - `--bootstrap-samples` `option`:bootstrap 重采样次数，默认 1000
-- `--budget-per-sample-ms` `option`:单 sample 时长上限 ms（必须 > 0，不传则无上限）
-- `--budget-per-sample-usd` `option`:单 sample 预算上限 USD（必须 > 0，不传则无上限）
+- `--budget-per-sample-ms` `option`:单用例时长上限 ms（必须 > 0，不传则无上限）
+- `--budget-per-sample-usd` `option`:单用例预算上限 USD（必须 > 0，不传则无上限）
 - `--budget-usd` `option`:总预算上限 USD（必须 > 0，不传则无上限）
 - `--concurrency` `option`:并发数，默认 1
 - `--config` `option`:eval.yaml 路径
-- `--control` `option`:control variant 表达式（仅 artifact 身份）
-- `--control-cwd` `option`:control 的 runtime context 目录
+- `--control` `option`:对照组（control）的 variant 表达式（仅 artifact 身份）
+- `--control-cwd` `option`:对照组（control）的 runtime context 目录
 - `--dry-run` `boolean`:只 plan 不实跑
 - `--effort` `option`:被测 LLM 扩展思考预算 low/medium/high/xhigh/max（默认 low；跨 effort 报告不严格可比）。
 - `--executor` `option`:执行器：claude / claude-sdk / codex / codex-sdk / anthropic-api / openai-api / 自定义命令。Codex 任务内自动用 codex；也可用 OMK_EXECUTOR 设置环境偏好。
@@ -90,7 +90,7 @@ omk eval [flags]
 - `--gold-dir` `option`:gold dataset 目录
 - `--holdout-ratio` `option`:留出比例 0-1（如 0.3）；切出 holdout 子集，对比 train/holdout 综合分检测过拟合
 - `--judge-models` `option`:评委配置，格式 executor:model[,...]，例 claude:haiku 或 codex:<model>（≥ 2 个 = ensemble）。默认跟随所选执行器；Codex 沿用被测模型。
-- `--judge-repeat` `option`:每个 dim 评 N 次
+- `--judge-repeat` `option`:每个维度由评委评价 N 次
 - `--lang` `option` (默认 `zh`):输出语言 zh|en，优先级 CLI > OMK_LANG env > zh。
 - `--layered-stats` `boolean`:输出分层统计
 - `--mcp-config` `option`:MCP 配置文件路径
@@ -99,30 +99,30 @@ omk eval [flags]
 - `--no-debias-length` `boolean`:关 length-debias（默认开）
 - `--no-diagnostic` `boolean`:关闭基于 Core 失败、缺失、排除与稳定 reason code 的诊断投影。
 - `--no-evidence` `boolean`:不把本次评测写成证据追加进受管记录(默认会为已 install 的 skill 自动写)。
-- `--no-gate` `boolean`:关 verdict gate
-- `--no-judge` `boolean`:跳过 LLM judge
+- `--no-gate` `boolean`:关闭判定门禁
+- `--no-judge` `boolean`:跳过 LLM 评委
 - `--no-serve` `boolean`:不启 report server
 - `--no-strict-baseline` `boolean`:关闭 baseline 隔离
 - `--output-dir` `option`:报告输出目录（默认项目级 .omk/eval）
-- `--repeat` `option`:每个 sample 重复跑 N 次
-- `--report-only` `boolean`:生成报告并打印 verdict，但始终 exit 0(不参与 CI gate）。
+- `--repeat` `option`:每个用例重复运行 N 次
+- `--report-only` `boolean`:生成报告并打印判定，但始终 exit 0(不参与 CI gate）。
 - `--resume` `option`:复用经过完整契约校验的 Core runId；拒绝时失败关闭
-- `--retry` `option`:失败 sample 重试次数
-- `--samples` `option`:用例路径。自动发现项目级或单 treatment 目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。
+- `--retry` `option`:单用例失败重试次数
+- `--samples` `option`:用例路径。自动发现项目级或单个实验组（treatment）目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。
 - `--skill-dir` `option`:skill 目录，默认 skills
 - `--skip-connectivity` `boolean`:跳 LLM 连通性预检
 - `--skip-doctor` `boolean`:escape hatch:跳 doctor 健康检查门禁（默认强制启用）。沙箱 mock 提供依赖时绕开 doctor 物理路径误报；garbage-in 风险自负。
 - `--strict-baseline` `boolean`:强制 baseline 隔离（default true）
-- `--threshold` `option`:verdict 阈值，默认 3.5
+- `--threshold` `option`:判定阈值，默认 3.5
 - `--timeout` `option`:单用例超时秒，默认 120
-- `--treatment` `option`:treatment variant 列表，逗号分隔（仅 artifact 身份）
-- `--treatment-cwd` `option`:treatment 的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）
+- `--treatment` `option`:实验组（treatment）的 variant 列表，逗号分隔（仅 artifact 身份）
+- `--treatment-cwd` `option`:实验组（treatment）的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）
 - `--trivial-diff` `option`:可忽略 diff 容差，0 表示不启用容差
 - `--verbose` `boolean`:详细日志
 
 **示例:**
 
-> 最简对照:baseline vs my-skill
+> 最简对照：baseline 作为对照组，my-skill 作为实验组
 
 ```bash
 omk eval --control baseline --treatment my-skill

package/dist/cli/commands/eval/index.js

@@ -174,13 +174,13 @@ async function runEval(_args, flags, lang) {
 }
 export default class Eval extends BaseCommand {
     static description = bilingual({
-        zh: '跑评测：对一个 control vs 多个 treatment skill 做对照试验，产 verdict 报告。',
+        zh: '跑评测：用一个对照组与多个实验组 skill 做对照试验，生成判定报告。',
         en: 'Run evaluation: control vs treatment(s) comparison, produce verdict report.',
     });
     static examples = [
         {
             description: bilingual({
-                zh: '最简对照:baseline vs my-skill',
+                zh: '最简对照：baseline 作为对照组，my-skill 作为实验组',
                 en: 'Minimal A/B: baseline vs my-skill',
             }),
             command: '<%= config.bin %> eval --control baseline --treatment my-skill',
@@ -197,23 +197,23 @@ export default class Eval extends BaseCommand {
         lang: LANG_FLAG,
         // ── 实验角色 ──
         control: Flags.string({
-            description: bilingual({ zh: 'control variant 表达式（仅 artifact 身份）', en: 'Control variant expr (artifact identity only)' }),
+            description: bilingual({ zh: '对照组（control）的 variant 表达式（仅 artifact 身份）', en: 'Control variant expr (artifact identity only)' }),
         }),
         treatment: Flags.string({
             description: bilingual({
-                zh: 'treatment variant 列表，逗号分隔（仅 artifact 身份）',
+                zh: '实验组（treatment）的 variant 列表，逗号分隔（仅 artifact 身份）',
                 en: 'Treatment variants, comma-separated (artifact identity only)',
             }),
         }),
         'control-cwd': Flags.string({
             description: bilingual({
-                zh: 'control 的 runtime context 目录',
+                zh: '对照组（control）的 runtime context 目录',
                 en: 'Runtime context dir for control',
             }),
         }),
         'treatment-cwd': Flags.string({
             description: bilingual({
-                zh: 'treatment 的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）',
+                zh: '实验组（treatment）的 runtime context 目录列表，逗号分隔、与 --treatment 按序对齐（空位 = 无 cwd）',
                 en: 'Runtime context dirs for treatments, comma-separated, index-aligned with --treatment (blank = none)',
             }),
         }),
@@ -222,7 +222,7 @@ export default class Eval extends BaseCommand {
         }),
         samples: Flags.string({
             description: bilingual({
-                zh: '用例路径。自动发现项目级或单 treatment 目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。',
+                zh: '用例路径。自动发现项目级或单个实验组（treatment）目录 skill 下的 eval-samples.json / eval-samples.yaml；显式路径可为 JSON / YAML 文件或分片目录。',
                 en: 'Samples path. Auto-discovers eval-samples.json / eval-samples.yaml at project scope or for a single directory-skill treatment; an explicit path may be a JSON / YAML file or split directory.',
             }),
         }),
@@ -256,7 +256,7 @@ export default class Eval extends BaseCommand {
         }),
         // ── 评测 toggle ──
         'no-judge': Flags.boolean({
-            description: bilingual({ zh: '跳过 LLM judge', en: 'Skip LLM judge' }),
+            description: bilingual({ zh: '跳过 LLM 评委', en: 'Skip LLM judge' }),
         }),
         'no-cache': Flags.boolean({
             description: bilingual({ zh: '跳过 executor cache', en: 'Skip executor cache' }),
@@ -277,7 +277,7 @@ export default class Eval extends BaseCommand {
         }),
         batch: Flags.boolean({
             description: bilingual({
-                zh: 'batch 模式:baseline vs 每个 skill',
+                zh: 'batch 模式：baseline 作为对照组，逐个 skill 作为实验组',
                 en: 'Batch mode: baseline vs each skill',
             }),
         }),
@@ -300,7 +300,7 @@ export default class Eval extends BaseCommand {
             description: bilingual({ zh: '详细日志', en: 'Verbose logging' }),
         }),
         retry: Flags.string({
-            description: bilingual({ zh: '失败 sample 重试次数', en: 'Per-sample retry count' }),
+            description: bilingual({ zh: '单用例失败重试次数', en: 'Per-sample retry count' }),
             parse: integerStringParser('--retry', { min: 0 }),
         }),
         resume: Flags.string({
@@ -333,7 +333,7 @@ export default class Eval extends BaseCommand {
         }),
         // ── eval-runner extra ──
         repeat: Flags.string({
-            description: bilingual({ zh: '每个 sample 重复跑 N 次', en: 'Repeat each sample N times' }),
+            description: bilingual({ zh: '每个用例重复运行 N 次', en: 'Repeat each sample N times' }),
             parse: integerStringParser('--repeat', { min: 1 }),
         }),
         'holdout-ratio': Flags.string({
@@ -344,7 +344,7 @@ export default class Eval extends BaseCommand {
             parse: numberStringParser('--holdout-ratio', { min: 0, max: 1 }),
         }),
         'judge-repeat': Flags.string({
-            description: bilingual({ zh: '每个 dim 评 N 次', en: 'Judge each dim N times' }),
+            description: bilingual({ zh: '每个维度由评委评价 N 次', en: 'Judge each dim N times' }),
             parse: integerStringParser('--judge-repeat', { min: 1 }),
         }),
         bootstrap: Flags.boolean({
@@ -365,16 +365,16 @@ export default class Eval extends BaseCommand {
             parse: numberStringParser('--budget-usd', { minExclusive: 0 }),
         }),
         'budget-per-sample-usd': Flags.string({
-            description: bilingual({ zh: '单 sample 预算上限 USD（必须 > 0，不传则无上限）', en: 'Per-sample budget cap USD (must be > 0; omit for no cap)' }),
+            description: bilingual({ zh: '单用例预算上限 USD（必须 > 0，不传则无上限）', en: 'Per-sample budget cap USD (must be > 0; omit for no cap)' }),
             parse: numberStringParser('--budget-per-sample-usd', { minExclusive: 0 }),
         }),
         'budget-per-sample-ms': Flags.string({
-            description: bilingual({ zh: '单 sample 时长上限 ms（必须 > 0，不传则无上限）', en: 'Per-sample time cap ms (must be > 0; omit for no cap)' }),
+            description: bilingual({ zh: '单用例时长上限 ms（必须 > 0，不传则无上限）', en: 'Per-sample time cap ms (must be > 0; omit for no cap)' }),
             parse: integerStringParser('--budget-per-sample-ms', { min: 1 }),
         }),
         threshold: Flags.string({
             description: bilingual({
-                zh: `verdict 阈值，默认 ${DEFAULT_GATE_THRESHOLD}`,
+                zh: `判定阈值，默认 ${DEFAULT_GATE_THRESHOLD}`,
                 en: `Verdict threshold, default ${DEFAULT_GATE_THRESHOLD}`,
             }),
             parse: numberStringParser('--threshold'),
@@ -385,12 +385,12 @@ export default class Eval extends BaseCommand {
         }),
         'report-only': Flags.boolean({
             description: bilingual({
-                zh: '生成报告并打印 verdict，但始终 exit 0(不参与 CI gate）。',
+                zh: '生成报告并打印判定，但始终 exit 0(不参与 CI gate）。',
                 en: 'Produce the report and print verdict, but always exit 0 (no CI gate).',
             }),
         }),
         'no-gate': Flags.boolean({
-            description: bilingual({ zh: '关 verdict gate', en: 'Disable verdict gate' }),
+            description: bilingual({ zh: '关闭判定门禁', en: 'Disable verdict gate' }),
         }),
         'no-evidence': Flags.boolean({
             description: bilingual({

package/dist/dsh-plugin/core-adapter.js

@@ -3,7 +3,7 @@ import { JsonValueSchema, RuntimeIdentitySchema, canonicalizeJson, deepFreezeCan
 import { ExecutionPortFailure, } from '../eval-core/execution/index.js';
 import { createDshHostRuntimeFingerprint } from '../executors/core/runtime-fingerprint.js';
 import { captureClaudeCliRunState, captureClaudeCliTarget, disposeClaudeCliTrial, openClaudeCliTrial, } from '../eval-workflows/runtime-adapter/adapters/claude/resources.js';
-import { createSameProcessExecutorAdapter } from '../eval-workflows/runtime-adapter/adapters/shared/same-process.js';
+import { createSameProcessExecutorAdapter } from '../eval-runtime/adapters/same-process.js';
 import { DSH_HOST_CORE_ADAPTER_IMPLEMENTATION_VERSION, dshHostCoreExecutorCapabilities, parseDshHostCoreResult, } from './core-protocol.js';
 import { supportsDshTraceEventType } from './trace-adapter.js';
 export { DSH_HOST_CORE_ADAPTER_IMPLEMENTATION_VERSION, createDshHostCoreSchemaValidators, } from './core-protocol.js';

package/dist/eval-core/analysis/runtime.js

@@ -260,20 +260,28 @@ function rowsForInputs(inputs) {
     }
     return [...byId.values()].sort((left, right) => compareStrings(left.rowId, right.rowId));
 }
-function observationCoverage(rows, includedRowIds, comparableRowIds) {
+function observationCoverage(rows, includedRowIds, comparableRowIds, notApplicableRowIds = []) {
     const included = new Set(includedRowIds);
     const comparable = new Set(comparableRowIds);
+    const notApplicable = new Set(notApplicableRowIds);
     const known = new Set(rows.map((row) => row.rowId));
     if (included.size !== includedRowIds.length
         || comparable.size !== comparableRowIds.length
+        || notApplicable.size !== notApplicableRowIds.length
         || [...included].some((rowId) => !known.has(rowId))
-        || [...comparable].some((rowId) => !included.has(rowId))) {
+        || [...comparable].some((rowId) => !included.has(rowId))
+        || [...notApplicable].some((rowId) => !known.has(rowId) || included.has(rowId))) {
         throw new TypeError('Analysis implementation returned invalid row membership.');
     }
+    if (rows.some((row) => (notApplicable.has(row.rowId)
+        && (row.rowStatus !== 'missing' || row.censored)))) {
+        throw new TypeError('Analysis implementation marked an ineligible row as not applicable.');
+    }
     const coverage = {
         planned: rows.length,
         observed: 0,
         missing: 0,
+        notApplicable: notApplicable.size,
         invalid: 0,
         evaluationFailed: 0,
         sourceUnavailable: 0,
@@ -284,6 +292,8 @@ function observationCoverage(rows, includedRowIds, comparableRowIds) {
         comparable: comparable.size,
     };
     for (const row of rows) {
+        if (notApplicable.has(row.rowId))
+            continue;
         if (row.rowStatus === 'observed')
             coverage.observed += 1;
         else if (row.rowStatus === 'missing')
@@ -319,6 +329,18 @@ function exclusionFacts(rows, includedRowIds) {
     }))
         .sort((left, right) => compareStrings(left.rowId, right.rowId));
 }
+function notApplicableFacts(rows, notApplicableRowIds) {
+    const notApplicable = new Set(notApplicableRowIds);
+    return rows
+        .filter((row) => notApplicable.has(row.rowId))
+        .map((row) => {
+        if (row.rowStatus === 'observed') {
+            throw new TypeError('Observed Analysis rows cannot be structurally not applicable.');
+        }
+        return { rowId: row.rowId, reasonCode: row.reasonCode };
+    })
+        .sort((left, right) => compareStrings(left.rowId, right.rowId));
+}
 function safeError(error) {
     if (error instanceof AnalysisPortFailure) {
         const parsed = EvaluationErrorSchema.safeParse(error.evaluationError);
@@ -583,6 +605,7 @@ async function runAnalysis(plan, ports, options, prepared, stream) {
                 inputReferences: binding.node.inputs,
                 coverage: emptyCoverage(rows),
                 exclusions: exclusionFacts(rows, []),
+                notApplicableRows: [],
                 assumptionChecks: [],
                 analysisMode: plan.analysis.analysisGraph.analysisMode,
                 derivedAt: ports.clock.timestamp(),
@@ -735,9 +758,10 @@ async function runAnalysis(plan, ports, options, prepared, stream) {
                 .map((row) => row.rowId);
             const includedRowIds = output.includedRowIds ?? observedRowIds;
             const comparableRowIds = output.comparableRowIds ?? includedRowIds;
+            const notApplicableRowIds = output.notApplicableRowIds ?? [];
             let coverage;
             try {
-                coverage = observationCoverage(rows, includedRowIds, comparableRowIds);
+                coverage = observationCoverage(rows, includedRowIds, comparableRowIds, notApplicableRowIds);
             }
             catch (error) {
                 const evaluationError = safeError(error);
@@ -784,6 +808,7 @@ async function runAnalysis(plan, ports, options, prepared, stream) {
                         ...base,
                         coverage,
                         exclusions: exclusionFacts(rows, includedRowIds),
+                        notApplicableRows: notApplicableFacts(rows, notApplicableRowIds),
                         assumptionChecks: checks,
                     }, runtimeDependencies, {
                         analysisStatus: 'inconclusive',
@@ -832,6 +857,7 @@ async function runAnalysis(plan, ports, options, prepared, stream) {
                     ...base,
                     coverage,
                     exclusions: exclusionFacts(rows, includedRowIds),
+                    notApplicableRows: notApplicableFacts(rows, notApplicableRowIds),
                     assumptionChecks: checks,
                 }, runtimeDependencies, {
                     analysisStatus: 'completed',
@@ -852,6 +878,7 @@ async function runAnalysis(plan, ports, options, prepared, stream) {
                     ...base,
                     coverage,
                     exclusions: exclusionFacts(rows, includedRowIds),
+                    notApplicableRows: notApplicableFacts(rows, notApplicableRowIds),
                     assumptionChecks: checks,
                 }, runtimeDependencies, {
                     analysisStatus: 'inconclusive',

package/dist/eval-core/analysis/types.d.ts

@@ -80,12 +80,14 @@ export type AnalysisNodeExecutionResult = {
     value: JsonValue;
     includedRowIds?: readonly Sha256Digest[];
     comparableRowIds?: readonly Sha256Digest[];
+    notApplicableRowIds?: readonly Sha256Digest[];
     assumptionChecks?: readonly Omit<AssumptionCheck, 'nodeId'>[];
 } | {
     analysisStatus: 'inconclusive';
     reasonCodes: readonly string[];
     includedRowIds?: readonly Sha256Digest[];
     comparableRowIds?: readonly Sha256Digest[];
+    notApplicableRowIds?: readonly Sha256Digest[];
     assumptionChecks?: readonly Omit<AssumptionCheck, 'nodeId'>[];
 };
 export declare const AnalysisNodeExecutionResultSchema: z.ZodDiscriminatedUnion<[z.ZodObject<{
@@ -101,6 +103,7 @@ export declare const AnalysisNodeExecutionResultSchema: z.ZodDiscriminatedUnion<
     value: z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>;
     includedRowIds: z.ZodOptional<z.ZodArray<z.ZodString>>;
     comparableRowIds: z.ZodOptional<z.ZodArray<z.ZodString>>;
+    notApplicableRowIds: z.ZodOptional<z.ZodArray<z.ZodString>>;
     assumptionChecks: z.ZodOptional<z.ZodArray<z.ZodObject<{
         details: z.ZodOptional<z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>>;
         reasonCode: z.ZodOptional<z.ZodString>;
@@ -116,6 +119,7 @@ export declare const AnalysisNodeExecutionResultSchema: z.ZodDiscriminatedUnion<
     reasonCodes: z.ZodArray<z.ZodString>;
     includedRowIds: z.ZodOptional<z.ZodArray<z.ZodString>>;
     comparableRowIds: z.ZodOptional<z.ZodArray<z.ZodString>>;
+    notApplicableRowIds: z.ZodOptional<z.ZodArray<z.ZodString>>;
     assumptionChecks: z.ZodOptional<z.ZodArray<z.ZodObject<{
         details: z.ZodOptional<z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>>;
         reasonCode: z.ZodOptional<z.ZodString>;

package/dist/eval-core/analysis/types.js

@@ -26,6 +26,7 @@ export const AnalysisNodeExecutionResultSchema = z.discriminatedUnion('analysisS
         value: JsonValueSchema,
         includedRowIds: z.array(Sha256DigestSchema).optional(),
         comparableRowIds: z.array(Sha256DigestSchema).optional(),
+        notApplicableRowIds: z.array(Sha256DigestSchema).optional(),
         assumptionChecks: z.array(PortAssumptionCheckSchema).optional(),
     }).strict(),
     z.object({
@@ -33,6 +34,7 @@ export const AnalysisNodeExecutionResultSchema = z.discriminatedUnion('analysisS
         reasonCodes: z.array(IdentifierSchema).min(1),
         includedRowIds: z.array(Sha256DigestSchema).optional(),
         comparableRowIds: z.array(Sha256DigestSchema).optional(),
+        notApplicableRowIds: z.array(Sha256DigestSchema).optional(),
         assumptionChecks: z.array(PortAssumptionCheckSchema).optional(),
     }).strict(),
 ]);

package/dist/eval-core/contracts/analysis-bundle.js

@@ -24,6 +24,7 @@ function compareStrings(left, right) {
 function assertObservationCoverage(coverage) {
     if (coverage.planned !== coverage.observed
         + coverage.missing
+        + coverage.notApplicable
         + coverage.invalid
         + coverage.evaluationFailed
         + coverage.sourceUnavailable
@@ -58,6 +59,20 @@ function assertRecords(bundle) {
         if (record.exclusions.length !== record.coverage.excluded) {
             throw new AnalysisBundleValidationError('ANALYSIS_BUNDLE_COVERAGE_INVALID', 'Analysis exclusions must account for every excluded observation row.');
         }
+        if (record.notApplicableRows.length !== record.coverage.notApplicable) {
+            throw new AnalysisBundleValidationError('ANALYSIS_BUNDLE_COVERAGE_INVALID', 'Analysis not-applicable rows must account for structural coverage.');
+        }
+        const exclusionById = new Map(record.exclusions.map((entry) => [entry.rowId, entry]));
+        for (let index = 0; index < record.notApplicableRows.length; index += 1) {
+            const fact = record.notApplicableRows[index];
+            const previousFact = record.notApplicableRows[index - 1];
+            if (previousFact !== undefined && compareStrings(previousFact.rowId, fact.rowId) >= 0) {
+                throw new AnalysisBundleValidationError('ANALYSIS_BUNDLE_COVERAGE_INVALID', 'Analysis not-applicable rows must use unique canonical row order.');
+            }
+            if (exclusionById.get(fact.rowId)?.reasonCode !== fact.reasonCode) {
+                throw new AnalysisBundleValidationError('ANALYSIS_BUNDLE_COVERAGE_INVALID', 'Every not-applicable row must be retained as a matching exclusion fact.');
+            }
+        }
         for (let exclusionIndex = 0; exclusionIndex < record.exclusions.length; exclusionIndex += 1) {
             const exclusion = record.exclusions[exclusionIndex];
             const previousExclusion = record.exclusions[exclusionIndex - 1];
@@ -282,10 +297,12 @@ function expectedRows(plan, execution, evaluation, node) {
     return rows.sort((left, right) => compareStrings(left.rowId, right.rowId));
 }
 function assertSourceCoverage(record, rows) {
+    const notApplicableIds = new Set(record.notApplicableRows.map((entry) => entry.rowId));
     const expected = {
         planned: rows.length,
         observed: rows.filter((row) => row.rowStatus === 'observed').length,
-        missing: rows.filter((row) => row.rowStatus === 'missing').length,
+        missing: rows.filter((row) => (row.rowStatus === 'missing' && !notApplicableIds.has(row.rowId))).length,
+        notApplicable: record.notApplicableRows.length,
         invalid: rows.filter((row) => row.rowStatus === 'invalid').length,
         evaluationFailed: rows.filter((row) => row.rowStatus === 'evaluation-failed').length,
         sourceUnavailable: rows.filter((row) => row.rowStatus === 'source-unavailable').length,
@@ -296,6 +313,7 @@ function assertSourceCoverage(record, rows) {
         planned: record.coverage.planned,
         observed: record.coverage.observed,
         missing: record.coverage.missing,
+        notApplicable: record.coverage.notApplicable,
         invalid: record.coverage.invalid,
         evaluationFailed: record.coverage.evaluationFailed,
         sourceUnavailable: record.coverage.sourceUnavailable,
@@ -308,6 +326,12 @@ function assertSourceCoverage(record, rows) {
     const byId = new Map(rows.map((row) => [row.rowId, row]));
     const exclusionById = new Map(record.exclusions.map((entry) => [entry.rowId, entry]));
     if (record.exclusions.some((entry) => !byId.has(entry.rowId))
+        || record.notApplicableRows.some((entry) => {
+            const row = byId.get(entry.rowId);
+            return row?.rowStatus !== 'missing'
+                || row.censored
+                || row.reasonCode !== entry.reasonCode;
+        })
         || rows.some((row) => row.rowStatus !== 'observed'
             && (exclusionById.get(row.rowId)?.reasonCode !== row.reasonCode))) {
         throw new AnalysisBundleValidationError('ANALYSIS_BUNDLE_SOURCE_MISMATCH', 'Analysis exclusions do not match source observation identities and reasons.');