oh-my-knowledge 0.52.3 → 0.53.0

package/README.md

@@ -159,7 +159,7 @@ RAG-specific evals: see RAGAS (separate niche, complementary to omk). Full compa
 |---|---|
 | **One-line verdict** | `omk eval` six-tier verdict + ship recommendation + exit-code routing; HTML pill shares the same rules |
 | **Six-dim evaluation** | Fact / Behavior / LLM-judge / Cost / Efficiency / Stability shown independently |
-| **Multi-executor** | Claude CLI / Claude SDK / Codex CLI / Codex SDK / OpenAI / Gemini / Anthropic API / any custom command |
+| **Multi-executor** | Claude CLI / Claude SDK / Codex CLI / Codex SDK / DeepSeek Harness / OpenAI / Anthropic API / any custom command |
 | **30+ assertion types** | substring, regex, JSON Schema, ROUGE/BLEU/Levenshtein similarity, agent tool-call assertions, semantic similarity, custom JS |
 | **Statistical rigor** | Bootstrap CI / length-debias / saturation curve on by default; Krippendorff α auto-computed with a gold set. [Details →](docs/explanation/statistical-rigor.md) |
 | **RAG metrics** | `faithfulness` / `answer_relevancy` / `context_recall` — anti-hallucination + answer relevance + context coverage |
@@ -177,6 +177,17 @@ RAG-specific evals: see RAGAS (separate niche, complementary to omk). Full compa
 | **Traceability** | reports carry CLI version, Node version, artifact version fingerprint, judge prompt hash |
 | **EN / ZH switch** | one-click language toggle in the HTML report |
 
+### Run inside an existing DeepSeek Harness
+
+Install OMK as a DSH bundle to reuse the profile's model, credentials, tools, and sandbox:
+
+```bash
+dsh plugin --profile web add oh-my-knowledge
+dsh --profile web
+```
+
+Then run `/omk eval eval.yaml` inside DSH. Each sample gets an isolated DSH session while OMK still owns the report and statistics. See the [executor guide](docs/reference/executors.md#deepseek-harness-prefer-the-host-plugin).
+
 ## Documentation
 
 The full docs are published at **[oh-my-knowledge.pages.dev](https://oh-my-knowledge.pages.dev)** — searchable, with an English / 简体中文 switcher. Key pages:
@@ -211,6 +222,7 @@ The full docs are published at **[oh-my-knowledge.pages.dev](https://oh-my-knowl
   - Codex: install and authenticate the Codex CLI (`npm i -g @openai/codex`); Codex tasks in the ChatGPT desktop app select it automatically
   - Claude: install and authenticate [Claude Code](https://claude.ai/code)
   - API / other executors: configure them as described in [Executors](docs/reference/executors.md)
+- Advanced `claude-sdk` / `codex-sdk` executors are optional and are not downloaded by the base OMK install. Install the matching SDK in the same local project or global npm prefix only when you select one; see [Executor prerequisites](docs/reference/executors.md#prerequisites).
 
 ## Security notice
 

package/README.zh.md

@@ -159,7 +159,7 @@ RAG 专项评测请看 RAGAS（独立 niche，跟 omk 互补）。完整对比
 |------|------|
 | **Verdict 一行结论** | `omk eval` 六档判定 + ship 建议 + exit code 路由，与 HTML 报告 verdict pill 共享规则 |
 | **六维评估** | 事实 / 行为 / LLM 评价 / 成本 / 效率 / 稳定性独立展示 |
-| **多执行器** | 支持 Claude CLI / Claude SDK / Codex CLI / Codex SDK / OpenAI / Gemini / Anthropic API 及自定义命令 |
+| **多执行器** | 支持 Claude CLI / Claude SDK / Codex CLI / Codex SDK / DeepSeek Harness / OpenAI / Anthropic API 及自定义命令 |
 | **30+ 种断言** | 包含子串、正则、JSON Schema、ROUGE/BLEU/Levenshtein 相似度、Agent 工具调用、语义相似度、自定义函数等 |
 | **统计严谨性** | Bootstrap CI / 长度去偏 / 饱和曲线默认开，Krippendorff α 提供 gold 集即自动计算。[详情 →](docs/zh/explanation/statistical-rigor.md) |
 | **RAG metrics** | `faithfulness` / `answer_relevancy` / `context_recall` 三 metric — 反幻觉 + 切题度 + context 覆盖 |
@@ -177,6 +177,17 @@ RAG 专项评测请看 RAGAS（独立 niche，跟 omk 互补）。完整对比
 | **可追溯性** | 报告含 CLI 版本、Node 版本、知识载体版本指纹、judge prompt hash |
 | **中英切换** | HTML 报告右上角一键切换语言 |
 
+### 在已有 DeepSeek Harness 中运行
+
+OMK 可以作为 DSH bundle 安装到现有 profile，直接复用其模型、凭证、工具与 sandbox：
+
+```bash
+dsh plugin --profile web add oh-my-knowledge
+dsh --profile web
+```
+
+进入 DSH 后运行 `/omk eval eval.yaml`。每条用例会使用独立 DSH session，报告仍由 OMK 生成。详见[执行器文档](docs/zh/reference/executors.md#deepseek-harness优先使用宿主插件)。
+
 ## 文档
 
 完整文档已发布到 **[oh-my-knowledge.pages.dev/zh](https://oh-my-knowledge.pages.dev/zh/)** —— 可搜索，可切换英文。重点页面：
@@ -211,6 +222,7 @@ RAG 专项评测请看 RAGAS（独立 niche，跟 omk 互补）。完整对比
   - Codex：安装并登录 Codex CLI（`npm i -g @openai/codex`）；ChatGPT desktop 的 Codex 任务会自动选择它
   - Claude：安装并登录 [Claude Code](https://claude.ai/code)
   - API / 其它执行器：按[执行器文档](docs/zh/reference/executors.md)配置
+- 高级 `claude-sdk`／`codex-sdk` 执行器是可选能力，OMK 基础安装不再下载它们。仅在明确选择对应 SDK 时，才在 OMK 所在的本地项目或全局 npm prefix 安装；详见[执行器前置要求](docs/zh/reference/executors.md#前置要求)。
 
 ## 安全说明
 

package/dist/assets/agent-skills/omk/SKILL.md

@@ -27,6 +27,10 @@ Codex 是 omk 的一等 runtime。运行在 Codex 任务中时，`omk eval` / `d
 
 普通终端想固定走 Codex 时，可以设置 `OMK_EXECUTOR=codex`；`OMK_MODEL` 可覆盖本机 Codex 配置，`OMK_JUDGE_MODELS` 可覆盖默认评委。逐次覆盖仍可使用 `--executor` / `--model` / `--judge-models`。Codex 不需要 Claude Code 风格的 `/omk` slash command，直接执行 CLI。
 
+### 在 DeepSeek Harness 中
+
+如果当前 DSH profile 已安装 `oh-my-knowledge` bundle，使用 `/omk eval <eval.yaml>`。该路径直接复用现有 DSH 的模型、凭证、工具与 sandbox，并为每条用例创建隔离 session；不要要求用户另起 DSH runtime。
+
 ## 第二步：理解用户意图
 
 根据用户的描述，匹配对应的操作：

package/dist/assets/agent-skills/omk/references/commands.md

@@ -85,7 +85,7 @@ omk eval [flags]
 - `--control-cwd` `option`:control 的 runtime context 目录
 - `--dry-run` `boolean`:只 plan 不实跑
 - `--effort` `option`:被测 LLM 扩展思考预算 low/medium/high/xhigh/max（默认 low；跨 effort 报告不严格可比）。
-- `--executor` `option`:执行器：claude / claude-sdk / codex / codex-sdk / openai-api / gemini / 自定义命令。Codex 任务内自动用 codex；也可用 OMK_EXECUTOR 设置环境偏好。
+- `--executor` `option`:执行器：claude / claude-sdk / codex / codex-sdk / anthropic-api / openai-api / 自定义命令。Codex 任务内自动用 codex；也可用 OMK_EXECUTOR 设置环境偏好。
 - `--global` `boolean`:报告写全局 ~/.oh-my-knowledge/reports，而非项目 .omk/
 - `--gold-dir` `option`:gold dataset 目录
 - `--holdout-ratio` `option`:留出比例 0-1（如 0.3）；切出 holdout 子集，对比 train/holdout 综合分检测过拟合

package/dist/authoring/evolver.js

@@ -17,7 +17,7 @@ import { toolCallStatus } from '../shared/tool-call-status.js';
 import { ownRecordValue, setOwnRecordValue } from '../shared/record-count.js';
 import { getJudgePromptHash } from '../grading/judge.js';
 import { getDiagnosticPromptHash, resolveDiagnosticTarget, } from '../grading/diagnostic.js';
-import { getExecutorRuntimeFingerprint } from '../executors/runtime-fingerprint.js';
+import { getExecutorRuntimeFingerprint } from '../executors/core/runtime-fingerprint.js';
 import { parseReportDocument } from '../eval-core/report-document.js';
 const IMPROVE_SYSTEM_PROMPT = `你是一个 AI 提示词改进专家。你的任务是分析评测结果中的薄弱环节，针对性地改进 skill（系统提示词），使其在评测中获得更高的分数。
 

package/dist/authoring/generator.js

@@ -1,5 +1,5 @@
 import { createExecutor } from '../executors/index.js';
-import { executorSupportsSampleMocks } from '../executors/capabilities.js';
+import { executorSupportsSampleMocks } from '../executors/core/capabilities.js';
 import { DEFAULT_GATE_THRESHOLD } from '../eval-core/verdict.js';
 import { sampleMockReferenceKeys } from '../shared/sample-contract.js';
 const SYSTEM_PROMPT = `你是一个评测用例生成器。你的任务是根据用户提供的 skill（系统提示词）内容，生成高质量的评测用例。

package/dist/cli/commands/eval/index.js

@@ -16,10 +16,11 @@ import { DEFAULT_GATE_THRESHOLD } from '../../../eval-core/verdict.js';
 import { EVALUATION_REPORT_SCHEMA_VERSION } from '../../../eval-core/evaluation-reporting.js';
 import { findSingleTreatmentDeprecatedSamplesHint, hasUsableSamplesPath, } from '../../../inputs/sample-locator.js';
 import { shellQuoteArg } from '../../../shared/shell-quote.js';
-const CLAUDE_EXECUTORS = new Set(['claude', 'claude-sdk']);
-const CODEX_EXECUTORS = new Set(['codex', 'codex-sdk']);
-const OPENAI_API_EXECUTORS = new Set(['openai-api']);
-const ANTHROPIC_API_EXECUTORS = new Set(['anthropic-api']);
+import { executorNamesForFamily } from '../../../executors/core/registry.js';
+const CLAUDE_EXECUTORS = executorNamesForFamily('claude');
+const CODEX_EXECUTORS = executorNamesForFamily('codex');
+const OPENAI_API_EXECUTORS = executorNamesForFamily('openai-api');
+const ANTHROPIC_API_EXECUTORS = executorNamesForFamily('anthropic-api');
 function isDryRunReport(report) {
     return Boolean(report && typeof report === 'object' && report.dryRun === true);
 }
@@ -559,8 +560,8 @@ export default class Eval extends BaseCommand {
         }),
         executor: Flags.string({
             description: bilingual({
-                zh: '执行器：claude / claude-sdk / codex / codex-sdk / openai-api / gemini / 自定义命令。Codex 任务内自动用 codex；也可用 OMK_EXECUTOR 设置环境偏好。',
-                en: 'Executor: claude / claude-sdk / codex / codex-sdk / openai-api / gemini / custom. Defaults to codex inside Codex tasks; OMK_EXECUTOR sets an environment preference.',
+                zh: '执行器：claude / claude-sdk / codex / codex-sdk / anthropic-api / openai-api / 自定义命令。Codex 任务内自动用 codex；也可用 OMK_EXECUTOR 设置环境偏好。',
+                en: 'Executor: claude / claude-sdk / codex / codex-sdk / anthropic-api / openai-api / custom. Defaults to codex inside Codex tasks; OMK_EXECUTOR sets an environment preference.',
             }),
         }),
         'judge-models': Flags.string({

package/dist/cli/commands/install.js

@@ -25,7 +25,11 @@ const TARGET_SPECS = {
 };
 function packagedOmkAgentSkillDir() {
     const here = dirname(fileURLToPath(import.meta.url));
-    return resolve(here, '..', '..', 'assets', 'agent-skills', 'omk');
+    const packaged = resolve(here, '..', '..', 'assets', 'agent-skills', 'omk');
+    if (existsSync(packaged))
+        return packaged;
+    const workspace = resolve(here, '..', '..', '..', '.agents', 'skills', 'omk');
+    return existsSync(workspace) ? workspace : packaged;
 }
 function knownTarget(target) {
     const home = homedir();

package/dist/cli/lib/generation-failure-hint.js

@@ -1,24 +1,22 @@
 import { tCli } from './i18n.js';
 import { codexExecutorFlags, codexModelFlagValue, codexModelHint } from './codex-model-hint.js';
 import { looksLikeLlmSetupFailure, looksLikeModelUnavailableFailure } from './llm-failure-classifier.js';
-const CLAUDE_SAMPLE_EXECUTORS = new Set(['claude', 'claude-sdk']);
-const CODEX_SAMPLE_EXECUTORS = new Set(['codex', 'codex-sdk']);
-const OPENAI_API_SAMPLE_EXECUTORS = new Set(['openai-api']);
-const ANTHROPIC_API_SAMPLE_EXECUTORS = new Set(['anthropic-api']);
+import { executorFamily } from '../../executors/core/registry.js';
 export function formatSampleGenerationFailureHint(message, executorName, lang, env = process.env) {
     const executor = executorName?.trim();
     if (!executor)
         return '';
     if (!looksLikeLlmSetupFailure(message))
         return '';
-    if (CLAUDE_SAMPLE_EXECUTORS.has(executor)) {
+    const family = executorFamily(executor);
+    if (family === 'claude') {
         return tCli('cli.gen.claude_auth_hint', lang, {
             codexFlags: codexExecutorFlags(env),
             codexModelHint: codexModelHint(lang, env),
             openaiFlags: '--executor openai-api --model <openai-model>',
         });
     }
-    if (CODEX_SAMPLE_EXECUTORS.has(executor)) {
+    if (family === 'codex') {
         if (looksLikeModelUnavailableFailure(message)) {
             return tCli('cli.gen.codex_model_hint', lang, {
                 codexFlags: codexExecutorFlags(env),
@@ -33,7 +31,7 @@ export function formatSampleGenerationFailureHint(message, executorName, lang, e
             openaiFlags: '--executor openai-api --model <openai-model>',
         });
     }
-    if (OPENAI_API_SAMPLE_EXECUTORS.has(executor)) {
+    if (family === 'openai-api') {
         if (looksLikeModelUnavailableFailure(message)) {
             return tCli('cli.gen.openai_api_model_hint', lang, {
                 claudeFlags: '--executor claude --model sonnet',
@@ -47,7 +45,7 @@ export function formatSampleGenerationFailureHint(message, executorName, lang, e
             codexModelHint: codexModelHint(lang, env),
         });
     }
-    if (ANTHROPIC_API_SAMPLE_EXECUTORS.has(executor)) {
+    if (family === 'anthropic-api') {
         if (looksLikeModelUnavailableFailure(message)) {
             return tCli('cli.gen.anthropic_api_model_hint', lang, {
                 claudeFlags: '--executor claude --model sonnet',

package/dist/cli/lib/runtime-defaults.d.ts

@@ -1,3 +1,5 @@
+export declare const DEFAULT_CLAUDE_MODEL = "sonnet";
+export declare const DEFAULT_CLAUDE_JUDGE_MODEL = "haiku";
 export interface RuntimeResolutionOptions {
     env?: NodeJS.ProcessEnv;
     commandExists?: (command: string, env: NodeJS.ProcessEnv) => boolean;

package/dist/cli/lib/runtime-defaults.js

@@ -1,7 +1,10 @@
 import { accessSync, constants } from 'node:fs';
 import { delimiter, join } from 'node:path';
 import { getCodexModelSuggestion } from './codex-model-hint.js';
-import { DEFAULT_MODEL, JUDGE_MODEL } from '../../executors/shared.js';
+import { executorFamily } from '../../executors/core/registry.js';
+// Claude 的模型 alias 只属于 CLI runtime 选择策略，不是执行器共享默认值。
+export const DEFAULT_CLAUDE_MODEL = 'sonnet';
+export const DEFAULT_CLAUDE_JUDGE_MODEL = 'haiku';
 function nonEmpty(value) {
     const trimmed = value?.trim();
     return trimmed ? trimmed : undefined;
@@ -27,7 +30,7 @@ export function isCodexHost(env = process.env) {
         || nonEmpty(env.CODEX_CI));
 }
 export function isCodexExecutor(executor) {
-    return executor === 'codex' || executor === 'codex-sdk';
+    return executorFamily(executor) === 'codex';
 }
 export function resolveCliExecutor(explicitExecutor, options = {}) {
     const env = options.env ?? process.env;
@@ -53,7 +56,7 @@ export function resolveCliModel(executor, explicitModel, options = {}) {
     if (envModel)
         return envModel;
     if (!isCodexExecutor(executor))
-        return DEFAULT_MODEL;
+        return DEFAULT_CLAUDE_MODEL;
     const suggestion = getCodexModelSuggestion(env);
     if (suggestion.fromConfig)
         return suggestion.model;
@@ -63,7 +66,7 @@ export function resolveCliModel(executor, explicitModel, options = {}) {
         : `The Codex executor needs an explicit model. Pass --model <model>, set OMK_MODEL, or configure a top-level model in ${suggestion.configPath}.`);
 }
 export function defaultJudgeModel(executor, taskModel) {
-    return isCodexExecutor(executor) ? taskModel : JUDGE_MODEL;
+    return isCodexExecutor(executor) ? taskModel : DEFAULT_CLAUDE_JUDGE_MODEL;
 }
 export function resolveRuntimeSelection(input, options = {}) {
     const executor = resolveCliExecutor(input.executor, options);

package/dist/dsh-plugin/cordis.patch.yml

@@ -0,0 +1,3 @@
+- insert:
+    - id: omk
+      name: oh-my-knowledge/dist/dsh-plugin/index.js

package/dist/dsh-plugin/host-executor.d.ts

@@ -0,0 +1,93 @@
+import type { ExecutorFn } from '../types/index.js';
+type UnknownRecord = Record<string, unknown>;
+export interface DshSessionLike {
+    readonly id: string;
+    readonly events: readonly UnknownRecord[];
+    readonly header: {
+        readonly cwd?: string;
+        readonly parentSession?: string;
+        readonly agentPreset?: string;
+    };
+}
+export interface DshAgentLike {
+    readonly id: string;
+    readonly options: {
+        readonly provider?: string;
+        readonly model?: string;
+    };
+    readonly ctx: object;
+    readonly session: DshSessionLike;
+    followup(message: UnknownRecord): void;
+    whenIdle(): Promise<void>;
+    /** `kind` mirrors DSH's host-owned cancellation protocol. */
+    cancel(cause: Readonly<Record<'kind', 'hook'> & {
+        reason: string;
+    }>): void;
+}
+interface DshAgentScopeLike {
+    readonly agent?: DshAgentLike;
+    readonly systemPrompt: {
+        section(section: {
+            readonly name: string;
+            readonly order: number;
+            readonly text: string;
+            readonly complete: true;
+        }): () => void;
+        suppressRuntimeContext(): () => void;
+    };
+    readonly tools?: {
+        get(name: string, agent?: DshAgentLike): unknown;
+        restrict(filter: {
+            readonly deny: readonly string[];
+        }): () => void;
+    };
+}
+interface DshAgentPresetsLike {
+    composedPreset(agentCtx: object): string | undefined;
+    composeFrom(agentCtx: object, parentCtx: object): string | undefined;
+}
+interface DshAgentHandleLike {
+    readonly agent: DshAgentLike;
+    dispose(): Promise<void>;
+}
+/** Minimal same-process DSH surface consumed by the OMK host adapter. */
+export interface DshHostContextLike {
+    readonly agents: {
+        create(options: {
+            readonly sessionId: string;
+            readonly meta: {
+                readonly cwd: string;
+                readonly parentSession?: string;
+                readonly agentPreset?: string;
+            };
+            readonly agentOptions: {
+                readonly provider?: string;
+                readonly model: string;
+            };
+            readonly setup: (ctx: DshAgentScopeLike) => void;
+        }): Promise<DshAgentHandleLike>;
+    };
+    readonly tools?: {
+        schemas(scope?: DshAgentLike): readonly UnknownRecord[];
+    };
+    readonly agentPresets: DshAgentPresetsLike;
+    on(event: 'session/event', listener: (session: DshSessionLike, entry: UnknownRecord) => void): () => void;
+    on(event: 'session/created', listener: (session: DshSessionLike) => void): () => void;
+}
+export interface DshHostExecutorOptions {
+    /** Interactive DSH session that initiated the measurement. */
+    parentAgent?: DshAgentLike;
+    /** Provider inherited by fresh measurement sessions. */
+    provider?: string;
+    /** Cancellation owned by the DSH command or embedding surface. */
+    signal?: AbortSignal;
+    /** Maximum time spent waiting for cancellation or disposal to settle. */
+    cleanupTimeoutMs?: number;
+}
+/**
+ * Build an OMK executor that creates fresh agents inside an already-running
+ * DSH plugin tree. The host keeps ownership of credentials, tools, policies,
+ * and persistence; OMK owns sample isolation and result projection.
+ */
+export declare function createDshHostExecutor(ctx: DshHostContextLike, options?: DshHostExecutorOptions): ExecutorFn;
+export {};

package/dist/dsh-plugin/host-executor.js

@@ -0,0 +1,232 @@
+import { randomUUID } from 'node:crypto';
+import { buildDshHostResult } from './protocol.js';
+import { createDshHostRuntimeFingerprint } from '../executors/core/runtime-fingerprint.js';
+import { DEFAULT_TIMEOUT_MS } from '../executors/core/limits.js';
+const DEFAULT_CLEANUP_TIMEOUT_MS = 5_000;
+function errorMessage(error) {
+    return error instanceof Error ? error.message : String(error);
+}
+function textFromContent(value) {
+    if (typeof value === 'string')
+        return value;
+    if (!Array.isArray(value))
+        return '';
+    return value.flatMap((block) => {
+        if (typeof block !== 'object' || block === null || Array.isArray(block))
+            return [];
+        const record = block;
+        return record.type === 'text' && typeof record.text === 'string' ? [record.text] : [];
+    }).join('');
+}
+function lastAssistantText(events) {
+    for (let index = events.length - 1; index >= 0; index -= 1) {
+        const event = events[index];
+        if (event?.type !== 'assistant/message')
+            continue;
+        const data = typeof event.data === 'object' && event.data !== null && !Array.isArray(event.data)
+            ? event.data
+            : undefined;
+        const message = typeof data?.message === 'object' && data.message !== null && !Array.isArray(data.message)
+            ? data.message
+            : undefined;
+        const text = textFromContent(message?.content);
+        if (text.length > 0)
+            return text;
+    }
+    return '';
+}
+function failureResult(startedAt, error) {
+    return {
+        ok: false,
+        output: null,
+        durationMs: Date.now() - startedAt,
+        durationApiMs: Date.now() - startedAt,
+        inputTokens: 0,
+        outputTokens: 0,
+        cacheReadTokens: 0,
+        cacheCreationTokens: 0,
+        tokenUsageReportedByExecutor: false,
+        costUSD: 0,
+        costReportedByExecutor: false,
+        stopReason: 'error',
+        numTurns: 0,
+        error: errorMessage(error),
+    };
+}
+function createPromptMessage(prompt) {
+    // `source.kind` mirrors the host-owned DSH message protocol and must retain its wire name.
+    return Object.freeze({
+        id: randomUUID(),
+        role: 'user',
+        content: [Object.freeze({ type: 'text', text: prompt })],
+        source: Object.freeze({ kind: 'user' }),
+    });
+}
+async function settleWithin(promise, timeoutMs) {
+    let timer;
+    const settled = await Promise.race([
+        promise.then(() => true),
+        new Promise((resolve) => {
+            timer = setTimeout(() => resolve(false), timeoutMs);
+        }),
+    ]);
+    if (timer !== undefined)
+        clearTimeout(timer);
+    return settled;
+}
+async function waitForIdle(agent, timeoutMs, signal, cleanupTimeoutMs) {
+    if (signal?.aborted) {
+        agent.cancel({ kind: 'hook', reason: 'OMK evaluation command was aborted' });
+        const settled = await settleWithin(agent.whenIdle(), cleanupTimeoutMs);
+        return { outcome: 'aborted', cleanupTimedOut: !settled };
+    }
+    let timer;
+    let removeAbortListener;
+    const outcome = await Promise.race([
+        agent.whenIdle().then(() => 'idle'),
+        new Promise((resolve) => {
+            timer = setTimeout(() => resolve('timeout'), timeoutMs);
+        }),
+        ...(signal === undefined ? [] : [new Promise((resolve) => {
+                const onAbort = () => resolve('aborted');
+                signal.addEventListener('abort', onAbort, { once: true });
+                removeAbortListener = () => signal.removeEventListener('abort', onAbort);
+            })]),
+    ]);
+    if (timer !== undefined)
+        clearTimeout(timer);
+    removeAbortListener?.();
+    if (outcome === 'idle')
+        return { outcome, cleanupTimedOut: false };
+    agent.cancel({
+        kind: 'hook',
+        reason: outcome === 'timeout'
+            ? `OMK sample timed out after ${timeoutMs}ms`
+            : 'OMK evaluation command was aborted',
+    });
+    const settled = await settleWithin(agent.whenIdle(), cleanupTimeoutMs);
+    return { outcome, cleanupTimedOut: !settled };
+}
+function assertSupportedIsolation(input) {
+    if (input.allowedSkills && input.allowedSkills.length > 0) {
+        throw new Error('dsh-host executor 不支持非空 skill 白名单；请注入待测 artifact，并使用 allowedSkills: [] 隔离环境 skill。');
+    }
+}
+/**
+ * Build an OMK executor that creates fresh agents inside an already-running
+ * DSH plugin tree. The host keeps ownership of credentials, tools, policies,
+ * and persistence; OMK owns sample isolation and result projection.
+ */
+export function createDshHostExecutor(ctx, options = {}) {
+    const provider = options.provider ?? options.parentAgent?.options.provider;
+    const activeAgentPreset = options.parentAgent
+        ? ctx.agentPresets.composedPreset(options.parentAgent.ctx)
+        : undefined;
+    // Preserve host schema order because DSH sends this order to the model; it is part of runtime identity.
+    const toolSchemas = ctx.tools?.schemas(options.parentAgent)
+        .filter((schema) => schema.name !== 'skill');
+    const runtimeFingerprint = (model) => (createDshHostRuntimeFingerprint(model, {
+        ...(provider ? { provider } : {}),
+        ...(activeAgentPreset
+            ? { agentPreset: activeAgentPreset }
+            : {}),
+        ...(toolSchemas ? { toolSchemas } : {}),
+    }));
+    const executor = async (input) => {
+        const startedAt = Date.now();
+        let handle;
+        const rootSessionId = `omk-${randomUUID()}`;
+        const descendants = new Set();
+        const orderedEvents = [];
+        const disposeCreated = ctx.on('session/created', (session) => {
+            const parent = session.header.parentSession;
+            if (parent !== rootSessionId && (parent === undefined || !descendants.has(parent)))
+                return;
+            descendants.add(String(session.id));
+        });
+        const disposeEvents = ctx.on('session/event', (session, event) => {
+            const sessionId = String(session.id);
+            if (sessionId === rootSessionId) {
+                orderedEvents.push({ sessionId, event, traceRole: 'main' });
+                return;
+            }
+            if (descendants.has(sessionId)) {
+                orderedEvents.push({ sessionId, event, traceRole: 'subagent' });
+            }
+        });
+        try {
+            assertSupportedIsolation(input);
+            const cwd = input.cwd ?? process.cwd();
+            handle = await ctx.agents.create({
+                sessionId: rootSessionId,
+                meta: {
+                    cwd,
+                    ...(options.parentAgent ? { parentSession: String(options.parentAgent.id) } : {}),
+                    ...(activeAgentPreset ? { agentPreset: activeAgentPreset } : {}),
+                },
+                agentOptions: {
+                    ...(provider ? { provider } : {}),
+                    model: input.model,
+                },
+                setup(agentCtx) {
+                    if (options.parentAgent) {
+                        const composedPreset = ctx.agentPresets.composeFrom(agentCtx, options.parentAgent.ctx);
+                        if (composedPreset !== activeAgentPreset) {
+                            throw new Error(`DSH agent preset changed during measurement setup: ${activeAgentPreset ?? 'none'} -> ${composedPreset ?? 'none'}`);
+                        }
+                    }
+                    agentCtx.systemPrompt.section({
+                        name: 'omk:evaluation',
+                        order: 0,
+                        text: input.system ?? '',
+                        complete: true,
+                    });
+                    agentCtx.systemPrompt.suppressRuntimeContext();
+                    const scopedAgent = agentCtx.agent;
+                    if (agentCtx.tools?.get('skill', scopedAgent) !== undefined) {
+                        agentCtx.tools.restrict({ deny: ['skill'] });
+                    }
+                },
+            });
+            handle.agent.followup(createPromptMessage(input.prompt));
+            const timeoutMs = input.timeoutMs ?? DEFAULT_TIMEOUT_MS;
+            const cleanupTimeoutMs = options.cleanupTimeoutMs ?? DEFAULT_CLEANUP_TIMEOUT_MS;
+            const { outcome, cleanupTimedOut } = await waitForIdle(handle.agent, timeoutMs, options.signal, cleanupTimeoutMs);
+            const wallClockDurationMs = Date.now() - startedAt;
+            const events = [...handle.agent.session.events];
+            const result = buildDshHostResult({
+                rootSessionId,
+                finalResponse: lastAssistantText(events),
+                events: orderedEvents,
+                childSessionIds: [...descendants],
+            }, wallClockDurationMs);
+            if (outcome === 'idle')
+                return result;
+            return {
+                ...result,
+                ok: false,
+                stopReason: outcome,
+                error: outcome === 'timeout'
+                    ? `dsh-host execution timed out after ${timeoutMs}ms${cleanupTimedOut ? `; cancellation did not settle within ${cleanupTimeoutMs}ms` : ''}`
+                    : `dsh-host execution aborted by its DSH command${cleanupTimedOut ? `; cancellation did not settle within ${cleanupTimeoutMs}ms` : ''}`,
+            };
+        }
+        catch (error) {
+            return failureResult(startedAt, error);
+        }
+        finally {
+            disposeEvents();
+            disposeCreated();
+            if (handle !== undefined) {
+                const disposal = handle.dispose().catch((error) => {
+                    process.stderr.write(`[dsh-host] 评测 session 关闭失败：${errorMessage(error)}\n`);
+                });
+                const disposed = await settleWithin(disposal, options.cleanupTimeoutMs ?? DEFAULT_CLEANUP_TIMEOUT_MS);
+                if (!disposed) {
+                    process.stderr.write('[dsh-host] 评测 session 关闭超出清理宽限期，已停止等待。\n');
+                }
+            }
+        }
+    };
+    return Object.assign(executor, { runtimeFingerprint });
+}

package/dist/dsh-plugin/index.d.ts

@@ -0,0 +1,28 @@
+import { type DshAgentLike, type DshHostContextLike } from './host-executor.js';
+interface DshCommandInvocationLike {
+    readonly agent: DshAgentLike;
+    readonly rawInput: string;
+    readonly signal: AbortSignal;
+}
+type DshCommandResultLike = Readonly<Record<'kind', 'success'> & {
+    text?: string;
+}> | Readonly<Record<'kind', 'error'> & {
+    text: string;
+}>;
+interface DshPluginContextLike extends DshHostContextLike {
+    readonly commands: {
+        register(definition: {
+            readonly name: string;
+            readonly description: string;
+            readonly input: {
+                readonly hint: string;
+            };
+            readonly handler: (invocation: DshCommandInvocationLike) => DshCommandResultLike | Promise<DshCommandResultLike>;
+        }): () => void;
+    };
+}
+export declare const name = "omk-dsh-plugin";
+export declare const inject: string[];
+/** Register `/omk eval <eval.yaml>` in every DSH command-capable surface. */
+export declare function apply(ctx: DshPluginContextLike): void;
+export {};