@dsh-enhanced/assistant-evaluation 0.1.23 → 0.1.30

package/README.md

@@ -77,9 +77,9 @@ projection 保存选中的 execution/objective/delivery component id，原始两
 ## 权限与数据边界
 
 - **文件系统：** 创建数据库父目录并写入本地 SQLite；新数据库目录使用 `0700`、数据库使用 `0600`。现有数据库若为符号链接、硬链接、非当前用户所有或对 group/other 开放，会拒绝启动。数据库启用 WAL、`synchronous=FULL`、迁移版本检查和 5 秒 busy timeout。
-- **网络：** 无。
-- **子进程：** 无。
-- **凭据：** 无。
+- **网络：** 默认 Evaluation 插件无网络访问；显式调用 experimental benchmark SDK 时，模型 delegate 或独立 holdout authority 是否联网由可信 Host/operator 的部署决定，SDK 本身只使用进程内调用或 stdio transport。
+- **子进程：** 默认 Evaluation 插件不创建子进程；显式调用 `./benchmark/holdout` 的 `openHoldoutProvider()` 才会启动 operator 指定的 authority executable。
+- **凭据：** 默认 Evaluation 插件不读取凭据；holdout provider 只收到 operator 显式给出的精确 environment，不继承 Host 环境，delegate 所需凭据仍由可信 Host 自行装配和撤销。
 - **浏览器：** 无。
 - **install script：** 无；仅有标准 TypeScript build/prepack 和仓库发布保护脚本。
 
@@ -92,6 +92,112 @@ scope、situation、producer/evaluator id、证据引用和指标属于本地评
 - scope 路径做绝对路径的语法规范化，不主动访问文件系统解析符号链接。
 - 当前面向单机个人助理，SQLite 依赖仓库兼容基线规定的 Node.js 版本。
 
+## 配对评测 Host SDK（实验性）
+
+`@dsh-enhanced/assistant-evaluation/benchmark` 提供 `parseBenchmarkPlan`、`benchmarkPlanDigest`、`benchmarkSchedule`、`BenchmarkStore`、`runBenchmark` 和 `benchmarkReport`。这是可独立导入的 Host 集成入口，不自动启用，也不注册模型工具。它使用单独的 benchmark SQLite 文件，不改变日常任务评价数据库。
+
+计划冻结数据集版本与摘要、逐题输入/验收摘要、模型、prompt、skills、tools、policy 和 runtime 的 SHA-256 配置/内容摘要。模型摘要应覆盖实际 provider、具体模型版本、采样参数和计价配置，不能只记录别名。能力比较要求模型相同；模型比较只能改变模型配置。消融必须与唯一候选保持版本一致，并只关闭记忆、规划、复核或成长中的一项。变体共享完全相同的预算与逐题 seed；执行顺序轮换。seed 记录不意味着模型服务一定支持确定性采样，adapter 必须如实记录提供商能力。
+
+可信 Host 先构造 `BenchmarkPlan`，创建 `new BenchmarkStore(absolutePrivateDatabasePath)`，再调用 `await runBenchmark(store, plan, executor, signal)`，用 `benchmarkReport(plan, store.results(plan.id))` 获取报告，最后关闭 store。`executor.execute(request)` 必须实际创建隔离且等价的任务环境，按冻结配置调用原生 AgentLoop、限制计量预算、收集 Host 计量、等待资源停止并使用独立 verifier 判定结果。请求只含计划中的任务身份及摘要，输入和答案由可信数据集提供方分别交给执行器与验收器；不得让模型自行填写 observation。控制器对版本/摘要漂移、缺失预算计量、超预算与未停止的执行返回 unknown。
+
+每个 cell 在执行前持久保存运行意图。相同计划可继续尚未开始的 cell，但完成的 cell 不再执行；同 ID 不接受修改后的计划或结果。启动时遇到运行中意图会拒绝，不能因为进程重启就重放可能有副作用的任务。管理员可以用 `store.interrupt(planId, timestamp)` 将遗留意图记为 interrupted/unknown；这不会停止实际进程，也不会恢复该计划的后续执行。先确认旧执行已停止，再创建新的完整比较计划，不能选择性重跑失败题来提高分数。
+
+超时和取消发出 AbortSignal 后有界返回，迟到结果不能改写账本；unknown 传输或生命周期状态阻止后续 cell。该机制不能强制终止同 UID 任意代码、子进程或远端调用，实际撤销/隔离仍由 Host adapter 和独立 broker 实现。数据库文件要求当前用户拥有、私有权限、非符号链接/硬链接；这不是对同 UID 对手的隔离。
+
+报告保留所有计划 cell 作为已验证成功率分母，unknown 和缺测单列，缺失费用/token/返工/人工介入量不会补成零。提供配对胜负、差值和均值/中位数/P95；区间按任务聚类 bootstrap，同题重复不当作独立任务。任一比较臂有缺测或 unknown 就不报告收益差值与区间，不能把基线的未知结果当作失败来制造增益；单臂有 unknown 也不报告其成功率区间。少于两个任务不提供区间。小样本或同质任务仍不足以证明泛化收益，报告始终 `promotionAuthorized: false`。
+
+**当前完成范围：** 冻结协议、持久账本、有界协调器、统计计算、原生 AgentLoop 执行器及 `dsh-benchmark` 命令。默认 `research-v1` 保留 8 道公开研究/注入开发题；`memory-v2` 增加 6 道公开记忆开发题。`strategy-v1` 增加 4 道公开合成 shell 开发题：整数汇总、接触区间合并、按频次/字典序词频和依赖拓扑排序。它们是开发语料，不是 holdout，也没有真实模型收益证据。
+
+### 独立留出 provider Host SDK（实验性）
+
+`@dsh-enhanced/assistant-evaluation/benchmark/holdout` 是 inert Host SDK：导入它不会挂载 Cordis 插件、启动 provider、创建数据库或注册模型工具。普通 `runIndependentHoldout()` 也只在可信 Host 显式调用时运行。若调用发生在插件生命周期内，应使用 `runIndependentHoldoutInContext(ctx, options)`；它把 benchmark journal、evidence store、authority/delegate session 和取消控制注册为当前 Cordis Fiber 的一个 owned effect。Fiber 卸载会停止新准入、abort 运行、等待有界清理并关闭两个 store；自然完成也会关闭资源并移除该 effect。`providerTimeoutMs` 为任意 Host provider 的启动、每个请求和关闭设置上限；在 abort 或期限后才解析的 provider 会只做一次有界 `close()`，不会再请求 manifest 或写入 store。它没有把 `ctx.isolate()` 当作进程、凭据或数据集隔离。
+
+operator 必须同时固定 `{ id, version, digest }` 数据集 pin 和 Ed25519 authority public key。authority 按顺序返回四类严格 envelope：
+
+- `manifest` 签名公开 dataset/case identity 及 input/acceptance commitments；
+- `input` 签名并绑定 exact manifest、plan、cell、input digest 和本次临时输入 bytes；
+- `verdict` 签名并绑定 exact input/acceptance/output digest、plan、cell 与判定；
+- `finish` 签名完整 cell 数和按计划顺序排列的 signed-verdict envelope digest 链。
+
+Host 不接受 authority 自报的资源用量：delegate 必须返回与冻结 variant 一致的版本、Host 侧 metrics、execution evidence digest 和 quiescence，runner 再核对每 cell 预算；墙钟 latency 由 runner 计量。这里的 Host 计量仍是可信同进程 adapter 的自证，不是独立外部 meter。`acceptanceDigest` 只承诺验收材料没有被事后替换，不加密低熵答案，也不能证明该材料在候选生成前已经冻结。
+
+私有 input 只在当前 delegate 调用期间存在，raw output 只用于本次 verdict 请求；SDK 在结算后尽力清零持有的 byte arrays。Benchmark SQLite、content-addressed evidence 与报告只保存 digests、Host metrics、执行证据引用以及 signed manifest/verdict/finish，不保存 signed input envelope、raw input 或 raw output。这个约束只覆盖 Host SDK 的持久化面，不能证明外部 authority、delegate、模型服务或同 UID 进程没有另行复制数据。
+
+恢复语义保持保守：完整 journal 加不可变 completion marker 可在不启动 authority/delegate 的情况下重验；已有 unknown 直接保持终态。completed prefix、遗留 running intent、缺失/漂移的 finish marker 都在 provider startup 前失败关闭，v1 不选择性续跑。若最后一个 cell 已发布 finish marker、但进程在 SQLite cell 终态提交前崩溃，重启仍看到 running intent；operator 必须先确认旧执行停止，再调用 `BenchmarkStore.interrupt()` 将其记为 unknown，不能用 marker 自动升级为成功。
+
+内置 provider transport 使用单请求 NDJSON、固定 `cwd: /`、完全显式的 child environment、长度/时间上限和 TERM→KILL 有界关闭。POSIX 关闭同时等待 leader/stdio 关闭和当前数值进程组消失，Windows 只管理直接 child 与其 stdio；继承管道仍未关闭时会有界报 `termination-unconfirmed`。普通 PID/PGID 信号不能约束脱离进程组的后代，也不能排除数值复用导致误发信号，因而不是无竞态进程身份或强 OS containment 证明。executable 只做绝对 canonical pathname 校验，没有 fd/inode 绑定，因此 pathname 在校验与 `spawn` 之间被替换不在当前保证内。现有真实子进程回归使用同 UID synthetic authority fixture；它验证 transport、签名链、无 raw I/O 持久化、零-spawn 重验和 fail-closed 恢复，不证明独立账号/机器/UID 部署、历史冻结透明度或真实隐藏性。目前也没有真实模型、真实独立 holdout 或 live benefit 证据，WP04 继续为实现中。
+
+### 原生策略比较的 Host 接口
+
+`strategy-v1` 已由 `dsh-benchmark corpus|doctor|plan|run|report` 支持。它冻结 `direct` 和 `adaptive-strategy` 两分支、策略执行限额、已安装能力来源与任务/验收摘要；每个 cell 在私有 state 目录保存可重读的详细观察和摘要。模型只能收到公开 objective、提示和示例；独立 verifier 的向量不进入 prompt。
+
+`parseStrategyBenchmarkPlan` 固定两个分支 `direct` / `adaptive-strategy`、共同预算、模型调用上限、单次输出和原生 Goal 轮数。共同 persona/工具/Policy/runtime 与策略 guide/tool/Policy/runtime 分别声明摘要，并派生两分支版本；任意额外版本差异会被拒绝。parser 验证声明一致，实际 executor 另外核对挂载的能力。`strategyBenchmarkJournalPlan` 将完整策略契约绑定到既有 journal，修改限额不能复用旧计划 ID。调用 `installStrategyBenchmarkRequestMeter` 会先核对 exact cell/计划/模型，再安装外层限额；executor 将同一 `maxGoalRounds` 交给真实 Goal。
+
+计量器必须在首次入站前安装在专用 Context，覆盖前台、父 Goal 和 child 的全部 `llm/stream` 与工具执行。仅接受可信输入上界和供应商输出上限；estimate/observed 配置显式拒绝。模型流在 dispatch 前共享原子预留，结束后根据完整 usage 结算；取消、缺 usage、异常和提前关闭保留预留，`assertComplete()` 拒绝未结清、空测量或尚在运行的工具。费用为 null 时只约束 token，不声称金额硬限。快照是进程内观察；executor 将它与计划/cell 绑定、持久保存，并与独立验收和资源停止证据核对。取得下游迭代器的 `dispatched` 不是实际付费 HTTP 证明。
+
+`createBenchmarkStrategyOwnerRuntime` 要求全新 Context、两个现存且独立的 canonical 私有目录：候选 workspace 和其外部 stateRoot。它在 stateRoot 独占创建 runtimeRoot，写入 JSONL Session、Delivery/Policy SQLite 及 spool；显式保存空会话头，使原生 Delivery 创建后可以真实恢复。它只配对本地合成 owner、注册本地回复捕获通道；该身份只用于评估，不是对任何真实用户/外部服务的授权，不发现或修改用户 profile，也不发外部通知。只暴露指定工具 allow-list，额外 Policy 规则由可信 Host 提供。模型 adapter 则拥有操作者声明的网络/凭据权限，由调用者安装和销毁。
+
+executor 在实际每个父请求上比较完整 persona、每个实际工具 schema，并通过 `assistantPolicy.inspectHostConfiguration()` 比较完整 Policy 配置；同时记录已解析安装源码的身份摘要。该检查发现计划与运行时的普通漂移，但不构成对同进程恶意 Host 代码或远端 provider 实现的形式证明。全流程 deadline 覆盖装配、调用、验证和关闭；不能证明停止、usage 或证据完整性时结果保持 `unknown`，不会重放 cell 或启动后续 cell。
+
+### 开发集命令
+
+```sh
+dsh-benchmark doctor
+dsh-benchmark corpus
+dsh-benchmark doctor --suite memory-v2
+dsh-benchmark corpus --suite memory-v2
+dsh-benchmark corpus --suite strategy-v1
+dsh-benchmark doctor --suite strategy-v1 --config ./docs/examples/strategy-benchmark.config.json
+dsh-benchmark plan --config ./docs/examples/strategy-benchmark.config.json --output ./private/strategy-plan.json
+dsh-benchmark run --config ./docs/examples/strategy-benchmark.config.json --adapter /absolute/trusted-adapter.mjs --database ./private/strategy.sqlite --output ./private/strategy-report.json
+dsh-benchmark report --database ./private/strategy.sqlite --plan strategy-public-development --config ./docs/examples/strategy-benchmark.config.json --output ./private/strategy-report-copy.json
+```
+
+`doctor --suite strategy-v1 --config` 会在不安装依赖、不拉取镜像的前提下检查 Host packages，并执行有界的离线 Docker/image probe；ready 不证明模型凭据或 provider 可用。`plan` 不调用模型。`run` 只装载操作者指定的 trusted adapter，费用和 token 限额来自配置；adapter 入口的 SHA-256 必须同时匹配 `adapterDigest` 与 `tokenCounterDigest`。策略 suite 固定 `inputLimitMode: "upper-bound"` 和 `outputLimitMode: "provider"`；现有 Codex 订阅 adapter 的 estimate/observed 模式不兼容策略运行。`report` 重新打开已保存的 cell 观察和摘要，并从 `--config` 重建当前 plan 进行比对；它不重新激活已关闭 runtime 或重放模型调用。
+
+配置示例、计量协议和限制见 [评测实施文档](../../docs/benchmark-implementation.md)。预算为每 cell 上限；金额单位为美元的百万分之一，计价为每百万 tokens 对应的该单位。`costUsdMicros: null` 明确选择仅 token 预算，模型输入/输出价格也可同时为 null；报告中的未知费用不会填成零。声明金额上限时必须提供非缓存输入、cache read、cache write、输出四类费率，且必须使用输入上界和提供商输出上限；输入预留采用三类输入的最高费率。DSH 的非缓存 input 与两类 cache 相加为实际输入，reasoning 已包含在 output 中，不重复计费。实际使用了缺价的 cache 时，费用保持 null。可信适配器须关闭隐藏自动重试。
+
+非策略原生 suite 仍可使用 estimate/observed 的 token-only 配置；这不扩展到 strategy-v1。策略运行创建私有 workspace/state 根并持久化证据，日常 profile 不会安装、启用或加载这些可选 Host peers。
+
+命令使用网络/凭据的权限由可信模型适配器决定；自身读取配置和模块，写独立私有 SQLite 与独占新建的 JSON 报告，创建并清理临时工作目录，不安装提供商、不修改日常 profile。报告不包含模型原始答案，判定绑定答案哈希与开发集验收摘要。`split: holdout` 只声明用途，同 UID 文件或公开仓库中的题目/答案不属于安全隐藏留出。
+
 ## 兼容性
 
+Goals 的独立 Verifier v2 回执使用 `goal-step` 任务身份，按 `goal:<goal-id>:definition:<version>` 归因；同一 ref 的前台或 Automation 任务不会与之合并。schema 10 迁移保留既有投影、证据与视图。原有 owner 反馈修订入口仍只支持前台和 Automation，目标步骤成功也不自动宣告整个长期目标完成。
+
 对齐仓库的 [DSH / Cordis 兼容性基线](../../docs/compatibility.md)。Host 提供 Cordis、Agent 和 ToolRuntime；插件自身携带 Schemastery。
+
+Host 可通过 `getTrustedAutomationRunLearningProjection({ scope, runId })` 精确读取一个 run 的 ready canonical learning projection，无需扫描受 limit 限制的原始 audit 列表。不存在或 objective-conflict 时不返回证明；返回值沿用同一 canonical projection、revision、digest 和 scope watermark。调用方须检查 disposition 和执行/目标状态，并在产生依赖该证据的写入时使用 `withTrustedLearningWriterFence`。
+
+`withTrustedCanonicalLearningWriterFence` 为依赖 Evaluation canonical 状态本身的 Host 写入提供同样的 scope watermark、精确 task tuple 和同步写锁校验，但不要求投递给 Evolution 的 outbox 已完成。`withTrustedLearningWriterFence` 继续为 Evolution 依赖方保留 projection-pending gate；两者都不会代替或伪造 outbox 投递。
+
+`getTrustedGoalOutcomeLearningProjection({ scope, assessmentId })` 是 whole-goal 消费者的 Host-only 精确读取入口。它只接受 `canonicalEvaluationHostScope()` 产生的 token，并且只查询相同 canonical workspace/preset 下 subject 恰为 `goal-outcome/<assessmentId>` 的当前 projection；缺失或跨 scope 返回 `undefined`。返回的 `version`、`digest`、`disposition` 与 scope watermark 是 Evaluation 当前状态，不把原始 immutable Verifier receipt 永久视为事实。该入口只证明 assessment 的 canonical 状态；owner、Goal、Session、run 和 outcome profile 的关联仍须由调用方从 Goals 的当前 Host snapshot 独立证明。
+
+`withTrustedCanonicalTaskWriterFence({ scope, scopeWatermark, evidence }, callback)` 是对应的同步 commit fence。它逐项核对 exact subject kind/ref、revision、digest 和 `upsert | retract` disposition，并在 Evaluation writer lock 持有期间执行同步 callback；scope 内任何任务推进 watermark，或该 assessment 被纠正、撤回、替换，都会在 callback 运行前返回 `evidence-changed` / `watermark-changed`。它不等待可选 Evolution projection outbox，因此可让下游在 exact retract 下原子移除旧观察；异步 callback 会失败并回滚。原 `withTrustedCanonicalLearningWriterFence` 继续只接受可推广的 `upsert`，其契约不被放宽。
+
+### Owner outcome revisions
+
+Delivery's authenticated capability can explicitly correct or withdraw one exact delivered result. Schema 8 retains immutable raw outcomes, linked owner revisions, provider command receipts (including rejected CAS attempts), and a single canonical task projection. Owner lanes include principal record id and version. Only explicitly linked predecessors are superseded; independent contradictory owner evidence stays quarantined. Withdrawal is an authoritative `unknown` tombstone, so earlier terminal/evaluator success cannot reappear. The revision, digest, audit and projection outbox commit in the same SQLite transaction.
+
+Legacy schema 7 owner rows are adopted lazily through the exact Host delivery capability and stable initial idempotency key. Adoption verifies the run and Outbox references without rewriting raw history. Host consumers receive revision notifications and must also revalidate durable proof on startup / dispatch; a failed listener cannot prevent other consumers or the projection outbox from progressing.
+
+整体目标的独立 v3 回执使用 `goal-outcome/<assessmentId>` subject，与同 ref 的步骤、前台和 Automation 分离，仍按完整目标定义归因。schema 11 在事务内迁移旧 task projections 并保留历史。Delivery v2 producer 可携带 current Goals process-local capability，把 exact whole-goal Verifier receipt 惰性采纳为 owner revision 1；Evaluation 不信任 Delivery metadata，而会通过当前 Goals provider 重建 proof，并从当前 Verifier `inspectAcceptedTask()` 独立核对 owner/scope、Goal definition、Session/native Goal、assessment、run、profile、contract、receipt 及 digest。provider generation、capability 或任一绑定变化时 fail closed。
+
+同一 `goal-outcome/<assessmentId>` 随后使用既有 owner CAS 协议：`correct` 必须匹配当前 version 与 previous status，`withdraw` 写入 canonical `unknown` retract tombstone，重复命令幂等，乱序或同版本冲突保留拒绝审计而不覆盖 current projection。baseline adoption 在一个事务中把 exact immutable Verifier outcome 认领为 revision 1；如果 Evaluation 在原 receipt 产生时离线且恢复前 receipt 已过期，则只在当前 Goals opaque proof、当前 Verifier 原始 service identity/generation 以及 exact contract/receipt/execution 全部重验后，同构补写原 `assistant-verifier:<receiptId>` outcome、projection、outbox、watermark 与 revision 1，不续期、改写或伪造 receipt。后续每次 correction/retract 的新 owner revision、task projection、scope watermark 和 projection outbox 在一个 Evaluation SQLite writer transaction 中提交。因此 Skills 的 exact canonical monitor 会看到 v2 correction/retract，而不会把原始 immutable receipt 永久当作 achieved。历史回执可以在有效期后接受 owner 标注，但每次 issue/inspect/append 都必须由当前 Goals/Verifier 重验其不可变身份；这不是放宽发送前 freshness。
+
+仓库回归使用本地 Delivery/Goals/Verifier/Evaluation fixture 验证上述 Host 协议；没有连接真实 Lark、真实模型或生产 deployment cohort，不能写成真实渠道验收或长期收益。
+
+
+`memory-v1` 保留首轮公开试验的原始题目与判定，可按原计划复现。`memory-v2` 是单独版本，明确当前资料 ID、Memory provenance URI 与 `claim:<key>` 的引用约定；改进题目说明后必须建立新计划，不会重写 v1 的失败结果。
+
+策略 Host 接口新增 `createStrategyGoalRuntime`、`strategyGoalTaskDigests` 和 `StrategyEvidenceStore`。单 cell 真实装配原生 Goal/driver/subagents、Goals、Delivery、Isolation 与 Verifier；模型通过真实工具创建目标并导出隔离产物，独立验证向后续轮次提供反馈。运行结束后读取当前 owner 路由授权的持久目标快照，等待资源清理成功，再保存内容寻址的私有证据；证据包含每轮 step/outcome 的独立回执、真实 triggerRunId、策略子会话和外层计量，读取时重验 plan/cell/digest。清理超时保留 unknown，不能据此重放或启动下一 cell。
+
+此接口运行本机 Docker 和离线任意候选 shell：输入复制入容器，验证向量只供独立 runner，Host 持久状态保存在候选工作区之外。调用方必须提供受信任 adapter factory、已有私有目录、固定本机镜像及明确限额；不查找或修改用户 profile，不发送真实渠道消息。`execute`/`close` 清理有等待上限，但不合作资源可能仍未停止。adapter factory 与初始装配仍由调用方的整体期限控制，不能把本接口计量窗口视为完整比较墙钟期限。目录权限不隔离同 UID 的可信 Host 代码。
+
+策略 executor 会核对实际挂载工具、完整 persona、完整 Policy 配置、运行时来源和 adapter；这些观察连同全流程故障证据写入 journal 外的私有证据目录。普通 Evaluation bundle 不自动激活上述可选 Host peers，且尚无真实模型收益结论。
+
+### DeepSeek strategy adapter
+
+The optional `./benchmark/deepseek` export supplies the trusted `createNativeAdapter` entry for strategy-v1. Explicitly install the matching `assistant-deepseek-budget` Host peer and provide the `DEEPSEEK_API_KEY` credential reference in the current credentials service or process environment. The factory registers no services or profile configuration; disposal shuts down its adapter. It uses the existing fixed HTTPS production transport and sends public task context and tool messages to that provider. No subprocess, credential file scan, model-generated adapter or automatic retry is added.
+
+The input reserve is the production contract's conservative 2,097,152-token upper bound per request; prices remain unknown. Use the [DeepSeek example and instructions](../../docs/goal-strategy-evaluation.md#deepseek-生产模型入口), rather than the small generic fixture budget. Source commitments include the selected DeepSeek and credentials packages. Runtime readiness, online model access and measured strategy benefit remain separate evidence.

package/bin/dsh-benchmark.js

@@ -0,0 +1,21 @@
+#!/usr/bin/env node
+const controller = new AbortController()
+const abort = () => controller.abort()
+process.once('SIGINT', abort)
+process.once('SIGTERM', abort)
+try {
+  const [major, minor] = process.versions.node.split('.').map(Number)
+  if (major < 22 || (major === 22 && minor < 19) || major === 23) throw new Error('unsupported-node-version')
+  const { benchmarkCli } = await import('../lib/benchmark/cli.js')
+  process.exitCode = await benchmarkCli(process.argv.slice(2), { stdout: text => process.stdout.write(text) }, controller.signal)
+} catch (error) {
+  // Host adapter exceptions can contain API keys, request bodies or credential paths.
+  // Do not print third-party error messages or stacks through the operator CLI.
+  const { BenchmarkError } = await import('../lib/benchmark/schema.js').catch(() => ({ BenchmarkError: class {} }))
+  const safe = error instanceof BenchmarkError ? error.message : 'benchmark command failed; Node.js ^22.19 or >=24 is required; check runtime dependencies, file paths and trusted adapter configuration'
+  process.stderr.write(`dsh-benchmark: ${safe}\n`)
+  process.exitCode = 1
+} finally {
+  process.removeListener('SIGINT', abort)
+  process.removeListener('SIGTERM', abort)
+}

package/lib/benchmark/cli.d.ts

@@ -0,0 +1,6 @@
+export interface BenchmarkCliOutput {
+    stdout(text: string): void;
+}
+/** Returns an exit status; errors are intentionally reduced to non-sensitive diagnostics by the bin. */
+export declare function benchmarkCli(argv: readonly string[], io: BenchmarkCliOutput, signal?: AbortSignal): Promise<number>;
+//# sourceMappingURL=cli.d.ts.map

package/lib/benchmark/cli.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../../src/benchmark/cli.ts"],"names":[],"mappings":"AAmHA,MAAM,WAAW,kBAAkB;IAAG,MAAM,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;CAAE;AAElE,wGAAwG;AACxG,wBAAsB,YAAY,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,EAAE,EAAE,EAAE,kBAAkB,EAAE,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,MAAM,CAAC,CA6IzH"}

package/lib/benchmark/cli.js

@@ -0,0 +1,332 @@
+var __rewriteRelativeImportExtension = (this && this.__rewriteRelativeImportExtension) || function (path, preserveJsx) {
+    if (typeof path === "string" && /^\.\.?\//.test(path)) {
+        return path.replace(/\.(tsx)$|((?:\.d)?)((?:\.[^./]+?)?)\.([cm]?)ts$/i, function (m, tsx, d, ext, cm) {
+            return tsx ? preserveJsx ? ".jsx" : ".js" : d && (!ext || !cm) ? m : (d + ext + "." + cm.toLowerCase() + "js");
+        });
+    }
+    return path;
+};
+import { createHash } from 'node:crypto';
+import { constants } from 'node:fs';
+import { lstat, mkdir, open, writeFile } from 'node:fs/promises';
+import { createRequire } from 'node:module';
+import { dirname, isAbsolute, resolve } from 'node:path';
+import { pathToFileURL } from 'node:url';
+import { developmentCorpus, developmentDataset } from './corpus.js';
+import { memoryDevelopmentCorpus, memoryDevelopmentDataset, memoryDevelopmentCorpusV2, memoryDevelopmentDatasetV2 } from './memory-corpus.js';
+import { benchmarkReport } from './report.js';
+import { benchmarkPlanDigest, benchmarkSchedule, BenchmarkError } from './schema.js';
+import { BenchmarkStore } from './store.js';
+import { runBenchmark } from './runner.js';
+const runtimePackages = ['@deepseek-ai/cordis', '@deepseek-ai/dsh-agent', '@deepseek-ai/dsh-agent-loop',
+    '@deepseek-ai/dsh-llm', '@deepseek-ai/dsh-session', '@deepseek-ai/dsh-session-projection', '@deepseek-ai/dsh-system-prompt', '@deepseek-ai/dsh-tools'];
+const help = `dsh-benchmark: public development benchmarks through the native DSH AgentLoop
+  corpus [--suite research-v1|memory-v1|memory-v2|strategy-v1]
+  doctor [--suite research-v1|memory-v1|memory-v2|strategy-v1] [--config FILE]
+  plan --config FILE [--output FILE]
+  run --config FILE --adapter ABSOLUTE_MODULE --database FILE [--output FILE]
+  report --database FILE --plan ID [--config FILE] [--output FILE]
+
+plan never invokes a model. run invokes the explicitly supplied trusted Host adapter.
+The adapter module must export createNativeAdapter and match both configured SHA-256 digests.
+Outputs are created exclusively (existing files are never overwritten).
+No hidden holdout, credentials, or model-provider installation is created by these commands.
+`;
+function options(args, allowed, required) {
+    const result = new Map();
+    for (let index = 0; index < args.length; index += 2) {
+        const key = args[index];
+        const value = args[index + 1];
+        if (!allowed.includes(key) || result.has(key) || value === undefined || value.startsWith('--') || value.trim() === '')
+            throw new BenchmarkError('invalid or duplicate benchmark argument');
+        result.set(key, value);
+    }
+    if (required.some(key => !result.has(key)))
+        throw new BenchmarkError('missing required benchmark argument; use --help');
+    return result;
+}
+async function boundedFile(path, limit, allowHardlinks = false) {
+    const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK);
+    try {
+        const before = await handle.stat();
+        if (!before.isFile() || (!allowHardlinks && before.nlink !== 1) || before.size > limit)
+            throw new BenchmarkError('benchmark input must be a bounded regular file');
+        const bytes = Buffer.alloc(limit + 1);
+        let total = 0;
+        while (total < bytes.length) {
+            const { bytesRead } = await handle.read(bytes, total, bytes.length - total, total);
+            if (bytesRead === 0)
+                break;
+            total += bytesRead;
+        }
+        const after = await handle.stat();
+        if (total > limit || before.size !== after.size || before.mtimeMs !== after.mtimeMs || before.ctimeMs !== after.ctimeMs)
+            throw new BenchmarkError('benchmark input changed while being read');
+        const data = bytes.subarray(0, total);
+        return { text: data.toString('utf8'), digest: createHash('sha256').update(data).digest('hex') };
+    }
+    finally {
+        await handle.close();
+    }
+}
+async function nativeModule() {
+    try {
+        return await import('./native.js');
+    }
+    catch (error) {
+        if (error && typeof error === 'object' && 'code' in error && error.code === 'ERR_MODULE_NOT_FOUND') {
+            throw new BenchmarkError('native DSH runtime dependencies are missing; run dsh-benchmark doctor and install the reported host packages');
+        }
+        throw new BenchmarkError('native benchmark runtime could not be loaded');
+    }
+}
+async function config(path) {
+    const file = await boundedFile(resolve(path), 131_072);
+    try {
+        return JSON.parse(file.text);
+    }
+    catch {
+        throw new BenchmarkError('benchmark config must be valid JSON');
+    }
+}
+/** Loads only an operator-selected Host module, not code produced by a benchmark candidate. */
+async function adapter(path, input) {
+    if (!isAbsolute(path))
+        throw new BenchmarkError('adapter module path must be absolute');
+    const captured = await boundedFile(path, 1_048_576, true);
+    if (captured.digest !== input.model.adapterDigest || captured.digest !== input.model.tokenCounterDigest)
+        throw new BenchmarkError('adapter module does not match frozen adapter/token-counter digests');
+    // Trusted installed adapters may be pnpm hardlinks. Config/output files still require one link.
+    // Digest and post-load check detect ordinary replacement; same-UID filesystem attacks and
+    // transitive module imports are outside this trusted Host extension boundary.
+    const imported = await import(__rewriteRelativeImportExtension(`${pathToFileURL(path).href}?benchmark=${captured.digest}`));
+    if ((await boundedFile(path, 1_048_576, true)).digest !== captured.digest)
+        throw new BenchmarkError('adapter module changed during import');
+    if (!imported || typeof imported !== 'object' || !('createNativeAdapter' in imported) || typeof imported.createNativeAdapter !== 'function')
+        throw new BenchmarkError('adapter module must export createNativeAdapter');
+    return imported.createNativeAdapter;
+}
+const strategyRuntimePackages = [...runtimePackages, '@deepseek-ai/dsh-goal', '@deepseek-ai/dsh-tool-goal', '@deepseek-ai/dsh-goal-round-driver',
+    '@deepseek-ai/dsh-subagent', '@deepseek-ai/dsh-session-persistence', '@deepseek-ai/dsh-session-persistence-jsonl', '@dsh-enhanced/assistant-delivery',
+    '@dsh-enhanced/assistant-policy', '@dsh-enhanced/assistant-goals', '@dsh-enhanced/assistant-isolation', '@dsh-enhanced/assistant-verifier'];
+async function strategyModules() {
+    try {
+        return await Promise.all([import('./strategy-config.js'), import('./strategy-corpus.js'), import('./strategy-executor.js'), import('./strategy-plan.js')]);
+    }
+    catch {
+        throw new BenchmarkError('strategy benchmark runtime dependencies are missing; run dsh-benchmark doctor and install the reported host packages');
+    }
+}
+async function strategyConfig(value) {
+    const [module] = await strategyModules();
+    return module.parseStrategyBenchmarkConfig(value);
+}
+function isStrategyConfig(value) {
+    return value !== null && typeof value === 'object' && value.suite === 'strategy-v1';
+}
+async function ensureNewOutput(path) {
+    if (path === undefined)
+        return;
+    try {
+        await lstat(resolve(path));
+    }
+    catch (error) {
+        if (error && typeof error === 'object' && 'code' in error && error.code === 'ENOENT')
+            return;
+        throw error;
+    }
+    throw new BenchmarkError('output path already exists; choose a new output file');
+}
+/** Returns an exit status; errors are intentionally reduced to non-sensitive diagnostics by the bin. */
+export async function benchmarkCli(argv, io, signal) {
+    const command = argv[0];
+    if (command === undefined || command === '--help' || command === 'help') {
+        io.stdout(help);
+        return 0;
+    }
+    let result;
+    let output;
+    let exitCode = 0;
+    if (command === 'corpus') {
+        const args = options(argv.slice(1), ['--suite'], []);
+        const suite = args.get('--suite') ?? 'research-v1';
+        if (suite === 'strategy-v1') {
+            const [, corpus] = await strategyModules();
+            result = { dataset: corpus.strategyDevelopmentDataset, tasks: corpus.strategyDevelopmentCorpus.map(task => ({ id: task.id, domain: task.domain, objective: task.objective })) };
+        }
+        else {
+            if (!['research-v1', 'memory-v1', 'memory-v2'].includes(suite))
+                throw new BenchmarkError('invalid native suite');
+            result = { dataset: suite === 'memory-v2' ? memoryDevelopmentDatasetV2 : suite === 'memory-v1' ? memoryDevelopmentDataset : developmentDataset, tasks: (suite === 'memory-v2' ? memoryDevelopmentCorpusV2 : suite === 'memory-v1' ? memoryDevelopmentCorpus : developmentCorpus).map(task => ({ id: task.id, domain: task.domain, objective: task.objective })) };
+        }
+    }
+    else if (command === 'doctor') {
+        const args = options(argv.slice(1), ['--suite', '--config'], []);
+        const suite = args.get('--suite') ?? 'research-v1';
+        const require = createRequire(import.meta.url);
+        if (suite === 'strategy-v1') {
+            const configPath = args.get('--config');
+            const configured = configPath === undefined ? undefined : await config(configPath);
+            const provider = configured?.model?.provider;
+            const selectedPackages = [...strategyRuntimePackages, ...(provider === 'deepseek-goal-metered' ? ['@dsh-enhanced/assistant-deepseek-budget', '@deepseek-ai/dsh-credentials'] : [])];
+            const packages = selectedPackages.map(name => {
+                try {
+                    require.resolve(name);
+                    return { name, available: true };
+                }
+                catch {
+                    return { name, available: false };
+                }
+            });
+            if (configPath === undefined) {
+                result = { ready: false, packages, isolation: { ready: false }, next: 'Provide --config to probe the immutable local image and Docker runtime. Dependencies alone do not establish strategy readiness.' };
+                exitCode = 2;
+            }
+            else if (!packages.every(entry => entry.available)) {
+                result = { ready: false, packages, isolation: { ready: false }, next: 'Install the unavailable strategy runtime packages before probing Docker or the configured image.' };
+                exitCode = 2;
+            }
+            else {
+                const input = await strategyConfig(configured);
+                let isolationReady = false;
+                try {
+                    const isolationPackage = '@dsh-enhanced/assistant-isolation';
+                    const isolation = await import(__rewriteRelativeImportExtension(isolationPackage));
+                    await isolation.probeIsolationRuntime(input.image, input.dockerPath);
+                    isolationReady = true;
+                }
+                catch { /* Keep diagnostics non-sensitive and do not attempt installation or image pulls. */ }
+                result = { ready: packages.every(entry => entry.available) && isolationReady, packages, isolation: { ready: isolationReady },
+                    next: 'A ready runtime does not prove model access. Configure a trusted adapter, token counter and frozen strategy limits.' };
+                exitCode = result.ready ? 0 : 2;
+            }
+        }
+        else {
+            if (!['research-v1', 'memory-v1', 'memory-v2'].includes(suite))
+                throw new BenchmarkError('invalid native suite');
+            const names = [...runtimePackages, ...(suite !== 'research-v1' ? ['@dsh-enhanced/personal-memory', '@dsh-enhanced/assistant-policy'] : [])];
+            const packages = names.map(name => {
+                try {
+                    require.resolve(name);
+                    return { name, available: true };
+                }
+                catch {
+                    return { name, available: false };
+                }
+            });
+            result = { ready: packages.every(entry => entry.available), packages, next: 'Configure a trusted adapter, model token counter and per-cell budget. A ready runtime does not prove model access.' };
+            exitCode = packages.every(entry => entry.available) ? 0 : 2;
+        }
+    }
+    else if (command === 'plan' || command === 'run') {
+        const fields = options(argv.slice(1), ['--config', '--adapter', '--database', '--output'], command === 'run' ? ['--config', '--adapter', '--database'] : ['--config']);
+        if (command === 'plan' && (fields.has('--adapter') || fields.has('--database')))
+            throw new BenchmarkError('plan accepts only --config and --output');
+        output = fields.get('--output');
+        await ensureNewOutput(output);
+        const raw = await config(fields.get('--config'));
+        if (isStrategyConfig(raw)) {
+            const input = await strategyConfig(raw);
+            const [, corpus, executor, strategyPlan] = await strategyModules();
+            const plan = executor.createStrategyBenchmarkPlan(input);
+            const journal = strategyPlan.strategyBenchmarkJournalPlan(plan);
+            const cells = benchmarkSchedule(journal);
+            if (command === 'plan') {
+                result = { plan, journalPlan: journal, planDigest: benchmarkPlanDigest(journal), plannedCells: cells.length,
+                    maximumCostUsdMicros: journal.budget.costUsdMicros === null ? null : (BigInt(cells.length) * BigInt(journal.budget.costUsdMicros)).toString(),
+                    inputLimitMode: 'upper-bound', outputLimitMode: 'provider', maximumInputTokens: cells.length * journal.budget.inputTokens,
+                    maximumOutputTokens: cells.length * journal.budget.outputTokens, observedInputTokenLimit: cells.length * journal.budget.inputTokens,
+                    observedOutputTokenLimit: cells.length * journal.budget.outputTokens };
+            }
+            else {
+                const store = new BenchmarkStore(resolve(fields.get('--database')));
+                try {
+                    store.create(journal);
+                    const factory = await adapter(fields.get('--adapter'), input);
+                    const tasks = Object.fromEntries(input.cases.map(caseId => [caseId, corpus.strategyDevelopmentTask(caseId)]));
+                    const strategyExecutor = executor.createStrategyBenchmarkExecutor({ plan, tasks, persona: input.persona, model: input.model, factory,
+                        workspaceDirectory: input.workspaceDirectory, stateDirectory: input.stateDirectory, image: input.image, dockerPath: input.dockerPath,
+                        stepMaxDurationMs: input.stepMaxDurationMs, ...(input.stopTimeoutMs === undefined ? {} : { stopTimeoutMs: input.stopTimeoutMs }) });
+                    const results = await runBenchmark(store, journal, strategyExecutor, signal);
+                    executor.verifyStrategyBenchmarkResults(plan, results, input.stateDirectory, input.workspaceDirectory);
+                    const report = benchmarkReport(journal, store.results(journal.id));
+                    result = report;
+                    exitCode = report.complete && report.variants.every(variant => variant.unknown === 0) ? 0 : 2;
+                }
+                finally {
+                    store.close();
+                }
+            }
+        }
+        else {
+            const input = raw;
+            const native = await nativeModule();
+            const plan = native.nativeBenchmarkPlan(input);
+            if (command === 'plan') {
+                result = { plan, planDigest: benchmarkPlanDigest(plan), plannedCells: benchmarkSchedule(plan).length,
+                    maximumCostUsdMicros: plan.budget.costUsdMicros === null ? null : (BigInt(benchmarkSchedule(plan).length) * BigInt(plan.budget.costUsdMicros)).toString(),
+                    inputLimitMode: input.model.inputLimitMode ?? 'upper-bound', outputLimitMode: input.model.outputLimitMode ?? 'provider',
+                    maximumInputTokens: input.model.inputLimitMode === 'estimate' ? null : benchmarkSchedule(plan).length * plan.budget.inputTokens,
+                    maximumOutputTokens: input.model.outputLimitMode === 'observed' ? null : benchmarkSchedule(plan).length * plan.budget.outputTokens,
+                    observedInputTokenLimit: benchmarkSchedule(plan).length * plan.budget.inputTokens,
+                    observedOutputTokenLimit: benchmarkSchedule(plan).length * plan.budget.outputTokens };
+            }
+            else {
+                const store = new BenchmarkStore(resolve(fields.get('--database')));
+                try {
+                    // Validate/freeze the plan before loading the Host adapter (which may initialize credentials).
+                    store.create(plan);
+                    const factory = await adapter(fields.get('--adapter'), input);
+                    await runBenchmark(store, plan, native.createNativeBenchmarkExecutor(input, factory), signal);
+                    const report = benchmarkReport(plan, store.results(plan.id));
+                    result = report;
+                    exitCode = report.complete && report.variants.every(variant => variant.unknown === 0) ? 0 : 2;
+                }
+                finally {
+                    store.close();
+                }
+            }
+        }
+    }
+    else if (command === 'report') {
+        const fields = options(argv.slice(1), ['--database', '--plan', '--config', '--output'], ['--database', '--plan']);
+        output = fields.get('--output');
+        await ensureNewOutput(output);
+        // Reporting must not silently create an empty database when the path is wrong.
+        const database = resolve(fields.get('--database'));
+        if (!(await lstat(database)).isFile())
+            throw new BenchmarkError('report requires an existing regular database');
+        const store = new BenchmarkStore(database);
+        try {
+            const plan = store.plan(fields.get('--plan'));
+            if (plan.dataset.id === 'dsh-strategy-development') {
+                const configPath = fields.get('--config');
+                if (configPath === undefined)
+                    throw new BenchmarkError('strategy report requires --config to reconstruct and verify evidence');
+                const input = await strategyConfig(await config(configPath));
+                const [, , executor, strategyPlan] = await strategyModules();
+                const strategy = executor.createStrategyBenchmarkPlan(input);
+                const journal = strategyPlan.strategyBenchmarkJournalPlan(strategy);
+                if (benchmarkPlanDigest(journal) !== benchmarkPlanDigest(plan))
+                    throw new BenchmarkError('strategy report config does not match the stored plan');
+                executor.verifyStrategyBenchmarkResults(strategy, store.results(plan.id), input.stateDirectory, input.workspaceDirectory);
+            }
+            result = benchmarkReport(plan, store.results(plan.id));
+        }
+        finally {
+            store.close();
+        }
+    }
+    else
+        throw new BenchmarkError('unknown benchmark command; use --help');
+    const text = `${JSON.stringify(result, null, 2)}\n`;
+    if (output === undefined)
+        io.stdout(text);
+    else {
+        const path = resolve(output);
+        await mkdir(dirname(path), { recursive: true, mode: 0o700 });
+        await writeFile(path, text, { flag: 'wx', mode: 0o600 });
+    }
+    return exitCode;
+}
+//# sourceMappingURL=cli.js.map

package/lib/benchmark/cli.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"cli.js","sourceRoot":"","sources":["../../src/benchmark/cli.ts"],"names":[],"mappings":";;;;;;;;AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAA;AACxC,OAAO,EAAE,SAAS,EAAE,MAAM,SAAS,CAAA;AACnC,OAAO,EAAE,KAAK,EAAE,KAAK,EAAE,IAAI,EAAE,SAAS,EAAE,MAAM,kBAAkB,CAAA;AAChE,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAA;AAC3C,OAAO,EAAE,OAAO,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,WAAW,CAAA;AACxD,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAA;AACxC,OAAO,EAAE,iBAAiB,EAAE,kBAAkB,EAAE,MAAM,aAAa,CAAA;AACnE,OAAO,EAAE,uBAAuB,EAAE,wBAAwB,EAAE,yBAAyB,EAAE,0BAA0B,EAAE,MAAM,oBAAoB,CAAA;AAC7I,OAAO,EAAE,eAAe,EAAE,MAAM,aAAa,CAAA;AAC7C,OAAO,EAAE,mBAAmB,EAAE,iBAAiB,EAAE,cAAc,EAAE,MAAM,aAAa,CAAA;AACpF,OAAO,EAAE,cAAc,EAAE,MAAM,YAAY,CAAA;AAC3C,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAA;AAG1C,MAAM,eAAe,GAAG,CAAC,qBAAqB,EAAE,wBAAwB,EAAE,6BAA6B;IACrG,sBAAsB,EAAE,0BAA0B,EAAE,qCAAqC,EAAE,gCAAgC,EAAE,wBAAwB,CAAU,CAAA;AACjK,MAAM,IAAI,GAAG;;;;;;;;;;;CAWZ,CAAA;AAED,SAAS,OAAO,CAAC,IAAuB,EAAE,OAA0B,EAAE,QAA2B;IAC/F,MAAM,MAAM,GAAG,IAAI,GAAG,EAAkB,CAAA;IACxC,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,IAAI,CAAC,MAAM,EAAE,KAAK,IAAI,CAAC,EAAE,CAAC;QACpD,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK,CAAE,CAAC;QAAC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC,CAAA;QACvD,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,IAAI,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,KAAK,KAAK,SAAS,IAAI,KAAK,CAAC,UAAU,CAAC,IAAI,CAAC,IAAI,KAAK,CAAC,IAAI,EAAE,KAAK,EAAE;YAAE,MAAM,IAAI,cAAc,CAAC,yCAAyC,CAAC,CAAA;QAC1L,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAA;IACxB,CAAC;IACD,IAAI,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QAAE,MAAM,IAAI,cAAc,CAAC,iDAAiD,CAAC,CAAA;IACvH,OAAO,MAAM,CAAA;AACf,CAAC;AAED,KAAK,UAAU,WAAW,CAAC,IAAY,EAAE,KAAa,EAAE,cAAc,GAAG,KAAK;IAC5E,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,IAAI,EAAE,SAAS,CAAC,QAAQ,GAAG,SAAS,CAAC,UAAU,GAAG,SAAS,CAAC,UAAU,CAAC,CAAA;IACjG,IAAI,CAAC;QACH,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,IAAI,EAAE,CAAA;QAClC,IAAI,CAAC,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC,cAAc,IAAI,MAAM,CAAC,KAAK,KAAK,CAAC,CAAC,IAAI,MAAM,CAAC,IAAI,GAAG,KAAK;YAAE,MAAM,IAAI,cAAc,CAAC,gDAAgD,CAAC,CAAA;QAClK,MAAM,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAA;QACrC,IAAI,KAAK,GAAG,CAAC,CAAA;QACb,OAAO,KAAK,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC;YAC5B,MAAM,EAAE,SAAS,EAAE,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,KAAK,EAAE,KAAK,CAAC,MAAM,GAAG,KAAK,EAAE,KAAK,CAAC,CAAA;YAClF,IAAI,SAAS,KAAK,CAAC;gBAAE,MAAK;YAC1B,KAAK,IAAI,SAAS,CAAA;QACpB,CAAC;QACD,MAAM,KAAK,GAAG,MAAM,MAAM,CAAC,IAAI,EAAE,CAAA;QACjC,IAAI,KAAK,GAAG,KAAK,IAAI,MAAM,CAAC,IAAI,KAAK,KAAK,CAAC,IAAI,IAAI,MAAM,CAAC,OAAO,KAAK,KAAK,CAAC,OAAO,IAAI,MAAM,CAAC,OAAO,KAAK,KAAK,CAAC,OAAO;YAAE,MAAM,IAAI,cAAc,CAAC,0CAA0C,CAAC,CAAA;QAC7L,MAAM,IAAI,GAAG,KAAK,CAAC,QAAQ,CAAC,CAAC,EAAE,KAAK,CAAC,CAAA;QACrC,OAAO,EAAE,IAAI,EAAE,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,EAAE,UAAU,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAAA;IACjG,CAAC;YAAS,CAAC;QAAC,MAAM,MAAM,CAAC,KAAK,EAAE,CAAA;IAAC,CAAC;AACpC,CAAC;AAED,KAAK,UAAU,YAAY;IACzB,IAAI,CAAC;QAAC,OAAO,MAAM,MAAM,CAAC,aAAa,CAAC,CAAA;IAAC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACxD,IAAI,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,IAAI,KAAK,IAAI,KAAK,CAAC,IAAI,KAAK,sBAAsB,EAAE,CAAC;YACnG,MAAM,IAAI,cAAc,CAAC,8GAA8G,CAAC,CAAA;QAC1I,CAAC;QACD,MAAM,IAAI,cAAc,CAAC,8CAA8C,CAAC,CAAA;IAC1E,CAAC;AACH,CAAC;AAED,KAAK,UAAU,MAAM,CAAC,IAAY;IAChC,MAAM,IAAI,GAAG,MAAM,WAAW,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,OAAO,CAAC,CAAA;IACtD,IAAI,CAAC;QAAC,OAAO,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAA0B,CAAA;IAAC,CAAC;IAAC,MAAM,CAAC;QAAC,MAAM,IAAI,cAAc,CAAC,qCAAqC,CAAC,CAAA;IAAC,CAAC;AACzI,CAAC;AAED,+FAA+F;AAC/F,KAAK,UAAU,OAAO,CAAC,IAAY,EAAE,KAA6C;IAChF,IAAI,CAAC,UAAU,CAAC,IAAI,CAAC;QAAE,MAAM,IAAI,cAAc,CAAC,sCAAsC,CAAC,CAAA;IACvF,MAAM,QAAQ,GAAG,MAAM,WAAW,CAAC,IAAI,EAAE,SAAS,EAAE,IAAI,CAAC,CAAA;IACzD,IAAI,QAAQ,CAAC,MAAM,KAAK,KAAK,CAAC,KAAK,CAAC,aAAa,IAAI,QAAQ,CAAC,MAAM,KAAK,KAAK,CAAC,KAAK,CAAC,kBAAkB;QAAE,MAAM,IAAI,cAAc,CAAC,oEAAoE,CAAC,CAAA;IACvM,gGAAgG;IAChG,0FAA0F;IAC1F,8EAA8E;IAC9E,MAAM,QAAQ,GAAY,MAAM,MAAM,kCAAC,GAAG,aAAa,CAAC,IAAI,CAAC,CAAC,IAAI,cAAc,QAAQ,CAAC,MAAM,EAAE,EAAC,CAAA;IAClG,IAAI,CAAC,MAAM,WAAW,CAAC,IAAI,EAAE,SAAS,EAAE,IAAI,CAAC,CAAC,CAAC,MAAM,KAAK,QAAQ,CAAC,MAAM;QAAE,MAAM,IAAI,cAAc,CAAC,sCAAsC,CAAC,CAAA;IAC3I,IAAI,CAAC,QAAQ,IAAI,OAAO,QAAQ,KAAK,QAAQ,IAAI,CAAC,CAAC,qBAAqB,IAAI,QAAQ,CAAC,IAAI,OAAO,QAAQ,CAAC,mBAAmB,KAAK,UAAU;QAAE,MAAM,IAAI,cAAc,CAAC,gDAAgD,CAAC,CAAA;IACvN,OAAO,QAAQ,CAAC,mBAA2C,CAAA;AAC7D,CAAC;AAED,MAAM,uBAAuB,GAAG,CAAC,GAAG,eAAe,EAAE,uBAAuB,EAAE,4BAA4B,EAAE,oCAAoC;IAC9I,2BAA2B,EAAE,sCAAsC,EAAE,4CAA4C,EAAE,kCAAkC;IACrJ,gCAAgC,EAAE,+BAA+B,EAAE,mCAAmC,EAAE,kCAAkC,CAAU,CAAA;AAEtJ,KAAK,UAAU,eAAe;IAC5B,IAAI,CAAC;QAAC,OAAO,MAAM,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,sBAAsB,CAAC,EAAE,MAAM,CAAC,sBAAsB,CAAC,EAAE,MAAM,CAAC,wBAAwB,CAAC,EAAE,MAAM,CAAC,oBAAoB,CAAC,CAAC,CAAC,CAAA;IAAC,CAAC;IAAC,MAAM,CAAC;QACxK,MAAM,IAAI,cAAc,CAAC,sHAAsH,CAAC,CAAA;IAClJ,CAAC;AACH,CAAC;AAED,KAAK,UAAU,cAAc,CAAC,KAAc;IAC1C,MAAM,CAAC,MAAM,CAAC,GAAG,MAAM,eAAe,EAAE,CAAA;IACxC,OAAO,MAAM,CAAC,4BAA4B,CAAC,KAAK,CAAC,CAAA;AACnD,CAAC;AAED,SAAS,gBAAgB,CAAC,KAAc;IACtC,OAAO,KAAK,KAAK,IAAI,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAK,KAA6B,CAAC,KAAK,KAAK,aAAa,CAAA;AAC9G,CAAC;AAED,KAAK,UAAU,eAAe,CAAC,IAAwB;IACrD,IAAI,IAAI,KAAK,SAAS;QAAE,OAAM;IAC9B,IAAI,CAAC;QAAC,MAAM,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAA;IAAC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QAChD,IAAI,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,IAAI,KAAK,IAAI,KAAK,CAAC,IAAI,KAAK,QAAQ;YAAE,OAAM;QAC5F,MAAM,KAAK,CAAA;IACb,CAAC;IACD,MAAM,IAAI,cAAc,CAAC,sDAAsD,CAAC,CAAA;AAClF,CAAC;AAID,wGAAwG;AACxG,MAAM,CAAC,KAAK,UAAU,YAAY,CAAC,IAAuB,EAAE,EAAsB,EAAE,MAAoB;IACtG,MAAM,OAAO,GAAG,IAAI,CAAC,CAAC,CAAC,CAAA;IACvB,IAAI,OAAO,KAAK,SAAS,IAAI,OAAO,KAAK,QAAQ,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;QAAC,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QAAC,OAAO,CAAC,CAAA;IAAC,CAAC;IACtG,IAAI,MAAe,CAAA;IACnB,IAAI,MAA0B,CAAA;IAC9B,IAAI,QAAQ,GAAG,CAAC,CAAA;IAChB,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QACzB,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,SAAS,CAAC,EAAE,EAAE,CAAC,CAAA;QACpD,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,IAAI,aAAa,CAAA;QAClD,IAAI,KAAK,KAAK,aAAa,EAAE,CAAC;YAC5B,MAAM,CAAC,EAAE,MAAM,CAAC,GAAG,MAAM,eAAe,EAAE,CAAA;YAC1C,MAAM,GAAG,EAAE,OAAO,EAAE,MAAM,CAAC,0BAA0B,EAAE,KAAK,EAAE,MAAM,CAAC,yBAAyB,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,EAAE,IAAI,CAAC,EAAE,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,IAAI,CAAC,SAAS,EAAE,CAAC,CAAC,EAAE,CAAA;QACjL,CAAC;aAAM,CAAC;YACN,IAAI,CAAC,CAAC,aAAa,EAAE,WAAW,EAAE,WAAW,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC;gBAAE,MAAM,IAAI,cAAc,CAAC,sBAAsB,CAAC,CAAA;YAChH,MAAM,GAAG,EAAE,OAAO,EAAE,KAAK,KAAK,WAAW,CAAC,CAAC,CAAC,0BAA0B,CAAC,CAAC,CAAC,KAAK,KAAK,WAAW,CAAC,CAAC,CAAC,wBAAwB,CAAC,CAAC,CAAC,kBAAkB,EAAE,KAAK,EAAE,CAAC,KAAK,KAAK,WAAW,CAAC,CAAC,CAAC,yBAAyB,CAAC,CAAC,CAAC,KAAK,KAAK,WAAW,CAAC,CAAC,CAAC,uBAAuB,CAAC,CAAC,CAAC,iBAAiB,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,EAAE,IAAI,CAAC,EAAE,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,SAAS,EAAE,IAAI,CAAC,SAAS,EAAE,CAAC,CAAC,EAAE,CAAA;QACnW,CAAC;IACH,CAAC;SAAM,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QAChC,MAAM,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,SAAS,EAAE,UAAU,CAAC,EAAE,EAAE,CAAC,CAAA;QAChE,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,SAAS,CAAC,IAAI,aAAa,CAAA;QAClD,MAAM,OAAO,GAAG,aAAa,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAA;QAC9C,IAAI,KAAK,KAAK,aAAa,EAAE,CAAC;YAC5B,MAAM,UAAU,GAAG,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,CAAA;YACvC,MAAM,UAAU,GAAG,UAAU,KAAK,SAAS,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,MAAM,MAAM,CAAC,UAAU,CAAC,CAAA;YAClF,MAAM,QAAQ,GAAI,UAA6D,EAAE,KAAK,EAAE,QAAQ,CAAA;YAChG,MAAM,gBAAgB,GAAG,CAAC,GAAG,uBAAuB,EAAE,GAAG,CAAC,QAAQ,KAAK,uBAAuB,CAAC,CAAC,CAAC,CAAC,yCAAyC,EAAE,8BAA8B,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAA;YACnL,MAAM,QAAQ,GAAG,gBAAgB,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE;gBAC3C,IAAI,CAAC;oBAAC,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;oBAAC,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,CAAA;gBAAC,CAAC;gBAAC,MAAM,CAAC;oBAAC,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,CAAA;gBAAC,CAAC;YAC7G,CAAC,CAAC,CAAA;YACF,IAAI,UAAU,KAAK,SAAS,EAAE,CAAC;gBAC7B,MAAM,GAAG,EAAE,KAAK,EAAE,KAAK,EAAE,QAAQ,EAAE,SAAS,EAAE,EAAE,KAAK,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,iIAAiI,EAAE,CAAA;gBACzM,QAAQ,GAAG,CAAC,CAAA;YACd,CAAC;iBAAM,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,CAAC;gBACrD,MAAM,GAAG,EAAE,KAAK,EAAE,KAAK,EAAE,QAAQ,EAAE,SAAS,EAAE,EAAE,KAAK,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,kGAAkG,EAAE,CAAA;gBAC1K,QAAQ,GAAG,CAAC,CAAA;YACd,CAAC;iBAAM,CAAC;gBACN,MAAM,KAAK,GAAG,MAAM,cAAc,CAAC,UAAU,CAAC,CAAA;gBAC9C,IAAI,cAAc,GAAG,KAAK,CAAA;gBAC1B,IAAI,CAAC;oBACH,MAAM,gBAAgB,GAAG,mCAAmC,CAAA;oBAC5D,MAAM,SAAS,GAAG,MAAM,MAAM,kCAAC,gBAAgB,EAAgF,CAAA;oBAC/H,MAAM,SAAS,CAAC,qBAAqB,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,UAAU,CAAC,CAAA;oBACpE,cAAc,GAAG,IAAI,CAAA;gBACvB,CAAC;gBAAC,MAAM,CAAC,CAAC,oFAAoF,CAAC,CAAC;gBAChG,MAAM,GAAG,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,CAAC,KAAK,CAAC,SAAS,CAAC,IAAI,cAAc,EAAE,QAAQ,EAAE,SAAS,EAAE,EAAE,KAAK,EAAE,cAAc,EAAE;oBAC1H,IAAI,EAAE,qHAAqH,EAAE,CAAA;gBAC/H,QAAQ,GAAI,MAA6B,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;YACzD,CAAC;QACH,CAAC;aAAM,CAAC;YACN,IAAI,CAAC,CAAC,aAAa,EAAE,WAAW,EAAE,WAAW,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC;gBAAE,MAAM,IAAI,cAAc,CAAC,sBAAsB,CAAC,CAAA;YAChH,MAAM,KAAK,GAAG,CAAC,GAAG,eAAe,EAAE,GAAG,CAAC,KAAK,KAAK,aAAa,CAAC,CAAC,CAAC,CAAC,+BAA+B,EAAE,gCAAgC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAA;YAC3I,MAAM,QAAQ,GAAG,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE;gBAChC,IAAI,CAAC;oBAAC,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;oBAAC,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,CAAA;gBAAC,CAAC;gBAAC,MAAM,CAAC;oBAAC,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,CAAA;gBAAC,CAAC;YAC7G,CAAC,CAAC,CAAA;YACF,MAAM,GAAG,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,CAAC,KAAK,CAAC,SAAS,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,oHAAoH,EAAE,CAAA;YAClM,QAAQ,GAAG,QAAQ,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;QAC7D,CAAC;IACH,CAAC;SAAM,IAAI,OAAO,KAAK,MAAM,IAAI,OAAO,KAAK,KAAK,EAAE,CAAC;QACnD,MAAM,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,UAAU,EAAE,WAAW,EAAE,YAAY,EAAE,UAAU,CAAC,EAAE,OAAO,KAAK,KAAK,CAAC,CAAC,CAAC,CAAC,UAAU,EAAE,WAAW,EAAE,YAAY,CAAC,CAAC,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAA;QACtK,IAAI,OAAO,KAAK,MAAM,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,WAAW,CAAC,IAAI,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC;YAAE,MAAM,IAAI,cAAc,CAAC,yCAAyC,CAAC,CAAA;QACpJ,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,UAAU,CAAC,CAAA;QAC/B,MAAM,eAAe,CAAC,MAAM,CAAC,CAAA;QAC7B,MAAM,GAAG,GAAG,MAAM,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,UAAU,CAAE,CAAC,CAAA;QACjD,IAAI,gBAAgB,CAAC,GAAG,CAAC,EAAE,CAAC;YAC1B,MAAM,KAAK,GAAG,MAAM,cAAc,CAAC,GAAG,CAAC,CAAA;YACvC,MAAM,CAAC,EAAE,MAAM,EAAE,QAAQ,EAAE,YAAY,CAAC,GAAG,MAAM,eAAe,EAAE,CAAA;YAClE,MAAM,IAAI,GAAG,QAAQ,CAAC,2BAA2B,CAAC,KAAK,CAAC,CAAA;YACxD,MAAM,OAAO,GAAG,YAAY,CAAC,4BAA4B,CAAC,IAAI,CAAC,CAAA;YAC/D,MAAM,KAAK,GAAG,iBAAiB,CAAC,OAAO,CAAC,CAAA;YACxC,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;gBACvB,MAAM,GAAG,EAAE,IAAI,EAAE,WAAW,EAAE,OAAO,EAAE,UAAU,EAAE,mBAAmB,CAAC,OAAO,CAAC,EAAE,YAAY,EAAE,KAAK,CAAC,MAAM;oBACzG,oBAAoB,EAAE,OAAO,CAAC,MAAM,CAAC,aAAa,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC,CAAC,QAAQ,EAAE;oBAC7I,cAAc,EAAE,aAAa,EAAE,eAAe,EAAE,UAAU,EAAE,kBAAkB,EAAE,KAAK,CAAC,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,WAAW;oBACzH,mBAAmB,EAAE,KAAK,CAAC,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,YAAY,EAAE,uBAAuB,EAAE,KAAK,CAAC,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,WAAW;oBACnI,wBAAwB,EAAE,KAAK,CAAC,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,YAAY,EAAE,CAAA;YAC1E,CAAC;iBAAM,CAAC;gBACN,MAAM,KAAK,GAAG,IAAI,cAAc,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,YAAY,CAAE,CAAC,CAAC,CAAA;gBACpE,IAAI,CAAC;oBACH,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,CAAA;oBACrB,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,WAAW,CAAE,EAAE,KAAK,CAAC,CAAA;oBAC9D,MAAM,KAAK,GAAG,MAAM,CAAC,WAAW,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,MAAM,EAAE,MAAM,CAAC,uBAAuB,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAA;oBAC7G,MAAM,gBAAgB,GAAG,QAAQ,CAAC,+BAA+B,CAAC,EAAE,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,OAAO;wBAClI,kBAAkB,EAAE,KAAK,CAAC,kBAAkB,EAAE,cAAc,EAAE,KAAK,CAAC,cAAc,EAAE,KAAK,EAAE,KAAK,CAAC,KAAK,EAAE,UAAU,EAAE,KAAK,CAAC,UAAU;wBACpI,iBAAiB,EAAE,KAAK,CAAC,iBAAiB,EAAE,GAAG,CAAC,KAAK,CAAC,aAAa,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,KAAK,CAAC,aAAa,EAAE,CAAC,EAAE,CAAC,CAAA;oBACrI,MAAM,OAAO,GAAG,MAAM,YAAY,CAAC,KAAK,EAAE,OAAO,EAAE,gBAAgB,EAAE,MAAM,CAAC,CAAA;oBAC5E,QAAQ,CAAC,8BAA8B,CAAC,IAAI,EAAE,OAAO,EAAE,KAAK,CAAC,cAAc,EAAE,KAAK,CAAC,kBAAkB,CAAC,CAAA;oBACtG,MAAM,MAAM,GAAG,eAAe,CAAC,OAAO,EAAE,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAA;oBAClE,MAAM,GAAG,MAAM,CAAA;oBACf,QAAQ,GAAG,MAAM,CAAC,QAAQ,IAAI,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC,OAAO,CAAC,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;gBAC/F,CAAC;wBAAS,CAAC;oBAAC,KAAK,CAAC,KAAK,EAAE,CAAA;gBAAC,CAAC;YAC7B,CAAC;QACH,CAAC;aAAM,CAAC;YACN,MAAM,KAAK,GAAG,GAA4B,CAAA;YAC1C,MAAM,MAAM,GAAG,MAAM,YAAY,EAAE,CAAA;YACnC,MAAM,IAAI,GAAG,MAAM,CAAC,mBAAmB,CAAC,KAAK,CAAC,CAAA;YAC9C,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;gBACvB,MAAM,GAAG,EAAE,IAAI,EAAE,UAAU,EAAE,mBAAmB,CAAC,IAAI,CAAC,EAAE,YAAY,EAAE,iBAAiB,CAAC,IAAI,CAAC,CAAC,MAAM;oBAClG,oBAAoB,EAAE,IAAI,CAAC,MAAM,CAAC,aAAa,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,MAAM,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC,CAAC,QAAQ,EAAE;oBACzJ,cAAc,EAAE,KAAK,CAAC,KAAK,CAAC,cAAc,IAAI,aAAa,EAAE,eAAe,EAAE,KAAK,CAAC,KAAK,CAAC,eAAe,IAAI,UAAU;oBACvH,kBAAkB,EAAE,KAAK,CAAC,KAAK,CAAC,cAAc,KAAK,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,WAAW;oBAC/H,mBAAmB,EAAE,KAAK,CAAC,KAAK,CAAC,eAAe,KAAK,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,iBAAiB,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,YAAY;oBAClI,uBAAuB,EAAE,iBAAiB,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,WAAW;oBACjF,wBAAwB,EAAE,iBAAiB,CAAC,IAAI,CAAC,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,YAAY,EAAE,CAAA;YACzF,CAAC;iBAAM,CAAC;gBACN,MAAM,KAAK,GAAG,IAAI,cAAc,CAAC,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,YAAY,CAAE,CAAC,CAAC,CAAA;gBACpE,IAAI,CAAC;oBACH,+FAA+F;oBAC/F,KAAK,CAAC,MAAM,CAAC,IAAI,CAAC,CAAA;oBAClB,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,WAAW,CAAE,EAAE,KAAK,CAAC,CAAA;oBAC9D,MAAM,YAAY,CAAC,KAAK,EAAE,IAAI,EAAE,MAAM,CAAC,6BAA6B,CAAC,KAAK,EAAE,OAAO,CAAC,EAAE,MAAM,CAAC,CAAA;oBAC7F,MAAM,MAAM,GAAG,eAAe,CAAC,IAAI,EAAE,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAA;oBAC5D,MAAM,GAAG,MAAM,CAAA;oBACf,QAAQ,GAAG,MAAM,CAAC,QAAQ,IAAI,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC,OAAO,CAAC,OAAO,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;gBAC/F,CAAC;wBAAS,CAAC;oBAAC,KAAK,CAAC,KAAK,EAAE,CAAA;gBAAC,CAAC;YAC7B,CAAC;QACH,CAAC;IACH,CAAC;SAAM,IAAI,OAAO,KAAK,QAAQ,EAAE,CAAC;QAChC,MAAM,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,YAAY,EAAE,QAAQ,EAAE,UAAU,EAAE,UAAU,CAAC,EAAE,CAAC,YAAY,EAAE,QAAQ,CAAC,CAAC,CAAA;QACjH,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC;QAAC,MAAM,eAAe,CAAC,MAAM,CAAC,CAAA;QAC9D,+EAA+E;QAC/E,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,YAAY,CAAE,CAAC,CAAA;QACnD,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,MAAM,EAAE;YAAE,MAAM,IAAI,cAAc,CAAC,8CAA8C,CAAC,CAAA;QAC/G,MAAM,KAAK,GAAG,IAAI,cAAc,CAAC,QAAQ,CAAC,CAAA;QAC1C,IAAI,CAAC;YACH,MAAM,IAAI,GAAG,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAE,CAAC,CAAA;YAC9C,IAAI,IAAI,CAAC,OAAO,CAAC,EAAE,KAAK,0BAA0B,EAAE,CAAC;gBACnD,MAAM,UAAU,GAAG,MAAM,CAAC,GAAG,CAAC,UAAU,CAAC,CAAA;gBACzC,IAAI,UAAU,KAAK,SAAS;oBAAE,MAAM,IAAI,cAAc,CAAC,sEAAsE,CAAC,CAAA;gBAC9H,MAAM,KAAK,GAAG,MAAM,cAAc,CAAC,MAAM,MAAM,CAAC,UAAU,CAAC,CAAC,CAAA;gBAC5D,MAAM,CAAC,EAAE,AAAD,EAAG,QAAQ,EAAE,YAAY,CAAC,GAAG,MAAM,eAAe,EAAE,CAAA;gBAC5D,MAAM,QAAQ,GAAG,QAAQ,CAAC,2BAA2B,CAAC,KAAK,CAAC,CAAA;gBAC5D,MAAM,OAAO,GAAG,YAAY,CAAC,4BAA4B,CAAC,QAAQ,CAAC,CAAA;gBACnE,IAAI,mBAAmB,CAAC,OAAO,CAAC,KAAK,mBAAmB,CAAC,IAAI,CAAC;oBAAE,MAAM,IAAI,cAAc,CAAC,uDAAuD,CAAC,CAAA;gBACjJ,QAAQ,CAAC,8BAA8B,CAAC,QAAQ,EAAE,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,EAAE,KAAK,CAAC,cAAc,EAAE,KAAK,CAAC,kBAAkB,CAAC,CAAA;YAC3H,CAAC;YACD,MAAM,GAAG,eAAe,CAAC,IAAI,EAAE,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAA;QACxD,CAAC;gBAAS,CAAC;YAAC,KAAK,CAAC,KAAK,EAAE,CAAA;QAAC,CAAC;IAC7B,CAAC;;QAAM,MAAM,IAAI,cAAc,CAAC,uCAAuC,CAAC,CAAA;IACxE,MAAM,IAAI,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,CAAA;IACnD,IAAI,MAAM,KAAK,SAAS;QAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAA;SACpC,CAAC;QAAC,MAAM,IAAI,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC;QAAC,MAAM,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;QAAC,MAAM,SAAS,CAAC,IAAI,EAAE,IAAI,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAA;IAAC,CAAC;IAC7J,OAAO,QAAQ,CAAA;AACjB,CAAC"}

package/lib/benchmark/corpus.d.ts

@@ -0,0 +1,28 @@
+import type { BenchmarkCase, BenchmarkPlan, BenchmarkVerdict } from './types.js';
+/** JSON values accepted by the public development-task judge. */
+export type BenchmarkJson = null | boolean | number | string | readonly BenchmarkJson[] | {
+    readonly [key: string]: BenchmarkJson;
+};
+export interface DevelopmentTask {
+    readonly id: string;
+    readonly domain: 'research' | 'injection';
+    readonly objective: string;
+    readonly sources: readonly {
+        readonly id: string;
+        readonly text: string;
+    }[];
+    readonly acceptance: {
+        readonly answer: BenchmarkJson;
+        readonly citations: readonly string[];
+    };
+}
+export declare const developmentCorpus: readonly DevelopmentTask[];
+export declare const developmentDataset: BenchmarkPlan['dataset'];
+/** Returns immutable copies so callers cannot mutate the corpus-derived benchmark manifest. */
+export declare function developmentCases(): readonly BenchmarkCase[];
+export declare function developmentPrompt(caseId: string): string;
+export declare function judgeDevelopmentResponse(caseId: string, output: string): {
+    verdict: BenchmarkVerdict;
+    evidenceDigest: string;
+};
+//# sourceMappingURL=corpus.d.ts.map

package/lib/benchmark/corpus.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"corpus.d.ts","sourceRoot":"","sources":["../../src/benchmark/corpus.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,aAAa,EAAE,aAAa,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAA;AAEhF,iEAAiE;AACjE,MAAM,MAAM,aAAa,GAAG,IAAI,GAAG,OAAO,GAAG,MAAM,GAAG,MAAM,GAAG,SAAS,aAAa,EAAE,GAAG;IAAE,QAAQ,EAAE,GAAG,EAAE,MAAM,GAAG,aAAa,CAAA;CAAE,CAAA;AAEnI,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;IACnB,QAAQ,CAAC,MAAM,EAAE,UAAU,GAAG,WAAW,CAAA;IACzC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAA;IAC1B,QAAQ,CAAC,OAAO,EAAE,SAAS;QAAE,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;KAAE,EAAE,CAAA;IAC3E,QAAQ,CAAC,UAAU,EAAE;QAAE,QAAQ,CAAC,MAAM,EAAE,aAAa,CAAC;QAAC,QAAQ,CAAC,SAAS,EAAE,SAAS,MAAM,EAAE,CAAA;KAAE,CAAA;CAC/F;AAwGD,eAAO,MAAM,iBAAiB,EAAE,SAAS,eAAe,EAA4G,CAAA;AAMpK,eAAO,MAAM,kBAAkB,EAAE,aAAa,CAAC,SAAS,CAEtD,CAAA;AAEF,+FAA+F;AAC/F,wBAAgB,gBAAgB,IAAI,SAAS,aAAa,EAAE,CAI3D;AAED,wBAAgB,iBAAiB,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,CAUxD;AAkBD,wBAAgB,wBAAwB,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG;IAAE,OAAO,EAAE,gBAAgB,CAAC;IAAC,cAAc,EAAE,MAAM,CAAA;CAAE,CAiB9H"}