oh-my-knowledge 1.0.0-beta.0 → 1.0.0-beta.10

package/README.md

@@ -8,236 +8,64 @@
 
 **English** | [简体中文](./README.zh.md)
 
-**Observe. Measure. Know.**
+**Observe. Measure. Know.** Make knowledge changes in your AI application evidence-backed.
 
-**OMK makes every knowledge change in your AI application evidence-backed.**
+OMK helps authors of prompts, RAG systems, skills, and agents compare versions, inspect evidence, and find knowledge gaps in real tasks. A controlled comparison keeps the **same model and evaluation samples, changing only the knowledge artifact**.
 
-Observe real-world performance, measure version differences, and determine whether the change is effective and the version is ready to ship.
+Version 1.0 is still **in Beta iteration**; APIs and storage contracts may change. Read the [migration guide](docs/guides/v1-preview-migration.md) before upgrading an older installation.
 
-**Same model. Same evaluation samples. Only the knowledge artifact changes.**
+![OMK: from controlled evaluation to real-world feedback](./docs/public/omk-knowledge-flow-en-animated.gif)
 
-**DeepSeek Harness users:** install OMK as a native bundle, reuse the current profile for controlled evaluations, and open persisted DSH task trajectories in Studio. [Set up the DSH host plugin →](docs/reference/executors.md#deepseek-harness-prefer-the-host-plugin)
+## Start with your goal
 
-![omk knowledge artifact evaluation flow: doctor / eval / observe / sample / evolve loop](./docs/public/omk-knowledge-flow-en-animated.gif)
-
-📖 **Full documentation: [oh-my-knowledge.pages.dev](https://oh-my-knowledge.pages.dev)** (searchable, English / 简体中文)
-
-## What OMK helps you know
-
-| Decision | Command | Evidence you get |
-|---|---|---|
-| Is this artifact coherent enough to evaluate? | `omk doctor` | structure, dependencies, safety, and measurability checks |
-| Is v2 actually better than v1? | `omk eval` | one-line verdict, confidence interval, failed samples, cost |
-| Why did it pass or fail? | `omk studio` | report view with scores, diagnostics, and examples |
-| Should this version become the accepted one? | `omk promote` / `omk evolve` | evidence-gated accept or generate a better candidate |
-| What happened during one real AI task? | `omk observe` / Studio Task Trajectory | a trace-backed view of the request, visible Knowledge, tool calls, results, response, and user correction |
-| What did real usage expose? | `omk observe` / `omk sample --from-traces` | production gaps drafted for review; reviewed drafts can become eval samples |
+| Goal | Entry point |
+|---|---|
+| Compare two skills and inspect the verdict, interval, and failed cases | [CLI quickstart](docs/quickstart-skill-eval.md) |
+| Add scoring and version comparisons to a Node.js service | [Service integration guide](docs/guides/eval-runtime.md), including an example without model credentials |
+| Inspect one task or find problems in historical logs | [Observation and task trajectories](docs/guides/observe-production.md) |
 
 ## Quick start
 
+You need Node.js >=22 and an authenticated model runtime; see [Requirements](#requirements). Install the Beta and preview your first comparison:
+
 ```bash
-npm i -g oh-my-knowledge
-omk init demo && cd demo
+npm i -g oh-my-knowledge@next
+omk init demo
+cd demo
 omk eval --control code-review-v1 --treatment code-review-v2 --dry-run
-omk eval --control code-review-v1 --treatment code-review-v2
 ```
 
-Runs out of the box — no edits needed first. `omk init` scaffolds two skill variants and three sample cases; `--dry-run` previews the sealed task plan and estimated calls; `omk eval` runs the controlled A/B and opens the authenticated Core run in Studio. Once it runs, swap in your own skills and cases. To start directly with the first-party 20-case pack, replace the first command with `omk init demo --samples 20`.
-
-Prerequisite: configure one authenticated model runtime (Codex CLI, Claude Code, or an API executor; see [Requirements](#requirements)). Inside a Codex task in the ChatGPT desktop app, omk automatically selects `codex`, reads the model from `~/.codex/config.toml`, and uses the same Codex model as the default judge. Claude is not required.
-
-To make Codex the default in regular terminals, add the preference to your shell profile (for example `~/.zshrc`):
+After checking the plan and estimated calls, run the evaluation (this calls the model):
 
 ```bash
-export OMK_EXECUTOR=codex
-# Optional: export OMK_MODEL="your-codex-model"
-```
-
-Without `OMK_MODEL`, omk reads the model from `~/.codex/config.toml`. You can still pass `--executor codex --model <codex-model>` per command. Pass `--judge-models` or set `OMK_JUDGE_MODELS` only when you want a different judge.
-
-> The default 3-case pack is a low-cost workflow check, so `UNDERPOWERED` is expected. `--samples 20` selects a first-party, difficulty-stratified starter pack that meets omk's registered sample-size floor. Its provenance is `llm-generated`: use it to learn the statistical workflow, then review and replace it with real domain cases before making a release decision.
-
-> The CLI notifies you when a newer version is available (at most once per 20h); set `OMK_SKIP_UPDATE_CHECK=1` to silence it permanently.
-
-Walkthrough: [5-minute quickstart guide](docs/quickstart-skill-eval.md) (recommended for first-time users; includes demo → own skill → verdict actions). More runnable examples (Skill Map, offline executor, agent runtime, RAG, Observe) live in the repo's [example gallery](examples/README.md).
-
-Deeper: [who omk is for](docs/explanation/who-omk-is-for.md) · [CLI reference](docs/reference/cli.md) · [how it works](docs/explanation/architecture.md) · [eval sample format](docs/reference/eval-sample-format.md) · [executors](docs/reference/executors.md) · [artifact layout](docs/reference/artifact-layout.md)
-
-## Inspect one Codex task
-
-If you only want to see what happened behind one Codex conversation, you do not need to run `observe ingest` first:
-
-```bash
-omk studio
+omk eval --control code-review-v1 --treatment code-review-v2
 ```
 
-Studio opens the local Codex conversation overview at `http://127.0.0.1:7799` by default. Select a conversation, then a task, to open **Task Trajectory**. Its four lanes — **Conversation, Actions, Results, and Knowledge** — show the request, AI responses, tool calls, tool returns, and observable context, with drill-downs into normalized events and raw logs.
-
-Running tasks are prioritized and update live. While **Following**, the trajectory advances smoothly as events arrive; after you inspect an earlier point, Studio keeps your position and offers **View updates**. Old logs without a terminal event are marked **End status not recorded** instead of remaining live forever.
-
-Task Trajectory only reconstructs facts observable in the log. It does not reveal or infer hidden reasoning. See [Observe production traces](docs/guides/observe-production.md#inspect-one-task) for the full model.
-
-## The OMK loop
-
-OMK is for authors and maintainers of LLM knowledge artifacts who need a release decision, not for passive end-users of a skill. The main loop is deliberately controlled:
-
-```text
-change a prompt / RAG / skill / agent artifact
-→ run omk doctor before evaluation
-→ run omk eval with the same model and the same samples
-→ read the report / Studio evidence
-→ promote a proven version or evolve a candidate
-→ observe real usage and draft gap-derived samples for review
-```
+The scaffold contains two skills and three cases. This small default set checks the workflow; `UNDERPOWERED` is expected. Starter cases are marked `llm-generated`. Even the full pack created with `omk init demo-full --samples 20` only meets the default heuristic evidence floor, not an a priori power plan or a release-evidence requirement. Review and replace starter cases with real domain cases before relying on the results.
 
-The first value is the pre-ship `doctor → eval` decision. The long-term value is the closed loop: `observe` surfaces production gaps, `sample --from-traces` drafts regression samples for human review, and reviewed drafts can become fixed eval samples that make the next `eval` harder to game.
+The [full walkthrough](docs/quickstart-skill-eval.md) covers runtime selection, your own skills, and interpreting results. The [example gallery](examples/README.md) offers more runnable scenarios.
 
 ## Use inside AI Coding Agents
 
-Install the official omk Agent Skill to let your coding agent run omk workflows from natural language:
-
 ```bash
 omk install omk-agent-skill
 ```
 
-By default, omk installs only into detected local targets it explicitly supports: Codex/AGENTS when `~/.codex` or `~/.agents` exists, and Claude Code when `~/.claude` exists. Use `--to all` to force every target omk currently knows, or `--dest` for a custom skill root.
-
-### Use inside Claude Code
-
-When the `omk` skill is available in Claude Code, you can invoke it directly:
-
-```bash
-/omk eval              # evaluate the artifact(s) in the current project
-/omk evolve            # auto-iterate to improve a skill
-/omk sample            # generate or fill test cases
-```
-
-These slash commands are natural-language entry points — the agent reads the conversation context to figure out which skill to operate on. You can also just say "compare v1 vs v2 for me" or "improve this artifact" and omk picks the right command.
+Then ask your coding agent: “Use omk to compare these two skills.” See the [quickstart](docs/quickstart-skill-eval.md#use-inside-an-agent) for installation targets and usage. DeepSeek Harness users can use the [host plugin](docs/reference/executors.md#deepseek-harness-prefer-the-host-plugin) to reuse their current profile.
 
-### Use inside Codex
+The [MCP integration](docs/guides/mcp-integration.md) accepts user-authorized knowledge feedback. It records partial evidence submitted at its tool boundary; it does not automatically monitor complete conversations.
 
-Codex does not support Claude Code style `/omk ...` slash commands. Ask the agent to run the `omk` CLI directly. Inside a Codex task, omk automatically selects the Codex runtime and locally configured model:
-
-```bash
-omk eval
-omk evolve skills/my-skill.md   # one-shot: doctor → (auto-generate samples if missing) → self-iterate
-omk sample skills/my-skill.md
-```
-
-You can also describe the goal in natural language, such as "compare v1 vs v2" or "generate test cases for this skill".
-
-`eval`, `doctor`, `sample`, `evolve`, and the LLM-enhanced observe review share the same runtime resolution. Once Codex is selected, the default judge reuses the evaluated Codex model instead of falling back to `claude:haiku`.
-
-> `omk evolve` is a one-shot loop: it runs the doctor gate first, auto-generates eval samples when the target skill has none, then self-iterates. For a brand-new skill, just run `omk evolve skills/foo.md`.
-
-## Why this tool
-
-Knowledge engineering creates a versioning problem: every prompt, RAG recipe, skill, agent, or workflow can change behavior without changing application code. When someone asks "can we ship v2, and why?", a prettier answer or a higher anecdotal success rate is not enough.
-
-omk treats the knowledge artifact as the variable under test: **same model, same evaluation samples, only the artifact changes.** That makes the comparison explainable, repeatable, and suitable for CI or release review.
-
-## Why omk over alternatives
-
-| | omk | promptfoo | DeepEval | LangSmith |
-|--|--|--|--|--|
-| Bootstrap CI | ✓ default | ✗ | ✗ | ✗ |
-| Krippendorff α (judge ↔ human) | ✓ with gold set | ✗ | ✗ | ✗ |
-| Length-debias judge prompt | ✓ default | ✗ | ✗ | ✗ |
-| Fail-closed evidence coverage | ✓ | ✗ | ✗ | ✗ |
-| Three-layer scoring isolation | ✓ | ✗ | partial | ✗ |
-| Per-variant skill isolation (construct validity) | ✓ default | ✗ | ✗ | ✗ |
-| Native Agent Skill | ✓ | ✗ | ✗ | ✗ |
-| Hosted SaaS dashboard | ✗ | ✗ | ✓ | ✓ |
-
-omk's moat is a **default-on safety net**: Bootstrap CI and length-debias are normal measurement behavior, missing evidence fails closed, and explicit Gold comparison provides judge ↔ human alpha calibration. Need a hosted SaaS dashboard? Choose LangSmith. Want quick local prompt iteration without statistics? Choose promptfoo. **Shipping to production and someone will ask "why should I trust this number?" Choose omk.**
-
-RAG-specific evals: see RAGAS (separate niche, complementary to omk). Full comparison with 7 tools across 25+ dimensions: [docs/reference/comparison.md](docs/reference/comparison.md).
-
-## Features
-
-| Feature | What it does |
-|---|---|
-| **Core release decision** | Six conclusions + stable reason codes + exit-code routing; Studio projects the same authenticated Decision |
-| **Five-layer evidence graph** | Assertion / LLM / Judge / Dimension / Composite stay distinct, with coverage, cost, status, and lineage kept orthogonal |
-| **Multi-executor** | Claude CLI / Claude SDK / Codex CLI / Codex SDK / DeepSeek Harness / OpenAI / Anthropic API / any custom command |
-| **30+ assertion types** | substring, regex, JSON Schema, ROUGE/BLEU/Levenshtein similarity, agent tool-call assertions, semantic similarity, custom JS |
-| **Statistical rigor** | Bootstrap comparison families, length-debias, fail-closed evidence coverage, and explicit Gold agreement calibration. [Details →](docs/explanation/statistical-rigor.md) |
-| **RAG metrics** | `faithfulness` / `answer_relevancy` / `context_recall` — anti-hallucination + answer relevance + context coverage |
-| **LLM health audit** | `omk doctor` grades 7 builtin dimensions; repeats the audit (`--repeat`) and merges findings by k/n consensus |
-| **Production observability** | normalize Codex, Claude Code, OpenClaw, and markdown logs into source-neutral Trace IR; measure per-skill outcomes / latency / token use / knowledge-gap signals |
-| **Active MCP knowledge feedback (experimental)** | an MCP client actively calls a tool to write user-authorized knowledge feedback into Observation Inbox; it does not monitor conversations, and every record is marked `coverage: partial` |
-| **Knowledge-gap detection** | severity-weighted signals quantify risk exposure instead of claiming completeness |
-| **Construct-validity isolation** | `--strict-baseline` (default ON) cuts three contamination channels so baseline doesn't silently see the skill it's being compared against |
-| **Git & remote sources** | install / eval from a local git ref or a remote git URL (`--git-url`); directory-skills run in a content-addressed **isolated copy** so `references/` assets are real measured input, not just `SKILL.md` |
-| **Evidence-gated management** | `omk install` registers a managed record; `omk eval` auto-writes evidence bound by content fingerprint, moving a skill `installed → measurable`; `omk list` surfaces each managed skill's status (installed / measurable / promoted / stale); `omk promote` accepts a version once its evidence passes the gate (default PROGRESS only); `omk rollback` revokes that acceptance, returning the skill to `measurable`. [spec →](docs/specs/evidence-gated-management.md) |
-| **Sample design science** | sample schema with `capability` / `difficulty` / `construct` / `provenance` metadata (HF Dataset Cards style); studio surfaces coverage breakdown plus `rubric_clarity_low` / `capability_thin` flags. [docs/specs/sample-design-spec.md](docs/specs/sample-design-spec.md) |
-| **Multi-judge ensemble** | `--judge-models claude:opus,openai-api:gpt-4o` cross-vendor scoring + agreement metrics |
-| **Multi-run variance** | `--repeat N` publishes independent Core runs and an Evaluation Series variance analysis |
-| **MCP URL fetching** | pull content from private-doc URLs via an MCP server (SSO-protected knowledge bases, etc.) |
-| **Auto analysis** | detects low-discrimination assertions, flat scores, all-pass / all-fail, expensive samples |
-| **Traceability** | reports carry CLI version, Node version, artifact version fingerprint, judge prompt hash |
-| **EN / ZH views** | bilingual local Studio views selected through the report URL |
-
-### Run inside an existing DeepSeek Harness
-
-Install OMK as a DSH bundle to reuse the profile's model, credentials, tools, and sandbox:
-
-```bash
-dsh plugin --profile web add oh-my-knowledge
-dsh --profile web
-```
-
-Inside DSH:
-
-- `/omk eval eval.yaml` runs every sample in an isolated DSH session while OMK owns the report and statistics;
-- `/omk observe` lists recent terminal sessions;
-- `/omk observe <session-id>` reads a consistent snapshot and returns its Studio Task Trajectory URL.
-
-Observe uses the profile's `sessionPersistence` directly, so users do not export or locate JSONL / SQLite files. The first version is offline-only and does not live-follow a session that is still being written. See the [executor guide](docs/reference/executors.md#deepseek-harness-prefer-the-host-plugin) and [observe guide](docs/guides/observe-production.md#inspect-a-task-inside-deepseek-harness).
-
-### Connect an MCP client (experimental)
-
-> **Positioning: OMK MCP is an active knowledge-feedback interface, not a conversation monitor.** OMK MCP alone cannot automatically monitor or subscribe to complete conversations in Codex, ChatGPT, or another client. OMK receives a record only after the client, model, or component actively calls `save_observation` with authorized content. An Agent Skill may automatically recognize a potential feedback moment, but saving it still requires user confirmation and an explicit MCP tool call.
-
-`omk-mcp` is a client-neutral stdio MCP server. Codex and other local MCP clients can start it directly; a private host can compose the exported Streamable HTTP adapter. The client calls `save_observation` only after the user explicitly asks to record feedback, appends the feedback and optional evidence under `.omk/observe/inbox/captures/`, and can render an inline MCP Apps review card for a human verdict and regression-sample draft.
-
-In Codex, explicitly invoke the OMK Skill to submit the current knowledge feedback:
-
-```text
-$omk feedback
-```
-
-The explicit invocation itself confirms the save. The agent selects the most recent clear issue from the visible conversation and calls `save_observation` with `confirmedByUser: true`; it asks first when the candidate is ambiguous. This shortcut is not a CLI command, and it does not automatically review the observation, draft a sample, or write to a gold set.
-
-```bash
-omk-mcp
-```
+## Use the evidence
 
-Every record carries `coverageStatus: partial`: OMK observes its tool boundary, submitted feedback, and optional evidence, but not the full conversation, other tool calls, or hidden reasoning. Continuous monitoring requires a host that is authorized to access an event stream and actively forwards those events; that is a host-integration capability, not an OMK MCP capability. Conversation IDs, turn IDs, and idempotency keys are hashed rather than persisted verbatim. For a private-host Streamable HTTP integration, see [Compose the OMK MCP integration](docs/guides/mcp-integration.md).
+`doctor` checks the artifact, `eval` compares versions, and `studio` presents results and raw evidence. When evidence meets the gate, `promote` can accept a version; `evolve` can generate a candidate. Gaps found by `observe` can become draft cases for review before the next evaluation.
 
-### Local storage
+Conclusions depend on the cases, scoring criteria, and execution environment. Observation signals are not causal conclusions, and generated cases are not an independent release-validation set. See [statistical rigor](docs/explanation/statistical-rigor.md) and the [three-stage workflow](docs/explanation/three-stage-workflow.md) for the method and limits.
 
-Project data uses the domain-oriented `.omk/` v2 layout: durable evidence lives under `eval/`, `doctor/`, and `observe/`; governance records live under `governance/`; backups remain recoverable; and only rebuildable work belongs in `state/`. Machine tools, tunnels, caches, and materialized copies stay under `~/.oh-my-knowledge/state/`, never in a project. This release neither reads nor migrates the earlier storage layout.
+Project evidence lives under `.omk/`; machine-level state lives under `~/.oh-my-knowledge/`. The current version neither reads nor migrates the old storage layout. Back up before upgrading and follow the [migration guide](docs/guides/v1-preview-migration.md) to establish new evidence.
 
 ## Documentation
 
-The full docs are published at **[oh-my-knowledge.pages.dev](https://oh-my-knowledge.pages.dev)** — searchable, with an English / 简体中文 switcher. Key pages:
-
-- **[How it works](docs/explanation/architecture.md)** — input compilation, sealed Core execution, analysis, persistence, and Studio projections
-- **[Eval sample format](docs/reference/eval-sample-format.md)** — sample schema, scoring formulas, 30+ assertion types, custom JS assertions
-- **[CLI reference](docs/reference/cli.md)** — all top-level commands with bash examples and flag tables
-- **[Migrate to the 1.0 preview](docs/guides/v1-preview-migration.md)** — install channel, storage reset, sample protocol, CLI automation, and embedded API changes since 0.54
-- **[Evaluation Core cutover](docs/guides/eval-core-cutover.md)** — `BREAKING-SCHEMA` storage, resume, Studio, Gold, managed-evidence, and evolve migration
-- **[Storage layout v2](docs/specs/storage-layout-spec.md)** — project/global domains, compatibility boundary, and Git policy
-- **[Executors](docs/reference/executors.md)** & **[artifact layout](docs/reference/artifact-layout.md)** — built-in / custom executors; how `variant` resolves to an artifact + runtime context
-- **[How-to guides](docs/guides/agent-eval.md)** — [evaluate an agent](docs/guides/agent-eval.md) (project runtime context) and [use non-Claude models](docs/guides/non-claude-models.md) (GLM / Qwen / DeepSeek / Moonshot / Ollama)
-- **[Observe & inspect task trajectories](docs/guides/observe-production.md)** — browse local Codex conversations, drill into one task, and follow its observable execution live
-- **[Quickstart](docs/quickstart-skill-eval.md)** — first-time five-minute walkthrough
-- **[Example gallery](https://github.com/lizhiyao/oh-my-knowledge/tree/main/examples)** — a set of runnable examples in the repo, arranged simplest-to-richest
-- **[Sample design spec](docs/specs/sample-design-spec.md)** — capability / construct / provenance metadata; industry-gap mapping
-- **[Statistical rigor](docs/explanation/statistical-rigor.md)** — why Bootstrap CI / Gold agreement / length-debias / evidence coverage matter
-- **[Comparison with 7 tools](docs/reference/comparison.md)** — 25+ dimensions across promptfoo / DeepEval / RAGAS / OpenAI Evals / LangSmith / lm-eval-harness / inspect-ai
-- **[Evidence-gated management](docs/specs/evidence-gated-management.md)** — managed records, lifecycle states (installed / measurable / promoted / stale), install → eval → measurable → promote → rollback
+[Documentation index](docs/README.md) · [Online docs](https://oh-my-knowledge.pages.dev) · [CLI reference](docs/reference/cli.md) · [Sample format](docs/reference/eval-sample-format.md) · [Executors](docs/reference/executors.md) · [How OMK understands knowledge](docs/explanation/knowledge.md)
 
 ## Environment variables
 

package/README.zh.md

@@ -8,236 +8,64 @@
 
 [English](./README.md) | **简体中文**
 
-**Observe. Measure. Know.**
+**Observe. Measure. Know.** 让 AI 应用的知识改动有据可依。
 
-**OMK，让 AI 应用的知识改动有据可依。**
+OMK 帮助 prompt、RAG、skill 和 agent 的作者比较版本、检查证据，并从真实任务中发现知识缺口。受控比较遵循：**相同模型，相同评测用例，只改变知识载体。**
 
-观测真实表现，量出版本差异，判断改动是否有效、版本能否发布。
+当前 1.0 仍处于 **Beta 迭代期**，接口与存储契约可能继续变化。从旧版本升级前，请先阅读[迁移指南](docs/zh/guides/v1-preview-migration.md)。
 
-**相同模型，相同评测用例，只改变知识载体。**
+![OMK：从受控评测到真实使用反馈](./docs/public/omk-knowledge-flow-animated.gif)
 
-**DeepSeek Harness 用户：** OMK 可作为原生 bundle 安装，复用当前 profile 做受控评测，并在 Studio 打开已持久化的 DSH 任务轨迹。[接入 DSH 宿主插件 →](docs/zh/reference/executors.md#deepseek-harness优先使用宿主插件)
+## 从你的目标开始
 
-![omk 知识载体评测流程：doctor / eval / observe / sample / evolve 闭环](./docs/public/omk-knowledge-flow-animated.gif)
-
-📖 **完整文档：[oh-my-knowledge.pages.dev/zh](https://oh-my-knowledge.pages.dev/zh/)**（可搜索，可切换英文）
-
-## OMK 让你知道什么
-
-| 决策问题 | 命令 | 你会得到的证据 |
-|------|------|------|
-| 这份知识载体是否清楚到值得评测？ | `omk doctor` | 结构、依赖、安全性、可测性检查 |
-| v2 是否真的优于 v1？ | `omk eval` | 一行 verdict、置信区间、失败样本、成本 |
-| 它为什么通过或失败？ | `omk studio` | 分数、诊断、样本证据的报告视图 |
-| 这个版本是否应成为接受版本？ | `omk promote` / `omk evolve` | 基于证据接受，或生成更好的候选版 |
-| 一次真实 AI 任务中发生了什么？ | `omk observe` / Studio 任务轨迹 | 请求、可见知识、工具调用与结果、回答和用户纠正的可核验轨迹 |
-| 真实使用暴露了哪些知识缺口？ | `omk observe` / `omk sample --from-traces` | 将线上缺口生成待复核草稿，复核后再沉淀为评测样本 |
+| 想做什么 | 入口 |
+|---|---|
+| 比较两版 skill，获得判定、置信区间与失败用例 | [命令行快速上手](docs/zh/quickstart-skill-eval.md) |
+| 在 Node.js 服务中接入评分与版本对比 | [服务接入指南](docs/zh/guides/eval-runtime.md)，含无需模型凭证的示例 |
+| 看清一次任务，或从历史日志发现问题 | [观测与任务轨迹](docs/zh/guides/observe-production.md) |
 
 ## 快速开始
 
-```bash
-npm i -g oh-my-knowledge
-omk init demo && cd demo
-omk eval --control code-review-v1 --treatment code-review-v2 --dry-run
-omk eval --control code-review-v1 --treatment code-review-v2
-```
-
-开箱即跑：`omk init` 脚手架好两版 skill 和三条评测用例，不用先改任何文件；`--dry-run` 预览 sealed task plan 与预估调用次数；`omk eval` 跑控制变量 A/B，并在 Studio 打开经过认证的 Core run。跑通后再把 skill 和用例换成你自己的。希望首次就使用官方 20 条完整起步集时，把第一条命令换成 `omk init demo --samples 20`。
-
-前置：准备一个已认证的模型 runtime（Codex CLI、Claude Code 或 API 执行器，见[系统要求](#系统要求)）。在 ChatGPT desktop 的 Codex 任务里，omk 会自动使用 `codex`，从 `~/.codex/config.toml` 读取模型，并默认用同一个 Codex 模型担任评委，不依赖 Claude。
-
-普通终端想固定使用 Codex，可以把偏好加入 shell 配置，例如 `~/.zshrc`：
+需要 Node.js >=22 和一个已认证的模型 runtime，详见[系统要求](#系统要求)。安装 Beta 并预览第一次比较：
 
 ```bash
-export OMK_EXECUTOR=codex
-# 可选：export OMK_MODEL="你的 Codex 模型"
+npm i -g oh-my-knowledge@next
+omk init demo
+cd demo
+omk eval --control code-review-v1 --treatment code-review-v2 --dry-run
 ```
 
-不设置 `OMK_MODEL` 时，omk 会读取 `~/.codex/config.toml` 的模型。也可以继续逐次显式传 `--executor codex --model <codex-model>`。自定义评委时再传 `--judge-models` 或设置 `OMK_JUDGE_MODELS`。
-
-> 默认 3 条用例是低成本流程检查，出现「数据不足（UNDERPOWERED）」符合预期。`--samples 20` 会选择经过难度分层的官方起步用例集，达到 omk 注册的样本量下限。其来源明确标记为 `llm-generated`：它适合学习统计流程，发布判断前仍应人工复核并替换为真实领域用例。
-
-> 命令行有新版本时会自动提示（每 20 小时最多一次）；想永久关闭该提醒，设环境变量 `OMK_SKIP_UPDATE_CHECK=1` 即可。
-
-手把手教程：[5 分钟快速上手](docs/zh/quickstart-skill-eval.md)（推荐第一次跑评测的用户，覆盖 demo → 自己的 skill → verdict 动作）。更多可跑示例（Skill Map、离线执行器、agent runtime、RAG、Observe）见仓库的[示例画廊](examples/README.zh.md)。
-
-深入：[为谁、解决什么](docs/zh/explanation/who-omk-is-for.md) · [CLI 参考](docs/zh/reference/cli.md) · [工作原理](docs/zh/explanation/architecture.md) · [评测用例格式](docs/zh/reference/eval-sample-format.md) · [执行器](docs/zh/reference/executors.md) · [知识载体布局](docs/zh/reference/artifact-layout.md)
-
-## 先看清一次 Codex 任务
-
-只想知道一次 Codex 对话背后发生了什么，不需要先运行 `observe ingest`：
+检查计划和预估调用次数后，执行评测（会调用模型）：
 
 ```bash
-omk studio
-```
-
-Studio 默认在 `http://127.0.0.1:7799` 打开本机 Codex 对话总览。先选择一段对话，再选择其中一次任务，即可进入「任务轨迹」：按**对话、执行、结果、知识**四条泳道查看请求、AI 回答、工具调用、工具返回和可见上下文，并可下钻到规范化事件与原始日志。
-
-进行中的任务会优先显示并实时更新。保持「跟随中」时，轨迹会随新事件平滑前进；手动查看历史位置后，页面保留当前位置并提示「查看更新」。旧日志如果没有记录结束事件，会标记为「未记录结束状态」，不会一直冒充进行中。
-
-任务轨迹只还原日志中可观测的事实，不展示或推断隐藏思维。完整说明见[观测与任务轨迹](docs/zh/guides/observe-production.md#查看一次任务)。
-
-## OMK 的闭环
-
-OMK 主要给 LLM 知识载体的作者 / 维护者用，帮他们做发布判断；它不是给被动安装 skill 的普通使用者用的。主流程刻意保持受控：
-
-```text
-改了一份 prompt / RAG / skill / agent 知识载体
-→ 先跑 omk doctor
-→ 用相同模型、相同评测用例跑 omk eval
-→ 看 report / Studio 里的证据
-→ 证据足够则 promote，证据不足则 evolve 候选版
-→ observe 真实使用，把缺口生成待复核评测草稿
+omk eval --control code-review-v1 --treatment code-review-v2
 ```
 
-第一价值是发布前的 `doctor → eval` 判断。长期价值是闭环：`observe` 暴露真实使用里的知识缺口，`sample --from-traces` 先生成待人工复核的评测用例草稿，复核后的草稿再沉淀为固定评测样本，下一次 `eval` 就更难被偶然样本骗过。
+脚手架包含两版 skill 和三条用例。默认小样本用于检查流程，出现 `UNDERPOWERED` 符合预期。官方起步用例标记为 `llm-generated`；即使用 `omk init demo-full --samples 20` 创建完整起步集，也只是达到默认启发式证据下限，不等于完成功效规划或具备发布证据。实际使用前应人工复核并替换为真实领域用例。
 
-## 在 AI Coding Agent 中使用
+[完整教程](docs/zh/quickstart-skill-eval.md)介绍模型选择、换成自己的 skill 和结果解读；[示例画廊](examples/README.zh.md)提供更多可运行场景。
 
-安装 omk 官方 Agent Skill 后，可以直接用自然语言让 coding agent 跑 omk 工作流：
+## 在 Agent 中使用
 
 ```bash
 omk install omk-agent-skill
 ```
 
-默认只会安装到本机已检测到、且 omk 明确支持的目标：检测到 `~/.codex` 或 `~/.agents` 时写入 Codex/AGENTS，检测到 `~/.claude` 时写入 Claude Code。要强制写入当前 omk 已知的全部目标，用 `--to all`；要指定自定义 skill 根目录，用 `--dest`。
-
-### 在 Claude Code 中使用
-
-当 `omk` skill 已在 Claude Code 中可用时，可以直接这样调用：
-
-```bash
-/omk eval              # 评测当前项目的知识载体
-/omk evolve            # 多轮自动迭代改进 skill
-/omk sample            # 生成或补齐评测用例
-```
-
-这些 slash command 是自然语言入口 —— agent 会从对话上下文里推断要操作哪个 skill。也可以直接说「帮我评测 v1 和 v2 的差异」、「改进一下这个知识载体」，omk 会自动理解意图并调用对应命令。
-
-### 在 Codex 中使用
+安装后，可以对 coding agent 说：“用 omk 比较这两版 skill。”安装目标与使用方法见[快速上手](docs/zh/quickstart-skill-eval.md#在-agent-中使用)。DeepSeek Harness 用户可直接使用[宿主插件](docs/zh/reference/executors.md#deepseek-harness优先使用宿主插件)，复用当前 profile。
 
-Codex 默认不支持 `/omk ...` 这种 Claude Code 风格的 slash command。直接让 agent 执行 `omk` CLI 即可；在 Codex 任务里，omk 会自动选择 Codex runtime 和本机配置的模型：
+[MCP 集成](docs/zh/guides/mcp-integration.md)提供用户授权的主动知识反馈入口。它仅记录提交到工具边界的部分证据，不自动监听完整对话。
 
-```bash
-omk eval
-omk evolve skills/my-skill.md   # 一键:体检 →(无用例则自动生成)→ 自迭代
-omk sample skills/my-skill.md
-```
-
-也可以直接用自然语言描述目标，例如「比较 v1 和 v2 的评测差异」、「为这个 skill 生成评测用例」。
-
-`eval`、`doctor`、`sample`、`evolve` 和 observe 的 LLM 增强复盘共用同一套 runtime 解析。Codex 被选中后，默认评委沿用被测 Codex 模型，不会回落到 `claude:haiku`。
-
-> `omk evolve` 是一键闭环：默认先跑 doctor 体检，目标 skill 没有评测用例时会自动生成一批，再进入多轮自迭代。全新 skill 直接 `omk evolve skills/foo.md` 即可。
-
-## 为什么需要这个工具
-
-知识工程带来的是一个版本治理问题：prompt、RAG 配方、skill、agent、workflow 都会改变模型行为，但这些改动未必体现在应用代码里。当有人追问「v2 能不能发、为什么」时，回答更顺眼、体感更好，远远不够。
-
-omk 把知识载体当作被测变量：**相同模型、相同评测用例，只改变知识载体。** 这样得到的对比才可解释、可复跑，也适合进入 CI 或发布评审。
-
-## 为什么选 omk
-
-| | omk | promptfoo | DeepEval | LangSmith |
-|--|--|--|--|--|
-| Bootstrap 置信区间 | ✓ 默认 | ✗ | ✗ | ✗ |
-| Krippendorff α（评委 ↔ 人工） | ✓ 加 gold 即开 | ✗ | ✗ | ✗ |
-| 长度去偏的评委 prompt | ✓ 默认 | ✗ | ✗ | ✗ |
-| 缺失证据失败关闭 | ✓ | ✗ | ✗ | ✗ |
-| 三层独立评分 | ✓ | ✗ | 部分 | ✗ |
-| 用例隔离(construct validity) | ✓ 默认 | ✗ | ✗ | ✗ |
-| 原生 Agent Skill | ✓ | ✗ | ✗ | ✗ |
-| 托管 SaaS 看板 | ✗ | ✗ | ✓ | ✓ |
-
-omk 的护城河是 **default-on 安全网**：Bootstrap CI 与长度去偏属于正常测量行为，缺失证据失败关闭，显式 Gold comparison 提供评委 ↔ 人工 alpha 校准。需要 SaaS 看板？选 LangSmith。要快速 prompt 迭代不要统计层？选 promptfoo。**要发到生产且会被问「为什么应该相信这个数字」？选 omk。**
-
-RAG 专项评测请看 RAGAS（独立 niche，跟 omk 互补）。完整对比（7 个工具 × 25+ 维度）： [docs/zh/reference/comparison.md](docs/zh/reference/comparison.md)
-
-## 特性
-
-| 特性 | 说明 |
-|------|------|
-| **Core 发布决定** | 六种结论 + 稳定 reason code + exit code 路由；Studio 投影同一份经过认证的 Decision |
-| **五层 evidence graph** | Assertion / LLM / Judge / Dimension / Composite 保持独立，coverage、成本、状态与 lineage 与分数正交 |
-| **多执行器** | 支持 Claude CLI / Claude SDK / Codex CLI / Codex SDK / DeepSeek Harness / OpenAI / Anthropic API 及自定义命令 |
-| **30+ 种断言** | 包含子串、正则、JSON Schema、ROUGE/BLEU/Levenshtein 相似度、Agent 工具调用、语义相似度、自定义函数等 |
-| **统计严谨性** | Bootstrap comparison family、长度去偏、缺失证据失败关闭与显式 Gold agreement 校准。[详情 →](docs/zh/explanation/statistical-rigor.md) |
-| **RAG metrics** | `faithfulness` / `answer_relevancy` / `context_recall` 三 metric — 反幻觉 + 切题度 + context 覆盖 |
-| **LLM 健康度审计** | `omk doctor` 给 7 个内置维度独立打分；重复采样（`--repeat`）+ k/n 共识归并 |
-| **线上 session 观测** | 将 Codex、Claude Code、OpenClaw 与 markdown 日志统一为 source-neutral Trace IR，测量各 skill 的执行结果、耗时、token 使用和知识缺口信号 |
-| **MCP 主动知识反馈（实验性）** | 由 MCP 客户端主动调用工具，把用户明确授权的 knowledge 反馈写入 Observation Inbox；不监听对话，并固定标记 `coverage: partial` |
-| **知识缺口识别** | 严重度加权的信号量化风险敞口，不宣称完备性 |
-| **用例隔离 (construct validity)** | `--strict-baseline`（默认开）三堵 baseline 拿到被测 skill 的污染路径 |
-| **Git / 远端源** | install / eval 支持本地 git ref 或远端 git URL（`--git-url`）；目录-skill 在内容寻址**隔离副本**里执行，`references/` 资产是真实测量输入，不只是 `SKILL.md` |
-| **证据门控管理** | `omk install` 登记受管记录；`omk eval` 按内容指纹自动写入证据，把 skill 从 `installed` 推到 `measurable`；`omk list` 查看各受管 skill 的状态（installed / measurable / promoted / stale）；`omk promote` 在证据过门禁（默认仅 PROGRESS）后把该版本接受为当前版本；`omk rollback` 撤销这次接受，让 skill 回到 `measurable`。[规范 →](docs/zh/specs/evidence-gated-management.md) |
-| **用例设计科学性** | Sample schema 加 `capability` / `difficulty` / `construct` / `provenance` 元数据字段（HF Dataset Cards 风），studio 输出 coverage 分桶 + `rubric_clarity_low` / `capability_thin` issue。[docs/zh/specs/sample-design-spec.md](docs/zh/specs/sample-design-spec.md) |
-| **多评委 ensemble** | `--judge-models claude:opus,openai-api:gpt-4o` 跨厂商评分 + agreement 度量 |
-| **多轮方差分析** | `--repeat N` 发布相互独立的 Core run 与 Evaluation Series 方差分析 |
-| **MCP URL 获取** | 通过 MCP Server 获取私有文档 URL 内容（SSO 保护的知识库等） |
-| **自动分析** | 检测低区分度断言、均匀分数、全通过/全失败、高成本用例 |
-| **可追溯性** | 报告含 CLI 版本、Node 版本、知识载体版本指纹、judge prompt hash |
-| **中英视图** | 通过报告 URL 选择中英文的本地 Studio 视图 |
-
-### 在已有 DeepSeek Harness 中运行
-
-OMK 可以作为 DSH bundle 安装到现有 profile，直接复用其模型、凭证、工具与 sandbox：
-
-```bash
-dsh plugin --profile web add oh-my-knowledge
-dsh --profile web
-```
-
-进入 DSH 后：
-
-- `/omk eval eval.yaml`：每条用例使用独立 DSH session，报告仍由 OMK 生成；
-- `/omk observe`：列出最近已结束的 session；
-- `/omk observe <session-id>`：只读摄取一致快照，并返回 Studio 任务轨迹链接。
-
-observe 直接使用 profile 的 `sessionPersistence`，无需导出或定位 JSONL／SQLite 文件；首版不实时跟随正在写入的 session。详见[执行器文档](docs/zh/reference/executors.md#deepseek-harness优先使用宿主插件)与[观测指南](docs/zh/guides/observe-production.md#在-deepseek-harness-中查看任务轨迹)。
-
-### 连接 MCP 客户端（实验性）
-
-> **定位：OMK MCP 是主动知识反馈接口，不是对话监听器。** 仅靠 OMK MCP 无法自动监听或订阅 Codex、ChatGPT 等客户端的完整对话。只有客户端、模型或 component 主动调用 `save_observation` 并提交授权内容后，OMK 才能收到并保存这条反馈。Agent Skill 可以自动识别潜在反馈时机，但识别结果仍须经过用户确认和一次显式 MCP 工具调用。
-
-`omk-mcp` 提供与客户端无关的 stdio MCP Server。Codex 等本地 MCP 客户端可直接启动它，私有宿主也可组合导出的 Streamable HTTP adapter。客户端只在用户明确要求记录时调用 `save_observation`，把反馈和可选证据追加到 `.omk/observe/inbox/captures/`，并可渲染对话内 MCP Apps 复核卡片，供人工确认问题和生成 regression sample 草稿。
-
-在 Codex 中可以显式调用 OMK Skill 快捷提交当前知识反馈：
-
-```text
-$omk feedback
-```
-
-这次显式调用本身视为保存确认。Agent 会从当前可见对话中选择最近一个明确问题，以 `confirmedByUser: true` 调用 `save_observation`；候选不明确时会先追问。该快捷入口不是 CLI 命令，也不会自动复核、生成 sample 或写入 gold set。
-
-```bash
-omk-mcp
-```
+## 如何使用证据
 
-每条记录都固定携带 `coverageStatus: partial`：已观测的是 OMK 工具边界、用户提交的反馈及可选证据；未观测的是完整对话、其他工具调用和隐藏推理。需要持续监听时，必须由有权访问事件流的宿主系统主动转交事件，这属于宿主集成能力，不属于 OMK MCP 自身能力。对话 ID、turn ID 与幂等键只用于生成哈希，不会原样落盘。私有宿主的 Streamable HTTP 组合方式见[组合 OMK MCP 集成](docs/zh/guides/mcp-integration.md)。
+`doctor` 检查知识载体，`eval` 做受控比较，`studio` 展示结果与原始证据。证据满足条件后可以 `promote` 接受版本，或用 `evolve` 生成候选；`observe` 暴露的缺口可转成待复核用例，再进入下一轮评测。
 
-### 本地存储
+结论受用例、评分准则和执行环境约束。观测信号不等于因果结论，生成的样本也不能直接充当独立发布验证集。原理与限制见[统计严谨性](docs/zh/explanation/statistical-rigor.md)和[三阶段工作流](docs/zh/explanation/three-stage-workflow.md)。
 
-项目数据采用领域化的 `.omk/` v2 布局：持久证据进入 `eval/`、`doctor/`、`observe/`，治理记录进入 `governance/`，备份保持可恢复，只有可重建工作进入 `state/`。机器工具、隧道、缓存和物化副本只能进入 `~/.oh-my-knowledge/state/`，不得写进项目。本期不读取、也不迁移旧存储布局。
+项目证据保存在 `.omk/`，机器级状态位于 `~/.oh-my-knowledge/`。当前版本不读取或迁移旧存储布局；升级前备份，按[迁移指南](docs/zh/guides/v1-preview-migration.md)重新建立证据。
 
 ## 文档
 
-完整文档已发布到 **[oh-my-knowledge.pages.dev/zh](https://oh-my-knowledge.pages.dev/zh/)** —— 可搜索，可切换英文。重点页面：
-
-- **[工作原理](docs/zh/explanation/architecture.md)** —— 输入编译、sealed Core 执行、分析、持久化与 Studio projection
-- **[评测用例格式](docs/zh/reference/eval-sample-format.md)** —— sample schema、评分公式、30+ 断言类型、自定义 JS 断言
-- **[CLI 参考](docs/zh/reference/cli.md)** —— 顶层命令的 bash 示例和 flag 表
-- **[迁移到 1.0 预览版](docs/zh/guides/v1-preview-migration.md)** —— 从 `0.54` 升级时的安装渠道、存储重建、用例协议、CLI 自动化与嵌入式 API 变化
-- **[Evaluation Core 生产切换](docs/zh/guides/eval-core-cutover.md)** —— `BREAKING-SCHEMA` 存储、resume、Studio、Gold、受管证据与 evolve 迁移
-- **[存储布局 v2](docs/zh/specs/storage-layout-spec.md)** —— 项目／全局领域、迁移兼容与 Git 策略
-- **[执行器](docs/zh/reference/executors.md)** & **[知识载体布局](docs/zh/reference/artifact-layout.md)** —— 内置 / 自定义执行器；variant 如何解析为 artifact + runtime context
-- **[操作指南](docs/zh/guides/agent-eval.md)** —— [评测 agent](docs/zh/guides/agent-eval.md)（项目 runtime context）与[使用非 Claude 模型](docs/zh/guides/non-claude-models.md)（GLM / 通义 / DeepSeek / Moonshot / Ollama）
-- **[观测与任务轨迹](docs/zh/guides/observe-production.md)** —— 浏览本机 Codex 对话，下钻一次任务，并实时跟随可观测执行过程
-- **[快速上手](docs/zh/quickstart-skill-eval.md)** —— 第一次跑评测的 5 分钟教程
-- **[示例画廊](https://github.com/lizhiyao/oh-my-knowledge/tree/main/examples)** —— 仓库里一组可直接跑的示例，按由简到全排成上手路径
-- **[用例设计规范](docs/zh/specs/sample-design-spec.md)** —— capability / construct / provenance 元数据；行业 gap 映射
-- **[统计严谨性](docs/zh/explanation/statistical-rigor.md)** —— 为什么 Bootstrap CI / Gold agreement / 长度去偏 / evidence coverage 重要
-- **[7 工具对比](docs/zh/reference/comparison.md)** —— promptfoo / DeepEval / RAGAS / OpenAI Evals / LangSmith / lm-eval-harness / inspect-ai 等 25+ 维度横评
-- **[证据门控管理](docs/zh/specs/evidence-gated-management.md)** —— 受管记录、生命周期状态（installed / measurable / promoted / stale）、install → eval → measurable → promote → rollback
+[完整文档与导航](docs/zh/README.md) · [在线文档](https://oh-my-knowledge.pages.dev/zh/) · [CLI 参考](docs/zh/reference/cli.md) · [用例格式](docs/zh/reference/eval-sample-format.md) · [执行器](docs/zh/reference/executors.md) · [OMK 如何理解知识](docs/zh/explanation/knowledge.md)
 
 ## 环境变量
 

package/dist/assets/agent-skills/omk/SKILL.md

@@ -26,7 +26,7 @@ argument-hint: "<doctor|eval|evolve|init|install|list|observe|promote|rollback|s
 除 `$omk feedback` 快捷入口外，运行 `which omk` 检查是否已安装。如果未安装，告诉用户：
 
 ```
-npm i oh-my-knowledge -g
+npm i -g oh-my-knowledge@next
 ```
 
 omk CLI 顶层命令包括：`init` / `install` / `list` / `promote` / `rollback` / `doctor` / `eval` / `observe` / `evolve` / `sample` / `studio`。没有 `bench` / `improve` / `gen-samples` 这些旧子命令名 —— 如果你在历史 SKILL / 文档里看到了，那是 v0.30 命令树重构之前的写法。
@@ -191,7 +191,7 @@ omk studio --host 0.0.0.0                 # 局域网访问（默认 127.0.0.1
 omk studio --no-open                      # 不自动开浏览器
 ```
 
-Studio 首页直接索引本机 Codex 对话。先选择对话，再选择任务查看四泳道任务轨迹；进行中的任务支持实时跟随。顶部「知识载体」入口用于浏览 doctor / eval / observe 报告，`/observe-inbox` 用于复核 observation。无需为了浏览本机 Codex 对话而先运行 `omk observe ingest`。
+Studio 首页直接索引本机 Codex 对话。先选择对话，再选择任务查看四泳道任务轨迹；进行中的任务支持实时跟随。顶部「知识载体」入口用于浏览 doctor / eval / observe 报告，`/observe/inbox` 用于复核 observation。无需为了浏览本机 Codex 对话而先运行 `omk observe ingest`。
 
 ## 第五步：解读结果
 
@@ -227,15 +227,23 @@ v2 比 v1 更好（verdict: PROGRESS，Δ=+0.7，95% CI [+0.3, +1.1]）：
 
 ## 指定工作目录（cwd）
 
-当评测用例需要模型读取特定仓库的代码时，可在 sample 中设置 `cwd` 字段：
+当评测用例需要模型读取特定仓库的代码时，可在 sample 中设置 `executionContext.cwd` 字段：
 
 ```yaml
-- sample_id: task-001
-  prompt: "实现用户登录功能，要求支持手机号和邮箱两种方式"
-  cwd: "/path/to/target-repo"
-  assertions:
-    - type: contains_all
-      values: ["auth.ts", "login.tsx"]
+schemaVersion: omk.eval-sample-set/v3
+samples:
+  - sampleId: task-001
+    input:
+      inputKind: text
+      text: 实现用户登录功能，要求支持手机号和邮箱两种方式
+    executionContext:
+      cwd: /path/to/target-repo
+    evaluationContext:
+      assertions:
+        - type: contains_all
+          values:
+            - auth.ts
+            - login.tsx
 ```
 
 `cwd` 会作为 executor 的工作目录，Codex / Claude 等 agent runtime 会在该目录下运行并读取仓库代码。适用于「给一个任务 query，断言应该修改哪些文件」的 A/B 评测场景。