agent-tune-kit 0.4.2__tar.gz → 0.4.4__tar.gz

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/.codex-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-tune-kit",
-  "version": "0.4.2",
+  "version": "0.4.4",
   "description": "Local Codex plugin for iterative Agent tuning with guided Skills, reusable runner templates, versioned results, and static validation.",
   "author": {
     "name": "hustyichi",

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agent-tune-kit
-Version: 0.4.2
+Version: 0.4.4
 Summary: Local Codex plugin for iterative Agent tuning with guided Skills, reusable runner templates, versioned results, and static validation.
 Project-URL: Homepage, https://github.com/hustyichi/agent-tune-kit
 Project-URL: Repository, https://github.com/hustyichi/agent-tune-kit
@@ -89,6 +89,27 @@ $atk-new-agent dataset is data/eval.csv
 
 Codex inspects the dataset, clarifies your intent, generates a minimal Agent project, and writes the interview and design notes to `.atk/specs/agent_spec.md`. This step does not write `.atk/datasets/dataset.csv`; `$atk-init` still owns dataset validation, normalization, and runner generation when connecting the Agent.
 
+If `.atk/datasets/dataset.csv` already exists but the expected-result semantics are missing or too weak for stable failure judgment, enrich the existing dataset before running evaluation:
+
+```text
+$atk-build-ground-truth
+```
+
+Codex asks you to choose one dataset-wide `ground_truth` style: exact answer or natural-language acceptance criteria. It then updates `.atk/datasets/dataset.csv` in place, preserving existing columns and `atk_id`. This is a dataset-only, pre-results step: it does not run the Agent, create `.atk/results/vN`, or write `failure_cases.csv`. If existing `eval_results.csv` was produced before this enrichment, rerun `$atk-run` before finding failures.
+Once the dataset is built, you can optionally review and quality-check it first:
+
+```text
+$atk-visualize-dataset
+```
+
+It renders `.atk/datasets/dataset.csv` into a local, single-file, offline HTML browser:
+
+```text
+.atk/datasets/dataset.html
+```
+
+The page uses a Dataset Visualizer-style offline static interface with Data List / Field Feature Analysis tabs, column visibility controls, search, category filtering, sorting, pagination, and a detail drawer. It highlights the input-vs-ground_truth comparison and auto-flags issues such as empty values, duplicate/missing `atk_id`, conflicting samples, and length outliers, so you can quickly confirm whether each ground_truth matches expectations. You can mark a verdict per row (matches / suspect / needs fix) and export `dataset_review.csv`, then return to `$atk-build-dataset` to fix it; reviewers do not need to install any frontend dependencies.
+
 ### 1. Initialize
 
 Tell Codex where your Agent starts and where the evaluation data lives:
@@ -226,6 +247,8 @@ Common output files:
 
 - `$atk-status`: inspect progress and suggest the next step.
 - `$atk-build-dataset`: build `.atk/datasets/dataset.csv` from business context, examples, or rules.
+- `$atk-build-ground-truth`: enrich an existing `.atk/datasets/dataset.csv` with a canonical `ground_truth` column.
+- `$atk-visualize-dataset`: render `.atk/datasets/dataset.csv` into a local HTML browser for quickly reviewing rows and confirming whether each ground_truth matches expectations.
 - `$atk-new-agent`: create a lightweight OpenAI-compatible Agent when you only have a dataset.
 - `$atk-init`: generate the test runner.
 - `$atk-run`: run evaluation and create a new result version.

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/README.en.md

@@ -75,6 +75,27 @@ $atk-new-agent dataset is data/eval.csv
 
 Codex inspects the dataset, clarifies your intent, generates a minimal Agent project, and writes the interview and design notes to `.atk/specs/agent_spec.md`. This step does not write `.atk/datasets/dataset.csv`; `$atk-init` still owns dataset validation, normalization, and runner generation when connecting the Agent.
 
+If `.atk/datasets/dataset.csv` already exists but the expected-result semantics are missing or too weak for stable failure judgment, enrich the existing dataset before running evaluation:
+
+```text
+$atk-build-ground-truth
+```
+
+Codex asks you to choose one dataset-wide `ground_truth` style: exact answer or natural-language acceptance criteria. It then updates `.atk/datasets/dataset.csv` in place, preserving existing columns and `atk_id`. This is a dataset-only, pre-results step: it does not run the Agent, create `.atk/results/vN`, or write `failure_cases.csv`. If existing `eval_results.csv` was produced before this enrichment, rerun `$atk-run` before finding failures.
+Once the dataset is built, you can optionally review and quality-check it first:
+
+```text
+$atk-visualize-dataset
+```
+
+It renders `.atk/datasets/dataset.csv` into a local, single-file, offline HTML browser:
+
+```text
+.atk/datasets/dataset.html
+```
+
+The page uses a Dataset Visualizer-style offline static interface with Data List / Field Feature Analysis tabs, column visibility controls, search, category filtering, sorting, pagination, and a detail drawer. It highlights the input-vs-ground_truth comparison and auto-flags issues such as empty values, duplicate/missing `atk_id`, conflicting samples, and length outliers, so you can quickly confirm whether each ground_truth matches expectations. You can mark a verdict per row (matches / suspect / needs fix) and export `dataset_review.csv`, then return to `$atk-build-dataset` to fix it; reviewers do not need to install any frontend dependencies.
+
 ### 1. Initialize
 
 Tell Codex where your Agent starts and where the evaluation data lives:
@@ -212,6 +233,8 @@ Common output files:
 
 - `$atk-status`: inspect progress and suggest the next step.
 - `$atk-build-dataset`: build `.atk/datasets/dataset.csv` from business context, examples, or rules.
+- `$atk-build-ground-truth`: enrich an existing `.atk/datasets/dataset.csv` with a canonical `ground_truth` column.
+- `$atk-visualize-dataset`: render `.atk/datasets/dataset.csv` into a local HTML browser for quickly reviewing rows and confirming whether each ground_truth matches expectations.
 - `$atk-new-agent`: create a lightweight OpenAI-compatible Agent when you only have a dataset.
 - `$atk-init`: generate the test runner.
 - `$atk-run`: run evaluation and create a new result version.

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/README.md

@@ -75,6 +75,27 @@ $atk-new-agent 数据集是 data/eval.csv
 
 Codex 会先理解数据集并确认你的意图，生成最小 Agent 项目，并把采访和设计信息写入 `.atk/specs/agent_spec.md`。这个步骤不会写 `.atk/datasets/dataset.csv`；后续仍由 `$atk-init` 负责接入 Agent 时的数据集校验、规范化和 runner 生成。
 
+如果 `.atk/datasets/dataset.csv` 已经存在，但缺少清晰的预期结果语义，导致后续异常判断不稳定，可以在跑评测前先补齐统一的 ground truth：
+
+```text
+$atk-build-ground-truth
+```
+
+Codex 会让你选择一种全局一致的 `ground_truth` 风格：精确答案，或自然语言验收标准。它会在保留原有列和 `atk_id` 的前提下就地更新 `.atk/datasets/dataset.csv`。这是 dataset-only / pre-results 步骤：不会运行 Agent，不会创建 `.atk/results/vN`，也不会写 `failure_cases.csv`。如果已有 `eval_results.csv` 是补齐 ground truth 之前生成的，请先重新运行 `$atk-run` 再找异常样本。
+构建好数据集后，可以选择先浏览和质检它：
+
+```text
+$atk-visualize-dataset
+```
+
+它会把 `.atk/datasets/dataset.csv` 生成一个本地、单文件、离线的 HTML 浏览页：
+
+```text
+.atk/datasets/dataset.html
+```
+
+页面采用 Dataset Visualizer 风格的离线静态界面，提供「数据列表 / 字段特征分析」双标签、字段显示控制、搜索、分类过滤、排序分页和详情抽屉，突出「输入 vs ground_truth」对照，并对空值、重复/缺失 `atk_id`、冲突样本、长度离群等问题自动打标签，便于快速确认 ground_truth 是否符合预期。你可以在页面里对每行裁决（符合预期 / 存疑 / 需修正）并导出 `dataset_review.csv`，再回到 `$atk-build-dataset` 修复；用户不需要安装任何前端依赖。
+
 ### 1. 初始化
 
 告诉 Codex 你的 Agent 从哪里启动、评估数据在哪里：
@@ -212,6 +233,8 @@ $atk-report
 
 - `$atk-status`：检查当前进度并提示下一步。
 - `$atk-build-dataset`：从业务描述、样例或规则构建 `.atk/datasets/dataset.csv`。
+- `$atk-build-ground-truth`：为现有 `.atk/datasets/dataset.csv` 补齐规范的 `ground_truth` 列。
+- `$atk-visualize-dataset`：将 `.atk/datasets/dataset.csv` 生成本地 HTML 浏览页，便于快速查看数据并确认 ground_truth 是否符合预期。
 - `$atk-new-agent`：当你只有数据集、还没有 Agent 项目时，生成一个轻量 OpenAI-compatible Agent。
 - `$atk-init`：生成测试脚本。
 - `$atk-run`：运行评测并生成新版本结果。

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/README.zh-CN.md

@@ -75,6 +75,27 @@ $atk-new-agent 数据集是 data/eval.csv
 
 Codex 会先理解数据集并确认你的意图，生成最小 Agent 项目，并把采访和设计信息写入 `.atk/specs/agent_spec.md`。这个步骤不会写 `.atk/datasets/dataset.csv`；后续仍由 `$atk-init` 负责接入 Agent 时的数据集校验、规范化和 runner 生成。
 
+如果 `.atk/datasets/dataset.csv` 已经存在，但缺少清晰的预期结果语义，导致后续异常判断不稳定，可以在跑评测前先补齐统一的 ground truth：
+
+```text
+$atk-build-ground-truth
+```
+
+Codex 会让你选择一种全局一致的 `ground_truth` 风格：精确答案，或自然语言验收标准。它会在保留原有列和 `atk_id` 的前提下就地更新 `.atk/datasets/dataset.csv`。这是 dataset-only / pre-results 步骤：不会运行 Agent，不会创建 `.atk/results/vN`，也不会写 `failure_cases.csv`。如果已有 `eval_results.csv` 是补齐 ground truth 之前生成的，请先重新运行 `$atk-run` 再找异常样本。
+构建好数据集后，可以选择先浏览和质检它：
+
+```text
+$atk-visualize-dataset
+```
+
+它会把 `.atk/datasets/dataset.csv` 生成一个本地、单文件、离线的 HTML 浏览页：
+
+```text
+.atk/datasets/dataset.html
+```
+
+页面采用 Dataset Visualizer 风格的离线静态界面，提供「数据列表 / 字段特征分析」双标签、字段显示控制、搜索、分类过滤、排序分页和详情抽屉，突出「输入 vs ground_truth」对照，并对空值、重复/缺失 `atk_id`、冲突样本、长度离群等问题自动打标签，便于快速确认 ground_truth 是否符合预期。你可以在页面里对每行裁决（符合预期 / 存疑 / 需修正）并导出 `dataset_review.csv`，再回到 `$atk-build-dataset` 修复；用户不需要安装任何前端依赖。
+
 ### 1. 初始化
 
 告诉 Codex 你的 Agent 从哪里启动、评估数据在哪里：
@@ -212,6 +233,8 @@ $atk-report
 
 - `$atk-status`：检查当前进度并提示下一步。
 - `$atk-build-dataset`：从业务描述、样例或规则构建 `.atk/datasets/dataset.csv`。
+- `$atk-build-ground-truth`：为现有 `.atk/datasets/dataset.csv` 补齐规范的 `ground_truth` 列。
+- `$atk-visualize-dataset`：将 `.atk/datasets/dataset.csv` 生成本地 HTML 浏览页，便于快速查看数据并确认 ground_truth 是否符合预期。
 - `$atk-new-agent`：当你只有数据集、还没有 Agent 项目时，生成一个轻量 OpenAI-compatible Agent。
 - `$atk-init`：生成测试脚本。
 - `$atk-run`：运行评测并生成新版本结果。

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/docs/codex_agent_tuning_prd.md

@@ -31,6 +31,7 @@
   - 用户需要准备需要调优的 Agent 服务以及可以用于评估的数据集
   - 系统需要在项目内维护统一父目录 `.atk/`
   - 用户可先通过 `$atk-build-dataset` 从业务描述、样例或验收规则构建初始 `.atk/datasets/dataset.csv`
+  - 当已有数据集缺少清晰预期结果语义时，用户可通过 `$atk-build-ground-truth` 为现有 `.atk/datasets/dataset.csv` 补齐规范 `ground_truth` 列，以提升后续异常判断稳定性
   - 用户提供或构建的数据集需要在生成测试脚本时由 `$atk-init` 校验/规范化写入 `.atk/datasets/`，并保证存在 `atk_id`
   - 所有共享脚本需要存储在 `.atk/runner/`
   - 所有版本化结果需要存储在 `.atk/results/{version}/`
@@ -61,7 +62,29 @@
   - `.atk/datasets/dataset.csv`
   - 下一步建议需根据 Agent 是否已存在分流：已有 Agent 时提示通过 `$atk-init` 接入新数据集并开始批量评测；没有 Agent 时提示先通过 `$atk-new-agent` 基于数据集创建 Agent；无法确认时同时说明两种可能路径
 
-### 2.1.1 ATK new Agent：仅有数据集时初始化 Agent 项目（可选前置 Skill）
+### 2.1.1 ATK build Ground Truth：为现有数据集补齐统一预期语义（可选前置 Skill）
+- **适用场景**：
+  - 用户已经有 `.atk/datasets/dataset.csv`，但缺少清晰的预期结果、答案或验收标准列
+  - 用户希望在不改变异常筛选 Skill 行为的前提下，提高后续 `$atk-find-failures` 对“Agent 输出 vs 预期结果”的判断稳定性
+- **入口**：
+  - Codex Skill：`$atk-build-ground-truth`
+  - 产品名：`ATK build Ground Truth`
+- **功能**：
+  - 读取现有 `.atk/datasets/dataset.csv`，要求存在非空、唯一、正整数 `atk_id`
+  - 检查 `expected`、`expected_output`、`label`、`answer`、`target`、`acceptance_criteria`、`notes` 以及其它业务列，识别可作为 ground truth 来源的字段
+  - 在生成前要求用户选择一种全局一致的 `ground_truth` 风格：精确答案，或自然语言验收标准；不得在同一数据集中静默混用两种风格
+  - 在保留原有列和相对顺序的基础上补齐规范 `ground_truth` 列；若该列已存在或会语义替换 expected-like 字段，必须展示候选修改摘要并要求用户显式确认
+  - 候选修改摘要应包含受影响行数和代表性示例；大数据集不需要逐行列出全部变更
+- **边界**：
+  - 这是 dataset-only / pre-results Skill；不创建 `.atk/results/vN`，不运行 Agent / `atk-run` / eval runner，不写 `failure_cases.csv`
+  - 第一版不改变 `$atk-find-failures` 行为，也不强制所有 ATK 数据集都必须拥有 `ground_truth`
+  - 不自动挖掘大规模生产日志，不在常规路径创建候选数据集文件，不发明缺失领域事实
+  - 当数据集包含多个互不兼容任务、所需领域事实缺失、已有 expected-like 语义冲突或风格选择不明确时，必须先向用户确认
+- **输出**：
+  - 更新后的 `.atk/datasets/dataset.csv`，保留原有列并包含规范 `ground_truth`
+  - 下一步建议：若现有 `eval_results.csv` 早于本次数据集补齐，提示重新运行 `$atk-run`；若当前评测结果已反映补齐后的数据集，可继续 `$atk-find-failures`
+
+### 2.1.2 ATK new Agent：仅有数据集时初始化 Agent 项目（可选前置 Skill）
 - **适用场景**：
   - 用户只有评估数据集，还没有可被 `$atk-init` 接入的本地 Agent 项目
   - 用户希望 Codex 先根据数据集和意图生成一个轻量初始 Agent，再进入现有 ATK 调优闭环
@@ -141,7 +164,7 @@
 - **写入行为**：规则执行与大模型模式写入的 `failure_cases.csv` 文件名一致；若当前版本目录下已存在该文件，默认覆盖、不做备份或合并，并按各 Skill 的确认规则在覆盖前提示风险
 - **异常定义来源**：
   - 规则模式由用户在 Skill 交互中显式给出
-  - 大模型模式由用户给出自然语言说明，或在用户未指定时由 Skill 基于"Agent 输出 vs 预期结果"自行推断
+  - 大模型模式由用户给出自然语言说明，或在用户未指定时由 Skill 基于"Agent 输出 vs 预期结果"自行推断；当数据集中存在由 `$atk-build-ground-truth` 生成的规范 `ground_truth` 列时，可作为优先的预期语义证据，但第一版不要求改变异常筛选实现行为
 - **数据确认机制**：当 `eval_results.csv` 字段含义不明、缺少预期结果或无法推断异常标准时，Skill 与用户确认后再继续
 - **输出**：
   - 规则初始化生成共享规则脚本 `.atk/runner/failure_rule.py`
@@ -229,6 +252,30 @@
   - 所有跨版本读取均为 best-effort：任意历史文件缺失/超限/损坏/无 `atk_id` 列时，对应视图降级并提示原因，绝不影响 HTML 写入
   - `agent_output_log_path` 等日志路径只在满足安全相对路径约束时生成可点击链接，否则仅作为证据文本展示
 
+### 2.5.2 数据集可视化与质检（可选 Codex Skill `atk-visualize-dataset`）
+- **功能**：
+  - 从 `.atk/datasets/dataset.csv` 生成便于人工浏览与质检的静态 HTML 浏览器 `.atk/datasets/dataset.html`
+  - 通过插件内固定 Python 标准库脚本生成 HTML，避免模型临场生成可视化、LLM 摘要、项目内模板漂移或额外依赖
+  - 该 Skill 是独立的可选审阅步骤，不并入 `atk-build-dataset` 或 `atk-init`，也不改变数据集构建、校验或评估语义
+  - 重点支持快速确认每行 ground_truth 是否符合预期，并发现 ground_truth 错误、缺失或不合理的样本
+  - 可在 `.atk/datasets/dataset.csv` 存在后随时执行；推荐在 `atk-build-dataset` 之后、`atk-init` 之前用于浏览与质检
+- **输入**：
+  - 必需：`.atk/datasets/dataset.csv`（非版本化，位于 `.atk/datasets/`，不走 `.atk/results/vN`）
+  - 自动识别 `atk_id` 列、输入列与 `expected|ground_truth|answer|label|target` 类期望列；无法识别时降级展示原始列并允许前端临时重映射
+- **输出文件**：
+  - `.atk/datasets/dataset.html`（单文件、离线、自包含）
+  - 不生成 metadata JSON、sidecar 数据文件或依赖文件；审阅结果由前端经 Blob 下载为 `dataset_review.csv`，不写入仓库
+- **HTML 生成要求**：
+  - 使用 Python 标准库读取 CSV，保留所有原始列并兼容不同数据集字段
+  - 对所有 CSV 派生内容进行 HTML 转义，并中和内联 JSON 中的 `</script>` 等风险序列
+  - 使用插件持有的内嵌 HTML/CSS/JS 与自包含离线资源构建，保留 `skills/atk-visualize-dataset/assets/vendor/echarts.min.js` 的单文件资产内联契约；零 CDN、零外链、零运行时依赖，用户不需要安装 Node、React、Vite 或任何前端依赖
+  - 提供尽可能还原 `dataset-visualize` 视觉与交互的一体化静态页面：橙色/石板色 sticky header、产品化卡片布局，以及两标签视图（`数据列表` / `字段特征分析`）
+  - `数据列表` 支持全字段搜索、动态分类 facet 过滤、字段显示控制、排序、分页、质量标签过滤、详情抽屉，并突出「输入 vs ground_truth」对照
+  - `字段特征分析` 提供字段角色映射、总行/总列/ground_truth/问题样本 KPI、分类分布摘要和逐字段统计；首版以核心视觉还原和静态阅读体验优先，不迁移浏览器上传、Gemini/AI Studio、数据集编辑或源码级 React 要求
+  - **数据集质检 lint**：逐行计算并提供过滤标签，包括空 ground_truth、空输入、重复/缺失/非正整数 `atk_id`、冲突样本（相同输入不同 ground_truth）、完全重复样本（输入+ground_truth 相同）、ground_truth 过短/过长（长度离群）
+  - **客户端审阅**：每行详情提供裁决（✅符合预期 / ⚠️存疑 / ❌需修正）与备注，按 `atk_id` 存入 `localStorage`；「导出审阅结果」经 Blob 下载 `dataset_review.csv`（含 atk_id、行号、裁决、备注、检测到的问题），完全离线、无后端；审阅结果回流既有 `$atk-build-dataset` 修复，不新增编辑类 Skill
+  - 数据集为空、表头无法解析或目标文件已存在且未确认覆盖时给出明确提示并安全退出，不破坏已有文件
+
 ### 2.6 Agent 调优模块（Codex Skill）
 - **功能**：
   - 以当前版本目录下的 `report.md` 作为本轮调优依据
@@ -269,7 +316,7 @@
 ## 4. 版本管理要求
 - **统一父目录**：
   - 所有调优相关产物统一放在 `.atk/` 下
-  - `.atk/datasets/` 存放 ATK 可运行数据集；可由 `$atk-build-dataset` 初始创建，并由 `$atk-init` 校验/规范化后供 runner 使用
+  - `.atk/datasets/` 存放 ATK 可运行数据集；可由 `$atk-build-dataset` 初始创建，可由 `$atk-build-ground-truth` 对现有 `dataset.csv` 补齐 `ground_truth`，并由 `$atk-init` 校验/规范化后供 runner 使用
   - `.atk/runner/` 存放跨版本共享脚本
   - `.atk/results/` 存放按版本隔离的结果
 - **版本命名**：
@@ -315,7 +362,7 @@
 ---
 
 ## 5. 使用示例（手工执行）
-1. 完成数据准备：本地待调优的 Agent 服务 + 评估数据集（默认 CSV）
+1. 完成数据准备：本地待调优的 Agent 服务 + 评估数据集（默认 CSV）；如只有业务描述可先执行 `$atk-build-dataset`，如已有数据集但预期语义薄弱可先执行 `$atk-build-ground-truth` 补齐 `ground_truth`
 2. 执行**批量测试脚本生成 Skill**，生成 `.atk/runner/eval_runner.py`；遇到 Agent 接入或字段不确定时按需与用户确认
 3. 执行 `atk-run`，由其运行 `.atk/runner/eval_runner.py`，脚本自动创建 `.atk/results/v1/` 并写入 `eval_results.csv`（及可选 `app.log`）
 4. 执行异常筛选（按需选择规则路径或大模型路径）：
@@ -334,7 +381,8 @@
 ```text
 /.atk/
 ├── datasets/
-│   └── dataset.csv             # ATK 可运行数据集（含 atk_id），可由 atk-build-dataset 创建并由 atk-init 校验/规范化
+│   ├── dataset.csv             # ATK 可运行数据集（含 atk_id），可由 atk-build-dataset 创建并由 atk-init 校验/规范化
+│   └── dataset.html            # 数据集可视化与质检页（可选，由 atk-visualize-dataset 生成）
 ├── runner/
 │   ├── eval_runner.py          # 跨版本共享测试脚本
 │   └── failure_rule.py          # 跨版本共享失败判定规则脚本（规则模式使用）
@@ -359,7 +407,7 @@
 
 ## 7. 产品交付要求
 1. **Codex Skill**：
-   - 模块化 Skill 模板，覆盖：测试脚本生成、异常筛选（规则模式与大模型模式两个独立入口）、归因分析与报告生成、可选异常样本 HTML 可视化、Agent 调优
+   - 模块化 Skill 模板，覆盖：数据集构建、现有数据集 ground truth 补齐、测试脚本生成、异常筛选（规则模式与大模型模式两个独立入口）、归因分析与报告生成、可选异常样本 HTML 可视化、Agent 调优
    - 内置数据不确定性提示机制，遇到无法可靠推断的字段、Agent 接入方式、异常标准或样本对应关系时引导用户确认
    - 所有模块兼容 `.atk/results/{version}/` 结构，按"数字最大的 `vN`"统一识别当前版本
    - 所有模块在常规流程中不要求用户指定版本号

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/docs/shared-versioning-and-confirmation.md

@@ -43,6 +43,14 @@ Non-goals for this pass:
 - Report: `.atk/results/vN/report.md`
 - Tuning plan: `.atk/results/vN/tuning_plan.md`
 
+## Dataset-only / pre-results Skills
+
+`atk-build-dataset`, `atk-build-ground-truth`, and `atk-new-agent` run before result-version lifecycle work. They do not create `.atk/results/vN`, do not run the Agent or eval runner, and do not write `failure_cases.csv`.
+
+`atk-build-ground-truth` is narrower than `atk-build-dataset`: it requires an existing `.atk/datasets/dataset.csv` with valid non-empty, unique positive-integer `atk_id` values, preserves all original columns, and appends or updates a canonical `ground_truth` column only after the user has selected one dataset-wide style: exact answer or natural-language acceptance criteria. It is optional; `ground_truth` is canonical when this Skill is used, not a universal requirement for every ATK dataset.
+
+If an existing `eval_results.csv` predates `ground_truth` enrichment, recommend rerunning `atk-run` before failure discovery so downstream `eval_results.csv` reflects the enriched dataset. If the current evaluation already reflects the enriched dataset, the user can continue to `atk-find-failures`.
+
 ## Current version vs new version creation
 
 All non-runner Skills use the current-version rule: the current version is the numerically largest existing `.atk/results/vN` directory where `N` is a positive integer. Do not filter current-version selection by required files. If the current version is missing the required input file for a module, stop and ask the user to repair or rerun that module; never fall back to an older version.
@@ -148,9 +156,11 @@ Ask before proceeding when any of these remain unresolved after inspection:
 - Agent invocation path, callable signature, required environment, or working directory;
 - target project Python runtime or import roots needed to load local Agent code;
 - dataset path, file format, encoding, delimiter, or field semantics;
+- `atk-build-ground-truth` global style choice is missing, would mix exact-answer and natural-language acceptance-criteria styles, or requires domain facts not present in the dataset/context;
 - an existing dataset column named `agent_output` conflicts with the required actual-output column, or `agent_output_log_path` conflicts with the required row-log evidence column;
 - app log source, Python `logging` logger names, or row-log capture method cannot be reliably captured or could alter Agent behavior;
 - failure criteria, expected-result columns, or pass/fail semantics are ambiguous;
+- existing `ground_truth` values or expected-like columns (`expected`, `expected_output`, `label`, `answer`, `target`, `acceptance_criteria`) would be overwritten, normalized, or semantically replaced;
 - current/previous-version sample matching is unreliable;
 - an existing `failure_rule.py` should be reused or updated by `atk-init-failure-rule`;
 - executing `failure_rule.py` would overwrite an existing `failure_cases.csv`;
@@ -161,6 +171,7 @@ Do not ask for confirmation for routine, reversible local file generation when t
 ## Per-Skill preconditions and failure behavior
 
 - `atk-status`: no version directory is required. It inspects `.atk/` state and recommends the next Skill or manual command without bypassing confirmation triggers.
+- `atk-build-ground-truth`: no version directory is required. Require existing `.atk/datasets/dataset.csv` and valid non-empty, unique positive-integer `atk_id`; if missing or invalid, stop with repair guidance to use `atk-build-dataset`/`atk-init` or fix `atk_id`. Inspect headers and representative rows before generating. Ask the user to choose one global `ground_truth` style (exact answer or natural-language acceptance criteria) before writing. Preserve original columns and append `ground_truth` if absent. If existing `ground_truth` or expected-like columns would be overwritten, normalized, or semantically replaced, show a candidate modification summary with affected row counts and representative examples, then ask for explicit confirmation. Do not create `.atk/results/vN`, run the Agent/eval runner, or write `failure_cases.csv`. Recommend rerunning `atk-run` when existing `eval_results.csv` predates enrichment; otherwise recommend `atk-find-failures` when failure discovery is next.
 - `atk-init`: no version directory is required. If Agent invocation, target runtime/import roots, dataset path/format, `atk_id` creation/validation, log source, Python `logging` logger names, existing `.atk/datasets/dataset.csv` overwrite semantics, or `agent_output` / `agent_output_log_path` column conflict cannot be inferred safely, ask the user to confirm before writing `.atk/runner/eval_runner.py`. Generated runners should support `--limit`/`--offset`/`--concurrency`, write results incrementally, require valid `atk_id`, add `agent_output_log_path`, create trustworthy row logs for configured same-process Python logging capture when an ATK row context is active, write same-process Python logging records to global `app.log`, keep stdout/stderr/subprocess/multiprocess/post-row background logs out of row files, and be import-checked under the inferred project runtime. If `atk-init` runs `.atk/runner/eval_runner.py --limit 1` or any other runner smoke command during verification, it must clean up the smoke-created result directory when that directory is confidently init-owned temporary output; if cleanup is unsafe because the directory pre-existed or contains user data, report the version impact instead of deleting it.
 - `atk-run`: require `.atk/runner/eval_runner.py`; execute it as the short command surface for batch testing using the target repository's Python runtime when available (`uv run python`, `.venv/bin/python`, Poetry, then `python3`). Pass through safe runner flags such as `--limit`, `--offset`, `--concurrency`, and `--only-failures`. The runner remains the only component that creates or reuses result versions. If the runner fails or no current `eval_results.csv` is produced, report the failure and do not clean up partial version directories. If `--only-failures` cannot resolve prior failed `atk_id` values back to `.atk/datasets/dataset.csv`, report the error and do not silently run the full dataset. If configured row logging is downgraded under `--concurrency > 1` because concurrent row logging is disabled, report that no `logs/row_*.log` files are expected and suggest serial execution or enabling the generated concurrent row-log flag for same-process Python logging evidence. If a partial `eval_results.csv` exists after interruption/failure, report it explicitly.
 - `atk-init-failure-rule`: require current `vN/eval_results.csv`; if no current version or missing `eval_results.csv`, stop with repair/rerun guidance. If existing `.atk/runner/failure_rule.py` exists, ask whether to reuse or update rule logic. This Skill generates or updates the rule script only; it does not write `failure_cases.csv`.
@@ -168,4 +179,5 @@ Do not ask for confirmation for routine, reversible local file generation when t
 - `atk-find-failures`: require current `vN/eval_results.csv`; if expected-result columns or failure criteria are ambiguous, ask for judgment. It writes `failure_cases.csv` in the current version and states that the file is overwritten. After writing `failure_cases.csv`, tell the user to run `atk-report` next.
 - `atk-report`: require current `eval_results.csv` and `failure_cases.csv`; row logs and `app.log` are optional. Prefer existing files referenced by `agent_output_log_path` for per-row failure attribution, then fall back to `app.log`. If previous version lacks `tuning_plan.md` or sample matching is unreliable, degrade to single-version or lower-confidence report with explicit explanation, not silent failure.
 - `atk-visualize-failures`: require current `failure_cases.csv`; optional same-version `report.md` context is best-effort and non-blocking. Generate current `failure_cases.html` only through the fixed plugin-owned stdlib script `scripts/generate_failure_browser.py`, with escaped dependency-free static HTML, expected-vs-actual detail review, search/filter/pagination, schema-adaptive role switching, and safe relative log links. Confirm before overwriting an existing `failure_cases.html` that may contain user edits. If `report.md` is missing or unparseable, continue and note that report context was skipped.
+- `atk-visualize-dataset`: no version directory is required; it operates on the non-versioned `.atk/datasets/` directory. Require `.atk/datasets/dataset.csv`; if it is missing or its header cannot be parsed, stop with guidance to run `atk-build-dataset` first. Generate `.atk/datasets/dataset.html` only through the fixed plugin-owned stdlib script `scripts/generate_dataset_browser.py`, with escaped dependency-free Dataset Visualizer-style static HTML, Data List / Field Feature Analysis tabs, column visibility controls, sorting/pagination, row detail drawer, input-vs-ground_truth comparison for confirming ground_truth correctness, dataset quality lint, schema-adaptive role switching, and client-side review export. Confirm before overwriting an existing `dataset.html` that may contain user edits. This Skill does not require user-installed frontend dependencies and does not read or write any `.atk/results/vN` version directory.
 - `atk-tune`: require current `report.md`; if missing, stop and tell the user to run report generation first. After changes, write `tuning_plan.md` with the exact headings `## 目标异常清单`, `## 调优手段`, and `## 关联改动`. Suggest user git commits/checkpoints; do not perform automatic Agent tuning workflow rollback/baseline restore.

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/docs/skill-template-pack-usage.md

@@ -115,6 +115,8 @@ Do not copy a single `skills/*` directory by itself; keep `skills/`, `templates/
 - `.codex-plugin/plugin.json` — local plugin manifest using `skills: "./skills/"`.
 - `skills/atk-status/SKILL.md` — guided router/status Skill that recommends the next step without bypassing confirmation gates.
 - `skills/atk-build-dataset/SKILL.md` — build `.atk/datasets/dataset.csv` from business context, examples, or rules before init.
+- `skills/atk-build-ground-truth/SKILL.md` — enrich an existing `.atk/datasets/dataset.csv` with a canonical `ground_truth` column before result-version work.
+- `skills/atk-visualize-dataset/SKILL.md` — run the fixed stdlib generator script to write `.atk/datasets/dataset.html` as an optional, dependency-free Dataset Visualizer-style static browser for `.atk/datasets/dataset.csv`, focused on confirming whether each ground_truth matches expectations.
 - `skills/atk-init/SKILL.md` — generate `.atk/runner/eval_runner.py`.
 - `skills/atk-run/SKILL.md` — run `.atk/runner/eval_runner.py` through a short Skill command and summarize the current results version.
 - `skills/atk-init-failure-rule/SKILL.md` — generate or update `.atk/runner/failure_rule.py` as the reusable failure rule script.
@@ -132,17 +134,19 @@ Do not copy a single `skills/*` directory by itself; keep `skills/`, `templates/
 ## Manual 2.2 → 2.6 loop
 
 1. Prepare the local Agent service and evaluation dataset. If you only have a business description, examples, or acceptance rules, trigger `atk-build-dataset` first. It asks 1-3 targeted questions when input fields or expected behavior are unclear, then writes `.atk/datasets/dataset.csv` directly with `atk_id`. Existing `dataset.csv` requires overwrite confirmation; the first version does not merge, append, or create a candidate dataset workflow. After dataset creation, it routes the next step by Agent availability: use `atk-init` when an Agent exists, use `atk-new-agent` when no Agent exists, and mention both paths when that cannot be confirmed.
-2. Trigger `atk-status` to inspect state and route to the right stage.
-3. Trigger `atk-init` in Codex. The Skill reads the Agent source and dataset, asks only about unsafe ambiguity, validates/normalizes `.atk/datasets/dataset.csv` as the ATK canonical runnable dataset with a required `atk_id` column, then writes `.atk/runner/eval_runner.py`.
-4. Trigger `atk-run`. It executes `python3 .atk/runner/eval_runner.py`; the runner creates or reuses a version directory and writes `.atk/results/vN/eval_results.csv` plus optional `app.log`. After a failure set exists, `atk-run --only-failures` reruns only rows whose `atk_id` appears in the latest prior `failure_cases.csv`, selecting those rows from `.atk/datasets/dataset.csv` rather than executing the result CSV as input. When same-process Python `logging` capture is configured, global logging records are written to `.atk/results/vN/app.log`; row-attributed records also create `.atk/results/vN/logs/row_{source_index:06d}.log` files and record relative POSIX paths in `agent_output_log_path` for serial runs and for `--concurrency > 1` while the generated concurrent row-log flag is enabled. The row router only writes records emitted under an active ATK row context; stdout/stderr, subprocess, multiprocess, and post-row background logs remain out of row-log scope. If concurrent row logging is disabled, `--concurrency > 1` visibly downgrades and no row logs are created while global Python logging still writes `app.log`.
-5. Choose one failure-finding entry:
+2. If `.atk/datasets/dataset.csv` already exists but expected-result semantics are weak, trigger `atk-build-ground-truth` before result-version work. It requires valid `atk_id`, asks for one dataset-wide style (`exact answer` or `natural-language acceptance criteria`), preserves existing columns, and writes or updates canonical `ground_truth` only after any overwrite/semantic-replacement risk is confirmed with a candidate modification summary containing affected row counts and representative examples. It does not create `.atk/results/vN`, run the Agent/eval runner, or write `failure_cases.csv`; rerun `atk-run` if existing `eval_results.csv` predates enrichment.
+3. Trigger `atk-status` to inspect state and route to the right stage.Trigger `atk-status` to inspect state and route to the right stage.
+   - Optionally trigger `atk-visualize-dataset` after the dataset exists to render `.atk/datasets/dataset.csv` into a single-file offline `.atk/datasets/dataset.html`. The Skill calls `skills/atk-visualize-dataset/scripts/generate_dataset_browser.py` to produce deterministic Dataset Visualizer-style HTML with Data List / Field Feature Analysis tabs, column visibility controls, sorting/pagination, row detail drawer, input-vs-ground_truth comparison, dataset quality lint, schema-adaptive role switching, and client-side review export, with no LLM summaries, project-local template, user-installed frontend dependencies, or sidecar metadata. It operates on the non-versioned `.atk/datasets/` directory and confirms before overwriting an existing `dataset.html`.
+4. Trigger `atk-init` in Codex. The Skill reads the Agent source and dataset, asks only about unsafe ambiguity, validates/normalizes `.atk/datasets/dataset.csv` as the ATK canonical runnable dataset with a required `atk_id` column, then writes `.atk/runner/eval_runner.py`.
+5. Trigger `atk-run`. It executes `python3 .atk/runner/eval_runner.py`; the runner creates or reuses a version directory and writes `.atk/results/vN/eval_results.csv` plus optional `app.log`. After a failure set exists, `atk-run --only-failures` reruns only rows whose `atk_id` appears in the latest prior `failure_cases.csv`, selecting those rows from `.atk/datasets/dataset.csv` rather than executing the result CSV as input. When same-process Python `logging` capture is configured, global logging records are written to `.atk/results/vN/app.log`; row-attributed records also create `.atk/results/vN/logs/row_{source_index:06d}.log` files and record relative POSIX paths in `agent_output_log_path` for serial runs and for `--concurrency > 1` while the generated concurrent row-log flag is enabled. The row router only writes records emitted under an active ATK row context; stdout/stderr, subprocess, multiprocess, and post-row background logs remain out of row-log scope. If concurrent row logging is disabled, `--concurrency > 1` visibly downgrades and no row logs are created while global Python logging still writes `app.log`.
+6. Choose one failure-finding entry:
    - Rule path: trigger `atk-init-failure-rule` to create/update `.atk/runner/failure_rule.py`, then trigger `atk-find-failures-by-rule` to execute it and write `failure_cases.csv`. If the script is missing, `atk-find-failures-by-rule` stops with guidance to run `atk-init-failure-rule` first.
    - Or trigger `atk-find-failures` to write `failure_cases.csv` directly from the current `eval_results.csv`.
-6. Trigger `atk-report` to create `.atk/results/vN/report.md`. From `v2` onward, it compares the current version with the previous existing version and reads the previous `tuning_plan.md` when available. Reports prefer row-specific files referenced by `agent_output_log_path` and fall back to `app.log` when row logs are unavailable.
-7. Optionally trigger `atk-visualize-failures` to create `.atk/results/vN/failure_cases.html`. It can run any time current `failure_cases.csv` exists; same-version `report.md` is optional best-effort context and never blocks CSV visualization. The Skill calls `skills/atk-visualize-failures/scripts/generate_failure_browser.py` to produce deterministic HTML with expected-vs-actual review, search/filter/pagination, schema-adaptive role switching, and no LLM summaries, project-local template, dependencies, or sidecar metadata.
-8. Trigger `atk-tune` to change the Agent and write `.atk/results/vN/tuning_plan.md`.
-9. Optionally create a user git commit/checkpoint. Agent tuning rollback remains user-git-only guidance; this plugin does not automate Agent code restore.
-10. Run the same loop again. The next test run creates `v{N+1}` when the current max version already has `eval_results.csv`.
+7. Trigger `atk-report` to create `.atk/results/vN/report.md`. From `v2` onward, it compares the current version with the previous existing version and reads the previous `tuning_plan.md` when available. Reports prefer row-specific files referenced by `agent_output_log_path` and fall back to `app.log` when row logs are unavailable.
+8. Optionally trigger `atk-visualize-failures` to create `.atk/results/vN/failure_cases.html`. It can run any time current `failure_cases.csv` exists; same-version `report.md` is optional best-effort context and never blocks CSV visualization. The Skill calls `skills/atk-visualize-failures/scripts/generate_failure_browser.py` to produce deterministic HTML with expected-vs-actual review, search/filter/pagination, schema-adaptive role switching, and no LLM summaries, project-local template, dependencies, or sidecar metadata.
+9. Trigger `atk-tune` to change the Agent and write `.atk/results/vN/tuning_plan.md`.
+10. Optionally create a user git commit/checkpoint. Agent tuning rollback remains user-git-only guidance; this plugin does not automate Agent code restore.
+11. Run the same loop again. The next test run creates `v{N+1}` when the current max version already has `eval_results.csv`.
 
 ## Version example: v1 → v2
 
@@ -160,7 +164,7 @@ Run:
 python3 scripts/validate_skill_pack.py
 git diff --check
 find skills templates docs scripts .codex-plugin -maxdepth 4 -type f | sort
-grep -R "agent_output\|failure_cases.csv\|failure_cases.html\|tuning_plan.md\|目标异常清单\|调优手段\|关联改动\|source.path" skills templates docs README.md README.zh-CN.md .codex-plugin scripts
+grep -R "agent_output\|ground_truth\|failure_cases.csv\|failure_cases.html\|tuning_plan.md\|目标异常清单\|调优手段\|关联改动\|source.path" skills templates docs README.md README.zh-CN.md .codex-plugin scripts
 ```
 
-`validate_skill_pack.py` checks required files, required sections, manifest fields, installer behavior, PRD traceability, canonical version helper names and snippets, output paths, non-goal boundaries, and each Skill's precondition/failure behavior.
+`validate_skill_pack.py` checks required files, required sections, manifest fields, installer behavior, PRD traceability, canonical version helper names and snippets, output paths, non-goal boundaries, dataset-only/pre-results classifications such as `atk-build-ground-truth`, and each Skill's precondition/failure behavior.

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "agent-tune-kit"
-version = "0.4.2"
+version = "0.4.4"
 description = "Local Codex plugin for iterative Agent tuning with guided Skills, reusable runner templates, versioned results, and static validation."
 readme = "README.en.md"
 requires-python = ">=3.11"

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/scripts/validate_skill_pack.py

@@ -19,6 +19,7 @@ REQUIRED_FILES = [
     ".codex-plugin/plugin.json",
     "skills/atk-status/SKILL.md",
     "skills/atk-build-dataset/SKILL.md",
+    "skills/atk-build-ground-truth/SKILL.md",
     "skills/atk-new-agent/SKILL.md",
     "skills/atk-init/SKILL.md",
     "skills/atk-run/SKILL.md",
@@ -32,6 +33,12 @@ REQUIRED_FILES = [
     "skills/atk-visualize-failures/assets/styles.css",
     "skills/atk-visualize-failures/assets/app.js",
     "skills/atk-visualize-failures/assets/vendor/echarts.min.js",
+    "skills/atk-visualize-dataset/SKILL.md",
+    "skills/atk-visualize-dataset/scripts/generate_dataset_browser.py",
+    "skills/atk-visualize-dataset/assets/page.html",
+    "skills/atk-visualize-dataset/assets/styles.css",
+    "skills/atk-visualize-dataset/assets/app.js",
+    "skills/atk-visualize-dataset/assets/vendor/echarts.min.js",
     "skills/atk-tune/SKILL.md",
     "templates/.atk/runner/eval_runner.py.md",
     "templates/.atk/runner/failure_rule.py.md",
@@ -59,8 +66,14 @@ REQUIRED_FILES = [
 
 SKILL_FILES = [path for path in REQUIRED_FILES if path.startswith("skills/") and path.endswith("/SKILL.md")]
 PRE_INIT_SKILL_FILES = {"skills/atk-build-dataset/SKILL.md", "skills/atk-new-agent/SKILL.md"}
+DATASET_ONLY_PRE_RESULTS_SKILL_FILES = {
+    "skills/atk-build-dataset/SKILL.md",
+    "skills/atk-build-ground-truth/SKILL.md",
+    "skills/atk-visualize-dataset/SKILL.md",
+}
+NON_LIFECYCLE_SKILL_FILES = PRE_INIT_SKILL_FILES | DATASET_ONLY_PRE_RESULTS_SKILL_FILES
 AGENT_SCAFFOLD_SKILL_FILES = {"skills/atk-new-agent/SKILL.md"}
-LIFECYCLE_SKILL_FILES = [path for path in SKILL_FILES if path not in PRE_INIT_SKILL_FILES]
+LIFECYCLE_SKILL_FILES = [path for path in SKILL_FILES if path not in NON_LIFECYCLE_SKILL_FILES]
 
 REQUIRED_SKILL_SECTIONS = [
     "## Purpose",
@@ -145,7 +158,7 @@ PRD_REFERENCES = [
 PER_FILE_PHRASES = {
     ".codex-plugin/plugin.json": [
         '"name": "agent-tune-kit"',
-        '"version": "0.4.2"',
+        '"version": "0.4.4"',
         '"skills": "./skills/"',
         '"displayName": "Agent Tune Kit"',
         '"defaultPrompt"',
@@ -290,6 +303,30 @@ PER_FILE_PHRASES = {
         "if Agent existence is unclear",
         "$atk-init",
     ],
+    "skills/atk-build-ground-truth/SKILL.md": [
+        "atk-build-ground-truth",
+        "dataset-only / pre-results",
+        "existing `.atk/datasets/dataset.csv`",
+        "ground_truth",
+        "valid `atk_id`",
+        "unique positive integers",
+        "exact answer",
+        "natural-language acceptance criteria",
+        "global ground-truth style",
+        "dataset-wide",
+        "candidate modification summary",
+        "affected row counts and representative examples",
+        "overwrite existing `ground_truth`",
+        "semantically replace expected-like fields",
+        "multiple incompatible Agent tasks",
+        "required domain facts are absent",
+        "Do not create `.atk/results/vN`",
+        "Do not run the Agent",
+        "Do not run `$atk-run`",
+        "Do not write `failure_cases.csv`",
+        "Do not change `atk-find-failures` behavior in v1",
+        "eval_results.csv` predates dataset enrichment",
+    ],
     "skills/atk-init/SKILL.md": [
         ".atk/runner/eval_runner.py",
         "Preserve all original dataset columns",
@@ -433,6 +470,60 @@ PER_FILE_PHRASES = {
         "No vN results directory exists",
         "Refusing to overwrite existing",
     ],
+    "skills/atk-visualize-dataset/SKILL.md": [
+        "atk-visualize-dataset",
+        ".atk/datasets/dataset.csv",
+        ".atk/datasets/dataset.html",
+        "scripts/generate_dataset_browser.py",
+        "csv.DictReader",
+        "html.escape",
+        "--dataset-path",
+        "--overwrite",
+        "input-vs-ground_truth comparison",
+        "ground_truth confirmation",
+        "dataset quality lint",
+        "client-side review export",
+        "dataset_review.csv",
+        "schema-adaptive role switching",
+        "row inspector details",
+        "ground_truth",
+        "atk_id",
+        "bundled offline ECharts",
+        "assets/vendor/echarts.min.js",
+        "Do not create `dataset_summary.json`",
+        "metadata JSON",
+        "Do not write outside the `.atk/datasets/` directory",
+        "never change `atk-build-dataset`",
+        "atk-build-dataset",
+        "atk-init",
+    ],
+    "skills/atk-visualize-dataset/scripts/generate_dataset_browser.py": [
+        'DATASETS_DIR = Path(".atk/datasets")',
+        'DATASET_FILENAME = "dataset.csv"',
+        'OUTPUT_FILENAME = "dataset.html"',
+        "def detect_atk_id_field(",
+        "def compute_issues(",
+        "def detect_roles(",
+        "def detect_facets(",
+        "csv.DictReader",
+        "html.escape",
+        "json.dumps",
+        "ensure_ascii=False",
+        "SNIPPET_MAX_CHARS = 240",
+        "PAGE_SIZES = [25, 50, 100, 250]",
+        "DEFAULT_PAGE_SIZE = 50",
+        "VENDOR_ECHARTS_NAME",
+        "--dataset-path",
+        "--overwrite",
+        "input-vs-ground_truth comparison",
+        "dataset quality lint",
+        "ground_truth confirmation",
+        "client-side review export",
+        "bundled offline ECharts",
+        "os.replace",
+        "Refusing to overwrite existing",
+        "run atk-build-dataset or atk-init first",
+    ],
     "skills/atk-tune/SKILL.md": [
         'require_current_file(current_dir, "report.md")',
         "## 目标异常清单",
@@ -721,7 +812,7 @@ def main() -> int:
         text = existing_texts.get(rel, "")
         require(text.startswith("---\n"), f"{rel} missing YAML front matter", errors)
         required_sections = (
-            REQUIRED_PRE_INIT_SCAFFOLD_SKILL_SECTIONS if rel in PRE_INIT_SKILL_FILES else REQUIRED_SKILL_SECTIONS
+            REQUIRED_PRE_INIT_SCAFFOLD_SKILL_SECTIONS if rel in NON_LIFECYCLE_SKILL_FILES else REQUIRED_SKILL_SECTIONS
         )
         for section in required_sections:
             require(section in text, f"{rel} missing section {section}", errors)
@@ -738,7 +829,7 @@ def main() -> int:
             errors,
         )
 
-    for rel in PRE_INIT_SKILL_FILES:
+    for rel in NON_LIFECYCLE_SKILL_FILES:
         text = existing_texts.get(rel, "")
         require(
             'RESULTS_DIR = Path(".atk/results")' not in text,
@@ -851,7 +942,7 @@ def main() -> int:
     )
     require(
         "def capture_python_app_logging" in runner_template
-        and "logging.FileHandler(log_path, mode=\"w\", encoding=\"utf-8\")" in runner_template
+        and 'logging.FileHandler(log_path, mode="w", encoding="utf-8")' in runner_template
         and "logger.addHandler(app_handler)" in runner_template
         and "logger.removeHandler(app_handler)" in runner_template,
         "runner template must write configured Python logging records to global app.log",

{agent_tune_kit-0.4.2 → agent_tune_kit-0.4.4}/skills/atk-build-dataset/SKILL.md

@@ -110,6 +110,9 @@ After writing the dataset, summarize:
 - input and expected-output columns;
 - coverage categories included;
 - any assumptions or unfilled domain facts;
+- optional review step: run `$atk-visualize-dataset` to open a local HTML browser of `.atk/datasets/dataset.csv` for
+  quickly inspecting rows, confirming whether each ground_truth matches expectations, and spotting incorrect, missing,
+  or unreasonable ground_truth before initializing evaluation;
 - next step based on Agent availability:
   - if an Agent exists: run `$atk-init`, using the newly created `.atk/datasets/dataset.csv` to initialize batch
     evaluation;