agent-tune-kit 2.2.1__tar.gz → 2.3.0__tar.gz

{agent_tune_kit-2.2.1 → agent_tune_kit-2.3.0}/.codex-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "agent-tune-kit",
-  "version": "2.2.1",
-  "description": "Local Codex plugin for iterative Agent tuning with guided Skills, reusable runner templates, versioned results, and static validation.",
+  "version": "2.3.0",
+  "description": "Local Codex plugin for batch Agent evaluation and session trace tuning with evidence-backed issues, focused changes, verification and offline review.",
   "author": {
     "name": "hustyichi",
     "email": "hustyichi@163.com",
@@ -15,13 +15,14 @@
     "codex-skills",
     "evaluation",
     "reporting",
-    "local-plugin"
+    "local-plugin",
+    "trace-tuning"
   ],
   "skills": "./skills/",
   "interface": {
     "displayName": "Agent Tune Kit",
-    "shortDescription": "Run a guided local Agent tuning loop",
-    "longDescription": "Agent Tune Kit provides Codex Skills for generating local test runners, finding failing cases, reporting regressions and improvements, and applying focused Agent tuning with versioned local artifacts.",
+    "shortDescription": "Evaluate Agents and tune session traces",
+    "longDescription": "Agent Tune Kit provides independent Codex Skills for batch evaluation and session trace tuning: capture traces, detect issues, investigate causes, apply focused changes, verify candidates, and review offline evidence.",
     "developerName": "hustyichi",
     "category": "Coding",
     "capabilities": [
@@ -31,9 +32,9 @@
     ],
     "websiteURL": "https://github.com/hustyichi/agent-tune-kit",
     "defaultPrompt": [
-      "Build an Agent tuning dataset",
+      "Analyze an Agent session trace",
       "Find Agent failure cases",
-      "Generate an Agent tuning report"
+      "Verify a focused Agent repair"
     ],
     "brandColor": "#334155",
     "screenshots": []

agent_tune_kit-2.3.0/PKG-INFO

@@ -0,0 +1,186 @@
+Metadata-Version: 2.5
+Name: agent-tune-kit
+Version: 2.3.0
+Summary: Local Codex plugin for iterative Agent tuning with guided Skills, reusable runner templates, versioned results, and static validation.
+Project-URL: Homepage, https://github.com/hustyichi/agent-tune-kit
+Project-URL: Repository, https://github.com/hustyichi/agent-tune-kit
+Author-email: hustyichi <hustyichi@163.com>
+License: UNLICENSED
+Keywords: agent-tune-kit,codex-skills,evaluation,local-plugin,reporting
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# Agent Tune Kit
+
+English | [简体中文](README.md)
+
+[![PyPI](https://img.shields.io/pypi/v/agent-tune-kit.svg)](https://pypi.org/project/agent-tune-kit/)
+
+Agent Tune Kit is a **local Codex plugin** for tuning an Agent's prompts, code, parameters, or tool configuration. Install it once and use either workflow independently.
+
+- [Evaluation-based tuning](#evaluation-based-tuning) starts with a set of samples and criteria. Run the Agent in batches, compare rounds, and check for regressions.
+- [Continuous tuning from session traces](#continuous-tuning-from-session-traces) starts with an existing session. Follow its execution trace to identify issues, then fix and verify them individually.
+
+## Install and Enable
+
+Run these commands in your terminal to install ATK and its bundled Skills.
+
+```sh
+uv tool install agent-tune-kit
+atk install
+```
+
+In Codex, enter `/plugins`, select and enable `Agent Tune Kit`, then restart Codex and open a new session.
+
+<details>
+<summary>Install with pipx or from local source</summary>
+
+To install with pipx, run the following commands.
+
+```sh
+pipx install agent-tune-kit
+atk install
+```
+
+To install from local source, update your chosen branch and run these commands from the **ATK repository root**. Rerun them after source updates, including when the version stays the same.
+
+```sh
+uv tool install --reinstall .
+atk install
+```
+
+</details>
+
+Finish active ATK tasks before reinstalling, and open a new session afterward to load updates. If you have installation problems, run `atk status` and confirm the plugin is enabled in Codex. [Installation and cache notes](docs/skill-template-pack-usage.md#local-plugin-install-and-smoke)
+
+## Evaluation-based tuning
+
+Use a set of evaluation samples to find Agent failures and verify changes in later rounds. Prepare your Agent's entry point and a set of samples, preferably in CSV format. Five to twenty samples are enough to get started. Add expected answers, labels, or acceptance criteria where available.
+
+Open your Agent project in Codex and send the following `$atk-…` instructions in the conversation.
+
+### 1. Prepare Data as Needed
+
+If you already have suitable data, go straight to connecting your Agent. To organize data or fill gaps in the evaluation criteria, send the following instruction.
+
+```text
+$atk-dataset Prepare data/eval.csv for evaluation. Preserve existing fields and answers. Fill gaps in the judgment criteria where the evidence supports an answer.
+```
+
+You can also start with business examples or request a correction, such as changing the expected result for atk_id 7. Keep your existing business column names. Answers without trustworthy support stay blank.
+
+Use `$atk-visualize-dataset` when you want a local page for browsing and reviewing data, then give feedback to `$atk-dataset`.
+
+### 2. Connect Your Agent
+
+Tell Codex where to find your Agent entry point and evaluation data.
+
+```text
+$atk-init My Agent entry point is scripts/agent.py and the evaluation dataset is data/eval.csv
+```
+
+Codex reads the project, prepares the adapter, and organizes a runnable dataset. If your Agent is already connected through a compatible adapter, go straight to evaluation.
+
+Evaluation appends five reserved columns called `agent_output`, `output_status`, `error_type`, `error_message`, and `log_path`. If evaluation reports a name collision, confirm a new name for the business column and update references in the adapter and judgment rules. [Reserved columns](docs/shared-versioning-and-confirmation.md#reserved-evaluation-output-columns)
+
+### 3. Evaluate, Diagnose, and Read the Report
+
+Send these instructions in order.
+
+```text
+$atk-eval
+$atk-diagnose
+$atk-report
+```
+
+| Instruction | What you get |
+| --- | --- |
+| `$atk-eval` | Actual Agent outputs and execution records for each sample |
+| `$atk-diagnose` | Passed, failed, or unknown judgments with reasons and an initial investigation of failures |
+| `$atk-report` | Main issues, cause analysis, and tuning recommendations |
+
+Use `$atk-visualize-failures` for a local offline page to search, filter, and review individual failures.
+
+Check judgment coverage when comparing pass results. Samples with insufficient evidence are marked unknown. Reported technical failures such as timeouts are counted separately and can be reviewed with `atk results inspect`.
+
+### 4. Tune from the Results
+
+```text
+$atk-optimize
+```
+
+Codex uses the report and failures to edit your Agent. It records why it made changes and what to verify next. You can save your current code with Git before tuning so you can compare and restore it.
+
+### 5. Rerun the Evaluation and Verify Changes
+
+After tuning, you can first rerun the previous round's confirmed failures.
+
+```text
+$atk-eval --only-failures
+$atk-diagnose
+$atk-report
+```
+
+A new evaluation does not overwrite existing nonempty result directories. From the second round onward, when prior tuning notes exist and conditions are comparable, the report identifies target issues as resolved, partially resolved, unresolved, or impossible to judge.
+
+To revisit unknown samples, technical failures, or other selected samples, use `$atk-eval --ids 7 9` separately. To check the effect on the full dataset, run `$atk-eval` without selection flags, then diagnose and generate the report.
+
+### Where to Find Evaluation Outputs
+
+Each round saves results under `.atk/results/vN/` in your Agent project. Files appear as you run the corresponding steps.
+
+| File | What it shows |
+| --- | --- |
+| `eval_results.csv` | Actual outputs and execution status. Follow `log_path` to find a sample's log |
+| `judgments.csv`, `failure_cases.csv` | Individual judgments and confirmed failures. Use `failure_cases.html` to browse failures |
+| `report.md`, `tuning_plan.md` | Issues, changes, and verification targets for the round. Later reports revisit those targets when conditions are comparable |
+
+[More batch evaluation options and outputs](docs/skill-template-pack-usage.md#manual-22--26-loop)
+
+![Evaluation-based tuning workflow, from preparing data and connecting an Agent through evaluation, diagnosis, reporting, tuning, and rerunning the evaluation.](docs/assets/architecture.en.svg)
+
+## Continuous tuning from session traces
+
+Start with an existing session and use its execution trace to check whether the Agent completed the task. Save diagnoses, fixes, and verification results by issue. Get the native Session ID, its source, and access to the trace, then open the Agent project in Codex. You can start without an evaluation dataset or runner.
+
+Run `atk trace --help` to check whether your CLI supports trace tuning. If the command is unavailable, follow the local source installation steps above with a version that supports trace tuning, then open a new Codex session.
+
+### 1. Prepare the Session Trace
+
+- **Langfuse v4**. Use `$atk-trace-init` to record project settings and configure read-only access on first setup. Configuration references credentials by environment-variable name. Then send `$atk-trace-capture Download session <session-id> from <start time> to <end time>`.
+- **Local files**. Send `$atk-trace-capture Import traces for session <session-id> from <file path>, with source <source>`. You can also provide files with verifiable ties to the session during analysis and let detection import them. Analyze an already archived session directly.
+
+The capture step downloads or imports data. Before saving, decide which content to retain and which fields to redact. ATK adds `.atk/traces/` to the project's `.gitignore`.
+
+### 2. Analyze and Review Evidence
+
+```text
+$atk-trace-detect Check whether session <session-id> completed its task correctly. List issues, evidence, and what you did not inspect.
+$atk-visualize-trace Open the execution lanes for session <session-id> and jump from issues to the relevant trace segments and evidence fields.
+```
+
+Detection lists issues, evidence, and what it inspected. Use Trace IDs to locate evidence within the session. You can generate a preview once traces are archived. Review any retained content in the HTML before sharing it.
+
+### 3. Investigate, Fix, and Verify an Issue as Needed
+
+```text
+$atk-trace-diagnose Investigate ISSUE-001 in session <session-id> and propose a repair and verification plan.
+$atk-trace-optimize Implement the selected plan for ISSUE-001 in session <session-id>.
+$atk-trace-verify Verify the candidate for ISSUE-001 in session <session-id> and check relevant behavior that should still work.
+```
+
+Start with diagnosis when you already have a specific issue and session evidence. Each verification records a conclusion per issue. If the evidence cannot establish which candidate loaded or whether key verification conditions were met, the outcome is `inconclusive`. As new issues surface, work through them individually or start another analysis with a new session trace.
+
+### Where to Find Trace Outputs
+
+Each session archive is stored under `.atk/traces/<session_id>/` in your Agent project. Files appear as you run the corresponding steps.
+
+| File or directory | What it shows |
+| --- | --- |
+| `session.json`, `raw/` | Session source, capture scope, inspection coverage, and retained traces |
+| `preview.html` | Offline browse: issue rail, swimlanes, Event list; jump by Issue segments |
+| `issues/<issue-id>/` | Issue evidence, diagnoses, and changes. Each verification's `result.json` records its status, conclusion, and evidence references |
+
+[Trace tuning guide for source configuration, supported formats, and archive layout](docs/trace-tuning.md)

agent_tune_kit-2.3.0/README.en.md

@@ -0,0 +1,172 @@
+# Agent Tune Kit
+
+English | [简体中文](README.md)
+
+[![PyPI](https://img.shields.io/pypi/v/agent-tune-kit.svg)](https://pypi.org/project/agent-tune-kit/)
+
+Agent Tune Kit is a **local Codex plugin** for tuning an Agent's prompts, code, parameters, or tool configuration. Install it once and use either workflow independently.
+
+- [Evaluation-based tuning](#evaluation-based-tuning) starts with a set of samples and criteria. Run the Agent in batches, compare rounds, and check for regressions.
+- [Continuous tuning from session traces](#continuous-tuning-from-session-traces) starts with an existing session. Follow its execution trace to identify issues, then fix and verify them individually.
+
+## Install and Enable
+
+Run these commands in your terminal to install ATK and its bundled Skills.
+
+```sh
+uv tool install agent-tune-kit
+atk install
+```
+
+In Codex, enter `/plugins`, select and enable `Agent Tune Kit`, then restart Codex and open a new session.
+
+<details>
+<summary>Install with pipx or from local source</summary>
+
+To install with pipx, run the following commands.
+
+```sh
+pipx install agent-tune-kit
+atk install
+```
+
+To install from local source, update your chosen branch and run these commands from the **ATK repository root**. Rerun them after source updates, including when the version stays the same.
+
+```sh
+uv tool install --reinstall .
+atk install
+```
+
+</details>
+
+Finish active ATK tasks before reinstalling, and open a new session afterward to load updates. If you have installation problems, run `atk status` and confirm the plugin is enabled in Codex. [Installation and cache notes](docs/skill-template-pack-usage.md#local-plugin-install-and-smoke)
+
+## Evaluation-based tuning
+
+Use a set of evaluation samples to find Agent failures and verify changes in later rounds. Prepare your Agent's entry point and a set of samples, preferably in CSV format. Five to twenty samples are enough to get started. Add expected answers, labels, or acceptance criteria where available.
+
+Open your Agent project in Codex and send the following `$atk-…` instructions in the conversation.
+
+### 1. Prepare Data as Needed
+
+If you already have suitable data, go straight to connecting your Agent. To organize data or fill gaps in the evaluation criteria, send the following instruction.
+
+```text
+$atk-dataset Prepare data/eval.csv for evaluation. Preserve existing fields and answers. Fill gaps in the judgment criteria where the evidence supports an answer.
+```
+
+You can also start with business examples or request a correction, such as changing the expected result for atk_id 7. Keep your existing business column names. Answers without trustworthy support stay blank.
+
+Use `$atk-visualize-dataset` when you want a local page for browsing and reviewing data, then give feedback to `$atk-dataset`.
+
+### 2. Connect Your Agent
+
+Tell Codex where to find your Agent entry point and evaluation data.
+
+```text
+$atk-init My Agent entry point is scripts/agent.py and the evaluation dataset is data/eval.csv
+```
+
+Codex reads the project, prepares the adapter, and organizes a runnable dataset. If your Agent is already connected through a compatible adapter, go straight to evaluation.
+
+Evaluation appends five reserved columns called `agent_output`, `output_status`, `error_type`, `error_message`, and `log_path`. If evaluation reports a name collision, confirm a new name for the business column and update references in the adapter and judgment rules. [Reserved columns](docs/shared-versioning-and-confirmation.md#reserved-evaluation-output-columns)
+
+### 3. Evaluate, Diagnose, and Read the Report
+
+Send these instructions in order.
+
+```text
+$atk-eval
+$atk-diagnose
+$atk-report
+```
+
+| Instruction | What you get |
+| --- | --- |
+| `$atk-eval` | Actual Agent outputs and execution records for each sample |
+| `$atk-diagnose` | Passed, failed, or unknown judgments with reasons and an initial investigation of failures |
+| `$atk-report` | Main issues, cause analysis, and tuning recommendations |
+
+Use `$atk-visualize-failures` for a local offline page to search, filter, and review individual failures.
+
+Check judgment coverage when comparing pass results. Samples with insufficient evidence are marked unknown. Reported technical failures such as timeouts are counted separately and can be reviewed with `atk results inspect`.
+
+### 4. Tune from the Results
+
+```text
+$atk-optimize
+```
+
+Codex uses the report and failures to edit your Agent. It records why it made changes and what to verify next. You can save your current code with Git before tuning so you can compare and restore it.
+
+### 5. Rerun the Evaluation and Verify Changes
+
+After tuning, you can first rerun the previous round's confirmed failures.
+
+```text
+$atk-eval --only-failures
+$atk-diagnose
+$atk-report
+```
+
+A new evaluation does not overwrite existing nonempty result directories. From the second round onward, when prior tuning notes exist and conditions are comparable, the report identifies target issues as resolved, partially resolved, unresolved, or impossible to judge.
+
+To revisit unknown samples, technical failures, or other selected samples, use `$atk-eval --ids 7 9` separately. To check the effect on the full dataset, run `$atk-eval` without selection flags, then diagnose and generate the report.
+
+### Where to Find Evaluation Outputs
+
+Each round saves results under `.atk/results/vN/` in your Agent project. Files appear as you run the corresponding steps.
+
+| File | What it shows |
+| --- | --- |
+| `eval_results.csv` | Actual outputs and execution status. Follow `log_path` to find a sample's log |
+| `judgments.csv`, `failure_cases.csv` | Individual judgments and confirmed failures. Use `failure_cases.html` to browse failures |
+| `report.md`, `tuning_plan.md` | Issues, changes, and verification targets for the round. Later reports revisit those targets when conditions are comparable |
+
+[More batch evaluation options and outputs](docs/skill-template-pack-usage.md#manual-22--26-loop)
+
+![Evaluation-based tuning workflow, from preparing data and connecting an Agent through evaluation, diagnosis, reporting, tuning, and rerunning the evaluation.](docs/assets/architecture.en.svg)
+
+## Continuous tuning from session traces
+
+Start with an existing session and use its execution trace to check whether the Agent completed the task. Save diagnoses, fixes, and verification results by issue. Get the native Session ID, its source, and access to the trace, then open the Agent project in Codex. You can start without an evaluation dataset or runner.
+
+Run `atk trace --help` to check whether your CLI supports trace tuning. If the command is unavailable, follow the local source installation steps above with a version that supports trace tuning, then open a new Codex session.
+
+### 1. Prepare the Session Trace
+
+- **Langfuse v4**. Use `$atk-trace-init` to record project settings and configure read-only access on first setup. Configuration references credentials by environment-variable name. Then send `$atk-trace-capture Download session <session-id> from <start time> to <end time>`.
+- **Local files**. Send `$atk-trace-capture Import traces for session <session-id> from <file path>, with source <source>`. You can also provide files with verifiable ties to the session during analysis and let detection import them. Analyze an already archived session directly.
+
+The capture step downloads or imports data. Before saving, decide which content to retain and which fields to redact. ATK adds `.atk/traces/` to the project's `.gitignore`.
+
+### 2. Analyze and Review Evidence
+
+```text
+$atk-trace-detect Check whether session <session-id> completed its task correctly. List issues, evidence, and what you did not inspect.
+$atk-visualize-trace Open the execution lanes for session <session-id> and jump from issues to the relevant trace segments and evidence fields.
+```
+
+Detection lists issues, evidence, and what it inspected. Use Trace IDs to locate evidence within the session. You can generate a preview once traces are archived. Review any retained content in the HTML before sharing it.
+
+### 3. Investigate, Fix, and Verify an Issue as Needed
+
+```text
+$atk-trace-diagnose Investigate ISSUE-001 in session <session-id> and propose a repair and verification plan.
+$atk-trace-optimize Implement the selected plan for ISSUE-001 in session <session-id>.
+$atk-trace-verify Verify the candidate for ISSUE-001 in session <session-id> and check relevant behavior that should still work.
+```
+
+Start with diagnosis when you already have a specific issue and session evidence. Each verification records a conclusion per issue. If the evidence cannot establish which candidate loaded or whether key verification conditions were met, the outcome is `inconclusive`. As new issues surface, work through them individually or start another analysis with a new session trace.
+
+### Where to Find Trace Outputs
+
+Each session archive is stored under `.atk/traces/<session_id>/` in your Agent project. Files appear as you run the corresponding steps.
+
+| File or directory | What it shows |
+| --- | --- |
+| `session.json`, `raw/` | Session source, capture scope, inspection coverage, and retained traces |
+| `preview.html` | Offline browse: issue rail, swimlanes, Event list; jump by Issue segments |
+| `issues/<issue-id>/` | Issue evidence, diagnoses, and changes. Each verification's `result.json` records its status, conclusion, and evidence references |
+
+[Trace tuning guide for source configuration, supported formats, and archive layout](docs/trace-tuning.md)

agent_tune_kit-2.3.0/README.md

@@ -0,0 +1,172 @@
+# Agent Tune Kit
+
+简体中文 | [English](README.en.md)
+
+[![PyPI](https://img.shields.io/pypi/v/agent-tune-kit.svg)](https://pypi.org/project/agent-tune-kit/)
+
+Agent Tune Kit 是一个**本地 Codex 插件**，用于调优 Agent 的提示词、代码、参数或工具配置。两条路线共用安装，安装后可按手头的材料独立使用。
+
+- [评测调优](#评测调优)从一组样本和评测标准出发，批量运行 Agent，比较多轮结果并做回归验证。
+- [轨迹持续调优](#轨迹持续调优)从一个已有会话出发，沿执行轨迹定位问题，逐项修复和验证。
+
+## 安装与启用
+
+在终端运行以下命令安装 ATK 和配套 Skills。
+
+```sh
+uv tool install agent-tune-kit
+atk install
+```
+
+在 Codex 中输入 `/plugins`，选择并启用 `Agent Tune Kit`，然后重启 Codex 并新建会话。
+
+<details>
+<summary>使用 pipx 或本地源码安装</summary>
+
+用 pipx 安装。
+
+```sh
+pipx install agent-tune-kit
+atk install
+```
+
+从本地源码安装时，先更新所需分支，再在 **ATK 仓库根目录**运行以下命令。更新源码后可重新执行，包括版本号未变的情况。
+
+```sh
+uv tool install --reinstall .
+atk install
+```
+
+</details>
+
+重装前先结束正在运行的 ATK 任务，安装后新建会话以加载更新。遇到安装问题时，运行 `atk status` 检查安装，并在 Codex 中确认插件已启用。[安装与缓存说明](docs/skill-template-pack-usage.md#local-plugin-install-and-smoke)
+
+## 评测调优
+
+用一组评测样本找出 Agent 的问题，并用后续评测验证改动。准备 Agent 的入口和一组样本，推荐用 CSV，5 到 20 条即可开始。每条样本可附上期望答案、标签或可验收的行为标准。
+
+在 Codex 中打开 Agent 项目，在对话中发送下面的 `$atk-…` 指令。
+
+### 1. 准备数据（按需）
+
+已有合适的数据可直接接入 Agent。需要整理数据或补充评测标准时，可以发送以下指令。
+
+```text
+$atk-dataset 使用 data/eval.csv 准备评测数据，保留原字段和已有答案，只补充缺失且有依据的判断标准。
+```
+
+也可以从业务示例开始，或直接提出修订，例如“修改 atk_id 为 7 的预期结果”。沿用已有的业务列名。缺少可信依据的答案会留空。
+
+需要人工浏览和复核时，可使用 `$atk-visualize-dataset` 生成本地数据浏览页，再把反馈交给 `$atk-dataset`。
+
+### 2. 接入 Agent
+
+告诉 Codex 你的 Agent 入口和评测数据位置。
+
+```text
+$atk-init 我的 Agent 入口是 scripts/agent.py，评测数据是 data/eval.csv
+```
+
+Codex 会读取项目、准备接入文件并整理可运行的数据集。已经完成接入且仍兼容时，可直接开始评测。
+
+评测会追加五个保留列，即 `agent_output`、`output_status`、`error_type`、`error_message` 和 `log_path`。如果评测提示列名冲突，确认新的业务列名并同步适配器和判定规则中的引用。[保留列说明](docs/shared-versioning-and-confirmation.md#reserved-evaluation-output-columns)
+
+### 3. 评测、诊断并查看报告
+
+依次发送以下指令。
+
+```text
+$atk-eval
+$atk-diagnose
+$atk-report
+```
+
+| 指令 | 你会得到什么 |
+| --- | --- |
+| `$atk-eval` | 每条样本的 Agent 实际输出和执行记录 |
+| `$atk-diagnose` | 通过、失败、未判定的结果及理由，以及失败原因的初步定位 |
+| `$atk-report` | 本轮主要问题、原因分析和调优建议 |
+
+需要逐条搜索、筛选和复核失败样本时，使用 `$atk-visualize-failures` 生成本地离线浏览页。
+
+比较通过情况时，也要看判定覆盖范围。依据不足的样本会标为未判定。已记录的超时等技术失败单独统计，可用 `atk results inspect` 查看。
+
+### 4. 根据结果调优
+
+```text
+$atk-optimize
+```
+
+Codex 会根据报告和失败样本修改 Agent，并记录改动原因和下一轮需要验证的问题。调优前可用 Git 保存当前代码，方便比较和恢复。
+
+### 5. 重跑并验证改动
+
+调优后，可以先重跑上一轮已确认的失败样本。
+
+```text
+$atk-eval --only-failures
+$atk-diagnose
+$atk-report
+```
+
+已有非空结果目录不会被下一轮评测覆盖。从第二轮开始，如果上一轮有调优记录且实验条件可比，报告会说明目标问题已解决、部分解决、未解决，还是无法判断。
+
+如需补测未判定、技术失败或其他指定样本，可单独使用 `$atk-eval --ids 7 9`。需要确认改动对整组样本的影响时，运行不带筛选参数的 `$atk-eval`，再诊断和生成报告。
+
+### 查看评测产物
+
+每轮结果保存在 Agent 项目的 `.atk/results/vN/` 下，文件随对应步骤生成。
+
+| 文件 | 用途 |
+| --- | --- |
+| `eval_results.csv` | 查看实际输出和执行状态，按 `log_path` 查找样本日志 |
+| `judgments.csv`、`failure_cases.csv` | 复核逐条判定和已确认的失败。可用 `failure_cases.html` 浏览失败样本 |
+| `report.md`、`tuning_plan.md` | 查看本轮问题、改动及验证目标。后续报告会在条件可比时核对这些目标 |
+
+[更多批量评测运行选项和产物](docs/skill-template-pack-usage.md#manual-22--26-loop)
+
+![评测调优流程，从准备数据和接入 Agent 到评测、诊断、报告、调优和复测。](docs/assets/architecture.svg)
+
+## 轨迹持续调优
+
+从一个已有会话出发，结合实际执行轨迹判断任务是否完成，再按问题保存诊断、修复和验证记录。准备原始 Session ID、来源和可读取的轨迹，直接在 Codex 中打开 Agent 项目开始。无需评测数据集或 runner。
+
+可用 `atk trace --help` 检查当前 CLI 是否支持轨迹调优。如果命令不可用，按上面的本地源码方式安装支持轨迹调优的版本，再新建 Codex 会话。
+
+### 1. 准备会话轨迹
+
+- **Langfuse v4**。首次接入可用 `$atk-trace-init` 记录项目和只读来源配置。配置只引用凭据的环境变量名。然后发送 `$atk-trace-capture 下载 <session-id> 的会话，时间范围是 <开始时间> 到 <结束时间>`。
+- **本地文件**。发送 `$atk-trace-capture 从 <文件路径> 导入 <session-id> 的轨迹，来源是 <来源>`。也可在分析时提供可验证会话归属的文件，由分析流程导入。已归档的会话可直接分析。
+
+采集步骤负责下载或导入。保存前确认哪些正文可以留存、哪些字段需要脱敏。`.atk/traces/` 会加入项目的 `.gitignore`。
+
+### 2. 分析并查看证据
+
+```text
+$atk-trace-detect 分析 <session-id> 的任务结果是否正确，列出问题、证据和未检查的范围。
+$atk-visualize-trace 打开 <session-id> 的执行泳道，从问题跳到相关轨迹片段和证据字段。
+```
+
+分析会列出问题、证据和检查范围。可用 Trace ID 定位会话内的证据。有归档轨迹即可生成预览，分享 HTML 前请核对其中留存的正文。
+
+### 3. 按问题调查、修复和验证（按需）
+
+```text
+$atk-trace-diagnose 调查 <session-id> 的 ISSUE-001，给出修复和验证方案。
+$atk-trace-optimize 实施 <session-id> 的 ISSUE-001 已选定的方案。
+$atk-trace-verify 验证 <session-id> 的 ISSUE-001 对应候选，并检查相关正常行为。
+```
+
+已有明确问题和会话证据时，可以从调查开始。每次验证按问题保存结论。证据不足以确认实际加载的候选或关键验证条件时，效果记为 `inconclusive`（无法判断）。后续发现新的问题，可继续按问题处理，或用新会话的轨迹开始下一次分析。
+
+### 查看轨迹产物
+
+会话归档保存在 Agent 项目的 `.atk/traces/<session_id>/` 下，文件随对应步骤生成。
+
+| 文件或目录 | 用途 |
+| --- | --- |
+| `session.json`、`raw/` | 查看来源、采集范围、分析覆盖和保留的轨迹 |
+| `preview.html` | 离线浏览：左侧问题栏、上方泳道、下方 Event；按 Issue 片段定位证据 |
+| `issues/<issue-id>/` | 查看问题证据、诊断和改动。各次验证的 `result.json` 记录验证状态、结论和证据引用 |
+
+[轨迹调优指南，包含来源配置、支持格式和归档结构](docs/trace-tuning.md)

{agent_tune_kit-2.2.1 → agent_tune_kit-2.3.0}/agent_tune_kit/__init__.py

@@ -1,3 +1,3 @@
 """Agent Tune Kit package."""
 
-__version__ = "2.2.1"
+__version__ = "2.3.0"

{agent_tune_kit-2.2.1 → agent_tune_kit-2.3.0}/agent_tune_kit/cli/main.py

@@ -7,10 +7,10 @@ from agent_tune_kit import installer
 from agent_tune_kit.errors import UserActionRequired
 from agent_tune_kit.execution.runtime import ExecutionFailed
 
-from . import dataset, evaluation, results, visualize
+from . import dataset, evaluation, results, trace, visualize
 from .output import emit
 
-HANDLERS = {"dataset": dataset, "eval": evaluation, "results": results, "visualize": visualize}
+HANDLERS = {"dataset": dataset, "eval": evaluation, "results": results, "visualize": visualize, "trace": trace}
 
 
 def main(argv=None):