niceeval 0.4.4 → 0.4.6

package/README.md

@@ -12,19 +12,16 @@
 
 </div>
 
-NiceEval is an Agent-Native eval tool inspired by [eve](https://eve.dev), built for the best possible DX.
+NiceEval is an agent eval tool that helps teams measure, evaluate, and improve AI in production. With NiceEval, teams can compare models, iterate on agents, catch regressions, and keep improving their AI applications using real user data.
 
-Thanks to its universal design, NiceEval can evaluate almost any agent application.
-Whether you need to evaluate coding agent plugins, Hooks, and Skills written for Claude Code / Codex, or your own AI agent application — both plug in easily.
-
-After the eval completes, it generates readable reports and lets you view agent behavior details. Convenient for debugging and understanding agent behavior.
+NiceEval is local-first at its core: your evals run in your own environment. When your team needs to share evals or track regressions, you can push a Report to platforms like BrainTrust, or export a custom report.
 
 ## Why NiceEval when DeepEval, LangFuse, and BrainTrust already exist
 
 NiceEval is an Agent-Native eval tool. The Dataset / golden pattern of building an Input and an Expected Output doesn't fit real agent evaluation.
 Agents today need to be evaluated at a finer grain — multi-turn conversations, multi-agent collaboration, tool calls, skill loading — and NiceEval does this better.
 
-It also coexists with LangFuse and BrainTrust: use them for tracing, or upload eval results to both (in progress).
+It also coexists with LangFuse and BrainTrust: use them for tracing, or upload eval results to both.
 
 ## Architecture
 
@@ -73,6 +70,17 @@ NiceEval supports two integration modes, depending on whether the agent under te
 - **Agent adapters** are the open boundary: you decide how to call the system under test.
 - Coding agents that need filesystem isolation run inside the **Docker Sandbox**; your own AI agent can connect directly, without Docker.
 
+## Core concepts at a glance
+
+| Concept | In one line |
+|---|---|
+| Eval | A test case: written in `evals/*.eval.ts`, describing what to check. |
+| Experiment | A checked-in run configuration: which Adapter, which model, which flags. |
+| Adapter | The layer that connects to the system under test: implement one `send`, get back a standard event stream. |
+| Sandbox | Only needed for coding agents that require an isolated workspace; a direct web agent doesn't need one. |
+| Tier | Three levels of Adapter integration effort: Tier 1 wires up `send` only, Tier 2 adds OTel for a call waterfall, Tier 3 makes invasive changes for feature A/B testing. |
+
+See the full glossary in the [architecture overview](https://niceeval.com/docs/concepts/overview).
 
 ## Example
 
@@ -126,9 +134,9 @@ READ https://niceeval.com/INIT.md and install niceeval for this repo.
 
 Start from the scenario that matches what you need to evaluate:
 
-- [Claude Code / Codex plugin eval](https://niceeval.com/docs/zh/example/claude-code-codex-plugin)
-- [Claude Code / Codex skill eval](https://niceeval.com/docs/zh/example/claude-code-codex-skill)
-- [AI Agent application eval](https://niceeval.com/docs/zh/example/ai-agent-application)
+- [Claude Code / Codex plugin eval](https://niceeval.com/docs/example/claude-code-codex-plugin)
+- [Claude Code / Codex skill eval](https://niceeval.com/docs/example/claude-code-codex-skill)
+- [AI Agent application eval](https://niceeval.com/docs/example/ai-agent-application)
 
 
 ## Roadmap
@@ -157,8 +165,9 @@ Official Adapters
 
 # Acknowledgements
 This project was inspired by — or had its code learned by AI from — the projects below:
-[eve](https://eve.dev)
-[agent eval](https://github.com/vercel-labs/agent-eval)
-[ponytail](https://github.com/DietrichGebert/ponytail)
+- [eve](https://eve.dev): the main DX and API inspiration
+- [agent eval](https://github.com/vercel-labs/agent-eval)
+- [ponytail](https://github.com/DietrichGebert/ponytail)
 
 Thanks to the following communities
+- WIP

package/README.zh.md

@@ -12,18 +12,16 @@
 
 </div>
 
-NiceEval 是一个受 [eve](https://eve.dev) 启发的 Agent-Native eval 工具，追求极致的DX。
+NiceEval 是一个 Agent 评估工具，帮助团队衡量、评估并改进生产环境中的 AI。借助 NiceEval，团队可以比较模型、迭代 Agent、发现回归问题，并利用真实用户数据持续改进 AI 应用。
+ 
+NiceEval 以本地优先为核心：你的评估在你自己的环境中运行。当你的团队需要分享评估、回归跟踪时，可以通过 Report 上报到 BrainTrust 等平台，或者自定义的报告导出。
 
-通过通用式的设计，使 NiceEval 可以为几乎所有 Agent 应用进行评估。
-无论你是需要评估为 Claude Code / Codex 写的 coding agent 插件、Hook 和 Skill，还是可以评估你自己的 AI Agent 应用都能轻松接入。
-
-在 eval 完成之后可以生成易读的报告与查看 Agent 的行为细节。方便 Debug 与理解 Agent 行为。
 
 ## 为什么有了 DeepEval、LangFuse、BrainTrust 还需要 NiceEval
 NiceEval 是一个 Agent-Native 的评估工具。Dataset / golden 那一套「构建 Input 与 Expected Output」的模式，并不适合真实的 Agent 评估。
 现在 Agent 需要在多轮对话、多 agent 协作、工具调用、Skill 加载等细粒度场景下进行评估，NiceEval 能做得更好。
 
-同时，NiceEval 也能与 LangFuse、BrainTrust 共存：可以用它们做 tracing，或者把评估结果上传到两者（这部分功能还在开发中）。
+同时，NiceEval 也能与 LangFuse、BrainTrust 共存：可以用它们做 tracing，或者把评估结果上传到两者。
 
 ## 架构
 
@@ -167,8 +165,9 @@ READ https://niceeval.com/INIT.md and install niceeval for this repo.
 
 # 感谢
 该项目受下面项目所启发或者是由AI从下面项目中学习代码所写
-[eve](https://eve.dev)
-[agent eval](https://github.com/vercel-labs/agent-eval)
-[ponytail](https://github.com/DietrichGebert/ponytail)
+- [eve](https://eve.dev): 主要的 DX 与 API 受 Eve 所启发
+- [agent eval](https://github.com/vercel-labs/agent-eval)
+- [ponytail](https://github.com/DietrichGebert/ponytail)
 
 感谢下列社区
+- WIP

package/docs-site/.mintignore

@@ -0,0 +1,13 @@
+# Mintlify automatically ignores these files and directories:
+# .git, .github, .claude, .agents, .idea, node_modules,
+# README.md, LICENSE.md, CHANGELOG.md, CONTRIBUTING.md
+
+# Draft content
+drafts/
+*.draft.mdx
+
+# Scraped source content (not documentation pages)
+sources/
+
+# Agent-only writing reference, not part of the published docs surface
+docs-ref/

package/docs-site/AGENTS.md

@@ -0,0 +1,46 @@
+# Mintlify 文档站指南
+
+`docs-site/` 是 NiceEval 的公开 Mintlify 文档站，面向正在学习和使用 NiceEval 的用户。这里的文档要回答“用户怎么完成任务”，不是记录内部设计讨论；内部设计、取舍和源码地图放在仓库根目录的 `docs/`。
+
+## 目录结构
+
+- `docs.json`：Mintlify 导航、主题、logo、navbar、redirects。
+- 顶层 `*.mdx`：英文入口页，例如 introduction、quickstart、installation。
+- `concepts/`：英文核心概念页，解释心智模型。
+- `guides/`：英文任务指南，按工作流组织。
+- `reference/`：英文 API / CLI 参考，列完整字段和选项。
+- `zh/`：中文文档。中文定位、概念命名和场景示例是公开叙事的准绳；英文页或 README 与中文冲突时，先按中文和当前代码核对，再同步其它入口。
+
+## 术语表
+
+- **NiceEval**：产品名。中文正文表达产品时用 `NiceEval`；命令、包名、配置文件、代码标识里用 `niceeval`。
+- **Eval**：一个评测用例。中文可以写“Eval”或“评估用例”
+- **Experiment**：可签入的运行配置。中文写“实验”或保留 `Experiment`，用于说明 agent、model、flags、runs、budget、sandbox 等运行维度。
+- **Adapter**：适配器，负责连接被测系统、鉴权、调用接口、把返回翻译成标准事件流。页面标题和导航可用 `Adapter`。
+- **Agent**：NiceEval 看到的被测对象连接。不要把 Agent 写成某个固定协议；具体协议属于 Adapter。
+- **Sandbox**：沙箱后端，回答“在哪里隔离运行”。不要和 Adapter 混成一层。
+- **Turn**：一次 `t.send()` / `t.respond()` 的结果。中文可写“一轮”或保留 `Turn`。
+- **StreamEvent / events**：标准事件流，是断言和报告读取的事实来源。
+- **HITL**：human-in-the-loop，人工介入。第一次出现时写全称或中文解释。
+- **OTel 接入**：Tier 2 的接入方式。只在讲 `send + OTel` 时使用，不要把 OTel 写进 Tier 1。
+- **Flags**：experiment 传入的 feature flags，经 `ctx.flags` 到 Adapter，经 `t.flags` 到 eval。不要写成 CLI flags，除非指命令行参数。
+- **Runner**：运行器。面向用户文档里避免写 “NiceEval core”；需要表达执行主体时写 NiceEval 或 runner。
+
+## 写作规则
+- 英语单词应该以大写开头
+- 只在 @docs-site/zh 下面更新中文版本，英语版本由其它 AI 翻译
+- 新增或重命名页面时同时更新 `docs.json`，必要时加 redirect，避免旧链接断掉。
+- 链接示例必须指向真实存在的 `examples/` 目录；当前完整示例主要在 `examples/zh/`。
+- 工作流写成 guide，字段全集写成 reference，概念边界写成 concepts。不要把一个页面同时写成教程、设计文档和 API 字典。
+- 命令、路径、flag、文件名、包名、代码标识用反引号。
+- 文案使用主动语态和短句。错误信息、限制和前置条件要直接说清楚下一步。
+- 写作指南: docs-site/docs-ref/00-index.md
+
+## 校验
+
+改 `docs-site/` 后，从仓库根目录运行：
+
+```sh
+PATH=/opt/homebrew/opt/node@22/bin:$PATH pnpm run docs:validate
+PATH=/opt/homebrew/opt/node@22/bin:$PATH pnpm run docs:links
+```

package/docs-site/concepts/adapter.mdx

@@ -0,0 +1,287 @@
+---
+title: "Adapter: Connect an Agent to NiceEval"
+sidebarTitle: "Adapter"
+description: "The Adapter is the adapter you write. This page explains what the send function receives and returns, how system-under-test configuration is passed in, the three integration tiers, and where the capabilities on t come from."
+---
+
+In NiceEval, an `Adapter` is the adaptation layer between the runner and the system under test. The Adapter implements the `send` function: it forwards eval-side actions such as `t.send()` to your application, then translates the application's response into NiceEval's standard `Turn`.
+
+Follow-up reading: [Write Send](/guides/write-send)
+
+## The adapter
+
+If the system under test speaks the standard OpenAI Chat Completions or Responses protocol, use an official adapter directly. For a homegrown frontend-backend protocol (HTTP, gRPC, WebSocket — anything works), write an Adapter: it knows how to authenticate, how to call your application, and how to translate the response into the standard event stream.
+
+One full round trip of `t.send()` crosses this boundary twice:
+
+![Full round trip of t.send: the eval calls t.send, the runner assembles TurnInput and ctx, the adapter calls your application and returns a Turn with the standard event stream.](/images/agent-turn-roundtrip-en.svg)
+
+Outbound: the driving verbs in the eval (`t.send` / `t.sendFile` / `t.respond`) assemble a `TurnInput` and the `ctx` for this conversation line, then call the Adapter's `send` once.
+
+Inbound: the Adapter translates the application's raw response into a `Turn` — where `events` is an array of event objects ordered by when they happened (real values below).
+
+The system under test's URL, auth, and protocol details are passed in through the Adapter factory's configuration. `defineExperiment` receives an already configured agent instance; the Adapter consumes that configuration inside the `send` closure, and forwards per-turn dynamic values (such as `ctx.model`, `ctx.flags`, `ctx.telemetry`) to the application with each request:
+
+```ts
+// agents/web-agent.ts
+import { defineAgent } from "niceeval/adapter";
+import type { Agent } from "niceeval/adapter";
+
+interface WebAgentOptions {
+  baseUrl: string;
+  apiKey?: string;
+}
+
+export function webAgent(options: WebAgentOptions): Agent {
+  const baseUrl = options.baseUrl.replace(/\/$/, "");
+
+  return defineAgent({
+    name: "web-agent",
+    async send(input, ctx) {
+      const response = await fetch(`${baseUrl}/api/turn`, {
+        method: "POST",
+        headers: {
+          "content-type": "application/json",
+          ...(options.apiKey ? { authorization: `Bearer ${options.apiKey}` } : {}),
+          ...ctx.telemetry?.headers,
+        },
+        body: JSON.stringify({
+          message: input.text,
+          files: input.files,
+          model: ctx.model,
+          flags: ctx.flags,
+        }),
+        signal: ctx.signal,
+      });
+
+      if (!response.ok) {
+        return {
+          status: "failed",
+          events: [{ type: "error", message: `HTTP ${response.status}` }],
+        };
+      }
+
+      const body = await response.json();
+      return {
+        status: "completed",
+        events: body.events,
+        data: body.data,
+        usage: body.usage,
+      };
+    },
+  });
+}
+
+// experiments/staging.ts
+import { defineExperiment } from "niceeval";
+import { webAgent } from "../agents/web-agent.ts";
+
+export default defineExperiment({
+  agent: webAgent({
+    baseUrl: "https://staging.example.com",
+    apiKey: process.env.STAGING_AGENT_API_KEY,
+  }),
+  model: "gpt-5.4",
+  flags: { promptVariant: "concise" },
+});
+```
+
+## Integration tiers
+
+Grouped by where the Adapter connects and what extra observability data it receives, integration comes in three tiers: **Tier 1 is send only** (not a single line of application code changes, and the full assertion set is already complete at this tier), **Tier 2 is send + OTel** (the application sends spans to NiceEval, buying the call waterfall in `niceeval view`), **Tier 3 is application changes + experiment flags** (feature A/B). What each tier costs, what it buys, and when to move up: see [Tier](/concepts/tier).
+
+The eval-side driving API — `t.send()`, `t.sendFile()`, `t.newSession()`, and HITL's `t.respond()` / `t.respondAll()` —
+all uniformly call the Adapter's `send`.
+
+How they converge and how to handle them in `send`: see [Write Send](/guides/write-send).
+
+## The send function
+
+Whether the Adapter connects to an HTTP service or a CLI in a sandbox, the interface exposed to the runner is exactly the same:
+
+```ts
+interface Agent {
+  readonly name: string;
+  send(input: TurnInput, ctx: AgentContext): Promise<Turn>;
+  // Optional: setup / teardown (once-per-sandbox lifecycle);
+  // optional observability members are not part of send — see "OTel Integration"
+}
+```
+
+`send` is the only function you must implement. There are only three types in its signature; take them one at a time.
+
+### Input: `TurnInput`
+
+```ts
+interface TurnInput {
+  readonly text: string;                          // Every turn: the text being sent; on answer turns it's the answer text (multiple answers joined by newlines)
+  readonly files?: readonly InputFile[];          // t.sendFile turns only: attached files (base64)
+  readonly responses?: readonly InputResponse[];  // Answer turns only (t.respond / t.respondAll): one structured answer per request
+  readonly outputSchema?: JsonSchema;             // Only when this turn declares output: the structured output requirement, lowered to JSON Schema
+}
+
+interface InputFile {
+  readonly filename?: string;     // Optional, for the adapter / model to reference
+  readonly mimeType: string;      // e.g. image/png
+  readonly dataBase64: string;    // base64 content, JSON-friendly, can go straight into the request body
+}
+
+interface InputResponse {
+  readonly requestId: string;    // Always present: which input.requested request this answers; used to match up when multiple requests are paused
+  readonly optionId?: string;    // Either this or text: the answer hit one of the request's option ids (approve / deny...)
+  readonly text?: string;        // Either this or optionId: a free-text answer (the request has no options, or the answer matches none)
+}
+```
+
+Whether the eval side calls `t.send()`, `t.sendFile()`, or `t.respond()`, it arrives at the Adapter as one ordinary `send` — the entire difference is in the fields:
+
+If your application interface does not accept files, ignore `files`.
+
+| Eval side | `text` | `files` | `responses` | `outputSchema` |
+|---|---|---|---|---|
+| `t.send(text)` | Sent text | — | — | Only when `output` is declared |
+| `t.sendFile(path, text?)` | Caption (may be empty string) | Present | — | Same |
+| `t.respond(...)` / `t.respondAll(...)` | Answer text | — | One per request | Same |
+
+`files`: ignore it if the interface does not accept files;
+`outputSchema`: forward it if the application interface accepts a schema (Chat Completions' `response_format`, Responses' `text.format`); it's fine if it doesn't — validation happens in the runner anyway (see the `Turn.data` rule below).
+
+#### Inputs for the different answers
+
+On HITL answer turns, the human verdict arrives in structured form via `input.responses` — the Adapter never needs to parse `text` to guess which sentence answers which request or whether it counts as approval. Every answer carries a `requestId`; `optionId` and `text` are mutually exclusive: if the answer hits one of the request's option ids it becomes `optionId` (the eval side has already validated it exists, so no typo can slip through silently); otherwise the whole sentence lands in `text` as free text. The four typical shapes:
+
+```ts
+// 1) A single pending request, answer hits an option (approve / deny work the same, only the optionId differs)
+await t.respond("approve");
+// → { text: "approve",
+//     responses: [{ requestId: "req_1", optionId: "approve" }] }
+
+// 2) Multiple requests paused at once — the object form matches them up explicitly
+await t.respond({ request, optionId: "deny" });
+// → { text: "deny",
+//     responses: [{ requestId: request.id, optionId: "deny" }] }
+
+// 3) The answer is none of the options → free text (requests waiting for extra info look like this)
+await t.respond("Change the recipient to ceo@corp.com");
+// → { text: "Change the recipient to ceo@corp.com",
+//     responses: [{ requestId: "req_1", text: "Change the recipient to ceo@corp.com" }] }
+
+// 4) respondAll: one answer per pending request, same optionId
+await t.respondAll("approve");
+// → { text: "approve\napprove",
+//     responses: [{ requestId: "req_1", optionId: "approve" },
+//                 { requestId: "req_2", optionId: "approve" }] }
+```
+
+The Adapter's corresponding obligations: hand the verdict back to the application by `requestId` (don't guess by order); for calls a human rejected, set the `action.result` event's `status` to `"rejected"` rather than `"failed"` — rejection is a human decision, not a tool failure, so `noFailedActions()` doesn't misfire and `calledTool(..., { status: "rejected" })` can assert it precisely.
+
+### Context: `AgentContext`
+
+```ts
+interface AgentContext {
+  // The three that are present every turn
+  readonly signal: AbortSignal;    // The runner's timeout and cancellation: pass it through to every request you make
+  readonly session: AgentSession;  // The state slot for this conversation line
+  log(msg: string): void;          // Writes to this turn's log, visible in niceeval view
+
+  // The three that appear per integration tier (see "Integration tiers")
+  readonly model?: string;         // Tier 1 model comparison: experiment.model passed through; forward it if the application interface accepts model selection
+  readonly reasoningEffort?: string;  // Reasoning effort: experiment.reasoningEffort passed through, same ownership as model
+  readonly telemetry?: Telemetry;  // Tier 2: this turn's W3C trace context (headers, a fresh traceparent per turn)
+  readonly flags: Readonly<Record<string, unknown>>;  // Tier 3 feature A/B: experiment.flags passed through; {} when unset
+
+  readonly sandbox?: Sandbox;      // Only Agents constructed by defineSandboxAgent receive this
+}
+
+interface AgentSession {
+  // Session continuation: server keeps the history (the shape for interfaces that accept a session id)
+  readonly id?: string;                      // The session id recorded for this line; undefined on a new conversation line
+  capture(id: string | undefined): void;     // Record the returned id; only lands if none has been recorded yet
+
+  // Session continuation: client carries the full history (Chat Completions shape)
+  history<TMsg>(): { get(): TMsg[]; commit(messages: TMsg[]): void };
+
+  // HITL: store and retrieve the paused-turn state (take clears on read — consumed once)
+  hold<T>(state: T): void;
+  take<T>(): T | undefined;
+
+  // Escape hatch: a free-form state slot, starts as {}, the framework never writes to it
+  readonly state: Record<string, unknown>;
+}
+```
+
+There are no flags to check on `ctx`. All three tier fields have "pass-through" semantics: `model` and `flags` are declared by the experiment and handed over verbatim by the runner — the Adapter's only job is to forward them to the application with the request, never to interpret them; `telemetry` only appears when OTel integration is configured, and inside `send` you only need to spread its `headers` into the request headers — the receiving endpoint is fixed in `defineConfig` and pointed at by the application at startup, not passed from here; see [OTel Integration](/guides/connect-otel).
+
+`session` is the private state of one conversation line, and NiceEval promises exactly one thing about it: **every `send` on the same conversation line receives the same `ctx.session`, and a new conversation line (the first turn of an eval, or after `t.newSession()`) receives a brand-new one.** The session continuation accessors (`id`/`capture`, `history`) and the HITL paused-turn accessors (`hold`/`take`) all live on it, and "the first turn" is simply the natural shape of a new conversation line — `id` is `undefined`, `history.get()` is an empty array, with no branch to write; `state` is the escape hatch beyond those accessors, and the framework never writes data into it.
+
+### Return: `Turn`
+
+```ts
+interface Turn {
+  readonly events: StreamEvent[];          // ★ The standard event stream — every Adapter's core product
+  readonly data?: JsonValue;               // This turn's structured output; only filled when input.outputSchema is present, validated by the runner against the declaration
+  readonly status: "completed" | "failed" | "waiting";  // waiting = paused for a human (HITL)
+  readonly usage?: Usage;
+}
+```
+
+`events` is just a **plain JS array**: one object per thing that happened this turn, in the order it actually happened. For a turn like "How warm is Beijing today?", where the agent checks the weather before answering, `events` looks like this:
+
+```ts
+[
+  { type: "action.called", callId: "c1", name: "get_weather", input: { city: "Beijing" } },
+  { type: "action.result", callId: "c1", output: { temp: 21 }, status: "completed" },
+  { type: "message", role: "assistant", text: "Beijing is 21°C today." },
+]
+```
+
+There are nine object types in total (`message`, `action.*`, `input.requested`… full list in the [Events Reference](/reference/events)). Assertions read this exact array: `t.calledTool("get_weather")` counts `action.called`, `t.reply` takes the last assistant `message`. Note that each `send` returns only **this turn's** array — stitching turns into a whole conversation line is the runner's job, covered in the next section. In most cases you don't hand-write these objects either: an official converter's return value is already a complete `Turn` with `events`, `usage`, and `status` filled in — how to pick a converter is covered in [Write Send](/guides/write-send).
+
+`data` is not a "put whatever" pocket; it has exactly one rule: **what goes in it is declared by the eval with a schema on the send; only a declaration produces data, and what you get is the declared type.**
+
+```ts
+// Eval side: the output declaration is both the requirement given to the application and the type source for turn.data
+const turn = await t.send("What's the total on this invoice?", {
+  output: z.object({ amount: z.number(), currency: z.string() }),
+});
+turn.data.amount;   // Typed as number — not unknown, no manual narrowing
+```
+
+The declaration, lowered to JSON Schema, reaches the Adapter via `input.outputSchema`; the Adapter puts the structured output from the application's response into `data` (e.g. `JSON.parse(reply.content)`). **Validation is enforced by the runner**: if `data` doesn't match the declaration, the turn fails immediately with the diff reported — there is no path where "a wrong object got stuffed in and the assertion silently passed". Conversely, turns with no `output` declaration have no `data`: applications that only return text never touch this field — don't copy the raw response body into it to fill space.
+
+### After send returns: where the four Turn fields go
+
+The moment `send` returns, the Adapter's work is done — everything after that is the runner's job. Each of the four fields has a definite destination:
+
+| Turn field | What the runner does with it | Where the eval author feels it |
+|---|---|---|
+| `events` | Appended to this conversation line's event stream; `deriveRunFacts` folds it into facts (`toolCalls`, `parked`, `messageCount`…); the last assistant `message` also updates `t.reply` | All scoped assertions: `t.calledTool()`, `t.messageIncludes()`…; `t.events` is directly queryable |
+| `status` | Records the turn's outcome; on `"waiting"`, collects this turn's `input.requested` into the pending list | `t.succeeded()` / `t.parked()`; after `"waiting"`, continue with `t.respond()` |
+| `usage` | Accumulated per turn into the conversation line and the whole run | `maxTokens` / `maxCost` scorers, usage in reports |
+| `data` | Validated against the `output` schema declared on the send — mismatch fails the turn immediately; saved only if it passes | `turn.data` (strongly typed by the declaration), `outputEquals` |
+
+The handle you get from `await t.send()` is a view onto this one Turn: turn-level assertions (things like `draft.parked()`) look only at this turn's `events`, while `t`-level assertions look at the whole conversation line's accumulation. So the Adapter never needs to "cooperate" with any assertion — fill in these four fields correctly, and assertions, scoring, and reporting all happen automatically downstream.
+
+## Where capabilities come from: proof by construction, not a questionnaire
+
+
+| What unlocks on `t` | What the evidence is | What you write |
+|---|---|---|
+| `t.send`, `t.sendFile`, `t.check`, `t.judge`… | Every Agent has them | Nothing |
+| Multiple `t.send()`, `t.reply`, `t.newSession()` | `send` wired up `ctx.session`'s continuation accessors (`history()` or `id` + `capture()`) — without them every turn is a fresh conversation | Nothing — the accessors are right on `ctx` |
+| `t.respond()` / `t.parked()` and other HITL | send has returned `"waiting"` + an `input.requested` event | Nothing — doing it is having it |
+| `t.calledTool()` / `t.toolOrder()` and other positive assertions | `action.*` present in events | Nothing — having events means you can assert |
+| `t.notCalledTool()` / `t.usedNoTools()` and other negative assertions being **trustworthy** | Events come from an official converter with a completeness proof | Nothing — the proof travels with the converter's return value |
+| `t.sandbox`, `t.sandbox.fileChanged()`, etc. | Constructed by `defineSandboxAgent` | Nothing |
+| `EvalResult.trace`, the `niceeval view` waterfall | A `tracing` block exists (spans only feed the waterfall, never assertions) | Nothing — you were writing the block anyway |
+
+The negative-assertion row is the only tiered one: `notCalledTool` asserts "it did not happen", which presupposes "the events are complete" — and that only holds when the source itself carries a completeness contract (SDK-native event stream pass-through, AI SDK's `result.steps`, Responses' `output`). Hand-written mappings carry no proof, so negative assertions warn at runtime that they are untrustworthy instead of silently passing. This also means there is no "declared but can't deliver" drift: trustworthiness follows where the events came from, not the author's self-assessment. The precise per-capability obligations are in the [Capabilities Reference](/reference/capabilities).
+
+
+## Related reading
+
+- [Write Send](/guides/write-send) — The hands-on tutorial: from sending one message to a full integration, seven steps, each unlocking a set of assertions.
+- [Connect Your Agent](/guides/connect-your-agent) — The integration overview: minimal integration, parameter channels, and the incremental map.
+- [Drive](/concepts/drive) — The eval-side view: how to use `t.send()`, `t.newSession()`, and HITL.
+- [Assert](/concepts/assert) — The complete assertion vocabulary driven by the standard event stream.
+- [Architecture Overview](/concepts/overview) — The four layers and their boundaries.