niceeval 0.4.4 → 0.4.5

package/docs-site/guides/write-send.mdx

@@ -0,0 +1,347 @@
+---
+title: "How to write send"
+sidebarTitle: "Write Send"
+description: "Hand-write the Adapter's send function along one storyline in seven steps: send a message and get the reply, continue from earlier messages, record usage, parse tools into events, human-in-the-loop (HITL), hook up OTel traces, and pass through experiment flags. Each step adds only a few lines to the previous step's code, and each step unlocks a family of assertions."
+---
+
+[Adapter](/concepts/adapter) covers the contract: `send` receives `TurnInput` and `AgentContext` and returns a `Turn`. This page covers how to write it — starting from the minimal "send one message, get the reply" form and building step by step to a complete integration. Each step adds only a few lines to the previous step's code; parts already shown are marked with `// …omitted` in the code, and each step ends by noting which assertions those lines unlock. Every step is immediately verifiable: write the newly unlocked assertions into an eval, rerun `npx niceeval exp`, and `niceeval view` shows the data this step added — multi-turn trajectories, usage, tool events, pending requests.
+
+Two principles run through the whole page:
+
+- **Always connect at the interface your user-facing frontend already uses.** Whatever endpoint the page calls and whatever format it receives, the Adapter calls that endpoint and receives that format — do not open a new endpoint for evals, and do not import app internals to call functions directly (for why, see [Connect your agent](/guides/connect-your-agent)).
+- **The only thing you truly hand-write is transport** — URL, auth, and request body differ per app anyway. Parsing (raw response → standard event stream) has official converters, exported from `niceeval/adapter`; the state slots for orchestration (session continuation, HITL pause/resume) hang off `ctx.session`, ready to use, with nothing extra to declare.
+
+![Full round trip of one t.send: the eval calls t.send, the runner assembles TurnInput and ctx, the adapter calls your app and returns a Turn with the standard event stream.](/images/agent-turn-roundtrip-en.svg)
+
+## Starting point: what shape is your app's interface?
+
+NiceEval does not invent protocols. Your app is almost certainly built on one of the industry protocols below (or a variant of one), and the official pieces cover exactly these shapes:
+
+| Protocol | Who defines it | In one sentence | Reference |
+|---|---|---|---|
+| **Chat Completions** | OpenAI; the industry's de facto standard — nearly every model provider offers a compatible endpoint | Stateless question-and-answer: the client sends the full `messages` list every turn and one JSON comes back | [API reference](https://platform.openai.com/docs/api-reference/chat) |
+| **Responses / Open Responses** | OpenAI (the Responses API); Open Responses is an open specification built on it, initiated by OpenAI and co-developed with Hugging Face and the community for multi-provider interoperability | Stateful and agent-oriented: requests carry `previous_response_id` so the server continues the history, and the `output` array promises a record of the full process | [API reference](https://platform.openai.com/docs/api-reference/responses) · [Open Responses spec](https://www.openresponses.org/specification) |
+| **AI SDK (Vercel)** | Vercel; the de facto standard of the TypeScript ecosystem | Not a wire protocol but an SDK: `generateText` returns a complete result, and `useChat` speaks the UI Message Stream protocol it defines | [generateText reference](https://ai-sdk.dev/docs/reference/ai-sdk-core/generate-text) · [UI Message Stream protocol](https://ai-sdk.dev/docs/ai-sdk-ui/stream-protocol) |
+
+The storyline below writes an app with a Chat Completions-shaped interface (the most common shape) from zero to complete; if your interface is Responses-shaped or streaming, the replacement pieces are given in steps two and four respectively.
+
+## Step 1: send one message, get the reply
+
+The minimal `send` does exactly three things: send `input.text` to the app's interface, put the reply into one `message` event, and report this turn's `status`:
+
+```ts
+// agents/chat-app.ts
+import { defineAgent } from "niceeval/adapter";
+
+const BASE_URL = "http://localhost:8080";   // to switch the URL per experiment, make it a factory option: see the integration guide
+
+export default defineAgent({
+  name: "chat-app",
+  async send(input, ctx) {
+    const res = await fetch(`${BASE_URL}/v1/chat/completions`, {  // ← the interface the frontend already uses
+      method: "POST",
+      headers: { "content-type": "application/json" },
+      body: JSON.stringify({ messages: [{ role: "user", content: input.text }] }),
+      signal: ctx.signal,                                         // ← runner timeout and cancellation
+    }).then((r) => r.json());
+
+    return {
+      status: "completed",
+      events: [{ type: "message", role: "assistant", text: res.choices[0].message.content }],
+    };
+  },
+});
+```
+
+**This step unlocks**: `t.reply`, `t.messageIncludes()`, and all the conversation material for judges.
+
+It has two obvious limitations: every turn is a brand-new conversation (the second `t.send` cannot continue from the first), and tool calls are completely invisible. The next two steps solve one each.
+
+## Step 2: continue from earlier messages
+
+The runner promises exactly one thing about sessions: **every `send` on the same session line gets the same `ctx.session`; a new session line (the eval's first turn, or after `t.newSession()`) gets a brand-new one.**
+
+How "earlier messages" get continued depends on the shape of the app's interface. The industry has exactly two patterns, and `ctx.session` provides a pair of accessors for each:
+
+- **Client carries the full history** (stateless server; the complete message list is sent every turn: the Chat Completions shape is the typical case) → `ctx.session.history<TMsg>()`
+- **Server keeps the history** (the interface takes a session id: the Responses shape's `previous_response_id`, the native sessions / threads of various SDKs) → `ctx.session.id` + `ctx.session.capture(id)`
+
+The storyline's interface is the former:
+
+```ts
+// agents/chat-app.ts
+import { defineAgent } from "niceeval/adapter";
+
+const BASE_URL = "http://localhost:8080";
+
+interface Msg { role: "user" | "assistant"; content: string | null }
+
+export default defineAgent({
+  name: "chat-app",
+  async send(input, ctx) {
+    const history = ctx.session.history<Msg>();  // ← this session line's history slot; empty on a new session line
+    const messages = [...history.get(), { role: "user" as const, content: input.text }];
+
+    const res = await fetch(`${BASE_URL}/v1/chat/completions`, {
+      // …method, headers, signal same as step one, omitted
+      body: JSON.stringify({ messages }),        // ← the change: send the full history
+    }).then((r) => r.json());
+
+    const reply = res.choices[0].message;
+    history.commit([...messages, reply]);        // ← write back; the next turn on the same line picks it up via get()
+
+    return {
+      status: "completed",
+      events: [{ type: "message", role: "assistant", text: reply.content }],
+    };
+  },
+});
+```
+
+Notice there is **no "first turn" branch**: `history.get()` on a new session line naturally returns an empty array — the "first send" form is the natural consequence of a new session line, not a condition for you to check. Nothing needs declaring on `defineAgent` either: connect `ctx.session` and multi-turn continues; skip it and every turn is a new conversation.
+
+If the app's interface takes a session id, the history lives on the server and the Adapter only records the id — just two changes inside `send`:
+
+```ts
+// Inside send:
+//   the request carries the previous turn's id … previous_response_id: ctx.session.id   ← naturally undefined on a new session line
+//   write back whatever id the response returns … ctx.session.capture(res.id)           ← the next turn continues from it
+```
+
+`capture` only lands when no id has been recorded yet; a backend re-sending the id (or even changing it due to a fork) will not overwrite the line being continued.
+
+**This step unlocks**: multi-turn conversations, and `t.newSession()` session isolation.
+
+## Step 3: record usage
+
+An agent that answers correctly but burns ten times the tokens should not get the same score as one that is frugal. Usage is `usage`, the fourth field on `Turn` alongside `events` and `status`: if the app's interface returns usage, report it truthfully, and the runner accumulates it turn by turn into the session line and the whole run. Chat Completions-shaped responses come with `usage` — copy it over; the rest of `send` is exactly the same as step two:
+
+```ts
+    // …transport and session same as step two, omitted
+    return {
+      status: "completed",
+      events: [{ type: "message", role: "assistant", text: reply.content }],
+      usage: {                                        // ← the change: report the usage the interface returned, as-is
+        inputTokens: res.usage.prompt_tokens,
+        outputTokens: res.usage.completion_tokens,
+      },
+    };
+```
+
+The full `Usage` fields also include the optional `cacheReadTokens` / `cacheWriteTokens`, the request count `requests`, and `costUSD` — if a gateway returns measured cost, fill that in; it takes precedence over price-table estimation. If the interface returns no usage, leave `usage` out entirely; other assertions are unaffected. This hand-copying is also transitional: the official converter in the next step fills in `usage` along with everything else.
+
+**This step unlocks**: the `t.maxTokens()` / `t.maxCost()` scorers (`maxCost` uses `costUSD` or the price table in the config), plus usage in reports and `niceeval view`.
+
+## Step 4: parse tools into events
+
+The app's response contains more than reply text — the `tool_calls` in a Chat Completions-shaped response record which tools this turn called. The Adapter's most important job is **normalizing the interface's response into the standard event stream**: one object per thing that happened this turn, ordered by actual occurrence in `Turn.events`, each object one of the nine types below (for the actual field values, [the contract page has a complete one-turn example](/concepts/adapter)):
+
+```ts
+type StreamEvent =
+  | { type: "message"; role: "assistant" | "user"; text: string }
+  | { type: "action.called"; callId: string; name: string; input: JsonValue; tool?: ToolName }
+  | { type: "action.result"; callId: string; output?: JsonValue;
+      status: "completed" | "failed" | "rejected" }
+  | { type: "subagent.called"; callId: string; name: string; remoteUrl?: string }
+  | { type: "subagent.completed"; callId: string; output?: JsonValue;
+      status: "completed" | "failed" }
+  | { type: "input.requested"; request: InputRequest }
+  | { type: "thinking"; text: string }
+  | { type: "compaction"; reason?: string }
+  | { type: "error"; message: string };
+```
+
+Parsing is one "response field → event" mapping. Hand-written it looks like this — the rest of `send` is exactly the same as step two:
+
+```ts
+    // …transport and session same as step two, usage same as step three, omitted
+    const reply = res.choices[0].message;
+
+    const events: StreamEvent[] = [];
+    for (const call of reply.tool_calls ?? []) {
+      events.push({ type: "action.called", callId: call.id,
+                    name: call.function.name, input: JSON.parse(call.function.arguments) });
+    }
+    if (reply.content) events.push({ type: "message", role: "assistant", text: reply.content });
+
+    return { events, status: "completed", usage };
+```
+
+But you usually **do not have to write this loop**. When the response is a standard shape, one line of official converter replaces all of the above — `events`, `status`, even step three's hand-copied `usage` are all in the return value; just `return` it:
+
+```ts
+    // …transport and session same as step two, omitted
+    return fromChatCompletion(res);            // ← the mapping above plus step three's usage, official edition
+```
+
+If the interface is not this shape, swap in the matching piece:
+
+| How the interface's data arrives | Official piece | What you write |
+|---|---|---|
+| Chat Completions-shaped response | `fromChatCompletion(res)` | Zero mapping |
+| Responses-shaped response | `fromResponses(res)` | Zero mapping |
+| AI SDK `generateText` complete result | `fromAiSdk(result)` | Zero mapping |
+| Streaming: SDK-native events passed through (one complete unit per frame) | `fromClaudeSdkMessages()` / `fromPiAgentEvents()` / `fromCodexThreadEvents()` | Zero mapping |
+| Streaming: the AI SDK's UI Message Stream (the `useChat` protocol) | `uiMessageStreamAgent()` | Zero mapping (the whole Agent is prebuilt — you do not even write send) |
+| Streaming: token-by-token / argument-by-argument deltas | The protocol's official reducer; only if there is none, `deltaStream({ toOps })` | One "frame type → operation" mapping table |
+| Final answer only | A few hand-written events | Start with one `message` (step one is this tier) |
+
+The prebuilt pieces go as far as "shape", not "protocol" — interfaces arrive in only these few shapes, one official piece each; only "delta stream with no official reducer from the protocol side" requires you to write a mapping, and what you write is a declaration (which operation this frame corresponds to) — concatenation, pairing, and landing timing are handled uniformly by `deltaStream`.
+
+Once normalized, whichever events you emit determine which assertion families eval authors can write:
+
+| Events you emit | Assertions unlocked |
+|---|---|
+| `message` | `t.reply`, `t.messageIncludes()`, judge conversation material |
+| `action.called` / `action.result` (paired by `callId`) | `t.calledTool()` / `t.toolOrder()` / `t.maxToolCalls()` / `t.noFailedActions()`… |
+| `input.requested` (together with status `"waiting"`) | `t.parked()`, `t.requireInputRequest()`, `t.respond()` (step five) |
+| `thinking` / `compaction` / `error` | The matching assertions and o11y counts |
+
+One honest disclosure: the Chat Completions shape **does not promise** that "response = full process record" (the app may well run the tool loop server-side and hand you only the final answer), so `fromChatCompletion`'s return carries no completeness proof — positive assertions like `calledTool` work as usual, while negative assertions like `notCalledTool` warn that they cannot be trusted. The Responses shape is the exact opposite: the protocol contract promises the `output` array records the full process, so `fromResponses`'s return carries a completeness proof and negative assertions are trustworthy. Same zero mapping — the difference in trustworthiness comes entirely from the interface shape itself.
+
+**This step unlocks**: the whole family of tool assertions — `t.calledTool()`, `t.toolOrder()`, `t.maxToolCalls()`, `t.noFailedActions()`, and more.
+
+## Step 5: HITL
+
+When the app stops mid-turn to wait for a human (tool approval, missing information), `send` has obligations on both sides:
+
+- **The pausing turn**: return `status: "waiting"`, and emit one `input.requested` event with a stable `id` per pending question — `t.parked()` and `t.requireInputRequest()` read them, and answers are matched by this `id`.
+- **The answer turn**: `t.respond(...)` in the eval reaches the Adapter as **just another ordinary `send`** (still the same session line, the same state); the human verdict arrives in structured form via `input.responses`, one entry per answer carrying `requestId` and `optionId` or `text` (shapes in [the answer input shapes](/concepts/adapter#input-turninput)). The Adapter hands the verdict back to the app first, then continues fetching the result. For calls a human rejected, set the `action.result` `status` to `"rejected"` rather than `"failed"` — a rejection is a human decision, not a tool failure, so `noFailedActions()` does not misfire.
+
+The "scene read halfway when the turn paused" (say, an SSE stream read halfway through) also lives on `ctx.session`: `ctx.session.hold(scene)` when pausing, `ctx.session.take()` at the start of the answer turn — taking clears it, single consumption.
+
+HITL almost always happens on streaming interfaces (pausing mid-stream), so this step's example switches to an app that passes native events through over SSE — it also exercises everything from the earlier steps (complete runnable version in the [tier1 examples](https://github.com/CorrectRoadH/niceeval/tree/main/examples/zh/tier1)):
+
+```ts
+// agents/my-app.ts
+import { defineAgent, sseJsonFrames, fromPiAgentEvents, driveFrameStream } from "niceeval/adapter";
+import type { AgentContext, PiAgentStream, SseFrameCursor } from "niceeval/adapter";
+import type { Turn, TurnInput } from "niceeval";
+
+const BASE_URL = "http://localhost:5299";               // the app's own port; evals do not manage the process
+
+interface Pending { cursor: SseFrameCursor<Frame>; stream: PiAgentStream; callId: string }
+
+function readStream(cursor: SseFrameCursor<Frame>, ctx: AgentContext, stream: PiAgentStream): Promise<Turn> {
+  return driveFrameStream(cursor, stream, ctx, (frame) => {
+    if (frame.type === "session") {
+      ctx.session.capture(frame.sessionId);             // ← step two: write back the returned id; the next turn continues from it
+      return;
+    }
+    if (frame.type === "approval_request") {            // ← this step: the app stops for approval; hold the scene
+      ctx.session.hold<Pending>({ cursor, stream, callId: frame.toolCallId });
+      return { pause: { id: frame.toolCallId, action: frame.toolName,
+                        options: [{ id: "approve" }, { id: "deny" }] } };
+      // driveFrameStream on pause: emits an input.requested, sets status to "waiting", stops reading the stream
+    }
+    if (frame.type === "server_error") return { fail: frame.message };
+  });
+}
+
+export default defineAgent({
+  name: "my-app",
+  async send(input: TurnInput, ctx: AgentContext): Promise<Turn> {
+    const held = ctx.session.take<Pending>();
+    if (held) {                                         // ← t.respond("approve"): an ordinary send; deliver the verdict first
+      const approved = input.responses?.[0]?.optionId === "approve";  // with multiple pending requests, match by requestId
+      if (!approved) held.stream.markRejected(held.callId);
+      await fetch(`${BASE_URL}/api/chat/approve`, { method: "POST", signal: ctx.signal,
+        body: JSON.stringify({ toolUseId: held.callId, approved }) });
+      return readStream(held.cursor, ctx, held.stream); //    then keep reading the same stream; no new request
+    }
+
+    const res = await fetch(`${BASE_URL}/api/chat`, {   // ← transport: the only part you truly hand-write
+      method: "POST",
+      body: JSON.stringify({
+        message: input.text,                            // ← step one: send the message
+        sessionId: ctx.session.id,                      // ← step two: absent on the first send, present automatically afterward
+      }),
+      signal: ctx.signal,
+    });
+    return readStream(sseJsonFrames<Frame>(res.body!), ctx, fromPiAgentEvents());  // ← step four: official converter, zero mapping
+  },
+});
+```
+
+For interfaces without HITL, delete the three held-scene parts (`Pending`, `hold`, and the opening `take` branch) and keep the rest unchanged. The complete mental model of pause / answer / resume is in [HITL](/concepts/hitl).
+
+**This step unlocks**: `t.parked()`, `t.requireInputRequest()`, `t.respond()` / `t.respondAll()`, and the precise assertion `calledTool(..., { status: "rejected" })`.
+
+## Step 6: hook up OTel traces
+
+If the app is already instrumented (standard OTel HTTP server instrumentation is enough), the integration comes in two halves — one is startup-time configuration, the other lives in `send`. Keeping them apart means separating "what never changes" from "what changes every turn".
+
+**The endpoint is startup-time configuration; it is not passed from send.** NiceEval's OTLP receiver address is the same on every run, so it does not go through `ctx`: pin the receiver port in `niceeval.config.ts`, point the app's OTel exporter at that fixed URL on startup, and never touch it again no matter how many eval runs follow:
+
+```ts
+// niceeval.config.ts
+import { defineConfig } from "niceeval";
+
+export default defineConfig({
+  telemetry: { port: 4318 },   // the receiver always listens on http://localhost:4318/v1/traces
+});
+```
+
+```bash
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318/v1/traces node server.js
+```
+
+**What send passes is this turn's trace context, not the endpoint.** `ctx.telemetry.headers` is a W3C `traceparent` header the runner generates fresh every turn — spread it into the request, and the spans your app produces this turn attach precisely to this turn's trace, with no misattribution when multiple evals run concurrently. Back on the storyline's chat-app, `send` gains exactly one line:
+
+```ts
+    const res = await fetch(`${BASE_URL}/v1/chat/completions`, {
+      // …rest same as step four, omitted
+      headers: { "content-type": "application/json", ...ctx.telemetry?.headers },  // ← the only added line
+    }).then((r) => r.json());
+```
+
+`ctx.telemetry` only exists when OTel integration is configured; spreading an `undefined` is safe when it is not, so the line can stay permanently. Spans still arrive without this header, but attribution degrades to time windows and that agent's turns fall back to serial execution — carrying it is what makes attribution accurate under concurrency.
+
+**This step unlocks**: the call waterfall in `niceeval view` — every model call and every tool execution inside the app, with duration and tokens, laid out as a per-turn timeline. Assertions are unchanged: the verdict evidence was already complete in the event mapping of the earlier steps; spans go only into the waterfall and never feed assertions. For how the receiver starts and how spans attribute to turns, see [OTel integration](/guides/connect-otel).
+
+## Step 7: pass through the experiment's flags (A/B comparison)
+
+Once the app exposes variants as switchable configuration, the experiment declares `flags`, and the runner hands them verbatim to `send` via `ctx.flags` every turn; the Adapter does not interpret their meaning, it only forwards them with the request — the app switches variants on the flag:
+
+```ts
+      // …rest of send same as before, omitted
+      body: JSON.stringify({
+        messages,
+        flags: ctx.flags,        // ← experiment.flags passed through as-is; {} when not configured
+      }),
+```
+
+```ts
+// experiments/concise.ts
+import { defineExperiment } from "niceeval";
+import chatApp from "../agents/chat-app.ts";
+
+export default defineExperiment({
+  agent: chatApp,
+  flags: { promptVariant: "concise" },   // ← reaches send via ctx.flags every turn
+});
+```
+
+Two experiment files, each declaring its own `flags`, running the same set of evals via `npx niceeval exp` — that is an A/B comparison. This is Tier 3 of the three integration tiers (it requires the app to cooperate by exposing switches); cost and payoff in [Tier](/concepts/tier); `flags` alongside `model`, `runs`, and the other experiment fields in [Write experiments](/guides/write-experiment).
+
+**This step unlocks**: score comparison across variants over the same set of evals.
+
+## Reference: the five t APIs as send sees them
+
+With all seven steps written, look back at the five driving APIs on the eval side — by the time they reach `send` they are just the same function receiving different fields; there is no second method to implement:
+
+| Eval side | Reaching the Adapter | What `send` receives |
+|---|---|---|
+| `t.send(text)` | One `send` | `input.text`; `ctx.session` is this session line's own state |
+| `t.sendFile(path, text?)` | One `send` | Same as above, plus `input.files` (base64 `InputFile[]`); put them into the request however the app's interface expects, or ignore them if it is not multimodal |
+| `t.newSession()` | Does not trigger a `send` by itself | Opens a second session line; that line's next first `send` gets **completely empty** state — exactly the same form as the eval's very first `send` |
+| `t.respond(...answers)` | One **ordinary** `send` | `input.text` = the answer text; `input.responses` has one `{ requestId, optionId }` per request (`{ requestId, text }` for free-text answers); still the same line, the same state |
+| `t.respondAll(optionId)` | One **ordinary** `send` | Same as above, with one answer per pending request; `optionId` is validated eval-side against every request — a typo throws instead of silently reaching the app |
+
+## Related reading
+
+- [Adapter](/concepts/adapter) — the contract itself: what `send` receives and returns, the three integration tiers, where capabilities come from.
+- [Connect your agent](/guides/connect-your-agent) — the integration picture: minimal integration, parameter channels, and the increment map.
+- [HITL](/concepts/hitl) — the complete concept of pausing for a human: handshake timing and both sides' obligations.
+- [Drive](/concepts/drive) — the eval-side view: how to use `t.send()`, `t.newSession()`, and HITL.
+- [Assert](/concepts/assert) — the complete assertion vocabulary driven by the standard event stream.

package/docs-site/images/adapter-tiers-zh.svg

@@ -0,0 +1,60 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1240 380" role="img" aria-labelledby="title desc">
+  <title id="title">接入分三个 Tier</title>
+  <desc id="desc">Tier 1 只接 send，应用零改动；Tier 2 加 OTel，事件免映射并带完整性证明；Tier 3 侵入改造，把变体暴露成 experiment flags 做 A/B。三档递进不互斥。</desc>
+  <defs>
+    <marker id="tierHead" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#8ab4f8" />
+    </marker>
+    <linearGradient id="card" x1="0" x2="1" y1="0" y2="1">
+      <stop offset="0" stop-color="#0d1117" />
+      <stop offset="1" stop-color="#111820" />
+    </linearGradient>
+    <style>
+      .tierTitle { fill: #e6edf3; font: 600 24px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+      .give { fill: #f0b429; font: 600 18px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+      .get { fill: #7ee787; font: 600 18px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+      .line { fill: #c9d1d9; font: 18px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+      .muted { fill: #8b949e; font: 18px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+      .box { fill: #161b22; stroke: #30363d; stroke-width: 1.5; }
+      .arrow { stroke: #8ab4f8; stroke-width: 3; fill: none; marker-end: url(#tierHead); }
+    </style>
+  </defs>
+
+  <rect x="1" y="1" width="1238" height="378" rx="24" fill="url(#card)" stroke="#30363d" stroke-width="2" />
+
+  <!-- Tier 1 -->
+  <rect x="40" y="44" width="352" height="252" rx="16" class="box" />
+  <text x="64" y="86" class="tierTitle">Tier 1 · 只接 send</text>
+  <text x="64" y="126" class="give">投入</text>
+  <text x="64" y="152" class="line">一个 Adapter 文件</text>
+  <text x="64" y="178" class="line">应用代码一行不改</text>
+  <text x="64" y="218" class="get">买到</text>
+  <text x="64" y="244" class="line">文本断言、judge、多轮、HITL</text>
+  <text x="64" y="270" class="line">手写映射的工具断言、模型对比</text>
+
+  <path d="M 402 170 H 434" class="arrow" />
+
+  <!-- Tier 2 -->
+  <rect x="444" y="44" width="352" height="252" rx="16" class="box" />
+  <text x="468" y="86" class="tierTitle">Tier 2 · send + OTel</text>
+  <text x="468" y="126" class="give">投入</text>
+  <text x="468" y="152" class="line">应用把 OTel span 发给 NiceEval</text>
+  <text x="468" y="178" class="line">已埋点的应用零改动</text>
+  <text x="468" y="218" class="get">买到</text>
+  <text x="468" y="244" class="line">niceeval view 的调用瀑布图</text>
+  <text x="468" y="270" class="line">每次模型 / 工具调用的耗时与 token</text>
+
+  <path d="M 806 170 H 838" class="arrow" />
+
+  <!-- Tier 3 -->
+  <rect x="848" y="44" width="352" height="252" rx="16" class="box" />
+  <text x="872" y="86" class="tierTitle">Tier 3 · 侵入改造</text>
+  <text x="872" y="126" class="give">投入</text>
+  <text x="872" y="152" class="line">应用内部把变体暴露成</text>
+  <text x="872" y="178" class="line">experiment 可传的 flags</text>
+  <text x="872" y="218" class="get">买到</text>
+  <text x="872" y="244" class="line">prompt / 工具集 / feature</text>
+  <text x="872" y="270" class="line">的 A/B 对比</text>
+
+  <text x="620" y="340" class="muted" text-anchor="middle">前两档无侵入：只对着前端本来就在用的接口收发。三档递进不互斥，已写的 eval 全程复用。</text>
+</svg>

package/docs-site/images/agent-turn-roundtrip-en.svg

@@ -0,0 +1,62 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1160 320" role="img" aria-labelledby="title desc">
+  <title id="title">One complete t.send round trip</title>
+  <desc id="desc">An eval calls t.send, the niceeval runner builds TurnInput and ctx, the adapter calls the application interface, and maps the raw response into a standard event-stream Turn.</desc>
+  <defs>
+    <marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#8ab4f8" />
+    </marker>
+    <linearGradient id="card" x1="0" x2="1" y1="0" y2="1">
+      <stop offset="0" stop-color="#0d1117" />
+      <stop offset="1" stop-color="#111820" />
+    </linearGradient>
+    <style>
+      .label { fill: #e6edf3; font: 600 24px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+      .small { fill: #c9d1d9; font: 22px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+      .muted { fill: #8b949e; font: 18px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+      .pill { fill: #1f2937; stroke: #374151; stroke-width: 1; }
+      .pillText { fill: #e5e7eb; font: 20px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+      .arrow { stroke: #8ab4f8; stroke-width: 3; fill: none; marker-end: url(#arrow); }
+      .returnArrow { stroke: #a78bfa; stroke-width: 3; fill: none; marker-end: url(#arrow); }
+      .lane { stroke: #30363d; stroke-width: 1; stroke-dasharray: 5 8; }
+    </style>
+  </defs>
+
+  <rect x="1" y="1" width="1158" height="318" rx="24" fill="url(#card)" stroke="#30363d" stroke-width="2" />
+
+  <text x="54" y="54" class="label">evals/*.eval.ts</text>
+  <text x="410" y="54" class="label">niceeval runner</text>
+  <text x="775" y="54" class="label">Adapter (you write)</text>
+  <text x="1060" y="42" class="muted">your app</text>
+  <text x="1060" y="66" class="muted">unchanged</text>
+
+  <line x1="342" y1="84" x2="342" y2="268" class="lane" />
+  <line x1="735" y1="84" x2="735" y2="268" class="lane" />
+  <line x1="965" y1="84" x2="965" y2="268" class="lane" />
+
+  <text x="54" y="126" class="small">await t.send("...")</text>
+  <path d="M 278 119 H 397" class="arrow" />
+  <text x="410" y="126" class="small">build TurnInput + ctx</text>
+  <path d="M 692 119 H 778" class="arrow" />
+  <text x="775" y="126" class="small">send(input, ctx)</text>
+  <path d="M 1008 119 H 1100" class="arrow" />
+
+  <text x="54" y="182" class="small">t.reply</text>
+  <path d="M 397 175 H 278" class="returnArrow" />
+  <text x="410" y="182" class="small">derive facts</text>
+  <path d="M 778 175 H 692" class="returnArrow" />
+  <text x="775" y="182" class="small">map to events</text>
+  <text x="1012" y="203" class="muted">response / SSE</text>
+  <path d="M 1100 175 H 1008" class="returnArrow" />
+
+  <text x="54" y="238" class="small">t.calledTool()...</text>
+  <text x="410" y="238" class="small">update reply and usage</text>
+  <text x="775" y="238" class="small">return Turn</text>
+
+  <rect x="49" y="278" width="118" height="29" rx="8" class="pill" />
+  <text x="63" y="299" class="pillText">t.send</text>
+  <rect x="175" y="278" width="136" height="29" rx="8" class="pill" />
+  <text x="189" y="299" class="pillText">t.sendFile</text>
+  <rect x="319" y="278" width="130" height="29" rx="8" class="pill" />
+  <text x="333" y="299" class="pillText">t.respond</text>
+  <text x="468" y="299" class="muted">share one runner event path; the adapter implements send once.</text>
+</svg>

package/docs-site/images/agent-turn-roundtrip-zh.svg

@@ -0,0 +1,77 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1240 340" role="img" aria-labelledby="title desc">
+  <title id="title">一次 t.send 的完整往返</title>
+  <desc id="desc">eval 调用 t.send，NiceEval 组装 TurnInput 和 ctx，adapter 调用应用接口，并把原始返回翻译成标准事件流 Turn。</desc>
+  <defs>
+    <marker id="rtHeadBlue" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#8ab4f8" />
+    </marker>
+    <marker id="rtHeadPurple" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="8" markerHeight="8" orient="auto-start-reverse">
+      <path d="M 0 0 L 10 5 L 0 10 z" fill="#a78bfa" />
+    </marker>
+    <linearGradient id="card" x1="0" x2="1" y1="0" y2="1">
+      <stop offset="0" stop-color="#0d1117" />
+      <stop offset="1" stop-color="#111820" />
+    </linearGradient>
+    <style>
+      .label { fill: #e6edf3; font: 600 24px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+      .small { fill: #c9d1d9; font: 22px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+      .muted { fill: #8b949e; font: 18px -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif; }
+      .pill { fill: #1f2937; stroke: #374151; stroke-width: 1; }
+      .pillText { fill: #e5e7eb; font: 20px ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
+      .box { fill: #161b22; stroke: #30363d; stroke-width: 1.5; }
+      .arrow { stroke: #8ab4f8; stroke-width: 3; fill: none; marker-end: url(#rtHeadBlue); }
+      .returnArrow { stroke: #a78bfa; stroke-width: 3; fill: none; marker-end: url(#rtHeadPurple); }
+      .lane { stroke: #30363d; stroke-width: 1; stroke-dasharray: 5 8; }
+    </style>
+  </defs>
+
+  <rect x="1" y="1" width="1238" height="338" rx="24" fill="url(#card)" stroke="#30363d" stroke-width="2" />
+
+  <text x="54" y="54" class="label">evals/*.eval.ts</text>
+  <text x="380" y="54" class="label">NiceEval</text>
+  <text x="730" y="54" class="label">Adapter（你写）</text>
+
+  <!-- 图例：去程 / 回程 -->
+  <path d="M 990 46 H 1020" class="arrow" />
+  <text x="1030" y="53" class="muted">去程</text>
+  <path d="M 1128 46 H 1098" class="returnArrow" />
+  <text x="1138" y="53" class="muted">回程</text>
+
+  <line x1="350" y1="84" x2="350" y2="268" class="lane" />
+  <line x1="700" y1="84" x2="700" y2="268" class="lane" />
+
+  <!-- 你的应用：箭头的真实落点 -->
+  <rect x="1030" y="90" width="170" height="178" rx="14" class="box" />
+  <text x="1115" y="130" class="label" text-anchor="middle">你的应用</text>
+  <text x="1115" y="164" class="muted" text-anchor="middle">前端在用的接口</text>
+  <text x="1115" y="192" class="muted" text-anchor="middle">响应 / SSE 流</text>
+  <text x="1115" y="238" class="muted" text-anchor="middle">一行不改</text>
+
+  <!-- 去程 -->
+  <text x="54" y="126" class="small">await t.send("...")</text>
+  <path d="M 312 119 H 372" class="arrow" />
+  <text x="380" y="126" class="small">组装 TurnInput + ctx</text>
+  <path d="M 650 119 H 722" class="arrow" />
+  <text x="730" y="126" class="small">send(input, ctx)</text>
+  <path d="M 950 119 H 1024" class="arrow" />
+
+  <!-- 回程 -->
+  <text x="54" y="182" class="small">t.reply</text>
+  <path d="M 372 175 H 312" class="returnArrow" />
+  <text x="380" y="182" class="small">折叠 events 成事实</text>
+  <path d="M 722 175 H 650" class="returnArrow" />
+  <text x="730" y="182" class="small">翻译成 events</text>
+  <path d="M 1024 175 H 950" class="returnArrow" />
+
+  <text x="54" y="238" class="small">t.calledTool()...</text>
+  <text x="380" y="238" class="small">更新 t.reply 与用量</text>
+  <text x="730" y="238" class="small">return Turn</text>
+
+  <rect x="49" y="284" width="118" height="29" rx="8" class="pill" />
+  <text x="63" y="305" class="pillText">t.send</text>
+  <rect x="175" y="284" width="136" height="29" rx="8" class="pill" />
+  <text x="189" y="305" class="pillText">t.sendFile</text>
+  <rect x="319" y="284" width="130" height="29" rx="8" class="pill" />
+  <text x="333" y="305" class="pillText">t.respond</text>
+  <text x="468" y="305" class="muted">都是运行器侧的统一事件入口；adapter 只实现 send。</text>
+</svg>