@cursor/july 0.1.108 → 0.1.109

package/dist/docs/llms-full.txt

@@ -4,349 +4,13 @@
 
 ---
 
-Source: /docs/ab.md
-
-# Live A/B metrics
-
-Use `defineAB` to compare variants on live agent sessions. New sessions
-receive a sticky assignment in each enrolled experiment. The Agent SDK folds
-their durable event streams into tool, token, failure, and wall-time
-metrics. You can send cumulative samples to your metrics backend and
-inspect aggregates in the playground.
-
-`defineAB` compares live variants through sticky assignment,
-instruction overlays, optional tool branches, and cumulative metrics.
-Metric callbacks observe the result without approving, rejecting, or
-failing a turn. Use [evals](/docs/evals.md) for pass/fail regression checks
-on fixed inputs.
-
-## Choose live A/B metrics or evals
-
-Both features read the session event stream, but they answer different
-questions.
-
-| | Live A/B metrics | Evals |
-| --- | --- | --- |
-| Question | How do variants compare on live sessions? | Does the agent still meet a fixed contract? |
-| Location | `agent/ab.ts` or `agent/ab/<name>.ts` | `evals/**/*.eval.ts` |
-| Input | Dev or production traffic | Frozen prompts and fixtures |
-| Output | Cumulative metrics by session and arm | Pass/fail assertions |
-| How it runs | Automatically on new live sessions | `agent-sdk eval` |
-
-There is no `agent-sdk ab` command or assertion API.
-
-## Define an experiment
-
-Author one experiment in `agent/ab.ts`, add more under
-`agent/ab/<name>.ts`, or use both forms. Each file defines one
-experiment. The experiment name comes from `name` when set. Otherwise,
-the Agent SDK uses `ab` for `agent/ab.ts` and the file stem for files under
-`agent/ab/`.
-
-```ts
-// agent/ab/concise-weather.ts
-import {
-  defineAB,
-  splitBySessionHash,
-} from "@cursor/july/ab";
-
-export default defineAB({
-  name: "concise-weather",
-  variants: {
-    control: {
-      label: "Baseline",
-    },
-    treatment: {
-      label: "Short replies",
-      description: "Adds a one-paragraph response limit.",
-      instructions: "Keep weather replies to one short paragraph.",
-    },
-  },
-  split: splitBySessionHash({
-    weights: { control: 1, treatment: 1 },
-    holdout: 0.1,
-  }),
-  derive: {
-    weatherCalls: (event) =>
-      event.type === "action.result" &&
-      event.data.toolName === "get_weather"
-        ? 1
-        : null,
-  },
-  onSample(sample) {
-    console.log(
-      sample.experiment,
-      sample.variant,
-      sample.metrics.toolCalls,
-      sample.metrics.wallTimeMs
-    );
-  },
-});
-```
-
-Every definition needs:
-
-- At least two variants. Variant keys cannot be empty or contain `/` or
-  `\`.
-- A `split` function that returns a variant key or `null`.
-- An `onSample` callback for completed or failed turns.
-
-`label` and `description` appear with the arm in result surfaces.
-`instructions` changes the prompt for sessions in that arm. `derive`
-adds custom counters.
-
-Duplicate experiment names are validation errors. Check discovery
-before you serve:
-
-```bash
-agent-sdk validate --dir .
-agent-sdk info --dir . --json
-```
-
-The `abs` field in `info` lists the discovered experiment names.
-
-## Assign sticky variants
-
-Enrollment happens once, when a live session is created and before its
-first turn:
-
-1. The Agent SDK records `session.started`.
-2. Each experiment runs its `split` function.
-3. The Agent SDK records one durable `ab.assigned` event per experiment.
-4. The selected arms become available on `session.abs`.
-5. Variant instruction overlays reach the first model turn.
-
-A split can return a variant key or `null`. A null assignment is a
-sticky skip for that experiment. It increments the experiment's
-`skipped` total, still appears in the snapshot's `sessions` list with
-`variant: null`, and does not collect arm metrics or call `onSample`.
-
-Use the split helper that matches your rollout:
-
-| Helper | Behavior |
-| --- | --- |
-| `splitBySessionHash({ weights?, holdout?, salt? })` | Hashes the session id into a reproducible arm; the recommended default |
-| `splitByRandom({ weights?, holdout? })` | Draws once when the session starts, then persists the result |
-| `splitAlways("control")` | Pins every new session to one arm |
-| `splitNone()` | Skips every new session without deleting the experiment |
-| `splitIf(predicate, inner)` | Runs `inner` only when the predicate passes |
-| Custom `split(ctx)` | Returns a declared variant key or `null` |
-
-The split context includes the agent name, channel id, session info,
-experiment name, and declared variant keys. For example, enroll only
-Slack sessions:
-
-```ts
-split: splitIf(
-  (ctx) => ctx.channel.id === "slack",
-  splitBySessionHash()
-),
-```
-
-Weights default to equal. Non-positive weights leave an arm out of the
-draw, and at least one arm must have a positive weight. `holdout` is the
-fraction of sessions assigned `null`, from `0` through `1`. Change
-`salt` to reshuffle future hash assignments without renaming the
-experiment.
-
-If a custom split throws or returns an unknown variant, the Agent SDK logs
-the error and records `variant: null`. The failed decision becomes a
-sticky skip instead of breaking the session.
-
-Enrollment only applies to new sessions. Adding an experiment does not
-assign existing conversations. Follow-ups keep the session's original
-arms. Keep experiment names and variant keys stable while you collect
-and compare results.
-
-## Change behavior by variant
-
-Variant instructions are appended to the agent's base instructions.
-Local sessions receive the merged instructions in `AGENTS.md` before
-every turn. Cloud sessions receive them in the first-turn preamble
-only. For cloud follow-ups, branch through `session.abs` when the arm
-must remain visible to deterministic behavior.
-
-Tools can branch on the assignment through `ctx.session.abs`. Hooks can
-read the same field for logging or export:
-
-```ts
-const treatment =
-  ctx.session.abs?.["concise-weather"] === "treatment";
-
-if (treatment) {
-  return conciseWeatherResult;
-}
-
-return baselineWeatherResult;
-```
-
-This makes the assignment available to deterministic code as well as
-the model prompt. Use both patterns together when one experiment must
-steer the prompt and host code at once.
-
-`defineAB` does not select a different model or runtime for each arm.
-Keep those settings in `agent/agent.ts`, or write explicit host logic
-when your experiment needs another behavior lever.
-
-The split and selected arm can affect agent behavior. `derive` and
-`onSample` only observe the resulting event stream. Errors in either
-callback are logged and never fail the turn.
-
-## Collect built-in and custom metrics
-
-Metrics accumulate for each session and experiment. When one session
-joins several experiments, every enrolled experiment folds the same
-turn and tool events into its own counters.
-
-| Metric | How the Agent SDK calculates it |
-| --- | --- |
-| `turns` | Adds one on `turn.completed` or `turn.failed` |
-| `turnFailures` | Adds one on `turn.failed` |
-| `toolCalls` | Adds one for each `action.result` |
-| `toolErrors` | Adds one when `action.result.data.isError` is true |
-| `inputTokens`, `outputTokens` | Adds usage from completed turns |
-| `cacheReadTokens`, `cacheWriteTokens` | Adds cache usage from completed turns |
-| `costUsd` | Sums the estimated turn cost recorded on `turn.completed` (turns whose model has no known rates contribute 0) |
-| `wallTimeMs` | Sums the time from `turn.started` to its completed or failed event |
-| `custom` | Sums finite numeric deltas returned by `derive` |
-
-`onSample` fires after every `turn.completed` and `turn.failed` event
-for an enrolled arm. The sample contains:
-
-| Field | Value |
-| --- | --- |
-| `experiment` | Experiment name |
-| `variant`, `variantLabel?` | Sticky arm and optional display label |
-| `sessionId`, `channelId` | Source session |
-| `metrics` | Cumulative metrics through this turn |
-| `reason` | `turn.completed` or `turn.failed` |
-| `at` | Terminal event timestamp |
-
-The metrics are cumulative, not per-turn deltas. A second sample from
-the same session includes the first turn's counts.
-
-Each `derive` extractor runs on every session event for its enrolled
-experiment, including streamed `message.appended` events. Keep it
-synchronous and cheap. Return a finite number to add a delta, or
-`null` to skip the event. Send samples to your metrics service from
-`onSample`; do not perform network or disk work in `derive`.
-
-Skipped sessions never call `onSample`. Errors from `derive` or
-`onSample` are logged, then metric collection continues.
-
-## Inspect assignments and results
-
-Open the playground's **A/Bs** tab to see aggregate arm totals and
-per-session assignments. The tab reads `GET /v1/abs`.
-
-The response has two views of the same durable data:
-
-| Field | Contents |
-| --- | --- |
-| `experiments` | Declared variants, skipped-session count, arm session counts, and aggregate metrics |
-| `sessions` | Visible sessions with their assignments and cumulative metrics |
-
-`GET /v1/abs` returns sessions visible to the current principal by
-default. In `--dev`, loopback requests include every session. Add
-`--allow-anonymous` to include every session from non-loopback callers
-too.
-
-The session event stream is the source of truth for assignment + fold.
-`GET /v1/abs` recomputes aggregates from those logs. Any
-`agent/storage.ts` exports samples and snapshots durably: an authored
-`abs` table when the backend has a native shape for it, or the table
-derived over the KV core otherwise. See
-[Storage](/docs/storage.md#eval-and-a-b-tables).
-
-## Configure the playground fold window
-
-Assignments and foldable metrics already persist in each session's
-event stream. The optional `agent/ab.config.ts` only caps how many
-sessions the playground and `GET /v1/abs` fold:
-
-```ts
-import { defineABConfig } from "@cursor/july/ab";
-
-export default defineABConfig({
-  // Optional. Defaults to 200. Only affects GET /v1/abs / A/Bs tab.
-  maxPlaygroundSessions: 500,
-});
-```
-
-`maxPlaygroundSessions` keeps the newest sessions in the fold. It does
-not prune session logs or change assignment. For export to S3, a DB, or
-your metrics vendor, send samples from `onSample` or declare a storage
-`abs` table.
-
-## Keep assignments durable
-
-The append-only event stream is the source of truth. Each
-`ab.assigned` event persists a variant key or null skip. Built-in
-metrics come from the turn and tool events that follow it.
-
-After a server restart or a parked session resumes, the live collector
-replays the stream to rebuild cumulative counters. Replay does not call
-`onSample` (or write to the storage `abs` table) for historical turns.
-Only a new completed or failed turn emits another sample.
-
-The snapshot API also replays `derive` across the full stream, so
-custom totals match the current extractor. Changing a derive function
-can change historical snapshot totals. Treat metric definitions as
-versioned experiment code.
-
-## Keep eval traffic separate
-
-Sessions created by `agent-sdk eval` and the playground Evals runner use
-`purpose: "eval"`. They skip A/B enrollment entirely:
-
-- No split function runs.
-- No `ab.assigned` event is recorded.
-- No `onSample` callback fires.
-- The session is omitted from `GET /v1/abs`.
-
-Ordinary chat, `agent-sdk run`, Slack, GitHub, and other channel sessions
-use the live purpose. You do not need `splitIf` to exclude eval traffic.
-
-## Know the boundaries
-
-`defineAB` provides sticky assignment, variant instructions,
-`session.abs` for tools, cumulative metrics, and local inspection. It
-does not provide:
-
-- A test command, assertion API, or pass/fail result
-- Statistical significance calculations
-- An experiment rollout or lifecycle service
-- Per-variant model or runtime configuration
-- A built-in analytics warehouse (bring your own via `onSample` or the
-  storage `abs` table)
-
-Use [evals](/docs/evals.md) to protect known behavior. Use `onSample` or a
-storage `abs` table when you need sample/snapshot exports beyond the
-session event log.
-
-## What's next
-
-Continue with these pages:
-
-- [Evals](/docs/evals.md): pass/fail regression checks on fixed inputs
-- [Hillclimbing](/docs/hillclimbing.md): improve an agent against fixed
-  fixtures
-- [Hooks](/docs/reference/hooks.md): other event-stream consumers
-- [Sessions and streaming](/docs/reference/sessions.md): the
-  `ab.assigned` event and durable log
-- [Playground](/docs/reference/playground.md): the A/Bs tab
-- [HTTP API](/docs/reference/http-api.md): `GET /v1/abs`
-- [Live A/B metrics skill](/docs/skills/ab.md): have a coding agent
-  wire an experiment
-
----
-
 Source: /docs/building-with-agents.md
 
 # Building agents with agents
 
 Give a coding agent the goal. The built-in skills guide it through
-scaffolding, channels, verification, evals, and measured improvement.
+scaffolding, channels, verification, deployment, evals, and measured
+improvement.
 
 ## What can a coding agent build for me?
 
@@ -393,12 +57,12 @@ The package ships task-specific guides under [`skills/`](/docs/skills/index.md):
 | Understand the project layout and runtimes | [`framework-map`](/docs/skills/framework-map.md) |
 | Create and verify a new agent | [`create-agent`](/docs/skills/create-agent.md) |
 | Write fixtures and regression checks | [`evals`](/docs/skills/evals.md) |
-| Live A/B metrics on traffic (`defineAB`) | [`ab`](/docs/skills/ab.md) |
 | Export OpenTelemetry traces | [`otel`](/docs/skills/otel.md) |
 | Improve an agent against fixed inputs | [`hillclimb`](/docs/skills/hillclimb.md) |
 | Add GitHub webhooks and replay events | [`github`](/docs/skills/github.md) |
 | Connect an agent to Slack | [`setup-slack`](/docs/skills/setup-slack.md) |
 | Authorize host MCP OAuth | [`mcp-auth`](/docs/skills/mcp-auth.md) |
+| Deploy with an attached service account | [`deploy`](/docs/skills/deploy.md) |
 | Diagnose a local run | [`debug`](/docs/skills/debug.md) |
 
 Point your coding agent at the matching `SKILL.md`. The guide contains
@@ -505,7 +169,6 @@ name. For example, `agent/tools/get_weather.ts` creates a tool named
 | `agent/mcp-connections/<name>.ts` | Tools from external MCP servers |
 | `agent/host-connections/<name>.ts` | Privileged MCP servers for host tools only |
 | `agent/channels/*.ts` | HTTP, Slack, and GitHub entry points |
-| `agent/ab.ts` or `agent/ab/*.ts` | Sticky variants and live performance metrics |
 | `agent/result.ts` | Optional host `commit` on the final assistant text |
 | `evals/**/*.eval.ts` | Repeatable checks at the project root |
 
@@ -541,8 +204,8 @@ Each session records an append-only event stream. It includes:
 - Turn completion and token usage
 
 Sessions and their event streams survive server restarts. The
-playground renders the stream. Evals assert against it. The
-`agent-sdk trajectory` command turns a saved stream into a short
+playground renders the stream. [Evals](/docs/evals.md) assert against it.
+The `agent-sdk trajectory` command turns a saved stream into a short
 summary.
 
 When a run surprises you, inspect its event stream first. See
@@ -636,7 +299,6 @@ See [Agent-to-agent](/docs/guides/agent-to-agent.md) for a complete example.
 - [Project layout](/docs/reference/project-layout.md)
 - [Sessions and streaming](/docs/reference/sessions.md)
 - [Channels](/docs/reference/channels.md)
-- [Live A/B metrics](/docs/ab.md)
 
 ---
 
@@ -3240,31 +2902,64 @@ Source: /docs/evals.md
 
 # Evals
 
-An eval is a repeatable check that runs your agent against a fixed input
-and gates the recorded trajectory: the run completed, the right tool
-ran, the reply has the right shape. Evals are how you know a prompt
-tweak helped, a refactor didn't regress the agent, and last month's fix
-is still holding.
+An eval sends a fixed message to your agent and asserts over the
+trajectory it records: the turn completed, the right tool ran with the
+right input, the reply has the right shape. Evals are how you know a
+prompt tweak helped, a refactor didn't regress the agent, and last
+month's fix still holds.
 
-Evals exercise the same surface your users hit. The runner starts (or
-targets) a real agent server, drives sessions over the public API, and
-grades what comes back. A passing eval means the agent started,
-accepted a message, and did what you asserted.
+Nothing is mocked. The runner starts (or targets) a real agent server,
+drives sessions over the public API, and grades the events it gets
+back. The model runs and server tools execute, so
+[keep side effects out of eval sessions](#keep-side-effects-out-of-eval-sessions)
+before you point an eval at an agent that posts anywhere.
 
-## Define evals with `defineEval`
+## Evals, hooks, or hillclimbing?
 
-The Agent SDK discovers evals under the project-root `evals/` directory,
-in `.eval.ts` or `.eval.js` files. That's a sibling of `agent/`, never
-inside it (`agent/evals/` is silently ignored). TypeScript is the normal
-authoring format.
+All three read the same session event stream. Pick by the question you
+are asking.
 
-The file path is the eval's identity, so you don't author an id.
-Directories group related evals: `evals/builds/api.eval.ts` becomes id
-`builds/api`. An `index` filename collapses to its directory, so
-`evals/builds/index.eval.ts` becomes `builds`.
+| You want to | Use |
+| --- | --- |
+| Gate one fixed input's behavior, locally and in CI | Evals (this page) |
+| Observe every live session: metrics, audit, alerts | [Hooks](/docs/reference/hooks.md) |
+| Improve an agent one measured round at a time | [Hillclimbing](/docs/hillclimbing.md); each kept win lands an eval |
+
+[Hooks, channel events, or evals?](/docs/reference/hooks.md#hooks-channel-events-or-evals)
+has the side-by-side table.
+
+### When not to write an eval
+
+- Test a server tool's own logic with
+  `agent-sdk call <tool> --dir . --input '{...}'` or a unit test. No
+  model turn, no credential.
+- Explore a prompt with `agent-sdk run --dir . --message "..."` and
+  read the trajectory. Write the eval once you know which decision to
+  gate.
+- Stop a bad turn while it runs with
+  [`needsApproval`](/docs/reference/tools.md#gate-a-tool-on-human-approval)
+  on the tool or [`defineResult`](/docs/reference/result.md). Evals grade
+  after the fact.
+
+## Write your first eval
 
-An eval is a single `async test(t)`. You drive the agent with `t` and
-assert on the run with the same `t`:
+Evals live under the project-root `evals/` directory, a sibling of
+`agent/`. `agent/evals/` is silently ignored. Discovery loads every
+`.eval.ts` (or `.eval.js`) file under `evals/`, plus one config file.
+
+```text
+my-agent/
+  agent/
+    agent.ts
+    tools/inspect_pr.ts
+  evals/
+    evals.config.ts      # required to run: maxConcurrency
+    readiness.eval.ts    # id: readiness
+    prs.eval.ts          # cases: prs/checkout, prs/search
+```
+
+An eval is a single `async test(t)`. You drive the agent with `t.send`
+and assert on the recorded run with the same `t`:
 
 ```ts
 // evals/readiness.eval.ts
@@ -3286,12 +2981,49 @@ export default defineEval({
 });
 ```
 
-One file can also hold several datapoints through `cases` (provide
-either `test` or `cases`, not both). Each case id becomes
-`<fileId>/<case.id>`:
+```ts
+// evals/evals.config.ts
+import { defineEvalConfig } from "@cursor/july/evals";
+
+export default defineEvalConfig({ maxConcurrency: 20 });
+```
+
+Run it under Node 22.13 or newer (never Bun) with a Cursor credential
+in place; see [Credentials](#credentials):
+
+```bash
+agent-sdk eval --dir . --list
+agent-sdk eval --dir . readiness
+```
+
+```text
+PASS  readiness (14.2s) — Inspects a PR without approving it.
+  ✓ succeeded
+  ✓ calledTool(inspect_pr)
+  ✓ notCalledTool(approve_pr)
+  ✓ check(includes)
+
+1 passed, 0 failed, 1 total
+artifacts: <project state directory>/evals/2026-09-11T15-02-11-402Z
+```
+
+Every local run writes each case's assertions, inputs, tool calls, and
+`t.log` lines under that artifacts directory. Open
+`evals/<case-id>.json` there when a case fails; see
+[Where results land](#where-results-land).
+
+## Name cases by path
+
+The file path is the eval's identity, so you don't author an id.
+`evals/builds/api.eval.ts` becomes `builds/api`. An `index` filename
+collapses to its directory: `evals/builds/index.eval.ts` becomes
+`builds`.
+
+One file can hold several datapoints through `cases`. Provide either
+`test` or `cases`, not both. Each case id becomes `<fileId>/<case.id>`:
 
 ```ts
-// evals/prs.eval.ts → prs/checkout, prs/search
+// evals/prs.eval.ts: prs/checkout, prs/search
 export default defineEval({
   tags: ["smoke", "prs"],
   cases: [
@@ -3320,93 +3052,75 @@ export default defineEval({
 });
 ```
 
-Case ids must be single path segments, unique within the file.
-Each case can set its own `description`, `tags`, `timeoutMs`, and
-`iterations`. A case-level value replaces the file-level value for that
-datapoint.
+Case ids are single path segments, unique within the file. A case can
+set its own `description`, `tags`, `timeoutMs`, `iterations`, `judge`,
+`reporters`, and `metadata`. A case-level value replaces the file-level
+one for that datapoint, except `metadata`, which merges with case keys
+winning, and `reporters`, which adds to the file's list. `metadata` is
+free-form data carried onto the result and every reporter.
+
+A file may instead export an array of `defineEval` calls to fan out
+over a dataset. Ids are then the file id plus a zero-padded index
+(`sql/0000`, `sql/0001`, ...); see [Load a dataset](#load-a-dataset).
+Prefer `cases` when datapoints are hand-written and deserve stable
+names.
 
 ### Iterations
 
-`iterations` (file or case, default `1`) runs a datapoint repeatedly.
-Discovery expands `iterations: 3` on case `nyc` to runnable ids
-`weather/nyc/1`, `weather/nyc/2`, `weather/nyc/3` (filter prefix
-`weather/nyc` still selects all three). Each expanded case exposes
-`t.iteration` / `t.iterations` on the test context. Cap is 100.
+`iterations` (file or case, default `1`, cap `100`) runs a datapoint
+repeatedly. Discovery expands `iterations: 3` on case `nyc` to runnable
+ids `weather/nyc/1`, `weather/nyc/2`, and `weather/nyc/3`. The filter
+`weather/nyc` still selects all three. Each expanded case exposes
+`t.iteration` and `t.iterations`.
 
-`maxConcurrency` counts **authored datapoints**, not expanded
-iterations: siblings `…/1`…`…/n` share one concurrency slot and run
-sequentially. A suite with 11 cases × 3 iterations and
-`maxConcurrency: 20` therefore has at most 11 cases in flight, not 33.
+`maxConcurrency` counts authored datapoints, not expanded iterations.
+Iterations of one datapoint share a concurrency slot and run in
+sequence, so a suite of 11 cases with 3 iterations each and
+`maxConcurrency: 20` has at most 11 cases in flight.
 
-## Configure eval runs
+## Drive the agent with `t.send`
 
-Each project with evals needs `evals/evals.config.ts` or
-`evals/evals.config.js`, and it must set `maxConcurrency`. Each case
-issues real model-provider requests, so concurrency is capped hard at
-200. Existing projects use 20. Discovery with `eval --list` works
-without this file, but running a case does not.
+`t.send(message, options?)` runs one turn and waits for it to settle:
+complete, park on an approval request, or fail. Several sends in one
+case share the session, which is how you write multi-turn evals.
 
-```ts
-import { defineEvalConfig } from "@cursor/july/evals";
+Each send resolves to a turn result: `message` (the assistant text),
+`sessionId`, `events`, `toolCalls` (tool names in order), `ok`, and
+`index`. The turn carries the same assertion vocabulary as `t`, scoped
+to that turn, so you can grade an intermediate turn before the next
+send overwrites `t.reply`. `turn.expectOk()` throws when the turn
+failed, for later steps that depend on it.
 
-export default defineEvalConfig({
-  maxConcurrency: 20, // required
-  // timeoutMs: 180_000, // optional project-wide default
-  // judge: { model: "..." }, // default judge model for t.judge.*
-  // reporters: [], // destinations that observe every case
-  // maxPlaygroundRuns: 50, // playground history only (default 20)
+Read the whole case with `t.reply` (last assistant text), `t.events`
+(every event so far), `t.turns` (settled turns, oldest first), and
+`t.sessionId`. `t.signal` aborts when the case hits its timeout; pass
+it to your own async work.
+
+Three options apply on the first send only, because they shape session
+creation:
+
+| Option | Effect |
+| --- | --- |
+| `workspaceFiles` | `{ path: contents }` seeded into the local session workspace. Prefer this over machine-local paths |
+| `workspaceDir` | Absolute harness cwd for the local runtime |
+| `cloud` | Per-session cloud options merged over the agent's static `cloud` config. Pin a fixture repo here for cloud evals instead of on the agent's default `cloud.repos`. On the cloud runtime, seeded files reach the agent as described under [Where does a turn run?](/docs/concepts.md#where-does-a-turn-run) |
+
+```ts
+await t.send("Review pr/diff.patch and post findings.", {
+  workspaceFiles: {
+    "pr/diff.patch": [
+      "diff --git a/app/routes/search.ts b/app/routes/search.ts",
+      "+res.send(`<h1>Results for ${req.query.q}</h1>`);",
+    ].join("\n"),
+  },
 });
 ```
 
-The timeout order is case or file `timeoutMs`, CLI `--timeout-ms`,
-project config `timeoutMs`, then the 180-second runner default.
-
-The optional fields:
+## Assert over the trajectory
 
-| Option | Default | Meaning |
-| --- | --- | --- |
-| `timeoutMs` | `180_000` | Project-wide per-case timeout |
-| `judge` | unset | Default judge model for `t.judge.*`; see [Judge free-form output](#judge-free-form-output) |
-| `reporters` | unset | Destinations that observe every case; `--skip-report` suppresses them |
-| `maxPlaygroundRuns` | `20` | Max batches in the playground / `/v1/dev/evals*` history (not CLI `eval`). Hard-capped at 500. |
-
-Reporters come from `@cursor/july/evals/reporters`: `JUnit` writes a
-JUnit XML file for CI, `Artifacts` writes per-case files, and
-`combineReporters` merges several into one (`renderJUnitXml` renders
-the XML for a custom destination). A file or case can add its own
-`reporters` on top of the config list.
-
-Playground batches survive restarts whenever `agent/storage.ts` exists
-with an `evals` table or a KV core providing `delete` and `list` (the
-table is derived over the core); see
-[Storage](/docs/storage.md#eval-and-a-b-tables). Without storage they live
-in process memory and disappear when `serve` exits. Navigating away
-and back still works while the process is up.
-
-## Drive and assert with `t`
-
-`t` is both the driver and the assertion surface. You write ordinary
-control flow, sending turns and asserting inline.
-
-Drive the agent with `t.send(message, options?)`. It runs one turn and
-waits for the session to park or fail. Multiple sends in one case share
-the session, which is how you write multi-turn evals.
-
-Each `t.send` resolves to a turn result with `message`,
-`sessionId`, `events`, `toolCalls`, `ok`, and `index`. The turn carries
-the same assertion vocabulary as `t`, scoped to that turn, so you can
-grade an intermediate turn before the next send overwrites `t.reply`.
-`turn.expectOk()` throws when the turn failed, for later
-steps that depend on it.
-
-Read the full case state with `t.reply` (the last assistant text),
-`t.events` (session events captured so far), `t.turns` (settled
-turns, oldest first), and `t.sessionId`. `t.signal` aborts when the
-case hits its timeout; pass it to your own async work. A thrown
-[turn result](/docs/reference/result.md) `commit` fails the turn, so
-`t.succeeded()` fails too.
-
-Assert with the gates:
+Assertions record; they never throw. One run reports every failure
+instead of dying on the first. Assertions on `t` read the whole run.
+Assertions on a turn read only that turn.
 
 | Gate | Checks |
 | --- | --- |
@@ -3415,158 +3129,483 @@ Assert with the gates:
 | `t.messageIncludes(token)` | the joined assistant text matches a string or `RegExp` |
 | `t.calledTool(name, matcher?)` | a matching call to `name` happened |
 | `t.notCalledTool(name)` | no request for `name`, in any lifecycle state |
-| `t.loadedSkill(name)` | the agent opened the skill's `SKILL.md` (read, grep, or shell `cat`) |
-| `t.toolOrder(names)` | tool requests appear in this relative order (extra calls allowed) |
+| `t.loadedSkill(name)` | the agent opened `skills/<name>/SKILL.md` (read, grep, or shell `cat`) |
+| `t.toolOrder(names)` | tool requests appear in this relative order; extra calls allowed |
 | `t.usedNoTools()` | no tool calls at all |
 | `t.maxToolCalls(max)` | at most `max` tool calls |
 | `t.noFailedActions()` | no tool call reported an error |
 | `t.calledSubagent(name, matcher?)` | a matching subagent delegation happened |
 | `t.taggedArtifact(kind?, predicate?)` | at least one [artifact](/docs/reference/artifacts.md) was tagged |
-| `t.event(type, matcher?)` | at least one matching event of `type` occurred |
-| `t.notEvent(type, matcher?)` | no matching event of `type` occurred |
+| `t.event(type, matcher?)` | at least one matching [event](/docs/reference/sessions.md#which-events-can-i-stream) of `type` |
+| `t.notEvent(type, matcher?)` | no matching event of `type` |
 | `t.eventOrder(matchers)` | matching event groups occur in this relative order |
 | `t.eventsSatisfy(label, predicate)` | your predicate over the typed event stream |
-| `t.check(value, expectation)` | any value, against a builder |
-| `t.score(name, value)` | records a 0–1 score you computed; soft until you add a bar |
-| `t.requireToolCall(name, matcher?)` | gates on a matching call and returns it, so later code can read its input and output |
-| `t.requireInputRequest(filter?)` | gates on exactly one pending approval request and returns it |
+| `t.check(value, expectation)` | any value, against a [builder](#grade-values-with-expectation-builders) |
+| `t.score(name, value)` | a 0-1 score you computed; soft until you add a bar |
+
+Three more assertions gate and return the matched fact. They stop the
+test body when nothing matches, without a duplicate execution error.
+`t.requireToolCall(name, matcher?)` returns the call so later code can
+read its `input` and `output`. `t.requireInputRequest(filter?)` returns
+the single pending approval request. `await t.require(value, expectation)`
+does the same for a value check.
+
+A case with no assertions passes when at least one turn completed. Add
+`t.succeeded()` and behavior gates anyway. They make the contract
+visible in review.
 
-Every gate returns a handle: `.soft()` demotes it to tracked-only,
-`.atLeast(0.7)` adds a soft score bar, and `.gate(0.8)` promotes a
-scored assertion into a hard gate.
+### What good cases assert
+
+Gate decisions and shape, not prose. Model wording varies run to run.
+Tool choice, tool avoidance, and output structure are the stable
+contract.
+
+1. `t.succeeded()`: always, first.
+2. The tool decision: `calledTool` for the intended path,
+   `notCalledTool` for the likely wrong alternative. The pair is
+   stronger than either alone.
+3. Output shape: a regex for the contract (`/ready|blocked/i`, a JSON
+   marker, a findings-block fence), never exact sentences.
+4. For structured output, parse `t.reply` and check fields with
+   `matches` or `satisfies` instead of substring-matching JSON.
+
+The common failure modes: asserting exact phrasing, packing more than
+about five gates into one case (split it), and cases that depend on
+live external state that drifts (pin the input).
+
+### Narrow tool assertions with matchers
 
 With no matcher, `calledTool` is request-based: a requested call counts
-even when its result has not arrived. Pass
-`t.calledTool("inspect_pr", { status: "completed" })` to require the
-call to return. `input`, `output`, and `count` matcher fields accept a
-literal, a `RegExp`, or a predicate.
-
-The expectation builders are `includes(string | RegExp)`,
-`equals(value)`, `matches(schema)`, `similarity(expected)`, and
-`satisfies(predicate, label)`. `includes` stringifies its input,
-`equals` compares values deeply, `matches` validates against a Standard
-Schema (or anything with `safeParse`, like Zod), `similarity` scores
-normalized text similarity, and `satisfies` runs your predicate. The
-plain function `normalizedSimilarity(actual, expected)` returns the
-same 0–1 score for use with `t.score`.
-
-A few more context members shape a case: `t.require(value, expectation)`
-records a gate and stops the test body when it fails, without a
-duplicate execution error. `t.skip(reason)` ends the case as skipped
-(reported separately, never changes the exit code; call it before
-sending messages). `t.metric(name, value)` records a structured score
-for the playground case card. `t.log(message)` records a debug line for
-the CLI and playground result.
-
-Three `t.send` options apply on session create (first `t.send` only):
-
-- `workspaceFiles`: `{ path: contents }`, seeded into the local session
-  workspace. Prefer this over machine-local paths.
-- `workspaceDir`: absolute harness cwd (local runtime).
-- `cloud`: per-session cloud options merged over the agent's static
-  `cloud` config (repos / env / …). Use a pinned `repos` override to
-  attach a fixture repo for cloud evals without putting it on the
-  agent's default `cloud.repos`. Cloud ignores `workspaceFiles` seeds.
-
-```ts
-const toolResults = t.events.filter((e) => e.type === "action.result");
+even before its result arrives. A matcher narrows it:
+
+```ts
+t.calledTool("inspect_pr", { status: "completed" });
+t.calledTool("apply_agents", { input: { verdict: "update" } });
+t.calledTool("bash", { input: { command: /^gh pr view/ }, count: 1 });
+t.calledTool("read_file", {
+  output: (value) => String(value).includes("TODO"),
+});
+```
+
+`input`, `output`, and `count` accept a literal, a `RegExp`, or a
+predicate. Object literals partial-deep-match, so `{ verdict: "update" }`
+matches arguments that also carry other keys. `status` is one of
+`completed`, `failed`, `pending`, or `rejected` (a human denied the
+approval). `calledSubagent` takes `{ output, status, count, callId }`.
+`event`, `notEvent`, and `eventOrder` take `{ data, count }`.
+
+### Grade values with expectation builders
+
+`t.check(value, expectation)` grades any value: `t.reply`, a parsed
+JSON field, a tool's output.
+
+| Builder | Checks | Severity |
+| --- | --- | --- |
+| `includes(string \| RegExp)` | substring or match; structured values are stringified first | gate |
+| `equals(value)` | deep equality | gate |
+| `matches(schema)` | a Standard Schema (Zod, Valibot, ...) or anything with `safeParse` | gate |
+| `similarity(expected)` | normalized text similarity, 0-1 | soft |
+| `satisfies(predicate, label)` | your predicate; `label` is the failure detail | gate |
+
+```ts
+import { matches, satisfies } from "@cursor/july/evals";
+import { z } from "zod";
+
+const verdict = JSON.parse(t.reply ?? "{}");
+t.check(
+  verdict,
+  matches(z.object({ ready: z.boolean(), blockers: z.array(z.string()) }))
+);
 t.check(
-  toolResults.length,
-  satisfies((n) => (n as number) <= 4, "at most 4 tool calls")
+  verdict.blockers.length,
+  satisfies((n) => (n as number) <= 3, "at most 3 blockers")
 );
 ```
 
-A case with no explicit gates falls back to whether at least one turn
-completed successfully. Add `t.succeeded()` and behavior-specific gates
-anyway. They make the contract visible during review.
+`normalizedSimilarity(actual, expected)` returns the same 0-1 score as
+`similarity`, for use with `t.score`.
+
+### Record without gating
+
+- `t.metric(name, value)` records a structured score or label. It shows
+  on the CLI result, the playground case card, JUnit output, and
+  artifacts.
+- `t.log(message)` records a debug line, streamed under `--verbose`.
+- `t.skip(reason)` ends the case as skipped. Skipped cases report
+  separately and never change the exit code. Call it before sending
+  messages.
+
+## Gates, soft scores, and verdicts
+
+Every assertion returns a handle, so severity rides on the assertion
+instead of a separate thresholds map:
+
+```ts
+t.succeeded(); // gate (default)
+t.calledTool("get_weather").soft(); // tracked, never fails
+t.check(t.reply, similarity("Sunny, 72°F")).atLeast(0.8); // soft, with a bar
+t.judge.closedQA("cites a source").gate(0.8); // promoted to a gate
+```
+
+- `.gate(threshold?)` is hard. A miss fails the case and `eval` exits 1.
+- `.soft(threshold?)` is tracked. With no threshold it never fails.
+- `.atLeast(threshold)` is soft with a bar. A miss marks the case
+  `scored`.
 
-### Judge free-form output
+Each case ends with one verdict:
 
-When wording matters and no regex captures it, `t.judge` grades the
-reply with an LLM. The built-in graders are `factuality(expected)`,
-`summarizes(expected)`, `closedQA(criteria)`, and `sql(expected)`. Each
-scores `t.reply` by default; pass `{ on }` to grade another value.
+| Verdict | Meaning | Exit code |
+| --- | --- | --- |
+| `passed` | every gate passed and no soft bar was missed | 0 |
+| `failed` | a gate failed, or the test body threw | 1 |
+| `scored` | only soft bars were missed | 0, or 1 under `--strict` |
+| `skipped` | `t.skip(reason)`, or a judge with no credentials | 0 |
+
+The CLI prints soft misses as `~` and gate misses as `✗`. Start a new
+benchmark with `t.score("recall", recall)` and `.atLeast()` so its
+number reports for a while without blocking merges. Add `--strict`
+once the bars are trustworthy.
+
+## Judge free-form output
+
+When wording matters and no regex captures it, `t.judge` grades with an
+LLM. The graders are `factuality(expected)`, `summarizes(expected)`,
+`closedQA(criteria)`, and `sql(expected)`. Each scores `t.reply` by
+default; pass `{ on }` to grade another value.
 
 ```ts
-t.judge.factuality("It is 54°F in NYC right now.").atLeast(0.7);
+const summary = await t.send("Why did CI fail on PR 42?");
+t.judge.factuality("The lint step failed on src/sidebar.ts.").atLeast(0.7);
+t.judge.closedQA("names the failing step", { on: summary.message }).gate(1);
 ```
 
 Judge assertions are soft by default, so a judge never fails a build
-until you give it a bar with `.atLeast(0.7)` or promote it with
-`.gate(0.8)`. The judge model comes from `defineEvalConfig({ judge })`,
+until you give it a bar with `.atLeast()` or promote it with `.gate()`.
+The recorded detail names the choice the judge made and its rationale.
+
+The judge model comes from `defineEvalConfig({ judge })`,
 `defineEval({ judge })`, a case-level `judge`, or a per-call
-`{ model }` override; the nearest one wins. For a domain-specific judge
-whose verdict is not a single score, `t.judge.model(prompt)` sends a
-raw prompt to the same model and returns the reply. You then record the
-parsed result with `t.score` or `t.check`.
+`{ model }`. The nearest one wins. A judge call with no model
+configured fails the case. A judge that cannot reach a model (no
+credential) ends the case as `skipped`, unless a deterministic gate
+already failed.
+
+For a domain-specific judge whose verdict is not a single score,
+`t.judge.model(prompt)` sends a raw prompt to the same model and
+returns the reply. Record the parsed result with `t.score` or
+`t.check`. Anything derived from the agent under test is untrusted
+input to your prompt: wrap it with `fenceUntrusted` and include
+`EVAL_JUDGE_INJECTION_GUARD`, as the built-in graders do.
+
+```ts
+import {
+  EVAL_JUDGE_INJECTION_GUARD,
+  fenceUntrusted,
+} from "@cursor/july/evals";
+
+const gold = ["XSS in search.ts", "open redirect in login.ts"];
+const reply = await t.judge.model(
+  [
+    "For each GOLD finding, answer whether SUBMISSION reports it.",
+    "Reply with one line per finding: <index> YES|NO.",
+    EVAL_JUDGE_INJECTION_GUARD,
+    fenceUntrusted("GOLD", gold.map((g, i) => `${i + 1}. ${g}`).join("\n")),
+    fenceUntrusted("SUBMISSION", t.reply ?? ""),
+  ].join("\n\n")
+);
+const hits = reply.match(/\bYES\b/g)?.length ?? 0;
+t.score("recall", hits / gold.length).atLeast(0.5);
+```
+
+## Keep side effects out of eval sessions
+
+Eval sessions run the real agent, tools included. A reviewer that
+comments on GitHub or posts to Slack will do so from an eval unless
+the tool checks the session's purpose. Eval sessions carry
+`purpose: "eval"`; live traffic carries `"live"`. Branch on it in the
+tool, hook, or result handler that actuates:
+
+```ts
+// agent/tools/post_findings.ts
+async execute({ findings }, ctx) {
+  if (ctx.session.purpose === "eval") {
+    return { posted: false, reason: "eval", count: findings.length };
+  }
+  // post the review
+}
+```
+
+Return a shaped result instead of throwing, so the eval can still
+assert `t.calledTool("post_findings", { input: ... })` on the decision.
+The same check belongs in [hooks](/docs/reference/hooks.md) that meter or
+page and in [`defineResult`](/docs/reference/result.md) commits.
+
+## Worked examples
+
+### Multi-turn: grade each turn
+
+```ts
+// evals/intro.eval.ts
+import { defineEval, includes, satisfies } from "@cursor/july/evals";
+
+export default defineEval({
+  description: "Introduces itself once; a repeat mention gets a short ack.",
+  async test(t) {
+    const intro = await t.send("Meet Jenny! @Jenny introduce yourself.");
+    intro.expectOk();
+    t.check(intro.message, includes(/jenny/i));
+
+    const repeat = await t.send("Meet, @Jenny!");
+    t.succeeded();
+    repeat.usedNoTools();
+    t.check(
+      repeat.message,
+      satisfies((r) => (r as string).trim().length <= 280, "short ack")
+    );
+    t.check(
+      repeat.message,
+      satisfies((r) => !/what i can do/i.test(r as string), "no second intro")
+    );
+  },
+});
+```
+
+`t.succeeded()` grades the whole session. `repeat.usedNoTools()` and the
+checks on `repeat.message` read only the second turn, even though
+`t.reply` now holds its text.
+
+### Approvals: assert the parked decision
+
+For a tool with `needsApproval`, the turn parks instead of finishing.
+Gate on `t.parked()` and on the arguments the model chose:
+
+```ts
+// evals/agents.eval.ts (one case; RULE and SLACK are fixture strings)
+{
+  id: "update-rule",
+  description: "A repeated billing rule parks the AGENTS.md write.",
+  async test(t) {
+    await t.send("Weekly AGENTS.md review. Read week/ and call apply_agents once.", {
+      workspaceFiles: {
+        "week/prs.md": RULE,
+        "week/slack.md": SLACK,
+        "week/tree/AGENTS.md.txt": "# API\n\nKeep handlers thin.\n",
+      },
+    });
+    t.parked();
+    t.calledTool("apply_agents", { input: { verdict: "update" } });
+  },
+},
+```
+
+`t.parked()` and `t.succeeded()` are exclusive: a parked run is a clean
+stop on an unanswered approval, not a completed one. Pair the parked
+case with a sibling that expects `verdict: "skip"` and `t.succeeded()`,
+so both branches stay pinned.
+
+## Pin fixtures
+
+A fixed input is what makes an eval repeatable. Pick the fixture by the
+surface under test.
+
+| Agent surface | Fixture |
+| --- | --- |
+| Chat or domain assistant | One canonical prompt string, chosen once and frozen |
+| Tool-heavy | Run `agent-sdk call <tool>` first to pin what the tool returns, then freeze the prompt that triggers it |
+| GitHub webhook | `agent-sdk github replay <pr> --events '*' --dry-run --out fixtures/github` snapshots real payloads for offline replay ([GitHub guide](/docs/guides/github.md#test-with-github-replay)) |
+| PR reviewer with host preparation | Diff, metadata, and gold labels pinned to commit SHAs; keep any live PR matrix small |
+| Workspace-dependent | `workspaceFiles` on the first `t.send`, never developer-machine paths |
+
+Tag the fast, reliably passing core `smoke` and run `--tag smoke` in
+the inner loop. Leave slow or drift-prone cases untagged for explicit
+runs.
+
+### Materialize API-backed fixtures
+
+An input that only points at external data (a pull request URL, a
+snapshot id, a pair of commit SHAs) is not self-contained. Fetch it once
+and commit the rendered fixture before you expand the suite:
+
+1. Save the diff, metadata, and labels under `fixtures/` at pinned
+   revisions.
+2. Seed those files with `workspaceFiles`, or read them from the
+   fixture directory.
+3. Assert decisions and output shape against the saved evidence.
+4. Keep a small `smoke` subset for any remaining live checks.
+
+`maxConcurrency` limits parallel datapoints, not the model or API
+fan-out inside one datapoint. Materialized fixtures keep a large suite
+from exhausting provider and GitHub rate limits.
+
+### Load a dataset
+
+Read committed fixtures with `loadJson`, `loadJsonl`, and `loadYaml`
+from `@cursor/july/evals/loaders`. Relative paths resolve against the
+project root the runner discovered, not the cwd the CLI ran from. Eval
+files are ES modules, so top-level `await` can load a dataset and fan
+one file out over it:
+
+```ts
+// evals/sql.eval.ts: sql/0000, sql/0001, ...
+import { defineEval, equals } from "@cursor/july/evals";
+import { loadYaml } from "@cursor/july/evals/loaders";
+
+const rows = await loadYaml<{ task: string; prompt: string; sql: string }[]>(
+  "evals/data/cases.yaml"
+);
+
+export default rows.map((row) =>
+  defineEval({
+    description: row.task,
+    async test(t) {
+      await t.send(row.prompt);
+      t.succeeded();
+      t.check(t.reply, equals(row.sql));
+    },
+  })
+);
+```
+
+## Configure eval runs
+
+`evals/evals.config.ts` (or `.js`) holds project-wide defaults. It must
+set `maxConcurrency`. Each case issues real model requests, so
+concurrency is hard-capped at 200; the templates use 10.
+`eval --list` works without the file. Running a case does not.
+
+```ts
+import { defineEvalConfig } from "@cursor/july/evals";
+
+export default defineEvalConfig({
+  maxConcurrency: 20,
+  timeoutMs: 180_000,
+  judge: { model: "gpt-5.4-mini" },
+});
+```
+
+| Option | Default | Meaning |
+| --- | --- | --- |
+| `maxConcurrency` | required | Datapoints in flight at once, 1-200 |
+| `timeoutMs` | `180_000` | Per-case timeout. Precedence: case or file `timeoutMs`, then `--timeout-ms`, then this value |
+| `judge` | unset | Default judge model for `t.judge.*` |
+| `reporters` | unset | Destinations that observe every case; `--skip-report` suppresses them |
+| `maxPlaygroundRuns` | `20` | Batches kept in playground history (cap 500). Counts `--prod` and `--url` batches, not the default local run |
 
-## Run evals from the CLI
+Reporters ship results somewhere; the runner still does the grading.
+`JUnit({ filePath, suiteName? })` writes JUnit XML and
+`Artifacts({ dir })` writes per-case files, both from
+`@cursor/july/evals/reporters`. A custom reporter is an object with any
+of `onRunStart`, `onEvalComplete`, and `onRunComplete`. A reporter that
+throws is logged and never fails the run. CI usually attaches the
+built-in two with `--junit` and `--artifacts` instead of `reporters`,
+so output paths stay with the pipeline, not the eval author.
 
-The `eval` command discovers, filters, and runs cases.
+Playground batches survive restarts when the project has
+[storage](/docs/storage.md#eval-table). Otherwise they live in process
+memory until `serve` exits.
 
-Run the CLI under Node 22.13 or newer. Do not use Bun. Its HTTP/2 client
-breaks tool-result streams and causes eval turns to fail.
+## Run evals from the CLI
 
 ```bash
-agent-sdk eval --dir . --list                         # discover only
-agent-sdk eval --dir .                                # run all
-agent-sdk eval --dir . builds/checkout                # one datapoint
-agent-sdk eval --dir . builds search                  # several ids or prefixes
-agent-sdk eval --dir . --tag smoke --tag pull-request # any matching tag
-agent-sdk eval --dir . --json --no-stream             # machine-readable results
-agent-sdk eval --dir . --verbose                      # logs + reply snippets
+agent-sdk eval --dir . --list                          # discover only
+agent-sdk eval --dir .                                 # run all
+agent-sdk eval --dir . builds/checkout                 # one datapoint
+agent-sdk eval --dir . builds search                   # several ids or prefixes
+agent-sdk eval --dir . --tag smoke --tag pull-request  # any matching tag
+agent-sdk eval --dir . --verbose                       # t.log lines + reply snippets
+agent-sdk eval --dir . --timeout-ms 300000             # override the per-case timeout
 ```
 
 Id filters use OR semantics. Each filter selects an exact id and its
-descendants. For example, `builds` selects `builds`,
-`builds/checkout`, and every other case below that path. Repeated tags
-also use OR semantics. When you provide both ids and tags, a case must
-match both groups.
+descendants: `builds` selects `builds`, `builds/checkout`, and every
+other case below that path. Repeated tags also use OR. With both ids
+and tags, a case must match both groups.
 
-`eval` boots an ephemeral server on port 0 with a temp state root
-outside the project, so cases don't inherit ambient monorepo rules and
-don't write into the project state directory. Point `--url` at a running server to eval
-a live agent instead:
+By default `eval` boots a throwaway server with its own state root, so
+cases don't inherit your checkout's `AGENTS.md` and session state stays
+out of the project. Artifacts still land in the project state
+directory; see [Where results land](#where-results-land). `--slug`
+picks the target in a multi-agent directory.
+
+`--url` runs the batch on a running server instead, the same way
+`--prod` does: that server discovers its own `evals/`, results land in
+its playground history, and the local-only flags (`--junit`,
+`--artifacts`, `--max-concurrency`, `--skip-report`) do not apply. See
+[Run evals in the playground or on a deployment](#run-evals-in-the-playground-or-on-a-deployment).
 
 ```bash
-agent-sdk eval --dir . \
-  --url http://127.0.0.1:3000/weather-agent \
+agent-sdk eval --url http://127.0.0.1:3000/weather-agent \
   --bearer-token "$AGENT_TOKEN"
 ```
 
-The eval definitions still come from `--dir`; `--url` only changes the
-agent that receives the turns. For a locally mounted multi-agent
-directory, `--slug weather-agent` chooses the target. Use
-`--state-root` to keep ephemeral session state at a chosen path,
-`--timeout-ms` to override the project timeout, and `--no-stream` to
-keep live progress off stderr. A TTY streams turn progress by default.
-`--verbose` still writes `t.log` lines to stderr and adds reply snippets
-to text results.
+See [CLI: eval](/docs/reference/cli.md#eval) for every flag.
+
+### Credentials
+
+Model turns need a
+[Cursor credential](/docs/reference/cli.md#environment-variables):
+`CURSOR_API_KEY`, `CURSOR_SERVICE_ACCOUNT_KEY`, or a saved
+`agent-sdk login`. The judge uses the same one. `eval --list` needs
+none.
 
-Model turns need a Cursor credential: `CURSOR_API_KEY`, then
-`CURSOR_API_KEY_FILE` (hosted default `/run/cursor/secrets/CURSOR_API_KEY`), then
-`CURSOR_SERVICE_ACCOUNT_KEY`, then `agent-sdk login`.
+### Where results land
 
-See [CLI: eval](/docs/reference/cli.md#eval) for flags and exit codes.
+Every local run writes artifacts to a timestamped directory under
+`evals/` in the project state directory, whatever `--state-root` says.
+`--artifacts <dir>` chooses the path and `--no-artifacts` skips them.
+The directory holds `summary.json`,
+`results.jsonl`, and `evals/<case-id>.json` with every assertion, the
+inputs, tool calls with arguments and output, the final text, and
+`t.log` lines. Start there when a case fails. `--out <file>` also
+writes the full results JSON to a path of your choice.
 
-### JSON results
+The artifact does not include the session's event stream. Pass
+`--state-root <path>` to keep the ephemeral server's
+[session data](/docs/reference/sessions.md#where-does-the-agent-sdk-store-session-data)
+on disk when you need the raw events.
 
-Use `--json --no-stream` in scripts and CI. The top-level result carries
-the totals and one result per case:
+## Run evals in CI
+
+Run the suite non-interactively, write JUnit for the CI annotations,
+and fail the job on a red gate:
+
+```bash
+# CURSOR_API_KEY comes from the CI secret store
+agent-sdk eval --dir . --json --no-stream \
+  --junit reports/evals.xml \
+  --artifacts reports/evals \
+  > reports/evals.json
+```
+
+The exit code follows the
+[verdict table](#gates-soft-scores-and-verdicts); `2` means nothing
+matched the selection. `--max-concurrency` overrides the project
+setting, for example to run lower on a shared runner.
+
+The JSON on stdout carries the totals and one result per case:
 
 ```json
 {
   "ok": true,
   "passed": 1,
   "failed": 0,
+  "scored": 0,
+  "skipped": 0,
+  "strict": false,
+  "artifactsDir": "/work/my-agent/reports/evals",
   "results": [
     {
       "id": "readiness",
+      "verdict": "passed",
       "ok": true,
-      "assertions": [{ "name": "succeeded", "passed": true }],
+      "assertions": [
+        { "name": "succeeded", "passed": true },
+        { "name": "calledTool(inspect_pr)", "passed": true }
+      ],
       "sessionId": "ses_123",
-      "inputs": ["Is checkout pull request 42 ready to approve?"],
+      "inputs": ["Is https://github.com/acme/checkout/pull/42 ready to approve?"],
       "toolCalls": [{ "toolName": "inspect_pr", "isError": false }],
+      "metrics": {},
       "logs": [],
       "durationMs": 12340
     }
@@ -3574,126 +3613,64 @@ the totals and one result per case:
 }
 ```
 
-Each case result can also include `description`, `finalText`, `tools`,
-`error`, and tool arguments or output. This shape lets CI report the
-failed assertion without parsing terminal text.
+Each result can also include `description`, `finalText`, `tools`,
+`error`, `skipReason`, `metadata`, `tags`, and tool `args` and `output`.
+A soft miss shows as `"severity": "soft"` with `score` and `threshold`
+on the assertion. This shape lets CI report the failed assertion
+without parsing terminal text.
+
+Keep CI green without weakening gates:
 
-## Run evals in the playground
+- Run `--tag smoke` on every push and the full suite on a schedule.
+- For probabilistic behavior, use `iterations` and a soft bar instead
+  of one hard gate.
 
-Start the server, open the playground, and choose **Evals**. You can run
-every case or one case, watch progress, and open the resulting session
-trace. The Evals tab works on a normal `serve`.
+## Run evals in the playground or on a deployment
+
+Start the server, open the playground, and choose **Evals**. Run every
+case or one case, watch progress, and open the resulting session trace.
+Playground runs target the live server instead of an ephemeral one, so
+their sessions appear in the session list. One batch runs at a time.
 
 ```bash
 agent-sdk serve --dir .
 ```
 
-Playground runs target the live server instead of an ephemeral one.
-Their sessions appear in the session list. One eval batch can run at a
-time. Persistence follows the rule under
-[Configure eval runs](#configure-eval-runs). See
-[Playground eval routes](/docs/reference/http-api.md#playground-eval-routes).
-The start request returns `202` while cases run in the background.
-Poll until the snapshot status becomes `completed`, `failed`, or `cancelled`.
-Configuration errors appear on a failed snapshot.
-
-On `--prod` / `--url`, the CLI prints the Eval ID as soon as the batch is
-accepted (and a Playground deep link with `?view=evals&evalRunId=…`):
+`--prod` (or `--url`) starts the same server-side batch on the team's
+hosted deployment (or the server you name), so results land in that
+server's playground history:
 
 ```bash
 agent-sdk eval --prod --slug vulnerability-scanner --tag deepsec
-# Eval ID: evalrun_…
-# Cancel:  agent-sdk eval cancel evalrun_… --prod --slug vulnerability-scanner
-# Playground: https://…/playground?view=evals&evalRunId=evalrun_…
-
-agent-sdk eval cancel evalrun_… --prod --slug vulnerability-scanner
-agent-sdk eval status evalrun_… --prod --slug vulnerability-scanner
+# Eval ID: <evalId>
+# Playground: <deployment>/playground?view=evals&evalRunId=<evalId>
+agent-sdk eval status <evalId> --prod --slug vulnerability-scanner
+agent-sdk eval cancel <evalId> --prod --slug vulnerability-scanner
 ```
 
-## What good cases assert
-
-Gate decisions and shape, not prose. Model wording varies run to run.
-Tool choice, tool avoidance, and output structure are the stable
-contract.
-
-1. `t.succeeded()`: always, first.
-2. The tool decision: `calledTool` for the intended path,
-   `notCalledTool` for the likely wrong alternative. The pair is
-   stronger than either alone.
-3. Output shape: a regex for the contract (`/ready|blocked/i`, a JSON
-   marker, a findings-block fence), never exact sentences.
-4. For structured output, parse `t.reply` and check fields with
-   `satisfies` instead of substring-matching JSON.
-
-The common failure modes: asserting exact phrasing, packing more than
-about five gates into one case (split it), and cases that depend on live
-external state that drifts (pin the input; see fixtures).
-
-## Pick fixtures by agent type
-
-The right fixture depends on the surface under test.
-
-| Agent surface | Fixture |
-| --- | --- |
-| Chat / domain assistant | A canonical prompt string, chosen once and frozen |
-| Tool-heavy | Run `agent-sdk call <tool>` first to pin what the tool returns, then freeze the prompt that triggers it |
-| GitHub webhook | `agent-sdk github replay <pr> --events '*' --dry-run --out fixtures/github` snapshots real payloads for offline replay ([GitHub guide](/docs/guides/github.md)) |
-| PR reviewer with host preparation | Diff, metadata, and gold labels pinned to commit SHAs; keep any live PR matrix small |
-| Workspace-dependent | `workspaceFiles` in `t.send` options, never developer-machine paths |
-
-Tag the fast, reliably passing core `smoke` and run `--tag smoke` in the
-inner loop. Leave slow or flaky-prone cases untagged for explicit runs.
-
-### Materialize API-backed fixtures
-
-An input that only points at external data, such as a pull request URL,
-snapshot id, or pair of commit SHAs, is not self-contained. Fetch it
-once and commit the rendered fixture before you expand the suite.
-
-1. Save the diff, metadata, and labels under `fixtures/` at pinned
-   revisions.
-2. Seed those files with `workspaceFiles`, or read them from the fixture
-   directory.
-3. Assert decisions and output shape against the saved evidence.
-4. Keep a small `smoke` subset for any remaining live pipeline checks.
-
-Read committed fixtures with `@cursor/july/evals/loaders`: `loadJson`,
-`loadJsonl`, and `loadYaml` resolve relative paths against the project
-root the runner discovered, not the cwd the CLI was invoked from
-(`resolveFixturePath` and `evalFixtureRoot` expose the same
-resolution for other file formats).
-
-`maxConcurrency` limits parallel datapoints. It does not limit model or
-API fan-out inside one datapoint. Materialized fixtures prevent a large
-suite from exhausting provider and GitHub rate limits. The
-[evals skill](/docs/skills/evals.md) has the full fixture workflow.
+The CLI prints the Eval ID as soon as the batch is accepted. Pass
+`--no-wait` to return right away and poll with `eval status` later; it
+exits `3` while the batch is still running. Hosted history follows
+`maxPlaygroundRuns` and the persistence rule under
+[Configure eval runs](#configure-eval-runs). The HTTP surface is under
+[Playground eval routes](/docs/reference/http-api.md#playground-eval-routes).
 
 ## Keep improvements with regression evals
 
-Every [hillclimb](/docs/hillclimbing.md) round that keeps a change must land
-an eval that would have failed before the change. If you can't express
-the improvement as a gate (a `calledTool` shift, a bounded
-`action.result` count, an output-shape regex), the improvement is
-unverified, and it'll regress silently.
+Every [hillclimb](/docs/hillclimbing.md) round that keeps a change must
+land an eval that would have failed before the change. If you can't
+express the improvement as a gate (a `calledTool` shift, a bounded
+`maxToolCalls`, an output-shape check), the improvement is unverified,
+and it'll regress silently.
 
-The rule cuts the other way too: never weaken an existing gate to make a
-round pass. That's the freeze line moving, and it turns your regression
-suite into a list of checks that no longer protect anything.
-
-## Compare variants on live traffic
-
-Use `defineAB` to compare variant metrics on live sessions. It is not a
-test runner and has no `agent-sdk ab` command. Keep `defineEval` as the
-regression ratchet. Eval sessions do not enroll or change live metrics.
-See [Live A/B metrics](/docs/ab.md) for assignment, behavior, collection,
-and inspection.
+The rule cuts the other way too: never weaken an existing gate to make
+a round pass. That's the freeze line moving, and it turns your
+regression suite into a list of checks that no longer protect anything.
 
 ## What's next
 
 Continue with these pages:
 
-- [Live A/B metrics](/docs/ab.md): sticky variants and cumulative metrics
-  on live sessions
 - [Hillclimbing](/docs/hillclimbing.md): the loop evals make trustworthy
 - [Building agents with agents](/docs/building-with-agents.md): have a
   coding agent write the first suite
@@ -3822,6 +3799,95 @@ Continue with these pages:
 
 ---
 
+Source: /docs/guides/bitbucket.md
+
+# Bitbucket agents
+
+Use `bitbucketChannel()` for Bitbucket Cloud and Bitbucket Data Center. The
+channel detects the source from each signed payload and normalizes Data Center
+events to the Bitbucket Cloud event vocabulary.
+
+## Define the channel
+
+Author `agent/channels/bitbucket.ts`:
+
+```ts
+import { bitbucketChannel } from "@cursor/july/channels/bitbucket";
+
+export default bitbucketChannel({});
+```
+
+Without `onPullRequest`, new pull requests start turns. Data Center updates
+also start turns when the source branch has new commits. Comments and pushes
+are opt-in through `onPullRequestComment` and `onPush`. Use `onEvent` for
+other event types. Set `webhookEvents` when managed event delivery must
+subscribe to an event that only `onEvent` handles.
+
+Filter comment hooks by author before starting a turn. This prevents comments
+posted by the agent from triggering another turn.
+
+`ctx.bitbucket.api` can read pull requests, post comments, and create build
+statuses. See the [Channels reference](/docs/reference/channels.md) for hook
+return values and session behavior.
+
+## Connect Bitbucket
+
+Set a repository hook secret and API token:
+
+```bash
+BITBUCKET_WEBHOOK_SECRET=...
+BITBUCKET_TOKEN=...
+```
+
+Add a repository webhook for
+`https://<your-host>/<slug>/v1/channels/bitbucket`. Use the same secret on
+both sides. The channel verifies the `X-Hub-Signature` HMAC before it parses
+the payload.
+
+Bitbucket Cloud uses its 2.0 API by default. Data Center also needs its REST
+API base:
+
+```ts
+export default bitbucketChannel({
+  apiBaseUrl: "https://bitbucket.example.com/rest/api/1.0",
+  repos: ["PLATFORM/api"],
+});
+```
+
+You can set `BITBUCKET_API_BASE_URL` instead.
+
+## Test locally
+
+Start the agent, then replay a pull request you can read:
+
+```bash
+agent-sdk dev
+agent-sdk bitbucket replay \
+  https://bitbucket.example.com/projects/PLATFORM/repos/api/pull-requests/42
+```
+
+Replay supports Bitbucket Cloud and Data Center URLs. `BITBUCKET_TOKEN` needs
+pull request read access. The command reads the pull request, creates a payload
+in the matching dialect, and sends it through the same channel route. Use
+`--events '*'` to replay the supported events declared by the channel. Use
+`--dry-run --out fixtures/bitbucket` to save fixtures.
+
+```bash
+agent-sdk bitbucket events --dir .
+agent-sdk bitbucket forward --dir .
+```
+
+`forward` prints the repository-hook and HTTPS tunnel setup for live
+deliveries.
+
+## Related
+
+- [Channels reference](/docs/reference/channels.md)
+- [Webhooks and custom channels](/docs/guides/webhooks.md)
+- [Evals](/docs/evals.md)
+
+---
+
 Source: /docs/guides/cloud-agents.md
 
 # Cursor cloud agents
@@ -4173,6 +4239,37 @@ snapshot in the wake.
 This is the preferred production path: no public URL, no repo admin
 webhook, and no inbound network for GitHub deliveries.
 
+## Connect GitHub Enterprise Server
+
+GitHub Enterprise Server uses the same `githubChannel()` hooks and normalized
+events. Connect it through the direct webhook route. Set the REST API base and
+credentials for your server:
+
+```ts
+export default githubChannel({
+  api: {
+    apiBaseUrl: "https://github.example.com/api/v3",
+  },
+  credentials: {
+    token: () => process.env.GITHUB_TOKEN,
+    webhookSecret: () => process.env.GITHUB_WEBHOOK_SECRET,
+  },
+  onPullRequest: (ctx, pr) =>
+    pr.action === "opened" ? { auth: defaultGitHubAuth(ctx) } : null,
+});
+```
+
+Add a repository webhook for
+`https://<your-host>/<slug>/v1/channels/github`. Use the same webhook secret
+on the server and in `GITHUB_WEBHOOK_SECRET`. The channel verifies
+`X-Hub-Signature-256` before it parses the payload.
+
+Cursor account event pull, `github replay`, and `github forward` target
+GitHub.com. Test Enterprise Server integrations by posting saved webhook
+fixtures to a local `--dev` server. Leave `GITHUB_WEBHOOK_SECRET` unset for
+this local test so the channel admits unsigned loopback deliveries. Use a real
+delivery from your server so the fixture matches its version.
+
 ## Define the channel
 
 Author `agent/channels/github.ts` with `githubChannel()` from
@@ -4380,6 +4477,103 @@ key. Handlers you author replace the matching defaults (same as
 
 ---
 
+Source: /docs/guides/gitlab.md
+
+# GitLab agents
+
+Use `gitlabChannel()` for GitLab.com and self-managed GitLab. The channel
+verifies project hooks, normalizes their payloads, and gives each hook a
+project-bound `ctx.gitlab` API client.
+
+## Define the channel
+
+Author `agent/channels/gitlab.ts`:
+
+```ts
+import { gitlabChannel } from "@cursor/july/channels/gitlab";
+
+export default gitlabChannel({});
+```
+
+Without `onMergeRequest`, new and reopened merge requests start turns.
+Updates start turns only when they include new commits. Notes, pushes, and
+pipelines are opt-in through `onNote`, `onPush`, and `onPipeline`.
+Use `onEvent` for other GitLab event types. Set `webhookEvents` when managed
+event delivery must subscribe to an event that only `onEvent` handles.
+
+Filter note hooks by author before starting a turn. This prevents comments
+posted by the agent from triggering another turn.
+
+`ctx.gitlab` can call the project REST API and create commit statuses. See the
+[Channels reference](/docs/reference/channels.md) for hook return values and
+session behavior.
+
+## Connect GitLab
+
+On Cursor-managed hosting, use the signed-in Cursor account:
+
+```ts
+export default gitlabChannel({
+  cursorAccount: {
+    projects: ["acme/platform"],
+  },
+});
+```
+
+For direct webhooks, set a project hook secret and API token:
+
+```bash
+GITLAB_WEBHOOK_SECRET=...
+GITLAB_TOKEN=...
+```
+
+Add a project webhook for
+`https://<your-host>/<slug>/v1/channels/gitlab`. Use the same value for the
+GitLab secret token and `GITLAB_WEBHOOK_SECRET`. The channel checks
+`X-Gitlab-Token` before it parses the payload.
+
+Self-managed GitLab also needs its REST API base:
+
+```ts
+export default gitlabChannel({
+  apiBaseUrl: "https://gitlab.example.com/api/v4",
+  projects: ["acme/platform"],
+});
+```
+
+You can set `GITLAB_API_BASE_URL` instead.
+
+## Test locally
+
+Start the agent, then replay a merge request you can read:
+
+```bash
+agent-sdk dev
+agent-sdk gitlab replay \
+  https://gitlab.example.com/acme/platform/-/merge_requests/42
+```
+
+`GITLAB_TOKEN` needs API read access. Replay reads the merge request,
+creates GitLab-shaped payloads, and sends them through the same channel route.
+Use `--events '*'` to replay the supported events declared by the channel.
+Use `--dry-run --out fixtures/gitlab` to save fixtures.
+
+```bash
+agent-sdk gitlab events --dir .
+agent-sdk gitlab forward --dir .
+```
+
+GitLab has no local webhook relay. `forward` prints the project-hook and HTTPS
+tunnel setup for live deliveries.
+
+## Related
+
+- [Channels reference](/docs/reference/channels.md)
+- [Webhooks and custom channels](/docs/guides/webhooks.md)
+- [Evals](/docs/evals.md)
+
+---
+
 Source: /docs/guides/grokbot-agents.md
 
 # Cursor Grok Bot agents
@@ -5395,7 +5589,8 @@ A custom channel gives the agent its own HTTP surface. You get routes
 with validated payloads, sessions keyed to something in your domain (a
 thread, a ticket, a PR), and replies delivered back to the caller. The
 [Slack](/docs/guides/slack.md) and [GitHub](/docs/guides/github.md) packs build on this
-mechanism. This page is the mechanism itself.
+mechanism. The [GitLab](/docs/guides/gitlab.md) and [Bitbucket](/docs/guides/bitbucket.md) packs
+use it too. This page is the mechanism itself.
 
 ## What you already have
 
@@ -5849,7 +6044,8 @@ from any PR you can read. See the [GitHub guide](/docs/guides/github.md).
 Continue with these pages:
 
 - [Channels reference](/docs/reference/channels.md): the full authoring API
-- [GitHub](/docs/guides/github.md) and [Slack](/docs/guides/slack.md): the packaged channels
+- [GitHub](/docs/guides/github.md), [GitLab](/docs/guides/gitlab.md),
+  [Bitbucket](/docs/guides/bitbucket.md), and [Slack](/docs/guides/slack.md): the packaged channels
 - [Sessions and streaming](/docs/reference/sessions.md): events your
   channel can subscribe to
 
@@ -5926,9 +6122,9 @@ Pin the input first. A moving fixture is noise. For GitHub agents, use `agent-sd
 
 ## How do I lock a hillclimb improvement with an eval?
 
-Every kept change needs an eval that would have failed before the change: a tool-choice gate, an `action.result` count bound, or an output-shape check. Run `agent-sdk eval --dir . --json` between rounds. Never weaken an existing gate to pass the round.
+Every kept change needs an eval that would have failed before the change: a tool-choice gate, a `maxToolCalls` bound, or an output-shape check. Run `agent-sdk eval --dir . --json` between rounds. Never weaken an existing gate to pass the round.
 
-Details live in [Evals](/docs/evals.md). The evals skill will author the case with you.
+Details live in [Keep improvements with regression evals](/docs/evals.md#keep-improvements-with-regression-evals). The evals skill will author the case with you.
 
 ## What habits help hillclimbing stay reliable?
 
@@ -5983,14 +6179,16 @@ npx @cursor/july docs
 | Turning a Cursor Automation into a project | [Convert a Cursor Automation](/docs/guides/convert-automation.md) |
 | Wiring an agent to Slack | [Slack guide](/docs/guides/slack.md) |
 | Starting from a packaged template | [Demo](/docs/templates/demo.md), [Grok Bot agents](/docs/templates/grokbot-agents.md), [Code wiki](/docs/templates/code-wiki.md), [Living AGENTS.md](/docs/templates/agents-md.md), [Security reviewer](/docs/templates/security-reviewer.md), [Security help](/docs/templates/security-help.md), [Triage](/docs/templates/triage.md), or [Agentic Owners](/docs/templates/agentic-owners.md) |
-| Wiring an agent to GitHub webhooks | [GitHub guide](/docs/guides/github.md) |
+| Wiring an agent to GitHub or GitHub Enterprise Server | [GitHub guide](/docs/guides/github.md) |
+| Wiring an agent to GitLab | [GitLab guide](/docs/guides/gitlab.md) |
+| Wiring an agent to Bitbucket | [Bitbucket guide](/docs/guides/bitbucket.md) |
 | Driving PRs from a cloud VM | [PR autofixer template](/docs/templates/pr-autofixer.md) |
 | Handing coding work to Cursor cloud agents | [Cursor cloud agents](/docs/guides/cloud-agents.md) |
 | Talking to your Grok Bot agents | [Cursor Grok Bot agents](/docs/guides/grokbot-agents.md) |
 | Letting an agent change its own source | [Self-improvement](/docs/guides/improve.md) |
 | Driving an agent from Linear (or another tracker) | [Webhooks guide: Linear example](/docs/guides/webhooks.md#example-linear-as-the-control-plane) |
 | Making an existing agent measurably better | [Evals](/docs/evals.md), then [Hillclimbing](/docs/hillclimbing.md) |
-| Comparing variants on live traffic | [Live A/B metrics](/docs/ab.md) |
+| Gating agent behavior in CI | [Run evals in CI](/docs/evals.md#run-evals-in-ci) |
 | Deploying with Cursor or on your own infrastructure | [Deployment](/docs/deployment.md) |
 | Debugging something that misbehaves | [Fix common agent problems](/docs/troubleshooting.md) |
 
@@ -6031,10 +6229,8 @@ npx @cursor/july docs
 
 - [Building agents with agents](/docs/building-with-agents.md): use a coding
   agent to scaffold, run, and iterate on your agent.
-- [Evals](/docs/evals.md): author `defineEval` cases, pick fixtures, and use
-  evals as regression checks.
-- [Live A/B metrics](/docs/ab.md): assign sticky variants and compare
-  cumulative metrics on live sessions.
+- [Evals](/docs/evals.md): author `defineEval` cases, assert over the
+  trajectory, and run them locally and in CI.
 - [Storage](/docs/storage.md): point durable storage at a backend you own
   with `defineStorage`.
 - [Hillclimbing](/docs/hillclimbing.md): measure and improve an agent
@@ -6046,6 +6242,10 @@ npx @cursor/july docs
   its own HTTP surface.
 - [GitHub](/docs/guides/github.md): trigger the agent from pull requests,
   CI, and comments.
+- [GitLab](/docs/guides/gitlab.md): trigger the agent from merge requests,
+  notes, pipelines, and pushes.
+- [Bitbucket](/docs/guides/bitbucket.md): trigger the agent from pull requests,
+  comments, and pushes.
 - [Slack](/docs/guides/slack.md): put the agent in Slack over Socket Mode.
 - [Human-in-the-loop approvals](/docs/guides/human-in-the-loop.md): park a
   tool call until a person signs off.
@@ -6862,7 +7062,8 @@ Source: /docs/reference/channels.md
 A channel is the surface an agent lives on. The built-in HTTP session
 channel is always mounted. Custom channels declare their own routes
 under `/v1/channels/<id>`. The Slack and GitHub packs are prebuilt
-channels with platform transports. This page is the authoring reference;
+channels with platform transports. GitLab and Bitbucket packs cover their
+hosted and self-managed products. This page is the authoring reference;
 for the walkthrough, see the [Webhooks guide](/docs/guides/webhooks.md).
 
 ## Built-in HTTP channel
@@ -6988,7 +7189,7 @@ Handlers receive the Fetch `Request` and an args object:
 `workspaceFiles`, `workspaceDir`, `cloud` (attach cloud repos for this
 session), `auth` (defaults to the request principal), `state` (starting
 channel state for new sessions), `title` (session display title), and
-`purpose` (`"eval"` skips sticky A/B enrollment).
+`purpose` (`"eval"` marks the session as regression traffic).
 
 ## Events
 
@@ -7069,7 +7270,19 @@ model turn), `{ task }` (host work), or `null`, and CLI tooling for
 replay and live forwarding. Author `agent/channels/github.ts` with
 `githubChannel()`. Opt-in `progress.commitStatus` and `progress.banner`
 converge a merge-box check and sticky PR comment from default stream
-events. Guide: [GitHub](/docs/guides/github.md).
+events. Supports GitHub.com and GitHub Enterprise Server. Guide:
+[GitHub](/docs/guides/github.md).
+
+**GitLab** (`@cursor/july/channels/gitlab`): verified project hooks for
+merge requests, notes, pipelines, pushes, and custom event types. Supports
+GitLab.com and self-managed GitLab. Author `agent/channels/gitlab.ts` with
+`gitlabChannel()`. Guide: [GitLab](/docs/guides/gitlab.md).
+
+**Bitbucket** (`@cursor/july/channels/bitbucket`): verified repository hooks
+for pull requests, comments, pushes, and custom event types. Supports
+Bitbucket Cloud and Bitbucket Data Center through one normalized hook API.
+Author `agent/channels/bitbucket.ts` with `bitbucketChannel()`. Guide:
+[Bitbucket](/docs/guides/bitbucket.md).
 
 **Deployments** (`@cursor/july/channels/deployments`): pull deploy
 events. Declare `events` and handle each one in `onEvent`. Each event
@@ -7491,8 +7704,10 @@ agent-sdk eval status <evalId> --prod --slug pr-approver
 agent-sdk eval cancel <evalId> --prod --slug pr-approver
 ```
 
-`eval` runs `evals/**/*.eval.{ts,js}` on an ephemeral server or against
-`--url`. Select one or more exact case IDs, file ID prefixes, or tags.
+`eval` runs `evals/**/*.eval.{ts,js}` on an ephemeral server. With
+`--prod` or `--url`, the target server runs its own evals as a
+server-side batch. Select one or more exact case IDs, file ID prefixes,
+or tags.
 Omit selectors to run all cases. Repeated `--tag` flags use OR matching.
 
 An eval run requires `evals/evals.config.{ts,js}` with `maxConcurrency`
@@ -7503,13 +7718,13 @@ between 1 and 200. Timeout priority is the case's `timeoutMs`, the CLI's
 | --- | --- |
 | `--list` | Print discovered cases without running. `--list --json` prints them as an array. |
 | `--tag <tag>` | Run cases with this tag. Repeated flags use OR matching. |
-| `--json` | Print `{ ok, passed, failed, results }`. |
+| `--json` | Print `{ ok, passed, failed, scored, skipped, strict, results }`; see [Run evals in CI](/docs/evals.md#run-evals-in-ci) for the result shape. |
 | `--verbose` | Stream `t.log` lines and reply snippets. |
 | `--no-stream` | Hide live progress on stderr. |
 | `--strict` | Exit `1` when a scored case misses a soft threshold. |
 | `--max-concurrency <n>` | Override `maxConcurrency` from `evals.config.ts`. |
 | `--junit <path>` | Write JUnit XML for CI annotations. |
-| `--artifacts <dir>` | Write run artifacts here. The default is a timestamped directory under `<state-root>/evals/`. |
+| `--artifacts <dir>` | Write run artifacts here. The default is a timestamped directory under `evals/` in the project state directory (not affected by `--state-root`). |
 | `--no-artifacts` | Skip run artifacts. |
 | `--skip-report` | Ignore reporters from `evals.config.ts` and eval files. |
 | `--out <path>` | Also write the full results JSON to this path (also for `eval status <evalId>`). |
@@ -8473,7 +8688,6 @@ namespace:
 | `subagents/<id>/` | subagent `<ns>__<id>` |
 | `instructions.md` / `.ts` / dir | appended to the agent's system prompt |
 | `sandbox/workspace/**` | seeded into each local session workspace |
-| `ab.ts` / `ab/<name>.ts` | A/B experiment `<ns>__<name>` |
 | `artifacts.ts` | artifact kinds `<ns>__<kind>` |
 
 The root agent still needs its own `instructions.md`. Extension
@@ -8490,7 +8704,6 @@ or is outside discovery:
 | `agent.ts` | One `defineAgent` runtime per agent |
 | `storage.ts` | One `host.kv` / `host.files` backend |
 | `otel.ts` | One OTLP exporter |
-| `ab.config.ts` | One experiment-platform config; `ab.ts` / `ab/` still merge |
 | `playground/` | Custom chips are a Vite glob of the agent tree, not a discovery walk |
 | `extensions/` | Nested mounts are not loaded |
 | `sandbox.ts` | Custom sandbox backends stay on the agent |
@@ -8531,7 +8744,6 @@ it alive.
 | `schedules/<name>.ts` | `disableSchedule()` |
 | `subagents/<id>.ts` | `disableSubagent()` |
 | `instructions.ts` | `disableInstructions()` |
-| `ab.ts` / `ab/<name>.ts` | `disableAB()` |
 | `artifacts.ts` | `disableArtifacts()` |
 
 `disable()` is the same brand as the slot helpers above and works in
@@ -8775,7 +8987,7 @@ exported from `@cursor/july`.
 
 | Member | What it is |
 | --- | --- |
-| `ctx.session` | Read-only session info: `id`, `channelId`, `mode` (`chat` or `task`), `purpose` (`live` or `eval`), `auth`, plus `title`, `sdkAgentId`, and `abs` when set |
+| `ctx.session` | Read-only session info: `id`, `channelId`, `mode` (`chat` or `task`), `purpose` (`live` or `eval`), `auth`, plus `title` and `sdkAgentId` when set |
 | `ctx.agent` | `{ name }` of the agent the event belongs to |
 | `ctx.channel` | `{ id, continuationToken }`. The token is `null` when the session can't take follow-ups |
 | `ctx.host.kv` | Durable JSON, shared by every session of the agent; the [storage backend](/docs/storage.md#author-kv-ctx-host-kv) decides whether it survives a hosted replace. Prefix keys with `ctx.session.id` for per-session state |
@@ -8806,17 +9018,17 @@ Each event reaches a hook at most once. A restart doesn't replay the log
 into hooks, so a mirror needs no dedupe, and the event log rather than
 the hook's copy is the source of truth.
 
-## Hooks, channel events, evals, or A/B?
+## Hooks, channel events, or evals?
 
 All of them consume the same stream, for different jobs:
 
-| | Hooks | Channel `events` | Evals | A/B (`defineAB`) |
-| --- | --- | --- | --- | --- |
-| Scope | every session of the agent | sessions the channel owns | one test turn | every live session; enrollment at creation, metrics on each turn |
-| Job | observe: audit, metrics, mirrors, derived state | deliver: replies back to the channel's surface | assert: gates over the trajectory | compare: sticky arms, then fold the stream into `onSample` metrics |
-| Context | `ctx.host`, `ctx.artifacts`, session info | `channel.state`, `setContinuationToken`, `ctx.host`, session info | the `t` assertion helpers | per-session samples in `onSample` |
-| Can affect the run | no | yes, it owns the surface | n/a | yes through arm instructions or `session.abs`; collection is observe-only |
-| Authored at | `agent/hooks/*.ts` | channel config | `evals/**/*.eval.ts` | [`agent/ab.ts` or `agent/ab/*.ts`](/docs/ab.md) |
+| | Hooks | Channel `events` | Evals |
+| --- | --- | --- | --- |
+| Scope | every session of the agent | sessions the channel owns | one test turn |
+| Job | observe: audit, metrics, mirrors, derived state | deliver: replies back to the channel's surface | assert: gates over the trajectory |
+| Context | `ctx.host`, `ctx.artifacts`, session info | `channel.state`, `setContinuationToken`, `ctx.host`, session info | the `t` assertion helpers |
+| Can affect the run | no | yes, it owns the surface | n/a |
+| Authored at | `agent/hooks/*.ts` | channel config | `evals/**/*.eval.ts` |
 
 ## When not to use a hook
 
@@ -8828,7 +9040,6 @@ All of them consume the same stream, for different jobs:
 | Block, approve, or rewrite a tool call | [`needsApproval`](/docs/reference/tools.md#gate-a-tool-on-human-approval) on the tool |
 | Act on the final assistant text, or fail a bad turn | [`defineResult`](/docs/reference/result.md) |
 | Gate a change on behavior | [Evals](/docs/evals.md) |
-| Compare two prompts on live traffic | [`defineAB`](/docs/ab.md) |
 
 ## Patterns
 
@@ -8954,8 +9165,6 @@ Continue with these pages:
 - [Deployment](/docs/deployment.md#observability): runtime logs and export
   paths
 - [Channels](/docs/reference/channels.md#events): the delivery-side counterpart
-- [Live A/B metrics](/docs/ab.md): sticky variants over the same event
-  stream
 
 ---
 
@@ -9110,12 +9319,11 @@ These read-only routes describe the running agent.
 
 | Route                  | What it does                                                                                                                                                                             |
 | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `GET /v1/info`         | The discovered surface: model, tools, skills, MCP connections, subagents, channels and routes (with schemas), schedules, hooks, A/B experiments, diagnostics |
+| `GET /v1/info`         | The discovered surface: model, tools, skills, MCP connections, subagents, channels and routes (with schemas), schedules, hooks, diagnostics |
 | `GET /v1/tools`        | The live tool catalog: authored server tools plus advertised MCP passthroughs under model-facing names, as light `{ name, title?, source? }` entries. `session` / `continuationToken` query parameters bind the listing to a session identity (advertised inventories can be tenant-scoped); a connection whose listing fails is skipped and reported in `connectionErrors` |
 | `GET /v1/tools/:name`  | One catalog tool's full description: description, execution, `needsApproval`, `effect`, input and output schemas, source connection. Same session binding as the listing; unknown names get `404` with the available names |
 | `GET /v1/health`       | Per-agent liveness, no auth                                                                                                                                                              |
 | `GET /v1/logs?after=N` | Recent server log lines, with a polling cursor                                                                                                                                           |
-| `GET /v1/abs`          | [Live A/B metrics](/docs/ab.md): per-session assignments and aggregate arm totals                                                                                                           |
 
 ## Artifacts
 
@@ -9174,7 +9382,7 @@ the final `completed` or `failed` status. Batch errors appear on the
 snapshot returned by the poll. Entries within `filterIds` and `tags`
 use OR semantics. When both fields are present, a case must match one
 entry from each field. Listed runs persist across restarts when storage is configured; see
-[Storage](/docs/storage.md#eval-and-a-b-tables). Otherwise they are
+[Storage](/docs/storage.md#eval-table). Otherwise they are
 process-memory only.
 
 ## Dev-mode routes
@@ -9336,8 +9544,6 @@ Use the playground to chat, try channel routes, and inspect sessions.
   an [extension](/docs/reference/extensions.md) cannot contribute them.
 - **Raw events pane**: flip it on to inspect the event stream.
 - **Logs tab**: recent server log lines, polled from `GET /v1/logs`.
-- **A/Bs tab**: per-session and aggregate
-  [live A/B metrics](/docs/ab.md) from `GET /v1/abs`.
 
 In multi-agent mode each agent has its own playground at
 `/<slug>/playground`, and `/` is an index of them all.
@@ -9360,8 +9566,6 @@ Continue with these pages:
 - [Sessions and streaming](/docs/reference/sessions.md): the streams it renders
 - [Human-in-the-loop](/docs/guides/human-in-the-loop.md): the approval
   buttons in context
-- [Live A/B metrics](/docs/ab.md): the assignments and results in the A/Bs
-  tab
 
 ---
 
@@ -9375,8 +9579,7 @@ how the Agent SDK loads it.
 
 ## Folder structure
 
-For the capabilities below, identity usually comes from the path. A/B
-experiments can override their file-derived name.
+For the capabilities below, identity comes from the path.
 
 | Path | Resolves to |
 | --- | --- |
@@ -9388,8 +9591,6 @@ experiments can override their file-derived name.
 | `agent/extensions/ci.ts` | extension mount `ci`; its contributions become `ci__<name>` |
 | `agent/extensions/notion.ts` | Cursor plugin mount `notion` (`cursorPlugin`); its skills, agents, and MCP servers become `notion__<name>` |
 | `agent/channels/drive.ts` | channel `drive`, routes under `/v1/channels/drive` |
-| `agent/ab.ts` | A/B experiment `ab` unless `name` overrides it |
-| `agent/ab/concise.ts` | A/B experiment `concise` unless `name` overrides it |
 
 The root agent takes its name from `package.json` `name`, falling back
 to the directory name. When serving multiple agents, the slug is the
@@ -9441,8 +9642,6 @@ Each path maps to a capability and a reference page.
 | `agent/channels/*.ts` | HTTP surfaces beyond the built-in session API; `slack.ts` and `github.ts` use the platform packs | [Channels](/docs/reference/channels.md) |
 | `agent/hooks/*.ts` | Observe-only event subscribers, never fatal | [Hooks](/docs/reference/hooks.md) |
 | `agent/otel.ts` | `defineOtel` OTLP export (traces, metrics, optional logs) | [OpenTelemetry](/docs/guides/opentelemetry.md) |
-| `agent/ab.ts`, `agent/ab/*.ts` | `defineAB` experiments with sticky variants and live metrics | [Live A/B metrics](/docs/ab.md) |
-| `agent/ab.config.ts` | `defineABConfig` shared A/B settings | [Live A/B metrics](/docs/ab.md) |
 | `agent/storage.ts` | `defineStorage` backend for the durable `host.kv` / `host.files` APIs | [Storage](/docs/storage.md) |
 | `agent/artifacts.ts` | `defineArtifacts` kinds, the `tag_artifact` opt-in, and retention | [Artifacts](/docs/reference/artifacts.md) |
 | `agent/result.ts` | `defineResult` host `commit` on the final assistant text | [Turn result](/docs/reference/result.md) |
@@ -9479,8 +9678,6 @@ Continue with these pages:
 
 - [Agent config](/docs/reference/agent-config.md): the runtime config at the root
 - [Tools](/docs/reference/tools.md): add typed actions under `agent/tools/`
-- [Live A/B metrics](/docs/ab.md): compare variants from `agent/ab.ts` or
-  `agent/ab/`
 - [Concepts](/docs/concepts.md): why the filesystem is the interface
 
 ---
@@ -9924,7 +10121,7 @@ within one session. The `at` field is an ISO-8601 timestamp.
 
 | Phase | Events | What they tell you |
 | --- | --- | --- |
-| Session | `session.started`, [`ab.assigned`](/docs/ab.md#assign-sticky-variants), `session.waiting`, `session.completed`, `session.failed` | Session creation, A/B enrollment, readiness, and task completion |
+| Session | `session.started`, `session.waiting`, `session.completed`, `session.failed` | Session creation, readiness, and task completion |
 | Agent | `agent.bound` | Cloud conversation URL |
 | Input | `message.received` | A user message was accepted |
 | Turn | `turn.queued`, `turn.started`, `turn.completed`, `turn.failed` | Queue position under a [`maxRunningTurns` cap](/docs/reference/agent-config.md#concurrency), then turn status, final result, and token usage |
@@ -10008,7 +10205,6 @@ view.
 
 - [HTTP API](/docs/reference/http-api.md)
 - [Hooks](/docs/reference/hooks.md)
-- [Live A/B metrics](/docs/ab.md)
 - [How the Agent SDK works](/docs/concepts.md)
 
 ---
@@ -10661,61 +10857,6 @@ inputs again, and adds an eval for each improvement you keep.
 
 ---
 
-Source: /docs/skills/ab.md
-
-# Agent SDK A/B metrics (`defineAB`)
-
-Live metrics plug-in. No `agent-sdk ab` CLI. No assertion API.
-Reference: `docs/ab.md`.
-
-| | `defineEval` | `defineAB` |
-| --- | --- | --- |
-| Job | Gates on frozen fixtures | Metrics on live runs |
-| Location | `evals/**/*.eval.ts` | `agent/ab.ts` or `agent/ab/<name>.ts` |
-| How it runs | `agent-sdk eval` | Under `serve` / `run` |
-
-```ts
-import { defineAB, splitBySessionHash } from "@cursor/july/ab";
-
-export default defineAB({
-  name: "concise-instructions",
-  variants: {
-    control: { label: "Baseline" },
-    treatment: {
-      label: "Shorter",
-      instructions: "Keep replies to one short paragraph.",
-    },
-  },
-  split: splitBySessionHash({ holdout: 0.1 }),
-  derive: {
-    weatherCalls: (event) =>
-      event.type === "action.result" && event.data.toolName === "get_weather"
-        ? 1
-        : null,
-  },
-  onSample(sample) {
-    console.log(sample.variant, sample.metrics.toolCalls, sample.metrics.wallTimeMs);
-  },
-});
-```
-
-```ts
-async execute(input, ctx) {
-  if (ctx.session.abs?.["concise-instructions"] === "treatment") {
-    // treatment-specific behavior
-  }
-}
-```
-
-Enrollment is at session creation. Eval sessions skip it. Do not
-use `splitIf` to filter evals. Split helpers and `onSample`
-fields: `docs/ab.md`.
-
-Pick a name, arm labels, a split, and a real `onSample` sink. Do
-not invent credentials.
-
----
-
 Source: /docs/skills/create-agent.md
 
 # Create an Agent SDK agent
@@ -10922,15 +11063,182 @@ trace.
 
 ---
 
+Source: /docs/skills/deploy.md
+
+# Deploy an Agent SDK agent
+
+Use this skill only after a person asks for a deployment. It deploys a
+pushed Git ref to Cursor-managed hosting. Local files never upload.
+
+Use the exact published `@cursor/july` release embedded in the installed
+skill. Do not use a floating npm tag or a workspace build of the CLI.
+
+## Gather the target
+
+Resolve these values from the request and checkout:
+
+- Agent directory. Default to the current directory only when it contains
+  one Agent SDK project.
+- Deployment slug. Default to the normalized directory name.
+- Git ref. Default to the exact pushed `HEAD` commit.
+- Team. Use the service account's team unless the request names another.
+- Cursor-event repositories and CLI-only egress domains.
+
+Ask one focused question when the target or requested ref is ambiguous.
+Do not ask for values the checkout or existing deployment supplies.
+
+## Use the attached service account
+
+Continue only when `CURSOR_SERVICE_ACCOUNT_KEY` is present. Never print
+the value. Do not run `agent-sdk login`.
+
+On Linux and macOS, run every Agent SDK command through this wrapper:
+
+```bash
+CURSOR_JULY_VERSION="__CURSOR_JULY_VERSION__"
+case "$CURSOR_JULY_VERSION" in
+  __CURSOR_JULY_*__)
+    echo "The Agent SDK deploy skill is not bound to a package version." >&2
+    exit 1
+    ;;
+esac
+
+run_agent_sdk() {
+  env -u NODE_OPTIONS -u CURSOR_API_KEY \
+    CURSOR_API_KEY_FILE=/dev/null \
+    npx --yes "@cursor/july@$CURSOR_JULY_VERSION" "$@"
+}
+```
+
+The installed copy pins the release it came from. The wrapper isolates that
+published CLI and the attached service account. The presence check prevents a
+stored personal login from being used when the attachment is missing.
+
+Verify the principal before any write:
+
+```bash
+test -n "${CURSOR_SERVICE_ACCOUNT_KEY:-}" || {
+  echo "No attached Cursor service account." >&2
+  exit 1
+}
+run_agent_sdk whoami --json
+```
+
+Continue only when `credentialSource` is `service-account`. Record the
+team and complete service-account ID for the final report. Stop on an
+authentication, team-access, repository-scope, or hosting-entitlement
+error.
+
+## Pin pushed source
+
+Find the Git root and commit:
+
+```bash
+GIT_ROOT="$(git -C "$AGENT_DIR" rev-parse --show-toplevel)"
+SOURCE_REF="$(git -C "$GIT_ROOT" rev-parse HEAD)"
+git -C "$GIT_ROOT" status --short
+git -C "$GIT_ROOT" branch -r --contains "$SOURCE_REF"
+```
+
+Use a ref from the request instead of `SOURCE_REF` when the person names
+one. Confirm the selected commit exists on the remote. If intended
+changes are uncommitted or unpushed, stop and explain they will not ship.
+Do not commit or push unless the request includes that work.
+
+The CLI infers the normalized HTTPS `origin`, agent path, and slug from
+`--dir`. Pass `--repo`, `--path`, or `--slug` when the request overrides
+the inferred value. Never print a remote URL that contains credentials.
+
+## Preserve deployment inputs
+
+A redeploy replaces its source, Cursor-event repositories, and
+CLI-supplied egress domains. Preserve the existing
+`cursorEventRepos` and `egressAllowedDomains` unless the request or
+project changes them. Reapply each value with:
+
+```text
+--cursor-events-repo <owner/repo>
+--allow-domain <hostname>
+```
+
+Read existing settings through a filter that selects only those two
+fields:
+
+```bash
+run_agent_sdk deployment "$SLUG" --json |
+  node -e '
+    let raw = "";
+    process.stdin.setEncoding("utf8");
+    process.stdin.on("data", (chunk) => { raw += chunk; });
+    process.stdin.on("end", () => {
+      const value = JSON.parse(raw);
+      console.log(JSON.stringify({
+        cursorEventRepos: value.cursorEventRepos ?? [],
+        egressAllowedDomains: value.egressAllowedDomains ?? [],
+      }, null, 2));
+    });
+  '
+```
+
+Never print or save the full response because it can contain short-lived
+engine-access headers.
+
+For a new SCM-channel deployment, ask which repositories should wake the
+agent when the project does not declare the answer. The source repository
+is not always the event repository.
+
+## Validate, deploy, and verify
+
+Validate with the same published package and principal:
+
+```bash
+run_agent_sdk validate --dir "$AGENT_DIR"
+```
+
+Stop on validation errors. Review hosting warnings about secret names,
+egress domains, and channel configuration before continuing.
+
+Deploy the pinned source. Add the preserved or requested repeatable
+flags:
+
+```bash
+run_agent_sdk deploy \
+  --dir "$AGENT_DIR" \
+  --ref "$SOURCE_REF"
+```
+
+The command waits for a terminal deployment state. Do not treat
+`pending`, `accepted`, or `deploying` as success. If the command is
+interrupted, resume inspection with:
+
+```bash
+run_agent_sdk deployment "$SLUG"
+```
+
+A first deployment prints a one-time alias token. Never paste it into
+chat or expose it in an agent-captured terminal. Before a first deploy,
+ask for a secure destination or ask the person to run the final command
+in a private terminal. When writing the token, use mode `0600` and print
+only the path. Redeploys do not print the token. JSON deployment output
+can also contain credentials, so filter it before display.
+
+Finish only when the deployment reports `running`. Report the slug,
+team, generation, pinned ref, and complete service-account ID. On
+failure, report the deployment status and sanitized error without
+retrying a different ref or principal.
+
+---
+
 Source: /docs/skills/evals.md
 
 # Agent SDK evals
 
 Fixed input, model turn, gates on the trajectory. Files live at
 project-root `evals/**/*.eval.ts`. `agent/evals/` is ignored.
+Guide: `docs/evals.md`.
 
-Live traffic variants: `skills/ab/SKILL.md`. That is not a test
-runner.
+Observing production: hooks. Tool logic without a model:
+`agent-sdk call <tool> --input '{...}'`.
 
 ```bash
 agent-sdk eval --dir . --list
@@ -10944,9 +11252,13 @@ agent-sdk eval --dir . --tag smoke
 | `evals/weather.eval.ts` + `test` | `weather` |
 | `evals/weather/nyc.eval.ts` + `test` | `weather/nyc` |
 | `evals/weather.eval.ts` + `{ id: "nyc" }` | `weather/nyc` |
+| `evals/sql.eval.ts` exporting an array of `defineEval` calls | `sql/0000`, `sql/0001`, ... |
 
 `eval` boots an ephemeral server and a temp state root. `--url`
-points at a running agent. Model turns need a Cursor credential (`CURSOR_API_KEY`, then `CURSOR_API_KEY_FILE`, then `CURSOR_SERVICE_ACCOUNT_KEY`, then `agent-sdk login`).
+points at a running agent. Model turns need a Cursor credential
+(`CURSOR_API_KEY`, then `CURSOR_API_KEY_FILE`, then
+`CURSOR_SERVICE_ACCOUNT_KEY`, then `agent-sdk login`); `--list` does
+not. Node 22.13+, never Bun.
 
 ## Seeding
 
@@ -10996,9 +11308,30 @@ export default defineEvalConfig({
 });
 ```
 
-Either `test(t)` or `cases`, not both. `t.send` waits for park/fail.
-`workspaceFiles` seeds the first turn. Assert with `t.succeeded()`,
-`calledTool` / `notCalledTool`, `t.check(t.reply, …)`, `t.metric`.
+Either `test(t)` or `cases`, not both. `t.send` waits for the turn to
+settle (complete, park, or fail) and returns it; `turn.calledTool(...)`
+reads only that turn, `t.*` reads the whole run. First-send options:
+`workspaceFiles`, `workspaceDir`, `cloud`.
+
+Gates: `t.succeeded()` / `t.parked()`,
+`calledTool(name, { input, output, status, count })` / `notCalledTool`,
+`toolOrder`, `maxToolCalls`, `taggedArtifact`, `event` / `notEvent`,
+`t.check(value, includes | equals | matches | similarity | satisfies)`.
+Records: `t.metric`, `t.log`, `t.score` (soft 0-1).
+
+Severity is on the handle: default gate; `.soft()` tracked;
+`.atLeast(0.8)` soft with a bar (verdict `scored`, exit 1 only under
+`--strict`); `.gate(0.8)` hard. `t.judge.factuality`, `summarizes`,
+`closedQA`, and `sql` are soft by default and need a judge model
+(`judge` in `evals.config.ts`, on the eval or case, or per-call
+`{ model }`).
+
+## Side effects
+
+Eval sessions run real tools. Guard actuation with
+`ctx.session.purpose === "eval"` in the tool, hook, or `defineResult`
+and return a shaped result so `calledTool` can still assert the
+decision.
 
 ## What to gate
 
@@ -11020,6 +11353,18 @@ drifting inputs (pin them).
 | GitHub | `github replay … --dry-run --out fixtures/github` |
 | Host-prep PR review | A team-owned PR; gate findings shape, not counts |
 | Workspace | `workspaceFiles` in `t.send` |
+| Dataset | `loadJson` / `loadJsonl` / `loadYaml` from `@cursor/july/evals/loaders`, paths from the project root |
+
+## Debug and CI
+
+A failed local run leaves `evals/<case-id>.json` (assertions, inputs,
+tool I/O, final text, logs) under `evals/<stamp>/` in the project state
+directory (the CLI prints the path); read it before editing the eval.
+`--state-root <path>` keeps the raw session events too.
+
+CI: `agent-sdk eval --dir . --json --no-stream --junit reports/evals.xml`.
+Exit `1` on a failed gate, `2` when nothing matched, `--strict` to fail
+on `scored`.
 
 Every kept hillclimb change lands an eval that would have failed
 before it. Never weaken a gate to pass a round.
@@ -11073,7 +11418,6 @@ Path is identity. Full list: README "Folder structure".
 | `agent/hooks/*.ts` | Observe-only |
 | `agent/artifacts.ts` | Durable tagged outputs (`defineArtifacts`) |
 | `agent/result.ts` | Host `commit` on the final assistant text (`defineResult`) |
-| `agent/ab.ts` or `agent/ab/*.ts` | Live A/B (`defineAB`) |
 | `agent/otel.ts` | OpenTelemetry (`defineOtel`) |
 | `agent/schedules/*` | Cron. Never auto-fire under `--dev` |
 | `agent/sandbox/workspace/` | Session seed files (local only) |
@@ -11121,11 +11465,11 @@ only this agent's directory.
 | --- | --- |
 | Scaffold | `skills/create-agent/SKILL.md` |
 | Evals | `skills/evals/SKILL.md` |
-| Live A/B | `skills/ab/SKILL.md` |
 | OpenTelemetry | `skills/otel/SKILL.md` |
 | GitHub | `skills/github/SKILL.md` |
 | Slack | `skills/setup-slack/SKILL.md` |
 | Host MCP OAuth | `skills/mcp-auth/SKILL.md` |
+| Managed deployment | `skills/deploy/SKILL.md` |
 | Local triage | `skills/debug/SKILL.md` |
 | Measured improvement | `skills/hillclimb/SKILL.md` |
 
@@ -11305,12 +11649,12 @@ to use each one.
 | [framework-map](/docs/skills/framework-map.md) | Learn the project layout and runtimes |
 | [create-agent](/docs/skills/create-agent.md) | Scaffold and verify a new agent |
 | [evals](/docs/skills/evals.md) | Write fixtures and regression checks |
-| [ab](/docs/skills/ab.md) | Compare variants on live traffic |
 | [otel](/docs/skills/otel.md) | Export OpenTelemetry traces |
 | [hillclimb](/docs/skills/hillclimb.md) | Improve an agent against fixed inputs |
 | [github](/docs/skills/github.md) | Add GitHub webhooks and replay events |
 | [setup-slack](/docs/skills/setup-slack.md) | Connect an agent to Slack |
 | [mcp-auth](/docs/skills/mcp-auth.md) | Authorize host MCP OAuth |
+| [deploy](/docs/skills/deploy.md) | Deploy with an attached service account |
 | [debug](/docs/skills/debug.md) | Diagnose a local run |
 
 ---
@@ -11605,8 +11949,8 @@ Source: /docs/storage.md
 # Storage
 
 The Agent SDK owns durable storage for sessions, continuation tokens,
-reminders, playground eval history, and live A/B samples. It chooses the
-keys, when to read and write, and how to restore after restart.
+reminders, and playground eval history. It chooses the keys, when to
+read and write, and how to restore after restart.
 
 The Agent SDK owns key encoding. Backends must accept the keys they are
 given. Do not fail `put` to enforce a shorter cap.
@@ -11640,9 +11984,9 @@ export default defineStorage({
 ## Which fields to provide
 
 Implement the small KV core (`put`/`get`/`delete`/`list` plus the `cas`
-group) and you get **full functionality**: eval-run and A/B history are
-derived over the core automatically. The dedicated `evals` / `abs` groups
-are backend-native optimizations, not required-or-lose-history hooks.
+group) and you get **full functionality**: eval-run history is derived
+over the core automatically. The dedicated `evals` group is a
+backend-native optimization, not a required-or-lose-history hook.
 
 | Field | Required | Role |
 | --- | --- | --- |
@@ -11653,36 +11997,28 @@ are backend-native optimizations, not required-or-lose-history hooks.
 | `delete` | For cleanup | Remove a key |
 | `name` | No | Label surfaced on `GET /v1/info` diagnostics |
 | `policy` | No | Timing knobs; see [Policy](#policy) |
-| `evals` | No | Backend-native eval-runs table; derived over the core when omitted. See [Eval and A/B tables](#eval-and-a-b-tables) |
-| `abs` | No | Backend-native A/B metrics table; derived over the core when omitted. See [Eval and A/B tables](#eval-and-a-b-tables) |
+| `evals` | No | Backend-native eval-runs table; derived over the core when omitted. See [Eval table](#eval-table) |
 
 A throwing `put` is logged and dropped. It never fails a turn. When
 resolving a missing continuation token, a throwing `get` fails the
 follow-up so a store outage does not open a new session. Return
 `undefined` only for a real miss.
 
-## Eval and A/B tables
+## Eval table
 
-Two dedicated table groups carry structured rows instead of opaque KV
-values. Both are **optional optimizations**: when a group is not
-authored, `defineStorage` derives it over the KV core, so a backend that
-implements only the core loses nothing. Author a group only when the
-backend has a better native shape (a real database table, an analytics
-pipeline). The built-in `fileKv` and `cursorHostedStorage` both do.
+The dedicated `evals` table group carries structured rows instead of
+opaque KV values. It is an **optional optimization**: when the group is
+not authored, `defineStorage` derives it over the KV core, so a backend
+that implements only the core loses nothing. Author the group only when
+the backend has a better native shape (a real database table, an
+analytics pipeline). The built-in `fileKv` and `cursorHostedStorage`
+both do.
 
 `evals` keeps playground eval batches across restarts (`put`, `delete`,
 `list` over run snapshots keyed by `runId`). A core missing `delete` or
 `list` leaves eval history in memory until restart.
 See [Evals](/docs/evals.md#configure-eval-runs).
 
-`abs` exports live A/B metrics: `putSample` appends one cumulative
-metric sample per enrolled experiment on each completed or failed turn;
-optional `putSnapshot` / `getSnapshot` store and serve back the latest
-aggregate so a replacement host can still serve the A/Bs surface.
-`putSample` and `putSnapshot` need only core `put`; `getSnapshot` needs
-core `get`. Session event logs remain the assignment source of truth
-either way. See [Live A/B metrics](/docs/ab.md).
-
 ## Policy
 
 Two knobs change behavior:
@@ -11717,8 +12053,7 @@ With `get` and `list`, serve can rebuild local state from your store:
 - At startup, the Agent SDK loads recent sessions up to the restore caps.
 - On demand, a missing continuation token resolves through the store
   and resumes that session.
-- Playground eval history and A/B aggregates can load from the same
-  sink.
+- Playground eval history can load from the same sink.
 
 A turn in flight at crash time is not replayed. The next follow-up
 resumes from the last flushed state.