@cursor/bdk 0.2.1

package/docs/reference/connections.md

@@ -0,0 +1,277 @@
+---
+title: "MCP connections"
+description: "Pull in tools from MCP servers: remote, stdio, the signed-in Cursor account's connectors, and peer agents."
+---
+
+# MCP connections
+
+An MCP connection gives the agent tools from an MCP server. Define one
+file per server under `bot/mcp-connections/`; the filename becomes the
+server name. Default-export `defineConnection` from
+`@cursor/bdk/connections`. The transport is remote HTTP, local stdio,
+the signed-in Cursor account's connectors, or a peer agent on the same
+host.
+
+Put a server in `bot/host-connections/` when only host tools should
+call it. Host connections use the same `defineConnection` shape and
+support `bdk mcp oauth`; the model and playground don't see them.
+
+The `@cursor/bdk` imports and `bdk` commands on this page are docs-ahead
+until publication; see [BDK package availability](/#bdk-package-availability).
+
+## Remote MCP server
+
+Point an MCP connection at a remote server with a URL.
+
+```ts
+import { defineConnection } from "@cursor/bdk/connections";
+
+export default defineConnection({
+  url: "https://mcp.linear.app/mcp",
+  headers: { authorization: `Bearer ${process.env.LINEAR_TOKEN}` },
+});
+```
+
+Tokens come from env vars. Never hardcode them in the file.
+
+## Host MCP OAuth
+
+For servers that speak OAuth, set `oauth: true` and authorize with the
+CLI or mid-run Connect. Tokens live in `mcp-auth.json` under the CLI
+config directory. `--store` copies them onto the deployment as
+`MCP_OAUTH_<NAME>_*` secrets.
+
+```ts
+export default defineConnection({
+  url: "https://mcp.example.com/inventory",
+  oauth: true,
+});
+```
+
+```text
+bdk mcp oauth inventory           # browser PKCE → local mcp-auth.json
+bdk mcp oauth inventory --store   # also upsert deployment secrets
+```
+
+Full walkthrough: [Host MCP OAuth](../guides/mcp-oauth.md). Companion
+skill: [`skills/mcp-auth/SKILL.md`](../../skills/mcp-auth/SKILL.md).
+
+## Per-session auth
+
+For http/sse connections whose credential depends on who the session is
+for, declare an `auth` callback instead of static headers. It runs
+host-side with the session's `SessionInfo` and returns headers merged
+over the static ones:
+
+```ts
+export default defineConnection({
+  url: "https://api.cursor.com/v1/mcp/plugins",
+  auth: async (session) => ({
+    headers: { Authorization: `Bearer ${await grantFor(session)}` },
+  }),
+  advertiseTools: true,
+});
+```
+
+The callback is evaluated on every turn, including reminder
+fires and post-restart follow-ups, so the identity always comes from the
+session itself, never from state parked in memory. The model never sees a
+tenant parameter and can never choose the tenant. A callback that throws
+fails the turn: a turn never silently runs without the connection's
+identity. `host.mcp` calls from server tools keep the static headers only.
+You can't combine `auth` with `oauth: true`; the host OAuth provider owns
+the Authorization header.
+
+Derive the identity from durable session facts: `session.auth`,
+`session.id`, or your channel's own session state. Do not key it off
+`session.continuationKey`. The HTTP channel rotates that key after every
+accepted follow-up, so a tenant mapping keyed on it breaks
+mid-conversation.
+
+`auth` works attached or advertised. Advertise the auth connection
+instead of attaching it next to a stateful stdio server.
+
+## Advertise tools {#advertise-tools}
+
+Set `advertiseTools: true` when the model should call an MCP server's
+tools by name. BDK preserves each tool's description, input
+and output schemas, and MCP annotations. Exposed names normalize invalid
+characters and add numeric suffixes to avoid collisions.
+
+```ts
+export default defineConnection({
+  url: "https://api.cursor.com/v1/mcp/plugins",
+  auth: async (session) => ({ headers: await tenantHeaders(session) }),
+  advertiseTools: true,
+});
+```
+
+Listing, authentication, invalid tool name, and name collision failures
+fail the turn by default. Set `optional: true` to omit an unavailable
+connection instead. Advertised tools behave like server tools, and
+[direct tool calls](./tools.md#call-a-tool-without-a-model-turn) use the
+same names.
+
+In a dry-run session, MCP tools marked read-only run normally. Tools
+marked as writes are stubbed. Tools without effect annotations are
+unavailable.
+
+## Filter connection tools
+
+Use `tools` the same way you allowlist harness tools on the agent. When
+set, the connection serves only those names. Use `disallowedTools` to
+drop names instead. The two combine as deny-wins. The model and
+`host.mcp` only see what remains. A model-visible filter requires
+`advertiseTools: true`.
+
+```ts
+export default defineConnection({
+  url: "https://mcp.example.com/inventory",
+  advertiseTools: true,
+  tools: ["search_skus", "get_stock"],
+});
+```
+
+```ts
+export default defineConnection({
+  url: "https://mcp.example.com/inventory",
+  advertiseTools: true,
+  disallowedTools: ["place_order", "delete_sku"],
+});
+```
+
+Names are the server's `tools/list` names. Unknown names are omitted. A
+filter that matches nothing on the server fails the turn unless
+`optional: true`. Combine with `effects: "read"` to keep only the listed
+tools the server classifies as reads.
+
+A list of names is an allowlist. An object of handlers authors TypeScript
+tools. On `host-connections/`, a name list restricts `host.mcp` without
+advertising.
+
+## Local stdio MCP server
+
+Run a local MCP server as a child process with `command`.
+
+```ts
+export default defineConnection({
+  command: "node",
+  args: ["--import", "tsx", "mcp/units-server.ts"],
+  // env, cwd
+});
+```
+
+This suits small purpose-built servers, like a `units` converter
+shipped next to the agent.
+
+To run TypeScript in the agent environment, author the tools on the
+connection instead. BDK packages authored tools as stdio MCP, so you
+write only `execute`.
+
+```ts
+export default defineConnection({
+  tools: {
+    probe_agent_tool: {
+      description: "Prove the tool ran in the agent environment.",
+      async execute({ label }) {
+        return { ok: true, label };
+      },
+    },
+  },
+});
+```
+
+## Cursor account MCP connection
+
+`{ cursorAccount: true }` forwards the MCP connectors the signed-in
+Cursor account already authorized (dashboard → MCP), such as Linear,
+Notion, and Slack. You don't configure tokens. Every tool runs on the
+Cursor backend with the account's stored OAuth credentials, so raw
+tokens never reach the serving host, session workspaces, or traces.
+
+By default the agent gets every connected HTTP/SSE connector on the
+account. Pass `servers: "*"` (or `["*"]`) for the same all-connectors
+behavior in an explicit form. Pass a name list when you want a smaller
+set.
+
+```ts
+// bot/mcp-connections/account.ts: every connected connector
+export default defineConnection({
+  cursorAccount: true,
+  advertiseTools: true,
+});
+```
+
+```ts
+// only Linear:
+export default defineConnection({
+  cursorAccount: true,
+  servers: ["Linear"],
+  advertiseTools: true,
+});
+```
+
+Name the file `account.ts`. `cursor.ts` collides with the IDE `cursor`
+MCP namespace. `advertiseTools: true` puts connector tools on local
+turns by name.
+
+The host must be signed in (`bdk login`, `CURSOR_API_KEY`, or
+`CURSOR_SERVICE_ACCOUNT_KEY`). `serve` fails at startup otherwise.
+
+Use a `{ command }` connection for stdio servers.
+
+> [!CAUTION]
+> Whoever can talk to the agent can drive these connectors, because they
+> are ordinary agent tools. `serve` refuses to start when
+> `--allow-anonymous` is combined with account MCP connections unless you
+> also pass `--allow-anonymous-cursor-account-mcp` (trusted boundary only;
+> for example an SSO proxy or the hosted alias token). Prefer
+> `--bearer-token` on shared hosts.
+
+## Peer MCP connection {#peer-mcp-connection}
+
+`{ agent: "<slug>" }` addresses another agent mounted on the same serve
+host. The model gets the peer's `ask` and `check` tools, plus
+`call_tool` when the peer has server tools, and can delegate work to it:
+
+```ts
+export default defineConnection({
+  agent: "weather-agent",
+  description: "Delegate weather questions to the weather agent.",
+});
+```
+
+Unknown slugs and self-references fail `serve` at startup. Walkthrough:
+[Peer agents](../guides/agent-to-agent.md#delegate-a-question-to-a-specialist).
+
+## Call MCP from host code
+
+A file under `bot/mcp-connections/` is available to the Cursor agent,
+to server tools through `ctx.host.mcp`, and to channel and schedule
+handlers through `args.host.mcp`. Host connections skip the model.
+
+```ts
+// bot/tools/search_linear.ts
+import { defineTool } from "@cursor/bdk/tools";
+import { z } from "zod";
+
+export default defineTool({
+  description: "Search Linear issues.",
+  inputSchema: z.object({ query: z.string() }),
+  async execute({ query }, ctx) {
+    return ctx.host.mcp.callTool("linear", "list_issues", { query });
+  },
+});
+```
+
+`host.mcp.names()` lists connection names.
+`listTools(name)` / `callTool(name, tool, args)` call into a named
+connection.
+
+## Related
+
+- [Host MCP OAuth](../guides/mcp-oauth.md): `mcp oauth`, Connect, `--store`
+- [Tools](./tools.md): authored tools that wrap MCP connections
+- [Webhooks](../guides/webhooks.md): calling MCP connections from handlers
+- [Peer agents](../guides/agent-to-agent.md): when a specialist is its
+  own agent

package/docs/reference/evals.md

@@ -0,0 +1,292 @@
+---
+title: "Evals"
+description: "Define eval cases, drive turns, assert trajectories, and interpret results."
+---
+
+# Evals
+
+Evals run fixed cases against an agent and record whether its turns,
+tools, events, and output meet a contract. Files under the project-root
+`evals/` directory define cases with `@cursor/bdk/evals`; the runner
+discovers their IDs, executes them, and reports every assertion. See the
+[Evals guide](../evals.md) for the regression workflow.
+
+The `@cursor/bdk` imports and `bdk` commands on this page are docs-ahead
+until publication; see [BDK package availability](/#bdk-package-availability).
+
+## Eval discovery and case IDs
+
+Eval files live under the project-root `evals/` directory and end in
+`.eval.ts` or `.eval.js`. The path under that directory becomes the
+eval ID:
+
+- `evals/readiness.eval.ts` becomes `readiness`.
+- `evals/builds/api.eval.ts` becomes `builds/api`.
+- `evals/builds/index.eval.ts` becomes `builds`.
+
+Export one `defineEval` with either `test` or `cases`, not both. A case
+ID appends one path segment to the file ID:
+
+```ts
+export default defineEval({
+  tags: ["smoke"],
+  cases: [
+    {
+      id: "checkout",
+      async test(t) {
+        await t.send("Review checkout PR 42.");
+        t.succeeded();
+      },
+    },
+    {
+      id: "search",
+      async test(t) {
+        await t.send("Review search PR 7.");
+        t.succeeded();
+      },
+    },
+  ],
+});
+```
+
+Those cases are `prs/checkout` and `prs/search` when the file is
+`evals/prs.eval.ts`. Case IDs must be unique single path segments.
+
+A file can also export an array of `defineEval` calls. The runner names
+them with zero-padded indexes such as `sql/0000`. Named `cases` give
+handwritten scenarios stable IDs; arrays fit loaded datasets.
+
+`iterations` repeats one datapoint from 1 to 100 times. For
+`iterations: 3`, `weather/nyc` expands to `weather/nyc/1`,
+`weather/nyc/2`, and `weather/nyc/3`; selecting `weather/nyc` runs all
+three.
+
+## Eval configuration
+
+Every running suite needs `evals/evals.config.ts` with
+`maxConcurrency`:
+
+```ts
+import { defineEvalConfig } from "@cursor/bdk/evals";
+
+export default defineEvalConfig({
+  maxConcurrency: 20,
+  timeoutMs: 180_000,
+  judge: { model: "gpt-5.4-mini" },
+});
+```
+
+| Option | Contract |
+| --- | --- |
+| `maxConcurrency` | Required; concurrent authored datapoints, from 1 to 200 |
+| `timeoutMs` | Per-case timeout; case/file, CLI, then config precedence |
+| `judge` | Default model for `t.judge` |
+| `reporters` | Objects notified as cases and runs complete |
+| `maxPlaygroundRuns` | Number of server-side batches kept in playground history; defaults to 20 |
+
+A case can override `description`, `tags`, `timeoutMs`, `iterations`,
+`judge`, `reporters`, and `metadata`. Case metadata merges over
+file-level metadata; case reporters add to the file's reporters.
+
+## Send turns in a case
+
+`await t.send(message, options?)` runs one turn and waits until it
+finishes, fails, or parks for approval. Several sends in one test share
+the session.
+
+The returned turn exposes its assistant `message`, `sessionId`,
+`events`, ordered `toolCalls`, `ok`, and one-based `index`. Assertions
+on the turn inspect only that turn; assertions on `t` inspect the whole
+case. Run values include `t.reply`, `t.events`, `t.turns`,
+`t.sessionId`, `t.iteration`, `t.iterations`, and the timeout
+`t.signal`.
+
+These options apply on the first send because they shape the session:
+
+| Option | Contract |
+| --- | --- |
+| `workspaceFiles` | Relative path-to-content map seeded into the session workspace |
+| `workspaceDir` | Absolute working directory for a new local session |
+
+Use `turn.expectOk()` when later test steps depend on the turn
+succeeding. It throws when the turn failed.
+
+## Trajectory assertions
+
+Assertions record failures without stopping the test, so one case
+reports every violated contract.
+
+| Assertion | Checks |
+| --- | --- |
+| `t.succeeded()` | Run finished without failure or an unanswered approval |
+| `t.parked()` | Run stopped on an unanswered approval |
+| `t.calledTool(name, matcher?)` | Matching tool request occurred |
+| `t.notCalledTool(name)` | No request for that tool occurred |
+| `t.loadedSkill(name)` | Agent opened a named skill |
+| `t.toolOrder(names)` | Tool requests appeared in relative order |
+| `t.usedNoTools()` | No tool was requested |
+| `t.maxToolCalls(max)` | Tool-call count stayed under the bound |
+| `t.noFailedActions()` | No tool result failed |
+| `t.calledSubagent(name, matcher?)` | Matching subagent call occurred |
+| `t.taggedArtifact(kind?, predicate?)` | Matching durable artifact was tagged |
+| `t.event(type, matcher?)` | Matching session event occurred |
+| `t.notEvent(type, matcher?)` | Matching session event did not occur |
+| `t.eventOrder(matchers)` | Event groups appeared in relative order |
+| `t.eventsSatisfy(label, predicate)` | Predicate accepted the event stream |
+| `t.check(value, expectation)` | Value met an expectation builder |
+| `t.score(name, value)` | Recorded a numeric score from 0 to 1 |
+
+`t.requireToolCall`, `t.requireInputRequest`, and `t.require` return the
+matched value and stop the remaining test body when no match exists.
+
+## Tool and event matchers
+
+Tool matchers can narrow by `input`, `output`, `status`, and `count`.
+Values accept literals, regular expressions, or predicates. Object
+literals partial-deep-match:
+
+```ts
+t.calledTool("inspect_pr", {
+  input: { verdict: "review" },
+  status: "completed",
+  count: 1,
+});
+```
+
+Tool status is `completed`, `failed`, `pending`, or `rejected`.
+Subagent matchers accept `output`, `status`, `count`, and `callId`.
+Event matchers accept `data` and `count`.
+
+## Expectation builders
+
+Import builders from `@cursor/bdk/evals`:
+
+| Builder | Default severity |
+| --- | --- |
+| `includes(string | RegExp)` | Hard gate |
+| `equals(value)` | Hard gate |
+| `matches(schema)` | Hard gate |
+| `satisfies(predicate, label)` | Hard gate |
+| `similarity(expected)` | Soft score |
+
+`matches` accepts Standard Schema implementations such as Zod.
+`normalizedSimilarity(value, expected)` returns the same 0-to-1 value
+used by `similarity`.
+
+## Severity and verdicts
+
+Every assertion returns a severity handle:
+
+```ts
+t.succeeded();
+t.calledTool("search").soft();
+t.check(t.reply, similarity("Expected answer")).atLeast(0.8);
+t.judge.closedQA("names the failing step").gate(1);
+```
+
+- `.gate(threshold?)` fails the case when it misses.
+- `.soft(threshold?)` records a value without failing.
+- `.atLeast(threshold)` records a soft bar; a miss produces a `scored`
+  verdict.
+
+Case verdicts are `passed`, `failed`, `scored`, or `skipped`. The CLI
+exits 1 for a failed gate. A scored result exits 0 unless `--strict` is
+set. Exit 2 means the selection matched no cases.
+
+## Judges
+
+Use a judge for meaning that deterministic checks cannot express:
+
+```ts
+t.judge.factuality(expected).atLeast(0.8);
+t.judge.summarizes(expected).atLeast(0.8);
+t.judge.closedQA("names the root cause").gate(1);
+t.judge.sql(expected).atLeast(0.9);
+```
+
+Each grades `t.reply` by default; pass `{ on }` to grade another value.
+Judge configuration can live in the project config, eval, case, or
+call, with the nearest value winning.
+
+`t.judge.model(prompt)` returns a raw model reply for custom grading.
+Treat agent output as untrusted input: wrap it with `fenceUntrusted` and
+include `EVAL_JUDGE_INJECTION_GUARD` in the judge prompt.
+
+## Datasets and fixtures
+
+Load committed JSON, JSONL, or YAML relative to the project root:
+
+```ts
+import { defineEval, includes } from "@cursor/bdk/evals";
+import { loadYaml } from "@cursor/bdk/evals/loaders";
+
+const rows = await loadYaml<
+  Array<{ task: string; prompt: string; expected: string }>
+>("evals/data/cases.yaml");
+
+export default rows.map(row =>
+  defineEval({
+    description: row.task,
+    async test(t) {
+      await t.send(row.prompt);
+      t.succeeded();
+      t.check(t.reply, includes(row.expected));
+    },
+  })
+);
+```
+
+## Reporters and results
+
+Built-in reporters include `JUnit({ filePath, suiteName? })` and
+`Artifacts({ dir })`. Custom reporters can expose `onRunStart`,
+`onEvalComplete`, and `onRunComplete`.
+
+JSON output contains run totals and one result per case. Results include
+the case ID, verdict, assertions, session ID, inputs, final text, tools,
+tool calls, metrics, logs, duration, metadata, tags, and any error or
+skip reason.
+
+## Select cases
+
+```text
+bdk eval --dir . --list
+bdk eval --dir .
+bdk eval --dir . builds/checkout
+bdk eval --dir . --tag smoke
+```
+
+ID filters match an exact ID and its descendants. Repeated IDs use OR;
+repeated tags use OR. When both are present, a case must match both
+groups.
+
+## Inspect run artifacts
+
+Default run artifacts go under `evals/<stamp>/` in the project state
+directory and include
+`summary.json`, `results.jsonl`, and `evals/<case-id>.json`.
+`--artifacts <dir>` chooses another destination; `--no-artifacts`
+disables them. This location is independent of `--state-root`.
+
+## Run evals against a server
+
+The playground **Evals** view starts batches on its running server.
+`--url` targets a named running server, and `--prod --slug <slug>`
+targets a hosted deployment.
+
+```text
+bdk eval --prod --slug vulnerability-scanner --tag smoke
+bdk eval status <eval-id> --prod --slug vulnerability-scanner
+bdk eval cancel <eval-id> --prod --slug vulnerability-scanner
+```
+
+Server-side batches use the target's discovered cases and return an eval
+ID for `status` or `cancel`. Local reporter and concurrency flags don't
+apply to `--url` or `--prod` runs.
+
+## Related
+
+- [Evals guide](../evals.md)
+- [CLI reference](./cli.md#eval)
+- [Sessions](./sessions.md)
+- [Artifacts](./artifacts.md)