@cursor/bdk 0.2.1

package/dist/docs/deployment.md

@@ -0,0 +1,211 @@
+# Deploy a BDK project
+
+Deployment turns a pushed Grok Bot Development Kit (BDK) project into a stable service.
+Cursor-managed hosting builds the selected Git ref and operates the
+runtime; self-hosting gives you control of the network, process, and
+filesystem.
+
+The `@cursor/bdk` imports and `bdk` commands in these examples are
+docs-ahead until publication; see
+[BDK package availability](/docs/index.md#bdk-package-availability).
+
+## Choose managed hosting or self-hosting
+
+| Option | Use it when | You manage |
+| --- | --- | --- |
+| Cursor-managed hosting | You want the shortest path from Git to a running agent | Source, declared egress, secrets, and agent behavior |
+| Self-hosting | You need your own network, proxy, filesystem, or process controls | The Node process, TLS, auth, secrets, state, monitoring, and upgrades |
+
+The rest of this guide starts with managed hosting. The
+[self-host recipe](#self-host-securely) uses the same project and HTTP
+API.
+
+## Deploy from Git
+
+Commit and push the exact ref you want to run, then sign in, validate,
+and deploy. The preferred CLI is `bdk`.
+
+> **Docs ahead:** `@cursor/bdk` is not on npm yet. This block shows future
+> CLI syntax and will not run until the package publishes.
+
+```text
+cd my-agent
+bdk login
+bdk whoami
+bdk validate
+bdk deploy
+```
+
+The deploy command waits until the engine is running and prints the stable
+deployment URL.
+
+`deploy` reads from Git; it does not upload your working tree. Inside a
+checkout, it infers the HTTPS origin, current ref, agent path, and slug.
+Pass them explicitly when the checkout is not the source you want:
+
+```text
+bdk deploy \
+  --repo https://github.com/acme/agents \
+  --ref main \
+  --path agents/weather \
+  --slug weather-agent
+```
+
+The source repository must be available through the team's repository
+integration, and a nested `--path` must contain an installable BDK
+project.
+
+## Add egress and secrets
+
+Declare the external hosts and secret names the agent needs. After the
+deployment exists, set each value at the hidden prompt and follow the
+CLI or dashboard instruction to redeploy.
+
+```ts
+// bot/agent.ts
+import { defineAgent } from "@cursor/bdk";
+
+export default defineAgent({
+  hosting: {
+    egressDomains: ["api.weather.example.com"],
+    secretNames: ["WEATHER_API_KEY"],
+  },
+});
+```
+
+```text
+bdk secrets set weather-agent WEATHER_API_KEY
+bdk secrets list weather-agent
+bdk deploy --dir . --slug weather-agent
+```
+
+Commit secret names, never values. `secrets set` accepts names and reads
+values without echoing them; `secrets list` returns names and creation
+times only. Do not pass `NAME=VALUE`, print a secret to verify it, or
+put names beginning with reserved `CURSOR_` in `secretNames`.
+
+Enter egress hosts without schemes, ports, or paths. The
+[Host MCP OAuth guide](/docs/guides/mcp-oauth.md) owns the extra secret and
+egress steps for OAuth-backed MCP servers.
+
+## Use the hosted agent
+
+The `--prod` commands authenticate through your Cursor credential and
+connect to the deployment directly. Open the playground, run a turn,
+then inspect the same session in logs.
+
+```text
+bdk playground --prod --slug weather-agent
+bdk run --prod --slug weather-agent \
+  --message "Forecast for Paris"
+bdk logs --prod --slug weather-agent
+bdk deployment weather-agent
+```
+
+External HTTP clients use the stable alias URL and send the alias token
+on every request:
+
+```bash
+curl "$AGENT_ALIAS_URL/v1/health" \
+  -H "X-Agent-Alias-Token: $AGENT_ALIAS_TOKEN"
+```
+
+The first deploy prints this token once. Store it outside terminal
+transcripts; it cannot be retrieved later. If it is lost or exposed,
+`bdk rotate-token weather-agent` replaces it and invalidates the
+old value.
+
+`deployment --json` can include short-lived access headers, so treat
+that output as a credential too.
+
+For a custom webhook whose provider cannot send the alias token, use
+`publicEndpoint()` only when the channel verifies the provider
+signature. That opens the custom channel path, not the built-in session
+or tool APIs. See [Webhooks](/docs/guides/webhooks.md).
+
+## Update the deployment
+
+Push the new ref and deploy the same slug. The stable URL remains while
+the new generation replaces the previous one.
+
+```text
+bdk deploy --dir . --slug weather-agent
+```
+
+A redeploy replaces its source, Cursor-event repository list, and any
+egress domains supplied through CLI flags. Repass
+`--cursor-events-repo` and other CLI-only settings each time. Keep
+source-controlled egress in the `hosting` block when possible.
+
+Use `bdk deployment weather-agent` and
+`bdk logs --prod --slug weather-agent` to inspect a rollout.
+The [CLI reference](/docs/reference/cli.md#deploy) owns `--no-wait`,
+multi-agent deployment, stop, and delete behavior.
+
+## Keep state across replacements
+
+Managed hosting persists framework-owned session, KV, and file state
+across normal runtime replacements. Files written elsewhere on the
+engine's local filesystem are disposable; use `ctx.host.kv` or
+`ctx.host.files` for durable application state.
+
+Parked approvals and in-memory reminder handlers do not survive a
+restart. Run the turn again or recreate the reminder after a
+replacement.
+
+## Self-host securely
+
+Run BDK under Node 22.13 or newer, never Bun. One supervised
+process can serve every agent under a directory while a persistent
+state root keeps sessions across restarts.
+
+```text
+export CURSOR_API_KEY="<cursor-api-key>"
+export BDK_BEARER_TOKEN="$(openssl rand -hex 32)"
+
+bdk serve \
+  --dir /srv/agents \
+  --host 127.0.0.1 \
+  --port 3000 \
+  --state-root /var/lib/bdk \
+  --bearer-token "$BDK_BEARER_TOKEN"
+```
+
+Before exposing the host:
+
+- Put TLS or an authenticating proxy in front of the loopback server.
+
+- Keep `--bearer-token` on shared hosts. Use `--allow-anonymous` only
+  behind a proxy that authenticates every caller.
+
+- Mount `--state-root` on a persistent volume outside the agent
+  repository and back it up.
+
+- Inject credentials at startup instead of baking them into an image.
+
+- Run the process under a supervisor and enable schedules on exactly
+  one serving process per project.
+
+Containers use the same command with `--host 0.0.0.0`, a mounted state
+volume, and an authenticating network boundary. The
+[Channels reference](/docs/reference/channels.md#auth-policies) covers
+per-caller and channel-specific auth.
+
+## Observability
+
+Use `bdk logs --prod` for runtime output,
+[OpenTelemetry](/docs/guides/opentelemetry.md) for traces and metrics, and
+[session traces](/docs/reference/sessions.md#inspect-a-saved-event-stream)
+for one conversation.
+
+## Related
+
+- [CLI reference](/docs/reference/cli.md#deploy): inspect, update, stop,
+  delete, and rotate deployments
+- [GitHub](/docs/guides/github.md): Cursor-event repositories and direct
+  webhook delivery
+- [Slack](/docs/guides/slack.md): provision a dedicated hosted Slack app
+- [Host MCP OAuth](/docs/guides/mcp-oauth.md): persist OAuth credentials
+  across hosted replacements
+- [Troubleshooting](/docs/troubleshooting.md): diagnose failed or stale
+  deployments

package/dist/docs/evals.html

@@ -0,0 +1,131 @@
+<!DOCTYPE html>
+<html lang="en-US" dir="ltr">
+  <head>
+    <meta charset="utf-8">
+    <meta name="viewport" content="width=device-width,initial-scale=1">
+    <title>Evals | Grok Bot Development Kit (BDK)</title>
+    <meta name="description" content="Protect agent decisions with fixed inputs, trajectory assertions, and regression checks that run locally or in CI.">
+    <meta name="generator" content="VitePress v1.6.4">
+    <link rel="preload stylesheet" href="/docs/assets/style.C130MVnH.css" as="style">
+    <link rel="preload stylesheet" href="/docs/vp-icons.css" as="style">
+    
+    <script type="module" src="/docs/assets/app.JVQvbRO-.js"></script>
+    <link rel="preload" href="/docs/assets/inter-roman-latin.Di8DUHzh.woff2" as="font" type="font/woff2" crossorigin="">
+    <link rel="modulepreload" href="/docs/assets/chunks/framework.BNw1pucY.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/theme.zK3OJa_C.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/katex.DkKDou_j.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/dagre-KV5264BT.DMGGa3uA.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/cose-bilkent-S5V4N54A.Buf3seya.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/c4Diagram-AHTNJAMY.D0AsQFSX.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/flowDiagram-DWJPFMVM.6_Aky-ut.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/erDiagram-SMLLAGMA.DQT1Hnwf.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/gitGraphDiagram-UUTBAWPF.CGd_eJbn.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/ganttDiagram-T4ZO3ILL.DHIwqnqN.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/infoDiagram-42DDH7IO.Cd4gMyAz.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/pieDiagram-DEJITSTG.DfAl9sy4.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/quadrantDiagram-34T5L4WZ.CNuqFRBj.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/xychartDiagram-5P7HB3ND.e8BPicch.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/requirementDiagram-MS252O5E.DJ5CmTCl.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/sequenceDiagram-FGHM5R23.BRn9WcrR.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/classDiagram-6PBFFD2Q.NPiy3Hz7.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/classDiagram-v2-HSJHXN6E.NPiy3Hz7.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/stateDiagram-FHFEXIEX.F12urmm7.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2._auCa3fR.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/journeyDiagram-VCZTEJTY.AWLXtelc.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/timeline-definition-GMOUNBTQ.B7GiQBeC.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/mindmap-definition-QFDTVHPH.By_RAMaP.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/kanban-definition-6JOO6SKY.CagPhAn5.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/sankeyDiagram-XADWPNL6.D-T_AKJj.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/diagram-TYMM5635.Dw8W9BTk.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/diagram-MMDJMWI5.DRvQV16Z.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/blockDiagram-DXYQGD6D.CQXAOmqd.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/diagram-5BDNPKRD.DDjnwRmY.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/architectureDiagram-Q4EWVU46.CAY_TPVz.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/ishikawaDiagram-UXIWVN3A.D1uw7MXq.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/vennDiagram-DHZGUBPP.D0niwuY_.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/diagram-G4DWMVQ6.RwyrIr8Q.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/wardleyDiagram-NUSXRM2D.DKaqpT7K.js">
+    <link rel="modulepreload" href="/docs/assets/chunks/virtual_mermaid-config.DDnGl6nM.js">
+    <link rel="modulepreload" href="/docs/assets/evals.md.CCAtNcuL.lean.js">
+    <script id="check-dark-mode">(()=>{const e=localStorage.getItem("vitepress-theme-appearance")||"auto",a=window.matchMedia("(prefers-color-scheme: dark)").matches;(!e||e==="auto"?a:e==="dark")&&document.documentElement.classList.add("dark")})();</script>
+    <script id="check-mac-os">document.documentElement.classList.toggle("mac",/Mac|iPhone|iPod|iPad/i.test(navigator.platform));</script>
+    <link rel="alternate" type="text/plain" href="/docs/llms.txt">
+    <link rel="alternate" type="text/markdown" href="/docs/evals.md">
+  </head>
+  <body>
+    <div id="app"><div class="Layout" data-v-76086e96><!--[--><!--]--><!--[--><span tabindex="-1" data-v-97fb5898></span><a href="#VPContent" class="VPSkipLink visually-hidden" data-v-97fb5898>Skip to content</a><!--]--><!----><header class="VPNav" data-v-76086e96 data-v-781b1279><div class="VPNavBar" data-v-781b1279 data-v-b33015df><div class="wrapper" data-v-b33015df><div class="container" data-v-b33015df><div class="title" data-v-b33015df><div class="VPNavBarTitle has-sidebar" data-v-b33015df data-v-a0c466dc><a class="title" href="/docs/" data-v-a0c466dc><!--[--><!--]--><!----><span data-v-a0c466dc>Grok Bot Development Kit (BDK)</span><!--[--><!--[--><!--[--><!--[--><span class="bdk-version" title="BDK 0.2.1 (@cursor/bdk package)" data-v-d89e5038>0.2.1</span><!--]--><!--]--><!--]--><!--]--></a></div></div><div class="content" data-v-b33015df><div class="content-body" data-v-b33015df><!--[--><!--]--><div class="VPNavBarSearch search" data-v-b33015df><!--[--><!----><div id="local-search"><button type="button" class="DocSearch DocSearch-Button" aria-label="Search"><span class="DocSearch-Button-Container"><span class="vp-icon DocSearch-Search-Icon"></span><span class="DocSearch-Button-Placeholder">Search</span></span><span class="DocSearch-Button-Keys"><kbd class="DocSearch-Button-Key"></kbd><kbd class="DocSearch-Button-Key">K</kbd></span></button></div><!--]--></div><nav aria-labelledby="main-nav-aria-label" class="VPNavBarMenu menu" data-v-b33015df data-v-8906c751><span id="main-nav-aria-label" class="visually-hidden" data-v-8906c751> Main Navigation </span><!--[--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/quickstart.html" tabindex="0" data-v-8906c751 data-v-e9bcc08e><!--[--><span data-v-e9bcc08e>Quickstart</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/guides/mcp-oauth.html" tabindex="0" data-v-8906c751 data-v-e9bcc08e><!--[--><span data-v-e9bcc08e>Guides</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/guides/github.html" tabindex="0" data-v-8906c751 data-v-e9bcc08e><!--[--><span data-v-e9bcc08e>Integrations</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/templates/security-reviewer.html" tabindex="0" data-v-8906c751 data-v-e9bcc08e><!--[--><span data-v-e9bcc08e>Templates</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/reference/project-layout.html" tabindex="0" data-v-8906c751 data-v-e9bcc08e><!--[--><span data-v-e9bcc08e>Reference</span><!--]--></a><!--]--><!--]--></nav><!----><div class="VPNavBarAppearance appearance" data-v-b33015df data-v-959ed110><button class="VPSwitch VPSwitchAppearance" type="button" role="switch" title aria-checked="false" data-v-959ed110 data-v-79ad6d7f data-v-0a895de9><span class="check" data-v-0a895de9><span class="icon" data-v-0a895de9><!--[--><span class="vpi-sun sun" data-v-79ad6d7f></span><span class="vpi-moon moon" data-v-79ad6d7f></span><!--]--></span></span></button></div><!----><div class="VPFlyout VPNavBarExtra extra" data-v-b33015df data-v-a713f501 data-v-1d7aa189><button type="button" class="button" aria-haspopup="true" aria-expanded="false" aria-label="extra navigation" data-v-1d7aa189><span class="vpi-more-horizontal icon" data-v-1d7aa189></span></button><div class="menu" data-v-1d7aa189><div class="VPMenu" data-v-1d7aa189 data-v-e34b7e14><!----><!--[--><!--[--><!----><div class="group" data-v-a713f501><div class="item appearance" data-v-a713f501><p class="label" data-v-a713f501>Appearance</p><div class="appearance-action" data-v-a713f501><button class="VPSwitch VPSwitchAppearance" type="button" role="switch" title aria-checked="false" data-v-a713f501 data-v-79ad6d7f data-v-0a895de9><span class="check" data-v-0a895de9><span class="icon" data-v-0a895de9><!--[--><span class="vpi-sun sun" data-v-79ad6d7f></span><span class="vpi-moon moon" data-v-79ad6d7f></span><!--]--></span></span></button></div></div></div><!----><!--]--><!--]--></div></div></div><!--[--><!--]--><button type="button" class="VPNavBarHamburger hamburger" aria-label="mobile navigation" aria-expanded="false" aria-controls="VPNavScreen" data-v-b33015df data-v-cb5d0edb><span class="container" data-v-cb5d0edb><span class="top" data-v-cb5d0edb></span><span class="middle" data-v-cb5d0edb></span><span class="bottom" data-v-cb5d0edb></span></span></button></div></div></div></div><div class="divider" data-v-b33015df><div class="divider-line" data-v-b33015df></div></div></div><!----></header><div class="VPLocalNav has-sidebar empty" data-v-76086e96 data-v-6f794310><div class="container" data-v-6f794310><button class="menu" aria-expanded="false" aria-controls="VPSidebarNav" data-v-6f794310><span class="vpi-align-left menu-icon" data-v-6f794310></span><span class="menu-text" data-v-6f794310>Menu</span></button><div class="VPLocalNavOutlineDropdown" style="--vp-vh:0px;" data-v-6f794310 data-v-bab84d9c><button data-v-bab84d9c>Return to top</button><!----></div></div></div><aside class="VPSidebar" data-v-76086e96 data-v-4daa293d><div class="curtain" data-v-4daa293d></div><nav class="nav" id="VPSidebarNav" aria-labelledby="sidebar-aria-label" tabindex="-1" data-v-4daa293d><span class="visually-hidden" id="sidebar-aria-label" data-v-4daa293d> Sidebar Navigation </span><!--[--><!--]--><!--[--><div class="no-transition group" data-v-095feacd><section class="VPSidebarItem level-0" data-v-095feacd data-v-d79cbf7e><div class="item" role="button" tabindex="0" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><h2 class="text" data-v-d79cbf7e>Getting started</h2><!----></div><div class="items" data-v-d79cbf7e><!--[--><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Overview</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/quickstart.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Building your first agent</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-095feacd><section class="VPSidebarItem level-0 collapsible" data-v-095feacd data-v-d79cbf7e><div class="item" role="button" tabindex="0" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><h2 class="text" data-v-d79cbf7e>Integrations</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-d79cbf7e><span class="vpi-chevron-right caret-icon" data-v-d79cbf7e></span></div></div><div class="items" data-v-d79cbf7e><!--[--><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/github.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>GitHub</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/slack.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Slack</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/webhooks.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Webhooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/convert-automation.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Cursor Automations</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/opentelemetry.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>OpenTelemetry</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/gitlab.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>GitLab</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/bitbucket.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Bitbucket</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-095feacd><section class="VPSidebarItem level-0 collapsible" data-v-095feacd data-v-d79cbf7e><div class="item" role="button" tabindex="0" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><h2 class="text" data-v-d79cbf7e>Extensions</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-d79cbf7e><span class="vpi-chevron-right caret-icon" data-v-d79cbf7e></span></div></div><div class="items" data-v-d79cbf7e><!--[--><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/extensions.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Overview</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/cloud-agents.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Cloud agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/grokbot-agents.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Grok Bot agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/improve.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Self-improvement</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/jev.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Jev</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-095feacd><section class="VPSidebarItem level-0 collapsible has-active" data-v-095feacd data-v-d79cbf7e><div class="item" role="button" tabindex="0" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><h2 class="text" data-v-d79cbf7e>Guides</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-d79cbf7e><span class="vpi-chevron-right caret-icon" data-v-d79cbf7e></span></div></div><div class="items" data-v-d79cbf7e><!--[--><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/mcp-oauth.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>MCP</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/hooks.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Hooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/guides/agent-to-agent.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Peer agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/deployment.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Deployment</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/evals.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Evals</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/hillclimbing.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Hillclimbing</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/building-with-agents.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Build with a coding agent</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/troubleshooting.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Troubleshooting</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-095feacd><section class="VPSidebarItem level-0 collapsible" data-v-095feacd data-v-d79cbf7e><div class="item" role="button" tabindex="0" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><h2 class="text" data-v-d79cbf7e>Templates</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-d79cbf7e><span class="vpi-chevron-right caret-icon" data-v-d79cbf7e></span></div></div><div class="items" data-v-d79cbf7e><!--[--><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/templates/security-reviewer.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Security reviewer</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/templates/thermo-review.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Thermo review</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/templates/thermo-quality-review.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Thermo quality review</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/templates/agentic-owners.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Agentic Approver</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/templates/triage.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Triage agent</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-095feacd><section class="VPSidebarItem level-0 collapsible" data-v-095feacd data-v-d79cbf7e><div class="item" role="button" tabindex="0" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><h2 class="text" data-v-d79cbf7e>Reference</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-d79cbf7e><span class="vpi-chevron-right caret-icon" data-v-d79cbf7e></span></div></div><div class="items" data-v-d79cbf7e><!--[--><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/project-layout.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Project layout</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/agent-config.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Agent config</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/instructions.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Instructions</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/tools.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Tools</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/prompt.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>prompt</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/skills.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Skills</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/connections.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>MCP Connections</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/subagents.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Subagents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/channels.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Channels</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/schedules.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Schedules & reminders</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/hooks.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Hooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/evals.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Evals</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/artifacts.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Artifacts</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/sessions.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Sessions & streaming</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/playground.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>Playground</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/cli.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>CLI</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-d79cbf7e><div class="item" data-v-d79cbf7e><div class="indicator" data-v-d79cbf7e></div><a class="VPLink link link" href="/docs/reference/http-api.html" data-v-d79cbf7e><!--[--><p class="text" data-v-d79cbf7e>HTTP API</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><!--]--><!--[--><!--]--></nav></aside><div class="VPContent has-sidebar" id="VPContent" data-v-76086e96 data-v-258cbba4><div class="VPDoc has-sidebar has-aside" data-v-258cbba4 data-v-afe580da><!--[--><!--]--><div class="container" data-v-afe580da><div class="aside" data-v-afe580da><div class="aside-curtain" data-v-afe580da></div><div class="aside-container" data-v-afe580da><div class="aside-content" data-v-afe580da><div class="VPDocAside" data-v-afe580da data-v-fb5275a4><!--[--><!--]--><!--[--><!--]--><nav aria-labelledby="doc-outline-aria-label" class="VPDocAsideOutline" data-v-fb5275a4 data-v-b3783f18><div class="content" data-v-b3783f18><div class="outline-marker" data-v-b3783f18></div><div aria-level="2" class="outline-title" id="doc-outline-aria-label" role="heading" data-v-b3783f18>On this page</div><ul class="VPDocOutlineItem root" data-v-b3783f18 data-v-72702c6e><!--[--><!--]--></ul></div></nav><!--[--><!--]--><div class="spacer" data-v-fb5275a4></div><!--[--><!--]--><!----><!--[--><!--]--><!--[--><!--]--></div></div></div></div><div class="content" data-v-afe580da><div class="content-container" data-v-afe580da><!--[--><!--]--><main class="main" data-v-afe580da><div style="position:relative;" class="vp-doc _docs_evals" data-v-afe580da><div><h1 id="evals" tabindex="-1">Evals <a class="header-anchor" href="#evals" aria-label="Permalink to &quot;Evals&quot;">​</a></h1><p>An eval sends a fixed input to the real agent and checks the resulting trajectory: whether the turn succeeded, which tools ran, and what shape the answer took. Use evals to protect behavior you already understand, not to discover what the prompt should do.</p><p>Evals run real model turns and real tools. Guard external writes before running a suite against an agent that can post, merge, or deploy.</p><p>The <code>@cursor/bdk</code> imports and <code>bdk</code> commands on this page are docs-ahead until publication; see <a href="/docs/#bdk-package-availability">BDK package availability</a>.</p><h2 id="protect-a-tool-decision" tabindex="-1">Protect a tool decision <a class="header-anchor" href="#protect-a-tool-decision" aria-label="Permalink to &quot;Protect a tool decision&quot;">​</a></h2><p>This smoke case asks for a PR verdict, requires the read-only inspection tool, and fails if the agent tries to approve. The CLI reports all three decisions together instead of stopping at the first miss.</p><p>Use the <a href="./guides/jev.html">Jev extension</a> when host code needs a typed choice, score, or boolean before it writes.</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">// evals/readiness.eval.ts</span></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { defineEval, includes } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;@cursor/bdk/evals&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
+<span class="line"></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">export</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> default</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> defineEval</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">({</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  tags: [</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;smoke&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">],</span></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">  async</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> test</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#E36209;--shiki-dark:#FFAB70;">t</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">) {</span></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">    await</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">send</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span></span>
+<span class="line"><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">      &quot;Is https://github.com/acme/checkout/pull/42 ready to approve?&quot;</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    );</span></span>
+<span class="line"></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">succeeded</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">();</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">calledTool</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;inspect_pr&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">notCalledTool</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;approve_pr&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">check</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(t.reply, </span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">includes</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">/</span><span style="--shiki-light:#032F62;--shiki-dark:#DBEDFF;">ready</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">|</span><span style="--shiki-light:#032F62;--shiki-dark:#DBEDFF;">blocked</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">|</span><span style="--shiki-light:#032F62;--shiki-dark:#DBEDFF;">approve</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">/</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">i</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">));</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  },</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><p>Evals live in the project-root <code>evals/</code> directory, beside <code>bot/</code>. Add the required concurrency config once:</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">// evals/evals.config.ts</span></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { defineEvalConfig } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;@cursor/bdk/evals&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
+<span class="line"></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">export</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> default</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> defineEvalConfig</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">({</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  maxConcurrency: </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">20</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>bdk eval --dir . --list</span></span>
+<span class="line"><span>bdk eval --dir . --tag smoke</span></span></code></pre></div><h2 id="check-answer-shape-or-quality" tabindex="-1">Check answer shape or quality <a class="header-anchor" href="#check-answer-shape-or-quality" aria-label="Permalink to &quot;Check answer shape or quality&quot;">​</a></h2><p>Prefer a deterministic shape check when the output has a contract. The failure tells you which field was wrong, and the case remains stable when the model changes its wording.</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { matches } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;@cursor/bdk/evals&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { z } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;zod&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
+<span class="line"></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">const</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> verdict</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> =</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> JSON</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">parse</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(t.reply </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">??</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;{}&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">check</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  verdict,</span></span>
+<span class="line"><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">  matches</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    z.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">object</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">({</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">      ready: z.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">boolean</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(),</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">      blockers: z.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">array</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(z.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">string</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">()),</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    })</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  )</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span></code></pre></div><p>Use a judge when correctness depends on meaning that a schema, regex, or known value cannot capture:</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">t.judge</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  .</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">factuality</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;The lint step failed on src/sidebar.ts.&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">)</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  .</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">atLeast</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">0.8</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span></code></pre></div><p>Judge checks are tracked scores until you give them a hard gate. Configure the judge model in <code>evals.config.ts</code>. The <a href="./reference/evals.html#judges">Evals reference</a> owns grader options and severity rules.</p><h2 id="test-a-conversation-or-approval" tabindex="-1">Test a conversation or approval <a class="header-anchor" href="#test-a-conversation-or-approval" aria-label="Permalink to &quot;Test a conversation or approval&quot;">​</a></h2><p>Several <code>t.send</code> calls in one case share a session. Assertions on a returned turn inspect only that turn, while assertions on <code>t</code> inspect the whole conversation.</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">const</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> first</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> =</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> await</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">send</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;Inspect PR 42.&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">first.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">calledTool</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;inspect_pr&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
+<span class="line"></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">const</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> followUp</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> =</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> await</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">send</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;Now summarize only the blockers.&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">followUp.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">usedNoTools</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">();</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">check</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(followUp.message, </span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">includes</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">/</span><span style="--shiki-light:#032F62;--shiki-dark:#DBEDFF;">blocker</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">/</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">i</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">));</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">succeeded</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">();</span></span></code></pre></div><p>For a tool that requires approval, the expected result is a parked turn instead of a completed one:</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">await</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">send</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;Apply the approved policy update.&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">parked</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">();</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">calledTool</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;apply_policy&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">, {</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  input: { verdict: </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;update&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> },</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><p>Pair an approval case with a safe branch that calls <code>t.notCalledTool(&quot;apply_policy&quot;)</code>, so both sides of the decision stay protected.</p><h2 id="pin-realistic-inputs" tabindex="-1">Pin realistic inputs <a class="header-anchor" href="#pin-realistic-inputs" aria-label="Permalink to &quot;Pin realistic inputs&quot;">​</a></h2><p>A useful eval changes only when the agent changes. Freeze canonical chat prompts, seed workspace evidence directly, and save webhook payloads instead of depending on a developer&#39;s checkout or live external state.</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">await</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">send</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;Review pr/diff.patch and report blockers.&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">, {</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  workspaceFiles: {</span></span>
+<span class="line"><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">    &quot;pr/diff.patch&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">: [</span></span>
+<span class="line"><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">      &quot;diff --git a/app/routes/search.ts b/app/routes/search.ts&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
+<span class="line"><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">      &quot;+res.send(`&lt;h1&gt;Results for ${req.query.q}&lt;/h1&gt;`);&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    ].</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">join</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">\n</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">),</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  },</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><p>For GitHub agents, snapshot a PR into committed fixtures:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>bdk github replay acme/checkout#42 \</span></span>
+<span class="line"><span>  --events &#39;*&#39; \</span></span>
+<span class="line"><span>  --dry-run \</span></span>
+<span class="line"><span>  --out fixtures/github</span></span></code></pre></div><p>Keep a small, fast <code>smoke</code> set for every change. Put larger datasets and drift-prone live cases behind an explicit selection. See <a href="./reference/evals.html#datasets-and-fixtures">Datasets and fixtures</a> for loaders and case expansion.</p><h2 id="run-and-debug-a-regression-suite" tabindex="-1">Run and debug a regression suite <a class="header-anchor" href="#run-and-debug-a-regression-suite" aria-label="Permalink to &quot;Run and debug a regression suite&quot;">​</a></h2><p>Run one case while iterating, then the smoke suite before review. A failure artifact records the input, assertions, tool calls, and final text, making it the first place to look before changing the prompt.</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>bdk eval --dir . readiness</span></span>
+<span class="line"><span>bdk eval --dir . --tag smoke</span></span>
+<span class="line"><span>bdk eval --dir . --json</span></span></code></pre></div><p>By default, results land under <code>evals/&lt;stamp&gt;/</code> in the project state directory. Open <code>evals/&lt;case-id&gt;.json</code> inside that run directory for the failed case. This artifact location is independent of <code>--state-root</code>; pass <code>--artifacts &lt;dir&gt;</code> to choose another destination.</p><p>Model turns need a credential. Resolution checks <code>CURSOR_API_KEY</code>, <code>CURSOR_API_KEY_FILE</code>, <code>CURSOR_SERVICE_ACCOUNT_KEY</code>, then a saved <code>bdk login</code>. <code>eval --list</code> only discovers cases and needs none.</p><h2 id="run-evals-in-ci" tabindex="-1">Run evals in CI <a class="header-anchor" href="#run-evals-in-ci" aria-label="Permalink to &quot;Run evals in CI&quot;">​</a></h2><p>Write machine-readable output and JUnit annotations, then let a failed gate fail the job:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span># CURSOR_API_KEY comes from the CI secret store.</span></span>
+<span class="line"><span>bdk eval --dir . --json --no-stream \</span></span>
+<span class="line"><span>  --junit reports/evals.xml \</span></span>
+<span class="line"><span>  --artifacts reports/evals \</span></span>
+<span class="line"><span>  &gt; reports/evals.json</span></span></code></pre></div><p>Run <code>--tag smoke</code> on pull requests and the full suite on a schedule when cost or latency makes every-push coverage impractical. Use soft scores for a new probabilistic benchmark until its threshold is trustworthy.</p><p>The <a href="./reference/cli.html#eval">CLI reference</a> covers selectors, hosted runs, exit codes, and JSON output.</p><h2 id="keep-side-effects-out-of-eval-sessions" tabindex="-1">Keep side effects out of eval sessions <a class="header-anchor" href="#keep-side-effects-out-of-eval-sessions" aria-label="Permalink to &quot;Keep side effects out of eval sessions&quot;">​</a></h2><p>Eval sessions carry <code>purpose: &quot;eval&quot;</code>. Check it at the deterministic write boundary and return the decision without performing the external action:</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">async </span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">execute</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">({ findings }, ctx) {</span></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">  if</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> (ctx.session.purpose </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">===</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> &quot;eval&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">) {</span></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">    return</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> {</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">      posted: </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">false</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">      reason: </span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">&quot;eval&quot;</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">      count: findings.</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">length</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">    };</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">  }</span></span>
+<span class="line"></span>
+<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">  return</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> await</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> postFindings</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(findings);</span></span>
+<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span></code></pre></div><p>This preserves the tool call in the trajectory, so the eval can still assert that the agent chose to post.</p><h2 id="evals-hooks-or-hillclimbing" tabindex="-1">Evals, hooks, or hillclimbing? <a class="header-anchor" href="#evals-hooks-or-hillclimbing" aria-label="Permalink to &quot;Evals, hooks, or hillclimbing?&quot;">​</a></h2><p>Use an eval to gate one fixed input. Use a <a href="./reference/hooks.html">hook</a> to observe every live session for metrics, auditing, or alerts. Use <a href="./hillclimbing.html">hillclimbing</a> to decide which source change improves a fixed set of inputs; every kept hillclimb change should add an eval.</p><h2 id="gates-and-scores" tabindex="-1">Gates and scores <a class="header-anchor" href="#gates-and-scores" aria-label="Permalink to &quot;Gates and scores&quot;">​</a></h2><p><code>t.succeeded()</code>, tool assertions, and deterministic checks are hard gates by default. A failed gate fails the case. <code>.soft()</code> records a measurement without blocking, while <code>.atLeast(threshold)</code> records a score and marks a miss as <code>scored</code>.</p><p>Start with decisions and output shape: the intended tool, the tempting wrong tool, and one structural check. Exact prose is rarely a stable contract.</p><h2 id="keep-improvements-with-regression-evals" tabindex="-1">Keep improvements with regression evals <a class="header-anchor" href="#keep-improvements-with-regression-evals" aria-label="Permalink to &quot;Keep improvements with regression evals&quot;">​</a></h2><p>Every kept improvement needs an eval that would have failed before the change. A tool-choice gate, bounded <code>maxToolCalls</code>, or output-shape check turns the improvement into a durable contract.</p><p>Never weaken an existing gate to make a new implementation pass. That moves the freeze line instead of proving the change.</p><h2 id="related" tabindex="-1">Related <a class="header-anchor" href="#related" aria-label="Permalink to &quot;Related&quot;">​</a></h2><ul><li><a href="./reference/evals.html">Evals reference</a>: complete authoring and runner contracts</li><li><a href="./../skills/evals/SKILL.html"><code>skills/evals/SKILL.md</code></a>: author and seed cases with a coding agent</li><li><a href="./hillclimbing.html">Hillclimbing</a>: measure a change and lock the win</li><li><a href="./reference/sessions.html">Sessions</a>: events available to trajectory assertions</li></ul></div></div></main><footer class="VPDocFooter" data-v-afe580da data-v-feab96e0><!--[--><!--]--><!----><nav class="prev-next" aria-labelledby="doc-footer-aria-label" data-v-feab96e0><span class="visually-hidden" id="doc-footer-aria-label" data-v-feab96e0>Pager</span><div class="pager" data-v-feab96e0><a class="VPLink link pager-link prev" href="/docs/deployment.html" data-v-feab96e0><!--[--><span class="desc" data-v-feab96e0>Previous page</span><span class="title" data-v-feab96e0>Deployment</span><!--]--></a></div><div class="pager" data-v-feab96e0><a class="VPLink link pager-link next" href="/docs/hillclimbing.html" data-v-feab96e0><!--[--><span class="desc" data-v-feab96e0>Next page</span><span class="title" data-v-feab96e0>Hillclimbing</span><!--]--></a></div></nav></footer><!--[--><!--]--></div></div></div><!--[--><!--]--></div></div><!----><!--[--><!--]--></div></div>
+    <script>window.__VP_HASH_MAP__=JSON.parse("{\"building-with-agents.md\":\"-fcjCmJU\",\"deployment.md\":\"nGo1yIsa\",\"evals.md\":\"CCAtNcuL\",\"guides_agent-to-agent.md\":\"CGkicXeB\",\"guides_bitbucket.md\":\"Cb3LPKt3\",\"guides_cloud-agents.md\":\"BuEkDCwo\",\"guides_convert-automation.md\":\"Ci-Y2iD5\",\"guides_github.md\":\"C2_DSWlE\",\"guides_gitlab.md\":\"BM27P43u\",\"guides_grokbot-agents.md\":\"yxVbSlKA\",\"guides_hooks.md\":\"BWoiNaMq\",\"guides_improve.md\":\"COtL-6ft\",\"guides_jev.md\":\"BNDm14Gy\",\"guides_mcp-oauth.md\":\"Cu6mhZ_3\",\"guides_opentelemetry.md\":\"DLlIB5rv\",\"guides_slack.md\":\"ZbHQETOT\",\"guides_webhooks.md\":\"D0furjw0\",\"hillclimbing.md\":\"Biy0m-Qb\",\"index.md\":\"CVc_oOA3\",\"quickstart.md\":\"wG4shE1f\",\"reference_agent-config.md\":\"DzGI777n\",\"reference_artifacts.md\":\"IFFC5Zln\",\"reference_channels.md\":\"mbNtKCiR\",\"reference_cli.md\":\"BK2QUH67\",\"reference_connections.md\":\"_XpKiOaF\",\"reference_evals.md\":\"DqNyHMA6\",\"reference_extensions.md\":\"CPPrkWlJ\",\"reference_hooks.md\":\"Dg3dEugd\",\"reference_http-api.md\":\"BZOvhCqY\",\"reference_instructions.md\":\"Bdwscs4G\",\"reference_playground.md\":\"BdQzRJc2\",\"reference_project-layout.md\":\"CpLQ9j4K\",\"reference_prompt.md\":\"BQZB4eaV\",\"reference_schedules.md\":\"BVU5EFbz\",\"reference_sessions.md\":\"BtpTjZmh\",\"reference_skills.md\":\"Cy00QMKS\",\"reference_subagents.md\":\"B8oyeXOI\",\"reference_tools.md\":\"hg2Wi5Bq\",\"templates_agentic-owners.md\":\"BykUiZ8j\",\"templates_security-reviewer.md\":\"DXMmKl_o\",\"templates_thermo-quality-review.md\":\"Dg1p1MsG\",\"templates_thermo-review.md\":\"Dlylkowj\",\"templates_triage.md\":\"CvJyx81N\",\"troubleshooting.md\":\"BzZSu1Gz\"}");window.__VP_SITE_DATA__=JSON.parse("{\"lang\":\"en-US\",\"dir\":\"ltr\",\"title\":\"Grok Bot Development Kit (BDK)\",\"description\":\"Build and serve bots and agents from versioned Markdown and TypeScript.\",\"base\":\"/docs/\",\"head\":[],\"router\":{\"prefetchLinks\":true},\"appearance\":true,\"themeConfig\":{\"sdkVersion\":\"0.2.1\",\"nav\":[{\"text\":\"Quickstart\",\"link\":\"/quickstart\"},{\"text\":\"Guides\",\"link\":\"/guides/mcp-oauth\"},{\"text\":\"Integrations\",\"link\":\"/guides/github\"},{\"text\":\"Templates\",\"link\":\"/templates/security-reviewer\"},{\"text\":\"Reference\",\"link\":\"/reference/project-layout\"}],\"search\":{\"provider\":\"local\"},\"outline\":{\"level\":[2,3]},\"sidebar\":[{\"text\":\"Getting started\",\"items\":[{\"text\":\"Overview\",\"link\":\"/\"},{\"text\":\"Building your first agent\",\"link\":\"/quickstart\"}]},{\"text\":\"Integrations\",\"collapsed\":false,\"items\":[{\"text\":\"GitHub\",\"link\":\"/guides/github\"},{\"text\":\"Slack\",\"link\":\"/guides/slack\"},{\"text\":\"Webhooks\",\"link\":\"/guides/webhooks\"},{\"text\":\"Cursor Automations\",\"link\":\"/guides/convert-automation\"},{\"text\":\"OpenTelemetry\",\"link\":\"/guides/opentelemetry\"},{\"text\":\"GitLab\",\"link\":\"/guides/gitlab\"},{\"text\":\"Bitbucket\",\"link\":\"/guides/bitbucket\"}]},{\"text\":\"Extensions\",\"collapsed\":false,\"items\":[{\"text\":\"Overview\",\"link\":\"/reference/extensions\"},{\"text\":\"Cloud agents\",\"link\":\"/guides/cloud-agents\"},{\"text\":\"Grok Bot agents\",\"link\":\"/guides/grokbot-agents\"},{\"text\":\"Self-improvement\",\"link\":\"/guides/improve\"},{\"text\":\"Jev\",\"link\":\"/guides/jev\"}]},{\"text\":\"Guides\",\"collapsed\":false,\"items\":[{\"text\":\"MCP\",\"link\":\"/guides/mcp-oauth\"},{\"text\":\"Hooks\",\"link\":\"/guides/hooks\"},{\"text\":\"Peer agents\",\"link\":\"/guides/agent-to-agent\"},{\"text\":\"Deployment\",\"link\":\"/deployment\"},{\"text\":\"Evals\",\"link\":\"/evals\"},{\"text\":\"Hillclimbing\",\"link\":\"/hillclimbing\"},{\"text\":\"Build with a coding agent\",\"link\":\"/building-with-agents\"},{\"text\":\"Troubleshooting\",\"link\":\"/troubleshooting\"}]},{\"text\":\"Templates\",\"collapsed\":false,\"items\":[{\"text\":\"Security reviewer\",\"link\":\"/templates/security-reviewer\"},{\"text\":\"Thermo review\",\"link\":\"/templates/thermo-review\"},{\"text\":\"Thermo quality review\",\"link\":\"/templates/thermo-quality-review\"},{\"text\":\"Agentic Approver\",\"link\":\"/templates/agentic-owners\"},{\"text\":\"Triage agent\",\"link\":\"/templates/triage\"}]},{\"text\":\"Reference\",\"collapsed\":false,\"items\":[{\"text\":\"Project layout\",\"link\":\"/reference/project-layout\"},{\"text\":\"Agent config\",\"link\":\"/reference/agent-config\"},{\"text\":\"Instructions\",\"link\":\"/reference/instructions\"},{\"text\":\"Tools\",\"link\":\"/reference/tools\"},{\"text\":\"prompt\",\"link\":\"/reference/prompt\"},{\"text\":\"Skills\",\"link\":\"/reference/skills\"},{\"text\":\"MCP Connections\",\"link\":\"/reference/connections\"},{\"text\":\"Subagents\",\"link\":\"/reference/subagents\"},{\"text\":\"Channels\",\"link\":\"/reference/channels\"},{\"text\":\"Schedules & reminders\",\"link\":\"/reference/schedules\"},{\"text\":\"Hooks\",\"link\":\"/reference/hooks\"},{\"text\":\"Evals\",\"link\":\"/reference/evals\"},{\"text\":\"Artifacts\",\"link\":\"/reference/artifacts\"},{\"text\":\"Sessions & streaming\",\"link\":\"/reference/sessions\"},{\"text\":\"Playground\",\"link\":\"/reference/playground\"},{\"text\":\"CLI\",\"link\":\"/reference/cli\"},{\"text\":\"HTTP API\",\"link\":\"/reference/http-api\"}]}]},\"locales\":{},\"scrollOffset\":134,\"cleanUrls\":false}");</script>
+    
+  </body>
+</html>