@grok/agent-sdk 0.2.27 → 0.2.29
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +42 -37
- package/CHANGELOG.md +1 -1
- package/README.md +15 -17
- package/dist/agent-v2-hosting.d.ts +13 -0
- package/dist/agent-v2-hosting.d.ts.map +1 -0
- package/dist/agent-v2-hosting.js +8 -0
- package/dist/artifacts.d.ts +1 -1
- package/dist/artifacts.js +1 -1
- package/dist/bin/agent-serve.d.ts +3 -15
- package/dist/bin/agent-serve.d.ts.map +1 -1
- package/dist/bin/agent-serve.js +30 -1628
- package/dist/channels/bitbucket/api.d.ts.map +1 -1
- package/dist/channels/bitbucket/api.js +3 -3
- package/dist/channels/bitbucket/binding.d.ts.map +1 -1
- package/dist/channels/bitbucket/binding.js +5 -5
- package/dist/channels/bitbucket/bitbucket-channel.d.ts.map +1 -1
- package/dist/channels/bitbucket/bitbucket-channel.js +1 -0
- package/dist/channels/bitbucket/bitbucket-webhook.d.ts.map +1 -1
- package/dist/channels/bitbucket/bitbucket-webhook.js +9 -6
- package/dist/channels/bitbucket/checks.d.ts.map +1 -1
- package/dist/channels/bitbucket/checks.js +4 -0
- package/dist/channels/bitbucket/events.d.ts.map +1 -1
- package/dist/channels/bitbucket/index.d.ts +1 -1
- package/dist/channels/bitbucket/index.js +1 -1
- package/dist/channels/bitbucket/replay.d.ts.map +1 -1
- package/dist/channels/bitbucket/replay.js +4 -3
- package/dist/channels/change-monitors/change-monitors-channel.d.ts.map +1 -1
- package/dist/channels/change-monitors/change-monitors-channel.js +15 -2
- package/dist/channels/change-monitors/claim.d.ts +5 -0
- package/dist/channels/change-monitors/claim.d.ts.map +1 -0
- package/dist/channels/change-monitors/claim.js +6 -0
- package/dist/channels/change-monitors/index.d.ts +1 -0
- package/dist/channels/change-monitors/index.d.ts.map +1 -1
- package/dist/channels/change-monitors/index.js +1 -0
- package/dist/channels/change-monitors/types.d.ts.map +1 -1
- package/dist/channels/deployments/deployments-channel.d.ts +1 -1
- package/dist/channels/deployments/deployments-channel.js +1 -1
- package/dist/channels/deployments/index.d.ts +1 -1
- package/dist/channels/deployments/index.js +1 -1
- package/dist/channels/deployments/types.d.ts.map +1 -1
- package/dist/channels/github/api.d.ts +11 -0
- package/dist/channels/github/api.d.ts.map +1 -1
- package/dist/channels/github/api.js +29 -4
- package/dist/channels/github/binding.d.ts +5 -3
- package/dist/channels/github/binding.d.ts.map +1 -1
- package/dist/channels/github/binding.js +16 -9
- package/dist/channels/github/github-channel.d.ts +7 -2
- package/dist/channels/github/github-channel.d.ts.map +1 -1
- package/dist/channels/github/github-channel.js +20 -11
- package/dist/channels/github/index.d.ts +1 -1
- package/dist/channels/github/index.js +1 -1
- package/dist/channels/github/progress.d.ts.map +1 -1
- package/dist/channels/github/progress.js +4 -0
- package/dist/channels/github/state.d.ts +2 -2
- package/dist/channels/github/state.d.ts.map +1 -1
- package/dist/channels/github/types.d.ts +7 -7
- package/dist/channels/github/types.d.ts.map +1 -1
- package/dist/channels/gitlab/api.d.ts +2 -2
- package/dist/channels/gitlab/api.d.ts.map +1 -1
- package/dist/channels/gitlab/api.js +25 -14
- package/dist/channels/gitlab/binding.d.ts.map +1 -1
- package/dist/channels/gitlab/binding.js +2 -3
- package/dist/channels/gitlab/checks.d.ts.map +1 -1
- package/dist/channels/gitlab/checks.js +4 -0
- package/dist/channels/gitlab/events.d.ts.map +1 -1
- package/dist/channels/gitlab/gitlab-channel.d.ts.map +1 -1
- package/dist/channels/gitlab/gitlab-channel.js +4 -7
- package/dist/channels/gitlab/gitlab-webhook.d.ts.map +1 -1
- package/dist/channels/gitlab/gitlab-webhook.js +11 -8
- package/dist/channels/gitlab/index.d.ts +1 -1
- package/dist/channels/gitlab/index.js +1 -1
- package/dist/channels/gitlab/replay.d.ts.map +1 -1
- package/dist/channels/gitlab/replay.js +2 -0
- package/dist/channels/gitlab/types.d.ts.map +1 -1
- package/dist/channels/issues/claim.d.ts +5 -0
- package/dist/channels/issues/claim.d.ts.map +1 -0
- package/dist/channels/issues/claim.js +6 -0
- package/dist/channels/issues/index.d.ts +1 -0
- package/dist/channels/issues/index.d.ts.map +1 -1
- package/dist/channels/issues/index.js +1 -0
- package/dist/channels/issues/issues-channel.d.ts.map +1 -1
- package/dist/channels/issues/issues-channel.js +16 -3
- package/dist/channels/issues/types.d.ts +2 -2
- package/dist/channels/issues/types.d.ts.map +1 -1
- package/dist/channels/origin/api.d.ts.map +1 -1
- package/dist/channels/origin/api.js +3 -3
- package/dist/channels/origin/index.d.ts +1 -1
- package/dist/channels/origin/index.js +1 -1
- package/dist/channels/origin/origin-channel.d.ts.map +1 -1
- package/dist/channels/origin/origin-channel.js +6 -6
- package/dist/channels/origin/origin-webhook.d.ts.map +1 -1
- package/dist/channels/origin/origin-webhook.js +2 -5
- package/dist/channels/pagerduty/api.d.ts.map +1 -1
- package/dist/channels/pagerduty/api.js +2 -1
- package/dist/channels/pagerduty/index.d.ts +1 -1
- package/dist/channels/pagerduty/index.js +1 -1
- package/dist/channels/pagerduty/pagerduty-channel.d.ts +1 -1
- package/dist/channels/pagerduty/pagerduty-channel.d.ts.map +1 -1
- package/dist/channels/pagerduty/pagerduty-channel.js +2 -1
- package/dist/channels/pagerduty/pagerduty-webhook.d.ts.map +1 -1
- package/dist/channels/pagerduty/pagerduty-webhook.js +20 -11
- package/dist/channels/pagerduty/payload.d.ts.map +1 -1
- package/dist/channels/pagerduty/payload.js +2 -3
- package/dist/channels/pagerduty/replay.d.ts +1 -1
- package/dist/channels/pagerduty/replay.d.ts.map +1 -1
- package/dist/channels/pagerduty/replay.js +8 -5
- package/dist/channels/pagerduty/self-user.d.ts.map +1 -1
- package/dist/channels/pagerduty/self-user.js +8 -9
- package/dist/channels/runs/runs-channel.d.ts.map +1 -1
- package/dist/channels/runs/runs-channel.js +8 -2
- package/dist/channels/runs/types.d.ts +2 -2
- package/dist/channels/runs/types.d.ts.map +1 -1
- package/dist/channels/slack/agentic-delivery.d.ts +20 -1
- package/dist/channels/slack/agentic-delivery.d.ts.map +1 -1
- package/dist/channels/slack/agentic-delivery.js +77 -12
- package/dist/channels/slack/agentic-signals.d.ts +2 -0
- package/dist/channels/slack/agentic-signals.d.ts.map +1 -1
- package/dist/channels/slack/agentic-signals.js +24 -7
- package/dist/channels/slack/api.d.ts +1 -1
- package/dist/channels/slack/api.d.ts.map +1 -1
- package/dist/channels/slack/api.js +12 -13
- package/dist/channels/slack/approvals.d.ts.map +1 -1
- package/dist/channels/slack/approvals.js +5 -4
- package/dist/channels/slack/attachments.d.ts.map +1 -1
- package/dist/channels/slack/attachments.js +34 -9
- package/dist/channels/slack/auth.d.ts.map +1 -1
- package/dist/channels/slack/auth.js +3 -1
- package/dist/channels/slack/bot-mentions.d.ts.map +1 -1
- package/dist/channels/slack/bot-mentions.js +2 -4
- package/dist/channels/slack/cli-project.d.ts.map +1 -1
- package/dist/channels/slack/cli-project.js +7 -10
- package/dist/channels/slack/constants.d.ts +7 -0
- package/dist/channels/slack/constants.d.ts.map +1 -1
- package/dist/channels/slack/constants.js +7 -0
- package/dist/channels/slack/defaults.d.ts +12 -0
- package/dist/channels/slack/defaults.d.ts.map +1 -1
- package/dist/channels/slack/defaults.js +10 -2
- package/dist/channels/slack/dispatch.d.ts +7 -1
- package/dist/channels/slack/dispatch.d.ts.map +1 -1
- package/dist/channels/slack/dispatch.js +88 -13
- package/dist/channels/slack/doctor.d.ts.map +1 -1
- package/dist/channels/slack/doctor.js +11 -11
- package/dist/channels/slack/inbound.d.ts +7 -0
- package/dist/channels/slack/inbound.d.ts.map +1 -1
- package/dist/channels/slack/inbound.js +25 -4
- package/dist/channels/slack/index.d.ts +2 -2
- package/dist/channels/slack/index.d.ts.map +1 -1
- package/dist/channels/slack/index.js +2 -2
- package/dist/channels/slack/init.d.ts +2 -0
- package/dist/channels/slack/init.d.ts.map +1 -1
- package/dist/channels/slack/init.js +10 -2
- package/dist/channels/slack/interactive.d.ts +6 -1
- package/dist/channels/slack/interactive.d.ts.map +1 -1
- package/dist/channels/slack/interactive.js +15 -25
- package/dist/channels/slack/manifest.d.ts.map +1 -1
- package/dist/channels/slack/manifest.js +15 -12
- package/dist/channels/slack/message-body.d.ts.map +1 -1
- package/dist/channels/slack/message-body.js +2 -4
- package/dist/channels/slack/nudge-carrier.d.ts.map +1 -1
- package/dist/channels/slack/nudge-carrier.js +27 -1
- package/dist/channels/slack/nudge-store.d.ts.map +1 -1
- package/dist/channels/slack/nudge-store.js +8 -3
- package/dist/channels/slack/placeholder.d.ts +3 -0
- package/dist/channels/slack/placeholder.d.ts.map +1 -1
- package/dist/channels/slack/placeholder.js +6 -5
- package/dist/channels/slack/recording-sink.d.ts.map +1 -1
- package/dist/channels/slack/recording-sink.js +4 -3
- package/dist/channels/slack/sender-name.d.ts.map +1 -1
- package/dist/channels/slack/sender-name.js +2 -3
- package/dist/channels/slack/slack-channel.d.ts +1 -3
- package/dist/channels/slack/slack-channel.d.ts.map +1 -1
- package/dist/channels/slack/slack-channel.js +63 -86
- package/dist/channels/slack/thread-context.d.ts.map +1 -1
- package/dist/channels/slack/thread-context.js +4 -9
- package/dist/channels/slack/tool-cards.d.ts.map +1 -1
- package/dist/channels/slack/tool-cards.js +4 -0
- package/dist/channels/slack/types.d.ts +19 -4
- package/dist/channels/slack/types.d.ts.map +1 -1
- package/dist/channels/slack/upload.d.ts +2 -0
- package/dist/channels/slack/upload.d.ts.map +1 -1
- package/dist/channels/slack/upload.js +49 -12
- package/dist/channels/slack/user-info.d.ts.map +1 -1
- package/dist/channels/slack/user-info.js +3 -6
- package/dist/channels.d.ts +11 -11
- package/dist/channels.d.ts.map +1 -1
- package/dist/channels.js +57 -30
- package/dist/connections.d.ts +1 -1
- package/dist/connections.js +1 -1
- package/dist/docs/404.html +4 -4
- package/dist/docs/architecture.svg +1 -1
- package/dist/docs/assets/{app.wA5xBBcd.js → app.DMrZKFjI.js} +4 -4
- package/dist/docs/assets/building-with-agents.md.Bf1yH2tG.js +9 -0
- package/dist/docs/assets/building-with-agents.md.Bf1yH2tG.lean.js +1 -0
- package/dist/docs/assets/changelog.md.DrLFQUlu.js +1 -0
- package/dist/docs/assets/changelog.md.DrLFQUlu.lean.js +1 -0
- package/dist/docs/assets/chunks/@localSearchIndexroot.Dpd_wxVU.js +1 -0
- package/dist/docs/assets/chunks/{VPLocalSearchBox.C29HLCVL.js → VPLocalSearchBox.DQvuw1KA.js} +1 -1
- package/dist/docs/assets/chunks/{arc.BOaoYWmh.js → arc.ahOG-Qza.js} +1 -1
- package/dist/docs/assets/chunks/{architectureDiagram-Q4EWVU46.MfG0itQh.js → architectureDiagram-Q4EWVU46.BcgfYNO_.js} +1 -1
- package/dist/docs/assets/chunks/{baseUniq.CUeJciCF.js → baseUniq.DeIYZrWh.js} +1 -1
- package/dist/docs/assets/chunks/{blockDiagram-DXYQGD6D.BrXMZRtK.js → blockDiagram-DXYQGD6D.HjJ2goMX.js} +1 -1
- package/dist/docs/assets/chunks/{c4Diagram-AHTNJAMY.DfkDHeB9.js → c4Diagram-AHTNJAMY.C1QTb9GM.js} +1 -1
- package/dist/docs/assets/chunks/channel._fZX_hvU.js +1 -0
- package/dist/docs/assets/chunks/{chunk-4BX2VUAB.xKz8CiXK.js → chunk-4BX2VUAB.UuZiDiVp.js} +1 -1
- package/dist/docs/assets/chunks/{chunk-4TB4RGXK.kkfBh4zk.js → chunk-4TB4RGXK.5A3YmkhI.js} +1 -1
- package/dist/docs/assets/chunks/{chunk-55IACEB6.BClJGb94.js → chunk-55IACEB6.B4uf1yYV.js} +1 -1
- package/dist/docs/assets/chunks/{chunk-EDXVE4YY.P9BpQ9L3.js → chunk-EDXVE4YY.CoXBce8t.js} +1 -1
- package/dist/docs/assets/chunks/{chunk-FMBD7UC4.R4iYi58S.js → chunk-FMBD7UC4.DnC5NO4e.js} +1 -1
- package/dist/docs/assets/chunks/{chunk-OYMX7WX6.Cpn8L1dw.js → chunk-OYMX7WX6.CzfHCeIi.js} +1 -1
- package/dist/docs/assets/chunks/{chunk-QZHKN3VN.BLftECqU.js → chunk-QZHKN3VN.C6tcwkVd.js} +1 -1
- package/dist/docs/assets/chunks/{chunk-YZCP3GAM.BWDV1AYt.js → chunk-YZCP3GAM._oboyZ1A.js} +1 -1
- package/dist/docs/assets/chunks/classDiagram-6PBFFD2Q.eClS1Q0E.js +1 -0
- package/dist/docs/assets/chunks/classDiagram-v2-HSJHXN6E.eClS1Q0E.js +1 -0
- package/dist/docs/assets/chunks/clone.W1Phdw9P.js +1 -0
- package/dist/docs/assets/chunks/{cose-bilkent-S5V4N54A.DGpAcJ55.js → cose-bilkent-S5V4N54A.BfPVKeId.js} +1 -1
- package/dist/docs/assets/chunks/{dagre-KV5264BT.BBXxGEaD.js → dagre-KV5264BT.CLMLc0lD.js} +1 -1
- package/dist/docs/assets/chunks/{diagram-5BDNPKRD.EKqOwmK2.js → diagram-5BDNPKRD.C7OF7UfH.js} +1 -1
- package/dist/docs/assets/chunks/{diagram-G4DWMVQ6.CB8pdkoZ.js → diagram-G4DWMVQ6.Dpao0Bl1.js} +1 -1
- package/dist/docs/assets/chunks/{diagram-MMDJMWI5.aHh4WGg9.js → diagram-MMDJMWI5.DLnM9eLp.js} +1 -1
- package/dist/docs/assets/chunks/{diagram-TYMM5635.YIsbRH20.js → diagram-TYMM5635.BcpVgA_M.js} +1 -1
- package/dist/docs/assets/chunks/{erDiagram-SMLLAGMA.B3bTCW_e.js → erDiagram-SMLLAGMA.CsL5JuCV.js} +1 -1
- package/dist/docs/assets/chunks/{flowDiagram-DWJPFMVM.M81hXIXZ.js → flowDiagram-DWJPFMVM.MFihq9AF.js} +1 -1
- package/dist/docs/assets/chunks/{ganttDiagram-T4ZO3ILL.BKg7WY--.js → ganttDiagram-T4ZO3ILL.CJeLodyG.js} +1 -1
- package/dist/docs/assets/chunks/{gitGraphDiagram-UUTBAWPF.CLEMZgO3.js → gitGraphDiagram-UUTBAWPF.1HhlE45Z.js} +1 -1
- package/dist/docs/assets/chunks/{graph.DS38W_Q-.js → graph.BS02KkWA.js} +1 -1
- package/dist/docs/assets/chunks/{infoDiagram-42DDH7IO.BTiWTi82.js → infoDiagram-42DDH7IO.fp5gvytQ.js} +1 -1
- package/dist/docs/assets/chunks/{ishikawaDiagram-UXIWVN3A.DcqY_KQL.js → ishikawaDiagram-UXIWVN3A.C6Wk1V4I.js} +1 -1
- package/dist/docs/assets/chunks/{journeyDiagram-VCZTEJTY.0lQNZ40U.js → journeyDiagram-VCZTEJTY.qJt4uJ_H.js} +1 -1
- package/dist/docs/assets/chunks/{kanban-definition-6JOO6SKY.L5sKlf7X.js → kanban-definition-6JOO6SKY.z2tG9kly.js} +1 -1
- package/dist/docs/assets/chunks/{layout.7u4gFhQa.js → layout.BPCkdejr.js} +1 -1
- package/dist/docs/assets/chunks/{linear.D5v33SOb.js → linear.CL3TC3u7.js} +1 -1
- package/dist/docs/assets/chunks/{min.Dc9uCxf9.js → min.SPrSp704.js} +1 -1
- package/dist/docs/assets/chunks/{mindmap-definition-QFDTVHPH.B56nGoB_.js → mindmap-definition-QFDTVHPH.DgrQRFZW.js} +1 -1
- package/dist/docs/assets/chunks/{pieDiagram-DEJITSTG.Cba8e8lS.js → pieDiagram-DEJITSTG.Ba7yxiW2.js} +1 -1
- package/dist/docs/assets/chunks/{quadrantDiagram-34T5L4WZ.2YjBycHF.js → quadrantDiagram-34T5L4WZ.Bvskad4p.js} +1 -1
- package/dist/docs/assets/chunks/{requirementDiagram-MS252O5E.D4P5RqVI.js → requirementDiagram-MS252O5E.Ce70ugV9.js} +1 -1
- package/dist/docs/assets/chunks/{sankeyDiagram-XADWPNL6.C0Hn8NUm.js → sankeyDiagram-XADWPNL6.BJ5OEULO.js} +1 -1
- package/dist/docs/assets/chunks/{sequenceDiagram-FGHM5R23.BZm7EaoT.js → sequenceDiagram-FGHM5R23.C8a1hDs6.js} +1 -1
- package/dist/docs/assets/chunks/{stateDiagram-FHFEXIEX.DcR4f3X2.js → stateDiagram-FHFEXIEX.D2D8A5CK.js} +1 -1
- package/dist/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.77yf4kRr.js +1 -0
- package/dist/docs/assets/chunks/{theme.DvfKzyXq.js → theme.DRciHmS1.js} +2 -2
- package/dist/docs/assets/chunks/{timeline-definition-GMOUNBTQ.CgDh96vg.js → timeline-definition-GMOUNBTQ.C4ePrY8-.js} +1 -1
- package/dist/docs/assets/chunks/{vennDiagram-DHZGUBPP.B4fagbC8.js → vennDiagram-DHZGUBPP.Dzso6dEA.js} +1 -1
- package/dist/docs/assets/chunks/{wardley-RL74JXVD.Ck68omim.js → wardley-RL74JXVD.djgXR0ar.js} +1 -1
- package/dist/docs/assets/chunks/{wardleyDiagram-NUSXRM2D.DBUg0bvz.js → wardleyDiagram-NUSXRM2D.Bf2ICxjh.js} +1 -1
- package/dist/docs/assets/chunks/{xychartDiagram-5P7HB3ND.DnKSGQE1.js → xychartDiagram-5P7HB3ND.DEH4BcVJ.js} +1 -1
- package/dist/docs/assets/deployment.md.KW0Ni_rY.js +31 -0
- package/dist/docs/assets/deployment.md.KW0Ni_rY.lean.js +1 -0
- package/dist/docs/assets/{evals.md.CHR34K1Y.js → evals.md.DiljU__Z.js} +10 -10
- package/dist/docs/assets/{guides_agent-to-agent.md.CwL5cwuE.js → guides_agent-to-agent.md.D1D5f6Jc.js} +7 -7
- package/dist/docs/assets/guides_agent-to-agent.md.D1D5f6Jc.lean.js +1 -0
- package/dist/docs/assets/{guides_bitbucket.md.CKUayYtD.js → guides_bitbucket.md.DkjkBRU8.js} +3 -3
- package/dist/docs/assets/guides_cloud-agents.md.Ceyjv0ul.js +9 -0
- package/dist/docs/assets/guides_cloud-agents.md.Ceyjv0ul.lean.js +1 -0
- package/dist/docs/assets/{guides_convert-automation.md.DMEX9nF-.js → guides_convert-automation.md.BjraCvok.js} +11 -11
- package/dist/docs/assets/guides_convert-automation.md.BjraCvok.lean.js +1 -0
- package/dist/docs/assets/{guides_github.md.TuNiZAOQ.js → guides_github.md.CgN5sNbW.js} +14 -14
- package/dist/docs/assets/{guides_github.md.TuNiZAOQ.lean.js → guides_github.md.CgN5sNbW.lean.js} +1 -1
- package/dist/docs/assets/{guides_gitlab.md.CtNI4fSH.js → guides_gitlab.md.DCE0p7_w.js} +3 -3
- package/dist/docs/assets/{guides_hooks.md.B2XfGX50.js → guides_hooks.md.BjeEiM7s.js} +4 -4
- package/dist/docs/assets/{guides_hooks.md.B2XfGX50.lean.js → guides_hooks.md.BjeEiM7s.lean.js} +1 -1
- package/dist/docs/assets/{guides_improve.md.CZKVt1cT.js → guides_improve.md.BFYtrtLE.js} +4 -4
- package/dist/docs/assets/{guides_improve.md.CZKVt1cT.lean.js → guides_improve.md.BFYtrtLE.lean.js} +1 -1
- package/dist/docs/assets/{guides_jev.md.D7XH4JH3.js → guides_jev.md.CEqLgjj9.js} +12 -12
- package/dist/docs/assets/{guides_jev.md.D7XH4JH3.lean.js → guides_jev.md.CEqLgjj9.lean.js} +1 -1
- package/dist/docs/assets/{guides_mcp-oauth.md.Cxq2TxBB.js → guides_mcp-oauth.md.DrGY92u4.js} +8 -8
- package/dist/docs/assets/{guides_mcp-oauth.md.Cxq2TxBB.lean.js → guides_mcp-oauth.md.DrGY92u4.lean.js} +1 -1
- package/dist/docs/assets/guides_migrate.md.RS9Eu-Xh.js +8 -0
- package/dist/docs/assets/guides_migrate.md.RS9Eu-Xh.lean.js +1 -0
- package/dist/docs/assets/guides_opentelemetry.md.CoZ1kNyd.js +28 -0
- package/dist/docs/assets/guides_opentelemetry.md.CoZ1kNyd.lean.js +1 -0
- package/dist/docs/assets/{guides_pagerduty.md.BHXMcMtx.js → guides_pagerduty.md.mnKXe9U0.js} +10 -10
- package/dist/docs/assets/{guides_slack.md.ClIyi7FI.js → guides_slack.md.BwXHURvY.js} +21 -8
- package/dist/docs/assets/{guides_slack.md.ClIyi7FI.lean.js → guides_slack.md.BwXHURvY.lean.js} +1 -1
- package/dist/docs/assets/{guides_webhooks.md.Cpdc3BDX.js → guides_webhooks.md.BJk63ZTs.js} +9 -5
- package/dist/docs/assets/{guides_webhooks.md.Cpdc3BDX.lean.js → guides_webhooks.md.BJk63ZTs.lean.js} +1 -1
- package/dist/docs/assets/{hillclimbing.md.Dxvd-0gT.js → hillclimbing.md.By61f3uq.js} +2 -2
- package/dist/docs/assets/index.md.CAruw_rp.js +26 -0
- package/dist/docs/assets/index.md.CAruw_rp.lean.js +1 -0
- package/dist/docs/assets/{quickstart.md.D8JDfkKO.js → quickstart.md.CljDaJBi.js} +12 -12
- package/dist/docs/assets/quickstart.md.CljDaJBi.lean.js +1 -0
- package/dist/docs/assets/reference_access.md.CEtibb4i.js +1 -0
- package/dist/docs/assets/reference_access.md.CEtibb4i.lean.js +1 -0
- package/dist/docs/assets/{reference_agent-config.md.CZrbHOfT.js → reference_agent-config.md.Ck29qYNj.js} +10 -8
- package/dist/docs/assets/{reference_agent-config.md.CZrbHOfT.lean.js → reference_agent-config.md.Ck29qYNj.lean.js} +1 -1
- package/dist/docs/assets/{reference_artifacts.md.DLV9MZES.js → reference_artifacts.md.C-hKR2hp.js} +2 -2
- package/dist/docs/assets/reference_channels.md.DsJfGSBJ.js +22 -0
- package/dist/docs/assets/{reference_channels.md.CNNIS6K7.lean.js → reference_channels.md.DsJfGSBJ.lean.js} +1 -1
- package/dist/docs/assets/reference_cli.md.BkdLRbXg.js +99 -0
- package/dist/docs/assets/reference_cli.md.BkdLRbXg.lean.js +1 -0
- package/dist/docs/assets/{reference_connections.md.BJE2Zllp.js → reference_connections.md.BoAotr7_.js} +7 -7
- package/dist/docs/assets/{reference_evals.md.Ccwp4x3o.js → reference_evals.md.2BXeMiO2.js} +31 -11
- package/dist/docs/assets/{reference_evals.md.Ccwp4x3o.lean.js → reference_evals.md.2BXeMiO2.lean.js} +1 -1
- package/dist/docs/assets/{reference_extensions.md.DKxTuCIw.js → reference_extensions.md.DI5X5x8p.js} +10 -10
- package/dist/docs/assets/{reference_extensions.md.DKxTuCIw.lean.js → reference_extensions.md.DI5X5x8p.lean.js} +1 -1
- package/dist/docs/assets/{reference_hooks.md.CROr0QgC.js → reference_hooks.md.B-bwJuiG.js} +5 -5
- package/dist/docs/assets/{reference_http-api.md.Dji-DD_Z.js → reference_http-api.md._RWELr3d.js} +1 -1
- package/dist/docs/assets/{reference_instructions.md.BYMVxAkw.js → reference_instructions.md.yL2hXLnl.js} +1 -1
- package/dist/docs/assets/reference_limits.md.BCCnvt3o.js +8 -0
- package/dist/docs/assets/reference_limits.md.BCCnvt3o.lean.js +1 -0
- package/dist/docs/assets/reference_memory.md.Dd7cyV9V.js +4 -0
- package/dist/docs/assets/reference_playground.md.CQiJk9CM.js +1 -0
- package/dist/docs/assets/{reference_project-layout.md.mFcS8yDD.js → reference_project-layout.md.BamcPDUn.js} +4 -4
- package/dist/docs/assets/{reference_prompt.md.DqPh71XN.js → reference_prompt.md.BMjm1Jp3.js} +2 -2
- package/dist/docs/assets/{reference_schedules.md.BqWnJES4.js → reference_schedules.md.RC47aFm4.js} +4 -4
- package/dist/docs/assets/{reference_sessions.md.Bi6lHttM.js → reference_sessions.md.C7KFhWXV.js} +1 -1
- package/dist/docs/assets/{reference_skills.md.Ckddg6KX.js → reference_skills.md.Dej2zQxi.js} +2 -2
- package/dist/docs/assets/{reference_skills.md.Ckddg6KX.lean.js → reference_skills.md.Dej2zQxi.lean.js} +1 -1
- package/dist/docs/assets/{reference_subagents.md.ChNGh6HK.js → reference_subagents.md.CAhvW4yS.js} +1 -1
- package/dist/docs/assets/{reference_tools.md.Bo0UfT0a.js → reference_tools.md.DQr7Fnhu.js} +10 -10
- package/dist/docs/assets/{reference_tools.md.Bo0UfT0a.lean.js → reference_tools.md.DQr7Fnhu.lean.js} +1 -1
- package/dist/docs/assets/{style.B9o0luvp.css → style.DK0YX1WL.css} +1 -1
- package/dist/docs/assets/{templates_agentic-owners.md.CTEnR0jp.js → templates_agentic-owners.md.Dh2nDSqg.js} +4 -4
- package/dist/docs/assets/templates_security-reviewer.md.DnwXdnB5.js +3 -0
- package/dist/docs/assets/templates_thermo-quality-review.md.MJIdHkzE.js +3 -0
- package/dist/docs/assets/{templates_thermo-quality-review.md.B77UyTrX.lean.js → templates_thermo-quality-review.md.MJIdHkzE.lean.js} +1 -1
- package/dist/docs/assets/templates_thermo-review.md.DfN1OFOl.js +3 -0
- package/dist/docs/assets/{templates_thermo-review.md.DRJ7mC5U.lean.js → templates_thermo-review.md.DfN1OFOl.lean.js} +1 -1
- package/dist/docs/assets/{templates_triage.md.CNN6b0JZ.js → templates_triage.md.BYWMloV4.js} +4 -4
- package/dist/docs/assets/troubleshooting.md.CGOdNVAL.js +1 -0
- package/dist/docs/building-with-agents.html +46 -46
- package/dist/docs/building-with-agents.md +20 -20
- package/dist/docs/changelog.html +39 -39
- package/dist/docs/changelog.md +57 -11
- package/dist/docs/deployment.html +54 -54
- package/dist/docs/deployment.md +115 -53
- package/dist/docs/evals.html +48 -48
- package/dist/docs/evals.md +15 -13
- package/dist/docs/guides/agent-to-agent.html +45 -45
- package/dist/docs/guides/agent-to-agent.md +7 -7
- package/dist/docs/guides/bitbucket.html +41 -41
- package/dist/docs/guides/bitbucket.md +5 -5
- package/dist/docs/guides/cloud-agents.html +41 -41
- package/dist/docs/guides/cloud-agents.md +3 -3
- package/dist/docs/guides/convert-automation.html +49 -49
- package/dist/docs/guides/convert-automation.md +16 -15
- package/dist/docs/guides/github.html +50 -50
- package/dist/docs/guides/github.md +17 -17
- package/dist/docs/guides/gitlab.html +41 -41
- package/dist/docs/guides/gitlab.md +5 -5
- package/dist/docs/guides/hooks.html +40 -40
- package/dist/docs/guides/hooks.md +3 -3
- package/dist/docs/guides/improve.html +41 -41
- package/dist/docs/guides/improve.md +2 -2
- package/dist/docs/guides/jev.html +48 -48
- package/dist/docs/guides/jev.md +15 -10
- package/dist/docs/guides/mcp-oauth.html +45 -45
- package/dist/docs/guides/mcp-oauth.md +10 -11
- package/dist/docs/guides/migrate.html +43 -42
- package/dist/docs/guides/migrate.md +36 -29
- package/dist/docs/guides/opentelemetry.html +50 -59
- package/dist/docs/guides/opentelemetry.md +61 -130
- package/dist/docs/guides/pagerduty.html +48 -48
- package/dist/docs/guides/pagerduty.md +18 -18
- package/dist/docs/guides/slack.html +58 -45
- package/dist/docs/guides/slack.md +87 -17
- package/dist/docs/guides/webhooks.html +46 -42
- package/dist/docs/guides/webhooks.md +61 -3
- package/dist/docs/hashmap.json +1 -1
- package/dist/docs/hillclimbing.html +39 -39
- package/dist/docs/hillclimbing.md +4 -4
- package/dist/docs/index.html +56 -57
- package/dist/docs/index.md +31 -31
- package/dist/docs/llms-full.txt +1333 -1417
- package/dist/docs/llms.txt +13 -14
- package/dist/docs/package/skills/create-agent/SKILL.md +178 -0
- package/dist/docs/package/skills/evals/SKILL.md +145 -0
- package/dist/docs/package/skills/github/SKILL.md +133 -0
- package/dist/docs/package/skills/hillclimb/SKILL.md +63 -0
- package/dist/docs/package/skills/mcp-auth/SKILL.md +60 -0
- package/dist/docs/package/skills/migrate-v2/SKILL.md +394 -0
- package/dist/docs/package/skills/setup-slack/SKILL.md +136 -0
- package/dist/docs/quickstart.html +50 -50
- package/dist/docs/quickstart.md +25 -21
- package/dist/docs/reference/access.html +60 -0
- package/dist/docs/reference/access.md +22 -0
- package/dist/docs/reference/agent-config.html +46 -44
- package/dist/docs/reference/agent-config.md +54 -12
- package/dist/docs/reference/artifacts.html +39 -39
- package/dist/docs/reference/artifacts.md +2 -2
- package/dist/docs/reference/channels.html +39 -39
- package/dist/docs/reference/channels.md +29 -9
- package/dist/docs/reference/cli.html +96 -100
- package/dist/docs/reference/cli.md +363 -366
- package/dist/docs/reference/connections.html +44 -44
- package/dist/docs/reference/connections.md +10 -10
- package/dist/docs/reference/evals.html +68 -48
- package/dist/docs/reference/evals.md +48 -13
- package/dist/docs/reference/extensions.html +47 -47
- package/dist/docs/reference/extensions.md +12 -12
- package/dist/docs/reference/hooks.html +42 -42
- package/dist/docs/reference/hooks.md +11 -11
- package/dist/docs/reference/http-api.html +39 -39
- package/dist/docs/reference/http-api.md +1 -1
- package/dist/docs/reference/instructions.html +39 -39
- package/dist/docs/reference/instructions.md +1 -1
- package/dist/docs/reference/limits.html +67 -0
- package/dist/docs/reference/limits.md +37 -0
- package/dist/docs/reference/memory.html +40 -40
- package/dist/docs/reference/memory.md +7 -7
- package/dist/docs/reference/playground.html +38 -38
- package/dist/docs/reference/playground.md +1 -1
- package/dist/docs/reference/project-layout.html +41 -41
- package/dist/docs/reference/project-layout.md +6 -6
- package/dist/docs/reference/prompt.html +39 -39
- package/dist/docs/reference/prompt.md +2 -2
- package/dist/docs/reference/schedules.html +42 -42
- package/dist/docs/reference/schedules.md +4 -4
- package/dist/docs/reference/sessions.html +39 -39
- package/dist/docs/reference/sessions.md +3 -3
- package/dist/docs/reference/skills.html +39 -39
- package/dist/docs/reference/skills.md +2 -2
- package/dist/docs/reference/subagents.html +39 -39
- package/dist/docs/reference/subagents.md +1 -1
- package/dist/docs/reference/tools.html +47 -47
- package/dist/docs/reference/tools.md +31 -21
- package/dist/docs/templates/agentic-owners.html +41 -41
- package/dist/docs/templates/agentic-owners.md +7 -7
- package/dist/docs/templates/security-reviewer.html +39 -39
- package/dist/docs/templates/security-reviewer.md +6 -6
- package/dist/docs/templates/thermo-quality-review.html +40 -40
- package/dist/docs/templates/thermo-quality-review.md +7 -7
- package/dist/docs/templates/thermo-review.html +40 -40
- package/dist/docs/templates/thermo-review.md +7 -7
- package/dist/docs/templates/triage.html +41 -41
- package/dist/docs/templates/triage.md +9 -9
- package/dist/docs/troubleshooting.html +38 -38
- package/dist/docs/troubleshooting.md +26 -24
- package/dist/evals/assertions.d.ts.map +1 -1
- package/dist/evals/assertions.js +2 -2
- package/dist/evals/expect.d.ts.map +1 -1
- package/dist/evals/expect.js +9 -1
- package/dist/evals/judge.js +2 -2
- package/dist/evals/loaders.d.ts +3 -3
- package/dist/evals/loaders.d.ts.map +1 -1
- package/dist/evals/loaders.js +6 -3
- package/dist/evals/matchers.d.ts +1 -1
- package/dist/evals/matchers.d.ts.map +1 -1
- package/dist/evals/matchers.js +40 -6
- package/dist/evals/reporters.d.ts +2 -2
- package/dist/evals/reporters.js +2 -2
- package/dist/evals/results.d.ts +1 -1
- package/dist/evals/results.js +1 -1
- package/dist/evals.d.ts +3 -3
- package/dist/evals.js +1 -1
- package/dist/executors.d.ts +1 -1
- package/dist/executors.d.ts.map +1 -1
- package/dist/executors.js +2 -2
- package/dist/extensions/cursor-cloud-agents/extension.d.ts +1 -1
- package/dist/extensions/cursor-cloud-agents/extension.js +1 -1
- package/dist/extensions/cursor-cloud-agents/lib/backend.d.ts.map +1 -1
- package/dist/extensions/cursor-cloud-agents/lib/backend.js +13 -2
- package/dist/extensions/cursor-cloud-agents/reminders/cloud-agent-stop-watch.js +3 -3
- package/dist/extensions/improve/extension.d.ts +1 -1
- package/dist/extensions/improve/extension.js +1 -1
- package/dist/extensions/jev/extension.d.ts +13 -3
- package/dist/extensions/jev/extension.d.ts.map +1 -1
- package/dist/extensions/jev/extension.js +5 -1
- package/dist/extensions/jev/lib/evaluate.d.ts +33 -6
- package/dist/extensions/jev/lib/evaluate.d.ts.map +1 -1
- package/dist/extensions/jev/lib/evaluate.js +44 -11
- package/dist/extensions/jev/skills/gated-write.md +1 -1
- package/dist/extensions/slack-communication/extension.d.ts +1 -1
- package/dist/extensions/slack-communication/extension.js +1 -1
- package/dist/extensions/slack-communication/skills/slack-voice.js +1 -1
- package/dist/extensions/slack-communication/tools/post-file.d.ts.map +1 -1
- package/dist/extensions/slack-communication/tools/post-file.js +45 -2
- package/dist/extensions.d.ts +2 -2
- package/dist/extensions.d.ts.map +1 -1
- package/dist/extensions.js +7 -3
- package/dist/files-backends/cursor-hosted.d.ts.map +1 -1
- package/dist/files-backends/cursor-hosted.js +5 -0
- package/dist/files-backends/local-fs.d.ts.map +1 -1
- package/dist/files-backends/local-fs.js +4 -3
- package/dist/filesystem.d.ts +24 -54
- package/dist/filesystem.d.ts.map +1 -1
- package/dist/filesystem.js +172 -104
- package/dist/hooks.d.ts +1 -1
- package/dist/hooks.js +1 -1
- package/dist/index.d.ts +11 -4
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +10 -3
- package/dist/internal/advertise-tools.d.ts +3 -3
- package/dist/internal/advertise-tools.d.ts.map +1 -1
- package/dist/internal/advertise-tools.js +9 -5
- package/dist/internal/artifacts-store.d.ts.map +1 -1
- package/dist/internal/artifacts-store.js +15 -6
- package/dist/internal/as-of.d.ts.map +1 -1
- package/dist/internal/as-of.js +4 -3
- package/dist/internal/authored-alias-hooks.d.ts +1 -20
- package/dist/internal/authored-alias-hooks.d.ts.map +1 -1
- package/dist/internal/authored-alias-hooks.js +54 -7
- package/dist/internal/authored-loaders.d.ts +2 -2
- package/dist/internal/authored-loaders.d.ts.map +1 -1
- package/dist/internal/authored-loaders.js +38 -13
- package/dist/internal/brand.d.ts.map +1 -1
- package/dist/internal/brand.js +25 -2
- package/dist/internal/builtin-tools/artifacts.d.ts.map +1 -1
- package/dist/internal/builtin-tools/artifacts.js +19 -3
- package/dist/internal/channel-dispatch.d.ts.map +1 -1
- package/dist/internal/channel-dispatch.js +1 -1
- package/dist/internal/channel-state.d.ts.map +1 -1
- package/dist/internal/channel-state.js +13 -8
- package/dist/internal/chat-client.d.ts.map +1 -1
- package/dist/internal/chat-client.js +14 -5
- package/dist/internal/cli/command.d.ts +134 -0
- package/dist/internal/cli/command.d.ts.map +1 -0
- package/dist/internal/cli/command.js +54 -0
- package/dist/internal/cli/commands/channels.d.ts +26 -0
- package/dist/internal/cli/commands/channels.d.ts.map +1 -0
- package/dist/internal/cli/commands/channels.js +524 -0
- package/dist/internal/cli/commands/develop.d.ts +15 -0
- package/dist/internal/cli/commands/develop.d.ts.map +1 -0
- package/dist/internal/cli/commands/develop.js +577 -0
- package/dist/internal/cli/commands/hosted.d.ts +22 -0
- package/dist/internal/cli/commands/hosted.d.ts.map +1 -0
- package/dist/internal/cli/commands/hosted.js +407 -0
- package/dist/internal/cli/commands/index.d.ts +4 -0
- package/dist/internal/cli/commands/index.d.ts.map +1 -0
- package/dist/internal/cli/commands/index.js +84 -0
- package/dist/internal/cli/commands/mcp.d.ts +6 -0
- package/dist/internal/cli/commands/mcp.d.ts.map +1 -0
- package/dist/internal/cli/commands/mcp.js +89 -0
- package/dist/internal/cli/commands/project.d.ts +5 -0
- package/dist/internal/cli/commands/project.d.ts.map +1 -0
- package/dist/internal/cli/commands/project.js +136 -0
- package/dist/internal/cli/commands/scaffold.d.ts +12 -0
- package/dist/internal/cli/commands/scaffold.d.ts.map +1 -0
- package/dist/internal/cli/commands/scaffold.js +282 -0
- package/dist/internal/cli/commands/serve.d.ts +11 -0
- package/dist/internal/cli/commands/serve.d.ts.map +1 -0
- package/dist/internal/cli/commands/serve.js +370 -0
- package/dist/internal/cli/commands/session.d.ts +7 -0
- package/dist/internal/cli/commands/session.d.ts.map +1 -0
- package/dist/internal/cli/commands/session.js +359 -0
- package/dist/internal/cli/dispatch.d.ts +46 -0
- package/dist/internal/cli/dispatch.d.ts.map +1 -0
- package/dist/internal/cli/dispatch.js +385 -0
- package/dist/internal/cli/errors.d.ts +51 -0
- package/dist/internal/cli/errors.d.ts.map +1 -0
- package/dist/internal/cli/errors.js +142 -0
- package/dist/internal/cli/flags.d.ts +186 -0
- package/dist/internal/cli/flags.d.ts.map +1 -0
- package/dist/internal/cli/flags.js +167 -0
- package/dist/internal/cli/help-command.d.ts +7 -0
- package/dist/internal/cli/help-command.d.ts.map +1 -0
- package/dist/internal/cli/help-command.js +60 -0
- package/dist/internal/cli/help.d.ts +63 -0
- package/dist/internal/cli/help.d.ts.map +1 -0
- package/dist/internal/cli/help.js +203 -0
- package/dist/internal/cli/io.d.ts +11 -0
- package/dist/internal/cli/io.d.ts.map +1 -0
- package/dist/internal/cli/io.js +33 -0
- package/dist/internal/cli/output.d.ts +54 -0
- package/dist/internal/cli/output.d.ts.map +1 -0
- package/dist/internal/cli/output.js +132 -0
- package/dist/internal/cli/value-kinds.d.ts +20 -0
- package/dist/internal/cli/value-kinds.d.ts.map +1 -0
- package/dist/internal/cli/value-kinds.js +70 -0
- package/dist/internal/cli-ax.d.ts +35 -20
- package/dist/internal/cli-ax.d.ts.map +1 -1
- package/dist/internal/cli-ax.js +439 -365
- package/dist/internal/cli-convert-automation.d.ts +3 -1
- package/dist/internal/cli-convert-automation.d.ts.map +1 -1
- package/dist/internal/cli-convert-automation.js +52 -20
- package/dist/internal/cli-cursor.js +5 -1
- package/dist/internal/cli-deploy.d.ts +23 -9
- package/dist/internal/cli-deploy.d.ts.map +1 -1
- package/dist/internal/cli-deploy.js +234 -508
- package/dist/internal/cli-docs.d.ts +1 -1
- package/dist/internal/cli-docs.js +3 -2
- package/dist/internal/cli-github.js +1 -0
- package/dist/internal/cli-images.d.ts.map +1 -1
- package/dist/internal/cli-images.js +44 -14
- package/dist/internal/cli-invocation.d.ts +22 -0
- package/dist/internal/cli-invocation.d.ts.map +1 -0
- package/dist/internal/cli-invocation.js +86 -0
- package/dist/internal/cli-manifest.d.ts +4 -1
- package/dist/internal/cli-manifest.d.ts.map +1 -1
- package/dist/internal/cli-manifest.js +16 -4
- package/dist/internal/cli-mcp-oauth.d.ts +7 -11
- package/dist/internal/cli-mcp-oauth.d.ts.map +1 -1
- package/dist/internal/cli-mcp-oauth.js +24 -104
- package/dist/internal/cli-mcp.d.ts +17 -28
- package/dist/internal/cli-mcp.d.ts.map +1 -1
- package/dist/internal/cli-mcp.js +64 -304
- package/dist/internal/cli-migrate.d.ts +3 -1
- package/dist/internal/cli-migrate.d.ts.map +1 -1
- package/dist/internal/cli-migrate.js +43 -15
- package/dist/internal/cli-pagerduty.d.ts +1 -1
- package/dist/internal/cli-pagerduty.js +1 -1
- package/dist/internal/cli-skills.d.ts +30 -2
- package/dist/internal/cli-skills.d.ts.map +1 -1
- package/dist/internal/cli-skills.js +63 -3
- package/dist/internal/cli-slack-context.d.ts +91 -0
- package/dist/internal/cli-slack-context.d.ts.map +1 -0
- package/dist/internal/cli-slack-context.js +112 -0
- package/dist/internal/cli-slack-manual.d.ts +2 -0
- package/dist/internal/cli-slack-manual.d.ts.map +1 -1
- package/dist/internal/cli-slack-manual.js +12 -0
- package/dist/internal/cli-slack.d.ts +1 -49
- package/dist/internal/cli-slack.d.ts.map +1 -1
- package/dist/internal/cli-slack.js +27 -147
- package/dist/internal/cloud-merge.d.ts +2 -4
- package/dist/internal/cloud-merge.d.ts.map +1 -1
- package/dist/internal/cloud-merge.js +2 -7
- package/dist/internal/continuation-channel.d.ts.map +1 -1
- package/dist/internal/continuation-channel.js +10 -22
- package/dist/internal/continuation-identity.d.ts +33 -12
- package/dist/internal/continuation-identity.d.ts.map +1 -1
- package/dist/internal/continuation-identity.js +161 -43
- package/dist/internal/continuation-registration.d.ts +26 -0
- package/dist/internal/continuation-registration.d.ts.map +1 -0
- package/dist/internal/continuation-registration.js +35 -0
- package/dist/internal/conversation-mirror.d.ts.map +1 -1
- package/dist/internal/conversation-mirror.js +23 -14
- package/dist/internal/convert-automation/convert-workflow.d.ts.map +1 -1
- package/dist/internal/convert-automation/convert-workflow.js +28 -22
- package/dist/internal/convert-automation/types.d.ts +1 -1
- package/dist/internal/convert-automation/types.js +1 -1
- package/dist/internal/cursor/account-mcp.js +1 -1
- package/dist/internal/cursor/backend-client.d.ts.map +1 -1
- package/dist/internal/cursor/backend-client.js +6 -8
- package/dist/internal/cursor/bound-secret-file.d.ts.map +1 -1
- package/dist/internal/cursor/bound-secret-file.js +2 -1
- package/dist/internal/cursor/credentials.d.ts +4 -17
- package/dist/internal/cursor/credentials.d.ts.map +1 -1
- package/dist/internal/cursor/credentials.js +11 -27
- package/dist/internal/cursor/session-token.d.ts.map +1 -1
- package/dist/internal/cursor/session-token.js +3 -4
- package/dist/internal/cursor/store-api-client.d.ts.map +1 -1
- package/dist/internal/cursor/store-api-client.js +3 -2
- package/dist/internal/cursor-agent-template.d.ts +1 -1
- package/dist/internal/cursor-agent-template.d.ts.map +1 -1
- package/dist/internal/cursor-agent-template.js +2 -2
- package/dist/internal/cursor-deployment-relay.d.ts.map +1 -1
- package/dist/internal/cursor-deployment-relay.js +24 -2
- package/dist/internal/cursor-event-relay.d.ts.map +1 -1
- package/dist/internal/cursor-event-relay.js +26 -2
- package/dist/internal/cursor-relay-core.d.ts.map +1 -1
- package/dist/internal/cursor-relay-core.js +22 -3
- package/dist/internal/deploy-client.d.ts +104 -94
- package/dist/internal/deploy-client.d.ts.map +1 -1
- package/dist/internal/deploy-client.js +143 -178
- package/dist/internal/deploy-manifest.d.ts +2 -2
- package/dist/internal/deploy-manifest.d.ts.map +1 -1
- package/dist/internal/deploy-manifest.js +6 -6
- package/dist/internal/describe-error.d.ts +4 -0
- package/dist/internal/describe-error.d.ts.map +1 -1
- package/dist/internal/describe-error.js +20 -0
- package/dist/internal/discovery/agent-config.d.ts +1 -5
- package/dist/internal/discovery/agent-config.d.ts.map +1 -1
- package/dist/internal/discovery/agent-config.js +5 -11
- package/dist/internal/discovery/agent.d.ts.map +1 -1
- package/dist/internal/discovery/agent.js +24 -30
- package/dist/internal/discovery/channels.d.ts.map +1 -1
- package/dist/internal/discovery/channels.js +2 -0
- package/dist/internal/discovery/connections.d.ts.map +1 -1
- package/dist/internal/discovery/connections.js +6 -4
- package/dist/internal/discovery/context.d.ts.map +1 -1
- package/dist/internal/discovery/context.js +6 -1
- package/dist/internal/discovery/extension-overlay.d.ts +1 -1
- package/dist/internal/discovery/extension-overlay.d.ts.map +1 -1
- package/dist/internal/discovery/extension-overlay.js +1 -0
- package/dist/internal/discovery/extensions.d.ts.map +1 -1
- package/dist/internal/discovery/extensions.js +12 -5
- package/dist/internal/discovery/fs.d.ts.map +1 -1
- package/dist/internal/discovery/fs.js +2 -1
- package/dist/internal/discovery/hooks.d.ts.map +1 -1
- package/dist/internal/discovery/hooks.js +1 -0
- package/dist/internal/discovery/info.d.ts.map +1 -1
- package/dist/internal/discovery/info.js +0 -3
- package/dist/internal/discovery/instructions.d.ts.map +1 -1
- package/dist/internal/discovery/instructions.js +4 -1
- package/dist/internal/discovery/instrumentation.js +1 -1
- package/dist/internal/discovery/modules.d.ts.map +1 -1
- package/dist/internal/discovery/modules.js +4 -1
- package/dist/internal/discovery/project.js +1 -1
- package/dist/internal/discovery/reminders.d.ts.map +1 -1
- package/dist/internal/discovery/reminders.js +1 -0
- package/dist/internal/discovery/runtime.d.ts +0 -16
- package/dist/internal/discovery/runtime.d.ts.map +1 -1
- package/dist/internal/discovery/runtime.js +2 -47
- package/dist/internal/discovery/schedules.d.ts.map +1 -1
- package/dist/internal/discovery/schedules.js +1 -0
- package/dist/internal/discovery/services.d.ts.map +1 -1
- package/dist/internal/discovery/services.js +3 -1
- package/dist/internal/discovery/skills.d.ts.map +1 -1
- package/dist/internal/discovery/skills.js +1 -0
- package/dist/internal/discovery/tools.d.ts +1 -1
- package/dist/internal/discovery/tools.d.ts.map +1 -1
- package/dist/internal/discovery/tools.js +3 -1
- package/dist/internal/discovery.d.ts +1 -1
- package/dist/internal/discovery.d.ts.map +1 -1
- package/dist/internal/discovery.js +1 -1
- package/dist/internal/distribution.d.ts +9 -14
- package/dist/internal/distribution.d.ts.map +1 -1
- package/dist/internal/distribution.js +54 -18
- package/dist/internal/docs-site.d.ts.map +1 -1
- package/dist/internal/docs-site.js +15 -2
- package/dist/internal/dry-run.d.ts +32 -0
- package/dist/internal/dry-run.d.ts.map +1 -0
- package/dist/internal/dry-run.js +62 -0
- package/dist/internal/duration.d.ts.map +1 -1
- package/dist/internal/duration.js +1 -0
- package/dist/internal/estimate-cost.d.ts.map +1 -1
- package/dist/internal/estimate-cost.js +12 -5
- package/dist/internal/eval-run-store.d.ts.map +1 -1
- package/dist/internal/eval-run-store.js +6 -5
- package/dist/internal/eval-runner.d.ts +0 -2
- package/dist/internal/eval-runner.d.ts.map +1 -1
- package/dist/internal/eval-runner.js +3 -3
- package/dist/internal/evals-client.d.ts.map +1 -1
- package/dist/internal/evals-client.js +60 -15
- package/dist/internal/event-log.d.ts.map +1 -1
- package/dist/internal/event-log.js +3 -11
- package/dist/internal/event-mapper.d.ts.map +1 -1
- package/dist/internal/event-mapper.js +33 -15
- package/dist/internal/expected-delivery-failure.d.ts.map +1 -1
- package/dist/internal/expected-delivery-failure.js +5 -4
- package/dist/internal/fetch-text.d.ts +8 -0
- package/dist/internal/fetch-text.d.ts.map +1 -0
- package/dist/internal/fetch-text.js +32 -0
- package/dist/internal/filesystem/errors.d.ts.map +1 -1
- package/dist/internal/filesystem/errors.js +8 -0
- package/dist/internal/filesystem/scratch.js +1 -4
- package/dist/internal/filesystem/sdk-schemas.d.ts +19 -19
- package/dist/internal/filesystem/sdk-schemas.d.ts.map +1 -1
- package/dist/internal/filesystem/sdk-schemas.js +15 -21
- package/dist/internal/filesystem/tools.d.ts +15 -4
- package/dist/internal/filesystem/tools.d.ts.map +1 -1
- package/dist/internal/filesystem/tools.js +33 -47
- package/dist/internal/filesystem/walk.d.ts.map +1 -1
- package/dist/internal/filesystem/walk.js +4 -0
- package/dist/internal/framework-file-storage.d.ts.map +1 -1
- package/dist/internal/framework-file-storage.js +2 -1
- package/dist/internal/frontmatter.d.ts.map +1 -1
- package/dist/internal/frontmatter.js +5 -1
- package/dist/internal/github-fanout.js +2 -1
- package/dist/internal/guest-network.d.ts.map +1 -1
- package/dist/internal/guest-network.js +6 -0
- package/dist/internal/handleAgentServeTrigger.d.ts +8 -0
- package/dist/internal/handleAgentServeTrigger.d.ts.map +1 -1
- package/dist/internal/handleAgentServeTrigger.js +27 -1
- package/dist/internal/host-kv.d.ts.map +1 -1
- package/dist/internal/host-kv.js +4 -2
- package/dist/internal/host-otel.d.ts.map +1 -1
- package/dist/internal/host-otel.js +2 -8
- package/dist/internal/host-platforms.d.ts.map +1 -1
- package/dist/internal/host-platforms.js +17 -10
- package/dist/internal/hosted-catch-protocol.d.ts.map +1 -1
- package/dist/internal/hosted-catch-protocol.js +26 -9
- package/dist/internal/hosted-catch.d.ts +13 -10
- package/dist/internal/hosted-catch.d.ts.map +1 -1
- package/dist/internal/hosted-catch.js +23 -11
- package/dist/internal/hosted-delivery-protocol.d.ts +1 -1
- package/dist/internal/hosted-delivery-protocol.d.ts.map +1 -1
- package/dist/internal/hosted-delivery-protocol.js +75 -45
- package/dist/internal/hosted-delivery.d.ts +1 -2
- package/dist/internal/hosted-delivery.d.ts.map +1 -1
- package/dist/internal/hosted-delivery.js +27 -32
- package/dist/internal/hosted-execution-diag.d.ts.map +1 -1
- package/dist/internal/hosted-execution-diag.js +11 -13
- package/dist/internal/hosted-execution-flush.js +1 -1
- package/dist/internal/hosted-execution-latency.d.ts.map +1 -1
- package/dist/internal/hosted-execution-latency.js +12 -1
- package/dist/internal/hosted-hints.d.ts +16 -0
- package/dist/internal/hosted-hints.d.ts.map +1 -0
- package/dist/internal/hosted-hints.js +23 -0
- package/dist/internal/hosted-managed-automation.d.ts.map +1 -1
- package/dist/internal/hosted-managed-automation.js +1 -0
- package/dist/internal/hosted-session-fixture.d.ts +26 -0
- package/dist/internal/hosted-session-fixture.d.ts.map +1 -0
- package/dist/internal/hosted-session-fixture.js +34 -0
- package/dist/internal/hosted-session-view.d.ts +48 -0
- package/dist/internal/hosted-session-view.d.ts.map +1 -0
- package/dist/internal/hosted-session-view.js +305 -0
- package/dist/internal/http-channel.d.ts.map +1 -1
- package/dist/internal/http-channel.js +8 -5
- package/dist/internal/http-control-plane-session.d.ts.map +1 -1
- package/dist/internal/http-control-plane-session.js +8 -9
- package/dist/internal/init-interview.js +2 -2
- package/dist/internal/init-project.d.ts +14 -16
- package/dist/internal/init-project.d.ts.map +1 -1
- package/dist/internal/init-project.js +63 -27
- package/dist/internal/init-scaffold.d.ts +2 -1
- package/dist/internal/init-scaffold.d.ts.map +1 -1
- package/dist/internal/init-scaffold.js +22 -9
- package/dist/internal/inline-stdio-define-stub.d.ts +1 -1
- package/dist/internal/inline-stdio-define-stub.d.ts.map +1 -1
- package/dist/internal/inline-stdio-define-stub.js +1 -1
- package/dist/internal/inline-stdio-mcp.d.ts.map +1 -1
- package/dist/internal/inline-stdio-mcp.js +31 -8
- package/dist/internal/install-cursor-skills.d.ts +22 -15
- package/dist/internal/install-cursor-skills.d.ts.map +1 -1
- package/dist/internal/install-cursor-skills.js +66 -31
- package/dist/internal/interview/agent-questions.d.ts.map +1 -1
- package/dist/internal/interview/agent-questions.js +4 -1
- package/dist/internal/interview/question.d.ts +1 -1
- package/dist/internal/interview/question.js +1 -1
- package/dist/internal/json-schema.d.ts +3 -2
- package/dist/internal/json-schema.d.ts.map +1 -1
- package/dist/internal/json-schema.js +83 -3
- package/dist/internal/json-value.d.ts +20 -0
- package/dist/internal/json-value.d.ts.map +1 -0
- package/dist/internal/json-value.js +56 -0
- package/dist/internal/logs-client.d.ts +4 -25
- package/dist/internal/logs-client.d.ts.map +1 -1
- package/dist/internal/logs-client.js +17 -174
- package/dist/internal/mcp-endpoint.d.ts +1 -1
- package/dist/internal/mcp-endpoint.js +2 -1
- package/dist/internal/mcp-host.d.ts.map +1 -1
- package/dist/internal/mcp-host.js +22 -9
- package/dist/internal/mcp-oauth.d.ts +2 -2
- package/dist/internal/mcp-oauth.d.ts.map +1 -1
- package/dist/internal/mcp-oauth.js +61 -8
- package/dist/internal/named-webhook-secret.d.ts +12 -0
- package/dist/internal/named-webhook-secret.d.ts.map +1 -1
- package/dist/internal/named-webhook-secret.js +18 -0
- package/dist/internal/otel/bind-telemetry.d.ts +26 -0
- package/dist/internal/otel/bind-telemetry.d.ts.map +1 -0
- package/dist/internal/otel/bind-telemetry.js +36 -0
- package/dist/internal/otel/collector.d.ts.map +1 -1
- package/dist/internal/otel/collector.js +94 -42
- package/dist/internal/otel/config.d.ts.map +1 -1
- package/dist/internal/otel/config.js +4 -3
- package/dist/internal/otel/conversation-span-processor.d.ts.map +1 -1
- package/dist/internal/otel/conversation-span-processor.js +5 -2
- package/dist/internal/otel/customer.d.ts +1 -5
- package/dist/internal/otel/customer.d.ts.map +1 -1
- package/dist/internal/otel/customer.js +35 -13
- package/dist/internal/otel/http.d.ts +1 -1
- package/dist/internal/otel/http.d.ts.map +1 -1
- package/dist/internal/otel/http.js +5 -3
- package/dist/internal/otel/propagation.d.ts +3 -2
- package/dist/internal/otel/propagation.d.ts.map +1 -1
- package/dist/internal/otel/propagation.js +3 -2
- package/dist/internal/otel/reserved-attributes.d.ts +8 -0
- package/dist/internal/otel/reserved-attributes.d.ts.map +1 -0
- package/dist/internal/otel/reserved-attributes.js +24 -0
- package/dist/internal/otel/runtime.d.ts.map +1 -1
- package/dist/internal/otel/runtime.js +9 -2
- package/dist/internal/otel/session-telemetry.d.ts +22 -0
- package/dist/internal/otel/session-telemetry.d.ts.map +1 -0
- package/dist/internal/otel/session-telemetry.js +30 -0
- package/dist/internal/otel/telemetry-attributes.d.ts +7 -0
- package/dist/internal/otel/telemetry-attributes.d.ts.map +1 -0
- package/dist/internal/otel/telemetry-attributes.js +35 -0
- package/dist/internal/otel/telemetry-processor.d.ts +7 -0
- package/dist/internal/otel/telemetry-processor.d.ts.map +1 -0
- package/dist/internal/otel/telemetry-processor.js +47 -0
- package/dist/internal/otel/tool-error-class.d.ts.map +1 -1
- package/dist/internal/otel/tool-error-class.js +4 -0
- package/dist/internal/package-inline-stdio.d.ts.map +1 -1
- package/dist/internal/package-inline-stdio.js +2 -2
- package/dist/internal/peer-connections.js +1 -1
- package/dist/internal/persistent-stage-log-mirror.js +2 -5
- package/dist/internal/platform-timers.d.ts.map +1 -1
- package/dist/internal/platform-timers.js +32 -1
- package/dist/internal/playground/toolchain.d.ts.map +1 -1
- package/dist/internal/playground/toolchain.js +16 -3
- package/dist/internal/playground-proxy.d.ts +6 -27
- package/dist/internal/playground-proxy.d.ts.map +1 -1
- package/dist/internal/playground-proxy.js +25 -83
- package/dist/internal/private-file.d.ts +2 -0
- package/dist/internal/private-file.d.ts.map +1 -0
- package/dist/internal/private-file.js +8 -0
- package/dist/internal/reminder-control-plane-protocol.d.ts.map +1 -1
- package/dist/internal/reminder-control-plane-protocol.js +4 -2
- package/dist/internal/reminder-runner.d.ts.map +1 -1
- package/dist/internal/reminder-runner.js +12 -25
- package/dist/internal/reminder-schedule.js +1 -1
- package/dist/internal/reminder-store.d.ts.map +1 -1
- package/dist/internal/reminder-store.js +16 -21
- package/dist/internal/request-headers.d.ts +2 -2
- package/dist/internal/request-headers.d.ts.map +1 -1
- package/dist/internal/request-headers.js +1 -1
- package/dist/internal/resolve-prod-target.d.ts +9 -65
- package/dist/internal/resolve-prod-target.d.ts.map +1 -1
- package/dist/internal/resolve-prod-target.js +20 -187
- package/dist/internal/resolved-connections.js +1 -1
- package/dist/internal/{factory-resource-channel.d.ts → resource-channel.d.ts} +10 -10
- package/dist/internal/resource-channel.d.ts.map +1 -0
- package/dist/internal/{factory-resource-channel.js → resource-channel.js} +9 -7
- package/dist/internal/run-client.d.ts.map +1 -1
- package/dist/internal/run-client.js +4 -1
- package/dist/internal/sdk-runner.d.ts +2 -2
- package/dist/internal/sdk-runner.d.ts.map +1 -1
- package/dist/internal/sdk-runner.js +19 -16
- package/dist/internal/sdk-subagent-inherit-support.d.ts.map +1 -1
- package/dist/internal/sdk-subagent-inherit-support.js +3 -2
- package/dist/internal/serve-restart.d.ts +3 -0
- package/dist/internal/serve-restart.d.ts.map +1 -1
- package/dist/internal/serve-restart.js +8 -4
- package/dist/internal/server.d.ts +5 -1
- package/dist/internal/server.d.ts.map +1 -1
- package/dist/internal/server.js +71 -64
- package/dist/internal/session-cost.d.ts +4 -4
- package/dist/internal/session-cost.d.ts.map +1 -1
- package/dist/internal/session-cost.js +5 -5
- package/dist/internal/session-engine.d.ts +22 -8
- package/dist/internal/session-engine.d.ts.map +1 -1
- package/dist/internal/session-engine.js +170 -75
- package/dist/internal/session-links.js +2 -1
- package/dist/internal/session-live-publisher.d.ts +0 -7
- package/dist/internal/session-live-publisher.d.ts.map +1 -1
- package/dist/internal/session-live-publisher.js +20 -7
- package/dist/internal/session-run-log.d.ts.map +1 -1
- package/dist/internal/session-run-log.js +19 -5
- package/dist/internal/session-store.d.ts.map +1 -1
- package/dist/internal/session-store.js +17 -3
- package/dist/internal/sessions-client.d.ts +2 -0
- package/dist/internal/sessions-client.d.ts.map +1 -1
- package/dist/internal/sessions-client.js +44 -10
- package/dist/internal/shallow-clone.d.ts +258 -0
- package/dist/internal/shallow-clone.d.ts.map +1 -0
- package/dist/internal/shallow-clone.js +644 -0
- package/dist/internal/slack-bot-scopes.d.ts +3 -0
- package/dist/internal/slack-bot-scopes.d.ts.map +1 -0
- package/dist/internal/slack-bot-scopes.js +2 -0
- package/dist/internal/slack-manifest-opt-in.d.ts +26 -0
- package/dist/internal/slack-manifest-opt-in.d.ts.map +1 -0
- package/dist/internal/slack-manifest-opt-in.js +63 -0
- package/dist/internal/slack-provision-client.d.ts +8 -24
- package/dist/internal/slack-provision-client.d.ts.map +1 -1
- package/dist/internal/slack-provision-client.js +21 -20
- package/dist/internal/slack-thread-key.d.ts +8 -3
- package/dist/internal/slack-thread-key.d.ts.map +1 -1
- package/dist/internal/slack-thread-key.js +6 -6
- package/dist/internal/sqlite-experimental-warning.js +1 -1
- package/dist/internal/start-log-store.d.ts.map +1 -1
- package/dist/internal/start-log-store.js +2 -5
- package/dist/internal/storage-coordinator.d.ts.map +1 -1
- package/dist/internal/storage-coordinator.js +49 -20
- package/dist/internal/stored-eval-run.d.ts +17 -0
- package/dist/internal/stored-eval-run.d.ts.map +1 -0
- package/dist/internal/stored-eval-run.js +305 -0
- package/dist/internal/stream-progress.d.ts.map +1 -1
- package/dist/internal/stream-progress.js +1 -1
- package/dist/internal/tcp-address.d.ts +4 -0
- package/dist/internal/tcp-address.d.ts.map +1 -0
- package/dist/internal/tcp-address.js +8 -0
- package/dist/internal/terminal-style.js +2 -2
- package/dist/internal/testing/pagerduty-continuation-fixtures.d.ts +1 -1
- package/dist/internal/testing/pagerduty-continuation-fixtures.js +1 -1
- package/dist/internal/testing/scm-continuation-fixtures.d.ts +1 -1
- package/dist/internal/testing/scm-continuation-fixtures.js +1 -1
- package/dist/internal/testing/slack-continuation-fixtures.d.ts +12 -6
- package/dist/internal/testing/slack-continuation-fixtures.d.ts.map +1 -1
- package/dist/internal/testing/slack-continuation-fixtures.js +74 -3
- package/dist/internal/tool-call-retry.d.ts +35 -7
- package/dist/internal/tool-call-retry.d.ts.map +1 -1
- package/dist/internal/tool-call-retry.js +108 -108
- package/dist/internal/tool-catalog.d.ts.map +1 -1
- package/dist/internal/tool-catalog.js +7 -2
- package/dist/internal/tool-policy.d.ts +20 -7
- package/dist/internal/tool-policy.d.ts.map +1 -1
- package/dist/internal/tool-policy.js +39 -11
- package/dist/internal/tool-result.d.ts +0 -5
- package/dist/internal/tool-result.d.ts.map +1 -1
- package/dist/internal/tool-result.js +26 -6
- package/dist/internal/trajectory.d.ts.map +1 -1
- package/dist/internal/trajectory.js +5 -1
- package/dist/internal/turn-outbox.d.ts.map +1 -1
- package/dist/internal/turn-outbox.js +1 -0
- package/dist/internal/update-check.d.ts +2 -0
- package/dist/internal/update-check.d.ts.map +1 -1
- package/dist/internal/update-check.js +4 -0
- package/dist/internal/workspace.d.ts +3 -4
- package/dist/internal/workspace.d.ts.map +1 -1
- package/dist/internal/workspace.js +5 -5
- package/dist/kv/memory-kv.d.ts +1 -1
- package/dist/kv/memory-kv.js +1 -1
- package/dist/kv.d.ts +9 -2
- package/dist/kv.d.ts.map +1 -1
- package/dist/kv.js +10 -2
- package/dist/memory.d.ts +1 -1
- package/dist/memory.js +1 -1
- package/dist/otel-public.d.ts +2 -1
- package/dist/otel-public.d.ts.map +1 -1
- package/dist/otel-public.js +2 -1
- package/dist/otel.d.ts.map +1 -1
- package/dist/otel.js +6 -2
- package/dist/playground/assets/cursor-icons-16-QUmxS7q3.woff2 +0 -0
- package/dist/playground/assets/index-Cl9zvmmf.js +62 -0
- package/dist/playground/assets/index-F8b5WvWi.css +1 -0
- package/dist/playground/index.html +2 -2
- package/dist/prompt.d.ts +1 -1
- package/dist/prompt.js +1 -1
- package/dist/public-deprecated.d.ts +11 -12
- package/dist/public-deprecated.d.ts.map +1 -1
- package/dist/public-deprecated.js +8 -9
- package/dist/reminders.d.ts +2 -2
- package/dist/reminders.js +2 -2
- package/dist/schedules.d.ts +1 -1
- package/dist/schedules.js +1 -1
- package/dist/scm.d.ts +2 -0
- package/dist/scm.d.ts.map +1 -0
- package/dist/scm.js +1 -0
- package/dist/skills.d.ts +1 -1
- package/dist/skills.js +1 -1
- package/dist/storage-backends/cursor-hosted.d.ts.map +1 -1
- package/dist/storage-backends/cursor-hosted.js +5 -3
- package/dist/storage-backends/file-kv.d.ts +3 -2
- package/dist/storage-backends/file-kv.d.ts.map +1 -1
- package/dist/storage-backends/file-kv.js +20 -8
- package/dist/storage.d.ts +2 -3
- package/dist/storage.d.ts.map +1 -1
- package/dist/storage.js +9 -5
- package/dist/telemetry.d.ts +12 -0
- package/dist/telemetry.d.ts.map +1 -0
- package/dist/telemetry.js +11 -0
- package/dist/tools.d.ts +1 -1
- package/dist/tools.d.ts.map +1 -1
- package/dist/tools.js +2 -1
- package/dist/types.d.ts +59 -46
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js +4 -4
- package/docs/README.md +33 -33
- package/docs/building-with-agents.md +16 -16
- package/docs/changelog.md +58 -12
- package/docs/deployment.md +116 -54
- package/docs/evals.md +14 -12
- package/docs/guides/agent-to-agent.md +8 -8
- package/docs/guides/bitbucket.md +5 -5
- package/docs/guides/cloud-agents.md +4 -4
- package/docs/guides/convert-automation.md +17 -16
- package/docs/guides/github.md +16 -16
- package/docs/guides/gitlab.md +5 -5
- package/docs/guides/hooks.md +3 -3
- package/docs/guides/improve.md +2 -2
- package/docs/guides/jev.md +15 -10
- package/docs/guides/mcp-oauth.md +9 -10
- package/docs/guides/migrate.md +35 -28
- package/docs/guides/opentelemetry.md +61 -130
- package/docs/guides/pagerduty.md +18 -18
- package/docs/guides/slack.md +85 -15
- package/docs/guides/webhooks.md +61 -3
- package/docs/hillclimbing.md +2 -2
- package/docs/quickstart.md +26 -22
- package/docs/reference/access.md +27 -0
- package/docs/reference/agent-config.md +54 -12
- package/docs/reference/artifacts.md +2 -2
- package/docs/reference/channels.md +29 -9
- package/docs/reference/cli.md +364 -367
- package/docs/reference/connections.md +9 -9
- package/docs/reference/evals.md +48 -13
- package/docs/reference/extensions.md +12 -12
- package/docs/reference/hooks.md +11 -11
- package/docs/reference/http-api.md +1 -1
- package/docs/reference/instructions.md +1 -1
- package/docs/reference/limits.md +42 -0
- package/docs/reference/memory.md +7 -7
- package/docs/reference/playground.md +1 -1
- package/docs/reference/project-layout.md +6 -6
- package/docs/reference/prompt.md +2 -2
- package/docs/reference/schedules.md +4 -4
- package/docs/reference/sessions.md +3 -3
- package/docs/reference/skills.md +2 -2
- package/docs/reference/subagents.md +1 -1
- package/docs/reference/tools.md +31 -21
- package/docs/templates/agentic-owners.md +7 -7
- package/docs/templates/security-reviewer.md +6 -6
- package/docs/templates/thermo-quality-review.md +7 -7
- package/docs/templates/thermo-review.md +7 -7
- package/docs/templates/triage.md +9 -9
- package/docs/troubleshooting.md +26 -24
- package/package.json +52 -99
- package/scripts/install-cursor-skills.mjs +2 -2
- package/skills/cli/SKILL.md +125 -0
- package/skills/create-agent/SKILL.md +29 -26
- package/skills/debug/SKILL.md +6 -6
- package/skills/deploy/SKILL.md +47 -44
- package/skills/evals/SKILL.md +13 -13
- package/skills/framework-map/SKILL.md +9 -8
- package/skills/github/SKILL.md +10 -10
- package/skills/hillclimb/SKILL.md +6 -6
- package/skills/mcp-auth/SKILL.md +3 -3
- package/skills/migrate-v2/SKILL.md +67 -70
- package/skills/otel/SKILL.md +36 -6
- package/skills/setup-slack/SKILL.md +11 -11
- package/templates/agentic-owners/README.md +6 -6
- package/templates/agentic-owners/bot/agent.ts +1 -1
- package/templates/agentic-owners/bot/channels/github.ts +1 -1
- package/templates/agentic-owners/bot/lib/github.ts +2 -2
- package/templates/agentic-owners/bot/lib/owners/parse.ts +1 -1
- package/templates/agentic-owners/bot/lib/owners/policyMode.ts +6 -2
- package/templates/agentic-owners/bot/lib/review.ts +7 -3
- package/templates/agentic-owners/bot/tools/record_review.ts +3 -3
- package/templates/agentic-owners/evals/evals.config.ts +1 -1
- package/templates/agentic-owners/evals/owners.eval.ts +1 -1
- package/templates/agentic-owners/package.json +3 -3
- package/templates/agents-md/README.md +5 -5
- package/templates/agents-md/bot/agent.ts +1 -1
- package/templates/agents-md/bot/channels/github.ts +1 -1
- package/templates/agents-md/bot/channels/queue.ts +5 -3
- package/templates/agents-md/bot/channels/slack.ts +1 -1
- package/templates/agents-md/bot/lib/collect.ts +1 -1
- package/templates/agents-md/bot/schedules/weekly.ts +6 -6
- package/templates/agents-md/bot/tools/apply_agents.ts +2 -2
- package/templates/agents-md/evals/agents.eval.ts +1 -1
- package/templates/agents-md/evals/evals.config.ts +1 -1
- package/templates/agents-md/package.json +3 -3
- package/templates/code-wiki/README.md +4 -4
- package/templates/code-wiki/bot/agent.ts +1 -1
- package/templates/code-wiki/bot/channels/github.ts +3 -3
- package/templates/code-wiki/bot/tools/apply_wiki.ts +3 -3
- package/templates/code-wiki/evals/evals.config.ts +1 -1
- package/templates/code-wiki/evals/wiki.eval.ts +1 -1
- package/templates/code-wiki/package.json +3 -3
- package/templates/demo/README.md +4 -4
- package/templates/demo/bot/agent.ts +1 -1
- package/templates/demo/bot/channels/github.ts +7 -3
- package/templates/demo/bot/channels/queue.ts +5 -5
- package/templates/demo/bot/lib/collect.ts +2 -2
- package/templates/demo/bot/tools/record_demo.ts +2 -2
- package/templates/demo/evals/demo.eval.ts +1 -1
- package/templates/demo/evals/evals.config.ts +1 -1
- package/templates/demo/package.json +3 -3
- package/templates/pr-autofixer/bot/agent.ts +1 -1
- package/templates/pr-autofixer/bot/channels/drive.ts +1 -1
- package/templates/pr-autofixer/bot/channels/github.ts +3 -3
- package/templates/pr-autofixer/bot/channels/slack.ts +1 -1
- package/templates/pr-autofixer/bot/hooks/bind-agent.ts +1 -1
- package/templates/pr-autofixer/bot/lib/affinity.ts +1 -1
- package/templates/pr-autofixer/bot/lib/drive.ts +2 -2
- package/templates/pr-autofixer/bot/lib/merge-watch.ts +4 -4
- package/templates/pr-autofixer/bot/lib/pr-state.ts +1 -1
- package/templates/pr-autofixer/bot/lib/start-drive.ts +2 -2
- package/templates/pr-autofixer/bot/lib/webhook-buffer.ts +1 -1
- package/templates/pr-autofixer/bot/lib/webhook-routing.ts +1 -1
- package/templates/pr-autofixer/bot/tools/drive_pr.ts +1 -1
- package/templates/pr-autofixer/evals/coordinator.eval.ts +1 -1
- package/templates/pr-autofixer/evals/evals.config.ts +1 -1
- package/templates/pr-autofixer/package.json +3 -3
- package/templates/security-help/.gitignore +4 -0
- package/templates/security-help/README.md +6 -6
- package/templates/security-help/bot/agent.ts +1 -1
- package/templates/security-help/bot/channels/github.ts +1 -1
- package/templates/security-help/bot/channels/slack.ts +1 -1
- package/templates/security-help/bot/lib/pr-first-pass.ts +3 -3
- package/templates/security-help/evals/evals.config.ts +1 -1
- package/templates/security-help/evals/security-help.eval.ts +1 -1
- package/templates/security-help/package.json +2 -2
- package/templates/security-reviewer/README.md +4 -4
- package/templates/security-reviewer/bot/agent.ts +1 -1
- package/templates/security-reviewer/bot/channels/github.ts +2 -2
- package/templates/security-reviewer/bot/tools/post_findings.ts +2 -2
- package/templates/security-reviewer/evals/evals.config.ts +1 -1
- package/templates/security-reviewer/evals/review.eval.ts +1 -1
- package/templates/security-reviewer/package.json +3 -3
- package/templates/thermo-quality-review/README.md +5 -5
- package/templates/thermo-quality-review/bot/agent.ts +1 -1
- package/templates/thermo-quality-review/bot/channels/github.ts +2 -2
- package/templates/thermo-quality-review/bot/tools/post_findings.ts +2 -2
- package/templates/thermo-quality-review/evals/evals.config.ts +1 -1
- package/templates/thermo-quality-review/evals/review.eval.ts +1 -1
- package/templates/thermo-quality-review/package.json +3 -3
- package/templates/thermo-review/README.md +5 -5
- package/templates/thermo-review/bot/agent.ts +1 -1
- package/templates/thermo-review/bot/channels/github.ts +2 -2
- package/templates/thermo-review/bot/tools/post_findings.ts +2 -2
- package/templates/thermo-review/evals/evals.config.ts +1 -1
- package/templates/thermo-review/evals/review.eval.ts +1 -1
- package/templates/thermo-review/package.json +3 -3
- package/templates/triage/README.md +5 -5
- package/templates/triage/bot/agent.ts +1 -1
- package/templates/triage/bot/channels/queue.ts +5 -5
- package/templates/triage/bot/channels/webhook.ts +5 -5
- package/templates/triage/bot/hooks/memory.ts +1 -1
- package/templates/triage/bot/lib/actuate.ts +2 -2
- package/templates/triage/bot/tools/record_triage.ts +1 -1
- package/templates/triage/evals/evals.config.ts +1 -1
- package/templates/triage/evals/triage.eval.ts +1 -1
- package/templates/triage/overlays/jira/README.md +4 -4
- package/templates/triage/overlays/jira/bot/mcp-connections/tracker.ts +1 -1
- package/templates/triage/overlays/linear/README.md +4 -4
- package/templates/triage/overlays/linear/bot/mcp-connections/tracker.ts +1 -1
- package/templates/triage/package.json +3 -3
- package/dist/bin/agent-sdk.js +0 -2
- package/dist/docs/assets/building-with-agents.md.FZXrY8Nk.js +0 -9
- package/dist/docs/assets/building-with-agents.md.FZXrY8Nk.lean.js +0 -1
- package/dist/docs/assets/changelog.md.BvHRIiMH.js +0 -1
- package/dist/docs/assets/changelog.md.BvHRIiMH.lean.js +0 -1
- package/dist/docs/assets/chunks/@localSearchIndexroot.BoDEJQWv.js +0 -1
- package/dist/docs/assets/chunks/channel.D2BTz0L-.js +0 -1
- package/dist/docs/assets/chunks/classDiagram-6PBFFD2Q.DqByliAC.js +0 -1
- package/dist/docs/assets/chunks/classDiagram-v2-HSJHXN6E.DqByliAC.js +0 -1
- package/dist/docs/assets/chunks/clone.CldT1lbQ.js +0 -1
- package/dist/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.Bb4oMe0P.js +0 -1
- package/dist/docs/assets/deployment.md.3MMWuPeV.js +0 -31
- package/dist/docs/assets/deployment.md.3MMWuPeV.lean.js +0 -1
- package/dist/docs/assets/guides_agent-to-agent.md.CwL5cwuE.lean.js +0 -1
- package/dist/docs/assets/guides_cloud-agents.md.BzbzcOyj.js +0 -9
- package/dist/docs/assets/guides_cloud-agents.md.BzbzcOyj.lean.js +0 -1
- package/dist/docs/assets/guides_convert-automation.md.DMEX9nF-.lean.js +0 -1
- package/dist/docs/assets/guides_export-grokbot.md.qo5bxi4h.js +0 -55
- package/dist/docs/assets/guides_export-grokbot.md.qo5bxi4h.lean.js +0 -1
- package/dist/docs/assets/guides_grokbot-agents.md.BwAykJ7U.js +0 -15
- package/dist/docs/assets/guides_grokbot-agents.md.BwAykJ7U.lean.js +0 -1
- package/dist/docs/assets/guides_migrate.md.CQlGaBKX.js +0 -7
- package/dist/docs/assets/guides_migrate.md.CQlGaBKX.lean.js +0 -1
- package/dist/docs/assets/guides_opentelemetry.md.D4wO7_8v.js +0 -37
- package/dist/docs/assets/guides_opentelemetry.md.D4wO7_8v.lean.js +0 -1
- package/dist/docs/assets/index.md.CAnsLl8D.js +0 -27
- package/dist/docs/assets/index.md.CAnsLl8D.lean.js +0 -1
- package/dist/docs/assets/quickstart.md.D8JDfkKO.lean.js +0 -1
- package/dist/docs/assets/reference_channels.md.CNNIS6K7.js +0 -22
- package/dist/docs/assets/reference_cli.md.1o_TqKdt.js +0 -103
- package/dist/docs/assets/reference_cli.md.1o_TqKdt.lean.js +0 -1
- package/dist/docs/assets/reference_memory.md.BwyJXYZe.js +0 -4
- package/dist/docs/assets/reference_playground.md.q4MI-0td.js +0 -1
- package/dist/docs/assets/templates_grokbot-agents.md.DgpPEs7B.js +0 -5
- package/dist/docs/assets/templates_grokbot-agents.md.DgpPEs7B.lean.js +0 -1
- package/dist/docs/assets/templates_security-reviewer.md.Gh6EF9Us.js +0 -3
- package/dist/docs/assets/templates_thermo-quality-review.md.B77UyTrX.js +0 -3
- package/dist/docs/assets/templates_thermo-review.md.DRJ7mC5U.js +0 -3
- package/dist/docs/assets/troubleshooting.md.DFnoTDlQ.js +0 -1
- package/dist/docs/guides/export-grokbot.html +0 -114
- package/dist/docs/guides/export-grokbot.md +0 -239
- package/dist/docs/guides/grokbot-agents.html +0 -74
- package/dist/docs/guides/grokbot-agents.md +0 -135
- package/dist/docs/templates/grokbot-agents.html +0 -64
- package/dist/docs/templates/grokbot-agents.md +0 -55
- package/dist/extensions/cursor-grokbot-agents/extension.d.ts +0 -77
- package/dist/extensions/cursor-grokbot-agents/extension.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/extension.js +0 -70
- package/dist/extensions/cursor-grokbot-agents/lib/agents.d.ts +0 -11
- package/dist/extensions/cursor-grokbot-agents/lib/agents.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/lib/agents.js +0 -31
- package/dist/extensions/cursor-grokbot-agents/lib/backend.d.ts +0 -56
- package/dist/extensions/cursor-grokbot-agents/lib/backend.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/lib/backend.js +0 -138
- package/dist/extensions/cursor-grokbot-agents/lib/contacts.d.ts +0 -28
- package/dist/extensions/cursor-grokbot-agents/lib/contacts.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/lib/contacts.js +0 -46
- package/dist/extensions/cursor-grokbot-agents/lib/conversation.d.ts +0 -53
- package/dist/extensions/cursor-grokbot-agents/lib/conversation.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/lib/conversation.js +0 -186
- package/dist/extensions/cursor-grokbot-agents/lib/inputs.d.ts +0 -4
- package/dist/extensions/cursor-grokbot-agents/lib/inputs.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/lib/inputs.js +0 -15
- package/dist/extensions/cursor-grokbot-agents/lib/lock.d.ts +0 -12
- package/dist/extensions/cursor-grokbot-agents/lib/lock.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/lib/lock.js +0 -24
- package/dist/extensions/cursor-grokbot-agents/skills/consult.md +0 -40
- package/dist/extensions/cursor-grokbot-agents/tools/ask.d.ts +0 -4
- package/dist/extensions/cursor-grokbot-agents/tools/ask.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/tools/ask.js +0 -26
- package/dist/extensions/cursor-grokbot-agents/tools/check.d.ts +0 -4
- package/dist/extensions/cursor-grokbot-agents/tools/check.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/tools/check.js +0 -21
- package/dist/extensions/cursor-grokbot-agents/tools/interrupt.d.ts +0 -4
- package/dist/extensions/cursor-grokbot-agents/tools/interrupt.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/tools/interrupt.js +0 -13
- package/dist/extensions/cursor-grokbot-agents/tools/list.d.ts +0 -4
- package/dist/extensions/cursor-grokbot-agents/tools/list.d.ts.map +0 -1
- package/dist/extensions/cursor-grokbot-agents/tools/list.js +0 -22
- package/dist/internal/cli-init-grokbot.d.ts +0 -29
- package/dist/internal/cli-init-grokbot.d.ts.map +0 -1
- package/dist/internal/cli-init-grokbot.js +0 -181
- package/dist/internal/convert-grokbot/answers.d.ts +0 -127
- package/dist/internal/convert-grokbot/answers.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/answers.js +0 -445
- package/dist/internal/convert-grokbot/bot.d.ts +0 -50
- package/dist/internal/convert-grokbot/bot.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/bot.js +0 -72
- package/dist/internal/convert-grokbot/checks.d.ts +0 -26
- package/dist/internal/convert-grokbot/checks.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/checks.js +0 -170
- package/dist/internal/convert-grokbot/convert.d.ts +0 -60
- package/dist/internal/convert-grokbot/convert.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/convert.js +0 -470
- package/dist/internal/convert-grokbot/handoff.d.ts +0 -44
- package/dist/internal/convert-grokbot/handoff.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/handoff.js +0 -160
- package/dist/internal/convert-grokbot/recipe.d.ts +0 -61
- package/dist/internal/convert-grokbot/recipe.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/recipe.js +0 -94
- package/dist/internal/convert-grokbot/routine.d.ts +0 -51
- package/dist/internal/convert-grokbot/routine.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/routine.js +0 -128
- package/dist/internal/convert-grokbot/slack.d.ts +0 -28
- package/dist/internal/convert-grokbot/slack.d.ts.map +0 -1
- package/dist/internal/convert-grokbot/slack.js +0 -50
- package/dist/internal/factory-resource-channel.d.ts.map +0 -1
- package/dist/internal/legacy-command-guard.d.ts +0 -24
- package/dist/internal/legacy-command-guard.d.ts.map +0 -1
- package/dist/internal/legacy-command-guard.js +0 -47
- package/dist/internal/runtime-dispatch-runner.d.ts +0 -25
- package/dist/internal/runtime-dispatch-runner.d.ts.map +0 -1
- package/dist/internal/runtime-dispatch-runner.js +0 -35
- package/dist/internal/testing/factory-resource-fixtures.d.ts +0 -64
- package/dist/internal/testing/factory-resource-fixtures.d.ts.map +0 -1
- package/dist/internal/testing/factory-resource-fixtures.js +0 -55
- package/dist/playground/assets/cursor-icons-16-DfKNljMZ.woff2 +0 -0
- package/dist/playground/assets/index-B3_YohwW.js +0 -62
- package/dist/playground/assets/index-Cn3-adsJ.css +0 -1
- package/docs/guides/export-grokbot.md +0 -244
- package/docs/guides/grokbot-agents.md +0 -140
- package/docs/templates/grokbot-agents.md +0 -60
- package/schemas/grokbot-answers.schema.json +0 -354
- package/templates/grokbot-agents/README.md +0 -27
- package/templates/grokbot-agents/bot/agent.ts +0 -6
- package/templates/grokbot-agents/bot/extensions/grokbot.ts +0 -5
- package/templates/grokbot-agents/bot/instructions.md +0 -7
- package/templates/grokbot-agents/evals/evals.config.ts +0 -5
- package/templates/grokbot-agents/evals/grokbot-agents.eval.ts +0 -18
- package/templates/grokbot-agents/init.json +0 -18
- package/templates/grokbot-agents/package.json +0 -17
- package/templates/grokbot-agents/tsconfig.json +0 -12
- /package/dist/docs/assets/{evals.md.CHR34K1Y.lean.js → evals.md.DiljU__Z.lean.js} +0 -0
- /package/dist/docs/assets/{guides_bitbucket.md.CKUayYtD.lean.js → guides_bitbucket.md.DkjkBRU8.lean.js} +0 -0
- /package/dist/docs/assets/{guides_gitlab.md.CtNI4fSH.lean.js → guides_gitlab.md.DCE0p7_w.lean.js} +0 -0
- /package/dist/docs/assets/{guides_pagerduty.md.BHXMcMtx.lean.js → guides_pagerduty.md.mnKXe9U0.lean.js} +0 -0
- /package/dist/docs/assets/{hillclimbing.md.Dxvd-0gT.lean.js → hillclimbing.md.By61f3uq.lean.js} +0 -0
- /package/dist/docs/assets/{reference_artifacts.md.DLV9MZES.lean.js → reference_artifacts.md.C-hKR2hp.lean.js} +0 -0
- /package/dist/docs/assets/{reference_connections.md.BJE2Zllp.lean.js → reference_connections.md.BoAotr7_.lean.js} +0 -0
- /package/dist/docs/assets/{reference_hooks.md.CROr0QgC.lean.js → reference_hooks.md.B-bwJuiG.lean.js} +0 -0
- /package/dist/docs/assets/{reference_http-api.md.Dji-DD_Z.lean.js → reference_http-api.md._RWELr3d.lean.js} +0 -0
- /package/dist/docs/assets/{reference_instructions.md.BYMVxAkw.lean.js → reference_instructions.md.yL2hXLnl.lean.js} +0 -0
- /package/dist/docs/assets/{reference_memory.md.BwyJXYZe.lean.js → reference_memory.md.Dd7cyV9V.lean.js} +0 -0
- /package/dist/docs/assets/{reference_playground.md.q4MI-0td.lean.js → reference_playground.md.CQiJk9CM.lean.js} +0 -0
- /package/dist/docs/assets/{reference_project-layout.md.mFcS8yDD.lean.js → reference_project-layout.md.BamcPDUn.lean.js} +0 -0
- /package/dist/docs/assets/{reference_prompt.md.DqPh71XN.lean.js → reference_prompt.md.BMjm1Jp3.lean.js} +0 -0
- /package/dist/docs/assets/{reference_schedules.md.BqWnJES4.lean.js → reference_schedules.md.RC47aFm4.lean.js} +0 -0
- /package/dist/docs/assets/{reference_sessions.md.Bi6lHttM.lean.js → reference_sessions.md.C7KFhWXV.lean.js} +0 -0
- /package/dist/docs/assets/{reference_subagents.md.ChNGh6HK.lean.js → reference_subagents.md.CAhvW4yS.lean.js} +0 -0
- /package/dist/docs/assets/{templates_agentic-owners.md.CTEnR0jp.lean.js → templates_agentic-owners.md.Dh2nDSqg.lean.js} +0 -0
- /package/dist/docs/assets/{templates_security-reviewer.md.Gh6EF9Us.lean.js → templates_security-reviewer.md.DnwXdnB5.lean.js} +0 -0
- /package/dist/docs/assets/{templates_triage.md.CNN6b0JZ.lean.js → templates_triage.md.BYWMloV4.lean.js} +0 -0
- /package/dist/docs/assets/{troubleshooting.md.DFnoTDlQ.lean.js → troubleshooting.md.CGOdNVAL.lean.js} +0 -0
package/dist/docs/evals.html
CHANGED
|
@@ -3,58 +3,58 @@
|
|
|
3
3
|
<head>
|
|
4
4
|
<meta charset="utf-8">
|
|
5
5
|
<meta name="viewport" content="width=device-width,initial-scale=1">
|
|
6
|
-
<title>Evals |
|
|
6
|
+
<title>Evals | Agent SDK</title>
|
|
7
7
|
<meta name="description" content="Protect agent decisions with fixed inputs, trajectory assertions, and regression checks that run locally or in CI.">
|
|
8
8
|
<meta name="generator" content="VitePress v1.6.4">
|
|
9
|
-
<link rel="preload stylesheet" href="/docs/assets/style.
|
|
9
|
+
<link rel="preload stylesheet" href="/docs/assets/style.DK0YX1WL.css" as="style">
|
|
10
10
|
<link rel="preload stylesheet" href="/docs/vp-icons.css" as="style">
|
|
11
11
|
|
|
12
|
-
<script type="module" src="/docs/assets/app.
|
|
12
|
+
<script type="module" src="/docs/assets/app.DMrZKFjI.js"></script>
|
|
13
13
|
<link rel="preload" href="/docs/assets/inter-roman-latin.Di8DUHzh.woff2" as="font" type="font/woff2" crossorigin="">
|
|
14
14
|
<link rel="modulepreload" href="/docs/assets/chunks/framework.BNw1pucY.js">
|
|
15
|
-
<link rel="modulepreload" href="/docs/assets/chunks/theme.
|
|
15
|
+
<link rel="modulepreload" href="/docs/assets/chunks/theme.DRciHmS1.js">
|
|
16
16
|
<link rel="modulepreload" href="/docs/assets/chunks/katex.DkKDou_j.js">
|
|
17
|
-
<link rel="modulepreload" href="/docs/assets/chunks/dagre-KV5264BT.
|
|
18
|
-
<link rel="modulepreload" href="/docs/assets/chunks/cose-bilkent-S5V4N54A.
|
|
19
|
-
<link rel="modulepreload" href="/docs/assets/chunks/c4Diagram-AHTNJAMY.
|
|
20
|
-
<link rel="modulepreload" href="/docs/assets/chunks/flowDiagram-DWJPFMVM.
|
|
21
|
-
<link rel="modulepreload" href="/docs/assets/chunks/erDiagram-SMLLAGMA.
|
|
22
|
-
<link rel="modulepreload" href="/docs/assets/chunks/gitGraphDiagram-UUTBAWPF.
|
|
23
|
-
<link rel="modulepreload" href="/docs/assets/chunks/ganttDiagram-T4ZO3ILL.
|
|
24
|
-
<link rel="modulepreload" href="/docs/assets/chunks/infoDiagram-42DDH7IO.
|
|
25
|
-
<link rel="modulepreload" href="/docs/assets/chunks/pieDiagram-DEJITSTG.
|
|
26
|
-
<link rel="modulepreload" href="/docs/assets/chunks/quadrantDiagram-34T5L4WZ.
|
|
27
|
-
<link rel="modulepreload" href="/docs/assets/chunks/xychartDiagram-5P7HB3ND.
|
|
28
|
-
<link rel="modulepreload" href="/docs/assets/chunks/requirementDiagram-MS252O5E.
|
|
29
|
-
<link rel="modulepreload" href="/docs/assets/chunks/sequenceDiagram-FGHM5R23.
|
|
30
|
-
<link rel="modulepreload" href="/docs/assets/chunks/classDiagram-6PBFFD2Q.
|
|
31
|
-
<link rel="modulepreload" href="/docs/assets/chunks/classDiagram-v2-HSJHXN6E.
|
|
32
|
-
<link rel="modulepreload" href="/docs/assets/chunks/stateDiagram-FHFEXIEX.
|
|
33
|
-
<link rel="modulepreload" href="/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.
|
|
34
|
-
<link rel="modulepreload" href="/docs/assets/chunks/journeyDiagram-VCZTEJTY.
|
|
35
|
-
<link rel="modulepreload" href="/docs/assets/chunks/timeline-definition-GMOUNBTQ.
|
|
36
|
-
<link rel="modulepreload" href="/docs/assets/chunks/mindmap-definition-QFDTVHPH.
|
|
37
|
-
<link rel="modulepreload" href="/docs/assets/chunks/kanban-definition-6JOO6SKY.
|
|
38
|
-
<link rel="modulepreload" href="/docs/assets/chunks/sankeyDiagram-XADWPNL6.
|
|
39
|
-
<link rel="modulepreload" href="/docs/assets/chunks/diagram-TYMM5635.
|
|
40
|
-
<link rel="modulepreload" href="/docs/assets/chunks/diagram-MMDJMWI5.
|
|
41
|
-
<link rel="modulepreload" href="/docs/assets/chunks/blockDiagram-DXYQGD6D.
|
|
42
|
-
<link rel="modulepreload" href="/docs/assets/chunks/diagram-5BDNPKRD.
|
|
43
|
-
<link rel="modulepreload" href="/docs/assets/chunks/architectureDiagram-Q4EWVU46.
|
|
44
|
-
<link rel="modulepreload" href="/docs/assets/chunks/ishikawaDiagram-UXIWVN3A.
|
|
45
|
-
<link rel="modulepreload" href="/docs/assets/chunks/vennDiagram-DHZGUBPP.
|
|
46
|
-
<link rel="modulepreload" href="/docs/assets/chunks/diagram-G4DWMVQ6.
|
|
47
|
-
<link rel="modulepreload" href="/docs/assets/chunks/wardleyDiagram-NUSXRM2D.
|
|
17
|
+
<link rel="modulepreload" href="/docs/assets/chunks/dagre-KV5264BT.CLMLc0lD.js">
|
|
18
|
+
<link rel="modulepreload" href="/docs/assets/chunks/cose-bilkent-S5V4N54A.BfPVKeId.js">
|
|
19
|
+
<link rel="modulepreload" href="/docs/assets/chunks/c4Diagram-AHTNJAMY.C1QTb9GM.js">
|
|
20
|
+
<link rel="modulepreload" href="/docs/assets/chunks/flowDiagram-DWJPFMVM.MFihq9AF.js">
|
|
21
|
+
<link rel="modulepreload" href="/docs/assets/chunks/erDiagram-SMLLAGMA.CsL5JuCV.js">
|
|
22
|
+
<link rel="modulepreload" href="/docs/assets/chunks/gitGraphDiagram-UUTBAWPF.1HhlE45Z.js">
|
|
23
|
+
<link rel="modulepreload" href="/docs/assets/chunks/ganttDiagram-T4ZO3ILL.CJeLodyG.js">
|
|
24
|
+
<link rel="modulepreload" href="/docs/assets/chunks/infoDiagram-42DDH7IO.fp5gvytQ.js">
|
|
25
|
+
<link rel="modulepreload" href="/docs/assets/chunks/pieDiagram-DEJITSTG.Ba7yxiW2.js">
|
|
26
|
+
<link rel="modulepreload" href="/docs/assets/chunks/quadrantDiagram-34T5L4WZ.Bvskad4p.js">
|
|
27
|
+
<link rel="modulepreload" href="/docs/assets/chunks/xychartDiagram-5P7HB3ND.DEH4BcVJ.js">
|
|
28
|
+
<link rel="modulepreload" href="/docs/assets/chunks/requirementDiagram-MS252O5E.Ce70ugV9.js">
|
|
29
|
+
<link rel="modulepreload" href="/docs/assets/chunks/sequenceDiagram-FGHM5R23.C8a1hDs6.js">
|
|
30
|
+
<link rel="modulepreload" href="/docs/assets/chunks/classDiagram-6PBFFD2Q.eClS1Q0E.js">
|
|
31
|
+
<link rel="modulepreload" href="/docs/assets/chunks/classDiagram-v2-HSJHXN6E.eClS1Q0E.js">
|
|
32
|
+
<link rel="modulepreload" href="/docs/assets/chunks/stateDiagram-FHFEXIEX.D2D8A5CK.js">
|
|
33
|
+
<link rel="modulepreload" href="/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.77yf4kRr.js">
|
|
34
|
+
<link rel="modulepreload" href="/docs/assets/chunks/journeyDiagram-VCZTEJTY.qJt4uJ_H.js">
|
|
35
|
+
<link rel="modulepreload" href="/docs/assets/chunks/timeline-definition-GMOUNBTQ.C4ePrY8-.js">
|
|
36
|
+
<link rel="modulepreload" href="/docs/assets/chunks/mindmap-definition-QFDTVHPH.DgrQRFZW.js">
|
|
37
|
+
<link rel="modulepreload" href="/docs/assets/chunks/kanban-definition-6JOO6SKY.z2tG9kly.js">
|
|
38
|
+
<link rel="modulepreload" href="/docs/assets/chunks/sankeyDiagram-XADWPNL6.BJ5OEULO.js">
|
|
39
|
+
<link rel="modulepreload" href="/docs/assets/chunks/diagram-TYMM5635.BcpVgA_M.js">
|
|
40
|
+
<link rel="modulepreload" href="/docs/assets/chunks/diagram-MMDJMWI5.DLnM9eLp.js">
|
|
41
|
+
<link rel="modulepreload" href="/docs/assets/chunks/blockDiagram-DXYQGD6D.HjJ2goMX.js">
|
|
42
|
+
<link rel="modulepreload" href="/docs/assets/chunks/diagram-5BDNPKRD.C7OF7UfH.js">
|
|
43
|
+
<link rel="modulepreload" href="/docs/assets/chunks/architectureDiagram-Q4EWVU46.BcgfYNO_.js">
|
|
44
|
+
<link rel="modulepreload" href="/docs/assets/chunks/ishikawaDiagram-UXIWVN3A.C6Wk1V4I.js">
|
|
45
|
+
<link rel="modulepreload" href="/docs/assets/chunks/vennDiagram-DHZGUBPP.Dzso6dEA.js">
|
|
46
|
+
<link rel="modulepreload" href="/docs/assets/chunks/diagram-G4DWMVQ6.Dpao0Bl1.js">
|
|
47
|
+
<link rel="modulepreload" href="/docs/assets/chunks/wardleyDiagram-NUSXRM2D.Bf2ICxjh.js">
|
|
48
48
|
<link rel="modulepreload" href="/docs/assets/chunks/virtual_mermaid-config.DDnGl6nM.js">
|
|
49
|
-
<link rel="modulepreload" href="/docs/assets/evals.md.
|
|
49
|
+
<link rel="modulepreload" href="/docs/assets/evals.md.DiljU__Z.lean.js">
|
|
50
50
|
<script id="check-dark-mode">(()=>{const e=localStorage.getItem("vitepress-theme-appearance")||"auto",a=window.matchMedia("(prefers-color-scheme: dark)").matches;(!e||e==="auto"?a:e==="dark")&&document.documentElement.classList.add("dark")})();</script>
|
|
51
51
|
<script id="check-mac-os">document.documentElement.classList.toggle("mac",/Mac|iPhone|iPod|iPad/i.test(navigator.platform));</script>
|
|
52
52
|
<link rel="alternate" type="text/plain" href="/docs/llms.txt">
|
|
53
53
|
<link rel="alternate" type="text/markdown" href="/docs/evals.md">
|
|
54
54
|
</head>
|
|
55
55
|
<body>
|
|
56
|
-
<div id="app"><div class="Layout" data-v-738a160a><!--[--><!--]--><!--[--><span tabindex="-1" data-v-a98c2453></span><a href="#VPContent" class="VPSkipLink visually-hidden" data-v-a98c2453>Skip to content</a><!--]--><!----><header class="VPNav" data-v-738a160a data-v-76ad4059><div class="VPNavBar" data-v-76ad4059 data-v-6fc2afcb><div class="wrapper" data-v-6fc2afcb><div class="container" data-v-6fc2afcb><div class="title" data-v-6fc2afcb><div class="VPNavBarTitle has-sidebar" data-v-6fc2afcb data-v-9e6caa3f><a class="title" href="/docs/" data-v-9e6caa3f><!--[--><!--]--><!----><span data-v-9e6caa3f>Bot Development Kit (BDK)</span><!--[--><!--[--><!--[--><!--[--><span class="bdk-version" title="BDK 0.2.27 (@cursor/bdk package)" data-v-d89e5038>0.2.27</span><!--]--><!--]--><!--]--><!--]--></a></div></div><div class="content" data-v-6fc2afcb><div class="content-body" data-v-6fc2afcb><!--[--><!--]--><div class="VPNavBarSearch search" data-v-6fc2afcb><!--[--><!----><div id="local-search"><button type="button" class="DocSearch DocSearch-Button" aria-label="Search"><span class="DocSearch-Button-Container"><span class="vp-icon DocSearch-Search-Icon"></span><span class="DocSearch-Button-Placeholder">Search</span></span><span class="DocSearch-Button-Keys"><kbd class="DocSearch-Button-Key"></kbd><kbd class="DocSearch-Button-Key">K</kbd></span></button></div><!--]--></div><nav aria-labelledby="main-nav-aria-label" class="VPNavBarMenu menu" data-v-6fc2afcb data-v-78f72f99><span id="main-nav-aria-label" class="visually-hidden" data-v-78f72f99> Main Navigation </span><!--[--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/quickstart.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Quickstart</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/guides/mcp-oauth.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Guides</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/guides/github.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Integrations</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/templates/security-reviewer.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Templates</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/reference/project-layout.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Reference</span><!--]--></a><!--]--><!--]--></nav><!----><div class="VPNavBarAppearance appearance" data-v-6fc2afcb data-v-bf7c3d0f><button class="VPSwitch VPSwitchAppearance" type="button" role="switch" title aria-checked="false" data-v-bf7c3d0f data-v-e9e166e4 data-v-74f7b0ff><span class="check" data-v-74f7b0ff><span class="icon" data-v-74f7b0ff><!--[--><span class="vpi-sun sun" data-v-e9e166e4></span><span class="vpi-moon moon" data-v-e9e166e4></span><!--]--></span></span></button></div><!----><div class="VPFlyout VPNavBarExtra extra" data-v-6fc2afcb data-v-799c9281 data-v-070c9767><button type="button" class="button" aria-haspopup="true" aria-expanded="false" aria-label="extra navigation" data-v-070c9767><span class="vpi-more-horizontal icon" data-v-070c9767></span></button><div class="menu" data-v-070c9767><div class="VPMenu" data-v-070c9767 data-v-c05d27ee><!----><!--[--><!--[--><!----><div class="group" data-v-799c9281><div class="item appearance" data-v-799c9281><p class="label" data-v-799c9281>Appearance</p><div class="appearance-action" data-v-799c9281><button class="VPSwitch VPSwitchAppearance" type="button" role="switch" title aria-checked="false" data-v-799c9281 data-v-e9e166e4 data-v-74f7b0ff><span class="check" data-v-74f7b0ff><span class="icon" data-v-74f7b0ff><!--[--><span class="vpi-sun sun" data-v-e9e166e4></span><span class="vpi-moon moon" data-v-e9e166e4></span><!--]--></span></span></button></div></div></div><!----><!--]--><!--]--></div></div></div><!--[--><!--]--><button type="button" class="VPNavBarHamburger hamburger" aria-label="mobile navigation" aria-expanded="false" aria-controls="VPNavScreen" data-v-6fc2afcb data-v-2caaf84e><span class="container" data-v-2caaf84e><span class="top" data-v-2caaf84e></span><span class="middle" data-v-2caaf84e></span><span class="bottom" data-v-2caaf84e></span></span></button></div></div></div></div><div class="divider" data-v-6fc2afcb><div class="divider-line" data-v-6fc2afcb></div></div></div><!----></header><div class="VPLocalNav has-sidebar empty" data-v-738a160a data-v-e0cbfd80><div class="container" data-v-e0cbfd80><button class="menu" aria-expanded="false" aria-controls="VPSidebarNav" data-v-e0cbfd80><span class="vpi-align-left menu-icon" data-v-e0cbfd80></span><span class="menu-text" data-v-e0cbfd80>Menu</span></button><div class="VPLocalNavOutlineDropdown" style="--vp-vh:0px;" data-v-e0cbfd80 data-v-36e0b8e9><button data-v-36e0b8e9>Return to top</button><!----></div></div></div><aside class="VPSidebar" data-v-738a160a data-v-2859715e><div class="curtain" data-v-2859715e></div><nav class="nav" id="VPSidebarNav" aria-labelledby="sidebar-aria-label" tabindex="-1" data-v-2859715e><span class="visually-hidden" id="sidebar-aria-label" data-v-2859715e> Sidebar Navigation </span><!--[--><!--]--><!--[--><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Getting started</h2><!----></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Overview</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/changelog.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Changelog</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/quickstart.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Building your first agent</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Integrations</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/github.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>GitHub</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/slack.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Slack</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/webhooks.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Webhooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/convert-automation.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Cursor Automations</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/opentelemetry.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>OpenTelemetry</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/gitlab.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>GitLab</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/bitbucket.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Bitbucket</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/pagerduty.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>PagerDuty</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Extensions</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/extensions.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Overview</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/cloud-agents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Cloud agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/grokbot-agents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Grok Bot agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/improve.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Self-improvement</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/jev.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Jev</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible has-active" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Guides</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/mcp-oauth.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>MCP</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/hooks.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Hooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/agent-to-agent.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Peer agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/deployment.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Deployment</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/export-grokbot.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Export a Grok Bot</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/migrate.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Early-alpha migration</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/evals.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Evals</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/hillclimbing.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Hillclimbing</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/building-with-agents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Build with a coding agent</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/troubleshooting.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Troubleshooting</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Templates</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/grokbot-agents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Grok Bot agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/security-reviewer.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Security reviewer</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/thermo-review.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Thermo review</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/thermo-quality-review.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Thermo quality review</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/agentic-owners.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Agentic Approver</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/triage.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Triage agent</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Reference</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/project-layout.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Project layout</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/agent-config.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Agent config</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/instructions.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Instructions</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/tools.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Tools</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/prompt.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>prompt</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/skills.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Skills</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/connections.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>MCP Connections</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/subagents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Subagents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/channels.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Channels</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/schedules.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Schedules & reminders</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/hooks.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Hooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/memory.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Memory</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/evals.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Evals</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/artifacts.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Artifacts</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/sessions.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Sessions & streaming</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/playground.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Playground</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/cli.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>CLI</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/http-api.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>HTTP API</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><!--]--><!--[--><!--]--></nav></aside><div class="VPContent has-sidebar" id="VPContent" data-v-738a160a data-v-9f462cdf><div class="VPDoc has-sidebar has-aside" data-v-9f462cdf data-v-995f90b4><!--[--><!--]--><div class="container" data-v-995f90b4><div class="aside" data-v-995f90b4><div class="aside-curtain" data-v-995f90b4></div><div class="aside-container" data-v-995f90b4><div class="aside-content" data-v-995f90b4><div class="VPDocAside" data-v-995f90b4 data-v-10889960><!--[--><!--]--><!--[--><!--]--><nav aria-labelledby="doc-outline-aria-label" class="VPDocAsideOutline" data-v-10889960 data-v-afb61a88><div class="content" data-v-afb61a88><div class="outline-marker" data-v-afb61a88></div><div aria-level="2" class="outline-title" id="doc-outline-aria-label" role="heading" data-v-afb61a88>On this page</div><ul class="VPDocOutlineItem root" data-v-afb61a88 data-v-0587367d><!--[--><!--]--></ul></div></nav><!--[--><!--]--><div class="spacer" data-v-10889960></div><!--[--><!--]--><!----><!--[--><!--]--><!--[--><!--]--></div></div></div></div><div class="content" data-v-995f90b4><div class="content-container" data-v-995f90b4><!--[--><!--]--><main class="main" data-v-995f90b4><div style="position:relative;" class="vp-doc _docs_evals" data-v-995f90b4><div><h1 id="evals" tabindex="-1">Evals <a class="header-anchor" href="#evals" aria-label="Permalink to "Evals""></a></h1><p>An eval sends a fixed input to the real agent and checks the resulting trajectory: whether the turn succeeded, which tools ran, and what shape the answer took. Use evals to protect behavior you already understand, not to discover what the prompt should do.</p><p>Evals run real model turns and real tools. Guard external writes before running a suite against an agent that can post, merge, or deploy.</p><h2 id="protect-a-tool-decision" tabindex="-1">Protect a tool decision <a class="header-anchor" href="#protect-a-tool-decision" aria-label="Permalink to "Protect a tool decision""></a></h2><p>This smoke case asks for a PR verdict, requires the read-only inspection tool, and fails if the agent tries to approve. The CLI reports all three decisions together instead of stopping at the first miss.</p><p>Use the <a href="./guides/jev.html">Jev extension</a> when host code needs a typed choice, score, or boolean before it writes.</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">// evals/readiness.eval.ts</span></span>
|
|
57
|
-
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { defineEval, includes } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "@
|
|
56
|
+
<div id="app"><div class="Layout" data-v-738a160a><!--[--><!--]--><!--[--><span tabindex="-1" data-v-a98c2453></span><a href="#VPContent" class="VPSkipLink visually-hidden" data-v-a98c2453>Skip to content</a><!--]--><!----><header class="VPNav" data-v-738a160a data-v-76ad4059><div class="VPNavBar" data-v-76ad4059 data-v-6fc2afcb><div class="wrapper" data-v-6fc2afcb><div class="container" data-v-6fc2afcb><div class="title" data-v-6fc2afcb><div class="VPNavBarTitle has-sidebar" data-v-6fc2afcb data-v-9e6caa3f><a class="title" href="/docs/" data-v-9e6caa3f><!--[--><!--]--><!----><span data-v-9e6caa3f>Agent SDK</span><!--[--><!--[--><!--[--><!--[--><span class="bdk-version" title="Agent SDK 0.2.29 (@grok/agent-sdk package)" data-v-a03aeddf>0.2.29</span><!--]--><!--]--><!--]--><!--]--></a></div></div><div class="content" data-v-6fc2afcb><div class="content-body" data-v-6fc2afcb><!--[--><!--]--><div class="VPNavBarSearch search" data-v-6fc2afcb><!--[--><!----><div id="local-search"><button type="button" class="DocSearch DocSearch-Button" aria-label="Search"><span class="DocSearch-Button-Container"><span class="vp-icon DocSearch-Search-Icon"></span><span class="DocSearch-Button-Placeholder">Search</span></span><span class="DocSearch-Button-Keys"><kbd class="DocSearch-Button-Key"></kbd><kbd class="DocSearch-Button-Key">K</kbd></span></button></div><!--]--></div><nav aria-labelledby="main-nav-aria-label" class="VPNavBarMenu menu" data-v-6fc2afcb data-v-78f72f99><span id="main-nav-aria-label" class="visually-hidden" data-v-78f72f99> Main Navigation </span><!--[--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/quickstart.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Quickstart</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/guides/mcp-oauth.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Guides</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/guides/github.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Integrations</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/templates/security-reviewer.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Templates</span><!--]--></a><!--]--><!--[--><a class="VPLink link VPNavBarMenuLink" href="/docs/reference/project-layout.html" tabindex="0" data-v-78f72f99 data-v-a7790978><!--[--><span data-v-a7790978>Reference</span><!--]--></a><!--]--><!--]--></nav><!----><div class="VPNavBarAppearance appearance" data-v-6fc2afcb data-v-bf7c3d0f><button class="VPSwitch VPSwitchAppearance" type="button" role="switch" title aria-checked="false" data-v-bf7c3d0f data-v-e9e166e4 data-v-74f7b0ff><span class="check" data-v-74f7b0ff><span class="icon" data-v-74f7b0ff><!--[--><span class="vpi-sun sun" data-v-e9e166e4></span><span class="vpi-moon moon" data-v-e9e166e4></span><!--]--></span></span></button></div><!----><div class="VPFlyout VPNavBarExtra extra" data-v-6fc2afcb data-v-799c9281 data-v-070c9767><button type="button" class="button" aria-haspopup="true" aria-expanded="false" aria-label="extra navigation" data-v-070c9767><span class="vpi-more-horizontal icon" data-v-070c9767></span></button><div class="menu" data-v-070c9767><div class="VPMenu" data-v-070c9767 data-v-c05d27ee><!----><!--[--><!--[--><!----><div class="group" data-v-799c9281><div class="item appearance" data-v-799c9281><p class="label" data-v-799c9281>Appearance</p><div class="appearance-action" data-v-799c9281><button class="VPSwitch VPSwitchAppearance" type="button" role="switch" title aria-checked="false" data-v-799c9281 data-v-e9e166e4 data-v-74f7b0ff><span class="check" data-v-74f7b0ff><span class="icon" data-v-74f7b0ff><!--[--><span class="vpi-sun sun" data-v-e9e166e4></span><span class="vpi-moon moon" data-v-e9e166e4></span><!--]--></span></span></button></div></div></div><!----><!--]--><!--]--></div></div></div><!--[--><!--]--><button type="button" class="VPNavBarHamburger hamburger" aria-label="mobile navigation" aria-expanded="false" aria-controls="VPNavScreen" data-v-6fc2afcb data-v-2caaf84e><span class="container" data-v-2caaf84e><span class="top" data-v-2caaf84e></span><span class="middle" data-v-2caaf84e></span><span class="bottom" data-v-2caaf84e></span></span></button></div></div></div></div><div class="divider" data-v-6fc2afcb><div class="divider-line" data-v-6fc2afcb></div></div></div><!----></header><div class="VPLocalNav has-sidebar empty" data-v-738a160a data-v-e0cbfd80><div class="container" data-v-e0cbfd80><button class="menu" aria-expanded="false" aria-controls="VPSidebarNav" data-v-e0cbfd80><span class="vpi-align-left menu-icon" data-v-e0cbfd80></span><span class="menu-text" data-v-e0cbfd80>Menu</span></button><div class="VPLocalNavOutlineDropdown" style="--vp-vh:0px;" data-v-e0cbfd80 data-v-36e0b8e9><button data-v-36e0b8e9>Return to top</button><!----></div></div></div><aside class="VPSidebar" data-v-738a160a data-v-2859715e><div class="curtain" data-v-2859715e></div><nav class="nav" id="VPSidebarNav" aria-labelledby="sidebar-aria-label" tabindex="-1" data-v-2859715e><span class="visually-hidden" id="sidebar-aria-label" data-v-2859715e> Sidebar Navigation </span><!--[--><!--]--><!--[--><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Getting started</h2><!----></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Overview</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/changelog.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Changelog</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/quickstart.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Building your first agent</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Integrations</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/github.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>GitHub</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/slack.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Slack</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/webhooks.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Webhooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/convert-automation.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Cursor Automations</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/opentelemetry.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>OpenTelemetry</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/gitlab.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>GitLab</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/bitbucket.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Bitbucket</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/pagerduty.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>PagerDuty</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Extensions</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/extensions.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Overview</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/cloud-agents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Cloud agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/improve.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Self-improvement</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/jev.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Jev</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible has-active" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Guides</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/mcp-oauth.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>MCP</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/hooks.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Hooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/agent-to-agent.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Peer agents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/deployment.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Deployment</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/guides/migrate.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Early-alpha migration</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/evals.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Evals</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/hillclimbing.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Hillclimbing</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/building-with-agents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Build with a coding agent</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/troubleshooting.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Troubleshooting</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Templates</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/security-reviewer.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Security reviewer</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/thermo-review.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Thermo review</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/thermo-quality-review.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Thermo quality review</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/agentic-owners.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Agentic Approver</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/templates/triage.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Triage agent</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><div class="no-transition group" data-v-8ed949fd><section class="VPSidebarItem level-0 collapsible" data-v-8ed949fd data-v-84325fb7><div class="item" role="button" tabindex="0" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><h2 class="text" data-v-84325fb7>Reference</h2><div class="caret" role="button" aria-label="toggle section" tabindex="0" data-v-84325fb7><span class="vpi-chevron-right caret-icon" data-v-84325fb7></span></div></div><div class="items" data-v-84325fb7><!--[--><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/project-layout.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Project layout</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/agent-config.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Agent config</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/instructions.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Instructions</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/tools.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Tools</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/prompt.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>prompt</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/skills.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Skills</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/connections.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>MCP Connections</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/subagents.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Subagents</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/channels.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Channels</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/schedules.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Schedules & reminders</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/hooks.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Hooks</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/memory.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Memory</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/evals.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Evals</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/artifacts.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Artifacts</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/sessions.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Sessions & streaming</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/playground.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Playground</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/cli.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>CLI</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/http-api.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>HTTP API</p><!--]--></a><!----></div><!----></div><div class="VPSidebarItem level-1 is-link" data-v-84325fb7><div class="item" data-v-84325fb7><div class="indicator" data-v-84325fb7></div><a class="VPLink link link" href="/docs/reference/access.html" data-v-84325fb7><!--[--><p class="text" data-v-84325fb7>Access</p><!--]--></a><!----></div><!----></div><!--]--></div></section></div><!--]--><!--[--><!--]--></nav></aside><div class="VPContent has-sidebar" id="VPContent" data-v-738a160a data-v-9f462cdf><div class="VPDoc has-sidebar has-aside" data-v-9f462cdf data-v-995f90b4><!--[--><!--]--><div class="container" data-v-995f90b4><div class="aside" data-v-995f90b4><div class="aside-curtain" data-v-995f90b4></div><div class="aside-container" data-v-995f90b4><div class="aside-content" data-v-995f90b4><div class="VPDocAside" data-v-995f90b4 data-v-10889960><!--[--><!--]--><!--[--><!--]--><nav aria-labelledby="doc-outline-aria-label" class="VPDocAsideOutline" data-v-10889960 data-v-afb61a88><div class="content" data-v-afb61a88><div class="outline-marker" data-v-afb61a88></div><div aria-level="2" class="outline-title" id="doc-outline-aria-label" role="heading" data-v-afb61a88>On this page</div><ul class="VPDocOutlineItem root" data-v-afb61a88 data-v-0587367d><!--[--><!--]--></ul></div></nav><!--[--><!--]--><div class="spacer" data-v-10889960></div><!--[--><!--]--><!----><!--[--><!--]--><!--[--><!--]--></div></div></div></div><div class="content" data-v-995f90b4><div class="content-container" data-v-995f90b4><!--[--><!--]--><main class="main" data-v-995f90b4><div style="position:relative;" class="vp-doc _docs_evals" data-v-995f90b4><div><h1 id="evals" tabindex="-1">Evals <a class="header-anchor" href="#evals" aria-label="Permalink to "Evals""></a></h1><p>An eval sends a fixed input to the real agent and checks the resulting trajectory: whether the turn succeeded, which tools ran, and what shape the answer took. Use evals to protect behavior you already understand, not to discover what the prompt should do.</p><p>Evals run real model turns and real tools. Guard external writes before running a suite against an agent that can post, merge, or deploy.</p><h2 id="protect-a-tool-decision" tabindex="-1">Protect a tool decision <a class="header-anchor" href="#protect-a-tool-decision" aria-label="Permalink to "Protect a tool decision""></a></h2><p>This smoke case asks for a PR verdict, requires the read-only inspection tool, and fails if the agent tries to approve. The CLI reports all three decisions together instead of stopping at the first miss.</p><p>Use the <a href="./guides/jev.html">Jev extension</a> when host code needs a typed choice, score, or boolean before it writes.</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">// evals/readiness.eval.ts</span></span>
|
|
57
|
+
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { defineEval, includes } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "@grok/agent-sdk/evals"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
|
|
58
58
|
<span class="line"></span>
|
|
59
59
|
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">export</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> default</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> defineEval</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">({</span></span>
|
|
60
60
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> tags: [</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">"smoke"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">],</span></span>
|
|
@@ -68,13 +68,13 @@
|
|
|
68
68
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">notCalledTool</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">"approve_pr"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
|
|
69
69
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> t.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">check</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(t.reply, </span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">includes</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">/</span><span style="--shiki-light:#032F62;--shiki-dark:#DBEDFF;">ready</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">|</span><span style="--shiki-light:#032F62;--shiki-dark:#DBEDFF;">blocked</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">|</span><span style="--shiki-light:#032F62;--shiki-dark:#DBEDFF;">approve</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">/</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">i</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">));</span></span>
|
|
70
70
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> },</span></span>
|
|
71
|
-
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><p>Evals live in the project-root <code>evals/</code> directory, beside <code>bot/</code>.
|
|
72
|
-
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { defineEvalConfig } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "@
|
|
71
|
+
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><p>Evals live in the project-root <code>evals/</code> directory, beside <code>bot/</code>. Blank <code>agent-sdk init</code> writes <code>evals/smoke.eval.ts</code>, which uses the scaffolded <code>echo</code> tool and asserts <code>t.succeeded()</code> and <code>t.calledTool("echo")</code>. It also writes this concurrency config:</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#6A737D;--shiki-dark:#6A737D;">// evals/evals.config.ts</span></span>
|
|
72
|
+
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { defineEvalConfig } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "@grok/agent-sdk/evals"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
|
|
73
73
|
<span class="line"></span>
|
|
74
74
|
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">export</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> default</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> defineEvalConfig</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">({</span></span>
|
|
75
75
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> maxConcurrency: </span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">20</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
|
|
76
|
-
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>
|
|
77
|
-
<span class="line"><span>
|
|
76
|
+
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>agent-sdk eval --dir . --list</span></span>
|
|
77
|
+
<span class="line"><span>agent-sdk eval --dir . --tag smoke</span></span></code></pre></div><h2 id="check-answer-shape-or-quality" tabindex="-1">Check answer shape or quality <a class="header-anchor" href="#check-answer-shape-or-quality" aria-label="Permalink to "Check answer shape or quality""></a></h2><p>Prefer a deterministic shape check when the output has a contract. The failure tells you which field was wrong, and the case remains stable when the model changes its wording.</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { matches } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "@grok/agent-sdk/evals"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
|
|
78
78
|
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">import</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> { z } </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">from</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "zod"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">;</span></span>
|
|
79
79
|
<span class="line"></span>
|
|
80
80
|
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">const</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> verdict</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> =</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;"> JSON</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">.</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">parse</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(t.reply </span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;">??</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "{}"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">);</span></span>
|
|
@@ -101,13 +101,13 @@
|
|
|
101
101
|
<span class="line"><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;"> "+res.send(`<h1>Results for ${req.query.q}</h1>`);"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">,</span></span>
|
|
102
102
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> ].</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">join</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">"</span><span style="--shiki-light:#005CC5;--shiki-dark:#79B8FF;">\n</span><span style="--shiki-light:#032F62;--shiki-dark:#9ECBFF;">"</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">),</span></span>
|
|
103
103
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;"> },</span></span>
|
|
104
|
-
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><p>For GitHub agents, snapshot a PR into committed fixtures:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>
|
|
104
|
+
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">});</span></span></code></pre></div><p>For GitHub agents, snapshot a PR into committed fixtures:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>agent-sdk github replay acme/checkout#42 \</span></span>
|
|
105
105
|
<span class="line"><span> --events '*' \</span></span>
|
|
106
106
|
<span class="line"><span> --dry-run \</span></span>
|
|
107
|
-
<span class="line"><span> --out fixtures/github</span></span></code></pre></div><p>Keep a small, fast <code>smoke</code> set for every change. Put larger datasets and drift-prone live cases behind an explicit selection. See <a href="./reference/evals.html#datasets-and-fixtures">Datasets and fixtures</a> for loaders and case expansion.</p><h2 id="run-and-debug-a-regression-suite" tabindex="-1">Run and debug a regression suite <a class="header-anchor" href="#run-and-debug-a-regression-suite" aria-label="Permalink to "Run and debug a regression suite""></a></h2><p>Run one case while iterating, then the smoke suite before review. A failure artifact records the input, assertions, tool calls, and final text, making it the first place to look before changing the prompt.</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>
|
|
108
|
-
<span class="line"><span>
|
|
109
|
-
<span class="line"><span>
|
|
110
|
-
<span class="line"><span>
|
|
107
|
+
<span class="line"><span> --out fixtures/github</span></span></code></pre></div><p>Keep a small, fast <code>smoke</code> set for every change. Put larger datasets and drift-prone live cases behind an explicit selection. See <a href="./reference/evals.html#datasets-and-fixtures">Datasets and fixtures</a> for loaders and case expansion.</p><h2 id="run-and-debug-a-regression-suite" tabindex="-1">Run and debug a regression suite <a class="header-anchor" href="#run-and-debug-a-regression-suite" aria-label="Permalink to "Run and debug a regression suite""></a></h2><p>Run one case while iterating, then the smoke suite before review. A failure artifact records the input, assertions, tool calls, and final text, making it the first place to look before changing the prompt.</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span>agent-sdk eval --dir . readiness</span></span>
|
|
108
|
+
<span class="line"><span>agent-sdk eval --dir . --tag smoke</span></span>
|
|
109
|
+
<span class="line"><span>agent-sdk eval --dir . --json</span></span></code></pre></div><p>By default, results land under <code>evals/<stamp>/</code> in the project state directory. Open <code>evals/<case-id>.json</code> inside that run directory for the failed case. This artifact location is independent of <code>--state-root</code>; pass <code>--artifacts <dir></code> to choose another destination.</p><p>Model turns need a credential. Resolution checks <code>CURSOR_API_KEY</code>, <code>CURSOR_API_KEY_FILE</code>, <code>CURSOR_SERVICE_ACCOUNT_KEY</code>, then a saved <code>agent-sdk login</code>. <code>eval --list</code> only discovers cases and needs none.</p><h2 id="run-evals-in-ci" tabindex="-1">Run evals in CI <a class="header-anchor" href="#run-evals-in-ci" aria-label="Permalink to "Run evals in CI""></a></h2><p>Write machine-readable output and JUnit annotations, then let a failed gate fail the job:</p><div class="language-text vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">text</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span># CURSOR_API_KEY comes from the CI secret store.</span></span>
|
|
110
|
+
<span class="line"><span>agent-sdk eval --dir . --json --no-stream \</span></span>
|
|
111
111
|
<span class="line"><span> --junit reports/evals.xml \</span></span>
|
|
112
112
|
<span class="line"><span> --artifacts reports/evals \</span></span>
|
|
113
113
|
<span class="line"><span> > reports/evals.json</span></span></code></pre></div><p>Run <code>--tag smoke</code> on pull requests and the full suite on a schedule when cost or latency makes every-push coverage impractical. Use soft scores for a new probabilistic benchmark until its threshold is trustworthy.</p><p>The <a href="./reference/cli.html#eval">CLI reference</a> covers selectors, server-side batches, exit codes, and JSON output.</p><h2 id="keep-side-effects-out-of-eval-sessions" tabindex="-1">Keep side effects out of eval sessions <a class="header-anchor" href="#keep-side-effects-out-of-eval-sessions" aria-label="Permalink to "Keep side effects out of eval sessions""></a></h2><p>Eval sessions carry <code>purpose: "eval"</code>. Check it at the deterministic write boundary and return the decision without performing the external action:</p><div class="language-ts vp-adaptive-theme"><button title="Copy Code" class="copy"></button><span class="lang">ts</span><pre class="shiki shiki-themes github-light github-dark vp-code" tabindex="0"><code><span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">async </span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;">execute</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">({ findings }, ctx) {</span></span>
|
|
@@ -121,7 +121,7 @@
|
|
|
121
121
|
<span class="line"></span>
|
|
122
122
|
<span class="line"><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> return</span><span style="--shiki-light:#D73A49;--shiki-dark:#F97583;"> await</span><span style="--shiki-light:#6F42C1;--shiki-dark:#B392F0;"> postFindings</span><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">(findings);</span></span>
|
|
123
123
|
<span class="line"><span style="--shiki-light:#24292E;--shiki-dark:#E1E4E8;">}</span></span></code></pre></div><p>This preserves the tool call in the trajectory, so the eval can still assert that the agent chose to post.</p><h2 id="evals-hooks-or-hillclimbing" tabindex="-1">Evals, hooks, or hillclimbing? <a class="header-anchor" href="#evals-hooks-or-hillclimbing" aria-label="Permalink to "Evals, hooks, or hillclimbing?""></a></h2><p>Use an eval to gate one fixed input. Use a <a href="./reference/hooks.html">hook</a> to observe every live session for metrics, auditing, or alerts. Use <a href="./hillclimbing.html">hillclimbing</a> to decide which source change improves a fixed set of inputs; every kept hillclimb change should add an eval.</p><h2 id="gates-and-scores" tabindex="-1">Gates and scores <a class="header-anchor" href="#gates-and-scores" aria-label="Permalink to "Gates and scores""></a></h2><p><code>t.succeeded()</code>, tool assertions, and deterministic checks are hard gates by default. A failed gate fails the case. <code>.soft()</code> records a measurement without blocking, while <code>.atLeast(threshold)</code> records a score and marks a miss as <code>scored</code>.</p><p>Start with decisions and output shape: the intended tool, the tempting wrong tool, and one structural check. Exact prose is rarely a stable contract.</p><h2 id="keep-improvements-with-regression-evals" tabindex="-1">Keep improvements with regression evals <a class="header-anchor" href="#keep-improvements-with-regression-evals" aria-label="Permalink to "Keep improvements with regression evals""></a></h2><p>Every kept improvement needs an eval that would have failed before the change. A tool-choice gate, bounded <code>maxToolCalls</code>, or output-shape check turns the improvement into a durable contract.</p><p>Never weaken an existing gate to make a new implementation pass. That moves the freeze line instead of proving the change.</p><h2 id="related" tabindex="-1">Related <a class="header-anchor" href="#related" aria-label="Permalink to "Related""></a></h2><ul><li><a href="./reference/evals.html">Evals reference</a>: complete authoring and runner contracts</li><li><a href="./../skills/evals/SKILL.html"><code>skills/evals/SKILL.md</code></a>: author and seed cases with a coding agent</li><li><a href="./hillclimbing.html">Hillclimbing</a>: measure a change and lock the win</li><li><a href="./reference/sessions.html">Sessions</a>: events available to trajectory assertions</li></ul></div></div></main><footer class="VPDocFooter" data-v-995f90b4 data-v-8edf72db><!--[--><!--]--><!----><nav class="prev-next" aria-labelledby="doc-footer-aria-label" data-v-8edf72db><span class="visually-hidden" id="doc-footer-aria-label" data-v-8edf72db>Pager</span><div class="pager" data-v-8edf72db><a class="VPLink link pager-link prev" href="/docs/guides/migrate.html" data-v-8edf72db><!--[--><span class="desc" data-v-8edf72db>Previous page</span><span class="title" data-v-8edf72db>Early-alpha migration</span><!--]--></a></div><div class="pager" data-v-8edf72db><a class="VPLink link pager-link next" href="/docs/hillclimbing.html" data-v-8edf72db><!--[--><span class="desc" data-v-8edf72db>Next page</span><span class="title" data-v-8edf72db>Hillclimbing</span><!--]--></a></div></nav></footer><!--[--><!--]--></div></div></div><!--[--><!--]--></div></div><!----><!--[--><!--]--></div></div>
|
|
124
|
-
<script>window.__VP_HASH_MAP__=JSON.parse("{\"building-with-agents.md\":\"
|
|
124
|
+
<script>window.__VP_HASH_MAP__=JSON.parse("{\"building-with-agents.md\":\"Bf1yH2tG\",\"changelog.md\":\"DrLFQUlu\",\"deployment.md\":\"KW0Ni_rY\",\"evals.md\":\"DiljU__Z\",\"guides_agent-to-agent.md\":\"D1D5f6Jc\",\"guides_bitbucket.md\":\"DkjkBRU8\",\"guides_cloud-agents.md\":\"Ceyjv0ul\",\"guides_convert-automation.md\":\"BjraCvok\",\"guides_github.md\":\"CgN5sNbW\",\"guides_gitlab.md\":\"DCE0p7_w\",\"guides_hooks.md\":\"BjeEiM7s\",\"guides_improve.md\":\"BFYtrtLE\",\"guides_jev.md\":\"CEqLgjj9\",\"guides_mcp-oauth.md\":\"DrGY92u4\",\"guides_migrate.md\":\"RS9Eu-Xh\",\"guides_opentelemetry.md\":\"CoZ1kNyd\",\"guides_pagerduty.md\":\"mnKXe9U0\",\"guides_slack.md\":\"BwXHURvY\",\"guides_webhooks.md\":\"BJk63ZTs\",\"hillclimbing.md\":\"By61f3uq\",\"index.md\":\"CAruw_rp\",\"quickstart.md\":\"CljDaJBi\",\"reference_access.md\":\"CEtibb4i\",\"reference_agent-config.md\":\"Ck29qYNj\",\"reference_artifacts.md\":\"C-hKR2hp\",\"reference_channels.md\":\"DsJfGSBJ\",\"reference_cli.md\":\"BkdLRbXg\",\"reference_connections.md\":\"BoAotr7_\",\"reference_evals.md\":\"2BXeMiO2\",\"reference_extensions.md\":\"DI5X5x8p\",\"reference_hooks.md\":\"B-bwJuiG\",\"reference_http-api.md\":\"_RWELr3d\",\"reference_instructions.md\":\"yL2hXLnl\",\"reference_limits.md\":\"BCCnvt3o\",\"reference_memory.md\":\"Dd7cyV9V\",\"reference_playground.md\":\"CQiJk9CM\",\"reference_project-layout.md\":\"BamcPDUn\",\"reference_prompt.md\":\"BMjm1Jp3\",\"reference_schedules.md\":\"RC47aFm4\",\"reference_sessions.md\":\"C7KFhWXV\",\"reference_skills.md\":\"Dej2zQxi\",\"reference_subagents.md\":\"CAhvW4yS\",\"reference_tools.md\":\"DQr7Fnhu\",\"templates_agentic-owners.md\":\"Dh2nDSqg\",\"templates_security-reviewer.md\":\"DnwXdnB5\",\"templates_thermo-quality-review.md\":\"MJIdHkzE\",\"templates_thermo-review.md\":\"DfN1OFOl\",\"templates_triage.md\":\"BYWMloV4\",\"troubleshooting.md\":\"CGOdNVAL\"}");window.__VP_SITE_DATA__=JSON.parse("{\"lang\":\"en-US\",\"dir\":\"ltr\",\"title\":\"Agent SDK\",\"description\":\"Build and serve bots and agents from versioned Markdown and TypeScript.\",\"base\":\"/docs/\",\"head\":[],\"router\":{\"prefetchLinks\":true},\"appearance\":true,\"themeConfig\":{\"sdkVersion\":\"0.2.29\",\"nav\":[{\"text\":\"Quickstart\",\"link\":\"/quickstart\"},{\"text\":\"Guides\",\"link\":\"/guides/mcp-oauth\"},{\"text\":\"Integrations\",\"link\":\"/guides/github\"},{\"text\":\"Templates\",\"link\":\"/templates/security-reviewer\"},{\"text\":\"Reference\",\"link\":\"/reference/project-layout\"}],\"search\":{\"provider\":\"local\"},\"outline\":{\"level\":[2,3]},\"sidebar\":[{\"text\":\"Getting started\",\"items\":[{\"text\":\"Overview\",\"link\":\"/\"},{\"text\":\"Changelog\",\"link\":\"/changelog\"},{\"text\":\"Building your first agent\",\"link\":\"/quickstart\"}]},{\"text\":\"Integrations\",\"collapsed\":false,\"items\":[{\"text\":\"GitHub\",\"link\":\"/guides/github\"},{\"text\":\"Slack\",\"link\":\"/guides/slack\"},{\"text\":\"Webhooks\",\"link\":\"/guides/webhooks\"},{\"text\":\"Cursor Automations\",\"link\":\"/guides/convert-automation\"},{\"text\":\"OpenTelemetry\",\"link\":\"/guides/opentelemetry\"},{\"text\":\"GitLab\",\"link\":\"/guides/gitlab\"},{\"text\":\"Bitbucket\",\"link\":\"/guides/bitbucket\"},{\"text\":\"PagerDuty\",\"link\":\"/guides/pagerduty\"}]},{\"text\":\"Extensions\",\"collapsed\":false,\"items\":[{\"text\":\"Overview\",\"link\":\"/reference/extensions\"},{\"text\":\"Cloud agents\",\"link\":\"/guides/cloud-agents\"},{\"text\":\"Self-improvement\",\"link\":\"/guides/improve\"},{\"text\":\"Jev\",\"link\":\"/guides/jev\"}]},{\"text\":\"Guides\",\"collapsed\":false,\"items\":[{\"text\":\"MCP\",\"link\":\"/guides/mcp-oauth\"},{\"text\":\"Hooks\",\"link\":\"/guides/hooks\"},{\"text\":\"Peer agents\",\"link\":\"/guides/agent-to-agent\"},{\"text\":\"Deployment\",\"link\":\"/deployment\"},{\"text\":\"Early-alpha migration\",\"link\":\"/guides/migrate\"},{\"text\":\"Evals\",\"link\":\"/evals\"},{\"text\":\"Hillclimbing\",\"link\":\"/hillclimbing\"},{\"text\":\"Build with a coding agent\",\"link\":\"/building-with-agents\"},{\"text\":\"Troubleshooting\",\"link\":\"/troubleshooting\"}]},{\"text\":\"Templates\",\"collapsed\":false,\"items\":[{\"text\":\"Security reviewer\",\"link\":\"/templates/security-reviewer\"},{\"text\":\"Thermo review\",\"link\":\"/templates/thermo-review\"},{\"text\":\"Thermo quality review\",\"link\":\"/templates/thermo-quality-review\"},{\"text\":\"Agentic Approver\",\"link\":\"/templates/agentic-owners\"},{\"text\":\"Triage agent\",\"link\":\"/templates/triage\"}]},{\"text\":\"Reference\",\"collapsed\":false,\"items\":[{\"text\":\"Project layout\",\"link\":\"/reference/project-layout\"},{\"text\":\"Agent config\",\"link\":\"/reference/agent-config\"},{\"text\":\"Instructions\",\"link\":\"/reference/instructions\"},{\"text\":\"Tools\",\"link\":\"/reference/tools\"},{\"text\":\"prompt\",\"link\":\"/reference/prompt\"},{\"text\":\"Skills\",\"link\":\"/reference/skills\"},{\"text\":\"MCP Connections\",\"link\":\"/reference/connections\"},{\"text\":\"Subagents\",\"link\":\"/reference/subagents\"},{\"text\":\"Channels\",\"link\":\"/reference/channels\"},{\"text\":\"Schedules & reminders\",\"link\":\"/reference/schedules\"},{\"text\":\"Hooks\",\"link\":\"/reference/hooks\"},{\"text\":\"Memory\",\"link\":\"/reference/memory\"},{\"text\":\"Evals\",\"link\":\"/reference/evals\"},{\"text\":\"Artifacts\",\"link\":\"/reference/artifacts\"},{\"text\":\"Sessions & streaming\",\"link\":\"/reference/sessions\"},{\"text\":\"Playground\",\"link\":\"/reference/playground\"},{\"text\":\"CLI\",\"link\":\"/reference/cli\"},{\"text\":\"HTTP API\",\"link\":\"/reference/http-api\"},{\"text\":\"Access\",\"link\":\"/reference/access\"}]}]},\"locales\":{},\"scrollOffset\":134,\"cleanUrls\":false}");</script>
|
|
125
125
|
|
|
126
126
|
</body>
|
|
127
127
|
</html>
|
package/dist/docs/evals.md
CHANGED
|
@@ -19,7 +19,7 @@ choice, score, or boolean before it writes.
|
|
|
19
19
|
|
|
20
20
|
```ts
|
|
21
21
|
// evals/readiness.eval.ts
|
|
22
|
-
import { defineEval, includes } from "@
|
|
22
|
+
import { defineEval, includes } from "@grok/agent-sdk/evals";
|
|
23
23
|
|
|
24
24
|
export default defineEval({
|
|
25
25
|
tags: ["smoke"],
|
|
@@ -37,11 +37,13 @@ export default defineEval({
|
|
|
37
37
|
```
|
|
38
38
|
|
|
39
39
|
Evals live in the project-root `evals/` directory, beside `bot/`.
|
|
40
|
-
|
|
40
|
+
Blank `agent-sdk init` writes `evals/smoke.eval.ts`, which uses the scaffolded
|
|
41
|
+
`echo` tool and asserts `t.succeeded()` and `t.calledTool("echo")`.
|
|
42
|
+
It also writes this concurrency config:
|
|
41
43
|
|
|
42
44
|
```ts
|
|
43
45
|
// evals/evals.config.ts
|
|
44
|
-
import { defineEvalConfig } from "@
|
|
46
|
+
import { defineEvalConfig } from "@grok/agent-sdk/evals";
|
|
45
47
|
|
|
46
48
|
export default defineEvalConfig({
|
|
47
49
|
maxConcurrency: 20,
|
|
@@ -49,8 +51,8 @@ export default defineEvalConfig({
|
|
|
49
51
|
```
|
|
50
52
|
|
|
51
53
|
```text
|
|
52
|
-
|
|
53
|
-
|
|
54
|
+
agent-sdk eval --dir . --list
|
|
55
|
+
agent-sdk eval --dir . --tag smoke
|
|
54
56
|
```
|
|
55
57
|
|
|
56
58
|
## Check answer shape or quality
|
|
@@ -60,7 +62,7 @@ failure tells you which field was wrong, and the case remains stable
|
|
|
60
62
|
when the model changes its wording.
|
|
61
63
|
|
|
62
64
|
```ts
|
|
63
|
-
import { matches } from "@
|
|
65
|
+
import { matches } from "@grok/agent-sdk/evals";
|
|
64
66
|
import { z } from "zod";
|
|
65
67
|
|
|
66
68
|
const verdict = JSON.parse(t.reply ?? "{}");
|
|
@@ -125,7 +127,7 @@ await t.send("Review pr/diff.patch and report blockers.", {
|
|
|
125
127
|
For GitHub agents, snapshot a PR into committed fixtures:
|
|
126
128
|
|
|
127
129
|
```text
|
|
128
|
-
|
|
130
|
+
agent-sdk github replay acme/checkout#42 \
|
|
129
131
|
--events '*' \
|
|
130
132
|
--dry-run \
|
|
131
133
|
--out fixtures/github
|
|
@@ -143,9 +145,9 @@ failure artifact records the input, assertions, tool calls, and final
|
|
|
143
145
|
text, making it the first place to look before changing the prompt.
|
|
144
146
|
|
|
145
147
|
```text
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
148
|
+
agent-sdk eval --dir . readiness
|
|
149
|
+
agent-sdk eval --dir . --tag smoke
|
|
150
|
+
agent-sdk eval --dir . --json
|
|
149
151
|
```
|
|
150
152
|
|
|
151
153
|
By default, results land under `evals/<stamp>/` in the project state
|
|
@@ -155,7 +157,7 @@ pass `--artifacts <dir>` to choose another destination.
|
|
|
155
157
|
|
|
156
158
|
Model turns need a credential. Resolution checks `CURSOR_API_KEY`,
|
|
157
159
|
`CURSOR_API_KEY_FILE`, `CURSOR_SERVICE_ACCOUNT_KEY`, then a saved
|
|
158
|
-
`
|
|
160
|
+
`agent-sdk login`. `eval --list` only discovers cases and needs none.
|
|
159
161
|
|
|
160
162
|
## Run evals in CI
|
|
161
163
|
|
|
@@ -164,7 +166,7 @@ gate fail the job:
|
|
|
164
166
|
|
|
165
167
|
```text
|
|
166
168
|
# CURSOR_API_KEY comes from the CI secret store.
|
|
167
|
-
|
|
169
|
+
agent-sdk eval --dir . --json --no-stream \
|
|
168
170
|
--junit reports/evals.xml \
|
|
169
171
|
--artifacts reports/evals \
|
|
170
172
|
> reports/evals.json
|
|
@@ -232,7 +234,7 @@ moves the freeze line instead of proving the change.
|
|
|
232
234
|
|
|
233
235
|
- [Evals reference](/docs/reference/evals.md): complete authoring and runner
|
|
234
236
|
contracts
|
|
235
|
-
- [`skills/evals/SKILL.md`](
|
|
237
|
+
- [`skills/evals/SKILL.md`](/docs/package/skills/evals/SKILL.md): author and seed
|
|
236
238
|
cases with a coding agent
|
|
237
239
|
- [Hillclimbing](/docs/hillclimbing.md): measure a change and lock the win
|
|
238
240
|
- [Sessions](/docs/reference/sessions.md): events available to trajectory
|