@grok/agent-sdk 0.2.28 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (528) hide show
  1. package/AGENTS.md +29 -29
  2. package/dist/bin/agent-serve.d.ts +2 -2
  3. package/dist/bin/agent-serve.js +3 -3
  4. package/dist/channels/pagerduty/api.js +1 -1
  5. package/dist/channels/pagerduty/pagerduty-channel.d.ts +1 -1
  6. package/dist/channels/pagerduty/pagerduty-channel.d.ts.map +1 -1
  7. package/dist/channels/pagerduty/pagerduty-channel.js +1 -1
  8. package/dist/channels/pagerduty/replay.d.ts +1 -1
  9. package/dist/channels/pagerduty/replay.js +4 -4
  10. package/dist/channels/slack/cli-project.js +1 -1
  11. package/dist/docs/404.html +2 -2
  12. package/dist/docs/assets/{app.DF9vlcdn.js → app.BzTUBOAD.js} +4 -4
  13. package/dist/docs/assets/building-with-agents.md.BqJ7AAjG.js +6 -0
  14. package/dist/docs/assets/{building-with-agents.md.Bf1yH2tG.lean.js → building-with-agents.md.BqJ7AAjG.lean.js} +1 -1
  15. package/dist/docs/assets/changelog.md.BWhC6DET.js +1 -0
  16. package/dist/docs/assets/changelog.md.BWhC6DET.lean.js +1 -0
  17. package/dist/docs/assets/chunks/@localSearchIndexroot.CWxwVoCG.js +1 -0
  18. package/dist/docs/assets/chunks/{VPLocalSearchBox.D8OuwKO0.js → VPLocalSearchBox.DsnRlUT5.js} +1 -1
  19. package/dist/docs/assets/chunks/{arc.DCq_TCYV.js → arc.DH8GOhiv.js} +1 -1
  20. package/dist/docs/assets/chunks/{architectureDiagram-Q4EWVU46.PWRNTPbk.js → architectureDiagram-Q4EWVU46.9m9ObyuE.js} +1 -1
  21. package/dist/docs/assets/chunks/{baseUniq.BeRPvRUL.js → baseUniq.BXHpx1lE.js} +1 -1
  22. package/dist/docs/assets/chunks/{blockDiagram-DXYQGD6D.BuRmZVSo.js → blockDiagram-DXYQGD6D.xY3EX59q.js} +1 -1
  23. package/dist/docs/assets/chunks/{c4Diagram-AHTNJAMY.7DKSMwuv.js → c4Diagram-AHTNJAMY.DnqgMVgc.js} +1 -1
  24. package/dist/docs/assets/chunks/channel.Cctv0f7X.js +1 -0
  25. package/dist/docs/assets/chunks/{chunk-4BX2VUAB.7VrO9-9i.js → chunk-4BX2VUAB.BjFxK2A3.js} +1 -1
  26. package/dist/docs/assets/chunks/{chunk-4TB4RGXK.CHystNE2.js → chunk-4TB4RGXK.BujS_Spt.js} +1 -1
  27. package/dist/docs/assets/chunks/{chunk-55IACEB6.Nuh9TbTq.js → chunk-55IACEB6.YbTjhdaM.js} +1 -1
  28. package/dist/docs/assets/chunks/{chunk-EDXVE4YY.CA0EKugm.js → chunk-EDXVE4YY.DnQ8t4GC.js} +1 -1
  29. package/dist/docs/assets/chunks/{chunk-FMBD7UC4.F9_mp7U8.js → chunk-FMBD7UC4.B67AwAmj.js} +1 -1
  30. package/dist/docs/assets/chunks/{chunk-OYMX7WX6.CEU4QfE1.js → chunk-OYMX7WX6.DrZIAFOm.js} +1 -1
  31. package/dist/docs/assets/chunks/{chunk-QZHKN3VN.B8h_x9_i.js → chunk-QZHKN3VN.BLmmydkn.js} +1 -1
  32. package/dist/docs/assets/chunks/{chunk-YZCP3GAM.Btfb-rKg.js → chunk-YZCP3GAM.DaoOfw0O.js} +1 -1
  33. package/dist/docs/assets/chunks/classDiagram-6PBFFD2Q.DU5sqchW.js +1 -0
  34. package/dist/docs/assets/chunks/classDiagram-v2-HSJHXN6E.DU5sqchW.js +1 -0
  35. package/dist/docs/assets/chunks/clone.B9rdNt6j.js +1 -0
  36. package/dist/docs/assets/chunks/{cose-bilkent-S5V4N54A.DKvc96Wa.js → cose-bilkent-S5V4N54A.DMQuNhM5.js} +1 -1
  37. package/dist/docs/assets/chunks/{dagre-KV5264BT.CMo0cy1b.js → dagre-KV5264BT.CZoGVuUf.js} +1 -1
  38. package/dist/docs/assets/chunks/{diagram-5BDNPKRD.DDgMOQhB.js → diagram-5BDNPKRD.CLt9uOyL.js} +1 -1
  39. package/dist/docs/assets/chunks/{diagram-G4DWMVQ6.BdJnoIsU.js → diagram-G4DWMVQ6.DpczEm1G.js} +1 -1
  40. package/dist/docs/assets/chunks/{diagram-MMDJMWI5.oSvhAZaL.js → diagram-MMDJMWI5.D4r13DvJ.js} +1 -1
  41. package/dist/docs/assets/chunks/{diagram-TYMM5635.C9zZLIE-.js → diagram-TYMM5635.CYU9eGFu.js} +1 -1
  42. package/dist/docs/assets/chunks/{erDiagram-SMLLAGMA.BNhVyccy.js → erDiagram-SMLLAGMA.QA9naWEw.js} +1 -1
  43. package/dist/docs/assets/chunks/{flowDiagram-DWJPFMVM.BEFLLBiU.js → flowDiagram-DWJPFMVM.BbNoVZYd.js} +1 -1
  44. package/dist/docs/assets/chunks/{ganttDiagram-T4ZO3ILL.BdUzXBgY.js → ganttDiagram-T4ZO3ILL.IHjDoG7r.js} +1 -1
  45. package/dist/docs/assets/chunks/{gitGraphDiagram-UUTBAWPF.Rd3LlOIJ.js → gitGraphDiagram-UUTBAWPF.B21VMdf5.js} +1 -1
  46. package/dist/docs/assets/chunks/{graph.DnJC1GpG.js → graph.DgJ4L-8V.js} +1 -1
  47. package/dist/docs/assets/chunks/{infoDiagram-42DDH7IO.xv0xntDJ.js → infoDiagram-42DDH7IO.CQjrcthc.js} +1 -1
  48. package/dist/docs/assets/chunks/{ishikawaDiagram-UXIWVN3A.BaGIlviU.js → ishikawaDiagram-UXIWVN3A.D4tPGtVo.js} +1 -1
  49. package/dist/docs/assets/chunks/{journeyDiagram-VCZTEJTY.Cof_h0fp.js → journeyDiagram-VCZTEJTY.BD3WaNbQ.js} +1 -1
  50. package/dist/docs/assets/chunks/{kanban-definition-6JOO6SKY.C5GozFuU.js → kanban-definition-6JOO6SKY.Ca0o4Tqz.js} +1 -1
  51. package/dist/docs/assets/chunks/{layout.zc9UShLc.js → layout.DXXqX4MQ.js} +1 -1
  52. package/dist/docs/assets/chunks/{linear.BoqrFVcJ.js → linear.5c61gNw4.js} +1 -1
  53. package/dist/docs/assets/chunks/{min.DwIfr8hG.js → min.B1-NAjHX.js} +1 -1
  54. package/dist/docs/assets/chunks/{mindmap-definition-QFDTVHPH.Dkq2yTtp.js → mindmap-definition-QFDTVHPH.BELznv3-.js} +1 -1
  55. package/dist/docs/assets/chunks/{pieDiagram-DEJITSTG.B4CpTmHK.js → pieDiagram-DEJITSTG.qPzO1mq5.js} +1 -1
  56. package/dist/docs/assets/chunks/{quadrantDiagram-34T5L4WZ.BqXnFC9C.js → quadrantDiagram-34T5L4WZ.CxIvzbjX.js} +1 -1
  57. package/dist/docs/assets/chunks/{requirementDiagram-MS252O5E.DhIN2z-E.js → requirementDiagram-MS252O5E.CqA0iJx8.js} +1 -1
  58. package/dist/docs/assets/chunks/{sankeyDiagram-XADWPNL6.BhxUAmg2.js → sankeyDiagram-XADWPNL6.XRTutBnb.js} +1 -1
  59. package/dist/docs/assets/chunks/{sequenceDiagram-FGHM5R23.BqgU7HU-.js → sequenceDiagram-FGHM5R23.E--7bRLQ.js} +1 -1
  60. package/dist/docs/assets/chunks/{stateDiagram-FHFEXIEX.B1BCO18N.js → stateDiagram-FHFEXIEX.C4vGu8zq.js} +1 -1
  61. package/dist/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.ECMu2BdJ.js +1 -0
  62. package/dist/docs/assets/chunks/{theme.oV7rddEz.js → theme.DMj_7q3B.js} +2 -2
  63. package/dist/docs/assets/chunks/{timeline-definition-GMOUNBTQ.oRI8UwMC.js → timeline-definition-GMOUNBTQ.C7LgNVAL.js} +1 -1
  64. package/dist/docs/assets/chunks/{vennDiagram-DHZGUBPP.DmuWrIxN.js → vennDiagram-DHZGUBPP.HUwoZcy_.js} +1 -1
  65. package/dist/docs/assets/chunks/{wardley-RL74JXVD.CEBjM5ov.js → wardley-RL74JXVD.BxFEFYZS.js} +1 -1
  66. package/dist/docs/assets/chunks/{wardleyDiagram-NUSXRM2D.m5zsCOF9.js → wardleyDiagram-NUSXRM2D.B2pSo5rB.js} +1 -1
  67. package/dist/docs/assets/chunks/{xychartDiagram-5P7HB3ND.cPN-587n.js → xychartDiagram-5P7HB3ND.NQhMiOOL.js} +1 -1
  68. package/dist/docs/assets/deployment.md.BtqJrHhp.js +31 -0
  69. package/dist/docs/assets/{deployment.md.DDmjAVbg.lean.js → deployment.md.BtqJrHhp.lean.js} +1 -1
  70. package/dist/docs/assets/evals.md.Dne6VRmZ.js +49 -0
  71. package/dist/docs/assets/evals.md.Dne6VRmZ.lean.js +1 -0
  72. package/dist/docs/assets/{guides_agent-to-agent.md.D1D5f6Jc.js → guides_agent-to-agent.md.BcmSqL2z.js} +8 -8
  73. package/dist/docs/assets/guides_agent-to-agent.md.BcmSqL2z.lean.js +1 -0
  74. package/dist/docs/assets/{guides_bitbucket.md.DkjkBRU8.js → guides_bitbucket.md.BiA6ZMFW.js} +27 -27
  75. package/dist/docs/assets/{guides_bitbucket.md.DkjkBRU8.lean.js → guides_bitbucket.md.BiA6ZMFW.lean.js} +1 -1
  76. package/dist/docs/assets/guides_cloud-agents.md.CiXAW6on.js +9 -0
  77. package/dist/docs/assets/{guides_cloud-agents.md.Ceyjv0ul.lean.js → guides_cloud-agents.md.CiXAW6on.lean.js} +1 -1
  78. package/dist/docs/assets/guides_convert-automation.md.BvTxsJ8J.js +26 -0
  79. package/dist/docs/assets/{guides_convert-automation.md.BjraCvok.lean.js → guides_convert-automation.md.BvTxsJ8J.lean.js} +1 -1
  80. package/dist/docs/assets/{guides_github.md.CgN5sNbW.js → guides_github.md.6jiEH7dg.js} +11 -19
  81. package/dist/docs/assets/{guides_github.md.CgN5sNbW.lean.js → guides_github.md.6jiEH7dg.lean.js} +1 -1
  82. package/dist/docs/assets/{guides_gitlab.md.DCE0p7_w.js → guides_gitlab.md.DcKusOcU.js} +63 -73
  83. package/dist/docs/assets/{guides_gitlab.md.DCE0p7_w.lean.js → guides_gitlab.md.DcKusOcU.lean.js} +1 -1
  84. package/dist/docs/assets/guides_improve.md.o3WOGR7n.js +16 -0
  85. package/dist/docs/assets/{guides_improve.md.BFYtrtLE.lean.js → guides_improve.md.o3WOGR7n.lean.js} +1 -1
  86. package/dist/docs/assets/{guides_jev.md.Ccw4pedT.js → guides_jev.md.DGCzkMUN.js} +20 -60
  87. package/dist/docs/assets/{guides_jev.md.Ccw4pedT.lean.js → guides_jev.md.DGCzkMUN.lean.js} +1 -1
  88. package/dist/docs/assets/{guides_mcp-oauth.md.ekpWOZCC.js → guides_mcp-oauth.md.C-Sx_IPa.js} +3 -5
  89. package/dist/docs/assets/{guides_mcp-oauth.md.ekpWOZCC.lean.js → guides_mcp-oauth.md.C-Sx_IPa.lean.js} +1 -1
  90. package/dist/docs/assets/guides_migrate.md.BbhHYeCr.js +3 -0
  91. package/dist/docs/assets/{guides_migrate.md.RS9Eu-Xh.lean.js → guides_migrate.md.BbhHYeCr.lean.js} +1 -1
  92. package/dist/docs/assets/guides_opentelemetry.md.BRSktBHt.js +26 -0
  93. package/dist/docs/assets/guides_opentelemetry.md.BRSktBHt.lean.js +1 -0
  94. package/dist/docs/assets/guides_pagerduty.md.pMXUkZZc.js +61 -0
  95. package/dist/docs/assets/guides_pagerduty.md.pMXUkZZc.lean.js +1 -0
  96. package/dist/docs/assets/guides_slack.md.BgIGG9ur.js +82 -0
  97. package/dist/docs/assets/{guides_slack.md.BqiGSYGX.lean.js → guides_slack.md.BgIGG9ur.lean.js} +1 -1
  98. package/dist/docs/assets/{guides_webhooks.md.BJk63ZTs.js → guides_webhooks.md.B5DXt2v1.js} +7 -7
  99. package/dist/docs/assets/{guides_webhooks.md.BJk63ZTs.lean.js → guides_webhooks.md.B5DXt2v1.lean.js} +1 -1
  100. package/dist/docs/assets/hillclimbing.md.3TkdF79A.js +5 -0
  101. package/dist/docs/assets/hillclimbing.md.3TkdF79A.lean.js +1 -0
  102. package/dist/docs/assets/index.md.DRfaoRCZ.js +20 -0
  103. package/dist/docs/assets/index.md.DRfaoRCZ.lean.js +1 -0
  104. package/dist/docs/assets/quickstart.md.r-oE8nkX.js +113 -0
  105. package/dist/docs/assets/{quickstart.md.DaLk4ZIF.lean.js → quickstart.md.r-oE8nkX.lean.js} +1 -1
  106. package/dist/docs/assets/reference_access.md.C5lblfUC.js +1 -0
  107. package/dist/docs/assets/{reference_access.md.CExlH_mR.lean.js → reference_access.md.C5lblfUC.lean.js} +1 -1
  108. package/dist/docs/assets/reference_agent-config.md.C0Tp_Fk9.js +27 -0
  109. package/dist/docs/assets/{reference_agent-config.md.11E_xe--.lean.js → reference_agent-config.md.C0Tp_Fk9.lean.js} +1 -1
  110. package/dist/docs/assets/reference_artifacts.md.BJpAI7h9.js +17 -0
  111. package/dist/docs/assets/{reference_artifacts.md.C-hKR2hp.lean.js → reference_artifacts.md.BJpAI7h9.lean.js} +1 -1
  112. package/dist/docs/assets/reference_channels.md.CPWOpdsC.js +21 -0
  113. package/dist/docs/assets/{reference_channels.md.DsJfGSBJ.lean.js → reference_channels.md.CPWOpdsC.lean.js} +1 -1
  114. package/dist/docs/assets/reference_cli.md.B8GdpaRj.js +83 -0
  115. package/dist/docs/assets/reference_cli.md.B8GdpaRj.lean.js +1 -0
  116. package/dist/docs/assets/reference_connections.md.DoY4vrDH.js +54 -0
  117. package/dist/docs/assets/{reference_connections.md.BoAotr7_.lean.js → reference_connections.md.DoY4vrDH.lean.js} +1 -1
  118. package/dist/docs/assets/reference_evals.md.DgfUtfIc.js +69 -0
  119. package/dist/docs/assets/{reference_evals.md.DYKjGFi-.lean.js → reference_evals.md.DgfUtfIc.lean.js} +1 -1
  120. package/dist/docs/assets/{reference_extensions.md.pEVj72Xo.js → reference_extensions.md.DHbW6eOf.js} +6 -8
  121. package/dist/docs/assets/{reference_extensions.md.pEVj72Xo.lean.js → reference_extensions.md.DHbW6eOf.lean.js} +1 -1
  122. package/dist/docs/assets/reference_hooks.md.CSWmOwMM.js +39 -0
  123. package/dist/docs/assets/reference_hooks.md.CSWmOwMM.lean.js +1 -0
  124. package/dist/docs/assets/reference_http-api.md.C7Cb0r2J.js +7 -0
  125. package/dist/docs/assets/{reference_http-api.md._RWELr3d.lean.js → reference_http-api.md.C7Cb0r2J.lean.js} +1 -1
  126. package/dist/docs/assets/reference_instructions.md.CV6Qq0UX.js +14 -0
  127. package/dist/docs/assets/reference_instructions.md.CV6Qq0UX.lean.js +1 -0
  128. package/dist/docs/assets/reference_limits.md.CbdtFr4T.js +8 -0
  129. package/dist/docs/assets/reference_limits.md.CbdtFr4T.lean.js +1 -0
  130. package/dist/docs/assets/reference_memory.md.CdoJISM-.js +3 -0
  131. package/dist/docs/assets/{reference_memory.md.Dd7cyV9V.lean.js → reference_memory.md.CdoJISM-.lean.js} +1 -1
  132. package/dist/docs/assets/reference_playground.md.DIUsRAcy.js +1 -0
  133. package/dist/docs/assets/{reference_playground.md.CQiJk9CM.lean.js → reference_playground.md.DIUsRAcy.lean.js} +1 -1
  134. package/dist/docs/assets/reference_project-layout.md.9DiKoE16.js +26 -0
  135. package/dist/docs/assets/reference_project-layout.md.9DiKoE16.lean.js +1 -0
  136. package/dist/docs/assets/reference_schedules.md.RhQ9ZLo1.js +45 -0
  137. package/dist/docs/assets/{reference_schedules.md.RC47aFm4.lean.js → reference_schedules.md.RhQ9ZLo1.lean.js} +1 -1
  138. package/dist/docs/assets/reference_sessions.md.NXVQt9mP.js +22 -0
  139. package/dist/docs/assets/{reference_sessions.md.C7KFhWXV.lean.js → reference_sessions.md.NXVQt9mP.lean.js} +1 -1
  140. package/dist/docs/assets/{reference_skills.md.Dej2zQxi.js → reference_skills.md.DzCPNPkG.js} +4 -4
  141. package/dist/docs/assets/{reference_skills.md.Dej2zQxi.lean.js → reference_skills.md.DzCPNPkG.lean.js} +1 -1
  142. package/dist/docs/assets/reference_subagents.md.Dvpq2sUu.js +10 -0
  143. package/dist/docs/assets/{reference_subagents.md.CAhvW4yS.lean.js → reference_subagents.md.Dvpq2sUu.lean.js} +1 -1
  144. package/dist/docs/assets/reference_tools.md.B2OaR1Nb.js +62 -0
  145. package/dist/docs/assets/{reference_tools.md.DQr7Fnhu.lean.js → reference_tools.md.B2OaR1Nb.lean.js} +1 -1
  146. package/dist/docs/assets/templates_agentic-owners.md.B6ZmKAzT.js +13 -0
  147. package/dist/docs/assets/{templates_agentic-owners.md.Dh2nDSqg.lean.js → templates_agentic-owners.md.B6ZmKAzT.lean.js} +1 -1
  148. package/dist/docs/assets/templates_reviews.md.dr9Xv_KM.js +10 -0
  149. package/dist/docs/assets/templates_reviews.md.dr9Xv_KM.lean.js +1 -0
  150. package/dist/docs/assets/troubleshooting.md.DfKKvW1R.js +3 -0
  151. package/dist/docs/assets/{troubleshooting.md.AGJ3jDzj.lean.js → troubleshooting.md.DfKKvW1R.lean.js} +1 -1
  152. package/dist/docs/building-with-agents.html +41 -44
  153. package/dist/docs/building-with-agents.md +11 -104
  154. package/dist/docs/changelog.html +37 -37
  155. package/dist/docs/changelog.md +5 -145
  156. package/dist/docs/deployment.html +54 -54
  157. package/dist/docs/deployment.md +86 -265
  158. package/dist/docs/evals.html +52 -71
  159. package/dist/docs/evals.md +33 -127
  160. package/dist/docs/guides/agent-to-agent.html +43 -43
  161. package/dist/docs/guides/agent-to-agent.md +27 -64
  162. package/dist/docs/guides/bitbucket.html +62 -62
  163. package/dist/docs/guides/bitbucket.md +73 -134
  164. package/dist/docs/guides/cloud-agents.html +37 -37
  165. package/dist/docs/guides/cloud-agents.md +23 -53
  166. package/dist/docs/guides/convert-automation.html +39 -56
  167. package/dist/docs/guides/convert-automation.md +31 -115
  168. package/dist/docs/guides/github.html +46 -54
  169. package/dist/docs/guides/github.md +49 -131
  170. package/dist/docs/guides/gitlab.html +98 -108
  171. package/dist/docs/guides/gitlab.md +104 -184
  172. package/dist/docs/guides/improve.html +38 -43
  173. package/dist/docs/guides/improve.md +24 -90
  174. package/dist/docs/guides/jev.html +55 -95
  175. package/dist/docs/guides/jev.md +57 -159
  176. package/dist/docs/guides/mcp-oauth.html +38 -40
  177. package/dist/docs/guides/mcp-oauth.md +34 -69
  178. package/dist/docs/guides/migrate.html +38 -43
  179. package/dist/docs/guides/migrate.md +56 -130
  180. package/dist/docs/guides/opentelemetry.html +41 -43
  181. package/dist/docs/guides/opentelemetry.md +32 -27
  182. package/dist/docs/guides/pagerduty.html +50 -49
  183. package/dist/docs/guides/pagerduty.md +65 -199
  184. package/dist/docs/guides/slack.html +51 -57
  185. package/dist/docs/guides/slack.md +76 -205
  186. package/dist/docs/guides/webhooks.html +42 -42
  187. package/dist/docs/guides/webhooks.md +64 -146
  188. package/dist/docs/hashmap.json +1 -1
  189. package/dist/docs/hillclimbing.html +40 -42
  190. package/dist/docs/hillclimbing.md +40 -102
  191. package/dist/docs/index.html +55 -62
  192. package/dist/docs/index.md +49 -82
  193. package/dist/docs/llms-full.txt +2292 -5830
  194. package/dist/docs/llms.txt +9 -15
  195. package/dist/docs/quickstart.html +96 -153
  196. package/dist/docs/quickstart.md +93 -178
  197. package/dist/docs/reference/access.html +36 -36
  198. package/dist/docs/reference/access.md +7 -12
  199. package/dist/docs/reference/agent-config.html +43 -56
  200. package/dist/docs/reference/agent-config.md +93 -176
  201. package/dist/docs/reference/artifacts.html +38 -39
  202. package/dist/docs/reference/artifacts.md +38 -62
  203. package/dist/docs/reference/channels.html +38 -39
  204. package/dist/docs/reference/channels.md +102 -151
  205. package/dist/docs/reference/cli.html +68 -84
  206. package/dist/docs/reference/cli.md +233 -698
  207. package/dist/docs/reference/connections.html +45 -57
  208. package/dist/docs/reference/connections.md +83 -164
  209. package/dist/docs/reference/evals.html +66 -53
  210. package/dist/docs/reference/evals.md +97 -128
  211. package/dist/docs/reference/extensions.html +41 -43
  212. package/dist/docs/reference/extensions.md +37 -70
  213. package/dist/docs/reference/hooks.html +40 -52
  214. package/dist/docs/reference/hooks.md +62 -133
  215. package/dist/docs/reference/http-api.html +38 -42
  216. package/dist/docs/reference/http-api.md +66 -118
  217. package/dist/docs/reference/instructions.html +39 -39
  218. package/dist/docs/reference/instructions.md +19 -29
  219. package/dist/docs/reference/limits.html +38 -38
  220. package/dist/docs/reference/limits.md +9 -21
  221. package/dist/docs/reference/memory.html +37 -38
  222. package/dist/docs/reference/memory.md +38 -75
  223. package/dist/docs/reference/playground.html +36 -36
  224. package/dist/docs/reference/playground.md +14 -22
  225. package/dist/docs/reference/project-layout.html +49 -48
  226. package/dist/docs/reference/project-layout.md +38 -84
  227. package/dist/docs/reference/schedules.html +46 -48
  228. package/dist/docs/reference/schedules.md +59 -90
  229. package/dist/docs/reference/sessions.html +37 -37
  230. package/dist/docs/reference/sessions.md +66 -113
  231. package/dist/docs/reference/skills.html +39 -39
  232. package/dist/docs/reference/skills.md +11 -31
  233. package/dist/docs/reference/subagents.html +37 -37
  234. package/dist/docs/reference/subagents.md +17 -31
  235. package/dist/docs/reference/tools.html +47 -64
  236. package/dist/docs/reference/tools.md +48 -130
  237. package/dist/docs/templates/agentic-owners.html +38 -38
  238. package/dist/docs/templates/agentic-owners.md +50 -74
  239. package/dist/docs/templates/reviews.html +69 -0
  240. package/dist/docs/templates/reviews.md +71 -0
  241. package/dist/docs/troubleshooting.html +38 -36
  242. package/dist/docs/troubleshooting.md +48 -101
  243. package/dist/evals.d.ts +2 -2
  244. package/dist/extensions/jev/extension.d.ts +12 -2
  245. package/dist/extensions/jev/extension.d.ts.map +1 -1
  246. package/dist/extensions/jev/extension.js +4 -0
  247. package/dist/extensions/jev/lib/evaluate.d.ts +31 -4
  248. package/dist/extensions/jev/lib/evaluate.d.ts.map +1 -1
  249. package/dist/extensions/jev/lib/evaluate.js +18 -4
  250. package/dist/internal/cli/command.d.ts +1 -1
  251. package/dist/internal/cli/command.js +1 -1
  252. package/dist/internal/cli/commands/channels.js +6 -6
  253. package/dist/internal/cli/commands/develop.d.ts.map +1 -1
  254. package/dist/internal/cli/commands/develop.js +23 -14
  255. package/dist/internal/cli/commands/hosted.js +4 -4
  256. package/dist/internal/cli/commands/index.d.ts +1 -1
  257. package/dist/internal/cli/commands/index.d.ts.map +1 -1
  258. package/dist/internal/cli/commands/index.js +1 -1
  259. package/dist/internal/cli/commands/mcp.js +1 -1
  260. package/dist/internal/cli/commands/project.js +4 -4
  261. package/dist/internal/cli/commands/scaffold.d.ts.map +1 -1
  262. package/dist/internal/cli/commands/scaffold.js +9 -6
  263. package/dist/internal/cli/commands/serve.d.ts +1 -1
  264. package/dist/internal/cli/commands/serve.js +5 -5
  265. package/dist/internal/cli/commands/session.d.ts.map +1 -1
  266. package/dist/internal/cli/commands/session.js +12 -7
  267. package/dist/internal/cli/dispatch.d.ts +3 -3
  268. package/dist/internal/cli/dispatch.js +3 -3
  269. package/dist/internal/cli/errors.d.ts +1 -1
  270. package/dist/internal/cli/errors.d.ts.map +1 -1
  271. package/dist/internal/cli/errors.js +1 -1
  272. package/dist/internal/cli/flags.d.ts +2 -2
  273. package/dist/internal/cli/flags.d.ts.map +1 -1
  274. package/dist/internal/cli/flags.js +2 -2
  275. package/dist/internal/cli/help-command.d.ts +1 -1
  276. package/dist/internal/cli/help-command.js +2 -2
  277. package/dist/internal/cli/help.d.ts +7 -7
  278. package/dist/internal/cli/help.d.ts.map +1 -1
  279. package/dist/internal/cli/help.js +8 -8
  280. package/dist/internal/cli-ax.d.ts.map +1 -1
  281. package/dist/internal/cli-ax.js +3 -0
  282. package/dist/internal/cli-convert-automation.d.ts +1 -1
  283. package/dist/internal/cli-convert-automation.js +1 -1
  284. package/dist/internal/cli-deploy.js +1 -1
  285. package/dist/internal/cli-docs.js +1 -1
  286. package/dist/internal/cli-images.js +1 -1
  287. package/dist/internal/cli-migrate.d.ts +1 -1
  288. package/dist/internal/cli-migrate.js +1 -1
  289. package/dist/internal/cli-pagerduty.d.ts +1 -1
  290. package/dist/internal/cli-pagerduty.js +1 -1
  291. package/dist/internal/cli-skills.js +1 -1
  292. package/dist/internal/convert-automation/types.d.ts +1 -1
  293. package/dist/internal/convert-automation/types.js +1 -1
  294. package/dist/internal/cursor/credentials.js +1 -1
  295. package/dist/internal/deploy-client.d.ts +2 -2
  296. package/dist/internal/deploy-client.d.ts.map +1 -1
  297. package/dist/internal/deploy-manifest.d.ts +1 -0
  298. package/dist/internal/deploy-manifest.d.ts.map +1 -1
  299. package/dist/internal/deploy-manifest.js +0 -1
  300. package/dist/internal/discovery/agent-config.d.ts +2 -6
  301. package/dist/internal/discovery/agent-config.d.ts.map +1 -1
  302. package/dist/internal/discovery/agent-config.js +8 -25
  303. package/dist/internal/discovery/agent.d.ts.map +1 -1
  304. package/dist/internal/discovery/agent.js +2 -3
  305. package/dist/internal/discovery/project.d.ts.map +1 -1
  306. package/dist/internal/discovery/project.js +15 -0
  307. package/dist/internal/init-project.d.ts +7 -3
  308. package/dist/internal/init-project.d.ts.map +1 -1
  309. package/dist/internal/init-project.js +60 -12
  310. package/dist/internal/init-scaffold.d.ts +2 -1
  311. package/dist/internal/init-scaffold.d.ts.map +1 -1
  312. package/dist/internal/init-scaffold.js +20 -7
  313. package/dist/internal/peer-connections.js +1 -1
  314. package/dist/internal/reminder-runner.js +1 -1
  315. package/dist/internal/server.d.ts +1 -1
  316. package/dist/internal/session-cost.d.ts +4 -4
  317. package/dist/internal/session-cost.d.ts.map +1 -1
  318. package/dist/internal/session-cost.js +4 -4
  319. package/dist/internal/session-engine.d.ts +1 -1
  320. package/dist/internal/session-engine.js +2 -2
  321. package/dist/internal/shallow-clone.js +1 -1
  322. package/dist/playground/assets/index-D6mVn9og.js +62 -0
  323. package/dist/playground/assets/index-DlALM6gr.css +1 -0
  324. package/dist/playground/index.html +2 -2
  325. package/dist/types.d.ts +9 -22
  326. package/dist/types.d.ts.map +1 -1
  327. package/docs/README.md +49 -82
  328. package/docs/building-with-agents.md +11 -104
  329. package/docs/changelog.md +6 -146
  330. package/docs/deployment.md +86 -265
  331. package/docs/evals.md +34 -128
  332. package/docs/guides/agent-to-agent.md +27 -64
  333. package/docs/guides/bitbucket.md +73 -134
  334. package/docs/guides/cloud-agents.md +23 -53
  335. package/docs/guides/convert-automation.md +31 -115
  336. package/docs/guides/github.md +49 -131
  337. package/docs/guides/gitlab.md +104 -184
  338. package/docs/guides/improve.md +24 -90
  339. package/docs/guides/jev.md +57 -159
  340. package/docs/guides/mcp-oauth.md +34 -69
  341. package/docs/guides/migrate.md +56 -130
  342. package/docs/guides/opentelemetry.md +37 -27
  343. package/docs/guides/pagerduty.md +68 -200
  344. package/docs/guides/slack.md +76 -205
  345. package/docs/guides/webhooks.md +64 -146
  346. package/docs/hillclimbing.md +41 -103
  347. package/docs/quickstart.md +93 -178
  348. package/docs/reference/access.md +7 -12
  349. package/docs/reference/agent-config.md +93 -176
  350. package/docs/reference/artifacts.md +38 -62
  351. package/docs/reference/channels.md +102 -151
  352. package/docs/reference/cli.md +233 -698
  353. package/docs/reference/connections.md +83 -164
  354. package/docs/reference/evals.md +97 -128
  355. package/docs/reference/extensions.md +37 -70
  356. package/docs/reference/hooks.md +61 -132
  357. package/docs/reference/http-api.md +66 -118
  358. package/docs/reference/instructions.md +20 -30
  359. package/docs/reference/limits.md +10 -22
  360. package/docs/reference/memory.md +38 -75
  361. package/docs/reference/playground.md +14 -22
  362. package/docs/reference/project-layout.md +38 -84
  363. package/docs/reference/schedules.md +59 -90
  364. package/docs/reference/sessions.md +66 -113
  365. package/docs/reference/skills.md +12 -32
  366. package/docs/reference/subagents.md +17 -31
  367. package/docs/reference/tools.md +48 -130
  368. package/docs/templates/agentic-owners.md +50 -74
  369. package/docs/templates/reviews.md +76 -0
  370. package/docs/troubleshooting.md +48 -101
  371. package/package.json +3 -9
  372. package/skills/cli/SKILL.md +15 -15
  373. package/skills/create-agent/SKILL.md +20 -19
  374. package/skills/debug/SKILL.md +4 -4
  375. package/skills/deploy/SKILL.md +2 -2
  376. package/skills/evals/SKILL.md +8 -8
  377. package/skills/framework-map/SKILL.md +3 -3
  378. package/skills/github/SKILL.md +7 -7
  379. package/skills/hillclimb/SKILL.md +4 -4
  380. package/skills/mcp-auth/SKILL.md +3 -3
  381. package/skills/migrate-v2/SKILL.md +40 -41
  382. package/skills/otel/SKILL.md +2 -2
  383. package/skills/setup-slack/SKILL.md +8 -8
  384. package/templates/agentic-owners/README.md +5 -5
  385. package/templates/agentic-owners/bot/tools/record_review.ts +1 -1
  386. package/templates/agentic-owners/package.json +1 -1
  387. package/templates/agents-md/README.md +4 -4
  388. package/templates/agents-md/package.json +1 -1
  389. package/templates/code-wiki/README.md +3 -3
  390. package/templates/code-wiki/package.json +1 -1
  391. package/templates/demo/README.md +3 -3
  392. package/templates/demo/package.json +1 -1
  393. package/templates/pr-autofixer/package.json +1 -1
  394. package/templates/security-help/README.md +5 -5
  395. package/templates/security-help/package.json +1 -1
  396. package/templates/security-reviewer/README.md +3 -3
  397. package/templates/security-reviewer/package.json +1 -1
  398. package/templates/thermo-quality-review/README.md +4 -4
  399. package/templates/thermo-quality-review/package.json +1 -1
  400. package/templates/thermo-review/README.md +4 -4
  401. package/templates/thermo-review/package.json +1 -1
  402. package/templates/triage/README.md +3 -3
  403. package/templates/triage/overlays/jira/README.md +3 -3
  404. package/templates/triage/overlays/linear/README.md +3 -3
  405. package/templates/triage/package.json +1 -1
  406. package/dist/docs/assets/building-with-agents.md.Bf1yH2tG.js +0 -9
  407. package/dist/docs/assets/changelog.md.fZ-NNFeJ.js +0 -1
  408. package/dist/docs/assets/changelog.md.fZ-NNFeJ.lean.js +0 -1
  409. package/dist/docs/assets/chunks/@localSearchIndexroot.mfzjiOsL.js +0 -1
  410. package/dist/docs/assets/chunks/channel.goFR5lbe.js +0 -1
  411. package/dist/docs/assets/chunks/classDiagram-6PBFFD2Q.B2yLtFja.js +0 -1
  412. package/dist/docs/assets/chunks/classDiagram-v2-HSJHXN6E.B2yLtFja.js +0 -1
  413. package/dist/docs/assets/chunks/clone.CztV97Wu.js +0 -1
  414. package/dist/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.CZlVHdDZ.js +0 -1
  415. package/dist/docs/assets/deployment.md.DDmjAVbg.js +0 -31
  416. package/dist/docs/assets/evals.md.OjKb7tNA.js +0 -68
  417. package/dist/docs/assets/evals.md.OjKb7tNA.lean.js +0 -1
  418. package/dist/docs/assets/guides_agent-to-agent.md.D1D5f6Jc.lean.js +0 -1
  419. package/dist/docs/assets/guides_cloud-agents.md.Ceyjv0ul.js +0 -9
  420. package/dist/docs/assets/guides_convert-automation.md.BjraCvok.js +0 -43
  421. package/dist/docs/assets/guides_grokbot-agents.md.D0PpM_Gf.js +0 -15
  422. package/dist/docs/assets/guides_grokbot-agents.md.D0PpM_Gf.lean.js +0 -1
  423. package/dist/docs/assets/guides_hooks.md.BjeEiM7s.js +0 -54
  424. package/dist/docs/assets/guides_hooks.md.BjeEiM7s.lean.js +0 -1
  425. package/dist/docs/assets/guides_improve.md.BFYtrtLE.js +0 -21
  426. package/dist/docs/assets/guides_migrate.md.RS9Eu-Xh.js +0 -8
  427. package/dist/docs/assets/guides_opentelemetry.md.CoZ1kNyd.js +0 -28
  428. package/dist/docs/assets/guides_opentelemetry.md.CoZ1kNyd.lean.js +0 -1
  429. package/dist/docs/assets/guides_pagerduty.md.mnKXe9U0.js +0 -60
  430. package/dist/docs/assets/guides_pagerduty.md.mnKXe9U0.lean.js +0 -1
  431. package/dist/docs/assets/guides_slack.md.BqiGSYGX.js +0 -88
  432. package/dist/docs/assets/hillclimbing.md.By61f3uq.js +0 -7
  433. package/dist/docs/assets/hillclimbing.md.By61f3uq.lean.js +0 -1
  434. package/dist/docs/assets/index.md.CVRjO09Y.js +0 -27
  435. package/dist/docs/assets/index.md.CVRjO09Y.lean.js +0 -1
  436. package/dist/docs/assets/quickstart.md.DaLk4ZIF.js +0 -170
  437. package/dist/docs/assets/reference_access.md.CExlH_mR.js +0 -1
  438. package/dist/docs/assets/reference_agent-config.md.11E_xe--.js +0 -40
  439. package/dist/docs/assets/reference_artifacts.md.C-hKR2hp.js +0 -18
  440. package/dist/docs/assets/reference_channels.md.DsJfGSBJ.js +0 -22
  441. package/dist/docs/assets/reference_cli.md.DQ3dCgmk.js +0 -99
  442. package/dist/docs/assets/reference_cli.md.DQ3dCgmk.lean.js +0 -1
  443. package/dist/docs/assets/reference_connections.md.BoAotr7_.js +0 -66
  444. package/dist/docs/assets/reference_evals.md.DYKjGFi-.js +0 -56
  445. package/dist/docs/assets/reference_hooks.md.B-bwJuiG.js +0 -51
  446. package/dist/docs/assets/reference_hooks.md.B-bwJuiG.lean.js +0 -1
  447. package/dist/docs/assets/reference_http-api.md._RWELr3d.js +0 -11
  448. package/dist/docs/assets/reference_instructions.md.yL2hXLnl.js +0 -14
  449. package/dist/docs/assets/reference_instructions.md.yL2hXLnl.lean.js +0 -1
  450. package/dist/docs/assets/reference_limits.md.Da1HegjB.js +0 -8
  451. package/dist/docs/assets/reference_limits.md.Da1HegjB.lean.js +0 -1
  452. package/dist/docs/assets/reference_memory.md.Dd7cyV9V.js +0 -4
  453. package/dist/docs/assets/reference_playground.md.CQiJk9CM.js +0 -1
  454. package/dist/docs/assets/reference_project-layout.md.BamcPDUn.js +0 -25
  455. package/dist/docs/assets/reference_project-layout.md.BamcPDUn.lean.js +0 -1
  456. package/dist/docs/assets/reference_prompt.md.BMjm1Jp3.js +0 -9
  457. package/dist/docs/assets/reference_prompt.md.BMjm1Jp3.lean.js +0 -1
  458. package/dist/docs/assets/reference_schedules.md.RC47aFm4.js +0 -47
  459. package/dist/docs/assets/reference_sessions.md.C7KFhWXV.js +0 -22
  460. package/dist/docs/assets/reference_subagents.md.CAhvW4yS.js +0 -10
  461. package/dist/docs/assets/reference_tools.md.DQr7Fnhu.js +0 -79
  462. package/dist/docs/assets/templates_agentic-owners.md.Dh2nDSqg.js +0 -13
  463. package/dist/docs/assets/templates_security-reviewer.md.DnwXdnB5.js +0 -3
  464. package/dist/docs/assets/templates_security-reviewer.md.DnwXdnB5.lean.js +0 -1
  465. package/dist/docs/assets/templates_thermo-quality-review.md.MJIdHkzE.js +0 -3
  466. package/dist/docs/assets/templates_thermo-quality-review.md.MJIdHkzE.lean.js +0 -1
  467. package/dist/docs/assets/templates_thermo-review.md.DfN1OFOl.js +0 -3
  468. package/dist/docs/assets/templates_thermo-review.md.DfN1OFOl.lean.js +0 -1
  469. package/dist/docs/assets/templates_triage.md.BYWMloV4.js +0 -9
  470. package/dist/docs/assets/templates_triage.md.BYWMloV4.lean.js +0 -1
  471. package/dist/docs/assets/troubleshooting.md.AGJ3jDzj.js +0 -1
  472. package/dist/docs/guides/grokbot-agents.html +0 -74
  473. package/dist/docs/guides/grokbot-agents.md +0 -129
  474. package/dist/docs/guides/hooks.html +0 -113
  475. package/dist/docs/guides/hooks.md +0 -116
  476. package/dist/docs/reference/prompt.html +0 -68
  477. package/dist/docs/reference/prompt.md +0 -47
  478. package/dist/docs/templates/security-reviewer.html +0 -62
  479. package/dist/docs/templates/security-reviewer.md +0 -81
  480. package/dist/docs/templates/thermo-quality-review.html +0 -62
  481. package/dist/docs/templates/thermo-quality-review.md +0 -75
  482. package/dist/docs/templates/thermo-review.html +0 -62
  483. package/dist/docs/templates/thermo-review.md +0 -75
  484. package/dist/docs/templates/triage.html +0 -68
  485. package/dist/docs/templates/triage.md +0 -101
  486. package/dist/extensions/cursor-grokbot-agents/extension.d.ts +0 -71
  487. package/dist/extensions/cursor-grokbot-agents/extension.d.ts.map +0 -1
  488. package/dist/extensions/cursor-grokbot-agents/extension.js +0 -64
  489. package/dist/extensions/cursor-grokbot-agents/lib/agents.d.ts +0 -11
  490. package/dist/extensions/cursor-grokbot-agents/lib/agents.d.ts.map +0 -1
  491. package/dist/extensions/cursor-grokbot-agents/lib/agents.js +0 -31
  492. package/dist/extensions/cursor-grokbot-agents/lib/backend.d.ts +0 -56
  493. package/dist/extensions/cursor-grokbot-agents/lib/backend.d.ts.map +0 -1
  494. package/dist/extensions/cursor-grokbot-agents/lib/backend.js +0 -139
  495. package/dist/extensions/cursor-grokbot-agents/lib/contacts.d.ts +0 -28
  496. package/dist/extensions/cursor-grokbot-agents/lib/contacts.d.ts.map +0 -1
  497. package/dist/extensions/cursor-grokbot-agents/lib/contacts.js +0 -46
  498. package/dist/extensions/cursor-grokbot-agents/lib/conversation.d.ts +0 -53
  499. package/dist/extensions/cursor-grokbot-agents/lib/conversation.d.ts.map +0 -1
  500. package/dist/extensions/cursor-grokbot-agents/lib/conversation.js +0 -186
  501. package/dist/extensions/cursor-grokbot-agents/lib/inputs.d.ts +0 -4
  502. package/dist/extensions/cursor-grokbot-agents/lib/inputs.d.ts.map +0 -1
  503. package/dist/extensions/cursor-grokbot-agents/lib/inputs.js +0 -15
  504. package/dist/extensions/cursor-grokbot-agents/lib/lock.d.ts +0 -12
  505. package/dist/extensions/cursor-grokbot-agents/lib/lock.d.ts.map +0 -1
  506. package/dist/extensions/cursor-grokbot-agents/lib/lock.js +0 -24
  507. package/dist/extensions/cursor-grokbot-agents/skills/consult.md +0 -40
  508. package/dist/extensions/cursor-grokbot-agents/tools/ask.d.ts +0 -4
  509. package/dist/extensions/cursor-grokbot-agents/tools/ask.d.ts.map +0 -1
  510. package/dist/extensions/cursor-grokbot-agents/tools/ask.js +0 -26
  511. package/dist/extensions/cursor-grokbot-agents/tools/check.d.ts +0 -4
  512. package/dist/extensions/cursor-grokbot-agents/tools/check.d.ts.map +0 -1
  513. package/dist/extensions/cursor-grokbot-agents/tools/check.js +0 -21
  514. package/dist/extensions/cursor-grokbot-agents/tools/interrupt.d.ts +0 -4
  515. package/dist/extensions/cursor-grokbot-agents/tools/interrupt.d.ts.map +0 -1
  516. package/dist/extensions/cursor-grokbot-agents/tools/interrupt.js +0 -13
  517. package/dist/extensions/cursor-grokbot-agents/tools/list.d.ts +0 -4
  518. package/dist/extensions/cursor-grokbot-agents/tools/list.d.ts.map +0 -1
  519. package/dist/extensions/cursor-grokbot-agents/tools/list.js +0 -22
  520. package/dist/playground/assets/index-BMpVzZzC.css +0 -1
  521. package/dist/playground/assets/index-_3zz3yh8.js +0 -62
  522. package/docs/guides/grokbot-agents.md +0 -134
  523. package/docs/guides/hooks.md +0 -121
  524. package/docs/reference/prompt.md +0 -52
  525. package/docs/templates/security-reviewer.md +0 -86
  526. package/docs/templates/thermo-quality-review.md +0 -80
  527. package/docs/templates/thermo-review.md +0 -80
  528. package/docs/templates/triage.md +0 -106
@@ -1,21 +1,13 @@
1
1
  # Evals
2
2
 
3
- An eval sends a fixed input to the real agent and checks the resulting
4
- trajectory: whether the turn succeeded, which tools ran, and what shape
5
- the answer took. Use evals to protect behavior you already understand,
6
- not to discover what the prompt should do.
3
+ An eval sends a fixed input to the real agent and checks which tools ran and what the answer looks
4
+ like. To improve an agent and lock each win with an eval, follow [Hillclimbing](/docs/hillclimbing.md).
7
5
 
8
- Evals run real model turns and real tools. Guard external writes before
9
- running a suite against an agent that can post, merge, or deploy.
6
+ Evals run real tools, so [guard any writes](#keep-side-effects-out-of-eval-sessions) first.
10
7
 
11
8
  ## Protect a tool decision
12
9
 
13
- This smoke case asks for a PR verdict, requires the read-only inspection
14
- tool, and fails if the agent tries to approve. The CLI reports all three
15
- decisions together instead of stopping at the first miss.
16
-
17
- Use the [Jev extension](/docs/guides/jev.md) when host code needs a typed
18
- choice, score, or boolean before it writes.
10
+ The case below passes only if the agent calls the inspection tool and doesn't approve.
19
11
 
20
12
  ```ts
21
13
  // evals/readiness.eval.ts
@@ -24,9 +16,7 @@ import { defineEval, includes } from "@grok/agent-sdk/evals";
24
16
  export default defineEval({
25
17
  tags: ["smoke"],
26
18
  async test(t) {
27
- await t.send(
28
- "Is https://github.com/acme/checkout/pull/42 ready to approve?"
29
- );
19
+ await t.send("Is https://github.com/acme/checkout/pull/42 ready to approve?");
30
20
 
31
21
  t.succeeded();
32
22
  t.calledTool("inspect_pr");
@@ -36,8 +26,7 @@ export default defineEval({
36
26
  });
37
27
  ```
38
28
 
39
- Evals live in the project-root `evals/` directory, beside `bot/`.
40
- Add the required concurrency config once:
29
+ Evals live in `evals/` beside `bot/`. Set how many cases run at once:
41
30
 
42
31
  ```ts
43
32
  // evals/evals.config.ts
@@ -48,52 +37,34 @@ export default defineEvalConfig({
48
37
  });
49
38
  ```
50
39
 
51
- ```text
40
+ ```bash
52
41
  agent-sdk eval --dir . --list
53
42
  agent-sdk eval --dir . --tag smoke
54
43
  ```
55
44
 
56
45
  ## Check answer shape or quality
57
46
 
58
- Prefer a deterministic shape check when the output has a contract. The
59
- failure tells you which field was wrong, and the case remains stable
60
- when the model changes its wording.
47
+ When the output has a contract, check its shape:
61
48
 
62
49
  ```ts
63
50
  import { matches } from "@grok/agent-sdk/evals";
64
51
  import { z } from "zod";
65
52
 
66
53
  const verdict = JSON.parse(t.reply ?? "{}");
67
- t.check(
68
- verdict,
69
- matches(
70
- z.object({
71
- ready: z.boolean(),
72
- blockers: z.array(z.string()),
73
- })
74
- )
75
- );
54
+ t.check(verdict, matches(z.object({ ready: z.boolean(), blockers: z.array(z.string()) })));
76
55
  ```
77
56
 
78
- Use a judge when correctness depends on meaning that a schema, regex, or
79
- known value cannot capture:
57
+ When correctness depends on meaning, use a judge. [Judges](/docs/reference/evals.md#judges) covers
58
+ models and grader options.
80
59
 
81
60
  ```ts
82
- t.judge
83
- .factuality("The lint step failed on src/sidebar.ts.")
84
- .atLeast(0.8);
61
+ t.judge.factuality("The lint step failed on src/sidebar.ts.").atLeast(0.8);
85
62
  ```
86
63
 
87
- Judge checks are tracked scores until you give them a hard gate.
88
- Configure the judge model in `evals.config.ts`. The
89
- [Evals reference](/docs/reference/evals.md#judges) owns grader options and
90
- severity rules.
91
-
92
64
  ## Test a conversation
93
65
 
94
- Several `t.send` calls in one case share a session. Assertions on a
95
- returned turn inspect only that turn, while assertions on `t` inspect
96
- the whole conversation.
66
+ Several `t.send` calls in one case share a session. An assertion on a returned turn checks only that
67
+ turn, and an assertion on `t` checks the whole conversation.
97
68
 
98
69
  ```ts
99
70
  const first = await t.send("Inspect PR 42.");
@@ -107,9 +78,7 @@ t.succeeded();
107
78
 
108
79
  ## Pin realistic inputs
109
80
 
110
- A useful eval changes only when the agent changes. Freeze canonical chat
111
- prompts, seed workspace evidence directly, and save webhook payloads
112
- instead of depending on a developer's checkout or live external state.
81
+ Seed the files the agent reads instead of depending on your checkout:
113
82
 
114
83
  ```ts
115
84
  await t.send("Review pr/diff.patch and report blockers.", {
@@ -122,47 +91,32 @@ await t.send("Review pr/diff.patch and report blockers.", {
122
91
  });
123
92
  ```
124
93
 
125
- For GitHub agents, snapshot a PR into committed fixtures:
94
+ Save a GitHub pull request's webhook payloads as fixtures:
126
95
 
127
- ```text
128
- agent-sdk github replay acme/checkout#42 \
129
- --events '*' \
130
- --dry-run \
131
- --out fixtures/github
96
+ ```bash
97
+ agent-sdk github replay https://github.com/acme/checkout/pull/42 \
98
+ --events '*' --dry-run --fixtures-dir fixtures/github
132
99
  ```
133
100
 
134
- Keep a small, fast `smoke` set for every change. Put larger datasets and
135
- drift-prone live cases behind an explicit selection. See
136
- [Datasets and fixtures](/docs/reference/evals.md#datasets-and-fixtures) for
137
- loaders and case expansion.
101
+ [Datasets and fixtures](/docs/reference/evals.md#datasets-and-fixtures) covers loaders for larger sets.
138
102
 
139
103
  ## Run and debug a regression suite
140
104
 
141
- Run one case while iterating, then the smoke suite before review. A
142
- failure artifact records the input, assertions, tool calls, and final
143
- text, making it the first place to look before changing the prompt.
105
+ Run one case while you iterate, then the smoke set before review:
144
106
 
145
- ```text
107
+ ```bash
146
108
  agent-sdk eval --dir . readiness
147
109
  agent-sdk eval --dir . --tag smoke
148
- agent-sdk eval --dir . --json
149
110
  ```
150
111
 
151
- By default, results land under `evals/<stamp>/` in the project state
152
- directory. Open `evals/<case-id>.json` inside that run directory for the
153
- failed case. This artifact location is independent of `--state-root`;
154
- pass `--artifacts <dir>` to choose another destination.
155
-
156
- Model turns need a credential. Resolution checks `CURSOR_API_KEY`,
157
- `CURSOR_API_KEY_FILE`, `CURSOR_SERVICE_ACCOUNT_KEY`, then a saved
158
- `agent-sdk login`. `eval --list` only discovers cases and needs none.
112
+ Each run writes a folder under `evals/` in the project state directory, or under `--artifacts <dir>`
113
+ if you pass it. Open `evals/<case-id>.json` in that folder to see a failed case's input, assertions,
114
+ tool calls, and final text. Model turns need a
115
+ [Cursor credential](/docs/reference/cli.md#environment-variables).
159
116
 
160
117
  ## Run evals in CI
161
118
 
162
- Write machine-readable output and JUnit annotations, then let a failed
163
- gate fail the job:
164
-
165
- ```text
119
+ ```bash
166
120
  # CURSOR_API_KEY comes from the CI secret store.
167
121
  agent-sdk eval --dir . --json --no-stream \
168
122
  --junit reports/evals.xml \
@@ -170,70 +124,22 @@ agent-sdk eval --dir . --json --no-stream \
170
124
  > reports/evals.json
171
125
  ```
172
126
 
173
- Run `--tag smoke` on pull requests and the full suite on a schedule when
174
- cost or latency makes every-push coverage impractical. Use soft scores
175
- for a new probabilistic benchmark until its threshold is trustworthy.
176
-
177
- The [CLI reference](/docs/reference/cli.md#eval) covers selectors,
178
- server-side batches, exit codes, and JSON output.
127
+ Run `--tag smoke` on pull requests and the full suite on a schedule. The
128
+ [CLI reference](/docs/reference/cli.md#eval) covers every flag.
179
129
 
180
130
  ## Keep side effects out of eval sessions
181
131
 
182
- Eval sessions carry `purpose: "eval"`. Check it at the deterministic
183
- write boundary and return the decision without performing the external
184
- action:
132
+ Eval sessions carry `purpose: "eval"`. Check it where the tool writes, and return the decision
133
+ without the external action:
185
134
 
186
135
  ```ts
187
136
  async execute({ findings }, ctx) {
188
137
  if (ctx.session.purpose === "eval") {
189
- return {
190
- posted: false,
191
- reason: "eval",
192
- count: findings.length,
193
- };
138
+ return { posted: false, reason: "eval", count: findings.length };
194
139
  }
195
140
 
196
141
  return await postFindings(findings);
197
142
  }
198
143
  ```
199
144
 
200
- This preserves the tool call in the trajectory, so the eval can still
201
- assert that the agent chose to post.
202
-
203
- ## Evals, hooks, or hillclimbing?
204
-
205
- Use an eval to gate one fixed input. Use a
206
- [hook](/docs/reference/hooks.md) to observe every live session for metrics,
207
- auditing, or alerts. Use [hillclimbing](/docs/hillclimbing.md) to decide
208
- which source change improves a fixed set of inputs; every kept
209
- hillclimb change should add an eval.
210
-
211
- ## Gates and scores
212
-
213
- `t.succeeded()`, tool assertions, and deterministic checks are hard
214
- gates by default. A failed gate fails the case. `.soft()` records a
215
- measurement without blocking, while `.atLeast(threshold)` records a
216
- score and marks a miss as `scored`.
217
-
218
- Start with decisions and output shape: the intended tool, the tempting
219
- wrong tool, and one structural check. Exact prose is rarely a stable
220
- contract.
221
-
222
- ## Keep improvements with regression evals
223
-
224
- Every kept improvement needs an eval that would have failed before the
225
- change. A tool-choice gate, bounded `maxToolCalls`, or output-shape check
226
- turns the improvement into a durable contract.
227
-
228
- Never weaken an existing gate to make a new implementation pass. That
229
- moves the freeze line instead of proving the change.
230
-
231
- ## Related
232
-
233
- - [Evals reference](/docs/reference/evals.md): complete authoring and runner
234
- contracts
235
- - [`skills/evals/SKILL.md`](https://github.com/cursor/cursor/blob/main/packages/agent-serve/skills/evals/SKILL.md): author and seed
236
- cases with a coding agent
237
- - [Hillclimbing](/docs/hillclimbing.md): measure a change and lock the win
238
- - [Sessions](/docs/reference/sessions.md): events available to trajectory
239
- assertions
145
+ The tool call stays in the trajectory, so the eval can still assert that the agent chose to post.