@grok/agent-sdk 0.2.29 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (451) hide show
  1. package/dist/channels/slack/cli-project.js +1 -1
  2. package/dist/channels/slack/slack-channel.d.ts.map +1 -1
  3. package/dist/channels/slack/slack-channel.js +27 -14
  4. package/dist/docs/404.html +2 -2
  5. package/dist/docs/assets/{app.DMrZKFjI.js → app.BSH__xV4.js} +4 -4
  6. package/dist/docs/assets/building-with-agents.md.Bo3ct-rf.js +6 -0
  7. package/dist/docs/assets/{building-with-agents.md.Bf1yH2tG.lean.js → building-with-agents.md.Bo3ct-rf.lean.js} +1 -1
  8. package/dist/docs/assets/changelog.md.b6H_ea44.js +1 -0
  9. package/dist/docs/assets/changelog.md.b6H_ea44.lean.js +1 -0
  10. package/dist/docs/assets/chunks/@localSearchIndexroot.CiRGcfZg.js +1 -0
  11. package/dist/docs/assets/chunks/{VPLocalSearchBox.DQvuw1KA.js → VPLocalSearchBox.Cadl7A0k.js} +1 -1
  12. package/dist/docs/assets/chunks/{arc.ahOG-Qza.js → arc.K7t9nNY6.js} +1 -1
  13. package/dist/docs/assets/chunks/{architectureDiagram-Q4EWVU46.BcgfYNO_.js → architectureDiagram-Q4EWVU46.SlHEknkX.js} +1 -1
  14. package/dist/docs/assets/chunks/{baseUniq.DeIYZrWh.js → baseUniq.DJLRMLkz.js} +1 -1
  15. package/dist/docs/assets/chunks/{blockDiagram-DXYQGD6D.HjJ2goMX.js → blockDiagram-DXYQGD6D.Bs6XxFpe.js} +1 -1
  16. package/dist/docs/assets/chunks/{c4Diagram-AHTNJAMY.C1QTb9GM.js → c4Diagram-AHTNJAMY.CyLg6Ovc.js} +1 -1
  17. package/dist/docs/assets/chunks/channel.CVTdqYx0.js +1 -0
  18. package/dist/docs/assets/chunks/{chunk-4BX2VUAB.UuZiDiVp.js → chunk-4BX2VUAB.DTQ4TinT.js} +1 -1
  19. package/dist/docs/assets/chunks/{chunk-4TB4RGXK.5A3YmkhI.js → chunk-4TB4RGXK.Es84IgcH.js} +1 -1
  20. package/dist/docs/assets/chunks/{chunk-55IACEB6.B4uf1yYV.js → chunk-55IACEB6.Clm3cGPQ.js} +1 -1
  21. package/dist/docs/assets/chunks/{chunk-EDXVE4YY.CoXBce8t.js → chunk-EDXVE4YY.DE4CRkS5.js} +1 -1
  22. package/dist/docs/assets/chunks/{chunk-FMBD7UC4.DnC5NO4e.js → chunk-FMBD7UC4.Bf8HCvqp.js} +1 -1
  23. package/dist/docs/assets/chunks/{chunk-OYMX7WX6.CzfHCeIi.js → chunk-OYMX7WX6.eub-7nSx.js} +1 -1
  24. package/dist/docs/assets/chunks/{chunk-QZHKN3VN.C6tcwkVd.js → chunk-QZHKN3VN.WyP9VAS_.js} +1 -1
  25. package/dist/docs/assets/chunks/{chunk-YZCP3GAM._oboyZ1A.js → chunk-YZCP3GAM.LO9ohhDE.js} +1 -1
  26. package/dist/docs/assets/chunks/classDiagram-6PBFFD2Q.CsJJyx6-.js +1 -0
  27. package/dist/docs/assets/chunks/classDiagram-v2-HSJHXN6E.CsJJyx6-.js +1 -0
  28. package/dist/docs/assets/chunks/clone.Dk8KvtOB.js +1 -0
  29. package/dist/docs/assets/chunks/{cose-bilkent-S5V4N54A.BfPVKeId.js → cose-bilkent-S5V4N54A.D5za6aH0.js} +1 -1
  30. package/dist/docs/assets/chunks/{dagre-KV5264BT.CLMLc0lD.js → dagre-KV5264BT.Dq1daUru.js} +1 -1
  31. package/dist/docs/assets/chunks/{diagram-5BDNPKRD.C7OF7UfH.js → diagram-5BDNPKRD.j1YL5BKS.js} +1 -1
  32. package/dist/docs/assets/chunks/{diagram-G4DWMVQ6.Dpao0Bl1.js → diagram-G4DWMVQ6.Bi1aLdhC.js} +1 -1
  33. package/dist/docs/assets/chunks/{diagram-MMDJMWI5.DLnM9eLp.js → diagram-MMDJMWI5.BOzMEkAr.js} +1 -1
  34. package/dist/docs/assets/chunks/{diagram-TYMM5635.BcpVgA_M.js → diagram-TYMM5635.Dx2FVhCr.js} +1 -1
  35. package/dist/docs/assets/chunks/{erDiagram-SMLLAGMA.CsL5JuCV.js → erDiagram-SMLLAGMA.DI4TNkye.js} +1 -1
  36. package/dist/docs/assets/chunks/{flowDiagram-DWJPFMVM.MFihq9AF.js → flowDiagram-DWJPFMVM.C58Z9uPK.js} +1 -1
  37. package/dist/docs/assets/chunks/{ganttDiagram-T4ZO3ILL.CJeLodyG.js → ganttDiagram-T4ZO3ILL.DXSpW809.js} +1 -1
  38. package/dist/docs/assets/chunks/{gitGraphDiagram-UUTBAWPF.1HhlE45Z.js → gitGraphDiagram-UUTBAWPF.B955oaFl.js} +1 -1
  39. package/dist/docs/assets/chunks/{graph.BS02KkWA.js → graph.BQXhKtxS.js} +1 -1
  40. package/dist/docs/assets/chunks/{infoDiagram-42DDH7IO.fp5gvytQ.js → infoDiagram-42DDH7IO.Dozj_-Gm.js} +1 -1
  41. package/dist/docs/assets/chunks/{ishikawaDiagram-UXIWVN3A.C6Wk1V4I.js → ishikawaDiagram-UXIWVN3A.D2140OzH.js} +1 -1
  42. package/dist/docs/assets/chunks/{journeyDiagram-VCZTEJTY.qJt4uJ_H.js → journeyDiagram-VCZTEJTY.DrqrMhCk.js} +1 -1
  43. package/dist/docs/assets/chunks/{kanban-definition-6JOO6SKY.z2tG9kly.js → kanban-definition-6JOO6SKY.Dv-LTrlw.js} +1 -1
  44. package/dist/docs/assets/chunks/{layout.BPCkdejr.js → layout.CGtHp49X.js} +1 -1
  45. package/dist/docs/assets/chunks/{linear.CL3TC3u7.js → linear.CHZLYB52.js} +1 -1
  46. package/dist/docs/assets/chunks/{min.SPrSp704.js → min.D4Vt3eIy.js} +1 -1
  47. package/dist/docs/assets/chunks/{mindmap-definition-QFDTVHPH.DgrQRFZW.js → mindmap-definition-QFDTVHPH.BBfpYTea.js} +1 -1
  48. package/dist/docs/assets/chunks/{pieDiagram-DEJITSTG.Ba7yxiW2.js → pieDiagram-DEJITSTG.BVYvElmY.js} +1 -1
  49. package/dist/docs/assets/chunks/{quadrantDiagram-34T5L4WZ.Bvskad4p.js → quadrantDiagram-34T5L4WZ.C6Gc2mdY.js} +1 -1
  50. package/dist/docs/assets/chunks/{requirementDiagram-MS252O5E.Ce70ugV9.js → requirementDiagram-MS252O5E.DWa8b4cn.js} +1 -1
  51. package/dist/docs/assets/chunks/{sankeyDiagram-XADWPNL6.BJ5OEULO.js → sankeyDiagram-XADWPNL6.Bi-g4ajv.js} +1 -1
  52. package/dist/docs/assets/chunks/{sequenceDiagram-FGHM5R23.C8a1hDs6.js → sequenceDiagram-FGHM5R23.DlU3vVN9.js} +1 -1
  53. package/dist/docs/assets/chunks/{stateDiagram-FHFEXIEX.D2D8A5CK.js → stateDiagram-FHFEXIEX.CeqJRqnU.js} +1 -1
  54. package/dist/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.DE0BnUZh.js +1 -0
  55. package/dist/docs/assets/chunks/{theme.DRciHmS1.js → theme.DGq7D4tt.js} +2 -2
  56. package/dist/docs/assets/chunks/{timeline-definition-GMOUNBTQ.C4ePrY8-.js → timeline-definition-GMOUNBTQ.Cqf2kjq9.js} +1 -1
  57. package/dist/docs/assets/chunks/{vennDiagram-DHZGUBPP.Dzso6dEA.js → vennDiagram-DHZGUBPP.BeOpE_lM.js} +1 -1
  58. package/dist/docs/assets/chunks/{wardley-RL74JXVD.djgXR0ar.js → wardley-RL74JXVD.FMQXA-6q.js} +1 -1
  59. package/dist/docs/assets/chunks/{wardleyDiagram-NUSXRM2D.Bf2ICxjh.js → wardleyDiagram-NUSXRM2D.BXS65dAC.js} +1 -1
  60. package/dist/docs/assets/chunks/{xychartDiagram-5P7HB3ND.DEH4BcVJ.js → xychartDiagram-5P7HB3ND.haEiSxqc.js} +1 -1
  61. package/dist/docs/assets/deployment.md.DJtuDVLh.js +31 -0
  62. package/dist/docs/assets/{deployment.md.KW0Ni_rY.lean.js → deployment.md.DJtuDVLh.lean.js} +1 -1
  63. package/dist/docs/assets/evals.md.Dne6VRmZ.js +49 -0
  64. package/dist/docs/assets/evals.md.Dne6VRmZ.lean.js +1 -0
  65. package/dist/docs/assets/{guides_agent-to-agent.md.D1D5f6Jc.js → guides_agent-to-agent.md.DcB3nhOz.js} +8 -8
  66. package/dist/docs/assets/guides_agent-to-agent.md.DcB3nhOz.lean.js +1 -0
  67. package/dist/docs/assets/{guides_bitbucket.md.DkjkBRU8.js → guides_bitbucket.md.BiA6ZMFW.js} +27 -27
  68. package/dist/docs/assets/{guides_bitbucket.md.DkjkBRU8.lean.js → guides_bitbucket.md.BiA6ZMFW.lean.js} +1 -1
  69. package/dist/docs/assets/guides_cloud-agents.md.CiXAW6on.js +9 -0
  70. package/dist/docs/assets/{guides_cloud-agents.md.Ceyjv0ul.lean.js → guides_cloud-agents.md.CiXAW6on.lean.js} +1 -1
  71. package/dist/docs/assets/guides_convert-automation.md.BvTxsJ8J.js +26 -0
  72. package/dist/docs/assets/{guides_convert-automation.md.BjraCvok.lean.js → guides_convert-automation.md.BvTxsJ8J.lean.js} +1 -1
  73. package/dist/docs/assets/{guides_github.md.CgN5sNbW.js → guides_github.md.CcuZ7keS.js} +11 -19
  74. package/dist/docs/assets/{guides_github.md.CgN5sNbW.lean.js → guides_github.md.CcuZ7keS.lean.js} +1 -1
  75. package/dist/docs/assets/{guides_gitlab.md.DCE0p7_w.js → guides_gitlab.md.DcKusOcU.js} +63 -73
  76. package/dist/docs/assets/{guides_gitlab.md.DCE0p7_w.lean.js → guides_gitlab.md.DcKusOcU.lean.js} +1 -1
  77. package/dist/docs/assets/guides_improve.md.o3WOGR7n.js +16 -0
  78. package/dist/docs/assets/{guides_improve.md.BFYtrtLE.lean.js → guides_improve.md.o3WOGR7n.lean.js} +1 -1
  79. package/dist/docs/assets/{guides_jev.md.CEqLgjj9.js → guides_jev.md.DGCzkMUN.js} +20 -60
  80. package/dist/docs/assets/{guides_jev.md.CEqLgjj9.lean.js → guides_jev.md.DGCzkMUN.lean.js} +1 -1
  81. package/dist/docs/assets/{guides_mcp-oauth.md.DrGY92u4.js → guides_mcp-oauth.md.GVPMKcYl.js} +3 -5
  82. package/dist/docs/assets/{guides_mcp-oauth.md.DrGY92u4.lean.js → guides_mcp-oauth.md.GVPMKcYl.lean.js} +1 -1
  83. package/dist/docs/assets/guides_migrate.md.B8CZxnnX.js +3 -0
  84. package/dist/docs/assets/{guides_migrate.md.RS9Eu-Xh.lean.js → guides_migrate.md.B8CZxnnX.lean.js} +1 -1
  85. package/dist/docs/assets/guides_opentelemetry.md.BRSktBHt.js +26 -0
  86. package/dist/docs/assets/guides_opentelemetry.md.BRSktBHt.lean.js +1 -0
  87. package/dist/docs/assets/guides_pagerduty.md.pMXUkZZc.js +61 -0
  88. package/dist/docs/assets/guides_pagerduty.md.pMXUkZZc.lean.js +1 -0
  89. package/dist/docs/assets/guides_slack.md.D7Rwyva4.js +82 -0
  90. package/dist/docs/assets/{guides_slack.md.BwXHURvY.lean.js → guides_slack.md.D7Rwyva4.lean.js} +1 -1
  91. package/dist/docs/assets/{guides_webhooks.md.BJk63ZTs.js → guides_webhooks.md.B5DXt2v1.js} +7 -7
  92. package/dist/docs/assets/{guides_webhooks.md.BJk63ZTs.lean.js → guides_webhooks.md.B5DXt2v1.lean.js} +1 -1
  93. package/dist/docs/assets/hillclimbing.md.M51bF-oT.js +5 -0
  94. package/dist/docs/assets/hillclimbing.md.M51bF-oT.lean.js +1 -0
  95. package/dist/docs/assets/index.md.DnJPj_B6.js +14 -0
  96. package/dist/docs/assets/index.md.DnJPj_B6.lean.js +1 -0
  97. package/dist/docs/assets/quickstart.md.r-oE8nkX.js +113 -0
  98. package/dist/docs/assets/{quickstart.md.CljDaJBi.lean.js → quickstart.md.r-oE8nkX.lean.js} +1 -1
  99. package/dist/docs/assets/reference_access.md.C5lblfUC.js +1 -0
  100. package/dist/docs/assets/reference_access.md.C5lblfUC.lean.js +1 -0
  101. package/dist/docs/assets/reference_agent-config.md.BSNp2uJ7.js +27 -0
  102. package/dist/docs/assets/{reference_agent-config.md.Ck29qYNj.lean.js → reference_agent-config.md.BSNp2uJ7.lean.js} +1 -1
  103. package/dist/docs/assets/reference_artifacts.md.BJpAI7h9.js +17 -0
  104. package/dist/docs/assets/{reference_artifacts.md.C-hKR2hp.lean.js → reference_artifacts.md.BJpAI7h9.lean.js} +1 -1
  105. package/dist/docs/assets/reference_channels.md.CPWOpdsC.js +21 -0
  106. package/dist/docs/assets/{reference_channels.md.DsJfGSBJ.lean.js → reference_channels.md.CPWOpdsC.lean.js} +1 -1
  107. package/dist/docs/assets/reference_cli.md.DHm6DmYk.js +83 -0
  108. package/dist/docs/assets/reference_cli.md.DHm6DmYk.lean.js +1 -0
  109. package/dist/docs/assets/reference_connections.md.DoY4vrDH.js +54 -0
  110. package/dist/docs/assets/{reference_connections.md.BoAotr7_.lean.js → reference_connections.md.DoY4vrDH.lean.js} +1 -1
  111. package/dist/docs/assets/reference_evals.md.DgfUtfIc.js +69 -0
  112. package/dist/docs/assets/{reference_evals.md.2BXeMiO2.lean.js → reference_evals.md.DgfUtfIc.lean.js} +1 -1
  113. package/dist/docs/assets/{reference_extensions.md.DI5X5x8p.js → reference_extensions.md.DHbW6eOf.js} +6 -8
  114. package/dist/docs/assets/{reference_extensions.md.DI5X5x8p.lean.js → reference_extensions.md.DHbW6eOf.lean.js} +1 -1
  115. package/dist/docs/assets/reference_hooks.md.vbIM394-.js +39 -0
  116. package/dist/docs/assets/reference_hooks.md.vbIM394-.lean.js +1 -0
  117. package/dist/docs/assets/reference_http-api.md.C7Cb0r2J.js +7 -0
  118. package/dist/docs/assets/{reference_http-api.md._RWELr3d.lean.js → reference_http-api.md.C7Cb0r2J.lean.js} +1 -1
  119. package/dist/docs/assets/reference_instructions.md.CV6Qq0UX.js +14 -0
  120. package/dist/docs/assets/reference_instructions.md.CV6Qq0UX.lean.js +1 -0
  121. package/dist/docs/assets/reference_limits.md.CbdtFr4T.js +8 -0
  122. package/dist/docs/assets/reference_limits.md.CbdtFr4T.lean.js +1 -0
  123. package/dist/docs/assets/reference_memory.md.CdoJISM-.js +3 -0
  124. package/dist/docs/assets/{reference_memory.md.Dd7cyV9V.lean.js → reference_memory.md.CdoJISM-.lean.js} +1 -1
  125. package/dist/docs/assets/reference_playground.md.DIUsRAcy.js +1 -0
  126. package/dist/docs/assets/{reference_playground.md.CQiJk9CM.lean.js → reference_playground.md.DIUsRAcy.lean.js} +1 -1
  127. package/dist/docs/assets/reference_project-layout.md.9DiKoE16.js +26 -0
  128. package/dist/docs/assets/reference_project-layout.md.9DiKoE16.lean.js +1 -0
  129. package/dist/docs/assets/reference_schedules.md.Dd8_d0WG.js +45 -0
  130. package/dist/docs/assets/{reference_schedules.md.RC47aFm4.lean.js → reference_schedules.md.Dd8_d0WG.lean.js} +1 -1
  131. package/dist/docs/assets/reference_sessions.md.NXVQt9mP.js +22 -0
  132. package/dist/docs/assets/{reference_sessions.md.C7KFhWXV.lean.js → reference_sessions.md.NXVQt9mP.lean.js} +1 -1
  133. package/dist/docs/assets/{reference_skills.md.Dej2zQxi.js → reference_skills.md.DzCPNPkG.js} +4 -4
  134. package/dist/docs/assets/{reference_skills.md.Dej2zQxi.lean.js → reference_skills.md.DzCPNPkG.lean.js} +1 -1
  135. package/dist/docs/assets/reference_subagents.md.Dvpq2sUu.js +10 -0
  136. package/dist/docs/assets/{reference_subagents.md.CAhvW4yS.lean.js → reference_subagents.md.Dvpq2sUu.lean.js} +1 -1
  137. package/dist/docs/assets/reference_tools.md.B2OaR1Nb.js +62 -0
  138. package/dist/docs/assets/{reference_tools.md.DQr7Fnhu.lean.js → reference_tools.md.B2OaR1Nb.lean.js} +1 -1
  139. package/dist/docs/assets/templates_agentic-owners.md.B6ZmKAzT.js +13 -0
  140. package/dist/docs/assets/{templates_agentic-owners.md.Dh2nDSqg.lean.js → templates_agentic-owners.md.B6ZmKAzT.lean.js} +1 -1
  141. package/dist/docs/assets/templates_reviews.md.dr9Xv_KM.js +10 -0
  142. package/dist/docs/assets/templates_reviews.md.dr9Xv_KM.lean.js +1 -0
  143. package/dist/docs/assets/troubleshooting.md.DfKKvW1R.js +3 -0
  144. package/dist/docs/assets/{troubleshooting.md.CGOdNVAL.lean.js → troubleshooting.md.DfKKvW1R.lean.js} +1 -1
  145. package/dist/docs/building-with-agents.html +41 -44
  146. package/dist/docs/building-with-agents.md +11 -104
  147. package/dist/docs/changelog.html +37 -37
  148. package/dist/docs/changelog.md +14 -145
  149. package/dist/docs/deployment.html +54 -54
  150. package/dist/docs/deployment.md +97 -270
  151. package/dist/docs/evals.html +52 -71
  152. package/dist/docs/evals.md +33 -129
  153. package/dist/docs/guides/agent-to-agent.html +43 -43
  154. package/dist/docs/guides/agent-to-agent.md +29 -64
  155. package/dist/docs/guides/bitbucket.html +62 -62
  156. package/dist/docs/guides/bitbucket.md +73 -134
  157. package/dist/docs/guides/cloud-agents.html +37 -37
  158. package/dist/docs/guides/cloud-agents.md +23 -53
  159. package/dist/docs/guides/convert-automation.html +39 -56
  160. package/dist/docs/guides/convert-automation.md +31 -115
  161. package/dist/docs/guides/github.html +46 -54
  162. package/dist/docs/guides/github.md +49 -131
  163. package/dist/docs/guides/gitlab.html +98 -108
  164. package/dist/docs/guides/gitlab.md +104 -184
  165. package/dist/docs/guides/improve.html +38 -43
  166. package/dist/docs/guides/improve.md +24 -90
  167. package/dist/docs/guides/jev.html +55 -95
  168. package/dist/docs/guides/jev.md +53 -160
  169. package/dist/docs/guides/mcp-oauth.html +38 -40
  170. package/dist/docs/guides/mcp-oauth.md +33 -67
  171. package/dist/docs/guides/migrate.html +38 -43
  172. package/dist/docs/guides/migrate.md +56 -130
  173. package/dist/docs/guides/opentelemetry.html +41 -43
  174. package/dist/docs/guides/opentelemetry.md +32 -27
  175. package/dist/docs/guides/pagerduty.html +50 -49
  176. package/dist/docs/guides/pagerduty.md +65 -199
  177. package/dist/docs/guides/slack.html +51 -57
  178. package/dist/docs/guides/slack.md +76 -204
  179. package/dist/docs/guides/webhooks.html +42 -42
  180. package/dist/docs/guides/webhooks.md +64 -146
  181. package/dist/docs/hashmap.json +1 -1
  182. package/dist/docs/hillclimbing.html +40 -42
  183. package/dist/docs/hillclimbing.md +40 -102
  184. package/dist/docs/index.html +49 -61
  185. package/dist/docs/index.md +42 -82
  186. package/dist/docs/llms-full.txt +2272 -5711
  187. package/dist/docs/llms.txt +9 -14
  188. package/dist/docs/quickstart.html +95 -152
  189. package/dist/docs/quickstart.md +91 -180
  190. package/dist/docs/reference/access.html +36 -36
  191. package/dist/docs/reference/access.md +7 -12
  192. package/dist/docs/reference/agent-config.html +45 -58
  193. package/dist/docs/reference/agent-config.md +92 -176
  194. package/dist/docs/reference/artifacts.html +38 -39
  195. package/dist/docs/reference/artifacts.md +38 -62
  196. package/dist/docs/reference/channels.html +38 -39
  197. package/dist/docs/reference/channels.md +102 -151
  198. package/dist/docs/reference/cli.html +68 -84
  199. package/dist/docs/reference/cli.md +232 -697
  200. package/dist/docs/reference/connections.html +45 -57
  201. package/dist/docs/reference/connections.md +83 -164
  202. package/dist/docs/reference/evals.html +47 -54
  203. package/dist/docs/reference/evals.md +75 -141
  204. package/dist/docs/reference/extensions.html +41 -43
  205. package/dist/docs/reference/extensions.md +37 -70
  206. package/dist/docs/reference/hooks.html +40 -52
  207. package/dist/docs/reference/hooks.md +62 -133
  208. package/dist/docs/reference/http-api.html +38 -42
  209. package/dist/docs/reference/http-api.md +66 -118
  210. package/dist/docs/reference/instructions.html +39 -39
  211. package/dist/docs/reference/instructions.md +19 -29
  212. package/dist/docs/reference/limits.html +38 -38
  213. package/dist/docs/reference/limits.md +9 -21
  214. package/dist/docs/reference/memory.html +37 -38
  215. package/dist/docs/reference/memory.md +38 -75
  216. package/dist/docs/reference/playground.html +36 -36
  217. package/dist/docs/reference/playground.md +14 -22
  218. package/dist/docs/reference/project-layout.html +49 -48
  219. package/dist/docs/reference/project-layout.md +38 -84
  220. package/dist/docs/reference/schedules.html +44 -46
  221. package/dist/docs/reference/schedules.md +59 -90
  222. package/dist/docs/reference/sessions.html +37 -37
  223. package/dist/docs/reference/sessions.md +66 -113
  224. package/dist/docs/reference/skills.html +39 -39
  225. package/dist/docs/reference/skills.md +11 -31
  226. package/dist/docs/reference/subagents.html +37 -37
  227. package/dist/docs/reference/subagents.md +17 -31
  228. package/dist/docs/reference/tools.html +47 -64
  229. package/dist/docs/reference/tools.md +48 -130
  230. package/dist/docs/templates/agentic-owners.html +38 -38
  231. package/dist/docs/templates/agentic-owners.md +50 -74
  232. package/dist/docs/templates/reviews.html +69 -0
  233. package/dist/docs/templates/reviews.md +71 -0
  234. package/dist/docs/troubleshooting.html +38 -36
  235. package/dist/docs/troubleshooting.md +48 -101
  236. package/dist/filesystem.d.ts +7 -320
  237. package/dist/filesystem.d.ts.map +1 -1
  238. package/dist/filesystem.js +10 -691
  239. package/dist/index.d.ts +1 -1
  240. package/dist/index.d.ts.map +1 -1
  241. package/dist/index.js +1 -1
  242. package/dist/internal/authored-alias-hooks.d.ts +6 -0
  243. package/dist/internal/authored-alias-hooks.d.ts.map +1 -1
  244. package/dist/internal/authored-alias-hooks.js +12 -0
  245. package/dist/internal/authored-loaders.js +10 -1
  246. package/dist/internal/cli/commands/scaffold.js +1 -1
  247. package/dist/internal/cli-deploy.js +1 -1
  248. package/dist/internal/cli-docs.js +1 -1
  249. package/dist/internal/cli-images.js +1 -1
  250. package/dist/internal/cli-skills.js +1 -1
  251. package/dist/internal/continuation-session-ids.d.ts +15 -0
  252. package/dist/internal/continuation-session-ids.d.ts.map +1 -0
  253. package/dist/internal/continuation-session-ids.js +43 -0
  254. package/dist/internal/cursor/credentials.js +1 -1
  255. package/dist/internal/deploy-manifest.d.ts +1 -0
  256. package/dist/internal/deploy-manifest.d.ts.map +1 -1
  257. package/dist/internal/deploy-manifest.js +0 -1
  258. package/dist/internal/discovery/agent-config.d.ts +2 -6
  259. package/dist/internal/discovery/agent-config.d.ts.map +1 -1
  260. package/dist/internal/discovery/agent-config.js +8 -25
  261. package/dist/internal/discovery/agent.d.ts.map +1 -1
  262. package/dist/internal/discovery/agent.js +2 -3
  263. package/dist/internal/discovery/project.d.ts.map +1 -1
  264. package/dist/internal/discovery/project.js +15 -0
  265. package/dist/internal/filesystem/walk.d.ts +1 -5
  266. package/dist/internal/filesystem/walk.d.ts.map +1 -1
  267. package/dist/internal/filesystem/walk.js +1 -9
  268. package/dist/internal/hosted-catch-protocol.d.ts +3 -3
  269. package/dist/internal/hosted-catch-protocol.js +3 -3
  270. package/dist/internal/hosted-fetch.d.ts +7 -0
  271. package/dist/internal/hosted-fetch.d.ts.map +1 -0
  272. package/dist/internal/hosted-fetch.js +26 -0
  273. package/dist/internal/init-project.d.ts.map +1 -1
  274. package/dist/internal/init-project.js +7 -5
  275. package/dist/internal/install-cursor-skills.d.ts +5 -6
  276. package/dist/internal/install-cursor-skills.d.ts.map +1 -1
  277. package/dist/internal/install-cursor-skills.js +19 -4
  278. package/dist/internal/mcp-endpoint.js +4 -4
  279. package/dist/internal/model-config.d.ts.map +1 -1
  280. package/dist/internal/model-config.js +11 -4
  281. package/dist/internal/platform-timers.d.ts +3 -7
  282. package/dist/internal/platform-timers.d.ts.map +1 -1
  283. package/dist/internal/platform-timers.js +10 -51
  284. package/dist/internal/reminder-runner.js +1 -1
  285. package/dist/internal/server.d.ts.map +1 -1
  286. package/dist/internal/server.js +6 -1
  287. package/dist/internal/session-engine.d.ts +3 -0
  288. package/dist/internal/session-engine.d.ts.map +1 -1
  289. package/dist/internal/session-engine.js +13 -4
  290. package/dist/internal/shallow-clone.js +1 -1
  291. package/dist/playground/assets/index-Bno25Cg0.css +1 -0
  292. package/dist/playground/assets/index-SGKsaznh.js +62 -0
  293. package/dist/playground/index.html +2 -2
  294. package/dist/types.d.ts +6 -17
  295. package/dist/types.d.ts.map +1 -1
  296. package/dist/types.js +2 -2
  297. package/docs/README.md +42 -82
  298. package/docs/building-with-agents.md +11 -104
  299. package/docs/changelog.md +15 -146
  300. package/docs/deployment.md +97 -270
  301. package/docs/evals.md +34 -130
  302. package/docs/guides/agent-to-agent.md +29 -64
  303. package/docs/guides/bitbucket.md +73 -134
  304. package/docs/guides/cloud-agents.md +23 -53
  305. package/docs/guides/convert-automation.md +31 -115
  306. package/docs/guides/github.md +49 -131
  307. package/docs/guides/gitlab.md +104 -184
  308. package/docs/guides/improve.md +24 -90
  309. package/docs/guides/jev.md +53 -160
  310. package/docs/guides/mcp-oauth.md +33 -67
  311. package/docs/guides/migrate.md +56 -130
  312. package/docs/guides/opentelemetry.md +37 -27
  313. package/docs/guides/pagerduty.md +68 -200
  314. package/docs/guides/slack.md +76 -204
  315. package/docs/guides/webhooks.md +64 -146
  316. package/docs/hillclimbing.md +41 -103
  317. package/docs/quickstart.md +91 -180
  318. package/docs/reference/access.md +7 -12
  319. package/docs/reference/agent-config.md +92 -176
  320. package/docs/reference/artifacts.md +38 -62
  321. package/docs/reference/channels.md +102 -151
  322. package/docs/reference/cli.md +232 -697
  323. package/docs/reference/connections.md +83 -164
  324. package/docs/reference/evals.md +75 -141
  325. package/docs/reference/extensions.md +37 -70
  326. package/docs/reference/hooks.md +61 -132
  327. package/docs/reference/http-api.md +66 -118
  328. package/docs/reference/instructions.md +20 -30
  329. package/docs/reference/limits.md +10 -22
  330. package/docs/reference/memory.md +38 -75
  331. package/docs/reference/playground.md +14 -22
  332. package/docs/reference/project-layout.md +38 -84
  333. package/docs/reference/schedules.md +59 -90
  334. package/docs/reference/sessions.md +66 -113
  335. package/docs/reference/skills.md +12 -32
  336. package/docs/reference/subagents.md +17 -31
  337. package/docs/reference/tools.md +48 -130
  338. package/docs/templates/agentic-owners.md +50 -74
  339. package/docs/templates/reviews.md +76 -0
  340. package/docs/troubleshooting.md +48 -101
  341. package/package.json +4 -4
  342. package/skills/cli/SKILL.md +1 -1
  343. package/skills/create-agent/SKILL.md +61 -7
  344. package/skills/debug/SKILL.md +1 -1
  345. package/skills/deploy/SKILL.md +3 -3
  346. package/skills/evals/SKILL.md +1 -1
  347. package/skills/framework-map/SKILL.md +2 -2
  348. package/skills/github/SKILL.md +1 -1
  349. package/skills/hillclimb/SKILL.md +1 -1
  350. package/skills/mcp-auth/SKILL.md +1 -1
  351. package/skills/migrate-v2/SKILL.md +2 -3
  352. package/skills/otel/SKILL.md +1 -1
  353. package/skills/setup-slack/SKILL.md +2 -2
  354. package/dist/docs/assets/building-with-agents.md.Bf1yH2tG.js +0 -9
  355. package/dist/docs/assets/changelog.md.DrLFQUlu.js +0 -1
  356. package/dist/docs/assets/changelog.md.DrLFQUlu.lean.js +0 -1
  357. package/dist/docs/assets/chunks/@localSearchIndexroot.Dpd_wxVU.js +0 -1
  358. package/dist/docs/assets/chunks/channel._fZX_hvU.js +0 -1
  359. package/dist/docs/assets/chunks/classDiagram-6PBFFD2Q.eClS1Q0E.js +0 -1
  360. package/dist/docs/assets/chunks/classDiagram-v2-HSJHXN6E.eClS1Q0E.js +0 -1
  361. package/dist/docs/assets/chunks/clone.W1Phdw9P.js +0 -1
  362. package/dist/docs/assets/chunks/stateDiagram-v2-QKLJ7IA2.77yf4kRr.js +0 -1
  363. package/dist/docs/assets/deployment.md.KW0Ni_rY.js +0 -31
  364. package/dist/docs/assets/evals.md.DiljU__Z.js +0 -68
  365. package/dist/docs/assets/evals.md.DiljU__Z.lean.js +0 -1
  366. package/dist/docs/assets/guides_agent-to-agent.md.D1D5f6Jc.lean.js +0 -1
  367. package/dist/docs/assets/guides_cloud-agents.md.Ceyjv0ul.js +0 -9
  368. package/dist/docs/assets/guides_convert-automation.md.BjraCvok.js +0 -43
  369. package/dist/docs/assets/guides_hooks.md.BjeEiM7s.js +0 -54
  370. package/dist/docs/assets/guides_hooks.md.BjeEiM7s.lean.js +0 -1
  371. package/dist/docs/assets/guides_improve.md.BFYtrtLE.js +0 -21
  372. package/dist/docs/assets/guides_migrate.md.RS9Eu-Xh.js +0 -8
  373. package/dist/docs/assets/guides_opentelemetry.md.CoZ1kNyd.js +0 -28
  374. package/dist/docs/assets/guides_opentelemetry.md.CoZ1kNyd.lean.js +0 -1
  375. package/dist/docs/assets/guides_pagerduty.md.mnKXe9U0.js +0 -60
  376. package/dist/docs/assets/guides_pagerduty.md.mnKXe9U0.lean.js +0 -1
  377. package/dist/docs/assets/guides_slack.md.BwXHURvY.js +0 -88
  378. package/dist/docs/assets/hillclimbing.md.By61f3uq.js +0 -7
  379. package/dist/docs/assets/hillclimbing.md.By61f3uq.lean.js +0 -1
  380. package/dist/docs/assets/index.md.CAruw_rp.js +0 -26
  381. package/dist/docs/assets/index.md.CAruw_rp.lean.js +0 -1
  382. package/dist/docs/assets/quickstart.md.CljDaJBi.js +0 -170
  383. package/dist/docs/assets/reference_access.md.CEtibb4i.js +0 -1
  384. package/dist/docs/assets/reference_access.md.CEtibb4i.lean.js +0 -1
  385. package/dist/docs/assets/reference_agent-config.md.Ck29qYNj.js +0 -40
  386. package/dist/docs/assets/reference_artifacts.md.C-hKR2hp.js +0 -18
  387. package/dist/docs/assets/reference_channels.md.DsJfGSBJ.js +0 -22
  388. package/dist/docs/assets/reference_cli.md.BkdLRbXg.js +0 -99
  389. package/dist/docs/assets/reference_cli.md.BkdLRbXg.lean.js +0 -1
  390. package/dist/docs/assets/reference_connections.md.BoAotr7_.js +0 -66
  391. package/dist/docs/assets/reference_evals.md.2BXeMiO2.js +0 -76
  392. package/dist/docs/assets/reference_hooks.md.B-bwJuiG.js +0 -51
  393. package/dist/docs/assets/reference_hooks.md.B-bwJuiG.lean.js +0 -1
  394. package/dist/docs/assets/reference_http-api.md._RWELr3d.js +0 -11
  395. package/dist/docs/assets/reference_instructions.md.yL2hXLnl.js +0 -14
  396. package/dist/docs/assets/reference_instructions.md.yL2hXLnl.lean.js +0 -1
  397. package/dist/docs/assets/reference_limits.md.BCCnvt3o.js +0 -8
  398. package/dist/docs/assets/reference_limits.md.BCCnvt3o.lean.js +0 -1
  399. package/dist/docs/assets/reference_memory.md.Dd7cyV9V.js +0 -4
  400. package/dist/docs/assets/reference_playground.md.CQiJk9CM.js +0 -1
  401. package/dist/docs/assets/reference_project-layout.md.BamcPDUn.js +0 -25
  402. package/dist/docs/assets/reference_project-layout.md.BamcPDUn.lean.js +0 -1
  403. package/dist/docs/assets/reference_prompt.md.BMjm1Jp3.js +0 -9
  404. package/dist/docs/assets/reference_prompt.md.BMjm1Jp3.lean.js +0 -1
  405. package/dist/docs/assets/reference_schedules.md.RC47aFm4.js +0 -47
  406. package/dist/docs/assets/reference_sessions.md.C7KFhWXV.js +0 -22
  407. package/dist/docs/assets/reference_subagents.md.CAhvW4yS.js +0 -10
  408. package/dist/docs/assets/reference_tools.md.DQr7Fnhu.js +0 -79
  409. package/dist/docs/assets/templates_agentic-owners.md.Dh2nDSqg.js +0 -13
  410. package/dist/docs/assets/templates_security-reviewer.md.DnwXdnB5.js +0 -3
  411. package/dist/docs/assets/templates_security-reviewer.md.DnwXdnB5.lean.js +0 -1
  412. package/dist/docs/assets/templates_thermo-quality-review.md.MJIdHkzE.js +0 -3
  413. package/dist/docs/assets/templates_thermo-quality-review.md.MJIdHkzE.lean.js +0 -1
  414. package/dist/docs/assets/templates_thermo-review.md.DfN1OFOl.js +0 -3
  415. package/dist/docs/assets/templates_thermo-review.md.DfN1OFOl.lean.js +0 -1
  416. package/dist/docs/assets/templates_triage.md.BYWMloV4.js +0 -9
  417. package/dist/docs/assets/templates_triage.md.BYWMloV4.lean.js +0 -1
  418. package/dist/docs/assets/troubleshooting.md.CGOdNVAL.js +0 -1
  419. package/dist/docs/guides/hooks.html +0 -113
  420. package/dist/docs/guides/hooks.md +0 -116
  421. package/dist/docs/package/skills/create-agent/SKILL.md +0 -178
  422. package/dist/docs/package/skills/evals/SKILL.md +0 -145
  423. package/dist/docs/package/skills/github/SKILL.md +0 -133
  424. package/dist/docs/package/skills/hillclimb/SKILL.md +0 -63
  425. package/dist/docs/package/skills/mcp-auth/SKILL.md +0 -60
  426. package/dist/docs/package/skills/migrate-v2/SKILL.md +0 -394
  427. package/dist/docs/package/skills/setup-slack/SKILL.md +0 -136
  428. package/dist/docs/reference/prompt.html +0 -68
  429. package/dist/docs/reference/prompt.md +0 -47
  430. package/dist/docs/templates/security-reviewer.html +0 -62
  431. package/dist/docs/templates/security-reviewer.md +0 -81
  432. package/dist/docs/templates/thermo-quality-review.html +0 -62
  433. package/dist/docs/templates/thermo-quality-review.md +0 -75
  434. package/dist/docs/templates/thermo-review.html +0 -62
  435. package/dist/docs/templates/thermo-review.md +0 -75
  436. package/dist/docs/templates/triage.html +0 -68
  437. package/dist/docs/templates/triage.md +0 -101
  438. package/dist/internal/filesystem/errors.d.ts +0 -64
  439. package/dist/internal/filesystem/errors.d.ts.map +0 -1
  440. package/dist/internal/filesystem/errors.js +0 -180
  441. package/dist/internal/filesystem/merge.d.ts +0 -34
  442. package/dist/internal/filesystem/merge.d.ts.map +0 -1
  443. package/dist/internal/filesystem/merge.js +0 -154
  444. package/dist/playground/assets/index-Cl9zvmmf.js +0 -62
  445. package/dist/playground/assets/index-F8b5WvWi.css +0 -1
  446. package/docs/guides/hooks.md +0 -121
  447. package/docs/reference/prompt.md +0 -52
  448. package/docs/templates/security-reviewer.md +0 -86
  449. package/docs/templates/thermo-quality-review.md +0 -80
  450. package/docs/templates/thermo-review.md +0 -80
  451. package/docs/templates/triage.md +0 -106
@@ -1,21 +1,13 @@
1
1
  # Evals
2
2
 
3
- An eval sends a fixed input to the real agent and checks the resulting
4
- trajectory: whether the turn succeeded, which tools ran, and what shape
5
- the answer took. Use evals to protect behavior you already understand,
6
- not to discover what the prompt should do.
3
+ An eval sends a fixed input to the real agent and checks which tools ran and what the answer looks
4
+ like. To improve an agent and lock each win with an eval, follow [Hillclimbing](/docs/hillclimbing.md).
7
5
 
8
- Evals run real model turns and real tools. Guard external writes before
9
- running a suite against an agent that can post, merge, or deploy.
6
+ Evals run real tools, so [guard any writes](#keep-side-effects-out-of-eval-sessions) first.
10
7
 
11
8
  ## Protect a tool decision
12
9
 
13
- This smoke case asks for a PR verdict, requires the read-only inspection
14
- tool, and fails if the agent tries to approve. The CLI reports all three
15
- decisions together instead of stopping at the first miss.
16
-
17
- Use the [Jev extension](/docs/guides/jev.md) when host code needs a typed
18
- choice, score, or boolean before it writes.
10
+ The case below passes only if the agent calls the inspection tool and doesn't approve.
19
11
 
20
12
  ```ts
21
13
  // evals/readiness.eval.ts
@@ -24,9 +16,7 @@ import { defineEval, includes } from "@grok/agent-sdk/evals";
24
16
  export default defineEval({
25
17
  tags: ["smoke"],
26
18
  async test(t) {
27
- await t.send(
28
- "Is https://github.com/acme/checkout/pull/42 ready to approve?"
29
- );
19
+ await t.send("Is https://github.com/acme/checkout/pull/42 ready to approve?");
30
20
 
31
21
  t.succeeded();
32
22
  t.calledTool("inspect_pr");
@@ -36,10 +26,7 @@ export default defineEval({
36
26
  });
37
27
  ```
38
28
 
39
- Evals live in the project-root `evals/` directory, beside `bot/`.
40
- Blank `agent-sdk init` writes `evals/smoke.eval.ts`, which uses the scaffolded
41
- `echo` tool and asserts `t.succeeded()` and `t.calledTool("echo")`.
42
- It also writes this concurrency config:
29
+ Evals live in `evals/` beside `bot/`. Set how many cases run at once:
43
30
 
44
31
  ```ts
45
32
  // evals/evals.config.ts
@@ -50,52 +37,34 @@ export default defineEvalConfig({
50
37
  });
51
38
  ```
52
39
 
53
- ```text
40
+ ```bash
54
41
  agent-sdk eval --dir . --list
55
42
  agent-sdk eval --dir . --tag smoke
56
43
  ```
57
44
 
58
45
  ## Check answer shape or quality
59
46
 
60
- Prefer a deterministic shape check when the output has a contract. The
61
- failure tells you which field was wrong, and the case remains stable
62
- when the model changes its wording.
47
+ When the output has a contract, check its shape:
63
48
 
64
49
  ```ts
65
50
  import { matches } from "@grok/agent-sdk/evals";
66
51
  import { z } from "zod";
67
52
 
68
53
  const verdict = JSON.parse(t.reply ?? "{}");
69
- t.check(
70
- verdict,
71
- matches(
72
- z.object({
73
- ready: z.boolean(),
74
- blockers: z.array(z.string()),
75
- })
76
- )
77
- );
54
+ t.check(verdict, matches(z.object({ ready: z.boolean(), blockers: z.array(z.string()) })));
78
55
  ```
79
56
 
80
- Use a judge when correctness depends on meaning that a schema, regex, or
81
- known value cannot capture:
57
+ When correctness depends on meaning, use a judge. [Judges](/docs/reference/evals.md#judges) covers
58
+ models and grader options.
82
59
 
83
60
  ```ts
84
- t.judge
85
- .factuality("The lint step failed on src/sidebar.ts.")
86
- .atLeast(0.8);
61
+ t.judge.factuality("The lint step failed on src/sidebar.ts.").atLeast(0.8);
87
62
  ```
88
63
 
89
- Judge checks are tracked scores until you give them a hard gate.
90
- Configure the judge model in `evals.config.ts`. The
91
- [Evals reference](/docs/reference/evals.md#judges) owns grader options and
92
- severity rules.
93
-
94
64
  ## Test a conversation
95
65
 
96
- Several `t.send` calls in one case share a session. Assertions on a
97
- returned turn inspect only that turn, while assertions on `t` inspect
98
- the whole conversation.
66
+ Several `t.send` calls in one case share a session. An assertion on a returned turn checks only that
67
+ turn, and an assertion on `t` checks the whole conversation.
99
68
 
100
69
  ```ts
101
70
  const first = await t.send("Inspect PR 42.");
@@ -109,9 +78,7 @@ t.succeeded();
109
78
 
110
79
  ## Pin realistic inputs
111
80
 
112
- A useful eval changes only when the agent changes. Freeze canonical chat
113
- prompts, seed workspace evidence directly, and save webhook payloads
114
- instead of depending on a developer's checkout or live external state.
81
+ Seed the files the agent reads instead of depending on your checkout:
115
82
 
116
83
  ```ts
117
84
  await t.send("Review pr/diff.patch and report blockers.", {
@@ -124,47 +91,32 @@ await t.send("Review pr/diff.patch and report blockers.", {
124
91
  });
125
92
  ```
126
93
 
127
- For GitHub agents, snapshot a PR into committed fixtures:
94
+ Save a GitHub pull request's webhook payloads as fixtures:
128
95
 
129
- ```text
130
- agent-sdk github replay acme/checkout#42 \
131
- --events '*' \
132
- --dry-run \
133
- --out fixtures/github
96
+ ```bash
97
+ agent-sdk github replay https://github.com/acme/checkout/pull/42 \
98
+ --events '*' --dry-run --fixtures-dir fixtures/github
134
99
  ```
135
100
 
136
- Keep a small, fast `smoke` set for every change. Put larger datasets and
137
- drift-prone live cases behind an explicit selection. See
138
- [Datasets and fixtures](/docs/reference/evals.md#datasets-and-fixtures) for
139
- loaders and case expansion.
101
+ [Datasets and fixtures](/docs/reference/evals.md#datasets-and-fixtures) covers loaders for larger sets.
140
102
 
141
103
  ## Run and debug a regression suite
142
104
 
143
- Run one case while iterating, then the smoke suite before review. A
144
- failure artifact records the input, assertions, tool calls, and final
145
- text, making it the first place to look before changing the prompt.
105
+ Run one case while you iterate, then the smoke set before review:
146
106
 
147
- ```text
107
+ ```bash
148
108
  agent-sdk eval --dir . readiness
149
109
  agent-sdk eval --dir . --tag smoke
150
- agent-sdk eval --dir . --json
151
110
  ```
152
111
 
153
- By default, results land under `evals/<stamp>/` in the project state
154
- directory. Open `evals/<case-id>.json` inside that run directory for the
155
- failed case. This artifact location is independent of `--state-root`;
156
- pass `--artifacts <dir>` to choose another destination.
157
-
158
- Model turns need a credential. Resolution checks `CURSOR_API_KEY`,
159
- `CURSOR_API_KEY_FILE`, `CURSOR_SERVICE_ACCOUNT_KEY`, then a saved
160
- `agent-sdk login`. `eval --list` only discovers cases and needs none.
112
+ Each run writes a folder under `evals/` in the project state directory, or under `--artifacts <dir>`
113
+ if you pass it. Open `evals/<case-id>.json` in that folder to see a failed case's input, assertions,
114
+ tool calls, and final text. Model turns need a
115
+ [Cursor credential](/docs/reference/cli.md#environment-variables).
161
116
 
162
117
  ## Run evals in CI
163
118
 
164
- Write machine-readable output and JUnit annotations, then let a failed
165
- gate fail the job:
166
-
167
- ```text
119
+ ```bash
168
120
  # CURSOR_API_KEY comes from the CI secret store.
169
121
  agent-sdk eval --dir . --json --no-stream \
170
122
  --junit reports/evals.xml \
@@ -172,70 +124,22 @@ agent-sdk eval --dir . --json --no-stream \
172
124
  > reports/evals.json
173
125
  ```
174
126
 
175
- Run `--tag smoke` on pull requests and the full suite on a schedule when
176
- cost or latency makes every-push coverage impractical. Use soft scores
177
- for a new probabilistic benchmark until its threshold is trustworthy.
178
-
179
- The [CLI reference](/docs/reference/cli.md#eval) covers selectors,
180
- server-side batches, exit codes, and JSON output.
127
+ Run `--tag smoke` on pull requests and the full suite on a schedule. The
128
+ [CLI reference](/docs/reference/cli.md#eval) covers every flag.
181
129
 
182
130
  ## Keep side effects out of eval sessions
183
131
 
184
- Eval sessions carry `purpose: "eval"`. Check it at the deterministic
185
- write boundary and return the decision without performing the external
186
- action:
132
+ Eval sessions carry `purpose: "eval"`. Check it where the tool writes, and return the decision
133
+ without the external action:
187
134
 
188
135
  ```ts
189
136
  async execute({ findings }, ctx) {
190
137
  if (ctx.session.purpose === "eval") {
191
- return {
192
- posted: false,
193
- reason: "eval",
194
- count: findings.length,
195
- };
138
+ return { posted: false, reason: "eval", count: findings.length };
196
139
  }
197
140
 
198
141
  return await postFindings(findings);
199
142
  }
200
143
  ```
201
144
 
202
- This preserves the tool call in the trajectory, so the eval can still
203
- assert that the agent chose to post.
204
-
205
- ## Evals, hooks, or hillclimbing?
206
-
207
- Use an eval to gate one fixed input. Use a
208
- [hook](/docs/reference/hooks.md) to observe every live session for metrics,
209
- auditing, or alerts. Use [hillclimbing](/docs/hillclimbing.md) to decide
210
- which source change improves a fixed set of inputs; every kept
211
- hillclimb change should add an eval.
212
-
213
- ## Gates and scores
214
-
215
- `t.succeeded()`, tool assertions, and deterministic checks are hard
216
- gates by default. A failed gate fails the case. `.soft()` records a
217
- measurement without blocking, while `.atLeast(threshold)` records a
218
- score and marks a miss as `scored`.
219
-
220
- Start with decisions and output shape: the intended tool, the tempting
221
- wrong tool, and one structural check. Exact prose is rarely a stable
222
- contract.
223
-
224
- ## Keep improvements with regression evals
225
-
226
- Every kept improvement needs an eval that would have failed before the
227
- change. A tool-choice gate, bounded `maxToolCalls`, or output-shape check
228
- turns the improvement into a durable contract.
229
-
230
- Never weaken an existing gate to make a new implementation pass. That
231
- moves the freeze line instead of proving the change.
232
-
233
- ## Related
234
-
235
- - [Evals reference](/docs/reference/evals.md): complete authoring and runner
236
- contracts
237
- - [`skills/evals/SKILL.md`](/docs/package/skills/evals/SKILL.md): author and seed
238
- cases with a coding agent
239
- - [Hillclimbing](/docs/hillclimbing.md): measure a change and lock the win
240
- - [Sessions](/docs/reference/sessions.md): events available to trajectory
241
- assertions
145
+ The tool call stays in the trajectory, so the eval can still assert that the agent chose to post.