@cursor/july 0.1.94 → 0.1.96
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +3 -4
- package/dist/channels/deployments/types.d.ts +1 -1
- package/dist/channels/deployments/types.d.ts.map +1 -1
- package/dist/channels/deployments/types.js +1 -0
- package/dist/channels/github/github-channel.d.ts.map +1 -1
- package/dist/channels/github/github-channel.js +14 -2
- package/dist/channels/github/types.d.ts +18 -3
- package/dist/channels/github/types.d.ts.map +1 -1
- package/dist/channels/origin/api.d.ts +59 -13
- package/dist/channels/origin/api.d.ts.map +1 -1
- package/dist/channels/origin/api.js +168 -27
- package/dist/channels/origin/binding.d.ts +2 -2
- package/dist/channels/origin/checks.d.ts +6 -6
- package/dist/channels/origin/checks.js +6 -6
- package/dist/channels/origin/cursor-account.d.ts +5 -3
- package/dist/channels/origin/cursor-account.d.ts.map +1 -1
- package/dist/channels/origin/cursor-account.js +5 -3
- package/dist/channels/origin/origin-channel.d.ts.map +1 -1
- package/dist/channels/origin/origin-channel.js +11 -3
- package/dist/channels/origin/origin-webhook.d.ts +11 -1
- package/dist/channels/origin/origin-webhook.d.ts.map +1 -1
- package/dist/channels/origin/origin-webhook.js +23 -3
- package/dist/channels/origin/types.d.ts +8 -0
- package/dist/channels/origin/types.d.ts.map +1 -1
- package/dist/channels/slack/api.d.ts +9 -0
- package/dist/channels/slack/api.d.ts.map +1 -1
- package/dist/channels/slack/api.js +36 -5
- package/dist/channels/slack/defaults.d.ts.map +1 -1
- package/dist/channels/slack/defaults.js +17 -46
- package/dist/channels/slack/dispatch.d.ts +7 -1
- package/dist/channels/slack/dispatch.d.ts.map +1 -1
- package/dist/channels/slack/dispatch.js +47 -23
- package/dist/channels/slack/inbound.d.ts +8 -0
- package/dist/channels/slack/inbound.d.ts.map +1 -1
- package/dist/channels/slack/inbound.js +32 -0
- package/dist/channels/slack/live-delivery.d.ts +26 -29
- package/dist/channels/slack/live-delivery.d.ts.map +1 -1
- package/dist/channels/slack/live-delivery.js +172 -73
- package/dist/channels/slack/manifest.d.ts.map +1 -1
- package/dist/channels/slack/manifest.js +5 -1
- package/dist/channels/slack/nudge-carrier.d.ts +2 -0
- package/dist/channels/slack/nudge-carrier.d.ts.map +1 -1
- package/dist/channels/slack/nudge-carrier.js +48 -1
- package/dist/channels/slack/pr-ref.d.ts +7 -1
- package/dist/channels/slack/pr-ref.d.ts.map +1 -1
- package/dist/channels/slack/pr-ref.js +42 -23
- package/dist/channels/slack/slack-channel.d.ts.map +1 -1
- package/dist/channels/slack/slack-channel.js +2 -2
- package/dist/docs/404.html +2 -2
- package/dist/docs/ab.html +4 -4
- package/dist/docs/assets/{app.CFDEas4I.js → app.jXleCmOc.js} +1 -1
- package/dist/docs/assets/chunks/@localSearchIndexroot.DFoGuETX.js +1 -0
- package/dist/docs/assets/chunks/{VPLocalSearchBox.B1IIYpYS.js → VPLocalSearchBox.8LhyglIT.js} +1 -1
- package/dist/docs/assets/chunks/{theme.Ct4NSiLm.js → theme.9ged-pr2.js} +2 -2
- package/dist/docs/assets/index.md.CI93Ltlb.js +5 -0
- package/dist/docs/assets/{index.md.DRakGHFe.lean.js → index.md.CI93Ltlb.lean.js} +1 -1
- package/dist/docs/assets/{reference_cli.md.Cw6_ICYG.js → reference_cli.md.CCnG-nJx.js} +4 -2
- package/dist/docs/assets/{reference_cli.md.Cw6_ICYG.lean.js → reference_cli.md.CCnG-nJx.lean.js} +1 -1
- package/dist/docs/assets/{reference_skills.md.8son6Hjm.js → reference_skills.md.BFW9retM.js} +3 -3
- package/dist/docs/assets/{templates_agentic-owners.md.DqtPdm6f.js → templates_agentic-owners.md.BZSH4N9z.js} +1 -1
- package/dist/docs/assets/templates_code-wiki.md.CDYZIBFN.js +2 -0
- package/dist/docs/assets/templates_code-wiki.md.CDYZIBFN.lean.js +1 -0
- package/dist/docs/assets/{templates_demo.md.DhFcWN6j.js → templates_demo.md.BeQX9V3H.js} +1 -1
- package/dist/docs/assets/{templates_pr-autofixer.md.DU7dQpor.js → templates_pr-autofixer.md.x5zl6-GT.js} +2 -2
- package/dist/docs/assets/{templates_pr-autofixer.md.DU7dQpor.lean.js → templates_pr-autofixer.md.x5zl6-GT.lean.js} +1 -1
- package/dist/docs/assets/templates_security-help.md.C3Ny_Qr2.js +4 -0
- package/dist/docs/assets/templates_security-help.md.C3Ny_Qr2.lean.js +1 -0
- package/dist/docs/assets/{templates_security-reviewer.md.CTa7u_l1.js → templates_security-reviewer.md.lshxbCLK.js} +2 -2
- package/dist/docs/assets/{templates_security-reviewer.md.CTa7u_l1.lean.js → templates_security-reviewer.md.lshxbCLK.lean.js} +1 -1
- package/dist/docs/assets/{templates_triage.md.CVlpctKS.js → templates_triage.md.Co4UNzkZ.js} +3 -3
- package/dist/docs/assets/{templates_triage.md.CVlpctKS.lean.js → templates_triage.md.Co4UNzkZ.lean.js} +1 -1
- package/dist/docs/building-with-agents.html +4 -4
- package/dist/docs/concepts.html +4 -4
- package/dist/docs/deployment.html +4 -4
- package/dist/docs/evals.html +4 -4
- package/dist/docs/guides/agent-to-agent.html +4 -4
- package/dist/docs/guides/cloud-runtime.html +4 -4
- package/dist/docs/guides/convert-automation.html +4 -4
- package/dist/docs/guides/github.html +4 -4
- package/dist/docs/guides/human-in-the-loop.html +4 -4
- package/dist/docs/guides/mcp-oauth.html +4 -4
- package/dist/docs/guides/opentelemetry.html +4 -4
- package/dist/docs/guides/slack.html +4 -4
- package/dist/docs/guides/webhooks.html +4 -4
- package/dist/docs/hashmap.json +1 -1
- package/dist/docs/hillclimbing.html +4 -4
- package/dist/docs/index.html +6 -6
- package/dist/docs/index.md +5 -1
- package/dist/docs/llms-full.txt +189 -26
- package/dist/docs/llms.txt +2 -0
- package/dist/docs/quickstart.html +4 -4
- package/dist/docs/reference/agent-config.html +4 -4
- package/dist/docs/reference/artifacts.html +4 -4
- package/dist/docs/reference/channels.html +4 -4
- package/dist/docs/reference/cli.html +8 -6
- package/dist/docs/reference/cli.md +4 -1
- package/dist/docs/reference/connections.html +4 -4
- package/dist/docs/reference/hooks.html +4 -4
- package/dist/docs/reference/http-api.html +4 -4
- package/dist/docs/reference/instructions.html +4 -4
- package/dist/docs/reference/playground.html +4 -4
- package/dist/docs/reference/project-layout.html +4 -4
- package/dist/docs/reference/prompt.html +4 -4
- package/dist/docs/reference/schedules.html +4 -4
- package/dist/docs/reference/sessions.html +4 -4
- package/dist/docs/reference/skills.html +7 -7
- package/dist/docs/reference/subagents.html +4 -4
- package/dist/docs/reference/tools.html +4 -4
- package/dist/docs/scaffolding-agents.html +4 -4
- package/dist/docs/storage.html +4 -4
- package/dist/docs/templates/agentic-owners.html +7 -7
- package/dist/docs/templates/agentic-owners.md +1 -1
- package/dist/docs/templates/code-wiki.html +28 -0
- package/dist/docs/templates/code-wiki.md +91 -0
- package/dist/docs/templates/demo.html +6 -6
- package/dist/docs/templates/demo.md +3 -2
- package/dist/docs/templates/pr-autofixer.html +6 -6
- package/dist/docs/templates/pr-autofixer.md +7 -13
- package/dist/docs/templates/security-help.html +30 -0
- package/dist/docs/templates/security-help.md +65 -0
- package/dist/docs/templates/security-reviewer.html +6 -6
- package/dist/docs/templates/security-reviewer.md +1 -2
- package/dist/docs/templates/triage.html +7 -7
- package/dist/docs/templates/triage.md +2 -6
- package/dist/docs/troubleshooting.html +4 -4
- package/dist/files-backends/cursor-hosted.d.ts +26 -0
- package/dist/files-backends/cursor-hosted.d.ts.map +1 -1
- package/dist/files-backends/cursor-hosted.js +32 -4
- package/dist/files.d.ts +2 -0
- package/dist/files.d.ts.map +1 -1
- package/dist/files.js +5 -0
- package/dist/internal/advertise-tools.d.ts +2 -0
- package/dist/internal/advertise-tools.d.ts.map +1 -1
- package/dist/internal/advertise-tools.js +15 -1
- package/dist/internal/bounded.d.ts +7 -0
- package/dist/internal/bounded.d.ts.map +1 -0
- package/dist/internal/bounded.js +32 -0
- package/dist/internal/cli-deploy.d.ts.map +1 -1
- package/dist/internal/cli-deploy.js +142 -7
- package/dist/internal/conversation-mirror.d.ts +85 -0
- package/dist/internal/conversation-mirror.d.ts.map +1 -0
- package/dist/internal/conversation-mirror.js +312 -0
- package/dist/internal/cursor/backend-client.d.ts.map +1 -1
- package/dist/internal/cursor/backend-client.js +22 -48
- package/dist/internal/cursor/session-token.d.ts +45 -0
- package/dist/internal/cursor/session-token.d.ts.map +1 -0
- package/dist/internal/cursor/session-token.js +99 -0
- package/dist/internal/cursor-agent-template.d.ts +1 -1
- package/dist/internal/cursor-agent-template.d.ts.map +1 -1
- package/dist/internal/cursor-agent-template.js +1 -0
- package/dist/internal/deferred-channel-session.d.ts +20 -0
- package/dist/internal/deferred-channel-session.d.ts.map +1 -0
- package/dist/internal/deferred-channel-session.js +62 -0
- package/dist/internal/deploy-client.d.ts +13 -1
- package/dist/internal/deploy-client.d.ts.map +1 -1
- package/dist/internal/deploy-client.js +11 -1
- package/dist/internal/deploy-manifest.d.ts.map +1 -1
- package/dist/internal/deploy-manifest.js +11 -5
- package/dist/internal/discovery.d.ts.map +1 -1
- package/dist/internal/discovery.js +22 -2
- package/dist/internal/framework-file-storage.d.ts +61 -0
- package/dist/internal/framework-file-storage.d.ts.map +1 -0
- package/dist/internal/framework-file-storage.js +204 -0
- package/dist/internal/framework-storage-selection.d.ts +38 -16
- package/dist/internal/framework-storage-selection.d.ts.map +1 -1
- package/dist/internal/framework-storage-selection.js +78 -17
- package/dist/internal/hosted-admission-context.d.ts +20 -0
- package/dist/internal/hosted-admission-context.d.ts.map +1 -0
- package/dist/internal/hosted-admission-context.js +31 -0
- package/dist/internal/hosted-delivery-protocol.d.ts +5 -0
- package/dist/internal/hosted-delivery-protocol.d.ts.map +1 -1
- package/dist/internal/hosted-delivery-protocol.js +33 -1
- package/dist/internal/hosted-delivery.d.ts +11 -2
- package/dist/internal/hosted-delivery.d.ts.map +1 -1
- package/dist/internal/hosted-delivery.js +79 -35
- package/dist/internal/hosted-managed-automation.d.ts +30 -0
- package/dist/internal/hosted-managed-automation.d.ts.map +1 -0
- package/dist/internal/hosted-managed-automation.js +58 -0
- package/dist/internal/init-project.d.ts.map +1 -1
- package/dist/internal/init-project.js +2 -0
- package/dist/internal/install-cursor-skills.d.ts.map +1 -1
- package/dist/internal/install-cursor-skills.js +1 -3
- package/dist/internal/managed-delivery.d.ts +80 -0
- package/dist/internal/managed-delivery.d.ts.map +1 -0
- package/dist/internal/managed-delivery.js +86 -0
- package/dist/internal/review-comments.d.ts.map +1 -1
- package/dist/internal/review-comments.js +10 -98
- package/dist/internal/runtime-dispatch-runner.d.ts +5 -1
- package/dist/internal/runtime-dispatch-runner.d.ts.map +1 -1
- package/dist/internal/runtime-dispatch-runner.js +7 -0
- package/dist/internal/scm/pr-url.d.ts +14 -0
- package/dist/internal/scm/pr-url.d.ts.map +1 -0
- package/dist/internal/scm/pr-url.js +65 -0
- package/dist/internal/sdk-runner.d.ts +25 -0
- package/dist/internal/sdk-runner.d.ts.map +1 -1
- package/dist/internal/sdk-runner.js +70 -0
- package/dist/internal/server.d.ts.map +1 -1
- package/dist/internal/server.js +11 -2
- package/dist/internal/session-engine.d.ts +13 -0
- package/dist/internal/session-engine.d.ts.map +1 -1
- package/dist/internal/session-engine.js +71 -3
- package/dist/internal/steering.d.ts +15 -0
- package/dist/internal/steering.d.ts.map +1 -0
- package/dist/internal/steering.js +18 -0
- package/dist/playground/assets/index-B1DDvBeC.css +1 -0
- package/dist/playground/assets/{index-B3JCyigB.js → index-Cx9CclK_.js} +47 -47
- package/dist/playground/index.html +2 -2
- package/dist/storage-backends/cursor-hosted.d.ts +8 -0
- package/dist/storage-backends/cursor-hosted.d.ts.map +1 -1
- package/dist/storage-backends/cursor-hosted.js +4 -3
- package/dist/types.d.ts +2 -1
- package/dist/types.d.ts.map +1 -1
- package/docs/README.md +5 -1
- package/docs/reference/cli.md +4 -1
- package/docs/templates/agentic-owners.md +1 -1
- package/docs/templates/code-wiki.md +96 -0
- package/docs/templates/demo.md +3 -2
- package/docs/templates/pr-autofixer.md +7 -13
- package/docs/templates/security-help.md +70 -0
- package/docs/templates/security-reviewer.md +1 -2
- package/docs/templates/triage.md +2 -6
- package/package.json +2 -2
- package/skills/ab/SKILL.md +13 -97
- package/skills/create-agent/SKILL.md +123 -223
- package/skills/debug/SKILL.md +34 -68
- package/skills/evals/SKILL.md +55 -180
- package/skills/framework-map/SKILL.md +83 -145
- package/skills/github/SKILL.md +53 -111
- package/skills/hillclimb/SKILL.md +40 -163
- package/skills/mcp-auth/SKILL.md +48 -107
- package/skills/otel/SKILL.md +22 -68
- package/skills/setup-slack/SKILL.md +77 -208
- package/src/channels/deployments/types.ts +1 -0
- package/src/channels/github/github-channel.ts +23 -8
- package/src/channels/github/types.ts +19 -2
- package/src/channels/origin/api.ts +242 -30
- package/src/channels/origin/binding.ts +2 -2
- package/src/channels/origin/checks.ts +6 -6
- package/src/channels/origin/cursor-account.ts +5 -3
- package/src/channels/origin/origin-channel.ts +13 -1
- package/src/channels/origin/origin-webhook.ts +27 -3
- package/src/channels/origin/types.ts +8 -0
- package/src/channels/slack/api.ts +41 -5
- package/src/channels/slack/defaults.ts +15 -49
- package/src/channels/slack/dispatch.ts +56 -22
- package/src/channels/slack/inbound.ts +43 -0
- package/src/channels/slack/live-delivery.ts +235 -114
- package/src/channels/slack/manifest.ts +5 -1
- package/src/channels/slack/nudge-carrier.ts +48 -1
- package/src/channels/slack/pr-ref.ts +56 -25
- package/src/channels/slack/slack-channel.ts +2 -0
- package/src/files-backends/cursor-hosted.ts +41 -2
- package/src/files.ts +11 -0
- package/src/internal/advertise-tools.ts +18 -2
- package/src/internal/bounded.ts +28 -0
- package/src/internal/cli-deploy.ts +183 -7
- package/src/internal/conversation-mirror.ts +406 -0
- package/src/internal/cursor/backend-client.ts +34 -58
- package/src/internal/cursor/session-token.ts +115 -0
- package/src/internal/cursor-agent-template.ts +1 -0
- package/src/internal/deferred-channel-session.ts +61 -0
- package/src/internal/deploy-client.ts +24 -1
- package/src/internal/deploy-manifest.ts +10 -5
- package/src/internal/discovery.ts +25 -2
- package/src/internal/framework-file-storage.ts +222 -0
- package/src/internal/framework-storage-selection.ts +101 -19
- package/src/internal/hosted-admission-context.ts +37 -0
- package/src/internal/hosted-delivery-protocol.ts +44 -1
- package/src/internal/hosted-delivery.ts +105 -40
- package/src/internal/hosted-managed-automation.ts +72 -0
- package/src/internal/init-project.ts +2 -0
- package/src/internal/install-cursor-skills.ts +1 -4
- package/src/internal/managed-delivery.ts +125 -0
- package/src/internal/review-comments.ts +10 -113
- package/src/internal/runtime-dispatch-runner.ts +10 -0
- package/src/internal/scm/pr-url.ts +95 -0
- package/src/internal/sdk-runner.ts +103 -0
- package/src/internal/server.ts +12 -0
- package/src/internal/session-engine.ts +76 -4
- package/src/internal/steering.ts +17 -0
- package/src/storage-backends/cursor-hosted.ts +10 -1
- package/src/types.ts +2 -1
- package/templates/agentic-owners/README.md +1 -1
- package/templates/agentic-owners/agent/agent.ts +0 -10
- package/templates/agentic-owners/agent/channels/github.ts +5 -14
- package/templates/agentic-owners/agent/lib/config.ts +0 -8
- package/templates/agentic-owners/agent/lib/review.ts +2 -15
- package/templates/agentic-owners/agent/tools/record_review.ts +2 -4
- package/templates/code-wiki/README.md +34 -0
- package/templates/code-wiki/agent/agent.ts +8 -0
- package/templates/code-wiki/agent/channels/github.ts +144 -0
- package/templates/code-wiki/agent/instructions.md +39 -0
- package/templates/code-wiki/agent/lib/repos.ts +5 -0
- package/templates/code-wiki/agent/lib/wiki.ts +22 -0
- package/templates/code-wiki/agent/tools/apply_wiki.ts +219 -0
- package/templates/code-wiki/evals/evals.config.ts +5 -0
- package/templates/code-wiki/evals/wiki.eval.ts +132 -0
- package/templates/code-wiki/init.json +25 -0
- package/templates/code-wiki/package.json +18 -0
- package/templates/code-wiki/tsconfig.json +12 -0
- package/templates/demo/agent/agent.ts +0 -10
- package/templates/pr-autofixer/README.md +0 -2
- package/templates/pr-autofixer/agent/agent.ts +0 -11
- package/templates/pr-autofixer/agent/channels/slack.ts +0 -7
- package/templates/pr-autofixer/agent/lib/pr-state.ts +5 -17
- package/templates/pr-autofixer/agent/lib/repos.ts +0 -1
- package/templates/security-help/README.md +2 -2
- package/templates/security-help/agent/agent.ts +1 -2
- package/templates/security-help/agent/channels/slack.ts +0 -3
- package/templates/security-help/agent/instructions.md +9 -10
- package/templates/security-help/agent/skills/access-request.md +1 -1
- package/templates/security-help/agent/skills/faq.md +31 -0
- package/templates/security-help/agent/skills/security-playbooks.md +1 -1
- package/templates/security-help/package.json +1 -2
- package/templates/security-reviewer/agent/agent.ts +0 -10
- package/templates/triage/agent/agent.ts +0 -10
- package/templates/triage/agent/channels/queue.ts +1 -1
- package/templates/triage/agent/channels/webhook.ts +1 -3
- package/dist/docs/assets/chunks/@localSearchIndexroot.DU3U2Ij2.js +0 -1
- package/dist/docs/assets/index.md.DRakGHFe.js +0 -5
- package/dist/internal/deployment-scm-event-stream.d.ts +0 -116
- package/dist/internal/deployment-scm-event-stream.d.ts.map +0 -1
- package/dist/internal/deployment-scm-event-stream.js +0 -474
- package/dist/playground/assets/index-D9N7-q97.css +0 -1
- package/src/internal/deployment-scm-event-stream.ts +0 -593
- package/templates/security-help/agent/knowledge/faq/approvals.md +0 -5
- package/templates/security-help/agent/knowledge/faq/channels.md +0 -6
- package/templates/security-help/agent/knowledge/faq/phishing.md +0 -10
- package/templates/security-help/agent/skills/security-first-pass.md +0 -15
- /package/dist/docs/assets/{reference_skills.md.8son6Hjm.lean.js → reference_skills.md.BFW9retM.lean.js} +0 -0
- /package/dist/docs/assets/{templates_agentic-owners.md.DqtPdm6f.lean.js → templates_agentic-owners.md.BZSH4N9z.lean.js} +0 -0
- /package/dist/docs/assets/{templates_demo.md.DhFcWN6j.lean.js → templates_demo.md.BeQX9V3H.lean.js} +0 -0
package/skills/ab/SKILL.md
CHANGED
|
@@ -1,35 +1,24 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: agentsdk-ab
|
|
3
3
|
description: >-
|
|
4
|
-
Author defineAB live A/B metrics
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
comparison; use defineEval for regression gates.
|
|
4
|
+
Author defineAB live A/B metrics under agent/ab. Sticky variants
|
|
5
|
+
and onSample on live runs. Not a test runner. Use when comparing
|
|
6
|
+
variants on traffic; use defineEval for regression gates.
|
|
8
7
|
---
|
|
9
8
|
|
|
10
9
|
# Agent SDK A/B metrics (`defineAB`)
|
|
11
10
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
`defineAB` is a **live metrics plug-in**. At session creation the engine runs
|
|
16
|
-
`split` and appends durable `ab.assigned` events; the collector folds the
|
|
17
|
-
session event stream and calls `onSample`. There is **no** `agent-sdk ab`
|
|
18
|
-
CLI and **no** assertion API.
|
|
19
|
-
|
|
20
|
-
Human-facing reference: [`docs/ab.md`](../../docs/ab.md).
|
|
11
|
+
Live metrics plug-in. No `agent-sdk ab` CLI. No assertion API.
|
|
12
|
+
Reference: `docs/ab.md`.
|
|
21
13
|
|
|
22
14
|
| | `defineEval` | `defineAB` |
|
|
23
15
|
| --- | --- | --- |
|
|
24
|
-
| Job |
|
|
16
|
+
| Job | Gates on frozen fixtures | Metrics on live runs |
|
|
25
17
|
| Location | `evals/**/*.eval.ts` | `agent/ab.ts` or `agent/ab/<name>.ts` |
|
|
26
|
-
| How it runs | `agent-sdk eval` |
|
|
27
|
-
| Driver | `t.send` + gates | `split` → `ab.assigned` + `onSample` |
|
|
28
|
-
|
|
29
|
-
## Authoring
|
|
18
|
+
| How it runs | `agent-sdk eval` | Under `serve` / `run` |
|
|
30
19
|
|
|
31
20
|
```ts
|
|
32
|
-
import { defineAB, splitBySessionHash
|
|
21
|
+
import { defineAB, splitBySessionHash } from "@cursor/july/ab";
|
|
33
22
|
|
|
34
23
|
export default defineAB({
|
|
35
24
|
name: "concise-instructions",
|
|
@@ -37,13 +26,11 @@ export default defineAB({
|
|
|
37
26
|
control: { label: "Baseline" },
|
|
38
27
|
treatment: {
|
|
39
28
|
label: "Shorter",
|
|
40
|
-
// Optional: appended to agent instructions for this arm.
|
|
41
29
|
instructions: "Keep replies to one short paragraph.",
|
|
42
30
|
},
|
|
43
31
|
},
|
|
44
32
|
split: splitBySessionHash({ holdout: 0.1 }),
|
|
45
33
|
derive: {
|
|
46
|
-
// Keep sync + cheap — runs on every event, including message deltas.
|
|
47
34
|
weatherCalls: (event) =>
|
|
48
35
|
event.type === "action.result" && event.data.toolName === "get_weather"
|
|
49
36
|
? 1
|
|
@@ -55,8 +42,6 @@ export default defineAB({
|
|
|
55
42
|
});
|
|
56
43
|
```
|
|
57
44
|
|
|
58
|
-
In tools, read the arm from session context:
|
|
59
|
-
|
|
60
45
|
```ts
|
|
61
46
|
async execute(input, ctx) {
|
|
62
47
|
if (ctx.session.abs?.["concise-instructions"] === "treatment") {
|
|
@@ -65,78 +50,9 @@ async execute(input, ctx) {
|
|
|
65
50
|
}
|
|
66
51
|
```
|
|
67
52
|
|
|
53
|
+
Enrollment is at session creation. Eval sessions skip it. Do not
|
|
54
|
+
use `splitIf` to filter evals. Split helpers and `onSample`
|
|
55
|
+
fields: `docs/ab.md`.
|
|
68
56
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
Optional project defaults in `agent/ab.config.ts` (retention + persistence):
|
|
72
|
-
|
|
73
|
-
```ts
|
|
74
|
-
import {
|
|
75
|
-
defineABConfig,
|
|
76
|
-
persistABSamplesToDir,
|
|
77
|
-
persistABSnapshotsToDir,
|
|
78
|
-
} from "@cursor/july/ab";
|
|
79
|
-
|
|
80
|
-
export default defineABConfig({
|
|
81
|
-
// maxPlaygroundSessions: 200, // optional; default 200; A/Bs tab / GET /v1/abs only
|
|
82
|
-
persistSamples: persistABSamplesToDir(".agent-serve/ab-samples"),
|
|
83
|
-
persistSnapshots: persistABSnapshotsToDir(".agent-serve/ab-snapshots"),
|
|
84
|
-
});
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
Session `events.ndjson` remains the assignment/metrics source of truth.
|
|
88
|
-
`persistSamples` / `persistSnapshots` are author-controlled exports (like
|
|
89
|
-
eval `persistRuns`).
|
|
90
|
-
|
|
91
|
-
### Stream model
|
|
92
|
-
|
|
93
|
-
Typical new-session prefix:
|
|
94
|
-
|
|
95
|
-
1. `session.started`
|
|
96
|
-
2. one `ab.assigned` per experiment (`variant` key or `null` skip)
|
|
97
|
-
3. `message.received` / `turn.started` / …
|
|
98
|
-
|
|
99
|
-
Enrollment is **at session creation** (before the first turn) so each arm can
|
|
100
|
-
vary instructions and tool behavior from a known assignment. Failed/`null`
|
|
101
|
-
splits persist as `ab.assigned` with `variant: null` (sticky skip). The
|
|
102
|
-
collector is a pure fold over the log. `SessionRecord.abs` is a convenience
|
|
103
|
-
mirror, not a metrics store. After park/restart it replays `events.ndjson` to
|
|
104
|
-
rebuild counters without re-firing `onSample`.
|
|
105
|
-
|
|
106
|
-
**Evals are separate:** the eval harness creates sessions with
|
|
107
|
-
`purpose: "eval"` (playground Evals / `agent-sdk eval`). Those skip
|
|
108
|
-
enrollment entirely — no `ab.assigned`, no `onSample`, omitted from
|
|
109
|
-
`GET /v1/abs`. Ordinary chat / `agent-sdk run` / Slack stay `"live"`.
|
|
110
|
-
Do not use `splitIf` to filter evals; the framework already does.
|
|
111
|
-
|
|
112
|
-
After enrollment, arms are on `session.abs` (experiment → variant | `null`)
|
|
113
|
-
for tools/hooks, and optional per-variant `instructions` are appended to the
|
|
114
|
-
agent prompt for that session (a treatment arm can, say, switch `get_weather`
|
|
115
|
-
to °C for its sessions).
|
|
116
|
-
|
|
117
|
-
### Split helpers
|
|
118
|
-
|
|
119
|
-
| Helper | Behavior |
|
|
120
|
-
| --- | --- |
|
|
121
|
-
| `splitBySessionHash({ weights?, holdout?, salt? })` | Sticky hash → arm |
|
|
122
|
-
| `splitByRandom({ weights?, holdout? })` | Random once, then sticky |
|
|
123
|
-
| `splitAlways("control")` | Pin all new sessions |
|
|
124
|
-
| `splitNone()` | Skip everyone (pause in place) |
|
|
125
|
-
| `splitIf(pred, inner)` | Skip unless `pred` |
|
|
126
|
-
| `(ctx) => …` | Custom — variant key or `null` |
|
|
127
|
-
|
|
128
|
-
### Built-in metrics
|
|
129
|
-
|
|
130
|
-
`turns`, `turnFailures`, `toolCalls`, `toolErrors`, token fields, `wallTimeMs`,
|
|
131
|
-
plus `custom.*` from `derive`. `onSample` fires on `turn.completed` / `turn.failed`.
|
|
132
|
-
|
|
133
|
-
Playground **A/Bs** tab and `GET /v1/abs` fold the same durable streams into
|
|
134
|
-
per-session assignments and aggregate arm totals (no separate metrics store).
|
|
135
|
-
|
|
136
|
-
### `derive` cost
|
|
137
|
-
|
|
138
|
-
`derive` extractors run on **every** session event (same cost model as hooks).
|
|
139
|
-
Keep them synchronous and cheap — no network, no disk, no heavy parsing.
|
|
140
|
-
|
|
141
|
-
Before wiring: pick experiment name + arm labels, a split helper (and who
|
|
142
|
-
to leave out), and a real `onSample` sink — do not invent credentials.
|
|
57
|
+
Pick a name, arm labels, a split, and a real `onSample` sink. Do
|
|
58
|
+
not invent credentials.
|
|
@@ -1,142 +1,99 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: agentsdk-create-agent
|
|
3
3
|
description: >-
|
|
4
|
-
Scaffold a new Agent SDK agent
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
creating a new agent.
|
|
8
|
-
paths:
|
|
9
|
-
- packages/agent-serve/**/*
|
|
4
|
+
Scaffold a new Agent SDK agent: short AskQuestion interview, then
|
|
5
|
+
verify a turn and hand off to hillclimb. Use when creating a new
|
|
6
|
+
agent. Ask only what the invoking message left open.
|
|
10
7
|
---
|
|
11
8
|
|
|
12
9
|
# Create an Agent SDK agent
|
|
13
10
|
|
|
14
|
-
|
|
15
|
-
|
|
11
|
+
1. **Interview.** Two `AskQuestion` rounds, then a plan gate.
|
|
12
|
+
2. **Scaffold.** `agent-sdk init`, then shape the files.
|
|
13
|
+
3. **Verify.** `validate` / `info` / `call`, then a model turn.
|
|
14
|
+
4. **Channels.** Slack, GitHub, webhook, or schedule as chosen.
|
|
15
|
+
5. **Hillclimb.** `skills/hillclimb/SKILL.md`.
|
|
16
16
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
wiring, `mcp-auth` for host MCP OAuth, `github` for webhook-driven
|
|
22
|
-
agents, `evals` for the eval API, `ab` for live A/B metrics, `otel` for
|
|
23
|
-
OpenTelemetry export, `hillclimb` for iteration once the agent runs.
|
|
24
|
-
|
|
25
|
-
Read `framework-map/SKILL.md` first if you haven't; treat the package
|
|
26
|
-
`AGENTS.md` and `README.md` as ground truth for
|
|
27
|
-
framework behavior. Run the CLI with Node, never Bun (Bun corrupts harness
|
|
28
|
-
tool-result streams): use the installed `agent-sdk` bin, or from a source
|
|
29
|
-
checkout `pnpm exec tsx src/bin/agent-serve.ts <command> …` — written as
|
|
30
|
-
`agent-sdk …` below.
|
|
17
|
+
Read `skills/framework-map/SKILL.md` if you have not. CLI is
|
|
18
|
+
`agent-sdk`. Public docs:
|
|
19
|
+
`node_modules/@cursor/july/dist/docs/llms.txt` or `/docs/llms.txt`
|
|
20
|
+
on a running host.
|
|
31
21
|
|
|
32
22
|
## Interview
|
|
33
23
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
- **Ask about decisions, not implementation.** Instructions wording, tool
|
|
48
|
-
names, file layout, env prefix, and eval shape are your job — derive them
|
|
49
|
-
from the purpose and show the result instead of asking.
|
|
50
|
-
- If `AskQuestion` is unavailable or the user says "just go", state the
|
|
51
|
-
defaults you are assuming in one short list and proceed.
|
|
52
|
-
|
|
53
|
-
### Round 1 — identity
|
|
24
|
+
`AskQuestion`: batched, recommended option first and labeled
|
|
25
|
+
`(recommended)`.
|
|
26
|
+
|
|
27
|
+
- Mine the invoking message. Target: two rounds, then files.
|
|
28
|
+
- Round 1 is identity. Round 2 is the surface, tailored by round 1.
|
|
29
|
+
- Multi-select for channels, MCP, capabilities. Other where a
|
|
30
|
+
custom answer is plausible.
|
|
31
|
+
- Ask decisions, not how to build it. File layout, tool names,
|
|
32
|
+
env prefix, and eval shape are yours.
|
|
33
|
+
- If `AskQuestion` is missing or the user skips, state defaults
|
|
34
|
+
in one list and proceed.
|
|
35
|
+
|
|
36
|
+
### Round 1: identity
|
|
54
37
|
|
|
55
38
|
| Question | Options |
|
|
56
39
|
| --- | --- |
|
|
57
|
-
| **Purpose** | chat assistant · PR / repo automation · domain assistant over APIs/tools · scheduled reporter · Other
|
|
58
|
-
| **Name** | 2–3
|
|
59
|
-
| **Location** | `./<slug>` (default) · another directory
|
|
40
|
+
| **Purpose** | chat assistant · PR / repo automation · domain assistant over APIs/tools · scheduled reporter · Other |
|
|
41
|
+
| **Name** | 2–3 slugs from the purpose + Other. Directory: `[A-Za-z0-9][A-Za-z0-9_-]*`, not `v1`/`playground`/`docs` |
|
|
42
|
+
| **Location** | `./<slug>` (default) · another directory |
|
|
60
43
|
|
|
61
|
-
### Round 2
|
|
44
|
+
### Round 2: surface
|
|
62
45
|
|
|
63
46
|
| Question | Options | Guidance |
|
|
64
47
|
| --- | --- | --- |
|
|
65
|
-
| **Runtime** | `local` (default) · `cloud` |
|
|
66
|
-
| **Model** | `grok-4.5`
|
|
67
|
-
| **Channels** (multi) | playground + HTTP
|
|
68
|
-
| **MCP
|
|
69
|
-
| **Capabilities** (multi) | server tools · agent tools
|
|
48
|
+
| **Runtime** | `local` (default) · `cloud` | Cloud needs `cloud.repos`. Approvals and sandbox seeds are local only. Recommend local unless the job needs a cloud checkout. |
|
|
49
|
+
| **Model** | `grok-4.5` + `effort=high`, `fast=true` · Other id | Params, not id suffixes: `{ id: "grok-4.5", params: [{ id: "effort", value: "high" }, { id: "fast", value: "true" }] }` |
|
|
50
|
+
| **Channels** (multi) | playground + HTTP (always on) · Slack · GitHub · custom webhook · schedule | Slack: `skills/setup-slack/SKILL.md`. GitHub: `skills/github/SKILL.md`. No Discord/Teams pack; offer custom webhook. |
|
|
51
|
+
| **MCP** (multi) | none · remote URL · local stdio · Cursor account connectors · Other | One file under `agent/mcp-connections/`. Account file is `account.ts`, never `cursor.ts`. Named local tools need `advertiseTools: true`. Privileged servers go in `agent/host-connections/`. |
|
|
52
|
+
| **Capabilities** (multi) | server tools · agent tools · skills · subagents · hooks · sandbox seeds · approvals · evals (recommend) | Always recommend one smoke eval. |
|
|
70
53
|
|
|
71
|
-
Close
|
|
72
|
-
|
|
73
|
-
write files after approval.
|
|
54
|
+
Close with the plan and file tree, then "Scaffold it" / "Adjust
|
|
55
|
+
something". Write files only after approval.
|
|
74
56
|
|
|
75
|
-
## Fill
|
|
57
|
+
## Fill the blanks
|
|
76
58
|
|
|
77
|
-
|
|
78
|
-
purpose, and reach for the proven shape instead of
|
|
79
|
-
inventing:
|
|
80
|
-
|
|
81
|
-
| Purpose looks like | Proven shape |
|
|
59
|
+
| Purpose | Shape |
|
|
82
60
|
| --- | --- |
|
|
83
|
-
| Slack chat
|
|
84
|
-
| PR review with host
|
|
85
|
-
| GitHub
|
|
86
|
-
| Domain assistant
|
|
87
|
-
| Wrap an existing pipeline
|
|
88
|
-
|
|
89
|
-
Defaults
|
|
90
|
-
|
|
91
|
-
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
(`~/.cache/agent-serve/<slug>`). Discovery does this for nested git
|
|
95
|
-
projects. Point cwd at a checkout only when the agent must inherit that
|
|
61
|
+
| Slack chat | `slackChannel({ envPrefix })` + suggested prompts. `agent-sdk slack create` mints the bot. Prefix: `skills/setup-slack/SKILL.md` |
|
|
62
|
+
| PR review with host prep | Channel `callTool` into a trimmed workspace `pr/` tree |
|
|
63
|
+
| GitHub automation | `githubChannel()` hooks, replay fixtures |
|
|
64
|
+
| Domain assistant | Server tools + one MCP connection + a skill |
|
|
65
|
+
| Wrap an existing pipeline | GitHub `{ task }` over a thin `agent/lib/` loop |
|
|
66
|
+
|
|
67
|
+
Defaults:
|
|
68
|
+
|
|
69
|
+
- Instructions + 1–2 tools + chosen channels + one smoke eval.
|
|
70
|
+
- `local.cwd` stays outside a monorepo checkout (default: a cache
|
|
71
|
+
directory under `~/.cache`) unless the agent must inherit that
|
|
96
72
|
tree.
|
|
97
|
-
-
|
|
98
|
-
|
|
99
|
-
-
|
|
100
|
-
`
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
summarizes, classifies, or composes replies is the model's job specified
|
|
113
|
-
badly, and it never comes off the books.
|
|
114
|
-
|
|
115
|
-
### The deterministic-path budget
|
|
116
|
-
|
|
117
|
-
TypeScript on the deterministic path is a permanent liability — every line
|
|
118
|
-
brings types, tests, restarts, and drift against the framework; prose is
|
|
119
|
-
cheap to change. Default every behavior to instructions/skills and promote
|
|
120
|
-
it to code only when it earns a row on the left:
|
|
121
|
-
|
|
122
|
-
| Code owns (earns its keep) | Model owns (keep out of code) |
|
|
73
|
+
- `agent/instructions.md`: identity, when to use which tool,
|
|
74
|
+
output shape.
|
|
75
|
+
- One tool per file. Prefer `execution: "server"` + zod
|
|
76
|
+
`inputSchema`. Gate side effects with `needsApproval: true`.
|
|
77
|
+
- Long prompts: `prompt\`…\`` / `prompt.lines\`…\`` from
|
|
78
|
+
`@cursor/july`.
|
|
79
|
+
- Host prep is for evidence (`callTool`, `workspaceFiles`), not
|
|
80
|
+
judgment, formatting, or replies.
|
|
81
|
+
|
|
82
|
+
### Deterministic-path budget
|
|
83
|
+
|
|
84
|
+
Default to instructions and skills. Promote to TypeScript only
|
|
85
|
+
when it earns the left column:
|
|
86
|
+
|
|
87
|
+
| Code owns | Model owns |
|
|
123
88
|
| --- | --- |
|
|
124
|
-
| Side-effect gates
|
|
125
|
-
|
|
|
126
|
-
| Auth
|
|
127
|
-
| Evidence seeding
|
|
128
|
-
| Hard
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
- **Prose in TS.** An `instructions.ts` / `prompt.ts` string builder is a
|
|
133
|
-
skill file wearing a trench coat — write `.md` and interpolate only what
|
|
134
|
-
is truly dynamic.
|
|
135
|
-
- **Inverted ratio.** A first cut whose `agent/lib/` + `tools/` dwarf its
|
|
136
|
-
instructions + skills (~3:1 or worse) is either wrapping an existing
|
|
137
|
-
deterministic pipeline (fine — keep the wrapper thin and say so) or
|
|
138
|
-
building a service with a vestigial model turn. Move procedure back into
|
|
139
|
-
prompts, or question whether this should be an agent at all.
|
|
89
|
+
| Side-effect gates (approve, deploy, post) | Human-facing output to a stated shape |
|
|
90
|
+
| Dedupe of external writes | Summarizing, classifying, wording |
|
|
91
|
+
| Auth and signature checks | What to do next from ground truth |
|
|
92
|
+
| Evidence seeding | Soft-failure retries |
|
|
93
|
+
| Hard API caps | Formatting under those caps |
|
|
94
|
+
|
|
95
|
+
Smells: prose builders in `.ts` (write `.md`); `agent/lib/` +
|
|
96
|
+
`tools/` dwarfing instructions + skills on a first cut.
|
|
140
97
|
|
|
141
98
|
## Scaffold
|
|
142
99
|
|
|
@@ -144,124 +101,67 @@ Two smells to check before scaffolding and again before hand-off:
|
|
|
144
101
|
agent-sdk init ./<slug>
|
|
145
102
|
```
|
|
146
103
|
|
|
147
|
-
`init` writes
|
|
148
|
-
`
|
|
149
|
-
`agent/instructions.md`, a demo `agent/tools/echo.ts`, and
|
|
150
|
-
`agent/hooks/memory.ts`. Existing files are left alone (`exist`). It then
|
|
151
|
-
runs `npm install`, and when the host is unsigned runs `login` and waits
|
|
152
|
-
before printing `cd` (when needed) and `agent-sdk dev`. Then shape it to the
|
|
153
|
-
plan:
|
|
154
|
-
|
|
155
|
-
1. `tsconfig.json` — init writes a strict, `noEmit` config covering `agent/`
|
|
156
|
-
and `evals/`; extend it only when the project needs more.
|
|
157
|
-
2. `agent/agent.ts` — chosen model/runtime via `defineAgent({...})`. Nested git checkouts get `local.cwd` under `~/.cache/agent-serve/<slug>` at discovery even if omitted. Add the `cloud: { repos: [...] }` block for cloud runtime.
|
|
158
|
-
3. `agent/instructions.md` — real instructions; delete or replace `echo.ts`
|
|
159
|
-
with the real tools.
|
|
160
|
-
4. Add the chosen `channels/`, `mcp-connections/`, `skills/`, `subagents/<id>/`
|
|
161
|
-
(needs `description`), `schedules/`, `hooks/`, `sandbox/workspace/`. For
|
|
162
|
-
Slack, do not hand-write the channel — `agent-sdk slack create --dir ./<slug> --name "<Name>"`
|
|
163
|
-
opens the dashboard wizard (env prefix from the directory name);
|
|
164
|
-
for a Slack app they own at api.slack.com, `agent-sdk slack init --manual --dir ./<slug> --name "<Name>"`;
|
|
165
|
-
customize `agent/channels/slack.ts` afterwards (e.g. `suggestedPrompts`).
|
|
166
|
-
5. `evals/evals.config.ts` — `defineEvalConfig({ maxConcurrency: 20 })`
|
|
167
|
-
(playground runs are memory-only unless you add `persistRuns`)
|
|
168
|
-
(`maxConcurrency` is required; hard-capped at 200 due to model provider
|
|
169
|
-
request limits) plus eval cases under `evals/**/*.eval.ts` — prefer one
|
|
170
|
-
suite file with `cases: [{ id, test }, …]` for related datapoints, or a
|
|
171
|
-
single `test` for one smoke. Assert `t.succeeded()` + `t.calledTool(...)`
|
|
172
|
-
(case id = file path, or `<fileId>/<case.id>`). Full assertion API and
|
|
173
|
-
fixture strategy: `evals/SKILL.md` (sibling skill).
|
|
174
|
-
|
|
175
|
-
Stick to deps the Agent SDK already
|
|
176
|
-
ships (`zod`, `@modelcontextprotocol/sdk`, `tsx`); a new npm dep needs its own
|
|
177
|
-
install story and is a smell for a first cut.
|
|
104
|
+
`init` writes the project, runs `npm install`, and may wait on
|
|
105
|
+
`login`. Then shape it:
|
|
178
106
|
|
|
179
|
-
|
|
107
|
+
1. `defineAgent` with the chosen model/runtime. Add
|
|
108
|
+
`cloud: { repos: [...] }` for cloud.
|
|
109
|
+
2. Real `instructions.md`. Replace `echo.ts`.
|
|
110
|
+
3. Chosen `channels/`, `mcp-connections/`, `skills/`,
|
|
111
|
+
`subagents/<id>/` (needs `description`), `schedules/`, `hooks/`.
|
|
112
|
+
Slack: `agent-sdk slack create --dir ./<slug> --name "<Name>"`,
|
|
113
|
+
not a hand-written channel. Manual Slack app:
|
|
114
|
+
`agent-sdk slack init --manual`.
|
|
115
|
+
4. `evals/evals.config.ts` with `maxConcurrency: 20` (required;
|
|
116
|
+
cap 200) plus `evals/**/*.eval.ts`. Assert `t.succeeded()` +
|
|
117
|
+
`t.calledTool(...)`. API: `skills/evals/SKILL.md`.
|
|
180
118
|
|
|
181
|
-
|
|
182
|
-
|
|
119
|
+
Stay on deps the Agent SDK already ships (`zod`,
|
|
120
|
+
`@modelcontextprotocol/sdk`, `tsx`).
|
|
121
|
+
|
|
122
|
+
## Verify
|
|
183
123
|
|
|
184
124
|
```bash
|
|
185
|
-
agent-sdk validate --dir ./<slug>
|
|
186
|
-
agent-sdk info --dir ./<slug> --json
|
|
187
|
-
agent-sdk call <tool> --dir ./<slug> --input '{…}'
|
|
125
|
+
agent-sdk validate --dir ./<slug>
|
|
126
|
+
agent-sdk info --dir ./<slug> --json
|
|
127
|
+
agent-sdk call <tool> --dir ./<slug> --input '{…}'
|
|
188
128
|
agent-sdk eval --dir ./<slug> --list
|
|
189
|
-
npx tsc --noEmit -p ./<slug>
|
|
129
|
+
npx tsc --noEmit -p ./<slug>
|
|
190
130
|
```
|
|
191
131
|
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
not satisfy `ToolExecuteResult` — returns must be JSON-shaped
|
|
195
|
-
(`ToolResultValue`), so use plain object literals or `type` aliases, not
|
|
196
|
-
`interface` types (their lack of an index signature fails assignability).
|
|
132
|
+
tsx does not typecheck. Tool `execute` must return JSON-shaped
|
|
133
|
+
values: object literals or `type` aliases, not `interface` types.
|
|
197
134
|
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
is forthcoming, finish every key-free check, confirm `run` fails with only
|
|
202
|
-
the clean API-key error, and hand these to the user as their next steps:
|
|
135
|
+
Model turns need `CURSOR_API_KEY`. Without one, finish the
|
|
136
|
+
key-free checks, confirm `run` fails with only the API-key error,
|
|
137
|
+
and hand these over:
|
|
203
138
|
|
|
204
139
|
```bash
|
|
205
|
-
agent-sdk run --dir ./<slug> --message "<fixture prompt>"
|
|
206
|
-
agent-sdk serve --dir ./<slug> --dev
|
|
207
|
-
#
|
|
140
|
+
agent-sdk run --dir ./<slug> --message "<fixture prompt>"
|
|
141
|
+
agent-sdk serve --dir ./<slug> --mode single --dev
|
|
142
|
+
# http://127.0.0.1:3000/playground
|
|
208
143
|
agent-sdk eval --dir ./<slug>
|
|
209
144
|
```
|
|
210
145
|
|
|
211
|
-
Serve only
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
## Hand off to hillclimbing
|
|
234
|
-
|
|
235
|
-
Scaffolding proves the agent runs; it says nothing about quality. Once the
|
|
236
|
-
smoke turn passes, propose the improvement loop — one last `AskQuestion` if
|
|
237
|
-
choices are open:
|
|
238
|
-
|
|
239
|
-
1. **Fixtures** — 1–3 fixed inputs worth optimizing against (a real PR URL, a
|
|
240
|
-
canonical Slack question, a saved webhook payload).
|
|
241
|
-
2. **Success criteria** — what "better" means first (correct tool choice,
|
|
242
|
-
fewer tool calls, faster wall time, output quality).
|
|
243
|
-
3. **Freeze line** — what must not change (API contract, output shape).
|
|
244
|
-
|
|
245
|
-
Those are exactly the preconditions in `skills/hillclimb/SKILL.md`; suggest
|
|
246
|
-
the user (or you) follow that skill with the fixtures you agreed on. Seed
|
|
247
|
-
the loop by pointing
|
|
248
|
-
at the smoke run's session (`.agent-serve/<slug>/sessions/<id>/events.ndjson`
|
|
249
|
-
or the trace under `.agent-serve/traces/`) as the baseline measurement — or,
|
|
250
|
-
when no API key was available, name the user's first real turn as the
|
|
251
|
-
baseline instead. For GitHub agents, snapshot replay fixtures now
|
|
252
|
-
(`agent-sdk github replay ... --dry-run --out fixtures/github`) so the
|
|
253
|
-
loop starts deterministic.
|
|
254
|
-
|
|
255
|
-
## Working agreements
|
|
256
|
-
|
|
257
|
-
- Interview → plan gate → scaffold → verify → channel setup → hillclimb
|
|
258
|
-
handoff. Do not skip verification because the files "look right".
|
|
259
|
-
- Fill blanks yourself. Re-ask only when an answer contradicts the framework,
|
|
260
|
-
such as approval-gated tools on the cloud runtime.
|
|
261
|
-
- Do not deploy, post to real Slack/GitHub surfaces beyond the smoke tests the
|
|
262
|
-
user asked for, or commit without being asked.
|
|
263
|
-
- Keep the first cut reviewable: if the plan grows past ~10 files, cut scope
|
|
264
|
-
and say what you deferred. Re-check the deterministic-path budget at
|
|
265
|
-
hand-off — behavior that crept into `lib/` during bring-up (formatting,
|
|
266
|
-
reply shaping, classification) goes back into instructions before you call
|
|
267
|
-
it done.
|
|
146
|
+
Serve only this agent's directory. Session files land in the
|
|
147
|
+
project state directory (`--state-root`). That is not harness cwd.
|
|
148
|
+
|
|
149
|
+
## Channel setup
|
|
150
|
+
|
|
151
|
+
- **Slack.** `skills/setup-slack/SKILL.md`
|
|
152
|
+
- **GitHub.** `skills/github/SKILL.md`
|
|
153
|
+
- **Custom webhook.** `POST /v1/channels/<id>/<route>`.
|
|
154
|
+
Loopback-only until you add `bearerAuth(...)`.
|
|
155
|
+
- **Schedules.** Never auto-fire under `--dev`.
|
|
156
|
+
`POST /v1/dev/schedules/<id>`.
|
|
157
|
+
|
|
158
|
+
## Hillclimb handoff
|
|
159
|
+
|
|
160
|
+
Once a smoke turn passes, agree fixtures, success criteria, and a
|
|
161
|
+
freeze line, then follow `skills/hillclimb/SKILL.md`. Seed from
|
|
162
|
+
the smoke session under the project state directory. GitHub:
|
|
163
|
+
snapshot `agent-sdk github replay ... --dry-run --out fixtures/github`.
|
|
164
|
+
|
|
165
|
+
Do not deploy or post to real Slack/GitHub beyond the smoke the
|
|
166
|
+
user asked for. If the plan grows past ~10 files, cut scope.
|
|
167
|
+
Re-check the budget at hand-off.
|