@cursor/july 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +40 -7
- package/README.md +33 -24
- package/dist/bin/agent-serve.d.ts +3 -1
- package/dist/bin/agent-serve.d.ts.map +1 -1
- package/dist/bin/agent-serve.js +466 -140
- package/dist/channels/github/api.d.ts.map +1 -1
- package/dist/channels/github/api.js +31 -14
- package/dist/channels/github/cursor-account.d.ts +43 -0
- package/dist/channels/github/cursor-account.d.ts.map +1 -0
- package/dist/channels/github/cursor-account.js +95 -0
- package/dist/channels/github/github-channel.d.ts.map +1 -1
- package/dist/channels/github/github-channel.js +46 -9
- package/dist/channels/github/index.d.ts +2 -2
- package/dist/channels/github/index.js +2 -2
- package/dist/channels/github/types.d.ts +17 -0
- package/dist/channels/github/types.d.ts.map +1 -1
- package/dist/channels/slack/slack-channel.d.ts +8 -2
- package/dist/channels/slack/slack-channel.d.ts.map +1 -1
- package/dist/channels/slack/slack-channel.js +8 -0
- package/dist/channels/slack/types.d.ts +24 -3
- package/dist/channels/slack/types.d.ts.map +1 -1
- package/dist/channels/slack/types.js +15 -1
- package/dist/docs/404.html +2 -2
- package/dist/docs/ab.html +8 -8
- package/dist/docs/assets/{ab.md.COdXkces.js → ab.md.BMCZ6Hd7.js} +3 -3
- package/dist/docs/assets/{ab.md.COdXkces.lean.js → ab.md.BMCZ6Hd7.lean.js} +1 -1
- package/dist/docs/assets/{app.DqfFEmJd.js → app.BR0RbIdx.js} +1 -1
- package/dist/docs/assets/chunks/@localSearchIndexroot.DtIOQzGj.js +1 -0
- package/dist/docs/assets/chunks/{VPLocalSearchBox.BaLEdS15.js → VPLocalSearchBox.qjsVTneI.js} +1 -1
- package/dist/docs/assets/chunks/{theme.CZRvu_0q.js → theme.2Kg8jIp_.js} +2 -2
- package/dist/docs/assets/{deployment.md.Dx1TYNk5.js → deployment.md.DTKwE15Z.js} +3 -3
- package/dist/docs/assets/{deployment.md.Dx1TYNk5.lean.js → deployment.md.DTKwE15Z.lean.js} +1 -1
- package/dist/docs/assets/{evals.md.DPZ_MAnI.js → evals.md.DAgEc_hL.js} +3 -3
- package/dist/docs/assets/{guides_agent-to-agent.md.CrtrsySy.js → guides_agent-to-agent.md.Bpzgq2Pq.js} +1 -1
- package/dist/docs/assets/{guides_cloud-runtime.md.CYlNMTNp.js → guides_cloud-runtime.md.gVzabdQL.js} +1 -1
- package/dist/docs/assets/{guides_github.md.DwbKhCeS.js → guides_github.md.DOOCpqsW.js} +11 -4
- package/dist/docs/assets/{guides_github.md.DwbKhCeS.lean.js → guides_github.md.DOOCpqsW.lean.js} +1 -1
- package/dist/docs/assets/{guides_human-in-the-loop.md.Dvuctx7s.js → guides_human-in-the-loop.md.DlUqsp1S.js} +2 -2
- package/dist/docs/assets/{guides_slack.md.bv41fHfW.js → guides_slack.md.CCwqHvSV.js} +4 -4
- package/dist/docs/assets/{guides_slack.md.bv41fHfW.lean.js → guides_slack.md.CCwqHvSV.lean.js} +1 -1
- package/dist/docs/assets/{guides_webhooks.md.hFTik3lf.js → guides_webhooks.md.B1EswtUu.js} +2 -2
- package/dist/docs/assets/index.md.m81y7TY7.js +20 -0
- package/dist/docs/assets/{index.md.BPKcj5AI.lean.js → index.md.m81y7TY7.lean.js} +1 -1
- package/dist/docs/assets/quickstart.md.BU6Iwi_9.js +204 -0
- package/dist/docs/assets/quickstart.md.BU6Iwi_9.lean.js +1 -0
- package/dist/docs/assets/{reference_agent-config.md.Bpd7HQwf.js → reference_agent-config.md.DrW2JUM8.js} +4 -4
- package/dist/docs/assets/{reference_agent-config.md.Bpd7HQwf.lean.js → reference_agent-config.md.DrW2JUM8.lean.js} +1 -1
- package/dist/docs/assets/{reference_channels.md.D7JTR03W.js → reference_channels.md.DdmiKgqf.js} +4 -4
- package/dist/docs/assets/{reference_channels.md.D7JTR03W.lean.js → reference_channels.md.DdmiKgqf.lean.js} +1 -1
- package/dist/docs/assets/{reference_cli.md.DA730zCu.js → reference_cli.md.Bv6pOxcF.js} +11 -6
- package/dist/docs/assets/{reference_cli.md.DA730zCu.lean.js → reference_cli.md.Bv6pOxcF.lean.js} +1 -1
- package/dist/docs/assets/{reference_connections.md.C3vNH_DE.js → reference_connections.md.zaEYCLHT.js} +1 -1
- package/dist/docs/assets/{reference_hooks.md.BCEc3MyM.js → reference_hooks.md.DyLVfE1O.js} +1 -1
- package/dist/docs/assets/{reference_hooks.md.BCEc3MyM.lean.js → reference_hooks.md.DyLVfE1O.lean.js} +1 -1
- package/dist/docs/assets/{reference_http-api.md.DBAahtdz.js → reference_http-api.md.Dx_nmDG6.js} +1 -1
- package/dist/docs/assets/{reference_instructions.md.BC05LEQ8.js → reference_instructions.md.CgoV-YEb.js} +9 -7
- package/dist/docs/assets/{reference_instructions.md.BC05LEQ8.lean.js → reference_instructions.md.CgoV-YEb.lean.js} +1 -1
- package/dist/docs/assets/{reference_schedules.md.D7qijxLk.js → reference_schedules.md.w_F2mXB6.js} +2 -2
- package/dist/docs/assets/{reference_skills.md.VQnlBT3Q.js → reference_skills.md.B_jHN7JL.js} +3 -3
- package/dist/docs/assets/{reference_skills.md.VQnlBT3Q.lean.js → reference_skills.md.B_jHN7JL.lean.js} +1 -1
- package/dist/docs/assets/{reference_subagents.md.CIRAVcPK.js → reference_subagents.md.zWAMNfi1.js} +1 -1
- package/dist/docs/assets/{reference_tools.md.DF5kwlt0.js → reference_tools.md.CqgJroI0.js} +2 -2
- package/dist/docs/assets/scaffolding-agents.md.C3pTrmoE.js +1 -0
- package/dist/docs/assets/scaffolding-agents.md.C3pTrmoE.lean.js +1 -0
- package/dist/docs/assets/storage.md.CVnInNiN.js +17 -0
- package/dist/docs/assets/storage.md.CVnInNiN.lean.js +1 -0
- package/dist/docs/building-with-agents.html +4 -4
- package/dist/docs/concepts.html +4 -4
- package/dist/docs/deployment.html +7 -7
- package/dist/docs/evals.html +7 -7
- package/dist/docs/guides/agent-to-agent.html +6 -6
- package/dist/docs/guides/cloud-runtime.html +5 -5
- package/dist/docs/guides/github.html +14 -7
- package/dist/docs/guides/human-in-the-loop.html +6 -6
- package/dist/docs/guides/slack.html +8 -8
- package/dist/docs/guides/webhooks.html +6 -6
- package/dist/docs/hashmap.json +1 -1
- package/dist/docs/hillclimbing.html +5 -5
- package/dist/docs/index.html +7 -7
- package/dist/docs/quickstart.html +195 -26
- package/dist/docs/reference/agent-config.html +7 -7
- package/dist/docs/reference/channels.html +8 -8
- package/dist/docs/reference/cli.html +14 -9
- package/dist/docs/reference/connections.html +5 -5
- package/dist/docs/reference/hooks.html +5 -5
- package/dist/docs/reference/http-api.html +6 -6
- package/dist/docs/reference/instructions.html +13 -11
- package/dist/docs/reference/playground.html +4 -4
- package/dist/docs/reference/project-layout.html +4 -4
- package/dist/docs/reference/schedules.html +7 -7
- package/dist/docs/reference/sessions.html +4 -4
- package/dist/docs/reference/skills.html +6 -6
- package/dist/docs/reference/subagents.html +6 -6
- package/dist/docs/reference/tools.html +7 -7
- package/dist/docs/scaffolding-agents.html +5 -5
- package/dist/docs/storage.html +41 -0
- package/dist/docs/troubleshooting.html +4 -4
- package/dist/index.d.ts +2 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -0
- package/dist/internal/chat-client.d.ts +29 -3
- package/dist/internal/chat-client.d.ts.map +1 -1
- package/dist/internal/chat-client.js +180 -27
- package/dist/internal/cli-ax.d.ts +75 -2
- package/dist/internal/cli-ax.d.ts.map +1 -1
- package/dist/internal/cli-ax.js +600 -52
- package/dist/internal/cli-cursor.d.ts.map +1 -1
- package/dist/internal/cli-cursor.js +8 -36
- package/dist/internal/cli-deploy.d.ts +108 -0
- package/dist/internal/cli-deploy.d.ts.map +1 -0
- package/dist/internal/cli-deploy.js +1009 -0
- package/dist/internal/cursor/backend-client.d.ts +10 -1
- package/dist/internal/cursor/backend-client.d.ts.map +1 -1
- package/dist/internal/cursor/backend-client.js +79 -1
- package/dist/internal/cursor/credentials.d.ts +9 -0
- package/dist/internal/cursor/credentials.d.ts.map +1 -1
- package/dist/internal/cursor/credentials.js +12 -0
- package/dist/internal/cursor/github-credentials.d.ts +44 -0
- package/dist/internal/cursor/github-credentials.d.ts.map +1 -0
- package/dist/internal/cursor/github-credentials.js +195 -0
- package/dist/internal/deploy-client.d.ts +184 -0
- package/dist/internal/deploy-client.d.ts.map +1 -0
- package/dist/internal/deploy-client.js +395 -0
- package/dist/internal/deploy-source.d.ts +35 -0
- package/dist/internal/deploy-source.d.ts.map +1 -0
- package/dist/internal/deploy-source.js +118 -0
- package/dist/internal/discovery.d.ts +11 -0
- package/dist/internal/discovery.d.ts.map +1 -1
- package/dist/internal/discovery.js +116 -19
- package/dist/internal/distribution.d.ts.map +1 -1
- package/dist/internal/distribution.js +1 -0
- package/dist/internal/eval-run-store.d.ts +22 -3
- package/dist/internal/eval-run-store.d.ts.map +1 -1
- package/dist/internal/eval-run-store.js +39 -20
- package/dist/internal/eval-runner.d.ts +2 -0
- package/dist/internal/eval-runner.d.ts.map +1 -1
- package/dist/internal/eval-runner.js +2 -0
- package/dist/internal/event-mapper.d.ts +54 -1
- package/dist/internal/event-mapper.d.ts.map +1 -1
- package/dist/internal/event-mapper.js +151 -41
- package/dist/internal/handleAgentServeTrigger.d.ts.map +1 -1
- package/dist/internal/handleAgentServeTrigger.js +10 -12
- package/dist/internal/host-platforms.d.ts +7 -2
- package/dist/internal/host-platforms.d.ts.map +1 -1
- package/dist/internal/host-platforms.js +15 -10
- package/dist/internal/hosting.d.ts +37 -0
- package/dist/internal/hosting.d.ts.map +1 -0
- package/dist/internal/hosting.js +67 -0
- package/dist/internal/init-project.d.ts +88 -1
- package/dist/internal/init-project.d.ts.map +1 -1
- package/dist/internal/init-project.js +221 -31
- package/dist/internal/install-cursor-skills.d.ts +64 -0
- package/dist/internal/install-cursor-skills.d.ts.map +1 -0
- package/dist/internal/install-cursor-skills.js +274 -0
- package/dist/internal/logs-client.d.ts +60 -0
- package/dist/internal/logs-client.d.ts.map +1 -0
- package/dist/internal/logs-client.js +311 -0
- package/dist/internal/open-browser.d.ts +5 -0
- package/dist/internal/open-browser.d.ts.map +1 -0
- package/dist/internal/open-browser.js +34 -0
- package/dist/internal/playground/toolchain.d.ts.map +1 -1
- package/dist/internal/playground/toolchain.js +16 -11
- package/dist/internal/playground-proxy.d.ts +69 -0
- package/dist/internal/playground-proxy.d.ts.map +1 -0
- package/dist/internal/playground-proxy.js +468 -0
- package/dist/internal/prompt-context.d.ts +35 -0
- package/dist/internal/prompt-context.d.ts.map +1 -0
- package/dist/internal/prompt-context.js +71 -0
- package/dist/internal/reminder-runner.d.ts +7 -0
- package/dist/internal/reminder-runner.d.ts.map +1 -1
- package/dist/internal/reminder-runner.js +50 -6
- package/dist/internal/reminder-store.d.ts +2 -0
- package/dist/internal/reminder-store.d.ts.map +1 -1
- package/dist/internal/reminder-store.js +18 -0
- package/dist/internal/request-headers.d.ts +11 -0
- package/dist/internal/request-headers.d.ts.map +1 -0
- package/dist/internal/request-headers.js +14 -0
- package/dist/internal/resolve-prod-target.d.ts +42 -0
- package/dist/internal/resolve-prod-target.d.ts.map +1 -0
- package/dist/internal/resolve-prod-target.js +111 -0
- package/dist/internal/run-client.d.ts +2 -2
- package/dist/internal/run-client.d.ts.map +1 -1
- package/dist/internal/run-client.js +38 -23
- package/dist/internal/server.d.ts.map +1 -1
- package/dist/internal/server.js +182 -45
- package/dist/internal/session-engine.d.ts +50 -1
- package/dist/internal/session-engine.d.ts.map +1 -1
- package/dist/internal/session-engine.js +267 -23
- package/dist/internal/sessions-client.d.ts +21 -0
- package/dist/internal/sessions-client.d.ts.map +1 -0
- package/dist/internal/sessions-client.js +168 -0
- package/dist/internal/storage-coordinator.d.ts +139 -0
- package/dist/internal/storage-coordinator.d.ts.map +1 -0
- package/dist/internal/storage-coordinator.js +499 -0
- package/dist/internal/stream-progress.d.ts.map +1 -1
- package/dist/internal/stream-progress.js +31 -3
- package/dist/internal/terminal-style.d.ts +22 -0
- package/dist/internal/terminal-style.d.ts.map +1 -0
- package/dist/internal/terminal-style.js +49 -0
- package/dist/internal/trajectory.d.ts +10 -5
- package/dist/internal/trajectory.d.ts.map +1 -1
- package/dist/internal/trajectory.js +82 -10
- package/dist/internal/workspace.d.ts +11 -0
- package/dist/internal/workspace.d.ts.map +1 -1
- package/dist/internal/workspace.js +38 -4
- package/dist/playground/assets/cursor-icons-outline-BxTT_FVJ.woff2 +0 -0
- package/dist/playground/assets/index-D-vo2lV_.css +1 -0
- package/dist/playground/assets/index-x60b9q2j.js +312 -0
- package/dist/playground/index.html +2 -2
- package/dist/storage.d.ts +204 -0
- package/dist/storage.d.ts.map +1 -0
- package/dist/storage.js +153 -0
- package/dist/types.d.ts +70 -4
- package/dist/types.d.ts.map +1 -1
- package/docs/README.md +3 -2
- package/docs/guides/github.md +43 -8
- package/docs/guides/slack.md +1 -1
- package/docs/quickstart.md +350 -57
- package/docs/reference/cli.md +51 -3
- package/docs/reference/instructions.md +8 -6
- package/docs/scaffolding-agents.md +1 -1
- package/docs/storage.md +98 -0
- package/package.json +10 -1
- package/skills/ab/SKILL.md +8 -8
- package/skills/create-agent/SKILL.md +28 -22
- package/skills/debug/SKILL.md +11 -11
- package/skills/evals/SKILL.md +16 -16
- package/skills/framework-map/SKILL.md +6 -6
- package/skills/github/SKILL.md +22 -14
- package/skills/hillclimb/SKILL.md +10 -10
- package/skills/setup-slack/SKILL.md +13 -13
- package/src/bin/agent-serve.ts +552 -179
- package/src/channels/github/api.ts +42 -23
- package/src/channels/github/cursor-account.ts +165 -0
- package/src/channels/github/github-channel.ts +66 -6
- package/src/channels/github/index.ts +2 -2
- package/src/channels/github/types.ts +19 -0
- package/src/channels/slack/slack-channel.ts +17 -3
- package/src/channels/slack/types.ts +44 -3
- package/src/index.ts +13 -0
- package/src/internal/chat-client.ts +252 -37
- package/src/internal/cli-ax.ts +722 -72
- package/src/internal/cli-cursor.ts +14 -42
- package/src/internal/cli-deploy.ts +1319 -0
- package/src/internal/cursor/backend-client.ts +103 -1
- package/src/internal/cursor/credentials.ts +14 -0
- package/src/internal/cursor/github-credentials.ts +248 -0
- package/src/internal/deploy-client.ts +632 -0
- package/src/internal/deploy-source.ts +133 -0
- package/src/internal/discovery.ts +141 -17
- package/src/internal/distribution.ts +1 -0
- package/src/internal/eval-run-store.ts +50 -20
- package/src/internal/eval-runner.ts +5 -0
- package/src/internal/event-mapper.ts +222 -42
- package/src/internal/handleAgentServeTrigger.ts +10 -12
- package/src/internal/host-platforms.ts +28 -11
- package/src/internal/hosting.ts +77 -0
- package/src/internal/init-project.ts +333 -51
- package/src/internal/install-cursor-skills.ts +327 -0
- package/src/internal/logs-client.ts +442 -0
- package/src/internal/open-browser.ts +41 -0
- package/src/internal/playground/toolchain.ts +15 -13
- package/src/internal/playground-proxy.ts +576 -0
- package/src/internal/prompt-context.ts +95 -0
- package/src/internal/reminder-runner.ts +50 -6
- package/src/internal/reminder-store.ts +21 -0
- package/src/internal/request-headers.ts +23 -0
- package/src/internal/resolve-prod-target.ts +150 -0
- package/src/internal/run-client.ts +39 -36
- package/src/internal/server.ts +223 -28
- package/src/internal/session-engine.ts +307 -10
- package/src/internal/sessions-client.ts +182 -0
- package/src/internal/storage-coordinator.ts +615 -0
- package/src/internal/stream-progress.ts +36 -2
- package/src/internal/terminal-style.ts +74 -0
- package/src/internal/trajectory.ts +91 -13
- package/src/internal/workspace.ts +42 -4
- package/src/storage.ts +325 -0
- package/src/types.ts +82 -8
- package/dist/docs/assets/chunks/@localSearchIndexroot.CcVk1uKq.js +0 -1
- package/dist/docs/assets/index.md.BPKcj5AI.js +0 -20
- package/dist/docs/assets/quickstart.md.tVPiGK_L.js +0 -35
- package/dist/docs/assets/quickstart.md.tVPiGK_L.lean.js +0 -1
- package/dist/docs/assets/scaffolding-agents.md.CyYfWGdc.js +0 -1
- package/dist/docs/assets/scaffolding-agents.md.CyYfWGdc.lean.js +0 -1
- package/dist/playground/assets/cursor-icons-outline-oY2V_mvK.woff2 +0 -0
- package/dist/playground/assets/index-1K-hG-7p.css +0 -1
- package/dist/playground/assets/index-FlWjhg3x.js +0 -79
- /package/dist/docs/assets/{evals.md.DPZ_MAnI.lean.js → evals.md.DAgEc_hL.lean.js} +0 -0
- /package/dist/docs/assets/{guides_agent-to-agent.md.CrtrsySy.lean.js → guides_agent-to-agent.md.Bpzgq2Pq.lean.js} +0 -0
- /package/dist/docs/assets/{guides_cloud-runtime.md.CYlNMTNp.lean.js → guides_cloud-runtime.md.gVzabdQL.lean.js} +0 -0
- /package/dist/docs/assets/{guides_human-in-the-loop.md.Dvuctx7s.lean.js → guides_human-in-the-loop.md.DlUqsp1S.lean.js} +0 -0
- /package/dist/docs/assets/{guides_webhooks.md.hFTik3lf.lean.js → guides_webhooks.md.B1EswtUu.lean.js} +0 -0
- /package/dist/docs/assets/{reference_connections.md.C3vNH_DE.lean.js → reference_connections.md.zaEYCLHT.lean.js} +0 -0
- /package/dist/docs/assets/{reference_http-api.md.DBAahtdz.lean.js → reference_http-api.md.Dx_nmDG6.lean.js} +0 -0
- /package/dist/docs/assets/{reference_schedules.md.D7qijxLk.lean.js → reference_schedules.md.w_F2mXB6.lean.js} +0 -0
- /package/dist/docs/assets/{reference_subagents.md.CIRAVcPK.lean.js → reference_subagents.md.zWAMNfi1.lean.js} +0 -0
- /package/dist/docs/assets/{reference_tools.md.DF5kwlt0.lean.js → reference_tools.md.CqgJroI0.lean.js} +0 -0
package/docs/storage.md
ADDED
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Storage"
|
|
3
|
+
description: "Point agentkit's durable storage at a backend you own with defineStorage."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Storage
|
|
7
|
+
|
|
8
|
+
agentkit owns durable storage for sessions, continuation tokens,
|
|
9
|
+
reminders, playground eval history, and live A/B samples. It chooses the
|
|
10
|
+
keys (under `agentkit/v1/`), when to read and write, and how to restore
|
|
11
|
+
after restart.
|
|
12
|
+
|
|
13
|
+
Keys have bounded length: caller-controlled segments (channel ids,
|
|
14
|
+
continuation tokens) are URI-encoded, and any segment past 256 encoded
|
|
15
|
+
bytes is replaced by its `sha256:…` digest — deterministically, so writes
|
|
16
|
+
and lookups always agree. Backends can rely on this instead of imposing
|
|
17
|
+
their own key-length caps (which would silently drop writes, since a
|
|
18
|
+
throwing `put` is at-most-once).
|
|
19
|
+
|
|
20
|
+
By default that storage lives under `--state-root` on local disk. Fine
|
|
21
|
+
for one machine; it does not survive replacing the host.
|
|
22
|
+
|
|
23
|
+
To keep the same framework storage across hosts, plug in a key-value
|
|
24
|
+
backend with `agent/storage.ts`. You provide `put` / `get` / `delete` /
|
|
25
|
+
`list`. agentkit does the rest.
|
|
26
|
+
|
|
27
|
+
```ts
|
|
28
|
+
// agent/storage.ts
|
|
29
|
+
import { defineStorage } from "@cursor/july/storage";
|
|
30
|
+
|
|
31
|
+
export default defineStorage({
|
|
32
|
+
put: (key, value) => db.upsert(key, value),
|
|
33
|
+
get: (key) => db.get(key),
|
|
34
|
+
delete: (key) => db.delete(key),
|
|
35
|
+
list: (prefix) => db.listByPrefix(prefix), // [{ key, value }], key order
|
|
36
|
+
});
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
> [!NOTE]
|
|
40
|
+
> Import paths here use `@cursor/july/storage`. On projects still
|
|
41
|
+
> using `@anysphere/agent-serve`, swap the import. See
|
|
42
|
+
> [Run the CLI](./README.md#run-the-cli) for the full rename table.
|
|
43
|
+
|
|
44
|
+
## Which methods to provide
|
|
45
|
+
|
|
46
|
+
| Method | Required | Role |
|
|
47
|
+
| --- | --- | --- |
|
|
48
|
+
| `put` | Yes | Write or update a value |
|
|
49
|
+
| `get` | For restore | Look up one key |
|
|
50
|
+
| `list` | For restore | Return entries under a prefix, in key order |
|
|
51
|
+
| `delete` | For cleanup | Remove a key |
|
|
52
|
+
|
|
53
|
+
A throwing `put` is logged and dropped. It never fails a turn. When
|
|
54
|
+
resolving a missing continuation token, a throwing `get` fails the
|
|
55
|
+
follow-up so a store outage does not open a new session. Return
|
|
56
|
+
`undefined` only for a real miss.
|
|
57
|
+
|
|
58
|
+
## Policy
|
|
59
|
+
|
|
60
|
+
Two knobs change behavior:
|
|
61
|
+
|
|
62
|
+
```ts
|
|
63
|
+
export default defineStorage({
|
|
64
|
+
policy: {
|
|
65
|
+
// Batch event writes while the session is busy (default: once per turn)
|
|
66
|
+
debounceMs: 30_000,
|
|
67
|
+
// Cap how much loads at serve start, or "off" to restore on demand
|
|
68
|
+
restore: { maxSessions: 500 },
|
|
69
|
+
},
|
|
70
|
+
// put / get / delete / list …
|
|
71
|
+
});
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
| Knob | Default | Meaning |
|
|
75
|
+
| --- | --- | --- |
|
|
76
|
+
| `debounceMs` | unset (once per turn) | Wait this long after activity before writing event batches |
|
|
77
|
+
| `restore` | caps below | How much to load at serve start |
|
|
78
|
+
| `restore.maxSessions` | `1000` | Max sessions loaded at serve start |
|
|
79
|
+
| `restore.maxAgeMs` | 30 days | Skip older sessions at serve start |
|
|
80
|
+
| `restore.maxTotalBytes` | 1 GiB | Stop loading once this budget is reached |
|
|
81
|
+
|
|
82
|
+
Set `restore: "off"` on high-traffic hosts. Sessions then load when a
|
|
83
|
+
follow-up arrives instead of at startup.
|
|
84
|
+
|
|
85
|
+
## Restore after restart
|
|
86
|
+
|
|
87
|
+
With `get` and `list`, serve can rebuild local state from your store:
|
|
88
|
+
|
|
89
|
+
- At startup, agentkit loads recent sessions up to the restore caps.
|
|
90
|
+
Local disk wins when both sides have the same session. Reminders
|
|
91
|
+
hydrate the same way into `--state-root/reminders`.
|
|
92
|
+
- On demand, a missing continuation token resolves through the store
|
|
93
|
+
and resumes that session.
|
|
94
|
+
- Playground eval history and A/B aggregates can load from the same
|
|
95
|
+
sink.
|
|
96
|
+
|
|
97
|
+
A turn in flight at crash time is not replayed. The next follow-up
|
|
98
|
+
resumes from the last flushed state.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@cursor/july",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
4
|
"description": "(early alpha) Filesystem-first framework for defining Cursor agents as markdown and TypeScript and serving them over channels with the Cursor SDK.",
|
|
5
5
|
"license": "SEE LICENSE IN LICENSE.md",
|
|
6
6
|
"repository": {
|
|
@@ -117,6 +117,13 @@
|
|
|
117
117
|
"import": "./dist/ab.js",
|
|
118
118
|
"default": "./dist/ab.js"
|
|
119
119
|
},
|
|
120
|
+
"./storage": {
|
|
121
|
+
"anysphere-source": "./src/storage.ts",
|
|
122
|
+
"bun": "./src/storage.ts",
|
|
123
|
+
"types": "./dist/storage.d.ts",
|
|
124
|
+
"import": "./dist/storage.js",
|
|
125
|
+
"default": "./dist/storage.js"
|
|
126
|
+
},
|
|
120
127
|
"./package.json": {
|
|
121
128
|
"anysphere-source": "./package.json",
|
|
122
129
|
"bun": "./package.json",
|
|
@@ -167,10 +174,12 @@
|
|
|
167
174
|
"@tanstack/react-virtual": "3.13.23",
|
|
168
175
|
"@types/ms": "^2.1.0",
|
|
169
176
|
"@types/node": "catalog:",
|
|
177
|
+
"@types/pg": "^8.16.0",
|
|
170
178
|
"@types/react": "^19.0.0",
|
|
171
179
|
"@types/react-dom": "^19.0.0",
|
|
172
180
|
"@typescript/native-preview": "7.0.0-dev.20260701.1",
|
|
173
181
|
"@vitejs/plugin-react": "^5.1.2",
|
|
182
|
+
"pg": "^8.18.0",
|
|
174
183
|
"pretty-ms": "^9.2.0",
|
|
175
184
|
"react": "^19.2.1",
|
|
176
185
|
"react-dom": "^19.2.1",
|
package/skills/ab/SKILL.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: agentkit-ab
|
|
3
3
|
description: >-
|
|
4
4
|
Author defineAB live A/B metrics plug-ins under agent/ab. Splits traffic into
|
|
5
5
|
sticky variants and calls onSample with performance metrics as the agent runs
|
|
@@ -7,11 +7,11 @@ description: >-
|
|
|
7
7
|
comparison; use defineEval for regression gates.
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
-
#
|
|
10
|
+
# agentkit A/B metrics (`defineAB`)
|
|
11
11
|
|
|
12
12
|
`defineAB` is a **live metrics plug-in**. At session creation the engine runs
|
|
13
13
|
`split` and appends durable `ab.assigned` events; the collector folds the
|
|
14
|
-
session event stream and calls `onSample`. There is **no** `
|
|
14
|
+
session event stream and calls `onSample`. There is **no** `agentkit ab`
|
|
15
15
|
CLI and **no** assertion API.
|
|
16
16
|
|
|
17
17
|
Human-facing reference: [`docs/ab.md`](../../docs/ab.md).
|
|
@@ -20,13 +20,13 @@ Human-facing reference: [`docs/ab.md`](../../docs/ab.md).
|
|
|
20
20
|
| --- | --- | --- |
|
|
21
21
|
| Job | Regression gates on frozen fixtures | Collect metrics on live runs |
|
|
22
22
|
| Location | `evals/**/*.eval.ts` | `agent/ab.ts` or `agent/ab/<name>.ts` |
|
|
23
|
-
| How it runs | `
|
|
23
|
+
| How it runs | `agentkit eval` | Automatically under `serve` / `run` |
|
|
24
24
|
| Driver | `t.send` + gates | `split` → `ab.assigned` + `onSample` |
|
|
25
25
|
|
|
26
26
|
## Authoring
|
|
27
27
|
|
|
28
28
|
```ts
|
|
29
|
-
import { defineAB, splitBySessionHash, splitIf } from "@
|
|
29
|
+
import { defineAB, splitBySessionHash, splitIf } from "@cursor/july/ab";
|
|
30
30
|
|
|
31
31
|
export default defineAB({
|
|
32
32
|
name: "concise-instructions",
|
|
@@ -72,7 +72,7 @@ import {
|
|
|
72
72
|
defineABConfig,
|
|
73
73
|
persistABSamplesToDir,
|
|
74
74
|
persistABSnapshotsToDir,
|
|
75
|
-
} from "@
|
|
75
|
+
} from "@cursor/july/ab";
|
|
76
76
|
|
|
77
77
|
export default defineABConfig({
|
|
78
78
|
// maxPlaygroundSessions: 200, // optional; default 200; A/Bs tab / GET /v1/abs only
|
|
@@ -101,9 +101,9 @@ mirror, not a metrics store. After park/restart it replays `events.ndjson` to
|
|
|
101
101
|
rebuild counters without re-firing `onSample`.
|
|
102
102
|
|
|
103
103
|
**Evals are separate:** the eval harness creates sessions with
|
|
104
|
-
`purpose: "eval"` (playground Evals / `
|
|
104
|
+
`purpose: "eval"` (playground Evals / `agentkit eval`). Those skip
|
|
105
105
|
enrollment entirely — no `ab.assigned`, no `onSample`, omitted from
|
|
106
|
-
`GET /v1/abs`. Ordinary chat / `
|
|
106
|
+
`GET /v1/abs`. Ordinary chat / `agentkit run` / Slack stay `"live"`.
|
|
107
107
|
Do not use `splitIf` to filter evals; the framework already does.
|
|
108
108
|
|
|
109
109
|
After enrollment, arms are on `session.abs` (experiment → variant | `null`)
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: agentkit-create-agent
|
|
3
3
|
description: >-
|
|
4
|
-
Scaffold a new
|
|
4
|
+
Scaffold a new agentkit agent through a guided AskQuestion interview —
|
|
5
5
|
purpose, name, runtime, model, channels, MCP connections, capabilities —
|
|
6
6
|
then verify it serves and hand off to hillclimbing. Read this skill when
|
|
7
7
|
creating a new agent.
|
|
@@ -9,7 +9,7 @@ paths:
|
|
|
9
9
|
- packages/agent-serve/**/*
|
|
10
10
|
---
|
|
11
11
|
|
|
12
|
-
# Create an
|
|
12
|
+
# Create an agentkit agent
|
|
13
13
|
|
|
14
14
|
Stand up one new agent project via a short interview, scaffold it, get
|
|
15
15
|
channels working
|
|
@@ -21,9 +21,9 @@ wiring, `github` for webhook-driven agents, `evals` for the eval API,
|
|
|
21
21
|
Read `framework-map/SKILL.md` first if you haven't; treat the package
|
|
22
22
|
`AGENTS.md` and `README.md` as ground truth for
|
|
23
23
|
framework behavior. Run the CLI with Node, never Bun (Bun corrupts harness
|
|
24
|
-
tool-result streams): use the installed `
|
|
24
|
+
tool-result streams): use the installed `agentkit` bin, or from a source
|
|
25
25
|
checkout `pnpm exec tsx src/bin/agent-serve.ts <command> …` — written as
|
|
26
|
-
`
|
|
26
|
+
`agentkit …` below.
|
|
27
27
|
|
|
28
28
|
## Interview
|
|
29
29
|
|
|
@@ -93,7 +93,7 @@ Defaults that make first cuts good:
|
|
|
93
93
|
(script) only when the tool must run where the agent runs (or runtime is
|
|
94
94
|
cloud). Gate side-effecting tools with `needsApproval: true`.
|
|
95
95
|
- **Env prefix** for Slack tokens = upper-snake slug (`my-agent` →
|
|
96
|
-
`MY_AGENT_SLACK_*`); `
|
|
96
|
+
`MY_AGENT_SLACK_*`); `agentkit slack init` derives it for you.
|
|
97
97
|
- **Host prep beats model wandering:** when the purpose has a deterministic
|
|
98
98
|
setup step (fetch a PR, seed files), do it in the channel handler via
|
|
99
99
|
`callTool` / `workspaceFiles` rather than instructing the model to do it.
|
|
@@ -101,21 +101,27 @@ Defaults that make first cuts good:
|
|
|
101
101
|
## Scaffold
|
|
102
102
|
|
|
103
103
|
```bash
|
|
104
|
-
|
|
104
|
+
agentkit init ./<slug>
|
|
105
105
|
```
|
|
106
106
|
|
|
107
|
-
`init` writes `package.json
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
107
|
+
`init` writes `package.json` (with `typescript` dev deps and a
|
|
108
|
+
`check` script), `tsconfig.json`, `agent/agent.ts`, `agent/instructions.md`,
|
|
109
|
+
a demo `agent/tools/echo.ts`, and empty capability folders
|
|
110
|
+
(`skills/`, `channels/`, `evals/`, …) each with a `.gitkeep`; existing
|
|
111
|
+
files are left alone (`exist`) and missing ones are filled in. It then
|
|
112
|
+
runs `npm install`, and when the host is unsigned runs `login` and waits
|
|
113
|
+
before printing `cd` (when needed) and bare `dev`. Then shape it to the
|
|
114
|
+
plan:
|
|
115
|
+
|
|
116
|
+
1. `tsconfig.json` — init writes a strict, `noEmit` config covering `agent/`
|
|
117
|
+
and `evals/`; extend it only when the project needs more.
|
|
112
118
|
2. `agent/agent.ts` — chosen model/runtime via `defineAgent({...})`; add the
|
|
113
119
|
`cloud: { repos: [...] }` block for cloud runtime.
|
|
114
120
|
3. `agent/instructions.md` — real instructions; delete or replace `echo.ts`
|
|
115
121
|
with the real tools.
|
|
116
122
|
4. Add the chosen `channels/`, `mcp-connections/`, `skills/`, `subagents/<id>/`
|
|
117
123
|
(needs `description`), `schedules/`, `hooks/`, `sandbox/workspace/`. For
|
|
118
|
-
Slack, do not hand-write the channel: `
|
|
124
|
+
Slack, do not hand-write the channel: `agentkit slack init --dir
|
|
119
125
|
./<slug> --name "<Name>"` generates `agent/channels/slack.ts` (with
|
|
120
126
|
the env prefix) plus manifests and `env.example`; customize the generated
|
|
121
127
|
file (e.g. `suggestedPrompts`) afterwards.
|
|
@@ -128,7 +134,7 @@ it to the plan:
|
|
|
128
134
|
(case id = file path, or `<fileId>/<case.id>`). Full assertion API and
|
|
129
135
|
fixture strategy: `evals/SKILL.md` (sibling skill).
|
|
130
136
|
|
|
131
|
-
Stick to deps
|
|
137
|
+
Stick to deps agentkit already
|
|
132
138
|
ships (`zod`, `@modelcontextprotocol/sdk`, `tsx`); a new npm dep needs its own
|
|
133
139
|
install story and is a smell for a first cut.
|
|
134
140
|
|
|
@@ -138,10 +144,10 @@ No API key needed for the structural half — run these first and fix every
|
|
|
138
144
|
error diagnostic:
|
|
139
145
|
|
|
140
146
|
```bash
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
147
|
+
agentkit validate --dir ./<slug> # zero errors (warnings explain runtime mismatches)
|
|
148
|
+
agentkit info --dir ./<slug> --json # surface matches the plan
|
|
149
|
+
agentkit call <tool> --dir ./<slug> --input '{…}' # server tools, deterministic, no model
|
|
150
|
+
agentkit eval --dir ./<slug> --list
|
|
145
151
|
npx tsc --noEmit -p ./<slug> # or the project's own type-check task
|
|
146
152
|
```
|
|
147
153
|
|
|
@@ -158,10 +164,10 @@ is forthcoming, finish every key-free check, confirm `run` fails with only
|
|
|
158
164
|
the clean API-key error, and hand these to the user as their next steps:
|
|
159
165
|
|
|
160
166
|
```bash
|
|
161
|
-
|
|
162
|
-
|
|
167
|
+
agentkit run --dir ./<slug> --message "<fixture prompt>" # JSON trajectory
|
|
168
|
+
agentkit serve --dir ./<slug> --dev
|
|
163
169
|
# playground: http://127.0.0.1:3000/<slug>/playground
|
|
164
|
-
|
|
170
|
+
agentkit eval --dir ./<slug>
|
|
165
171
|
```
|
|
166
172
|
|
|
167
173
|
Serve only the new agent's directory during bring-up — pointing `serve` at a
|
|
@@ -204,7 +210,7 @@ at the smoke run's session (`.agent-serve/<slug>/sessions/<id>/events.ndjson`
|
|
|
204
210
|
or the trace under `.agent-serve/traces/`) as the baseline measurement — or,
|
|
205
211
|
when no API key was available, name the user's first real turn as the
|
|
206
212
|
baseline instead. For GitHub agents, snapshot replay fixtures now
|
|
207
|
-
(`
|
|
213
|
+
(`agentkit github replay ... --dry-run --out fixtures/github`) so the
|
|
208
214
|
loop starts deterministic.
|
|
209
215
|
|
|
210
216
|
## Working agreements
|
package/skills/debug/SKILL.md
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: agentkit-debug
|
|
3
3
|
description: >-
|
|
4
|
-
Diagnose a misbehaving
|
|
4
|
+
Diagnose a misbehaving agentkit agent, server, or playground — blank
|
|
5
5
|
playground, sessions missing, HMR not reloading, failing reads/retry
|
|
6
6
|
loops, webhook 401s, 403/409 responses, approvals or reminders lost,
|
|
7
|
-
orphaned processes. Use when an
|
|
7
|
+
orphaned processes. Use when an agentkit project runs but behaves
|
|
8
8
|
wrong locally.
|
|
9
9
|
---
|
|
10
10
|
|
|
11
|
-
# Debugging
|
|
11
|
+
# Debugging agentkit locally
|
|
12
12
|
|
|
13
13
|
Read `framework-map/SKILL.md` (sibling skill) first if you don't know the
|
|
14
14
|
project structure or session model. Everything below assumes Node/tsx (`pnpm exec tsx
|
|
@@ -18,7 +18,7 @@ src/bin/agent-serve.ts ...` in the everysphere monorepo).
|
|
|
18
18
|
|
|
19
19
|
```bash
|
|
20
20
|
# 1. Is discovery clean? serve refuses to start on error diagnostics.
|
|
21
|
-
|
|
21
|
+
agentkit validate --dir <project>
|
|
22
22
|
|
|
23
23
|
# 2. What is actually running, and where?
|
|
24
24
|
lsof -iTCP:3000 -sTCP:LISTEN; lsof -iTCP:5273 -sTCP:LISTEN
|
|
@@ -37,11 +37,11 @@ curl -sN 'localhost:3000/<slug>/v1/session/<id>/stream?startIndex=0' | head -50
|
|
|
37
37
|
|
|
38
38
|
| Symptom | Cause / fix |
|
|
39
39
|
| --- | --- |
|
|
40
|
-
| Playground blank or "no agents" even though Vite assets are built | The SPA is static; it needs the
|
|
41
|
-
| Edits under `playground/src` don't show up in the browser | You're on the **static** `dist/playground` bundle, not HMR. Only `serve --dev` starts Vite HMR (`:5273`) and prints `playground
|
|
40
|
+
| Playground blank or "no agents" even though Vite assets are built | The SPA is static; it needs the agentkit backend on `:3000`. Start `serve` — building `dist/playground/` alone serves nothing. |
|
|
41
|
+
| Edits under `playground/src` don't show up in the browser | You're on the **static** `dist/playground` bundle, not HMR. Only `serve --dev` / `dev` starts Vite HMR (`:5273`) and prints that as `playground` — open that URL, not `:3000`. In the monorepo, `mise //packages/agent-serve:start` = all examples + multi-agent HMR; pin one slug with `AGENT_SERVE_BASE=/<slug>`. |
|
|
42
42
|
| Webhook / schedule sessions exist on disk but the playground session list is empty | The list shows the calling principal's sessions. `--dev` (loopback) or `--allow-anonymous` (trusted shared host) switches it to `includeAll`. Otherwise deep-link `/<slug>/playground?sessionId=ses_...` or read `sessions/` on disk. |
|
|
43
43
|
| Every built-in read/grep the model makes fails; turns crawl through retry loops | You ran the CLI under **Bun**. Kill it, rerun under Node/tsx. (`NGHTTP2_FRAME_SIZE_ERROR` in SDK logs is the tell.) |
|
|
44
|
-
| `gh webhook forward` / `
|
|
44
|
+
| `gh webhook forward` / `agentkit github forward` deliveries all 401 — but hook creation succeeded | `GITHUB_TOKEN`/`GH_TOKEN` in the env. The relay authenticates with the gh CLI login and rejects env tokens. `GITHUB_TOKEN= GH_TOKEN= agentkit github forward ...` |
|
|
45
45
|
| `Hook already exists` starting a forwarder | GitHub allows one forwarder per repo. Use one `github forward --dir <parent>` (it fans out to every matching channel) instead of N processes; kill stale forwarders. |
|
|
46
46
|
| Agent's answers reference monorepo rules / AGENTS.md it shouldn't know | Session workspace sits inside the monorepo, so the harness loaded ancestor config. `defineAgent({ local: { cwd } })` outside the repo, or `--state-root` under `/tmp`. |
|
|
47
47
|
| Port 3000/5273 in use; stray processes after crashes | `lsof -iTCP:3000 -sTCP:LISTEN`, kill the pids; also check companion processes (e.g. vite). |
|
|
@@ -49,11 +49,11 @@ curl -sN 'localhost:3000/<slug>/v1/session/<id>/stream?startIndex=0' | head -50
|
|
|
49
49
|
| Schedule / reminder never fires under `--dev` | Dev never auto-fires. `POST /<slug>/v1/dev/schedules/<id>` or `POST /<slug>/v1/dev/reminders/<id>` (list at `GET /v1/dev/reminders`). |
|
|
50
50
|
| Reminder disarmed after restart with `handler_lost_on_restart` | `run`-handler reminders are in-memory; re-arm them from the code path that created them (enroll hook / policy), or use prompt-based reminders. |
|
|
51
51
|
| `409` on a follow-up | Stale `continuationToken` (each accepted follow-up rotates it), busy session, or a task/schedule session (not followable). |
|
|
52
|
-
| `409 session_busy` on `
|
|
52
|
+
| `409 session_busy` on `agentkit call --session` | Session-bound deterministic calls serialize with model turns; wait or use an ephemeral call (drop `--session`). |
|
|
53
53
|
| `403` on stream/follow-up | Caller principal ≠ session owner. In dev, create and inspect with the same auth; beyond loopback pass `--bearer-token` and send it. |
|
|
54
54
|
| Works on localhost, 401/blocked through a tunnel or LAN | Default `localDevStrict()` auth only admits direct loopback, **rejects proxy-forwarding headers** (`X-Forwarded-For`, ...), and requires a loopback `Host`. Use `--bearer-token <secret>` (or authored `bearerAuth`) — `--allow-anonymous` only for trusted-network demos (and never with Cursor account MCP connections). |
|
|
55
55
|
| Channel route won't compile: body/query schema type error | `GET` requires a Zod `querySchema`, `POST`/`PUT`/`PATCH` a Zod `bodySchema` — plain JSON Schema objects don't type-check. Use `z.object({})` / `z.unknown()` for open surfaces. Empty POST bodies are coerced to `{}` before validation. |
|
|
56
|
-
| Slack channel prints `channel idle ... missing credentials` | Expected: tokens absent. Multi-agent needs `<PREFIX>_SLACK_BOT_TOKEN` + `<PREFIX>_SLACK_APP_TOKEN` per agent. `
|
|
56
|
+
| Slack channel prints `channel idle ... missing credentials` | Expected: tokens absent. Multi-agent needs `<PREFIX>_SLACK_BOT_TOKEN` + `<PREFIX>_SLACK_APP_TOKEN` per agent. `agentkit slack doctor --prefix <PREFIX>`. |
|
|
57
57
|
| Turn fails immediately with an API-key error | Model turns need `CURSOR_API_KEY`; everything structural (validate/info/call/serve bring-up) doesn't. |
|
|
58
58
|
| Server tools / skills / sandbox silently absent | Runtime is `cloud` — those are local-only. `validate` prints exactly this warning; read it. |
|
|
59
59
|
| `validate` clean, `run` works, CI typecheck fails | tsx never typechecked it. See invariant 3 in `framework-map/SKILL.md` (JSON-shaped tool returns; `type` not `interface`). |
|
|
@@ -67,5 +67,5 @@ curl -sN 'localhost:3000/<slug>/v1/session/<id>/stream?startIndex=0' | head -50
|
|
|
67
67
|
preempted it — that's the designed behavior, not a crash.
|
|
68
68
|
- Escapes outside the session workspace in read/grep paths mean the
|
|
69
69
|
harness is fighting your evidence layout, not that the model is broken.
|
|
70
|
-
- `
|
|
70
|
+
- `agentkit trajectory --events <file>` renders any saved NDJSON; the
|
|
71
71
|
playground "Open trace" does the same visually.
|
package/skills/evals/SKILL.md
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: agentkit-evals
|
|
3
3
|
description: >-
|
|
4
|
-
Author and run
|
|
4
|
+
Author and run agentkit defineEval cases (single- or multi-datapoint).
|
|
5
5
|
Use when writing, fixing, seeding, or hillclimbing evals; AskQuestion
|
|
6
6
|
whether to generate samples or upload data. Materialize API-backed
|
|
7
7
|
fixtures first. Live A/B metrics: defineAB (skills/ab), not defineEval.
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
-
#
|
|
10
|
+
# agentkit evals
|
|
11
11
|
|
|
12
12
|
Evals are the ratchet that makes iteration trustworthy: a fixed input, a
|
|
13
13
|
model turn, and gates over the recorded trajectory. They live at the
|
|
@@ -29,13 +29,13 @@ and does not replace `defineEval` gates.
|
|
|
29
29
|
(`evals/weather.eval.ts` + `{ id: "nyc" }` → `weather/nyc`).
|
|
30
30
|
|
|
31
31
|
```bash
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
32
|
+
agentkit eval --dir . --list # all datapoints
|
|
33
|
+
agentkit eval --dir . --json # run all
|
|
34
|
+
agentkit eval --dir . weather/nyc # one datapoint
|
|
35
|
+
agentkit eval --dir . weather # every datapoint in that file
|
|
36
|
+
agentkit eval --dir . weather forecast # several files
|
|
37
|
+
agentkit eval --dir . --tag smoke
|
|
38
|
+
agentkit eval --dir . --verbose
|
|
39
39
|
```
|
|
40
40
|
|
|
41
41
|
`eval` (like `run`) boots an ephemeral server on port 0 with a temp state
|
|
@@ -65,7 +65,7 @@ If `AskQuestion` is unavailable, ask the same fork in plain chat and wait.
|
|
|
65
65
|
|
|
66
66
|
Tell the user the **shape** of the data (see “The API” below) and **how to
|
|
67
67
|
upload it**. Do not invent cases until they provide files (or paste content
|
|
68
|
-
to write). After they add data, run `
|
|
68
|
+
to write). After they add data, run `agentkit eval --dir <project>
|
|
69
69
|
--list` to confirm discovery, then wire any missing gates.
|
|
70
70
|
|
|
71
71
|
If the uploaded data is **API-backed** (PR URLs, pinned SHAs, gold labels,
|
|
@@ -79,7 +79,7 @@ fixtures first** — see “When eval data needs API calls to render” below.
|
|
|
79
79
|
3. Optional fixtures beside the case or under `fixtures/`.
|
|
80
80
|
4. Tell the agent once the files are in place — or paste case bodies in chat
|
|
81
81
|
and ask the agent to write the file.
|
|
82
|
-
5. Verify: `
|
|
82
|
+
5. Verify: `agentkit eval --dir <project> --list` shows the new ids.
|
|
83
83
|
|
|
84
84
|
### Step 2b — user chose agent-generated samples
|
|
85
85
|
|
|
@@ -95,7 +95,7 @@ fixtures first** — see “When eval data needs API calls to render” below.
|
|
|
95
95
|
3. **Append** — prefer adding entries to an existing file’s `cases` array
|
|
96
96
|
when the suite fits; otherwise create a new `evals/<suite>.eval.ts` with
|
|
97
97
|
`cases`. Never overwrite or weaken an existing datapoint. Case ids must
|
|
98
|
-
not collide with `
|
|
98
|
+
not collide with `agentkit eval --dir <project> --list`.
|
|
99
99
|
4. Create `evals/evals.config.ts` if missing (`maxConcurrency: 20` is fine
|
|
100
100
|
for now; hard limit 200 for model provider request limits).
|
|
101
101
|
5. Re-list to confirm, then optionally run `--tag smoke` if you tagged any.
|
|
@@ -103,7 +103,7 @@ fixtures first** — see “When eval data needs API calls to render” below.
|
|
|
103
103
|
## The API
|
|
104
104
|
|
|
105
105
|
```ts
|
|
106
|
-
import { defineEval, includes, equals, satisfies } from "@
|
|
106
|
+
import { defineEval, includes, equals, satisfies } from "@cursor/july/evals";
|
|
107
107
|
|
|
108
108
|
// Single datapoint (case id = file path under evals/)
|
|
109
109
|
export default defineEval({
|
|
@@ -152,7 +152,7 @@ Project-wide defaults in `evals/evals.config.ts` (required once):
|
|
|
152
152
|
import {
|
|
153
153
|
defineEvalConfig,
|
|
154
154
|
persistEvalRunsToDir,
|
|
155
|
-
} from "@
|
|
155
|
+
} from "@cursor/july/evals";
|
|
156
156
|
|
|
157
157
|
export default defineEvalConfig({
|
|
158
158
|
maxConcurrency: 20, // required; hard-capped at 200 (model provider limits)
|
|
@@ -193,7 +193,7 @@ input instead — see fixtures).
|
|
|
193
193
|
| Agent surface | Fixture source |
|
|
194
194
|
| --- | --- |
|
|
195
195
|
| Chat / domain assistant | A canonical prompt string, chosen once and frozen |
|
|
196
|
-
| Tool-heavy | `
|
|
196
|
+
| Tool-heavy | `agentkit call <tool> --dir . --input '{...}'` first, then the prompt that should trigger it |
|
|
197
197
|
| GitHub webhook | `github replay … --dry-run --out fixtures/github` (see `packages/agent-serve/skills/github/SKILL.md`) |
|
|
198
198
|
| PR reviewer with host prep | A small fixed PR the team controls; assert on findings shape, not counts |
|
|
199
199
|
| Workspace-dependent | `workspaceFiles` in `t.send` — never machine-local paths |
|
|
@@ -1,22 +1,22 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: agentkit-framework-map
|
|
3
3
|
description: >-
|
|
4
|
-
Orientation for @
|
|
4
|
+
Orientation for @cursor/july — folder structure, local vs cloud
|
|
5
5
|
runtimes, sessions, state layout, and invariants (Node not Bun, root
|
|
6
6
|
evals/, typecheck after tsx). Use when creating, editing, or running
|
|
7
|
-
|
|
7
|
+
agentkit projects with agent/agent.ts and agent/instructions.md.
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
-
#
|
|
10
|
+
# agentkit framework map
|
|
11
11
|
|
|
12
|
-
`@
|
|
12
|
+
`@cursor/july` serves agents defined as ordinary files: markdown
|
|
13
13
|
for prose, TypeScript for typed behavior, under an `agent/` directory. The
|
|
14
14
|
framework discovers the files, compiles a manifest, and serves the agent
|
|
15
15
|
over HTTP/Slack/GitHub channels, with the Cursor SDK + harness as the
|
|
16
16
|
execution engine. Ground truth is the package `README.md` (full reference)
|
|
17
17
|
and `AGENTS.md` (coding-agent loop) — this skill is the map, not the spec.
|
|
18
18
|
|
|
19
|
-
Run the CLI as `
|
|
19
|
+
Run the CLI as `agentkit <cmd>`. In the everysphere monorepo there is no
|
|
20
20
|
installed bin — use:
|
|
21
21
|
|
|
22
22
|
```bash
|
package/skills/github/SKILL.md
CHANGED
|
@@ -1,26 +1,27 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: agentkit-github
|
|
3
3
|
description: >-
|
|
4
|
-
Build and test GitHub-webhook-driven
|
|
4
|
+
Build and test GitHub-webhook-driven agentkit agents: githubChannel
|
|
5
5
|
hooks (auth turn vs host task), signature modes, and testing tiers
|
|
6
6
|
(fixtures, github replay, github forward, serve --cursor-events). Use
|
|
7
7
|
when wiring GitHub events, replaying PR webhooks, or debugging delivery.
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
-
# GitHub channels in
|
|
10
|
+
# GitHub channels in agentkit
|
|
11
11
|
|
|
12
12
|
Author `agent/channels/github.ts` with `githubChannel()` from
|
|
13
|
-
`@
|
|
13
|
+
`@cursor/july/channels/github`. It mounts
|
|
14
14
|
`POST /<slug>/v1/channels/github` and publishes the events it dispatches on
|
|
15
15
|
(derived from declared hooks, or pinned via `webhookEvents`) so the CLI can
|
|
16
16
|
auto-derive forwarding.
|
|
17
17
|
|
|
18
18
|
```ts
|
|
19
|
-
import { defaultGitHubAuth, githubChannel } from "@
|
|
19
|
+
import { defaultGitHubAuth, githubChannel } from "@cursor/july/channels/github";
|
|
20
20
|
|
|
21
21
|
export default githubChannel({
|
|
22
22
|
botName: "my-agent", // or GITHUB_APP_SLUG; used to ignore self-comments
|
|
23
|
-
|
|
23
|
+
cursorAccount: { repos: ["owner/repo"] }, // permissions?: "read" | "pr-write" | "contents-write"
|
|
24
|
+
|
|
24
25
|
onPullRequest: (ctx, pr) =>
|
|
25
26
|
pr.action === "opened" ? { auth: defaultGitHubAuth(ctx) } : null,
|
|
26
27
|
onCheckSuite: (ctx, suite) =>
|
|
@@ -44,6 +45,13 @@ the model should reason about the event.
|
|
|
44
45
|
|
|
45
46
|
## Auth modes
|
|
46
47
|
|
|
48
|
+
- **`cursorAccount`** → the signed-in Cursor principal supplies both the SCM
|
|
49
|
+
event stream and a short-lived, repo-scoped GitHub credential. It works in
|
|
50
|
+
local dev (`agentkit login`) and hosted deployments (`CURSOR_API_KEY`).
|
|
51
|
+
`ctx.github`, `ctx.host.github`, and child `gh` commands share the lease.
|
|
52
|
+
Use `{ repos: ["owner/repo"] }`, or `true` with `serve --cursor-events
|
|
53
|
+
--repo owner/repo`. One host credential covers up to 20 repositories under
|
|
54
|
+
one GitHub owner; it cannot span owners.
|
|
47
55
|
- **Secret set** → route auth is `allowAll()` + `X-Hub-Signature-256`
|
|
48
56
|
verification before parsing; the HMAC becomes the request principal.
|
|
49
57
|
- **No secret** → `localDevStrict()` (loopback only) — except under
|
|
@@ -73,9 +81,9 @@ is fine here), synthesizes GitHub-shaped payloads, signs them when a secret
|
|
|
73
81
|
is configured, and POSTs them at the channel. Fully deterministic.
|
|
74
82
|
|
|
75
83
|
```bash
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
84
|
+
agentkit github replay https://github.com/owner/repo/pull/123 --dir <project>
|
|
85
|
+
agentkit github replay owner/repo#123 --dir <project> --events '*' --conclusion failure
|
|
86
|
+
agentkit github replay owner/repo#123 --dir <project> --events '*' --dry-run --out fixtures/github
|
|
79
87
|
```
|
|
80
88
|
|
|
81
89
|
`--events` defaults to `pull_request` (`'*'` = the channel's declared
|
|
@@ -88,9 +96,9 @@ Wraps `gh webhook forward`: registers a real webhook and relays deliveries
|
|
|
88
96
|
to loopback. URL + events auto-derived; repo inferred from the git remote.
|
|
89
97
|
|
|
90
98
|
```bash
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
GITHUB_TOKEN= GH_TOKEN=
|
|
99
|
+
agentkit github doctor --install # one-time: gh + cli/gh-webhook
|
|
100
|
+
agentkit github events --dir <parent> --json # what would be forwarded
|
|
101
|
+
GITHUB_TOKEN= GH_TOKEN= agentkit github forward --dir <project>
|
|
94
102
|
```
|
|
95
103
|
|
|
96
104
|
Traps, in order of hours lost:
|
|
@@ -113,11 +121,11 @@ Traps, in order of hours lost:
|
|
|
113
121
|
### 4. `serve --cursor-events` — pull from Cursor (`/v0/scm-events`)
|
|
114
122
|
|
|
115
123
|
```bash
|
|
116
|
-
|
|
124
|
+
agentkit serve --dir <project> --cursor-events --repo owner/repo
|
|
117
125
|
```
|
|
118
126
|
|
|
119
127
|
Reads the stream as the host's Cursor user, so it **requires sign-in**
|
|
120
|
-
(`
|
|
128
|
+
(`agentkit login` / `CURSOR_API_KEY`) — `serve` refuses to start signed
|
|
121
129
|
out instead of running a relay that can never receive events. No public URL
|
|
122
130
|
or repo admin. Registers before consuming; offset + consumer id under
|
|
123
131
|
`<state-root>/cursor-events/`. `CURSOR_API_BASE_URL` for a non-default
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: agentkit-hillclimb
|
|
3
3
|
description: >-
|
|
4
|
-
Iterate on a specific
|
|
4
|
+
Iterate on a specific agentkit agent by running the local server,
|
|
5
5
|
sending real requests, measuring correctness/efficiency, and proposing
|
|
6
6
|
the next harness or prompt change. Read this skill when improving an
|
|
7
7
|
existing agent.
|
|
@@ -9,9 +9,9 @@ paths:
|
|
|
9
9
|
- packages/agent-serve/**/*
|
|
10
10
|
---
|
|
11
11
|
|
|
12
|
-
#
|
|
12
|
+
# Agentkit hillclimb
|
|
13
13
|
|
|
14
|
-
Optimize one
|
|
14
|
+
Optimize one agentkit agent project
|
|
15
15
|
with a tight measure → change → remeasure loop. Do not redesign the
|
|
16
16
|
whole framework; climb one concrete failure mode at a time. Sibling skills:
|
|
17
17
|
`framework-map` (orientation), `evals` (the ratchet), `ab` (live variant
|
|
@@ -36,15 +36,15 @@ If those are missing, ask before editing.
|
|
|
36
36
|
A climb over a moving input is noise. Before round 1:
|
|
37
37
|
|
|
38
38
|
- **GitHub agents** — replay, don't wait for live events:
|
|
39
|
-
`
|
|
39
|
+
`agentkit github replay <pr> --dir <project>` synthesizes signed,
|
|
40
40
|
GitHub-shaped payloads from read access only, identically every run;
|
|
41
41
|
`--dry-run --out fixtures/github` snapshots them for offline replay.
|
|
42
42
|
- **Tool behavior** — isolate host tools from the model with
|
|
43
|
-
`
|
|
43
|
+
`agentkit call <tool> --dir <project> --input '{...}'` (schema-validated,
|
|
44
44
|
in-process, no turn). If the tool output is wrong, no prompt change fixes it.
|
|
45
|
-
- **Chat agents** — `
|
|
45
|
+
- **Chat agents** — `agentkit run --dir <project> --message "<fixture>"`
|
|
46
46
|
gives a JSON trajectory plus an NDJSON trace under
|
|
47
|
-
`<project>/.agent-serve/traces/`; `
|
|
47
|
+
`<project>/.agent-serve/traces/`; `agentkit trajectory --events <file>`
|
|
48
48
|
re-summarizes any saved trace.
|
|
49
49
|
- **Manual probes** — the playground Try modal on any channel route
|
|
50
50
|
remembers your last body per endpoint and has Copy curl; a successful Try
|
|
@@ -59,7 +59,7 @@ Repeat until the user stops or fixtures meet the criteria.
|
|
|
59
59
|
- Serve with Node (never Bun):
|
|
60
60
|
|
|
61
61
|
```bash
|
|
62
|
-
|
|
62
|
+
agentkit serve --dir <project> --dev
|
|
63
63
|
```
|
|
64
64
|
|
|
65
65
|
- Hit the agent the way a user would (channel HTTP, playground, Slack in
|
|
@@ -119,7 +119,7 @@ State the hypothesis in one sentence: *“If we X, metric Y should move because
|
|
|
119
119
|
- **Ratchet every kept change**: land an eval that would have failed before
|
|
120
120
|
it (tool-choice gate, `action.result` count bound, output-shape regex —
|
|
121
121
|
see `evals/SKILL.md`), and never weaken an existing gate to make a round
|
|
122
|
-
pass. `
|
|
122
|
+
pass. `agentkit eval --dir <project> --json` is the regression check
|
|
123
123
|
between rounds.
|
|
124
124
|
|
|
125
125
|
Present a short round report to the user before starting the next climb.
|