@andreprado/agentkit 0.1.0-alpha.18 → 0.1.0-alpha.19

package/src/templates/skills/agentkit-capsule/SKILL.md

@@ -24,9 +24,10 @@ Use this first inside an AgentKit Agent Capsule.
 - Add docs, FAQs, prices, policies, or CSV facts: `skills/agentkit-knowledge/SKILL.md`
 - Switch from `test/fake` to a real model provider: `skills/agentkit-provider/SKILL.md`
 - Add or run evals: `skills/agentkit-evals/SKILL.md`
+- Improve from hosted or local production evidence: `skills/agentkit-improve/SKILL.md`
 - Prepare hosted deploy: `skills/agentkit-deploy/SKILL.md`
 - Work with secrets, external APIs, public access, channels, or real data: `skills/agentkit-security/SKILL.md`
-- Add or debug website, Telegram, or WhatsApp channels: `skills/agentkit-channels/SKILL.md`
+- Add or debug website, Telegram, WhatsApp, or Discord channels: `skills/agentkit-channels/SKILL.md`
 - Investigate command failures: `skills/agentkit-troubleshooting/SKILL.md`
 
 For a compact docs map, read `references/docs-router.md`.
@@ -53,6 +54,12 @@ If you change behavior, add or update an eval and run:
 npm run eval
 ```
 
+If the change fixes production behavior, also collect or use an improve bundle and run:
+
+```sh
+npm run agentkit -- replay .agentkit/improve/<run> --against local
+```
+
 ## Rules
 
 - Keep `.env`, `.agentkit/`, and `node_modules/` out of commits.

package/src/templates/skills/agentkit-capsule/references/docs-router.md

@@ -8,8 +8,7 @@ Prefer the narrowest source that covers the task.
 - Evals and deterministic side-effect guards: `docs/guides/run-evals.md`
 - Real provider setup: `docs/guides/use-provider.md`
 - Deploy readiness, managed secrets, smoke checks, hosted UI: `docs/guides/prepare-deploy.md`
-- Channels: `docs/guides/add-channel.md`, `connect-telegram.md`, `connect-whatsapp-zapster.md`, `debug-channel.md`
+- Channels: `docs/guides/add-channel.md`, `connect-discord.md`, `connect-telegram.md`, `connect-whatsapp-zapster.md`, `debug-channel.md`
 - Security: `docs/guides/security-rules.md`
 
 Use `npm run agentkit -- docs full` only for a complete-contract audit, framework internals, or a behavior not covered by the task guide.
-

package/src/templates/skills/agentkit-channels/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: agentkit-channels
-description: Use when adding, connecting, testing, buffering, transcribing audio, or debugging AgentKit website, Telegram, or WhatsApp channels, including channel config helpers, provider secrets, webhook setup, channel tests, delivery logs, burst-message buffers, and transcription provider secrets.
+description: Use when adding, connecting, testing, buffering, transcribing audio, or debugging AgentKit website, Telegram, WhatsApp, or Discord channels, including channel config helpers, provider secrets, webhook setup, channel tests, delivery logs, burst-message buffers, and transcription provider secrets.
 ---
 
 # AgentKit Channels
@@ -49,6 +49,8 @@ V1 providers:
 
 Telegram voice notes are usually OGG/Opus, so use Groq for the default Telegram voice-note path in V1. Zapster audio needs a usable HTTPS Zapster media download URL in the webhook payload; arbitrary hosts are rejected before bearer auth is sent. Hosted channel creation requires the transcription secret automatically when the channel enables transcription. Webhooks only enqueue audio jobs; download and transcription run in the retryable channel worker before the agent run.
 
+Discord channels do not support audio in V1.
+
 ## Buffering
 
 Enable `buffer.mode: "debounce"` when clients send several short messages in a row and the agent should answer once.
@@ -78,6 +80,8 @@ npm run agentkit -- channels list
 npm run agentkit -- channels add website website-chat
 npm run agentkit -- channels connect telegram support-telegram
 npm run agentkit -- channels add whatsapp support-whatsapp --provider zapster
+npm run agentkit -- channels connect discord support-discord
+npm run agentkit -- channels connect discord server-discord --mode bot
 npm run agentkit -- channels doctor support-telegram
 npm run agentkit -- channels test support-telegram --message "hello"
 npm run agentkit -- channels test-audio support-telegram --fixture voice-note
@@ -87,6 +91,7 @@ npm run agentkit -- channels deliveries list support-telegram
 
 ## References
 
+- `references/discord.md`
 - `references/telegram.md`
 - `references/whatsapp-zapster.md`
 - `references/channel-buffering.md`

package/src/templates/skills/agentkit-channels/references/channel-debugging.md

@@ -46,7 +46,8 @@ Common errors:
 
 - `channel_not_found`: webhook URL points to an unknown channel. `channels test` is the official synthetic smoke and should resolve the current `.agentkit/deploy.json` channel.
 - `channel_secret_missing`: required hosted secret is not set.
-- `channel_signature_invalid`: webhook secret, token, or origin header mismatch.
+- `channel_signature_invalid`: webhook secret, token, origin header, or Discord public key mismatch.
+- Discord bot mode with no deliveries: confirm `--mode bot`, `DISCORD_BOT_TOKEN`, Message Content Intent, server install, and channel permissions for View Channel, Read Message History, and Send Messages.
 - `channel_payload_invalid`: malformed or unsupported provider payload.
 - `channel_event_duplicate`: provider retry; do not create a second run.
 - `audio_received`: audio message was accepted and normalized.

package/src/templates/skills/agentkit-channels/references/discord.md

@@ -0,0 +1,93 @@
+# Discord Channel
+
+Discord supports two modes:
+
+- `interactions`: slash commands through the Discord Interactions Endpoint URL.
+- `bot`: normal server messages through a Discord Bot user and Gateway connection.
+
+Use bot mode when the owner asks for the agent to answer without slash commands.
+
+## Slash Commands
+
+Required secret:
+
+```txt
+DISCORD_PUBLIC_KEY
+```
+
+Config:
+
+```ts
+discordChannel({ name: "support-discord" })
+```
+
+Commands:
+
+```sh
+agentkit deploy
+agentkit secret set DISCORD_PUBLIC_KEY --stdin
+agentkit channels connect discord support-discord
+agentkit channels test support-discord --message "hello"
+agentkit channels deliveries list support-discord
+```
+
+Paste the printed webhook URL into the application's Interactions Endpoint URL. Register a slash command with a string option named `message`, `text`, `prompt`, `question`, `query`, or `input`.
+
+## Bot Server Messages
+
+Required secret:
+
+```txt
+DISCORD_BOT_TOKEN
+```
+
+Config:
+
+```ts
+discordChannel({
+  name: "server-discord",
+  mode: "bot",
+})
+```
+
+Commands:
+
+```sh
+agentkit deploy
+agentkit secret set DISCORD_BOT_TOKEN --stdin
+agentkit channels connect discord server-discord --mode bot
+agentkit channels test server-discord --message "hello"
+agentkit channels deliveries list server-discord
+```
+
+In the Discord Developer Portal, open the Bot page, enable Message Content Intent, install the app into the server, and grant the bot `View Channel`, `Read Message History`, and `Send Messages` in the channels it should answer.
+
+Bot mode processes every readable non-bot text message Discord sends over the Gateway. Keep server permissions narrow if the agent should answer only in specific channels.
+
+## Buffering
+
+Buffer rapid Discord messages:
+
+```ts
+discordChannel({
+  name: "server-discord",
+  mode: "bot",
+  buffer: {
+    mode: "debounce",
+    quietWindowMs: 1500,
+    maxWaitMs: 8000,
+    maxMessages: 20,
+    maxChars: 8000,
+  },
+})
+```
+
+## Runtime Behavior
+
+Slash-command mode validates `X-Signature-Ed25519` and `X-Signature-Timestamp` against the raw body and `DISCORD_PUBLIC_KEY`, returns `type: 1` for Discord `PING`, acknowledges slash commands with a deferred response, then sends the final answer as an interaction follow-up.
+
+Bot mode connects to Discord Gateway with `DISCORD_BOT_TOKEN`, requests guild message and message-content intents, ignores bot-authored messages, normalizes `MESSAGE_CREATE`, and sends the final answer through `/channels/<channel_id>/messages`.
+
+All outbound Discord sends use `allowed_mentions: { parse: [] }`. Discord interaction tokens and bot tokens must not appear in delivery, queue, buffer, or doctor API responses.
+
+Discord audio is not supported in V1; do not configure `audio` on `discordChannel`.

package/src/templates/skills/agentkit-deploy/SKILL.md

@@ -26,6 +26,12 @@ npm run agentkit -- deploy --dry-run
 npm run agentkit -- deploy doctor
 ```
 
+If this deploy fixes production behavior, replay the collected evidence first:
+
+```sh
+npm run agentkit -- replay .agentkit/improve/<run> --against local
+```
+
 ## Hosted Flow
 
 ```sh

package/src/templates/skills/agentkit-evals/SKILL.md

@@ -16,9 +16,10 @@ Use evals after chat works and before claiming behavior is stable.
 5. Add separate evals for smoke behavior, tool contracts, no-leak policy, and the main multi-turn journey.
 6. For date-sensitive flows, set top-level `now` to an ISO timestamp with `Z` or a numeric offset so today, tomorrow, weekdays, and tool date validation stay deterministic.
 7. Use `turns` for full conversation flows, such as user asks, agent calls a tool, then the answer follows the required format.
-8. Convert real failures into regression tests with `npm run agentkit -- eval from-conversation <conversation-id>`.
-9. Do not put secrets or real client PII in evals.
-10. For tools that write externally, delete, charge money, send email, or call real customer systems, branch on `ctx.runtime.environment === "eval"` inside the registered tool.
+8. Convert local failures into regression tests with `npm run agentkit -- eval from-conversation <conversation-id>`.
+9. Convert hosted or local production evidence into regression tests with `npm run agentkit -- improve collect --deploy --since 24h`, then `npm run agentkit -- improve evals .agentkit/improve/<run>`.
+10. Do not put secrets or real client PII in evals.
+11. For tools that write externally, delete, charge money, send email, or call real customer systems, branch on `ctx.runtime.environment === "eval"` inside the registered tool.
 
 ## Assertion Shape
 
@@ -97,4 +98,12 @@ npm run typecheck
 npm run eval
 ```
 
+On Windows PowerShell, if `npm.ps1` is blocked with `PSSecurityException`, use `npm.cmd run typecheck` and `npm.cmd run eval`.
+
+When evals came from an improve bundle, also run:
+
+```sh
+npm run agentkit -- replay .agentkit/improve/<run> --against local
+```
+
 If eval output changes after switching providers, keep deterministic smoke evals on `test/fake` and add provider-specific evals separately.

package/src/templates/skills/agentkit-improve/SKILL.md

@@ -0,0 +1,86 @@
+---
+name: agentkit-improve
+description: Use when improving an AgentKit Agent Capsule from hosted or local production evidence, including collected traces, generated regression evals, local replay, channel failures, or post-deploy behavior fixes.
+---
+
+# AgentKit Improve
+
+Use this when production or local conversation evidence should drive a fix.
+
+## Boundary
+
+AgentKit Cloud exports evidence. The local coding agent edits the capsule, writes evals, runs replay, and deploys. Do not expect hosted AgentKit Cloud to change source files.
+
+## Workflow
+
+1. Collect evidence:
+
+```sh
+npm run agentkit -- improve collect --deploy --since 24h
+```
+
+Hosted conversation reads require a deploy access token even when a deploy manifest says `access.mode: "public"`. If collection fails with auth, refresh the local token:
+
+```sh
+npm run agentkit -- access token create agentkit-chat-ui --out .agentkit/chat-access-token.json
+```
+
+For one known conversation:
+
+```sh
+npm run agentkit -- improve collect --deploy --conversation-id <conversation-id>
+```
+
+2. Read the generated report:
+
+```txt
+.agentkit/improve/<run>/report.json
+.agentkit/improve/<run>/traces/
+```
+
+3. Generate regression evals:
+
+```sh
+npm run agentkit -- improve evals .agentkit/improve/<run>
+```
+
+4. Patch the capsule. Likely files:
+
+```txt
+prompts/instructions.md
+agentkit.config.ts
+tools/
+knowledge/
+evals/
+```
+
+5. Verify:
+
+```sh
+npm run typecheck
+npm run agentkit -- inspect
+npm run eval
+npm run agentkit -- replay .agentkit/improve/<run> --against local
+```
+
+6. Deploy only after local evals and replay pass:
+
+```sh
+npm run agentkit -- deploy --smoke "hello"
+```
+
+## Rules
+
+- Keep `.agentkit/improve/` out of commits.
+- Review generated evals before committing them.
+- AgentKit redacts common email, phone, bearer token, and key patterns in generated eval text, but you must still remove or generalize domain-specific client PII.
+- If a tool writes externally, deletes, charges money, sends email, or touches real customer systems, make the tool branch on `ctx.runtime.environment === "eval"`.
+- Do not paste secret values into reports, prompts, evals, or Knowledge files.
+- Do not try to read the hosted database directly. Use authenticated AgentKit CLI/API routes only.
+- If replay uses a real provider instead of `test/fake`, say that in the final response.
+
+## References
+
+- `references/trace-packets.md`
+- `references/replay-side-effects.md`
+- `templates/regression.eval.md`

package/src/templates/skills/agentkit-improve/references/replay-side-effects.md

@@ -0,0 +1,18 @@
+# Replay Side Effects
+
+Replay runs collected user turns through the local capsule with:
+
+```txt
+ctx.runtime.environment === "eval"
+ctx.runtime.invocation === "eval"
+```
+
+Tools still execute. Any tool that writes externally, deletes data, charges money, sends email, sends messages, or calls a real customer system must guard eval mode:
+
+```ts
+if (ctx.runtime.environment === "eval") {
+  return { ok: true, evalFixture: true };
+}
+```
+
+Do not rely on prompt text alone to prevent side effects.

package/src/templates/skills/agentkit-improve/references/trace-packets.md

@@ -0,0 +1,22 @@
+# Trace Packets
+
+`agentkit improve collect` writes an ignored evidence bundle:
+
+```txt
+.agentkit/improve/<run>/
+  bundle.json
+  report.json
+  traces/
+```
+
+Use `report.json` for a quick index and `traces/<trace_id>.json` for the full conversation trace.
+
+The bundle can contain hosted or local traces. Treat both as sensitive source material. Do not commit `.agentkit/improve/`.
+
+Generated evals belong in:
+
+```txt
+evals/regressions/
+```
+
+Before committing generated evals, remove real client PII and replace brittle exact response assertions with the important behavior when needed. AgentKit redacts common email, phone, bearer token, and key patterns in generated eval text, but it cannot know every domain-specific identifier.

package/src/templates/skills/agentkit-improve/templates/regression.eval.md

@@ -0,0 +1,18 @@
+```ts
+import { defineEval } from "@andreprado/agentkit";
+
+export default defineEval({
+  name: "production regression",
+  turns: [
+    {
+      input: "User message from the production trace.",
+      expect: {
+        response: {
+          containsAny: ["required phrase", "acceptable alternative"],
+          notRegex: ["API_KEY|secret|token"],
+        },
+      },
+    },
+  ],
+});
+```

package/src/templates/skills/agentkit-integrations/SKILL.md

@@ -58,6 +58,7 @@ npm run agentkit -- integrations connect composio --toolkit gmail
 
 - Do not add `COMPOSIO_API_KEY` to `.env.schema` for managed Composio.
 - Do not ask the owner for a Composio key when using managed Composio.
+- Do not ask the owner for Composio auth config ids; AgentKit Cloud resolves toolkit auth configs.
 - Managed Composio requires a non-anonymous hosted deploy and `managed_composio` entitlement.
 - The generated tool is `agentkit_composio_execute`.
 - Use one Composio settings profile per agent.

package/src/templates/skills/agentkit-knowledge/SKILL.md

@@ -31,10 +31,13 @@ npm run agentkit -- knowledge inspect
 npm run agentkit -- knowledge search "refund policy" --top-k 3
 ```
 
+On Windows PowerShell, if `npm.ps1` is blocked with `PSSecurityException`, use `npm.cmd run agentkit -- knowledge sync` and `npm.cmd run agentkit -- knowledge search "refund policy" --top-k 3`.
+
+Local lexical search uses SQLite FTS5 when available. If the local SQLite build does not provide FTS5, AgentKit automatically uses a plain SQLite fallback table and simpler text matching.
+
 ## Safety
 
 - Do not put secrets, credentials, `.env` contents, or private tokens in Knowledge files.
 - Treat committed Knowledge files as repo content.
 - Use a private repo for private business docs.
 - Do not expose raw retrieval JSON, scores, chunk IDs, or tool output objects to users.
-

package/src/templates/skills/agentkit-provider/SKILL.md

@@ -42,6 +42,8 @@ provider: { name: "openrouter", model: "gpt-4o-mini" },
 secrets: ["OPENROUTER_API_KEY"],
 ```
 
+Prefer OpenRouter model ids or aliases known to the installed Pi SDK, such as `~google/gemini-flash-latest`. If a raw OpenRouter id is newer than Pi's registry, AgentKit passes it through to OpenRouter with conservative unknown-model metadata; OpenRouter can still reject invalid, inaccessible, or unsupported models.
+
 ## Verification
 
 ```sh
@@ -51,7 +53,8 @@ npm run chat -- --message "hello"
 npm run dev
 ```
 
+On Windows PowerShell, if `npm.ps1` is blocked with `PSSecurityException`, use `npm.cmd run typecheck`, `npm.cmd run agentkit -- inspect`, and `npm.cmd run chat -- --message "hello"`.
+
 Open the printed `Chat:` URL and report it to the owner.
 
 Do not import provider SDKs in the capsule. AgentKit resolves providers internally through Pi-backed adapters.
-

package/src/templates/skills/agentkit-security/SKILL.md

@@ -40,7 +40,9 @@ skills/
 - Add `permissions` for external capabilities.
 - Add timeouts to network tools.
 - Remove client PII before writing evals.
-- Treat public deploy URLs as transport, not access control.
+- Keep `.agentkit/improve/` bundles out of commits and review generated regression evals before committing.
+- Guard replay/eval mode inside external write tools with `ctx.runtime.environment === "eval"`.
+- Treat hosted deploy URLs as addresses, not access control. Hosted chat, conversation reads, and trace reads require a deploy access token even if a config says `access.mode: "public"`.
 
 ## Checks
 
@@ -52,4 +54,3 @@ git diff --check
 ```
 
 Expected: secret names may appear, secret values do not.
-

package/src/templates/skills/agentkit-troubleshooting/SKILL.md

@@ -42,6 +42,14 @@ For deploy issues:
 npm run agentkit -- deploy doctor
 ```
 
+For production behavior issues:
+
+```sh
+npm run agentkit -- improve collect --deploy --since 24h
+npm run agentkit -- improve evals .agentkit/improve/<run>
+npm run agentkit -- replay .agentkit/improve/<run> --against local
+```
+
 ## Common Causes
 
 - missing dependencies: run `npm install`;
@@ -50,4 +58,5 @@ npm run agentkit -- deploy doctor
 - schema missing: run `db migrate` and check `schema.sql`;
 - tool validation failed: check `inputSchema` and `outputSchema`;
 - channel secret missing: set hosted managed secret, not source files;
+- production behavior drift: collect an improve bundle and convert it to regression evals before patching;
 - local AgentKit skills are stale: run `npm run agentkit -- skills sync`.

package/src/templates/support.ts

@@ -111,7 +111,9 @@ export default defineAgent({
       },
       {
         path: "tools/lookup-order.ts",
-        contents: `const orders: Record<string, { status: string; eta: string }> = {
+        contents: `import type { AgentTool } from "@andreprado/agentkit";
+
+const orders: Record<string, { status: string; eta: string }> = {
   A100: { status: "preparing", eta: "today" },
   B200: { status: "shipped", eta: "tomorrow" },
 };
@@ -157,7 +159,7 @@ export const lookupOrder = {
       eta: order.eta,
     };
   },
-};
+} satisfies AgentTool;
 `,
       },
       {