@andreprado/agentkit 0.1.0-alpha.2 → 0.1.0-alpha.21

package/src/templates/skills/agentkit-improve/references/replay-side-effects.md

@@ -0,0 +1,18 @@
+# Replay Side Effects
+
+Replay runs collected user turns through the local capsule with:
+
+```txt
+ctx.runtime.environment === "eval"
+ctx.runtime.invocation === "eval"
+```
+
+Tools still execute. Any tool that writes externally, deletes data, charges money, sends email, sends messages, or calls a real customer system must guard eval mode:
+
+```ts
+if (ctx.runtime.environment === "eval") {
+  return { ok: true, evalFixture: true };
+}
+```
+
+Do not rely on prompt text alone to prevent side effects.

package/src/templates/skills/agentkit-improve/references/trace-packets.md

@@ -0,0 +1,22 @@
+# Trace Packets
+
+`agentkit improve collect` writes an ignored evidence bundle:
+
+```txt
+.agentkit/improve/<run>/
+  bundle.json
+  report.json
+  traces/
+```
+
+Use `report.json` for a quick index and `traces/<trace_id>.json` for the full conversation trace.
+
+The bundle can contain hosted or local traces. Treat both as sensitive source material. Do not commit `.agentkit/improve/`.
+
+Generated evals belong in:
+
+```txt
+evals/regressions/
+```
+
+Before committing generated evals, remove real client PII and replace brittle exact response assertions with the important behavior when needed. AgentKit redacts common email, phone, bearer token, and key patterns in generated eval text, but it cannot know every domain-specific identifier.

package/src/templates/skills/agentkit-improve/templates/regression.eval.md

@@ -0,0 +1,18 @@
+```ts
+import { defineEval } from "@andreprado/agentkit";
+
+export default defineEval({
+  name: "production regression",
+  turns: [
+    {
+      input: "User message from the production trace.",
+      expect: {
+        response: {
+          containsAny: ["required phrase", "acceptable alternative"],
+          notRegex: ["API_KEY|secret|token"],
+        },
+      },
+    },
+  ],
+});
+```

package/src/templates/skills/agentkit-integrations/SKILL.md

@@ -0,0 +1,76 @@
+---
+name: agentkit-integrations
+description: Use when adding, inspecting, connecting, or troubleshooting AgentKit-managed integrations such as managed Composio in an Agent Capsule.
+---
+
+# AgentKit Integrations
+
+Use this when the owner asks for managed connected apps, Gmail/Calendar/Slack/Linear through AgentKit, or paid AgentKit-managed Composio.
+
+## Managed Composio
+
+Managed Composio is a paid hosted AgentKit feature. Use it when the owner wants AgentKit to manage OAuth/connect links, per-agent connected app state, deploy readiness, and hosted Composio credentials.
+
+Use BYO `defineTool` instead when the owner wants to bring their own Composio account/API key.
+
+## Config
+
+Edit `agentkit.config.ts`:
+
+```ts
+import { composioManaged, defineAgent } from "@andreprado/agentkit";
+
+export default defineAgent({
+  integrations: [
+    composioManaged({
+      toolkits: ["gmail", "googlecalendar"],
+      tools: {
+        gmail: ["GMAIL_FETCH_EMAILS", "GMAIL_SEND_EMAIL"],
+        googlecalendar: [
+          "GOOGLECALENDAR_EVENTS_LIST",
+          "GOOGLECALENDAR_CREATE_EVENT",
+          "GOOGLECALENDAR_UPDATE_EVENT",
+        ],
+      },
+      confirmExternalWrites: true,
+    }),
+  ],
+});
+```
+
+Keep the action list explicit. Do not expose the whole Composio catalog by default.
+
+For Google Calendar, do not configure create-only access. Include `GOOGLECALENDAR_EVENTS_LIST` so the agent can inspect availability before writing. For `GOOGLECALENDAR_CREATE_EVENT`, pass UTC `start_datetime` and explicit `event_duration_minutes` or `event_duration_hour`; AgentKit blocks Composio's implicit 30-minute duration default.
+
+Managed Composio write actions require tool input `confirmed: true` by default. Set it only after the owner/user confirms the exact external change. Use `confirmExternalWrites: false` only when the capsule implements an equivalent confirmation guard elsewhere.
+
+## Commands
+
+```sh
+npm run agentkit -- inspect
+npm run agentkit -- deploy doctor
+npm run agentkit -- deploy
+npm run agentkit -- integrations status --toolkit googlecalendar
+npm run agentkit -- integrations connect composio --toolkit gmail
+```
+
+## Rules
+
+- Do not add `COMPOSIO_API_KEY` to `.env.schema` for managed Composio.
+- Do not ask the owner for a Composio key when using managed Composio.
+- Do not ask the owner for Composio auth config ids; AgentKit Cloud resolves toolkit auth configs.
+- Managed Composio requires a non-anonymous hosted deploy and `managed_composio` entitlement.
+- The generated tool is `agentkit_composio_execute`.
+- Use one Composio settings profile per agent.
+
+## Verification
+
+Expected `inspect` output includes:
+
+```txt
+integrations[0].provider = composio
+tools includes agentkit_composio_execute
+managedSecrets includes COMPOSIO_API_KEY
+```
+
+If `deploy doctor` reports `managed_composio_entitlement_required`, the owner must log in with a paid AgentKit Cloud account or ask an operator to grant it.

package/src/templates/skills/agentkit-knowledge/SKILL.md

@@ -0,0 +1,43 @@
+---
+name: agentkit-knowledge
+description: Use when adding AgentKit Knowledge sources for grounded answers from local Markdown, text, or CSV files, configuring retrieval or embeddings, syncing/searching Knowledge, or deciding between Knowledge and tools.
+---
+
+# AgentKit Knowledge
+
+Use Knowledge for committed reference facts: FAQs, policies, prices, service descriptions, procedures, and CSV tables.
+
+Use tools instead for live records, authorization-sensitive data, customer-specific data, payments, orders, or writes.
+
+## Workflow
+
+1. Create files under `knowledge/`.
+2. Configure `knowledge.sources` in `agentkit.config.ts`.
+3. Use lexical search by default. Add embeddings only when needed.
+4. Run sync and search before relying on answers.
+
+## Templates
+
+- `templates/faq.md`
+- `templates/policies.md`
+- `templates/prices.csv`
+
+## Commands
+
+```sh
+npm run agentkit -- knowledge add knowledge/faq.md
+npm run agentkit -- knowledge sync
+npm run agentkit -- knowledge inspect
+npm run agentkit -- knowledge search "refund policy" --top-k 3
+```
+
+On Windows PowerShell, if `npm.ps1` is blocked with `PSSecurityException`, use `npm.cmd run agentkit -- knowledge sync` and `npm.cmd run agentkit -- knowledge search "refund policy" --top-k 3`.
+
+Local lexical search uses SQLite FTS5 when available. If the local SQLite build does not provide FTS5, AgentKit automatically uses a plain SQLite fallback table and simpler text matching.
+
+## Safety
+
+- Do not put secrets, credentials, `.env` contents, or private tokens in Knowledge files.
+- Treat committed Knowledge files as repo content.
+- Use a private repo for private business docs.
+- Do not expose raw retrieval JSON, scores, chunk IDs, or tool output objects to users.

package/src/templates/skills/agentkit-knowledge/templates/faq.md

@@ -0,0 +1,14 @@
+# FAQ
+
+## What services do you offer?
+
+Replace this answer with the owner's real services.
+
+## What are your hours?
+
+Replace this answer with the owner's real business hours.
+
+## How should users contact a human?
+
+Replace this answer with the owner's real escalation path.
+

package/src/templates/skills/agentkit-knowledge/templates/policies.md

@@ -0,0 +1,14 @@
+# Policies
+
+## Cancellation
+
+Replace this with the real cancellation policy.
+
+## Privacy
+
+Replace this with the real privacy policy.
+
+## Escalation
+
+Replace this with the real human handoff policy.
+

package/src/templates/skills/agentkit-knowledge/templates/prices.csv

@@ -0,0 +1,3 @@
+item,price,notes
+Example service,100,Replace with real prices before using in production
+

package/src/templates/skills/agentkit-prompts/SKILL.md

@@ -0,0 +1,47 @@
+---
+name: agentkit-prompts
+description: Use when writing or revising AgentKit prompt files, especially prompts/instructions.md, including agent role, behavior, boundaries, escalation rules, tool-use policy, Knowledge policy, and user-facing tone.
+---
+
+# AgentKit Prompts
+
+Use this when editing `prompts/instructions.md`.
+
+## Prompt Checklist
+
+Include only behavior the runtime should apply on every conversation:
+
+- agent role and audience;
+- domain-specific goals;
+- what information to collect;
+- when to use tools;
+- when to search Knowledge;
+- how to interpret scheduling language such as today, tomorrow, and next Friday;
+- what the agent must not claim;
+- escalation and safety boundaries;
+- response style.
+
+AgentKit injects the current timestamp, local date, weekday, and timezone dynamically at runtime. Do not hardcode today's date in `prompts/instructions.md`; set `timeZone` in `agentkit.config.ts` when a scheduling agent needs a specific business/user timezone.
+
+Keep operational secrets, provider details, and implementation notes out of prompts.
+
+## Tool And Knowledge Policy
+
+- Use tools for actions, live data, authorization-sensitive records, payments, orders, booking, and writes.
+- Use Knowledge for committed reference facts such as FAQs, prices, services, policies, procedures, and CSV tables.
+- Do not expose raw tool output, retrieval JSON, chunk IDs, scores, secrets, logs, or internal labels.
+
+## Templates
+
+- `templates/knowledge-grounded-faq.instructions.md`
+- `../agentkit-build-agent/templates/support-agent.instructions.md`
+- `../agentkit-build-agent/templates/appointment-intake.instructions.md`
+
+## Verification
+
+```sh
+npm run chat -- --message "hello"
+npm run eval
+```
+
+If the provider is still `test/fake`, say prompt behavior was not tested with a real model.

package/src/templates/skills/agentkit-prompts/templates/knowledge-grounded-faq.instructions.md

@@ -0,0 +1,11 @@
+You are a knowledge-grounded FAQ agent.
+
+Use the configured Knowledge search before answering business-specific factual questions about services, prices, policies, procedures, or support rules.
+
+Rules:
+- Answer from the retrieved source material when available.
+- If the answer is not in the available sources, say that you do not have enough information.
+- Do not expose raw retrieval JSON, scores, chunk IDs, or internal source metadata.
+- Do not use Knowledge for secrets, credentials, live customer records, payments, or authorization-sensitive data.
+- Use tools for live or customer-specific lookups.
+

package/src/templates/skills/agentkit-provider/SKILL.md

@@ -0,0 +1,60 @@
+---
+name: agentkit-provider
+description: Use when switching an AgentKit capsule from the deterministic test/fake provider to a real Pi-backed provider such as OpenAI, Anthropic, or OpenRouter, or when verifying provider secrets and model behavior.
+---
+
+# AgentKit Provider
+
+Use this when `test/fake` is no longer enough.
+
+## Rule
+
+Do not choose a real provider automatically. Ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, or another supported provider.
+
+## Workflow
+
+1. Edit `agentkit.config.ts`.
+2. Add required secret names to `secrets`.
+3. Add names to `.env.schema`.
+4. Set local secret values in ignored `.env` through AgentKit commands.
+5. Run inspect, chat, and UI checks.
+
+## Config Examples
+
+OpenAI:
+
+```ts
+provider: { name: "openai", model: "gpt-4o-mini" },
+secrets: ["OPENAI_API_KEY"],
+```
+
+Anthropic:
+
+```ts
+provider: { name: "anthropic", model: "claude-3-5-haiku-latest" },
+secrets: ["ANTHROPIC_API_KEY"],
+```
+
+OpenRouter:
+
+```ts
+provider: { name: "openrouter", model: "gpt-4o-mini" },
+secrets: ["OPENROUTER_API_KEY"],
+```
+
+Prefer OpenRouter model ids or aliases known to the installed Pi SDK, such as `~google/gemini-flash-latest`. If a raw OpenRouter id is newer than Pi's registry, AgentKit passes it through to OpenRouter with conservative unknown-model metadata; OpenRouter can still reject invalid, inaccessible, or unsupported models.
+
+## Verification
+
+```sh
+npm run typecheck
+npm run agentkit -- inspect
+npm run chat -- --message "hello"
+npm run dev
+```
+
+On Windows PowerShell, if `npm.ps1` is blocked with `PSSecurityException`, use `npm.cmd run typecheck`, `npm.cmd run agentkit -- inspect`, and `npm.cmd run chat -- --message "hello"`.
+
+Open the printed `Chat:` URL and report it to the owner.
+
+Do not import provider SDKs in the capsule. AgentKit resolves providers internally through Pi-backed adapters.

package/src/templates/skills/agentkit-security/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: agentkit-security
+description: Use before or during AgentKit work involving secrets, external APIs, tools, evals from real data, public access, hosted deploys, or messaging channels.
+---
+
+# AgentKit Security
+
+Use this as a guardrail skill.
+
+## Never Commit
+
+```txt
+.env
+.agentkit/
+node_modules/
+```
+
+Secret values must not appear in:
+
+```txt
+agentkit.config.ts
+.env.schema
+AGENTS.md
+AGENTKIT.md
+CLAUDE.md
+README.md
+prompts/
+evals/
+docs/
+skills/
+```
+
+## Rules
+
+- `.env.schema` stores secret names only.
+- Ignored `.env` stores local development values only.
+- Hosted production uses managed secrets.
+- Tools receive only secrets listed in that tool's `secrets` field.
+- Prefer `ctx.secrets` over direct `process.env` reads in tools.
+- Add `permissions` for external capabilities.
+- Add timeouts to network tools.
+- Remove client PII before writing evals.
+- Keep `.agentkit/improve/` bundles out of commits and review generated regression evals before committing.
+- Guard replay/eval mode inside external write tools with `ctx.runtime.environment === "eval"`.
+- Treat hosted deploy URLs as addresses, not access control. Hosted chat, conversation reads, and trace reads require a deploy access token even if a config says `access.mode: "public"`.
+
+## Checks
+
+```sh
+git status --short
+npm run agentkit -- env list
+npm run agentkit -- inspect
+git diff --check
+```
+
+Expected: secret names may appear, secret values do not.

package/src/templates/skills/agentkit-tools/SKILL.md

@@ -0,0 +1,37 @@
+---
+name: agentkit-tools
+description: Use when adding, changing, registering, or testing AgentKit TypeScript tools with defineTool, including external APIs, local actions, input/output schemas, tool secrets, permissions, timeouts, and eval-safe side effects.
+---
+
+# AgentKit Tools
+
+Use this when the agent needs code, an API, live data, a write, or an external action.
+
+## Workflow
+
+1. Create or edit `tools/<name>.ts`.
+2. Export a `defineTool` tool with `name`, `description`, `inputSchema`, and usually `outputSchema`.
+3. Add `secrets`, `permissions`, and `timeoutMs` when needed.
+4. Register the tool in `agentkit.config.ts`.
+5. Keep secret names in `.env.schema`; values stay in ignored `.env` or hosted managed secrets.
+6. Use `ctx.clock` for date-sensitive tool logic instead of calling `new Date()` directly.
+7. Add eval guards for destructive or external side effects.
+
+## Examples
+
+- `examples/lookup-order.tool.md`: simple lookup
+- `examples/database-write.tool.md`: write through `ctx.db`
+- `examples/eval-safe-external-action.tool.md`: safe eval branch for external side effects
+
+Use `skills/agentkit-database/SKILL.md` for database-backed tools.
+Use `skills/agentkit-security/SKILL.md` before adding external APIs or secrets.
+
+## Verification
+
+```sh
+npm run typecheck
+npm run agentkit -- tool <tool_name> --input '<json>'
+npm run agentkit -- inspect
+```
+
+Expected: input/output validation passes, tool calls persist, and no secret values appear in output or stored data.

package/src/templates/skills/agentkit-tools/examples/database-write.tool.md

@@ -0,0 +1,35 @@
+# Database Write Tool
+
+Copy the code into a real tool file and make sure `schema.sql` contains the target table.
+
+```ts
+import { defineTool } from "@andreprado/agentkit";
+
+export const saveNote = defineTool({
+  name: "save_note",
+  description: "Saves a note in the AgentKit-managed database.",
+  inputSchema: {
+    type: "object",
+    properties: {
+      content: { type: "string" },
+    },
+    required: ["content"],
+    additionalProperties: false,
+  },
+  outputSchema: {
+    type: "object",
+    properties: {
+      id: { type: "string" },
+      content: { type: "string" },
+    },
+    required: ["id", "content"],
+    additionalProperties: false,
+  },
+  async execute(input: { content: string }, ctx) {
+    const id = crypto.randomUUID();
+    await ctx.db.execute("INSERT INTO notes (id, content) VALUES (?, ?)", [id, input.content]);
+    return { id, content: input.content };
+  },
+});
+```
+

package/src/templates/skills/agentkit-tools/examples/eval-safe-external-action.tool.md

@@ -0,0 +1,37 @@
+# Eval-Safe External Action Tool
+
+Use this pattern for tools that send email, call customer systems, charge money, delete data, or perform other external side effects.
+
+```ts
+import { defineTool } from "@andreprado/agentkit";
+
+export const sendFollowupEmail = defineTool({
+  name: "send_followup_email",
+  description: "Sends a follow-up email after explicit confirmation.",
+  secrets: ["EMAIL_API_KEY"],
+  permissions: ["email:send"],
+  inputSchema: {
+    type: "object",
+    properties: {
+      email: { type: "string" },
+      confirmed: { type: "boolean" },
+    },
+    required: ["email", "confirmed"],
+    additionalProperties: false,
+  },
+  async execute(input: { email: string; confirmed: boolean }, ctx) {
+    if (!input.confirmed) {
+      return { sent: false, reason: "confirmation_required" };
+    }
+
+    if (ctx.runtime.environment === "eval") {
+      return { sent: false, evalFixture: true, email: input.email };
+    }
+
+    const apiKey = ctx.secrets.EMAIL_API_KEY;
+    // Call the real email provider with apiKey here.
+    return { sent: true, evalFixture: false, email: input.email };
+  },
+});
+```
+

package/src/templates/skills/agentkit-tools/examples/lookup-order.tool.md

@@ -0,0 +1,46 @@
+# Lookup Order Tool
+
+Copy the code into `tools/lookup-order.ts` and register `lookupOrder` in `agentkit.config.ts`.
+
+```ts
+import { defineTool } from "@andreprado/agentkit";
+
+const orders: Record<string, { status: string; eta: string }> = {
+  A100: { status: "preparing", eta: "today" },
+  B200: { status: "shipped", eta: "tomorrow" },
+};
+
+export const lookupOrder = defineTool({
+  name: "lookup_order",
+  description: "Looks up a demo support order by order id.",
+  inputSchema: {
+    type: "object",
+    properties: {
+      orderId: { type: "string" },
+    },
+    required: ["orderId"],
+    additionalProperties: false,
+  },
+  outputSchema: {
+    type: "object",
+    properties: {
+      orderId: { type: "string" },
+      found: { type: "boolean" },
+      status: { type: "string" },
+      eta: { type: "string" },
+    },
+    required: ["orderId", "found", "status", "eta"],
+    additionalProperties: false,
+  },
+  execute(input: { orderId: string }) {
+    const order = orders[input.orderId];
+    return {
+      orderId: input.orderId,
+      found: Boolean(order),
+      status: order?.status ?? "unknown",
+      eta: order?.eta ?? "unknown",
+    };
+  },
+});
+```
+

package/src/templates/skills/agentkit-troubleshooting/SKILL.md

@@ -0,0 +1,76 @@
+---
+name: agentkit-troubleshooting
+description: Use when an AgentKit command, local chat, tool, eval, Knowledge sync, channel, or deploy fails and the coding agent needs to diagnose from CLI output and route to the right narrow skill or docs guide.
+---
+
+# AgentKit Troubleshooting
+
+Use this when something fails.
+
+## Workflow
+
+1. Read the exact error code and message.
+2. Run the narrow inspect command before guessing.
+3. Route to a task skill when the failure points to config, tools, database, provider, evals, deploy, Knowledge, or channels.
+4. Load `llms-full.txt` only when the narrow skill and guide do not explain the behavior.
+5. If the failure appears to be an AgentKit bug, missing docs, or unclear recovery path, create a local feedback draft after diagnosis.
+
+## First Commands
+
+```sh
+npm run agentkit -- inspect
+npm run agentkit -- skills status
+npm run typecheck
+npm run agentkit -- env list
+git status --short
+```
+
+For chat issues:
+
+```sh
+npm run chat -- --message "hello"
+```
+
+For tool issues:
+
+```sh
+npm run agentkit -- tool <tool_name> --input '<json>'
+```
+
+For deploy issues:
+
+```sh
+npm run agentkit -- deploy doctor
+```
+
+For AgentKit product feedback:
+
+```sh
+npm run agentkit -- feedback create --about last-run --kind bug --summary "short concrete summary"
+```
+
+For production behavior issues:
+
+```sh
+npm run agentkit -- improve collect --deploy --since 24h
+npm run agentkit -- improve evals .agentkit/improve/<run>
+npm run agentkit -- replay .agentkit/improve/<run> --against local
+```
+
+## Common Causes
+
+- missing dependencies: run `npm install`;
+- missing provider key: set ignored `.env` and confirm with `inspect`;
+- provider not chosen: stay on `test/fake` or ask the owner;
+- schema missing: run `db migrate` and check `schema.sql`;
+- tool validation failed: check `inputSchema` and `outputSchema`;
+- channel secret missing: set hosted managed secret, not source files;
+- production behavior drift: collect an improve bundle and convert it to regression evals before patching;
+- local AgentKit skills are stale: run `npm run agentkit -- skills sync`.
+
+## Feedback Rules
+
+- `feedback create` writes a local draft only; it does not send anything.
+- Review the draft before `feedback send`.
+- Sending requires `agentkit login --token agk_user_...`.
+- Do not paste `.env` values, provider keys, cookies, client PII, or full private transcripts into feedback.