@andreprado/agentkit 0.1.0-alpha.21 → 0.1.0-alpha.23

package/src/templates/skills/agentkit-deploy/SKILL.md

@@ -47,4 +47,6 @@ Open the printed `Chat:` URL and report it to the owner.
 
 ## Production Handoff
 
+`npm run agentkit -- deploy` prints a production handoff after a successful deploy. Use it as the source of truth for the deploy URL, UI command, secret status, database/schema artifact, integration connect commands, smoke status, and next recommended command.
+
 Report changed files, required env/secret names, database schema changes, deploy order, smoke checks, rollback concerns, and whether the provider was still `test/fake`.

package/src/templates/skills/agentkit-evals/SKILL.md

@@ -7,6 +7,8 @@ description: Use when adding, editing, or running AgentKit eval files, including
 
 Use evals after chat works and before claiming behavior is stable.
 
+`npm run eval` uses temporary local SQLite storage for eval execution. Eval conversations and tool calls do not write to the normal `.agentkit/agentkit.db`, so evals can run while local chat or `npm run dev` is using the development database.
+
 ## Workflow
 
 1. Create or edit `evals/<name>.eval.ts`.
@@ -21,6 +23,19 @@ Use evals after chat works and before claiming behavior is stable.
 10. Do not put secrets or real client PII in evals.
 11. For tools that write externally, delete, charge money, send email, or call real customer systems, branch on `ctx.runtime.environment === "eval"` inside the registered tool.
 
+## What To Test
+
+Create evals proactively from the brief and spec. Common high-value evals:
+
+- identity and scope: the agent says who it is and refuses out-of-scope work;
+- intake: required fields such as name, phone, email, account id, date, or budget are collected before action;
+- confirmation: external writes, deletes, messages, charges, and bookings do not happen before explicit confirmation;
+- privacy: raw tool output, full calendars, internal IDs, retrieval metadata, secrets, and stack traces are not shown to the client;
+- timezone and schedule: eval `now` is frozen, `timeZone` is honored, and tool payloads use the intended local date/time;
+- defaults and constraints: durations, allowed hours, allowed regions, max/min values, and business rules are asserted;
+- unhappy paths: rate limits, timeouts, missing auth, unavailable slots, empty results, and validation errors produce safe user-facing responses;
+- regression: every real conversation bug gets the smallest eval that would have failed before the fix.
+
 ## Assertion Shape
 
 Use this shape first:

package/src/templates/skills/agentkit-improve/SKILL.md

@@ -38,13 +38,23 @@ npm run agentkit -- improve collect --deploy --conversation-id <conversation-id>
 .agentkit/improve/<run>/traces/
 ```
 
-3. Generate regression evals:
+3. Before patching, identify the smallest testable lesson from each relevant trace:
+
+- Did the agent miss a required field?
+- Did it expose raw tool output, a full schedule, an internal id, or a technical error?
+- Did it write externally without confirmation?
+- Did it use the wrong timezone, duration, business hour, or availability assumption?
+- Did a tool error or provider limit produce a bad client response?
+
+4. Generate regression evals:
 
 ```sh
 npm run agentkit -- improve evals .agentkit/improve/<run>
 ```
 
-4. Patch the capsule. Likely files:
+5. Review or rewrite generated evals so they assert the behavior, not brittle transcript wording. If the bug involved a tool call, assert the persisted tool call input or absence of the unsafe call.
+
+6. Patch the capsule. Likely files:
 
 ```txt
 prompts/instructions.md
@@ -54,7 +64,7 @@ knowledge/
 evals/
 ```
 
-5. Verify:
+7. Verify:
 
 ```sh
 npm run typecheck
@@ -63,7 +73,7 @@ npm run eval
 npm run agentkit -- replay .agentkit/improve/<run> --against local
 ```
 
-6. Deploy only after local evals and replay pass:
+8. Deploy only after local evals and replay pass:
 
 ```sh
 npm run agentkit -- deploy --smoke "hello"

package/src/templates/skills/agentkit-integrations/SKILL.md

@@ -44,6 +44,18 @@ For Google Calendar, do not configure create-only access. Include `GOOGLECALENDA
 
 Managed Composio write actions require tool input `confirmed: true` by default. Set it only after the owner/user confirms the exact external change. Use `confirmExternalWrites: false` only when the capsule implements an equivalent confirmation guard elsewhere.
 
+## Testability
+
+Do not rely on the real connected app for ordinary evals. When adding an integration, also add deterministic coverage for:
+
+- the safe path, such as free/busy before calendar create;
+- missing confirmation before an external write;
+- provider errors such as 429, timeout, missing auth, empty result, or unavailable slot;
+- privacy rules, such as not showing a full calendar or raw provider payload to the client;
+- payload invariants, such as timezone conversion, duration, recipients, or record ids.
+
+Use eval-safe branches inside capsule tools, local fixtures, or `test/fake` behavior when the hosted integration cannot run locally without touching the real provider.
+
 ## Commands
 
 ```sh
@@ -54,6 +66,16 @@ npm run agentkit -- integrations status --toolkit googlecalendar
 npm run agentkit -- integrations connect composio --toolkit gmail
 ```
 
+`integrations connect composio` requires a hosted deploy because the connect link is deploy-scoped. After declaring `composioManaged(...)`, tell the owner the sequence is:
+
+```sh
+npm run agentkit -- deploy doctor
+npm run agentkit -- deploy
+npm run agentkit -- integrations connect composio --toolkit googlecalendar
+```
+
+The deploy handoff prints the connect command for each configured toolkit.
+
 ## Rules
 
 - Do not add `COMPOSIO_API_KEY` to `.env.schema` for managed Composio.

package/src/templates/skills/agentkit-provider/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: agentkit-provider
-description: Use when switching an AgentKit capsule from the deterministic test/fake provider to a real Pi-backed provider such as OpenAI, Anthropic, or OpenRouter, or when verifying provider secrets and model behavior.
+description: Use when switching an AgentKit capsule from the deterministic test/fake provider to a real Pi-backed provider such as OpenAI, Anthropic, OpenRouter, OpenCode Zen, or OpenCode Go, or when verifying provider secrets and model behavior.
 ---
 
 # AgentKit Provider
@@ -9,7 +9,7 @@ Use this when `test/fake` is no longer enough.
 
 ## Rule
 
-Do not choose a real provider automatically. Ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, or another supported provider.
+Do not choose a real provider automatically. Ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, OpenCode Zen, OpenCode Go, or another supported provider.
 
 ## Workflow
 
@@ -44,6 +44,24 @@ secrets: ["OPENROUTER_API_KEY"],
 
 Prefer OpenRouter model ids or aliases known to the installed Pi SDK, such as `~google/gemini-flash-latest`. If a raw OpenRouter id is newer than Pi's registry, AgentKit passes it through to OpenRouter with conservative unknown-model metadata; OpenRouter can still reject invalid, inaccessible, or unsupported models.
 
+OpenCode Zen:
+
+```ts
+provider: { name: "opencode", model: "big-pickle" },
+secrets: ["OPENCODE_API_KEY"],
+```
+
+Use an OpenCode Zen model id listed by the installed Pi SDK, such as `big-pickle`, `deepseek-v4-flash-free`, `claude-sonnet-4-5`, or `gpt-5.4-mini`.
+
+OpenCode Go:
+
+```ts
+provider: { name: "opencode-go", model: "deepseek-v4-flash" },
+secrets: ["OPENCODE_API_KEY"],
+```
+
+Use an OpenCode Go model id listed by the installed Pi SDK, such as `deepseek-v4-flash`, `deepseek-v4-pro`, `glm-5.1`, `kimi-k2.6`, `minimax-m2.7`, or `qwen3.6-plus`.
+
 ## Verification
 
 ```sh

package/src/templates/skills/agentkit-tools/SKILL.md

@@ -16,6 +16,8 @@ Use this when the agent needs code, an API, live data, a write, or an external a
 5. Keep secret names in `.env.schema`; values stay in ignored `.env` or hosted managed secrets.
 6. Use `ctx.clock` for date-sensitive tool logic instead of calling `new Date()` directly.
 7. Add eval guards for destructive or external side effects.
+8. Add deterministic fixtures, fake branches, or direct tool inputs for important success and failure paths.
+9. Add evals that assert the tool is called with safe inputs, or not called when confirmation/intake is missing.
 
 ## Examples
 

package/src/templates/support.ts

@@ -64,6 +64,7 @@ node_modules/
 OPENAI_API_KEY=
 ANTHROPIC_API_KEY=
 OPENROUTER_API_KEY=
+OPENCODE_API_KEY=
 `,
       },
       {
@@ -193,7 +194,7 @@ This is an AgentKit support Agent Capsule.
 
 ## Coding Agent Workflow
 
-When the owner opens this folder in Codex, Claude Code, or another coding agent and asks for a specific support agent, treat that request as the product brief.
+When the owner opens this folder in Codex, Claude Code, or another coding agent and asks for a specific support agent, treat that request as the product brief. The owner should not need to run a separate AgentKit wizard or prepare a brief file.
 
 Start building immediately:
 
@@ -203,11 +204,26 @@ Start building immediately:
 - Edit \`prompts/instructions.md\` for support behavior.
 - Edit \`agentkit.config.ts\` for provider, tools, secrets, access, and storage.
 - Add or replace TypeScript tools under \`tools/\` when the requested support agent needs actions or external data.
+- When the support agent needs to save durable records, complete the full slice: schema/migration, tool, config registration, prompt instructions, direct tool check, and eval.
 - Add \`sync.ts\`, \`seed.sql\`, and ordered \`migrations/*.sql\` when the support agent depends on external catalogs or production-shaped data changes.
 - Do not wait for a wizard or recipe. AgentKit provides the scaffold and contract; you decide the implementation from the owner's brief.
 - Ask follow-up questions only when missing information blocks a safe local implementation.
 - State assumptions in the final response.
 
+## Proactive Agent Builder Contract
+
+Do not only edit prompts. For every meaningful requirement in the owner's request or \`AGENT_SPEC.md\`, decide what should enforce it:
+
+- spec entry for the product contract;
+- prompt instruction for behavior, tone, boundaries, intake, and escalation;
+- tool plus config registration for actions, live data, external writes, or authorization-sensitive data;
+- schema/migration plus tool for durable records;
+- eval for privacy, confirmation, required fields, date/time behavior, business rules, and regressions;
+- fixture, seed data, fake branch, or direct tool check for integrations and failure paths;
+- deploy/readiness check for hosted secrets, channels, integrations, or production access.
+
+If a rule protects privacy, money, bookings, external writes, customer data, business hours, or safety, it must have an eval or deterministic check before you call the capsule done. If a real conversation exposes a bug, convert it into the smallest regression eval before or alongside the fix.
+
 ## Local Commands
 
 - \`npm install\`: restore capsule dependencies if this capsule used \`--no-install\`, install failed, or \`node_modules\` was deleted.
@@ -229,7 +245,7 @@ Start building immediately:
 - Local UI: run \`npm run dev\`, open the printed \`Chat:\` URL, and tell the owner the exact URL.
 - Hosted UI: after \`npm run agentkit -- deploy\`, run \`npm run agentkit -- chat-ui --deploy\`, open the printed \`Chat:\` URL, and tell the owner it is connected to the hosted deploy.
 - \`test/fake\` is deterministic. It is useful for scaffold checks, direct tool checks, and fake-provider evals, but it does not validate natural conversation quality.
-- Before claiming real conversation behavior is tested, ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, or another supported provider. Do not choose for them.
+- Before claiming real conversation behavior is tested, ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, OpenCode Zen, OpenCode Go, or another supported provider. Do not choose for them.
 
 ## Hosted Deploy
 
@@ -262,7 +278,9 @@ This folder is an AgentKit support Agent Capsule.
 
 ## Start Here
 
-If the owner asks you to build an agent in natural language, that request is the brief. Do not ask them to fill another file first.
+If the owner asks you to build an agent in natural language, that request is the brief. Do not ask them to run a wizard or fill another file first.
+
+Build a testable capsule, not only a prompt.
 
 Example owner request:
 
@@ -274,7 +292,10 @@ Turn the request into a working local capsule:
 - Create or update \`AGENT_SPEC.md\` with \`npm run agentkit -- spec init --brief "<owner request>"\`. The owner gives the general idea; the coding agent turns it into the structured contract.
 - Update \`agentkit.config.ts\` when tools, secrets, provider, or access rules change.
 - Add, replace, or remove TypeScript tools under \`tools/\` for real actions or external data.
+- For durable records, implement the full schema/tool/prompt/eval slice instead of only adding a table or only adding a tool.
 - Use \`npm run agentkit -- sync init\` when the agent needs catalog sync, fixture seed data, or ordered migrations.
+- For every privacy, confirmation, required-intake, timezone, business-hour, integration-error, or no-leak rule, add an eval, fixture, fake branch, or direct tool check.
+- Convert failed or surprising real conversations into regression evals with \`npm run agentkit -- eval from-conversation <conversation-id>\`.
 - Keep the first version runnable with \`test/fake\` unless the owner explicitly asks for a real provider.
 - Do not use a wizard or recipe. Build the capsule directly from the scaffold, the AgentKit contract, and the owner's brief.
 - Make practical assumptions and list them in your final response.
@@ -291,6 +312,8 @@ npm run eval
 
 \`test/fake\` proves the scaffold and deterministic tool paths. It does not prove natural conversation quality.
 
+Before saying the agent is done, make sure important requirements have matching checks. Prompt-only changes are not enough for privacy, external writes, bookings, customer data, business hours, or integration failures.
+
 \`agentkit new\` installs dependencies by default. Run \`npm install\` only if the capsule was created with \`--no-install\`, install failed, or \`node_modules\` was deleted.
 
 Set local development secrets without opening code:
@@ -320,7 +343,7 @@ npm run agentkit -- chat-ui --deploy
 
 Open the printed \`Chat:\` URL and tell the owner this local UI is connected to the hosted deploy.
 
-Before claiming real conversation behavior has been tested, ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, or another supported provider. Do not choose for them. After they choose, update \`agentkit.config.ts\`, \`.env.schema\`, local secrets, hosted secrets if deploying, then rerun chat/UI checks.
+Before claiming real conversation behavior has been tested, ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, OpenCode Zen, OpenCode Go, or another supported provider. Do not choose for them. After they choose, update \`agentkit.config.ts\`, \`.env.schema\`, local secrets, hosted secrets if deploying, then rerun chat/UI checks.
 
 If you add a tool, also run a fake-provider tool smoke test:
 
@@ -348,7 +371,7 @@ The recommended dual-storage pattern is:
 4. Use \`npm run agentkit -- db migrate\`, \`db reset --yes\`, \`db seed\`, and \`db shell\` for local database setup and inspection.
 5. Run \`npm run agentkit -- deploy\`. AgentKit migrates/provisions hosted storage internally.
 
-\`schema.sql\` is an idempotent bootstrap file in v1. Use \`CREATE TABLE IF NOT EXISTS\`, \`CREATE INDEX IF NOT EXISTS\`, and safe additive changes. AgentKit does not run destructive schema changes or ordered \`migrations/*.sql\` automatically yet.
+\`schema.sql\` is an idempotent bootstrap file. Use \`CREATE TABLE IF NOT EXISTS\`, \`CREATE INDEX IF NOT EXISTS\`, and safe additive changes. Use ordered \`migrations/*.sql\` for production-shaped schema evolution; \`npm run agentkit -- db migrate\` applies unapplied local migrations before \`schema.sql\`.
 
 ## Hosted Deploy
 
@@ -374,7 +397,7 @@ Use AgentKit conventions when editing this support capsule.
 - The agent contract lives in \`agentkit.config.ts\`.
 - The example tool lives in \`tools/lookup-order.ts\`.
 - The default provider is \`test/fake\`, which can call tools from JSON messages during local tests.
-- Ask the owner which real provider to use before switching from \`test/fake\`; do not choose OpenRouter, OpenAI, or Anthropic automatically.
+- Ask the owner which real provider to use before switching from \`test/fake\`; do not choose OpenRouter, OpenAI, Anthropic, OpenCode Zen, or OpenCode Go automatically.
 - Keep required local secret names in \`.env.schema\` and values in ignored \`.env\`. AgentKit local commands load \`.env\` directly.
 - Treat the owner's natural-language request as the brief and start implementing inside this capsule.
 - Start with \`skills/agentkit-capsule/SKILL.md\` when the task is not obvious.
@@ -398,7 +421,7 @@ npm run dev
 \`agentkit new\` installs dependencies by default. Run \`npm install\` only if this capsule was created with \`--no-install\`, install failed, or \`node_modules\` was deleted.
 
 The support template includes a local \`lookup_order\` TypeScript tool and uses \`test/fake\` by default.
-\`test/fake\` does not validate real conversation quality. The owner must choose OpenRouter, OpenAI, Anthropic, or another supported provider before real model behavior is tested.
+\`test/fake\` does not validate real conversation quality. The owner must choose OpenRouter, OpenAI, Anthropic, OpenCode Zen, OpenCode Go, or another supported provider before real model behavior is tested.
 
 For UI testing, run \`npm run dev\` and open the printed \`Chat:\` URL. After hosted deploy, run \`npm run agentkit -- chat-ui --deploy\` and open its printed \`Chat:\` URL.
 `,