@andreprado/agentkit 0.1.0-alpha.10

package/docs/guides/create-agent.md

@@ -0,0 +1,220 @@
+# Create An Agent Capsule
+
+## Goal
+
+Create a local AgentKit Agent Capsule that installs, typechecks, chats, stores conversations, and exposes inspectable runtime state.
+
+## When To Use This
+
+Use this when starting a new agent from zero or when verifying that the scaffold still works.
+
+## Commands
+
+From a clean working directory:
+
+```sh
+rm -rf /tmp/agentkit-demo
+mkdir -p /tmp/agentkit-demo
+cd /tmp/agentkit-demo
+
+npx @andreprado/agentkit@alpha new demo --template blank
+cd demo
+npm run typecheck
+npm run chat -- --message "hello"
+npm run agentkit -- conversations list
+npm run agentkit -- inspect
+```
+
+Then open the folder in Codex, Claude Code, or another coding agent and ask directly:
+
+```txt
+Develop an appointment and intake agent for an ophthalmology office.
+```
+
+For the support template:
+
+```sh
+npx @andreprado/agentkit@alpha new support-demo --template support
+cd support-demo
+npm run agentkit -- tool lookup_order --input '{"orderId":"A100"}'
+```
+
+## Files Created Or Edited
+
+Generated files:
+
+```txt
+agentkit.config.ts
+package.json
+tsconfig.json
+.env.schema
+.gitignore
+AGENTS.md
+AGENTKIT.md
+CLAUDE.md
+README.md
+src/agent.ts
+prompts/instructions.md
+evals/smoke.eval.ts
+```
+
+AgentKit also runs `git init` in the generated capsule when `.git` does not already exist.
+
+The `support` template also creates:
+
+```txt
+tools/lookup-order.ts
+```
+
+Runtime files created after chat:
+
+```txt
+.agentkit/agentkit.db
+.agentkit/agentkit.db-shm
+.agentkit/agentkit.db-wal
+```
+
+## Handoff To A Coding Agent
+
+The owner does not need to fill a separate brief file. The natural-language request they type into Codex, Claude Code, or another coding agent is the brief.
+
+Primary flow:
+
+```txt
+Develop an appointment and intake agent for an ophthalmology office.
+```
+
+The generated `AGENTS.md`, `AGENTKIT.md`, `CLAUDE.md`, and `skills/` pack tell the coding agent which files to edit, which task skill to load, and which verification commands to run. There is no wizard or recipe layer: the coding agent edits the capsule directly from the scaffold, contract, and owner request. The default router is `skills/agentkit-capsule/SKILL.md`; `llms-full.txt` is reserved for complete-contract checks.
+
+After the owner gives the general idea, the coding agent should create or update the implementation contract itself:
+
+```sh
+npm run agentkit -- spec init --brief "Develop an appointment and intake agent for an ophthalmology office."
+npm run agentkit -- spec check
+```
+
+`AGENT_SPEC.md` is an internal working contract for the coding agent. It is not a form the owner must fill before work starts.
+
+Optional shortcut when copying a prompt into another coding agent:
+
+```sh
+npm run agentkit -- handoff codex "Develop an appointment and intake agent for an ophthalmology office."
+npm run agentkit -- handoff claude "Develop an appointment and intake agent for an ophthalmology office."
+```
+
+## Minimal Working Example
+
+`agentkit.config.ts` for the offline fake provider:
+
+```ts
+import { defineAgent } from "@andreprado/agentkit";
+
+export default defineAgent({
+  name: "demo",
+  runtime: "edge",
+  provider: {
+    name: "test",
+    model: "fake",
+  },
+  instructions: "./prompts/instructions.md",
+  secrets: [],
+  tools: [],
+  access: {
+    mode: "private",
+  },
+  storage: {
+    driver: "agentkit",
+    path: ".agentkit/agentkit.db",
+    database: {
+      driver: "turso",
+      schema: "./schema.sql",
+    },
+    files: {
+      driver: "r2",
+      bucket: "agentkit-demo-files",
+      prefix: "demo",
+    },
+  },
+});
+```
+
+Expected chat output:
+
+```txt
+Echo: hello
+```
+
+## Testing With A UI
+
+Local UI:
+
+```sh
+npm run dev
+```
+
+Open the printed `Chat:` URL and tell the owner the exact URL.
+
+Hosted deploy UI:
+
+```sh
+npm run agentkit -- deploy
+npm run agentkit -- chat-ui --deploy
+```
+
+Open the printed `Chat:` URL and tell the owner this local UI is connected to the hosted deploy.
+
+`test/fake` is deterministic. It validates the scaffold, direct tool checks, and fake-provider evals, but it does not validate natural conversation quality. Before claiming real conversation behavior is tested, ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, or another supported provider.
+
+## Safety Rules
+
+- Do not commit `.env`.
+- Do not commit `.agentkit/`.
+- Do not put secret values in `agentkit.config.ts`.
+- Keep `agentkit.config.ts` declarative.
+- Put behavior in prompts and tools.
+- Run commands from the capsule root.
+
+## Verification
+
+```sh
+npm run typecheck
+npm run chat -- --message "hello"
+npm run eval
+npm run agentkit -- conversations list
+npm run agentkit -- inspect
+```
+
+Expected checks:
+
+- TypeScript exits with code `0`.
+- Chat prints `Echo: hello`.
+- Evals report `1 passed, 0 failed`.
+- Conversation list shows one row with `2` messages.
+- Inspect prints JSON with `agent`, `runtime`, `provider`, `tools`, `storage`, and `secrets`.
+
+## Troubleshooting
+
+`Cannot find module '@andreprado/agentkit'`:
+
+```sh
+npm install
+npm run typecheck
+```
+
+`agentkit new` installs dependencies by default. Run this if the scaffold was created with `--no-install`, the install failed, or `node_modules` was deleted. Make sure `tsconfig.json` has `moduleResolution: "Bundler"`.
+
+`No agentkit.config.ts found`:
+
+Run the command from the generated capsule root.
+
+`Prompt file not found`:
+
+Check the `instructions` path in `agentkit.config.ts`.
+
+`Unknown template`:
+
+Use `blank` or `support`. `research` is planned but not implemented.
+
+## Backend Contracts Used
+
+Local commands use local files and SQLite. Hosted deploy sends a build artifact to AgentKit Cloud with `agentkit deploy`; users do not create hosted projects or infrastructure by hand.

package/docs/guides/prepare-deploy.md

@@ -0,0 +1,209 @@
+# Prepare Deploy
+
+## Goal
+
+Prepare an Agent Capsule so a coding agent can run the same workflow a user will run:
+
+```sh
+agentkit deploy
+```
+
+The agent should not choose hosting targets or infrastructure providers. AgentKit owns that routing.
+
+## When To Use This
+
+Use this before asking a coding agent to make a capsule deployable, or before testing the end-to-end user flow from a clean scaffold.
+
+## User Workflow
+
+From the capsule root:
+
+```sh
+npm run typecheck
+npm run agentkit -- inspect
+npm run agentkit -- db migrate
+npm run chat -- --message "hello"
+```
+
+All scaffold, local dev, chat, eval, inspect, database, and build commands are token-free. Only hosted production deploy and hosted control-plane mutations require an invited AgentKit Cloud token:
+
+```sh
+npm run agentkit -- login --token agk_user_...
+npm run agentkit -- deploy doctor
+npm run agentkit -- secret set OPENAI_API_KEY --from-local-env
+npm run agentkit -- deploy --smoke "hello"
+npm run agentkit -- chat-ui --deploy
+```
+
+If the capsule uses OpenAI locally, put the user’s local key in `.env`:
+
+```sh
+OPENAI_API_KEY=sk-...
+```
+
+`.env` is local-only. Hosted deploy secrets are handled by AgentKit outside the capsule.
+Closed-alpha hosted deploys require an invited account token with `cloudflare_deploy_alpha`.
+
+## What The Agent Should Edit
+
+Review and edit:
+
+```txt
+agentkit.config.ts
+prompts/instructions.md
+tools/
+evals/
+.env.schema
+AGENTS.md
+CLAUDE.md
+README.md
+schema.sql
+```
+
+Never commit or deploy:
+
+```txt
+.env
+.agentkit/
+node_modules/
+```
+
+## Deploy-Ready Config Shape
+
+Use the AgentKit-managed storage contract. Do not make the user or the coding agent choose a backend target or infrastructure provider.
+
+```ts
+export default defineAgent({
+  name: "support-agent",
+  runtime: "edge",
+  provider: {
+    name: "openai",
+    model: "gpt-4o-mini",
+  },
+  instructions: "./prompts/instructions.md",
+  secrets: ["OPENAI_API_KEY"],
+  tools: [],
+  access: {
+    mode: "private",
+  },
+  storage: {
+    driver: "agentkit",
+  },
+});
+```
+
+This config is user-facing because it describes the agent contract: provider, prompts, tools, access, secrets, and managed storage. The concrete deploy target, database vendor, file bucket, and runtime host are not user-facing.
+
+If the generated capsule already includes a managed database block, keep the `schema` path correct and put tables in `schema.sql`. The concrete database vendor is an AgentKit implementation detail.
+
+## Database Rules
+
+For agent-owned tables:
+
+- Put tables and indexes in `schema.sql`.
+- Keep `schema.sql` idempotent.
+- Prefer `CREATE TABLE IF NOT EXISTS` and `CREATE INDEX IF NOT EXISTS`.
+- Use `agentkit db migrate` before local tool/chat testing.
+- Use `ctx.db` inside tools. `ctx.database` and `ctx.storage.sql` are aliases.
+- Do not import local database drivers from tools.
+
+Local commands use `.agentkit/agentkit.db`. Hosted deploy uses AgentKit-managed infrastructure.
+
+## Verification
+
+Run this before saying the capsule is deploy-ready:
+
+```sh
+npm run typecheck
+npm run agentkit -- inspect
+npm run agentkit -- db migrate
+npm run chat -- --message "hello"
+npm run agentkit -- deploy --dry-run
+npm run agentkit -- deploy doctor
+```
+
+For a final end-to-end test, run:
+
+```sh
+npm run agentkit -- login --token agk_user_...
+npm run agentkit -- secret sync --from-local
+npm run agentkit -- secret list
+npm run agentkit -- deploy --smoke "hello"
+npm run agentkit -- deploy status
+npm run agentkit -- chat-ui --deploy
+npm run agentkit -- access token list
+```
+
+For private hosted deploys, `npm run agentkit -- deploy` writes the local chat/UI access token to `.agentkit/chat-access-token.json`. Use `npm run agentkit -- deploy --smoke "hello"` for the official hosted chat smoke, and use `npm run agentkit -- chat-ui --deploy` for hosted UI testing. Use `npm run agentkit -- access token create <name> --out <path>` only for additional clients.
+
+When the UI is running, open the printed `Chat:` URL and tell the owner the exact URL. If the capsule is still on `test/fake`, say the UI was tested only with the deterministic fake provider.
+
+## Readiness Checklist
+
+```txt
+No .env committed
+No .agentkit committed
+All required secret names are declared
+Prompt path exists
+Tools have input schemas
+Dangerous tools have permissions
+Provider model is supported by AgentKit
+schema.sql is idempotent
+README/AGENTS/CLAUDE match the capsule
+```
+
+## Troubleshooting
+
+`agentkit deploy` fails before publishing:
+
+Read the AgentKit error code first. Common fixes are:
+
+- run `npm run typecheck`;
+- run `npm run agentkit -- inspect`;
+- make tools edge-safe;
+- declare missing secret names in `agentkit.config.ts`;
+- keep production secret values out of source files.
+
+Missing local OpenAI key:
+
+```sh
+echo 'OPENAI_API_KEY=sk-...' >> .env
+```
+
+Missing hosted secret:
+
+Set the secret in AgentKit Cloud or the configured control plane:
+
+```sh
+npm run agentkit -- secret set OPENAI_API_KEY --from-local-env
+```
+
+To sync all declared user-managed secrets present in local `.env`:
+
+```sh
+npm run agentkit -- secret sync --from-local
+```
+
+Do not write hosted secret values into the capsule.
+
+Before a hosted production deploy, run:
+
+```sh
+npm run agentkit -- deploy doctor
+```
+
+The doctor checks AgentKit Cloud login, alpha deploy entitlement, online deploy capacity, declared hosted secrets, local `.env` names that have not been uploaded with `agentkit secret set`, and private-access runtime token handling. `agentkit deploy` runs the same readiness check automatically before building and uploading the artifact.
+
+`alpha_access_required`:
+
+Log in with an invited alpha account:
+
+```sh
+npm run agentkit -- login --token agk_user_...
+```
+
+## Operator Notes
+
+This section is for AgentKit maintainers, not for capsule-building agents.
+
+`agentkit deploy` sends the capsule artifact to the configured AgentKit Cloud API. The control plane owns infrastructure selection, provisioning, managed secrets, and public URL creation. Use `AGENTKIT_CLOUD_API_URL` only when testing a non-default control plane.

package/docs/guides/run-evals.md

@@ -0,0 +1,179 @@
+# Run Evals
+
+## Goal
+
+Prepare and run evals for an Agent Capsule.
+
+## When To Use This
+
+Use this after chat works and before changing prompts, tools, or provider config.
+
+## Commands
+
+Generated file:
+
+```txt
+evals/smoke.eval.ts
+```
+
+Run evals:
+
+```sh
+agentkit eval run
+```
+
+Create an eval from a stored conversation:
+
+```sh
+agentkit conversations list
+agentkit conversations trace <conversation-id>
+agentkit eval from-conversation <conversation-id>
+agentkit eval run
+```
+
+## Files Created Or Edited
+
+Create or edit:
+
+```txt
+evals/<name>.eval.ts
+```
+
+Use conversations as source material:
+
+```txt
+.agentkit/agentkit.db
+```
+
+Do not edit `.agentkit/agentkit.db` by hand.
+
+## Minimal Working Example
+
+Generated smoke eval:
+
+```ts
+export default {
+  name: "smoke",
+  input: "Say hello in one short sentence.",
+  expect: {
+    contains: "hello",
+  },
+};
+```
+
+Supported assertion types:
+
+```txt
+contains
+not_contains
+regex
+matches_regex
+persisted_tool_call
+```
+
+Multi-turn conversation evals use `turns`:
+
+```ts
+export default {
+  name: "buyer under budget",
+  turns: [
+    {
+      input: "I want a house up to 600k near Pinheiros.",
+      expect: {
+        persisted_tool_call: {
+          name: "buscar_imoveis",
+          status: "completed",
+          input: { maxPrice: 600000 },
+        },
+      },
+    },
+    {
+      input: "Show me the best two.",
+      expect: {
+        contains: ["Pinheiros", "R$"],
+      },
+    },
+  ],
+};
+```
+
+`persisted_tool_call` validates the tool call saved in local SQLite `tool_calls`, not a provider-specific raw response shape. It can be a tool name string or an object with `name`, `input`, `output`, `rendered`, `status`, and/or `visibility`.
+
+Use response assertions and persisted tool assertions together when internal operational output must not leak:
+
+```ts
+export default {
+  name: "triage lead",
+  input: '{"tool":"triage_real_estate_lead","input":{"email":"ada@example.com"}}',
+  expect: {
+    not_contains: ["hot", "score"],
+    persisted_tool_call: {
+      name: "triage_real_estate_lead",
+      status: "completed",
+      visibility: "internal",
+      output: { status: "hot" },
+    },
+  },
+};
+```
+
+`tool_call` remains accepted as a backwards-compatible alias, but new evals should use `persisted_tool_call`.
+
+Safe external-tool pattern:
+
+```ts
+export const sendFollowupEmail = defineTool({
+  name: "send_followup_email",
+  description: "Sends a follow-up email.",
+  inputSchema: {
+    type: "object",
+    properties: {
+      email: { type: "string" },
+    },
+    required: ["email"],
+    additionalProperties: false,
+  },
+  async execute(input: { email: string }, ctx) {
+    if (ctx.runtime.environment === "eval") {
+      return { sent: false, evalFixture: true, email: input.email };
+    }
+
+    // Real provider call here.
+    return { sent: true, evalFixture: false, email: input.email };
+  },
+});
+```
+
+## Safety Rules
+
+- Do not put provider keys in eval files.
+- Do not put real client PII in eval files.
+- Prefer small deterministic assertions.
+- Keep eval tool calls deterministic and non-destructive.
+- For tools that would write, delete, charge money, send email, or call a real customer system, branch inside the registered tool on `ctx.runtime.environment === "eval"` and return safe fixture output.
+- Treat conversations as source material, not as automatically safe training data.
+
+## Verification
+
+```sh
+npm run typecheck
+npm run eval
+```
+
+## Troubleshooting
+
+`agentkit eval` is unknown:
+
+Run `npm install` in the capsule and make sure it depends on the AgentKit version that includes `agentkit eval run`.
+
+Eval output changes after provider switch:
+
+Use `test/fake` for deterministic smoke tests, then add provider-specific evals separately.
+
+Tool eval hits a real API:
+
+Make the registered tool return deterministic fixture output when `ctx.runtime.environment === "eval"`, then rerun the eval suite. AgentKit does not expose a separate eval-only mock registry yet.
+
+## Backend Contracts Used
+
+Future hosted evals use `EvalCase` and `EvalResult` resources. The current local implementation stores conversations, messages, runs, and tool calls only.