@andreprado/agentkit 0.1.0-alpha.2 → 0.1.0-alpha.20

package/docs/llms-full.txt

@@ -30,59 +30,96 @@ The capsule root is the runtime boundary. Run AgentKit commands from the directo
 
 ## Current Command Surface
 
-Implemented local commands:
+Current commands:
 
 ```sh
-agentkit new <name> --template blank
-agentkit new <name> --template support
-agentkit chat --message "hello"
-agentkit tool <name> --input <path-or-json>
+agentkit new <name> [--template blank|support|dentista] [--no-install]
+agentkit dev [--port <number>]
+agentkit open
+agentkit chat-ui --deploy
+agentkit chat-ui --deploy [--port <number>] [--token-file <path>]
+agentkit chat --message <text> [--conversation-id <id>]
+agentkit tool <name> [--input <path-or-json>]
+agentkit knowledge add <path-or-url>
+agentkit knowledge sync
+agentkit knowledge inspect
+agentkit knowledge search <query> [--top-k <number>]
+agentkit db migrate
+agentkit db reset --yes
+agentkit db shell
+agentkit db seed [--file <path>]
 agentkit eval run
+agentkit eval from-conversation <conversation-id> [--out <path>] [--force]
+agentkit improve collect [--deploy] [--since <duration|iso>] [--conversation-id <id>] [--out <directory>]
+agentkit improve evals <bundle-dir-or-json> [--force]
+agentkit replay <bundle-dir-or-json> --against local
+agentkit feedback create [--about last-run|deploy|manual] [--kind bug|confusion|missing_docs|feature_request|deploy_issue|runtime_issue|other] --summary <text> [--message <text>] [--out <path>]
+agentkit feedback preview [draft.json]
+agentkit feedback send [draft.json] [--about last-run|deploy|manual] [--kind <kind>] --summary <text> [--message <text>] [--api <url>] [--save]
 agentkit conversations list
 agentkit conversations show <conversation-id>
+agentkit conversations trace <conversation-id> [--deploy]
 agentkit channels list
-agentkit channels add website website-chat
-agentkit channels add telegram support-telegram
-agentkit channels add whatsapp support-whatsapp --provider zapster
-agentkit channels setup <name>
-agentkit channels setup <name> --apply
-agentkit channels status <name>
-agentkit channels test <name> --message "hello"
-agentkit channels test <name> --fixture ./fixtures/provider-event.json
-agentkit channels deliveries list <name>
-agentkit channels deliveries show <delivery-id>
+agentkit channels add <website|telegram|whatsapp|discord|slack> <name> [--provider zapster|meta] [--mode interactions|bot] [--api <url>]
+agentkit channels connect <website|telegram|whatsapp|discord|slack> <name> [--provider zapster|meta] [--mode interactions|bot] [--api <url>]
+agentkit channels setup <name> [--apply] [--api <url>]
+agentkit channels status <name> [--api <url>]
+agentkit channels test <name> [--message <text>] [--fixture <path>] [--api <url>]
+agentkit channels test-audio <name> [--fixture voice-note|audio-file|<path>] [--api <url>]
+agentkit channels deliveries list <name> [--api <url>]
+agentkit channels deliveries show <delivery-id> [--api <url>]
+agentkit integrations status [--toolkit <slug>] [--api <url>]
+agentkit integrations connect composio [--toolkit <slug>] [--callback-url <url>] [--api <url>]
+agentkit skills status
+agentkit skills sync
 agentkit inspect
-agentkit env set <NAME> <VALUE>
-agentkit env list
-agentkit env unset <NAME>
-agentkit docs path
-agentkit docs llms
-agentkit docs full
-agentkit handoff codex [goal]
-agentkit handoff claude [goal]
-agentkit dev
-agentkit dev --port 4124
-agentkit build
-agentkit deploy
-agentkit deploy --dry-run
+agentkit build [--target cloudflare|container]
+agentkit billing checkout --slots <count> [--email <email>] [--api <url>]
+agentkit billing status <billing-intent-id> --secret <secret> [--api <url>]
+agentkit billing claim <billing-intent-id> --secret <secret> [--token-name <name>] [--api <url>]
+agentkit billing portal [--api <url>]
 agentkit login --token <token>
 agentkit logout
+agentkit account token create <name> [--api <url>] [--use] [--out <path>]
+agentkit account token list [--api <url>]
+agentkit account token revoke <token-id> [--api <url>]
+agentkit deploy [--target cloudflare|vps] [--host <host>] [--api <url>] [--dry-run] [--anonymous] [--local-wrangler] [--smoke <message>]
+agentkit deploy doctor [--api <url>] [--anonymous]
+agentkit deploy smoke [--message <text>] [--api <url>]
 agentkit deploy status
 agentkit deploy pause
 agentkit deploy resume
+agentkit transcribe smoke [--provider test|openai|groq] [--model <model>] [--fixture <audio-file>]
 agentkit secret set <NAME> <VALUE>
+agentkit secret set <NAME> --stdin
+agentkit secret set <NAME> --from-env [ENV_NAME]
+agentkit secret set <NAME> --from-local-env
+agentkit secret sync --from-local
 agentkit secret list
 agentkit secret unset <NAME>
-agentkit access token create <name>
+agentkit access token create <name> [--api <url>] [--out <path>]
 agentkit access token list
 agentkit access token revoke <token-id>
+agentkit env set <NAME> <VALUE>
+agentkit env set <NAME> --stdin
+agentkit env set <NAME> --from-env [ENV_NAME]
+agentkit env list
+agentkit env unset <NAME>
+agentkit docs path
+agentkit docs llms
+agentkit docs full
+agentkit handoff codex [goal]
+agentkit handoff claude [goal]
+agentkit help commands
 ```
 
-Planned commands described by the contract but not implemented yet:
+Prefer `env set --stdin` or `--from-env` for local secret values, and prefer `secret set --stdin`, `--from-env`, or `--from-local-env` for hosted secrets. Inline `<VALUE>` forms exist for simple non-sensitive values, but agents should avoid putting secrets in shell history.
 
-```sh
-agentkit eval create-from-conversation <conversation-id>
-```
+Use `agentkit feedback create` when AgentKit itself fails, confuses the coding agent, or lacks docs. Drafts are local files under `.agentkit/feedback/` and are not sent automatically. Use `agentkit feedback send` only after reviewing the draft and logging in with `agentkit login --token agk_user_...`. Feedback upload uses authenticated AgentKit Cloud account auth, redacts common token/key patterns, does not upload arbitrary files, and does not cause the deployed agent runtime to phone home.
+
+On Windows PowerShell, if `npm.ps1` or `npx.ps1` is blocked with `PSSecurityException`, run capsule scripts through the `.cmd` shims instead of changing the workflow. Examples: `npx.cmd @andreprado/agentkit@alpha new demo --template blank`, `npm.cmd run agentkit -- inspect`, `npm.cmd run agentkit -- knowledge sync`, and `npm.cmd run eval`.
+
+Managed Composio is configured with `composioManaged({...})` in `agentkit.config.ts` and is paid hosted AgentKit infrastructure. It requires a non-anonymous AgentKit Cloud deploy with `managed_composio`, uses one Composio settings profile per agent, injects `COMPOSIO_API_KEY` as an AgentKit-managed secret, resolves toolkit auth configs from AgentKit Cloud, validates toolkit readiness during `agentkit deploy doctor`, and exposes the generated `agentkit_composio_execute` tool only for explicit configured action slugs. Calendar starters should include `GOOGLECALENDAR_EVENTS_LIST`, `GOOGLECALENDAR_CREATE_EVENT`, and `GOOGLECALENDAR_UPDATE_EVENT`; create-event calls must pass UTC `start_datetime` plus explicit duration. Managed external write actions require tool input `confirmed: true` by default unless the integration sets `confirmExternalWrites: false`. See `docs/guides/add-managed-composio.md`.
 
 ## Create And Test A Capsule
 
@@ -95,7 +132,6 @@ cd /tmp/agentkit-demo
 
 npx @andreprado/agentkit@alpha new demo --template blank
 cd demo
-npm install
 npm run chat -- --message "hello"
 npm run chat -- --message "second message"
 npm run eval
@@ -117,7 +153,6 @@ The primary flow is:
 ```sh
 agentkit new eye-office-agent --template blank
 cd eye-office-agent
-npm install
 ```
 
 Then open the folder in Codex, Claude Code, or another coding agent and ask directly:
@@ -126,7 +161,7 @@ Then open the folder in Codex, Claude Code, or another coding agent and ask dire
 Develop an appointment and intake agent for an ophthalmology office.
 ```
 
-The coding agent should infer the first useful version, edit `prompts/instructions.md`, `agentkit.config.ts`, and `tools/`, then run the verification commands before finishing.
+The coding agent should infer the first useful version, edit `prompts/instructions.md`, `agentkit.config.ts`, `schema.sql`, `tools/`, and `evals/`, then run the verification commands before finishing. Do not wait for a wizard or recipe. AgentKit provides the scaffold and contract; the coding agent implements directly in the capsule.
 
 Optional handoff shortcut:
 
@@ -135,7 +170,9 @@ npm run agentkit -- handoff codex "Develop an appointment and intake agent for a
 npm run agentkit -- handoff claude "Develop an appointment and intake agent for an ophthalmology office."
 ```
 
-The command prints a ready-to-paste prompt that points the coding agent at `AGENTKIT.md` and the packaged `llms-full.txt` contract.
+The command prints a ready-to-paste prompt that points the coding agent at `AGENTKIT.md`, the repo-local `skills/agentkit-capsule/SKILL.md` router when present, and the packaged `llms.txt` docs router. Load `llms-full.txt` only when a skill or ambiguous framework behavior requires the complete contract.
+
+The generated docs and handoff prompt must make UI testing explicit. For local UI testing, run `npm run dev`, open the printed `Chat:` URL, and tell the owner the exact URL. For hosted UI testing after deploy, run `npm run agentkit -- chat-ui --deploy`, open the printed `Chat:` URL, and tell the owner it is connected to the hosted deploy.
 
 ## Agent Config
 
@@ -151,6 +188,7 @@ export default defineAgent({
     name: "test",
     model: "fake",
   },
+  timeZone: "America/New_York",
   instructions: "./prompts/instructions.md",
   secrets: [],
   tools: [],
@@ -163,6 +201,8 @@ export default defineAgent({
 });
 ```
 
+`timeZone` is optional and must be an IANA time zone when set. AgentKit injects dynamic runtime context into every chat run: current ISO timestamp, local date, local weekday, local date/time, and timezone. Use `timeZone` for scheduling, appointments, reminders, deadlines, and any prompt behavior that interprets "today", "tomorrow", weekdays, or relative dates. Do not hardcode today's date in `prompts/instructions.md`. If `timeZone` is omitted, AgentKit falls back to `AGENTKIT_TIME_ZONE`, then valid `TZ`, then the runtime default timezone.
+
 Valid runtime values:
 
 ```txt
@@ -203,7 +243,11 @@ provider: {
 secrets: [],
 ```
 
-Use OpenAI locally:
+`test/fake` is deterministic. It is useful for scaffold checks, direct tool checks, and fake-provider evals, but it does not validate natural conversation quality.
+
+Before claiming real conversation behavior has been tested, ask the owner which provider to use: OpenRouter, OpenAI, Anthropic, or another supported provider. Do not choose for them. After the owner chooses, update `agentkit.config.ts`, `.env.schema`, local secrets, hosted secrets if deploying, then rerun chat/UI checks.
+
+OpenAI example:
 
 ```ts
 provider: {
@@ -224,6 +268,75 @@ npm run chat -- --message "hello"
 
 If a provider key is missing, the runtime returns `secret_not_found`.
 
+For OpenRouter, prefer model ids or aliases known to the installed Pi SDK, such as `~google/gemini-flash-latest`. If an OpenRouter id is newer than Pi's registry, AgentKit passes the raw id through to OpenRouter with conservative unknown-model metadata. OpenRouter can still reject invalid, inaccessible, or unsupported models, and unknown-model cost/capability metadata is not authoritative.
+
+## Knowledge Contract
+
+Knowledge is AgentKit's native retrieval layer for facts the agent should ground in source files. Use it for FAQs, prices, policies, service descriptions, procedures, CSV tables, and reference docs. Do not put secrets, credentials, `.env` contents, or live customer/payment records in Knowledge. Use tools for live or authorization-sensitive data.
+
+Configure Knowledge in `agentkit.config.ts`:
+
+```ts
+knowledge: {
+  sources: [
+    "knowledge/faq.md",
+    { path: "knowledge/prices.csv", title: "Prices" },
+  ],
+  retrieval: {
+    topK: 8,
+    hybrid: false,
+  },
+},
+```
+
+Local Knowledge supports `.md`, `.markdown`, `.txt`, and `.csv` sources inside the Agent Capsule. Markdown chunks follow headings, text chunks follow paragraphs, and CSV chunks preserve row data with headers.
+
+Embeddings are configured separately from the chat provider. The default provider is `none`, which gives local lexical search without an API key. For OpenAI embeddings:
+
+```ts
+knowledge: {
+  sources: ["knowledge/faq.md"],
+  embedding: {
+    provider: "openai",
+    model: "text-embedding-3-small",
+    secret: "KNOWLEDGE_OPENAI_API_KEY",
+  },
+  retrieval: {
+    topK: 8,
+    hybrid: true,
+  },
+},
+```
+
+Set the local embedding secret with `agentkit env set KNOWLEDGE_OPENAI_API_KEY --stdin`. Do not commit the value.
+
+Knowledge commands:
+
+```sh
+agentkit knowledge add knowledge/faq.md
+agentkit knowledge sync
+agentkit knowledge inspect
+agentkit knowledge search "refund policy" --top-k 3
+```
+
+`knowledge add` indexes one local path. `knowledge sync` indexes all configured `knowledge.sources` and skips unchanged files by content hash. `agentkit dev` and `agentkit chat` also sync configured Knowledge automatically before local runs. `knowledge inspect` lists indexed sources and chunk counts. `knowledge search` validates retrieval before relying on the agent. Local lexical search uses SQLite FTS5 when the local SQLite build provides it; when it does not, AgentKit automatically keeps indexing and searching with a normal SQLite table and simpler text matching. When embeddings are configured locally, AgentKit stores canonical chunks in `.agentkit/agentkit.db`, rebuilds a local libSQL vector sidecar at `.agentkit/agentkit.vectors.db`, uses native `libsql_vector_idx` semantic search, and falls back to stored JSON embeddings if the native vector path is unavailable.
+
+When `knowledge` is configured, AgentKit automatically registers the internal chat tool `agentkit_search_knowledge` and appends a prompt policy. The policy tells the agent to search before answering business-specific factual questions and not to expose raw retrieval JSON, scores, chunk IDs, or tool output objects. With `test/fake`, verify the internal tool directly:
+
+```sh
+agentkit chat --message '{"tool":"agentkit_search_knowledge","input":{"query":"refund policy","topK":1}}'
+```
+
+Expected output:
+
+```txt
+Tool agentkit_search_knowledge: completed
+```
+
+Cloudflare Knowledge deploys require `storage.driver: "agentkit"` and `storage.database.driver: "turso"`. `agentkit deploy doctor` and `agentkit build --target cloudflare` fail clearly when Knowledge is configured without Turso. Cloudflare artifacts include the Knowledge manifest, required embedding secret names, internal Knowledge schema, packaged local source contents, prompt policy, and hosted `agentkit_search_knowledge` runtime. During `agentkit deploy`, AgentKit Cloud applies the Knowledge schema, chunks packaged local sources, creates embeddings when configured, deletes stale hosted sources, and syncs sources, chunks, embedding metadata, FTS rows, and a native Turso `libsql_vector_idx` index into the project Turso database before publishing the Worker. Local `agentkit knowledge add/sync`, `agentkit dev`, and `agentkit chat` index configured Knowledge into local SQLite and the local libSQL vector sidecar; hosted deploy syncs configured local Knowledge sources automatically from the deploy artifact so private source material and embedding secrets do not move into client code. Hosted semantic search uses Turso native vector search when embeddings are configured and falls back to stored JSON embeddings if the native vector path is unavailable.
+
+Full guide: `docs/guides/add-knowledge.md`.
+
 ## Channel Contract
 
 Channels are hosted inbound/outbound conversation transports. They are separate from tools: channels receive user messages, while tools let the agent call external systems.
@@ -231,7 +344,7 @@ Channels are hosted inbound/outbound conversation transports. They are separate
 Use these helpers in `agentkit.config.ts`:
 
 ```ts
-import { defineAgent, telegramChannel, whatsappChannel, websiteChannel } from "@andreprado/agentkit";
+import { defineAgent, discordChannel, slackChannel, telegramChannel, whatsappChannel, websiteChannel } from "@andreprado/agentkit";
 
 export default defineAgent({
   name: "support-agent",
@@ -244,6 +357,9 @@ export default defineAgent({
     websiteChannel({ name: "website-chat" }),
     telegramChannel({ name: "support-telegram" }),
     whatsappChannel({ name: "support-whatsapp", provider: "zapster" }),
+    discordChannel({ name: "support-discord" }),
+    discordChannel({ name: "server-discord", mode: "bot" }),
+    slackChannel({ name: "support-slack" }),
   ],
   access: { mode: "public" },
   storage: { driver: "agentkit" },
@@ -257,15 +373,34 @@ Rules:
 - Config stores secret names only, never secret values.
 - AgentKit owns channel webhook URLs, dedupe, identities, queue state, and delivery logs.
 - Do not store channel plumbing in the user's Turso database.
+- Use `buffer.mode: "debounce"` when a channel should coalesce rapid client messages into one agent run.
+- Buffered deliveries show `buffered`, then flush to one `queued` run after `quietWindowMs`, `maxWaitMs`, `maxMessages`, or `maxChars`.
+
+Channel buffer example:
+
+```ts
+whatsappChannel({
+  name: "support-whatsapp",
+  provider: "zapster",
+  buffer: {
+    mode: "debounce",
+    quietWindowMs: 2500,
+    maxWaitMs: 12000,
+    maxMessages: 20,
+    maxChars: 8000,
+  },
+})
+```
 
 Useful guides:
 
 - Add a channel: `docs/guides/add-channel.md`
+- Connect Discord: `docs/guides/connect-discord.md`
+- Connect Slack: `docs/guides/connect-slack.md`
 - Connect Telegram: `docs/guides/connect-telegram.md`
 - Connect WhatsApp through Zapster: `docs/guides/connect-whatsapp-zapster.md`
 - Debug a channel: `docs/guides/debug-channel.md`
 - Channel security: `docs/guides/channel-security.md`
-- Production handoff: `docs/guides/channels-production-handoff.md`
 
 Telegram required secrets:
 
@@ -278,7 +413,20 @@ Zapster WhatsApp required secrets:
 
 ```txt
 ZAPSTER_API_KEY
-ZAPSTER_WEBHOOK_SECRET
+ZAPSTER_INSTANCE_ID
+ZAPSTER_WEBHOOK_ID
+```
+
+Discord slash-command required secret:
+
+```txt
+DISCORD_PUBLIC_KEY
+```
+
+Discord bot-mode required secret:
+
+```txt
+DISCORD_BOT_TOKEN
 ```
 
 Common channel verification:
@@ -288,13 +436,20 @@ agentkit inspect
 agentkit deploy
 agentkit channels list
 agentkit channels add telegram support-telegram
+agentkit channels connect discord support-discord
+agentkit channels connect discord server-discord --mode bot
+agentkit channels connect slack support-slack
 agentkit channels setup support-telegram
 agentkit channels test support-telegram --message "hello"
+agentkit channels test-audio support-telegram --fixture voice-note
+agentkit transcribe smoke --provider groq
 agentkit channels deliveries list support-telegram
 agentkit channels deliveries show <delivery-id>
 ```
 
-`channels setup` is read-only by default. `channels setup <telegram-name> --apply` calls Telegram `setWebhook` and requires `TELEGRAM_BOT_TOKEN` plus `TELEGRAM_WEBHOOK_SECRET`.
+`channels connect` creates or refreshes the channel resource, validates secrets, runs provider setup when supported, then runs the official synthetic smoke. `channels setup` is read-only by default. `channels setup <telegram-name> --apply` calls Telegram `setWebhook` and requires `TELEGRAM_BOT_TOKEN` plus `TELEGRAM_WEBHOOK_SECRET`.
+
+Discord slash-command mode validates `X-Signature-Ed25519` and `X-Signature-Timestamp` against `DISCORD_PUBLIC_KEY`, answers signed `PING` requests with `type: 1`, acknowledges slash commands with a deferred response, then sends the final answer as an interaction follow-up. Discord bot mode uses `DISCORD_BOT_TOKEN`, Discord Gateway `MESSAGE_CREATE`, Message Content Intent, and `/channels/<channel_id>/messages` bot replies. Discord channels support buffering but do not support `audio` in V1.
 
 Default tests are offline. Real provider smoke tests are opt-in:
 
@@ -368,6 +523,7 @@ Tool runtime rules:
 - tools that need SQL use canonical `ctx.db`; `ctx.database` and `ctx.storage.sql` are supported aliases;
 - tools can use `ctx.db.batch([...])` for atomic writes; local tools can also use `ctx.db.transaction(async (tx) => ...)`;
 - tools can inspect `ctx.runtime` with `{ environment, invocation, target, database }`;
+- tools can use `ctx.clock` for the same runtime clock injected into the agent prompt, including `now`, `isoTimestamp`, `timeZone`, `localDate`, `localWeekday`, and `localDateTime`;
 - tools must not import local database drivers or Node-only APIs. Use AgentKit runtime services instead.
 
 ## Database Tools And Dual Storage
@@ -540,6 +696,8 @@ Run:
 agentkit dev
 ```
 
+If the default local port is occupied, `agentkit dev` chooses the next available port and prints the actual URLs. If the occupied port is another AgentKit server, the output names the agent using it.
+
 Expected output:
 
 ```txt
@@ -551,6 +709,12 @@ Inspect:   http://localhost:4123/_agentkit
 Storage:   .agentkit/agentkit.db
 ```
 
+To reopen the local UI for the current capsule after the server is already running:
+
+```sh
+agentkit open
+```
+
 Endpoints:
 
 ```txt
@@ -558,6 +722,7 @@ GET  /_agentkit
 POST /v1/chat
 GET  /v1/conversations
 GET  /v1/conversations/:id
+GET  /v1/conversations/:id/trace
 ```
 
 Chat request:
@@ -568,6 +733,8 @@ curl -X POST http://localhost:4123/v1/chat \
   -d '{"message":{"role":"user","content":"hello"}}'
 ```
 
+The chat response includes `conversationId`. Send the same `conversationId` on the next `POST /v1/chat` request, or pass it to `agentkit chat --conversation-id <id>`, to continue with the persisted message history.
+
 ## Conversations
 
 After chat:
@@ -575,6 +742,7 @@ After chat:
 ```sh
 agentkit conversations list
 agentkit conversations show <conversation-id>
+agentkit conversations trace <conversation-id>
 ```
 
 List output columns:
@@ -583,7 +751,7 @@ List output columns:
 id	title	updated_at	messages
 ```
 
-Show output includes the conversation id, title, updated timestamp, and each message as `role: content`.
+Show output includes the conversation id, title, updated timestamp, and each message as `role: content`. Trace output includes stored messages and tool calls for the conversation.
 
 ## Evals
 
@@ -602,14 +770,83 @@ npm run eval
 Supported assertion types:
 
 ```txt
-contains
-not_contains
-regex
-matches_regex
-persisted_tool_call
+response.contains
+response.containsAll
+response.containsAny
+response.caseInsensitiveContains
+response.notContains
+response.regex
+response.matchesRegex
+response.notRegex
+response.maxLength
+tools.called
+tools.calledOnce
+tools.count
+tools.order
+tools.persisted
 ```
 
-`persisted_tool_call` validates the tool call saved in local SQLite `tool_calls`, not a provider-specific raw response shape. It can be a tool name string or an object with `name`, `input`, `output`, `status`, and/or `visibility`. `tool_call` remains accepted as a backwards-compatible alias.
+Import `defineEval` from `@andreprado/agentkit` when writing new evals. `tools.persisted` validates the tool call saved in local SQLite `tool_calls`, not a provider-specific raw response shape. It can be a tool name string or an object with `name`, `input`, `output`, `rendered`, `status`, and/or `visibility`. `tool_call` and `persisted_tool_call` remain accepted as backwards-compatible aliases, but new evals should use `tools.persisted`.
+
+For date-sensitive evals, set top-level `now` to an ISO timestamp with an explicit timezone designator such as `Z` or `-05:00`. AgentKit uses that fixed clock for every turn and tool call in the eval so "today", "tomorrow", and weekdays remain deterministic while normal chat continues to use the real current date.
+
+Evals run the normal capsule tools. If a tool would write externally, delete, charge money, send email, or call a real customer system, make its `execute` implementation branch on `ctx.runtime.environment === "eval"` and return deterministic non-destructive output for eval runs. Do not invent an eval-only mock API; keep the behavior inside the registered tool contract unless AgentKit adds a first-class mock facility later.
+
+## Improve From Production
+
+Use AgentKit Improve when a hosted or local conversation should become a reproducible local fix loop. Hosted AgentKit Cloud exports evidence; the local coding agent edits the Agent Capsule, writes evals, replays, and deploys.
+
+Collect hosted evidence from the last deploy:
+
+```sh
+agentkit improve collect --deploy --since 24h
+```
+
+When the CLI is logged in to AgentKit Cloud, hosted collection first exports deploy evidence such as failed channel deliveries, deploy errors, and conversation IDs from the control plane, then reads replayable conversation traces from the deployed runtime. Without Cloud auth, it falls back to deploy-token conversation trace reads.
+
+Hosted conversation reads require a deploy access token even when the chat endpoint is public. `agentkit deploy` normally writes `.agentkit/chat-access-token.json`; use `agentkit access token create agentkit-chat-ui --out .agentkit/chat-access-token.json` to refresh it.
+
+Collect one hosted conversation:
+
+```sh
+agentkit improve collect --deploy --conversation-id <conversation-id>
+```
+
+Collect local evidence:
+
+```sh
+agentkit improve collect --since 7d
+```
+
+The command writes ignored local state:
+
+```txt
+.agentkit/improve/<run>/
+  bundle.json
+  report.json
+  traces/
+```
+
+Generate committed regression evals:
+
+```sh
+agentkit improve evals .agentkit/improve/<run>
+```
+
+Then replay before deploying:
+
+```sh
+agentkit replay .agentkit/improve/<run> --against local
+npm run eval
+agentkit deploy --smoke "hello"
+```
+
+Replay runs collected user turns through the local capsule with `ctx.runtime.environment === "eval"` and `ctx.runtime.invocation === "eval"`. Generated evals live under `evals/regressions/`; review them before committing, especially when traces contain real client details or overly strict prose assertions.
+
+Full guides:
+
+- `docs/guides/improve-from-production.md`
+- `docs/guides/replay-production-traces.md`
 
 ## Security Rules
 
@@ -634,7 +871,7 @@ prompts/
 docs/
 ```
 
-Local `.env` is development only. Use `.env.schema` as the committed secret-name contract; local AgentKit commands load `.env` directly. Hosted alpha deploys use `agentkit login --token ...` and managed secrets through `agentkit secret set/list/unset`.
+Local `.env` is development only. Use `.env.schema` as the committed secret-name contract; local AgentKit commands load `.env` directly. Hosted deploys require an AgentKit Cloud account with `cloudflare_deploy_alpha` or purchased/manual deploy slots. First-time paid access uses `agentkit billing checkout --slots <count>` and `agentkit billing claim billint_... --secret bsec_...`; existing accounts use `agentkit login --token ...`. Hosted secrets use `agentkit secret set/list/unset` or `agentkit secret sync --from-local`. Prefer `--stdin`, `--from-env`, `--from-local-env`, or sync from local `.env` so secret values do not appear in shell history. Inline `<VALUE>` forms exist only for compatibility and simple non-sensitive values.
 
 Tools are a security boundary. A tool must declare every secret it needs. The runtime injects only tool-declared secrets.
 
@@ -646,22 +883,31 @@ Current flow:
 
 ```sh
 agentkit deploy --dry-run
+agentkit billing checkout --slots 1 --email user@example.com
+agentkit billing claim billint_... --secret bsec_...
 agentkit login --token agk_user_...
+agentkit account token create new-laptop --use
 agentkit deploy doctor
-agentkit deploy
+agentkit deploy --smoke "hello"
 agentkit deploy status
+agentkit deploy smoke --message "hello"
+agentkit chat-ui --deploy
 ```
 
-`agentkit deploy doctor` checks AgentKit Cloud login, `cloudflare_deploy_alpha`, hosted secrets, local `.env` names that still need `agentkit secret set`, and private-access runtime token handling. `agentkit deploy` sends the capsule to AgentKit Cloud and runs the same readiness check automatically before building and uploading. Production alpha deploys require an account with `cloudflare_deploy_alpha`; local commands and dry-run builds do not require login. AgentKit owns infrastructure selection, backend migration, managed secrets, and public URL creation.
+`agentkit deploy doctor` checks AgentKit Cloud login, hosted deploy entitlement, online deploy capacity, hosted secrets, local `.env` names that still need `agentkit secret set`, managed Composio API/auth-config readiness by toolkit, and private-access runtime token handling. `agentkit deploy` sends the capsule to AgentKit Cloud, runs the same readiness check automatically before building and uploading, updates the current project deploy slot by default, and writes the local chat/UI deploy access token to `.agentkit/chat-access-token.json` for hosted deploys. `agentkit deploy --smoke "hello"` deploys and then tests `/v1/chat` with the deploy access token. `agentkit deploy smoke --message "hello"` repeats that smoke against the last local deploy. `agentkit chat-ui --deploy` serves a local UI pointed at the hosted deploy using that token without exposing it to browser code, shows the conversation id and tool calls, and supports starting a new conversation. Use `agentkit conversations trace <conversation-id> --deploy` to pull hosted conversation messages and tool calls from the last deploy. Production deploys require an account with `cloudflare_deploy_alpha` or purchased/manual deploy slots; local commands and dry-run builds do not require login. AgentKit owns infrastructure selection, backend migration, managed secrets, and public URL creation.
 
-The CLI defaults to the hosted AgentKit Cloud API at `https://agentkit-cloud.aibuilders.com.br`. Use `AGENTKIT_CLOUD_API_URL` or `agentkit deploy --api <url>` only for local or alternate control-plane tests.
+The CLI defaults to the hosted AgentKit Cloud API at `https://agentkit-cloud.aibuilders.com.br`. Use `AGENTKIT_CLOUD_API_URL` or `agentkit deploy --api <url>` only when the owner gives you a non-default AgentKit Cloud API URL.
 
 Account/access flow:
 
 ```sh
-agentkit secret set OPENAI_API_KEY
+agentkit secret set OPENAI_API_KEY --from-local-env
+agentkit secret sync --from-local
 agentkit secret list
-agentkit access token create website-chat
+agentkit account token list
+agentkit skills status
+agentkit skills sync
+agentkit access token create website-chat --out .agentkit/website-chat-access-token.json
 agentkit access token list
 ```
 
@@ -710,6 +956,8 @@ npm install
 npm run typecheck
 ```
 
+`agentkit new` installs dependencies by default. Run this if the scaffold used `--no-install`, the install failed, or `node_modules` was deleted.
+
 Make sure `tsconfig.json` contains:
 
 ```json