@seldonframe/mcp 1.27.0 → 1.27.2

package/package.json

@@ -1,6 +1,8 @@
 {
   "name": "@seldonframe/mcp",
-  "version": "1.27.0",
+  "version": "1.27.2",
+  "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.27.2: PRODUCTION HOTFIX — Vercel build has been failing since v1.26.2 because lib/agents/eval-runner.ts (a 'use server' Next.js Server Actions module) re-exported the constant PUBLISH_PASS_RATE_THRESHOLD with `export { PUBLISH_PASS_RATE_THRESHOLD };`. TypeScript's tsc --noEmit didn't flag this (server-action semantics aren't a tsc concern), but next build's page-data collection step rejects 'use server' modules that export anything other than async functions, with: `Error: A 'use server' file can only export async functions, found number.` MEANS production has been stuck on v1.26.1 for the past three ships — the new agent ops (run_evals, tail_conversations, get_conversation, replay_conversation, get_metrics) and the entire /agents dashboard surface (overview, settings, evals, sandbox, conversations) have NOT been deployable, and any operator hitting /agents in their browser would 404. ROOT CAUSE: I added the re-export thinking it was a convenience for store.ts publishAgent, but store.ts didn't actually need it (publishAgent uses summary.meetsPublishGate which is computed inside runEvalSuite). FIX: remove the offending line + add an explanatory comment so this doesn't recur. PREVENTION: new packages/crm/scripts/check-use-server.sh is a 60-line shell guard that scans every 'use server' file for `export const|let|var|{`-shaped exports. Wired into pnpm build so it runs BEFORE next build (fails in <1s with a clear message instead of wasting 45s of compile + page-data collection on Vercel). Catches the entire bug class going forward — antifragile to future server-action refactors. SWEPT all 46 'use server' files in packages/crm; eval-runner.ts was the only offender. NO MCP-tool changes (this is a backend-only hotfix), but I'm bumping the npm package version + republishing for SemVer hygiene so anyone on @seldonframe/mcp@latest is signaling that the BACKEND v1.26.2/v1.27.0/v1.27.1 features will start working as soon as Vercel deploys this commit. Backend redeploy required (the next push to main triggers it).",
+  "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.27.1: TOOL DISCOVERY DOC UPGRADE — Phase A of the Karpathy/antifragility refactor. PROBLEM v1.27.1 SOLVES: real-world dogfooding of v1.27.0 surfaced that Claude Code couldn't DISCOVER the agent tools we'd just shipped. Asked to 'create a chatbot for Cypress Pine HVAC and put it on the homepage,' Claude Code spent 33s + 7 tool calls inspecting list_blocks, find no chat-widget block type, and concluded 'SF doesn't expose a landing-page chatbot widget' — falling back to suggesting send_conversation_turn (the v1.18 SMS conversation primitive). The agent tools shipped in v1.26.x (configure_llm_provider, create_agent, publish_agent, list_agents, update_agent_blueprint) and v1.26.2 (run_agent_evals, tail_agent_conversations, get_agent_conversation, replay_conversation, get_agent_metrics) WERE there, just not surfaced through the noise of ~50 flat tools. WHY THIS HAPPENS: tools.js is 200KB+ flat catalog; welcome.js (the MCP `instructions` payload Claude Code reads as system context) was 309 lines all about the v1.18-era workspace creation flow with no mention of agents as a top-level primitive; tool descriptions described WHAT each tool does instead of WHEN to call it. PHILOSOPHY APPLIED — Karpathy 'LLM is the scheduler, don't compete with it': the antifragile fix is BETTER CONTEXT, not server-side keyword matchers. As Claude gets better, richer descriptions help more, not less. This contrasts with the originally-planned discover_tools server-side matcher (cut from Phase C — would compete with LLM judgment and DEGRADE as models improve). Thin harnesses + fat skills: harness = the 10 tool definitions, unchanged; skill = the prose descriptions + welcome.js capability map, dramatically expanded. WHAT SHIPS: (1) WELCOME.JS CAPABILITY MAP. New top-level table mapping operator-language ('add a chatbot to my website', 'reply to inbound SMS automatically', 'add hero/services/faq section') to the right SF primitive (AGENT, CONVERSATION, BLOCK) with the entry-point tool. CRITICAL ANTI-PATTERN section explicitly tells the LLM: chat widgets are NOT blocks — don't list_blocks looking for chat. WRONG path → RIGHT path side-by-side. Plus a 'canonical short flow — build a website chatbot' walkthrough showing the 5-call sequence (configure_llm_provider → create_agent → publish_agent test → operator sandbox-tests → publish_agent live). Plus the observability tool list with one-line summaries of when to call each. (2) FIRST_CALL_BANNER reorganized as a capability map first (a/b/c/d/e/f primitives with what-they-are summaries), chatbot canonical flow second, workspace flow third (legacy reference). The banner is what Claude Code shows on first use — anchoring it on capability decisions BEFORE workspace creation re-orients the entire mental model. (3) USE-WHEN TRIGGER PHRASES on every v1.26.x agent tool description. Each description now LEADS with 'USE WHEN USER SAYS:' followed by 4-6 example natural-language phrases, then the technical doc. This makes semantic-match win during tool selection. configure_llm_provider, create_agent, list_agents, publish_agent, update_agent_blueprint, run_agent_evals, tail_agent_conversations, get_agent_conversation, replay_conversation, get_agent_metrics — all 10 updated. create_agent additionally calls out the DON'T-CONFUSE-WITH cases (list_blocks, send_conversation_turn) inline. (4) VERSION constant bumped 1.18.1 → 1.27.1 (was stale 9 versions back). NO new tools, NO new endpoints, NO new migrations, NO new env vars. Backend redeploy NOT required (this only touches the MCP package). MCP republish only. EXPECTED IMPACT: the same 'create a chatbot for Cypress Pine HVAC' prompt that wasted 7 tool calls in v1.27.0 should now resolve in 3-5 calls (configure_llm_provider [if needed] → create_agent → publish_agent test). Phase B (v1.28.0 — build_website_chatbot skill bundle collapsing the 4-call sequence into 1, plus optional auto-inject onto a landing page block) and Phase C (v1.29.0 — get_skill_guide markdown how-tos, voice/email skill bundles, optional tool-name prefix renames) follow.",
   "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.27.0: AGENTS GET A PROPER HOME — discoverable nav + full in-dashboard agent management. PROBLEM v1.27 SOLVES: after v1.26.2 the chat-AI agents lived at /admin/agents (not in the sidebar nav, type-the-URL discovery), confusable with the existing /agents/runs (workflow run log) and /automations (workflow archetype configurator). SF clients had to use Claude Code MCP for ALL agent edits — no in-dashboard editing path. WHAT SHIPS: (1) NAV REWORK. /admin/agents promoted to /agents (operator-facing top-level route, not SF-internal). 'Agents' added to sidebar nav between Bookings and Pages with Bot icon (lucide-react). The existing /agents/runs (workflow runs log, 15 cross-refs) coexists at the same level — Next.js resolves the static 'runs' segment before falling through to dynamic '[id]', so /agents (chat AI list), /agents/<uuid>/* (chat AI detail), and /agents/runs (workflow runs) all route correctly. (2) AGENT DETAIL SHELL. New /agents/[id]/layout.tsx renders the agent header (← All agents back link, name, status pill, version, archetype, slug) + AgentTabs nav with 5 tabs: Overview / Sandbox / Conversations / Settings / Evals. Each child route is a tab body. (3) OVERVIEW TAB (/agents/[id]/page.tsx). At-a-glance stat grid (24h conversations, validator pass rate, eval pass rate, avg latency); daily-token-usage progress bar (green/amber/rose at 70/90%); 3 most recent conversations with quality marks visible; embed snippet copy block. OverviewActions client component lets operator toggle status (draft/test/live/paused) — 'live' triggers the eval gate server-side; failure surfaces the failing scenarios in-line with hint to fix in Settings. (4) SETTINGS TAB (/agents/[id]/settings + settings-client.tsx). INLINE BLUEPRINT EDITOR replicating MCP update_agent_blueprint without leaving the dashboard. Greeting textarea, capability checklist (5 capabilities: look_up_availability, book_appointment, find_my_existing_appointment, escalate_to_human, provide_faq_answer), FAQ rows (add/edit/delete), pricing facts rows (add/edit/delete with label+amount+currency), optional change-note input, Save button. saveAgentBlueprintAction validates with Zod, calls updateAgentBlueprint server action (bumps current_version + writes new agent_versions row), revalidates path, returns new version number. (5) EVALS TAB (/agents/[id]/evals + evals-client.tsx). Lists all 8 platform-owned scenarios (description + severity pill + category) with latest result inline (passed / failed / not run); failed rows show failure reasons + collapsible 'Agent response' detail. 'Run evals now' button calls runEvalsAction → runEvalSuite → returns updated summary. Header shows aggregate pass count + meets-gate badge + last-run timestamp. (6) CONVERSATIONS TAB (existing, polished). Same expand-in-place transcript as v1.26.2 plus QualityMarker client component on every expanded conversation: good/bad pill toggles + notes input. markConversationQualityAction writes operator_quality + operator_notes (already in schema since v1.26.0); collapsed-row preview shows the quality mark + first 60 chars of notes so operators can scan the list for already-reviewed chats. (7) SERVER ACTIONS (lib/agents/actions.ts). saveAgentBlueprintAction, setAgentStatusAction, runEvalsAction, markConversationQualityAction. All workspace-scoped via getOrgId(). Zod-validated input. Each calls revalidatePath so the dashboard reflects new state without hard refresh. (8) DUPLICATE-HEADER TRIM. /agents/[id]/test/page.tsx and /agents/[id]/conversations/page.tsx had their own h1 + back-link UI (from v1.26.2 when they were at /admin/agents); v1.27.0 removes those since the layout.tsx provides them at the parent level. Cleaner, no duplication. NO new migrations (v1.26.0 0044 still covers everything; operator_quality + operator_notes columns already there). NO new env vars. Backend redeploy + npm publish required. DEFERRED to v1.28: real Anthropic-streaming-passthrough on /turn (mid-turn tool_use streaming); voice-receptionist + sms-followup-bot specialized eval scenarios; per-scenario eval history graph; conversation-detail standalone page; 'fold operator-marked good/bad chats into future eval scenarios' (signal pipeline). The existing /agents/runs workflow run log will move to /automations/runs in v1.28 once we've audited the 15 cross-refs.",
   "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.26.2: AGENTS GO SAFE — eval-gated publish + SSE streaming + admin debug surfaces + 5 new debug MCP tools. WHAT SHIPS: (1) EVAL SUITE. New lib/agents/eval-scenarios.ts ships 8 platform-owned scenarios per archetype (website-chatbot first; voice + sms inherit until v1.27/v1.28 specialization): 5 critical (3 prompt-injection + PII safety probes + 2 pricing-discipline checks — refuses-invented-price, refuses-competitor-match) and 3 warning (off-topic refusal, greeting-within-cap, escalation-on-complex-legal). Each scenario = id + description + userMessages[] + expected{responseContains|responseLacks|toolCallsRequired|validatorsAllPassed}. New lib/agents/eval-runner.ts runs each scenario as an ephemeral test conversation (channelMeta.eval_run=true so they don't pollute tail_conversations), captures actuals, persists to agent_evals, returns aggregate {totalRun, passed, failed, passRate, meetsPublishGate, results[]}. (2) EVAL-GATED PUBLISH. lib/agents/store.publishAgent now runs the eval suite before flipping status='live' and rejects with error='eval_gate_failed' when passRate < 87.5% (≥7/8 of 8 default scenarios). Other transitions (draft, test, paused) unrestricted. SF-controlled emergency override via {force:true} (logged). evalSummary returned alongside ok=true so the operator sees what passed. (3) SSE STREAMING ON /turn. POST /api/v1/public/agent/<slug>/turn now branches on Accept: text/event-stream | body.stream=true | ?stream=1: emits 'start' (conversation_id), 'delta' chunks (~28-char word-aware splits, ~22ms gap for typewriter UX), 'done' (conversation_id + validators_critical_failed). Runtime still buffers full response for validator gating BEFORE any byte streams (critical-fail still replaces the whole text). Real Anthropic-passthrough streaming with mid-turn tool-use events queued for v1.27 alongside multi-step tool-call streaming. embed.js (v1.26.2) consumes SSE: assistant message DOM node grows in place as deltas arrive, falls back to JSON if Content-Type isn't text/event-stream. (4) ADMIN DEBUG SURFACES. /(dashboard)/admin/agents — workspace agent roster with status pills, daily-token-usage bars, quick links to /test + /conversations. /(dashboard)/admin/agents/[id]/test — interactive sandbox client (TestSandboxClient) consuming SSE, identical UX to embed widget; surfaces conversation_id so operator can call get_agent_conversation MCP tool to inspect. /(dashboard)/admin/agents/[id]/conversations — server-rendered list of recent customer chats (eval-runs filtered by default), expand-in-place to show full transcript with tool_calls + tool_results + validator pass/fail per assistant turn. (5) 5 NEW DEBUG MCP TOOLS. run_agent_evals({agent_id}) — manual eval suite trigger (publish auto-runs too). tail_agent_conversations({agent_id, limit?, include_eval_runs?}) — newest-first conversation list with first_user_message preview. get_agent_conversation({conversation_id}) — full transcript with all tool_calls + tool_results + validator results per turn. replay_conversation({conversation_id}) — re-run user messages against current blueprint, returns original_turns + replay_turns side-by-side for regression-testing blueprint changes. get_agent_metrics({agent_id, since_hours?}) — aggregate stats over a time window: conversations + turns + tokens_in/out + avg_latency_ms + validator_pass_rate (% of assistant turns where ALL validators passed) + eval_pass_rate (latest result per scenario). NO new migrations (0044 from v1.26.0 still covers all five agent tables). NO new env vars. Backend redeploy + npm publish required. DEFERRED TO v1.27: real Anthropic-streaming-passthrough on /turn (mid-turn tool_use streaming); voice-receptionist + sms-followup-bot specialized eval scenarios; operator-quality marking (good/bad/notes) on /conversations; conversation-detail standalone page (/conversations/[convId]/page.tsx); per-scenario eval history graph.",
   "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.26.1: AGENTS GO LIVE — BYOK + embed.js + 5 new MCP tools. WHAT SHIPS: (1) BYOK (BRING-YOUR-OWN-KEY). SF clients (the agencies + their HVAC/dental/coaching operators) use Claude Code or Cursor to build agents — they already have Anthropic API keys. SF doesn't markup LLM cost; SF charges per agent turn (separate revenue line). lib/agents/runtime.ts swaps from process.env.ANTHROPIC_API_KEY to getAIClient({ orgId }) — the same BYOK helper that lib/ai/client.ts already exposes for the rest of the platform (encrypted at rest in organizations.integrations.anthropic.apiKey with v1. prefix; decryptIfNeeded resolves at call time). When a workspace hasn't configured a key, executeTurn returns { ok: false, reason: 'llm_not_configured', fallbackMessage } — the public turn endpoint surfaces a graceful 'I'm not set up to chat yet — please reach out directly' instead of crashing. (2) EMBED.JS — single-line install on operator's website. New endpoint GET /api/v1/public/agent/<orgSlug>--<agentSlug>/embed.js returns a self-contained IIFE (~3KB) that injects a bottom-right chat bubble + chat panel. Click bubble → panel opens → user types → POST to /turn → response appears. Anonymous session id stored in localStorage (sf_agent_session_<agentSlug>) so multi-message sessions thread together server-side via agent_conversations.anonymous_session_id. Themed via agent's blueprint.greeting + workspace's theme.primaryColor. Returns no-op script (still 200) when agent not 'live' so operator's site console stays clean during draft → test → live progression. CSS namespaced .sf-agent-* (zero collision risk on operator's site). Operator's install snippet: <script src=\"https://app.seldonframe.com/api/v1/public/agent/cypress-pine-hvac--default/embed.js\" async></script>. (3) 5 NEW MCP TOOLS so SF clients build + ship agents from Claude Code without touching the dashboard: configure_llm_provider({ provider: 'anthropic'|'openai', api_key }) — first-run setup, encrypts + writes to organizations.integrations[provider].apiKey. create_agent({ name, archetype, channel, capabilities?, faq?, pricing_facts?, greeting? }) — wraps lib/agents/store.createAgent, returns { agent, embed_url, turn_url, next_steps[] } so the SF client immediately gets the embed snippet to drop on their operator's site. update_agent_blueprint({ agent_id, patch, publish_notes? }) — wraps updateAgentBlueprint, bumps current_version + writes new agent_versions row. publish_agent({ agent_id, status: draft|test|live|paused }) — flips status. list_agents() — workspace-scoped roster with id/name/slug/channel/archetype/status/version/tokens/budget/created. (4) /api/v1/agents route extended with op='set_llm_key' handler so the new MCP tool routes through one endpoint. Validates provider ∈ {anthropic, openai}, api_key length ≥ 10, encrypts via encryptValue (returns 503 encryption_unavailable when ENCRYPTION_KEY env not set), merges into organizations.integrations preserving other providers' entries (kit, mailchimp, etc.). NO new migrations (0044 from v1.26.0 covers all agent tables). NO new env vars beyond ENCRYPTION_KEY (already required for the existing BYOK path). Backend redeploy + npm publish required. DEFERRED TO v1.26.2: SSE streaming on /turn; eval suite runner (executes scenarios from blocks/agent-website-chatbot/SKILL.md against the live agent); eval-gated publish (require ≥7/8 scenarios pass before flipping to live); /admin/agents/[id]/{conversations,test} debug surfaces; MCP debug tools (tail_agent_conversations, get_agent_conversation, replay_conversation, get_agent_metrics).",

package/src/tools.js

@@ -4051,11 +4051,12 @@ export const TOOLS = [
   {
     name: "configure_llm_provider",
     description:
-      "Set the LLM API key for this workspace's agents (BYOK — Bring Your Own Key). " +
+      "USE WHEN USER SAYS: 'set up Anthropic key for my agents', 'add my OpenAI key', 'configure BYOK for agents', 'why is my chatbot saying it's not configured?' " +
+      "FIRST-RUN setup BEFORE create_agent. Sets the LLM API key for this workspace's agents (BYOK — Bring Your Own Key). " +
       "The OPERATOR pays the LLM provider directly (Anthropic / OpenAI / etc.); SF charges separately for agent platform usage. " +
       "Stored encrypted at rest using the deployment's ENCRYPTION_KEY. " +
       "Operators get keys from console.anthropic.com (recommended for v1.26.x — best tool-use support) or platform.openai.com. " +
-      "Call this BEFORE create_agent — agents fail-graceful with 'I'm not set up yet' if no key configured.",
+      "Skip if the workspace already has a key — agents fail-graceful with 'I'm not set up yet' if no key configured, so a 'not configured' chatbot response means CALL THIS TOOL.",
     inputSchema: obj(
       {
         workspace_id: str("Workspace id (bearer workspace)."),
@@ -4088,12 +4089,14 @@ export const TOOLS = [
   {
     name: "create_agent",
     description:
-      "Create a new agent for this workspace. Agents are conversational interfaces (web chat, voice, SMS) that answer FAQs, book appointments, and escalate to humans — composed from typed primitives + the workspace's Soul (industry, voice, services). " +
+      "USE WHEN USER SAYS: 'add a chatbot to my website', 'add an AI assistant to my landing page', 'put a chat widget on my site', 'create a website chatbot', 'add an AI agent that answers customer questions', 'I want chat on my homepage', 'build me a chatbot for [business]'. " +
+      "DON'T confuse with: list_blocks (chat widgets are NOT a block type — agents are a separate primitive); send_conversation_turn (that's for inbound SMS/email auto-reply, NOT a website widget). If the operator wants chat on their website, THIS is the tool. " +
+      "Creates a new agent for this workspace. Agents are conversational interfaces (web chat, voice, SMS) that answer FAQs, book appointments, and escalate to humans — composed from typed primitives + the workspace's Soul (industry, voice, services). " +
       "WHAT GETS COMPOSED AUTOMATICALLY: persona derived from soul.industry + soul.voice; FAQ knowledge from your `faq` array; pricing facts from `pricing_facts` (validators block any $-amount the agent invents that's not in this list); typed tools (look_up_availability, book_appointment, find_my_existing_appointment, escalate_to_human, provide_faq_answer). " +
-      "WHAT YOU PROVIDE: name, archetype (website-chatbot for v1.26.x; voice-receptionist + sms-followup-bot queued), channel (web_chat / voice / sms / email), inline FAQ pairs, allowed pricing facts, optional greeting. " +
-      "STATUS LIFECYCLE: created in 'draft' (not callable). Flip to 'test' to chat with it in sandbox (tool calls return synthetic results — no real bookings). Flip to 'live' once you're confident. v1.26.2 will add eval-gating that blocks 'live' until a test scenario suite passes. " +
+      "WHAT YOU PROVIDE: name, archetype (website-chatbot for v1.26.x+; voice-receptionist + sms-followup-bot queued), channel (web_chat / voice / sms / email), inline FAQ pairs, allowed pricing facts, optional greeting. " +
+      "STATUS LIFECYCLE: created in 'draft' (not callable). Flip to 'test' to chat with it in sandbox. Flip to 'live' once you're confident — v1.26.2+ eval-gates 'live' until 8-scenario suite passes ≥87.5%. " +
       "SAFETY: response validators run on every turn — quotes_only_from_soul_pricing (critical, blocks hallucinated $X), no_prompt_injection_echo (critical), no_pii_leak (critical), no_avoid_words (warning), response_length_under_cap (warning). Critical fail = agent says 'let me check + escalate' instead of sending the bad response. " +
-      "OUTPUT: the agent's embed URL (one-line <script> for the operator's website) and turn URL (POST endpoint for direct API integration).",
+      "OUTPUT: the agent's embed URL (one-line <script> for the operator's website) and turn URL (POST endpoint for direct API integration). Tell the operator to drop the script tag on their site OR show them the dashboard sandbox at /agents/[id]/test to chat with it.",
     inputSchema: obj(
       {
         workspace_id: str("Workspace id (bearer workspace)."),
@@ -4171,7 +4174,8 @@ export const TOOLS = [
   {
     name: "list_agents",
     description:
-      "List all agents in this workspace with status (draft/test/live/paused), version, daily token usage vs budget, and metadata. Use to find an agent_id before calling publish_agent or update_agent_blueprint, or to audit which agents are live across a workspace.",
+      "USE WHEN USER SAYS: 'show me my agents', 'which chatbots do I have', 'list agents in this workspace', 'is the HVAC chatbot live yet?'. " +
+      "Lists all agents in the workspace with status (draft/test/live/paused), version, daily token usage vs budget, and metadata. Use to find an agent_id before calling publish_agent / update_agent_blueprint / get_agent_metrics, or to audit which agents are live across a workspace.",
     inputSchema: obj(
       { workspace_id: str("Workspace id (bearer workspace).") },
       ["workspace_id"],
@@ -4189,7 +4193,9 @@ export const TOOLS = [
   {
     name: "publish_agent",
     description:
-      "Change an agent's status: draft → test (sandboxed playground; tools return synthetic results), test → live (real bookings, real escalations, customer-facing), live → paused (chat bubble disabled). v1.26.2 will add eval-gating that blocks 'live' until the agent's eval suite passes; for v1.26.x, the operator-side responsibility is to test in 'test' mode before flipping to 'live'.",
+      "USE WHEN USER SAYS: 'put my chatbot live', 'enable the agent', 'switch agent to test mode', 'pause the chatbot', 'go live with the assistant', 'turn off the chatbot temporarily'. " +
+      "Changes an agent's status: draft → test (sandboxed playground), test → live (real bookings, real escalations, customer-facing), live → paused (chat bubble disabled). " +
+      "EVAL GATE (v1.26.2+): flipping to 'live' AUTO-RUNS the 8-scenario eval suite — rejects with error='eval_gate_failed' if pass rate < 87.5%. The response includes evalSummary so you can show the operator which scenarios failed and route them to /agents/[id]/settings to fix. Use { force: true } to bypass (logged; SF emergencies only).",
     inputSchema: obj(
       {
         workspace_id: str("Workspace id (bearer workspace)."),
@@ -4215,8 +4221,10 @@ export const TOOLS = [
   {
     name: "update_agent_blueprint",
     description:
-      "Update an agent's blueprint (FAQ, pricing facts, greeting, capabilities). Bumps current_version + writes a new agent_versions row for rollback. The agent's status is unchanged — flip to test/live separately when you're ready. " +
+      "USE WHEN USER SAYS: 'add this FAQ to the chatbot', 'update agent pricing', 'change the greeting', 'remove booking from the agent', 'the chatbot answer for X is wrong, fix it', 'add another service to the agent'. " +
+      "Updates an agent's blueprint (FAQ, pricing facts, greeting, capabilities). Bumps current_version + writes a new agent_versions row for rollback. The agent's status is unchanged — flip to test/live separately. " +
       "PATCH SEMANTICS: arrays REPLACE (not merge). If you want to ADD a single FAQ pair, fetch the current blueprint first via list_agents, append your new pair, and submit the full updated array. " +
+      "After a blueprint change, RE-RUN evals before promoting to live (use run_agent_evals or just call publish_agent({status:'live'}) which auto-runs them). " +
       "Common reasons to call this: operator added new FAQ entries; pricing changed; greeting needs A/B testing; restricting capabilities (e.g. removing book_appointment to make agent answer-only).",
     inputSchema: obj(
       {
@@ -4260,8 +4268,9 @@ export const TOOLS = [
   {
     name: "run_agent_evals",
     description:
-      "Run the platform's safety + behavior eval suite against this agent. The 8-scenario suite covers prompt-injection probes (ignore-instructions, role-swap), PII probes (customer-list leak), pricing discipline (refuses invented prices, refuses competitor match), scope refusal (off-topic), greeting + escalation. Each scenario runs through the live blueprint as an ephemeral test conversation, results persist to agent_evals. " +
-      "publish_agent with status='live' AUTOMATICALLY runs this and gates on ≥87.5% pass. Call this directly to dry-run before publishing or to verify after a blueprint update.",
+      "USE WHEN USER SAYS: 'test the chatbot against safety scenarios', 'run evals', 'check if my agent passes the safety suite', 'is the chatbot safe?', 'why did my agent fail the publish gate?'. " +
+      "Runs the platform's 8-scenario safety + behavior eval suite against this agent: prompt-injection probes (ignore-instructions, role-swap), PII probes (customer-list leak), pricing discipline (refuses invented prices, refuses competitor match), scope refusal (off-topic), greeting + escalation. Each scenario runs through the live blueprint as an ephemeral test conversation; results persist to agent_evals. " +
+      "publish_agent({status:'live'}) AUTOMATICALLY runs this and gates on ≥87.5% pass. Call THIS tool directly to dry-run before publishing or to verify after a blueprint update.",
     inputSchema: obj(
       {
         workspace_id: str("Workspace id (bearer workspace)."),
@@ -4282,7 +4291,8 @@ export const TOOLS = [
   {
     name: "tail_agent_conversations",
     description:
-      "List recent conversations for an agent — newest first. Excludes eval-runs and replay-runs by default (set include_eval_runs=true to see them). Each row includes status, turn_count, tokens, llm_cost_cents, and the customer's first message preview so you can spot patterns (most common questions, escalations, etc.) without opening each transcript. " +
+      "USE WHEN USER SAYS: 'show me recent chats with the agent', 'what are customers asking the chatbot', 'tail conversations', 'list the latest 20 chatbot sessions', 'what's been happening on the agent today?'. " +
+      "Lists recent conversations for an agent — newest first. Excludes eval-runs and replay-runs by default (set include_eval_runs=true to see them). Each row includes status, turn_count, tokens, llm_cost_cents, and the customer's first message preview so you can spot patterns (most common questions, escalations, etc.) without opening each transcript. " +
       "Use BEFORE get_agent_conversation to pick which conversation to drill into.",
     inputSchema: obj(
       {
@@ -4317,7 +4327,8 @@ export const TOOLS = [
   {
     name: "get_agent_conversation",
     description:
-      "Fetch the full transcript of a single conversation: every turn (user + assistant), all tool_calls (look_up_availability, book_appointment, escalate_to_human, etc.) with their inputs, all tool_results with success/error, validator_results per assistant turn (which validators passed/failed), tokens, latency, model. " +
+      "USE WHEN USER SAYS: 'show me that conversation in detail', 'why did the chatbot say X?', 'debug this chat', 'what tools did the agent call in conversation Y', 'show the validator results for conversation Z'. " +
+      "Fetches the full transcript of a single conversation: every turn (user + assistant), all tool_calls (look_up_availability, book_appointment, escalate_to_human, etc.) with their inputs, all tool_results with success/error, validator_results per assistant turn (which validators passed/failed), tokens, latency, model. " +
       "Use this to debug WHY an agent gave a specific answer — was it a tool failure? a validator gating? wrong info in blueprint? Pair with replay_conversation to test a blueprint fix.",
     inputSchema: obj(
       {
@@ -4344,7 +4355,8 @@ export const TOOLS = [
   {
     name: "replay_conversation",
     description:
-      "Replay a past conversation's user messages against the agent's CURRENT blueprint, returning the original responses + the new responses side-by-side. Lets you regression-test a blueprint change ('did adding this FAQ break the booking flow?') without touching production. " +
+      "USE WHEN USER SAYS: 'will my new FAQ break the booking flow?', 'replay this chat against the new blueprint', 'regression test the chatbot', 'test if my recent change still answers this conversation correctly'. " +
+      "Replays a past conversation's user messages against the agent's CURRENT blueprint, returning the original responses + the new responses side-by-side. Lets you regression-test a blueprint change without touching production. " +
       "Creates a new ephemeral test-status conversation tagged with replay_of=<original>; the original is untouched.",
     inputSchema: obj(
       {
@@ -4369,8 +4381,9 @@ export const TOOLS = [
   {
     name: "get_agent_metrics",
     description:
+      "USE WHEN USER SAYS: 'is my chatbot healthy?', 'how's the agent performing?', 'show me agent stats', 'what's my chatbot's pass rate this week?', 'agent dashboard ping'. " +
       "Aggregate health stats for an agent over a time window: conversations + turns count, tokens (in/out), avg latency, validator pass rate (% of assistant turns where ALL validators passed), latest eval pass rate (last result per scenario). " +
-      "Use as a dashboard ping — 'is my agent healthy?' If validator_pass_rate drops or eval_pass_rate falls below the 87.5% gate, the agent shouldn't be promoted to live.",
+      "Use as a dashboard ping. If validator_pass_rate drops or eval_pass_rate falls below the 87.5% gate, the agent shouldn't be promoted to live.",
     inputSchema: obj(
       {
         workspace_id: str("Workspace id (bearer workspace)."),

package/src/welcome.js

@@ -8,7 +8,7 @@
 // stripped. `create_full_workspace` is the only workspace-creation
 // path mentioned anywhere in this briefing.
 
-export const VERSION = "1.18.1";
+export const VERSION = "1.27.2";
 
 export const WELCOME_MARKDOWN = `# SeldonFrame — create a real Business OS in one conversation
 
@@ -19,6 +19,102 @@ CRM, and AI agents — all on a real subdomain.
 
 ---
 
+## Capability map — pick the right primitive BEFORE you explore tools
+
+SeldonFrame has SEVEN top-level primitives. They are NOT the same as
+each other. Pick correctly from this map FIRST. Don't go fishing in
+\`tools/list\` looking for the right one — most mistakes happen because
+the wrong primitive was chosen and the LLM got stuck searching for a
+tool that doesn't exist in that primitive.
+
+| Operator says…                                                 | Primitive            | Entry-point tools                                                     |
+| -------------------------------------------------------------- | -------------------- | --------------------------------------------------------------------- |
+| "Build me a website / business / CRM"                          | **WORKSPACE**        | \`create_workspace_v2\` then block tools                              |
+| "Add an AI chatbot to my website / landing page"               | **AGENT** (web chat) | \`create_agent\` ({ archetype: "website-chatbot" }) → embed.js script |
+| "Build me a 24/7 AI receptionist for my phone"                 | AGENT (voice)        | (v1.28+ — voice archetype shipping soon)                              |
+| "Reply to inbound customer SMS / email automatically"          | **CONVERSATION**     | \`send_conversation_turn\` (one-shot Soul-aware reply)                |
+| "Add a hero / services / FAQ / CTA section to a page"          | **BLOCK**            | \`get_block_skill\` + \`persist_block\`                               |
+| "Send a campaign email / SMS blast"                            | **MESSAGING**        | \`send_email\` / \`send_sms\`                                         |
+| "Update contacts / deals / bookings in the CRM"                | **CRM**              | \`list_contacts\`, \`create_deal\`, \`move_deal_stage\`, etc.         |
+
+### CRITICAL anti-pattern: chat widgets are NOT blocks
+
+If the operator says "add a chatbot to the website," "add chat to my
+landing page," or "I want an AI assistant on my site," **do NOT look
+in the blocks catalog**. Blocks are static page sections (hero,
+services, faq, cta, booking, intake). The chat-widget primitive is a
+separate concept called an **AGENT**, shipped in v1.26+.
+
+WRONG path (don't do this):
+- \`list_blocks\` → search for "chat" → conclude "no chat block exists" → propose SMS workaround
+
+RIGHT path:
+- \`create_agent({ archetype: "website-chatbot", faq, pricing_facts, greeting })\`
+- → returns \`embed_url\`
+- Drop \`<script src="EMBED_URL" async></script>\` on any page (or use the inline-edit option in v1.28+)
+
+The same applies to voice (use AGENT with archetype="voice-receptionist"
+when shipping) and SMS auto-reply (use the CONVERSATION primitive's
+\`send_conversation_turn\`, not an AGENT).
+
+---
+
+## Build a website chatbot — the canonical short flow
+
+Most operators asking for "AI on my website" want this exact flow.
+~5 tool calls, end-to-end:
+
+\`\`\`
+1. configure_llm_provider({ provider: "anthropic", api_key })
+   // Skip if workspace already has a key. SF uses BYOK — operator pays
+   // Anthropic directly for the LLM; SF charges per agent turn separately.
+
+2. create_agent({
+     workspace_id,
+     name: "Cypress Pine HVAC Helper",
+     archetype: "website-chatbot",
+     channel: "web_chat",
+     greeting: "Hi! Asking about HVAC service in Phoenix? I can book you in.",
+     faq: [
+       { q: "What areas do you service?", a: "Phoenix, Scottsdale, Tempe..." },
+       { q: "Do you offer emergency service?", a: "Yes — 24/7..." }
+     ],
+     pricing_facts: [
+       { label: "Service call", amount: 89, currency: "USD" },
+       { label: "AC tune-up", amount: 149, currency: "USD" }
+     ]
+   })
+   // → returns { agent, embed_url, turn_url, next_steps }
+
+3. publish_agent({ workspace_id, agent_id, status: "test" })
+   // Sandbox-test the agent in the dashboard before flipping to live.
+
+4. (Operator tests in /agents/[id]/test sandbox.)
+
+5. publish_agent({ workspace_id, agent_id, status: "live" })
+   // AUTO-RUNS the 8-scenario eval gate. Fails if pass rate < 87.5%.
+   // Surfaces failing scenarios so the operator can fix in /agents/[id]/settings.
+\`\`\`
+
+Then the operator drops the embed snippet onto their site, OR you can
+help them edit a block on the SF-hosted landing page to include it.
+
+After publish, observability tools the operator can call from Claude
+Code at any time:
+- \`list_agents\` — workspace roster
+- \`tail_agent_conversations\` — recent customer chats with first-message preview
+- \`get_agent_conversation\` — full transcript with tool calls + validator results
+- \`get_agent_metrics\` — health stats over a time window
+- \`run_agent_evals\` — manual eval suite trigger
+- \`replay_conversation\` — re-run a past chat against current blueprint
+
+The dashboard surfaces (\`/agents\`, \`/agents/[id]/test\`, \`/agents/[id]/settings\`,
+\`/agents/[id]/evals\`, \`/agents/[id]/conversations\`) provide the same view
+without leaving the browser — operators iterate FAQ/pricing/greeting inline
+and Save bumps a new blueprint version.
+
+---
+
 ## NEVER CREATE LOCAL FILES
 
 SeldonFrame workspaces are created via the \`create_full_workspace\`
@@ -306,4 +402,4 @@ admin dashboard. Pre-fills their email automatically.
 <https://seldonframe.com> · **Discord:** <https://discord.gg/sbVUu976NW>
 `;
 
-export const FIRST_CALL_BANNER = `🚀 SeldonFrame v1.18.1 is connected. PREFERRED workspace creation: create_workspace_v2 → IN PARALLEL for all 7 recommended_blocks (hero, services, about, faq, cta, booking, intake): get_block_skill + persist_block → complete_workspace_v2 → finalize_workspace({ workspace_id, email }). The v2 flow puts YOUR LLM in charge of every operator-facing surface using one SKILL.md per block. Each block's prop schema is server-validated. Run blocks in PARALLEL (Promise.all) — sequential takes 60+ seconds. v1.10+ TIER 2 CUSTOMIZE TOOLS: regenerate_block (re-do one block with operator instructions; thin-harness — server bundles context, your LLM generates), upload_workspace_image (set logo/hero_background; v1.10.1+ accepts image_url or local_file_path — DON'T base64 unless you have to, the encoded string eats your tool-call token budget). v1.11+ STRUCTURAL PRIMITIVES: get_landing_structure → move_section → delete_section. INDEX-based, handle duplicate section types, atomic. v1.12+ COMPOSITE TREES: add_composite_section / update_composite_section — manifest ANY block (comparison, pricing, "how it works," stats, custom CTAs) from 12 low-level primitives. Server validates + renders; YOUR LLM composes. Read the SKILL.md via get_block_skill('composite') before composing. Every URL is real. NEVER create local files. Skipping finalize_workspace leaves the operator with no admin login.`;
+export const FIRST_CALL_BANNER = `🚀 SeldonFrame v1.27.2 is connected. CAPABILITY MAP — pick the right primitive BEFORE exploring tools: (a) "build me a website" → WORKSPACE → create_workspace_v2 + block tools. (b) "add a chatbot to my website / landing page" → AGENT (web chat) → create_agent({archetype:"website-chatbot"}) → embed.js script. CRITICAL ANTI-PATTERN: chat widgets are NOT blocks. Don't search list_blocks for chat — chat agents are a separate primitive (v1.26+). (c) "auto-reply to inbound SMS/email" → CONVERSATION → send_conversation_turn (one-shot Soul-aware reply, not a website widget). (d) "add hero/services/faq/cta section" → BLOCK → get_block_skill + persist_block. (e) "send campaign email/sms" → MESSAGING. (f) CRM ops → list_contacts/create_deal/etc. CHATBOT CANONICAL FLOW (5 calls): configure_llm_provider (BYOK Anthropic, skip if already set) → create_agent (faq + pricing_facts + greeting) → publish_agent({status:"test"}) → operator sandbox-tests at /agents/[id]/test → publish_agent({status:"live"}) auto-runs 8-scenario eval gate (≥87.5% pass required). Observability tools after publish: list_agents, tail_agent_conversations, get_agent_conversation, get_agent_metrics, run_agent_evals, replay_conversation. Dashboard surfaces /agents, /agents/[id]/test, /agents/[id]/settings, /agents/[id]/evals, /agents/[id]/conversations let operators iterate without leaving the browser. WORKSPACE FLOW (legacy reference): create_workspace_v2 → IN PARALLEL for all 7 recommended_blocks: get_block_skill + persist_block → complete_workspace_v2 → finalize_workspace({workspace_id, email}). v1.10+ TIER 2 CUSTOMIZE: regenerate_block, upload_workspace_image (image_url preferred over base64). v1.11+ STRUCTURAL: get_landing_structure, move_section, delete_section. v1.12+ COMPOSITES: add_composite_section / update_composite_section. Run blocks in PARALLEL (Promise.all). Skipping finalize_workspace leaves the operator with no admin login. Every URL is real. NEVER create local files.`; PREFERRED workspace creation: create_workspace_v2 → IN PARALLEL for all 7 recommended_blocks (hero, services, about, faq, cta, booking, intake): get_block_skill + persist_block → complete_workspace_v2 → finalize_workspace({ workspace_id, email }). The v2 flow puts YOUR LLM in charge of every operator-facing surface using one SKILL.md per block. Each block's prop schema is server-validated. Run blocks in PARALLEL (Promise.all) — sequential takes 60+ seconds. v1.10+ TIER 2 CUSTOMIZE TOOLS: regenerate_block (re-do one block with operator instructions; thin-harness — server bundles context, your LLM generates), upload_workspace_image (set logo/hero_background; v1.10.1+ accepts image_url or local_file_path — DON'T base64 unless you have to, the encoded string eats your tool-call token budget). v1.11+ STRUCTURAL PRIMITIVES: get_landing_structure → move_section → delete_section. INDEX-based, handle duplicate section types, atomic. v1.12+ COMPOSITE TREES: add_composite_section / update_composite_section — manifest ANY block (comparison, pricing, "how it works," stats, custom CTAs) from 12 low-level primitives. Server validates + renders; YOUR LLM composes. Read the SKILL.md via get_block_skill('composite') before composing. Every URL is real. NEVER create local files. Skipping finalize_workspace leaves the operator with no admin login.`;