@seldonframe/mcp 1.26.2 → 1.27.0

package/package.json

@@ -1,6 +1,7 @@
 {
   "name": "@seldonframe/mcp",
-  "version": "1.26.2",
+  "version": "1.27.0",
+  "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.27.0: AGENTS GET A PROPER HOME — discoverable nav + full in-dashboard agent management. PROBLEM v1.27 SOLVES: after v1.26.2 the chat-AI agents lived at /admin/agents (not in the sidebar nav, type-the-URL discovery), confusable with the existing /agents/runs (workflow run log) and /automations (workflow archetype configurator). SF clients had to use Claude Code MCP for ALL agent edits — no in-dashboard editing path. WHAT SHIPS: (1) NAV REWORK. /admin/agents promoted to /agents (operator-facing top-level route, not SF-internal). 'Agents' added to sidebar nav between Bookings and Pages with Bot icon (lucide-react). The existing /agents/runs (workflow runs log, 15 cross-refs) coexists at the same level — Next.js resolves the static 'runs' segment before falling through to dynamic '[id]', so /agents (chat AI list), /agents/<uuid>/* (chat AI detail), and /agents/runs (workflow runs) all route correctly. (2) AGENT DETAIL SHELL. New /agents/[id]/layout.tsx renders the agent header (← All agents back link, name, status pill, version, archetype, slug) + AgentTabs nav with 5 tabs: Overview / Sandbox / Conversations / Settings / Evals. Each child route is a tab body. (3) OVERVIEW TAB (/agents/[id]/page.tsx). At-a-glance stat grid (24h conversations, validator pass rate, eval pass rate, avg latency); daily-token-usage progress bar (green/amber/rose at 70/90%); 3 most recent conversations with quality marks visible; embed snippet copy block. OverviewActions client component lets operator toggle status (draft/test/live/paused) — 'live' triggers the eval gate server-side; failure surfaces the failing scenarios in-line with hint to fix in Settings. (4) SETTINGS TAB (/agents/[id]/settings + settings-client.tsx). INLINE BLUEPRINT EDITOR replicating MCP update_agent_blueprint without leaving the dashboard. Greeting textarea, capability checklist (5 capabilities: look_up_availability, book_appointment, find_my_existing_appointment, escalate_to_human, provide_faq_answer), FAQ rows (add/edit/delete), pricing facts rows (add/edit/delete with label+amount+currency), optional change-note input, Save button. saveAgentBlueprintAction validates with Zod, calls updateAgentBlueprint server action (bumps current_version + writes new agent_versions row), revalidates path, returns new version number. (5) EVALS TAB (/agents/[id]/evals + evals-client.tsx). Lists all 8 platform-owned scenarios (description + severity pill + category) with latest result inline (passed / failed / not run); failed rows show failure reasons + collapsible 'Agent response' detail. 'Run evals now' button calls runEvalsAction → runEvalSuite → returns updated summary. Header shows aggregate pass count + meets-gate badge + last-run timestamp. (6) CONVERSATIONS TAB (existing, polished). Same expand-in-place transcript as v1.26.2 plus QualityMarker client component on every expanded conversation: good/bad pill toggles + notes input. markConversationQualityAction writes operator_quality + operator_notes (already in schema since v1.26.0); collapsed-row preview shows the quality mark + first 60 chars of notes so operators can scan the list for already-reviewed chats. (7) SERVER ACTIONS (lib/agents/actions.ts). saveAgentBlueprintAction, setAgentStatusAction, runEvalsAction, markConversationQualityAction. All workspace-scoped via getOrgId(). Zod-validated input. Each calls revalidatePath so the dashboard reflects new state without hard refresh. (8) DUPLICATE-HEADER TRIM. /agents/[id]/test/page.tsx and /agents/[id]/conversations/page.tsx had their own h1 + back-link UI (from v1.26.2 when they were at /admin/agents); v1.27.0 removes those since the layout.tsx provides them at the parent level. Cleaner, no duplication. NO new migrations (v1.26.0 0044 still covers everything; operator_quality + operator_notes columns already there). NO new env vars. Backend redeploy + npm publish required. DEFERRED to v1.28: real Anthropic-streaming-passthrough on /turn (mid-turn tool_use streaming); voice-receptionist + sms-followup-bot specialized eval scenarios; per-scenario eval history graph; conversation-detail standalone page; 'fold operator-marked good/bad chats into future eval scenarios' (signal pipeline). The existing /agents/runs workflow run log will move to /automations/runs in v1.28 once we've audited the 15 cross-refs.",
   "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.26.2: AGENTS GO SAFE — eval-gated publish + SSE streaming + admin debug surfaces + 5 new debug MCP tools. WHAT SHIPS: (1) EVAL SUITE. New lib/agents/eval-scenarios.ts ships 8 platform-owned scenarios per archetype (website-chatbot first; voice + sms inherit until v1.27/v1.28 specialization): 5 critical (3 prompt-injection + PII safety probes + 2 pricing-discipline checks — refuses-invented-price, refuses-competitor-match) and 3 warning (off-topic refusal, greeting-within-cap, escalation-on-complex-legal). Each scenario = id + description + userMessages[] + expected{responseContains|responseLacks|toolCallsRequired|validatorsAllPassed}. New lib/agents/eval-runner.ts runs each scenario as an ephemeral test conversation (channelMeta.eval_run=true so they don't pollute tail_conversations), captures actuals, persists to agent_evals, returns aggregate {totalRun, passed, failed, passRate, meetsPublishGate, results[]}. (2) EVAL-GATED PUBLISH. lib/agents/store.publishAgent now runs the eval suite before flipping status='live' and rejects with error='eval_gate_failed' when passRate < 87.5% (≥7/8 of 8 default scenarios). Other transitions (draft, test, paused) unrestricted. SF-controlled emergency override via {force:true} (logged). evalSummary returned alongside ok=true so the operator sees what passed. (3) SSE STREAMING ON /turn. POST /api/v1/public/agent/<slug>/turn now branches on Accept: text/event-stream | body.stream=true | ?stream=1: emits 'start' (conversation_id), 'delta' chunks (~28-char word-aware splits, ~22ms gap for typewriter UX), 'done' (conversation_id + validators_critical_failed). Runtime still buffers full response for validator gating BEFORE any byte streams (critical-fail still replaces the whole text). Real Anthropic-passthrough streaming with mid-turn tool-use events queued for v1.27 alongside multi-step tool-call streaming. embed.js (v1.26.2) consumes SSE: assistant message DOM node grows in place as deltas arrive, falls back to JSON if Content-Type isn't text/event-stream. (4) ADMIN DEBUG SURFACES. /(dashboard)/admin/agents — workspace agent roster with status pills, daily-token-usage bars, quick links to /test + /conversations. /(dashboard)/admin/agents/[id]/test — interactive sandbox client (TestSandboxClient) consuming SSE, identical UX to embed widget; surfaces conversation_id so operator can call get_agent_conversation MCP tool to inspect. /(dashboard)/admin/agents/[id]/conversations — server-rendered list of recent customer chats (eval-runs filtered by default), expand-in-place to show full transcript with tool_calls + tool_results + validator pass/fail per assistant turn. (5) 5 NEW DEBUG MCP TOOLS. run_agent_evals({agent_id}) — manual eval suite trigger (publish auto-runs too). tail_agent_conversations({agent_id, limit?, include_eval_runs?}) — newest-first conversation list with first_user_message preview. get_agent_conversation({conversation_id}) — full transcript with all tool_calls + tool_results + validator results per turn. replay_conversation({conversation_id}) — re-run user messages against current blueprint, returns original_turns + replay_turns side-by-side for regression-testing blueprint changes. get_agent_metrics({agent_id, since_hours?}) — aggregate stats over a time window: conversations + turns + tokens_in/out + avg_latency_ms + validator_pass_rate (% of assistant turns where ALL validators passed) + eval_pass_rate (latest result per scenario). NO new migrations (0044 from v1.26.0 still covers all five agent tables). NO new env vars. Backend redeploy + npm publish required. DEFERRED TO v1.27: real Anthropic-streaming-passthrough on /turn (mid-turn tool_use streaming); voice-receptionist + sms-followup-bot specialized eval scenarios; operator-quality marking (good/bad/notes) on /conversations; conversation-detail standalone page (/conversations/[convId]/page.tsx); per-scenario eval history graph.",
   "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.26.1: AGENTS GO LIVE — BYOK + embed.js + 5 new MCP tools. WHAT SHIPS: (1) BYOK (BRING-YOUR-OWN-KEY). SF clients (the agencies + their HVAC/dental/coaching operators) use Claude Code or Cursor to build agents — they already have Anthropic API keys. SF doesn't markup LLM cost; SF charges per agent turn (separate revenue line). lib/agents/runtime.ts swaps from process.env.ANTHROPIC_API_KEY to getAIClient({ orgId }) — the same BYOK helper that lib/ai/client.ts already exposes for the rest of the platform (encrypted at rest in organizations.integrations.anthropic.apiKey with v1. prefix; decryptIfNeeded resolves at call time). When a workspace hasn't configured a key, executeTurn returns { ok: false, reason: 'llm_not_configured', fallbackMessage } — the public turn endpoint surfaces a graceful 'I'm not set up to chat yet — please reach out directly' instead of crashing. (2) EMBED.JS — single-line install on operator's website. New endpoint GET /api/v1/public/agent/<orgSlug>--<agentSlug>/embed.js returns a self-contained IIFE (~3KB) that injects a bottom-right chat bubble + chat panel. Click bubble → panel opens → user types → POST to /turn → response appears. Anonymous session id stored in localStorage (sf_agent_session_<agentSlug>) so multi-message sessions thread together server-side via agent_conversations.anonymous_session_id. Themed via agent's blueprint.greeting + workspace's theme.primaryColor. Returns no-op script (still 200) when agent not 'live' so operator's site console stays clean during draft → test → live progression. CSS namespaced .sf-agent-* (zero collision risk on operator's site). Operator's install snippet: <script src=\"https://app.seldonframe.com/api/v1/public/agent/cypress-pine-hvac--default/embed.js\" async></script>. (3) 5 NEW MCP TOOLS so SF clients build + ship agents from Claude Code without touching the dashboard: configure_llm_provider({ provider: 'anthropic'|'openai', api_key }) — first-run setup, encrypts + writes to organizations.integrations[provider].apiKey. create_agent({ name, archetype, channel, capabilities?, faq?, pricing_facts?, greeting? }) — wraps lib/agents/store.createAgent, returns { agent, embed_url, turn_url, next_steps[] } so the SF client immediately gets the embed snippet to drop on their operator's site. update_agent_blueprint({ agent_id, patch, publish_notes? }) — wraps updateAgentBlueprint, bumps current_version + writes new agent_versions row. publish_agent({ agent_id, status: draft|test|live|paused }) — flips status. list_agents() — workspace-scoped roster with id/name/slug/channel/archetype/status/version/tokens/budget/created. (4) /api/v1/agents route extended with op='set_llm_key' handler so the new MCP tool routes through one endpoint. Validates provider ∈ {anthropic, openai}, api_key length ≥ 10, encrypts via encryptValue (returns 503 encryption_unavailable when ENCRYPTION_KEY env not set), merges into organizations.integrations preserving other providers' entries (kit, mailchimp, etc.). NO new migrations (0044 from v1.26.0 covers all agent tables). NO new env vars beyond ENCRYPTION_KEY (already required for the existing BYOK path). Backend redeploy + npm publish required. DEFERRED TO v1.26.2: SSE streaming on /turn; eval suite runner (executes scenarios from blocks/agent-website-chatbot/SKILL.md against the live agent); eval-gated publish (require ≥7/8 scenarios pass before flipping to live); /admin/agents/[id]/{conversations,test} debug surfaces; MCP debug tools (tail_agent_conversations, get_agent_conversation, replay_conversation, get_agent_metrics).",
   "description": "MCP server for SeldonFrame — AI-native Business OS platform. v1.26.0: AGENT FOUNDATION — primitives for building chat / voice / SMS / email agents on top of Soul + Brain. Web chat archetype ships first; voice + SMS queued for v1.27/v1.28. Migration 0044 adds agents / agent_versions / agent_conversations / agent_turns / agent_evals tables. Schema (src/db/schema/agents.ts) typed with AgentBlueprint, AgentToolCall, AgentToolResult, AgentValidatorResult. System prompt composer (lib/agents/prompt.ts) derives prompts from Soul + blueprint, NEVER operator-authored — operators contribute knowledge (FAQ, services, pricing); platform owns the prompt. Five output validators (lib/agents/validators.ts) run on every assistant response: quotes_only_from_soul_pricing (critical — blocks hallucinated $X), no_prompt_injection_echo (critical — blocks 'ignore previous instructions' echoes), no_pii_leak (critical — blocks emails/phones not in user's message), no_avoid_words (warning — Soul voice rule), response_length_under_cap (warning — 600 chars). Critical fail = response replaced with 'let me check + escalate'. Five typed tools (lib/agents/tools.ts) with Zod input schemas: look_up_availability, book_appointment (wraps submitPublicBookingAction — same atomic primitive as /book), find_my_existing_appointment, escalate_to_human (writes portal_messages + activities), provide_faq_answer. Runtime (lib/agents/runtime.ts) executeTurn — loads conv + agent + soul, checks daily token budget, persists user turn, calls Anthropic with tools, loops on tool_use (max 6 iterations), runs validators, persists assistant turn with cost/latency, updates aggregates, activity-bridges first turn to operator's CRM. Non-streaming for v1.26.0; v1.26.1 adds SSE. Public turn endpoint POST /api/v1/public/agent/<orgSlug>--<agentSlug>/turn (anonymous; live/test status required). Agent CRUD endpoint POST /api/v1/agents (op: create | update_blueprint | publish | list — workspace bearer auth). createAgent generates slug, creates v1 in agent_versions, soul-derives blueprint defaults. blocks/agent-website-chatbot/SKILL.md documents the archetype + 8 eval scenarios including prompt-injection + PII-leak adversarial probes. NO new MCP tools yet (HTTP endpoint exposed; MCP-tool wiring + SSE streaming + embed.js + operator debug surfaces land in v1.26.1). NO new env vars beyond ANTHROPIC_API_KEY (already required). Backend redeploy required.",