@ema.co/mcp-toolkit 2026.1.25 → 2026.1.26-3

package/docs/proposals/HANDOFF-tool-restructure.md

@@ -1,526 +0,0 @@
-# MCP Tool Restructure: Session Handoff
-
-**Date**: 2026-01-18  
-**Branch**: `feat/mcp-tool-restructure`  
-**Base**: `main` (PR #49 merged)
-**Status**: **IN PROGRESS** - V2 tools implemented, testing
-
----
-
-## Executive Summary
-
-Restructure MCP tools from **7 mode-based tools** to **4 LLM-optimized tools**.
-
-| Before (V1) | After (V2) |
-|-------------|------------|
-| 7 tools with mode params | 4 focused tools |
-| `persona(mode="list")` | `persona()` |
-| `data(mode="upload")` | `persona(id, data={upload:...})` |
-| `action`, `template`, `reference` | `catalog(type="actions")` |
-
-**Why**: Minimal tool count optimizes LLM tool selection. Data under persona (always persona-scoped). Catalog consolidates all reference data.
-
----
-
-## Current State
-
-### Implemented (V2 - Default)
-
-| Tool | Purpose | Key Operations |
-|------|---------|----------------|
-| `persona` | AI Employee entity + data | query, create, update, delete, analyze, sanitize, snapshot/history/restore, data ops |
-| `catalog` | Reference data | actions, templates, widgets, voices, patterns, concepts |
-| `sync` | Cross-env operations | run, status |
-| `env` | Environment info | list |
-
-**Total**: 4 tools exposed to LLM
-
-### Feature Flags
-
-| Flag | Effect |
-|------|--------|
-| (default) | V2 tools only (4 tools) |
-| `EMA_ENABLE_V1_TOOLS=true` | V1 mode-based tools (7 tools) |
-| `EMA_ENABLE_LEGACY_TOOLS=true` | Add legacy individual tools |
-
-### Files Changed
-
-| File | Status | Description |
-|------|--------|-------------|
-| `src/mcp/tools-v2.ts` | **NEW** | V2 tool definitions (4 tools) |
-| `src/mcp/server.ts` | **MODIFIED** | V2 routing, persona v2 handler, catalog handler |
-
-### V2 Persona Operations
-
-```typescript
-// Query (list/get)
-persona()                           // list all
-persona(id="abc")                   // get one
-persona(type="voice")               // filter
-
-// Create
-persona(create={name:"Bot", type:"voice"})
-persona(create={name:"Bot", from:"template-id"})  // from template
-persona(create={name:"Bot", from:"other-id"})     // clone
-
-// Update
-persona(id="abc", update={config:{...}})
-persona(id="abc", update={input:"add HITL"})
-
-// Delete
-persona(id="abc", delete=true)
-
-// Analyze
-persona(id="abc", analyze=true)
-persona(id="abc", analyze=true, fix=true)
-
-// History (snapshots)
-persona(id="abc", snapshot="message")   // create
-persona(id="abc", history=true)         // list
-persona(id="abc", restore="v3")         // rollback
-
-// Compare
-persona(id="abc", compare="def")
-
-// Data (owned by persona)
-persona(id="abc", data={list:true})
-persona(id="abc", data={upload:"/path/to/file"})
-persona(id="abc", data={generate:"5 customers"})
-persona(id="abc", data={embed:true})
-persona(id="abc", data={search:"pricing"})
-```
-
-### V2 Catalog Operations
-
-```typescript
-catalog(type="actions")                    // list actions
-catalog(type="actions", id="knowledge_search")  // get one
-catalog(type="actions", recommend="email")  // recommendations
-catalog(type="templates")                  // list templates
-catalog(type="widgets")                    // list widgets
-catalog(type="patterns")                   // list patterns
-catalog(type="concepts")                   // platform terms
-```
-
----
-
-## Research Summary
-
-### Industry Patterns
-
-| Server | Pattern | Tools | Example |
-|--------|---------|-------|---------|
-| Playwright | `browser_{action}` | 15+ | `browser_click`, `browser_type` |
-| Firecrawl | `firecrawl_{action}` | 6 | `firecrawl_scrape`, `firecrawl_crawl` |
-| Filesystem | `{action}` | 10+ | `read_file`, `write_file` |
-| Supabase | `{resource}_{action}` | 10+ | `list_tables`, `execute_sql` |
-
-### MCP Spec Guidance
-
-- **Tools**: Model-controlled, LLM invokes automatically
-- **Prompts**: User-controlled, user triggers explicitly
-- **Tool names**: `A-Z`, `a-z`, `0-9`, `_`, `-`, `.` allowed
-
-### Ema Backend Convention
-
-OpenAPI uses `{Verb}{Resource}`:
-- `GetPersona`, `ListPersonas`, `UpdateAiEmployee`
-
-MCP convention `{resource}_{action}` is compatible (just inverted order).
-
----
-
-## Proposed Tool Inventory
-
-### Persona Tools (9)
-
-```
-persona_list      - List all personas (basic info)
-persona_get       - Get persona details (with workflow)
-persona_create    - Create from template or clone
-persona_update    - Update config/workflow
-persona_delete    - Delete persona
-persona_clone     - Clone existing persona
-persona_compare   - Compare two personas
-persona_sanitize  - Clean up for export
-persona_analyze   - Analyze workflow/config/data issues
-```
-
-### Persona Version Tools (5)
-
-```
-persona_version_list    - List versions
-persona_version_get     - Get specific version
-persona_version_create  - Create snapshot
-persona_version_restore - Restore from version
-persona_version_compare - Compare versions
-```
-
-### Data Tools (5)
-
-```
-data_list    - List data sources
-data_get     - Get data source details
-data_upload  - Upload new data
-data_delete  - Delete data source
-data_search  - Search across data
-```
-
-### Document Tools (4)
-
-```
-document_create     - Create document (async)
-document_get        - Get document status/content
-document_regenerate - Regenerate document
-document_update     - Update with instructions
-```
-
-### Workflow Tools (4)
-
-```
-workflow_analyze  - Detect issues
-workflow_optimize - Suggest/apply fixes
-workflow_compile  - Compile spec to def
-workflow_explain  - Explain in natural language
-```
-
-### Sync Tools (3)
-
-```
-sync_run       - Execute sync
-sync_status    - Check sync status
-sync_configure - Update sync config
-```
-
-### Action Tools (3)
-
-```
-action_list      - List available actions
-action_get       - Get action details
-action_recommend - Recommend actions for intent
-```
-
-### Template Tools (2)
-
-```
-template_list - List templates by type
-template_get  - Get template details
-```
-
-### Utility Tools (2)
-
-```
-env_list        - List environments
-reference_get   - Get platform reference info
-```
-
-**Total: 37 tools**
-
----
-
-## Implementation Strategy
-
-### Phase 1: Define New Tools
-
-Create `src/mcp/tools-v2.ts` with new tool definitions:
-
-```typescript
-export const PERSONA_TOOLS = [
-  {
-    name: "persona_list",
-    description: "List all AI Employee personas",
-    inputSchema: {
-      type: "object",
-      properties: {
-        env: { type: "string", description: "Environment" },
-        type: { 
-          type: "string", 
-          enum: ["voice", "chat", "dashboard"],
-          description: "Filter by persona type"
-        }
-      }
-    }
-  },
-  // ... more tools
-];
-```
-
-### Phase 2: Update Routing
-
-In `src/mcp/server.ts`, parse tool names:
-
-```typescript
-function parseToolName(name: string): { resource: string; action: string } {
-  const parts = name.split('_');
-  if (parts.length === 2) {
-    return { resource: parts[0], action: parts[1] };
-  }
-  // Handle persona_version_* pattern
-  if (parts.length === 3 && parts[0] === 'persona' && parts[1] === 'version') {
-    return { resource: 'persona_version', action: parts[2] };
-  }
-  throw new Error(`Invalid tool name: ${name}`);
-}
-
-// Route to handlers
-const { resource, action } = parseToolName(toolName);
-switch (resource) {
-  case 'persona':
-    return handlePersona({ ...args, _action: action }, client, env);
-  case 'persona_version':
-    return handlePersonaVersion({ ...args, _action: action }, client, env);
-  // ...
-}
-```
-
-### Phase 3: Backwards Compatibility
-
-Keep old tools working during migration:
-
-```typescript
-// Old tool calls route to new
-if (toolName === 'persona') {
-  const action = args.mode || 'list';
-  console.warn(`DEPRECATED: Use persona_${action} instead of persona(mode="${action}")`);
-  return handlePersona({ ...args, _action: action }, client, env);
-}
-```
-
-### Phase 4: Prompts for Workflows
-
-Add user-triggered workflow prompts:
-
-```typescript
-export const WORKFLOW_PROMPTS = [
-  {
-    name: "create_voice_ai",
-    description: "Guided workflow to create a Voice AI persona",
-    arguments: [
-      { name: "name", description: "Persona name", required: true },
-      { name: "description", description: "What should it do?", required: true }
-    ]
-  },
-  {
-    name: "troubleshoot",
-    description: "Diagnose and fix persona issues",
-    arguments: [
-      { name: "persona", description: "Persona ID or name", required: true }
-    ]
-  }
-];
-```
-
----
-
-## Key Design Decisions
-
-### 1. Tool Name Format: `{resource}_{action}`
-
-**Chosen**: `persona_list`, `data_upload`, `sync_run`
-
-**Why**:
-- Alphabetically groups related tools
-- Matches Playwright, Supabase patterns
-- Clear, predictable naming
-
-**Rejected**:
-- `listPersonas` (camelCase) - doesn't group
-- `persona.list` (dot notation) - uncommon
-- `persona(mode="list")` (current) - param sprawl
-
-### 2. Nested Resources: `persona_version_*`
-
-**Chosen**: `persona_version_list`, `persona_version_create`
-
-**Why**:
-- Clear hierarchy
-- Still groups alphabetically
-- Avoids ambiguity
-
-**Alternative considered**: `version_list` with `persona_id` param - less discoverable
-
-### 3. Document vs Data
-
-**Chosen**: Separate `document_*` tools for doc generation
-
-**Why**:
-- Different API (Ema Document Generation API)
-- Different workflow (async, polling)
-- Clearer separation of concerns
-
-### 4. Workflow as Separate Resource
-
-**Chosen**: `workflow_analyze`, `workflow_optimize`
-
-**Why**:
-- Workflow operations are complex enough to warrant separation
-- Prevents persona_* bloat
-- Clear when you're operating on workflow vs persona config
-
----
-
-## Migration Path
-
-### For Users
-
-```typescript
-// Old (deprecated)
-persona(mode="list", type="voice")
-
-// New
-persona_list(type="voice")
-```
-
-### Deprecation Timeline
-
-1. **Phase 1**: New tools available, old tools warn
-2. **Phase 2**: Old tools in docs marked deprecated
-3. **Phase 3**: Old tools removed (major version bump)
-
----
-
-## Files to Modify
-
-| File | Change |
-|------|--------|
-| `src/mcp/tools-v2.ts` | **NEW** - All new tool definitions |
-| `src/mcp/server.ts` | Update routing to handle new tool names |
-| `src/mcp/handlers-consolidated.ts` | Minor updates (accept `_action` param) |
-| `src/mcp/prompts.ts` | Add workflow prompts |
-| `docs/mcp-tools-guide.md` | Full rewrite for new tools |
-| `.cursor/rules/ema/mcp/RULE.md` | Update tool reference |
-
----
-
-## Testing Strategy
-
-1. **Unit tests**: Each new tool definition validates
-2. **Integration tests**: Tool routing works correctly
-3. **Backwards compat tests**: Old tool calls still work
-4. **Deprecation tests**: Warnings are emitted
-
----
-
-## Open Questions
-
-1. **Should `env` be a tool or implicit?**
-   - Every tool needs env - maybe it should be in transport headers?
-
-2. **Prompt implementation detail**
-   - Should prompts call tools internally or return guidance for LLM to call?
-
-3. **Tool count concern**
-   - 37 tools - is this too many for some MCP clients?
-
----
-
-## DONE: Workflow Fix - Cascading Dependencies (Implemented)
-
-**Problem**: Removing "orphan" nodes created dangling references in consumer nodes.
-
-**Solution implemented**:
-1. `detectOrphanNodes()` in `knowledge.ts` now calls `findNodeConsumers()` to track dependent nodes
-2. `applySimpleFixes()` in `handlers-consolidated.ts` now:
-   - Removes orphan nodes
-   - Cleans up `incoming_edges` that reference removed nodes
-   - Cleans up `inputBindings` that reference removed nodes
-3. Issue reason now includes dependent nodes: `"Removing will also clean dangling refs in: node1, node2"`
-
-**Files changed**:
-- `src/sdk/knowledge.ts`: Added `findNodeConsumers()`, updated `detectOrphanNodes()`
-- `src/mcp/handlers-consolidated.ts`: Implemented `applySimpleFixes()` with cascading cleanup
-
----
-
-## TODO: Extract knowledge.ts to Config Files
-
-**Problem**: `knowledge.ts` has 4,370 lines of hardcoded data that should be config-driven.
-
-**Current state (hardcoded)**:
-| Data | Lines | Should Be |
-|------|-------|-----------|
-| `AGENT_CATALOG` | ~2000 | `resources/config/agents.json` |
-| `WORKFLOW_PATTERNS` | ~300 | `resources/config/patterns.json` |
-| `PLATFORM_CONCEPTS` | ~400 | `resources/config/concepts.json` |
-| `WIDGET_CATALOG` | ~100 | Proto/OpenAPI derived |
-| `QUALIFYING_QUESTIONS` | ~100 | `resources/config/questions.json` |
-
-**Migration plan**:
-1. Create JSON config files in `resources/config/`
-2. Add loader functions that read and cache configs at startup
-3. Update knowledge.ts exports to use loaded configs
-4. Add JSON schema validation for configs
-5. Update `npm run generate:types` to also validate configs against schemas
-
-**Note**: Some data is domain knowledge (glossary, best practices) that's NOT derivable from specs.
-This is intentional - it should be config, not code, but can't be auto-generated.
-
----
-
-## DONE: Generate Enum Mappings from OpenAPI
-
-- Modified `scripts/generate-types.ts` to extract enum labels from `x-enum-varnames`
-- Generated `PersonaTriggerTypeEnumLabels` and `PersonaTriggerTypeEnumValues`
-- Updated `handlers-consolidated.ts` to use generated mapping (no hardcoding)
-
----
-
-## DONE: Deprecation Tracking for Old Param Names
-
-Added deprecation warnings for legacy param names:
-- `identifier` → `id`
-- `clone_from` → `from`
-- `template_id` → `from`
-- `clone_data` → `include_data`
-
-Warnings logged to console. Target removal: v2.0.0
-
----
-
-## TODO: Test Coverage Gaps
-
-**Problem**: 631 tests exist but recent bugs (e.g., `proto_config` ignored without explicit `mode`) weren't caught.
-
-**Root cause**: Tests validate happy paths and individual functions, but miss:
-- **Integration gaps**: Handler → SDK → API round-trips
-- **Param interaction**: What happens when params combine unexpectedly
-- **Mode inference edge cases**: The auto-detection logic was never tested for ambiguity
-- **Real API contracts**: Tests mock responses, don't validate actual API shape
-
-**Needed**:
-1. **Negative tests**: "What should NOT work" - explicit failure cases
-2. **Ambiguity tests**: Multiple valid interpretations → should clarify, not guess
-3. **Contract tests**: Validate mocks match real API responses (use HAR captures)
-4. **E2E smoke tests**: Against real sandbox env, not mocks
-5. **Regression tests**: Every bug fix needs a test that would have caught it
-
-**Priority**: Address during or after restructure - new tools need proper test discipline from the start.
-
----
-
-## Quick Start
-
-```bash
-# 1. You're on this branch
-git checkout feat/mcp-tool-restructure
-
-# 2. Start with tools-v2.ts
-# Create new tool definitions following the inventory above
-
-# 3. Update server.ts routing
-# Parse {resource}_{action} and route
-
-# 4. Run tests
-npm run build && npm test
-
-# 5. Update docs last
-```
-
----
-
-## Reference
-
-- **Full proposal**: `docs/proposals/mcp-tool-restructure-2026-01.md`
-- **MCP spec**: https://modelcontextprotocol.io/specification/2025-11-25/server/tools
-- **Playwright MCP**: https://github.com/microsoft/playwright-mcp
-- **Current tools**: `src/mcp/tools-consolidated.ts`