ema-mcp-toolkit 0.2.0

package/docs/tool-consolidation-proposal.md

@@ -0,0 +1,427 @@
+# MCP Tool Consolidation Proposal
+
+> Consolidate 45 tools into 9 core commands following Unix CLI patterns.
+
+## Status: Implementation Complete
+
+Files created:
+- `src/mcp/tools-consolidated.ts` - New tool definitions
+- `src/mcp/handlers-consolidated.ts` - New handlers with mode-based dispatch
+
+## Design Principles
+
+1. **Consistent terminology**: Use "persona" everywhere (not "ai_employee")
+2. **Get/List as flags**: `persona("id")` vs `persona(all=True)`
+3. **Related functions as modes**: `workflow(mode="analyze")` includes detect, validate, suggest
+4. **Flags over separate tools**: `template(type="voice")` not `get_voice_template()`
+
+---
+
+## Proposed Tool Structure
+
+### 1. `persona` - AI Employee Management
+
+**Replaces:** `find_personas`, `get_persona`, `create_ai_employee`, `update_ai_employee`, `compare_ai_employees`
+
+```typescript
+interface PersonaArgs {
+  // Identifier (optional - if omitted with all=true, lists all)
+  id?: string;  // ID or exact name
+  
+  // Mode
+  mode?: "get" | "create" | "update" | "delete" | "compare" | "sync";
+  
+  // List/Search flags
+  all?: boolean;
+  query?: string;
+  status?: "active" | "inactive" | "draft";
+  trigger_type?: "voice" | "chat" | "dashboard";
+  limit?: number;
+  
+  // Get flags
+  include_workflow?: boolean;
+  include_fingerprint?: boolean;
+  
+  // Create/Update flags
+  name?: string;
+  description?: string;
+  template_id?: string;
+  source_persona_id?: string;
+  
+  // Compare/Sync flags
+  compare_to?: string;
+  target_env?: string;
+  dry_run?: boolean;
+  
+  // Environment
+  env?: string;
+}
+```
+
+**Usage Examples:**
+```bash
+# Get single
+persona("IT Support Bot")
+persona("abc-123", include_workflow=True)
+
+# List/Search
+persona(all=True)
+persona(query="Support", status="active")
+
+# Create
+persona(mode="create", name="New Bot", template_id="voice")
+
+# Update
+persona("abc-123", mode="update", name="Renamed")
+
+# Compare
+persona("abc-123", mode="compare", compare_to="def-456")
+
+# Sync
+persona("abc-123", mode="sync", target_env="dev", dry_run=True)
+```
+
+---
+
+### 2. `workflow` - Workflow Generation & Analysis
+
+**Replaces:** `workflow`, `deploy_workflow`, `analyze_workflow`, `detect_workflow_issues`, `validate_workflow_connections`, `suggest_workflow_fixes`, `get_workflow_metrics`, `compare_workflow_versions`, `compile_workflow`
+
+```typescript
+interface WorkflowArgs {
+  // Input (one of these)
+  input?: string | object;  // Natural language or spec
+  persona_id?: string;      // For existing workflow operations
+  workflow_def?: object;    // For direct analysis
+  
+  // Mode
+  mode?: "generate" | "analyze" | "deploy" | "compare" | "compile";
+  
+  // Generate flags
+  persona_type?: "voice" | "chat" | "dashboard";
+  use_autobuilder?: boolean;
+  
+  // Analyze flags (what to include in analysis)
+  include?: ("issues" | "connections" | "fixes" | "metrics")[];
+  // Default: all of them
+  
+  // Deploy flags
+  auto_fix?: boolean;
+  validate_first?: boolean;
+  
+  // Compare flags
+  compare_to?: string;  // Another persona_id
+  
+  // Environment
+  env?: string;
+}
+```
+
+**Usage Examples:**
+```bash
+# Generate
+workflow("IT helpdesk with KB search and tickets")
+workflow(input={intents: [...]}, persona_type="voice")
+
+# Analyze (unified - returns issues, connections, fixes, metrics)
+workflow(persona_id="abc-123", mode="analyze")
+
+# Analyze with specific includes
+workflow(persona_id="abc-123", mode="analyze", include=["issues", "metrics"])
+
+# Analyze from JSON (not deployed)
+workflow(workflow_def={...}, mode="analyze")
+
+# Deploy
+workflow(persona_id="abc-123", mode="deploy", workflow_def={...}, auto_fix=True)
+
+# Compare versions
+workflow(persona_id="abc-123", mode="compare", compare_to="def-456")
+
+# Compile spec to workflow_def
+workflow(input={nodes: [...]}, mode="compile")
+```
+
+---
+
+### 3. `action` - Workflow Actions/Agents
+
+**Replaces:** `find_workflow_actions`, `get_workflow_action`, `list_auto_builder_agents`, `get_auto_builder_agent`, `suggest_agents_for_use_case`
+
+```typescript
+interface ActionArgs {
+  // Identifier (optional)
+  id?: string;  // Action ID or name
+  
+  // List/Search flags
+  all?: boolean;
+  category?: string;
+  query?: string;
+  persona_id?: string;  // Actions in this workflow
+  enabled?: boolean;
+  
+  // Suggestion mode
+  suggest_for?: string;  // Use case description
+  
+  // Include flags
+  include_docs?: boolean;  // Include full documentation
+  
+  // Environment
+  env?: string;
+}
+```
+
+**Usage Examples:**
+```bash
+# Get single
+action("chat_categorizer")
+action("abc-123-uuid")
+action("chat_categorizer", include_docs=True)
+
+# List/Search
+action(all=True)
+action(category="routing")
+action(persona_id="abc-123")  # Actions in workflow
+action(query="search")
+
+# Suggest for use case
+action(suggest_for="IT helpdesk with ServiceNow")
+```
+
+---
+
+### 4. `template` - Templates & Patterns
+
+**Replaces:** `get_workflow_pattern`, `list_workflow_patterns`, `get_widget_reference`, `get_voice_persona_template`, `list_ai_employee_templates`
+
+```typescript
+interface TemplateArgs {
+  // What to get
+  type?: "voice" | "chat" | "dashboard";  // Persona config template
+  pattern?: string;  // Workflow pattern name
+  widgets?: "voice" | "chat" | "dashboard";  // Widget reference
+  
+  // List flags
+  all?: boolean;
+  patterns?: boolean;  // List workflow patterns
+  
+  // Environment
+  env?: string;
+}
+```
+
+**Usage Examples:**
+```bash
+# Persona config templates
+template(type="voice")
+template(type="chat")
+
+# Workflow patterns
+template(pattern="intent-routing")
+template(patterns=True)  # List all patterns
+template(patterns=True, type="voice")  # Filter by persona type
+
+# Widget reference
+template(widgets="voice")
+
+# List all persona templates
+template(all=True)
+```
+
+---
+
+### 5. `knowledge` - Data Sources & KB
+
+**Replaces:** `upload_data_source`, `delete_data_source`, `list_data_sources`, `get_embedding_status`, `toggle_embedding`
+
+```typescript
+interface KnowledgeArgs {
+  persona_id: string;
+  
+  // Mode
+  mode?: "list" | "upload" | "delete" | "status" | "toggle";
+  
+  // Upload flags
+  file_path?: string;
+  tags?: string;
+  
+  // Delete flags
+  file_id?: string;
+  
+  // Toggle flags
+  embedding_enabled?: boolean;
+  
+  // Environment
+  env?: string;
+}
+```
+
+**Usage Examples:**
+```bash
+# List files
+knowledge(persona_id="abc-123", mode="list")
+
+# Upload
+knowledge(persona_id="abc-123", mode="upload", file_path="/path/to/file.pdf")
+
+# Delete
+knowledge(persona_id="abc-123", mode="delete", file_id="file-123")
+
+# Embedding status
+knowledge(persona_id="abc-123", mode="status")
+
+# Toggle embedding
+knowledge(persona_id="abc-123", mode="toggle", embedding_enabled=True)
+```
+
+---
+
+### 6. `reference` - Platform Knowledge
+
+**Replaces:** `get_platform_concept`, `list_platform_concepts`, `get_common_mistakes`, `get_debug_checklist`, `get_workflow_execution_model`, `get_auto_builder_guidance`, `get_qualifying_questions`, `validate_workflow_prompt`, `check_type_compatibility`
+
+```typescript
+interface ReferenceArgs {
+  // What to get (one of these)
+  concept?: string;       // Platform concept (e.g., "HITL", "Workflow")
+  guidance?: string;      // Topic guidance (e.g., "categorizer-routing")
+  questions?: boolean;    // Qualifying questions
+  mistakes?: boolean;     // Common mistakes
+  checklist?: boolean;    // Debug checklist
+  execution?: boolean;    // Workflow execution model
+  
+  // List flags
+  concepts?: boolean;     // List all concepts
+  
+  // Questions filter
+  category?: string;      // Filter questions by category
+  required_only?: boolean;
+  
+  // Type compatibility check
+  check_types?: { source: string; target: string };
+  
+  // Prompt validation
+  validate_prompt?: string;
+}
+```
+
+**Usage Examples:**
+```bash
+# Concepts
+reference(concept="HITL")
+reference(concepts=True)
+
+# Guidance
+reference(guidance="categorizer-routing")
+reference(guidance="type-compatibility")
+
+# Questions
+reference(questions=True)
+reference(questions=True, category="Voice", required_only=True)
+
+# Debugging
+reference(mistakes=True)
+reference(checklist=True)
+reference(execution=True)
+
+# Validation
+reference(check_types={source: "CHAT_CONVERSATION", target: "TEXT_WITH_SOURCES"})
+reference(validate_prompt="Create a bot that...")
+```
+
+---
+
+### 7. `sync` - Environment Sync
+
+**Replaces:** `sync`, `sync_info`
+
+```typescript
+interface SyncArgs {
+  // What to sync
+  id?: string;  // Persona ID or name
+  
+  // Mode
+  mode?: "run" | "status" | "config";
+  
+  // Sync flags
+  target_env: string;
+  source_env?: string;
+  scope?: "one" | "all";
+  dry_run?: boolean;
+  include_status?: boolean;
+  
+  // Status flags
+  list_synced?: boolean;
+  master_env?: string;
+}
+```
+
+**Usage Examples:**
+```bash
+# Sync single
+sync(id="IT Support", target_env="dev")
+sync(id="abc-123", target_env="staging", dry_run=True)
+
+# Sync all
+sync(target_env="dev", scope="all")
+
+# Status
+sync(mode="status", id="abc-123")
+sync(mode="status", list_synced=True, env="dev")
+
+# Config
+sync(mode="config")
+```
+
+---
+
+### 8. `env` - Environment Management
+
+**Replaces:** `list_environments`
+
+```typescript
+interface EnvArgs {
+  all?: boolean;  // List all environments
+}
+```
+
+**Usage Examples:**
+```bash
+env(all=True)  # List available environments
+```
+
+---
+
+## Summary: Before & After
+
+| Before (45+ tools) | After (8 tools) |
+|-------------------|-----------------|
+| `find_personas`, `get_persona`, `create_ai_employee`, `update_ai_employee`, `compare_ai_employees` | `persona` |
+| `workflow`, `deploy_workflow`, `analyze_workflow`, `detect_workflow_issues`, `validate_workflow_connections`, `suggest_workflow_fixes`, `get_workflow_metrics`, `compare_workflow_versions`, `compile_workflow` | `workflow` |
+| `find_workflow_actions`, `get_workflow_action`, `list_auto_builder_agents`, `get_auto_builder_agent`, `suggest_agents_for_use_case` | `action` |
+| `get_workflow_pattern`, `list_workflow_patterns`, `get_widget_reference`, `get_voice_persona_template`, `list_ai_employee_templates` | `template` |
+| `upload_data_source`, `delete_data_source`, `list_data_sources`, `get_embedding_status`, `toggle_embedding` | `knowledge` |
+| `get_platform_concept`, `list_platform_concepts`, `get_common_mistakes`, `get_debug_checklist`, `get_workflow_execution_model`, `get_auto_builder_guidance`, `get_qualifying_questions`, `validate_workflow_prompt`, `check_type_compatibility` | `reference` |
+| `sync`, `sync_info` | `sync` |
+| `list_environments` | `env` |
+
+**Result: 45+ tools → 8 commands**
+
+---
+
+## Migration Strategy
+
+1. **Phase 1**: Create new consolidated tools alongside existing
+2. **Phase 2**: Mark old tools as deprecated with pointer to new
+3. **Phase 3**: Remove deprecated tools after transition period
+
+## Backward Compatibility
+
+Old tool names can be aliases:
+```typescript
+// In server.ts
+get_persona: (args) => persona({ ...args, mode: "get" }),
+find_personas: (args) => persona({ ...args, all: true }),
+analyze_workflow: (args) => workflow({ ...args, mode: "analyze" }),
+// etc.
+```

package/package.json

@@ -0,0 +1,95 @@
+{
+  "name": "ema-mcp-toolkit",
+  "version": "0.2.0",
+  "description": "Ema AI Employee toolkit - MCP server, CLI, and SDK for managing AI Employees across environments",
+  "type": "module",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "bin": {
+    "ema": "dist/cli/index.js",
+    "ema-mcp": "dist/mcp/server.js",
+    "ema-mcp-toolkit": "dist/mcp/server.js"
+  },
+  "files": [
+    "dist",
+    "resources",
+    "docs",
+    "config.example.yaml",
+    "README.md"
+  ],
+  "exports": {
+    ".": {
+      "import": "./dist/index.js",
+      "types": "./dist/index.d.ts"
+    },
+    "./sdk": {
+      "import": "./dist/sdk/index.js",
+      "types": "./dist/sdk/index.d.ts"
+    },
+    "./mcp": {
+      "import": "./dist/mcp/index.js",
+      "types": "./dist/mcp/index.d.ts"
+    }
+  },
+  "scripts": {
+    "build": "tsc",
+    "dev": "tsx src/index.ts",
+    "start": "node dist/index.js",
+    "mcp": "tsx src/mcp/server.ts",
+    "stop": "lsof -ti:8080 | xargs kill -9 2>/dev/null; pkill -f 'tsx.*src/(index|mcp)' || pkill -f 'node.*dist/(index|mcp)' || true",
+    "restart": "npm run stop && npm run dev",
+    "restart:mcp": "npm run stop && npm run mcp",
+    "cli": "tsx src/cli/index.ts",
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc --noEmit",
+    "precommit": "npm run typecheck && npm run test",
+    "prepare": "husky",
+    "prepublishOnly": "npm run build && npm run test",
+    "generate:types": "tsx scripts/generate-types.ts",
+    "generate:types:oats": "openapi-typescript https://staging.ema.co/api/openapi.json -o src/sdk/generated/api.d.ts",
+    "check:contracts": "tsx scripts/check-api-contracts.ts"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/ema-co/ema-mcp-toolkit.git"
+  },
+  "keywords": [
+    "ema",
+    "mcp",
+    "model-context-protocol",
+    "ai-assistant",
+    "cursor",
+    "claude",
+    "workflow",
+    "ai-employee"
+  ],
+  "author": "Ema Inc.",
+  "license": "UNLICENSED",
+  "bugs": {
+    "url": "https://github.com/ema-co/ema-mcp-toolkit/issues"
+  },
+  "homepage": "https://github.com/ema-co/ema-mcp-toolkit#readme",
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "dependencies": {
+    "@modelcontextprotocol/sdk": "^1.25.2",
+    "better-sqlite3": "^11.7.0",
+    "fastify": "^5.2.0",
+    "js-yaml": "^4.1.0",
+    "node-cron": "^3.0.3",
+    "zod": "^3.24.1"
+  },
+  "devDependencies": {
+    "@types/better-sqlite3": "^7.6.11",
+    "@types/js-yaml": "^4.0.9",
+    "@types/node": "^22.10.2",
+    "@types/node-cron": "^3.0.11",
+    "husky": "^9.1.7",
+    "openapi-typescript": "^7.0.0",
+    "tsx": "^4.19.2",
+    "typescript": "^5.7.2",
+    "vitest": "^4.0.16"
+  }
+}

package/resources/templates/chat-ai/README.md

@@ -0,0 +1,119 @@
+# {Persona Name}
+
+**Type**: Chat AI
+**Created**: {Date}
+**Version**: 1.0
+
+## Overview
+
+{Brief description of what this Chat AI Employee does}
+
+## Deployment Steps
+
+### Step 1: Create Workflow in Auto Builder
+
+1. Go to [Ema Auto Builder](https://builder.ema.co/)
+2. Create new AI Employee → Select **Chat AI**
+3. Copy contents of `workflow-prompt.md` into the prompt field
+4. Click "Generate Workflow"
+5. Review generated nodes and verify:
+   - All categorizers have edges for every category
+   - Search nodes connect to respond nodes correctly
+   - Fallback paths are present
+
+### Step 2: Configure Chat Settings
+
+1. Navigate to "Chat Settings" section
+2. Apply settings from `persona-config.json`:
+
+| Field | Source |
+|-------|--------|
+| Name | `chatbotSdkConfig.name` |
+| Theme Color | `chatbotSdkConfig.theme.primaryColor` |
+| Logo | `chatbotSdkConfig.logo` |
+| Allowed Domains | `chatbotSdkConfig.allowedDomains` |
+
+### Step 3: Configure Knowledge Base
+
+1. Navigate to "Knowledge Base" section
+2. Upload documents from `docs/` folder
+3. Configure tags from `persona-config.json`:
+   - Tag Types: `fileTagging.tagTypes`
+4. Enable chunking: `fileUpload.useChunking`
+
+### Step 4: Configure Feedback
+
+1. Navigate to "Feedback" section
+2. Apply settings:
+   - Question: `feedbackMessage.message.question`
+   - Frequency: `feedbackMessage.feedbackFrequency`
+
+### Step 5: Deploy Widget
+
+1. Get embed code from "Deploy" section
+2. Add to your website:
+
+```html
+<script src="https://widget.ema.co/chat/{persona-id}.js"></script>
+```
+
+### Step 6: Test
+
+1. **Preview Mode**: Use the built-in chat preview
+2. **Test Scenarios**:
+   - Test knowledge base queries
+   - Test each category branch
+   - Test fallback handling
+   - Test source citations
+3. **Edge Cases**:
+   - Test with off-topic queries
+   - Test with multi-turn conversations
+   - Test with complex questions
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `workflow-prompt.md` | Auto Builder prompt with safeguards |
+| `persona-config.json` | Chat AI configuration |
+| `proto-config.json` | Full API-deployable config |
+| `docs/` | Knowledge base documents |
+
+## Testing Checklist
+
+- [ ] Workflow generates without validation errors
+- [ ] Knowledge search returns relevant results
+- [ ] Source citations appear correctly
+- [ ] All category paths work
+- [ ] Fallback provides helpful response
+- [ ] Widget loads on allowed domains
+- [ ] Feedback collection works
+
+## Maintenance
+
+| Change Type | Action |
+|-------------|--------|
+| Workflow logic | Update `workflow-prompt.md`, re-generate |
+| Chat settings | Update `persona-config.json`, apply in UI |
+| Knowledge content | Update docs in `docs/`, re-upload |
+| Allowed domains | Update `chatbotSdkConfig.allowedDomains` |
+
+## Troubleshooting
+
+### Widget doesn't load
+
+- Check `allowedDomains` includes your domain
+- Verify embed code is correct
+- Check browser console for errors
+
+### Poor search results
+
+- Review document chunking settings
+- Add more specific tags
+- Improve document structure
+
+### Missing source citations
+
+- Ensure `disableSources: false`
+- Verify documents have clear headings
+- Check that search is returning results