@chandshantanu/agentbuilder-mcp-server 0.2.0

package/AGENT_CREATION_TEST_PLAN.md

@@ -0,0 +1,514 @@
+# Agent Creation MCP Tools - Integration Test Plan
+
+## Overview
+This document outlines the integration tests for the new agent creation tools added to the AgentBuilder MCP server.
+
+## Test Environment Setup
+
+### Prerequisites
+1. Backend API running at `https://agentsapi.chatslytics.com`
+2. Valid MCP connection ID (format: `mcp_conn_xxx`)
+3. Seller account credentials: `shantanu.chandra@banxwayglobal.com`
+4. Claude Code with MCP server configured
+
+### Setup Steps
+```bash
+# 1. Build MCP server
+cd agentbuilder-mcp-server
+npm run build
+
+# 2. Verify connection
+# In Claude Code, run:
+# set_connection with connection_id: mcp_conn_xxx
+```
+
+## Test Cases
+
+### Test 1: Create Agent (create_agent)
+
+**Objective**: Create a new agent with basic information
+
+**Input**:
+```json
+{
+  "name": "Test Instagram DM Agent",
+  "description": "Test agent for Instagram DM sales automation",
+  "type": "conversational",
+  "category": "Sales",
+  "tags": ["instagram", "sales", "automation"],
+  "is_public": false
+}
+```
+
+**Expected Output**:
+- ✅ Status: Success
+- ✅ Agent created with draft status
+- ✅ Returns agent_id and slug
+- ✅ Next steps displayed
+
+**Validation**:
+- Agent appears in database with status "draft"
+- Creator ID matches authenticated user
+- Slug is URL-friendly
+
+---
+
+### Test 2: Set Wizard Configuration (set_agent_wizard_config)
+
+**Objective**: Configure onboarding wizard for the agent
+
+**Prerequisites**: Completed Test 1 (have agent_id)
+
+**Input**:
+```json
+{
+  "agent_id": "<from_test_1>",
+  "wizard_configuration": {
+    "required_oauth_providers": ["facebook", "instagram"],
+    "required_credentials": [
+      {
+        "key": "openrouter_api_key",
+        "label": "OpenRouter API Key",
+        "type": "api_key",
+        "required": true,
+        "placeholder": "sk-or-v1-...",
+        "help_text": "For AI model inference",
+        "validation_regex": "^sk-or-v1-"
+      }
+    ],
+    "required_platform_config": [
+      {
+        "key": "platform_type",
+        "label": "E-Commerce Platform",
+        "type": "select",
+        "options": ["shopify", "woocommerce"],
+        "required": true
+      },
+      {
+        "key": "shopify_store_url",
+        "label": "Shopify Store URL",
+        "type": "url",
+        "required": true,
+        "depends_on": "platform_type=shopify",
+        "placeholder": "https://store.myshopify.com"
+      }
+    ],
+    "custom_wizard_steps": [
+      {
+        "step_id": "instagram_account",
+        "title": "Select Instagram Account",
+        "description": "Choose which Instagram account to use",
+        "component_name": "InstagramPageSelector",
+        "after_step": "oauth",
+        "required": true
+      }
+    ],
+    "configuration_defaults": {
+      "sync_frequency": "realtime",
+      "product_fields": ["name", "price", "description"]
+    }
+  }
+}
+```
+
+**Expected Output**:
+- ✅ Status: Success
+- ✅ Wizard configuration saved
+- ✅ Validation passed
+- ✅ OAuth providers validated
+- ✅ Credential fields validated
+
+**Validation**:
+- Run validation checks:
+  - All OAuth providers are valid
+  - Credential fields have required properties
+  - Platform config types are valid
+  - Conditional logic (depends_on) references existing fields
+
+**Error Cases**:
+- Invalid OAuth provider → Error with list of valid providers
+- Missing required field in credential → Error specifying which field
+- Select field without options → Error
+- Invalid depends_on reference → Warning
+
+---
+
+### Test 3: Set Agent Graph (set_agent_graph)
+
+**Objective**: Define LangGraph workflow for agent execution
+
+**Prerequisites**: Completed Test 1 and Test 2
+
+**Input**:
+```json
+{
+  "agent_id": "<from_test_1>",
+  "graph": {
+    "nodes": [
+      {
+        "id": "webhook_receiver",
+        "type": "webhook",
+        "config": {
+          "path": "/webhook/instagram"
+        }
+      },
+      {
+        "id": "classify_intent",
+        "type": "llm_classifier",
+        "config": {
+          "model": "openai/gpt-4o",
+          "categories": ["product_inquiry", "support", "general"]
+        }
+      },
+      {
+        "id": "search_products",
+        "type": "vector_search",
+        "config": {
+          "index_name": "products",
+          "top_k": 5
+        }
+      },
+      {
+        "id": "generate_response",
+        "type": "llm_chat",
+        "config": {
+          "model": "openai/gpt-4o"
+        }
+      },
+      {
+        "id": "send_reply",
+        "type": "api_call",
+        "config": {
+          "method": "POST",
+          "url": "https://graph.facebook.com/v18.0/{page_id}/messages"
+        }
+      }
+    ],
+    "edges": [
+      {
+        "from": "webhook_receiver",
+        "to": "classify_intent"
+      },
+      {
+        "from": "classify_intent",
+        "to": "search_products",
+        "condition": "intent == 'product_inquiry'"
+      },
+      {
+        "from": "search_products",
+        "to": "generate_response"
+      },
+      {
+        "from": "generate_response",
+        "to": "send_reply"
+      }
+    ],
+    "entry_point": "webhook_receiver"
+  }
+}
+```
+
+**Expected Output**:
+- ✅ Status: Success
+- ✅ Graph saved
+- ✅ Node types validated
+- ✅ Edge references validated
+- ✅ Entry point validated
+
+**Validation**:
+- All node types exist in ToolRegistry
+- All edges reference existing nodes
+- Entry point exists in nodes
+- No circular dependencies (warning)
+
+**Error Cases**:
+- Invalid node type → Error with list of valid types
+- Edge referencing non-existent node → Error
+- Missing entry_point → Error
+- Entry point not in nodes → Error
+
+---
+
+### Test 4: Validate Agent Configuration (validate_agent_configuration)
+
+**Objective**: Pre-publish validation check
+
+**Prerequisites**: Completed Tests 1, 2, and 3
+
+**Input**:
+```json
+{
+  "agent_id": "<from_test_1>"
+}
+```
+
+**Expected Output**:
+```json
+{
+  "valid": true,
+  "errors": [],
+  "warnings": [
+    "Agent has no icon_url. Consider adding one for better visibility.",
+    "Agent has no banner_url. Consider adding one for marketplace display.",
+    "Testing status unknown. Ensure agent has been tested before publishing."
+  ],
+  "checklist": {
+    "wizard_configuration": true,
+    "agent_graph": true,
+    "pricing": true,
+    "metadata": true,
+    "testing": false
+  },
+  "agent_id": "<agent_id>",
+  "agent_name": "Test Instagram DM Agent"
+}
+```
+
+**Validation Checks**:
+- ✅ Wizard configuration complete
+- ✅ Agent graph defined and valid
+- ✅ Pricing configured
+- ✅ Basic metadata present (name, description, category, type)
+- ⚠️ Optional: icon_url, banner_url (warnings only)
+- ⚠️ Testing status (informational)
+
+**Error Cases (if validation fails)**:
+- Missing wizard config → `valid: false`, error in list
+- Missing graph → `valid: false`, error in list
+- Invalid pricing → `valid: false`, error in list
+- Missing metadata → `valid: false`, error in list
+
+---
+
+### Test 5: Publish Agent (publish_agent)
+
+**Objective**: Publish agent to marketplace
+
+**Prerequisites**: Completed Tests 1-4 with valid=true
+
+**Input**:
+```json
+{
+  "agent_id": "<from_test_1>"
+}
+```
+
+**Expected Output (for Seller)**:
+```json
+{
+  "success": true,
+  "message": "Agent submitted for review",
+  "agent_id": "<agent_id>",
+  "agent_name": "Test Instagram DM Agent",
+  "status": "pending_review",
+  "marketplace_url": "/agents/test-instagram-dm-agent"
+}
+```
+
+**Expected Output (for Admin)**:
+```json
+{
+  "success": true,
+  "message": "Agent published successfully",
+  "agent_id": "<agent_id>",
+  "agent_name": "Test Instagram DM Agent",
+  "status": "active",
+  "marketplace_url": "/agents/test-instagram-dm-agent"
+}
+```
+
+**Validation**:
+- Seller: Status changes to "pending_review"
+- Admin: Status changes to "active"
+- Agent appears in marketplace (for active)
+- Admins notified (for pending_review)
+
+**Error Cases**:
+- Validation fails → Error with validation errors
+- User not owner → 403 Forbidden
+- Agent already published → Error
+
+---
+
+### Test 6: Update Agent (update_agent)
+
+**Objective**: Update agent metadata
+
+**Prerequisites**: Completed Test 1
+
+**Input**:
+```json
+{
+  "agent_id": "<from_test_1>",
+  "updates": {
+    "description": "Updated: Advanced Instagram DM sales automation with AI",
+    "tags": ["instagram", "sales", "automation", "ai"],
+    "icon_url": "https://example.com/icon.png",
+    "banner_url": "https://example.com/banner.png"
+  }
+}
+```
+
+**Expected Output**:
+- ✅ Status: Success
+- ✅ Agent updated
+- ✅ Returns updated agent data
+
+**Validation**:
+- Updates applied to database
+- updated_at timestamp changed
+
+---
+
+### Test 7: End-to-End Agent Creation Flow
+
+**Objective**: Complete workflow from creation to publishing
+
+**Steps**:
+1. ✅ Create agent (`create_agent`)
+2. ✅ Set wizard configuration (`set_agent_wizard_config`)
+3. ✅ Set agent graph (`set_agent_graph`)
+4. ✅ Add media (icon, banner) (`update_agent`)
+5. ✅ Validate agent (`validate_agent_configuration`)
+6. ✅ Publish agent (`publish_agent`)
+7. ✅ Verify in marketplace (manual check)
+
+**Success Criteria**:
+- All steps complete without errors
+- Agent visible in marketplace (if admin) or pending review (if seller)
+- Agent can be subscribed to by users
+- Configuration wizard renders correctly for subscribers
+
+---
+
+## Error Handling Tests
+
+### Test 8: Authorization Errors
+
+**Test Cases**:
+1. Basic user tries to create agent → 403 Forbidden
+2. Non-owner tries to update agent → 403 Forbidden
+3. Non-owner tries to publish agent → 403 Forbidden
+
+### Test 9: Validation Errors
+
+**Test Cases**:
+1. Invalid wizard config → 400 Bad Request with errors
+2. Invalid graph structure → 400 Bad Request with errors
+3. Publish without wizard config → 400 Bad Request with validation errors
+
+### Test 10: Not Found Errors
+
+**Test Cases**:
+1. Update non-existent agent → 404 Not Found
+2. Validate non-existent agent → 404 Not Found
+3. Publish non-existent agent → 404 Not Found
+
+---
+
+## Performance Tests
+
+### Test 11: Large Graph Performance
+
+**Objective**: Test with large LangGraph workflow
+
+**Input**: Graph with 50+ nodes, 100+ edges
+
+**Expected**: Should complete within 5 seconds
+
+### Test 12: Complex Wizard Configuration
+
+**Objective**: Test with extensive wizard config
+
+**Input**:
+- 5+ OAuth providers
+- 20+ credential fields
+- 30+ platform config fields
+- 10+ custom steps
+
+**Expected**: Should validate and save within 3 seconds
+
+---
+
+## Regression Tests
+
+### Test 13: Existing Functionality Unaffected
+
+**Verify**:
+- ✅ list_agents still works
+- ✅ get_agent still works
+- ✅ execute_agent still works
+- ✅ deploy_agent still works
+- ✅ Dashboard tools still work
+
+---
+
+## Test Execution Checklist
+
+### Pre-Testing
+- [ ] Backend API is running
+- [ ] Database is accessible
+- [ ] MCP server built and configured
+- [ ] Connection established
+- [ ] Seller account ready
+
+### During Testing
+- [ ] Record all test results
+- [ ] Capture error messages
+- [ ] Note response times
+- [ ] Screenshot UI if applicable
+
+### Post-Testing
+- [ ] Clean up test data
+- [ ] Document any issues found
+- [ ] Report bugs to development team
+- [ ] Update documentation as needed
+
+---
+
+## Test Data Cleanup
+
+After testing, clean up with:
+
+```javascript
+// Delete test agents
+db.agents.deleteMany({ name: /^Test/ })
+
+// Or specific test agent
+db.agents.deleteOne({ _id: ObjectId("<test_agent_id>") })
+```
+
+---
+
+## Success Metrics
+
+- ✅ All 13 test cases pass
+- ✅ No backend errors in logs
+- ✅ Response times < 5 seconds
+- ✅ Validation catches all error cases
+- ✅ User experience is smooth
+- ✅ Documentation is accurate
+
+---
+
+## Known Limitations
+
+1. **Testing Status**: Cannot automatically validate if agent has been tested
+2. **Media Upload**: Media URLs must be provided (no upload in MCP)
+3. **OAuth Flow**: OAuth providers must be pre-configured in backend
+
+---
+
+## Future Enhancements
+
+1. Add automated test runner (Jest/Mocha)
+2. Add mock backend for unit testing
+3. Add performance benchmarks
+4. Add load testing (100+ concurrent agent creations)
+5. Add E2E tests with actual agent deployment
+
+---
+
+**Last Updated**: 2026-02-06
+**Test Plan Version**: 1.0.0
+**Author**: Development Team