miadi 1.0.14 → 2.0.1

package/MCP_REMOTE_SERVER_SPEC.md

@@ -1,373 +0,0 @@
-# Miadi MCP Remote Server Specification
-## Natural Language-Driven Software Development Prompt
-
-**Context**: Transform the existing Miadi MCP server from stdio-based local operation to HTTP-based remote server with OAuth 2.1 authentication, enabling claude.ai connector integration while preserving all 26 existing tools and tool selection capabilities.
-
-**Objective**: Create a production-ready remote MCP server that exposes Miadi Three-Pathway Agent System API through secure HTTP endpoints, supporting OAuth authentication flow with claude.ai callback integration and existing EH_TOKEN user validation.
-
----
-
-## Core Requirements Specification
-
-### 1. Transport Layer Migration
-**From**: `StdioServerTransport` (JSON-RPC over stdin/stdout)  
-**To**: HTTP-based JSON-RPC server with Express.js framework
-
-**Implementation**: 
-- Express.js server listening on configurable port (default: 3000)
-- POST endpoint `/mcp/jsonrpc` handling MCP JSON-RPC protocol
-- Support for both single requests and batch requests
-- Proper CORS headers for cross-origin requests
-- Content-Type validation for `application/json`
-
-### 2. OAuth 2.1 Resource Server Implementation
-**Specification Compliance**: MCP OAuth 2.1 with Resource Server pattern (RFC8707)
-
-**Authentication Flow**:
-```
-1. Client discovers server via /.well-known/oauth-protected-resource
-2. Dynamic Client Registration (RFC7591) for automatic client setup
-3. Authorization Code + PKCE flow for secure token exchange
-4. Resource Indicators for token scoping to specific MCP server
-5. JWT token validation on every request
-6. EH_TOKEN integration for Miadi user database validation
-```
-
-**Endpoints Required**:
-- `/.well-known/oauth-protected-resource` - Resource Server Metadata
-- `/oauth/authorize` - Authorization endpoint (redirect to identity provider)
-- `/oauth/token` - Token exchange endpoint
-- `/oauth/userinfo` - User information endpoint for validation
-
-### 3. User Authentication Integration
-**Existing System**: Miadi API uses `EH_TOKEN` Bearer authentication  
-**Integration Pattern**: OAuth JWT contains user identifier → validate against EH_TOKEN system
-
-**User Validation Flow**:
-```
-1. OAuth token validated (signature, expiration, scope)
-2. Extract user identifier from JWT claims
-3. Call Miadi API with EH_TOKEN to validate user access
-4. Cache validation results (5-minute TTL)
-5. Reject requests for unauthorized users
-```
-
-### 4. Tool System Preservation
-**Requirement**: All existing functionality must be preserved
-- 26 tools across 6 categories (memory, session, capability, ai, workflow, forge)
-- Environment variable-based tool selection system
-- Complete Zod schemas and validation
-- Error handling and logging
-- Tool usage tracking and rate limiting
-
-### 5. Deployment Architecture
-**Local Development**: Ngrok tunnel for public endpoint exposure  
-**Production**: Cloud hosting with proper SSL/TLS termination
-
-**Ngrok Configuration**:
-```bash
-ngrok http 3000 --domain=<custom-domain>
-# Exposes local server at https://<custom-domain>.ngrok-free.app
-```
-
-**Environment Variables**:
-```bash
-# Existing Miadi API config
-EH_TOKEN=<miadi-api-token>
-EH_API_URL=<miadi-api-base-url>
-
-# OAuth configuration
-OAUTH_CLIENT_ID=<dynamic-or-configured>
-OAUTH_CLIENT_SECRET=<if-confidential-client>
-OAUTH_ISSUER=<identity-provider-url>
-JWT_SECRET=<token-validation-secret>
-
-# Server configuration
-MCP_PORT=3000
-MCP_HOST=0.0.0.0
-NGROK_AUTHTOKEN=<ngrok-token>
-
-# Tool selection (preserved)
-MIADI_TOOLS_ENABLED=<categories-or-tools>
-MIADI_TOOLS_DISABLED=<categories-or-tools>
-```
-
----
-
-## Technical Implementation Specification
-
-### File Structure
-```
-src/
-├── index.ts                 # Main server entry (HTTP instead of stdio)
-├── http-transport.ts        # Express.js HTTP transport implementation
-├── oauth/
-│   ├── resource-server.ts   # OAuth 2.1 Resource Server implementation
-│   ├── middleware.ts        # Authentication middleware
-│   ├── user-validator.ts    # EH_TOKEN integration
-│   └── metadata.ts          # /.well-known/oauth-protected-resource
-├── tool-registry.ts         # Existing (preserved)
-├── utils.ts                 # Existing (preserved) 
-└── tools/                   # Existing tool modules (preserved)
-```
-
-### Core Components
-
-#### 1. HTTP Transport Layer (`http-transport.ts`)
-```typescript
-interface HttpMcpTransport {
-  // Express.js server with JSON-RPC handling
-  server: Express.Application
-  port: number
-  
-  // Methods
-  start(): Promise<void>
-  stop(): Promise<void>
-  handleJsonRpc(req: Request, res: Response): Promise<void>
-}
-```
-
-#### 2. OAuth Resource Server (`oauth/resource-server.ts`)
-```typescript
-interface OAuthResourceServer {
-  // OAuth 2.1 Resource Server implementation
-  validateToken(token: string): Promise<TokenValidationResult>
-  getUserFromToken(token: string): Promise<MiadiUser>
-  generateMetadata(): ResourceServerMetadata
-  
-  // Integration with Miadi user system
-  validateUserAccess(userId: string): Promise<boolean>
-}
-```
-
-#### 3. Authentication Middleware (`oauth/middleware.ts`)
-```typescript
-interface AuthMiddleware {
-  // Express middleware for token validation
-  authenticate(req: Request, res: Response, next: NextFunction): Promise<void>
-  
-  // User context injection
-  injectUser(req: AuthenticatedRequest): void
-}
-```
-
-### MCP JSON-RPC Protocol Adaptation
-
-**Request Format** (unchanged from stdio):
-```json
-{
-  "jsonrpc": "2.0",
-  "id": "request-id",
-  "method": "tools/call",
-  "params": {
-    "name": "miadi-get-memory",
-    "arguments": {
-      "key": "session:active",
-      "type": "auto"
-    }
-  }
-}
-```
-
-**HTTP Headers Required**:
-```
-Authorization: Bearer <jwt-token>
-Content-Type: application/json
-Accept: application/json
-```
-
-**Response Format** (unchanged):
-```json
-{
-  "jsonrpc": "2.0",
-  "id": "request-id",
-  "result": {
-    "content": [
-      {
-        "type": "text",
-        "text": "<tool-result-json>"
-      }
-    ],
-    "_meta": {
-      "tool": "miadi-get-memory",
-      "timestamp": "2025-01-08T..."
-    }
-  }
-}
-```
-
----
-
-## Security Specifications
-
-### 1. OAuth 2.1 Security Requirements
-- **PKCE mandatory** for all authorization flows
-- **Resource Indicators** (RFC8707) for token scoping
-- **JWT validation** with signature verification and expiration checks
-- **HTTPS enforcement** in production (ngrok provides SSL termination)
-- **CORS policy** restricting origins to claude.ai domains
-
-### 2. Rate Limiting
-- **Per-user limits**: 100 requests per minute (existing implementation)
-- **Global limits**: 1000 requests per minute per server instance
-- **Tool-specific limits**: AI tools (10/min), others (50/min)
-
-### 3. User Authorization
-- **EH_TOKEN validation** for every authenticated request
-- **User scope validation** against Miadi user permissions
-- **Session invalidation** on user access revocation
-- **Audit logging** for all tool usage with user identification
-
-### 4. Error Security
-- **No sensitive data** in error responses
-- **Generic error messages** for authentication failures
-- **Request ID tracking** for debugging without exposing internals
-
----
-
-## Claude.ai Integration Specification
-
-### Connector Registration
-**Claude.ai Settings** → **Connectors** → **Add Custom Connector**
-
-**Configuration Fields**:
-```
-Name: Miadi Agent System
-Base URL: https://<ngrok-domain>.ngrok-free.app
-Description: Access to Miadi Three-Pathway Agent System with memory operations, session management, and AI integration
-Icon: <miadi-logo-url>
-```
-
-### OAuth Discovery Flow
-1. Claude.ai fetches `/.well-known/oauth-protected-resource` from server
-2. Discovers authorization and token endpoints
-3. Initiates Dynamic Client Registration if supported
-4. Redirects user to authorization endpoint with PKCE challenge
-5. User authenticates with identity provider
-6. Authorization code returned to claude.ai callback
-7. Token exchange with PKCE verifier
-8. Claude.ai stores access token for MCP tool invocations
-
-### Tool Invocation Pattern
-```
-Claude.ai → HTTP POST /mcp/jsonrpc
-Headers: Authorization: Bearer <jwt>
-Body: MCP JSON-RPC tool call
-← Tool result with Miadi agent data
-```
-
----
-
-## Testing and Validation Specification
-
-### 1. Unit Tests
-- OAuth token validation functions
-- User authorization against EH_TOKEN
-- HTTP transport JSON-RPC handling
-- Tool registry with HTTP context
-- Error handling and security edge cases
-
-### 2. Integration Tests
-- Complete OAuth flow with test identity provider
-- Tool execution through HTTP transport
-- User restriction enforcement
-- Rate limiting behavior
-- CORS and security headers
-
-### 3. End-to-End Tests
-- Ngrok tunnel establishment
-- Claude.ai connector registration
-- Full authentication and tool invocation flow
-- Error scenarios and recovery
-- Performance under load
-
-### 4. Security Testing
-- JWT token tampering attempts
-- Unauthorized user access attempts
-- Rate limiting bypass attempts
-- CORS policy violation tests
-- Input validation and injection tests
-
----
-
-## Performance and Monitoring Specification
-
-### 1. Metrics Collection
-- **Request latency**: P50, P95, P99 response times
-- **Authentication latency**: OAuth validation timing
-- **Tool execution time**: Per-tool performance tracking
-- **Error rates**: Authentication failures, tool errors, rate limits
-- **User activity**: Tool usage patterns per user
-
-### 2. Logging Requirements
-- **Structured logging** with request IDs and user context
-- **Security events**: Authentication failures, unauthorized access
-- **Performance events**: Slow requests, rate limit hits
-- **Business events**: Tool usage, session management actions
-
-### 3. Health Endpoints
-- `/health` - Basic server health check
-- `/metrics` - Prometheus-compatible metrics
-- `/oauth/health` - OAuth provider connectivity
-- `/miadi/health` - Miadi API connectivity
-
----
-
-## Migration and Rollback Specification
-
-### 1. Backward Compatibility
-- **Stdio transport preserved** for local development
-- **Environment variable** to select transport mode
-- **Tool registry unchanged** - no breaking changes to tool interface
-- **Configuration migration** from stdio to HTTP with sensible defaults
-
-### 2. Deployment Strategy
-- **Blue-green deployment** for production environments
-- **Canary rollout** for gradual user migration
-- **Rollback procedures** with automatic health check triggers
-- **Configuration validation** before server startup
-
-### 3. Migration Steps
-1. **Development**: Local HTTP server with ngrok tunnel
-2. **Staging**: Cloud deployment with test OAuth provider
-3. **Production**: Production OAuth integration and claude.ai connector
-4. **Validation**: User acceptance testing and performance verification
-5. **Rollout**: Gradual user migration with monitoring
-
----
-
-## Implementation Acceptance Criteria
-
-### ✅ Core Functionality
-- [ ] HTTP server replaces stdio transport completely
-- [ ] All 26 tools function identically through HTTP
-- [ ] Tool selection system works with HTTP transport
-- [ ] Error handling and logging preserved
-
-### ✅ OAuth Integration
-- [ ] OAuth 2.1 Resource Server fully compliant
-- [ ] Dynamic Client Registration working
-- [ ] JWT token validation secure and performant
-- [ ] EH_TOKEN user validation integrated
-
-### ✅ Security Requirements
-- [ ] All requests require valid authentication
-- [ ] User authorization enforced for all tools
-- [ ] Rate limiting and CORS protection active
-- [ ] Security headers properly configured
-
-### ✅ Claude.ai Compatibility
-- [ ] Connector registration successful
-- [ ] OAuth flow completes without errors
-- [ ] All tools accessible through claude.ai interface
-- [ ] Performance acceptable for interactive use
-
-### ✅ Production Readiness
-- [ ] Ngrok tunnel stable and performant
-- [ ] Health checks and monitoring operational
-- [ ] Error handling graceful and informative
-- [ ] Documentation complete and accurate
-
----
-
-This specification provides the complete natural language-driven blueprint for transforming the Miadi MCP server into a production-ready remote server with OAuth authentication and claude.ai integration while preserving all existing functionality and security requirements.