@goonnguyen/human-mcp 1.3.0 → 2.0.0

package/docs/codebase-summary.md

@@ -1,321 +0,0 @@
-# Human MCP - Codebase Summary
-
-## Overview
-
-Human MCP is a Model Context Protocol server that provides AI coding agents with visual analysis capabilities for debugging UI issues, processing screenshots, videos, and GIFs using Google Gemini AI. This summary provides a comprehensive overview of the codebase structure and key components.
-
-## Project Statistics
-
-- **Language**: TypeScript/JavaScript (Bun runtime)
-- **Total Source Files**: ~65 files
-- **Main Package**: @modelcontextprotocol/sdk, Google Generative AI, Zod, Sharp, fluent-ffmpeg
-- **Architecture**: MCP Server with plugin-based tools
-- **Build Tool**: Bun with TypeScript compilation
-
-## Directory Structure
-
-```
-human-mcp/
-├── .claude/                    # Claude Code agent configurations
-├── .github/workflows/          # CI/CD automation
-├── .serena/                   # Serena MCP tool configuration
-├── docs/                      # Project documentation (NEW)
-├── examples/                  # Usage examples
-├── src/                       # Source code
-│   ├── index.ts              # Entry point
-│   ├── server.ts             # MCP server setup
-│   ├── tools/eyes/           # Vision analysis tools
-│   ├── prompts/              # Pre-built debugging prompts
-│   ├── resources/            # MCP resources
-│   ├── types/                # TypeScript definitions
-│   └── utils/                # Core utilities
-├── tests/                    # Test suites
-├── dist/                     # Built output
-└── Configuration files       # package.json, tsconfig.json, etc.
-```
-
-## Core Components
-
-### 1. MCP Server (`src/server.ts`, `src/index.ts`)
-
-**Purpose**: Initializes and starts the MCP server with stdio transport
-**Key Functions**:
-- Creates McpServer instance with metadata
-- Registers tools, prompts, and resources
-- Handles server lifecycle and error management
-
-**Architecture Pattern**: Server initialization with dependency injection
-
-```typescript
-export async function createServer() {
-  const config = loadConfig();
-  const server = new McpServer({
-    name: "human-mcp",
-    version: "1.0.0",
-  });
-
-  await registerEyesTool(server, config);
-  await registerPrompts(server);
-  await registerResources(server);
-  
-  return server;
-}
-```
-
-### 2. Vision Analysis Tools (`src/tools/eyes/`)
-
-**Primary Tool**: `eyes.analyze` - Multi-modal visual content analysis
-**Secondary Tool**: `eyes.compare` - Image comparison and difference detection
-
-#### Tool Structure:
-- **`index.ts`**: Tool registration and orchestration
-- **`schemas.ts`**: Zod validation schemas for inputs/outputs
-- **`processors/`**: Media-specific processing logic
-  - `image.ts`: Direct image analysis
-  - `video.ts`: Video frame extraction and analysis
-  - `gif.ts`: GIF frame extraction and sequence analysis
-- **`utils/`**: Tool utilities
-  - `gemini-client.ts`: Google Gemini API integration
-  - `formatters.ts`: Output formatting and structuring
-
-#### Key Features:
-- **Multi-format Support**: Images (PNG, JPEG, WebP), Videos (MP4, WebM, MOV, AVI), GIFs
-- **Analysis Types**: general, ui_debug, error_detection, accessibility, performance, layout
-- **Input Sources**: File paths, URLs, base64 data URIs
-- **Comparison Types**: pixel, structural, semantic differences
-
-### 3. Pre-built Debugging Workflows (`src/prompts/`)
-
-**Component**: Debugging prompt templates for common UI analysis scenarios
-
-**Available Prompts**:
-- `debug_ui_screenshot`: Layout and rendering issue detection
-- `analyze_error_recording`: Temporal error pattern analysis
-- `accessibility_audit`: WCAG compliance and accessibility checking
-- `performance_visual_audit`: Performance indicator analysis
-- `layout_comparison`: Before/after layout difference analysis
-
-**Integration**: Prompts are registered as MCP prompt resources with templating support
-
-### 4. Configuration Management (`src/utils/config.ts`)
-
-**Pattern**: Environment-driven configuration with Zod validation
-**Key Features**:
-- **Required**: Google Gemini API key
-- **Optional**: Model selection, timeouts, caching, logging levels
-- **Validation**: Runtime configuration validation with meaningful error messages
-- **Defaults**: Sensible defaults for all optional settings
-
-**Configuration Schema**:
-```typescript
-const ConfigSchema = z.object({
-  gemini: z.object({
-    apiKey: z.string().min(1, "Google Gemini API key is required"),
-    model: z.string().default("gemini-2.5-flash"),
-  }),
-  server: z.object({
-    requestTimeout: z.number().default(300000),
-    fetchTimeout: z.number().default(60000),
-    // ... other server config
-  }),
-  // ... security, logging config
-});
-```
-
-### 5. Error Handling & Logging (`src/utils/`)
-
-**Error Handling (`errors.ts`)**:
-- Centralized error processing with MCP compliance
-- Structured error responses with meaningful messages
-- Error categorization and appropriate HTTP status mapping
-
-**Logging (`logger.ts`)**:
-- Structured logging with configurable levels
-- Context-aware logging with request tracking
-- Performance metrics and timing information
-- Privacy-conscious logging (no sensitive data)
-
-### 6. Media Processing Architecture
-
-#### Image Processing (`src/tools/eyes/processors/image.ts`)
-- Direct Gemini Vision API integration
-- Support for all major image formats
-- Base64 and URL input handling
-- OCR and element detection capabilities
-
-#### Video Processing (`src/tools/eyes/processors/video.ts`)
-- ffmpeg integration via fluent-ffmpeg
-- Frame extraction with configurable sampling
-- Temporal analysis for error detection
-- Support for common video formats
-
-#### GIF Processing (`src/tools/eyes/processors/gif.ts`)
-- Sharp library for frame extraction
-- Animation sequence understanding
-- Frame-by-frame analysis capabilities
-- Support for both static and animated GIFs
-
-### 7. Type System (`src/types/`, `src/tools/eyes/schemas.ts`)
-
-**Type Safety Features**:
-- Comprehensive TypeScript type definitions
-- Zod runtime validation schemas
-- Input/output type inference
-- MCP protocol compliance types
-
-**Key Schemas**:
-- `EyesInputSchema`: Visual analysis input validation
-- `EyesOutputSchema`: Structured analysis output format
-- `CompareInputSchema`: Image comparison input validation
-- `Config`: Environment configuration typing
-
-### 8. Testing Infrastructure (`tests/`)
-
-**Test Structure**:
-- **Unit Tests**: Individual function and utility testing
-- **Integration Tests**: End-to-end MCP server functionality
-- **Setup**: Centralized test environment configuration
-- **Coverage**: Core utilities and error handling
-
-**Testing Tools**:
-- Bun built-in test runner
-- MCP inspector for manual testing
-- Mock implementations for external services
-
-## Key Dependencies
-
-### Runtime Dependencies
-- **@modelcontextprotocol/sdk**: MCP protocol implementation
-- **@google/generative-ai**: Google Gemini API client
-- **zod**: Runtime type validation and parsing
-- **sharp**: Image processing and manipulation
-- **fluent-ffmpeg**: Video processing wrapper for ffmpeg
-
-### Development Dependencies
-- **typescript**: Static type checking and compilation
-- **@modelcontextprotocol/inspector**: Interactive MCP tool testing
-- **semantic-release**: Automated version management and publishing
-- **@types/*****: TypeScript type definitions for Node.js libraries
-
-### System Dependencies
-- **Bun Runtime**: JavaScript/TypeScript runtime environment
-- **ffmpeg**: Video processing system dependency
-- **Node.js**: Alternative runtime compatibility
-
-## Configuration & Environment
-
-### Required Environment Variables
-- `GOOGLE_GEMINI_API_KEY`: Google Gemini API access key (required)
-
-### Optional Environment Variables
-- `GOOGLE_GEMINI_MODEL`: AI model selection (default: gemini-2.5-flash)
-- `LOG_LEVEL`: Logging verbosity (debug, info, warn, error)
-- `REQUEST_TIMEOUT`: Operation timeout in milliseconds (default: 300000)
-- `FETCH_TIMEOUT`: HTTP request timeout in milliseconds (default: 60000)
-- `ENABLE_CACHING`: Enable response caching (default: true)
-- `CACHE_TTL`: Cache time-to-live in seconds (default: 3600)
-
-### TypeScript Configuration
-- **Target**: ESNext with bundler module resolution
-- **Strict Mode**: All strict type checking options enabled
-- **Path Mapping**: `@/*` aliases for clean imports
-- **No Emit**: Bun handles compilation directly
-
-## Architecture Patterns
-
-### 1. Plugin-based Tool Architecture
-Tools are registered dynamically with the MCP server, allowing for easy extension and modification without changing core server code.
-
-### 2. Strategy Pattern for Media Processing
-Different processors handle different media types, allowing for specialized optimization and feature sets per media type.
-
-### 3. Configuration-driven Development
-All runtime behavior configurable through environment variables with validation and sensible defaults.
-
-### 4. Error-first Design
-Comprehensive error handling at every layer with structured error responses and logging.
-
-### 5. Schema-driven Validation
-All external inputs validated through Zod schemas with TypeScript type inference for compile-time safety.
-
-## Integration Points
-
-### MCP Client Integration
-The server exposes standard MCP protocol endpoints via stdio transport, making it compatible with any MCP-enabled AI agent or client.
-
-### Google Gemini AI Integration
-Direct integration with Google's Gemini API for visual analysis, with configurable model selection and comprehensive error handling.
-
-### System Tool Integration
-Integration with system-level tools (ffmpeg for video processing, Sharp for image processing) with proper error handling and fallback mechanisms.
-
-## Development Workflow
-
-### Development Commands
-```bash
-bun run dev        # Development server with hot reload
-bun run build      # Production build
-bun run start      # Run production build
-bun test           # Run test suite
-bun run typecheck  # TypeScript type checking
-bun run inspector  # MCP tool inspector for testing
-```
-
-### Testing Strategy
-- **Unit Testing**: Individual function testing with mocks
-- **Integration Testing**: Full MCP server workflow testing
-- **Manual Testing**: Interactive testing via MCP inspector
-- **Configuration Testing**: Environment variable validation testing
-
-## Performance Characteristics
-
-### Response Times
-- **Image Analysis**: 10-30 seconds depending on detail level
-- **Video Processing**: 1-3 minutes for typical clips
-- **GIF Analysis**: 30 seconds to 2 minutes depending on frame count
-- **Image Comparison**: 15-45 seconds for detailed comparison
-
-### Memory Usage
-- **Base Server**: ~50-100MB
-- **Image Processing**: +20-100MB per operation
-- **Video Processing**: +100-500MB depending on video size
-- **Concurrent Operations**: Scales linearly with request count
-
-### Scalability Considerations
-- **Stateless Design**: No persistent state between requests
-- **Rate Limiting**: Configurable limits to prevent API abuse
-- **Resource Cleanup**: Proper cleanup of temporary files and memory
-- **Concurrent Request Handling**: Built-in MCP protocol concurrency support
-
-## Security Features
-
-### API Key Management
-- Environment variable based configuration only
-- No hardcoded credentials anywhere in codebase
-- Validation of required credentials at startup
-
-### Input Validation
-- All external inputs validated through Zod schemas
-- File path sanitization for local file access
-- URL validation for remote content fetching
-- Content size limits to prevent abuse
-
-### Rate Limiting & Abuse Prevention
-- Configurable rate limiting per time window
-- Request size limits for large media files
-- Timeout mechanisms to prevent resource exhaustion
-
-## Future Extension Points
-
-The codebase is designed for easy extension in several areas:
-
-1. **Additional AI Models**: Easy integration of new AI vision models beyond Gemini
-2. **New Media Types**: Plugin architecture supports adding new media processors
-3. **Enhanced Analysis Types**: New analysis types can be added to existing processors
-4. **Transport Protocols**: Support for additional MCP transport methods
-5. **Caching Strategies**: More sophisticated caching implementations
-6. **Monitoring & Metrics**: Enhanced observability and performance monitoring
-
-## Summary
-
-Human MCP represents a well-architected, extensible solution for bringing visual analysis capabilities to AI agents through the Model Context Protocol. The codebase demonstrates modern TypeScript best practices, robust error handling, comprehensive configuration management, and a clean separation of concerns that enables both reliability and extensibility.

package/docs/project-overview-pdr.md

@@ -1,286 +0,0 @@
-# Human MCP - Project Overview & Product Development Requirements
-
-## Project Overview
-
-**Human MCP** is a Model Context Protocol (MCP) server that provides AI coding agents with advanced visual analysis capabilities for debugging UI issues, processing screenshots, videos, and GIFs using Google Gemini AI. It bridges the gap between AI agents and human-like visual perception, enabling sophisticated multimodal debugging workflows.
-
-### Vision Statement
-**"Bringing Human Capabilities to Coding Agents"**
-
-To transform AI coding agents with comprehensive human-like sensory capabilities, enabling sophisticated multimodal analysis, debugging workflows, and content understanding. Human MCP bridges the gap between artificial intelligence and human perception through advanced visual analysis, document understanding, audio processing, speech generation, and content creation capabilities.
-
-### Core Purpose
-- **Phase 1 (Complete)**: Advanced visual analysis capabilities for images, videos, and GIFs
-- **Phase 2 (Q1 2025)**: Document understanding and structured data extraction  
-- **Phase 3 (Q2 2025)**: Audio processing and speech-to-text capabilities
-- **Phase 4 (Q3 2025)**: Speech generation and text-to-speech features
-- **Phase 5 (Q4 2025)**: Content generation including image and video creation
-
-For detailed development roadmap, see **[Project Roadmap](project-roadmap.md)**.
-
-### Google Gemini Documentation
-- [Gemini API](https://ai.google.dev/gemini-api/docs?hl=en)
-- [Gemini Models](https://ai.google.dev/gemini-api/docs/models)
-- [Video Understanding](https://ai.google.dev/gemini-api/docs/video-understanding?hl=en)
-- [Image Understanding](https://ai.google.dev/gemini-api/docs/image-understanding)
-- [Document Understanding](https://ai.google.dev/gemini-api/docs/document-processing)
-- [Audio Understanding](https://ai.google.dev/gemini-api/docs/audio)
-- [Speech Generation](https://ai.google.dev/gemini-api/docs/speech-generation)
-- [Image Generation](https://ai.google.dev/gemini-api/docs/image-generation)
-- [Video Generation](https://ai.google.dev/gemini-api/docs/video)
-
-## Product Development Requirements (PDR)
-
-### 1. Functional Requirements
-
-#### 1.1 Core MCP Tools
-
-**FR-1.1: Visual Analysis Tool (`eyes.analyze`)**
-- **Requirement**: Process images, videos, and GIFs with AI-powered visual analysis
-- **Input Types**: File paths, URLs, base64 data URIs
-- **Media Support**: PNG, JPEG, WebP, GIF (images), MP4, WebM, MOV, AVI (videos), animated GIFs
-- **Analysis Types**: general, ui_debug, error_detection, accessibility, performance, layout
-- **Detail Levels**: quick, detailed
-- **Output**: Structured analysis with detected elements, debugging insights, and recommendations
-
-**FR-1.2: Image Comparison Tool (`eyes.compare`)**
-- **Requirement**: Compare two images to identify visual differences
-- **Comparison Types**: pixel (exact differences), structural (layout changes), semantic (content meaning)
-- **Output**: Summary, specific differences, impact assessment, recommendations
-- **Use Cases**: Before/after comparisons, regression testing, layout validation
-
-#### 1.2 Media Processing Capabilities
-
-**FR-2.1: Image Processing**
-- Support standard image formats (PNG, JPEG, WebP, static GIF)
-- Handle various input sources (file paths, URLs, base64)
-- Extract visual elements and metadata
-- Perform OCR and text extraction when requested
-
-**FR-2.2: Video Processing**
-- Support common video formats (MP4, WebM, MOV, AVI)
-- Frame extraction using ffmpeg via fluent-ffmpeg
-- Configurable frame sampling (max_frames parameter)
-- Temporal analysis for error detection and workflow understanding
-
-**FR-2.3: GIF Processing**
-- Animated GIF frame extraction using Sharp library
-- Frame-by-frame analysis capabilities
-- Animation sequence understanding
-- Support for both animated and static GIFs
-
-#### 1.3 Pre-built Debugging Workflows
-
-**FR-3.1: Debugging Prompts**
-- UI screenshot debugging with layout issue detection
-- Error recording analysis for temporal error patterns
-- Accessibility audits with WCAG compliance checking
-- Performance visual audits for loading and render issues
-- Layout comparison for responsive design validation
-
-**FR-3.2: Resource Documentation**
-- Comprehensive MCP tool documentation
-- Usage examples and integration guides
-- Best practices for visual debugging workflows
-- API reference and configuration options
-
-### 2. Non-Functional Requirements
-
-#### 2.1 Performance Requirements
-
-**NFR-1.1: Response Time**
-- Quick analysis mode: < 10 seconds for images
-- Detailed analysis mode: < 30 seconds for images
-- Video processing: < 2 minutes for 30-second clips
-- Request timeout: 5 minutes (configurable)
-- Fetch timeout: 60 seconds for HTTP requests
-
-**NFR-1.2: Scalability**
-- Support concurrent requests through MCP protocol
-- Configurable rate limiting (default: 100 requests/minute)
-- Memory-efficient media processing
-- Streaming support for large files
-
-#### 2.2 Reliability Requirements
-
-**NFR-2.1: Error Handling**
-- Comprehensive error catching and logging
-- Graceful degradation for unsupported media types
-- Retry mechanisms for network requests
-- Structured error responses with meaningful messages
-
-**NFR-2.2: Data Security**
-- Secure handling of API keys and credentials
-- No persistent storage of processed media
-- Optional request/response logging with privacy controls
-- Rate limiting to prevent abuse
-
-#### 2.3 Integration Requirements
-
-**NFR-3.1: MCP Protocol Compliance**
-- Full Model Context Protocol specification adherence
-- Stdio transport for command-line integration
-- Proper tool registration and schema validation
-- Compatible with MCP-enabled AI agents and clients
-
-**NFR-3.2: External Dependencies**
-- Google Gemini API integration with configurable models
-- ffmpeg for video processing capabilities
-- Sharp library for image manipulation
-- Zod for runtime type validation
-
-### 3. Technical Requirements
-
-#### 3.1 Runtime Environment
-
-**TR-1.1: Runtime Platform**
-- Bun runtime environment (JavaScript/TypeScript)
-- Node.js compatibility for broader deployment
-- ESNext module system with bundler resolution
-- TypeScript with strict type checking
-
-**TR-1.2: System Dependencies**
-- ffmpeg installed and accessible in PATH
-- Internet connectivity for Gemini API access
-- File system access for local media processing
-- Minimum 512MB RAM for media processing
-
-#### 3.2 Configuration Management
-
-**TR-2.1: Environment Configuration**
-- Required: `GOOGLE_GEMINI_API_KEY`
-- Optional: Model selection, timeout settings, caching options
-- Zod-based configuration validation
-- Environment variable override support
-
-**TR-2.2: Runtime Configuration**
-- Default Gemini model: gemini-2.5-flash
-- Configurable request and fetch timeouts
-- Enable/disable caching with TTL settings
-- Logging level configuration (debug, info, warn, error)
-
-### 4. Development Requirements
-
-#### 4.1 Code Quality Standards
-
-**DR-1.1: TypeScript Standards**
-- Strict type checking enabled
-- Path mapping with `@/*` aliases
-- Comprehensive type definitions for all APIs
-- Zod schemas for runtime validation
-
-**DR-1.2: Error Handling Patterns**
-- Centralized error handling via utils/errors.ts
-- Structured error responses with MCP compliance
-- Comprehensive logging with configurable levels
-- Graceful error recovery where possible
-
-#### 4.1 Testing Requirements
-
-**DR-2.1: Test Coverage**
-- Unit tests for core utilities and processors
-- Integration tests for MCP server functionality
-- Manual testing via MCP inspector tool
-- Configuration validation testing
-
-**DR-2.2: Development Tools**
-- MCP inspector for interactive tool testing
-- Hot reload development server
-- TypeScript compilation checking
-- Build process for production deployment
-
-### 5. Deployment Requirements
-
-#### 5.1 Distribution
-
-**DP-1.1: Package Distribution**
-- npm package with semantic versioning
-- Automated release process via GitHub Actions
-- Comprehensive README and documentation
-- Example usage and integration guides
-
-**DP-1.2: Installation Requirements**
-- Bun or Node.js runtime environment
-- ffmpeg system dependency
-- Google Gemini API key setup
-- MCP client configuration
-
-### 6. Success Metrics
-
-#### 6.1 Functional Metrics
-- **Tool Adoption**: Number of MCP clients integrating Human MCP
-- **Processing Success Rate**: >95% successful analysis completion
-- **Response Time**: <30 seconds for detailed image analysis
-- **Error Rate**: <2% unhandled errors in production use
-
-#### 6.2 Quality Metrics
-- **Code Coverage**: >80% test coverage for core functionality
-- **Documentation Coverage**: 100% API documentation completeness
-- **User Satisfaction**: Positive feedback from integration partners
-- **Performance**: Memory usage <100MB for typical operations
-
-### 7. Constraints and Limitations
-
-#### 7.1 Technical Constraints
-- **Gemini API Dependency**: Requires active Google Gemini API key
-- **System Dependencies**: Requires ffmpeg for video processing
-- **Memory Limitations**: Large media files may require streaming
-- **Network Dependency**: Requires internet access for AI processing
-
-#### 7.2 Operational Constraints
-- **Rate Limiting**: Subject to Gemini API quotas and limits
-- **Cost Considerations**: AI API usage costs scale with usage
-- **Privacy**: Processed content sent to Google's AI services
-- **Regional Availability**: Limited by Gemini API geographic availability
-
-### 8. Future Roadmap
-
-**Current Status**: Phase 1 Complete - Visual Analysis Foundation (v1.2.1)
-
-#### 8.1 Phase 2: Document Understanding (Q1 2025)
-- **Document Analysis**: PDF, Word, Excel, PowerPoint processing
-- **Structured Data Extraction**: Schema-based data extraction from documents
-- **Multi-format Support**: Text, markdown, and document format analysis
-- **Document Comparison**: Cross-document analysis and comparison
-
-#### 8.2 Phase 3: Audio Processing (Q2 2025)
-- **Speech-to-Text**: Advanced transcription with speaker identification
-- **Audio Analysis**: Content classification and quality assessment  
-- **Audio Comparison**: A/B testing and regression detection for audio content
-- **Multi-format Support**: WAV, MP3, AAC, OGG, FLAC processing
-
-#### 8.3 Phase 4: Speech Generation (Q3 2025)
-- **Text-to-Speech**: High-quality speech synthesis with customizable voices
-- **Technical Narration**: Code explanation and documentation narration
-- **Multi-language Support**: International speech generation capabilities
-- **Voice Customization**: Configurable speech parameters and effects
-
-#### 8.4 Phase 5: Content Generation (Q4 2025)
-- **Image Generation**: AI-powered image creation using Google Imagen
-- **Video Generation**: Video content creation using Google Veo3
-- **Batch Processing**: Automated content generation workflows
-- **Style Customization**: Artistic and technical style controls
-
-For complete roadmap details, timeline, and technical specifications, see **[Project Roadmap](project-roadmap.md)**.
-
-### 9. Risk Assessment
-
-#### 9.1 Technical Risks
-- **High**: Gemini API changes breaking compatibility
-- **Medium**: ffmpeg dependency issues across platforms
-- **Low**: Memory constraints with large media files
-
-#### 9.2 Business Risks
-- **High**: Changes to Gemini API pricing or availability
-- **Medium**: Competition from similar visual analysis tools
-- **Low**: MCP protocol evolution requiring updates
-
-#### 9.3 Mitigation Strategies
-- **Multi-provider Support**: Implement additional AI model backends
-- **Graceful Degradation**: Fallback processing modes for limited environments
-- **Documentation**: Comprehensive setup guides and troubleshooting
-- **Community**: Open-source development with contributor engagement
-
-## Conclusion
-
-Human MCP represents a significant advancement in AI-assisted visual debugging and analysis. By providing sophisticated computer vision capabilities through the Model Context Protocol, it enables AI agents to perform human-like visual analysis tasks, significantly improving debugging workflows and development productivity. The project's modular architecture, comprehensive error handling, and extensive configuration options make it suitable for both individual developers and enterprise deployments.