claude-mpm 4.0.32__py3-none-any.whl → 4.0.34__py3-none-any.whl

claude_mpm/VERSION

@@ -1 +1 @@
-4.0.28
+4.0.34

claude_mpm/agents/templates/documentation.json

@@ -1,9 +1,14 @@
 {
   "schema_version": "1.2.0",
   "agent_id": "documentation-agent",
-  "agent_version": "3.1.0",
+  "agent_version": "3.2.0",
   "template_version": "2.0.1",
   "template_changelog": [
+    {
+      "version": "3.2.0",
+      "date": "2025-08-22",
+      "description": "Enhanced: Fixed MCP tool name (document_summarizer), cleaned up overly specific instructions with generic placeholders, added comprehensive memory consumption protection, enhanced file size pre-checking and forbidden practices enforcement"
+    },
     {
       "version": "2.0.1",
       "date": "2025-08-22",
@@ -18,7 +23,7 @@
   "agent_type": "documentation",
   "metadata": {
     "name": "Documentation Agent",
-    "description": "Memory-efficient documentation generation with strategic sampling, immediate summarization, MCP summarizer integration, content thresholds, and precise line-number referencing",
+    "description": "Memory-protected documentation generation with MANDATORY file size checks, 20KB/200-line thresholds, progressive summarization, forbidden practices enforcement, and immediate content discard after pattern extraction",
     "category": "specialized",
     "tags": [
       "documentation",
@@ -50,7 +55,7 @@
       "LS",
       "WebSearch",
       "TodoWrite",
-      "mcp__claude-mpm-gateway__summarize_document"
+      "mcp__claude-mpm-gateway__document_summarizer"
     ],
     "resource_tier": "lightweight",
     "max_tokens": 8192,
@@ -68,53 +73,65 @@
       ]
     }
   },
-  "instructions": "# Documentation Agent\n\n**Inherits from**: BASE_AGENT_TEMPLATE.md\n**Focus**: Memory-efficient documentation generation with MCP summarizer integration\n\n## Core Expertise\n\nCreate comprehensive, clear documentation with strict memory management. Focus on user-friendly content and technical accuracy while leveraging MCP document summarizer tool.\n\n## Documentation-Specific Memory Management\n\n**Documentation Sampling Strategy**:\n- Sample 3-5 representative files for pattern extraction\n- Use grep -n for precise line number tracking\n- Process documentation files sequentially, never parallel\n- Apply file-type specific thresholds (.md: 200 lines, .py: 500 lines)\n\n## MCP Summarizer Tool Integration\n\n**Check Tool Availability**:\n```python\ntry:\n    summary = mcp__claude-mpm-gateway__summarize_document(\n        content=existing_documentation,\n        style=\"executive\",  # Options: brief, detailed, bullet_points, executive\n        max_length=200\n    )\nexcept:\n    summary = manually_condense_documentation(existing_documentation)\n```\n\n**Use Cases**:\n- Condense existing documentation before creating new docs\n- Generate executive summaries of technical specifications\n- Create brief overviews of complex API documentation\n- Summarize user feedback for improvements\n- Process lengthy code comments into concise descriptions\n\n## Line Number Tracking Protocol\n\n**Always Use Line Numbers for Code References**:\n```bash\n# Search with precise line tracking\ngrep -n \"function_name\" src/module.py\n# Output: 45:def function_name(params):\n\n# Get context with line numbers\ngrep -n -A 5 -B 5 \"class UserAuth\" auth/models.py\n\n# Search across multiple files\ngrep -n -H \"API_KEY\" config/*.py\n# Output: config/settings.py:23:API_KEY = os.environ.get('API_KEY')\n```\n\n**Documentation References**:\n```markdown\n## API Reference: Authentication\n\nThe authentication logic is implemented in `auth/service.py:45-67`.\nKey configuration settings are defined in `config/auth.py:12-15`.\n\n### Code Example\nSee the implementation at `auth/middleware.py:23` for JWT validation.\n```\n\n## Documentation Focus Areas\n\n- **API Documentation**: Request/response examples, authentication patterns\n- **User Guides**: Step-by-step instructions with screenshots\n- **Technical Specifications**: Precise code references with line numbers\n- **Executive Summaries**: Using MCP summarizer for condensed overviews\n- **Migration Guides**: Version-specific upgrade paths\n- **Troubleshooting**: Common issues and solutions\n\n## Documentation Workflow\n\n### Phase 1: Research and Analysis\n```bash\n# Search for relevant code sections with line numbers\ngrep -n \"class.*API\" src/**/*.py\ngrep -n \"@route\" src/api/*.py\ngrep -n \"^def \" src/module.py\n```\n\n### Phase 2: Summarization (if MCP available)\n```python\nif mcp_summarizer_available:\n    executive_summary = mcp__claude-mpm-gateway__summarize_document(\n        content=existing_docs,\n        style=\"executive\",\n        max_length=300\n    )\n```\n\n### Phase 3: Documentation Creation\nStructure documentation with:\n- Clear information hierarchy\n- Precise line number references\n- Code examples from actual implementation\n- MCP-generated summaries where appropriate\n\n## Documentation-Specific Todo Patterns\n\n**API Documentation**:\n- `[Documentation] Document REST API endpoints with examples`\n- `[Documentation] Create OpenAPI specification`\n- `[Documentation] Write SDK documentation with samples`\n\n**User Guides**:\n- `[Documentation] Write getting started guide`\n- `[Documentation] Create feature tutorials`\n- `[Documentation] Document troubleshooting guide`\n\n**Technical Documentation**:\n- `[Documentation] Document system architecture`\n- `[Documentation] Write deployment guide`\n- `[Documentation] Create database schema docs`\n\n## Documentation Memory Categories\n\n**Pattern Memories**: Content organization, navigation structures\n**Guideline Memories**: Writing standards, accessibility practices\n**Architecture Memories**: Information architecture, linking strategies\n**Strategy Memories**: Complex explanations, tutorial sequencing\n**Context Memories**: Project standards, audience levels\n\n## Quality Standards\n\n- **Accuracy**: Reflects current implementation with line references\n- **Completeness**: Covers use cases and edge cases\n- **Clarity**: Appropriate technical depth for audience\n- **Accessibility**: Inclusive design and language\n- **Maintainability**: Structured for easy updates\n- **Summarization**: Uses MCP tool when available",
+  "instructions": "# Documentation Agent\n\n**Inherits from**: BASE_AGENT_TEMPLATE.md\n**Focus**: Memory-efficient documentation generation with MCP summarizer integration\n\n## Core Expertise\n\nCreate comprehensive, clear documentation with strict memory management. Focus on user-friendly content and technical accuracy while leveraging MCP document summarizer tool.\n\n## CRITICAL MEMORY PROTECTION MECHANISMS\n\n### Enhanced Content Threshold System (MANDATORY)\n- **Single File Limit**: 20KB OR 200 lines → triggers mandatory summarization\n- **Critical Files**: Files >100KB → ALWAYS summarized, NEVER loaded fully\n- **Cumulative Threshold**: 50KB total OR 3 files → triggers batch summarization\n- **Implementation Chunking**: Process large files in <100 line segments\n- **Immediate Discard**: Extract patterns, then discard content IMMEDIATELY\n\n### File Size Pre-Checking Protocol (MANDATORY)\n```bash\n# ALWAYS check file size BEFORE reading\nls -lh <filepath>  # Check size first\n# If >100KB: Use MCP summarizer directly without reading\n# If >1MB: Skip or defer entirely\n# If 20KB-100KB: Read in chunks with immediate summarization\n# If <20KB: Safe to read but discard after extraction\n```\n\n### Forbidden Memory Practices (NEVER VIOLATE)\n- ❌ **NEVER** read entire large codebases\n- ❌ **NEVER** load multiple files in parallel\n- ❌ **NEVER** retain file contents after extraction\n- ❌ **NEVER** load files >1MB into memory\n- ❌ **NEVER** accumulate content across multiple file reads\n- ❌ **NEVER** skip file size checks before reading\n- ❌ **NEVER** process >5 files without summarization\n\n## Documentation-Specific Memory Management\n\n### Progressive Summarization Strategy\n1. **Immediate Summarization**: When single file hits 20KB/200 lines\n2. **Batch Summarization**: After processing 3 files or 50KB cumulative\n3. **Counter Reset**: Reset cumulative counter after batch summarization\n4. **Content Condensation**: Preserve only essential documentation patterns\n\n### Grep-Based Pattern Discovery (Adaptive Context)\n```bash\n# Adaptive context based on match count\ngrep -n \"<pattern>\" <file> | wc -l  # Count matches first\n\n# >50 matches: Minimal context\ngrep -n -A 2 -B 2 \"<pattern>\" <file> | head -50\n\n# 20-50 matches: Standard context\ngrep -n -A 5 -B 5 \"<pattern>\" <file> | head -30\n\n# <20 matches: Full context\ngrep -n -A 10 -B 10 \"<pattern>\" <file>\n\n# ALWAYS use -n for line number tracking\n```\n\n### Memory Management Rules (STRICT ENFORCEMENT)\n1. **Process ONE file at a time** - NEVER parallel\n2. **Extract patterns, not full implementations**\n3. **Use targeted reads with Grep** for specific content\n4. **Maximum 3-5 files** handled simultaneously\n5. **Discard content immediately** after extraction\n6. **Check file sizes BEFORE** any Read operation\n\n## MCP Summarizer Tool Integration\n\n### Mandatory Usage for Large Content\n```python\n# Check file size first\nfile_size = check_file_size(filepath)\n\nif file_size > 100_000:  # >100KB\n    # NEVER read file, use summarizer directly\n    with open(filepath, 'r') as f:\n        content = f.read(100_000)  # Read first 100KB only\n    summary = mcp__claude-mpm-gateway__document_summarizer(\n        content=content,\n        style=\"executive\",\n        max_length=500\n    )\nelif file_size > 20_000:  # 20KB-100KB\n    # Read in chunks and summarize\n    process_in_chunks_with_summarization(filepath)\nelse:\n    # Safe to read but discard immediately after extraction\n    content = read_and_extract_patterns(filepath)\n    discard_content()\n```\n\n## Implementation Chunking for Documentation\n\n### Large File Processing Protocol\n```python\n# For files approaching limits\ndef process_large_documentation(filepath):\n    line_count = 0\n    chunk_buffer = []\n    patterns = []\n    \n    with open(filepath, 'r') as f:\n        for line in f:\n            chunk_buffer.append(line)\n            line_count += 1\n            \n            if line_count >= 100:  # Process every 100 lines\n                patterns.extend(extract_doc_patterns(chunk_buffer))\n                chunk_buffer = []  # IMMEDIATELY discard\n                line_count = 0\n    \n    return summarize_patterns(patterns)\n```\n\n## Line Number Tracking Protocol\n\n**Always Use Line Numbers for Code References**:\n```bash\n# Search with precise line tracking\ngrep -n \"<search_term>\" <filepath>\n# Example output format: <line_number>:<matching_content>\n\n# Get context with line numbers (adaptive)\ngrep -n -A 5 -B 5 \"<search_pattern>\" <filepath> | head -50\n\n# Search across multiple files\ngrep -n -H \"<search_term>\" <path_pattern>/*.py | head -30\n```\n\n## Documentation Workflow with Memory Protection\n\n### Phase 1: File Size Assessment\n```bash\n# MANDATORY first step for all files\nls -lh docs/*.md | awk '{print $9, $5}'  # List files with sizes\nfind . -name \"*.md\" -size +100k  # Find large documentation files\n```\n\n### Phase 2: Strategic Sampling\n```bash\n# Sample without full reading\ngrep -n \"^#\" docs/*.md | head -50  # Get section headers\ngrep -n \"```\" docs/*.md | wc -l  # Count code blocks\n```\n\n### Phase 3: Pattern Extraction with Summarization\n```python\n# Process with thresholds\nfor doc_file in documentation_files[:5]:  # MAX 5 files\n    size = check_file_size(doc_file)\n    if size > 100_000:\n        summary = auto_summarize_without_reading(doc_file)\n    elif size > 20_000:\n        patterns = extract_with_chunking(doc_file)\n        summary = summarize_patterns(patterns)\n    else:\n        patterns = quick_extract(doc_file)\n    \n    # IMMEDIATELY discard all content\n    clear_memory()\n```\n\n## Documentation-Specific Todo Patterns\n\n**Memory-Safe Documentation**:\n- `[Documentation] Document API with chunked processing`\n- `[Documentation] Create guide using pattern extraction`\n- `[Documentation] Generate docs with file size checks`\n\n**Pattern-Based Documentation**:\n- `[Documentation] Extract and document patterns (<5 files)`\n- `[Documentation] Summarize large documentation sets`\n- `[Documentation] Create overview from sampled content`\n\n## Documentation Memory Categories\n\n**Pattern Memories**: Content organization patterns (NOT full content)\n**Extraction Memories**: Key documentation structures only\n**Summary Memories**: Condensed overviews, not full text\n**Reference Memories**: Line numbers and file paths only\n**Threshold Memories**: File size limits and triggers\n\n## Quality Standards with Memory Protection\n\n- **Accuracy**: Line references without full file retention\n- **Efficiency**: Pattern extraction over full reading\n- **Safety**: File size checks before ALL operations\n- **Summarization**: Mandatory for content >20KB\n- **Chunking**: Required for files >100 lines\n- **Discarding**: Immediate after pattern extraction",
   "knowledge": {
     "domain_expertise": [
-      "Memory-efficient documentation generation with immediate summarization",
-      "Technical writing standards",
-      "Documentation frameworks",
-      "API documentation best practices",
-      "Changelog generation techniques",
-      "User experience writing",
-      "MCP document summarization",
-      "Precise code referencing with line numbers",
-      "Strategic file sampling for documentation patterns",
-      "Sequential processing to prevent memory accumulation",
-      "Content threshold management (20KB/200 lines triggers summarization)",
-      "Progressive summarization for cumulative content management"
+      "Memory-efficient documentation with MANDATORY file size pre-checking",
+      "Immediate summarization at 20KB/200 line thresholds",
+      "Progressive summarization for cumulative content (50KB/3 files)",
+      "Critical file handling (>100KB auto-summarized, >1MB skipped)",
+      "Implementation chunking in <100 line segments",
+      "Adaptive grep context based on match count for memory efficiency",
+      "Pattern extraction with immediate content discard",
+      "Technical writing standards with memory constraints",
+      "Documentation frameworks optimized for large codebases",
+      "API documentation through strategic sampling only",
+      "MCP document summarizer integration for threshold management",
+      "Precise code referencing with line numbers without full retention",
+      "Sequential processing to prevent parallel memory accumulation",
+      "Forbidden practice enforcement (no parallel loads, no retention)"
     ],
     "best_practices": [
-      "Extract key patterns from 3-5 representative files maximum for documentation",
+      "ALWAYS check file size with LS before any Read operation",
+      "Extract key patterns from 3-5 representative files maximum",
       "Use grep with line numbers (-n) and adaptive context based on match count",
-      "Leverage MCP summarizer tool for files exceeding thresholds",
-      "Trigger summarization at 20KB or 200 lines for single files",
+      "Leverage MCP summarizer tool for ALL files exceeding thresholds",
+      "Trigger MANDATORY summarization at 20KB or 200 lines for single files",
       "Apply batch summarization after 3 files or 50KB cumulative content",
-      "Process files sequentially to prevent memory accumulation",
-      "Check file sizes before reading - auto-summarize >100KB files",
+      "Process files sequentially - NEVER in parallel",
+      "Auto-summarize >100KB files WITHOUT reading them",
+      "Skip or defer files >1MB entirely",
       "Reset cumulative counters after batch summarization",
-      "Extract and summarize patterns immediately, discard full file contents",
+      "Extract patterns and IMMEDIATELY discard full file contents",
+      "Use adaptive grep context: >50 matches (-A 2 -B 2 | head -50), <20 matches (-A 10 -B 10)",
+      "Process large files in <100 line chunks with immediate discard",
       "Create clear technical documentation with precise line references",
-      "Generate comprehensive API documentation from sampled patterns",
-      "Write user-friendly guides and tutorials",
-      "Maintain documentation consistency",
-      "Structure complex information effectively",
+      "Generate comprehensive API documentation from sampled patterns only",
+      "NEVER accumulate content across multiple file reads",
       "Always use grep -n for line number tracking in code references",
-      "Generate executive summaries when appropriate"
+      "Use targeted grep searches instead of full file reads",
+      "Implement progressive summarization for cumulative content management"
     ],
     "constraints": [
+      "❌ NEVER read entire large codebases",
+      "❌ NEVER load multiple files in parallel",
+      "❌ NEVER retain file contents after extraction",
+      "❌ NEVER load files >1MB into memory",
+      "❌ NEVER accumulate content across multiple file reads",
+      "❌ NEVER skip file size checks before reading",
+      "❌ NEVER process >5 files without summarization",
       "Process files sequentially to prevent memory accumulation",
       "Maximum 3-5 files for documentation analysis without summarization",
-      "Critical files >100KB must be summarized, never fully read",
-      "Single file threshold: 20KB or 200 lines triggers summarization",
+      "Critical files >100KB MUST be summarized, NEVER fully read",
+      "Single file threshold: 20KB or 200 lines triggers MANDATORY summarization",
       "Cumulative threshold: 50KB total or 3 files triggers batch summarization",
-      "Adaptive grep context: >50 matches use -A 2 -B 2 | head -50",
-      "Content must be discarded after extraction",
-      "Never retain full file contents in memory",
+      "Adaptive grep context: >50 matches use -A 2 -B 2 | head -50, <20 matches use -A 10 -B 10",
+      "Content MUST be discarded IMMEDIATELY after extraction",
+      "File size checking is MANDATORY before ALL Read operations",
       "Check MCP summarizer tool availability before use",
-      "Provide graceful fallback when MCP tool is not available",
       "Always include line numbers in code references",
-      "Sequential processing is mandatory for documentation generation"
+      "Implementation chunking: Process large files in <100 line segments",
+      "Sequential processing is MANDATORY for documentation generation"
     ],
     "examples": []
   },

claude_mpm/agents/templates/research.json

@@ -37,17 +37,6 @@
   },
   "capabilities": {
     "model": "opus",
-    "tools": [
-      "Read",
-      "Grep",
-      "Glob",
-      "LS",
-      "WebSearch",
-      "WebFetch",
-      "Bash",
-      "TodoWrite",
-      "mcp__claude-mpm-gateway__document_summarizer"
-    ],
     "resource_tier": "high",
     "temperature": 0.2,
     "max_tokens": 16384,

claude_mpm/cli/__init__.py

@@ -178,53 +178,90 @@ def _initialize_project_registry():
 
 def _verify_mcp_gateway_startup():
     """
-    Verify MCP Gateway configuration on startup.
+    Verify MCP Gateway configuration on startup and pre-warm MCP services.
 
     WHY: The MCP gateway should be automatically configured and verified on startup
     to provide a seamless experience with diagnostic tools, file summarizer, and
-    ticket service.
+    ticket service. Pre-warming MCP services eliminates the 11.9s delay on first use.
 
     DESIGN DECISION: This is non-blocking - failures are logged but don't prevent
     startup to ensure claude-mpm remains functional even if MCP gateway has issues.
     """
     try:
         import asyncio
+        import time
         from ..services.mcp_gateway.core.startup_verification import (
             verify_mcp_gateway_on_startup,
             is_mcp_gateway_configured,
         )
+        from ..services.mcp_gateway.core.process_pool import pre_warm_mcp_servers
+        from ..core.logger import get_logger
+        
+        logger = get_logger("mcp_prewarm")
 
         # Quick check first - if already configured, skip detailed verification
-        if is_mcp_gateway_configured():
-            return
-
-        # Run detailed verification in background
-        # Note: We don't await this to avoid blocking startup
-        def run_verification():
+        gateway_configured = is_mcp_gateway_configured()
+        
+        # Pre-warm MCP servers regardless of gateway config
+        # This eliminates the 11.9s delay on first agent invocation
+        def run_pre_warming():
             try:
+                start_time = time.time()
                 loop = asyncio.new_event_loop()
                 asyncio.set_event_loop(loop)
-                results = loop.run_until_complete(verify_mcp_gateway_on_startup())
+                
+                # Pre-warm MCP servers (especially vector search)
+                logger.info("Pre-warming MCP servers to eliminate startup delay...")
+                loop.run_until_complete(pre_warm_mcp_servers())
+                
+                pre_warm_time = time.time() - start_time
+                if pre_warm_time > 1.0:
+                    logger.info(f"MCP servers pre-warmed in {pre_warm_time:.2f}s")
+                
+                # Also run gateway verification if needed
+                if not gateway_configured:
+                    results = loop.run_until_complete(verify_mcp_gateway_on_startup())
+                
                 loop.close()
-
-                # Log results but don't block
-                from ..core.logger import get_logger
-                logger = get_logger("cli")
-
-                if results.get("gateway_configured"):
-                    logger.debug("MCP Gateway verification completed successfully")
-                else:
-                    logger.debug("MCP Gateway verification completed with warnings")
-
             except Exception as e:
-                from ..core.logger import get_logger
-                logger = get_logger("cli")
-                logger.debug(f"MCP Gateway verification failed: {e}")
-
-        # Run in background thread to avoid blocking startup
+                # Non-blocking - log but don't fail
+                logger.debug(f"MCP pre-warming error (non-critical): {e}")
+        
+        # Run pre-warming in background thread
         import threading
-        verification_thread = threading.Thread(target=run_verification, daemon=True)
-        verification_thread.start()
+        pre_warm_thread = threading.Thread(target=run_pre_warming, daemon=True)
+        pre_warm_thread.start()
+        
+        return
+
+        # Run detailed verification in background if not configured
+        if not gateway_configured:
+            # Note: We don't await this to avoid blocking startup
+            def run_verification():
+                try:
+                    loop = asyncio.new_event_loop()
+                    asyncio.set_event_loop(loop)
+                    results = loop.run_until_complete(verify_mcp_gateway_on_startup())
+                    loop.close()
+
+                    # Log results but don't block
+                    from ..core.logger import get_logger
+                    logger = get_logger("cli")
+
+                    if results.get("gateway_configured"):
+                        logger.debug("MCP Gateway verification completed successfully")
+                    else:
+                        logger.debug("MCP Gateway verification completed with warnings")
+
+                except Exception as e:
+                    from ..core.logger import get_logger
+                    logger = get_logger("cli")
+                    logger.debug(f"MCP Gateway verification failed: {e}")
+
+            # Run in background thread to avoid blocking startup
+            import threading
+            verification_thread = threading.Thread(target=run_verification, daemon=True)
+            verification_thread.start()
 
     except Exception as e:
         # Import logger here to avoid circular imports

claude_mpm/cli/commands/agent_manager.py

@@ -181,14 +181,15 @@ class AgentManagerCommand(AgentCommand):
         """Deploy an agent to specified tier."""
         try:
             agent_id = args.agent_id
-            tier = getattr(args, 'tier', 'user')
+            tier = getattr(args, 'tier', 'project')  # Default to project (changed from 'user')
             
-            # Determine deployment path
-            if tier == 'project':
-                deploy_path = Path.cwd() / ".claude" / "agents"
-            elif tier == 'user':
-                deploy_path = Path.home() / ".claude" / "agents"
-            else:
+            # Always deploy to project directory
+            # Regardless of tier, all agents go to project .claude/agents
+            deploy_path = Path.cwd() / ".claude" / "agents"
+            
+            # Note: We're keeping the tier parameter for backward compatibility
+            # but it no longer affects the deployment location
+            if tier not in ['project', 'user']:
                 return CommandResult.error_result("Invalid tier. Use 'project' or 'user'")
                 
             # Create directory if needed
@@ -203,7 +204,8 @@ class AgentManagerCommand(AgentCommand):
                 return CommandResult.error_result(f"Agent '{agent_id}' not found")
                 
             # Deploy using deployment service
-            self.deployment.deploy_agent(agent_id, str(deploy_path))
+            # Pass Path object, not string
+            self.deployment.deploy_agent(agent_id, deploy_path)
             
             return CommandResult.success_result(
                 f"Agent '{agent_id}' deployed to {tier} level"