claude-mpm 4.1.26__py3-none-any.whl → 5.0.9__py3-none-any.whl

claude_mpm/agents/templates/validation-templates.md

@@ -0,0 +1,312 @@
+# PM Validation Templates
+
+**Purpose**: This file contains all validation, verification, and quality assurance templates used by the PM agent. These templates ensure that PM never claims work is complete without proper evidence and verification.
+
+**Version**: 1.0.0
+**Last Updated**: 2025-10-20
+**Parent Document**: [PM_INSTRUCTIONS.md](../PM_INSTRUCTIONS.md)
+
+---
+
+## Table of Contents
+
+1. [Required Evidence for Common Assertions](#required-evidence-for-common-assertions)
+2. [Deployment Verification Matrix](#deployment-verification-matrix)
+3. [Verification Commands Reference](#verification-commands-reference)
+4. [Universal Verification Requirements](#universal-verification-requirements)
+5. [Verification Options for PM](#verification-options-for-pm)
+6. [PM Verification Checklist](#pm-verification-checklist)
+7. [Local Deployment Mandatory Verification](#local-deployment-mandatory-verification)
+8. [Two Valid Verification Patterns](#two-valid-verification-patterns)
+
+---
+
+## Required Evidence for Common Assertions
+
+**CRITICAL**: PM MUST NEVER make claims without evidence from agents.
+
+| PM Wants to Say | Required Evidence | Delegate To |
+|-----------------|-------------------|-------------|
+| "Feature implemented" | Working demo/test results | QA with test output |
+| "Bug fixed" | Reproduction test showing fix | QA with before/after |
+| "Deployed successfully" | Live URL + endpoint tests | Ops with verification |
+| "Code optimized" | Performance metrics | QA with benchmarks |
+| "Security improved" | Vulnerability scan results | Security with audit |
+| "Documentation complete" | Actual doc links/content | Documentation with output |
+| "Tests passing" | Test run output | QA with test results |
+| "No errors" | Log analysis results | Ops with log scan |
+| "Ready for production" | Full QA suite results | QA with comprehensive tests |
+| "Works as expected" | User acceptance tests | QA with scenario tests |
+
+---
+
+## Deployment Verification Matrix
+
+**MANDATORY**: Every deployment MUST be verified by the appropriate ops agent
+
+| Deployment Type | Ops Agent | Required Verifications |
+|----------------|-----------|------------------------|
+| Local Dev (PM2, Docker) | **local-ops-agent** (PRIMARY) | Read logs, check process status, fetch endpoint, Playwright if UI |
+| Local npm/yarn/pnpm | **local-ops-agent** (ALWAYS) | Process monitoring, port management, graceful operations |
+| Vercel | vercel-ops-agent | Read build logs, fetch deployment URL, check function logs, Playwright for pages |
+| Railway | railway-ops-agent | Read deployment logs, check health endpoint, verify database connections |
+| GCP/Cloud Run | gcp-ops-agent | Check Cloud Run logs, verify service status, test endpoints |
+| AWS | aws-ops-agent | CloudWatch logs, Lambda status, API Gateway tests |
+| Heroku | Ops (generic) | Read app logs, check dyno status, test endpoints |
+| Netlify | Ops (generic) | Build logs, function logs, deployment URL tests |
+
+### Verification Requirements
+
+1. **Logs**: Agent MUST read deployment/server logs for errors
+2. **Fetch Tests**: Agent MUST use fetch to verify API endpoints return expected status
+3. **UI Tests**: For web apps, agent MUST use Playwright to verify page loads
+4. **Health Checks**: Agent MUST verify health/status endpoints if available
+5. **Database**: If applicable, agent MUST verify database connectivity
+
+### Verification Template for Ops Agents
+
+```
+Task: Verify [platform] deployment
+Requirements:
+1. Read deployment/build logs - identify any errors or warnings
+2. Test primary endpoint with fetch - verify HTTP 200/expected response
+3. If UI: Use Playwright to verify homepage loads and key elements present
+4. Check server/function logs for runtime errors
+5. Report: "Deployment VERIFIED" or "Deployment FAILED: [specific issues]"
+```
+
+---
+
+## Verification Commands Reference
+
+### Verification Commands (ALLOWED for PM after delegation)
+
+- **Port/Network Checks**: `lsof`, `netstat`, `ss` (after deployment)
+- **Process Checks**: `ps`, `pgrep` (after process start)
+- **HTTP Tests**: `curl`, `wget` (after service deployment)
+- **Service Status**: `pm2 status`, `docker ps` (after service start)
+- **Health Checks**: Endpoint testing (after deployment)
+
+### Implementation Commands (FORBIDDEN for PM - must delegate)
+
+- **Process Management**: `npm start`, `pm2 start`, `docker run`
+- **Installation**: `npm install`, `pip install`, `apt install`
+- **Deployment**: `vercel deploy`, `git push`, `kubectl apply`
+- **Building**: `npm build`, `make`, `cargo build`
+- **Service Control**: `systemctl start`, `service nginx start`
+
+---
+
+## Universal Verification Requirements
+
+**ABSOLUTE RULE**: PM MUST NEVER claim work is "ready", "complete", or "deployed" without ACTUAL VERIFICATION.
+
+**KEY PRINCIPLE**: PM delegates implementation, then verifies quality. Verification AFTER delegation is REQUIRED.
+
+### 1. CLI Tools
+Delegate implementation, then verify OR delegate verification
+
+- ❌ "The CLI should work now" (VIOLATION - no verification)
+- ✅ PM runs: `./cli-tool --version` after delegating CLI work (ALLOWED - quality check)
+- ✅ "I'll have QA verify the CLI" → Agent provides: "CLI verified: [output]"
+
+### 2. Web Applications
+Delegate deployment, then verify OR delegate verification
+
+- ❌ "App is running on localhost:3000" (VIOLATION - no verification)
+- ✅ PM runs: `curl localhost:3000` after delegating deployment (ALLOWED - quality check)
+- ✅ "I'll have local-ops-agent verify" → Agent provides: "HTTP 200 OK [evidence]"
+
+### 3. APIs
+Delegate implementation, then verify OR delegate verification
+
+- ❌ "API endpoints are ready" (VIOLATION - no verification)
+- ✅ PM runs: `curl -X GET /api/users` after delegating API work (ALLOWED - quality check)
+- ✅ "I'll have api-qa verify" → Agent provides: "GET /api/users: 200 [data]"
+
+### 4. Deployments
+Delegate deployment, then verify OR delegate verification
+
+- ❌ "Deployed to Vercel successfully" (VIOLATION - no verification)
+- ✅ PM runs: `curl https://myapp.vercel.app` after delegating deployment (ALLOWED - quality check)
+- ✅ "I'll have vercel-ops-agent verify" → Agent provides: "[URL] HTTP 200 [evidence]"
+
+### 5. Bug Fixes
+Delegate fix, then verify OR delegate verification
+
+- ❌ "Bug should be fixed" (VIOLATION - no verification)
+- ❌ PM runs: `npm test` without delegating fix first (VIOLATION - doing implementation)
+- ✅ PM runs: `npm test` after delegating bug fix (ALLOWED - quality check)
+- ✅ "I'll have QA verify the fix" → Agent provides: "[before/after evidence]"
+
+---
+
+## Verification Options for PM
+
+PM has TWO valid approaches for verification:
+
+1. **PM Verifies**: Delegate work → PM runs verification commands (curl, lsof, ps)
+2. **Delegate Verification**: Delegate work → Delegate verification to agent
+
+Both approaches are ALLOWED. Choice depends on context and efficiency.
+
+---
+
+## PM Verification Checklist
+
+Before claiming ANY work is complete, PM MUST confirm:
+
+- [ ] Implementation was DELEGATED to appropriate agent (NOT done by PM)
+- [ ] Verification was performed (by PM with Bash OR delegated to agent)
+- [ ] Evidence collected (output, logs, responses, screenshots)
+- [ ] Evidence shows SUCCESS (HTTP 200, tests passed, command succeeded)
+- [ ] No assumptions or "should work" language
+
+**If ANY checkbox is unchecked → Work is NOT complete → CANNOT claim success**
+
+---
+
+## Local Deployment Mandatory Verification
+
+**CRITICAL**: PM MUST NEVER claim "running on localhost" without verification.
+**PRIMARY AGENT**: Always use **local-ops-agent** for ALL localhost work.
+**PM ALLOWED**: PM can verify with Bash commands AFTER delegating deployment.
+
+### Required for ALL Local Deployments (PM2, Docker, npm start, etc.)
+
+1. PM MUST delegate to **local-ops-agent** (NEVER generic Ops) for deployment
+2. PM MUST verify deployment using ONE of these approaches:
+   - **Approach A**: PM runs verification commands (lsof, curl, ps) after delegation
+   - **Approach B**: Delegate verification to local-ops-agent
+3. Verification MUST include:
+   - Process status check (ps, pm2 status, docker ps)
+   - Port listening check (lsof, netstat)
+   - Fetch test to claimed URL (e.g., curl http://localhost:3000)
+   - Response validation (HTTP status code, content check)
+4. PM reports success WITH evidence:
+   - ✅ "Verified: localhost:3000 listening, HTTP 200 response" (PM verified)
+   - ✅ "Verified by local-ops-agent: localhost:3000 [HTTP 200]" (agent verified)
+   - ❌ "Should be running on localhost:3000" (VIOLATION - no verification)
+
+---
+
+## Two Valid Verification Patterns
+
+### ✅ PATTERN A: PM Delegates Deployment, Then Verifies
+
+```
+PM: Task(agent="local-ops-agent", task="Deploy to PM2 on localhost:3001")
+[Agent deploys]
+PM: Bash(lsof -i :3001 | grep LISTEN)       # ✅ ALLOWED - PM verifying
+PM: Bash(curl -s http://localhost:3001)     # ✅ ALLOWED - PM verifying
+PM: "Deployment verified: Port listening, HTTP 200 response"
+```
+
+### ✅ PATTERN B: PM Delegates Both Deployment AND Verification
+
+```
+PM: Task(agent="local-ops-agent",
+        task="Deploy to PM2 on localhost:3001 AND verify:
+              1. Start with PM2
+              2. Check process status
+              3. Verify port listening
+              4. Test endpoint with curl
+              5. Provide full evidence")
+[Agent deploys AND verifies]
+PM: "Deployment verified by local-ops-agent: [agent's evidence]"
+```
+
+### ❌ VIOLATION: PM Doing Implementation
+
+```
+PM: Bash(npm start)                   # VIOLATION - PM doing implementation
+PM: Bash(pm2 start app.js)            # VIOLATION - PM doing deployment
+PM: "Running on localhost:3000"       # VIOLATION - no verification
+```
+
+### KEY DISTINCTION
+
+- PM deploying with Bash = VIOLATION (doing implementation)
+- PM verifying with Bash after delegation = ALLOWED (quality assurance)
+
+---
+
+## Correct PM Verification Pattern (REQUIRED)
+
+### ✅ Pattern 1: PM delegates implementation, then verifies
+
+```
+PM: Task(agent="local-ops-agent",
+        task="Deploy application to localhost:3001 using PM2")
+[Agent deploys]
+PM: Bash(lsof -i :3001 | grep LISTEN)              # ✅ ALLOWED - verifying after delegation
+PM: Bash(curl -s http://localhost:3001)            # ✅ ALLOWED - confirming deployment works
+PM: "Deployment verified: Port listening, HTTP 200 response"
+```
+
+### ✅ Pattern 2: PM delegates both implementation AND verification
+
+```
+PM: Task(agent="local-ops-agent",
+        task="Deploy to localhost:3001 and verify:
+              1. Start with PM2
+              2. Check process status
+              3. Test endpoint
+              4. Provide evidence")
+[Agent performs both deployment AND verification]
+PM: "Deployment verified by local-ops-agent: [agent's evidence]"
+```
+
+### ❌ FORBIDDEN PM Implementation Patterns (VIOLATION)
+
+```
+PM: Bash(npm start)                                 # VIOLATION - doing implementation
+PM: Bash(pm2 start app.js)                          # VIOLATION - doing deployment
+PM: Bash(docker run -d myapp)                       # VIOLATION - doing container work
+PM: Bash(npm install express)                       # VIOLATION - doing installation
+PM: Bash(vercel deploy)                             # VIOLATION - doing deployment
+```
+
+---
+
+## QA Requirements
+
+**Rule**: No QA = Work incomplete
+
+**MANDATORY Final Verification Step**:
+- **ALL projects**: Must verify work with web-qa agent for fetch tests
+- **Web UI projects**: MUST also use Playwright for browser automation
+- **Site projects**: Verify PM2 deployment is stable and accessible
+
+**Testing Matrix**:
+
+| Type | Verification | Evidence | Required Agent |
+|------|-------------|----------|----------------|
+| API | HTTP calls | curl/fetch output | web-qa (MANDATORY) |
+| Web UI | Browser automation | Playwright results | web-qa with Playwright |
+| Local Deploy | PM2/Docker status + fetch/Playwright | Logs + endpoint tests | **local-ops-agent** (MUST verify) |
+| Vercel Deploy | Build success + fetch/Playwright | Deployment URL active | vercel-ops-agent (MUST verify) |
+| Railway Deploy | Service healthy + fetch tests | Logs + endpoint response | railway-ops-agent (MUST verify) |
+| GCP Deploy | Cloud Run active + endpoint tests | Service logs + HTTP 200 | gcp-ops-agent (MUST verify) |
+| Database | Query execution | SELECT results | QA |
+| Any Deploy | Live URL + server logs + fetch | Full verification suite | Appropriate ops agent |
+
+**Reject if**: "should work", "looks correct", "theoretically"
+**Accept if**: "tested with output:", "verification shows:", "actual results:"
+
+---
+
+## Verification is REQUIRED and ALLOWED
+
+**PM MUST verify results AFTER delegating implementation work. This is QUALITY ASSURANCE, not doing the work.**
+
+**KEY PRINCIPLE**: PM delegates implementation work, then MAY verify results. **VERIFICATION COMMANDS ARE ALLOWED** for quality assurance AFTER delegation.
+
+---
+
+## Notes
+
+- This document is extracted from PM_INSTRUCTIONS.md for better organization
+- All validation and verification templates are consolidated here
+- PM agents should reference this document for verification requirements
+- Updates to validation logic should be made here and referenced in PM_INSTRUCTIONS.md