qa360 2.2.1 → 2.2.7

package/docs/mcp-tools.md

@@ -1,1131 +0,0 @@
-# QA360 MCP Tools Reference
-
-## Overview
-
-This document provides detailed specifications for all QA360 MCP tools, including argument schemas, return types, error codes, and usage examples.
-
-## Tool Categories
-
-- [Core Execution](#core-execution-tools)
-- [History & Analytics](#history--analytics-tools)
-- [Pack Management](#pack-management-tools)
-- [Secrets Management](#secrets-management-tools)
-- [Server & Vault](#server--vault-tools)
-- [Triage & Support](#triage--support-tools)
-
----
-
-## Core Execution Tools
-
-### qa360.run
-
-Execute a QA360 pack and return comprehensive results.
-
-**Permission Required:** `--allow-run`
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "packPath": {
-      "type": "string",
-      "description": "Path to the QA360 pack file",
-      "pattern": "^[^\\0]+\\.(yml|yaml)$"
-    },
-    "runKey": {
-      "type": "string",
-      "description": "Unique key for idempotent runs",
-      "pattern": "^[a-zA-Z0-9:._-]+$",
-      "maxLength": 100
-    },
-    "strict": {
-      "type": "boolean",
-      "description": "Enable strict mode validation",
-      "default": true
-    },
-    "weights": {
-      "type": "object",
-      "description": "Custom weights for trust score calculation",
-      "additionalProperties": {
-        "type": "number",
-        "minimum": 0,
-        "maximum": 1
-      }
-    }
-  },
-  "required": ["packPath"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "runId": "string",
-  "status": "passed|failed|cancelled|error",
-  "trust": {
-    "score": "number (0-100)",
-    "weights": "object"
-  },
-  "proof": {
-    "pdf": {
-      "sha256": "string",
-      "path": "string"
-    }
-  },
-  "gates": [
-    {
-      "name": "string",
-      "status": "string",
-      "duration_ms": "number",
-      "key_metrics": "object"
-    }
-  ]
-}
-```
-
-**Example Usage:**
-```json
-{
-  "packPath": "./security-pack.yml",
-  "runKey": "ci:security:v1.2.0",
-  "strict": true,
-  "weights": {
-    "security": 0.4,
-    "performance": 0.3,
-    "ui": 0.3
-  }
-}
-```
-
-**Error Codes:**
-- `QA360_PERMISSION_DENIED`: Requires --allow-run flag
-- `QA360_PACK_NOT_FOUND`: Pack file doesn't exist
-- `QA360_PACK_INVALID`: Pack validation failed
-- `QA360_EXECUTION_FAILED`: Test execution error
-
----
-
-### qa360.report.sign
-
-Sign a test report with Ed25519 cryptographic signature.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "runId": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    }
-  },
-  "required": ["runId"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "signature": "string",
-  "algorithm": "ed25519",
-  "runId": "string"
-}
-```
-
-**Error Codes:**
-- `QA360_RUN_NOT_FOUND`: Run ID doesn't exist
-- `QA360_ALREADY_SIGNED`: Run already has signature
-- `QA360_SIGNING_FAILED`: Cryptographic signing error
-
----
-
-### qa360.verify
-
-Verify proof integrity and cryptographic signature.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "runId": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    },
-    "proofPath": {
-      "type": "string",
-      "description": "Path to proof file"
-    }
-  }
-}
-```
-
-**Return Type:**
-```json
-{
-  "verified": "boolean",
-  "algorithm": "string",
-  "message": "string",
-  "runId": "string"
-}
-```
-
-**Error Codes:**
-- `QA360_INVALID_ARGUMENTS`: Neither runId nor proofPath provided
-- `QA360_RUN_NOT_FOUND`: Run doesn't exist
-- `QA360_PROOF_NOT_FOUND`: Proof file doesn't exist
-- `QA360_VERIFICATION_FAILED`: Signature verification failed
-
----
-
-## History & Analytics Tools
-
-### qa360.history.list
-
-List test runs with comprehensive filtering options.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "limit": {
-      "type": "integer",
-      "minimum": 1,
-      "maximum": 100,
-      "default": 20
-    },
-    "status": {
-      "type": "string",
-      "enum": ["passed", "failed", "error", "cancelled", "any"],
-      "default": "any"
-    },
-    "gate": {
-      "type": "string",
-      "pattern": "^[a-zA-Z0-9_-]+$"
-    },
-    "since": {
-      "type": "string",
-      "format": "date-time"
-    }
-  }
-}
-```
-
-**Return Type:**
-```json
-{
-  "runs": [
-    {
-      "id": "string",
-      "started_at": "number",
-      "status": "string",
-      "trust_score": "number",
-      "gates_summary": "object"
-    }
-  ]
-}
-```
-
-**Example Usage:**
-```json
-{
-  "limit": 10,
-  "status": "failed",
-  "gate": "security",
-  "since": "2025-01-01T00:00:00Z"
-}
-```
-
----
-
-### qa360.history.get
-
-Get detailed information about a specific test run.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "runId": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    }
-  },
-  "required": ["runId"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "run": {
-    "id": "string",
-    "started_at": "number",
-    "ended_at": "number",
-    "status": "string",
-    "trust_score": "number",
-    "pack_path": "string",
-    "pack_hash": "string",
-    "run_key": "string"
-  },
-  "gates": [
-    {
-      "name": "string",
-      "status": "string",
-      "duration_ms": "number",
-      "metrics": "object"
-    }
-  ],
-  "findings": [
-    {
-      "gate": "string",
-      "severity": "string",
-      "rule": "string",
-      "location": "string",
-      "message": "string"
-    }
-  ],
-  "artifacts": [
-    {
-      "label": "string",
-      "sha256": "string",
-      "mime_type": "string",
-      "size_bytes": "number"
-    }
-  ]
-}
-```
-
----
-
-### qa360.history.diff
-
-Compare two test runs to identify changes and regressions.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "runA": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    },
-    "runB": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    }
-  },
-  "required": ["runA", "runB"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "delta": {
-    "trust": "number",
-    "gates": {
-      "gate_name": {
-        "status_change": "string",
-        "duration_change": "number"
-      }
-    },
-    "findings": {
-      "added": ["array"],
-      "removed": ["array"],
-      "regressed": ["array"]
-    }
-  }
-}
-```
-
----
-
-### qa360.history.trend
-
-Analyze trends over time for performance and quality metrics.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "gate": {
-      "type": "string",
-      "enum": ["perf", "ui", "api", "a11y", "sast", "dast", "deps", "secrets"]
-    },
-    "window": {
-      "type": "integer",
-      "minimum": 5,
-      "maximum": 200,
-      "default": 20
-    }
-  }
-}
-```
-
-**Return Type:**
-```json
-{
-  "series": [
-    {
-      "timestamp": "number",
-      "value": "number",
-      "metadata": "object"
-    }
-  ],
-  "summary": {
-    "average": "number",
-    "trend": "up|down|stable",
-    "window_size": "number"
-  }
-}
-```
-
----
-
-### qa360.history.export
-
-Export a complete run bundle with all artifacts and metadata.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "runId": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    },
-    "bundlePath": {
-      "type": "string",
-      "pattern": "^[^\\0]+\\.zip$"
-    }
-  },
-  "required": ["runId", "bundlePath"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "bundlePath": "string",
-  "size_bytes": "number",
-  "contents": ["array of strings"]
-}
-```
-
----
-
-### qa360.history.gc
-
-Garbage collection to manage vault size and remove orphaned data.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "keepLast": {
-      "type": "integer",
-      "minimum": 1,
-      "maximum": 1000,
-      "default": 50
-    },
-    "maxBytes": {
-      "type": "string",
-      "pattern": "^\\d+(\\.\\d+)?(B|KB|MB|GB|TB)$"
-    },
-    "dryRun": {
-      "type": "boolean",
-      "default": false
-    }
-  }
-}
-```
-
-**Return Type:**
-```json
-{
-  "removed": {
-    "runs": "number",
-    "artifacts": "number",
-    "bytesFreed": "number"
-  },
-  "dryRun": "boolean"
-}
-```
-
----
-
-### qa360.history.pin / qa360.history.unpin
-
-Pin or unpin runs to protect them from garbage collection.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "runId": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    }
-  },
-  "required": ["runId"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "success": "boolean",
-  "runId": "string",
-  "pinned": "boolean",
-  "message": "string"
-}
-```
-
----
-
-## Pack Management Tools
-
-### qa360.pack.ask
-
-Generate a QA360 pack from natural language query using AI.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "query": {
-      "type": "string",
-      "minLength": 10,
-      "maxLength": 1000
-    },
-    "context": {
-      "type": "object",
-      "properties": {
-        "targets": "object"
-      }
-    }
-  },
-  "required": ["query"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "pack": "string (YAML)",
-  "hints": {
-    "tools_needed": ["array of strings"],
-    "estimated_duration": "string",
-    "complexity": "simple|medium|complex"
-  }
-}
-```
-
-**Example Usage:**
-```json
-{
-  "query": "Test my e-commerce checkout flow for security vulnerabilities and performance under load",
-  "context": {
-    "targets": {
-      "url": "https://shop.example.com",
-      "checkout_url": "https://shop.example.com/checkout"
-    }
-  }
-}
-```
-
----
-
-### qa360.pack.validate
-
-Validate a QA360 pack configuration against the schema.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "pack": {
-      "type": "string",
-      "description": "YAML pack content"
-    },
-    "path": {
-      "type": "string",
-      "description": "Path to pack file"
-    }
-  }
-}
-```
-
-**Return Type:**
-```json
-{
-  "valid": "boolean",
-  "errors": [
-    {
-      "code": "string",
-      "message": "string",
-      "line": "number",
-      "column": "number"
-    }
-  ],
-  "warnings": [
-    {
-      "code": "string",
-      "message": "string",
-      "line": "number",
-      "column": "number"
-    }
-  ]
-}
-```
-
-**Validation Error Codes:**
-- `PACK_EMPTY`: Pack file is empty or invalid YAML
-- `PACK_NO_NAME`: Missing required name field
-- `PACK_NO_VERSION`: Missing required version field
-- `PACK_NO_GATES`: Missing or invalid gates configuration
-- `GATE_INVALID`: Gate configuration is not an object
-- `GATE_NO_ADAPTER`: Gate missing adapter specification
-- `GATE_UNKNOWN_ADAPTER`: Unknown adapter type
-- `BUDGET_INVALID_DURATION`: Invalid budget duration value
-
----
-
-### qa360.pack.lint
-
-Lint a QA360 pack and suggest improvements and best practices.
-
-**Arguments Schema:** Same as `qa360.pack.validate`
-
-**Return Type:** Same as `qa360.pack.validate` plus:
-```json
-{
-  "auto_fixes": ["array of suggested fixes"]
-}
-```
-
-**Lint Warning Codes:**
-- `PACK_NO_DESCRIPTION`: Consider adding description field
-- `PACK_NO_METADATA`: Consider adding metadata section
-- `PACK_TOO_MANY_GATES`: Too many gates, consider splitting
-- `GATE_NAME_SPACES`: Gate names should use underscores/hyphens
-
----
-
-### qa360.pack.migrate
-
-Migrate a QA360 pack to the latest version format.
-
-**Arguments Schema:** Same as `qa360.pack.validate`
-
-**Return Type:** Same as `qa360.pack.lint`
-
-**Migration Actions:**
-- Update deprecated adapter names
-- Add missing required fields
-- Modernize configuration format
-- Apply current best practices
-
----
-
-## Secrets Management Tools
-
-**All secrets tools require `--allow-secrets` flag**
-
-### qa360.secrets.add
-
-Add a secret to the encrypted vault.
-
-**Permission Required:** `--allow-secrets`
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "name": {
-      "type": "string",
-      "pattern": "^[a-zA-Z0-9_-]+$",
-      "minLength": 1,
-      "maxLength": 100
-    },
-    "value": {
-      "type": "string",
-      "minLength": 1,
-      "maxLength": 10000
-    },
-    "description": {
-      "type": "string",
-      "maxLength": 500
-    }
-  },
-  "required": ["name", "value"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "success": "boolean",
-  "name": "string",
-  "message": "string",
-  "created_at": "number"
-}
-```
-
----
-
-### qa360.secrets.list
-
-List all secrets with names and metadata (values redacted).
-
-**Permission Required:** `--allow-secrets`
-
-**Return Type:**
-```json
-{
-  "secrets": [
-    {
-      "name": "string",
-      "description": "string",
-      "created_at": "number",
-      "last_used": "number"
-    }
-  ]
-}
-```
-
----
-
-### qa360.secrets.remove
-
-Remove a secret from the vault.
-
-**Permission Required:** `--allow-secrets`
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "name": {
-      "type": "string",
-      "pattern": "^[a-zA-Z0-9_-]+$"
-    }
-  },
-  "required": ["name"]
-}
-```
-
----
-
-### qa360.secrets.doctor
-
-Check secrets vault health and provide recommendations.
-
-**Permission Required:** `--allow-secrets`
-
-**Return Type:**
-```json
-{
-  "health": "healthy|warning|error",
-  "secret_count": "number",
-  "issues": ["array of issues"],
-  "recommendations": ["array of recommendations"],
-  "vault_location": "string",
-  "encryption": "string"
-}
-```
-
----
-
-## Server & Vault Tools
-
-### qa360.serve.health
-
-Check QA360 server health status.
-
-**Return Type:**
-```json
-{
-  "status": "healthy|degraded|unhealthy",
-  "server": "running|not_running",
-  "url": "string",
-  "version": "string",
-  "uptime": "number"
-}
-```
-
----
-
-### qa360.serve.diag
-
-Get detailed server diagnostics and system information.
-
-**Return Type:**
-```json
-{
-  "health": "healthy|degraded|unhealthy",
-  "version": "string",
-  "uptime_ms": "number",
-  "metrics": {
-    "memory": "object",
-    "process": "object",
-    "mcp": "object"
-  },
-  "vault_stats": "object"
-}
-```
-
----
-
-### qa360.serve.metrics
-
-Get server metrics and performance data.
-
-**Return Type:**
-```json
-{
-  "timestamp": "number",
-  "server_type": "string",
-  "uptime_ms": "number",
-  "memory_usage": "object",
-  "mcp_metrics": {
-    "tools_registered": "number",
-    "total_calls": "number",
-    "success_rate": "number",
-    "top_tools": "array"
-  },
-  "security": "object"
-}
-```
-
----
-
-### qa360.vault.get_run
-
-Get run data from the evidence vault (read-only).
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "runId": {
-      "type": "string",
-      "pattern": "^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$"
-    }
-  },
-  "required": ["runId"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "run": "object (run details)",
-  "gates": "array",
-  "findings": "array (redacted)",
-  "metadata": {
-    "total_gates": "number",
-    "total_findings": "number",
-    "passed_gates": "number",
-    "failed_gates": "number",
-    "critical_findings": "number",
-    "high_findings": "number"
-  }
-}
-```
-
----
-
-### qa360.vault.get_artifacts
-
-Get artifacts metadata for a run (content not included for security).
-
-**Arguments Schema:** Same as `qa360.vault.get_run`
-
-**Return Type:**
-```json
-{
-  "run_id": "string",
-  "total_artifacts": "number",
-  "artifacts": [
-    {
-      "label": "string",
-      "sha256": "string",
-      "mime_type": "string",
-      "size_bytes": "number",
-      "original_name": "string",
-      "cas_path": "string",
-      "created_at": "number",
-      "access_note": "string"
-    }
-  ],
-  "summary": {
-    "total_size_bytes": "number",
-    "mime_types": "array",
-    "labels": "array",
-    "oldest_artifact": "number",
-    "newest_artifact": "number"
-  }
-}
-```
-
----
-
-## Triage & Support Tools
-
-### qa360.triage.explain_code
-
-Explain error codes and provide remediation guidance.
-
-**Arguments Schema:**
-```json
-{
-  "type": "object",
-  "properties": {
-    "code": {
-      "type": "string",
-      "pattern": "^[A-Z]{2,4}\\d{3,4}$",
-      "description": "Error code (e.g., SG001, ZAP002)"
-    },
-    "context": {
-      "type": "object",
-      "description": "Additional context for explanation"
-    }
-  },
-  "required": ["code"]
-}
-```
-
-**Return Type:**
-```json
-{
-  "explanation": {
-    "cause": "string",
-    "severity": "low|medium|high|critical",
-    "category": "string"
-  },
-  "actions": [
-    {
-      "type": "fix|investigate|ignore",
-      "description": "string",
-      "command": "string"
-    }
-  ],
-  "references": [
-    {
-      "title": "string",
-      "url": "string"
-    }
-  ]
-}
-```
-
-**Supported Error Code Prefixes:**
-- `SG`: Semgrep (Static Analysis Security Testing)
-- `ZAP`: OWASP ZAP (Dynamic Application Security Testing)
-- `GL`: Gitleaks (Secret Detection)
-- `OSV`: OSV Scanner (Dependency Vulnerabilities)
-- `PA`: Playwright (UI Testing)
-- `K6`/`PU`: K6/Performance (Load Testing)
-
-**Example Usage:**
-```json
-{
-  "code": "SG001",
-  "context": {
-    "file": "/src/config.js",
-    "line": 42,
-    "rule": "hardcoded-secret"
-  }
-}
-```
-
----
-
-## Common Patterns
-
-### Error Handling
-
-All tools return errors in this standardized format:
-
-```json
-{
-  "error": {
-    "code": "QA360_ERROR_CODE",
-    "message": "Human readable description",
-    "hint": "Suggested resolution"
-  }
-}
-```
-
-### Pagination
-
-Tools that return lists support pagination:
-
-```json
-{
-  "limit": 20,
-  "offset": 0,
-  "total": 150,
-  "has_more": true
-}
-```
-
-### Filtering
-
-Many tools support filtering with these common patterns:
-
-```json
-{
-  "status": "passed|failed|error|cancelled|any",
-  "since": "ISO 8601 date string",
-  "until": "ISO 8601 date string",
-  "gate": "gate name",
-  "severity": "low|medium|high|critical"
-}
-```
-
-### Redaction
-
-Sensitive data is automatically redacted in responses:
-- Signatures → `***REDACTED***`
-- Secrets → `***`
-- Tokens → `Bearer ***`
-- Passwords → `***`
-
----
-
-## Rate Limiting
-
-All tools are subject to rate limiting:
-
-- Default: 5 requests per second
-- Configurable via `--rate-limit` flag
-- Heavy operations (run, export, gc) have additional concurrency limits
-- Rate limit headers included in responses
-
----
-
-## Security Considerations
-
-### Input Validation
-
-- All arguments validated against JSON Schema
-- Path traversal prevention
-- SQL injection protection
-- Command injection prevention
-
-### Output Sanitization
-
-- Automatic PII redaction
-- Sensitive field masking
-- Safe error messages
-- Audit trail logging
-
-### Access Control
-
-- Tool-level permissions (`--allow-run`, `--allow-secrets`)
-- Path sandboxing
-- Concurrency limiting
-- Rate limiting
-
----
-
-## Performance Notes
-
-### Heavy Operations
-
-These tools may take longer and consume more resources:
-- `qa360.run`: Executes full test suite
-- `qa360.history.export`: Creates ZIP bundles
-- `qa360.history.gc`: Processes large datasets
-- `qa360.pack.ask`: AI-powered generation
-
-### Optimization Tips
-
-- Use `limit` parameters to reduce response size
-- Implement client-side caching for static data
-- Use `dryRun` for preview operations
-- Pin important runs before garbage collection
-
----
-
-## Integration Examples
-
-### Batch Operations
-
-```javascript
-// Get recent failed runs and explain their errors
-const failedRuns = await mcp.call('qa360.history.list', {
-  status: 'failed',
-  limit: 5
-});
-
-for (const run of failedRuns.runs) {
-  const details = await mcp.call('qa360.history.get', {
-    runId: run.id
-  });
-  
-  for (const finding of details.findings) {
-    if (finding.rule.match(/^[A-Z]{2,4}\d{3,4}$/)) {
-      const explanation = await mcp.call('qa360.triage.explain_code', {
-        code: finding.rule,
-        context: { location: finding.location }
-      });
-      console.log(`${finding.rule}: ${explanation.explanation.cause}`);
-    }
-  }
-}
-```
-
-### Workflow Automation
-
-```javascript
-// Complete testing workflow
-async function runQualityCheck(packPath, runKey) {
-  // 1. Validate pack
-  const validation = await mcp.call('qa360.pack.validate', { 
-    path: packPath 
-  });
-  
-  if (!validation.valid) {
-    throw new Error(`Pack validation failed: ${validation.errors[0].message}`);
-  }
-  
-  // 2. Run tests
-  const result = await mcp.call('qa360.run', { 
-    packPath, 
-    runKey,
-    strict: true 
-  });
-  
-  // 3. Sign results
-  await mcp.call('qa360.report.sign', { 
-    runId: result.runId 
-  });
-  
-  // 4. Export for archival
-  await mcp.call('qa360.history.export', {
-    runId: result.runId,
-    bundlePath: `./reports/${runKey}.zip`
-  });
-  
-  return result;
-}
-```
-
-This comprehensive reference covers all available MCP tools with their complete specifications, usage patterns, and integration examples.