qa360 2.2.1 → 2.2.13

package/docs/mcp-server.md

@@ -1,651 +0,0 @@
-# QA360 MCP Server
-
-## Overview
-
-The QA360 MCP (Model Context Protocol) Server enables seamless integration between QA360 and LLMs/IDEs. It exposes QA360's testing capabilities through a standardized protocol that can be consumed by Claude, VSCode, and other MCP-compatible clients.
-
-## Installation
-
-### Prerequisites
-
-- Node.js 18+ 
-- QA360 Core installed and configured
-- MCP-compatible client (Claude Desktop, VSCode with MCP extension, etc.)
-
-### Install from NPM
-
-```bash
-npm install -g @qa360/mcp-server
-```
-
-### Build from Source
-
-```bash
-git clone https://github.com/qa360/qa360-core
-cd qa360-core/packages/mcp-server-qa360
-npm install
-npm run build
-npm link
-```
-
-## Quick Start
-
-### 1. Start the MCP Server
-
-```bash
-# Basic usage (read-only tools)
-qa360-mcp
-
-# Enable all tools (requires explicit opt-in)
-qa360-mcp --allow-run --allow-secrets
-
-# Custom configuration
-qa360-mcp --allow-run --max-concurrency 5 --rate-limit 10
-```
-
-### 2. Configure Your MCP Client
-
-#### Claude Desktop
-
-Add to `~/.config/claude-desktop/claude_desktop_config.json`:
-
-```json
-{
-  "mcpServers": {
-    "qa360": {
-      "command": "qa360-mcp",
-      "args": ["--allow-run"],
-      "env": {
-        "QA360_LOG_LEVEL": "info"
-      }
-    }
-  }
-}
-```
-
-#### VSCode with MCP Extension
-
-Add to VSCode settings:
-
-```json
-{
-  "mcp.servers": [
-    {
-      "name": "qa360",
-      "command": "qa360-mcp",
-      "args": ["--allow-run", "--allow-secrets"],
-      "cwd": "${workspaceFolder}"
-    }
-  ]
-}
-```
-
-### 3. Test the Connection
-
-In your MCP client, try:
-
-```
-List my recent QA360 test runs
-```
-
-or
-
-```
-Generate a QA360 pack to test my web application for security issues
-```
-
-## Available Tools
-
-### Core Execution Tools
-
-#### `qa360.run`
-Execute a QA360 pack and return results.
-
-**Arguments:**
-- `packPath` (required): Path to pack file
-- `runKey` (optional): Unique key for idempotent runs
-- `strict` (optional): Enable strict validation (default: true)
-- `weights` (optional): Custom trust score weights
-
-**Example:**
-```json
-{
-  "packPath": "./my-pack.yml",
-  "runKey": "ci-build-123",
-  "strict": true
-}
-```
-
-**Requires:** `--allow-run` flag
-
-#### `qa360.report.sign`
-Sign a test report with Ed25519 signature.
-
-**Arguments:**
-- `runId` (required): Run ID to sign
-
-#### `qa360.verify`
-Verify proof integrity and signature.
-
-**Arguments:**
-- `runId` OR `proofPath` (required): Run ID or proof file path
-
-### History & Analytics Tools
-
-#### `qa360.history.list`
-List test runs with optional filters.
-
-**Arguments:**
-- `limit` (optional): Max runs to return (default: 20)
-- `status` (optional): Filter by status (passed|failed|error|cancelled|any)
-- `gate` (optional): Filter by gate name
-- `since` (optional): Show runs since date (ISO format)
-
-#### `qa360.history.get`
-Get detailed information about a specific run.
-
-**Arguments:**
-- `runId` (required): Run ID to retrieve
-
-#### `qa360.history.diff`
-Compare two test runs to identify changes.
-
-**Arguments:**
-- `runA` (required): First run ID
-- `runB` (required): Second run ID
-
-#### `qa360.history.trend`
-Analyze trends over time for performance and quality metrics.
-
-**Arguments:**
-- `gate` (optional): Focus on specific gate (perf|ui|api|a11y|sast|dast|deps|secrets)
-- `window` (optional): Number of runs to analyze (default: 20)
-
-#### `qa360.history.export`
-Export a complete run bundle with all artifacts.
-
-**Arguments:**
-- `runId` (required): Run ID to export
-- `bundlePath` (required): Output bundle path (.zip)
-
-#### `qa360.history.gc`
-Garbage collection to manage vault size.
-
-**Arguments:**
-- `keepLast` (optional): Number of recent runs to keep (default: 50)
-- `maxBytes` (optional): Maximum vault size (e.g., "5GB")
-- `dryRun` (optional): Show what would be deleted (default: false)
-
-#### `qa360.history.pin` / `qa360.history.unpin`
-Pin/unpin runs to protect from garbage collection.
-
-**Arguments:**
-- `runId` (required): Run ID to pin/unpin
-
-### Pack Management Tools
-
-#### `qa360.pack.ask`
-Generate a QA360 pack from natural language query.
-
-**Arguments:**
-- `query` (required): Natural language description
-- `context` (optional): Additional context
-
-**Example:**
-```json
-{
-  "query": "Test my web login form and check for security vulnerabilities",
-  "context": {
-    "targets": {
-      "url": "https://myapp.com/login"
-    }
-  }
-}
-```
-
-#### `qa360.pack.validate`
-Validate a QA360 pack configuration.
-
-**Arguments:**
-- `pack` OR `path` (required): YAML content or file path
-
-#### `qa360.pack.lint`
-Lint a QA360 pack and suggest improvements.
-
-**Arguments:**
-- `pack` OR `path` (required): YAML content or file path
-
-#### `qa360.pack.migrate`
-Migrate a QA360 pack to latest version.
-
-**Arguments:**
-- `pack` OR `path` (required): YAML content or file path
-
-### Secrets Management Tools
-
-#### `qa360.secrets.add`
-Add a secret to the secure vault.
-
-**Arguments:**
-- `name` (required): Secret name
-- `value` (required): Secret value
-- `description` (optional): Description
-
-**Requires:** `--allow-secrets` flag
-
-#### `qa360.secrets.list`
-List all secrets (names only, values redacted).
-
-**Requires:** `--allow-secrets` flag
-
-#### `qa360.secrets.remove`
-Remove a secret from the vault.
-
-**Arguments:**
-- `name` (required): Secret name to remove
-
-**Requires:** `--allow-secrets` flag
-
-#### `qa360.secrets.doctor`
-Check secrets vault health and integrity.
-
-**Requires:** `--allow-secrets` flag
-
-### Server & Vault Tools
-
-#### `qa360.serve.health`
-Check QA360 server health status.
-
-#### `qa360.serve.diag`
-Get detailed server diagnostics.
-
-#### `qa360.serve.metrics`
-Get server metrics and performance data.
-
-#### `qa360.vault.get_run`
-Get run data from the evidence vault.
-
-**Arguments:**
-- `runId` (required): Run ID to retrieve
-
-#### `qa360.vault.get_artifacts`
-Get artifacts for a run from the vault.
-
-**Arguments:**
-- `runId` (required): Run ID to get artifacts for
-
-### Triage & Support Tools
-
-#### `qa360.triage.explain_code`
-Explain error codes and suggest remediation actions.
-
-**Arguments:**
-- `code` (required): Error code (e.g., SG001, ZAP002, GL003)
-- `context` (optional): Additional context
-
-**Example:**
-```json
-{
-  "code": "SG001",
-  "context": {
-    "file": "/src/config.js",
-    "line": 42
-  }
-}
-```
-
-## Security & Configuration
-
-### Security Flags
-
-The MCP server implements multiple security layers:
-
-#### `--allow-run`
-Enables the `qa360.run` tool. **Required explicit opt-in** due to potential system impact.
-
-```bash
-qa360-mcp --allow-run
-```
-
-#### `--allow-secrets`
-Enables all `qa360.secrets.*` tools. **Required explicit opt-in** for security.
-
-```bash
-qa360-mcp --allow-secrets
-```
-
-### Rate Limiting
-
-Control request rate to prevent abuse:
-
-```bash
-qa360-mcp --rate-limit 10  # 10 requests per second
-```
-
-### Concurrency Control
-
-Limit concurrent heavy operations:
-
-```bash
-qa360-mcp --max-concurrency 3  # Max 3 concurrent runs
-```
-
-### Audit Logging
-
-All tool calls are logged for security and compliance:
-
-```bash
-qa360-mcp --audit-path ~/.qa360/mcp-audit.log
-```
-
-Disable audit logging:
-
-```bash
-qa360-mcp --no-audit
-```
-
-### Path Sandboxing
-
-The server restricts file access to allowed paths:
-
-- Default: Current working directory
-- Prevents path traversal attacks
-- Validates all file operations
-
-### Data Redaction
-
-Sensitive information is automatically redacted from:
-- Audit logs
-- Tool responses
-- Error messages
-
-Patterns redacted:
-- Bearer tokens
-- API keys
-- Passwords
-- Secrets
-
-## Configuration File
-
-Create `~/.qa360/mcp-config.json` for persistent settings:
-
-```json
-{
-  "allowRun": true,
-  "allowSecrets": false,
-  "maxConcurrency": 2,
-  "rateLimit": {
-    "requests": 5,
-    "window": 1
-  },
-  "sandbox": {
-    "allowedPaths": [
-      "/home/user/projects",
-      "/opt/qa360-packs"
-    ],
-    "requireQA360Dir": true
-  },
-  "audit": {
-    "enabled": true,
-    "logPath": "~/.qa360/mcp-audit.log"
-  }
-}
-```
-
-## Error Handling
-
-### Common Error Codes
-
-- `QA360_PERMISSION_DENIED`: Tool requires explicit permission flag
-- `QA360_RATE_LIMIT_EXCEEDED`: Too many requests
-- `QA360_CONCURRENCY_LIMIT`: Too many concurrent operations
-- `QA360_PATH_ACCESS_DENIED`: File path not allowed
-- `QA360_VAULT_NOT_FOUND`: No QA360 vault initialized
-- `QA360_RUN_NOT_FOUND`: Specified run ID doesn't exist
-- `QA360_INVALID_ARGUMENTS`: Tool arguments validation failed
-
-### Error Response Format
-
-```json
-{
-  "error": {
-    "code": "QA360_PERMISSION_DENIED",
-    "message": "qa360.run tool requires --allow-run flag",
-    "hint": "Restart the MCP server with --allow-run flag"
-  }
-}
-```
-
-## Integration Examples
-
-### CI/CD Pipeline Integration
-
-```yaml
-# GitHub Actions
-name: QA360 MCP Integration
-on: [push]
-
-jobs:
-  qa360-test:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v3
-      
-      - name: Setup QA360 MCP
-        run: |
-          npm install -g @qa360/mcp-server
-          qa360-mcp --allow-run &
-          sleep 2
-      
-      - name: Run Tests via MCP
-        run: |
-          # Use MCP client to trigger tests
-          mcp-call qa360.run '{"packPath": "./qa360-pack.yml", "runKey": "ci-${{ github.run_id }}"}'
-```
-
-### Claude Desktop Workflow
-
-1. **Generate Pack:**
-   ```
-   Create a QA360 pack to test my e-commerce checkout flow for security and performance issues
-   ```
-
-2. **Execute Tests:**
-   ```
-   Run the pack I just created and show me the results
-   ```
-
-3. **Analyze Results:**
-   ```
-   Compare my last two test runs and show me what changed
-   ```
-
-4. **Triage Issues:**
-   ```
-   Explain error code SG001 and tell me how to fix it
-   ```
-
-### VSCode Integration
-
-With the MCP extension installed:
-
-1. Open Command Palette (`Cmd+Shift+P`)
-2. Type "MCP: QA360"
-3. Select desired action
-4. Follow prompts for arguments
-
-## Troubleshooting
-
-### Server Won't Start
-
-```bash
-# Check if port is available
-lsof -i :stdio
-
-# Verify QA360 Core installation
-qa360 --version
-
-# Check permissions
-ls -la ~/.qa360/
-```
-
-### Tool Permission Errors
-
-```bash
-# Enable required permissions
-qa360-mcp --allow-run --allow-secrets
-
-# Check current configuration
-qa360-mcp --dry-run
-```
-
-### Connection Issues
-
-```bash
-# Test MCP server directly
-echo '{"jsonrpc":"2.0","method":"tools/list","id":1}' | qa360-mcp
-
-# Verify client configuration
-cat ~/.config/claude-desktop/claude_desktop_config.json
-```
-
-### Performance Issues
-
-```bash
-# Increase rate limits
-qa360-mcp --rate-limit 20 --max-concurrency 5
-
-# Check audit log size
-ls -lh ~/.qa360/mcp-audit.log
-
-# Rotate logs
-qa360-mcp --rotate-logs
-```
-
-### Vault Issues
-
-```bash
-# Initialize vault if missing
-qa360 run --url https://example.com --pack ./basic-pack.yml
-
-# Check vault health
-qa360-mcp --vault-doctor
-
-# Repair vault
-qa360 doctor --fix
-```
-
-## Best Practices
-
-### Security
-
-1. **Principle of Least Privilege:** Only enable required tools
-2. **Regular Auditing:** Monitor audit logs for suspicious activity
-3. **Path Restrictions:** Use specific allowed paths, not broad directories
-4. **Secrets Management:** Use `qa360.secrets.*` tools instead of environment variables
-5. **Rate Limiting:** Set appropriate limits for your use case
-
-### Performance
-
-1. **Concurrency Limits:** Balance performance vs resource usage
-2. **Vault Maintenance:** Regular garbage collection with `qa360.history.gc`
-3. **Pack Optimization:** Use `qa360.pack.lint` to optimize pack configurations
-4. **Monitoring:** Use `qa360.serve.metrics` to track performance
-
-### Reliability
-
-1. **Error Handling:** Always check tool responses for errors
-2. **Idempotent Operations:** Use `runKey` for repeatable test execution
-3. **Backup Strategy:** Pin important runs with `qa360.history.pin`
-4. **Health Checks:** Regular `qa360.serve.health` monitoring
-
-### Development Workflow
-
-1. **Pack Development:** Use `qa360.pack.ask` for initial generation, then `qa360.pack.validate` and `qa360.pack.lint`
-2. **Testing:** Use `runKey` for consistent test execution during development
-3. **Debugging:** Use `qa360.triage.explain_code` for error resolution
-4. **Analysis:** Use `qa360.history.trend` for performance tracking
-
-## API Reference
-
-### Tool Schema Validation
-
-All tools use JSON Schema validation. Invalid arguments will be rejected with detailed error messages.
-
-### Response Format
-
-All tools return responses in this format:
-
-```json
-{
-  "success": true,
-  "data": {
-    // Tool-specific response data
-  },
-  "metadata": {
-    "timestamp": 1640995200000,
-    "duration_ms": 1500,
-    "tool": "qa360.run"
-  }
-}
-```
-
-### Error Format
-
-```json
-{
-  "success": false,
-  "error": {
-    "code": "QA360_ERROR_CODE",
-    "message": "Human readable error message",
-    "hint": "Suggestion for resolution"
-  }
-}
-```
-
-## Contributing
-
-### Development Setup
-
-```bash
-git clone https://github.com/qa360/qa360-core
-cd qa360-core/packages/mcp-server-qa360
-npm install
-npm run dev
-```
-
-### Testing
-
-```bash
-npm test                    # Unit tests
-npm run test:integration    # Integration tests
-npm run test:coverage       # Coverage report
-```
-
-### Adding New Tools
-
-1. Create tool implementation in `src/tools/`
-2. Add to `ToolRegistry`
-3. Add JSON schema validation
-4. Write tests
-5. Update documentation
-
-## Changelog
-
-### v0.9.0-mcp
-- Initial MCP server implementation
-- 24 tools covering run, history, pack, secrets, serve, vault, and triage
-- Security features: rate limiting, sandboxing, audit logging
-- Comprehensive test suite
-- Full documentation
-
-## License
-
-MIT License - see LICENSE file for details.
-
-## Support
-
-- Documentation: https://qa360.ai/docs/mcp
-- Issues: https://github.com/qa360/qa360-core/issues
-- Discord: https://discord.gg/qa360
-- Email: support@qa360.ai