qa360 2.1.7 → 2.2.0

package/docs/history.md

@@ -0,0 +1,565 @@
+# QA360 History Commands
+
+## Overview
+
+The QA360 History commands provide comprehensive analysis and management of test execution data stored in the Evidence Vault. These commands enable teams to track trends, compare runs, and maintain their testing history.
+
+## Commands Reference
+
+### `qa360 history list`
+
+List test runs with optional filtering.
+
+```bash
+qa360 history list [options]
+```
+
+**Options:**
+- `-l, --limit <number>`: Maximum number of runs to show (default: 20)
+- `-s, --status <status>`: Filter by status (passed|failed|cancelled|error)
+- `-g, --gate <gate>`: Filter by gate name
+- `--since <date>`: Show runs since date (ISO format)
+- `--json`: Output as JSON
+
+**Examples:**
+
+```bash
+# List last 10 runs
+qa360 history list --limit 10
+
+# Show only failed runs
+qa360 history list --status failed
+
+# Filter by specific gate
+qa360 history list --gate ui
+
+# Show runs since specific date
+qa360 history list --since "2025-10-01"
+
+# JSON output for automation
+qa360 history list --json --limit 5
+```
+
+**Sample Output:**
+
+```
+📋 QA360 Run History
+
+┌─────────┬─────────────────────┬──────────┬───────┬──────────┬─────────────┬────────┐
+│ Run ID  │ Started             │ Status   │ Trust │ Duration │ Pack        │ Pinned │
+├─────────┼─────────────────────┼──────────┼───────┼──────────┼─────────────┼────────┤
+│ a1b2c3d │ 2025-10-24 15:30:45 │ ✅ PASSED │ 87%   │ 45s      │ web-app.yml │        │
+│ e4f5g6h │ 2025-10-24 14:22:10 │ ❌ FAILED │ 42%   │ 32s      │ api-test.yml│        │
+│ i7j8k9l │ 2025-10-24 13:15:33 │ ✅ PASSED │ 92%   │ 38s      │ web-app.yml │ 📌     │
+└─────────┴─────────────────────┴──────────┴───────┴──────────┴─────────────┴────────┘
+```
+
+### `qa360 history get`
+
+Get detailed information about a specific run.
+
+```bash
+qa360 history get <runId> [options]
+```
+
+**Options:**
+- `--json`: Output as JSON
+
+**Examples:**
+
+```bash
+# Get run details
+qa360 history get a1b2c3d4
+
+# JSON output
+qa360 history get a1b2c3d4 --json
+```
+
+**Sample Output:**
+
+```
+🔍 Run Details: a1b2c3d4-e5f6-7890-abcd-ef1234567890
+
+📊 Overview:
+  Status: ✅ PASSED
+  Trust Score: 87%
+  Started: 2025-10-24 15:30:45
+  Ended: 2025-10-24 15:31:30
+  Duration: 45s
+  Pack: ./packs/web-app.yml
+  Pack Hash: abc123def456...
+  Run Key: ci-build-789
+
+🚪 Gates:
+  ui: ✅ PASSED (2500ms)
+    interactions: 12
+    coverage_percent: 85
+  api: ✅ PASSED (1200ms)
+    endpoints_tested: 8
+    response_time_p95: 150
+  sast: ⚠️ WARN (800ms)
+    findings_total: 2
+    high_severity: 0
+
+🔍 Findings:
+  sast:
+    🟡 MEDIUM semgrep:hardcoded-url: Hardcoded URL detected
+      Location: /src/config.js:42
+    🔵 LOW eslint:no-console: Console statement found
+      Location: /src/debug.js:15
+
+📎 Artifacts:
+  proof_pdf: a1b2c3d4e5f6... (2.3 MB)
+    Original: proof-2025-10-24.pdf
+  ui_screenshot_final: f6e5d4c3b2a1... (456 KB)
+    Original: final-state.png
+```
+
+### `qa360 history diff`
+
+Compare two test runs to identify changes.
+
+```bash
+qa360 history diff <runIdA> <runIdB> [options]
+```
+
+**Options:**
+- `--json`: Output as JSON
+
+**Examples:**
+
+```bash
+# Compare two runs
+qa360 history diff a1b2c3d4 e5f6g7h8
+
+# JSON output for automation
+qa360 history diff a1b2c3d4 e5f6g7h8 --json
+```
+
+**Sample Output:**
+
+```
+🔄 Run Comparison
+
+📊 Overview:
+  Run A: a1b2c3d4 (passed, trust: 87%)
+  Run B: e5f6g7h8 (failed, trust: 42%)
+  Trust Delta: 📉 -45%
+
+🚪 Gate Changes:
+  ~ ui: passed → failed
+    Duration: ⬆️ +500ms
+  ~ api: passed → passed
+    Duration: ⬇️ -200ms
+  + sast: warn (new gate)
+
+🔍 Finding Changes:
+  + sast: 🔴 HIGH semgrep:sql-injection SQL injection vulnerability
+  + ui: 🟠 MEDIUM accessibility:missing-alt Missing alt text
+  - api: 🔵 LOW openapi:deprecated-endpoint (resolved)
+```
+
+### `qa360 history trend`
+
+Analyze trends over time for performance and quality metrics.
+
+```bash
+qa360 history trend [options]
+```
+
+**Options:**
+- `-g, --gate <gate>`: Focus on specific gate
+- `-w, --window <number>`: Number of runs to analyze (default: 20)
+- `--json`: Output as JSON
+- `--csv <path>`: Export to CSV file
+
+**Examples:**
+
+```bash
+# General trend analysis
+qa360 history trend
+
+# Focus on UI gate performance
+qa360 history trend --gate ui --window 30
+
+# Export trend data to CSV
+qa360 history trend --csv ./trends.csv
+
+# JSON output for dashboards
+qa360 history trend --json --window 50
+```
+
+**Sample Output:**
+
+```
+📈 Trend Analysis (20 runs)
+
+📊 Summary:
+  Average Trust Score: 78%
+  Average Findings: 3
+  Success Rate: 85%
+  Average UI Duration: 2200ms
+
+📈 Recent Trend:
+  2025-10-24: ✅ PASSED Trust: 87% Findings: 2
+  2025-10-24: ❌ FAILED Trust: 42% Findings: 8
+  2025-10-23: ✅ PASSED Trust: 91% Findings: 1
+  2025-10-23: ✅ PASSED Trust: 83% Findings: 3
+  2025-10-23: ⚠️ WARN   Trust: 76% Findings: 5
+  2025-10-22: ✅ PASSED Trust: 89% Findings: 2
+  2025-10-22: ✅ PASSED Trust: 92% Findings: 1
+  2025-10-21: ✅ PASSED Trust: 85% Findings: 4
+  2025-10-21: ❌ FAILED Trust: 38% Findings: 12
+  2025-10-20: ✅ PASSED Trust: 88% Findings: 2
+```
+
+### `qa360 history export`
+
+Export a complete run bundle with all artifacts and metadata.
+
+```bash
+qa360 history export <runId> --bundle <path>
+```
+
+**Options:**
+- `-b, --bundle <path>`: Output bundle path (.zip) **[Required]**
+
+**Examples:**
+
+```bash
+# Export single run
+qa360 history export a1b2c3d4 --bundle ./exports/run-a1b2c3d4.zip
+
+# Export for compliance audit
+qa360 history export baseline-run --bundle ./audit/baseline-evidence.zip
+```
+
+**Bundle Contents:**
+```
+run-a1b2c3d4.zip
+├── run.json              # Complete run metadata
+├── proof.pdf             # Signed proof document
+├── proof.json            # Proof metadata and signature
+├── artifacts/
+│   ├── ui_screenshot_1.png
+│   ├── ui_screenshot_2.png
+│   ├── sast_report.json
+│   └── performance_trace.json
+└── verification/
+    ├── signature.txt      # Ed25519 signature
+    └── public_key.pem     # Public key for verification
+```
+
+### `qa360 history gc`
+
+Garbage collection to manage vault size and remove orphaned data.
+
+```bash
+qa360 history gc [options]
+```
+
+**Options:**
+- `--keep-last <number>`: Number of recent runs to keep (default: 50)
+- `--max-bytes <size>`: Maximum vault size (e.g., 5GB)
+- `--dry-run`: Show what would be deleted without deleting
+
+**Examples:**
+
+```bash
+# Dry run to preview cleanup
+qa360 history gc --keep-last 100 --dry-run
+
+# Clean up keeping last 50 runs
+qa360 history gc --keep-last 50
+
+# Limit vault size to 2GB
+qa360 history gc --max-bytes 2GB
+
+# Conservative cleanup with size limit
+qa360 history gc --keep-last 200 --max-bytes 10GB
+```
+
+**Sample Output:**
+
+```
+🧹 Starting garbage collection...
+
+📊 Vault Analysis:
+  Total runs: 150
+  Total artifacts: 1,247
+  Current size: 8.3 GB
+  Pinned runs: 5
+
+🗑️ Cleanup Plan:
+  Runs to delete: 50 (keeping last 100)
+  Orphaned artifacts: 23
+  Space to free: 2.1 GB
+
+⚠️ Protected from deletion:
+  📌 baseline-v1.0 (pinned)
+  📌 release-candidate (pinned)
+  📌 security-audit-2025 (pinned)
+
+✅ Garbage collection completed!
+  Removed: 50 runs, 23 orphaned artifacts
+  Freed: 2.1 GB
+  New vault size: 6.2 GB
+```
+
+### `qa360 history pin` / `qa360 history unpin`
+
+Protect important runs from garbage collection.
+
+```bash
+qa360 history pin <runId>
+qa360 history unpin <runId>
+```
+
+**Examples:**
+
+```bash
+# Pin important baseline run
+qa360 history pin baseline-v2.0
+
+# Pin security audit run
+qa360 history pin security-audit-2025
+
+# Unpin old baseline
+qa360 history unpin baseline-v1.0
+```
+
+## Use Cases
+
+### Development Workflow
+
+```bash
+# Check recent test results
+qa360 history list --limit 5
+
+# Compare current run with previous
+qa360 history diff $(qa360 history list --json --limit 1 | jq -r '.[0].id') \
+                   $(qa360 history list --json --limit 2 | jq -r '.[1].id')
+
+# Track UI performance trends
+qa360 history trend --gate ui --window 20
+```
+
+### CI/CD Integration
+
+```bash
+# In CI pipeline - check if run already exists
+if qa360 history list --json | jq -e ".[] | select(.run_key == \"ci-$BUILD_ID\")" > /dev/null; then
+  echo "Run already exists, skipping..."
+  exit 0
+fi
+
+# Export results for artifact storage
+qa360 history export $RUN_ID --bundle ./artifacts/qa360-results.zip
+```
+
+### Release Management
+
+```bash
+# Pin release candidate run
+qa360 history pin $RELEASE_RUN_ID
+
+# Compare release candidate with previous release
+qa360 history diff $RELEASE_RUN_ID $PREVIOUS_RELEASE_RUN_ID
+
+# Export release evidence
+qa360 history export $RELEASE_RUN_ID --bundle ./release-evidence.zip
+```
+
+### Performance Monitoring
+
+```bash
+# Monitor UI performance trends
+qa360 history trend --gate ui --csv ./metrics/ui-performance.csv
+
+# Check for performance regressions
+qa360 history list --gate perf --limit 10 --json | \
+  jq '.[] | select(.trust_score < 80)'
+
+# Weekly performance report
+qa360 history trend --window 50 --json > weekly-report.json
+```
+
+### Security Auditing
+
+```bash
+# Find runs with security findings
+qa360 history list --json | \
+  jq '.[] | select(.critical_findings > 0)'
+
+# Export security audit trail
+qa360 history export security-audit-run --bundle ./audit/security-evidence.zip
+
+# Track security improvements
+qa360 history trend --gate sast --window 30
+```
+
+### Compliance & Documentation
+
+```bash
+# Pin compliance runs
+qa360 history pin compliance-q4-2025
+
+# Export evidence for audit
+qa360 history export compliance-q4-2025 --bundle ./compliance/q4-evidence.zip
+
+# Generate compliance report
+qa360 history trend --json --window 100 > compliance-metrics.json
+```
+
+## JSON Output Format
+
+All history commands support `--json` output for automation:
+
+```json
+{
+  "runs": [
+    {
+      "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+      "run_key": "ci-build-789",
+      "started_at": 1729781445000,
+      "ended_at": 1729781490000,
+      "status": "passed",
+      "trust_score": 87,
+      "pack_path": "./packs/web-app.yml",
+      "pack_hash": "abc123def456...",
+      "pinned": false,
+      "gates": [
+        {
+          "name": "ui",
+          "status": "passed",
+          "duration_ms": 2500,
+          "metrics": {
+            "interactions": 12,
+            "coverage_percent": 85
+          }
+        }
+      ],
+      "findings": [
+        {
+          "gate": "sast",
+          "severity": "medium",
+          "rule": "semgrep:hardcoded-url",
+          "location": "/src/config.js:42",
+          "message": "Hardcoded URL detected"
+        }
+      ],
+      "artifacts": [
+        {
+          "sha256": "a1b2c3d4e5f6...",
+          "label": "proof_pdf",
+          "mime_type": "application/pdf",
+          "size_bytes": 2415616,
+          "original_name": "proof-2025-10-24.pdf"
+        }
+      ]
+    }
+  ]
+}
+```
+
+## Error Handling
+
+### Common Errors
+
+1. **No vault found**: Run a test first to initialize the vault
+2. **Run not found**: Check run ID format and existence
+3. **Permission denied**: Ensure write access to `.qa360` directory
+4. **Disk full**: Run garbage collection or free disk space
+
+### Error Messages
+
+```bash
+# Vault not initialized
+Error: No QA360 vault found. Run a test first to initialize the vault.
+
+# Invalid run ID
+Error: Run not found: invalid-run-id
+
+# Permission issues
+Error: Permission denied accessing vault. Check .qa360 directory permissions.
+
+# Disk space issues
+Error: Insufficient disk space for export. Free space or use --max-bytes option.
+```
+
+## Best Practices
+
+### Regular Maintenance
+
+```bash
+# Weekly cleanup
+qa360 history gc --keep-last 100 --max-bytes 5GB
+
+# Monthly trend analysis
+qa360 history trend --window 200 --csv monthly-trends.csv
+
+# Quarterly compliance export
+qa360 history export compliance-run --bundle quarterly-audit.zip
+```
+
+### Performance Optimization
+
+- Use `--limit` to avoid large result sets
+- Pin important runs before garbage collection
+- Export large result sets to CSV for analysis
+- Use JSON output for programmatic processing
+
+### Security Considerations
+
+- Exported bundles contain sensitive test data
+- Store exports securely with appropriate access controls
+- Regularly audit pinned runs and unpin when no longer needed
+- Monitor vault size to prevent disk exhaustion
+
+### Integration Tips
+
+- Use run keys for idempotent CI/CD integration
+- Combine with `jq` for JSON processing in scripts
+- Set up automated garbage collection in maintenance windows
+- Create dashboards using trend data exports
+
+## Troubleshooting
+
+### Performance Issues
+
+```bash
+# Check vault size
+qa360 history stats
+
+# Analyze slow queries
+sqlite3 .qa360/vault.db "EXPLAIN QUERY PLAN SELECT * FROM runs ORDER BY started_at DESC LIMIT 10;"
+
+# Optimize database
+sqlite3 .qa360/vault.db "VACUUM; ANALYZE;"
+```
+
+### Data Integrity
+
+```bash
+# Verify vault integrity
+qa360 doctor --vault
+
+# Check for orphaned artifacts
+qa360 history gc --dry-run --orphans-only
+
+# Verify artifact checksums
+qa360 history verify --artifacts
+```
+
+### Recovery
+
+```bash
+# Restore from backup
+cp vault-backup.db .qa360/vault.db
+
+# Rebuild indexes
+sqlite3 .qa360/vault.db "REINDEX;"
+
+# Verify recovery
+qa360 history list --limit 1
+```