qa360 2.2.1 → 2.2.13

package/docs/vault.md

@@ -1,491 +0,0 @@
-# QA360 Evidence Vault
-
-## Overview
-
-The QA360 Evidence Vault is a comprehensive storage system for test execution data, providing:
-
-- **SQLite WAL database** for structured data storage
-- **Content-Addressable Storage (CAS)** for file deduplication
-- **Idempotent runs** via run keys
-- **Cryptographic signatures** for proof integrity
-- **Historical analysis** and trend tracking
-
-## Architecture
-
-### Core Components
-
-1. **EvidenceVault**: Main vault engine with SQLite WAL backend
-2. **ContentAddressableStorage**: File storage with SHA256-based deduplication
-3. **SecurityRedactor**: Automatic PII removal from stored data
-4. **CLI History Commands**: Query and analysis interface
-
-### Database Schema
-
-```sql
--- Core run records
-runs (
-    id TEXT PRIMARY KEY,           -- UUID v4
-    run_key TEXT UNIQUE,          -- Idempotence key
-    started_at INTEGER,           -- Epoch milliseconds
-    ended_at INTEGER,             -- Epoch milliseconds
-    pack_hash TEXT,               -- SHA256 of normalized pack.yml
-    status TEXT,                  -- running|passed|failed|cancelled|error
-    trust_score INTEGER,          -- 0-100
-    signature_hex TEXT,           -- Ed25519 signature
-    proof_pdf_path TEXT,          -- CAS path to signed proof
-    pinned INTEGER DEFAULT 0      -- GC protection
-)
-
--- Test gate results
-gates (
-    run_id TEXT,                  -- Foreign key to runs
-    name TEXT,                    -- api_smoke|ui|perf|sast|dast|a11y|deps|secrets
-    status TEXT,                  -- passed|failed|skipped|warn
-    duration_ms INTEGER,          -- Execution time
-    metrics_json TEXT,            -- JSON metrics (p95, a11y score, etc.)
-    budgets_json TEXT             -- JSON budget definitions
-)
-
--- Security/quality findings
-findings (
-    run_id TEXT,                  -- Foreign key to runs
-    gate TEXT,                    -- Gate that produced finding
-    severity TEXT,                -- info|low|medium|high|critical
-    rule TEXT,                    -- Rule identifier (semgrep:xxx, zap:yyy)
-    location TEXT,                -- File path, URL, or selector
-    message TEXT,                 -- Redacted finding message
-    fingerprint TEXT              -- Unique fingerprint for deduplication
-)
-
--- File artifacts (CAS metadata)
-artifacts (
-    sha256 TEXT PRIMARY KEY,      -- Content hash
-    mime_type TEXT,               -- MIME type
-    size_bytes INTEGER,           -- File size
-    cas_path TEXT,                -- CAS storage path
-    created_at INTEGER            -- Creation timestamp
-)
-
--- Run-artifact relationships
-run_artifacts (
-    run_id TEXT,                  -- Foreign key to runs
-    sha256 TEXT,                  -- Foreign key to artifacts
-    label TEXT,                   -- proof_pdf, ui_screenshot_login, etc.
-    PRIMARY KEY (run_id, sha256, label)
-)
-```
-
-## Usage
-
-### Basic Operations
-
-```typescript
-import { EvidenceVault } from '@qa360/core';
-
-// Open vault
-const vault = await EvidenceVault.open('/path/to/.qa360');
-
-// Begin run with idempotence
-const { runId, isReused } = await vault.beginRun({
-  run_key: 'ci-build-123',  // Optional idempotence key
-  pack_path: './pack.yml',
-  pack_hash: 'sha256-of-normalized-pack',
-  weights: { ui: 0.4, api: 0.6 }
-});
-
-// Record gate execution
-await vault.recordGate(runId, {
-  name: 'ui',
-  status: 'passed',
-  duration_ms: 2500,
-  metrics_json: JSON.stringify({
-    interactions: 12,
-    coverage_percent: 85
-  })
-});
-
-// Record security finding
-await vault.recordFinding(runId, {
-  gate: 'sast',
-  severity: 'high',
-  rule: 'semgrep:hardcoded-secret',
-  location: '/src/config.js:42',
-  message: 'Hardcoded API key detected'
-});
-
-// Store proof PDF
-const proofPdf = await fs.readFile('./proof.pdf');
-const artifact = await vault.storeArtifact(
-  runId,
-  proofPdf,
-  'application/pdf',
-  'proof_pdf',
-  'proof.pdf'
-);
-
-// Finish run
-await vault.finishRun(runId, {
-  status: 'passed',
-  trust_score: 87,
-  signature: 'ed25519-signature-hex',
-  proof_pdf_sha: artifact.sha256
-});
-
-// Close vault
-await vault.close();
-```
-
-### CLI Commands
-
-```bash
-# List recent runs
-qa360 history list --limit 10
-
-# Get detailed run information
-qa360 history get <runId>
-
-# Compare two runs
-qa360 history diff <runIdA> <runIdB>
-
-# Show trends over time
-qa360 history trend --gate ui --window 20
-
-# Export run bundle
-qa360 history export <runId> --bundle ./export.zip
-
-# Garbage collection
-qa360 history gc --keep-last 50 --dry-run
-
-# Pin/unpin runs
-qa360 history pin <runId>
-qa360 history unpin <runId>
-```
-
-## Content-Addressable Storage
-
-### File Organization
-
-```
-.qa360/runs/cas/
-├── aa/
-│   ├── bb/
-│   │   ├── aabbcc1234567890abcdef...  # SHA256 filename
-│   │   └── aabbcc9876543210fedcba...
-│   └── cc/
-└── ff/
-    └── ee/
-```
-
-### Benefits
-
-- **Deduplication**: Identical files stored only once
-- **Integrity**: SHA256 verification prevents corruption
-- **Scalability**: Sharded directory structure
-- **Immutability**: Content cannot be modified without changing hash
-
-### Usage
-
-```typescript
-import { ContentAddressableStorage } from '@qa360/core';
-
-const cas = new ContentAddressableStorage('/path/to/.qa360/runs');
-
-// Save file
-const content = Buffer.from('Hello, World!');
-const artifact = await cas.saveArtifact(content, 'text/plain');
-console.log(artifact.sha256); // Content hash
-console.log(artifact.exists);  // false (first save)
-
-// Retrieve file
-const retrieved = await cas.getArtifact(artifact.sha256);
-console.log(retrieved.equals(content)); // true
-
-// Check existence
-console.log(cas.exists(artifact.sha256)); // true
-
-// Get statistics
-const stats = await cas.getStats();
-console.log(`${stats.totalFiles} files, ${stats.totalBytes} bytes`);
-```
-
-## Idempotence
-
-### Run Keys
-
-Run keys enable idempotent test execution:
-
-```typescript
-// First execution
-const result1 = await vault.beginRun({
-  run_key: 'ci-pr-456',
-  pack_path: './pack.yml',
-  pack_hash: 'abc123'
-});
-console.log(result1.isReused); // false
-
-// Subsequent execution with same key
-const result2 = await vault.beginRun({
-  run_key: 'ci-pr-456',
-  pack_path: './pack.yml',
-  pack_hash: 'abc123'
-});
-console.log(result2.isReused); // true
-console.log(result1.runId === result2.runId); // true
-```
-
-### Pack Hash Normalization
-
-Pack hashes ensure consistent idempotence:
-
-```typescript
-const pack1 = `
-name: test
-version: 1.0.0
-# Comment 1
-description: Test pack
-`;
-
-const pack2 = `
-name: test
-version: 1.0.0
-description: Test pack
-# Comment 2
-`;
-
-const hash1 = EvidenceVault.calculatePackHash(pack1);
-const hash2 = EvidenceVault.calculatePackHash(pack2);
-console.log(hash1 === hash2); // true (comments ignored)
-```
-
-## Security & Redaction
-
-### Automatic PII Removal
-
-The vault automatically redacts sensitive information:
-
-```typescript
-// Input finding
-await vault.recordFinding(runId, {
-  gate: 'sast',
-  severity: 'high',
-  rule: 'hardcoded-secret',
-  message: 'API key found: sk_live_1234567890abcdef',
-  raw_output: 'Full tool output with sk_live_1234567890abcdef'
-});
-
-// Stored finding (redacted)
-const findings = await vault.getFindings(runId);
-console.log(findings[0].message); // "API key found: ***"
-```
-
-### Redaction Rules
-
-- **Passwords**: `"password": "value"` → `"password": "***"`
-- **API Keys**: `sk_live_abc123` → `sk_live_***`
-- **Tokens**: `Bearer abc123` → `Bearer ***`
-- **Email**: `user@domain.com` → `***@domain.com`
-- **Credit Cards**: `4111-1111-1111-1111` → `****-****-****-****`
-
-## Performance & Optimization
-
-### SQLite WAL Mode
-
-```sql
-PRAGMA journal_mode = WAL;           -- Write-Ahead Logging
-PRAGMA journal_size_limit = 67108864; -- 64MB limit
-PRAGMA synchronous = NORMAL;         -- Balanced durability/performance
-PRAGMA cache_size = 10000;           -- 10K pages in memory
-PRAGMA foreign_keys = ON;            -- Referential integrity
-```
-
-### Indexing Strategy
-
-```sql
--- Optimized for common queries
-CREATE INDEX idx_runs_started_at ON runs(started_at DESC);
-CREATE INDEX idx_runs_status ON runs(status);
-CREATE INDEX idx_gates_run_name ON gates(run_id, name);
-CREATE INDEX idx_findings_run_gate ON findings(run_id, gate);
-CREATE INDEX idx_findings_severity ON findings(severity);
-```
-
-### Views for Analytics
-
-```sql
--- Recent runs with summary
-CREATE VIEW v_recent_runs AS
-SELECT 
-    r.id,
-    r.started_at,
-    r.status,
-    r.trust_score,
-    COUNT(g.id) as gate_count,
-    COUNT(CASE WHEN g.status = 'passed' THEN 1 END) as gates_passed,
-    COUNT(f.id) as finding_count
-FROM runs r
-LEFT JOIN gates g ON r.id = g.run_id
-LEFT JOIN findings f ON r.id = f.run_id
-GROUP BY r.id
-ORDER BY r.started_at DESC;
-```
-
-## Backup & Restore
-
-### Database Backup
-
-```bash
-# Create backup
-sqlite3 .qa360/vault.db ".backup backup.db"
-
-# Restore from backup
-sqlite3 .qa360/vault.db ".restore backup.db"
-```
-
-### CAS Backup
-
-```bash
-# Backup CAS files
-tar -czf cas-backup.tar.gz .qa360/runs/cas/
-
-# Restore CAS files
-tar -xzf cas-backup.tar.gz
-```
-
-### Full Vault Export
-
-```bash
-# Export complete vault
-qa360 history export --all --bundle complete-vault.zip
-
-# Import vault
-qa360 history import --bundle complete-vault.zip
-```
-
-## Monitoring & Maintenance
-
-### Vault Statistics
-
-```typescript
-const stats = await vault.getStats();
-console.log(`
-Total runs: ${stats.totalRuns}
-Total gates: ${stats.totalGates}
-Total findings: ${stats.totalFindings}
-Total artifacts: ${stats.totalArtifacts}
-Vault size: ${ContentAddressableStorage.formatBytes(stats.vaultSizeBytes)}
-Date range: ${stats.oldestRun} - ${stats.newestRun}
-`);
-```
-
-### Garbage Collection
-
-```bash
-# Dry run to see what would be deleted
-qa360 history gc --keep-last 100 --max-bytes 5GB --dry-run
-
-# Actually perform cleanup
-qa360 history gc --keep-last 100 --max-bytes 5GB
-
-# Protect important runs from GC
-qa360 history pin <important-run-id>
-```
-
-### Health Checks
-
-```bash
-# Verify vault integrity
-qa360 doctor --vault
-
-# Check for orphaned artifacts
-qa360 history gc --dry-run --orphans-only
-
-# Verify artifact integrity
-qa360 history verify --all-artifacts
-```
-
-## Integration Examples
-
-### CI/CD Pipeline
-
-```yaml
-# GitHub Actions
-- name: Run QA360 Tests
-  run: |
-    qa360 run --run-key "ci-${{ github.run_id }}" \
-              --pack ./qa360-pack.yml
-    
-- name: Upload Results
-  if: always()
-  uses: actions/upload-artifact@v3
-  with:
-    name: qa360-results
-    path: .qa360/
-```
-
-### Prometheus Metrics
-
-```bash
-# Export metrics for monitoring
-curl http://localhost:4060/metrics | grep qa360_
-```
-
-### Trend Analysis
-
-```typescript
-// Get performance trends
-const runs = await vault.listRuns({ limit: 50 });
-const uiGates = await Promise.all(
-  runs.map(r => vault.getGates(r.id).then(gates => 
-    gates.find(g => g.name === 'ui')
-  ))
-);
-
-const avgDuration = uiGates
-  .filter(g => g && g.duration_ms)
-  .reduce((sum, g) => sum + g.duration_ms!, 0) / uiGates.length;
-
-console.log(`Average UI test duration: ${avgDuration}ms`);
-```
-
-## Troubleshooting
-
-### Common Issues
-
-1. **Database locked**: Ensure proper vault closure
-2. **Disk space**: Monitor CAS growth and run GC
-3. **Performance**: Check SQLite WAL file size
-4. **Corruption**: Use SQLite integrity check
-
-### Diagnostic Commands
-
-```bash
-# Check vault health
-qa360 doctor --vault
-
-# Verify database integrity
-sqlite3 .qa360/vault.db "PRAGMA integrity_check;"
-
-# Check WAL file size
-ls -lh .qa360/vault.db-wal
-
-# Analyze disk usage
-qa360 history stats --detailed
-```
-
-## Best Practices
-
-1. **Regular GC**: Schedule garbage collection
-2. **Pin important runs**: Protect baseline runs from cleanup
-3. **Monitor growth**: Track vault size over time
-4. **Backup strategy**: Regular database and CAS backups
-5. **Run keys**: Use meaningful, unique identifiers
-6. **Artifact labeling**: Use descriptive labels for artifacts
-
-## API Reference
-
-See the TypeScript interfaces in `@qa360/core` for complete API documentation:
-
-- `EvidenceVault`: Main vault interface
-- `ContentAddressableStorage`: File storage interface
-- `RunRecord`, `GateRecord`, `FindingRecord`: Data structures
-- `VaultConfig`, `BeginRunOptions`, `FinishRunOptions`: Configuration types