qa360 2.2.1 → 2.2.7

package/docs/proofs.md

@@ -1,670 +0,0 @@
-# QA360 Cryptographic Proofs
-
-## Overview
-
-QA360 generates cryptographically signed proofs of test execution, providing tamper-evident evidence of quality assurance activities. These proofs combine comprehensive test results with Ed25519 digital signatures to ensure authenticity and integrity.
-
-## Proof Architecture
-
-### Components
-
-1. **Proof Document (PDF)**: Human-readable test report with visual evidence
-2. **Proof Metadata (JSON)**: Machine-readable test data and metrics
-3. **Digital Signature**: Ed25519 cryptographic signature ensuring integrity
-4. **Evidence Vault Link**: Connection to detailed execution data in the vault
-
-### Trust Chain
-
-```
-Pack Configuration → Test Execution → Results Collection → Proof Generation → Digital Signing → Vault Storage
-```
-
-## Proof Generation Process
-
-### 1. Test Execution
-
-```typescript
-// During test execution
-const { runId } = await vault.beginRun({
-  run_key: 'release-v2.1.0',
-  pack_path: './qa360-pack.yml',
-  pack_hash: EvidenceVault.calculatePackHash(packContent)
-});
-
-// Record test results
-await vault.recordGate(runId, {
-  name: 'ui',
-  status: 'passed',
-  duration_ms: 2500,
-  metrics_json: JSON.stringify({ coverage: 85, interactions: 12 })
-});
-```
-
-### 2. Proof Document Creation
-
-```typescript
-// Generate comprehensive proof
-const proofData = {
-  run_id: runId,
-  timestamp: Date.now(),
-  pack_hash: packHash,
-  gates: await vault.getGates(runId),
-  findings: await vault.getFindings(runId),
-  artifacts: await vault.getRunArtifacts(runId),
-  trust_score: calculateTrustScore(gates, findings),
-  metadata: {
-    version: '1.0.0',
-    generator: 'qa360-core',
-    environment: process.env.NODE_ENV
-  }
-};
-
-// Create PDF proof document
-const proofPdf = await generateProofPDF(proofData);
-```
-
-### 3. Digital Signing
-
-```typescript
-import { sign } from 'crypto';
-
-// Load Ed25519 private key
-const privateKey = await loadPrivateKey('.qa360/keys/private.pem');
-
-// Create signature of proof JSON
-const proofJson = JSON.stringify(proofData, null, 2);
-const signature = sign('ed25519', Buffer.from(proofJson), privateKey);
-
-// Store signed proof
-await vault.finishRun(runId, {
-  status: 'passed',
-  trust_score: proofData.trust_score,
-  signature: signature.toString('hex'),
-  proof_pdf_sha: await vault.storeArtifact(
-    runId, 
-    proofPdf, 
-    'application/pdf', 
-    'proof_pdf'
-  ).then(a => a.sha256)
-});
-```
-
-## Proof Structure
-
-### JSON Proof Format
-
-```json
-{
-  "proof_version": "1.0.0",
-  "run_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
-  "timestamp": 1729781445000,
-  "pack": {
-    "path": "./qa360-pack.yml",
-    "hash": "sha256:abc123def456...",
-    "version": "1.2.0"
-  },
-  "execution": {
-    "started_at": 1729781445000,
-    "ended_at": 1729781490000,
-    "duration_ms": 45000,
-    "environment": {
-      "node_version": "18.20.5",
-      "platform": "darwin",
-      "ci": true,
-      "runner": "github-actions"
-    }
-  },
-  "results": {
-    "status": "passed",
-    "trust_score": 87,
-    "gates": [
-      {
-        "name": "ui",
-        "status": "passed",
-        "duration_ms": 2500,
-        "metrics": {
-          "interactions": 12,
-          "coverage_percent": 85,
-          "accessibility_score": 92
-        },
-        "budgets": {
-          "max_duration_ms": 5000,
-          "min_coverage_percent": 80
-        }
-      },
-      {
-        "name": "api",
-        "status": "passed",
-        "duration_ms": 1200,
-        "metrics": {
-          "endpoints_tested": 8,
-          "response_time_p95": 150,
-          "success_rate": 100
-        }
-      },
-      {
-        "name": "sast",
-        "status": "warn",
-        "duration_ms": 800,
-        "metrics": {
-          "files_scanned": 156,
-          "rules_applied": 234
-        }
-      }
-    ],
-    "findings": [
-      {
-        "gate": "sast",
-        "severity": "medium",
-        "rule": "semgrep:hardcoded-url",
-        "location": "/src/config.js:42",
-        "message": "Hardcoded URL detected",
-        "fingerprint": "abc123def456"
-      }
-    ],
-    "coverage": {
-      "ui_interactions": 85,
-      "api_endpoints": 100,
-      "security_rules": 95,
-      "overall": 87
-    }
-  },
-  "artifacts": [
-    {
-      "label": "proof_pdf",
-      "sha256": "def456abc123...",
-      "mime_type": "application/pdf",
-      "size_bytes": 2415616
-    },
-    {
-      "label": "ui_screenshot_final",
-      "sha256": "789abc123def...",
-      "mime_type": "image/png",
-      "size_bytes": 456789
-    }
-  ],
-  "signature": {
-    "algorithm": "ed25519",
-    "public_key": "302a300506032b657003210012345...",
-    "signature": "abcdef123456789...",
-    "signed_data_hash": "sha256:fedcba987654321..."
-  },
-  "verification": {
-    "vault_location": ".qa360/vault.db",
-    "proof_pdf_sha256": "def456abc123...",
-    "qr_code": "https://verify.qa360.ai/proof/a1b2c3d4"
-  }
-}
-```
-
-### PDF Proof Document
-
-The PDF proof contains:
-
-1. **Executive Summary**
-   - Overall test status and trust score
-   - Key metrics and findings summary
-   - Execution timeline and environment
-
-2. **Detailed Results**
-   - Gate-by-gate breakdown
-   - Performance metrics and budgets
-   - Security findings with severity levels
-
-3. **Visual Evidence**
-   - Screenshots of test execution
-   - Charts and graphs of metrics
-   - Error highlights and annotations
-
-4. **Technical Details**
-   - Pack configuration used
-   - Environment and tool versions
-   - Execution logs and traces
-
-5. **Verification Information**
-   - Digital signature details
-   - QR code for online verification
-   - Instructions for proof validation
-
-## Verification Process
-
-### Command Line Verification
-
-```bash
-# Verify proof from run ID
-qa360 verify <runId>
-
-# Verify exported proof bundle
-qa360 verify ./proof-bundle.zip
-
-# Verify PDF proof directly
-qa360 verify ./proof.pdf
-```
-
-### Programmatic Verification
-
-```typescript
-import { verify } from 'crypto';
-import { EvidenceVault } from '@qa360/core';
-
-async function verifyProof(runId: string): Promise<boolean> {
-  const vault = await EvidenceVault.open('.qa360');
-  
-  // Get run and proof data
-  const run = await vault.getRun(runId);
-  if (!run || !run.signature_hex) {
-    throw new Error('Run not found or not signed');
-  }
-  
-  // Reconstruct proof JSON
-  const [gates, findings, artifacts] = await Promise.all([
-    vault.getGates(runId),
-    vault.getFindings(runId),
-    vault.getRunArtifacts(runId)
-  ]);
-  
-  const proofData = {
-    run_id: runId,
-    timestamp: run.started_at,
-    // ... reconstruct complete proof object
-  };
-  
-  // Verify signature
-  const publicKey = await loadPublicKey('.qa360/keys/public.pem');
-  const proofJson = JSON.stringify(proofData, null, 2);
-  const signature = Buffer.from(run.signature_hex, 'hex');
-  
-  return verify('ed25519', Buffer.from(proofJson), publicKey, signature);
-}
-```
-
-### Online Verification
-
-```bash
-# Generate verification URL
-qa360 verify --online <runId>
-# Output: https://verify.qa360.ai/proof/a1b2c3d4-e5f6-7890-abcd-ef1234567890
-
-# Embed QR code in proof
-qa360 generate-proof --qr-code <runId>
-```
-
-## Key Management
-
-### Key Generation
-
-```bash
-# Generate Ed25519 key pair
-qa360 keys generate --algorithm ed25519
-
-# Specify custom location
-qa360 keys generate --output ./custom-keys/
-
-# Generate with passphrase protection
-qa360 keys generate --passphrase
-```
-
-### Key Storage
-
-```
-.qa360/keys/
-├── private.pem          # Ed25519 private key (600 permissions)
-├── public.pem           # Ed25519 public key
-├── key.fingerprint      # Key fingerprint for identification
-└── key.metadata.json    # Key generation metadata
-```
-
-### Key Rotation
-
-```bash
-# Generate new key pair
-qa360 keys rotate
-
-# Backup old keys
-qa360 keys backup --output ./key-backup/
-
-# Update vault with new key
-qa360 keys update-vault
-```
-
-## Trust Score Calculation
-
-### Algorithm
-
-```typescript
-function calculateTrustScore(gates: GateRecord[], findings: FindingRecord[]): number {
-  let score = 100;
-  
-  // Gate failures reduce trust
-  gates.forEach(gate => {
-    switch (gate.status) {
-      case 'failed':
-        score -= 20;
-        break;
-      case 'warn':
-        score -= 5;
-        break;
-      case 'skipped':
-        score -= 2;
-        break;
-    }
-  });
-  
-  // Security findings reduce trust
-  findings.forEach(finding => {
-    switch (finding.severity) {
-      case 'critical':
-        score -= 25;
-        break;
-      case 'high':
-        score -= 15;
-        break;
-      case 'medium':
-        score -= 5;
-        break;
-      case 'low':
-        score -= 1;
-        break;
-    }
-  });
-  
-  // Performance budget violations
-  gates.forEach(gate => {
-    if (gate.budgets_json) {
-      const budgets = JSON.parse(gate.budgets_json);
-      const metrics = JSON.parse(gate.metrics_json || '{}');
-      
-      // Check budget violations
-      if (budgets.max_duration_ms && metrics.duration_ms > budgets.max_duration_ms) {
-        score -= 10;
-      }
-      if (budgets.min_coverage_percent && metrics.coverage_percent < budgets.min_coverage_percent) {
-        score -= 15;
-      }
-    }
-  });
-  
-  return Math.max(0, Math.min(100, score));
-}
-```
-
-### Trust Levels
-
-- **90-100%**: Excellent - Production ready
-- **80-89%**: Good - Minor issues, acceptable for release
-- **70-79%**: Fair - Some concerns, review required
-- **60-69%**: Poor - Significant issues, not recommended
-- **0-59%**: Critical - Major problems, do not release
-
-## Compliance & Auditing
-
-### Regulatory Requirements
-
-QA360 proofs support various compliance frameworks:
-
-- **SOC 2**: Evidence of security controls testing
-- **ISO 27001**: Information security management validation
-- **GDPR**: Data protection impact assessments
-- **HIPAA**: Healthcare application security verification
-- **PCI DSS**: Payment system security compliance
-
-### Audit Trail
-
-```bash
-# Generate audit report
-qa360 audit generate --from "2025-01-01" --to "2025-12-31"
-
-# Export compliance evidence
-qa360 audit export --standard soc2 --output ./compliance/
-
-# Verify audit trail integrity
-qa360 audit verify --all
-```
-
-### Evidence Package
-
-```
-compliance-package.zip
-├── executive-summary.pdf
-├── detailed-results/
-│   ├── security-testing.pdf
-│   ├── performance-testing.pdf
-│   └── accessibility-testing.pdf
-├── proofs/
-│   ├── proof-run-1.pdf
-│   ├── proof-run-2.pdf
-│   └── ...
-├── verification/
-│   ├── public-keys.pem
-│   ├── signatures.txt
-│   └── verification-instructions.md
-└── metadata/
-    ├── test-configurations.json
-    ├── environment-details.json
-    └── compliance-mapping.json
-```
-
-## Integration Examples
-
-### CI/CD Pipeline
-
-```yaml
-# GitHub Actions
-name: QA360 Testing with Proof Generation
-
-on: [push, pull_request]
-
-jobs:
-  qa360-test:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v3
-      
-      - name: Setup QA360
-        run: |
-          npm install -g @qa360/cli
-          qa360 doctor --fix
-      
-      - name: Run Tests with Proof
-        run: |
-          qa360 run --pack ./qa360-pack.yml \
-                    --run-key "ci-${{ github.run_id }}" \
-                    --sign
-        env:
-          QA360_SIGNING_KEY: ${{ secrets.QA360_SIGNING_KEY }}
-      
-      - name: Export Proof
-        if: always()
-        run: |
-          RUN_ID=$(qa360 history list --json --limit 1 | jq -r '.[0].id')
-          qa360 history export $RUN_ID --bundle ./proof-bundle.zip
-      
-      - name: Upload Proof
-        if: always()
-        uses: actions/upload-artifact@v3
-        with:
-          name: qa360-proof
-          path: proof-bundle.zip
-      
-      - name: Verify Proof
-        run: |
-          RUN_ID=$(qa360 history list --json --limit 1 | jq -r '.[0].id')
-          qa360 verify $RUN_ID
-```
-
-### Release Process
-
-```bash
-#!/bin/bash
-# release-with-proof.sh
-
-VERSION=$1
-RUN_KEY="release-$VERSION"
-
-echo "🚀 Starting QA360 testing for release $VERSION"
-
-# Run comprehensive tests
-qa360 run --pack ./release-pack.yml \
-          --run-key "$RUN_KEY" \
-          --sign
-
-# Get run ID
-RUN_ID=$(qa360 history list --json --limit 1 | jq -r '.[0].id')
-
-# Verify proof
-if qa360 verify $RUN_ID; then
-  echo "✅ Proof verified successfully"
-else
-  echo "❌ Proof verification failed"
-  exit 1
-fi
-
-# Pin release run
-qa360 history pin $RUN_ID
-
-# Export release evidence
-qa360 history export $RUN_ID --bundle "./releases/$VERSION-evidence.zip"
-
-echo "📦 Release $VERSION evidence package created"
-echo "🔗 Run ID: $RUN_ID"
-echo "📄 Proof bundle: ./releases/$VERSION-evidence.zip"
-```
-
-### Compliance Automation
-
-```bash
-#!/bin/bash
-# compliance-check.sh
-
-COMPLIANCE_STANDARD=$1  # soc2, iso27001, etc.
-PERIOD_START=$2
-PERIOD_END=$3
-
-echo "📋 Generating compliance report for $COMPLIANCE_STANDARD"
-
-# Get all runs in period
-qa360 history list --since "$PERIOD_START" \
-                   --json > runs-$PERIOD_START-$PERIOD_END.json
-
-# Verify all proofs
-FAILED_VERIFICATIONS=0
-while read -r run_id; do
-  if ! qa360 verify "$run_id"; then
-    echo "❌ Verification failed for run: $run_id"
-    ((FAILED_VERIFICATIONS++))
-  fi
-done < <(jq -r '.[].id' runs-$PERIOD_START-$PERIOD_END.json)
-
-if [ $FAILED_VERIFICATIONS -eq 0 ]; then
-  echo "✅ All proofs verified successfully"
-  
-  # Generate compliance package
-  qa360 audit generate --standard "$COMPLIANCE_STANDARD" \
-                       --from "$PERIOD_START" \
-                       --to "$PERIOD_END" \
-                       --output "./compliance/$COMPLIANCE_STANDARD-$(date +%Y%m%d).zip"
-  
-  echo "📦 Compliance package generated"
-else
-  echo "❌ $FAILED_VERIFICATIONS proof verification failures"
-  exit 1
-fi
-```
-
-## Security Considerations
-
-### Key Security
-
-- Store private keys with restricted permissions (600)
-- Use hardware security modules (HSM) for production
-- Implement key rotation policies
-- Backup keys securely and separately from proofs
-
-### Proof Integrity
-
-- Verify signatures before trusting proof data
-- Check proof timestamps for replay attacks
-- Validate pack hashes for configuration integrity
-- Cross-reference vault data with proof claims
-
-### Distribution Security
-
-- Use secure channels for proof distribution
-- Implement access controls for proof archives
-- Log all proof access and verification attempts
-- Consider proof encryption for sensitive environments
-
-## Troubleshooting
-
-### Common Issues
-
-1. **Signature verification fails**
-   - Check key pair consistency
-   - Verify proof data integrity
-   - Ensure correct algorithm (Ed25519)
-
-2. **Missing proof artifacts**
-   - Check vault integrity
-   - Verify CAS file existence
-   - Run garbage collection analysis
-
-3. **Invalid trust scores**
-   - Review gate results and findings
-   - Check budget configurations
-   - Validate scoring algorithm
-
-### Diagnostic Commands
-
-```bash
-# Check key integrity
-qa360 keys verify
-
-# Validate proof structure
-qa360 verify --validate-only <runId>
-
-# Debug signature process
-qa360 verify --debug <runId>
-
-# Check vault consistency
-qa360 doctor --vault --proofs
-```
-
-## Best Practices
-
-### Proof Generation
-
-1. **Consistent environments**: Use reproducible test environments
-2. **Comprehensive coverage**: Include all relevant test gates
-3. **Meaningful run keys**: Use descriptive, unique identifiers
-4. **Regular verification**: Verify proofs immediately after generation
-
-### Key Management
-
-1. **Secure storage**: Protect private keys with appropriate access controls
-2. **Regular rotation**: Implement key rotation schedules
-3. **Backup strategy**: Maintain secure key backups
-4. **Access logging**: Log all key usage and access
-
-### Compliance
-
-1. **Documentation**: Maintain clear proof generation procedures
-2. **Retention policies**: Define proof retention and disposal policies
-3. **Regular audits**: Verify proof integrity and compliance
-4. **Training**: Ensure team understands proof verification process
-
-## Future Enhancements
-
-### Planned Features
-
-- **Multi-signature support**: Require multiple signatures for critical proofs
-- **Timestamping service**: RFC 3161 timestamp integration
-- **Blockchain anchoring**: Immutable proof registration
-- **Hardware security**: HSM and smart card integration
-- **Zero-knowledge proofs**: Privacy-preserving verification
-
-### Integration Roadmap
-
-- **PKI integration**: Enterprise certificate authority support
-- **Cloud KMS**: AWS KMS, Azure Key Vault, Google Cloud KMS
-- **Compliance frameworks**: Additional standards support
-- **Verification services**: Hosted verification endpoints