qa360 2.1.2 → 2.1.4

package/docs/archive/TEST-01_COVERAGE_PLAN.md

@@ -0,0 +1,423 @@
+# 📊 TEST-01 COVERAGE PLAN - DAY 3
+
+**Date**: December 23, 2024  
+**Current Status**: Baseline established  
+**Target**: 80% overall coverage
+
+---
+
+## 📈 CURRENT COVERAGE BASELINE
+
+### Overall Metrics
+- **Statements**: 8.54%
+- **Branches**: 4.75%
+- **Functions**: 9.05%
+- **Lines**: 8.62%
+
+**Status**: 🔴 Critical - Needs significant work
+
+### By Package Analysis
+
+| Package | Statements | Branches | Functions | Lines | Status |
+|---------|-----------|----------|-----------|-------|--------|
+| **core** | ~5-10% | ~2-5% | ~5-10% | ~5-10% | 🔴 CRITICAL |
+| **cli** | ~10-15% | ~5-10% | ~10-15% | ~10-15% | 🔴 CRITICAL |
+| **mcp** | N/A | N/A | N/A | N/A | ⚪ Not tested |
+
+Legend:
+- 🔴 <60% - Needs significant work (CURRENT STATE)
+- 🟡 60-75% - Needs improvement  
+- 🟢 >75% - Good baseline
+
+### Coverage Highlights
+**What IS tested (195 passing tests):**
+- ✅ CLI doctor command (QA360Doctor)
+- ✅ Proof system (bundle, schema, canonicalize) - **63% coverage**
+- ✅ CLI utilities and helpers
+- ✅ Configuration loading
+
+**What is NOT tested (0% coverage):**
+- ❌ All adapters (Playwright, k6, Semgrep, ZAP, OSV, Gitleaks)
+- ❌ Evidence Vault (sqlite3)
+- ❌ Secrets management (crypto, manager)
+- ❌ Security redactor
+- ❌ Server/observability endpoints
+- ❌ Hooks runner & compose
+- ❌ Pack validation & migration
+- ❌ CLI commands (run, verify, serve, report, secrets, ask)
+
+---
+
+## 🎯 COVERAGE TARGETS
+
+### Day 4 Goals (Aggressive but Achievable)
+| Package | Current | Day 4 Target | Gap | Priority |
+|---------|---------|--------------|-----|----------|
+| **core** | ~8% | 60% | +52% | 🔥 CRITICAL |
+| **cli** | ~12% | 50% | +38% | 🔥 HIGH |
+| **mcp** | 0% | 30% | +30% | 🟡 MEDIUM |
+| **Overall** | 8.54% | 55% | +46.5% | 🔥 CRITICAL |
+
+### Week 2 Final Targets
+- Core: 80%
+- CLI: 75%
+- MCP: 70%
+- Overall: 75%
+
+**Rationale**: Day 4 establishes foundation (55%), Day 5 pushes to target (75%)
+
+---
+
+## 🚨 CRITICAL GAPS (0% Coverage)
+
+### Core Package - ZERO Coverage Files
+1. **core/src/adapters/** (ALL 7 adapters - 0%)
+   - `playwright-api.ts` (328 lines) - API testing adapter
+   - `playwright-ui.ts` (445 lines) - UI testing adapter
+   - `k6-perf.ts` (474 lines) - Performance testing
+   - `semgrep-sast.ts` (405 lines) - SAST scanning
+   - `gitleaks-secrets.ts` (514 lines) - Secrets scanning
+   - `zap-dast.ts` (543 lines) - DAST scanning
+   - `osv-deps.ts` (460 lines) - Dependency scanning
+
+2. **core/src/vault/** (Evidence storage - 0%)
+   - `index.ts` (663 lines) - SQLite vault operations
+   - `cas.ts` (314 lines) - Content-addressable storage
+
+3. **core/src/secrets/** (0%)
+   - `crypto.ts` (293 lines) - AES-256 encryption
+   - `manager.ts` (269 lines) - Secrets CRUD
+
+4. **core/src/security/** (0%)
+   - `redactor.ts` (319 lines) - PII masking
+   - `redaction-patterns-extended.ts` (277 lines)
+
+5. **core/src/serve/** (Observability - 0%)
+   - `server.ts` (228 lines) - HTTP server
+   - `health-checker.ts` (271 lines) - Health checks
+   - `diagnostics-collector.ts` (199 lines)
+   - `metrics-collector.ts` (383 lines)
+   - `process-manager.ts` (263 lines)
+
+6. **core/src/hooks/** (0%)
+   - `runner.ts` (362 lines) - Hook execution
+   - `compose.ts` (266 lines) - Docker Compose
+
+7. **core/src/pack/** (0%)
+   - `validator.ts` (321 lines) - Pack validation
+   - `migrator.ts` (351 lines) - Pack migration
+
+8. **core/src/runner/** (0%)
+   - `phase3-runner.ts` (454 lines) - Advanced orchestration
+
+9. **core/src/types/** (0%)
+   - `trust-score.ts` (253 lines) - Trust calculations
+
+### CLI Package - ZERO Coverage Commands
+1. **cli/src/commands/** (Most commands 0%)
+   - `run.ts` (379 lines) - Main execution command
+   - `verify.ts` (284 lines) - Proof verification
+   - `serve.ts` (192 lines) - Server command
+   - `report.ts` (440 lines) - Report generation
+   - `secrets.ts` (328 lines) - Secrets management
+   - `ask.ts` (356 lines) - AI test generation
+   - `history.ts` (387 lines) - Run history
+
+**Total untested files: ~40+ files with 0% coverage**
+
+---
+
+## ⚠️ LOW COVERAGE (<50%)
+
+### Files Needing Significant Testing
+1. **core/src/proof/signer.ts** (35.8%)
+   - Sign operations: 64% uncovered
+   - Ed25519 signature generation
+   - Private key operations
+
+2. **core/src/proof/verifier.ts** (43.05%)
+   - Verify operations: 57% uncovered
+   - Signature validation
+   - Certificate chain verification
+
+3. **core/src/index.ts** (0% - exports only)
+   - Main package exports
+   - Low priority (mostly re-exports)
+
+---
+
+## 📋 DAY 4 ACTION PLAN
+
+### Phase 1: Core Foundations (4 hours)
+
+#### High Priority - Critical Path Testing
+- [ ] **Proof System** (90 min)
+  - Complete signer.ts coverage (35% → 80%)
+  - Complete verifier.ts coverage (43% → 80%)
+  - Target: Proof system to 75% overall
+
+- [ ] **Vault Operations** (60 min)
+  - Test vault CRUD (store, retrieve, list)
+  - Test CAS operations
+  - Mock sqlite3 appropriately
+  - Target: vault/ to 60%
+
+- [ ] **Secrets Management** (45 min)
+  - Test crypto encrypt/decrypt
+  - Test manager CRUD operations
+  - Target: secrets/ to 60%
+
+- [ ] **Pack Validation** (45 min)
+  - Test validator.ts (schema checks)
+  - Test basic validation flows
+  - Target: pack/ to 50%
+
+### Phase 2: CLI Commands (3 hours)
+
+#### Medium Priority - User-Facing Features
+- [ ] **doctor command** (30 min)
+  - Already well tested (✅)
+  - Add edge case tests
+  - Target: 90%+
+
+- [ ] **run command** (60 min)
+  - Test basic execution flow
+  - Mock adapters
+  - Test error handling
+  - Target: 50%
+
+- [ ] **verify command** (45 min)
+  - Test proof verification flow
+  - Integration with proof system
+  - Target: 50%
+
+- [ ] **serve command** (45 min)
+  - Test server startup
+  - Mock endpoints
+  - Target: 40%
+
+### Phase 3: Quick Wins (2 hours)
+
+#### Low Priority - Easy Coverage Boosts
+- [ ] **Security Redactor** (30 min)
+  - Test pattern matching
+  - Test redaction logic
+  - Target: 60%
+
+- [ ] **Adapters Smoke Tests** (60 min)
+  - Basic instantiation tests for all 7
+  - Mock external tools
+  - Target: 20% each (just structure)
+
+- [ ] **Type Utilities** (30 min)
+  - Test trust-score calculations
+  - Pure logic, easy to test
+  - Target: 70%
+
+---
+
+## 🎯 TESTING STRATEGY
+
+### Unit Tests Focus
+**Pure Logic (Easy):**
+- ✅ Proof canonicalization (already 90%)
+- ✅ Proof schema validation (already 96%)
+- 🎯 Trust score calculations
+- 🎯 Security redaction patterns
+- 🎯 Pack validation rules
+
+**Core Operations (Medium):**
+- 🎯 Vault CRUD (mock sqlite3)
+- 🎯 Secrets encrypt/decrypt (real crypto)
+- 🎯 Proof sign/verify (real Ed25519)
+- 🎯 Pack migration logic
+
+**Integrations (Hard):**
+- 🎯 CLI command flows
+- 🎯 Adapter orchestration
+- 🎯 Server endpoints
+
+### Integration Tests
+**Critical Paths:**
+- 🎯 Full run → vault → report flow
+- 🎯 Sign → verify → bundle flow
+- 🎯 Doctor → fix → validate flow
+
+### Mock Strategy
+**What to Mock:**
+- ✅ File system operations (fs.readFile, writeFile)
+- ✅ External tools (playwright, k6, zap, etc.)
+- ✅ Network calls (HTTP requests)
+- ✅ Child processes (spawn, exec)
+
+**What to Keep Real:**
+- ✅ Crypto operations (Ed25519, AES-256)
+- ✅ SQLite in-memory (use :memory:)
+- ✅ Core logic (proof, validation)
+
+---
+
+## 🚀 QUICK WINS FOR DAY 4
+
+### 1. Proof System (Already 63% - Easy Push)
+**File**: `core/src/proof/signer.ts`  
+**Effort**: 30 min  
+**Impact**: +10% overall  
+**Why**: Core functionality, pure logic, already partially tested
+
+### 2. Proof System (Part 2)
+**File**: `core/src/proof/verifier.ts`  
+**Effort**: 30 min  
+**Impact**: +10% overall  
+**Why**: Completes proof system to 80%+
+
+### 3. Security Redactor
+**File**: `core/src/security/redactor.ts`  
+**Effort**: 30 min  
+**Impact**: +5% overall  
+**Why**: Pure string manipulation, easy test cases
+
+### 4. Trust Score Calculations
+**File**: `core/src/types/trust-score.ts`  
+**Effort**: 30 min  
+**Impact**: +3% overall  
+**Why**: Pure math, no dependencies
+
+### 5. Pack Validator Basics
+**File**: `core/src/pack/validator.ts`  
+**Effort**: 45 min  
+**Impact**: +5% overall  
+**Why**: Schema validation, clear test cases
+
+### 6. Vault Smoke Tests
+**File**: `core/src/vault/index.ts`  
+**Effort**: 45 min  
+**Impact**: +8% overall  
+**Why**: Critical path, use in-memory sqlite
+
+### 7. Secrets Crypto
+**File**: `core/src/secrets/crypto.ts`  
+**Effort**: 30 min  
+**Impact**: +4% overall  
+**Why**: Encryption/decryption, testable
+
+### 8. CLI Run Command Basics
+**File**: `cli/src/commands/run.ts`  
+**Effort**: 45 min  
+**Impact**: +6% overall  
+**Why**: Main user command, high visibility
+
+**Total Quick Wins**: 51% potential coverage gain in ~5 hours
+
+---
+
+## 🎯 SUCCESS CRITERIA (DAY 4)
+
+### Primary Goals
+- [ ] **Overall coverage: 55%+** (current 8.54%)
+- [ ] **Core proof system: 75%+** (current 63%)
+- [ ] **Vault operations: 60%+** (current 0%)
+- [ ] **Secrets management: 60%+** (current 0%)
+- [ ] **CLI doctor: 90%+** (current ~80%)
+
+### Secondary Goals
+- [ ] All critical paths have at least smoke tests
+- [ ] No 0% coverage in core proof/vault/secrets
+- [ ] Pack validation basics covered
+- [ ] CLI run command basics covered
+
+### Stretch Goals
+- [ ] Overall coverage: 60%+
+- [ ] 2+ adapters with smoke tests (20%+)
+- [ ] Server health checks tested
+
+---
+
+## 📊 ESTIMATED EFFORT
+
+### By Area
+| Area | Tests Needed | Est. Time | Impact |
+|------|--------------|-----------|--------|
+| **Proof completion** | 10 tests | 1 hour | +20% |
+| **Vault basics** | 15 tests | 1.5 hours | +8% |
+| **Secrets basics** | 10 tests | 1 hour | +4% |
+| **Pack validation** | 8 tests | 1 hour | +5% |
+| **CLI commands** | 20 tests | 2 hours | +10% |
+| **Quick wins** | 15 tests | 1.5 hours | +8% |
+
+**Total Day 4**: ~60-70 new tests, 8 hours, targeting 55% coverage
+
+---
+
+## 📝 TESTING PRIORITIES
+
+### 🔥 P0 - Must Have (Day 4)
+1. Proof signer/verifier completion
+2. Vault CRUD operations
+3. Secrets encryption
+4. CLI run command basics
+
+### 🔶 P1 - Should Have (Day 4-5)
+1. Pack validation
+2. Security redactor
+3. Trust score calculations
+4. CLI verify command
+5. Adapter smoke tests
+
+### 🟡 P2 - Nice to Have (Day 5+)
+1. Server endpoints
+2. Hooks runner
+3. Process manager
+4. CLI report generation
+5. Full adapter coverage
+
+---
+
+## 🎓 LESSONS LEARNED
+
+### What Worked Well
+- ✅ Doctor command has good test coverage
+- ✅ Proof system partially covered (63%)
+- ✅ Test infrastructure stable (195/195 passing)
+
+### What Needs Attention
+- ⚠️ 40+ files with 0% coverage
+- ⚠️ Adapters completely untested
+- ⚠️ Vault/secrets (critical paths) untested
+- ⚠️ Most CLI commands untested
+
+### Key Insights
+1. **Focus on pure logic first** (proof, crypto, validation)
+2. **Mock external dependencies** (fs, network, tools)
+3. **Use in-memory sqlite** for vault tests
+4. **Test critical paths** (run → vault → report)
+5. **Quick wins matter** (8% → 55% is achievable in Day 4)
+
+---
+
+## 📈 COVERAGE ROADMAP
+
+### Day 4 Target: 55%
+- Proof: 75%
+- Vault: 60%
+- Secrets: 60%
+- Pack: 50%
+- CLI: 40%
+
+### Day 5 Target: 75%
+- Core: 80%
+- CLI: 75%
+- Adapters: 40%
+- Server: 60%
+
+### Week 2 Final: 80%
+- Core: 85%
+- CLI: 80%
+- MCP: 70%
+- Adapters: 60%
+
+---
+
+**Coverage baseline established by TEST-01 on Day 3**  
+**Status**: 🔴 8.54% → 🟢 Target 55% (Day 4)  
+**Ready for**: Aggressive coverage push on Day 4

package/docs/budgets-advanced.md

@@ -0,0 +1,308 @@
+# QA360 Phase 4 - Budgets Avancés & Trust Score
+
+Guide complet des budgets granulaires et du système de Trust Score pondéré.
+
+## 📊 Schéma Budgets Avancés
+
+### Structure Complète
+
+```yaml
+version: 1
+name: "enterprise-app"
+gates: ["ui", "api", "perf", "a11y", "sast", "deps", "secrets", "dast"]
+
+budgets:
+  perf:
+    p95_ms: 800          # P95 response time budget
+    p90_ms: 600          # P90 response time budget  
+    p50_ms: 300          # P50 response time budget
+    max_errors_rate: 0.1 # 10% max error rate
+  
+  a11y:
+    min_score: 90        # Minimum accessibility score
+    max_violations:
+      critical: 0        # Zero critical violations
+      serious: 2         # Max 2 serious violations
+      moderate: 5        # Max 5 moderate violations
+      minor: 10          # Max 10 minor violations
+
+security:
+  sast:                  # Static Application Security Testing
+    max_high: 0          # Zero high severity findings
+    max_critical: 0      # Zero critical findings
+    max_medium: 3        # Max 3 medium findings
+  
+  deps:                  # Dependency vulnerabilities
+    max_high: 0          # Zero high severity vulns
+    max_critical: 0      # Zero critical vulns
+    max_medium: 2        # Max 2 medium vulns
+  
+  secrets:               # Secret detection
+    max_findings: 0      # Zero secrets in code
+  
+  dast:                  # Dynamic Application Security Testing
+    max_high: 0          # Zero high severity alerts
+    max_critical: 0      # Zero critical alerts
+    max_medium: 1        # Max 1 medium alert
+    timeout_ms: 300000   # 5 minutes ZAP scan timeout
+
+execution:
+  on_failure: "continue" # continue | stop
+  mode: "strict"         # strict | soft
+```
+
+### Modes d'Exécution
+
+#### Mode Strict (Défaut)
+```yaml
+execution:
+  mode: "strict"
+```
+- Échec immédiat si budget dépassé
+- Exit code 1 pour CI/CD
+- Blocage du pipeline
+
+#### Mode Soft
+```yaml
+execution:
+  mode: "soft"
+```
+- Transformation en warnings
+- Dégradation du Trust Score
+- Pipeline continue
+
+## 🎯 Trust Score Pondéré
+
+### Pondération Par Défaut
+
+```typescript
+const DEFAULT_WEIGHTS = {
+  ui: 20,      // Tests UI/UX
+  api: 20,     // Tests API
+  perf: 15,    // Performance
+  a11y: 10,    // Accessibilité
+  sast: 15,    // Sécurité statique
+  deps: 10,    // Vulnérabilités dépendances
+  secrets: 5,  // Détection secrets
+  dast: 5      // Sécurité dynamique
+};
+```
+
+### Personnalisation Weights
+
+```bash
+# Via CLI
+qa360 run --url <url> --weights "ui:25,api:25,perf:20,sast:20,dast:10"
+
+# Via pack.yml
+weights:
+  ui: 25
+  api: 25
+  perf: 20
+  sast: 20
+  dast: 10
+```
+
+### Calcul Trust Score
+
+```
+Trust Score = Σ(Gate Score × Weight) / 100
+
+Exemple:
+- UI: 85% × 20% = 17.0
+- API: 90% × 20% = 18.0  
+- Perf: 75% × 15% = 11.25
+- A11y: 95% × 10% = 9.5
+- SAST: 100% × 15% = 15.0
+- Deps: 100% × 10% = 10.0
+- Secrets: 100% × 5% = 5.0
+- DAST: 90% × 5% = 4.5
+
+Trust Score = 90.25%
+```
+
+## 🔧 Exemples par Secteur
+
+### E-commerce
+```yaml
+budgets:
+  perf:
+    p95_ms: 500    # Performance critique
+    max_errors_rate: 0.01
+  a11y:
+    min_score: 95  # Accessibilité légale
+security:
+  sast:
+    max_high: 0    # Sécurité paiements
+  dast:
+    max_high: 0
+
+weights:
+  perf: 30       # Performance prioritaire
+  ui: 25
+  sast: 20       # Sécurité critique
+  api: 15
+  dast: 10
+```
+
+### SaaS B2B
+```yaml
+budgets:
+  perf:
+    p95_ms: 1000   # Tolérance plus élevée
+  a11y:
+    min_score: 85  # Standard business
+security:
+  sast:
+    max_high: 1    # Tolérance limitée
+  secrets:
+    max_findings: 0 # Zero secret
+
+weights:
+  api: 30        # API-first
+  ui: 25
+  sast: 20
+  perf: 15
+  deps: 10
+```
+
+### Application Interne
+```yaml
+budgets:
+  perf:
+    p95_ms: 2000   # Réseau interne
+  a11y:
+    min_score: 70  # Standard interne
+security:
+  sast:
+    max_high: 3    # Plus permissif
+  dast:
+    max_medium: 5
+
+weights:
+  ui: 40         # UX prioritaire
+  api: 30
+  perf: 20
+  sast: 10
+```
+
+## 📈 Recommandations Trust Score
+
+### Excellent (90-100%)
+- ✅ Prêt pour production
+- ✅ Qualité enterprise
+- ✅ Sécurité optimale
+
+### Good (80-89%)
+- ✅ Production acceptable
+- ⚠️ Améliorations mineures
+- ✅ Sécurité correcte
+
+### Acceptable (70-79%)
+- ⚠️ Production conditionnelle
+- 🔧 Améliorations requises
+- ⚠️ Révision sécurité
+
+### Needs Improvement (50-69%)
+- ❌ Production non recommandée
+- 🔧 Corrections majeures
+- 🔍 Audit sécurité
+
+### Critical (<50%)
+- 🚨 Blocage production
+- 🔧 Refactoring requis
+- 🚨 Failles critiques
+
+## 🛠️ Intégration CI/CD
+
+### GitHub Actions
+```yaml
+- name: QA360 Quality Gate
+  run: |
+    qa360 run --url ${{ env.STAGING_URL }} \
+      --mode strict \
+      --weights "perf:25,sast:25,ui:20,api:20,dast:10" \
+      --report-format json
+    
+    # Vérifier Trust Score
+    TRUST_SCORE=$(jq '.trustScore.weightedScore' qa360-report.json)
+    if [ $TRUST_SCORE -lt 80 ]; then
+      echo "Trust Score too low: $TRUST_SCORE%"
+      exit 1
+    fi
+```
+
+### GitLab CI
+```yaml
+qa360_quality_gate:
+  script:
+    - qa360 run --url $STAGING_URL --mode strict
+    - qa360 report --format badge --output trust-score.svg
+  artifacts:
+    reports:
+      junit: qa360-junit.xml
+    paths:
+      - trust-score.svg
+```
+
+## 🎨 Badges & Rapports
+
+### Badge Trust Score
+```markdown
+![Trust Score](https://img.shields.io/badge/Trust%20Score-85%25-green)
+```
+
+### Rapport Exécutif
+```bash
+qa360 report --format executive --output executive-report.pdf
+```
+
+### Dashboard Multi-Projets
+```bash
+qa360 dashboard --input "./projects/**/qa360-report.json" \
+  --output dashboard.html \
+  --weights "perf:30,sast:25,ui:25,api:20"
+```
+
+## 🔍 Monitoring & Alertes
+
+### Métriques Prometheus
+```
+qa360_trust_score{project="app",environment="prod"} 85
+qa360_gate_score{gate="perf",project="app"} 78
+qa360_budget_exceeded{gate="sast",severity="high"} 1
+```
+
+### Alertes Slack
+```yaml
+alerts:
+  trust_score_threshold: 80
+  webhook_url: "https://hooks.slack.com/..."
+  channels:
+    critical: "#security-alerts"
+    warning: "#qa-team"
+```
+
+## 📚 Bonnes Pratiques
+
+### 1. Définition Budgets
+- Commencer permissif, durcir progressivement
+- Adapter selon criticité métier
+- Réviser trimestriellement
+
+### 2. Pondération Weights
+- Aligner sur priorités business
+- Ajuster selon phase projet
+- Documenter les choix
+
+### 3. Monitoring Continu
+- Tracking évolution Trust Score
+- Alertes sur régressions
+- Rapports réguliers stakeholders
+
+### 4. Amélioration Continue
+- Analyse tendances
+- Identification goulots
+- Formation équipes
+
+Ce système de budgets avancés et Trust Score pondéré transforme QA360 en véritable plateforme de gouvernance qualité enterprise.