qa360 2.1.7 → 2.2.1

package/docs/serve.md

@@ -0,0 +1,494 @@
+# QA360 Serve - Serveur d'Observabilité
+
+Le module `serve` de QA360 Core fournit un serveur HTTP léger d'observabilité et de contrôle pour le runner QA360.
+
+## 🎯 Objectifs
+
+- **Readiness rapide** : Vérification de l'état du système pour CI/CD
+- **Diagnostics actionnables** : Informations de debug avec solutions
+- **Métriques Prometheus** : Exposition des métriques au format OpenMetrics
+- **Arrêt propre** : Cancel safe des sous-processus sans zombies
+
+## 🚀 Démarrage Rapide
+
+### Démarrer le serveur
+
+```bash
+# Serveur basique sur port 4060
+qa360 serve
+
+# Configuration avancée
+qa360 serve --port 8080 --host 0.0.0.0 --verbose
+
+# Désactiver les métriques
+qa360 serve --no-metrics
+
+# Check de readiness pour CI
+qa360 serve --readiness-check
+```
+
+### Endpoints disponibles
+
+| Endpoint | Méthode | Description |
+|----------|---------|-------------|
+| `/health` | GET | Liveness & readiness checks |
+| `/diag` | GET | Diagnostic structuré avec solutions |
+| `/metrics` | GET | Métriques OpenMetrics/Prometheus |
+| `/cancel/:runId` | POST | Arrêt propre d'un run |
+
+## 📊 Endpoints Détaillés
+
+### GET /health
+
+Retourne l'état de santé du système avec checks de readiness.
+
+**Réponse exemple :**
+```json
+{
+  "status": "ok",
+  "uptime_s": 123,
+  "version": "0.9.0-core",
+  "commit": "abc1234",
+  "env": {
+    "node": "18.20.0",
+    "platform": "darwin"
+  },
+  "checks": {
+    "config": "ok",
+    "tools": {
+      "playwright": "ok",
+      "k6": "missing",
+      "zap": "ok"
+    },
+    "secrets": "ok",
+    "disk_free_mb": 2048,
+    "tmp_writable": true
+  },
+  "readiness": {
+    "ready": true,
+    "reason": null
+  }
+}
+```
+
+**Codes d'erreur :**
+- `HLT001` : Répertoire temporaire non accessible en écriture
+- `HLT002` : Erreur de configuration (.qa360 manquant)
+- `HLT003` : Espace disque insuffisant (< 100MB)
+
+### GET /diag
+
+Diagnostic actionnable avec problèmes détectés et solutions.
+
+**Réponse exemple :**
+```json
+{
+  "problems": [
+    {
+      "code": "DOC201",
+      "title": "k6 not found",
+      "fix": "qa360 doctor --fix tools=k6"
+    },
+    {
+      "code": "SEC101",
+      "title": "secrets file missing",
+      "fix": "qa360 secrets add API_TOKEN"
+    }
+  ],
+  "suggestions": [
+    "Enable Docker fallback: qa360 run --docker",
+    "Reduce perf budget p95_ms to 900 for demo profile"
+  ],
+  "last_runs": [
+    {
+      "runId": "run-abc123",
+      "trust": 92,
+      "ts": "2024-01-15T10:30:00Z",
+      "status": "success"
+    }
+  ]
+}
+```
+
+### GET /metrics
+
+Métriques au format OpenMetrics/Prometheus.
+
+**Exemple de sortie :**
+```
+# HELP qa360_uptime_seconds QA360 server uptime in seconds
+# TYPE qa360_uptime_seconds counter
+qa360_uptime_seconds 1234
+
+# HELP qa360_runs_total Total number of QA360 runs
+# TYPE qa360_runs_total counter
+qa360_runs_total 42
+
+# HELP qa360_last_run_trust_score Trust score of last run
+# TYPE qa360_last_run_trust_score gauge
+qa360_last_run_trust_score 93
+
+# HELP qa360_gate_duration_ms Duration of gate execution in milliseconds
+# TYPE qa360_gate_duration_ms gauge
+qa360_gate_duration_ms{gate="api"} 1530
+qa360_gate_duration_ms{gate="ui"} 2100
+
+# HELP qa360_security_findings Number of security findings by type
+# TYPE qa360_security_findings gauge
+qa360_security_findings{kind="sast_high"} 0
+qa360_security_findings{kind="dast_high"} 0
+
+# HELP qa360_perf_p95_ms Performance P95 latency in milliseconds
+# TYPE qa360_perf_p95_ms gauge
+qa360_perf_p95_ms 710
+```
+
+### POST /cancel/:runId
+
+Arrêt gracieux d'un run en cours.
+
+**Réponse exemple :**
+```json
+{
+  "cancelled": true,
+  "runId": "run-abc123"
+}
+```
+
+## 🔧 Configuration
+
+### Options CLI
+
+```bash
+qa360 serve [options]
+
+Options:
+  -p, --port <port>      Server port (default: 4060)
+  -h, --host <host>      Server host (default: 127.0.0.1)
+  --no-metrics          Disable metrics endpoint
+  --readiness-check     Perform readiness check and exit
+  -v, --verbose         Verbose logging
+```
+
+### Variables d'environnement
+
+```bash
+# Configuration serveur
+QA360_SERVE_PORT=4060
+QA360_SERVE_HOST=127.0.0.1
+QA360_SERVE_METRICS=true
+
+# Pour les checks
+QA360_CONFIG_DIR=.qa360
+QA360_SECRETS_FILE=secrets.json
+```
+
+## 🔍 Readiness Check pour CI
+
+Le mode `--readiness-check` permet de valider l'état du système avant d'exécuter des tests.
+
+```bash
+# Check de readiness
+qa360 serve --readiness-check
+
+# Exit codes:
+# 0 = Ready (système prêt)
+# 1 = Not ready (problème détecté)
+```
+
+**Exemple d'utilisation en CI :**
+
+```yaml
+# GitHub Actions
+- name: QA360 Readiness Check
+  run: qa360 serve --readiness-check
+  
+- name: Run Tests
+  if: success()
+  run: qa360 run pack.yml
+```
+
+## 🐳 Intégration Docker
+
+```dockerfile
+# Dockerfile
+FROM node:18-alpine
+RUN npm install -g @qa360/cli
+EXPOSE 4060
+HEALTHCHECK --interval=30s --timeout=3s \
+  CMD qa360 serve --readiness-check || exit 1
+CMD ["qa360", "serve", "--host", "0.0.0.0"]
+```
+
+## 📈 Intégration Prometheus
+
+### Configuration Prometheus
+
+```yaml
+# prometheus.yml
+scrape_configs:
+  - job_name: 'qa360'
+    static_configs:
+      - targets: ['localhost:4060']
+    metrics_path: '/metrics'
+    scrape_interval: 30s
+```
+
+### Alertes recommandées
+
+```yaml
+# alerts.yml
+groups:
+  - name: qa360
+    rules:
+      - alert: QA360Down
+        expr: up{job="qa360"} == 0
+        for: 1m
+        labels:
+          severity: critical
+        annotations:
+          summary: "QA360 server is down"
+          
+      - alert: QA360TrustScoreLow
+        expr: qa360_last_run_trust_score < 70
+        for: 5m
+        labels:
+          severity: warning
+        annotations:
+          summary: "QA360 trust score is low ({{ $value }})"
+```
+
+## 🔄 Graceful Shutdown
+
+Le serveur gère proprement l'arrêt des processus :
+
+1. **SIGTERM** envoyé aux processus enfants
+2. **Délai de grâce** de 10 secondes
+3. **SIGKILL** si nécessaire
+4. **Nettoyage Docker** (compose down)
+
+```javascript
+// Exemple d'intégration
+process.on('SIGTERM', async () => {
+  console.log('Shutting down gracefully...');
+  await server.stop();
+  process.exit(0);
+});
+```
+
+## 🧪 Tests et Validation
+
+### Tests unitaires
+
+```bash
+# Tests du module serve
+npm test -- --testPathPattern="serve"
+
+# Tests d'intégration
+npm test -- --testPathPattern="integration"
+```
+
+### Tests manuels
+
+```bash
+# Démarrer le serveur
+qa360 serve --port 4061 --verbose &
+SERVER_PID=$!
+
+# Tester les endpoints
+curl http://localhost:4061/health
+curl http://localhost:4061/diag
+curl http://localhost:4061/metrics
+
+# Tester le cancel
+curl -X POST http://localhost:4061/cancel/test-run-123
+
+# Arrêter le serveur
+kill $SERVER_PID
+```
+
+## 📋 Exemples CI/CD
+
+### GitHub Actions
+
+```yaml
+name: QA360 with Serve
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: '18'
+          
+      - name: Install QA360
+        run: npm install -g @qa360/cli
+        
+      - name: QA360 Readiness Check
+        run: qa360 serve --readiness-check
+        
+      - name: Start QA360 Server
+        run: |
+          qa360 serve --port 4060 &
+          echo $! > qa360.pid
+          
+      - name: Wait for server
+        run: |
+          timeout 30 bash -c 'until curl -f http://localhost:4060/health; do sleep 1; done'
+          
+      - name: Run Tests
+        run: qa360 run pack.yml
+        
+      - name: Get Metrics
+        run: curl http://localhost:4060/metrics
+        
+      - name: Stop Server
+        run: |
+          if [ -f qa360.pid ]; then
+            kill $(cat qa360.pid)
+          fi
+```
+
+### GitLab CI
+
+```yaml
+# .gitlab-ci.yml
+stages:
+  - readiness
+  - test
+  - metrics
+
+readiness:
+  stage: readiness
+  script:
+    - npm install -g @qa360/cli
+    - qa360 serve --readiness-check
+  only:
+    - merge_requests
+    - main
+
+qa360_test:
+  stage: test
+  script:
+    - qa360 serve --port 4060 &
+    - SERVER_PID=$!
+    - sleep 5
+    - qa360 run pack.yml
+    - kill $SERVER_PID
+  artifacts:
+    reports:
+      junit: qa360-results.xml
+    paths:
+      - qa360-report.html
+    expire_in: 1 week
+
+metrics:
+  stage: metrics
+  script:
+    - qa360 serve --port 4060 &
+    - sleep 2
+    - curl http://localhost:4060/metrics > qa360-metrics.txt
+  artifacts:
+    paths:
+      - qa360-metrics.txt
+```
+
+### Jenkins Pipeline
+
+```groovy
+pipeline {
+    agent any
+    
+    stages {
+        stage('Readiness') {
+            steps {
+                sh 'npm install -g @qa360/cli'
+                sh 'qa360 serve --readiness-check'
+            }
+        }
+        
+        stage('Test') {
+            steps {
+                script {
+                    sh 'qa360 serve --port 4060 &'
+                    env.SERVER_PID = sh(
+                        script: 'echo $!',
+                        returnStdout: true
+                    ).trim()
+                }
+                
+                sh 'sleep 5'
+                sh 'qa360 run pack.yml'
+            }
+            
+            post {
+                always {
+                    script {
+                        if (env.SERVER_PID) {
+                            sh "kill ${env.SERVER_PID} || true"
+                        }
+                    }
+                }
+            }
+        }
+    }
+}
+```
+
+## 🔧 Troubleshooting
+
+### Problèmes courants
+
+**Port déjà utilisé :**
+```bash
+# Changer le port
+qa360 serve --port 4061
+
+# Ou tuer le processus existant
+lsof -ti:4060 | xargs kill -9
+```
+
+**Permissions insuffisantes :**
+```bash
+# Vérifier les permissions
+qa360 doctor --fix
+
+# Ou utiliser un port non privilégié
+qa360 serve --port 8080
+```
+
+**Outils manquants :**
+```bash
+# Diagnostic complet
+curl http://localhost:4060/diag
+
+# Auto-fix
+qa360 doctor --fix
+```
+
+### Logs de debug
+
+```bash
+# Mode verbose
+qa360 serve --verbose
+
+# Logs JSON structurés
+qa360 serve --verbose 2>&1 | jq .
+```
+
+## 🎯 Bonnes Pratiques
+
+1. **Utilisez --readiness-check** en CI avant les tests
+2. **Configurez des alertes** Prometheus pour la surveillance
+3. **Testez le graceful shutdown** dans vos pipelines
+4. **Surveillez les métriques** de performance et trust score
+5. **Utilisez des ports non privilégiés** (> 1024)
+6. **Configurez des timeouts** appropriés en CI
+7. **Archivez les métriques** pour l'analyse historique
+
+Le serveur QA360 Serve fournit une observabilité complète et un contrôle fin pour l'intégration en environnements de production et CI/CD.

package/docs/unit-test-adapters.md

@@ -0,0 +1,168 @@
+# Unit Test Adapters
+
+QA360 v2.2 introduces support for ingesting unit test results from popular testing frameworks. This enables you to track unit test metrics alongside your E2E and integration tests.
+
+## Supported Frameworks
+
+| Framework  | Result Format      | Coverage Format      | Status |
+|-----------|--------------------|---------------------|--------|
+| Vitest    | JSON (`--reporter=json`) | `coverage/coverage-final.json` | ✅ Supported |
+| Jest      | JSON (`--json`)    | `coverage/coverage-summary.json` | ✅ Supported |
+| pytest    | JUnit XML (`--junitxml`) | coverage.py XML | ✅ Supported |
+
+## Usage
+
+### Vitest
+
+```yaml
+# qa360.config.yml
+packs:
+  - name: unit-tests
+    adapters:
+      - name: vitest
+        config:
+          results: test-results/vitest-results.json
+          coverage: coverage/coverage-final.json
+          thresholds:
+            statements: 80
+            branches: 75
+            functions: 80
+            lines: 80
+```
+
+Generate results:
+```bash
+vitest run --reporter=json --outputFile=test-results/vitest-results.json
+vitest run --coverage
+```
+
+### Jest
+
+```yaml
+# qa360.config.yml
+packs:
+  - name: unit-tests
+    adapters:
+      - name: jest
+        config:
+          results: test-results/jest-results.json
+          coverage: coverage/coverage-summary.json
+          thresholds:
+            statements: 80
+            branches: 75
+            functions: 80
+            lines: 80
+```
+
+Generate results:
+```bash
+jest --json --outputFile=test-results/jest-results.json
+jest --coverage
+```
+
+### pytest
+
+```yaml
+# qa360.config.yml
+packs:
+  - name: unit-tests
+    adapters:
+      - name: pytest
+        config:
+          results: test-results/junit.xml
+          coverage: coverage.xml
+          thresholds:
+            statements: 80
+            branches: 75
+            functions: 80
+            lines: 80
+```
+
+Generate results:
+```bash
+pytest --junitxml=test-results/junit.xml
+pytest --cov=src --cov-report=xml
+```
+
+## API Usage
+
+You can also use the adapters directly in your code:
+
+```typescript
+import { VitestAdapter, JestAdapter, PytestAdapter } from 'qa360-core';
+
+// Vitest
+const vitest = new VitestAdapter(process.cwd());
+const result = await vitest.execute({
+  results: 'test-results/vitest-results.json',
+  coverage: 'coverage/coverage-final.json',
+  thresholds: { statements: 80 }
+});
+
+console.log(`Passed: ${result.passed}/${result.total}`);
+console.log(`Coverage: ${result.coverage?.statements.percentage}%`);
+```
+
+## Result Structure
+
+All adapters return a consistent `UnitTestExecutionResult`:
+
+```typescript
+interface UnitTestExecutionResult {
+  adapter: string;           // 'vitest' | 'jest' | 'pytest'
+  total: number;             // Total tests
+  passed: number;            // Passed tests
+  failed: number;            // Failed tests
+  skipped: number;           // Skipped tests
+  pending: number;           // Pending tests
+  duration: number;          // Total duration (ms)
+  tests: UnitTestResult[];   // Individual test results
+  coverage?: CoverageStats;  // Coverage metrics (if available)
+  thresholdResults?: {       // Threshold pass/fail status
+    statements: boolean;
+    branches: boolean;
+    functions: boolean;
+    lines: boolean;
+  };
+  success: boolean;          // Overall success
+}
+```
+
+## Coverage Thresholds
+
+Thresholds can be enforced for any coverage metric. If a threshold is not met, the overall result will be marked as failed:
+
+```yaml
+thresholds:
+  statements: 80   # 80% statement coverage required
+  branches: 75     # 75% branch coverage required
+  functions: 80    # 80% function coverage required
+  lines: 80        # 80% line coverage required
+```
+
+## Test Distribution Score
+
+Unit tests contribute to the Test Pyramid Analysis. The ideal distribution is:
+- Unit Tests: 70%
+- Integration Tests: 20%
+- E2E Tests: 10%
+
+Run the pyramid analysis to see your distribution:
+```bash
+qa360 analyze --pyramid
+```
+
+## Future Enhancements
+
+Planned support for:
+- Go testing (`go test -json`)
+- JUnit (Java)
+- NUnit / xUnit (.NET)
+- RSpec (Ruby)
+- JUnit XML format (universal)
+
+## See Also
+
+- [Test Pyramid Philosophy](QA360_TESTING_PHILOSOPHY.md)
+- [Pack v2 Configuration](pack-v2.md)
+- [Coverage Analytics](../core/src/coverage/README.md)