qa360 2.2.1 → 2.2.13

package/docs/rfc/proof-bundle-v1.md

@@ -1,787 +0,0 @@
-# RFC: QA360 Proof Bundle v1
-
-**Status**: Draft  
-**Version**: 1.0.0  
-**Date**: 2025-10-26  
-**Authors**: QA360 Core Team
-
----
-
-## Table of Contents
-
-1. [Motivation & Scope](#1-motivation--scope)
-2. [Terminology](#2-terminology)
-3. [Data Model](#3-data-model)
-4. [Canonicalization](#4-canonicalization)
-5. [Signature Procedure](#5-signature-procedure)
-6. [Verification Procedure](#6-verification-procedure)
-7. [JSON Schema](#7-json-schema)
-8. [Compatibility & Extensions](#8-compatibility--extensions)
-9. [Security Considerations](#9-security-considerations)
-10. [Examples](#10-examples)
-11. [Annexes](#11-annexes)
-
----
-
-## 1. Motivation & Scope
-
-### 1.1 Problem Statement
-
-Software quality testing generates artifacts (reports, metrics, logs) but lacks:
-- **Verifiable integrity**: Can results be tampered with?
-- **Portable proofs**: Can results be verified offline, cross-OS?
-- **Legal admissibility**: Can results serve as evidence in audits?
-
-### 1.2 Solution
-
-QA360 Proof Bundle v1 defines a **cryptographically signed, self-contained proof** of test execution that is:
-
-- ✅ **Local-first**: No cloud dependencies
-- ✅ **Cross-platform**: Same hash on Windows/macOS/Linux
-- ✅ **Verifiable offline**: Ed25519 signature + SHA-256 hashes
-- ✅ **Forward-compatible**: Reserved fields for RFC 3161 timestamps, DID/Sigstore identities
-
-### 1.3 Non-Goals (Phase 1)
-
-- ❌ RFC 3161 timestamp verification (Phase 2)
-- ❌ Sigstore/DID identity binding (Phase 2)
-- ❌ Multi-signature support (Phase 3)
-- ❌ Cloud storage/distribution (Phase 4)
-
----
-
-## 2. Terminology
-
-| Term | Definition |
-|------|------------|
-| **Proof Bundle** | JSON document containing run metadata, results, artifacts, and signature |
-| **Canonical Form** | Deterministic JSON serialization (sorted keys, no whitespace) |
-| **Run ID** | UUID v4 uniquely identifying a test execution |
-| **Signer ID** | Identity of the signing entity (default: `local@qa360`) |
-| **Artifact** | File produced during test run (report, screenshot, log) |
-| **Trust Score** | Numeric quality metric (0-100) |
-| **Gate** | Individual test category (api_smoke, perf, sast, etc.) |
-
----
-
-## 3. Data Model
-
-### 3.1 Top-Level Structure
-
-```json
-{
-  "spec": "qa360.proof.v1",
-  "run": { ... },
-  "artifacts": [ ... ],
-  "results": { ... },
-  "signing": { ... },
-  "signature": "base64-encoded-ed25519-signature"
-}
-```
-
-### 3.2 Field Specifications
-
-#### 3.2.1 `spec` (required)
-
-- **Type**: `string`
-- **Value**: `"qa360.proof.v1"` (immutable)
-- **Purpose**: Version identifier for proof format
-
-#### 3.2.2 `run` (required)
-
-```json
-{
-  "id": "uuid-v4",
-  "startedAt": "2025-10-26T12:34:56Z",
-  "finishedAt": "2025-10-26T12:35:42Z",
-  "environment": {
-    "os": "windows|linux|darwin",
-    "node": "20.19.0",
-    "arch": "x64|arm64",
-    "ci": false
-  },
-  "packHash": "sha256-<64-hex-chars>",
-  "ciContext": {
-    "provider": null
-  }
-}
-```
-
-**Constraints**:
-- `id`: UUID v4 format
-- `startedAt`, `finishedAt`: ISO 8601 UTC timestamps
-- `packHash`: SHA-256 of canonicalized pack.yml
-
-#### 3.2.3 `artifacts` (required, can be empty array)
-
-```json
-[
-  {
-    "name": "report.html",
-    "sha256": "sha256-<64-hex-chars>",
-    "size": 12345,
-    "path": ".qa360/artifacts/report.html"
-  }
-]
-```
-
-**Constraints**:
-- `name`: Relative filename
-- `sha256`: SHA-256 hash (hex, lowercase)
-- `size`: Bytes (integer ≥ 0)
-- `path`: Relative path from proof bundle location
-
-#### 3.2.4 `results` (required)
-
-```json
-{
-  "trustScore": 87,
-  "gates": [
-    {
-      "name": "api_smoke",
-      "status": "pass|fail|skip",
-      "metrics": {
-        "p95_ms": 142,
-        "success_rate": 0.997
-      }
-    }
-  ]
-}
-```
-
-**Constraints**:
-- `trustScore`: Integer 0-100
-- `gates[].status`: Enum `pass|fail|skip`
-- `gates[].metrics`: Optional object (gate-specific)
-
-#### 3.2.5 `signing` (required)
-
-```json
-{
-  "algo": "ed25519",
-  "signerId": "local@qa360",
-  "timestamp": {
-    "type": "none",
-    "token": null
-  },
-  "identity": {
-    "type": "none",
-    "evidence": null
-  }
-}
-```
-
-**Constraints**:
-- `algo`: Fixed `"ed25519"` (Phase 1)
-- `signerId`: String identifier (default: `local@qa360`)
-- `timestamp.type`: `"none"` (Phase 1), `"rfc3161"` (Phase 2)
-- `identity.type`: `"none"` (Phase 1), `"did"|"sigstore"` (Phase 2)
-
-#### 3.2.6 `signature` (required)
-
-- **Type**: `string`
-- **Format**: Base64-encoded Ed25519 signature (88 chars)
-- **Computed over**: SHA-256 hash of canonical JSON (excluding `signature` field)
-
----
-
-## 4. Canonicalization
-
-### 4.1 Purpose
-
-Ensure **deterministic serialization** across platforms, languages, and JSON libraries.
-
-### 4.2 Algorithm
-
-```
-CANONICAL_JSON(obj):
-  1. Remove "signature" field if present
-  2. Sort all object keys alphabetically (recursive)
-  3. Encode strings as UTF-8 NFC (Unicode normalization)
-  4. Numbers: decimal notation (no scientific notation)
-  5. Booleans/null: JSON literals (true, false, null)
-  6. Omit empty optional objects/arrays
-  7. No whitespace (compact form)
-  8. Terminate with single \n
-```
-
-### 4.3 Example
-
-**Input**:
-```json
-{
-  "run": { "id": "abc", "startedAt": "2025-01-01T00:00:00Z" },
-  "spec": "qa360.proof.v1"
-}
-```
-
-**Canonical**:
-```
-{"run":{"id":"abc","startedAt":"2025-01-01T00:00:00Z"},"spec":"qa360.proof.v1"}
-```
-
-### 4.4 Implementation Notes
-
-- Use `JSON.stringify()` with custom replacer (sort keys)
-- Apply Unicode NFC normalization (`String.prototype.normalize('NFC')`)
-- Verify no BOM (Byte Order Mark) in UTF-8 encoding
-
----
-
-## 5. Signature Procedure
-
-### 5.1 Key Generation (One-time)
-
-```bash
-# Generate Ed25519 keypair
-qa360 doctor --init-keys
-
-# Stores:
-# ~/.qa360/keys/ed25519.key (private, chmod 600)
-# ~/.qa360/keys/ed25519.pub (public, chmod 644)
-```
-
-### 5.2 Signing Algorithm
-
-```
-SIGN(proof_bundle):
-  1. canonical = CANONICAL_JSON(proof_bundle)
-  2. hash = SHA256(canonical)
-  3. sig = Ed25519.sign(hash, private_key)
-  4. proof_bundle.signature = base64(sig)
-  5. return proof_bundle
-```
-
-### 5.3 Libraries
-
-- **Node.js**: `tweetnacl` (Ed25519), `crypto` (SHA-256)
-- **No network calls**: All operations local
-
----
-
-## 6. Verification Procedure
-
-### 6.1 Algorithm
-
-```
-VERIFY(proof_bundle):
-  1. Extract signature_b64 = proof_bundle.signature
-  2. Remove proof_bundle.signature
-  3. canonical = CANONICAL_JSON(proof_bundle)
-  4. hash = SHA256(canonical)
-  5. sig = base64_decode(signature_b64)
-  6. public_key = load_from(~/.qa360/keys/ed25519.pub)
-  7. valid = Ed25519.verify(sig, hash, public_key)
-  8. IF NOT valid: RETURN ERROR "Invalid signature"
-  9. Validate JSON Schema (AJV)
-  10. Verify artifact hashes (if artifacts present)
-  11. RETURN SUCCESS
-```
-
-### 6.2 Exit Codes
-
-| Code | Meaning |
-|------|---------|
-| 0 | Proof verified successfully |
-| 1 | Invalid signature |
-| 2 | Schema validation failed |
-| 3 | Artifact hash mismatch |
-| 4 | Missing public key |
-
-### 6.3 CLI Output
-
-```bash
-$ qa360 verify .qa360/proofs/abc-123.json
-
-✅ Proof verified: OK
-🔏 Signer: local@qa360
-🔐 Hash: sha256-a1b2c3...
-📦 Artifacts: 7 (all verified)
-⏱️  Run: 2025-10-26T12:34:56Z → 2025-10-26T12:35:42Z
-🎯 Trust Score: 87/100
-```
-
----
-
-## 7. JSON Schema
-
-### 7.1 Complete Schema (AJV v2020-12)
-
-```json
-{
-  "$schema": "https://json-schema.org/draft/2020-12/schema",
-  "$id": "https://qa360.ai/schemas/proof-bundle-v1.json",
-  "title": "QA360 Proof Bundle v1",
-  "type": "object",
-  "required": ["spec", "run", "artifacts", "results", "signing", "signature"],
-  "additionalProperties": false,
-  "properties": {
-    "spec": {
-      "type": "string",
-      "const": "qa360.proof.v1"
-    },
-    "run": {
-      "type": "object",
-      "required": ["id", "startedAt", "finishedAt", "environment", "packHash"],
-      "additionalProperties": false,
-      "properties": {
-        "id": {
-          "type": "string",
-          "format": "uuid"
-        },
-        "startedAt": {
-          "type": "string",
-          "format": "date-time"
-        },
-        "finishedAt": {
-          "type": "string",
-          "format": "date-time"
-        },
-        "environment": {
-          "type": "object",
-          "required": ["os", "node", "arch", "ci"],
-          "additionalProperties": false,
-          "properties": {
-            "os": {
-              "type": "string",
-              "enum": ["windows", "linux", "darwin"]
-            },
-            "node": {
-              "type": "string",
-              "pattern": "^\\d+\\.\\d+\\.\\d+$"
-            },
-            "arch": {
-              "type": "string",
-              "enum": ["x64", "arm64"]
-            },
-            "ci": {
-              "type": "boolean"
-            }
-          }
-        },
-        "packHash": {
-          "type": "string",
-          "pattern": "^sha256-[0-9a-f]{64}$"
-        },
-        "ciContext": {
-          "type": "object",
-          "properties": {
-            "provider": {
-              "type": ["string", "null"]
-            }
-          }
-        }
-      }
-    },
-    "artifacts": {
-      "type": "array",
-      "items": {
-        "type": "object",
-        "required": ["name", "sha256", "size"],
-        "additionalProperties": false,
-        "properties": {
-          "name": {
-            "type": "string",
-            "minLength": 1
-          },
-          "sha256": {
-            "type": "string",
-            "pattern": "^sha256-[0-9a-f]{64}$"
-          },
-          "size": {
-            "type": "integer",
-            "minimum": 0
-          },
-          "path": {
-            "type": "string"
-          }
-        }
-      }
-    },
-    "results": {
-      "type": "object",
-      "required": ["trustScore", "gates"],
-      "additionalProperties": false,
-      "properties": {
-        "trustScore": {
-          "type": "integer",
-          "minimum": 0,
-          "maximum": 100
-        },
-        "gates": {
-          "type": "array",
-          "items": {
-            "type": "object",
-            "required": ["name", "status"],
-            "additionalProperties": false,
-            "properties": {
-              "name": {
-                "type": "string",
-                "minLength": 1
-              },
-              "status": {
-                "type": "string",
-                "enum": ["pass", "fail", "skip"]
-              },
-              "metrics": {
-                "type": "object"
-              }
-            }
-          }
-        }
-      }
-    },
-    "signing": {
-      "type": "object",
-      "required": ["algo", "signerId", "timestamp", "identity"],
-      "additionalProperties": false,
-      "properties": {
-        "algo": {
-          "type": "string",
-          "const": "ed25519"
-        },
-        "signerId": {
-          "type": "string",
-          "minLength": 1
-        },
-        "timestamp": {
-          "type": "object",
-          "required": ["type"],
-          "properties": {
-            "type": {
-              "type": "string",
-              "enum": ["none", "rfc3161"]
-            },
-            "token": {
-              "type": ["string", "null"]
-            }
-          }
-        },
-        "identity": {
-          "type": "object",
-          "required": ["type"],
-          "properties": {
-            "type": {
-              "type": "string",
-              "enum": ["none", "did", "sigstore"]
-            },
-            "evidence": {
-              "type": ["string", "object", "null"]
-            }
-          }
-        }
-      }
-    },
-    "signature": {
-      "type": "string",
-      "pattern": "^[A-Za-z0-9+/]{86}==$"
-    }
-  }
-}
-```
-
----
-
-## 8. Compatibility & Extensions
-
-### 8.1 Forward Compatibility
-
-**Reserved fields** for future phases (no implementation required in Phase 1):
-
-#### 8.1.1 RFC 3161 Timestamp (Phase 2)
-
-```json
-{
-  "signing": {
-    "timestamp": {
-      "type": "rfc3161",
-      "token": "base64-encoded-tsa-response"
-    }
-  }
-}
-```
-
-**Verification** (Phase 2):
-- Parse TSA response
-- Verify TSA signature chain
-- Validate timestamp against run.finishedAt
-
-#### 8.1.2 DID/Sigstore Identity (Phase 2)
-
-```json
-{
-  "signing": {
-    "identity": {
-      "type": "did",
-      "evidence": "did:key:z6MkhaXgBZDvotDkL5257faiztiGiC2QtKLGpbnnEGta2doK"
-    }
-  }
-}
-```
-
-**Verification** (Phase 2):
-- Resolve DID document
-- Verify public key matches signature
-- Validate identity claims
-
-### 8.2 Backward Compatibility
-
-- **v1.0.0**: Initial release (this RFC)
-- **v1.1.0+**: May add optional fields, MUST NOT remove required fields
-- **v2.0.0**: Breaking changes allowed (new `spec` value)
-
-### 8.3 Extension Points
-
-| Field | Purpose | Phase |
-|-------|---------|-------|
-| `ciContext.provider` | CI/CD metadata | 1 |
-| `timestamp.token` | TSA response | 2 |
-| `identity.evidence` | DID/Sigstore | 2 |
-| `attestations` | Multi-sig | 3 |
-
----
-
-## 9. Security Considerations
-
-### 9.1 Threat Model
-
-| Threat | Mitigation |
-|--------|------------|
-| **Tampered results** | Ed25519 signature detects any modification |
-| **Artifact substitution** | SHA-256 hashes verify artifact integrity |
-| **Replay attacks** | UUID v4 run ID + timestamps |
-| **Key compromise** | Rotation policy (Phase 2) |
-| **Time manipulation** | RFC 3161 TSA (Phase 2) |
-
-### 9.2 Key Storage
-
-**Local keys** (Phase 1):
-```
-~/.qa360/keys/
-├── ed25519.key  (chmod 600, never commit)
-└── ed25519.pub  (chmod 644, shareable)
-```
-
-**Best practices**:
-- ✅ Generate keys with `qa360 doctor --init-keys`
-- ✅ Backup private key securely
-- ✅ Never commit private key to git
-- ✅ Use environment variables in CI (`QA360_PRIVATE_KEY`)
-
-### 9.3 Redaction
-
-**Sensitive data** in artifacts:
-- Automatically redacted by QA360 Core (20+ patterns)
-- Passwords, tokens, API keys, PII
-- Redaction applied BEFORE hashing
-
-### 9.4 Audit Trail
-
-**Proof bundles** are immutable:
-- Stored in `.qa360/proofs/<runId>.json`
-- Never modified after creation
-- Retention policy: 90 days (configurable)
-
----
-
-## 10. Examples
-
-### 10.1 Valid Proof Bundle
-
-```json
-{
-  "spec": "qa360.proof.v1",
-  "run": {
-    "id": "550e8400-e29b-41d4-a716-446655440000",
-    "startedAt": "2025-10-26T12:34:56Z",
-    "finishedAt": "2025-10-26T12:35:42Z",
-    "environment": {
-      "os": "linux",
-      "node": "20.19.0",
-      "arch": "x64",
-      "ci": false
-    },
-    "packHash": "sha256-a1b2c3d4e5f6789012345678901234567890123456789012345678901234abcd",
-    "ciContext": {
-      "provider": null
-    }
-  },
-  "artifacts": [
-    {
-      "name": "report.html",
-      "sha256": "sha256-1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
-      "size": 12345,
-      "path": ".qa360/artifacts/report.html"
-    }
-  ],
-  "results": {
-    "trustScore": 87,
-    "gates": [
-      {
-        "name": "api_smoke",
-        "status": "pass",
-        "metrics": {
-          "p95_ms": 142,
-          "success_rate": 0.997
-        }
-      }
-    ]
-  },
-  "signing": {
-    "algo": "ed25519",
-    "signerId": "local@qa360",
-    "timestamp": {
-      "type": "none",
-      "token": null
-    },
-    "identity": {
-      "type": "none",
-      "evidence": null
-    }
-  },
-  "signature": "SGVsbG8gV29ybGQhIFRoaXMgaXMgYSBmYWtlIHNpZ25hdHVyZSBmb3IgZGVtbyBwdXJwb3Nlcy4K=="
-}
-```
-
-### 10.2 Invalid Proof (Tampered)
-
-**Scenario**: `trustScore` modified from 87 to 95
-
-**Verification result**:
-```
-❌ Proof verification FAILED
-🔐 Signature mismatch (hash changed)
-📝 Expected: sha256-a1b2c3...
-📝 Got:      sha256-d4e5f6...
-```
-
-### 10.3 Future: RFC 3161 Timestamp
-
-```json
-{
-  "signing": {
-    "timestamp": {
-      "type": "rfc3161",
-      "token": "MIIBsAYJKoZIhvcNAQcCoIIBo..."
-    }
-  }
-}
-```
-
-**Note**: Token verification NOT implemented in Phase 1.
-
----
-
-## 11. Annexes
-
-### 11.1 SHA-256 Format
-
-**Pattern**: `sha256-[0-9a-f]{64}`
-
-**Example**: `sha256-a1b2c3d4e5f6789012345678901234567890123456789012345678901234abcd`
-
-**Computation**:
-```javascript
-const crypto = require('crypto');
-const hash = crypto.createHash('sha256')
-  .update(data, 'utf8')
-  .digest('hex');
-const formatted = `sha256-${hash}`;
-```
-
-### 11.2 Unicode Normalization
-
-**NFC (Canonical Composition)**:
-```javascript
-const normalized = str.normalize('NFC');
-```
-
-**Why**: Ensures `é` (U+00E9) and `é` (U+0065 U+0301) produce same hash.
-
-### 11.3 Error Codes
-
-| Code | Symbol | Meaning |
-|------|--------|---------|
-| 0 | `PROOF_OK` | Verification successful |
-| 1 | `PROOF_INVALID_SIG` | Signature verification failed |
-| 2 | `PROOF_INVALID_SCHEMA` | JSON schema validation failed |
-| 3 | `PROOF_ARTIFACT_MISMATCH` | Artifact hash doesn't match |
-| 4 | `PROOF_MISSING_KEY` | Public key not found |
-| 5 | `PROOF_MALFORMED` | Invalid JSON structure |
-
-### 11.4 CLI Reference
-
-```bash
-# Verify single proof
-qa360 verify .qa360/proofs/abc-123.json
-
-# Verify all proofs in directory
-qa360 verify .qa360/proofs/
-
-# JSON output
-qa360 verify proof.json --json
-
-# Strict mode (require RFC 3161 timestamp)
-qa360 verify proof.json --strict
-```
-
-### 11.5 Test Vectors
-
-**Canonical JSON**:
-```
-Input:  {"b": 2, "a": 1}
-Output: {"a":1,"b":2}\n
-```
-
-**SHA-256**:
-```
-Input:  {"a":1,"b":2}\n
-Output: sha256-559aead08264d5795d3909718cdd05abd49572e84fe55590eef31a88a08fdffd
-```
-
----
-
-## Appendix: Implementation Checklist
-
-### Phase 1 (RFC + Core)
-
-- [ ] `core/src/proof/bundle.ts` - Bundle creation
-- [ ] `core/src/proof/canonicalize.ts` - Canonical JSON
-- [ ] `core/src/proof/signer.ts` - Ed25519 sign/verify
-- [ ] `core/src/proof/verifier.ts` - Full verification
-- [ ] `core/src/proof/schema.ts` - AJV validation
-
-### Phase 2 (CLI)
-
-- [ ] `cli/src/commands/verify.ts` - Verification command
-- [ ] `cli/src/commands/doctor.ts` - Add proof system check
-- [ ] Key generation (`--init-keys`)
-
-### Phase 3 (Tests)
-
-- [ ] `tests/e2e/proof-bundle.test.ts` - E2E tests
-- [ ] Cross-OS validation (Windows/macOS/Linux)
-- [ ] Roundtrip sign/verify
-
-### Phase 4 (Examples)
-
-- [ ] `examples/proofs/httpbin-proof.json`
-- [ ] `examples/proofs/e2e-playwright-proof.json`
-- [ ] `examples/proofs/multi-adapter-proof.json`
-
----
-
-## References
-
-- [RFC 3161](https://www.rfc-editor.org/rfc/rfc3161) - Time-Stamp Protocol (TSP)
-- [Ed25519](https://ed25519.cr.yp.to/) - High-speed high-security signatures
-- [JSON Canonicalization](https://www.rfc-editor.org/rfc/rfc8785) - JCS (RFC 8785)
-- [JSON Schema](https://json-schema.org/draft/2020-12/schema) - v2020-12
-- [DID](https://www.w3.org/TR/did-core/) - Decentralized Identifiers
-- [Sigstore](https://www.sigstore.dev/) - Software signing service
-
----
-
-**Document Status**: ✅ Ready for Implementation  
-**Next Steps**: Phase 2 - Core Implementation (`bundle.ts`, `signer.ts`, `verifier.ts`)
-