qa360 2.1.7 → 2.2.0

package/docs/rfc/proof-bundle-v1.md

@@ -0,0 +1,787 @@
+# RFC: QA360 Proof Bundle v1
+
+**Status**: Draft  
+**Version**: 1.0.0  
+**Date**: 2025-10-26  
+**Authors**: QA360 Core Team
+
+---
+
+## Table of Contents
+
+1. [Motivation & Scope](#1-motivation--scope)
+2. [Terminology](#2-terminology)
+3. [Data Model](#3-data-model)
+4. [Canonicalization](#4-canonicalization)
+5. [Signature Procedure](#5-signature-procedure)
+6. [Verification Procedure](#6-verification-procedure)
+7. [JSON Schema](#7-json-schema)
+8. [Compatibility & Extensions](#8-compatibility--extensions)
+9. [Security Considerations](#9-security-considerations)
+10. [Examples](#10-examples)
+11. [Annexes](#11-annexes)
+
+---
+
+## 1. Motivation & Scope
+
+### 1.1 Problem Statement
+
+Software quality testing generates artifacts (reports, metrics, logs) but lacks:
+- **Verifiable integrity**: Can results be tampered with?
+- **Portable proofs**: Can results be verified offline, cross-OS?
+- **Legal admissibility**: Can results serve as evidence in audits?
+
+### 1.2 Solution
+
+QA360 Proof Bundle v1 defines a **cryptographically signed, self-contained proof** of test execution that is:
+
+- ✅ **Local-first**: No cloud dependencies
+- ✅ **Cross-platform**: Same hash on Windows/macOS/Linux
+- ✅ **Verifiable offline**: Ed25519 signature + SHA-256 hashes
+- ✅ **Forward-compatible**: Reserved fields for RFC 3161 timestamps, DID/Sigstore identities
+
+### 1.3 Non-Goals (Phase 1)
+
+- ❌ RFC 3161 timestamp verification (Phase 2)
+- ❌ Sigstore/DID identity binding (Phase 2)
+- ❌ Multi-signature support (Phase 3)
+- ❌ Cloud storage/distribution (Phase 4)
+
+---
+
+## 2. Terminology
+
+| Term | Definition |
+|------|------------|
+| **Proof Bundle** | JSON document containing run metadata, results, artifacts, and signature |
+| **Canonical Form** | Deterministic JSON serialization (sorted keys, no whitespace) |
+| **Run ID** | UUID v4 uniquely identifying a test execution |
+| **Signer ID** | Identity of the signing entity (default: `local@qa360`) |
+| **Artifact** | File produced during test run (report, screenshot, log) |
+| **Trust Score** | Numeric quality metric (0-100) |
+| **Gate** | Individual test category (api_smoke, perf, sast, etc.) |
+
+---
+
+## 3. Data Model
+
+### 3.1 Top-Level Structure
+
+```json
+{
+  "spec": "qa360.proof.v1",
+  "run": { ... },
+  "artifacts": [ ... ],
+  "results": { ... },
+  "signing": { ... },
+  "signature": "base64-encoded-ed25519-signature"
+}
+```
+
+### 3.2 Field Specifications
+
+#### 3.2.1 `spec` (required)
+
+- **Type**: `string`
+- **Value**: `"qa360.proof.v1"` (immutable)
+- **Purpose**: Version identifier for proof format
+
+#### 3.2.2 `run` (required)
+
+```json
+{
+  "id": "uuid-v4",
+  "startedAt": "2025-10-26T12:34:56Z",
+  "finishedAt": "2025-10-26T12:35:42Z",
+  "environment": {
+    "os": "windows|linux|darwin",
+    "node": "20.19.0",
+    "arch": "x64|arm64",
+    "ci": false
+  },
+  "packHash": "sha256-<64-hex-chars>",
+  "ciContext": {
+    "provider": null
+  }
+}
+```
+
+**Constraints**:
+- `id`: UUID v4 format
+- `startedAt`, `finishedAt`: ISO 8601 UTC timestamps
+- `packHash`: SHA-256 of canonicalized pack.yml
+
+#### 3.2.3 `artifacts` (required, can be empty array)
+
+```json
+[
+  {
+    "name": "report.html",
+    "sha256": "sha256-<64-hex-chars>",
+    "size": 12345,
+    "path": ".qa360/artifacts/report.html"
+  }
+]
+```
+
+**Constraints**:
+- `name`: Relative filename
+- `sha256`: SHA-256 hash (hex, lowercase)
+- `size`: Bytes (integer ≥ 0)
+- `path`: Relative path from proof bundle location
+
+#### 3.2.4 `results` (required)
+
+```json
+{
+  "trustScore": 87,
+  "gates": [
+    {
+      "name": "api_smoke",
+      "status": "pass|fail|skip",
+      "metrics": {
+        "p95_ms": 142,
+        "success_rate": 0.997
+      }
+    }
+  ]
+}
+```
+
+**Constraints**:
+- `trustScore`: Integer 0-100
+- `gates[].status`: Enum `pass|fail|skip`
+- `gates[].metrics`: Optional object (gate-specific)
+
+#### 3.2.5 `signing` (required)
+
+```json
+{
+  "algo": "ed25519",
+  "signerId": "local@qa360",
+  "timestamp": {
+    "type": "none",
+    "token": null
+  },
+  "identity": {
+    "type": "none",
+    "evidence": null
+  }
+}
+```
+
+**Constraints**:
+- `algo`: Fixed `"ed25519"` (Phase 1)
+- `signerId`: String identifier (default: `local@qa360`)
+- `timestamp.type`: `"none"` (Phase 1), `"rfc3161"` (Phase 2)
+- `identity.type`: `"none"` (Phase 1), `"did"|"sigstore"` (Phase 2)
+
+#### 3.2.6 `signature` (required)
+
+- **Type**: `string`
+- **Format**: Base64-encoded Ed25519 signature (88 chars)
+- **Computed over**: SHA-256 hash of canonical JSON (excluding `signature` field)
+
+---
+
+## 4. Canonicalization
+
+### 4.1 Purpose
+
+Ensure **deterministic serialization** across platforms, languages, and JSON libraries.
+
+### 4.2 Algorithm
+
+```
+CANONICAL_JSON(obj):
+  1. Remove "signature" field if present
+  2. Sort all object keys alphabetically (recursive)
+  3. Encode strings as UTF-8 NFC (Unicode normalization)
+  4. Numbers: decimal notation (no scientific notation)
+  5. Booleans/null: JSON literals (true, false, null)
+  6. Omit empty optional objects/arrays
+  7. No whitespace (compact form)
+  8. Terminate with single \n
+```
+
+### 4.3 Example
+
+**Input**:
+```json
+{
+  "run": { "id": "abc", "startedAt": "2025-01-01T00:00:00Z" },
+  "spec": "qa360.proof.v1"
+}
+```
+
+**Canonical**:
+```
+{"run":{"id":"abc","startedAt":"2025-01-01T00:00:00Z"},"spec":"qa360.proof.v1"}
+```
+
+### 4.4 Implementation Notes
+
+- Use `JSON.stringify()` with custom replacer (sort keys)
+- Apply Unicode NFC normalization (`String.prototype.normalize('NFC')`)
+- Verify no BOM (Byte Order Mark) in UTF-8 encoding
+
+---
+
+## 5. Signature Procedure
+
+### 5.1 Key Generation (One-time)
+
+```bash
+# Generate Ed25519 keypair
+qa360 doctor --init-keys
+
+# Stores:
+# ~/.qa360/keys/ed25519.key (private, chmod 600)
+# ~/.qa360/keys/ed25519.pub (public, chmod 644)
+```
+
+### 5.2 Signing Algorithm
+
+```
+SIGN(proof_bundle):
+  1. canonical = CANONICAL_JSON(proof_bundle)
+  2. hash = SHA256(canonical)
+  3. sig = Ed25519.sign(hash, private_key)
+  4. proof_bundle.signature = base64(sig)
+  5. return proof_bundle
+```
+
+### 5.3 Libraries
+
+- **Node.js**: `tweetnacl` (Ed25519), `crypto` (SHA-256)
+- **No network calls**: All operations local
+
+---
+
+## 6. Verification Procedure
+
+### 6.1 Algorithm
+
+```
+VERIFY(proof_bundle):
+  1. Extract signature_b64 = proof_bundle.signature
+  2. Remove proof_bundle.signature
+  3. canonical = CANONICAL_JSON(proof_bundle)
+  4. hash = SHA256(canonical)
+  5. sig = base64_decode(signature_b64)
+  6. public_key = load_from(~/.qa360/keys/ed25519.pub)
+  7. valid = Ed25519.verify(sig, hash, public_key)
+  8. IF NOT valid: RETURN ERROR "Invalid signature"
+  9. Validate JSON Schema (AJV)
+  10. Verify artifact hashes (if artifacts present)
+  11. RETURN SUCCESS
+```
+
+### 6.2 Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | Proof verified successfully |
+| 1 | Invalid signature |
+| 2 | Schema validation failed |
+| 3 | Artifact hash mismatch |
+| 4 | Missing public key |
+
+### 6.3 CLI Output
+
+```bash
+$ qa360 verify .qa360/proofs/abc-123.json
+
+✅ Proof verified: OK
+🔏 Signer: local@qa360
+🔐 Hash: sha256-a1b2c3...
+📦 Artifacts: 7 (all verified)
+⏱️  Run: 2025-10-26T12:34:56Z → 2025-10-26T12:35:42Z
+🎯 Trust Score: 87/100
+```
+
+---
+
+## 7. JSON Schema
+
+### 7.1 Complete Schema (AJV v2020-12)
+
+```json
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://qa360.ai/schemas/proof-bundle-v1.json",
+  "title": "QA360 Proof Bundle v1",
+  "type": "object",
+  "required": ["spec", "run", "artifacts", "results", "signing", "signature"],
+  "additionalProperties": false,
+  "properties": {
+    "spec": {
+      "type": "string",
+      "const": "qa360.proof.v1"
+    },
+    "run": {
+      "type": "object",
+      "required": ["id", "startedAt", "finishedAt", "environment", "packHash"],
+      "additionalProperties": false,
+      "properties": {
+        "id": {
+          "type": "string",
+          "format": "uuid"
+        },
+        "startedAt": {
+          "type": "string",
+          "format": "date-time"
+        },
+        "finishedAt": {
+          "type": "string",
+          "format": "date-time"
+        },
+        "environment": {
+          "type": "object",
+          "required": ["os", "node", "arch", "ci"],
+          "additionalProperties": false,
+          "properties": {
+            "os": {
+              "type": "string",
+              "enum": ["windows", "linux", "darwin"]
+            },
+            "node": {
+              "type": "string",
+              "pattern": "^\\d+\\.\\d+\\.\\d+$"
+            },
+            "arch": {
+              "type": "string",
+              "enum": ["x64", "arm64"]
+            },
+            "ci": {
+              "type": "boolean"
+            }
+          }
+        },
+        "packHash": {
+          "type": "string",
+          "pattern": "^sha256-[0-9a-f]{64}$"
+        },
+        "ciContext": {
+          "type": "object",
+          "properties": {
+            "provider": {
+              "type": ["string", "null"]
+            }
+          }
+        }
+      }
+    },
+    "artifacts": {
+      "type": "array",
+      "items": {
+        "type": "object",
+        "required": ["name", "sha256", "size"],
+        "additionalProperties": false,
+        "properties": {
+          "name": {
+            "type": "string",
+            "minLength": 1
+          },
+          "sha256": {
+            "type": "string",
+            "pattern": "^sha256-[0-9a-f]{64}$"
+          },
+          "size": {
+            "type": "integer",
+            "minimum": 0
+          },
+          "path": {
+            "type": "string"
+          }
+        }
+      }
+    },
+    "results": {
+      "type": "object",
+      "required": ["trustScore", "gates"],
+      "additionalProperties": false,
+      "properties": {
+        "trustScore": {
+          "type": "integer",
+          "minimum": 0,
+          "maximum": 100
+        },
+        "gates": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "required": ["name", "status"],
+            "additionalProperties": false,
+            "properties": {
+              "name": {
+                "type": "string",
+                "minLength": 1
+              },
+              "status": {
+                "type": "string",
+                "enum": ["pass", "fail", "skip"]
+              },
+              "metrics": {
+                "type": "object"
+              }
+            }
+          }
+        }
+      }
+    },
+    "signing": {
+      "type": "object",
+      "required": ["algo", "signerId", "timestamp", "identity"],
+      "additionalProperties": false,
+      "properties": {
+        "algo": {
+          "type": "string",
+          "const": "ed25519"
+        },
+        "signerId": {
+          "type": "string",
+          "minLength": 1
+        },
+        "timestamp": {
+          "type": "object",
+          "required": ["type"],
+          "properties": {
+            "type": {
+              "type": "string",
+              "enum": ["none", "rfc3161"]
+            },
+            "token": {
+              "type": ["string", "null"]
+            }
+          }
+        },
+        "identity": {
+          "type": "object",
+          "required": ["type"],
+          "properties": {
+            "type": {
+              "type": "string",
+              "enum": ["none", "did", "sigstore"]
+            },
+            "evidence": {
+              "type": ["string", "object", "null"]
+            }
+          }
+        }
+      }
+    },
+    "signature": {
+      "type": "string",
+      "pattern": "^[A-Za-z0-9+/]{86}==$"
+    }
+  }
+}
+```
+
+---
+
+## 8. Compatibility & Extensions
+
+### 8.1 Forward Compatibility
+
+**Reserved fields** for future phases (no implementation required in Phase 1):
+
+#### 8.1.1 RFC 3161 Timestamp (Phase 2)
+
+```json
+{
+  "signing": {
+    "timestamp": {
+      "type": "rfc3161",
+      "token": "base64-encoded-tsa-response"
+    }
+  }
+}
+```
+
+**Verification** (Phase 2):
+- Parse TSA response
+- Verify TSA signature chain
+- Validate timestamp against run.finishedAt
+
+#### 8.1.2 DID/Sigstore Identity (Phase 2)
+
+```json
+{
+  "signing": {
+    "identity": {
+      "type": "did",
+      "evidence": "did:key:z6MkhaXgBZDvotDkL5257faiztiGiC2QtKLGpbnnEGta2doK"
+    }
+  }
+}
+```
+
+**Verification** (Phase 2):
+- Resolve DID document
+- Verify public key matches signature
+- Validate identity claims
+
+### 8.2 Backward Compatibility
+
+- **v1.0.0**: Initial release (this RFC)
+- **v1.1.0+**: May add optional fields, MUST NOT remove required fields
+- **v2.0.0**: Breaking changes allowed (new `spec` value)
+
+### 8.3 Extension Points
+
+| Field | Purpose | Phase |
+|-------|---------|-------|
+| `ciContext.provider` | CI/CD metadata | 1 |
+| `timestamp.token` | TSA response | 2 |
+| `identity.evidence` | DID/Sigstore | 2 |
+| `attestations` | Multi-sig | 3 |
+
+---
+
+## 9. Security Considerations
+
+### 9.1 Threat Model
+
+| Threat | Mitigation |
+|--------|------------|
+| **Tampered results** | Ed25519 signature detects any modification |
+| **Artifact substitution** | SHA-256 hashes verify artifact integrity |
+| **Replay attacks** | UUID v4 run ID + timestamps |
+| **Key compromise** | Rotation policy (Phase 2) |
+| **Time manipulation** | RFC 3161 TSA (Phase 2) |
+
+### 9.2 Key Storage
+
+**Local keys** (Phase 1):
+```
+~/.qa360/keys/
+├── ed25519.key  (chmod 600, never commit)
+└── ed25519.pub  (chmod 644, shareable)
+```
+
+**Best practices**:
+- ✅ Generate keys with `qa360 doctor --init-keys`
+- ✅ Backup private key securely
+- ✅ Never commit private key to git
+- ✅ Use environment variables in CI (`QA360_PRIVATE_KEY`)
+
+### 9.3 Redaction
+
+**Sensitive data** in artifacts:
+- Automatically redacted by QA360 Core (20+ patterns)
+- Passwords, tokens, API keys, PII
+- Redaction applied BEFORE hashing
+
+### 9.4 Audit Trail
+
+**Proof bundles** are immutable:
+- Stored in `.qa360/proofs/<runId>.json`
+- Never modified after creation
+- Retention policy: 90 days (configurable)
+
+---
+
+## 10. Examples
+
+### 10.1 Valid Proof Bundle
+
+```json
+{
+  "spec": "qa360.proof.v1",
+  "run": {
+    "id": "550e8400-e29b-41d4-a716-446655440000",
+    "startedAt": "2025-10-26T12:34:56Z",
+    "finishedAt": "2025-10-26T12:35:42Z",
+    "environment": {
+      "os": "linux",
+      "node": "20.19.0",
+      "arch": "x64",
+      "ci": false
+    },
+    "packHash": "sha256-a1b2c3d4e5f6789012345678901234567890123456789012345678901234abcd",
+    "ciContext": {
+      "provider": null
+    }
+  },
+  "artifacts": [
+    {
+      "name": "report.html",
+      "sha256": "sha256-1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef",
+      "size": 12345,
+      "path": ".qa360/artifacts/report.html"
+    }
+  ],
+  "results": {
+    "trustScore": 87,
+    "gates": [
+      {
+        "name": "api_smoke",
+        "status": "pass",
+        "metrics": {
+          "p95_ms": 142,
+          "success_rate": 0.997
+        }
+      }
+    ]
+  },
+  "signing": {
+    "algo": "ed25519",
+    "signerId": "local@qa360",
+    "timestamp": {
+      "type": "none",
+      "token": null
+    },
+    "identity": {
+      "type": "none",
+      "evidence": null
+    }
+  },
+  "signature": "SGVsbG8gV29ybGQhIFRoaXMgaXMgYSBmYWtlIHNpZ25hdHVyZSBmb3IgZGVtbyBwdXJwb3Nlcy4K=="
+}
+```
+
+### 10.2 Invalid Proof (Tampered)
+
+**Scenario**: `trustScore` modified from 87 to 95
+
+**Verification result**:
+```
+❌ Proof verification FAILED
+🔐 Signature mismatch (hash changed)
+📝 Expected: sha256-a1b2c3...
+📝 Got:      sha256-d4e5f6...
+```
+
+### 10.3 Future: RFC 3161 Timestamp
+
+```json
+{
+  "signing": {
+    "timestamp": {
+      "type": "rfc3161",
+      "token": "MIIBsAYJKoZIhvcNAQcCoIIBo..."
+    }
+  }
+}
+```
+
+**Note**: Token verification NOT implemented in Phase 1.
+
+---
+
+## 11. Annexes
+
+### 11.1 SHA-256 Format
+
+**Pattern**: `sha256-[0-9a-f]{64}`
+
+**Example**: `sha256-a1b2c3d4e5f6789012345678901234567890123456789012345678901234abcd`
+
+**Computation**:
+```javascript
+const crypto = require('crypto');
+const hash = crypto.createHash('sha256')
+  .update(data, 'utf8')
+  .digest('hex');
+const formatted = `sha256-${hash}`;
+```
+
+### 11.2 Unicode Normalization
+
+**NFC (Canonical Composition)**:
+```javascript
+const normalized = str.normalize('NFC');
+```
+
+**Why**: Ensures `é` (U+00E9) and `é` (U+0065 U+0301) produce same hash.
+
+### 11.3 Error Codes
+
+| Code | Symbol | Meaning |
+|------|--------|---------|
+| 0 | `PROOF_OK` | Verification successful |
+| 1 | `PROOF_INVALID_SIG` | Signature verification failed |
+| 2 | `PROOF_INVALID_SCHEMA` | JSON schema validation failed |
+| 3 | `PROOF_ARTIFACT_MISMATCH` | Artifact hash doesn't match |
+| 4 | `PROOF_MISSING_KEY` | Public key not found |
+| 5 | `PROOF_MALFORMED` | Invalid JSON structure |
+
+### 11.4 CLI Reference
+
+```bash
+# Verify single proof
+qa360 verify .qa360/proofs/abc-123.json
+
+# Verify all proofs in directory
+qa360 verify .qa360/proofs/
+
+# JSON output
+qa360 verify proof.json --json
+
+# Strict mode (require RFC 3161 timestamp)
+qa360 verify proof.json --strict
+```
+
+### 11.5 Test Vectors
+
+**Canonical JSON**:
+```
+Input:  {"b": 2, "a": 1}
+Output: {"a":1,"b":2}\n
+```
+
+**SHA-256**:
+```
+Input:  {"a":1,"b":2}\n
+Output: sha256-559aead08264d5795d3909718cdd05abd49572e84fe55590eef31a88a08fdffd
+```
+
+---
+
+## Appendix: Implementation Checklist
+
+### Phase 1 (RFC + Core)
+
+- [ ] `core/src/proof/bundle.ts` - Bundle creation
+- [ ] `core/src/proof/canonicalize.ts` - Canonical JSON
+- [ ] `core/src/proof/signer.ts` - Ed25519 sign/verify
+- [ ] `core/src/proof/verifier.ts` - Full verification
+- [ ] `core/src/proof/schema.ts` - AJV validation
+
+### Phase 2 (CLI)
+
+- [ ] `cli/src/commands/verify.ts` - Verification command
+- [ ] `cli/src/commands/doctor.ts` - Add proof system check
+- [ ] Key generation (`--init-keys`)
+
+### Phase 3 (Tests)
+
+- [ ] `tests/e2e/proof-bundle.test.ts` - E2E tests
+- [ ] Cross-OS validation (Windows/macOS/Linux)
+- [ ] Roundtrip sign/verify
+
+### Phase 4 (Examples)
+
+- [ ] `examples/proofs/httpbin-proof.json`
+- [ ] `examples/proofs/e2e-playwright-proof.json`
+- [ ] `examples/proofs/multi-adapter-proof.json`
+
+---
+
+## References
+
+- [RFC 3161](https://www.rfc-editor.org/rfc/rfc3161) - Time-Stamp Protocol (TSP)
+- [Ed25519](https://ed25519.cr.yp.to/) - High-speed high-security signatures
+- [JSON Canonicalization](https://www.rfc-editor.org/rfc/rfc8785) - JCS (RFC 8785)
+- [JSON Schema](https://json-schema.org/draft/2020-12/schema) - v2020-12
+- [DID](https://www.w3.org/TR/did-core/) - Decentralized Identifiers
+- [Sigstore](https://www.sigstore.dev/) - Software signing service
+
+---
+
+**Document Status**: ✅ Ready for Implementation  
+**Next Steps**: Phase 2 - Core Implementation (`bundle.ts`, `signer.ts`, `verifier.ts`)
+