qa360 2.1.2 → 2.1.4

package/docs/DEVELOPMENT.md

@@ -0,0 +1,331 @@
+# QA360 - Development Guide
+
+**For contributors and developers working on QA360 codebase**
+
+---
+
+## Quick Start
+
+```bash
+# Clone and install
+git clone https://github.com/xyqotech/qa360.git
+cd qa360
+pnpm install
+
+# Build
+pnpm build
+
+# Run tests
+pnpm test
+
+# Run CLI locally
+node cli/bin/qa360.js --help
+```
+
+---
+
+## Project Structure
+
+```
+qa360/
+├── cli/                    # @qa360/cli - User-facing CLI
+│   ├── src/
+│   │   ├── commands/      # CLI command implementations
+│   │   └── index.ts       # CLI entry point
+│   └── package.json
+│
+├── core/                   # qa360-core - Core engine (no CLI deps)
+│   ├── src/
+│   │   ├── ai/            # Multi-provider LLM
+│   │   ├── auth/          # 9 auth providers
+│   │   ├── generation/    # AI test generation
+│   │   ├── pack-v2/       # Pack v2 validation
+│   │   ├── repair/        # Auto-repair module
+│   │   ├── tdi/           # Test Data Intelligence
+│   │   ├── tui/           # Terminal UI (Phase 8)
+│   │   ├── dashboard/     # Web dashboard (Phase 8)
+│   │   ├── vault/         # Evidence vault (SQLite)
+│   │   ├── proof/         # Ed25519 signing
+│   │   ├── adapters/      # Test adapters
+│   │   └── index.ts       # Core exports
+│   └── package.json
+│
+├── packages/mcp/          # MCP server integration
+├── docs/                  # Documentation (3 files only)
+│   ├── STATUS.md          # Project status (THIS IS THE SOURCE)
+│   ├── DEVELOPMENT.md     # This file
+│   └── ARCHITECTURE.md    # Technical architecture
+└── examples/              # Example configurations
+```
+
+---
+
+## Common Commands
+
+### Development
+```bash
+pnpm build           # Build all packages
+pnpm dev             # Watch mode
+pnpm clean           # Clean build artifacts
+pnpm type-check      # TypeScript validation
+pnpm lint            # ESLint
+pnpm lint:fix        # Auto-fix linting
+```
+
+### Testing
+```bash
+pnpm test            # All tests
+pnpm test:core       # Core only
+pnpm test:cli        # CLI only
+pnpm test:coverage   # With coverage
+```
+
+### Running QA360
+```bash
+qa360 run qa360.yml              # Execute tests
+qa360 verify .qa360/runs/proof.json  # Verify proof
+qa360 doctor --fix                # Health check
+qa360 pack validate               # Validate config
+```
+
+---
+
+## Phase Roadmap
+
+**Current Phase: 8 of 9**
+
+| Phase | Name | Status |
+|-------|------|--------|
+| 0-5 | Core Platform | ✅ Complete |
+| 6 | Test Data Intelligence | ✅ Complete |
+| 7 | Auto-Repair Module | ✅ Complete |
+| 8 | TUI & Dashboard | ✅ Complete |
+| 9 | Parallel CI | ⏳ Next |
+
+See [STATUS.md](./STATUS.md) for detailed progress.
+
+---
+
+## Adding a New Feature
+
+### 1. Create a branch
+```bash
+git checkout develop
+git pull origin develop
+git checkout -b feature/my-feature
+```
+
+### 2. Implementation
+- Add code to `core/src/` for business logic
+- Add CLI commands to `cli/src/commands/`
+- Write tests in `__tests__/` directories
+
+### 3. Test
+```bash
+pnpm type-check
+pnpm lint
+pnpm test
+```
+
+### 4. Commit
+```bash
+git add .
+git commit -m "feat: description of feature"
+```
+
+### 5. Push and create PR
+```bash
+git push origin feature/my-feature
+gh pr create --title "feat: description" --body "..."
+```
+
+---
+
+## Publication Workflow
+
+**IMPORTANT**: Follow this exact order for releases:
+
+```bash
+# 1. Commit to develop
+git checkout develop
+git pull origin develop
+# ... make changes ...
+git add .
+git commit -m "feat: description"
+git push origin develop
+
+# 2. Wait for CI green
+gh run list --branch develop --limit 1
+gh run watch <run-id>
+
+# 3. Merge to main
+git checkout main
+git pull origin main
+git merge develop --no-ff -m "Merge branch 'develop'"
+git push origin main
+
+# 4. Wait for CI green on main
+gh run list --branch main --limit 1
+gh run watch <run-id>
+
+# 5. Version bump (code changes only)
+cd cli
+npm version patch  # or minor/major
+git add package.json
+git commit -m "chore(cli): bump version to X.Y.Z"
+git push origin develop
+
+# 6. Merge version bump to main
+git checkout main && git merge develop --no-ff
+git push origin main
+
+# 7. Tag and publish
+git tag -a vX.Y.Z -m "Release vX.Y.Z"
+git push origin vX.Y.Z
+npm publish --access public
+```
+
+---
+
+## Module Reference
+
+### AI Module (`core/src/ai/`)
+- `OllamaProvider` - Local LLM (Ollama)
+- `OpenAIProvider` - GPT-4 models
+- `AnthropicProvider` - Claude models
+- `MockProvider` - Testing mock
+- `ProviderFactory` - Auto fallback chain
+
+### Auth Module (`core/src/auth/`)
+9 providers: JWT, OAuth2, API Key, Bearer, Basic, TOTP, UI Login, GCP ADC, AWS IAM, Azure AD
+
+### Generation Module (`core/src/generation/`)
+- `PromptBuilder` - Few-shot prompts
+- `CodeFormatter` - Multi-language formatting
+- `TestOptimizer` - Deduplication
+- `CodeGenerator` - AI generation with timeout
+
+### Repair Module (`core/src/repair/`)
+- `ErrorAnalyzer` - Error classification (8 types)
+- `SuggestionEngine` - LLM fix suggestions
+- `CodeFixer` - Apply fixes with backup
+- `TestRepairer` - Orchestrate repair cycles
+
+### TDI Module (`core/src/tdi/`)
+- `TestAnalyzer` - Parse and detect frameworks
+- `CoverageEngine` - Coverage analysis
+- `DuplicationDetector` - Find duplicate tests
+- `SuggestionGenerator` - Improvement suggestions
+- `HealthCalculator` - Test health score
+
+---
+
+## Testing Guidelines
+
+### Test Structure
+```
+core/src/module/
+├── index.ts
+├── __tests__/
+│   └── module.test.ts
+```
+
+### Coverage Targets
+- Core modules: 75%+
+- New features: 80%+
+- Critical paths (proof, vault): 85%+
+
+### Test Quality
+- Unit tests for pure functions
+- Integration tests for workflows
+- Mock external dependencies (LLM, filesystem)
+- Use temporary directories for file operations
+
+---
+
+## Error Handling
+
+Every error must be:
+1. **Explicit** - Clear what went wrong
+2. **Pedagogical** - Explains why
+3. **Guiding** - Suggests fix
+
+```typescript
+throw new QA360Error(
+  'PACK_VALIDATION_FAILED',
+  'Pack file validation failed: missing required field "name"',
+  {
+    suggestion: 'Add a "name" field to your pack.yml',
+    example: 'name: "My API Tests"',
+    docs: 'https://qa360.ai/docs/pack-schema'
+  }
+);
+```
+
+---
+
+## Conventional Commits
+
+```
+feat(scope): description     # New feature
+fix(scope): description      # Bug fix
+docs(scope): description     # Documentation
+test(scope): description     # Tests only
+refactor(scope): description # Refactoring
+chore(scope): description    # Build/config/deps
+```
+
+Examples:
+- `feat(core): add auto-repair module`
+- `fix(cli): resolve verification edge case`
+- `docs: update STATUS.md for Phase 7`
+
+---
+
+## Key Dependencies
+
+| Package | Purpose | Version |
+|---------|---------|---------|
+| commander | CLI framework | ^12.1.0 |
+| chalk | Terminal colors | ^5.3.0 |
+| vitest | Test runner | ^1.6.0 |
+| typescript | Language | ^5.3.3 |
+| tweetnacl | Ed25519 crypto | ^1.0.3 |
+| better-sqlite3 | SQLite | ^9.4.3 |
+| ollama | LLM client (optional) | ^0.5.1 |
+
+---
+
+## Environment Variables
+
+```bash
+# LLM Provider
+LLM_PROVIDER=ollama|openai|anthropic|mock
+LLM_MODEL=llama3.2
+LLM_BASE_URL=http://localhost:11434
+LLM_API_KEY=sk-xxx
+
+# QA360
+QA360_HOME=~/.qa360
+QA360_VERBOSE=1
+```
+
+---
+
+## Documentation Files
+
+Only 3 documentation files exist (by design):
+
+1. **STATUS.md** - Project status, phases, metrics (THE SOURCE OF TRUTH)
+2. **DEVELOPMENT.md** - This file (developer guide)
+3. **ARCHITECTURE.md** - Technical architecture details
+
+All other docs are archived or should be integrated into these 3 files.
+
+---
+
+## Support
+
+- **Issues**: https://github.com/xyqotech/qa360/issues
+- **Discussions**: https://github.com/xyqotech/qa360/discussions
+- **License**: MIT

package/docs/DEVELOPMENT_HISTORY.md

@@ -0,0 +1,345 @@
+# QA360 Development History
+
+This document provides a historical record of QA360's development phases.
+
+> **Note**: For current project status, see [STATUS.md](./STATUS.md).
+
+---
+
+## Phase 1: Runner CLI "Zero Friction"
+
+**Completed**: October 2024
+
+### Objectives
+- Develop a CLI enabling "1 command • 1 pack • 1 signed proof"
+- Implement anti-friction features for seamless user experience
+
+### Delivered Features
+
+#### Core Commands
+- `qa360 doctor` - System diagnostics with auto-fix capabilities
+- `qa360 ask` - Pack generation from natural language
+- `qa360 run` - Test execution with hooks support
+- `qa360 report --sign` - PDF generation with Ed25519 signatures
+- `qa360 verify` - Cryptographic proof verification
+- `qa360 explain` - Error code explanations
+
+#### Anti-Friction Features
+- **Portable cache** (`.qa360/tools/`) for downloaded tools
+- **Docker fallback** for missing tools
+- **Pedagogical error messages** (QX001-QX007)
+- **Targeted retry** with exponential backoff
+- **Automatic log redaction** for PII and secrets
+
+### Architecture
+- Monorepo structure with CLI and Core separation
+- Ed25519 cryptographic signatures
+- Trust Score calculation
+- PDF report generation with Puppeteer
+
+### Metrics
+- **Tests**: 26/28 passing (93% success)
+- **Commands**: 7 implemented
+- **Error codes**: QX001-QX007 with guidance
+- **Performance**: <1s pack generation, <100ms verification
+
+---
+
+## Phase 2: Pack v1 + Encrypted Secrets
+
+**Completed**: October 2024
+
+### Objectives
+- Standardize Pack v1 schema
+- Implement encrypted secrets management
+- Ensure deterministic proof generation
+
+### Delivered Features
+
+#### Pack Schema v1
+- Official JSON schema (`pack.schema.json`)
+- Required fields: version, name, gates
+- Supported gates: api_smoke, ui, perf, sast, dast, a11y
+- Validation with 15+ business rules (QP### error codes)
+- Automatic migration from v0.9 to v1
+
+#### Pack CLI Commands
+- `qa360 pack validate` - Schema and business rule validation
+- `qa360 pack lint` - Auto-correction of format issues
+- `qa360 pack migrate` - Legacy to v1 migration with backup
+
+#### Secrets Management
+- **Encryption**: AES-256-GCM with PBKDF2 (100k iterations)
+- **Key derivation**: System keychain (macOS/Windows/Linux)
+- **Storage**: `.qa360/secrets.enc` with 600 permissions
+- **Integrity**: SHA-256 checksum + GCM authentication
+
+#### Secrets CLI Commands
+- `qa360 secrets add` - Secure secret addition
+- `qa360 secrets list` - List secret names (values hidden)
+- `qa360 secrets remove` - Delete with confirmation
+- `qa360 secrets doctor` - System diagnostics
+- `qa360 secrets export --redacted` - Masked export for debugging
+
+#### Secret Interpolation
+- Syntax: `${{ secrets.NAME }}` in pack.yml
+- Automatic validation before execution
+- Zero secrets leakage in logs/reports
+
+#### Security Redaction
+- 15+ patterns for masking (passwords, tokens, API keys, JWT, etc.)
+- Applied to logs, reports, JSON objects, errors
+- Length-preserving or fixed-format masking
+
+### Architecture
+- **PackValidator** - Schema + business rule validation
+- **PackMigrator** - Automatic version migration
+- **SecretsManager** - CRUD with secure caching
+- **SecretsCrypto** - AES-256-GCM encryption
+- **SecurityRedactor** - Automatic PII masking
+
+### Metrics
+- **Tests**: 79/87 passing (91% success)
+- **Commands**: 12 new commands (pack + secrets)
+- **Error codes**: 20+ QP### codes
+- **Security**: AES-256-GCM, PBKDF2 100k iterations
+- **Performance**: <100ms validation, <1s encryption
+
+### Impact
+- **Determinism**: Same pack.yml → same proof everywhere
+- **Security**: Secrets encrypted locally, never in VCS
+- **Portability**: Pack v1 works on all environments
+- **Compliance**: Military-grade encryption, strict permissions
+
+---
+
+## Phase 3: Declarative Hooks + Phase3Runner
+
+**Completed**: December 2024
+
+### Objectives
+- Implement Phase3Runner orchestration engine
+- Connect CLI commands to core engine
+- Establish cryptographic proof foundation
+
+### Delivered Features
+
+#### Phase3Runner
+- Gate orchestration with lifecycle management
+- Ed25519 real signatures (production-ready)
+- Evidence Vault with SQLite storage
+- Trust Score calculation
+- Proof bundle generation
+
+#### CLI Integration
+- `qa360 run` connected to Phase3Runner
+- `qa360 verify` connected to Ed25519 verifier
+- Phase3 proof format support
+- Graceful error handling
+
+#### Evidence System
+- SQLite with WAL mode
+- Content-addressable storage (CAS)
+- Immutable audit trail
+- Historical run tracking
+
+### Architecture
+- **Phase3Runner** - Core orchestration engine
+- **ProofGenerator** - Cryptographic proof creation
+- **SignatureManager** - Ed25519 signing/verification
+- **EvidenceVault** - SQLite storage layer
+
+### Metrics
+- **Completion**: Phase3Runner fully operational
+- **Signatures**: Ed25519 real implementation
+- **Storage**: SQLite WAL mode
+- **Integration**: CLI commands connected
+
+---
+
+## Phase 4: OOTB Adapters
+
+**Completed**: December 2024
+
+### Objectives
+- Implement production-ready test adapters
+- Support multiple testing domains (API, UI, performance, security)
+- Enable comprehensive quality gates
+
+### Delivered Adapters (7 total)
+
+1. **playwright-api** - REST/GraphQL API testing
+2. **playwright-ui** - UI + Accessibility testing (axe-core)
+3. **k6-perf** - Load & performance testing
+4. **semgrep-sast** - Static security analysis
+5. **zap-dast** - Dynamic security scanning (Docker/local, **REAL implementation**)
+6. **gitleaks-secrets** - Secret detection in code
+7. **osv-deps** - Dependency vulnerability scanning
+
+### Features
+- Unified adapter interface
+- Docker and local execution support
+- Budget threshold validation
+- Comprehensive error codes
+- Production-ready implementations
+
+### Metrics
+- **Adapters**: 7 production-ready
+- **Coverage**: API, UI, performance, security, accessibility, dependencies
+- **Docker**: Full container support
+- **Local**: Native execution support
+
+---
+
+## Phase 5: Health/Diag/Metrics Endpoints
+
+**Completed**: December 2024
+
+### Objectives
+- Implement observability server
+- Enable CI/CD readiness checks
+- Provide Prometheus metrics
+
+### Delivered Features
+
+#### Observability Server
+- `qa360 serve` command with HTTP server
+- `/health` - Liveness and readiness checks
+- `/diag` - Diagnostic with actionable solutions
+- `/metrics` - OpenMetrics/Prometheus format
+- `/cancel/:runId` - Graceful shutdown
+
+#### Health Checks
+- System validation (tmp, config, disk space)
+- Tool detection (Playwright, k6, ZAP)
+- Secret accessibility checks
+- Readiness scoring
+
+#### Metrics
+- System metrics (uptime, version, runs)
+- Gate metrics (duration, failures by type)
+- Security metrics (findings by severity)
+- Performance metrics (P95, P90, P50, error rate)
+
+### Architecture
+- **QA360Server** - HTTP routing and error handling
+- **HealthChecker** - Liveness/readiness validation
+- **DiagnosticsCollector** - Problem detection + solutions
+- **MetricsCollector** - OpenMetrics/Prometheus export
+- **ProcessManager** - Graceful shutdown management
+
+### Metrics
+- **Endpoints**: 4 operational
+- **CI/CD**: Readiness check with exit 0/1
+- **Monitoring**: Prometheus-compatible
+- **Shutdown**: Graceful with cleanup
+
+---
+
+## Phase 6: Evidence Vault + SQLite WAL History
+
+**Completed**: December 2024
+
+### Objectives
+- Implement persistent evidence storage
+- Enable historical run tracking
+- Provide content-addressable storage
+
+### Delivered Features
+
+#### Evidence Vault
+- SQLite database with WAL mode
+- Content-addressable storage (SHA-256 based)
+- Immutable audit trail
+- Run metadata tracking
+
+#### Features
+- Historical run queries
+- Trend analysis support
+- Artifact deduplication
+- Integrity verification
+
+### Architecture
+- **EvidenceVault** - SQLite storage layer
+- **HistoryManager** - Run tracking
+- **AuditTrail** - Immutable records
+
+---
+
+## Current Status
+
+As of December 2024:
+- **Phases 1-6**: ✅ Complete
+- **Phase 7**: 🚧 In Progress (Complete Proof Bundle - 90%)
+- **Phase 8+**: ⏳ Planned
+
+### Key Metrics
+- **412 tests** passing (100% success rate)
+- **7 adapters** in production
+- **9 CLI commands** fully functional
+- **~92% overall completion**
+
+For detailed current status, see [STATUS.md](./STATUS.md).
+
+---
+
+## Technology Stack Evolution
+
+### Core Technologies
+- **Runtime**: Node.js 18+
+- **Language**: TypeScript 5+
+- **Testing**: Jest
+- **Monorepo**: pnpm workspaces
+
+### Security
+- **Signatures**: Ed25519 (tweetnacl)
+- **Hashing**: SHA-256
+- **Encryption**: AES-256-GCM
+- **Key Derivation**: PBKDF2 (100k iterations)
+- **Validation**: JSON Schema (AJV 2020-12)
+
+### Storage
+- **Database**: SQLite with WAL mode
+- **Content Addressing**: SHA-256 based CAS
+- **Permissions**: Strict 600 for sensitive files
+
+### Adapters
+- **UI Testing**: Playwright
+- **API Testing**: Playwright
+- **Performance**: K6
+- **SAST**: Semgrep
+- **DAST**: OWASP ZAP
+- **Secrets**: Gitleaks
+- **Dependencies**: OSV
+
+---
+
+## Lessons Learned
+
+### What Worked Well
+1. **Incremental phases** - Clear milestones and deliverables
+2. **Security-first** - Built-in encryption and redaction from start
+3. **Local-first** - No cloud dependencies, fully portable
+4. **Developer experience** - Anti-friction features throughout
+
+### Challenges Overcome
+1. **ESM/Jest compatibility** - Resolved with inline schemas and mocks
+2. **Cross-platform secrets** - Unified keychain approach
+3. **Docker integration** - Seamless fallback mechanisms
+4. **Proof determinism** - Canonical JSON and stable hashing
+
+### Future Improvements
+1. Test coverage expansion (55% → 75%+)
+2. Additional adapters (Appium for mobile)
+3. Enhanced TUI/UI dashboard
+4. Multi-language SDK support
+
+---
+
+## References
+
+- [Architecture Overview](./ARCHITECTURE.md)
+- [Current Status](./STATUS.md)
+- [Pack v1 Specification](./pack-v1.md)
+- [Secrets Management](./secrets.md)
+- [Proof Bundles](./proofs.md)