icdev 0.0.3__py3-none-any.whl

docs/implementation-plan-architecture-evolution.md

@@ -0,0 +1,614 @@
+# CUI // SP-CTI
+# ICDEV Architecture Evolution — Implementation Plan
+
+**Date:** 2026-03-08
+**Classification:** CUI // SP-CTI
+**Based on:** 3 research documents (3,300+ lines, 100+ sources)
+**Scope:** Software architecture, design patterns, and development best practices for ICDEV
+
+---
+
+## Executive Summary
+
+This plan consolidates findings from three research reports into 6 implementation phases spanning ~12 weeks. Each phase delivers independently valuable capabilities while building toward the target architecture. Work items are ordered by impact-to-effort ratio with compliance and security improvements front-loaded.
+
+**Current State:** ICDEV is a 6-layer GOTCHA-based, 12-agent platform with 613 Python files, 42 tool directories, 348+ DB tables, 12 MCP servers, and 50+ DataConnectors. Architecture is solid but lacks formal patterns for resilience, observability, agent interoperability, and FedRAMP 20x readiness.
+
+**Target State:** Industry-standard C4-documented, DDD-bounded, resilient, FedRAMP 20x-ready platform with formal A2A/MCP compliance, semantic caching, corrective RAG, and property-based test coverage.
+
+---
+
+## Phase 1 — Foundation: Resilience & Code Quality (Weeks 1-2)
+
+**Goal:** Harden core infrastructure with circuit breakers, error hierarchy, DLQ, and pipeline gates.
+
+### 1.1 Circuit Breaker in LLM Router
+**Priority:** Critical | **Effort:** 1 day | **Impact:** Prevents cascade failures
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/core/circuit_breaker.py` (new) |
+| **Integrates with** | `tools/llm/router.py` — wrap provider probing |
+| **States** | Closed → Open (after 5 failures in 60s) → Half-Open (probe after 300s) |
+| **Config** | `args/resilience_config.yaml` — per-provider thresholds |
+| **Tests** | `tests/test_circuit_breaker.py` — 17 tests (state transitions, threading, reset) |
+| **ADR** | D-ARCH-2: Circuit breaker with 3 states in LLM router |
+
+**Implementation steps:**
+1. Create `tools/core/circuit_breaker.py` with `CircuitBreaker` class (3 states, thread-safe)
+2. Create `args/resilience_config.yaml` with per-provider configuration
+3. Integrate into `router.py` `_availability_cache` — replace boolean cache with breaker state
+4. Add `_failure_counts` and `_breaker_state` dicts to router
+5. Write tests covering: normal flow, trip threshold, half-open recovery, concurrent access
+6. Update `tools/manifest.md`
+
+### 1.2 Structured Error Hierarchy
+**Priority:** High | **Effort:** 0.5 day | **Impact:** Consistent debugging across 12 agents
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/core/errors.py` (new) |
+| **Classes** | 14 error classes across 5 domains (agent, compliance, LLM, storage, security) |
+| **Pattern** | All inherit `ICDEVError` with `code`, `retryable`, `context` fields |
+| **Tests** | `tests/test_core_errors.py` — 16 tests |
+
+**Key error classes:**
+- `ICDEVError` (base) → `AgentUnavailableError`, `AgentTimeoutError`
+- `ComplianceGateError` → `CUIMarkingMissingError`, `STIGFindingError`
+- `LLMFallbackExhaustedError` → `LLMProviderError`, `LLMBudgetExceededError`
+- `StorageConnectionError` → `StorageMigrationError`
+- `SecurityGateError` → `PromptInjectionError`, `TrustScoreError`
+
+### 1.3 Dead Letter Queue for Agent Tasks
+**Priority:** High | **Effort:** 1 day | **Impact:** Prevents lost work
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/core/task_dlq.py` (new) |
+| **Table** | `agent_task_dlq` (append-only, NIST AU compliant) |
+| **Pattern** | Dispatch → retry (3x, exponential backoff + jitter) → DLQ |
+| **Config** | `args/resilience_config.yaml` — max_retries, backoff_base, jitter |
+| **Tests** | `tests/test_task_dlq.py` — 8 tests |
+| **Dashboard** | Add DLQ widget to `/health` endpoint |
+
+### 1.4 Pipeline Gates YAML
+**Priority:** High | **Effort:** 0.5 day | **Impact:** Enforces gate ordering
+
+| Item | Detail |
+|------|--------|
+| **File** | `args/pipeline_gates.yaml` (new) |
+| **Gates** | pre_commit, build, pre_deploy, post_deploy |
+| **Enforcer** | `tools/ci/gate_enforcer.py` — reads YAML, runs tools, blocks on failures |
+| **Compliance** | Maps to cATO Evaluation Criteria stages |
+
+### 1.5 Dependency Injection Container
+**Priority:** Medium | **Effort:** 0.5 day | **Impact:** Test isolation
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/core/container.py` (new) |
+| **Pattern** | Lightweight `ServiceContainer` with `register()` / `resolve()` |
+| **Services** | db, llm_router, audit, event_bus |
+| **Tests** | `tests/test_container.py` — 9 tests |
+
+### 1.6 Graceful Degradation Matrix
+**Priority:** Medium | **Effort:** 0.5 day | **Impact:** Documents failure modes
+
+| Item | Detail |
+|------|--------|
+| **File** | `args/degradation_matrix.yaml` (new) |
+| **Scenarios** | 8 failure scenarios (cloud LLM, PostgreSQL, Ollama, network, each agent type) |
+| **Consumed by** | Orchestrator agent for automated degradation decisions |
+
+### Phase 1 Deliverables
+
+| Deliverable | Files | Tests |
+|------------|-------|-------|
+| Circuit breaker | `tools/core/circuit_breaker.py`, `args/resilience_config.yaml` | 17 |
+| Error hierarchy | `tools/core/errors.py` | 16 |
+| Dead letter queue | `tools/core/task_dlq.py` | 8 |
+| DI container | `tools/core/container.py` | 9 |
+| Pipeline gates | `args/pipeline_gates.yaml`, `tools/ci/gate_enforcer.py` | 5 |
+| Degradation matrix | `args/degradation_matrix.yaml` | — |
+| **Total** | **8 new files** | **55 tests** |
+
+---
+
+## Phase 2 — Architecture Visibility: C4, DDD, ADRs (Weeks 3-4)
+
+**Goal:** Formalize architecture with C4 diagrams, bounded contexts, domain events, and extracted ADRs.
+
+### 2.1 C4 Architecture-as-Code (Structurizr DSL)
+**Priority:** High | **Effort:** 2 days | **Impact:** PR-reviewable architecture changes
+
+| Item | Detail |
+|------|--------|
+| **File** | `docs/architecture/icdev.dsl` (new) |
+| **Levels** | L1: System Context (10 external actors), L2: Container (17 containers), L3: Component (Compliance Agent detailed) |
+| **Export** | PlantUML, Mermaid, PNG/SVG |
+| **ADR** | D-ARCH-1: Structurizr DSL for architecture-as-code |
+
+**C4 Level mapping to GOTCHA:**
+
+| GOTCHA Layer | C4 Level | Contents |
+|---|---|---|
+| Goals, Context, Hard Prompts | N/A (non-runtime) | Process definitions |
+| Orchestration (Claude) | L2 Container | Orchestrator Agent |
+| Tools | L3 Components | 255+ tool scripts |
+| Args | L3 (config artifacts) | 35 YAML/JSON configs |
+
+### 2.2 Domain-Driven Design: Bounded Contexts
+**Priority:** High | **Effort:** 2 days | **Impact:** Architectural clarity and maintainability
+
+**12 bounded contexts (matching tools/ structure):**
+
+| Bounded Context | Directory | Core Aggregate | Key Invariant |
+|---|---|---|---|
+| Compliance | `tools/compliance/` | `ComplianceControl` | Cannot be "Implemented" without non-expired evidence |
+| Security | `tools/security/` | `ThreatModel` | Every STRIDE threat maps to >= 1 NIST control |
+| Requirements | `tools/requirements/` | `IntakeSession` | Cannot pass Stage 3 without readiness >= 0.7 |
+| Simulation | `tools/simulation/` | `Scenario` | Monte Carlo requires >= 1000 iterations |
+| DevSecOps | `tools/devsecops/` | `DevSecOpsProfile` | IL4+ requires ZTA maturity >= Advanced |
+| Supply Chain | `tools/supply_chain/` | `DependencyGraph` | All vendors must have SCRM assessment |
+| MBSE | `tools/mbse/` | `DigitalThread` | Every model element must have trace link |
+| Embedded | `tools/embedded/` + `fleet/` + `edge_ai/` | `Device` | Deployed firmware must have SBOM |
+| Observability | `tools/observability/` | `Trace` | Active tracing required for production |
+| GovCon | `tools/govcon/` | `Proposal` | Read-only bridge to compliance (D-WG-8) |
+| CloudForge | `tools/cloudforge/` | `Runbook` | DAG execution via Kahn's algorithm |
+| Knowledge | `tools/memory/` + `tools/rag/` | `MemoryEntry` | Hybrid search: 0.7 BM25 + 0.3 semantic |
+
+**Context map relationships:**
+
+```
+Compliance <--[Conformist]--> Security
+Requirements <--[Customer/Supplier]--> Compliance
+Requirements <--[Customer/Supplier]--> Simulation
+MBSE <--[Partnership]--> Requirements
+Supply Chain <--[ACL]--> External Vendors
+Embedded <--[ACL]--> Compliance (different frameworks)
+GovCon <--[Published Language]--> Compliance (read-only)
+CloudForge <--[Shared Kernel]--> DevSecOps
+```
+
+**Deliverable:** `docs/architecture/bounded-contexts.md` documenting all 12 contexts, aggregates, invariants, and relationships.
+
+### 2.3 Domain Event Bus
+**Priority:** High | **Effort:** 2 days | **Impact:** Reactive cross-context communication
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/events/event_bus.py` (new) |
+| **Pattern** | In-process pub/sub via `queue.Queue`; NATS for K8s deployment |
+| **Events** | 11 domain events (see table below) |
+| **Audit** | Events simultaneously written to append-only audit trail (D6) |
+| **ADR** | D-ARCH-4: Domain event bus (in-process Queue, NATS for K8s) |
+
+**Core domain events:**
+
+| Event | Source | Consumers | Reaction |
+|---|---|---|---|
+| `ControlImplemented` | Compliance | Security, GovCon | Update threat posture; update claims |
+| `ControlEvidenceExpired` | Compliance | cATO, Dashboard | Flag re-collection; alert ISSO |
+| `VulnerabilityDiscovered` | Security | Compliance, Supply Chain | Create POAM; flag vendor |
+| `ThreatModelCreated` | Security | Compliance, DevSecOps | Map threats; update policies |
+| `RequirementDecomposed` | Requirements | MBSE, Simulation | Create trace links; seed scenarios |
+| `ATOBoundaryChanged` | Requirements | Compliance, DevSecOps | Re-assess controls; regen policies |
+| `FirmwareDeployed` | Embedded | Compliance, Security | Regen SBOM; trigger scan |
+| `TrustScoreDecayed` | Security | Orchestrator | Restrict agent permissions |
+| `SagaStepCompleted` | Orchestrator | Audit, Monitor | Log progress |
+| `SagaCompensationTriggered` | Orchestrator | Affected agents | Execute rollback |
+| `PeerInteractionRecorded` | Any agent | Orchestrator, Audit | Verify authorization |
+
+### 2.4 Extract ADRs from CLAUDE.md
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Discoverability
+
+| Item | Detail |
+|------|--------|
+| **Directory** | `docs/adr/` (new) |
+| **Format** | One file per decision: `docs/adr/D-DB-20-postgresql-primary.md` |
+| **Fields** | Status (Accepted/Deprecated/Superseded), Date, Context, Decision, Consequences |
+| **Grouping** | D-DB-*, D-CF-*, D-INV-*, D-SBD-*, D-ARCH-*, D-WG-*, D-MKT-*, D-HARNESS-* |
+| **Cross-ref** | Keep summary table in CLAUDE.md, link to full ADRs |
+
+### Phase 2 Deliverables
+
+| Deliverable | Files |
+|------------|-------|
+| C4 Structurizr DSL | `docs/architecture/icdev.dsl` |
+| Bounded contexts doc | `docs/architecture/bounded-contexts.md` |
+| Domain event bus | `tools/events/event_bus.py`, `tools/events/__init__.py` |
+| ADR directory | `docs/adr/` (40+ individual ADR files) |
+| **Total** | **45+ new files** |
+
+---
+
+## Phase 3 — Compliance Acceleration: FedRAMP 20x & cATO (Weeks 5-7)
+
+**Goal:** Position ICDEV for 3-month FedRAMP authorization via KSI emission, OSCAL enhancement, and control inheritance formalization.
+
+### 3.1 FedRAMP 20x KSI Emitter
+**Priority:** Critical | **Effort:** 3 days | **Impact:** 3-month authorization vs 18+ months
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/compliance/fedramp_20x_ksi_emitter.py` (new) |
+| **Input** | cATO Live Engine evidence streams |
+| **Output** | Machine-readable JSON per KSI category |
+| **Categories** | Vulnerability management, config compliance, access control, encryption, patch currency |
+| **Endpoint** | Automated validation endpoint for 3PAO |
+| **ADR** | D-INV-50: FedRAMP 20x KSI emitter for continuous posture reporting |
+
+**Architecture:**
+```
+Evidence Collectors (existing)
+    → cATO Live Engine (existing, D-INV-1)
+        → KSI Emitter (NEW)
+            → Machine-readable KSI JSON
+            → 3PAO validation endpoint
+            → FedRAMP continuous reporting
+```
+
+### 3.2 OSCAL Enhancements
+**Priority:** High | **Effort:** 2 days | **Impact:** Machine-readable compliance packages
+
+| Enhancement | File | Detail |
+|---|---|---|
+| Component Definitions | `tools/compliance/oscal_generator.py` | Generate reusable component definitions for ICDEV platform controls |
+| Profile generation | `tools/compliance/oscal_generator.py` | Tailored baselines per impact level (IL2, IL4, IL5) |
+| Schema validation gate | `tools/compliance/oscal_validator.py` (new) | Validate against NIST OSCAL schemas before output |
+| 20x metadata extensions | `tools/compliance/oscal_generator.py` | FedRAMP 20x metadata in all OSCAL packages |
+
+### 3.3 Control Inheritance Chain
+**Priority:** High | **Effort:** 2 days | **Impact:** Automated compliance for child apps
+
+**Three-tier inheritance:**
+```
+Tier 1: AWS GovCloud → 46+ inherited controls (PE-*, SC-7 partial, SC-39)
+Tier 2: ICDEV Platform → AC-2, AC-3, AU-2, AU-3, CM-2, IA-2, SI-2 (implemented by tools)
+Tier 3: Child Apps → Application-specific controls (inherited via Golden Path)
+```
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/compliance/inheritance_engine.py` (new) |
+| **Data** | `args/control_inheritance.yaml` — declares inherited/shared/app-specific per control |
+| **Output** | OSCAL Component Definitions with inheritance annotations |
+| **Integration** | Crosswalk engine auto-populates inheritance during SSP generation |
+
+### 3.4 RAGAS Quality Gate for Narratives
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Reduces hallucination in AI-generated narratives
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/compliance/narrative_quality_gate.py` (new) |
+| **Metrics** | context_relevancy, faithfulness, answer_relevancy (RAGAS) |
+| **Gate** | Score >= 0.75 required before narrative enters approval queue |
+| **Integration** | Hooks into `narrative_workflow.py` between generation and pending_review |
+
+### 3.5 AI RMF Crosswalk
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Automated AI governance compliance
+
+| Item | Detail |
+|------|--------|
+| **File** | `context/compliance/ai_rmf_crosswalk.yaml` (new) |
+| **Framework key** | `ai_rmf` in crosswalk engine |
+| **Mapping** | GOVERN → policy_generator, sbd_assessor; MAP → ai_bom_generator; MEASURE → atlas_assessor, owasp_llm_assessor, agent_shap; MANAGE → prompt_injection_detector, ai_telemetry_logger |
+| **Integration** | `crosswalk_engine.py` — running any mapped tool auto-populates AI RMF status |
+
+### Phase 3 Deliverables
+
+| Deliverable | Files | Tests |
+|------------|-------|-------|
+| KSI emitter | `tools/compliance/fedramp_20x_ksi_emitter.py` | 12 |
+| OSCAL enhancements | `tools/compliance/oscal_validator.py` + edits | 8 |
+| Inheritance engine | `tools/compliance/inheritance_engine.py`, `args/control_inheritance.yaml` | 10 |
+| RAGAS quality gate | `tools/compliance/narrative_quality_gate.py` | 6 |
+| AI RMF crosswalk | `context/compliance/ai_rmf_crosswalk.yaml` + crosswalk edits | 4 |
+| **Total** | **5 new files + 4 edits** | **40 tests** |
+
+---
+
+## Phase 4 — Agent Evolution: A2A, MCP, Trust (Weeks 7-9)
+
+**Goal:** Adopt industry-standard agent protocols, implement trust tiers, and enable peer-to-peer communication.
+
+### 4.1 A2A v0.2 Agent Card Migration
+**Priority:** High | **Effort:** 2 days | **Impact:** Ecosystem interoperability
+
+| Item | Detail |
+|------|--------|
+| **Files** | `tools/agent/cards/*.json` (update all 12) |
+| **Schema** | Linux Foundation A2A v0.2 with `authentication`, `capabilities`, `skills` fields |
+| **Task lifecycle** | submitted → working → input-required → completed/failed |
+| **Table** | Update `agent_tasks` to use formal lifecycle states |
+| **ADR** | D-ARCH-5: A2A spec-compliant Agent Cards |
+
+### 4.2 ATLAS Saga Coordinator
+**Priority:** High | **Effort:** 3 days | **Impact:** Automated workflow recovery
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/orchestration/saga_coordinator.py` (new) |
+| **Tables** | `saga_executions`, `saga_steps` (append-only) |
+| **Pattern** | Orchestration-based saga (centralized coordinator in Orchestrator) |
+| **Compensations** | Each ATLAS phase registers a rollback action |
+| **ADR** | D-ARCH-6: ATLAS saga coordinator with compensation registry |
+
+**ATLAS saga steps and compensations:**
+
+| Phase | Action | Compensation |
+|---|---|---|
+| M-ATLAS Model | Import SysML/ReqIF | Remove imported model elements |
+| Architect | System design, decompose | Archive design artifacts |
+| Trace | Traceability matrix | Remove trace links for this session |
+| Link | Wire components, DI | Unregister components |
+| Assemble | Build, test, integrate | Rollback to pre-build state |
+| Stress_test | Load test, security scan | Mark results as invalidated |
+
+### 4.3 CSA Agentic Trust Framework Tiers
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Industry-standard trust model
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/security/agent_trust_scorer.py` (update) |
+| **Tiers** | Intern (read-only) → Junior (recommend, require approval) → Senior (execute approved types) → Principal (full autonomy in bounded context) |
+| **Promotion** | Score-based with minimum task count and time-in-tier requirements |
+| **Decay** | Trust score decays 5%/week without positive signals |
+| **ADR** | D-ARCH-7: CSA ATF trust tiers |
+
+### 4.4 Selective Peer-to-Peer Agent Channels
+**Priority:** Medium | **Effort:** 2 days | **Impact:** Reduces orchestrator bottleneck
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/orchestration/peer_channels.py` (new) |
+| **Authorized pairs** | Security↔Compliance, Builder↔Knowledge, Requirements↔Simulation, MBSE↔Requirements, DevSecOps↔CloudForge |
+| **Observability** | All peer interactions reported to Orchestrator via `PeerInteractionRecorded` event |
+| **ADR** | D-ARCH-8: Selective peer-to-peer agent channels |
+
+### 4.5 Lazy MCP Tool Loading
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Reduces context window waste
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/mcp/tool_registry.py` (new) |
+| **Pattern** | Load tool definitions on-demand based on declared session purpose |
+| **Mapping** | Session purpose → relevant MCP servers → load only those tools |
+| **Savings** | ~60% context reduction (from 80+ tools to ~20-30 per session) |
+
+### Phase 4 Deliverables
+
+| Deliverable | Files | Tests |
+|------------|-------|-------|
+| A2A Agent Cards | 12 updated card files | 12 |
+| Saga coordinator | `tools/orchestration/saga_coordinator.py` | 15 |
+| Trust tiers | `tools/security/agent_trust_scorer.py` (update) | 8 |
+| Peer channels | `tools/orchestration/peer_channels.py` | 10 |
+| Lazy MCP loading | `tools/mcp/tool_registry.py` | 6 |
+| **Total** | **3 new + 13 updated** | **51 tests** |
+
+---
+
+## Phase 5 — Intelligence: Semantic Cache, RAG, Budget Routing (Weeks 9-11)
+
+**Goal:** Reduce LLM costs 40-60% via semantic caching and budget-aware routing; improve RAG accuracy with corrective retrieval.
+
+### 5.1 Semantic Caching in LLM Router
+**Priority:** Critical | **Effort:** 3 days | **Impact:** 40-60% token cost reduction
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/llm/semantic_cache.py` (new) |
+| **Table** | `llm_semantic_cache` (query_embedding, response, ttl, hit_count) |
+| **Tiers** | Exact match (sub-ms) → Semantic match (cosine > 0.92) → Cache miss (invoke LLM) |
+| **TTL** | 30 days for compliance, 7 days for code analysis, 1 day for dynamic queries |
+| **Embeddings** | Reuse existing `text-embedding-3-small` infrastructure |
+| **Integration** | `router.py` checks cache before provider invocation |
+
+**Expected impact by function tier:**
+
+| Tier | Cache Hit Rate | Token Savings |
+|---|---|---|
+| Scanner (compliance_export, narrative_generation) | 60-70% | High — these are repetitive |
+| Worker (code_generation, wg_rewrite) | 20-30% | Moderate |
+| Planner (intake_persona_response) | 5-10% | Low — highly contextual |
+
+### 5.2 Budget-Aware Routing
+**Priority:** High | **Effort:** 1 day | **Impact:** Per-project cost control
+
+| Item | Detail |
+|------|--------|
+| **Config** | `args/llm_config.yaml` — add `token_budgets` section per project |
+| **Logic** | Track cumulative spend in audit trail; auto-downgrade tier when approaching budget |
+| **Thresholds** | 80% budget → warn; 90% → downgrade Worker→Scanner; 100% → block Planner |
+| **Dashboard** | Token budget widget on project status page |
+
+### 5.3 Corrective RAG for Compliance Retrieval
+**Priority:** High | **Effort:** 2 days | **Impact:** Reduces hallucination in narratives
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/rag/corrective_rag.py` (new) |
+| **Pattern** | Retrieve → Evaluate relevance → Correct (re-query/refine/fallback) → Generate |
+| **Evaluator** | Lightweight classifier scoring document relevance (0-1) |
+| **Thresholds** | > 0.8: use as-is; 0.5-0.8: refine query; < 0.5: decompose into sub-queries |
+| **Integration** | `narrative_workflow.py` uses corrective RAG instead of direct hybrid_search |
+
+### 5.4 Prompt Versioning and Provenance
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Reproducibility and A/B testing
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/llm/prompt_registry.py` (new) |
+| **Table** | `prompt_versions` (prompt_id, content_hash, version, metrics, active) |
+| **Pattern** | SHA-256 content hash (mirrors D-INV-5 template provenance) |
+| **A/B** | Route 10% traffic to candidate prompt, compare RAGAS scores |
+| **Rollback** | Revert to last known-good prompt on regression detection |
+
+### 5.5 Output Verification Gate (Prompt Injection Layer 3)
+**Priority:** High | **Effort:** 1 day | **Impact:** Closes prompt injection defense gap
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/security/output_verifier.py` (new) |
+| **Checks** | Leaked system prompts, tool credentials, DB connection strings, classification overflow |
+| **Integration** | Post-generation gate in `router.py` |
+| **Goal-lock** | Once session purpose declared, reject deviating tool invocations |
+
+### Phase 5 Deliverables
+
+| Deliverable | Files | Tests |
+|------------|-------|-------|
+| Semantic cache | `tools/llm/semantic_cache.py` | 12 |
+| Budget routing | `args/llm_config.yaml` update, `router.py` update | 8 |
+| Corrective RAG | `tools/rag/corrective_rag.py` | 10 |
+| Prompt registry | `tools/llm/prompt_registry.py` | 8 |
+| Output verifier | `tools/security/output_verifier.py` | 10 |
+| **Total** | **4 new + 2 updated** | **48 tests** |
+
+---
+
+## Phase 6 — Testing & DevSecOps Maturity (Weeks 11-12)
+
+**Goal:** Property-based testing, contract tests, SLSA Level 2, compliance sidecar, and feature flags.
+
+### 6.1 Property-Based Testing (Hypothesis)
+**Priority:** Critical | **Effort:** 2 days | **Impact:** 50x more bugs per test (OOPSLA 2025)
+
+| Item | Detail |
+|------|--------|
+| **File** | `tests/test_compliance_properties.py` (new) |
+| **Targets** | Crosswalk engine, NIST lookup, ZTA scorer, control mapper, SbD assessor |
+| **Pattern** | Generate random valid control IDs → assert invariants always hold |
+| **Dependency** | `hypothesis` package |
+
+**Example properties:**
+- Any valid NIST control maps to >= 1 framework
+- Crosswalk is idempotent (running twice produces same result)
+- ZTA maturity score is always in [0.0, 1.0]
+- SbD assessment never produces empty requirements list
+- Trust score decay is monotonically decreasing
+
+### 6.2 A2A Contract Tests
+**Priority:** High | **Effort:** 1 day | **Impact:** Prevents integration failures
+
+| Item | Detail |
+|------|--------|
+| **Directory** | `tests/contracts/` (new) |
+| **Pattern** | Consumer-driven contracts (Pact-style) with JSON Schema |
+| **Coverage** | All 12 agent interfaces (method + params_schema + result_schema) |
+| **Tests** | 27 tests across 9 agent contracts |
+
+### 6.3 Snapshot Tests for Compliance Artifacts
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Catches unintended output regressions
+
+| Item | Detail |
+|------|--------|
+| **File** | `tests/test_compliance_snapshots.py` (new) |
+| **Targets** | SSP, POAM, SBOM, OSCAL outputs |
+| **Dependency** | `syrupy` pytest plugin |
+| **Pattern** | Normalize timestamps/UUIDs → compare against snapshot |
+
+### 6.4 SLSA Level 2 Supply Chain Security
+**Priority:** High | **Effort:** 2 days | **Impact:** Provenance attestation
+
+| Item | Detail |
+|------|--------|
+| **File** | `.github/workflows/slsa-build.yml` (new) |
+| **Level 1** | Build provenance JSON from audit trail |
+| **Level 2** | GitHub Actions OIDC + Sigstore cosign for artifact signing |
+| **Verification** | `tools/supply_chain/slsa_verifier.py` — verify provenance attestations |
+
+### 6.5 Compliance Sidecar Decorator
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Eliminates duplicated CUI marking
+
+| Item | Detail |
+|------|--------|
+| **File** | `tools/core/compliance_sidecar.py` (new) |
+| **Pattern** | `@compliance_sidecar` decorator wrapping tool entry points |
+| **Enforces** | CUI marking, audit trail logging, classification validation |
+| **Config** | Reads classification policy from `args/compliance_config.yaml` |
+| **ADR** | D-ARCH-3: Compliance sidecar decorator for all tools |
+
+### 6.6 Feature Flags with Environment Profiles
+**Priority:** Medium | **Effort:** 1 day | **Impact:** Safer deployments
+
+| Item | Detail |
+|------|--------|
+| **File** | `args/feature_flags.yaml` (new) |
+| **Profiles** | `args/environments/{dev,staging,production,air-gap}.yaml` |
+| **Reader** | `tools/core/feature_flags.py` — reads YAML, env var override |
+| **Flags** | cato_live_streaming, two_tier_llm, forge_marketplace, air_gap_mode |
+
+### Phase 6 Deliverables
+
+| Deliverable | Files | Tests |
+|------------|-------|-------|
+| Property-based tests | `tests/test_compliance_properties.py` | 15 |
+| Contract tests | `tests/contracts/` | 27 |
+| Snapshot tests | `tests/test_compliance_snapshots.py` | 8 |
+| SLSA Level 2 | `.github/workflows/slsa-build.yml`, `tools/supply_chain/slsa_verifier.py` | 6 |
+| Compliance sidecar | `tools/core/compliance_sidecar.py` | 8 |
+| Feature flags | `args/feature_flags.yaml`, `tools/core/feature_flags.py`, `args/environments/*.yaml` | 10 |
+| **Total** | **10 new files** | **74 tests** |
+
+---
+
+## Architecture Decisions Registry
+
+All new decisions proposed in this plan:
+
+| ID | Decision | Phase | Rationale |
+|---|---|---|---|
+| D-ARCH-1 | Structurizr DSL for architecture-as-code | 2 | Version-controlled, diff-able diagrams |
+| D-ARCH-2 | Circuit breaker (3-state) in LLM router | 1 | Faster recovery, prevents cascade failures |
+| D-ARCH-3 | Compliance sidecar decorator for all tools | 6 | Centralize CUI marking, eliminate duplication |
+| D-ARCH-4 | Domain event bus (Queue in-process, NATS for K8s) | 2 | Reactive cross-context communication |
+| D-ARCH-5 | A2A v0.2 spec-compliant Agent Cards | 4 | Industry interoperability, AAIF alignment |
+| D-ARCH-6 | ATLAS saga coordinator with compensation registry | 4 | Automated workflow recovery |
+| D-ARCH-7 | CSA ATF trust tiers (Intern/Junior/Senior/Principal) | 4 | Industry-standard agent trust model |
+| D-ARCH-8 | Selective peer-to-peer agent channels | 4 | Reduce orchestrator bottleneck |
+| D-ARCH-9 | Semantic caching (3-tier) in LLM router | 5 | 40-60% token cost reduction |
+| D-ARCH-10 | FedRAMP 20x KSI emitter for continuous posture | 3 | 3-month authorization cycles |
+| D-ARCH-11 | Corrective RAG for compliance retrieval | 5 | Reduces hallucination in narratives |
+| D-ARCH-12 | Output verification gate (injection Layer 3) | 5 | Closes prompt injection defense gap |
+| D-ARCH-13 | SLSA Level 2 supply chain provenance | 6 | Signed artifacts, verifiable builds |
+| D-ARCH-14 | Property-based testing for rule engines | 6 | 50x more bugs per test |
+
+---
+
+## Summary
+
+| Phase | Weeks | Focus | New Files | Tests | Key Outcome |
+|---|---|---|---|---|---|
+| 1 | 1-2 | Resilience & Code Quality | 8 | 55 | Circuit breaker, DLQ, error hierarchy |
+| 2 | 3-4 | Architecture Visibility | 45+ | — | C4 diagrams, DDD, ADRs, event bus |
+| 3 | 5-7 | Compliance Acceleration | 5 | 40 | FedRAMP 20x KSI, OSCAL, inheritance |
+| 4 | 7-9 | Agent Evolution | 16 | 51 | A2A v0.2, saga, trust tiers, peer channels |
+| 5 | 9-11 | Intelligence | 6 | 48 | Semantic cache, corrective RAG, budget routing |
+| 6 | 11-12 | Testing & DevSecOps | 10 | 74 | Property tests, SLSA L2, sidecar, flags |
+| **Total** | **12 weeks** | | **~90 files** | **268 tests** | |
+
+---
+
+## Future Roadmap (Beyond 12 Weeks)
+
+Items deferred due to higher effort or lower immediate impact:
+
+| Item | Effort | Impact | Source |
+|---|---|---|---|
+| GraphRAG for crosswalk engine (transitive control queries) | High | High | AI Patterns Report §2.2 |
+| Plan-and-Execute delegation (50-60% additional token savings) | High | High | Architecture Report §3.2 |
+| Service mesh self-consumption (Istio on own K8s deployment) | High | Medium | Architecture Report §2.7 |
+| Event replay and snapshot capabilities | High | Medium | Architecture Report §2.4 |
+| MCP server consolidation (12 → ~8) | Medium | Medium | AI Patterns Report §1.2 |
+| Federated learning coordinator for fleet | High | Medium | AI Patterns Report §5.2 |
+| Chaos engineering with LitmusChaos | Medium | Medium | Dev Practices Report §6.1 |
+| ArgoCD GitOps deployment | Medium | Medium | Dev Practices Report §2.4 |
+| CQRS read model for compliance dashboards | Medium | Medium | Architecture Report §2.5 |
+| External system integration ACL standardization | High | Medium | Architecture Report §4.5 |
+
+---
+
+*Generated from research conducted 2026-03-08. Sources: 3 research documents, 100+ cited references.*
+*CUI // SP-CTI*