icdev 0.0.3__py3-none-any.whl

goals/connector_forge.md

@@ -0,0 +1,133 @@
+# CUI // SP-CTI
+# Goal: Connector Forge — Dynamic DataBridge Connector Generation
+
+## Purpose
+
+Generate, validate, sandbox, and publish new DataBridge connectors on demand from API specs, WSDL definitions, or structured YAML input. Extends DataBridge's 47 hand-coded connectors with an infinite long tail via AI-assisted code generation.
+
+## When to Use
+
+- User needs a DataBridge connector for a product/service that doesn't have one yet
+- User has an OpenAPI spec, WSDL definition, or API documentation URL
+- User wants to import a community-shared connector from the marketplace
+- User wants to promote a sandboxed connector to production
+
+## 6-Stage Pipeline
+
+```
+INPUT PARSING → BASE CLASS SELECTION → CODE GENERATION → STATIC VALIDATION → SANDBOX → INTEGRATION TEST
+  (spec_parser)    (base_selector)      (code_generator)  (static_validator)  (sandbox_mgr)  (int_tester)
+```
+
+### Stage 1: Spec Parser
+Parse OpenAPI JSON/YAML, WSDL XML, HTML docs, or structured YAML into `ForgeApiManifest`.
+
+### Stage 2: Base Selector
+Deterministic protocol→base class mapping. Maps to one of 8 base classes: SaaSBase, SoapBase, SQLBase, FsspecBase, MessagingBase, HealthBase, EmailBase, DataConnector.
+
+### Stage 3: Code Generator
+Render skeleton from Jinja2 template, optionally enhance with two-tier LLM (qwen3 draft → Claude review).
+
+### Stage 4: Static Validator
+6-gate validation: py_compile, ruff, AST ABC check, bandit SAST, secret scan, import whitelist.
+
+### Stage 5: Sandbox
+Docker container (--network none, --memory 256m) with subprocess fallback. Probes import/instantiate/connect/health/list_tables.
+
+### Stage 6: Integration Test
+Evaluate sandbox results. Pass requires successful import + instantiation.
+
+## Workflow
+
+### Generate a connector from spec
+```bash
+# Via MCP
+echo '{"jsonrpc":"2.0","id":1,"method":"forge_from_spec","params":{"content":"<openapi-json>","connector_name":"my_api"}}' | python tools/mcp/connector_forge_server.py
+
+# Via Python
+from tools.databridge.forge.forge_agent import forge_from_spec
+result = forge_from_spec(content="...", connector_name="my_api", use_llm=False, run_sandbox_flag=False)
+```
+
+### Promote to production
+```bash
+from tools.databridge.forge.promoter import promote_connector
+result = promote_connector(connector_id="forge-abc123", promoted_by="admin")
+```
+
+### Publish to marketplace
+```bash
+from tools.databridge.forge.marketplace_publisher import publish_connector
+result = publish_connector(connector_id="forge-abc123", marketplace_url="https://marketplace.icdev.ai")
+```
+
+### Import from marketplace
+```bash
+from tools.databridge.forge.import_handler import import_community_connector
+result = import_community_connector(slug="databridge-connector-acme-crm")
+```
+
+## Promotion State Machine
+
+```
+sandboxed → promoted → published → deprecated
+```
+
+- **sandboxed**: Just generated, passed validation. Cannot be used in production sync jobs.
+- **promoted**: Human-approved. Loaded into in-memory connector registry via `load_forge_connectors()`.
+- **published**: Shared on marketplace.icdev.ai for community use.
+- **deprecated**: Retired. No longer loaded into registry.
+
+## Expected Outputs
+
+| Operation | Success Output |
+|-----------|---------------|
+| forge_from_spec | `{status: "sandboxed", connector_id, validation, sandbox, integration}` |
+| forge_promote | `{status: "ok", new_status: "promoted"}` |
+| forge_publish | `{status: "ok", slug, artifact_id}` |
+| forge_import | `{status: "ok", connector_id, new_status: "sandboxed"}` |
+| forge_list | `{connectors: [...], count: N}` |
+
+## Edge Cases
+
+- **No LLM available**: Falls back to template-only generation (still produces valid connector skeleton)
+- **No Docker**: Falls back to subprocess sandbox with restricted PYTHONPATH
+- **Air-gapped**: All parsing uses stdlib (xml.etree, json, html.parser). Jinja2 has string-replacement fallback
+- **Validation failure**: Pipeline stops after Stage 4, returns detailed gate results
+- **Sandbox timeout**: 30-second default, configurable in `args/databridge_config.yaml` under `forge.sandbox`
+- **Import validation fail**: Community connector rejected, not stored in DB
+
+## Configuration
+
+`args/databridge_config.yaml` under `forge:` block — sandbox settings, validation flags, import whitelist, promotion config.
+
+## Architecture Decisions
+
+- D-CF-1: `forge/` is a subpackage of `tools/databridge/`
+- D-CF-2: Two-tier LLM (qwen3 drafts, Claude reviews)
+- D-CF-3: Inline Jinja2 templates with string-replacement fallback
+- D-CF-4: Docker primary, subprocess fallback for sandbox
+- D-CF-5: Two new ConnectorType enum values: SOAP, HEALTH
+- D-CF-6: Promotion state machine: sandboxed → promoted → published → deprecated
+- D-CF-7: 8 new audit event types
+- D-CF-8: Marketplace install via ASSET_TYPE_DIRS["databridge_connector"]
+- D-CF-9: MCP server with 8 tools
+- D-CF-10: Config in databridge_config.yaml under forge: block
+
+## Tools
+
+| Tool | Path |
+|------|------|
+| Forge Agent | `tools/databridge/forge/forge_agent.py` |
+| Spec Parser | `tools/databridge/forge/spec_parser.py` |
+| Base Selector | `tools/databridge/forge/base_selector.py` |
+| Code Generator | `tools/databridge/forge/code_generator.py` |
+| Static Validator | `tools/databridge/forge/static_validator.py` |
+| Sandbox Manager | `tools/databridge/forge/sandbox_manager.py` |
+| Integration Tester | `tools/databridge/forge/integration_tester.py` |
+| Promoter | `tools/databridge/forge/promoter.py` |
+| Publisher | `tools/databridge/forge/marketplace_publisher.py` |
+| Import Handler | `tools/databridge/forge/import_handler.py` |
+| Templates | `tools/databridge/forge/templates/__init__.py` |
+| MCP Server | `tools/mcp/connector_forge_server.py` |
+| A2A Card | `tools/agent/cards/connector_forge_card.json` |

goals/databridge.md

@@ -0,0 +1,128 @@
+# CUI // SP-CTI
+# Goal: DataBridge — Universal Data & Storage Connector
+
+## Purpose
+Enable ICDEV and child applications to connect to any database, data store,
+cloud storage, streaming platform, SaaS API, or file source through a
+unified Apache Arrow-based pipeline.
+
+## When to Use
+- User needs to read/write data from external sources
+- User wants to set up data integrations (ETL/ELT)
+- User needs to map schemas between sources
+- User needs to query across multiple data sources (DuckDB analytics)
+- User needs PII detection or CUI field-level marking on data flows
+
+## Workflow
+
+### 1. Create Connection
+```bash
+# Via dashboard: /databridge → "Add Connection"
+# Via API: POST /api/databridge/connections
+python tools/databridge/connection_manager.py --list --json
+```
+
+### 2. Test Connection
+```bash
+python tools/databridge/connection_manager.py --test <connection_id> --json
+```
+
+### 3. Infer Schema
+```bash
+python tools/databridge/schema_engine.py --get <connection_id> <table_name> --json
+```
+
+### 4. Sync Data (Read)
+```bash
+python tools/databridge/sync_engine.py --read <connection_id> <table_name> --json
+```
+
+### 5. Create Mapping (Optional)
+```bash
+# Via dashboard: /databridge/mappings → visual drag-and-drop editor
+# Via API: POST /api/databridge/mappings
+python tools/databridge/mapping_engine.py --list --json
+```
+
+### 6. Execute Mapping
+```bash
+python tools/databridge/mapping_engine.py --execute <mapping_id> --json
+```
+
+### 7. Query with DuckDB (Optional)
+```bash
+python tools/databridge/analytics.py --query "SELECT * FROM source LIMIT 10" --json
+```
+
+### 8. PII Scan (Optional)
+```bash
+python tools/databridge/pii_detector.py --scan <connection_id> <table_name> --json
+```
+
+## Tools
+
+| Tool | Purpose |
+|------|---------|
+| `tools/databridge/connector.py` | ABC base class + universal types |
+| `tools/databridge/connection_manager.py` | CRUD connections, health checks |
+| `tools/databridge/schema_engine.py` | Schema inference, versioning |
+| `tools/databridge/arrow_pipeline.py` | Arrow transform pipeline |
+| `tools/databridge/sync_engine.py` | Sync orchestrator |
+| `tools/databridge/registry.py` | Connector discovery |
+| `tools/databridge/mapping_engine.py` | Schema crosswalks + transforms |
+| `tools/databridge/transforms.py` | Transform function library |
+| `tools/databridge/format_converter.py` | Format conversion |
+| `tools/databridge/analytics.py` | DuckDB analytics |
+| `tools/databridge/pii_detector.py` | PII detection (Presidio) |
+| `tools/databridge/data_profiler.py` | Data quality profiling |
+| `tools/databridge/stream_manager.py` | Streaming connector manager |
+| `tools/databridge/relay_server.py` | On-prem WebSocket relay |
+
+## Configuration
+- `args/databridge_config.yaml` — module settings, tier limits, PII, CUI, FIPS
+
+## Connectors (22 built-in)
+
+| Category | Connectors | Tier |
+|----------|-----------|------|
+| Database | SQLite, PostgreSQL, MySQL, MSSQL, Oracle | SQLite=free |
+| Cloud Storage | Local FS, S3, Azure Blob, GCS, HDFS | Local=free |
+| File | CSV, JSON, Parquet, Avro, Excel | CSV/JSON=free |
+| Streaming | Kafka, Kinesis, CDC | All paid |
+| SaaS API | Salesforce, ServiceNow, Jira, SAP | All paid |
+
+## Design Decisions
+- D-DB-1: Independent `tools/databridge/` directory
+- D-DB-2: ABC `DataConnector` + concrete implementations
+- D-DB-3: Apache Arrow universal in-memory format
+- D-DB-4: fsspec universal filesystem abstraction
+- D-DB-5: DuckDB local analytics
+- D-DB-6: Append-only sync logs (NIST AU)
+- D-DB-7: Versioned schema mappings (SHA-256)
+- D-DB-8: CUI field-level marking (IL4+ only)
+- D-DB-9: Presidio PII detection
+- D-DB-10: On-prem agent (outbound WebSocket)
+- D-DB-11: Free tier: 3 connectors, no streaming
+- D-DB-12: YAML config in DB column
+- D-DB-13: Secret references, never plaintext
+- D-DB-14: Deterministic schema inference
+- D-DB-15: Transform pipeline = Arrow compute DAG
+- D-DB-17: RAG integration with on/off toggle
+- D-DB-18: Graceful optional imports
+- D-DB-19: Visual mapping in vanilla JS + SVG
+- D-DB-20: FIPS 140-3 TLS optional
+
+## Edge Cases
+- Connection fails mid-sync → log partial results, mark connection as error
+- Schema drift detected → create new version, alert user
+- PII detected in non-CUI data → flag_only mode (no masking unless configured)
+- Free tier limit hit → return error with upgrade prompt
+- Heavy dependency not installed → graceful skip, connector unavailable
+- On-prem agent disconnects → auto-reconnect with exponential backoff
+
+## Compliance
+- All sync logs are append-only (NIST AU-2, AU-3)
+- CUI field marking at IL4+ (NIST SI-12)
+- PII detection satisfies NIST SI-18
+- Schema versioning satisfies NIST CM-3
+- Secret references satisfy NIST SC-28

goals/deploy_workflow.md

@@ -0,0 +1,390 @@
+# Goal: Infrastructure Deployment
+
+## Description
+
+Generate all infrastructure-as-code artifacts (Terraform, Ansible, Kubernetes manifests, CI/CD pipeline), verify all pre-deployment gates, commit to GitLab, and monitor the deployment pipeline through all 7 stages to production.
+
+**Why this matters:** Manual deployments are unreproducible, error-prone, and unauditable. Infrastructure-as-code ensures every deployment is identical, testable, and traceable. In government environments, every change must be tracked and reversible.
+
+---
+
+## Prerequisites
+
+- [ ] Project initialized (`goals/init_project.md` completed)
+- [ ] All tests pass (`goals/tdd_workflow.md` completed)
+- [ ] Security scan gates pass (`goals/security_scan.md` — 0 critical, 0 secrets)
+- [ ] Compliance artifacts current (`goals/compliance_workflow.md` — within 30 days)
+- [ ] ATO status is READY or existing ATO is valid
+- [ ] Target environment defined (dev, staging, production)
+- [ ] GitLab repository configured
+- [ ] `memory/MEMORY.md` loaded (session context)
+
+**HARD STOP: Do not proceed if any prerequisite fails. Deployment without passing gates is a compliance violation.**
+
+---
+
+## Deployment Pipeline Overview
+
+```mermaid
+flowchart LR
+    TF["Terraform\nGenerate IaC"]:::blue --> AN["Ansible\nPlaybooks"]:::blue
+    AN --> K8["K8s\nManifests"]:::blue
+    K8 --> PL["Pipeline\nGenerate CI/CD"]:::blue
+    PL --> GATES{"10 Pre-Deploy\nGates"}:::yellow
+    GATES -->|ALL PASS| COMMIT["Commit &\nPush"]:::green
+    GATES -->|ANY FAIL| BLOCKED["BLOCKED\nRemediate"]:::red
+    COMMIT --> EXEC["Pipeline\nExecution"]:::blue
+    EXEC --> HC{"Health\nCheck"}:::yellow
+    HC -->|PASS| AUDIT["Audit\nLogged"]:::green
+    HC -->|FAIL| ROLLBACK["Rollback"]:::red
+
+    classDef green fill:#1a3a2d,stroke:#28a745,color:#e0e0e0
+    classDef red fill:#3a1a1a,stroke:#dc3545,color:#e0e0e0
+    classDef yellow fill:#3a3a1a,stroke:#ffc107,color:#e0e0e0
+    classDef blue fill:#1a3a5c,stroke:#4a90d9,color:#e0e0e0
+```
+
+```mermaid
+flowchart LR
+    S1["1. Build"]:::blue -.->|"tests pass\ncoverage >= 80%"| S2["2. Test"]:::blue
+    S2 -.->|"0 critical/high\nSAST findings"| S3["3. SAST"]:::blue
+    S3 -.->|"0 critical/high\ndep vulns"| S4["4. Deps"]:::blue
+    S4 -.->|"0 critical\ncontainer vulns"| S5["5. Container"]:::blue
+    S5 -.->|"0 CAT1 STIGs\nCUI markings"| S6["6. Compliance"]:::blue
+    S6 -.->|"manual approval\nfor production"| S7["7. Deploy"]:::green
+
+    classDef green fill:#1a3a2d,stroke:#28a745,color:#e0e0e0
+    classDef blue fill:#1a3a5c,stroke:#4a90d9,color:#e0e0e0
+```
+
+---
+
+## Process
+
+### Step 1: Generate Terraform Configuration
+
+**Tool:** `python tools/infra/terraform_generator.py --project <name>`
+
+**Expected output:**
+```
+Terraform files generated: projects/<name>/infrastructure/terraform/
+
+Files:
+  - main.tf              # Provider, backend, core resources
+  - variables.tf         # Input variables
+  - outputs.tf           # Output values
+  - networking.tf        # VPC, subnets, security groups
+  - compute.tf           # EC2/ECS/EKS resources
+  - storage.tf           # S3, RDS, EBS
+  - iam.tf               # IAM roles, policies (least privilege)
+  - security_groups.tf   # Network ACLs
+  - terraform.tfvars     # Environment-specific values
+
+Provider: AWS GovCloud (us-gov-west-1)
+Backend: S3 + DynamoDB state locking
+```
+
+**Security requirements for generated Terraform:**
+- All S3 buckets: encryption enabled, public access blocked, versioning on
+- All security groups: no 0.0.0.0/0 ingress (except ALB on 443)
+- IAM: least-privilege policies, no wildcard (*) actions
+- RDS: encryption at rest, no public accessibility
+- VPC: private subnets for compute, public only for load balancers
+
+**Error handling:**
+- Missing provider credentials → fail clearly, do not generate with placeholder keys
+- Unsupported resource type → document limitation, suggest manual creation
+- State backend not configured → generate backend config, warn user to initialize
+
+**Verify:** `terraform validate` passes. `terraform plan` shows expected resources. No security group allows unrestricted ingress.
+
+---
+
+### Step 2: Generate Ansible Playbooks
+
+**Tool:** `python tools/infra/ansible_generator.py --project <name>`
+
+**Expected output:**
+```
+Ansible files generated: projects/<name>/infrastructure/ansible/
+
+Files:
+  - site.yml                    # Master playbook
+  - inventory/
+  │   ├── production.yml        # Production hosts
+  │   ├── staging.yml           # Staging hosts
+  │   └── group_vars/
+  │       └── all.yml           # Shared variables
+  - roles/
+  │   ├── hardening/            # STIG hardening role
+  │   │   ├── tasks/main.yml
+  │   │   └── handlers/main.yml
+  │   ├── application/          # App deployment role
+  │   │   ├── tasks/main.yml
+  │   │   └── templates/
+  │   └── monitoring/           # Monitoring agent role
+  │       └── tasks/main.yml
+
+STIG hardening checks: <count> automated
+```
+
+**Security requirements for generated Ansible:**
+- STIG hardening role applies all applicable STIG checks
+- No plaintext passwords in playbooks (use Ansible Vault)
+- SSH key-based authentication only
+- Audit logging enabled on all managed hosts
+- Firewall rules applied matching Terraform security groups
+
+**Error handling:**
+- Ansible not installed → provide installation instructions
+- Missing vault password → warn, generate without secrets (user adds later)
+- Invalid YAML syntax → validate before writing, fix
+
+**Verify:** `ansible-playbook --syntax-check site.yml` passes. Vault-encrypted files are not plaintext.
+
+---
+
+### Step 3: Generate Kubernetes Manifests
+
+**Tool:** `python tools/infra/k8s_generator.py --project <name>`
+
+**Expected output:**
+```
+Kubernetes manifests generated: projects/<name>/infrastructure/k8s/
+
+Files:
+  - namespace.yaml              # Isolated namespace
+  - deployment.yaml             # App deployment
+  - service.yaml                # ClusterIP/LoadBalancer service
+  - ingress.yaml                # Ingress with TLS
+  - configmap.yaml              # Non-sensitive config
+  - secret.yaml                 # Sensitive config (sealed)
+  - hpa.yaml                    # Horizontal Pod Autoscaler
+  - networkpolicy.yaml          # Network isolation
+  - poddisruptionbudget.yaml   # Availability guarantee
+  - serviceaccount.yaml         # RBAC service account
+  - rbac.yaml                   # Role and RoleBinding
+
+Security settings applied:
+  - runAsNonRoot: true
+  - readOnlyRootFilesystem: true
+  - allowPrivilegeEscalation: false
+  - resource limits set
+  - network policies enforced
+```
+
+**Security requirements for generated K8s manifests:**
+- Pods run as non-root user
+- Read-only root filesystem
+- No privilege escalation
+- Resource limits defined (CPU and memory)
+- Network policies restrict pod-to-pod traffic
+- Secrets use SealedSecrets or external secret management
+- No `latest` tag — all images pinned to specific versions
+
+**Error handling:**
+- No Dockerfile → generate one first (`tools/infra/dockerfile_generator.py`)
+- Invalid manifest syntax → `kubectl apply --dry-run=client -f <file>` to validate
+- Missing namespace → create namespace manifest first
+
+**Verify:** `kubectl apply --dry-run=client -f .` passes for all manifests. Security context is set on all pods.
+
+---
+
+### Step 4: Generate CI/CD Pipeline
+
+**Tool:** `python tools/infra/pipeline_generator.py --project <name>`
+
+**Expected output:**
+```
+Pipeline generated: projects/<name>/.gitlab-ci.yml
+
+Stages (7):
+  1. build        — Compile, package, create container image
+  2. test         — Unit tests, integration tests, coverage check
+  3. sast         — Static analysis (bandit/eslint-security)
+  4. dependency   — pip-audit/npm audit
+  5. container    — trivy container scan
+  6. compliance   — STIG check, CUI marking verification
+  7. deploy       — Terraform apply, Ansible run, K8s deploy
+
+Gates between stages:
+  - test → sast: all tests pass, coverage >= 80%
+  - sast → dependency: 0 critical/high SAST findings
+  - dependency → container: 0 critical/high dependency vulns
+  - container → compliance: 0 critical container vulns
+  - compliance → deploy: 0 CAT1 STIGs, CUI markings present
+  - deploy: manual trigger for production (automatic for dev/staging)
+
+Artifacts:
+  - Test reports (JUnit XML)
+  - Coverage reports (Cobertura)
+  - Scan results (JSON)
+  - SBOM (CycloneDX JSON)
+```
+
+**Pipeline requirements:**
+- Each stage fails fast (no continuing past failures)
+- Production deployment requires manual approval
+- Rollback procedure documented in pipeline comments
+- All artifacts preserved for 90 days
+- Pipeline variables use CI/CD variables (never hardcoded)
+
+**Error handling:**
+- GitLab CI not available → generate pipeline file anyway, warn about manual execution
+- Missing CI/CD variables → document required variables in pipeline comments
+- Pipeline too slow → add caching for dependencies
+
+**Verify:** `.gitlab-ci.yml` is valid YAML. Pipeline lint passes (`gitlab-ci-lint` or GitLab API).
+
+---
+
+### Step 5: Verify All Pre-Deployment Gates
+
+**Action:** Final gate check before committing to deployment.
+
+```
+=== PRE-DEPLOYMENT GATE CHECK ===
+
+Gate 1: All tests pass                    [PASS/FAIL]
+Gate 2: Coverage >= 80%                   [PASS/FAIL]
+Gate 3: SAST — 0 critical/high           [PASS/FAIL]
+Gate 4: Dependencies — 0 critical/high    [PASS/FAIL]
+Gate 5: Secrets — 0 detected             [PASS/FAIL]
+Gate 6: Container — 0 critical/high      [PASS/FAIL] (or N/A)
+Gate 7: STIG — 0 CAT1                    [PASS/FAIL]
+Gate 8: CUI markings present             [PASS/FAIL]
+Gate 9: SBOM current (< 30 days)         [PASS/FAIL]
+Gate 10: ATO status valid                [PASS/FAIL]
+
+Overall: <ALL GATES PASS | BLOCKED — gates X, Y, Z failed>
+```
+
+**If ANY gate fails:** STOP. Do not deploy. Document which gates failed and what remediation is needed. Return to the appropriate workflow to fix.
+
+---
+
+### Step 6: Commit to GitLab
+
+**Action:** Stage all infrastructure files and commit.
+
+```bash
+git add projects/<name>/infrastructure/
+git add projects/<name>/.gitlab-ci.yml
+git commit -m "feat(<name>): infrastructure-as-code for <environment> deployment
+
+- Terraform: AWS GovCloud resources
+- Ansible: STIG-hardened configuration
+- K8s: Security-hardened manifests
+- Pipeline: 7-stage CI/CD with security gates
+
+All pre-deployment gates passed.
+Scan date: <YYYY-MM-DD>"
+
+git push origin <branch>
+```
+
+**Error handling:**
+- Git not initialized → `git init`, configure remote
+- Push rejected → pull first, resolve conflicts, re-push
+- Large files → check for binaries, use `.gitignore`
+
+**Verify:** Commit exists in remote repository. Pipeline triggered.
+
+---
+
+### Step 7: Monitor Pipeline Execution
+
+**Action:** Watch the 7-stage pipeline for completion.
+
+```
+Pipeline status:
+  Stage 1 (build):      [PASS] — 2m 15s
+  Stage 2 (test):       [PASS] — 4m 30s
+  Stage 3 (sast):       [PASS] — 1m 45s
+  Stage 4 (dependency): [PASS] — 0m 55s
+  Stage 5 (container):  [PASS] — 3m 20s
+  Stage 6 (compliance): [PASS] — 2m 10s
+  Stage 7 (deploy):     [PASS] — 5m 00s  (manual approval for prod)
+
+Total time: 20m 35s
+Status: SUCCESS
+```
+
+**If pipeline fails:**
+1. Identify which stage failed
+2. Read the stage logs
+3. Determine if the failure is in code, infra, or pipeline config
+4. Fix the issue
+5. Re-push and re-trigger pipeline
+6. If production deploy fails → execute rollback (`tools/infra/rollback.py --project <name> --environment <env>`)
+
+**Post-deployment health check:**
+```bash
+python tools/monitor/health_checker.py --url <deployed-url> --retries 5
+```
+
+---
+
+### Step 8: Log to Audit Trail
+
+**Tool:** `python tools/audit/audit_logger.py --event "deployment_complete" --actor "orchestrator" --action "deploy" --project <name>`
+
+**Tool:** `python tools/memory/memory_write.py --content "Deployed <name> to <environment>. Pipeline: all 7 stages passed. Health check: <status>" --type event --importance 8`
+
+---
+
+## Success Criteria
+
+- [ ] Terraform configuration generated and validated
+- [ ] Ansible playbooks generated with STIG hardening
+- [ ] Kubernetes manifests generated with security contexts
+- [ ] CI/CD pipeline generated with 7 stages and gates
+- [ ] All 10 pre-deployment gates pass
+- [ ] Code committed and pushed to GitLab
+- [ ] Pipeline completes all 7 stages successfully
+- [ ] Health check passes post-deployment
+- [ ] Audit trail entry logged
+
+---
+
+## Edge Cases & Notes
+
+1. **Rollback procedure:** If deployment breaks production, execute `python tools/infra/rollback.py --project <name> --environment production`. This reverts to the last known-good deployment. Test rollback in staging first.
+2. **Blue-green deployments:** For zero-downtime, use blue-green strategy. Both versions run simultaneously; traffic switches after health check passes.
+3. **Canary deployments:** Route 5% of traffic to new version first. Monitor error rates. If stable for 15 minutes, increase to 100%.
+4. **Terraform state:** State files contain sensitive information. Store in encrypted S3 bucket with state locking via DynamoDB. Never commit state files to git.
+5. **Secret injection:** Pipeline secrets come from GitLab CI/CD variables or AWS Secrets Manager. Never bake secrets into images or manifests.
+6. **Multi-environment:** Dev deploys automatically on merge. Staging deploys automatically on tag. Production requires manual approval.
+7. **Disaster recovery:** Terraform enables full infrastructure recreation. Document RTO (Recovery Time Objective) and RPO (Recovery Point Objective).
+8. **Cost management:** Terraform `plan` shows estimated costs. Review before applying to production.
+
+---
+
+## GOTCHA Layer Mapping
+
+| Step | GOTCHA Layer | Component |
+|------|-------------|-----------|
+| Generate Terraform | Tools | terraform_generator.py |
+| Generate Ansible | Tools | ansible_generator.py |
+| Generate K8s | Tools | k8s_generator.py |
+| Generate pipeline | Tools | pipeline_generator.py |
+| Gate evaluation | Orchestration | AI (you) |
+| Environment config | Args | terraform.tfvars, inventory |
+| Infrastructure patterns | Context | GovCloud reference |
+| Deployment decisions | Orchestration | AI (you) |
+
+---
+
+## Related Files
+
+- **Tools:** `tools/infra/terraform_generator.py`, `tools/infra/ansible_generator.py`, `tools/infra/k8s_generator.py`, `tools/infra/pipeline_generator.py`, `tools/infra/rollback.py`
+- **Depends on:** `goals/tdd_workflow.md`, `goals/security_scan.md`, `goals/compliance_workflow.md`
+- **Feeds into:** `goals/monitoring.md` (post-deploy observability)
+- **Database:** `data/icdev.db` (deployments table)
+
+---
+
+## Changelog
+
+- 2026-02-14: Initial creation