icdev 0.0.3__py3-none-any.whl

goals/tdd_workflow.md

@@ -0,0 +1,403 @@
+# Goal: TDD Workflow (RED -> GREEN -> REFACTOR)
+
+## Description
+
+Implement features using strict Test-Driven Development: write tests FIRST, watch them fail, write the minimum code to pass, then refactor. This workflow enforces the discipline that separates production-grade code from "vibe coding."
+
+**Why this matters:** Code without tests is a liability. In government/DoD environments, untested code does not ship. TDD catches defects at the cheapest point in the lifecycle — before they exist.
+
+**The iron rule:** No code is written until a failing test demands it.
+
+---
+
+## Prerequisites
+
+- [ ] Project initialized (`goals/init_project.md` completed)
+- [ ] Project exists in DB with valid project ID
+- [ ] Requirement or user story clearly defined (what, not how)
+- [ ] Test framework installed (pytest, behave)
+- [ ] `memory/MEMORY.md` loaded (session context)
+
+---
+
+## Process
+
+### Step 1: Write Gherkin Feature File (Specification)
+
+**Tool:** `python tools/builder/test_writer.py --project <name> --requirement "<requirement text>"`
+
+**Expected output:**
+```
+Feature file created: projects/<name>/tests/features/<feature-name>.feature
+
+Feature: <Feature Name>
+  As a <role>
+  I want <capability>
+  So that <benefit>
+
+  Scenario: <Happy path>
+    Given <precondition>
+    When <action>
+    Then <expected result>
+
+  Scenario: <Error case>
+    Given <precondition>
+    When <invalid action>
+    Then <error handling>
+
+  Scenario: <Edge case>
+    Given <boundary condition>
+    When <action>
+    Then <correct behavior>
+```
+
+**Minimum scenarios per feature:** 3 (happy path, error case, edge case)
+
+**Error handling:**
+- Ambiguous requirement → ask user for clarification before generating
+- Feature file already exists → append new scenarios, do not overwrite
+- Generated scenarios too vague → regenerate with more specific requirement
+
+**Verify:** Feature file parses correctly with `behave --dry-run`.
+
+#### BDD Step Alignment Rules
+
+When writing Gherkin features and step definitions, follow these rules to prevent mismatches:
+
+1. **Match step definitions to tool return signatures.** Read the tool function's return type before writing Then steps. If a function returns `{"impacts": [...]}`, the step must access `result["impacts"]`, not iterate over `result` directly.
+
+2. **Use the exact field names from tool output.** If the tool returns `impact_severity`, do not write a step that checks `impact_score`. Run the tool once and inspect the output dict keys.
+
+3. **Trailing colons for data tables.** When a Gherkin step uses a data table, the step text MUST end with a colon. Example: `When I add the following entities:` (not `When I add the following entities`).
+
+4. **Article variants need separate decorators.** Behave treats `a` and `an` as different text. Either use a regex step (`@given('(?:a|an) "{type}" entity')`) or register both variants.
+
+5. **CHECK constraints in test DB.** If the tool code validates against a Python constant (e.g., `ENTITY_TYPES`), ensure the test database schema uses the same constant. Import `SCHEMA_SQL` from the init script rather than duplicating SQL in `conftest.py` or `environment.py`.
+
+6. **Tolerance for probabilistic outputs.** Monte Carlo and simulation results are non-deterministic. Use range checks (`assert 0.3 < value < 0.7`) not exact equality. Histogram bin counts should use `>=` thresholds, not `==`.
+
+---
+
+### Step 2: Generate pytest Test Cases (Step Definitions)
+
+**Tool:** `python tools/builder/test_writer.py --project <name> --requirement "<requirement text>" --output pytest`
+
+**Expected output:**
+```
+Test files created:
+  projects/<name>/tests/unit/test_<feature>.py
+  projects/<name>/tests/integration/test_<feature>_integration.py
+  projects/<name>/tests/features/steps/<feature>_steps.py
+
+Tests generated:
+  - test_<function>_happy_path
+  - test_<function>_invalid_input
+  - test_<function>_edge_case
+  - test_<function>_boundary_values
+  Total: <count> tests
+```
+
+**Test structure requirements:**
+- Each test follows Arrange-Act-Assert pattern
+- Tests are independent (no shared mutable state)
+- Test names describe the behavior, not the implementation
+- Fixtures defined for common setup/teardown
+
+**Error handling:**
+- Cannot determine test structure → generate skeleton tests with TODO markers
+- Import errors in generated tests → fix imports before proceeding
+
+**Verify:** Tests are syntactically valid Python (`python -m py_compile <file>`).
+
+---
+
+### Step 3: RED — Run Tests (They Must Fail)
+
+**Tool:** `pytest projects/<name>/tests/ -v --tb=short`
+
+**Expected output:**
+```
+projects/<name>/tests/unit/test_<feature>.py::test_happy_path FAILED
+projects/<name>/tests/unit/test_<feature>.py::test_invalid_input FAILED
+projects/<name>/tests/unit/test_<feature>.py::test_edge_case FAILED
+
+========================= X failed in Y.YYs =========================
+```
+
+**Critical check: ALL tests MUST fail.**
+
+- If any test passes → the test is not testing new behavior. Investigate:
+  - Is the test trivially true? (e.g., `assert True`)
+  - Does the implementation already exist?
+  - Is the test importing wrong module?
+- If tests error (not fail) → fix test syntax/imports, re-run
+- ImportError is acceptable at this stage (module doesn't exist yet)
+
+**Error handling:**
+- Tests pass unexpectedly → STOP. Do not proceed. Diagnose why.
+- Test framework not installed → `pip install pytest behave`
+- Syntax errors in tests → fix before proceeding
+
+**This step confirms the tests are actually testing something meaningful.**
+
+---
+
+### Step 4: Generate Implementation Code
+
+**Tool:** `python tools/builder/code_generator.py --project <name> --spec "projects/<name>/tests/features/<feature-name>.feature"`
+
+**Expected output:**
+```
+Code generated:
+  projects/<name>/src/<module>.py
+
+Functions/classes created:
+  - <function_name>(): <description>
+  - <ClassName>: <description>
+
+Lines of code: <count>
+Complexity: <low|medium|high>
+```
+
+**Code generation rules:**
+- Write the MINIMUM code to make tests pass
+- No speculative features (YAGNI — You Ain't Gonna Need It)
+- Follow project language conventions
+- Include type hints (Python) or JSDoc (JavaScript)
+- No hardcoded values — use configuration from args/
+
+**Error handling:**
+- Generated code has syntax errors → regenerate with corrections
+- Generated code is overly complex → simplify, remember YAGNI
+- Missing dependencies → add to requirements.txt, install
+
+**Verify:** Code is syntactically valid (`python -m py_compile <file>`).
+
+---
+
+### Step 5: GREEN — Run Tests (They Must Pass)
+
+**Tool:** `pytest projects/<name>/tests/ -v --tb=short --cov=projects/<name>/src --cov-report=term-missing`
+
+**Expected output:**
+```
+projects/<name>/tests/unit/test_<feature>.py::test_happy_path PASSED
+projects/<name>/tests/unit/test_<feature>.py::test_invalid_input PASSED
+projects/<name>/tests/unit/test_<feature>.py::test_edge_case PASSED
+
+---------- coverage: ----------
+Name                          Stmts   Miss  Cover   Missing
+------------------------------------------------------------
+projects/<name>/src/<module>     42      3    93%    67-69
+------------------------------------------------------------
+TOTAL                            42      3    93%
+
+========================= X passed in Y.YYs =========================
+```
+
+**Gates:**
+- ALL tests must pass (0 failures)
+- Coverage must be >= 80% (target 90%+)
+- No warnings that indicate test issues
+
+**If tests fail:**
+1. Read the failure output carefully
+2. Determine if the bug is in the code or the test
+3. Fix the CODE first (tests define the contract)
+4. Only fix tests if they contain genuine errors (wrong assertions, not wrong expectations)
+5. Re-run until green
+
+**Error handling:**
+- Flaky tests (pass sometimes, fail sometimes) → investigate non-determinism, fix root cause
+- Coverage below 80% → write additional tests for uncovered lines
+- Tests pass but behavior is wrong → tests are incomplete, go back to Step 1
+
+**Do not proceed until ALL tests pass with >= 80% coverage.**
+
+---
+
+### Step 6: REFACTOR — Lint and Format
+
+**Tool (lint):** `python tools/builder/linter.py --project <name> --fix`
+
+**Expected output:**
+```
+Linting results:
+  Files checked: <count>
+  Issues found: <count>
+  Issues fixed: <count>
+  Remaining: <count>
+
+  Rules applied: flake8 (Python) / eslint (JavaScript)
+```
+
+**Tool (format):** `python tools/builder/formatter.py --project <name>`
+
+**Expected output:**
+```
+Formatting results:
+  Files formatted: <count>
+  Changes made: <count>
+
+  Formatter: black (Python) / prettier (JavaScript)
+```
+
+**After refactoring, re-run tests:**
+```
+pytest projects/<name>/tests/ -v --tb=short
+```
+
+**Critical: Tests must still pass after refactoring.** If refactoring breaks tests, the refactoring introduced a bug — revert and try again.
+
+**Refactoring guidelines:**
+- Extract repeated code into functions
+- Rename variables for clarity
+- Reduce complexity (cyclomatic complexity < 10)
+- Remove dead code
+- Add docstrings to public functions
+
+**Error handling:**
+- Linting finds issues that --fix cannot resolve → manual intervention required
+- Formatter changes break tests → formatter config may conflict with test expectations, investigate
+- Refactoring changes behavior → REVERT. Refactoring must be behavior-preserving.
+
+---
+
+### Step 7: Log to Audit Trail
+
+**Tool:** `python tools/audit/audit_logger.py --event "tdd_cycle_complete" --actor "orchestrator" --action "implement" --project <name>`
+
+**Tool:** `python tools/memory/memory_write.py --content "TDD cycle complete for <feature> in project <name>. Tests: <count> passing, coverage: <pct>%" --type event --importance 5`
+
+**Expected output:** Audit entry and memory entry logged.
+
+---
+
+## Success Criteria
+
+- [ ] Gherkin feature file exists with >= 3 scenarios
+- [ ] All pytest tests pass (0 failures)
+- [ ] Code coverage >= 80%
+- [ ] Linting clean (0 remaining issues)
+- [ ] Formatting applied
+- [ ] Tests still pass after refactoring
+- [ ] Audit trail entry logged
+
+---
+
+## Edge Cases & Notes
+
+1. **Existing code without tests:** If asked to add tests to existing code, still follow TDD — write the test, verify it exercises the existing behavior (it should pass), then write a NEW test for the new behavior (it should fail), then implement.
+2. **Test-first resistance:** Users may want to write code first. Explain the workflow but respect their choice. Log which approach was used.
+3. **Integration tests:** These follow the same RED-GREEN-REFACTOR cycle but may require mocks for external services. Use `unittest.mock` or `pytest-mock`.
+4. **Database tests:** Use an in-memory SQLite database for test isolation. Never test against production data.
+5. **Coverage exceptions:** Some lines (e.g., `if __name__ == "__main__"`) can be excluded from coverage. Use `# pragma: no cover` sparingly and only with justification.
+6. **Flaky test policy:** A test that fails intermittently is worse than no test. Fix flaky tests immediately or quarantine them with `@pytest.mark.skip(reason="flaky — tracking in issue #X")`.
+7. **Large features:** Break into multiple TDD cycles. Each cycle should be completable in < 30 minutes. If a feature takes longer, decompose it.
+
+---
+
+## The TDD Cycle Visualized
+
+```mermaid
+stateDiagram-v2
+    [*] --> RED : Write failing test
+    RED --> GREEN : Write minimum code
+    GREEN --> REFACTOR : Clean up
+    REFACTOR --> RED : Next requirement
+    REFACTOR --> [*] : Feature complete
+
+    state RED {
+        [*] --> WriteGherkin : Feature file
+        WriteGherkin --> WritePytest : Step definitions
+        WritePytest --> RunFailing : Must FAIL
+    }
+    state GREEN {
+        [*] --> GenerateCode : Minimum implementation
+        GenerateCode --> RunPassing : Must PASS
+        RunPassing --> CheckCoverage : Coverage >= 80%
+    }
+    state REFACTOR {
+        [*] --> Lint : flake8 / ruff
+        Lint --> Format : black
+        Format --> ReRun : Tests still pass?
+    }
+
+    classDef red fill:#3a1a1a,stroke:#dc3545,color:#e0e0e0
+    classDef green fill:#1a3a2d,stroke:#28a745,color:#e0e0e0
+    classDef blue fill:#1a3a5c,stroke:#4a90d9,color:#e0e0e0
+
+    class RED red
+    class GREEN green
+    class REFACTOR blue
+```
+
+```mermaid
+sequenceDiagram
+    participant O as Orchestrator
+    participant TW as test_writer.py
+    participant PT as pytest
+    participant CG as code_generator.py
+    participant LN as linter.py
+    participant FM as formatter.py
+    participant AL as audit_logger.py
+
+    rect rgb(58, 26, 26)
+        Note over O,PT: RED Phase
+        O->>TW: Write Gherkin + pytest tests
+        TW-->>O: Feature file + test cases
+        O->>PT: Run tests (must FAIL)
+        PT-->>O: All tests FAILED
+    end
+
+    rect rgb(26, 58, 45)
+        Note over O,PT: GREEN Phase
+        O->>CG: Generate minimum code
+        CG-->>O: Implementation module
+        O->>PT: Run tests (must PASS)
+        PT-->>O: All tests PASSED (coverage >= 80%)
+    end
+
+    rect rgb(26, 58, 92)
+        Note over O,FM: REFACTOR Phase
+        O->>LN: Lint code (flake8 / ruff)
+        LN-->>O: Clean (0 issues)
+        O->>FM: Format code (black)
+        FM-->>O: Formatted
+        O->>PT: Re-run tests (still pass?)
+        PT-->>O: All tests PASSED
+    end
+
+    O->>AL: Log TDD cycle to audit trail
+    AL-->>O: Audit entry recorded
+```
+
+---
+
+## GOTCHA Layer Mapping
+
+| Step | GOTCHA Layer | Component |
+|------|-------------|-----------|
+| Define requirement | Goals | This document |
+| Write feature file | Tools | test_writer.py |
+| Generate tests | Tools | test_writer.py |
+| Run tests (RED) | Tools | pytest |
+| Generate code | Tools | code_generator.py |
+| Run tests (GREEN) | Tools | pytest |
+| Lint + format | Tools | linter.py, formatter.py |
+| Decide test strategy | Orchestration | AI (you) |
+| Code style settings | Args | linter config, formatter config |
+
+---
+
+## Related Files
+
+- **Tools:** `tools/builder/test_writer.py`, `tools/builder/code_generator.py`, `tools/builder/linter.py`, `tools/builder/formatter.py`
+- **Context:** `context/coding_standards.md` (if exists)
+- **Hard Prompts:** `hardprompts/test_generation.md`, `hardprompts/code_generation.md`
+
+---
+
+## Changelog
+
+- 2026-02-14: Initial creation

goals/template_exchange.md

@@ -0,0 +1,77 @@
+# CUI // SP-CTI
+
+# F2: Community Compliance Template Exchange
+
+## Purpose
+
+Enable sharing, discovery, and reuse of compliance templates (SSP sections, POAM entries, control narratives) across projects and tenants. Community-driven with ratings and quality scoring.
+
+## Prerequisites
+
+- `data/icdev.db` initialized with compliance tables
+- At least one project created via `tools/agent/core_mcp_server.py`
+
+## Workflow Steps
+
+### 1. Create Template
+```bash
+python tools/compliance/template_exchange.py --create --project-id "sparkpilot" --name "AC-2 Narrative" --framework "NIST 800-53" --control-id "AC-2" --content-file /path/to/template.md --json
+```
+**Expected output:** JSON with template ID, metadata, version, and publication status (draft).
+
+### 2. List and Search Templates
+```bash
+python tools/compliance/template_exchange.py --list --framework "NIST 800-53" --json
+python tools/compliance/template_exchange.py --search --query "access control" --json
+```
+**Expected output:** JSON array of templates with ID, name, framework, rating, download count.
+
+### 3. Rate Template
+```bash
+python tools/compliance/template_exchange.py --rate --template-id "tpl-001" --rating 4 --comment "Clear and thorough" --json
+```
+**Expected output:** JSON with updated average rating and total review count.
+
+### 4. Publish Template
+```bash
+python tools/compliance/template_exchange.py --publish --template-id "tpl-001" --json
+```
+**Expected output:** JSON with publication status, visibility scope, and shareable reference ID.
+
+### 5. Import Template
+```bash
+python tools/compliance/template_exchange.py --import --template-id "tpl-001" --target-project "sparkpilot" --json
+```
+**Expected output:** JSON with imported template ID, adaptation notes, and control mapping status.
+
+## Decision Reference
+
+| Decision | Description |
+|----------|-------------|
+| D-INV-5 | Templates stored as versioned rows in SQLite -- full history preserved |
+| D-INV-6 | Rating uses 1-5 integer scale with weighted average (recent reviews weighted higher) |
+| D-INV-7 | Search uses BM25 keyword matching on template name, description, and content |
+| D-INV-8 | Cross-project import creates a copy -- no shared mutable state between projects |
+
+## Edge Cases
+
+- Importing a template for a different framework maps controls via crosswalk engine
+- Empty search query returns top-rated templates
+- Duplicate template name within same project appends version suffix
+- Deleted source template does not affect already-imported copies
+
+## Tier Gating
+
+| Capability | Community | Pro |
+|------------|-----------|-----|
+| Create templates | 5 max | Unlimited |
+| Search and browse | Yes | Yes |
+| Rate templates | Yes | Yes |
+| Publish to exchange | Yes | Yes |
+| Cross-project import | No | Yes |
+
+## Security
+
+- Template content scanned for secrets before publication
+- CUI markings enforced on all templates at publication time
+- Audit trail logged for all create/publish/import operations

goals/thread_heatmap.md

@@ -0,0 +1,77 @@
+# CUI // SP-CTI
+
+# F5: Digital Thread Gap Heatmap
+
+## Purpose
+
+Visualize traceability coverage across the digital thread (requirements -> design -> code -> test -> compliance). Identify orphaned artifacts, missing links, and coverage gaps as a heatmap matrix to prioritize remediation efforts.
+
+## Prerequisites
+
+- `data/icdev.db` initialized with MBSE tables
+- Digital thread links populated via `tools/mbse/digital_thread.py`
+- At least one model imported via `xmi_parser.py` or `reqif_parser.py`
+
+## Workflow Steps
+
+### 1. Generate Heatmap
+```bash
+python tools/mbse/thread_heatmap.py --heatmap --project-id "sparkpilot" --json
+```
+**Expected output:** JSON with heatmap matrix (rows = source artifacts, columns = target artifact types), cell values as coverage percentages (0.0-1.0), and overall thread health score.
+
+### 2. Get Gap Matrix
+```bash
+python tools/mbse/thread_heatmap.py --gap-matrix --project-id "sparkpilot" --json
+```
+**Expected output:** JSON array of gap entries with source artifact ID, missing link types, severity (critical/high/medium/low), and suggested remediation.
+
+### 3. Get Orphaned Artifacts
+```bash
+python tools/mbse/thread_heatmap.py --orphans --project-id "sparkpilot" --json
+```
+**Expected output:** JSON array of artifacts with zero inbound or outbound links, grouped by artifact type.
+
+### 4. Check Coverage Threshold
+```bash
+python tools/mbse/thread_heatmap.py --coverage-check --project-id "sparkpilot" --threshold 0.8 --json
+```
+**Expected output:** JSON with pass/fail status, current coverage percentage, threshold, and list of below-threshold artifact pairs.
+
+### 5. Get Historical Trend
+```bash
+python tools/mbse/thread_heatmap.py --trend --project-id "sparkpilot" --window-days 30 --json
+```
+**Expected output:** JSON array of daily coverage snapshots showing improvement or regression.
+
+## Decision Reference
+
+| Decision | Description |
+|----------|-------------|
+| D-INV-17 | Heatmap computed from digital_thread_links table via SQL aggregation -- no external deps |
+| D-INV-18 | Gap severity based on artifact type: requirement->test gaps are CRITICAL, design->code gaps are HIGH |
+| D-INV-19 | Orphan detection uses bidirectional link check -- both unlinked sources and unlinked targets |
+| D-INV-20 | Coverage threshold configurable per project, defaults to 0.8 |
+
+## Edge Cases
+
+- Empty digital thread returns heatmap with all zeros and setup instructions
+- Single artifact type returns 1x1 matrix with self-link coverage
+- Deleted artifacts excluded from heatmap but logged in gap report
+- Coverage check with threshold 0.0 always passes (used for baseline)
+
+## Tier Gating
+
+| Capability | Community | Pro |
+|------------|-----------|-----|
+| Heatmap generation | Yes | Yes |
+| Gap matrix | Yes | Yes |
+| Orphan detection | Yes | Yes |
+| CI/CD gate integration | No | Yes |
+| Historical trending | No | Yes |
+
+## Security
+
+- Heatmap data is read-only aggregation -- no mutations
+- Coverage check results logged to audit trail
+- CUI markings applied to exported heatmap reports

goals/threat_modeler.md

@@ -0,0 +1,77 @@
+# CUI // SP-CTI
+
+# F7: STRIDE Threat Modeler
+
+## Purpose
+
+Systematic threat modeling using the STRIDE methodology (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege). Maps identified threats to NIST 800-53 controls and auto-generates POAM entries for unmitigated risks.
+
+## Prerequisites
+
+- `data/icdev.db` initialized with project and compliance tables
+- System architecture documented (components, data flows, trust boundaries)
+- Control mappings populated via `tools/compliance/control_mapper.py`
+
+## Workflow Steps
+
+### 1. Create Threat Model
+```bash
+python tools/security/threat_modeler.py --create --project-id "sparkpilot" --name "API Gateway Model" --json
+```
+**Expected output:** JSON with model ID, name, creation timestamp, and status (draft).
+
+### 2. Analyze Components
+```bash
+python tools/security/threat_modeler.py --analyze --model-id "tm-001" --components /path/to/components.json --json
+```
+**Expected output:** JSON with STRIDE analysis per component: threat ID, category (S/T/R/I/D/E), description, likelihood (1-5), impact (1-5), risk score, and mitigation status.
+
+### 3. Map Threats to NIST Controls
+```bash
+python tools/security/threat_modeler.py --map-controls --model-id "tm-001" --json
+```
+**Expected output:** JSON mapping each threat to recommended NIST 800-53 controls, with implementation status from existing compliance data.
+
+### 4. Auto-Generate POAM Entries
+```bash
+python tools/security/threat_modeler.py --auto-poam --model-id "tm-001" --json
+```
+**Expected output:** JSON with generated POAM entries for unmitigated threats, including milestone dates and responsible parties.
+
+### 5. Get Model Summary
+```bash
+python tools/security/threat_modeler.py --summary --model-id "tm-001" --json
+```
+**Expected output:** JSON with threat counts by STRIDE category, risk distribution, mitigation coverage, and residual risk score.
+
+## Decision Reference
+
+| Decision | Description |
+|----------|-------------|
+| D-INV-25 | STRIDE analysis uses deterministic rule engine per component type -- air-gap safe |
+| D-INV-26 | Risk score = likelihood x impact (1-25 scale), thresholds: LOW < 6, MEDIUM < 12, HIGH < 20, CRITICAL >= 20 |
+| D-INV-27 | NIST mapping uses crosswalk engine for multi-framework propagation |
+| D-INV-28 | POAM entries auto-generated with 90-day default milestones for HIGH/CRITICAL |
+
+## Edge Cases
+
+- Component with no applicable STRIDE threats returns clean report
+- Missing control mapping triggers crosswalk engine lookup before failing
+- Duplicate threat model name within project appends version suffix
+- Empty component list returns model shell with instructions
+
+## Tier Gating
+
+| Capability | Community | Pro |
+|------------|-----------|-----|
+| STRIDE analysis | Yes | Yes |
+| NIST control mapping | Yes | Yes |
+| POAM auto-generation | Yes | Yes |
+| ATT&CK mapping | No | Yes |
+| Auto-remediation suggestions | No | Yes |
+
+## Security
+
+- Threat models and POAM entries are append-only (NIST AU compliant)
+- Model data scoped to project -- no cross-project access
+- CUI markings applied to all exported threat model artifacts