icdev 1.0.0__py3-none-any.whl

icdev/data/docs/features/phase-01-gotcha-framework.md

@@ -0,0 +1,249 @@
+# Phase 1 — GOTCHA Framework Foundation
+
+**CUI // SP-CTI**
+
+| Field | Value |
+|-------|-------|
+| Phase | 1 |
+| Title | GOTCHA Framework Foundation |
+| Status | Implemented |
+| Priority | P0 |
+| Dependencies | None |
+| Author | ICDEV Architect Agent |
+| Date | 2026-02-23 |
+
+---
+
+## 1. Problem Statement
+
+Large Language Models are probabilistic systems. When used directly for business logic, accuracy compounds negatively across steps: 90% accuracy per step yields only ~59% accuracy over 5 sequential steps. In government and DoD environments, this compounding unreliability is unacceptable. Systems handling CUI, FOUO, or SECRET information must produce deterministic, auditable, and reproducible results.
+
+The fundamental challenge is separating what LLMs are good at (orchestration, decision-making, natural language understanding) from what must be deterministic (business logic, compliance checks, security scanning, infrastructure generation). Without a structured framework, AI-assisted development devolves into "vibe coding" -- producing demos that cannot survive production scrutiny, compliance audits, or operational stress.
+
+ICDEV requires a foundational architecture that enforces this separation of concerns while providing a complete operational model: persistent memory across sessions, manifest-driven tool discovery, goal-driven workflows, configurable behavior, reusable context, and templated prompts. This is the GOTCHA framework.
+
+---
+
+## 2. Goals
+
+1. Establish a 6-layer architecture (Goals, Orchestration, Tools, Args, Context, Hard Prompts) that enforces separation between probabilistic AI orchestration and deterministic tool execution
+2. Implement a manifest system (`goals/manifest.md`, `tools/manifest.md`) that serves as a single source of truth for all available workflows and tools
+3. Build a dual-storage memory system (markdown files for human readability + SQLite databases for machine searchability) with 6 memory types
+4. Define a session start protocol that loads long-term context, daily logs, and project state before any work begins
+5. Create a guardrails system that prevents common failure modes (tool duplication, unverified output chaining, destructive operations)
+6. Provide configurable behavior through YAML/JSON args files that change system behavior without modifying goals or tools
+7. Enable static reference material (context/) and reusable LLM instruction templates (hardprompts/) for consistent outputs
+
+---
+
+## 3. Architecture
+
+### 3.1 The 6 Layers
+
+```
++---------------------------------------------------------------+
+|                    GOTCHA Framework                             |
+|                                                                |
+|  +-------------------+    +-----------------------------+      |
+|  | GOALS             |    | ORCHESTRATION (LLM)         |      |
+|  | goals/            |    | Read goal -> decide tool    |      |
+|  | Process defs,     |--->| order -> apply args ->      |      |
+|  | workflows,        |    | reference context ->        |      |
+|  | expected outputs  |    | handle errors               |      |
+|  +-------------------+    +----+---+----+----+----------+      |
+|                                |   |    |    |                 |
+|          +-----------+---------+   |    |    +--------+        |
+|          v           v             v    v             v        |
+|  +------------+ +---------+ +---------+ +-----------+          |
+|  | TOOLS      | | ARGS    | | CONTEXT | | HARD      |          |
+|  | tools/     | | args/   | | context/| | PROMPTS   |          |
+|  | Python     | | YAML/   | | Static  | | hardprompts/         |
+|  | scripts,   | | JSON    | | ref     | | Reusable  |          |
+|  | one job    | | behavior| | material| | LLM       |          |
+|  | each,      | | settings| | tone,   | | templates |          |
+|  | determin-  | |         | | samples | |           |          |
+|  | istic      | |         | |         | |           |          |
+|  +------------+ +---------+ +---------+ +-----------+          |
++---------------------------------------------------------------+
+```
+
+### 3.2 Memory System
+
+```
++-------------------------------------+
+|         MEMORY SYSTEM               |
+|                                     |
+|  +---------------+  +------------+  |
+|  | Markdown      |  | SQLite     |  |
+|  | (Human)       |  | (Machine)  |  |
+|  |               |  |            |  |
+|  | MEMORY.md     |  | memory.db  |  |
+|  | logs/YYYY-    |  |  - entries |  |
+|  |   MM-DD.md    |  |  - logs    |  |
+|  |               |  |  - access  |  |
+|  +---------------+  +------------+  |
+|                                     |
+|  Types: fact, preference, event,    |
+|         insight, task, relationship |
+|                                     |
+|  Search: hybrid = 0.7*BM25 +       |
+|          0.3*semantic (vector)      |
+|                                     |
+|  Embeddings: text-embedding-3-small |
+|              1536 dims, BLOB store  |
++-------------------------------------+
+```
+
+### 3.3 Key Files
+
+| File | Purpose |
+|------|---------|
+| `goals/manifest.md` | Index of all goal workflows |
+| `tools/manifest.md` | Master list of all tools |
+| `memory/MEMORY.md` | Curated long-term facts and preferences |
+| `memory/logs/YYYY-MM-DD.md` | Daily session logs |
+| `.env` | API keys and environment variables |
+| `.tmp/` | Disposable scratch work |
+| `args/project_defaults.yaml` | Default TDD, compliance, security, infra settings |
+
+---
+
+## 4. Requirements
+
+### 4.1 Layer Separation
+
+#### REQ-01-001: Deterministic Tool Execution
+The system SHALL execute all business logic through deterministic Python scripts in `tools/`, never through direct LLM generation of business-critical outputs.
+
+#### REQ-01-002: Single Responsibility Tools
+Each tool in `tools/` SHALL perform exactly one job, accepting structured inputs and producing structured outputs.
+
+#### REQ-01-003: Orchestration Layer
+The AI orchestration layer SHALL read goals, decide tool execution order, apply args, reference context, and handle errors without directly executing business logic.
+
+#### REQ-01-004: Configurable Behavior
+The system SHALL support YAML/JSON configuration files in `args/` that modify tool behavior without requiring changes to goal definitions or tool code.
+
+### 4.2 Manifest System
+
+#### REQ-01-005: Goal Manifest
+The system SHALL maintain a `goals/manifest.md` file indexing all available goal workflows with file paths and descriptions.
+
+#### REQ-01-006: Tool Manifest
+The system SHALL maintain a `tools/manifest.md` file indexing all available tools, checked before creating any new script.
+
+#### REQ-01-007: Manifest Verification
+The orchestration layer SHALL check the relevant manifest before starting any task or creating any new tool.
+
+### 4.3 Memory System
+
+#### REQ-01-008: Dual Storage
+The memory system SHALL store data in both markdown files (human-readable) and SQLite databases (machine-searchable).
+
+#### REQ-01-009: Memory Types
+The memory system SHALL support 6 memory types: fact, preference, event, insight, task, and relationship.
+
+#### REQ-01-010: Hybrid Search
+The memory system SHALL support hybrid search combining BM25 keyword search (weight 0.7) and semantic vector search (weight 0.3), configurable via CLI flags.
+
+#### REQ-01-011: Embedding Storage
+The memory system SHALL store OpenAI text-embedding-3-small embeddings (1536 dimensions) as BLOBs in SQLite for vector similarity search.
+
+#### REQ-01-012: Session Start Protocol
+The system SHALL load `memory/MEMORY.md`, today's daily log, and yesterday's log at the start of every session before performing any work.
+
+### 4.4 Guardrails
+
+#### REQ-01-013: Tool Duplication Prevention
+The system SHALL check `tools/manifest.md` before writing any new script to prevent duplicate tool creation.
+
+#### REQ-01-014: Output Verification
+The system SHALL verify tool output format before chaining output into another tool.
+
+#### REQ-01-015: Intermediate Output Preservation
+When a workflow fails mid-execution, the system SHALL preserve intermediate outputs before retrying.
+
+---
+
+## 5. Database Schema
+
+### Tables
+
+| Table | Purpose |
+|-------|---------|
+| `memory_entries` | All memory entries with embeddings, types, importance scores |
+| `daily_logs` | Daily session log entries linked to markdown files |
+| `memory_access_log` | Tracks when and how memory entries are accessed |
+| `tasks` | Activity tracking in `data/activity.db` |
+
+---
+
+## 6. Tools
+
+| Tool | Purpose |
+|------|---------|
+| `tools/memory/memory_read.py` | Load all memory (markdown format) |
+| `tools/memory/memory_write.py` | Write to daily log + DB, update MEMORY.md sections |
+| `tools/memory/memory_db.py` | Keyword search across memory database |
+| `tools/memory/semantic_search.py` | Semantic vector search (requires OpenAI key) |
+| `tools/memory/hybrid_search.py` | Combined keyword + semantic search (best results) |
+| `tools/memory/embed_memory.py` | Generate embeddings for all memory entries |
+| `tools/memory/time_decay.py` | Time-decay scoring for memory ranking |
+| `tools/db/init_icdev_db.py` | Initialize the ICDEV database |
+| `tools/project/project_create.py` | Create new project records |
+| `tools/project/project_list.py` | List all projects |
+| `tools/project/project_status.py` | Get project status |
+
+---
+
+## 7. Architecture Decisions
+
+| ID | Decision | Rationale |
+|----|----------|-----------|
+| D1 | SQLite for ICDEV internals; PostgreSQL for apps ICDEV builds | Zero-config portability for the framework itself; production-grade DB for generated applications |
+| D6 | Audit trail is append-only/immutable (no UPDATE/DELETE) | Satisfies NIST 800-53 AU controls; ensures tamper-evident audit records |
+| D168 | Time-decay uses exponential formula with per-type half-lives | Memory relevance degrades naturally; events decay fast (7d), facts slowly (90d), preferences slowest (180d) |
+
+---
+
+## 8. Security Gate
+
+**No dedicated security gate for Phase 1.**
+
+Phase 1 establishes the foundational architecture. Security gates are enforced starting in Phase 5 (Security Scanning) and Phase 7 (Code Review Gates). However, Phase 1 establishes the critical guardrail:
+
+- Audit trail is append-only -- no UPDATE/DELETE operations permitted on audit tables
+- Memory system uses SHA-256 hashing for entry integrity
+- `.env` files excluded from version control
+
+---
+
+## 9. Commands
+
+```bash
+# Initialize framework (first run)
+/initialize                    # Custom slash command -- sets up all dirs, manifests, memory, databases
+
+# Memory system
+python tools/memory/memory_read.py --format markdown          # Load all memory
+python tools/memory/memory_write.py --content "text" --type event  # Write to daily log + DB
+python tools/memory/memory_write.py --content "text" --type fact --importance 7  # Store a fact
+python tools/memory/memory_write.py --update-memory --content "text" --section user_preferences  # Update MEMORY.md
+python tools/memory/memory_db.py --action search --query "keyword"   # Keyword search
+python tools/memory/semantic_search.py --query "concept"             # Semantic search
+python tools/memory/hybrid_search.py --query "query"                 # Combined keyword + semantic
+python tools/memory/embed_memory.py --all                            # Generate embeddings for all entries
+python tools/memory/time_decay.py --rank --query "keyword" --top-k 10 --json  # Time-decay ranked search
+
+# Database initialization
+python tools/db/init_icdev_db.py                    # Initialize ICDEV database
+
+# Project management
+python tools/project/project_create.py --name "my-app" --type microservice
+python tools/project/project_list.py
+python tools/project/project_status.py --project-id "proj-123"
+
+# Health check
+python tools/testing/health_check.py                 # Full system health check
+python tools/testing/health_check.py --json           # JSON output
+```

icdev/data/docs/features/phase-02-atlas-build-workflow.md

@@ -0,0 +1,223 @@
+# Phase 2 — ATLAS Build Workflow
+
+**CUI // SP-CTI**
+
+| Field | Value |
+|-------|-------|
+| Phase | 2 |
+| Title | ATLAS Build Workflow |
+| Status | Implemented |
+| Priority | P0 |
+| Dependencies | Phase 1 (GOTCHA Framework Foundation) |
+| Author | ICDEV Architect Agent |
+| Date | 2026-02-23 |
+
+---
+
+## 1. Problem Statement
+
+Building production-grade applications with AI assistance requires more than code generation. Without a structured build methodology, AI-assisted development produces "vibe code" -- applications that demo well but fail under production scrutiny. Common failure modes include: building before designing (leading to full rewrites), skipping connection validation (hours wasted on broken integrations), no data modeling (schema changes cascading into UI rewrites), and no testing (shipping broken code).
+
+Government and DoD systems demand a rigorous, repeatable build process that produces auditable, traceable, and production-ready artifacts. The build process must enforce design-before-code, validate all integrations before implementation, and stress-test before delivery. Furthermore, Model-Based Systems Engineering (MBSE) projects require an additional pre-phase that imports authoritative system models and establishes a digital thread before any design decisions are made.
+
+ATLAS provides this structured 5-step build methodology (Architect, Trace, Link, Assemble, Stress-test), with M-ATLAS extending it to 6 steps by adding a Model pre-phase for MBSE-enabled projects. Each step maps directly to a GOTCHA layer, ensuring the framework's separation of concerns is maintained throughout the build lifecycle.
+
+---
+
+## 2. Goals
+
+1. Define a repeatable 5-step build process (ATLAS) that prevents common "vibe coding" anti-patterns
+2. Ensure all applications start with architecture decisions (Architect) before any code is written
+3. Require data schema definition, integration mapping, and technology stack approval (Trace) before implementation
+4. Validate all external connections (APIs, databases, MCP servers, OAuth flows) before building (Link)
+5. Enforce a layered build order: database first, backend second, frontend last (Assemble)
+6. Require functional, integration, and edge case testing before delivery (Stress-test)
+7. Support MBSE-enabled projects via M-ATLAS variant that adds a Model pre-phase for SysML/DOORS NG import
+8. Integrate acceptance criteria validation (V&V) as a mandatory gate in the Stress-test phase
+
+---
+
+## 3. Architecture
+
+### 3.1 ATLAS Pipeline
+
+```
++-------+    +-------+    +-------+    +----------+    +------------+
+|   A   |--->|   T   |--->|   L   |--->|    A     |--->|     S      |
+|Archi- |    |Trace  |    |Link   |    |Assemble  |    |Stress-test |
+|tect   |    |       |    |       |    |          |    |            |
+|Define |    |Data   |    |Validate|   |Build     |    |Test        |
+|problem,|   |schema,|    |connec-|    |layers:   |    |functional, |
+|users,  |    |integ- |    |tions  |    |DB->API-> |   |integration,|
+|success |    |rations|    |before |    |UI        |    |edge cases  |
+|metrics |    |stack  |    |build  |    |          |    |+ V&V gate  |
++-------+    +-------+    +-------+    +----------+    +------------+
+                                            ^                |
+                                            |  Issues found  |
+                                            +----------------+
+```
+
+### 3.2 M-ATLAS Variant (MBSE-Enabled)
+
+```
++-------+    +-------+    +-------+    +-------+    +----------+    +------------+
+|   M   |--->|   A   |--->|   T   |--->|   L   |--->|    A     |--->|     S      |
+|Model  |    |Archi- |    |Trace  |    |Link   |    |Assemble  |    |Stress-test |
+|       |    |tect   |    |       |    |       |    |          |    |            |
+|Import |    |System |    |Schema |    |Validate|   |Build w/  |    |Test incl.  |
+|XMI/   |    |design |    |+ model|    |+ model|    |model-gen |    |model-gen   |
+|ReqIF, |    |from   |    |trace- |    |code   |    |scaffold  |    |test stubs  |
+|digital|    |model  |    |ability|    |maps   |    |          |    |            |
+|thread |    |       |    |       |    |       |    |          |    |            |
++-------+    +-------+    +-------+    +-------+    +----------+    +------------+
+```
+
+### 3.3 GOTCHA Layer Mapping
+
+| ATLAS Step | GOTCHA Layer | Role |
+|------------|--------------|------|
+| Architect | Goals | Define the process, problem, success criteria |
+| Trace | Context | Reference patterns, data schemas, integrations |
+| Link | Args | Environment setup, connection validation |
+| Assemble | Tools | Deterministic execution of build steps |
+| Stress-test | Orchestration | AI validates results, evaluates acceptance criteria |
+
+---
+
+## 4. Requirements
+
+### 4.1 Architect Phase
+
+#### REQ-02-001: Problem Definition
+The system SHALL require a clear one-sentence problem statement before proceeding to any subsequent build phase.
+
+#### REQ-02-002: User Identification
+The system SHALL require identification of specific target users (not "everyone") as part of the architecture phase.
+
+#### REQ-02-003: Success Metrics
+The system SHALL require measurable success criteria that can be objectively evaluated during the Stress-test phase.
+
+#### REQ-02-004: Agentic Fitness Assessment
+The system SHALL evaluate each component's fitness for agentic architecture via `agentic_fitness.py`, scoring 6 dimensions (data_complexity, decision_complexity, user_interaction, integration_density, compliance_sensitivity, scale_variability) before architecture decisions.
+
+### 4.2 Trace Phase
+
+#### REQ-02-005: Data Schema First
+The system SHALL require data schema definition (tables, relationships, field types) before any code generation begins.
+
+#### REQ-02-006: Integration Mapping
+The system SHALL require documentation of all external service connections (APIs, databases, MCP servers) with authentication types and availability status.
+
+#### REQ-02-007: Technology Stack Approval
+The system SHALL propose a technology stack and obtain user approval before proceeding to the Link phase.
+
+#### REQ-02-008: Edge Case Documentation
+The system SHALL document potential failure modes (rate limits, auth expiry, timeouts, invalid input) during the Trace phase.
+
+### 4.3 Link Phase
+
+#### REQ-02-009: Connection Validation
+The system SHALL validate all database connections, API endpoints, MCP servers, and OAuth flows before proceeding to the Assemble phase.
+
+#### REQ-02-010: Validation Completeness
+All items on the connection validation checklist SHALL show green status before the Assemble phase begins.
+
+### 4.4 Assemble Phase
+
+#### REQ-02-011: Build Order Enforcement
+The system SHALL enforce the build order: database schema first, backend API routes second, frontend UI last.
+
+#### REQ-02-012: GOTCHA Separation
+Generated code SHALL follow GOTCHA separation of concerns -- UI components, business logic, and data access in distinct layers.
+
+### 4.5 Stress-test Phase
+
+#### REQ-02-013: Functional Testing
+The system SHALL verify all user-facing functionality (buttons, data persistence, navigation, error states) during the Stress-test phase.
+
+#### REQ-02-014: Integration Testing
+The system SHALL verify all external connections hold under test conditions (API calls succeed, auth persists, rate limits respected).
+
+#### REQ-02-015: Acceptance Validation Gate
+The system SHALL run acceptance criteria validation (`acceptance_validator.py`) with blocking conditions: 0 failed acceptance criteria, 0 pages with error patterns (500, tracebacks, JS errors).
+
+---
+
+## 5. Database Schema
+
+### Tables
+
+| Table | Purpose |
+|-------|---------|
+| `projects` | Project metadata including type, classification, status |
+| `agents` | Agent registry for multi-agent orchestration |
+| `audit_trail` | Append-only log of all build actions and decisions |
+| `agentic_fitness_assessments` | Fitness scoring results for agentic architecture decisions |
+
+---
+
+## 6. Tools
+
+| Tool | Purpose |
+|------|---------|
+| `tools/builder/agentic_fitness.py` | 6-dimension fitness scoring for agentic architecture |
+| `tools/builder/app_blueprint.py` | Generate blueprint from fitness scorecard + user decisions |
+| `tools/builder/scaffolder.py` | Scaffold project structure based on type and blueprint |
+| `tools/builder/code_generator.py` | Generate implementation code from test specs |
+| `tools/builder/test_writer.py` | Generate Gherkin features and pytest test cases |
+| `tools/builder/linter.py` | Run language-appropriate linting (flake8/ruff, eslint) |
+| `tools/builder/formatter.py` | Run language-appropriate formatting (black, prettier) |
+| `tools/builder/language_support.py` | Detect project languages and list supported languages |
+| `tools/testing/acceptance_validator.py` | Validate acceptance criteria against test evidence |
+
+---
+
+## 7. Architecture Decisions
+
+| ID | Decision | Rationale |
+|----|----------|-----------|
+| D9 | M-ATLAS adds "Model" pre-phase to ATLAS (backward compatible) | Skips if no model exists; ATLAS starts at Architect for non-MBSE projects |
+| D44 | Flag-based (`--agentic`) for backward compatibility | Omitting the flag produces identical output to pre-agentic behavior |
+| D46 | Fitness scoring: weighted rule-based + optional LLM override | Deterministic scoring with escape hatch for nuanced assessments |
+| D47 | Blueprint-driven generation -- single config drives all generators | One source of truth for the entire generation pipeline |
+
+---
+
+## 8. Security Gate
+
+**Acceptance Validation Gate:**
+- 0 failed acceptance criteria
+- 0 pages rendering with error patterns (500, tracebacks, JS errors)
+- Plan MUST contain `## Acceptance Criteria` section
+- V&V before handoff -- NEVER declare a fix/feature complete based solely on API validation; browser verification required for UI changes
+
+---
+
+## 9. Commands
+
+```bash
+# Agentic fitness assessment (Step 0)
+python tools/builder/agentic_fitness.py --spec "..." --project-id "proj-123" --json
+
+# Blueprint generation
+python tools/builder/app_blueprint.py --fitness-scorecard sc.json \
+  --user-decisions '{}' --app-name "my-app" --json
+
+# Project scaffolding
+python tools/builder/scaffolder.py --project-path /tmp --name "my-app" \
+  --type api --agentic --fitness-scorecard sc.json
+
+# Language detection
+python tools/builder/language_support.py --detect "/path/to/project"
+python tools/builder/language_support.py --list
+
+# M-ATLAS Model phase (MBSE-enabled projects)
+python tools/mbse/xmi_parser.py --project-id "proj-123" --file model.xmi --json
+python tools/mbse/reqif_parser.py --project-id "proj-123" --file reqs.reqif --json
+python tools/mbse/digital_thread.py --project-id "proj-123" auto-link --json
+python tools/mbse/model_code_generator.py --project-id "proj-123" --language python --output ./src
+python tools/mbse/model_control_mapper.py --project-id "proj-123" --map-all --json
+
+# Build workflow skill
+/icdev-build    # Build code using true TDD via M-ATLAS workflow
+```