project-iris 0.0.1

package/src/iris_bundle/frameworks/iris-core/memory-bank/intents/005-testing-strategy/units.md

@@ -0,0 +1,332 @@
+# Units: Testing Strategy
+
+## Unit Decomposition
+
+This intent is decomposed into five implementation units covering six testing layers.
+
+**See**: `memory-bank/research/test_strategy/testing-strategy.md` for detailed pyramid, workflows, and CI/CD integration.
+
+---
+
+## 6-Layer Testing Pyramid
+
+```mermaid
+graph TB
+    subgraph pyramid [" "]
+        direction TB
+        L6["🔍 Release Validation<br/><i>Human Review + Grok 4.1 Free</i>"]
+        L5["🤖 Agent Behavior<br/><i>Promptfoo + OpenRouter Free</i>"]
+        L4["🔄 E2E Workflow<br/><i>Custom Test Harness</i>"]
+        L3["⚙️ Integration<br/><i>BATS + Node.js</i>"]
+        L2["📋 Schema/Contract<br/><i>JSON Schema + markdownlint</i>"]
+        L1["🧪 Unit Tests<br/><i>Vitest</i>"]
+    end
+    L6 --> L5 --> L4 --> L3 --> L2 --> L1
+```
+
+| Layer | Speed | Cost | Determinism | Technology |
+|-------|-------|------|-------------|------------|
+| Unit Tests | ~ms | Free | 100% | Vitest |
+| Schema/Contract | ~ms | Free | 100% | JSON Schema, markdownlint |
+| Integration | ~sec | Free | 100% | BATS |
+| E2E Workflow | ~min | Free | ~95% | Custom harness |
+| Agent Behavior | ~min | Free | ~70-85% | Promptfoo + OpenRouter |
+| Release Validation | ~hr | Free | ~70-85% + human | Grok 4.1 Fast |
+
+**Total Cost**: $0 using OpenRouter free tier models.
+
+---
+
+## Unit 1: Specification Contract Testing
+
+### Description
+
+Schema validation and contract enforcement for all markdown-based specifications.
+
+### Responsibilities
+
+- Validate `memory-bank.yaml` against JSON Schema
+- Enforce required sections in intent artifacts
+- Validate unit brief template compliance
+- Enforce bolt template structure
+- Run markdown linting for structure consistency
+
+### Components
+
+| Component | File | Description |
+|-----------|------|-------------|
+| Schema Validator | `__tests__/unit/schema-validation/` | JSON Schema validation |
+| Intent Validator | `__tests__/unit/schema-validation/intent-schema.test.ts` | Intent structure checks |
+| Unit Validator | `__tests__/unit/schema-validation/unit-schema.test.ts` | Unit brief validation |
+| Markdown Linter | `.markdownlint.yaml` | Markdown structure rules |
+
+### Dependencies
+
+- JSON Schema library (Ajv)
+- markdownlint
+- remark-lint
+
+---
+
+## Unit 2: CLI & Installer Testing
+
+### Description
+
+Traditional testing for deterministic CLI commands and installer operations.
+
+### Responsibilities
+
+- Unit test installer components
+- Integration test slash commands
+- Snapshot test generated files
+- Property-based test Memory Bank operations
+
+### Components
+
+| Component | File | Description |
+|-----------|------|-------------|
+| Installer Tests | `__tests__/unit/installers/` | Unit tests for installers |
+| CLI Integration | `__tests__/integration/cli/` | BATS command tests |
+| Snapshot Tests | `__tests__/unit/template-generation/` | Generated file snapshots |
+| Memory Bank Tests | `__tests__/integration/memory-bank/` | CRUD operation tests |
+
+### Dependencies
+
+- Jest/Vitest
+- BATS (Bash Automated Testing System)
+- Mock filesystem library
+
+---
+
+## Unit 3: Agent Behavior Evaluation
+
+### Description
+
+Non-deterministic testing for LLM-driven agent outputs using multiple evaluation strategies.
+
+**See**: `memory-bank/research/test_strategy/promptfoo-tutorial.md` for Promptfoo configuration and assertions.
+
+### Responsibilities
+
+- Mock LLM responses for deterministic CI runs
+- Evaluate outputs via LLM-as-judge
+- Compute semantic similarity against golden examples
+- Assert behavioral properties (not exact matches)
+- Handle test flakiness from non-determinism
+
+### Components
+
+| Component | File | Description |
+|-----------|------|-------------|
+| **Promptfoo Config** | `promptfoo.yaml` | Declarative test configuration |
+| Mock Responses | `__tests__/fixtures/mock-responses/` | Recorded LLM outputs |
+| LLM Judge Rubrics | `__tests__/evaluation/rubrics/` | Evaluation criteria |
+| Semantic Similarity | `__tests__/evaluation/` | Embedding comparison |
+| Behavioral Assertions | `__tests__/e2e/` | Property-based checks |
+
+### Dependencies
+
+- **Promptfoo** (primary evaluation framework)
+- **OpenRouter** (LLM-as-judge provider, free tier)
+  - `x-ai/grok-4.1-fast:free` - Coding agents (2M context)
+  - `qwen/qwen3-coder:free` - Code-specialized
+- vcrpy/responses for mocking
+
+### Evaluation Rubrics
+
+| Rubric | File | Criteria |
+|--------|------|----------|
+| Intent Quality | `rubrics/intent-quality.yaml` | Completeness, clarity, testability |
+| Unit Completeness | `rubrics/unit-completeness.yaml` | Required sections, dependencies |
+| Bolt Validity | `rubrics/bolt-validity.yaml` | Stage definitions, acceptance criteria |
+
+---
+
+## Unit 4: Golden Dataset Management
+
+### Description
+
+Curated test cases with known-good outputs for regression testing.
+
+**See**: `memory-bank/research/test_strategy/promptfoo-irismd-tutorial.md` for detailed golden dataset and fixture guide.
+
+### Responsibilities
+
+- Maintain input/output examples per agent
+- Track baseline quality scores
+- Detect regressions from baseline
+- Version golden datasets with code
+- Manage pre-configured memory bank fixtures
+
+### Components
+
+| Component | File | Description |
+|-----------|------|-------------|
+| Inception Dataset | `__tests__/evaluation/golden-datasets/inception/` | Inception agent examples |
+| Construction Dataset | `__tests__/evaluation/golden-datasets/construction/` | Construction agent examples |
+| Operations Dataset | `__tests__/evaluation/golden-datasets/operations/` | Operations agent examples |
+| Master Dataset | `__tests__/evaluation/golden-datasets/master/` | Routing examples |
+| Baselines | `__tests__/evaluation/regression/baselines/` | Historical scores |
+| **Test Fixtures** | `__tests__/fixtures/memory-bank-states/` | Pre-configured memory bank states |
+
+### Dataset Structure
+
+```text
+golden-datasets/
+├── inception/
+│   ├── inputs/
+│   │   ├── 001-simple-feature.txt
+│   │   ├── 002-complex-system.txt
+│   │   └── ...
+│   └── outputs/
+│       ├── 001-simple-feature-intent.md
+│       ├── 002-complex-system-intent.md
+│       └── ...
+├── construction/
+│   ├── inputs/
+│   └── outputs/
+└── ...
+```
+
+### Test Fixtures (Memory Bank States)
+
+Pre-configured memory bank states for reproducible testing:
+
+| Fixture | Description | Tests |
+|---------|-------------|-------|
+| `01-empty-project/` | Fresh project, no irismd init | Project initialization |
+| `02-initialized-project/` | After `irismd install` | Agent routing, context loading |
+| `03-inception-started/` | Intent created, no units | Unit decomposition, requirements |
+| `04-inception-complete/` | Full intent with units and stories | Transition to construction |
+| `05-construction-in-progress/` | Bolts planned, stage 2 active | Bolt execution, stage advancement |
+| `06-construction-complete/` | All bolts completed | Transition to operations |
+
+**Structure**:
+
+```text
+__tests__/fixtures/memory-bank-states/
+├── 01-empty-project/
+│   └── (empty)
+├── 02-initialized-project/
+│   └── .irismd/
+│       ├── memory-bank.yaml
+│       └── context-config.yaml
+├── 03-inception-started/
+│   └── .irismd/
+│       ├── memory-bank.yaml
+│       └── memory-bank/
+│           └── intents/
+│               └── user-auth/
+│                   └── requirements.md
+...
+```
+
+### Minimum Coverage
+
+| Agent | Minimum Examples | Target Coverage |
+|-------|-----------------|-----------------|
+| Master Agent | 20 | 100+ |
+| Inception Agent | 10 | 50+ |
+| Construction Agent | 10 | 50+ |
+| Operations Agent | 5 | 25+ |
+
+---
+
+## Unit 5: CI/CD Integration
+
+### Description
+
+Pipeline automation for test execution and quality gate enforcement.
+
+### Responsibilities
+
+- Trigger tests on PR and merge events
+- Enforce quality thresholds
+- Report results to PRs
+- Block releases on failures
+- Support human approval gates
+
+### Components
+
+| Component | File | Description |
+|-----------|------|-------------|
+| PR Workflow | `.github/workflows/pr-tests.yml` | Fast deterministic tests |
+| Main Workflow | `.github/workflows/main-tests.yml` | Agent evaluation on merge |
+| Nightly Workflow | `.github/workflows/nightly-eval.yml` | Full golden dataset |
+| Quality Gates | `.github/workflows/quality-gates.yml` | Threshold enforcement |
+
+### Pipeline Tiers
+
+| Tier | Trigger | Tests | Duration |
+|------|---------|-------|----------|
+| Tier 1 | Every PR | Schema, CLI, Snapshots | ~2 min |
+| Tier 2 | Merge to main | + Quick agent eval | ~5 min |
+| Tier 3 | Nightly | Full golden dataset | ~30 min |
+| Tier 4 | Release | Full suite + human review | ~1 hour |
+
+### Quality Thresholds
+
+| Metric | Target | Alert |
+|--------|--------|-------|
+| Schema Validation | 100% | < 100% |
+| CLI Test Pass Rate | 100% | < 100% |
+| Agent Quality Score | > 0.85 | < 0.80 |
+| Semantic Similarity | > 0.90 | < 0.85 |
+| Regression from Baseline | 0% | > 5% |
+
+---
+
+## Unit Dependency Graph
+
+```text
+┌─────────────────────────────────────────────────────────────┐
+│                    CI/CD Integration                        │
+│                       (Unit 5)                              │
+│                    Orchestrates all                         │
+└───────────────────────────┬─────────────────────────────────┘
+                            │ triggers
+        ┌───────────────────┼───────────────────┐
+        │                   │                   │
+        ▼                   ▼                   ▼
+┌───────────────┐  ┌───────────────┐  ┌───────────────────────┐
+│ Specification │  │ CLI/Installer │  │ Agent Behavior        │
+│ Contract      │  │ Testing       │  │ Evaluation            │
+│ Testing       │  │ (Unit 2)      │  │ (Unit 3)              │
+│ (Unit 1)      │  │               │  │                       │
+│               │  │ - Unit tests  │  │ - Mock responses      │
+│ - Schema      │  │ - Integration │  │ - LLM-as-judge        │
+│ - Linting     │  │ - Snapshots   │  │ - Semantic similarity │
+└───────────────┘  └───────────────┘  └───────────┬───────────┘
+                                                  │ uses
+                                                  ▼
+                                      ┌───────────────────────┐
+                                      │ Golden Dataset        │
+                                      │ Management            │
+                                      │ (Unit 4)              │
+                                      │                       │
+                                      │ - Test cases          │
+                                      │ - Baselines           │
+                                      │ - Regression tracking │
+                                      └───────────────────────┘
+```
+
+---
+
+## Implementation Order & Status
+
+| # | Unit | Status | Notes |
+|---|------|--------|-------|
+| 1 | Specification Contract Testing | ✅ Complete | 7 tests, YAML + MD validators |
+| 2 | CLI/Installer Testing | ⏳ Pending | Next priority |
+| 3 | Golden Dataset Management | ⏳ Pending | Required for Unit 4 |
+| 4 | Agent Behavior Evaluation | ⏳ Pending | Depends on golden datasets |
+| 5 | CI/CD Integration | 🚧 In Progress | GitHub Action created |
+
+### Completed Work (Unit 1)
+
+- ✅ `src/lib/yaml-validator.ts` - YAML config validation
+- ✅ `src/lib/markdown-validator.ts` - Markdown template validation
+- ✅ `src/__tests__/schemas/` - 5 YAML schema definitions
+- ✅ `src/__tests__/unit/schema-validation/` - 7 passing tests
+- ✅ `.markdownlint.yaml` - Markdown linting rules
+- ✅ `.github/workflows/test.yml` - CI workflow (pending commit)

package/src/iris_bundle/frameworks/iris-core/memory-bank/intents/006-brownfield-support/requirements.md

@@ -0,0 +1,214 @@
+# Intent: Brownfield Application Support
+
+## Overview
+
+Add support for brownfield application development workflows in irismd. This enables working with existing codebases - enhancements, migrations, modernization, defect fixes, and refactoring.
+
+Currently irismd only supports greenfield (new application) workflows. This intent adds:
+
+1. Level 1 Plan generation based on intent type
+2. Elevation phase (code archaeology → living specification)
+3. Different workflow paths for non-greenfield scenarios
+
+---
+
+## Related Research
+
+- `/memory-bank/research/unified-modernization-model.md` - Core model: UNDERSTAND → EXTRACT → CRYSTALLIZE → EVOLVE
+- `/memory-bank/research/vibe-to-production-academic-research.md` - Academic foundation (40+ papers)
+- `/memory-bank/research/vibe-to-spec-flow-options.md` - Flow options analysis
+
+---
+
+## Background: Level 1 Plan (AI-DLC Concept)
+
+Per AI-DLC specification:
+> "Given a business intent (ex. Green-Field development, Brown-Field enhancement, modernization, or defect fixing), AI-DLC begins by prompting AI to generate a **Level 1 Plan** that outlines the workflow to implement the intent."
+
+### Intent Types and Their Level 1 Plans
+
+| Intent Type | Level 1 Plan Workflow |
+|-------------|----------------------|
+| **Green-Field** | Inception → Construction → Operations |
+| **Brown-Field Enhancement** | Elevation → Inception → Construction → Operations |
+| **Defect Fix** | Elevation (targeted) → Construction bolt |
+| **Refactoring** | Elevation → Construction bolt(s) |
+| **Migration/Modernization** | Elevation → Inception → Construction → Operations |
+| **Vibe-to-Spec Conversion** | Elevation → Inception → Construction → Operations |
+
+### Key Insight from Research
+
+> "Legacy modernization and vibe-coding-to-spec conversion are fundamentally the SAME problem: existing code with embedded knowledge that needs to be preserved while adopting spec-driven development."
+
+---
+
+## The Unified Model: Elevation Phase
+
+From `/memory-bank/research/unified-modernization-model.md`:
+
+```text
+ANY Existing Codebase (Legacy OR Vibe-Coded)
+                ↓
+    [1. UNDERSTAND] - Code Archaeology
+                ↓
+    [2. EXTRACT] - Specification Mining
+                ↓
+    [3. CRYSTALLIZE] - Living Specification
+                ↓
+    [4. EVOLVE] - Incremental AI-DLC Adoption
+```
+
+### Elevation = UNDERSTAND + EXTRACT + CRYSTALLIZE
+
+The Elevation phase combines these three steps to prepare existing code for AI-DLC:
+
+1. **UNDERSTAND**: Analyze codebase structure, behavior, and domain knowledge
+2. **EXTRACT**: Mine specifications from code (user stories, contracts, tests)
+3. **CRYSTALLIZE**: Transform into AI-DLC compatible artifacts
+
+---
+
+## Functional Requirements
+
+### FR-1: Intent Type Classification
+
+- FR-1.1: System SHALL classify user intent into: Green-Field, Brown-Field Enhancement, Defect Fix, Refactoring, Migration, Vibe-to-Spec
+- FR-1.2: Classification SHALL happen when user describes their intent
+- FR-1.3: Classification SHALL be confirmed with user before proceeding
+- FR-1.4: For Green-Field, Level 1 Plan is implicit (always Inception → Construction → Operations)
+
+### FR-2: Level 1 Plan Generation (Non-Greenfield Only)
+
+- FR-2.1: System SHALL generate Level 1 Plan only for non-greenfield intents
+- FR-2.2: Level 1 Plan SHALL outline workflow phases specific to intent type
+- FR-2.3: Level 1 Plan SHALL be presented to user for validation
+- FR-2.4: Level 1 Plan SHALL be stored in memory-bank/intents/{intent}/level-1-plan.md
+
+### FR-3: Elevation Phase
+
+- FR-3.1: System SHALL support Elevation phase for brownfield intents
+- FR-3.2: Elevation SHALL be divided into: UNDERSTAND, EXTRACT, CRYSTALLIZE sub-phases
+- FR-3.3: UNDERSTAND SHALL produce: structure analysis, behavior analysis, knowledge map
+- FR-3.4: EXTRACT SHALL produce: intent, requirements, contracts, golden tests
+- FR-3.5: CRYSTALLIZE SHALL produce: AI-DLC compatible specifications with traceability
+- FR-3.6: Each sub-phase output SHALL be validated by human before proceeding
+
+### FR-4: Defect Fix Workflow
+
+- FR-4.1: System SHALL support streamlined defect fix workflow
+- FR-4.2: Defect fix SHALL skip full Inception (no units decomposition needed)
+- FR-4.3: Defect fix SHALL perform TARGETED elevation of affected code only
+- FR-4.4: Defect fix SHALL execute single targeted construction bolt
+
+### FR-5: Refactoring Workflow
+
+- FR-5.1: System SHALL support refactoring workflow
+- FR-5.2: Refactoring SHALL preserve existing behavior (no new features)
+- FR-5.3: Refactoring SHALL require golden tests before changes
+- FR-5.4: Refactoring SHALL use targeted elevation of affected code
+
+### FR-6: Migration/Modernization Workflow
+
+- FR-6.1: System SHALL support migration workflow
+- FR-6.2: Migration SHALL perform full codebase elevation
+- FR-6.3: Migration SHALL go through Inception for target architecture
+- FR-6.4: Migration SHALL support Strangler Fig pattern for incremental migration
+
+---
+
+## Non-Functional Requirements
+
+### NFR-1: Backwards Compatibility
+
+- Existing greenfield workflows SHALL continue unchanged
+- Greenfield SHALL NOT require Level 1 Plan approval (implicit)
+
+### NFR-2: Traceability
+
+- Extracted specifications SHALL trace back to source code locations
+- Golden tests SHALL validate behavioral equivalence
+- Confidence scores SHALL indicate extraction certainty
+
+### NFR-3: Incremental Adoption
+
+- Elevation SHALL be incremental (can stop and resume)
+- Teams SHALL be able to elevate one module at a time
+- Progress SHALL be persisted in memory-bank
+
+---
+
+## Acceptance Criteria
+
+### AC-1: Intent Classification
+
+- GIVEN a user describes "fix the login bug"
+- WHEN system classifies the intent
+- THEN intent type is "Defect Fix" with streamlined Level 1 Plan
+
+### AC-2: Level 1 Plan for Brownfield
+
+- GIVEN a user describes "add payment feature to existing e-commerce app"
+- WHEN system generates Level 1 Plan
+- THEN plan shows: Elevation → Inception → Construction → Operations
+
+### AC-3: Greenfield No Level 1 Plan
+
+- GIVEN a user describes "build a new todo app from scratch"
+- WHEN system classifies as Green-Field
+- THEN NO explicit Level 1 Plan is shown (implicit standard workflow)
+
+### AC-4: Elevation Phase Outputs
+
+- GIVEN a brownfield intent with existing codebase
+- WHEN Elevation phase completes
+- THEN memory-bank contains: understanding/, extracted/, and AI-DLC intent structure
+
+### AC-5: Defect Fix Streamlined
+
+- GIVEN a defect fix intent
+- WHEN user proceeds through workflow
+- THEN full Inception is skipped, only targeted elevation + construction bolt occur
+
+---
+
+## Memory Bank Extension
+
+```text
+memory-bank/
+├── elevation/                       # NEW: Elevation phase artifacts
+│   └── {project-name}/
+│       ├── understanding/
+│       │   ├── structure-report.md
+│       │   ├── behavior-report.md
+│       │   └── knowledge-map.md
+│       ├── extracted/
+│       │   ├── intent.md
+│       │   ├── requirements.md
+│       │   ├── contracts/
+│       │   └── golden-tests/
+│       └── traceability/
+│           └── matrix.md
+│
+├── intents/
+│   └── {intent}/
+│       ├── level-1-plan.md          # NEW: Only for non-greenfield
+│       ├── requirements.md
+│       └── ...
+```
+
+---
+
+## Out of Scope (v1)
+
+- Automated code analysis tools integration (SonarQube, etc.)
+- Multi-repository brownfield support
+- Legacy language migration (COBOL, etc.)
+- Database schema migration tooling
+
+---
+
+## Dependencies
+
+- 001-multi-agent-orchestration (agents must support brownfield workflows)
+- 003-memory-bank-system (store elevation artifacts)
+- 004-standards-system (elevation templates)