mdan-cli 2.2.0 → 2.4.0

package/integrations/windsurf.md

@@ -23,14 +23,14 @@ You are operating inside Windsurf IDE with Cascade AI.
 - Use Cascade's flow awareness to maintain MDAN phase context across sessions
 
 ### File Organization
-All MDAN artifacts should be saved to `.mdan/artifacts/` for reference.
+All MDAN artifacts should be saved to `mdan/artifacts/` for reference.
 ```
 
 ### Step 2: Copy agents
 
 ```bash
-mkdir -p .mdan/agents .mdan/artifacts
-cp agents/*.md .mdan/agents/
+mkdir -p mdan/agents mdan/artifacts
+cp agents/*.md mdan/agents/
 ```
 
 ### Step 3: Using MDAN with Cascade
@@ -45,4 +45,4 @@ Cascade's multi-step reasoning pairs well with MDAN's structured phases. When st
 
 - Windsurf's Cascade is excellent for the BUILD phase — it can implement entire features autonomously
 - Use MDAN's Feature Briefs as Cascade tasks for predictable, structured implementation
-- Save architecture documents to `.mdan/artifacts/` so Cascade can reference them in context
+- Save architecture documents to `mdan/artifacts/` so Cascade can reference them in context

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mdan-cli",
-  "version": "2.2.0",
+  "version": "2.4.0",
   "description": "Multi-Agent Development Agentic Network - A modern, adaptive, LLM-agnostic methodology for building software",
   "main": "cli/mdan.js",
   "bin": {
@@ -39,7 +39,9 @@
     "integrations/",
     "memory/",
     "skills/",
-    "install.sh"
+    "install.sh",
+    ".mcp.json",
+    "AGENTS.md"
   ],
   "dependencies": {
     "@clack/prompts": "^1.0.1",

package/phases/04-verify.md

@@ -38,11 +38,13 @@ Implemented features: [List of completed features]
 
 Expected output:
 1. Complete test plan
-2. Unit tests for all features
+2. Unit tests for all features (80%+ coverage)
 3. Integration tests for critical flows
 4. E2E test scenarios
-5. Performance test criteria
-6. Test results summary
+5. Scenario tests (Better Agents format)
+6. Evaluation datasets (if RAG/ML features)
+7. Performance test criteria
+8. Test results summary
 ```
 
 ---
@@ -77,6 +79,8 @@ Testing:
 [ ] Test coverage meets target (e.g., 80%)
 [ ] Integration tests pass
 [ ] At least 3 E2E scenarios pass
+[ ] Scenario tests pass (Better Agents format)
+[ ] Evaluation benchmarks pass (if RAG/ML features)
 [ ] Performance criteria are met
 
 Security:
@@ -99,3 +103,5 @@ Quality:
 |---|---|---|---|
 | Test Plan | `templates/TEST-PLAN.md` | Test Agent | Complete |
 | Security Report | `templates/SECURITY-REVIEW.md` | Security Agent | Signed off |
+| Scenarios | `templates/tests/scenarios/*.test.md` | Test Agent | Pass |
+| Evaluations | `templates/tests/evaluations/*.md` | Test Agent | Pass thresholds |

package/templates/prompts/README.md

@@ -0,0 +1,108 @@
+# Prompts Versioning Index
+
+> Version-controlled prompts for MDAN agents
+
+## Overview
+
+Prompts are versioned using YAML files for better collaboration, tracking, and rollback capabilities.
+
+## Available Prompts
+
+| Prompt | Version | Agent |
+|--------|---------|-------|
+| [orchestrator.yaml](prompts/orchestrator.yaml) | 2.2.0 | MDAN Core |
+| [dev-agent.yaml](prompts/dev-agent.yaml) | 2.0.0 | Haytame (Dev) |
+| [product-agent.yaml](prompts/product-agent.yaml) | 2.0.0 | Khalil (Product) |
+| [architect-agent.yaml](prompts/architect-agent.yaml) | 2.0.0 | Reda (Architect) |
+| [test-agent.yaml](prompts/test-agent.yaml) | 2.0.0 | Youssef (Test) |
+
+## Prompt Format
+
+Each prompt file follows this structure:
+
+```yaml
+handle: agent-handle
+scope: PROJECT  # or GLOBAL
+model: openai/gpt-4o
+version: 1.0.0
+last_updated: "2026-02-24"
+maintainer: username
+
+description: Brief description
+
+system_prompt: |
+  Full prompt content with:
+  - Role definition
+  - Capabilities
+  - Constraints
+  - Output format
+
+capabilities:
+  - Capability 1
+  - Capability 2
+
+constraints:
+  - Constraint 1
+  - Constraint 2
+
+changelog:
+  - version: 1.0.0
+    date: "2026-02-24"
+    changes:
+      - Change 1
+      - Change 2
+```
+
+## Using Prompts
+
+### CLI Commands
+
+```bash
+# List all prompts
+mdan prompt list
+
+# Show specific prompt
+mdan prompt show orchestrator
+
+# Compare versions
+mdan prompt diff orchestrator 2.1.0 2.2.0
+
+# Rollback prompt
+mdan prompt rollback orchestrator 2.1.0
+```
+
+### Integration with IDE
+
+Prompts are synced to:
+- `.claude/skills/` - For Claude/Cursor
+- `.windsurfrules` - For Windsurf
+- `.github/copilot-instructions.md` - For Copilot
+
+## Registry
+
+The `prompts.json` file tracks all prompts:
+
+```json
+{
+  "prompts": {
+    "orchestrator": {
+      "version": "2.2.0",
+      "active": true
+    }
+  }
+}
+```
+
+## Best Practices
+
+1. **Version bump on any change** - Even small tweaks warrant a version bump
+2. **Document changes** - Always add changelog entries
+3. **Test prompts** - Validate prompts before releasing
+4. **Use semantic versioning** - MAJOR for breaking, MINOR for features, PATCH for fixes
+
+## Adding New Prompts
+
+1. Create YAML file in `templates/prompts/`
+2. Add entry to `prompts.json`
+3. Update registry version
+4. Commit with descriptive message

package/templates/prompts/dev-agent.yaml

@@ -0,0 +1,85 @@
+handle: dev-agent
+scope: PROJECT
+model: openai/gpt-4o
+version: 2.0.0
+last_updated: "2026-02-24"
+maintainer: khalilbenaz
+
+description: MDAN Dev Agent (Haytame) - Senior full-stack developer responsible for implementation, code quality, and technical decisions.
+
+system_prompt: |
+  [MDAN-AGENT]
+  NAME: Dev Agent (Haytame)
+  VERSION: 2.0.0
+  ROLE: Senior Full-Stack Developer responsible for implementation
+  PHASE: BUILD
+  REPORTS_TO: MDAN Core
+
+  You are Haytame, a senior full-stack developer with 10+ years of experience.
+  You write clean, maintainable, production-ready code. You care about:
+  - Code readability over cleverness
+  - Testing as a safety net
+  - Security by default
+  - Performance optimization when needed
+
+  Your philosophy:
+  - "Code is read more than written"
+  - "The best code is no code"
+  - "Always consider the next developer"
+  - "Security is not optional"
+
+capabilities:
+  - Implement features from user stories
+  - Write unit, integration, and e2e tests
+  - Create API endpoints
+  - Design database schemas
+  - Set up CI/CD pipelines
+  - Perform code reviews
+  - Refactor existing code
+  - Optimize performance
+  - Fix bugs
+
+constraints:
+  - NEVER skip tests
+  - NEVER commit secrets/keys to repository
+  - NEVER bypass security checks
+  - ALWAYS use type hints (TypeScript/Python)
+  - ALWAYS handle errors explicitly
+  - NEVER expose stack traces to users
+  - ALWAYS use environment variables for config
+
+input_format: |
+  MDAN Core provides:
+  - User stories with acceptance criteria
+  - Architecture document
+  - UX designs/wireframes
+  - Previous implementation context (if any)
+
+output_format: |
+  Produce implementation artifacts:
+  - Implementation Plan (file structure, dependencies, sequence)
+  - Code Files (source, types, configs)
+  - Tests (unit 80%+, integration, e2e)
+  - Setup Instructions (env vars, run locally, migrations)
+
+quality_checklist:
+  - All acceptance criteria implemented
+  - Code follows style guide
+  - No TODO/FIXME in production
+  - All tests pass
+  - No security vulnerabilities
+  - Error handling comprehensive
+  - Logging appropriate
+  - Performance addressed
+
+changelog:
+  - version: 2.0.0
+    date: "2026-02-24"
+    changes:
+      - Added implementation plan format
+      - Added setup instructions section
+      - Added coding standards reference
+  - version: 1.0.0
+    date: "2025-10-01"
+    changes:
+      - Initial release

package/templates/prompts/orchestrator.yaml

@@ -0,0 +1,97 @@
+handle: orchestrator
+scope: PROJECT
+model: openai/gpt-4o
+version: 2.2.0
+last_updated: "2026-02-24"
+maintainer: khalilbenaz
+
+description: |
+  MDAN Core v2 - Adaptive central orchestrator that manages the entire
+  development lifecycle. Handles project profiling, session memory,
+  and agent coordination.
+
+system_prompt: |
+  You are MDAN Core v2, the adaptive central orchestrator of the MDAN method.
+
+  You do everything MDAN Core v1 does, plus two critical upgrades:
+
+  1. PROJECT PROFILE DETECTION — You detect the type of project early in DISCOVER 
+     and automatically adapt the agent workflow, phase depth, and artifact requirements.
+
+  2. SESSION MEMORY — You can resume any project from a MDAN-STATE.json file, 
+     reconstructing full context and picking up exactly where work left off.
+
+  Your responsibilities:
+  - Guide user through 5 phases: DISCOVER, DESIGN, BUILD, VERIFY, SHIP
+  - Activate appropriate agents based on project profile
+  - Maintain session state in MDAN-STATE.json
+  - Ensure quality gates are met before phase transitions
+  - Support slash commands: /mdan-help, /party
+
+  Project profiles you can detect:
+  - MICRO: Solo dev, <1 week, single feature
+  - STANDARD: Small team, 2-8 weeks (default)
+  - PRODUCT: Team 5+, multi-month, stakeholders
+  - ENTERPRISE: Large team, compliance, SLA
+  - API-ONLY: No frontend, developer-focused
+  - DATA/ML: Machine learning, data pipelines
+
+constraints: |
+  - Never skip quality gates
+  - Always ask for user name in first interaction
+  - Use agent names: Khalil (Product), Reda (Architect), Jihane (UX),
+    Haytame (Dev), Youssef (Test), Said (Security), Anas (DevOps), Amina (Doc)
+  - Log all agent activations with version numbers
+
+output_format: |
+  Always include these in your responses:
+  - Current phase indicator
+  - Next recommended action
+  - Quality gate status
+
+examples:
+  - scenario: Start new project
+    user: "I want to build a SaaS app"
+    response: |
+      [MDAN CORE — PROJECT PROFILE DETECTED]
+      
+      Based on your description, I'm classifying this as a STANDARD project.
+      
+      This means:
+      - Full 5-phase workflow
+      - All agents will be activated
+      - Standard artifact depth
+      
+      Does this feel right? Also, before we proceed, what is your name?
+
+  - scenario: Resume session
+    user: "Here's my MDAN-STATE.json"
+    response: |
+      [MDAN CORE — SESSION RESUMED]
+      
+      📁 Project: my-saas (STANDARD)
+      📍 Current Phase: DESIGN
+      ...
+      
+      → Recommended next action: Complete Architecture document
+      
+      Shall I continue?
+
+changelog:
+  - version: 2.2.0
+    date: "2026-02-24"
+    changes:
+      - Added project profile detection
+      - Added session resume protocol
+      - Added agent version tracking
+      - Added quality gates for each profile
+  - version: 2.1.0
+    date: "2025-12-01"
+    changes:
+      - Added /party command
+      - Improved /mdan-help
+  - version: 2.0.0
+    date: "2025-10-01"
+    changes:
+      - Initial v2 release
+      - 5-phase workflow

package/templates/prompts.json

@@ -0,0 +1,81 @@
+{
+  "registry_version": "1.0",
+  "last_updated": "2026-02-24",
+  "prompts": {
+    "orchestrator": {
+      "handle": "orchestrator",
+      "version": "2.2.0",
+      "file": "templates/prompts/orchestrator.yaml",
+      "active": true,
+      "model": "openai/gpt-4o",
+      "changelog": [
+        {
+          "version": "2.2.0",
+          "date": "2026-02-24",
+          "changes": ["Added project profile detection", "Added session resume protocol"]
+        }
+      ]
+    },
+    "dev-agent": {
+      "handle": "dev-agent",
+      "version": "2.0.0",
+      "file": "templates/prompts/dev-agent.yaml",
+      "active": true,
+      "model": "openai/gpt-4o",
+      "changelog": [
+        {
+          "version": "2.0.0",
+          "date": "2026-02-24",
+          "changes": ["Added implementation plan format", "Added setup instructions"]
+        }
+      ]
+    },
+    "product-agent": {
+      "handle": "product-agent",
+      "version": "2.0.0",
+      "file": "templates/prompts/product-agent.yaml",
+      "active": true,
+      "model": "openai/gpt-4o",
+      "changelog": [
+        {
+          "version": "2.0.0",
+          "date": "2026-02-24",
+          "changes": ["Initial release with MoSCoW, risk matrix"]
+        }
+      ]
+    },
+    "architect-agent": {
+      "handle": "architect-agent",
+      "version": "2.0.0",
+      "file": "templates/prompts/architect-agent.yaml",
+      "active": true,
+      "model": "openai/gpt-4o",
+      "changelog": [
+        {
+          "version": "2.0.0",
+          "date": "2026-02-24",
+          "changes": ["Added Mermaid diagrams", "Added ADR format"]
+        }
+      ]
+    },
+    "test-agent": {
+      "handle": "test-agent",
+      "version": "2.0.0",
+      "file": "templates/prompts/test-agent.yaml",
+      "active": true,
+      "model": "openai/gpt-4o",
+      "changelog": [
+        {
+          "version": "2.0.0",
+          "date": "2026-02-24",
+          "changes": ["Added scenarios support", "Added evaluations"]
+        }
+      ]
+    }
+  },
+  "sync_settings": {
+    "auto_sync": false,
+    "langwatch_integration": false,
+    "commit_on_change": true
+  }
+}

package/templates/tests/evaluations/README.md

@@ -0,0 +1,80 @@
+# Test Evaluations Index
+
+> Structured benchmarking for agent components
+
+## Overview
+
+Evaluations provide quantitative testing for specific components of your agent pipeline. Unlike scenarios (end-to-end), evaluations test isolated components.
+
+## Available Evaluations
+
+| Evaluation | Description | Metrics |
+|------------|-------------|---------|
+| [rag_eval.md](rag_eval.md) | RAG correctness | F1, Precision, Recall, Context Relevance |
+| [classification_eval.md](classification_eval.md) | Routing/categorization | Accuracy, F1, Precision, Recall |
+
+## Adding New Evaluations
+
+1. Copy an existing template:
+   ```bash
+   cp rag_eval.md my_eval.md
+   ```
+
+2. Customize:
+   - Update dataset format
+   - Set target thresholds
+   - Add domain-specific checks
+
+3. Run:
+   ```python
+   import langwatch
+   results = langwatch.evaluate(
+       dataset="my-dataset",
+       evaluator="my_eval",
+       metrics=["accuracy", "f1"]
+   )
+   ```
+
+## Built-in Evaluators
+
+LangWatch provides extensive evaluators:
+
+| Evaluator | Description |
+|-----------|-------------|
+| `rag_correctness` | Retrieval and generation quality |
+| `classification_accuracy` | Routing and categorization |
+| `answer_correctness` | Factual accuracy |
+| `safety_check` | Jailbreak, PII, toxicity |
+| `format_validation` | JSON, XML, markdown structure |
+| `tool_calling` | Correct tool selection and args |
+| `latency` | Response time benchmarking |
+
+## Running Evaluations
+
+```bash
+# Python
+pytest tests/evaluations/ -v
+
+# JavaScript
+npm test -- tests/evaluations/
+
+# With LangWatch
+langwatch evaluate --dataset my-project
+```
+
+## Integration with MDAN
+
+During VERIFY phase:
+1. Test Agent identifies components needing evaluation
+2. Creates evaluation datasets from PRD/user stories
+3. Runs relevant evaluations
+4. Reports metrics in quality gate
+5. Fails if thresholds not met
+
+## Best Practices
+
+- Create evaluation datasets from real user queries
+- Include edge cases and negative examples
+- Set realistic thresholds (not 100%)
+- Track metrics over time (regression detection)
+- Run evaluations in CI/CD pipeline

package/templates/tests/evaluations/classification_eval.md

@@ -0,0 +1,136 @@
+# Classification Evaluation Template
+
+> Benchmark routing accuracy and categorization correctness
+
+## Metadata
+
+| Field | Value |
+|-------|-------|
+| eval_name | classification_accuracy |
+| version | 1.0.0 |
+| metrics | Accuracy, Precision, Recall, F1 Score |
+
+## Purpose
+
+Evaluate how well the agent correctly routes requests, categorizes inputs, or makes routing decisions.
+
+## Use Cases
+
+- Intent classification (chatbot routing)
+- Content moderation
+- Priority/ticket categorization
+- Language detection
+- Sentiment analysis
+
+## Dataset Format
+
+```json
+[
+  {
+    "input": "I want to get a refund",
+    "expected_category": "refund_request",
+    "expected_confidence": 0.90
+  },
+  {
+    "input": "How do I change my password?",
+    "expected_category": "account_settings",
+    "expected_confidence": 0.85
+  }
+]
+```
+
+## Evaluation Metrics
+
+| Metric | Target | Description |
+|--------|--------|-------------|
+| Accuracy | ≥0.95 | % of correct classifications |
+| Macro F1 | ≥0.90 | F1 averaged across all categories |
+| Precision (per class) | ≥0.85 | True positives / predicted positives |
+| Recall (per class) | ≥0.85 | True positives / actual positives |
+| Confidence Calibration | ≤0.10 | Mean absolute error between confidence and accuracy |
+
+## Evaluation Code
+
+### Python (LangWatch)
+
+```python
+import langwatch
+
+results = langwatch.evaluate(
+    dataset="intent-classification",
+    evaluator="classification_accuracy",
+    metrics=["accuracy", "precision", "recall", "f1_macro", "calibration"]
+)
+
+print(f"Accuracy: {results.accuracy}")
+print(f"Macro F1: {results.f1_macro}")
+print(f"Precision: {results.precision}")
+print(f"Recall: {results.recall}")
+```
+
+### JavaScript/TypeScript
+
+```typescript
+import { evaluate } from "@langwatch/evaluators";
+
+const results = await evaluate({
+  dataset: "intent-classification",
+  evaluator: "classification_accuracy",
+  metrics: ["accuracy", "precision", "recall", "f1_macro"],
+});
+
+console.log(`Accuracy: ${results.accuracy}`);
+```
+
+## Pass/Fail Criteria
+
+| Metric | Threshold | Status |
+|--------|-----------|--------|
+| Accuracy | ≥0.95 | ✅ Pass |
+| Accuracy | 0.90-0.94 | ⚠️ Warning |
+| Accuracy | <0.90 | ❌ Fail |
+| Macro F1 | ≥0.90 | ✅ Pass |
+| Macro F1 | <0.85 | ❌ Fail |
+| Confidence Error | ≤0.10 | ✅ Pass |
+| Confidence Error | >0.15 | ⚠️ Warning |
+
+## Per-Class Analysis
+
+Generate confusion matrix:
+
+|  | Predicted A | Predicted B | Predicted C |
+|--|-------------|-------------|-------------|
+| Actual A | 45 | 3 | 2 |
+| Actual B | 5 | 38 | 7 |
+| Actual C | 1 | 4 | 40 |
+
+Identify:
+- **High-confusion pairs**: A↔B need better differentiation
+- **Low-recall classes**: More training data needed
+- **Low-precision classes**: Overlapping with other categories
+
+## Common Issues
+
+### Low Precision (many false positives)
+- Add negative examples
+- Make categories more distinct
+- Add disambiguation prompts
+
+### Low Recall (many false negatives)
+- Add more training data
+- Expand category definitions
+- Check for data quality issues
+
+### Poor Calibration
+- Retrain with temperature scaling
+- Add more diverse examples
+- Use calibration-aware loss
+
+## Integration with MDAN
+
+During VERIFY phase, Test Agent should:
+1. Identify all classification/routing points in the system
+2. Create evaluation datasets from real user queries
+3. Run classification evaluation
+4. Report per-class performance
+5. Fail if overall accuracy < 90%