@thedecipherist/mdd 1.5.1 → 1.5.3

package/commands/mdd-import-spec.md

@@ -2,7 +2,7 @@
 
 Triggered when arguments start with `import-spec`.
 
-Reads one or more large spec or prompt documents — the kind produced by extended brainstorming sessions with Claude — and converts them into properly structured MDD feature docs. Every decision in the spec is preserved. Duplicate or overlapping topics are merged intelligently. Path grouping determines whether to create initiative/waves or flat docs.
+Reads one or more large spec or prompt documents — the kind produced by extended brainstorming sessions with Claude — and converts them into properly structured MDD initiatives, waves, and feature docs. Every decision in the spec is preserved. Duplicate or overlapping topics are merged intelligently. Features are numbered and waved in **build dependency order**, not spec-reading order.
 
 ---
 
@@ -39,25 +39,29 @@ Reading <filename> (<N> lines)...
 Full file read. Proceeding to feature extraction.
 ```
 
-If multiple files are provided, merge all content into a single working document after all files are fully read. Tag each section internally with its source filename (e.g. `<!-- source: rawpg-prompt-driver.md -->`) for traceability — these tags are used in the merge summary and content mapping display but are never written to output docs.
+If multiple files are provided, merge all content into a single working document after all files are fully read. Tag each section internally with its source filename (e.g. `<!-- source: spec.md -->`) for traceability — these tags are used in the merge summary and content mapping display but are never written to output docs.
 
 ---
 
-### Phase IS2 — Feature Extraction + Path Grouping
+### Phase IS2 — Feature Extraction, Classification + Path Grouping
 
-This phase runs in two steps. Path grouping always runs before complexity determination because the number of distinct top-level path areas is the primary signal for initiative-scale scope.
+This phase runs in four steps. Build-order classification (IS2c) always precedes structure determination (IS2d) because the correct numbering depends on understanding what you build vs. what you reference.
 
-#### Step IS2a — Extract features
+#### Step IS2a — Extract features + record source line ranges
 
-Read the entire merged spec. Identify every distinct feature, system, subsystem, or bounded capability described. A "feature" is any topic that could become a standalone MDD doc — something with a purpose, decisions, constraints, and a clear scope.
+Read the entire merged working document. Identify every distinct feature, system, subsystem, or bounded capability described. A "feature" is any topic that could become a standalone MDD doc — something with a purpose, decisions, constraints, and a clear scope.
 
-For each identified feature:
+For each identified feature, record:
 - Name / title
-- Core purpose (1–2 sentences distilled from the spec)
-- All key decisions, constraints, business rules, and edge cases mentioned in the spec
+- **The exact line ranges in the source file(s) that cover this feature.** Record every range, including secondary ranges where the feature is discussed again (e.g. a directive mentioned in the syntax section AND in the security section AND in the changelog). Format: `lines 254–340, 2401–2450, 4725–4741`
+- A brief extraction note: what types of content are in those ranges (e.g. "syntax section: options table, processing pipeline, scope model; security section: filesystem confinement rules; changelog: file resolution model decisions")
 - Dependencies on other features identified in the same spec
 
-Track which spec sections contribute to each feature. When multiple spec sections cover the same concept (same feature re-discussed with refinements), **merge them** — do not create duplicate docs. When versions of the same decision conflict, keep the most specific or most recent version and note the discarded variant.
+This line-range map is used in IS4 to re-read the source before writing each doc. It is the most important output of IS2a. Do not skip or estimate ranges — check your chunk boundaries if unsure.
+
+Track which spec sections contribute to each feature. When multiple spec sections cover the same concept (same feature re-discussed with refinements), **merge them** — do not create duplicate docs. Record all contributing ranges. When versions of the same decision conflict, keep the most specific or most recent version and note the discarded variant.
+
+**Changelog and review-pass sections** (sections titled "v1.0 Review", "Changelog", "Decision Log", or similar retrospective formats) are not standalone features — they are design decision records. For each decision in such a section, identify which feature it belongs to and add its line range to that feature's source ranges. Do not create a standalone "Changelog" feature doc.
 
 #### Step IS2b — Assign paths
 
@@ -68,7 +72,47 @@ For each identified feature, determine its `path` value:
 - Title Case, 1–3 levels, `/`-separated
 - Siblings must use identical parent spelling (if `Auth/Login` exists, new auth docs use `Auth`, not `Authentication`)
 
-#### Step IS2c — Determine output structure
+#### Step IS2c — Build-order classification
+
+This is the most important step. Features are numbered in the order a developer would actually build them — not the order they appear in the spec.
+
+**Classify every feature as one of two types:**
+
+**COMPONENT** — Something that results in code: a module, class, package, service, server, or runtime. The thing you sit down and write. Examples: Parser, Template Engine, Renderer, MCP Server, CLI.
+
+**SPEC** — Something that describes behaviour a COMPONENT must implement: a directive definition, an API contract, a security rule, a caching rule, a language feature. You consult a SPEC doc while building the COMPONENT that implements it. Examples: `@include` directive spec, Security Config spec, Cache Modes spec.
+
+**Order COMPONENT features by build dependency:**
+
+Start from the foundation (the component everything else depends on) and work outward. Ask: "What must exist before I can build this?" The component with no dependencies comes first. The component that depends on everything comes last.
+
+Example for a language toolchain:
+```
+Parser         — foundation, no deps, must exist first
+Stripper       — depends on Parser AST
+Renderer       — depends on Parser AST, no connection deps
+Template Engine — depends on Parser + Renderer
+MCP Server     — depends on Template Engine
+Hook           — depends on MCP Server
+CLI            — depends on all of the above
+```
+
+**Assign SPEC features to waves:** A SPEC feature belongs in the wave of the COMPONENT that implements it. The SPEC doc gets created alongside its implementing COMPONENT's wave so that both exist when building begins. The COMPONENT doc lists SPEC docs in its `depends_on`.
+
+**Determine wave breakdown:** Group COMPONENTs (and their associated SPECs) into waves by build phase. Each wave should have a clear demo-state — a thing you can actually demonstrate when the wave is done.
+
+Example wave structure for a language toolchain:
+```
+Wave 1 — Foundation:    Parser + all language directive SPECs
+Wave 2 — Static Output: Stripper + Renderer
+Wave 3 — Engine:        Template Engine + Caching SPECs
+Wave 4 — Live Data:     MCP Server + Hook + Security SPECs
+Wave 5 — CLI:           CLI + Distribution
+```
+
+**If no COMPONENT/SPEC distinction applies** (e.g. a feature-extension spec for an existing product), order features by: user-facing value first, infrastructure last. Earlier features should be shippable without later features.
+
+#### Step IS2d — Determine output structure
 
 Count distinct root-level path segments (top-level areas):
 
@@ -78,7 +122,64 @@ Count distinct root-level path segments (top-level areas):
 | 1–2 root areas AND 4–7 features | Waves + Feature docs (no initiative wrapper) |
 | Any root area count AND 1–3 features | Flat feature docs only |
 
-These thresholds are guidelines — apply judgment. A 3-feature spec spanning radically different domains can still warrant waves.
+These thresholds are guidelines — apply judgment.
+
+---
+
+### Phase IS2.5 — CLAUDE.md Check
+
+Before showing the dry-run preview, check the project's CLAUDE.md.
+
+Run: `[ -f CLAUDE.md ] && wc -l CLAUDE.md || echo "missing"`
+
+**If CLAUDE.md is missing or under 10 lines:**
+
+Ask the user via AskUserQuestion:
+
+> "This looks like a new project. Want me to create a CLAUDE.md with a description of what `<product name>` is, its architecture, and how the MDD docs map to the codebase? This gives Claude the context it needs to build from the docs correctly."
+
+Options:
+- **"Yes, create CLAUDE.md"** — proceed to draft it from the spec, include in IS4 write step
+- **"No, skip"** — continue without creating CLAUDE.md
+
+**If CLAUDE.md already exists and is substantial:** skip this check entirely.
+
+**CLAUDE.md content to draft (if approved):**
+
+Derive from the spec:
+
+```markdown
+# CLAUDE.md
+
+## What This Project Is
+
+<Product name> — <one paragraph elevator pitch. What does it do, who is it for, what problem does it solve. Write this in plain terms, not marketing language.>
+
+## Core Philosophy
+
+<The fundamental design principles that drive all decisions. Extract from the spec's philosophy/principles sections. These are the "why" behind architecture choices.>
+
+## Architecture Overview
+
+<The major components and how they relate. For a toolchain: list each component, one line description, what it depends on. For an API: list major subsystems. Make it concrete.>
+
+## Tech Stack
+
+<Language, frameworks, package manager, test runner — derive from spec if stated, leave blank if not.>
+
+## What the MDD Docs Represent
+
+This project uses MDD (Manual-Driven Development). The `.mdd/` directory contains:
+- `.mdd/initiatives/` — the top-level initiative(s) defining the full scope
+- `.mdd/waves/` — waves within each initiative, each with a concrete demo-state
+- `.mdd/docs/` — individual feature docs, one per buildable unit or behavioural spec
+
+**The docs are numbered in build dependency order.** `/mdd 01` is always the first thing to build. Read the wave file before starting a wave to understand what "done" looks like.
+
+## Key Constraints
+
+<Any hard rules derived from the spec: immutable rules, security constraints, "never do X", platform requirements. These are the lines that cannot be crossed.>
+```
 
 ---
 
@@ -91,16 +192,23 @@ Before writing any files, display the complete proposed structure and wait for e
 
 Source: <filename(s)>
 Features identified: <N>   Merged: <N> duplicate/overlapping topics
+CLAUDE.md: <"will be created" | "already exists, skipping" | "skipping (user declined)">
+
+Proposed structure: Initiative "<name>" → <N> Waves → <N> Feature docs
 
-Proposed structure: <Flat docs | Waves | Initiative: "<name>" → Waves>
+Initiative: .mdd/initiatives/<slug>.md
 
-Path Tree:
-  <Root Area 1>
-    ├── <Section>              → <NN>-<slug>    (draft)
-    └── <Section>
-         └── <Sub-section>    → <NN>-<slug>    (draft)
-  <Root Area 2>
-    └── <Section>              → <NN>-<slug>    (draft)
+Waves (in build order):
+  Wave 1 — <name>   (.mdd/waves/<slug>-wave-1.md)
+    Demo-state: <what you can demonstrate when this wave is done>
+    Features:
+      <NN> <slug>   [COMPONENT]   <path>
+      <NN> <slug>   [SPEC]        <path>
+  Wave 2 — <name>   (.mdd/waves/<slug>-wave-2.md)
+    Demo-state: <demo-state>
+    Features:
+      <NN> <slug>   [COMPONENT]   <path>
+      ...
 
 IDs assigned: <NN>–<NN> (continuing from existing <prev>)
 
@@ -112,10 +220,11 @@ Content mapping:
   <NN>-<slug>:   §<Spec Section> + §<Spec Section>
   <NN>-<slug>:   §<Spec Section>
 
-Adjust paths, titles, or grouping? (approve / adjust / abort)
+Is the build order correct? Does each wave's demo-state make sense?
+(approve / adjust / abort)
 ```
 
-**If the user says "adjust":** accept their description of changes, re-run IS2 with that feedback applied, then show the preview again. Repeat until the user approves.
+**If the user says "adjust":** accept their description of changes — reordering waves, reclassifying features, changing demo-states, renaming — re-run IS2 with that feedback applied, then show the preview again. Repeat until the user approves.
 
 **If the user says "abort":** stop. Write nothing.
 
@@ -125,22 +234,131 @@ Adjust paths, titles, or grouping? (approve / adjust / abort)
 
 ### Phase IS4 — Write Files
 
-**If initiative-scale was detected:** Create `.mdd/initiative.md` with:
-- `id`, `title`, `status: planning`, `created: <today>`
-- A brief initiative description derived from the spec's overall theme
-- Wave breakdown: one wave per major root path area or logical phase, each listing its feature doc slugs
+Write in this order: CLAUDE.md (if approved) → initiative → waves → feature docs.
+
+#### CLAUDE.md (if approved in IS2.5)
+
+Write the drafted CLAUDE.md to the project root. Report: `✅ CLAUDE.md created`
+
+#### Initiative file
+
+Create `.mdd/initiatives/<slug>.md`:
+
+```markdown
+---
+id: <slug>
+title: <Full Initiative Title>
+status: active
+version: 1
+hash:
+created: <today YYYY-MM-DD>
+---
+
+# <Full Initiative Title>
+
+## Overview
+
+<Comprehensive description of what this initiative delivers. This should be 3-6 paragraphs:
+- What the product/system is and what it does
+- Why it exists and what problem it solves
+- The core design philosophy and principles that drive all decisions
+- The major components or areas being built
+- What "done" looks like for the full initiative
+Do NOT write "brief" — write enough that a developer reading only this file understands what they are building and why.>
+
+## Open Product Questions
+(none — imported from spec)
+
+## Waves
+| Wave | File | Demo-state | Status |
+|------|------|------------|--------|
+<one row per wave>
+| Wave 1 | waves/<slug>-wave-1.md | <demo-state> | planned |
+| Wave 2 | waves/<slug>-wave-2.md | <demo-state> | planned |
+```
+
+Compute and write the `hash:` field after writing (hash of file content excluding the hash line).
+
+#### Wave files
+
+For each wave, create `.mdd/waves/<slug>-wave-N.md`:
+
+```markdown
+---
+id: <slug>-wave-N
+title: "Wave N: <Wave Title>"
+initiative: <initiative-slug>
+initiative_version: 1
+status: planned
+depends_on: <none | slug of previous wave>
+demo_state: "<concrete thing you can demonstrate when this wave is complete>"
+created: <today YYYY-MM-DD>
+hash:
+---
+
+# Wave N: <Wave Title>
+
+## Demo-State
+
+<demo-state>
+*(This wave is not complete until this can be manually demonstrated.)*
+
+## Features
+
+| # | Feature | Doc | Type | Status | Depends on |
+|---|---------|-----|------|--------|------------|
+| 1 | <slug> | docs/<NN>-<slug>.md | COMPONENT | planned | — |
+| 2 | <slug> | docs/<NN>-<slug>.md | SPEC | planned | <dep or —> |
+
+## Open Research
+
+(none — imported from spec)
+```
+
+Compute and write the `hash:` field after writing.
+
+#### Feature docs
 
-**For each feature doc in the approved plan:**
+For each feature in the approved plan, in wave order:
 
 1. Auto-number continuing from the highest existing doc number in `.mdd/docs/`
-2. Write a complete MDD feature doc at `.mdd/docs/<NN>-<slug>.md` using the canonical frontmatter structure:
+
+2. **Re-read the source before writing.** Look up the line ranges recorded for this feature in IS2a. Re-read each range from the original spec file now — do not write from the IS2a extraction summary alone. The summary identified what exists; the re-read provides the actual content. Use the same chunked read strategy as IS1 if a range is large.
+
+   ```
+   Re-reading source for <slug>...
+     lines <X>–<Y>  (<section name>)  ✓
+     lines <X>–<Y>  (<section name>)  ✓
+   Writing doc...
+   ```
+
+3. **Run the completeness checklist against the freshly-read source before writing the doc.** For each item present in the source, it must appear in the doc:
+
+   - [ ] Every options / parameters table — every row, every column, every default value
+   - [ ] Every CLI subcommand and its flags (including rare/advanced flags)
+   - [ ] Every config JSON structure — exact keys, types, nesting, defaults
+   - [ ] Every TypeScript interface and type alias defined in the spec
+   - [ ] Every AST node type and its fields
+   - [ ] Every error message format and the exact condition that triggers it
+   - [ ] Every behavioral table (evaluation tables, state tables, format tables, platform tables)
+   - [ ] Every "always" / "never" / "must not" / "only valid when" rule
+   - [ ] Every example that illustrates an edge case or non-obvious behaviour
+   - [ ] Every named distinction where two similar things behave differently (e.g. `column` singular vs `columns` plural)
+
+   If any item is present in the source but not yet captured in your notes, add it now before writing the doc. Do not skip this step.
+
+4. Write `.mdd/docs/<NN>-<slug>.md`:
 
 ```markdown
 ---
 id: <NN>-<slug>
 title: <Feature Title>
+type: <COMPONENT | SPEC>
+initiative: <initiative-slug>
+wave: <wave-slug>
+wave_status: planned
 edition: <project name or "Both">
-depends_on: [<IDs of other imported features this one depends on>]
+depends_on: [<IDs of docs this one depends on>]
 source_files: []
 routes: []
 models: []
@@ -159,27 +377,27 @@ known_issues: []
 
 ## Purpose
 
-<2–3 sentences from the merged spec content>
+<Full description of what this feature is and does. Include the core design rationale — the "why" behind the decisions. Do not limit this to a sentence count. If the spec says a lot about this feature, capture it all here. A developer reading only this doc should fully understand what they are building.>
 
 ## Architecture
 
-<How this feature fits into the system, derived from spec decisions>
+<How this feature fits into the system. For a COMPONENT: describe its responsibilities, inputs, outputs, and what it must never do. For a SPEC: describe the behaviour contract the implementing COMPONENT must satisfy. Include TypeScript interfaces, AST node types, and data structures if the spec defines them.>
 
 ## Data Model
 
-<If the spec describes data structures — omit section if truly not applicable>
+<Data structures, schemas, config formats described in the spec. Include the exact field names, types, and constraints. Omit section only if the spec truly defines no data structures for this feature.>
 
-## API Endpoints
+## API / Interface
 
-<If the spec describes endpoints — omit section if truly not applicable>
+<Public interface this feature exposes: function signatures, tool names, command syntax, config keys. Omit if not applicable.>
 
 ## Business Rules
 
-<All decisions, constraints, validation rules, edge cases from the spec>
+<Every decision, constraint, validation rule, edge case, error behaviour, and "never do X" described in the spec for this feature. This section should be exhaustive — if it is in the spec, it is here. Use bullet points or numbered lists for clarity.>
 
 ## Dependencies
 
-<Other features this one requires, by doc ID>
+<Other feature docs this one requires. List by ID and title. For SPECs, name the COMPONENT that implements this spec.>
 
 ## Known Issues
 
@@ -187,30 +405,35 @@ known_issues: []
 ```
 
 3. Tags: 4–8 domain-concept keywords. NOT file paths or spec section names.
-4. `depends_on`: if feature A was described in the spec as depending on feature B (also imported), use the IDs assigned in this run.
+4. `depends_on`: populate from the build order analysis. COMPONENT docs list the SPEC docs they implement. SPEC docs list other SPECs they depend on (e.g. the expression system spec is depended on by the filter spec).
 
 **Progress report as you write:**
 ```
-Writing docs...
-  ✅ <NN>-<slug>.md   (<path>)
-  ✅ <NN>-<slug>.md   (<path>)
+Writing files...
+  ✅ CLAUDE.md
+  ✅ initiatives/markdownai.md
+  ✅ waves/markdownai-wave-1.md  (Wave 1 — Foundation)
+  ✅ waves/markdownai-wave-2.md  (Wave 2 — Static Pipeline)
+  ...
+  ✅ docs/01-<slug>.md   [COMPONENT]  <path>
+  ✅ docs/02-<slug>.md   [SPEC]       <path>
   ...
 ```
 
 ---
 
-### Phase IS5 — Rebuild .startup.md
+### Phase IS5 — Rebuild .startup.md + Connections
 
-Trigger the `.mdd/.startup.md` rebuild:
-- Rebuild the auto-generated zone (Project Snapshot, Features Documented list with IDs, status, and tags)
+Rebuild `.mdd/.startup.md`:
+- Rebuild the auto-generated zone (Project Snapshot, Features Documented list with IDs, status, and tags; Ops Runbooks)
+- Add initiative and wave summary to the Features section: show initiative title, each wave with status, feature count
 - Preserve the Notes zone exactly as-is
 - Update the generated date and current branch
 
-Then regenerate connections.md:
+Then regenerate `.mdd/connections.md`:
 
-**Regenerate `.mdd/connections.md`:**
-Read all `.mdd/docs/*.md` (excluding `archive/`) — frontmatter only (id, title, status, path, depends_on, source_files). Never read doc bodies. Then:
-- **Path tree:** sort docs by path alphabetically, then by id within the same path. Render as indented tree using `├──` / `└──` characters. Each leaf: `<path-leaf-segment>  <id>  <status>`.
+Read all `.mdd/docs/*.md` (excluding `archive/`) — frontmatter only (id, title, type, status, path, depends_on, wave, source_files). Never read doc bodies. Then:
+- **Path tree:** sort docs by path alphabetically, then by id within the same path. Render as indented tree using `├──` / `└──` characters. Each leaf: `<path-leaf-segment>  <id>  <type>  <status>`.
 - **Mermaid graph:** one node per doc (short node ID), one `-->` edge per `depends_on` entry, `:::<status>` suffix on each node. Include `classDef complete fill:#00e5cc,color:#000`, `in_progress fill:#ffaa00,color:#000`, `draft fill:#888,color:#fff`, `deprecated fill:#555,color:#aaa`.
 - **Source overlap:** build map of source_file → docs that reference it. Include only files with 2+ docs.
 - **Warnings:** broken `depends_on` refs (target doesn't exist), circular dependencies, docs missing `path`.
@@ -221,21 +444,23 @@ Then report:
 ```
 ✅ Import Spec Complete
 
-Source:    <filename(s)>
-Created:   <N> feature docs
-Structure: <Flat | Waves | Initiative + Waves>
+Source:      <filename(s)>
+CLAUDE.md:   <created | skipped>
+Initiative:  .mdd/initiatives/<slug>.md
+Waves:       <N> wave files created
+Docs:        <N> feature docs created  (<N> COMPONENT, <N> SPEC)
 
-Docs created:
-  <NN>-<slug>   <path>   draft
-  <NN>-<slug>   <path>   draft
-  ...
+Structure:
+  <initiative title>
+    Wave 1 — <name>   (<N> features)
+    Wave 2 — <name>   (<N> features)
+    ...
 
-Startup:   .mdd/.startup.md rebuilt
+Startup:     .mdd/.startup.md rebuilt
 Connections: .mdd/connections.md updated
 
 Next steps:
-  /mdd <NN>         — start building any imported feature
-  /mdd audit        — run a full audit across all imported docs
-  /mdd upgrade      — add path fields to any pre-existing docs that are missing them
-  /mdd rebuild-tags — regenerate tags if any look thin
+  /mdd plan-execute <slug>-wave-1   — start building Wave 1
+  /mdd audit                        — run a full audit across all imported docs
+  /mdd <NN>                         — jump directly to any feature doc
 ```

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@thedecipherist/mdd",
-  "version": "1.5.1",
+  "version": "1.5.3",
   "description": "MDD — Manual-Driven Development workflow for Claude Code",
   "type": "module",
   "bin": {