specweave 0.22.13 → 0.22.14

package/plugins/specweave/commands/specweave-import-docs.md

@@ -1,136 +1,326 @@
 ---
 name: specweave:import-docs
-description: Import brownfield documentation from Notion exports, Confluence, GitHub Wiki, or any markdown folder. Automatically classifies files as specs, modules, team docs, or legacy.
+description: Intelligently import ANY documentation (Notion, Confluence, Evernote, Google Docs, markdown) with AI-powered content analysis and automatic living docs integration.
 ---
 
-# Import Brownfield Documentation
+# Import Documentation (AI-Powered)
 
-Import existing documentation from Notion exports, Confluence, GitHub Wiki, or any markdown folder.
+**One command. Any source. Intelligent classification.**
+
+Import documentation from **anywhere** - Notion, Confluence, Evernote, Google Docs, or plain markdown. SpecWeave uses AI to understand content, classify documents, and merge intelligently into living docs.
+
+## Philosophy
+
+**You shouldn't need to tell SpecWeave where docs came from.** Just point it at a folder of markdown files, and SpecWeave figures out:
+- What type of document it is (spec, architecture, guide)
+- Where it should go in living docs
+- How to merge with existing content
+- What needs manual review
 
 ## What This Does
 
-1. **Analyzes markdown files** in source directory (recursively)
-2. **Classifies files** based on content:
-   - **Specs** - Contains "user story", "acceptance criteria", "feature"
-   - **Modules** - Contains "module", "component", "architecture"
-   - **Team** - Contains "onboarding", "convention", "workflow"
-   - **Legacy** - Everything else (no strong match)
-3. **Copies files** to appropriate destinations
-4. **Creates migration report** with classification details
-5. **Updates config** with import history
+### 1. **Smart Source Detection** (Zero Config)
+SpecWeave auto-detects export format:
+- Notion export? → Detects `.csv` metadata, nested folders
+- Confluence? → Detects `index.html`, attachment structure
+- Evernote? → Detects `.enex` format (converts to markdown)
+- Google Docs? → Detects HTML export structure
+- Plain markdown? → Works with any folder
+
+**You don't specify `--source`** - SpecWeave figures it out.
+
+### 2. **AI-Powered Content Classification**
+LLM analyzes ACTUAL content (not just keywords):
+- **Feature Specs** - User stories, acceptance criteria, business requirements
+- **Architecture** - Technical design, system diagrams, ADRs
+- **API Docs** - Endpoints, request/response formats, authentication
+- **User Guides** - How-tos, tutorials, onboarding
+- **Team Process** - Conventions, workflows, PR guidelines
+- **Legacy** - Needs review (ambiguous content)
+
+**Confidence Scoring**: 90%+ = Auto-classify, <90% = Flag for review
+
+### 3. **Intelligent Merging** (Not Overwriting)
+When docs already exist:
+- **Specs**: Merge user stories (preserve existing, add new)
+- **Architecture**: Create versioned files (design-v1.md, design-v2.md)
+- **Guides**: Detect duplicates, suggest consolidation
+- **Legacy**: Never overwrite (safe copy to legacy/)
+
+### 4. **Living Docs Integration**
+Automatically updates:
+- Feature registry with imported specs
+- Architecture index with design docs
+- Cross-reference links between documents
+- Traceability metadata (source, import date)
+
+### 5. **Migration Report**
+Shows what happened:
+- Classification decisions with confidence scores
+- Merge conflicts detected
+- Files that need manual review
+- Suggested next steps
 
 ## Usage
 
+**Simplest form** (SpecWeave does everything):
 ```bash
-/specweave:import-docs <source-path> [options]
+/specweave:import-docs /path/to/docs
 ```
 
-### Options
+That's it! No flags, no config, no source type. Just point and import.
+
+### Options (All Optional)
 
-- `--source=<type>` - Source type: `notion`, `confluence`, `wiki`, `custom` (required)
-- `--project=<id>` - Target project ID (default: active project)
-- `--preserve-structure` - Preserve original folder structure
-- `--dry-run` - Preview classification without importing
+- `--project=<id>` - Target project (default: active project from config)
+- `--dry-run` - Preview without making changes
+- `--confidence=<threshold>` - Min confidence for auto-classification (default: 90)
+- `--merge-strategy=<smart|skip|overwrite>` - How to handle conflicts (default: smart)
 
 ## Examples
 
-### Example 1: Notion Export
+### Example 1: Simplest Import (Zero Config)
 
 ```bash
-# Export Notion workspace to /tmp/notion-export/
-# Then import:
-
-/specweave:import-docs /tmp/notion-export/ --source=notion
-
-# Result:
-# 📊 Analysis Results:
-#    Total files: 47
-#    - Specs: 12 files → specs/
-#    - Modules: 18 files → modules/
-#    - Team docs: 5 files → team/
-#    - Legacy: 12 files → legacy/notion/
-# ✅ Import complete!
+# Just downloaded Notion export to /tmp/notion-export/
+# No need to specify it's from Notion!
+
+/specweave:import-docs /tmp/notion-export
+
+# Output:
+# 🔍 Analyzing /tmp/notion-export...
+#    ✓ Detected source: Notion export (found Database.csv)
+#    ✓ Found 47 markdown files
+#
+# 🤖 AI Classification (using Claude)...
+#    ✓ Analyzed 47 files
+#
+# 📊 Classification Results:
+#    Feature Specs: 12 files (avg confidence: 95%)
+#      • Product Requirements.md → FS-048
+#      • User Stories.md → FS-049
+#      ...
+#
+#    Architecture: 18 files (avg confidence: 92%)
+#      • System Design.md → architecture/design/
+#      • API Architecture.md → architecture/api/
+#      ...
+#
+#    User Guides: 5 files (avg confidence: 88%)
+#      • Getting Started.md → guides/
+#      ...
+#
+#    Needs Review: 12 files (confidence <90%)
+#      • Meeting Notes.md → legacy/ (68% confidence)
+#      ...
+#
+# 🔗 Living Docs Integration...
+#    ✓ Created 12 feature specs (FS-048 through FS-059)
+#    ✓ Updated architecture index
+#    ✓ Created 23 cross-reference links
+#
+# ✅ Import complete! 35/47 files auto-classified (74%)
+#    💡 Review 12 files in legacy/ for manual classification
 ```
 
-### Example 2: Confluence Export
+### Example 2: Dry Run (Preview)
 
 ```bash
-/specweave:import-docs /path/to/confluence/ --source=confluence --project=web-app
+# Preview what would happen WITHOUT making changes
+/specweave:import-docs /path/to/docs --dry-run
+
+# Output shows:
+# - Source detection results
+# - AI classification decisions
+# - Where files would go
+# - Merge conflicts that would occur
+# - NO actual changes made
+```
+
+### Example 3: Confluence Export (Auto-Detected)
 
-# Imports to: projects/web-app/specs/, modules/, team/, legacy/confluence/
+```bash
+# Exported Confluence space to /path/to/confluence/
+# SpecWeave auto-detects it's Confluence!
+
+/specweave:import-docs /path/to/confluence
+
+# Output:
+# 🔍 Analyzing /path/to/confluence...
+#    ✓ Detected source: Confluence export (found index.html)
+#    ✓ Converting HTML to markdown (38 files)...
+#    ✓ Found 38 markdown files
+#
+# [Rest of AI classification...]
 ```
 
-### Example 3: Dry Run (Preview)
+### Example 4: Evernote Export (Auto-Converts)
 
 ```bash
-/specweave:import-docs /tmp/docs/ --source=custom --dry-run
+# Exported Evernote notebook to /tmp/evernote.enex
+# SpecWeave auto-detects and converts!
+
+/specweave:import-docs /tmp/evernote.enex
+
+# Output:
+# 🔍 Analyzing /tmp/evernote.enex...
+#    ✓ Detected source: Evernote export (.enex format)
+#    ✓ Converting ENEX to markdown (142 notes)...
+#    ✓ Extracted 142 markdown files
+#
+# [Rest of AI classification...]
+```
 
-# Shows classification without importing files
-# Use this to preview results before actual import
+### Example 5: Intelligent Merging
+
+```bash
+# You already have FS-048 in living docs
+# SpecWeave merges intelligently (doesn't overwrite!)
+
+/specweave:import-docs /tmp/updated-requirements
+
+# Output:
+# 🔍 Analyzing /tmp/updated-requirements...
+#    ✓ Detected source: Plain markdown folder
+#
+# 🤖 AI Classification...
+#    ✓ Feature Requirements.md → FS-048 (ALREADY EXISTS)
+#
+# 🔀 Intelligent Merging...
+#    FS-048: User Authentication
+#      Existing: 3 user stories (US-001, US-002, US-003)
+#      Imported: 5 user stories (US-001, US-002, US-004, US-005, US-006)
+#
+#      Merge strategy:
+#      ✓ US-001: Keep existing (no changes)
+#      ✓ US-002: Merge (updated acceptance criteria)
+#      ✓ US-003: Keep existing (not in import)
+#      ✓ US-004: Add new (from import)
+#      ✓ US-005: Add new (from import)
+#      ✓ US-006: Add new (from import)
+#
+#      Result: 6 user stories total (3 preserved, 1 merged, 3 new)
+#
+# ✅ Smart merge complete! No data lost.
 ```
 
-### Example 4: Preserve Structure
+### Example 6: Lower Confidence Threshold
 
 ```bash
-/specweave:import-docs /path/to/wiki/ --source=wiki --preserve-structure
+# Accept files with 75%+ confidence (vs default 90%)
+# More aggressive auto-classification
 
-# Preserves original folder structure:
-# legacy/wiki/engineering/backend/auth.md
-# legacy/wiki/engineering/frontend/components.md
+/specweave:import-docs /tmp/messy-docs --confidence=75
+
+# Output:
+# [More files auto-classified, fewer in "Needs Review"]
 ```
 
-## Supported Sources
+## Supported Sources (All Auto-Detected)
+
+SpecWeave automatically detects the source format. You never need to specify `--source`.
 
 ### Notion
-- **Export Format**: Markdown & CSV
-- **Steps**:
-  1. In Notion: Settings → Export → Export all workspace content
-  2. Choose "Markdown & CSV" format
-  3. Download ZIP file
-  4. Extract to folder (e.g., `/tmp/notion-export/`)
-  5. Run import command
+**Auto-Detection**: Finds `Database.csv` or Notion-specific folder structure
+**Export Steps**:
+  1. Notion → Settings → Export → Export all workspace content
+  2. Choose "Markdown & CSV"
+  3. Extract ZIP to folder
+  4. Run: `/specweave:import-docs /path/to/export`
 
 ### Confluence
-- **Export Format**: HTML or Markdown
-- **Steps**:
+**Auto-Detection**: Finds `index.html` or Confluence metadata
+**Export Steps**:
   1. Space tools → Content Tools → Export
   2. Choose HTML or Markdown
-  3. Extract exported files
-  4. Run import command
+  3. Extract files
+  4. Run: `/specweave:import-docs /path/to/export`
+
+### Evernote
+**Auto-Detection**: Finds `.enex` file format
+**Export Steps**:
+  1. Evernote → File → Export → Export as ENEX
+  2. Save .enex file
+  3. Run: `/specweave:import-docs /path/to/export.enex`
+
+**Note**: SpecWeave auto-converts ENEX to markdown
+
+### Google Docs
+**Auto-Detection**: Finds HTML export structure
+**Export Steps**:
+  1. File → Download → Web Page (.html)
+  2. Extract ZIP
+  3. Run: `/specweave:import-docs /path/to/export`
 
 ### GitHub Wiki
-- **Export Format**: Git repository
-- **Steps**:
-  1. Clone wiki: `git clone https://github.com/user/repo.wiki.git`
-  2. Run import command on cloned directory
-
-### Custom (Any Markdown)
-- Any folder containing `.md` or `.markdown` files
-- Recursive search through subdirectories
-
-## Classification Algorithm
-
-Files are classified using keyword analysis:
-
-### Specs (70%+ confidence)
-- Keywords: "user story", "acceptance criteria", "feature", "requirement"
-- Patterns: "As a [user], I want [goal]", "Given-When-Then"
-- Example: Feature specs, PRDs, user stories
-
-### Modules (70%+ confidence)
-- Keywords: "module", "component", "service", "architecture"
-- Patterns: API docs, technical design, integration points
-- Example: Auth module, payment processing, ML pipeline
-
-### Team (70%+ confidence)
-- Keywords: "onboarding", "convention", "workflow", "pr process"
-- Patterns: Team processes, coding standards, deployment guides
-- Example: Onboarding guide, code review checklist
-
-### Legacy (<70% confidence)
-- No strong match to above categories
-- Uncertain classification
-- Requires manual review
+**Auto-Detection**: Finds `.git` folder with wiki structure
+**Export Steps**:
+  1. Clone: `git clone https://github.com/user/repo.wiki.git`
+  2. Run: `/specweave:import-docs ./repo.wiki`
+
+### Plain Markdown
+**Auto-Detection**: Finds `.md` or `.markdown` files
+**No export needed** - just point at any folder
+
+## AI Classification (How It Works)
+
+SpecWeave uses Claude to analyze **actual content**, not just keywords.
+
+### What Claude Analyzes
+
+For each file, Claude examines:
+1. **Document Structure**: Headings, lists, tables, code blocks
+2. **Content Patterns**: User stories, acceptance criteria, technical terms
+3. **Writing Style**: Business vs technical language
+4. **Relationships**: References to other documents
+5. **Intent**: What problem is this document solving?
+
+### Classification Categories
+
+**Feature Specs** (90%+ confidence required)
+- **Indicators**: User stories, acceptance criteria, business requirements
+- **Example Content**: "As a user, I want...", "Given-When-Then", "AC-001"
+- **Living Docs Destination**: `.specweave/docs/internal/specs/`
+
+**Architecture Docs** (90%+ confidence)
+- **Indicators**: System design, technical decisions, diagrams
+- **Example Content**: "Architecture", "Component diagram", "Database schema"
+- **Living Docs Destination**: `.specweave/docs/internal/architecture/`
+
+**API Documentation** (90%+ confidence)
+- **Indicators**: Endpoints, request/response, authentication
+- **Example Content**: "POST /api/users", "Authorization: Bearer", "HTTP 200"
+- **Living Docs Destination**: `.specweave/docs/internal/architecture/api/`
+
+**User Guides** (90%+ confidence)
+- **Indicators**: Step-by-step instructions, tutorials, screenshots
+- **Example Content**: "Getting Started", "How to...", "Step 1:"
+- **Living Docs Destination**: `.specweave/docs/public/guides/`
+
+**Team Process** (90%+ confidence)
+- **Indicators**: Conventions, workflows, team agreements
+- **Example Content**: "PR process", "Code review checklist", "Onboarding"
+- **Living Docs Destination**: `.specweave/docs/internal/team/`
+
+**ADRs** (95%+ confidence - strict!)
+- **Indicators**: "Architecture Decision Record", "ADR-XXXX", "Status: Accepted"
+- **Example Content**: "## Context", "## Decision", "## Consequences"
+- **Living Docs Destination**: `.specweave/docs/internal/architecture/adr/`
+
+**Needs Manual Review** (<90% confidence)
+- **Indicators**: Ambiguous content, mixed types, meeting notes
+- **Example Content**: Anything Claude isn't confident about
+- **Living Docs Destination**: `.specweave/docs/internal/legacy/needs-review/`
+
+### Confidence Scoring
+
+Claude provides a confidence score (0-100%) for each classification:
+
+- **95-100%**: Extremely confident (rare, only for very clear docs like ADRs)
+- **90-94%**: High confidence (default threshold for auto-classification)
+- **80-89%**: Medium confidence (flagged for review, not auto-classified)
+- **70-79%**: Low confidence (definitely needs review)
+- **<70%**: Very uncertain (goes to legacy/)
 
 ## Destination Folders
 

package/plugins/specweave/commands/specweave-progress.md

@@ -17,120 +17,68 @@ Simple, fast progress check for all active increments.
 
 ```bash
 #!/bin/bash
+#
+# Enhanced progress tracking with User Story grouping (v0.23.0+)
+# Uses TypeScript script for accurate task parsing and US-level progress
+#
 
-echo ""
-echo "📊 Increment Progress"
-echo "================================"
-echo ""
-
-# Counters
-active_count=0
-other_count=0
-
-# Scan all increments
-for dir in .specweave/increments/*/; do
-  [ ! -d "$dir" ] && continue
-
-  increment=$(basename "$dir")
-  metadata="$dir/metadata.json"
-
-  # Skip if no metadata
-  [ ! -f "$metadata" ] && continue
-
-  # Get status
-  inc_status=$(jq -r '.status' "$metadata" 2>/dev/null)
-
-  # Skip completed/archived
-  [ "$inc_status" = "completed" ] && continue
-  [ "$inc_status" = "archived" ] && continue
-
-  # Count for summary
-  if [ "$inc_status" = "in-progress" ]; then
-    active_count=$((active_count + 1))
-  else
-    other_count=$((other_count + 1))
-  fi
-
-  # Get task stats from tasks.md
-  tasks_file="$dir/tasks.md"
-  if [ -f "$tasks_file" ]; then
-    # Count tasks (headers with T-NNN format - both ### and ####)
-    total=$(grep -cE '^#{3,4}\s*T-[0-9]' "$tasks_file" 2>/dev/null | tr -d '\n' || echo "0")
-    # Count completed (various formats)
-    completed=$(grep -cE '(✅ COMPLETE|\[COMPLETED\]|\[x\] Completed)' "$tasks_file" 2>/dev/null | tr -d '\n' || echo "0")
-
-    # Ensure we have valid numbers (fallback to 0 if empty)
-    total=${total:-0}
-    completed=${completed:-0}
-
-    if [ "$total" -gt 0 ] 2>/dev/null; then
-      percent=$((completed * 100 / total))
-    else
-      percent=0
-    fi
-  else
-    total=0
-    completed=0
-    percent=0
-  fi
-
-  # Display based on status
-  if [ "$inc_status" = "in-progress" ]; then
-    echo "🟢 ACTIVE: $increment"
-    echo "   Status: $inc_status"
-    echo "   Tasks: $completed/$total completed ($percent%)"
-    echo "   Next: /specweave:do $increment"
-    echo ""
-  else
-    echo "⏸️  $inc_status: $increment"
-    echo "   Tasks: $completed/$total ($percent%)"
-    echo ""
-  fi
-done
-
-echo "================================"
-echo "Summary:"
-echo "  Active increments: $active_count"
-echo "  Other non-completed: $other_count"
-
-if [ "$active_count" -eq 0 ]; then
-  echo ""
-  echo "💡 No active work. Run /specweave:increment to start new work"
-elif [ "$active_count" -gt 0 ]; then
-  echo ""
-  echo "💡 Continue with /specweave:do"
-fi
-
-echo ""
+# Call TypeScript progress script
+npx tsx "$(dirname "${BASH_SOURCE[0]}")"/../../../scripts/show-progress.ts "$@"
 ```
 
 ## Example Output
 
+### Legacy Format (no User Stories)
 ```
 📊 Increment Progress
-================================
+============================================================
 
 🟢 ACTIVE: 0037-project-specific-tasks
-   Status: in-progress
-   Tasks: 72/85 completed (84%)
-   Next: /specweave:do 0037-project-specific-tasks
+   ████████████████████████░░░░░░ 84% (72/85 tasks)
 
-⏸️  planning: 0039-ultra-smart-next-command
-   Tasks: 0/45 (0%)
+   Next: /specweave:do 0037-project-specific-tasks
 
-================================
+============================================================
 Summary:
   Active increments: 1
-  Other non-completed: 1
+  Other non-completed: 0
 
 💡 Continue with /specweave:do
 ```
 
+### Enhanced Format (with User Story grouping)
+```
+📊 Increment Progress
+============================================================
+
+⏸️  ACTIVE: 0047-us-task-linkage
+   ██████████████████░░░░░░░░░░░░ 59% (13/22 tasks)
+
+   Progress by User Story:
+   ✅ US-001: ████████████████████ 100% (4/4)
+   ✅ US-002: ████████████████████ 100% (3/3)
+   ├─ US-003: ████████████░░░░░░░░ 60% (3/5)
+   ✅ US-004: ████████████████████ 100% (3/3)
+   ├─ US-005: ░░░░░░░░░░░░░░░░░░░░ 0% (0/4)
+   ├─ US-006: ░░░░░░░░░░░░░░░░░░░░ 0% (0/3)
+
+   Resume: /specweave:resume 0047-us-task-linkage
+
+============================================================
+Summary:
+  Active increments: 0
+  Other non-completed: 1
+
+💡 No active work. Run /specweave:increment to start new work
+```
+
 ## What It Shows
 
-- **Active increments** (in-progress): Shown first with green indicator
-- **Other non-completed**: planning, paused, blocked, etc.
-- **Task completion**: X/Y completed (Z%)
-- **Next action**: Which command to run
+- **Overall progress**: Visual bar + percentage + task count
+- **Per-User Story progress** (if US linkage exists): Completion status for each US
+- **Progress bars**: Color-coded (green ≥80%, yellow 50-79%, red <50%)
+- **Completion indicators**: ✅ for 100% complete USs
+- **Orphan tasks warning**: If tasks exist without User Story linkage
+- **Next action**: Command to continue work
 
-**Note**: Skips completed and archived increments.
+**Note**: Skips completed and archived increments. Automatically detects and displays US-level progress for increments using US-task linkage (v0.23.0+).