specfact-cli 0.4.2__py3-none-any.whl → 0.6.8__py3-none-any.whl

specfact_cli/resources/prompts/specfact-plan-compare.md

@@ -0,0 +1,571 @@
+---
+description: Compare manual and auto-derived plans to detect deviations and inconsistencies.
+---
+# SpecFact Compare Plan Command
+
+## User Input
+
+```text
+$ARGUMENTS
+```
+
+You **MUST** consider the user input before proceeding (if not empty).
+
+## ⚠️ CRITICAL: CLI Usage Enforcement
+
+**YOU MUST ALWAYS USE THE SPECFACT CLI**. Never create artifacts directly or implement functionality.
+
+### Rules
+
+1. **ALWAYS execute CLI first**: Run `specfact plan compare` before any comparison - execute the CLI command before any other operations
+2. **NEVER write code**: Do not implement comparison logic - the CLI handles this
+3. **NEVER create YAML/JSON directly**: All comparison reports must be CLI-generated
+4. **NEVER bypass CLI validation**: CLI ensures schema compliance and metadata - use it, don't bypass its validation
+5. **Use CLI output as grounding**: Parse CLI output, don't regenerate or recreate it - use the CLI output as the source of truth
+6. **NEVER read artifacts directly**: Do NOT read plan bundle files directly to extract information. Use CLI commands to get plan information. The CLI provides all necessary data through its output.
+7. **No internal knowledge required**: You should NOT need to know about internal implementation details (PlanBundle model, Deviation class, etc.). All operations must be performed via CLI commands.
+
+### What Happens If You Don't Follow This
+
+- ❌ Artifacts may not match CLI schema versions
+- ❌ Missing metadata and telemetry
+- ❌ Format inconsistencies
+- ❌ Validation failures
+- ❌ Works only in Copilot mode, fails in CI/CD
+- ❌ Breaks when CLI internals change
+- ❌ Requires knowledge of internal code structure
+- ❌ Out-of-sync information if bundle files are read directly
+
+## ⏸️ Wait States: User Input Required
+
+**When user input is required, you MUST wait for the user's response.**
+
+### Wait State Rules
+
+1. **Never assume**: If input is missing, ask and wait
+2. **Never continue**: Do not proceed until user responds
+3. **Be explicit**: Clearly state what information you need
+4. **Provide options**: Give examples or default suggestions
+
+## Goal
+
+Compare a manual plan bundle with an auto-derived plan bundle to detect deviations, mismatches, and missing features. This command helps identify gaps between planned features and actual implementation, ensuring alignment between specification and code.
+
+**Note**: This is a **read-only comparison** operation - it generates comparison reports but does not modify plan bundles.
+
+## Action Required
+
+### Use CLI to Resolve Plan Numbers/Names
+
+**⚠️ CRITICAL**: If user provides plan numbers (e.g., "19 vs 20") or plan names, you MUST use the CLI to resolve them to actual file paths. NEVER search for files directly.
+
+### Step 1: Resolve Plan Paths Using CLI
+
+**If user input contains plan numbers** (e.g., "19 vs 20", "compare 1 and 2"):
+
+1. **List available plans using CLI**:
+
+   ```bash
+   specfact plan select
+   ```
+
+   - Parse the CLI table output to get plan names for the specified numbers
+   - Extract the full plan file names from the table
+
+2. **Get full plan paths using CLI**:
+
+   ```bash
+   specfact plan select <plan_number>
+   ```
+
+   - This will output the full plan name/path
+   - Use this to construct the full path: `.specfact/plans/<plan_name>`
+
+**If user input contains plan names** (e.g., "main.bundle.yaml vs auto-derived.bundle.yaml"):
+
+- Use the plan names directly (may need to add `.bundle.yaml` suffix if missing)
+- Verify paths exist by attempting to use them with the CLI
+
+**If arguments provided as paths**: Use them directly.
+
+**If arguments missing**: Ask user interactively for each missing argument and **WAIT for their response**:
+
+1. **Manual plan path**: "Which manual plan to compare? (Enter plan number, plan name, or path. Default: .specfact/plans/main.bundle.yaml)"
+   - **[WAIT FOR USER RESPONSE - DO NOT CONTINUE]**
+
+2. **Auto plan path**: "Which auto-derived plan to compare? (Enter plan number, plan name, or path. Default: latest in .specfact/plans/)"
+   - **[WAIT FOR USER RESPONSE - DO NOT CONTINUE]**
+
+3. **Output format**: "Output format? (1) Markdown, (2) JSON, (3) YAML (default: markdown)"
+   - **[WAIT FOR USER RESPONSE - DO NOT CONTINUE]**
+
+4. **Output file**: "Save report to file? (optional, default: auto-generated with timestamp)"
+   - **[WAIT FOR USER RESPONSE - DO NOT CONTINUE]**
+
+**Only execute CLI after** resolving plan paths and getting necessary information from user.
+
+## Operating Constraints
+
+**STRICTLY READ-ONLY**: Do **not** modify the codebase. All comparison reports must be generated by the specfact CLI.
+
+**Mode Auto-Detection**: The CLI automatically detects operational mode (CI/CD or CoPilot) based on environment. No need to specify `--mode` flag. Mode is detected from:
+
+- Environment variables (`SPECFACT_MODE`)
+- CoPilot API availability
+- IDE integration (VS Code/Cursor with CoPilot)
+- Defaults to CI/CD mode if none detected
+
+## Command
+
+```bash
+specfact plan compare [--manual PATH] [--auto PATH] [--format {markdown|json|yaml}] [--out PATH]
+```
+
+**Note**: Mode is auto-detected by the CLI. No need to specify `--mode` flag.
+
+**CRITICAL**: Always execute this CLI command. Never create comparison reports directly.
+
+## Quick Reference
+
+**Arguments:**
+
+- `--manual PATH` - Manual plan bundle path (default: `.specfact/plans/main.bundle.yaml`) - **ASK USER if default not found**
+- `--auto PATH` - Auto-derived plan bundle path (default: latest in `.specfact/reports/brownfield/`) - **ASK USER if default not found**
+- `--format {markdown|json|yaml}` - Output format (default: `markdown`) - **ASK USER if not specified**
+- `--out PATH` - Output file path (optional, default: auto-generated in `.specfact/reports/comparison/`)
+
+**What it does:**
+
+1. Loads and validates both plan bundles (manual and auto-derived)
+2. Compares features and stories between plans
+3. Detects deviations (missing features, mismatches, etc.)
+4. Assigns severity (HIGH, MEDIUM, LOW) to each deviation
+5. Generates structured comparison report (Markdown, JSON, or YAML)
+6. Displays summary in console with deviation counts and severity breakdown
+7. Optionally applies enforcement rules (if `.specfact/enforcement.yaml` exists)
+
+## Interactive Flow
+
+### Use CLI to Resolve Plan Paths
+
+**⚠️ CRITICAL**: **NEVER search for files directly**. Always use CLI commands to resolve plan numbers/names to file paths.
+
+**Step 1**: Parse user input to identify plan selections.
+
+- **If user input contains plan numbers** (e.g., "19 vs 20", "compare 1 and 2"):
+  1. **Execute CLI to list plans**:
+
+     ```bash
+     specfact plan select
+     ```
+
+  2. **Parse CLI table output** to extract plan names for the specified numbers
+  3. **Get full plan paths** by executing:
+
+     ```bash
+     specfact plan select <plan_number>
+     ```
+
+     - Parse the CLI output to get the full plan name
+     - Construct full path: `.specfact/plans/<plan_name>`
+
+- **If user input contains plan names** (e.g., "main.bundle.yaml vs auto-derived.bundle.yaml"):
+  - Use plan names directly (may need to add `.bundle.yaml` suffix if missing)
+  - Construct full path: `.specfact/plans/<plan_name>`
+
+- **If user input contains full paths**: Use them directly
+
+**Step 2**: Resolve manual plan path.
+
+- **If plan number/name provided**: Use CLI to resolve (see Step 1)
+- **If missing**: Check if default path (`.specfact/plans/main.bundle.yaml`) exists using CLI
+  - **Verify using CLI**: Attempt to use the path with `specfact plan compare` - if it fails, the file doesn't exist
+  - **If not exists**: Ask user and **WAIT**:
+
+    ```text
+    "Manual plan not found at default location. Enter plan number, plan name, or path to manual plan bundle, 
+    or create one with `specfact plan init --interactive`?
+    [WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
+    ```
+
+**Step 3**: Resolve auto plan path.
+
+- **If plan number/name provided**: Use CLI to resolve (see Step 1)
+- **If missing**: Use CLI to find latest auto-derived plan
+  - **Execute CLI to list plans**:
+
+    ```bash
+    specfact plan select
+    ```
+
+  - **Parse CLI output** to find latest auto-derived plan (by modification date)
+  - **If found**: Ask user and **WAIT**:
+
+    ```text
+    "Use latest auto-derived plan: [PLAN_NAME]? (y/n, or enter different plan number/name/path)
+    [WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
+    ```
+
+  - **If not found**: Ask user and **WAIT**:
+
+    ```text
+    "No auto-derived plans found. Enter plan number, plan name, or path to auto-derived plan bundle, 
+    or generate one with `specfact import from-code --repo . --name my-project`?
+    [WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
+    ```
+
+**Step 3**: Check if `--format` is specified.
+
+- **If missing**: Ask user and **WAIT**:
+
+  ```text
+  "Output format? (1) Markdown, (2) JSON, (3) YAML (default: markdown)
+  [WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
+  ```
+
+- **If provided**: Use specified format
+
+**Step 4**: Check if `--out` is specified.
+
+- **If missing**: Ask user and **WAIT**:
+
+  ```text
+  "Save report to file? (y/n, default: auto-generated in .specfact/reports/comparison/)
+  [WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
+  ```
+
+  - **If yes**: Generate default path with timestamp and format extension
+  - **If no**: Skip file output (display in console only)
+- **If provided**: Use specified path
+
+**Step 5**: Confirm execution.
+
+- Show summary and **WAIT**:
+
+  ```text
+  "Will compare [MANUAL_PATH] vs [AUTO_PATH] and save report as [OUT_PATH] in [FORMAT] format. 
+  Continue? (y/n)
+  [WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
+  ```
+
+- **If yes**: Execute CLI command
+- **If no**: Cancel or ask for changes
+
+**Step 6**: Execute CLI command with confirmed arguments.
+
+**⚠️ CRITICAL**: Use the resolved file paths (not plan numbers) in the CLI command:
+
+```bash
+specfact plan compare --manual <MANUAL_PATH> --auto <AUTO_PATH> --format <FORMAT> --out <OUT_PATH>
+```
+
+**Example**: If user said "19 vs 20", and CLI resolved them to:
+
+- Plan 19: `specfact-import-test-v2.2025-11-17T13-53-31.bundle.yaml`
+- Plan 20: `specfact-import-test-v2.2025-11-17T13-53-31.enriched.2025-11-17T13-55-40.bundle.yaml`
+
+Then execute:
+
+```bash
+specfact plan compare --manual .specfact/plans/specfact-import-test-v2.2025-11-17T13-53-31.bundle.yaml --auto .specfact/plans/specfact-import-test-v2.2025-11-17T13-53-31.enriched.2025-11-17T13-55-40.bundle.yaml
+```
+
+**Capture CLI output**:
+
+- Comparison report path (if `--out` specified)
+- Deviation counts and severity breakdown
+- Console output with summary
+- Any error messages or warnings
+
+## Expected Output
+
+**Console output (with deviations):**
+
+```bash
+SpecFact CLI - Plan Comparator
+
+Manual Plan: .specfact/plans/main.bundle.yaml
+Auto Plan: .specfact/reports/brownfield/auto-derived-2025-11-02T12-00-00.bundle.yaml
+Total Deviations: 15
+
+Comparison Results
+
+Manual Plan: .specfact/plans/main.bundle.yaml
+Auto Plan: .specfact/reports/brownfield/auto-derived-2025-11-02T12-00-00.bundle.yaml
+Total Deviations: 15
+
+Deviation Summary:
+  🔴 HIGH: 3
+  🟡 MEDIUM: 8
+  🔵 LOW: 4
+
+Deviations by Type and Severity
+┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
+┃ Severity  ┃ Type              ┃ Description                        ┃ Location       ┃
+┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
+│ 🔴 HIGH   │ Missing Feature   │ FEATURE-005 exists in auto but not │ features[4]    │
+│           │                   │ in manual                           │                │
+│ 🔴 HIGH   │ Critical Mismatch │ STORY-003 acceptance criteria differ│ features[0].   │
+│           │                   │ significantly                       │ stories[2]     │
+└───────────┴───────────────────┴────────────────────────────────────┴───────────────┘
+
+✓ Report written to: .specfact/reports/comparison/deviations-2025-11-02T12-00-00.md
+```
+
+**Console output (no deviations):**
+
+```bash
+No deviations found! Plans are identical.
+```
+
+**Console output (enforcement blocked):**
+
+```bash
+Enforcement Rules
+Using enforcement config: .specfact/enforcement.yaml
+
+🚫 [HIGH] missing_feature: BLOCK
+❌ Enforcement BLOCKED: 1 deviation(s) violate quality gates
+Fix the blocking deviations or adjust enforcement config
+```
+
+## Execution Steps
+
+### 1. Execute CLI Command (REQUIRED)
+
+**ALWAYS execute the specfact CLI** to perform the comparison:
+
+```bash
+specfact plan compare --manual <manual_path> --auto <auto_path> --format <format> --out <output_path>
+```
+
+**The CLI performs**:
+
+- Plan bundle loading and validation
+- Feature and story comparison
+- Deviation detection and severity assignment
+- Report generation (Markdown, JSON, or YAML)
+- Enforcement rule checking (if `.specfact/enforcement.yaml` exists)
+
+**Capture CLI output**:
+
+- Comparison report path (if `--out` specified)
+- Deviation counts (HIGH, MEDIUM, LOW)
+- Console summary
+- Any error messages or warnings
+
+**If CLI execution fails**:
+
+- Report the error to the user
+- Do not attempt to create comparison reports manually
+- Suggest fixes based on error message
+
+### 2. Present Results
+
+**Present the CLI-generated comparison report** to the user:
+
+- **Report location**: Show where the CLI wrote the report file (if `--out` specified)
+- **Deviation summary**: Show counts by severity (HIGH, MEDIUM, LOW)
+- **Key deviations**: Highlight critical deviations from CLI output
+- **Next steps**: Suggest actions based on deviations found
+
+### 3. Validate Plan Bundles (Reference - CLI Handles This)
+
+Load and validate both plan bundles:
+
+- **Schema validation**: Use Pydantic models to validate structure
+- **Required fields**: Check all required fields are present
+- **Data types**: Validate data types match schema
+- **Report errors**: If validation fails, report error and exit
+
+### 4. Compare Plans
+
+Perform comprehensive comparison:
+
+#### 4.1 Feature Comparison
+
+For each feature in both plans:
+
+- **Feature matching**: Match features by key (exact match) or title (fuzzy match)
+- **Feature deviations**:
+  - **Missing in manual**: Features in auto plan but not in manual
+  - **Missing in auto**: Features in manual plan but not in auto
+  - **Title mismatch**: Same key but different titles
+  - **Outcomes mismatch**: Different expected outcomes
+  - **Acceptance criteria mismatch**: Different acceptance criteria
+  - **Confidence mismatch**: Significant confidence difference (> 0.3)
+
+#### 4.2 Story Comparison
+
+For each story in matched features:
+
+- **Story matching**: Match stories by key (exact match) or title (fuzzy match)
+- **Story deviations**:
+  - **Missing in manual**: Stories in auto plan but not in manual
+  - **Missing in auto**: Stories in manual plan but not in auto
+  - **Title mismatch**: Same key but different titles
+  - **Acceptance criteria mismatch**: Different acceptance criteria
+  - **Confidence mismatch**: Significant confidence difference (> 0.3)
+
+#### 4.3 Structure Comparison
+
+- **Feature count**: Compare total feature counts
+- **Story count**: Compare total story counts per feature
+- **Coverage gaps**: Identify areas with no coverage in either plan
+
+### 5. Assign Severity
+
+Classify each deviation by severity:
+
+- **HIGH**:
+  - Missing features/stories in manual plan (potential oversight)
+  - Critical acceptance criteria mismatch
+  - Confidence difference > 0.5
+  - Structural inconsistencies (orphaned stories, duplicate keys)
+
+- **MEDIUM**:
+  - Title mismatches (possible naming inconsistencies)
+  - Acceptance criteria differences (non-critical)
+  - Confidence difference 0.3-0.5
+  - Outcome mismatches
+
+- **LOW**:
+  - Minor title variations
+  - Confidence difference < 0.3
+  - Cosmetic differences (formatting, whitespace)
+  - Missing optional fields
+
+### 6. Generate Comparison Report
+
+Create structured report based on format:
+
+#### Markdown Format
+
+```markdown
+# Plan Comparison Report
+
+**Manual Plan**: `/path/to/manual.bundle.yaml`
+**Auto Plan**: `/path/to/auto.bundle.yaml`
+**Timestamp**: `2025-11-02T12:00:00Z`
+**Total Deviations**: `15`
+
+## Summary
+
+- 🔴 **HIGH**: 3
+- 🟡 **MEDIUM**: 8
+- 🔵 **LOW**: 4
+
+## Deviations
+
+### HIGH Severity
+
+| ID | Type | Description | Location |
+|----|------|-------------|----------|
+| H1 | Missing Feature | FEATURE-005 exists in auto but not in manual | features[4] |
+| H2 | Critical Mismatch | STORY-003 acceptance criteria differ significantly | features[0].stories[2] |
+
+### MEDIUM Severity
+
+...
+
+### LOW Severity
+
+...
+```
+
+#### JSON Format
+
+```json
+{
+  "manual_plan": "/path/to/manual.bundle.yaml",
+  "auto_plan": "/path/to/auto.bundle.yaml",
+  "timestamp": "2025-11-02T12:00:00Z",
+  "total_deviations": 15,
+  "severity_counts": {
+    "HIGH": 3,
+    "MEDIUM": 8,
+    "LOW": 4
+  },
+  "deviations": [
+    {
+      "id": "H1",
+      "severity": "HIGH",
+      "type": "missing_feature",
+      "description": "FEATURE-005 exists in auto but not in manual",
+      "location": "features[4]"
+    }
+  ]
+}
+```
+
+#### YAML Format
+
+Same structure as JSON, in YAML format.
+
+### 7. Output Results
+
+- **Console output**: Display summary with deviation counts and severity breakdown
+- **Table view**: Show detailed deviation table in console with deviations grouped by severity
+- **Comparison report**: Write to specified output path (if `--out` provided)
+
+### 8. Apply Enforcement Rules (Optional)
+
+If enforcement config exists (`.specfact/enforcement.yaml`):
+
+- **Load config**: Parse enforcement configuration
+- **Check blocking**: Identify deviations that should block (based on severity)
+- **Report blocking**: If blocking deviations found, report error and exit with code 1
+- **Report passing**: If no blocking deviations, report success
+
+**Note**: Finding deviations is a successful comparison result. Exit code 0 indicates successful execution (even if deviations were found). Use the report file, stdout, or enforcement config to determine if deviations are critical.
+
+## Output Format
+
+### Deviation Structure
+
+Each deviation includes:
+
+- **ID**: Unique identifier (e.g., "H1", "M5", "L2")
+- **Severity**: HIGH, MEDIUM, or LOW
+- **Type**: Deviation type (e.g., "missing_feature", "title_mismatch")
+- **Description**: Human-readable description
+- **Location**: Path to deviation in plan structure (e.g., "features[0].stories[2]")
+
+### Report Structure
+
+- **Header**: Manual plan path, auto plan path, timestamp
+- **Summary**: Total deviations, severity counts
+- **Deviations by Severity**: Grouped deviations with details
+- **Coverage Gaps**: Features/stories missing in either plan
+- **Recommendations**: Suggestions for resolving deviations
+
+## Guidelines
+
+### Feature Matching
+
+- **Exact match**: Same feature key (preferred)
+- **Fuzzy match**: Similar feature titles (fallback, lower confidence)
+- **No match**: Treat as missing feature
+
+### Story Matching
+
+- **Exact match**: Same story key within same feature (preferred)
+- **Fuzzy match**: Similar story titles within same feature (fallback)
+- **No match**: Treat as missing story
+
+### Severity Assignment
+
+- Use consistent criteria for severity classification
+- Consider business impact when assigning severity
+- Err on side of higher severity for missing features/stories
+
+### Error Handling
+
+- **Missing files**: Report error with helpful suggestions
+- **Invalid format**: Report error and exit
+- **Validation failures**: Report validation errors and exit
+- **Permission errors**: Report error and exit gracefully
+
+## Context
+
+{ARGS}