specfact-cli 0.4.2__py3-none-any.whl → 0.6.8__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (66) hide show
  1. specfact_cli/__init__.py +1 -1
  2. specfact_cli/agents/analyze_agent.py +2 -3
  3. specfact_cli/analyzers/__init__.py +2 -1
  4. specfact_cli/analyzers/ambiguity_scanner.py +601 -0
  5. specfact_cli/analyzers/code_analyzer.py +462 -30
  6. specfact_cli/analyzers/constitution_evidence_extractor.py +491 -0
  7. specfact_cli/analyzers/contract_extractor.py +419 -0
  8. specfact_cli/analyzers/control_flow_analyzer.py +281 -0
  9. specfact_cli/analyzers/requirement_extractor.py +337 -0
  10. specfact_cli/analyzers/test_pattern_extractor.py +330 -0
  11. specfact_cli/cli.py +151 -206
  12. specfact_cli/commands/constitution.py +281 -0
  13. specfact_cli/commands/enforce.py +42 -34
  14. specfact_cli/commands/import_cmd.py +481 -152
  15. specfact_cli/commands/init.py +224 -55
  16. specfact_cli/commands/plan.py +2133 -547
  17. specfact_cli/commands/repro.py +100 -78
  18. specfact_cli/commands/sync.py +701 -186
  19. specfact_cli/enrichers/constitution_enricher.py +765 -0
  20. specfact_cli/enrichers/plan_enricher.py +294 -0
  21. specfact_cli/importers/speckit_converter.py +364 -48
  22. specfact_cli/importers/speckit_scanner.py +65 -0
  23. specfact_cli/models/plan.py +42 -0
  24. specfact_cli/resources/mappings/node-async.yaml +49 -0
  25. specfact_cli/resources/mappings/python-async.yaml +47 -0
  26. specfact_cli/resources/mappings/speckit-default.yaml +82 -0
  27. specfact_cli/resources/prompts/specfact-enforce.md +185 -0
  28. specfact_cli/resources/prompts/specfact-import-from-code.md +626 -0
  29. specfact_cli/resources/prompts/specfact-plan-add-feature.md +188 -0
  30. specfact_cli/resources/prompts/specfact-plan-add-story.md +212 -0
  31. specfact_cli/resources/prompts/specfact-plan-compare.md +571 -0
  32. specfact_cli/resources/prompts/specfact-plan-init.md +531 -0
  33. specfact_cli/resources/prompts/specfact-plan-promote.md +352 -0
  34. specfact_cli/resources/prompts/specfact-plan-review.md +1276 -0
  35. specfact_cli/resources/prompts/specfact-plan-select.md +401 -0
  36. specfact_cli/resources/prompts/specfact-plan-update-feature.md +242 -0
  37. specfact_cli/resources/prompts/specfact-plan-update-idea.md +211 -0
  38. specfact_cli/resources/prompts/specfact-repro.md +268 -0
  39. specfact_cli/resources/prompts/specfact-sync.md +497 -0
  40. specfact_cli/resources/schemas/deviation.schema.json +61 -0
  41. specfact_cli/resources/schemas/plan.schema.json +204 -0
  42. specfact_cli/resources/schemas/protocol.schema.json +53 -0
  43. specfact_cli/resources/templates/github-action.yml.j2 +140 -0
  44. specfact_cli/resources/templates/plan.bundle.yaml.j2 +141 -0
  45. specfact_cli/resources/templates/pr-template.md.j2 +58 -0
  46. specfact_cli/resources/templates/protocol.yaml.j2 +24 -0
  47. specfact_cli/resources/templates/telemetry.yaml.example +35 -0
  48. specfact_cli/sync/__init__.py +10 -1
  49. specfact_cli/sync/watcher.py +268 -0
  50. specfact_cli/telemetry.py +440 -0
  51. specfact_cli/utils/acceptance_criteria.py +127 -0
  52. specfact_cli/utils/enrichment_parser.py +445 -0
  53. specfact_cli/utils/feature_keys.py +12 -3
  54. specfact_cli/utils/ide_setup.py +170 -0
  55. specfact_cli/utils/structure.py +179 -2
  56. specfact_cli/utils/yaml_utils.py +33 -0
  57. specfact_cli/validators/repro_checker.py +22 -1
  58. specfact_cli/validators/schema.py +15 -4
  59. specfact_cli-0.6.8.dist-info/METADATA +456 -0
  60. specfact_cli-0.6.8.dist-info/RECORD +99 -0
  61. {specfact_cli-0.4.2.dist-info → specfact_cli-0.6.8.dist-info}/entry_points.txt +1 -0
  62. specfact_cli-0.6.8.dist-info/licenses/LICENSE.md +202 -0
  63. specfact_cli-0.4.2.dist-info/METADATA +0 -370
  64. specfact_cli-0.4.2.dist-info/RECORD +0 -62
  65. specfact_cli-0.4.2.dist-info/licenses/LICENSE.md +0 -61
  66. {specfact_cli-0.4.2.dist-info → specfact_cli-0.6.8.dist-info}/WHEEL +0 -0
@@ -0,0 +1,1276 @@
1
+ ---
2
+ description: "Review plan bundle to identify and resolve ambiguities, fill gaps, and prepare for promotion"
3
+ ---
4
+
5
+ # SpecFact Review Plan Bundle Command
6
+
7
+ ## User Input
8
+
9
+ ```text
10
+ $ARGUMENTS
11
+ ```
12
+
13
+ You **MUST** consider the user input before proceeding (if not empty).
14
+
15
+ ## ⚠️ CRITICAL: CLI Usage Enforcement
16
+
17
+ **YOU MUST ALWAYS USE THE SPECFACT CLI**. Never create artifacts directly.
18
+
19
+ ### Rules
20
+
21
+ 1. **ALWAYS execute CLI first**: Run `specfact plan review` before any analysis - execute the CLI command before any other operations
22
+ 2. **NEVER write code**: Do not implement review logic - the CLI handles this
23
+ 3. **NEVER create YAML/JSON directly**: All plan bundle updates must be CLI-generated
24
+ 4. **NEVER bypass CLI validation**: CLI ensures schema compliance and metadata - use it, don't bypass its validation
25
+ 5. **Use CLI output as grounding**: Parse CLI output, don't regenerate or recreate it - use the CLI output as the source of truth
26
+ 6. **NEVER manipulate internal code**: Do NOT use Python code to directly modify PlanBundle objects, Feature objects, Clarification objects, or any internal data structures. The CLI is THE interface - use it exclusively.
27
+ 7. **No internal knowledge required**: You should NOT need to know about internal implementation details (PlanBundle model, Feature class, AmbiguityScanner, etc.). All operations must be performed via CLI commands.
28
+ 8. **NEVER read artifacts directly**: Do NOT read plan bundle files directly to extract information unless for display purposes (e.g., showing plan details to user). Use CLI commands (`specfact plan review --list-questions`, `specfact plan select`) to get plan information.
29
+
30
+ ### What Happens If You Don't Follow This
31
+
32
+ - ❌ Artifacts may not match CLI schema versions
33
+ - ❌ Missing metadata and telemetry
34
+ - ❌ Format inconsistencies
35
+ - ❌ Validation failures
36
+ - ❌ Works only in Copilot mode, fails in CI/CD
37
+ - ❌ Breaks when CLI internals change
38
+ - ❌ Requires knowledge of internal code structure
39
+
40
+ ### Available CLI Commands for Plan Updates
41
+
42
+ **For updating idea section (OPTIONAL - business metadata)**:
43
+
44
+ - `specfact plan update-idea --title <title> --narrative <narrative> --target-users <users> --value-hypothesis <hypothesis> --constraints <constraints> --plan <path>`
45
+ - Updates idea section metadata (optional business context, not technical implementation)
46
+ - **Note**: Idea section is OPTIONAL - provides business context and metadata
47
+ - All parameters are optional - use only what you need
48
+ - Works in CI/CD, Copilot, and interactive modes
49
+ - Example: `specfact plan update-idea --target-users "Developers, DevOps" --value-hypothesis "Reduce technical debt" --constraints "Python 3.11+, Maintain backward compatibility"`
50
+
51
+ **For updating features**:
52
+
53
+ - `specfact plan update-feature --key <key> --title <title> --outcomes <outcomes> --acceptance <acceptance> --constraints <constraints> --confidence <confidence> --draft/--no-draft --plan <path>`
54
+ - **Boolean flags**: `--draft` sets True, `--no-draft` sets False, omit to leave unchanged
55
+ - ❌ **WRONG**: `--draft true` or `--draft false` (Typer boolean flags don't accept values)
56
+ - ✅ **CORRECT**: `--draft` (sets True) or `--no-draft` (sets False)
57
+ - Updates existing feature metadata (title, outcomes, acceptance criteria, constraints, confidence, draft status)
58
+ - Works in CI/CD, Copilot, and interactive modes
59
+ - Example: `specfact plan update-feature --key FEATURE-001 --title "New Title" --outcomes "Outcome 1, Outcome 2"`
60
+
61
+ **For adding features**:
62
+
63
+ - `specfact plan add-feature --key <key> --title <title> --outcomes <outcomes> --acceptance <acceptance> --plan <path>`
64
+
65
+ **For adding stories**:
66
+
67
+ - `specfact plan add-story --feature <feature-key> --key <story-key> --title <title> --acceptance <acceptance> --story-points <points> --value-points <points> --plan <path>`
68
+
69
+ **For updating stories**:
70
+
71
+ - `specfact plan update-story --feature <feature-key> --key <story-key> --title <title> --acceptance <acceptance> --story-points <points> --value-points <points> --confidence <confidence> --draft/--no-draft --plan <path>`
72
+ - **Boolean flags**: `--draft` sets True, `--no-draft` sets False, omit to leave unchanged
73
+ - ❌ **WRONG**: `--draft true` or `--draft false` (Typer boolean flags don't accept values)
74
+ - ✅ **CORRECT**: `--draft` (sets True) or `--no-draft` (sets False)
75
+ - Updates existing story metadata (title, acceptance criteria, story points, value points, confidence, draft status)
76
+ - Works in CI/CD, Copilot, and interactive modes
77
+ - Example: `specfact plan update-story --feature FEATURE-001 --key STORY-001 --acceptance "Given X, When Y, Then Z" --story-points 5`
78
+
79
+ **❌ FORBIDDEN**: Direct Python code manipulation like:
80
+
81
+ ```python
82
+ # ❌ NEVER DO THIS:
83
+ from specfact_cli.models.plan import PlanBundle, Feature
84
+ plan_bundle.features[0].title = "New Title" # Direct manipulation
85
+ generator.generate(plan_bundle, plan_path) # Bypassing CLI
86
+ ```
87
+
88
+ **✅ CORRECT**: Use CLI commands:
89
+
90
+ ```bash
91
+ # ✅ ALWAYS DO THIS:
92
+ specfact plan update-feature --key FEATURE-001 --title "New Title" --plan <path>
93
+ ```
94
+
95
+ ## ⏸️ Wait States: User Input Required
96
+
97
+ **When user input is required, you MUST wait for the user's response.**
98
+
99
+ ### Wait State Rules
100
+
101
+ 1. **Never assume**: If input is missing, ask and wait
102
+ 2. **Never continue**: Do not proceed until user responds
103
+ 3. **Be explicit**: Clearly state what information you need
104
+ 4. **Provide options**: Give examples or default suggestions
105
+
106
+ ## Goal
107
+
108
+ Review a plan bundle to identify ambiguities, missing information, and unknowns. Systematically resolve these through targeted questions to make the plan ready for promotion (draft → review → approved).
109
+
110
+ **Note**: This review workflow is expected to run BEFORE promoting from `draft` to `review` stage. If the user explicitly states they are skipping review (e.g., exploratory spike), you may proceed, but must warn that promotion readiness may be incomplete.
111
+
112
+ **Automatic Enrichment Strategy**:
113
+
114
+ The CLI now supports automatic enrichment via `--auto-enrich` flag. Use this when:
115
+
116
+ 1. **User explicitly requests enrichment**: "enrich", "auto-fix", "improve quality", "fix vague criteria"
117
+ 2. **Plan quality indicators suggest it**: Vague acceptance criteria, incomplete requirements, generic tasks detected
118
+ 3. **After Spec-Kit sync**: If user mentions issues from `/speckit.analyze` (vague acceptance criteria, incomplete requirements)
119
+
120
+ **Enrichment Workflow**:
121
+
122
+ 1. **Run with `--auto-enrich`**: Execute `specfact plan review --auto-enrich` to automatically fix common issues
123
+ 2. **Review enrichment results**: Analyze what was enhanced and verify improvements
124
+ 3. **LLM reasoning**: Use your reasoning to:
125
+ - Verify enhancements are contextually appropriate
126
+ - Identify any generic improvements that need refinement
127
+ - Suggest specific manual improvements for edge cases
128
+ 4. **Follow-up enrichment**: If auto-enrichment made generic improvements, use CLI commands to refine them:
129
+ - `specfact plan update-feature` to add specific file paths, method names, or component references to feature-level acceptance criteria
130
+ - `specfact plan update-story` to refine story-level acceptance criteria with specific actions, method calls, and testable assertions
131
+ - `specfact plan update-feature` to add domain-specific constraints
132
+
133
+ **Example Enrichment Flow**:
134
+
135
+ ```bash
136
+ # Step 1: Auto-enrich to fix common issues
137
+ specfact plan review --auto-enrich --plan <plan_path>
138
+
139
+ # Step 2: LLM analyzes results and suggests refinements
140
+ # "Auto-enrichment enhanced 8 acceptance criteria. The Given/When/Then format is good,
141
+ # but we should make the 'When' clause more specific. For example, 'When they interact
142
+ # with the system' could be 'When they call the configure() method with valid parameters'."
143
+
144
+ # Step 3: Manual refinement using CLI commands
145
+ specfact plan update-feature --key FEATURE-001 --acceptance "Given a developer wants to configure Git operations, When they call the configure() method with valid parameters, Then the configuration is validated and stored" --plan <plan_path>
146
+ ```
147
+
148
+ ## Operating Constraints
149
+
150
+ **STRICTLY READ-WRITE**: This command modifies plan bundle metadata and content. All updates must be performed by the specfact CLI.
151
+
152
+ **Command**: `specfact plan review [--auto-enrich]`
153
+
154
+ **Mode Auto-Detection**: The CLI automatically detects operational mode (CI/CD or CoPilot) based on environment. No need to specify `--mode` flag. Mode is detected from:
155
+
156
+ - Environment variables (`SPECFACT_MODE`)
157
+ - CoPilot API availability
158
+ - IDE integration (VS Code/Cursor with CoPilot)
159
+ - Defaults to CI/CD mode if none detected
160
+
161
+ **Mode-Specific Behavior**:
162
+
163
+ - **CLI Mode**: Interactive Q&A with free-text input, simple multiple-choice when applicable
164
+ - **Copilot Mode**: Reasoning-enhanced questions with recommendations, similar to Spec-Kit clarify
165
+
166
+ **Auto-Enrichment Feature**:
167
+
168
+ The `--auto-enrich` flag automatically enhances the plan bundle before scanning for ambiguities:
169
+
170
+ - **Vague acceptance criteria** (e.g., "is implemented", "is functional", "works") → Converted to testable Given/When/Then format
171
+ - **Incomplete requirements** (e.g., "System MUST Helper class") → Enhanced with verbs and actions (e.g., "System MUST provide a Helper class for [feature] operations")
172
+ - **Generic tasks** (e.g., "Implement [story]") → Enhanced with implementation details (file paths, methods, components)
173
+
174
+ **⚠️ IMPORTANT LIMITATION**: Auto-enrichment creates **generic templates** (e.g., "Given a user wants to use {story}, When they interact with the system, Then {story} works correctly"). These are NOT testable and MUST be refined by LLM with code-specific details. The LLM MUST automatically refine all generic criteria after auto-enrichment runs (see "LLM Post-Enrichment Analysis & Automatic Refinement" section below).
175
+
176
+ **When to Use Auto-Enrichment**:
177
+
178
+ - **Before first review**: Use `--auto-enrich` when reviewing a plan bundle imported from code or Spec-Kit to automatically fix common quality issues
179
+ - **After sync from Spec-Kit**: If `/speckit.analyze` reports vague acceptance criteria or incomplete requirements, use `--auto-enrich` to fix them automatically
180
+ - **Before promotion**: Use `--auto-enrich` to improve plan quality before promoting from `draft` to `review` stage
181
+ - **LLM Reasoning**: In Copilot mode, analyze the plan bundle first, then decide if auto-enrichment would be beneficial based on the coverage summary
182
+
183
+ ## What This Command Does
184
+
185
+ The `specfact plan review` command:
186
+
187
+ 1. **Analyzes** the plan bundle for ambiguities using a structured taxonomy
188
+ 2. **Identifies** missing information, unclear requirements, and unknowns
189
+ 3. **Asks** targeted questions (max 5 per session) to resolve ambiguities
190
+ 4. **Integrates** answers back into the plan bundle incrementally
191
+ 5. **Validates** plan bundle structure after each update
192
+ 6. **Reports** coverage summary and promotion readiness
193
+
194
+ ## Execution Steps
195
+
196
+ ### ⚠️ **CRITICAL: Copilot Mode Workflow**
197
+
198
+ In Copilot mode, follow this three-phase workflow:
199
+
200
+ 1. **Phase 1: Get Questions** - Execute `specfact plan review --list-questions` to get questions in JSON format
201
+ 2. **Phase 2: Ask User** - Present questions to user one at a time, collect answers
202
+ 3. **Phase 3: Feed Answers** - Write answers to a JSON file, then execute `specfact plan review --answers answers.json` to integrate answers
203
+
204
+ **⚠️ IMPORTANT**: Always use a JSON file path (not inline JSON string) to avoid parsing issues and ensure proper formatting.
205
+
206
+ **Never create clarifications directly in YAML**. Always use the CLI to integrate answers.
207
+
208
+ ### 1. Parse Arguments and Load Plan Bundle
209
+
210
+ **Parse user input** to extract:
211
+
212
+ - Plan bundle path (default: active plan or latest in `.specfact/plans/`)
213
+ - Max questions per session (default: 5)
214
+ - Category focus (optional, to focus on specific taxonomy category)
215
+ - Auto-enrichment flag (if user requests automatic enrichment or if plan appears to need it)
216
+
217
+ **LLM Reasoning for Auto-Enrichment**:
218
+
219
+ In Copilot mode, you should **reason about whether auto-enrichment is needed**:
220
+
221
+ 1. **Check if plan was imported from code or Spec-Kit**: If the user mentions "after sync" or "imported from code", auto-enrichment is likely beneficial
222
+ 2. **Analyze plan quality indicators**: If you see patterns like:
223
+ - Vague acceptance criteria ("is implemented", "is functional")
224
+ - Incomplete requirements ("System MUST Helper class")
225
+ - Generic tasks without implementation details
226
+ Then suggest using `--auto-enrich`
227
+ 3. **User intent**: If user explicitly requests "enrich", "improve", "fix vague criteria", or mentions issues from `/speckit.analyze`, use `--auto-enrich`
228
+
229
+ **Decision Flow**:
230
+
231
+ ```text
232
+ IF user mentions "after sync" OR "imported" OR "vague criteria" OR "incomplete requirements":
233
+ → Use --auto-enrich flag
234
+ ELSE IF plan appears to have quality issues (based on initial scan):
235
+ → Suggest --auto-enrich to user and wait for confirmation
236
+ ELSE:
237
+ → Proceed with normal review (no auto-enrichment)
238
+ ```
239
+
240
+ **WAIT STATE**: If plan path is unclear, ask the user:
241
+
242
+ ```text
243
+ "Which plan bundle would you like to review?
244
+ (Enter path, 'active plan', or 'latest')
245
+ [WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
246
+ ```
247
+
248
+ **In Copilot Mode**: Use `--list-questions` to get questions in structured format:
249
+
250
+ ```bash
251
+ # Get questions as JSON (for Copilot mode)
252
+ specfact plan review --list-questions --plan <plan_path> --max-questions 5
253
+
254
+ # With auto-enrichment (if needed)
255
+ specfact plan review --auto-enrich --list-questions --plan <plan_path> --max-questions 5
256
+ ```
257
+
258
+ **In CI/CD Mode**: Use `--non-interactive` flag:
259
+
260
+ ```bash
261
+ # Non-interactive mode (for automation)
262
+ specfact plan review --non-interactive --plan <plan_path> --answers '{"Q001": "answer1", "Q002": "answer2"}'
263
+
264
+ # With auto-enrichment
265
+ specfact plan review --auto-enrich --non-interactive --plan <plan_path> --answers '{"Q001": "answer1"}'
266
+ ```
267
+
268
+ **Capture from CLI**:
269
+
270
+ - Plan bundle loaded successfully
271
+ - **Deduplication summary**: "✓ Removed N duplicate features from plan bundle" (if duplicates were found)
272
+ - Current stage (should be `draft` for review)
273
+ - Existing clarifications (if any)
274
+ - **Auto-enrichment summary** (if `--auto-enrich` was used):
275
+ - Features updated
276
+ - Stories updated
277
+ - Acceptance criteria enhanced
278
+ - Requirements enhanced
279
+ - Tasks enhanced
280
+ - List of changes made
281
+ - Questions list (if `--list-questions` used)
282
+ - **Coverage Summary**: Pay special attention to Partial categories - they indicate areas that could be enriched but don't block promotion
283
+
284
+ **⚠️ CRITICAL: Automatic Refinement After Auto-Enrichment**:
285
+
286
+ **If auto-enrichment was used, you MUST automatically refine generic acceptance criteria BEFORE proceeding with questions.**
287
+
288
+ **Step 1: Identify Generic Criteria** (from auto-enrichment output):
289
+
290
+ Look for patterns in the "Changes made" list:
291
+
292
+ - Generic templates: "Given a user wants to use {story}, When they interact with the system, Then {story} works correctly"
293
+ - Vague actions: "interact with the system", "perform the action", "access the system"
294
+ - Vague outcomes: "works correctly", "is functional", "works as expected"
295
+
296
+ **Step 2: Research Codebase** (for each story with generic criteria):
297
+
298
+ - Find the actual class and method names
299
+ - Identify method signatures and parameters
300
+ - Check test files for actual test patterns
301
+ - Understand return values and assertions
302
+
303
+ **Step 3: Generate Code-Specific Criteria** (replace generic with specific):
304
+
305
+ - Replace "interact with the system" → specific method calls with parameters
306
+ - Replace "works correctly" → specific return values, state changes, or assertions
307
+ - Add class names, method signatures, file paths where relevant
308
+
309
+ **Step 4: Apply Refinements** (use CLI commands):
310
+
311
+ ```bash
312
+ # For story-level acceptance criteria, use update-story:
313
+ specfact plan update-story --feature <feature-key> --key <story-key> --acceptance "<refined-code-specific-criteria>" --plan <path>
314
+
315
+ # For feature-level acceptance criteria, use update-feature:
316
+ specfact plan update-feature --key <feature-key> --acceptance "<refined-code-specific-criteria>" --plan <path>
317
+ ```
318
+
319
+ **Step 5: Verify** (before proceeding):
320
+
321
+ - All generic criteria replaced with code-specific criteria
322
+ - All criteria mention specific methods, classes, or file paths
323
+ - All criteria are testable (can be verified with automated tests)
324
+
325
+ **Only after Step 5 is complete, proceed with questions.**
326
+
327
+ **Understanding Deduplication**:
328
+
329
+ The CLI automatically deduplicates features during review using normalized key matching:
330
+
331
+ 1. **Exact matches**: Features with identical normalized keys are automatically deduplicated
332
+ - Example: `FEATURE-001` and `001_FEATURE_NAME` normalize to the same key
333
+ 2. **Prefix matches**: Abbreviated class names vs full Spec-Kit directory names
334
+ - Example: `FEATURE-IDEINTEGRATION` (from code analysis) vs `041_IDE_INTEGRATION_SYSTEM` (from Spec-Kit)
335
+ - Only matches when at least one key has a numbered prefix (Spec-Kit origin) to avoid false positives
336
+ - Requires minimum 10 characters, 6+ character difference, and <75% length ratio
337
+
338
+ **LLM Semantic Deduplication**:
339
+
340
+ After automated deduplication, you should review the plan bundle for **semantic/logical duplicates** that automated matching might miss:
341
+
342
+ 1. **Review feature titles and descriptions**: Look for features that represent the same functionality with different names
343
+ - Example: "Git Operations Manager" vs "Git Operations Handler" (both handle git operations)
344
+ - Example: "Telemetry Settings" vs "Telemetry Configuration" (both configure telemetry)
345
+ 2. **Check feature stories**: Features with overlapping or identical user stories may be duplicates
346
+ 3. **Analyze acceptance criteria**: Features with similar acceptance criteria covering the same functionality
347
+ 4. **Check code references**: If multiple features reference the same code files/modules, they might be the same feature
348
+ 5. **Suggest consolidation**: When semantic duplicates are found:
349
+ - Use `specfact plan update-feature` to merge information into one feature
350
+ - Use `specfact plan add-feature` to create a consolidated feature if needed
351
+ - Document which features were consolidated and why
352
+
353
+ **Example Semantic Duplicate Detection**:
354
+
355
+ ```text
356
+ After review, analyze the plan bundle and identify:
357
+ - Features with similar titles but different keys
358
+ - Features covering the same code modules
359
+ - Features with overlapping user stories or acceptance criteria
360
+ - Features that represent the same functionality
361
+
362
+ If semantic duplicates are found, suggest consolidation:
363
+ "Found semantic duplicates: FEATURE-GITOPERATIONS and FEATURE-GITOPERATIONSHANDLER
364
+ both cover git operations. Should I consolidate these into a single feature?"
365
+ ```
366
+
367
+ **Understanding Auto-Enrichment Output**:
368
+
369
+ When `--auto-enrich` is used, the CLI will output:
370
+
371
+ ```bash
372
+ Auto-enriching plan bundle (enhancing vague acceptance criteria, incomplete requirements, generic tasks)...
373
+ ✓ Auto-enriched plan bundle: 2 features, 5 stories updated
374
+ - Enhanced 8 acceptance criteria
375
+ - Enhanced 3 requirements
376
+ - Enhanced 4 tasks
377
+
378
+ Changes made:
379
+ - Feature FEATURE-001: Enhanced requirement 'System MUST Helper class' → 'System MUST provide a Helper class for git operations operations'
380
+ - Story STORY-001: Enhanced acceptance criteria 'is implemented' → 'Given a developer wants to use configure git operations, When they interact with the system, Then configure git operations is functional and verified'
381
+ ...
382
+ ```
383
+
384
+ **Understanding CLI Output**:
385
+
386
+ When the CLI reports "No critical ambiguities detected. Plan is ready for promotion" but shows ⚠️ Partial categories, this means:
387
+
388
+ - **Critical categories** (Functional Scope, Feature Completeness, Constraints) are all Clear or Partial (not Missing)
389
+ - **Partial categories** are not critical enough to block promotion, but enrichment would improve plan quality
390
+ - The plan can be promoted, but consider enriching Partial categories for better completeness
391
+
392
+ **LLM Post-Enrichment Analysis & Automatic Refinement**:
393
+
394
+ **⚠️ CRITICAL**: After auto-enrichment runs, you MUST automatically refine the generic acceptance criteria with code-specific, testable details. The auto-enrichment creates generic templates (e.g., "Given a user wants to use {story}, When they interact with the system, Then {story} works correctly"), but these are NOT testable. You should IMMEDIATELY replace them with specific, code-based criteria.
395
+
396
+ **Why This Matters**:
397
+
398
+ - **Generic criteria are NOT testable**: "When they interact with the system" cannot be verified
399
+ - **Test-based criteria are better**: "When extract_article_viii_evidence() is called" is specific and testable
400
+ - **Auto-enrichment makes things worse**: It replaces test-based criteria with generic templates
401
+ - **LLM reasoning is required**: Only LLM can understand codebase context and create specific criteria
402
+
403
+ **Automatic Refinement Workflow (MANDATORY after auto-enrichment)**:
404
+
405
+ 1. **Parse auto-enrichment output**: Identify which acceptance criteria were enhanced (look for generic patterns like "interact with the system", "works correctly", "is functional and verified")
406
+ 2. **Research codebase context**: For each enhanced story, find the actual:
407
+ - Class names and method signatures (e.g., `ContractFirstTestManager.extract_article_viii_evidence()`)
408
+ - File paths and module structure (e.g., `src/specfact_cli/enrichers/plan_enricher.py`)
409
+ - Test patterns and validation logic (check test files for actual test cases)
410
+ - Actual behavior and return values (e.g., returns `dict` with `'status'` key)
411
+ 3. **Generate code-specific criteria**: Replace generic templates with specific, testable criteria:
412
+ - **Generic (BAD)**: "Given a user wants to use as a developer, i can configure contract first test manager, When they interact with the system, Then as a developer, i can configure contract first test manager works correctly"
413
+ - **Code-specific (GOOD)**: "Given a ContractFirstTestManager instance is available, When extract_article_viii_evidence(repo_path: Path) is called, Then the method returns a dict with 'status' key equal to 'PASS' or 'FAIL' and 'frameworks_detected' list"
414
+ 4. **Apply refinements automatically**: Use `specfact plan update-feature` to replace ALL generic criteria with code-specific ones BEFORE asking questions
415
+ 5. **Verify testability**: Ensure all refined criteria can be verified with automated tests (include specific method names, parameters, return values, assertions)
416
+
417
+ **Example Automatic Refinement Process**:
418
+
419
+ ```markdown
420
+ 1. Auto-enrichment enhanced: "is implemented" → "Given a user wants to use configure git operations, When they interact with the system, Then configure git operations works correctly"
421
+
422
+ 2. LLM Analysis:
423
+ - Story: "As a developer, I can configure Contract First Test Manager"
424
+ - Feature: "Contract First Test Manager"
425
+ - Research codebase: Find `ContractFirstTestManager` class and its methods
426
+
427
+ 3. Codebase Research:
428
+ - Find: `src/specfact_cli/enrichers/plan_enricher.py` with `PlanEnricher` class
429
+ - Methods: `enrich_plan()`, `_enhance_vague_acceptance_criteria()`, etc.
430
+ - Test patterns: Check test files for actual test cases
431
+
432
+ 4. Generate Code-Specific Criteria:
433
+ - "Given a developer wants to configure Contract First Test Manager, When they call `PlanEnricher.enrich_plan(plan_bundle: PlanBundle)` with a valid plan bundle, Then the method returns an enrichment summary dict with 'features_updated' and 'stories_updated' counts"
434
+
435
+ 5. Apply via CLI:
436
+ ```bash
437
+ # For story-level acceptance criteria:
438
+ specfact plan update-story --feature FEATURE-CONTRACTFIRSTTESTMANAGER --key STORY-001 --acceptance "Given a developer wants to configure Contract First Test Manager, When they call PlanEnricher.enrich_plan(plan_bundle: PlanBundle) with a valid plan bundle, Then the method returns an enrichment summary dict with 'features_updated' and 'stories_updated' counts" --plan <path>
439
+
440
+ # For feature-level acceptance criteria:
441
+ specfact plan update-feature --key FEATURE-CONTRACTFIRSTTESTMANAGER --acceptance "Given a developer wants to configure Contract First Test Manager, When they call PlanEnricher.enrich_plan(plan_bundle: PlanBundle) with a valid plan bundle, Then the method returns an enrichment summary dict with 'features_updated' and 'stories_updated' counts" --plan <path>
442
+ ```
443
+
444
+ **When to Apply Automatic Refinement**:
445
+
446
+ - **MANDATORY after auto-enrichment**: If `--auto-enrich` was used, you MUST automatically refine ALL generic criteria BEFORE asking questions. Do not proceed with questions until generic criteria are replaced.
447
+ - **During review**: When questions ask about vague acceptance criteria, provide code-specific refinements immediately
448
+ - **Before promotion**: Ensure all acceptance criteria are code-specific and testable (no generic placeholders)
449
+
450
+ **Refinement Priority**:
451
+
452
+ 1. **High Priority (Do First)**: Criteria containing generic patterns:
453
+ - "interact with the system"
454
+ - "works correctly" / "works as expected" / "is functional"
455
+ - "perform the action"
456
+ - "access the system"
457
+ - Any criteria that doesn't mention specific methods, classes, or file paths
458
+
459
+ 2. **Medium Priority**: Criteria that are testable but could be more specific:
460
+ - Add method signatures
461
+ - Add parameter types
462
+ - Add return value assertions
463
+ - Add file path references
464
+
465
+ 3. **Low Priority**: Criteria that are already code-specific:
466
+ - Preserve test-based criteria (don't replace with generic)
467
+ - Only enhance if missing important details
468
+
469
+ **Refinement Quality Checklist**:
470
+
471
+ - ✅ **Specific method names**: Include actual class.method() signatures
472
+ - ✅ **Specific file paths**: Reference actual code locations when relevant
473
+ - ✅ **Testable outcomes**: Include specific return values, state changes, or observable behaviors
474
+ - ✅ **Domain-specific**: Use terminology from the actual codebase
475
+ - ✅ **No generic placeholders**: Avoid "interact with the system", "works correctly", "is functional"
476
+
477
+ ### 2. Get Questions from CLI (Copilot Mode) or Analyze Directly (Interactive Mode)
478
+
479
+ **⚠️ CRITICAL**: In Copilot mode, you MUST use `--list-questions` to get questions from the CLI, then ask the user, then feed answers back via `--answers`.
480
+
481
+ **Step 2a: Get Questions (Copilot Mode)**:
482
+
483
+ ```bash
484
+ # Execute CLI to get questions in JSON format
485
+ specfact plan review --list-questions --plan <plan_path> --max-questions 5
486
+ ```
487
+
488
+ **Parse JSON output**:
489
+
490
+ ```json
491
+ {
492
+ "questions": [
493
+ {
494
+ "id": "Q001",
495
+ "category": "Feature/Story Completeness",
496
+ "question": "What user stories are needed for feature FEATURE-IDEINTEGRATION?",
497
+ "impact": 0.9,
498
+ "uncertainty": 0.8,
499
+ "related_sections": ["features.FEATURE-IDEINTEGRATION.stories"]
500
+ },
501
+ ...
502
+ ],
503
+ "total": 5
504
+ }
505
+ ```
506
+
507
+ **Step 2b: Analyze Plan Bundle for Ambiguities (Interactive Mode Only)**:
508
+
509
+ **CLI Mode**: The CLI performs structured ambiguity scan using taxonomy categories:
510
+
511
+ 1. **Functional Scope & Behavior**
512
+ - Core user goals & success criteria
513
+ - Explicit out-of-scope declarations
514
+ - User roles / personas differentiation
515
+
516
+ 2. **Domain & Data Model**
517
+ - Entities, attributes, relationships
518
+ - Identity & uniqueness rules
519
+ - Lifecycle/state transitions
520
+ - Data volume / scale assumptions
521
+
522
+ 3. **Interaction & UX Flow**
523
+ - Critical user journeys / sequences
524
+ - Error/empty/loading states
525
+ - Accessibility or localization notes
526
+
527
+ 4. **Non-Functional Quality Attributes**
528
+ - Performance (latency, throughput targets)
529
+ - Scalability (horizontal/vertical, limits)
530
+ - Reliability & availability (uptime, recovery expectations)
531
+ - Observability (logging, metrics, tracing signals)
532
+ - Security & privacy (authN/Z, data protection, threat assumptions)
533
+ - Compliance / regulatory constraints (if any)
534
+
535
+ 5. **Integration & External Dependencies**
536
+ - External services/APIs and failure modes
537
+ - Data import/export formats
538
+ - Protocol/versioning assumptions
539
+
540
+ 6. **Edge Cases & Failure Handling**
541
+ - Negative scenarios
542
+ - Rate limiting / throttling
543
+ - Conflict resolution (e.g., concurrent edits)
544
+
545
+ 7. **Constraints & Tradeoffs**
546
+ - Technical constraints (language, storage, hosting)
547
+ - Explicit tradeoffs or rejected alternatives
548
+
549
+ 8. **Terminology & Consistency**
550
+ - Canonical glossary terms
551
+ - Avoided synonyms / deprecated terms
552
+
553
+ 9. **Completion Signals**
554
+ - Acceptance criteria testability
555
+ - Measurable Definition of Done style indicators
556
+
557
+ 10. **Feature/Story Completeness**
558
+ - Missing acceptance criteria
559
+ - Unclear story outcomes
560
+ - Incomplete feature constraints
561
+
562
+ **For each category**, mark status: **Clear** / **Partial** / **Missing**
563
+
564
+ **Copilot Mode**: In addition to CLI analysis, you can:
565
+
566
+ - Research codebase for additional context
567
+ - Analyze similar implementations for best practices
568
+ - Provide reasoning for question prioritization
569
+
570
+ ### 3. Generate Question Queue
571
+
572
+ **Prioritize questions** by (Impact × Uncertainty) heuristic:
573
+
574
+ - Maximum 5 questions per session
575
+ - Only include questions whose answers materially impact:
576
+ - Architecture decisions
577
+ - Data modeling
578
+ - Task decomposition
579
+ - Test design
580
+ - UX behavior
581
+ - Operational readiness
582
+ - Compliance validation
583
+
584
+ **Exclude**:
585
+
586
+ - Questions already answered in existing clarifications
587
+ - Trivial stylistic preferences
588
+ - Plan-level execution details (unless blocking correctness)
589
+ - Speculative tech stack questions (unless blocking functional clarity)
590
+
591
+ **If no valid questions exist**, analyze the coverage summary:
592
+
593
+ **Understanding Coverage Status**:
594
+
595
+ - **✅ Clear**: Category has no ambiguities or all findings are resolved
596
+ - **⚠️ Partial**: Category has some findings, but they're not high-priority enough to generate questions (low impact × uncertainty score)
597
+ - **❌ Missing**: Category has critical findings that block promotion (high impact × uncertainty)
598
+
599
+ **Critical vs Important Categories**:
600
+
601
+ - **Critical categories** (block promotion if Missing):
602
+ - Functional Scope & Behavior
603
+ - Feature/Story Completeness
604
+ - Constraints & Tradeoffs
605
+
606
+ - **Important categories** (warn if Missing or Partial, but don't block):
607
+ - Domain & Data Model
608
+ - Interaction & UX Flow
609
+ - Integration & External Dependencies
610
+ - Edge Cases & Failure Handling
611
+ - Completion Signals
612
+
613
+ **When No Questions Are Generated**:
614
+
615
+ 1. **If Partial categories exist**: Explain what "Partial" means and provide enrichment guidance:
616
+ - **Partial = Some gaps exist but not critical enough for questions**
617
+ - **Action**: Use CLI commands to manually enrich these categories (see "Manual Enrichment" section below)
618
+ - **Example**: If "Completion Signals: Partial", many stories have acceptance criteria but they're not testable (missing "must", "should", "verify", "validate", "check" keywords)
619
+
620
+ 2. **If Missing critical categories**: Report warning and suggest using `specfact plan update-idea` or `specfact plan update-feature` to fill gaps
621
+ - **Note**: Idea section is OPTIONAL - provides business context, not technical implementation
622
+ - Report: "No high-priority questions generated, but missing critical categories detected. Consider using `specfact plan update-idea` to add optional business metadata."
623
+
624
+ 3. **If all categories are Clear**: Report: "No critical ambiguities detected. Plan is ready for promotion."
625
+
626
+ **Spec-Kit Sync Integration**:
627
+
628
+ If the user mentions they plan to sync to Spec-Kit later (e.g., "I'll sync to spec-kit after review"), you should:
629
+
630
+ 1. **Reassure them**: The `specfact sync spec-kit` command automatically generates all required Spec-Kit fields:
631
+ - Frontmatter (Feature Branch, Created date, Status) in spec.md
632
+ - INVSEST criteria in spec.md
633
+ - Scenarios (Primary, Alternate, Exception, Recovery) in spec.md
634
+ - Constitution Check (Article VII, VIII, IX) in plan.md
635
+ - Phases (Phase 0, 1, 2, -1) in plan.md and tasks.md
636
+ - Technology Stack in plan.md
637
+ - Story mappings ([US1], [US2]) in tasks.md
638
+
639
+ 2. **Focus on plan bundle enrichment**: During review, focus on enriching the plan bundle itself (acceptance criteria, constraints, stories) rather than worrying about Spec-Kit-specific formatting
640
+
641
+ 3. **Explain the workflow**:
642
+ - Review enriches plan bundle → Sync generates complete Spec-Kit artifacts → Optional customization if needed
643
+
644
+ **Enrichment Strategy for Partial Categories**:
645
+
646
+ When categories are marked as "Partial", use this two-phase approach:
647
+
648
+ **Phase 1: Automatic Enrichment** (use `--auto-enrich` flag):
649
+
650
+ - **Completion Signals (Partial)**: Auto-enriches vague acceptance criteria ("is implemented", "is functional") → Given/When/Then format
651
+ - **Feature Completeness (Partial)**: Auto-enriches incomplete requirements ("System MUST Helper class") → Complete requirements with verbs
652
+ - **Feature Completeness (Partial)**: Auto-enriches generic tasks → Tasks with implementation details
653
+
654
+ **Phase 2: LLM-Enhanced Manual Refinement** (after auto-enrichment):
655
+
656
+ After auto-enrichment, use LLM reasoning to refine generic improvements:
657
+
658
+ - **Completion Signals (Partial)**: Review auto-enriched Given/When/Then scenarios and refine with specific actions:
659
+ - Generic: "When they interact with the system"
660
+ - Refined: "When they call the `configure()` method with valid parameters"
661
+ - Use: `specfact plan update-story --feature <feature-key> --key <story-key> --acceptance "<refined criteria>" --plan <path>` for story-level criteria
662
+ - Use: `specfact plan update-feature --key <key> --acceptance "<refined criteria>" --plan <path>` for feature-level criteria
663
+
664
+ - **Edge Cases (Partial)**: Add domain-specific edge cases:
665
+ - Use `specfact plan update-feature` to add edge case acceptance criteria
666
+ - Add keywords like "edge", "corner", "boundary", "limit", "invalid", "null", "empty"
667
+ - Example: Add "Given invalid Git repository path, When configure() is called, Then system returns clear error message"
668
+
669
+ - **Integration (Partial)**: Add specific external dependency constraints:
670
+ - Use `specfact plan update-feature --constraints` to add external dependency constraints
671
+ - Example: `--constraints "API rate limits: 100 req/min, Timeout: 30s, Retry: 3 attempts"`
672
+
673
+ - **Data Model (Partial)**: Add specific data model constraints:
674
+ - Use `specfact plan update-feature --constraints` to add data model constraints
675
+ - Example: `--constraints "Entity uniqueness: email must be unique, Max records: 10,000 per user"`
676
+
677
+ - **Interaction/UX (Partial)**: Add specific error handling scenarios:
678
+ - Use `specfact plan update-feature` to add error handling acceptance criteria
679
+ - Add keywords like "error", "empty", "invalid", "validation", "failure"
680
+ - Example: Add "Given user submits invalid input, When validation runs, Then system displays clear error message with field-specific guidance"
681
+
682
+ **LLM Reasoning for Refinement**:
683
+
684
+ When refining auto-enriched content, consider:
685
+
686
+ 1. **Context from codebase**: Research the actual codebase structure to suggest accurate file paths, method names, and component references
687
+ 2. **Domain knowledge**: Use domain-specific terminology and patterns appropriate for the feature
688
+ 3. **Testability**: Ensure refined acceptance criteria are truly testable (can be verified with automated tests)
689
+ 4. **Specificity**: Replace generic placeholders with specific, actionable details
690
+
691
+ ### 4. Sequential Questioning Loop
692
+
693
+ **⚠️ CRITICAL**: In Copilot mode, you MUST:
694
+
695
+ 1. Get questions via `--list-questions` (already done in Step 2a)
696
+ 2. Ask the user each question (this step)
697
+ 3. Collect all answers
698
+ 4. Feed answers back to CLI via `--answers` (Step 5)
699
+
700
+ **Present EXACTLY ONE question at a time.**
701
+
702
+ #### CLI Mode Format
703
+
704
+ **For multiple-choice questions**:
705
+
706
+ ```text
707
+ Q: [Question text]
708
+
709
+ Options:
710
+ A) [Option A description]
711
+ B) [Option B description]
712
+ C) [Option C description]
713
+ D) [Option D description] (if needed)
714
+ E) [Option E description] (if needed)
715
+ F) [Free text answer (<=5 words)]
716
+
717
+ Enter option letter (A-F) or provide your own short answer:
718
+ [WAIT FOR USER RESPONSE - DO NOT CONTINUE]
719
+ ```
720
+
721
+ **For short-answer questions**:
722
+
723
+ ```text
724
+ Q: [Question text]
725
+
726
+ Format: Short answer (<=5 words)
727
+
728
+ Enter your answer:
729
+ [WAIT FOR USER RESPONSE - DO NOT CONTINUE]
730
+ ```
731
+
732
+ #### Copilot Mode Format
733
+
734
+ **For multiple-choice questions**:
735
+
736
+ ```text
737
+ **Recommended:** Option [X] - <reasoning (1-2 sentences explaining why this is the best choice)>
738
+
739
+ | Option | Description |
740
+ |--------|-------------|
741
+ | A | <Option A description> |
742
+ | B | <Option B description> |
743
+ | C | <Option C description> |
744
+ | D | <Option D description> (if needed) |
745
+ | E | <Option E description> (if needed) |
746
+ | Short | Provide a different short answer (<=5 words) |
747
+
748
+ You can reply with the option letter (e.g., "A"), accept the recommendation by saying "yes" or "recommended", or provide your own short answer.
749
+ [WAIT FOR USER RESPONSE - DO NOT CONTINUE]
750
+ ```
751
+
752
+ **For short-answer questions**:
753
+
754
+ ```text
755
+ **Suggested:** <your proposed answer> - <brief reasoning>
756
+
757
+ Format: Short answer (<=5 words). You can accept the suggestion by saying "yes" or "suggested", or provide your own answer.
758
+ [WAIT FOR USER RESPONSE - DO NOT CONTINUE]
759
+ ```
760
+
761
+ **After user answers**:
762
+
763
+ - If user replies with "yes", "recommended", or "suggested", use your previously stated recommendation/suggestion as the answer
764
+ - Otherwise, validate the answer (maps to option or fits <=5 word constraint)
765
+ - If ambiguous, ask for quick disambiguation (same question, don't advance)
766
+ - Once satisfactory, record answer and move to next question
767
+
768
+ **Stop asking when**:
769
+
770
+ - All critical ambiguities resolved early (remaining queued items become unnecessary), OR
771
+ - User signals completion ("done", "good", "no more"), OR
772
+ - You reach 5 asked questions
773
+
774
+ **Never reveal future queued questions in advance.**
775
+
776
+ ### 5. Feed Answers Back to CLI (Copilot Mode) or Integrate Directly (Interactive Mode)
777
+
778
+ **⚠️ CRITICAL**: In Copilot mode, after collecting all answers from the user, you MUST feed them back to the CLI using `--answers`:
779
+
780
+ **Step 1: Create answers JSON file** (ALWAYS use file, not inline JSON):
781
+
782
+ ```bash
783
+ # Create answers.json file with all answers
784
+ cat > answers.json << 'EOF'
785
+ {
786
+ "Q001": "Developers, DevOps engineers",
787
+ "Q002": "Yes",
788
+ "Q003": "Yes",
789
+ "Q004": "Yes",
790
+ "Q005": "Yes"
791
+ }
792
+ EOF
793
+ ```
794
+
795
+ **Step 2: Feed answers to CLI** (using file path - RECOMMENDED):
796
+
797
+ ```bash
798
+ # Feed all answers back to CLI (Copilot mode) - using file path (RECOMMENDED)
799
+ specfact plan review --plan <plan_path> --answers answers.json
800
+ ```
801
+
802
+ **⚠️ AVOID inline JSON strings** - They can cause parsing issues with special characters, quotes, and Rich markup:
803
+
804
+ ```bash
805
+ # ❌ NOT RECOMMENDED: Inline JSON string (may have parsing issues)
806
+ specfact plan review --plan <plan_path> --answers '{"Q001": "answer1", "Q002": "answer2"}'
807
+ ```
808
+
809
+ **Format**: The `--answers` parameter accepts either:
810
+
811
+ - **✅ JSON file path** (RECOMMENDED): Path to a JSON file containing question_id -> answer mappings
812
+ - More reliable parsing
813
+ - Easier to validate JSON syntax
814
+ - Avoids shell escaping issues
815
+ - Better for complex answers with special characters
816
+
817
+ - **⚠️ JSON string** (NOT RECOMMENDED): Direct JSON object (may have Rich markup parsing issues, shell escaping problems)
818
+ - Only use for simple, single-answer cases
819
+ - Requires careful quote escaping
820
+ - Can fail with special characters
821
+
822
+ **JSON Structure**:
823
+
824
+ - Keys: Question IDs (e.g., "Q001", "Q002")
825
+ - Values: Answer strings (≤5 words recommended)
826
+
827
+ **⚠️ CRITICAL: Boolean-Like Answer Values**:
828
+
829
+ When providing answers that are boolean-like strings (e.g., "Yes", "No", "True", "False", "On", "Off"), ensure they are:
830
+
831
+ 1. **Always quoted in JSON**: Use `"Yes"` not `Yes` (JSON requires quotes for strings)
832
+ 2. **Provided as strings**: Never use JSON booleans `true`/`false` - always use string values `"Yes"`/`"No"`
833
+
834
+ **❌ WRONG** (causes YAML validation errors):
835
+
836
+ ```json
837
+ {
838
+ "Q001": "Developers, DevOps engineers",
839
+ "Q002": true, // ❌ JSON boolean - will cause validation error
840
+ "Q003": Yes // ❌ Unquoted string - invalid JSON
841
+ }
842
+ ```
843
+
844
+ **✅ CORRECT**:
845
+
846
+ ```json
847
+ {
848
+ "Q001": "Developers, DevOps engineers",
849
+ "Q002": "Yes", // ✅ Quoted string
850
+ "Q003": "No" // ✅ Quoted string
851
+ }
852
+ ```
853
+
854
+ **Why This Matters**:
855
+
856
+ - YAML parsers interpret unquoted "Yes", "No", "True", "False", "On", "Off" as boolean values
857
+ - The CLI expects all answers to be strings (validated with `isinstance(answer, str)`)
858
+ - Boolean values in JSON will cause validation errors: "Answer for Q002 must be a non-empty string"
859
+ - The YAML serializer now automatically quotes boolean-like strings, but JSON parsing must still provide strings
860
+
861
+ **Example JSON file** (`answers.json`):
862
+
863
+ ```json
864
+ {
865
+ "Q001": "Developers, DevOps engineers",
866
+ "Q002": "Yes",
867
+ "Q003": "Yes",
868
+ "Q004": "Yes",
869
+ "Q005": "Yes"
870
+ }
871
+ ```
872
+
873
+ **Usage**:
874
+
875
+ ```bash
876
+ # ✅ RECOMMENDED: Using file path
877
+ specfact plan review --plan <plan_path> --answers answers.json
878
+
879
+ # ⚠️ NOT RECOMMENDED: Using JSON string (only for simple cases)
880
+ specfact plan review --plan <plan_path> --answers '{"Q001": "answer1"}'
881
+ ```
882
+
883
+ **Validation After Feeding Answers**:
884
+
885
+ After feeding answers, always verify the plan bundle is valid:
886
+
887
+ ```bash
888
+ # Verify plan bundle is valid (should not show validation errors)
889
+ specfact plan review --plan <plan_path> --list-questions --max-questions 1
890
+ ```
891
+
892
+ If you see validation errors like "Input should be a valid string", check:
893
+
894
+ 1. All answers in JSON file are quoted strings (not booleans)
895
+ 2. JSON file syntax is valid (use `python3 -m json.tool answers.json` to validate)
896
+ 3. No unquoted boolean-like strings ("Yes", "No", "True", "False")
897
+
898
+ **In Interactive Mode**: The CLI automatically integrates answers after each question.
899
+
900
+ **After CLI processes answers** (Copilot mode), the CLI will:
901
+
902
+ 1. **Update plan bundle sections** based on answer:
903
+ - **Functional ambiguity** → `features[].acceptance[]` or `idea.narrative`
904
+ - **Data model** → `features[].constraints[]` or new feature
905
+ - **Non-functional** → `features[].constraints[]` or `idea.constraints[]`
906
+ - **Edge cases** → `features[].acceptance[]` or `stories[].acceptance[]`
907
+ - **Terminology** → Normalize across plan bundle
908
+
909
+ 2. **Add clarification to plan bundle**:
910
+ - Ensure `clarifications.sessions[]` exists
911
+ - Create session for today's date if not present
912
+ - Add clarification with:
913
+ - Unique ID (Q001, Q002, etc.)
914
+ - Category
915
+ - Question text
916
+ - Answer
917
+ - Integration points (e.g., `["features.FEATURE-001.acceptance"]`)
918
+ - Timestamp
919
+
920
+ 3. **Save plan bundle** (CLI automatically saves after each integration)
921
+
922
+ 4. **Validate plan bundle**:
923
+ - Structure is valid
924
+ - No contradictory statements
925
+ - Terminology consistency
926
+ - Clarifications properly formatted
927
+
928
+ **Preserve formatting**: Do not reorder unrelated sections; keep heading hierarchy intact.
929
+
930
+ **Keep each integration minimal and testable** (avoid narrative drift).
931
+
932
+ ### 6. Validation
933
+
934
+ **After EACH write plus final pass**:
935
+
936
+ - Clarifications session contains exactly one entry per accepted answer (no duplicates)
937
+ - Total asked (accepted) questions ≤ 5
938
+ - Updated sections contain no lingering vague placeholders
939
+ - No contradictory earlier statement remains
940
+ - Plan bundle structure is valid
941
+ - Terminology consistency: same canonical term used across all updated sections
942
+
943
+ ### 7. Report Completion
944
+
945
+ **After questioning loop ends or early termination**:
946
+
947
+ **If questions were asked and answered**:
948
+
949
+ ```markdown
950
+ ✓ Review complete!
951
+
952
+ **Questions Asked**: 3
953
+ **Plan Bundle**: `.specfact/plans/specfact-import-test.2025-11-17T12-21-48.bundle.yaml`
954
+ **Sections Touched**:
955
+ - `features.FEATURE-001.acceptance`
956
+ - `features.FEATURE-002.constraints`
957
+ - `idea.constraints`
958
+
959
+ **Coverage Summary**:
960
+
961
+ | Category | Status | Notes |
962
+ |----------|--------|-------|
963
+ | Functional Scope | ✅ Clear | Resolved (was Partial, now addressed) |
964
+ | Data Model | ✅ Clear | Already sufficient |
965
+ | Non-Functional | ✅ Clear | Resolved (was Missing, now addressed) |
966
+ | Edge Cases | ⚠️ Partial | Deferred (exceeds question quota, see enrichment guide) |
967
+ | Completion Signals | ⚠️ Partial | Some stories need testable acceptance criteria |
968
+ | Terminology | ✅ Clear | Already sufficient |
969
+
970
+ **Next Steps**:
971
+ - Plan is ready for promotion to `review` stage
972
+ - Run: `/specfact-cli/specfact-plan-promote review`
973
+ - Optional: Enrich Partial categories using CLI commands (see Manual Enrichment section)
974
+ ```
975
+
976
+ **If no questions were generated but Partial categories exist**:
977
+
978
+ ```markdown
979
+ ✓ Review analysis complete!
980
+
981
+ **Plan Bundle**: `.specfact/plans/specfact-import-test.2025-11-17T12-21-48.bundle.yaml`
982
+ **Status**: No critical ambiguities detected (all critical categories are Clear)
983
+
984
+ **Coverage Summary**:
985
+
986
+ | Category | Status | Meaning |
987
+ |----------|--------|---------|
988
+ | Functional Scope | ✅ Clear | No ambiguities detected |
989
+ | Data Model | ⚠️ Partial | Some features mention data but lack constraints (not critical) |
990
+ | Interaction/UX | ⚠️ Partial | Some stories mention UX but lack error handling (not critical) |
991
+ | Integration | ⚠️ Partial | Some features mention integration but lack constraints (not critical) |
992
+ | Edge Cases | ⚠️ Partial | Some stories have <3 acceptance criteria (not critical) |
993
+ | Completion Signals | ⚠️ Partial | Some acceptance criteria are not testable (not critical) |
994
+ | Constraints | ✅ Clear | No ambiguities detected |
995
+
996
+ **Understanding "Partial" Status**:
997
+
998
+ ⚠️ **Partial** means the category has some gaps, but they're not high-priority enough to generate questions. The plan can still be promoted, but enrichment would improve quality.
999
+
1000
+ **Enrichment Options**:
1001
+
1002
+ - **Automatic Enrichment** (recommended first step): Use `--auto-enrich` flag to automatically fix vague acceptance criteria, incomplete requirements, and generic tasks
1003
+
1004
+ ```bash
1005
+ specfact plan review --auto-enrich --plan <plan_path>
1006
+ ```
1007
+
1008
+ - **LLM-Enhanced Refinement** (after auto-enrichment): Use LLM reasoning to:
1009
+
1010
+ - Review auto-enrichment results and verify contextually appropriate improvements
1011
+ - Identify generic improvements that need domain-specific refinement
1012
+ - Suggest specific manual improvements using CLI commands
1013
+
1014
+ - **Manual Enrichment**: Use `specfact plan update-feature` commands to add missing constraints/acceptance criteria with specific details
1015
+
1016
+ - **Defer**: Proceed with promotion and enrich later during implementation (not recommended if Partial categories are high-impact)
1017
+
1018
+ **Next Steps**:
1019
+
1020
+ - Plan can be promoted (no critical blockers)
1021
+ - Optional: Run enrichment to improve Partial categories
1022
+ - Run: `/specfact-cli/specfact-plan-promote review`
1023
+
1024
+ **If Outstanding or Deferred remain**:
1025
+
1026
+ - Recommend whether to proceed to promotion or run review again
1027
+ - Flag high-impact deferred items with rationale
1028
+ - Explain what "Partial" means and when enrichment is recommended vs optional
1029
+
1030
+ ## Guidelines
1031
+
1032
+ ### Question Quality
1033
+
1034
+ - **High Impact**: Questions that materially affect implementation or validation
1035
+ - **Actionable**: Answers that can be integrated into plan bundle sections
1036
+ - **Focused**: One question per ambiguity, not multiple related questions
1037
+ - **Testable**: Answers that lead to measurable acceptance criteria
1038
+
1039
+ ### Integration Quality
1040
+
1041
+ - **Immediate**: Integrate after each answer, don't batch
1042
+ - **Atomic**: Save plan bundle after each integration
1043
+ - **Minimal**: Keep integrations concise and testable
1044
+ - **Consistent**: Use same terminology across all sections
1045
+
1046
+ ### Promotion Readiness
1047
+
1048
+ A plan is ready for promotion when:
1049
+
1050
+ - All critical ambiguities resolved
1051
+ - Acceptance criteria are testable
1052
+ - Constraints are explicit and measurable
1053
+ - Terminology is consistent
1054
+ - No contradictory statements remain
1055
+
1056
+ ### LLM Reasoning for Continuous Improvement
1057
+
1058
+ **After auto-enrichment, use LLM reasoning to further improve the plan**:
1059
+
1060
+ 1. **Analyze Enrichment Quality**:
1061
+ - Review each enhanced acceptance criteria: Is the Given/When/Then scenario specific enough?
1062
+ - Check enhanced requirements: Do they capture the full intent with appropriate verbs?
1063
+ - Evaluate task enhancements: Are file paths and method names accurate for the codebase?
1064
+
1065
+ 2. **Identify Generic Patterns**:
1066
+ - Look for placeholder text like "interact with the system" → Suggest specific actions
1067
+ - Find generic file paths like "src/specfact_cli/..." → Research actual codebase structure
1068
+ - Detect vague component names → Suggest specific class/function names from codebase
1069
+
1070
+ 3. **Research Codebase Context**:
1071
+ - Search for actual file paths, method names, and component structures
1072
+ - Identify domain-specific patterns and terminology
1073
+ - Understand the actual implementation approach to suggest accurate refinements
1074
+
1075
+ 4. **Propose Specific Refinements**:
1076
+ - Use `specfact plan update-feature` to refine generic Given/When/Then scenarios
1077
+ - Add specific file paths, method names, or component references to tasks
1078
+ - Enhance requirements with domain-specific details
1079
+
1080
+ 5. **Validate Improvements**:
1081
+ - Ensure all refinements are testable and measurable
1082
+ - Verify terminology consistency across all enhancements
1083
+ - Check that refinements align with codebase structure and patterns
1084
+
1085
+ ## Troubleshooting
1086
+
1087
+ ### Common Errors and Solutions
1088
+
1089
+ #### Error: "Plan validation failed: Validation error: Input should be a valid string"
1090
+
1091
+ **Cause**: Answers in clarifications section are stored as booleans instead of strings.
1092
+
1093
+ **Symptoms**:
1094
+
1095
+ - Error message: `clarifications.sessions.0.questions.X.answer: Input should be a valid string`
1096
+ - Plan bundle fails to load or validate
1097
+
1098
+ **Solution**:
1099
+
1100
+ 1. **Check JSON file format**:
1101
+
1102
+ ```bash
1103
+ # Validate JSON syntax
1104
+ python3 -m json.tool answers.json
1105
+ ```
1106
+
1107
+ 2. **Ensure all answers are quoted strings**:
1108
+
1109
+ ```json
1110
+ {
1111
+ "Q001": "Developers, DevOps engineers", // ✅ Quoted string
1112
+ "Q002": "Yes", // ✅ Quoted string (not true or unquoted Yes)
1113
+ "Q003": "No" // ✅ Quoted string (not false or unquoted No)
1114
+ }
1115
+ ```
1116
+
1117
+ 3. **Fix existing plan bundle** (if already corrupted):
1118
+
1119
+ ```bash
1120
+ # Use sed to quote unquoted "Yes" values in YAML
1121
+ sed -i "s/^ answer: Yes$/ answer: 'Yes'/" .specfact/plans/<plan>.bundle.yaml
1122
+ sed -i "s/^ answer: No$/ answer: 'No'/" .specfact/plans/<plan>.bundle.yaml
1123
+ ```
1124
+
1125
+ 4. **Verify fix**:
1126
+
1127
+ ```bash
1128
+ # Check that all answers are strings
1129
+ python3 -c "import yaml; data = yaml.safe_load(open('.specfact/plans/<plan>.bundle.yaml')); print('All strings:', all(isinstance(q['answer'], str) for s in data['clarifications']['sessions'] for q in s['questions']))"
1130
+ ```
1131
+
1132
+ #### Error: "Invalid JSON in --answers"
1133
+
1134
+ **Cause**: JSON syntax error in answers file or inline JSON string.
1135
+
1136
+ **Solution**:
1137
+
1138
+ 1. **Validate JSON syntax**:
1139
+
1140
+ ```bash
1141
+ python3 -m json.tool answers.json
1142
+ ```
1143
+
1144
+ 2. **Check for common issues**:
1145
+ - Missing quotes around string values
1146
+ - Trailing commas
1147
+ - Unclosed brackets or braces
1148
+ - Special characters not escaped
1149
+
1150
+ 3. **Use file path instead of inline JSON** (recommended):
1151
+
1152
+ ```bash
1153
+ # ✅ Better: Use file
1154
+ specfact plan review --answers answers.json
1155
+
1156
+ # ⚠️ Avoid: Inline JSON (can have escaping issues)
1157
+ specfact plan review --answers '{"Q001": "answer"}'
1158
+ ```
1159
+
1160
+ #### Error: "Answer for Q002 must be a non-empty string"
1161
+
1162
+ **Cause**: Answer value is not a string (e.g., boolean `true`/`false` or `null`).
1163
+
1164
+ **Solution**:
1165
+
1166
+ 1. **Ensure all answers are strings in JSON**:
1167
+
1168
+ ```json
1169
+ {
1170
+ "Q002": "Yes" // ✅ String
1171
+ }
1172
+ ```
1173
+
1174
+ Not:
1175
+
1176
+ ```json
1177
+ {
1178
+ "Q002": true // ❌ Boolean
1179
+ "Q002": null // ❌ Null
1180
+ }
1181
+ ```
1182
+
1183
+ 2. **Validate before feeding to CLI**:
1184
+
1185
+ ```bash
1186
+ # Check all values are strings
1187
+ python3 -c "import json; data = json.load(open('answers.json')); print('All strings:', all(isinstance(v, str) for v in data.values()))"
1188
+ ```
1189
+
1190
+ #### Error: "Feature 'FEATURE-001' not found in plan"
1191
+
1192
+ **Cause**: Feature key doesn't exist in plan bundle.
1193
+
1194
+ **Solution**:
1195
+
1196
+ 1. **List available features**:
1197
+
1198
+ ```bash
1199
+ specfact plan select --list-features
1200
+ ```
1201
+
1202
+ 2. **Use correct feature key** (case-sensitive, exact match required)
1203
+
1204
+ #### Error: "Story 'STORY-001' not found in feature 'FEATURE-001'"
1205
+
1206
+ **Cause**: Story key doesn't exist in the specified feature.
1207
+
1208
+ **Solution**:
1209
+
1210
+ 1. **List stories in feature**:
1211
+
1212
+ ```bash
1213
+ # Check plan bundle YAML for story keys
1214
+ grep -A 5 "key: FEATURE-001" .specfact/plans/<plan>.bundle.yaml | grep "key: STORY"
1215
+ ```
1216
+
1217
+ 2. **Use correct story key** (case-sensitive, exact match required)
1218
+
1219
+ ### Prevention Checklist
1220
+
1221
+ Before feeding answers to CLI:
1222
+
1223
+ - [ ] **JSON file syntax is valid** (use `python3 -m json.tool` to validate)
1224
+ - [ ] **All answer values are quoted strings** (not booleans, not null)
1225
+ - [ ] **Boolean-like strings are quoted** ("Yes", "No", "True", "False", "On", "Off")
1226
+ - [ ] **Using file path** (not inline JSON string) for complex answers
1227
+ - [ ] **No trailing commas** in JSON
1228
+ - [ ] **All question IDs match** (Q001, Q002, etc. from `--list-questions` output)
1229
+
1230
+ After feeding answers:
1231
+
1232
+ - [ ] **Plan bundle validates** (run `specfact plan review --list-questions --max-questions 1`)
1233
+ - [ ] **No validation errors** in CLI output
1234
+ - [ ] **All clarifications saved** (check `clarifications.sessions` in YAML)
1235
+
1236
+ **Example LLM Reasoning Process**:
1237
+
1238
+ ```text
1239
+ 1. Auto-enrichment enhanced: "is implemented" → "Given a developer wants to use configure git operations, When they interact with the system, Then configure git operations is functional and verified"
1240
+
1241
+ 2. LLM Analysis:
1242
+
1243
+ - ✅ Given clause is contextually appropriate
1244
+ - ⚠️ When clause is too generic ("interact with the system")
1245
+ - ✅ Then clause captures the outcome
1246
+
1247
+ 3. Codebase Research:
1248
+
1249
+ - Search for Git operations configuration methods
1250
+ - Find: `src/specfact_cli/utils/git_operations.py` with `configure()` method
1251
+ - Identify: Method signature `configure(repo_path: Path, config: dict) -> bool`
1252
+
1253
+ 4. Proposed Refinement:
1254
+
1255
+ - "Given a developer wants to configure Git operations, When they call configure(repo_path, config) with valid parameters, Then the method returns True and configuration is persisted"
1256
+
1257
+ 5. Execute Refinement:
1258
+
1259
+ ```bash
1260
+ specfact plan update-feature --key FEATURE-001 --acceptance "Given a developer wants to configure Git operations, When they call configure(repo_path, config) with valid parameters, Then the method returns True and configuration is persisted" --plan <path>
1261
+ ```
1262
+
1263
+ **Continuous Improvement Loop**:
1264
+
1265
+ 1. Run `--auto-enrich` to fix common issues automatically
1266
+ 2. Use LLM reasoning to analyze enrichment results
1267
+ 3. Research codebase for specific details
1268
+ 4. Propose and execute refinements using CLI commands
1269
+ 5. Re-run review to verify improvements
1270
+ 6. Iterate until plan quality meets promotion standards
1271
+
1272
+ ## Context
1273
+
1274
+ {ARGS}
1275
+
1276
+ --- End Command ---