specfact-cli 0.4.2__py3-none-any.whl → 0.6.8__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- specfact_cli/__init__.py +1 -1
- specfact_cli/agents/analyze_agent.py +2 -3
- specfact_cli/analyzers/__init__.py +2 -1
- specfact_cli/analyzers/ambiguity_scanner.py +601 -0
- specfact_cli/analyzers/code_analyzer.py +462 -30
- specfact_cli/analyzers/constitution_evidence_extractor.py +491 -0
- specfact_cli/analyzers/contract_extractor.py +419 -0
- specfact_cli/analyzers/control_flow_analyzer.py +281 -0
- specfact_cli/analyzers/requirement_extractor.py +337 -0
- specfact_cli/analyzers/test_pattern_extractor.py +330 -0
- specfact_cli/cli.py +151 -206
- specfact_cli/commands/constitution.py +281 -0
- specfact_cli/commands/enforce.py +42 -34
- specfact_cli/commands/import_cmd.py +481 -152
- specfact_cli/commands/init.py +224 -55
- specfact_cli/commands/plan.py +2133 -547
- specfact_cli/commands/repro.py +100 -78
- specfact_cli/commands/sync.py +701 -186
- specfact_cli/enrichers/constitution_enricher.py +765 -0
- specfact_cli/enrichers/plan_enricher.py +294 -0
- specfact_cli/importers/speckit_converter.py +364 -48
- specfact_cli/importers/speckit_scanner.py +65 -0
- specfact_cli/models/plan.py +42 -0
- specfact_cli/resources/mappings/node-async.yaml +49 -0
- specfact_cli/resources/mappings/python-async.yaml +47 -0
- specfact_cli/resources/mappings/speckit-default.yaml +82 -0
- specfact_cli/resources/prompts/specfact-enforce.md +185 -0
- specfact_cli/resources/prompts/specfact-import-from-code.md +626 -0
- specfact_cli/resources/prompts/specfact-plan-add-feature.md +188 -0
- specfact_cli/resources/prompts/specfact-plan-add-story.md +212 -0
- specfact_cli/resources/prompts/specfact-plan-compare.md +571 -0
- specfact_cli/resources/prompts/specfact-plan-init.md +531 -0
- specfact_cli/resources/prompts/specfact-plan-promote.md +352 -0
- specfact_cli/resources/prompts/specfact-plan-review.md +1276 -0
- specfact_cli/resources/prompts/specfact-plan-select.md +401 -0
- specfact_cli/resources/prompts/specfact-plan-update-feature.md +242 -0
- specfact_cli/resources/prompts/specfact-plan-update-idea.md +211 -0
- specfact_cli/resources/prompts/specfact-repro.md +268 -0
- specfact_cli/resources/prompts/specfact-sync.md +497 -0
- specfact_cli/resources/schemas/deviation.schema.json +61 -0
- specfact_cli/resources/schemas/plan.schema.json +204 -0
- specfact_cli/resources/schemas/protocol.schema.json +53 -0
- specfact_cli/resources/templates/github-action.yml.j2 +140 -0
- specfact_cli/resources/templates/plan.bundle.yaml.j2 +141 -0
- specfact_cli/resources/templates/pr-template.md.j2 +58 -0
- specfact_cli/resources/templates/protocol.yaml.j2 +24 -0
- specfact_cli/resources/templates/telemetry.yaml.example +35 -0
- specfact_cli/sync/__init__.py +10 -1
- specfact_cli/sync/watcher.py +268 -0
- specfact_cli/telemetry.py +440 -0
- specfact_cli/utils/acceptance_criteria.py +127 -0
- specfact_cli/utils/enrichment_parser.py +445 -0
- specfact_cli/utils/feature_keys.py +12 -3
- specfact_cli/utils/ide_setup.py +170 -0
- specfact_cli/utils/structure.py +179 -2
- specfact_cli/utils/yaml_utils.py +33 -0
- specfact_cli/validators/repro_checker.py +22 -1
- specfact_cli/validators/schema.py +15 -4
- specfact_cli-0.6.8.dist-info/METADATA +456 -0
- specfact_cli-0.6.8.dist-info/RECORD +99 -0
- {specfact_cli-0.4.2.dist-info → specfact_cli-0.6.8.dist-info}/entry_points.txt +1 -0
- specfact_cli-0.6.8.dist-info/licenses/LICENSE.md +202 -0
- specfact_cli-0.4.2.dist-info/METADATA +0 -370
- specfact_cli-0.4.2.dist-info/RECORD +0 -62
- specfact_cli-0.4.2.dist-info/licenses/LICENSE.md +0 -61
- {specfact_cli-0.4.2.dist-info → specfact_cli-0.6.8.dist-info}/WHEEL +0 -0
|
@@ -0,0 +1,1276 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "Review plan bundle to identify and resolve ambiguities, fill gaps, and prepare for promotion"
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
# SpecFact Review Plan Bundle Command
|
|
6
|
+
|
|
7
|
+
## User Input
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
$ARGUMENTS
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
You **MUST** consider the user input before proceeding (if not empty).
|
|
14
|
+
|
|
15
|
+
## ⚠️ CRITICAL: CLI Usage Enforcement
|
|
16
|
+
|
|
17
|
+
**YOU MUST ALWAYS USE THE SPECFACT CLI**. Never create artifacts directly.
|
|
18
|
+
|
|
19
|
+
### Rules
|
|
20
|
+
|
|
21
|
+
1. **ALWAYS execute CLI first**: Run `specfact plan review` before any analysis - execute the CLI command before any other operations
|
|
22
|
+
2. **NEVER write code**: Do not implement review logic - the CLI handles this
|
|
23
|
+
3. **NEVER create YAML/JSON directly**: All plan bundle updates must be CLI-generated
|
|
24
|
+
4. **NEVER bypass CLI validation**: CLI ensures schema compliance and metadata - use it, don't bypass its validation
|
|
25
|
+
5. **Use CLI output as grounding**: Parse CLI output, don't regenerate or recreate it - use the CLI output as the source of truth
|
|
26
|
+
6. **NEVER manipulate internal code**: Do NOT use Python code to directly modify PlanBundle objects, Feature objects, Clarification objects, or any internal data structures. The CLI is THE interface - use it exclusively.
|
|
27
|
+
7. **No internal knowledge required**: You should NOT need to know about internal implementation details (PlanBundle model, Feature class, AmbiguityScanner, etc.). All operations must be performed via CLI commands.
|
|
28
|
+
8. **NEVER read artifacts directly**: Do NOT read plan bundle files directly to extract information unless for display purposes (e.g., showing plan details to user). Use CLI commands (`specfact plan review --list-questions`, `specfact plan select`) to get plan information.
|
|
29
|
+
|
|
30
|
+
### What Happens If You Don't Follow This
|
|
31
|
+
|
|
32
|
+
- ❌ Artifacts may not match CLI schema versions
|
|
33
|
+
- ❌ Missing metadata and telemetry
|
|
34
|
+
- ❌ Format inconsistencies
|
|
35
|
+
- ❌ Validation failures
|
|
36
|
+
- ❌ Works only in Copilot mode, fails in CI/CD
|
|
37
|
+
- ❌ Breaks when CLI internals change
|
|
38
|
+
- ❌ Requires knowledge of internal code structure
|
|
39
|
+
|
|
40
|
+
### Available CLI Commands for Plan Updates
|
|
41
|
+
|
|
42
|
+
**For updating idea section (OPTIONAL - business metadata)**:
|
|
43
|
+
|
|
44
|
+
- `specfact plan update-idea --title <title> --narrative <narrative> --target-users <users> --value-hypothesis <hypothesis> --constraints <constraints> --plan <path>`
|
|
45
|
+
- Updates idea section metadata (optional business context, not technical implementation)
|
|
46
|
+
- **Note**: Idea section is OPTIONAL - provides business context and metadata
|
|
47
|
+
- All parameters are optional - use only what you need
|
|
48
|
+
- Works in CI/CD, Copilot, and interactive modes
|
|
49
|
+
- Example: `specfact plan update-idea --target-users "Developers, DevOps" --value-hypothesis "Reduce technical debt" --constraints "Python 3.11+, Maintain backward compatibility"`
|
|
50
|
+
|
|
51
|
+
**For updating features**:
|
|
52
|
+
|
|
53
|
+
- `specfact plan update-feature --key <key> --title <title> --outcomes <outcomes> --acceptance <acceptance> --constraints <constraints> --confidence <confidence> --draft/--no-draft --plan <path>`
|
|
54
|
+
- **Boolean flags**: `--draft` sets True, `--no-draft` sets False, omit to leave unchanged
|
|
55
|
+
- ❌ **WRONG**: `--draft true` or `--draft false` (Typer boolean flags don't accept values)
|
|
56
|
+
- ✅ **CORRECT**: `--draft` (sets True) or `--no-draft` (sets False)
|
|
57
|
+
- Updates existing feature metadata (title, outcomes, acceptance criteria, constraints, confidence, draft status)
|
|
58
|
+
- Works in CI/CD, Copilot, and interactive modes
|
|
59
|
+
- Example: `specfact plan update-feature --key FEATURE-001 --title "New Title" --outcomes "Outcome 1, Outcome 2"`
|
|
60
|
+
|
|
61
|
+
**For adding features**:
|
|
62
|
+
|
|
63
|
+
- `specfact plan add-feature --key <key> --title <title> --outcomes <outcomes> --acceptance <acceptance> --plan <path>`
|
|
64
|
+
|
|
65
|
+
**For adding stories**:
|
|
66
|
+
|
|
67
|
+
- `specfact plan add-story --feature <feature-key> --key <story-key> --title <title> --acceptance <acceptance> --story-points <points> --value-points <points> --plan <path>`
|
|
68
|
+
|
|
69
|
+
**For updating stories**:
|
|
70
|
+
|
|
71
|
+
- `specfact plan update-story --feature <feature-key> --key <story-key> --title <title> --acceptance <acceptance> --story-points <points> --value-points <points> --confidence <confidence> --draft/--no-draft --plan <path>`
|
|
72
|
+
- **Boolean flags**: `--draft` sets True, `--no-draft` sets False, omit to leave unchanged
|
|
73
|
+
- ❌ **WRONG**: `--draft true` or `--draft false` (Typer boolean flags don't accept values)
|
|
74
|
+
- ✅ **CORRECT**: `--draft` (sets True) or `--no-draft` (sets False)
|
|
75
|
+
- Updates existing story metadata (title, acceptance criteria, story points, value points, confidence, draft status)
|
|
76
|
+
- Works in CI/CD, Copilot, and interactive modes
|
|
77
|
+
- Example: `specfact plan update-story --feature FEATURE-001 --key STORY-001 --acceptance "Given X, When Y, Then Z" --story-points 5`
|
|
78
|
+
|
|
79
|
+
**❌ FORBIDDEN**: Direct Python code manipulation like:
|
|
80
|
+
|
|
81
|
+
```python
|
|
82
|
+
# ❌ NEVER DO THIS:
|
|
83
|
+
from specfact_cli.models.plan import PlanBundle, Feature
|
|
84
|
+
plan_bundle.features[0].title = "New Title" # Direct manipulation
|
|
85
|
+
generator.generate(plan_bundle, plan_path) # Bypassing CLI
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**✅ CORRECT**: Use CLI commands:
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
# ✅ ALWAYS DO THIS:
|
|
92
|
+
specfact plan update-feature --key FEATURE-001 --title "New Title" --plan <path>
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
## ⏸️ Wait States: User Input Required
|
|
96
|
+
|
|
97
|
+
**When user input is required, you MUST wait for the user's response.**
|
|
98
|
+
|
|
99
|
+
### Wait State Rules
|
|
100
|
+
|
|
101
|
+
1. **Never assume**: If input is missing, ask and wait
|
|
102
|
+
2. **Never continue**: Do not proceed until user responds
|
|
103
|
+
3. **Be explicit**: Clearly state what information you need
|
|
104
|
+
4. **Provide options**: Give examples or default suggestions
|
|
105
|
+
|
|
106
|
+
## Goal
|
|
107
|
+
|
|
108
|
+
Review a plan bundle to identify ambiguities, missing information, and unknowns. Systematically resolve these through targeted questions to make the plan ready for promotion (draft → review → approved).
|
|
109
|
+
|
|
110
|
+
**Note**: This review workflow is expected to run BEFORE promoting from `draft` to `review` stage. If the user explicitly states they are skipping review (e.g., exploratory spike), you may proceed, but must warn that promotion readiness may be incomplete.
|
|
111
|
+
|
|
112
|
+
**Automatic Enrichment Strategy**:
|
|
113
|
+
|
|
114
|
+
The CLI now supports automatic enrichment via `--auto-enrich` flag. Use this when:
|
|
115
|
+
|
|
116
|
+
1. **User explicitly requests enrichment**: "enrich", "auto-fix", "improve quality", "fix vague criteria"
|
|
117
|
+
2. **Plan quality indicators suggest it**: Vague acceptance criteria, incomplete requirements, generic tasks detected
|
|
118
|
+
3. **After Spec-Kit sync**: If user mentions issues from `/speckit.analyze` (vague acceptance criteria, incomplete requirements)
|
|
119
|
+
|
|
120
|
+
**Enrichment Workflow**:
|
|
121
|
+
|
|
122
|
+
1. **Run with `--auto-enrich`**: Execute `specfact plan review --auto-enrich` to automatically fix common issues
|
|
123
|
+
2. **Review enrichment results**: Analyze what was enhanced and verify improvements
|
|
124
|
+
3. **LLM reasoning**: Use your reasoning to:
|
|
125
|
+
- Verify enhancements are contextually appropriate
|
|
126
|
+
- Identify any generic improvements that need refinement
|
|
127
|
+
- Suggest specific manual improvements for edge cases
|
|
128
|
+
4. **Follow-up enrichment**: If auto-enrichment made generic improvements, use CLI commands to refine them:
|
|
129
|
+
- `specfact plan update-feature` to add specific file paths, method names, or component references to feature-level acceptance criteria
|
|
130
|
+
- `specfact plan update-story` to refine story-level acceptance criteria with specific actions, method calls, and testable assertions
|
|
131
|
+
- `specfact plan update-feature` to add domain-specific constraints
|
|
132
|
+
|
|
133
|
+
**Example Enrichment Flow**:
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
# Step 1: Auto-enrich to fix common issues
|
|
137
|
+
specfact plan review --auto-enrich --plan <plan_path>
|
|
138
|
+
|
|
139
|
+
# Step 2: LLM analyzes results and suggests refinements
|
|
140
|
+
# "Auto-enrichment enhanced 8 acceptance criteria. The Given/When/Then format is good,
|
|
141
|
+
# but we should make the 'When' clause more specific. For example, 'When they interact
|
|
142
|
+
# with the system' could be 'When they call the configure() method with valid parameters'."
|
|
143
|
+
|
|
144
|
+
# Step 3: Manual refinement using CLI commands
|
|
145
|
+
specfact plan update-feature --key FEATURE-001 --acceptance "Given a developer wants to configure Git operations, When they call the configure() method with valid parameters, Then the configuration is validated and stored" --plan <plan_path>
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
## Operating Constraints
|
|
149
|
+
|
|
150
|
+
**STRICTLY READ-WRITE**: This command modifies plan bundle metadata and content. All updates must be performed by the specfact CLI.
|
|
151
|
+
|
|
152
|
+
**Command**: `specfact plan review [--auto-enrich]`
|
|
153
|
+
|
|
154
|
+
**Mode Auto-Detection**: The CLI automatically detects operational mode (CI/CD or CoPilot) based on environment. No need to specify `--mode` flag. Mode is detected from:
|
|
155
|
+
|
|
156
|
+
- Environment variables (`SPECFACT_MODE`)
|
|
157
|
+
- CoPilot API availability
|
|
158
|
+
- IDE integration (VS Code/Cursor with CoPilot)
|
|
159
|
+
- Defaults to CI/CD mode if none detected
|
|
160
|
+
|
|
161
|
+
**Mode-Specific Behavior**:
|
|
162
|
+
|
|
163
|
+
- **CLI Mode**: Interactive Q&A with free-text input, simple multiple-choice when applicable
|
|
164
|
+
- **Copilot Mode**: Reasoning-enhanced questions with recommendations, similar to Spec-Kit clarify
|
|
165
|
+
|
|
166
|
+
**Auto-Enrichment Feature**:
|
|
167
|
+
|
|
168
|
+
The `--auto-enrich` flag automatically enhances the plan bundle before scanning for ambiguities:
|
|
169
|
+
|
|
170
|
+
- **Vague acceptance criteria** (e.g., "is implemented", "is functional", "works") → Converted to testable Given/When/Then format
|
|
171
|
+
- **Incomplete requirements** (e.g., "System MUST Helper class") → Enhanced with verbs and actions (e.g., "System MUST provide a Helper class for [feature] operations")
|
|
172
|
+
- **Generic tasks** (e.g., "Implement [story]") → Enhanced with implementation details (file paths, methods, components)
|
|
173
|
+
|
|
174
|
+
**⚠️ IMPORTANT LIMITATION**: Auto-enrichment creates **generic templates** (e.g., "Given a user wants to use {story}, When they interact with the system, Then {story} works correctly"). These are NOT testable and MUST be refined by LLM with code-specific details. The LLM MUST automatically refine all generic criteria after auto-enrichment runs (see "LLM Post-Enrichment Analysis & Automatic Refinement" section below).
|
|
175
|
+
|
|
176
|
+
**When to Use Auto-Enrichment**:
|
|
177
|
+
|
|
178
|
+
- **Before first review**: Use `--auto-enrich` when reviewing a plan bundle imported from code or Spec-Kit to automatically fix common quality issues
|
|
179
|
+
- **After sync from Spec-Kit**: If `/speckit.analyze` reports vague acceptance criteria or incomplete requirements, use `--auto-enrich` to fix them automatically
|
|
180
|
+
- **Before promotion**: Use `--auto-enrich` to improve plan quality before promoting from `draft` to `review` stage
|
|
181
|
+
- **LLM Reasoning**: In Copilot mode, analyze the plan bundle first, then decide if auto-enrichment would be beneficial based on the coverage summary
|
|
182
|
+
|
|
183
|
+
## What This Command Does
|
|
184
|
+
|
|
185
|
+
The `specfact plan review` command:
|
|
186
|
+
|
|
187
|
+
1. **Analyzes** the plan bundle for ambiguities using a structured taxonomy
|
|
188
|
+
2. **Identifies** missing information, unclear requirements, and unknowns
|
|
189
|
+
3. **Asks** targeted questions (max 5 per session) to resolve ambiguities
|
|
190
|
+
4. **Integrates** answers back into the plan bundle incrementally
|
|
191
|
+
5. **Validates** plan bundle structure after each update
|
|
192
|
+
6. **Reports** coverage summary and promotion readiness
|
|
193
|
+
|
|
194
|
+
## Execution Steps
|
|
195
|
+
|
|
196
|
+
### ⚠️ **CRITICAL: Copilot Mode Workflow**
|
|
197
|
+
|
|
198
|
+
In Copilot mode, follow this three-phase workflow:
|
|
199
|
+
|
|
200
|
+
1. **Phase 1: Get Questions** - Execute `specfact plan review --list-questions` to get questions in JSON format
|
|
201
|
+
2. **Phase 2: Ask User** - Present questions to user one at a time, collect answers
|
|
202
|
+
3. **Phase 3: Feed Answers** - Write answers to a JSON file, then execute `specfact plan review --answers answers.json` to integrate answers
|
|
203
|
+
|
|
204
|
+
**⚠️ IMPORTANT**: Always use a JSON file path (not inline JSON string) to avoid parsing issues and ensure proper formatting.
|
|
205
|
+
|
|
206
|
+
**Never create clarifications directly in YAML**. Always use the CLI to integrate answers.
|
|
207
|
+
|
|
208
|
+
### 1. Parse Arguments and Load Plan Bundle
|
|
209
|
+
|
|
210
|
+
**Parse user input** to extract:
|
|
211
|
+
|
|
212
|
+
- Plan bundle path (default: active plan or latest in `.specfact/plans/`)
|
|
213
|
+
- Max questions per session (default: 5)
|
|
214
|
+
- Category focus (optional, to focus on specific taxonomy category)
|
|
215
|
+
- Auto-enrichment flag (if user requests automatic enrichment or if plan appears to need it)
|
|
216
|
+
|
|
217
|
+
**LLM Reasoning for Auto-Enrichment**:
|
|
218
|
+
|
|
219
|
+
In Copilot mode, you should **reason about whether auto-enrichment is needed**:
|
|
220
|
+
|
|
221
|
+
1. **Check if plan was imported from code or Spec-Kit**: If the user mentions "after sync" or "imported from code", auto-enrichment is likely beneficial
|
|
222
|
+
2. **Analyze plan quality indicators**: If you see patterns like:
|
|
223
|
+
- Vague acceptance criteria ("is implemented", "is functional")
|
|
224
|
+
- Incomplete requirements ("System MUST Helper class")
|
|
225
|
+
- Generic tasks without implementation details
|
|
226
|
+
Then suggest using `--auto-enrich`
|
|
227
|
+
3. **User intent**: If user explicitly requests "enrich", "improve", "fix vague criteria", or mentions issues from `/speckit.analyze`, use `--auto-enrich`
|
|
228
|
+
|
|
229
|
+
**Decision Flow**:
|
|
230
|
+
|
|
231
|
+
```text
|
|
232
|
+
IF user mentions "after sync" OR "imported" OR "vague criteria" OR "incomplete requirements":
|
|
233
|
+
→ Use --auto-enrich flag
|
|
234
|
+
ELSE IF plan appears to have quality issues (based on initial scan):
|
|
235
|
+
→ Suggest --auto-enrich to user and wait for confirmation
|
|
236
|
+
ELSE:
|
|
237
|
+
→ Proceed with normal review (no auto-enrichment)
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
**WAIT STATE**: If plan path is unclear, ask the user:
|
|
241
|
+
|
|
242
|
+
```text
|
|
243
|
+
"Which plan bundle would you like to review?
|
|
244
|
+
(Enter path, 'active plan', or 'latest')
|
|
245
|
+
[WAIT FOR USER RESPONSE - DO NOT CONTINUE]"
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
**In Copilot Mode**: Use `--list-questions` to get questions in structured format:
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
# Get questions as JSON (for Copilot mode)
|
|
252
|
+
specfact plan review --list-questions --plan <plan_path> --max-questions 5
|
|
253
|
+
|
|
254
|
+
# With auto-enrichment (if needed)
|
|
255
|
+
specfact plan review --auto-enrich --list-questions --plan <plan_path> --max-questions 5
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
**In CI/CD Mode**: Use `--non-interactive` flag:
|
|
259
|
+
|
|
260
|
+
```bash
|
|
261
|
+
# Non-interactive mode (for automation)
|
|
262
|
+
specfact plan review --non-interactive --plan <plan_path> --answers '{"Q001": "answer1", "Q002": "answer2"}'
|
|
263
|
+
|
|
264
|
+
# With auto-enrichment
|
|
265
|
+
specfact plan review --auto-enrich --non-interactive --plan <plan_path> --answers '{"Q001": "answer1"}'
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
**Capture from CLI**:
|
|
269
|
+
|
|
270
|
+
- Plan bundle loaded successfully
|
|
271
|
+
- **Deduplication summary**: "✓ Removed N duplicate features from plan bundle" (if duplicates were found)
|
|
272
|
+
- Current stage (should be `draft` for review)
|
|
273
|
+
- Existing clarifications (if any)
|
|
274
|
+
- **Auto-enrichment summary** (if `--auto-enrich` was used):
|
|
275
|
+
- Features updated
|
|
276
|
+
- Stories updated
|
|
277
|
+
- Acceptance criteria enhanced
|
|
278
|
+
- Requirements enhanced
|
|
279
|
+
- Tasks enhanced
|
|
280
|
+
- List of changes made
|
|
281
|
+
- Questions list (if `--list-questions` used)
|
|
282
|
+
- **Coverage Summary**: Pay special attention to Partial categories - they indicate areas that could be enriched but don't block promotion
|
|
283
|
+
|
|
284
|
+
**⚠️ CRITICAL: Automatic Refinement After Auto-Enrichment**:
|
|
285
|
+
|
|
286
|
+
**If auto-enrichment was used, you MUST automatically refine generic acceptance criteria BEFORE proceeding with questions.**
|
|
287
|
+
|
|
288
|
+
**Step 1: Identify Generic Criteria** (from auto-enrichment output):
|
|
289
|
+
|
|
290
|
+
Look for patterns in the "Changes made" list:
|
|
291
|
+
|
|
292
|
+
- Generic templates: "Given a user wants to use {story}, When they interact with the system, Then {story} works correctly"
|
|
293
|
+
- Vague actions: "interact with the system", "perform the action", "access the system"
|
|
294
|
+
- Vague outcomes: "works correctly", "is functional", "works as expected"
|
|
295
|
+
|
|
296
|
+
**Step 2: Research Codebase** (for each story with generic criteria):
|
|
297
|
+
|
|
298
|
+
- Find the actual class and method names
|
|
299
|
+
- Identify method signatures and parameters
|
|
300
|
+
- Check test files for actual test patterns
|
|
301
|
+
- Understand return values and assertions
|
|
302
|
+
|
|
303
|
+
**Step 3: Generate Code-Specific Criteria** (replace generic with specific):
|
|
304
|
+
|
|
305
|
+
- Replace "interact with the system" → specific method calls with parameters
|
|
306
|
+
- Replace "works correctly" → specific return values, state changes, or assertions
|
|
307
|
+
- Add class names, method signatures, file paths where relevant
|
|
308
|
+
|
|
309
|
+
**Step 4: Apply Refinements** (use CLI commands):
|
|
310
|
+
|
|
311
|
+
```bash
|
|
312
|
+
# For story-level acceptance criteria, use update-story:
|
|
313
|
+
specfact plan update-story --feature <feature-key> --key <story-key> --acceptance "<refined-code-specific-criteria>" --plan <path>
|
|
314
|
+
|
|
315
|
+
# For feature-level acceptance criteria, use update-feature:
|
|
316
|
+
specfact plan update-feature --key <feature-key> --acceptance "<refined-code-specific-criteria>" --plan <path>
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
**Step 5: Verify** (before proceeding):
|
|
320
|
+
|
|
321
|
+
- All generic criteria replaced with code-specific criteria
|
|
322
|
+
- All criteria mention specific methods, classes, or file paths
|
|
323
|
+
- All criteria are testable (can be verified with automated tests)
|
|
324
|
+
|
|
325
|
+
**Only after Step 5 is complete, proceed with questions.**
|
|
326
|
+
|
|
327
|
+
**Understanding Deduplication**:
|
|
328
|
+
|
|
329
|
+
The CLI automatically deduplicates features during review using normalized key matching:
|
|
330
|
+
|
|
331
|
+
1. **Exact matches**: Features with identical normalized keys are automatically deduplicated
|
|
332
|
+
- Example: `FEATURE-001` and `001_FEATURE_NAME` normalize to the same key
|
|
333
|
+
2. **Prefix matches**: Abbreviated class names vs full Spec-Kit directory names
|
|
334
|
+
- Example: `FEATURE-IDEINTEGRATION` (from code analysis) vs `041_IDE_INTEGRATION_SYSTEM` (from Spec-Kit)
|
|
335
|
+
- Only matches when at least one key has a numbered prefix (Spec-Kit origin) to avoid false positives
|
|
336
|
+
- Requires minimum 10 characters, 6+ character difference, and <75% length ratio
|
|
337
|
+
|
|
338
|
+
**LLM Semantic Deduplication**:
|
|
339
|
+
|
|
340
|
+
After automated deduplication, you should review the plan bundle for **semantic/logical duplicates** that automated matching might miss:
|
|
341
|
+
|
|
342
|
+
1. **Review feature titles and descriptions**: Look for features that represent the same functionality with different names
|
|
343
|
+
- Example: "Git Operations Manager" vs "Git Operations Handler" (both handle git operations)
|
|
344
|
+
- Example: "Telemetry Settings" vs "Telemetry Configuration" (both configure telemetry)
|
|
345
|
+
2. **Check feature stories**: Features with overlapping or identical user stories may be duplicates
|
|
346
|
+
3. **Analyze acceptance criteria**: Features with similar acceptance criteria covering the same functionality
|
|
347
|
+
4. **Check code references**: If multiple features reference the same code files/modules, they might be the same feature
|
|
348
|
+
5. **Suggest consolidation**: When semantic duplicates are found:
|
|
349
|
+
- Use `specfact plan update-feature` to merge information into one feature
|
|
350
|
+
- Use `specfact plan add-feature` to create a consolidated feature if needed
|
|
351
|
+
- Document which features were consolidated and why
|
|
352
|
+
|
|
353
|
+
**Example Semantic Duplicate Detection**:
|
|
354
|
+
|
|
355
|
+
```text
|
|
356
|
+
After review, analyze the plan bundle and identify:
|
|
357
|
+
- Features with similar titles but different keys
|
|
358
|
+
- Features covering the same code modules
|
|
359
|
+
- Features with overlapping user stories or acceptance criteria
|
|
360
|
+
- Features that represent the same functionality
|
|
361
|
+
|
|
362
|
+
If semantic duplicates are found, suggest consolidation:
|
|
363
|
+
"Found semantic duplicates: FEATURE-GITOPERATIONS and FEATURE-GITOPERATIONSHANDLER
|
|
364
|
+
both cover git operations. Should I consolidate these into a single feature?"
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
**Understanding Auto-Enrichment Output**:
|
|
368
|
+
|
|
369
|
+
When `--auto-enrich` is used, the CLI will output:
|
|
370
|
+
|
|
371
|
+
```bash
|
|
372
|
+
Auto-enriching plan bundle (enhancing vague acceptance criteria, incomplete requirements, generic tasks)...
|
|
373
|
+
✓ Auto-enriched plan bundle: 2 features, 5 stories updated
|
|
374
|
+
- Enhanced 8 acceptance criteria
|
|
375
|
+
- Enhanced 3 requirements
|
|
376
|
+
- Enhanced 4 tasks
|
|
377
|
+
|
|
378
|
+
Changes made:
|
|
379
|
+
- Feature FEATURE-001: Enhanced requirement 'System MUST Helper class' → 'System MUST provide a Helper class for git operations operations'
|
|
380
|
+
- Story STORY-001: Enhanced acceptance criteria 'is implemented' → 'Given a developer wants to use configure git operations, When they interact with the system, Then configure git operations is functional and verified'
|
|
381
|
+
...
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
**Understanding CLI Output**:
|
|
385
|
+
|
|
386
|
+
When the CLI reports "No critical ambiguities detected. Plan is ready for promotion" but shows ⚠️ Partial categories, this means:
|
|
387
|
+
|
|
388
|
+
- **Critical categories** (Functional Scope, Feature Completeness, Constraints) are all Clear or Partial (not Missing)
|
|
389
|
+
- **Partial categories** are not critical enough to block promotion, but enrichment would improve plan quality
|
|
390
|
+
- The plan can be promoted, but consider enriching Partial categories for better completeness
|
|
391
|
+
|
|
392
|
+
**LLM Post-Enrichment Analysis & Automatic Refinement**:
|
|
393
|
+
|
|
394
|
+
**⚠️ CRITICAL**: After auto-enrichment runs, you MUST automatically refine the generic acceptance criteria with code-specific, testable details. The auto-enrichment creates generic templates (e.g., "Given a user wants to use {story}, When they interact with the system, Then {story} works correctly"), but these are NOT testable. You should IMMEDIATELY replace them with specific, code-based criteria.
|
|
395
|
+
|
|
396
|
+
**Why This Matters**:
|
|
397
|
+
|
|
398
|
+
- **Generic criteria are NOT testable**: "When they interact with the system" cannot be verified
|
|
399
|
+
- **Test-based criteria are better**: "When extract_article_viii_evidence() is called" is specific and testable
|
|
400
|
+
- **Auto-enrichment makes things worse**: It replaces test-based criteria with generic templates
|
|
401
|
+
- **LLM reasoning is required**: Only LLM can understand codebase context and create specific criteria
|
|
402
|
+
|
|
403
|
+
**Automatic Refinement Workflow (MANDATORY after auto-enrichment)**:
|
|
404
|
+
|
|
405
|
+
1. **Parse auto-enrichment output**: Identify which acceptance criteria were enhanced (look for generic patterns like "interact with the system", "works correctly", "is functional and verified")
|
|
406
|
+
2. **Research codebase context**: For each enhanced story, find the actual:
|
|
407
|
+
- Class names and method signatures (e.g., `ContractFirstTestManager.extract_article_viii_evidence()`)
|
|
408
|
+
- File paths and module structure (e.g., `src/specfact_cli/enrichers/plan_enricher.py`)
|
|
409
|
+
- Test patterns and validation logic (check test files for actual test cases)
|
|
410
|
+
- Actual behavior and return values (e.g., returns `dict` with `'status'` key)
|
|
411
|
+
3. **Generate code-specific criteria**: Replace generic templates with specific, testable criteria:
|
|
412
|
+
- **Generic (BAD)**: "Given a user wants to use as a developer, i can configure contract first test manager, When they interact with the system, Then as a developer, i can configure contract first test manager works correctly"
|
|
413
|
+
- **Code-specific (GOOD)**: "Given a ContractFirstTestManager instance is available, When extract_article_viii_evidence(repo_path: Path) is called, Then the method returns a dict with 'status' key equal to 'PASS' or 'FAIL' and 'frameworks_detected' list"
|
|
414
|
+
4. **Apply refinements automatically**: Use `specfact plan update-feature` to replace ALL generic criteria with code-specific ones BEFORE asking questions
|
|
415
|
+
5. **Verify testability**: Ensure all refined criteria can be verified with automated tests (include specific method names, parameters, return values, assertions)
|
|
416
|
+
|
|
417
|
+
**Example Automatic Refinement Process**:
|
|
418
|
+
|
|
419
|
+
```markdown
|
|
420
|
+
1. Auto-enrichment enhanced: "is implemented" → "Given a user wants to use configure git operations, When they interact with the system, Then configure git operations works correctly"
|
|
421
|
+
|
|
422
|
+
2. LLM Analysis:
|
|
423
|
+
- Story: "As a developer, I can configure Contract First Test Manager"
|
|
424
|
+
- Feature: "Contract First Test Manager"
|
|
425
|
+
- Research codebase: Find `ContractFirstTestManager` class and its methods
|
|
426
|
+
|
|
427
|
+
3. Codebase Research:
|
|
428
|
+
- Find: `src/specfact_cli/enrichers/plan_enricher.py` with `PlanEnricher` class
|
|
429
|
+
- Methods: `enrich_plan()`, `_enhance_vague_acceptance_criteria()`, etc.
|
|
430
|
+
- Test patterns: Check test files for actual test cases
|
|
431
|
+
|
|
432
|
+
4. Generate Code-Specific Criteria:
|
|
433
|
+
- "Given a developer wants to configure Contract First Test Manager, When they call `PlanEnricher.enrich_plan(plan_bundle: PlanBundle)` with a valid plan bundle, Then the method returns an enrichment summary dict with 'features_updated' and 'stories_updated' counts"
|
|
434
|
+
|
|
435
|
+
5. Apply via CLI:
|
|
436
|
+
```bash
|
|
437
|
+
# For story-level acceptance criteria:
|
|
438
|
+
specfact plan update-story --feature FEATURE-CONTRACTFIRSTTESTMANAGER --key STORY-001 --acceptance "Given a developer wants to configure Contract First Test Manager, When they call PlanEnricher.enrich_plan(plan_bundle: PlanBundle) with a valid plan bundle, Then the method returns an enrichment summary dict with 'features_updated' and 'stories_updated' counts" --plan <path>
|
|
439
|
+
|
|
440
|
+
# For feature-level acceptance criteria:
|
|
441
|
+
specfact plan update-feature --key FEATURE-CONTRACTFIRSTTESTMANAGER --acceptance "Given a developer wants to configure Contract First Test Manager, When they call PlanEnricher.enrich_plan(plan_bundle: PlanBundle) with a valid plan bundle, Then the method returns an enrichment summary dict with 'features_updated' and 'stories_updated' counts" --plan <path>
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
**When to Apply Automatic Refinement**:
|
|
445
|
+
|
|
446
|
+
- **MANDATORY after auto-enrichment**: If `--auto-enrich` was used, you MUST automatically refine ALL generic criteria BEFORE asking questions. Do not proceed with questions until generic criteria are replaced.
|
|
447
|
+
- **During review**: When questions ask about vague acceptance criteria, provide code-specific refinements immediately
|
|
448
|
+
- **Before promotion**: Ensure all acceptance criteria are code-specific and testable (no generic placeholders)
|
|
449
|
+
|
|
450
|
+
**Refinement Priority**:
|
|
451
|
+
|
|
452
|
+
1. **High Priority (Do First)**: Criteria containing generic patterns:
|
|
453
|
+
- "interact with the system"
|
|
454
|
+
- "works correctly" / "works as expected" / "is functional"
|
|
455
|
+
- "perform the action"
|
|
456
|
+
- "access the system"
|
|
457
|
+
- Any criteria that doesn't mention specific methods, classes, or file paths
|
|
458
|
+
|
|
459
|
+
2. **Medium Priority**: Criteria that are testable but could be more specific:
|
|
460
|
+
- Add method signatures
|
|
461
|
+
- Add parameter types
|
|
462
|
+
- Add return value assertions
|
|
463
|
+
- Add file path references
|
|
464
|
+
|
|
465
|
+
3. **Low Priority**: Criteria that are already code-specific:
|
|
466
|
+
- Preserve test-based criteria (don't replace with generic)
|
|
467
|
+
- Only enhance if missing important details
|
|
468
|
+
|
|
469
|
+
**Refinement Quality Checklist**:
|
|
470
|
+
|
|
471
|
+
- ✅ **Specific method names**: Include actual class.method() signatures
|
|
472
|
+
- ✅ **Specific file paths**: Reference actual code locations when relevant
|
|
473
|
+
- ✅ **Testable outcomes**: Include specific return values, state changes, or observable behaviors
|
|
474
|
+
- ✅ **Domain-specific**: Use terminology from the actual codebase
|
|
475
|
+
- ✅ **No generic placeholders**: Avoid "interact with the system", "works correctly", "is functional"
|
|
476
|
+
|
|
477
|
+
### 2. Get Questions from CLI (Copilot Mode) or Analyze Directly (Interactive Mode)
|
|
478
|
+
|
|
479
|
+
**⚠️ CRITICAL**: In Copilot mode, you MUST use `--list-questions` to get questions from the CLI, then ask the user, then feed answers back via `--answers`.
|
|
480
|
+
|
|
481
|
+
**Step 2a: Get Questions (Copilot Mode)**:
|
|
482
|
+
|
|
483
|
+
```bash
|
|
484
|
+
# Execute CLI to get questions in JSON format
|
|
485
|
+
specfact plan review --list-questions --plan <plan_path> --max-questions 5
|
|
486
|
+
```
|
|
487
|
+
|
|
488
|
+
**Parse JSON output**:
|
|
489
|
+
|
|
490
|
+
```json
|
|
491
|
+
{
|
|
492
|
+
"questions": [
|
|
493
|
+
{
|
|
494
|
+
"id": "Q001",
|
|
495
|
+
"category": "Feature/Story Completeness",
|
|
496
|
+
"question": "What user stories are needed for feature FEATURE-IDEINTEGRATION?",
|
|
497
|
+
"impact": 0.9,
|
|
498
|
+
"uncertainty": 0.8,
|
|
499
|
+
"related_sections": ["features.FEATURE-IDEINTEGRATION.stories"]
|
|
500
|
+
},
|
|
501
|
+
...
|
|
502
|
+
],
|
|
503
|
+
"total": 5
|
|
504
|
+
}
|
|
505
|
+
```
|
|
506
|
+
|
|
507
|
+
**Step 2b: Analyze Plan Bundle for Ambiguities (Interactive Mode Only)**:
|
|
508
|
+
|
|
509
|
+
**CLI Mode**: The CLI performs structured ambiguity scan using taxonomy categories:
|
|
510
|
+
|
|
511
|
+
1. **Functional Scope & Behavior**
|
|
512
|
+
- Core user goals & success criteria
|
|
513
|
+
- Explicit out-of-scope declarations
|
|
514
|
+
- User roles / personas differentiation
|
|
515
|
+
|
|
516
|
+
2. **Domain & Data Model**
|
|
517
|
+
- Entities, attributes, relationships
|
|
518
|
+
- Identity & uniqueness rules
|
|
519
|
+
- Lifecycle/state transitions
|
|
520
|
+
- Data volume / scale assumptions
|
|
521
|
+
|
|
522
|
+
3. **Interaction & UX Flow**
|
|
523
|
+
- Critical user journeys / sequences
|
|
524
|
+
- Error/empty/loading states
|
|
525
|
+
- Accessibility or localization notes
|
|
526
|
+
|
|
527
|
+
4. **Non-Functional Quality Attributes**
|
|
528
|
+
- Performance (latency, throughput targets)
|
|
529
|
+
- Scalability (horizontal/vertical, limits)
|
|
530
|
+
- Reliability & availability (uptime, recovery expectations)
|
|
531
|
+
- Observability (logging, metrics, tracing signals)
|
|
532
|
+
- Security & privacy (authN/Z, data protection, threat assumptions)
|
|
533
|
+
- Compliance / regulatory constraints (if any)
|
|
534
|
+
|
|
535
|
+
5. **Integration & External Dependencies**
|
|
536
|
+
- External services/APIs and failure modes
|
|
537
|
+
- Data import/export formats
|
|
538
|
+
- Protocol/versioning assumptions
|
|
539
|
+
|
|
540
|
+
6. **Edge Cases & Failure Handling**
|
|
541
|
+
- Negative scenarios
|
|
542
|
+
- Rate limiting / throttling
|
|
543
|
+
- Conflict resolution (e.g., concurrent edits)
|
|
544
|
+
|
|
545
|
+
7. **Constraints & Tradeoffs**
|
|
546
|
+
- Technical constraints (language, storage, hosting)
|
|
547
|
+
- Explicit tradeoffs or rejected alternatives
|
|
548
|
+
|
|
549
|
+
8. **Terminology & Consistency**
|
|
550
|
+
- Canonical glossary terms
|
|
551
|
+
- Avoided synonyms / deprecated terms
|
|
552
|
+
|
|
553
|
+
9. **Completion Signals**
|
|
554
|
+
- Acceptance criteria testability
|
|
555
|
+
- Measurable Definition of Done style indicators
|
|
556
|
+
|
|
557
|
+
10. **Feature/Story Completeness**
|
|
558
|
+
- Missing acceptance criteria
|
|
559
|
+
- Unclear story outcomes
|
|
560
|
+
- Incomplete feature constraints
|
|
561
|
+
|
|
562
|
+
**For each category**, mark status: **Clear** / **Partial** / **Missing**
|
|
563
|
+
|
|
564
|
+
**Copilot Mode**: In addition to CLI analysis, you can:
|
|
565
|
+
|
|
566
|
+
- Research codebase for additional context
|
|
567
|
+
- Analyze similar implementations for best practices
|
|
568
|
+
- Provide reasoning for question prioritization
|
|
569
|
+
|
|
570
|
+
### 3. Generate Question Queue
|
|
571
|
+
|
|
572
|
+
**Prioritize questions** by (Impact × Uncertainty) heuristic:
|
|
573
|
+
|
|
574
|
+
- Maximum 5 questions per session
|
|
575
|
+
- Only include questions whose answers materially impact:
|
|
576
|
+
- Architecture decisions
|
|
577
|
+
- Data modeling
|
|
578
|
+
- Task decomposition
|
|
579
|
+
- Test design
|
|
580
|
+
- UX behavior
|
|
581
|
+
- Operational readiness
|
|
582
|
+
- Compliance validation
|
|
583
|
+
|
|
584
|
+
**Exclude**:
|
|
585
|
+
|
|
586
|
+
- Questions already answered in existing clarifications
|
|
587
|
+
- Trivial stylistic preferences
|
|
588
|
+
- Plan-level execution details (unless blocking correctness)
|
|
589
|
+
- Speculative tech stack questions (unless blocking functional clarity)
|
|
590
|
+
|
|
591
|
+
**If no valid questions exist**, analyze the coverage summary:
|
|
592
|
+
|
|
593
|
+
**Understanding Coverage Status**:
|
|
594
|
+
|
|
595
|
+
- **✅ Clear**: Category has no ambiguities or all findings are resolved
|
|
596
|
+
- **⚠️ Partial**: Category has some findings, but they're not high-priority enough to generate questions (low impact × uncertainty score)
|
|
597
|
+
- **❌ Missing**: Category has critical findings that block promotion (high impact × uncertainty)
|
|
598
|
+
|
|
599
|
+
**Critical vs Important Categories**:
|
|
600
|
+
|
|
601
|
+
- **Critical categories** (block promotion if Missing):
|
|
602
|
+
- Functional Scope & Behavior
|
|
603
|
+
- Feature/Story Completeness
|
|
604
|
+
- Constraints & Tradeoffs
|
|
605
|
+
|
|
606
|
+
- **Important categories** (warn if Missing or Partial, but don't block):
|
|
607
|
+
- Domain & Data Model
|
|
608
|
+
- Interaction & UX Flow
|
|
609
|
+
- Integration & External Dependencies
|
|
610
|
+
- Edge Cases & Failure Handling
|
|
611
|
+
- Completion Signals
|
|
612
|
+
|
|
613
|
+
**When No Questions Are Generated**:
|
|
614
|
+
|
|
615
|
+
1. **If Partial categories exist**: Explain what "Partial" means and provide enrichment guidance:
|
|
616
|
+
- **Partial = Some gaps exist but not critical enough for questions**
|
|
617
|
+
- **Action**: Use CLI commands to manually enrich these categories (see "Manual Enrichment" section below)
|
|
618
|
+
- **Example**: If "Completion Signals: Partial", many stories have acceptance criteria but they're not testable (missing "must", "should", "verify", "validate", "check" keywords)
|
|
619
|
+
|
|
620
|
+
2. **If Missing critical categories**: Report warning and suggest using `specfact plan update-idea` or `specfact plan update-feature` to fill gaps
|
|
621
|
+
- **Note**: Idea section is OPTIONAL - provides business context, not technical implementation
|
|
622
|
+
- Report: "No high-priority questions generated, but missing critical categories detected. Consider using `specfact plan update-idea` to add optional business metadata."
|
|
623
|
+
|
|
624
|
+
3. **If all categories are Clear**: Report: "No critical ambiguities detected. Plan is ready for promotion."
|
|
625
|
+
|
|
626
|
+
**Spec-Kit Sync Integration**:
|
|
627
|
+
|
|
628
|
+
If the user mentions they plan to sync to Spec-Kit later (e.g., "I'll sync to spec-kit after review"), you should:
|
|
629
|
+
|
|
630
|
+
1. **Reassure them**: The `specfact sync spec-kit` command automatically generates all required Spec-Kit fields:
|
|
631
|
+
- Frontmatter (Feature Branch, Created date, Status) in spec.md
|
|
632
|
+
- INVSEST criteria in spec.md
|
|
633
|
+
- Scenarios (Primary, Alternate, Exception, Recovery) in spec.md
|
|
634
|
+
- Constitution Check (Article VII, VIII, IX) in plan.md
|
|
635
|
+
- Phases (Phase 0, 1, 2, -1) in plan.md and tasks.md
|
|
636
|
+
- Technology Stack in plan.md
|
|
637
|
+
- Story mappings ([US1], [US2]) in tasks.md
|
|
638
|
+
|
|
639
|
+
2. **Focus on plan bundle enrichment**: During review, focus on enriching the plan bundle itself (acceptance criteria, constraints, stories) rather than worrying about Spec-Kit-specific formatting
|
|
640
|
+
|
|
641
|
+
3. **Explain the workflow**:
|
|
642
|
+
- Review enriches plan bundle → Sync generates complete Spec-Kit artifacts → Optional customization if needed
|
|
643
|
+
|
|
644
|
+
**Enrichment Strategy for Partial Categories**:
|
|
645
|
+
|
|
646
|
+
When categories are marked as "Partial", use this two-phase approach:
|
|
647
|
+
|
|
648
|
+
**Phase 1: Automatic Enrichment** (use `--auto-enrich` flag):
|
|
649
|
+
|
|
650
|
+
- **Completion Signals (Partial)**: Auto-enriches vague acceptance criteria ("is implemented", "is functional") → Given/When/Then format
|
|
651
|
+
- **Feature Completeness (Partial)**: Auto-enriches incomplete requirements ("System MUST Helper class") → Complete requirements with verbs
|
|
652
|
+
- **Feature Completeness (Partial)**: Auto-enriches generic tasks → Tasks with implementation details
|
|
653
|
+
|
|
654
|
+
**Phase 2: LLM-Enhanced Manual Refinement** (after auto-enrichment):
|
|
655
|
+
|
|
656
|
+
After auto-enrichment, use LLM reasoning to refine generic improvements:
|
|
657
|
+
|
|
658
|
+
- **Completion Signals (Partial)**: Review auto-enriched Given/When/Then scenarios and refine with specific actions:
|
|
659
|
+
- Generic: "When they interact with the system"
|
|
660
|
+
- Refined: "When they call the `configure()` method with valid parameters"
|
|
661
|
+
- Use: `specfact plan update-story --feature <feature-key> --key <story-key> --acceptance "<refined criteria>" --plan <path>` for story-level criteria
|
|
662
|
+
- Use: `specfact plan update-feature --key <key> --acceptance "<refined criteria>" --plan <path>` for feature-level criteria
|
|
663
|
+
|
|
664
|
+
- **Edge Cases (Partial)**: Add domain-specific edge cases:
|
|
665
|
+
- Use `specfact plan update-feature` to add edge case acceptance criteria
|
|
666
|
+
- Add keywords like "edge", "corner", "boundary", "limit", "invalid", "null", "empty"
|
|
667
|
+
- Example: Add "Given invalid Git repository path, When configure() is called, Then system returns clear error message"
|
|
668
|
+
|
|
669
|
+
- **Integration (Partial)**: Add specific external dependency constraints:
|
|
670
|
+
- Use `specfact plan update-feature --constraints` to add external dependency constraints
|
|
671
|
+
- Example: `--constraints "API rate limits: 100 req/min, Timeout: 30s, Retry: 3 attempts"`
|
|
672
|
+
|
|
673
|
+
- **Data Model (Partial)**: Add specific data model constraints:
|
|
674
|
+
- Use `specfact plan update-feature --constraints` to add data model constraints
|
|
675
|
+
- Example: `--constraints "Entity uniqueness: email must be unique, Max records: 10,000 per user"`
|
|
676
|
+
|
|
677
|
+
- **Interaction/UX (Partial)**: Add specific error handling scenarios:
|
|
678
|
+
- Use `specfact plan update-feature` to add error handling acceptance criteria
|
|
679
|
+
- Add keywords like "error", "empty", "invalid", "validation", "failure"
|
|
680
|
+
- Example: Add "Given user submits invalid input, When validation runs, Then system displays clear error message with field-specific guidance"
|
|
681
|
+
|
|
682
|
+
**LLM Reasoning for Refinement**:
|
|
683
|
+
|
|
684
|
+
When refining auto-enriched content, consider:
|
|
685
|
+
|
|
686
|
+
1. **Context from codebase**: Research the actual codebase structure to suggest accurate file paths, method names, and component references
|
|
687
|
+
2. **Domain knowledge**: Use domain-specific terminology and patterns appropriate for the feature
|
|
688
|
+
3. **Testability**: Ensure refined acceptance criteria are truly testable (can be verified with automated tests)
|
|
689
|
+
4. **Specificity**: Replace generic placeholders with specific, actionable details
|
|
690
|
+
|
|
691
|
+
### 4. Sequential Questioning Loop
|
|
692
|
+
|
|
693
|
+
**⚠️ CRITICAL**: In Copilot mode, you MUST:
|
|
694
|
+
|
|
695
|
+
1. Get questions via `--list-questions` (already done in Step 2a)
|
|
696
|
+
2. Ask the user each question (this step)
|
|
697
|
+
3. Collect all answers
|
|
698
|
+
4. Feed answers back to CLI via `--answers` (Step 5)
|
|
699
|
+
|
|
700
|
+
**Present EXACTLY ONE question at a time.**
|
|
701
|
+
|
|
702
|
+
#### CLI Mode Format
|
|
703
|
+
|
|
704
|
+
**For multiple-choice questions**:
|
|
705
|
+
|
|
706
|
+
```text
|
|
707
|
+
Q: [Question text]
|
|
708
|
+
|
|
709
|
+
Options:
|
|
710
|
+
A) [Option A description]
|
|
711
|
+
B) [Option B description]
|
|
712
|
+
C) [Option C description]
|
|
713
|
+
D) [Option D description] (if needed)
|
|
714
|
+
E) [Option E description] (if needed)
|
|
715
|
+
F) [Free text answer (<=5 words)]
|
|
716
|
+
|
|
717
|
+
Enter option letter (A-F) or provide your own short answer:
|
|
718
|
+
[WAIT FOR USER RESPONSE - DO NOT CONTINUE]
|
|
719
|
+
```
|
|
720
|
+
|
|
721
|
+
**For short-answer questions**:
|
|
722
|
+
|
|
723
|
+
```text
|
|
724
|
+
Q: [Question text]
|
|
725
|
+
|
|
726
|
+
Format: Short answer (<=5 words)
|
|
727
|
+
|
|
728
|
+
Enter your answer:
|
|
729
|
+
[WAIT FOR USER RESPONSE - DO NOT CONTINUE]
|
|
730
|
+
```
|
|
731
|
+
|
|
732
|
+
#### Copilot Mode Format
|
|
733
|
+
|
|
734
|
+
**For multiple-choice questions**:
|
|
735
|
+
|
|
736
|
+
```text
|
|
737
|
+
**Recommended:** Option [X] - <reasoning (1-2 sentences explaining why this is the best choice)>
|
|
738
|
+
|
|
739
|
+
| Option | Description |
|
|
740
|
+
|--------|-------------|
|
|
741
|
+
| A | <Option A description> |
|
|
742
|
+
| B | <Option B description> |
|
|
743
|
+
| C | <Option C description> |
|
|
744
|
+
| D | <Option D description> (if needed) |
|
|
745
|
+
| E | <Option E description> (if needed) |
|
|
746
|
+
| Short | Provide a different short answer (<=5 words) |
|
|
747
|
+
|
|
748
|
+
You can reply with the option letter (e.g., "A"), accept the recommendation by saying "yes" or "recommended", or provide your own short answer.
|
|
749
|
+
[WAIT FOR USER RESPONSE - DO NOT CONTINUE]
|
|
750
|
+
```
|
|
751
|
+
|
|
752
|
+
**For short-answer questions**:
|
|
753
|
+
|
|
754
|
+
```text
|
|
755
|
+
**Suggested:** <your proposed answer> - <brief reasoning>
|
|
756
|
+
|
|
757
|
+
Format: Short answer (<=5 words). You can accept the suggestion by saying "yes" or "suggested", or provide your own answer.
|
|
758
|
+
[WAIT FOR USER RESPONSE - DO NOT CONTINUE]
|
|
759
|
+
```
|
|
760
|
+
|
|
761
|
+
**After user answers**:
|
|
762
|
+
|
|
763
|
+
- If user replies with "yes", "recommended", or "suggested", use your previously stated recommendation/suggestion as the answer
|
|
764
|
+
- Otherwise, validate the answer (maps to option or fits <=5 word constraint)
|
|
765
|
+
- If ambiguous, ask for quick disambiguation (same question, don't advance)
|
|
766
|
+
- Once satisfactory, record answer and move to next question
|
|
767
|
+
|
|
768
|
+
**Stop asking when**:
|
|
769
|
+
|
|
770
|
+
- All critical ambiguities resolved early (remaining queued items become unnecessary), OR
|
|
771
|
+
- User signals completion ("done", "good", "no more"), OR
|
|
772
|
+
- You reach 5 asked questions
|
|
773
|
+
|
|
774
|
+
**Never reveal future queued questions in advance.**
|
|
775
|
+
|
|
776
|
+
### 5. Feed Answers Back to CLI (Copilot Mode) or Integrate Directly (Interactive Mode)
|
|
777
|
+
|
|
778
|
+
**⚠️ CRITICAL**: In Copilot mode, after collecting all answers from the user, you MUST feed them back to the CLI using `--answers`:
|
|
779
|
+
|
|
780
|
+
**Step 1: Create answers JSON file** (ALWAYS use file, not inline JSON):
|
|
781
|
+
|
|
782
|
+
```bash
|
|
783
|
+
# Create answers.json file with all answers
|
|
784
|
+
cat > answers.json << 'EOF'
|
|
785
|
+
{
|
|
786
|
+
"Q001": "Developers, DevOps engineers",
|
|
787
|
+
"Q002": "Yes",
|
|
788
|
+
"Q003": "Yes",
|
|
789
|
+
"Q004": "Yes",
|
|
790
|
+
"Q005": "Yes"
|
|
791
|
+
}
|
|
792
|
+
EOF
|
|
793
|
+
```
|
|
794
|
+
|
|
795
|
+
**Step 2: Feed answers to CLI** (using file path - RECOMMENDED):
|
|
796
|
+
|
|
797
|
+
```bash
|
|
798
|
+
# Feed all answers back to CLI (Copilot mode) - using file path (RECOMMENDED)
|
|
799
|
+
specfact plan review --plan <plan_path> --answers answers.json
|
|
800
|
+
```
|
|
801
|
+
|
|
802
|
+
**⚠️ AVOID inline JSON strings** - They can cause parsing issues with special characters, quotes, and Rich markup:
|
|
803
|
+
|
|
804
|
+
```bash
|
|
805
|
+
# ❌ NOT RECOMMENDED: Inline JSON string (may have parsing issues)
|
|
806
|
+
specfact plan review --plan <plan_path> --answers '{"Q001": "answer1", "Q002": "answer2"}'
|
|
807
|
+
```
|
|
808
|
+
|
|
809
|
+
**Format**: The `--answers` parameter accepts either:
|
|
810
|
+
|
|
811
|
+
- **✅ JSON file path** (RECOMMENDED): Path to a JSON file containing question_id -> answer mappings
|
|
812
|
+
- More reliable parsing
|
|
813
|
+
- Easier to validate JSON syntax
|
|
814
|
+
- Avoids shell escaping issues
|
|
815
|
+
- Better for complex answers with special characters
|
|
816
|
+
|
|
817
|
+
- **⚠️ JSON string** (NOT RECOMMENDED): Direct JSON object (may have Rich markup parsing issues, shell escaping problems)
|
|
818
|
+
- Only use for simple, single-answer cases
|
|
819
|
+
- Requires careful quote escaping
|
|
820
|
+
- Can fail with special characters
|
|
821
|
+
|
|
822
|
+
**JSON Structure**:
|
|
823
|
+
|
|
824
|
+
- Keys: Question IDs (e.g., "Q001", "Q002")
|
|
825
|
+
- Values: Answer strings (≤5 words recommended)
|
|
826
|
+
|
|
827
|
+
**⚠️ CRITICAL: Boolean-Like Answer Values**:
|
|
828
|
+
|
|
829
|
+
When providing answers that are boolean-like strings (e.g., "Yes", "No", "True", "False", "On", "Off"), ensure they are:
|
|
830
|
+
|
|
831
|
+
1. **Always quoted in JSON**: Use `"Yes"` not `Yes` (JSON requires quotes for strings)
|
|
832
|
+
2. **Provided as strings**: Never use JSON booleans `true`/`false` - always use string values `"Yes"`/`"No"`
|
|
833
|
+
|
|
834
|
+
**❌ WRONG** (causes YAML validation errors):
|
|
835
|
+
|
|
836
|
+
```json
|
|
837
|
+
{
|
|
838
|
+
"Q001": "Developers, DevOps engineers",
|
|
839
|
+
"Q002": true, // ❌ JSON boolean - will cause validation error
|
|
840
|
+
"Q003": Yes // ❌ Unquoted string - invalid JSON
|
|
841
|
+
}
|
|
842
|
+
```
|
|
843
|
+
|
|
844
|
+
**✅ CORRECT**:
|
|
845
|
+
|
|
846
|
+
```json
|
|
847
|
+
{
|
|
848
|
+
"Q001": "Developers, DevOps engineers",
|
|
849
|
+
"Q002": "Yes", // ✅ Quoted string
|
|
850
|
+
"Q003": "No" // ✅ Quoted string
|
|
851
|
+
}
|
|
852
|
+
```
|
|
853
|
+
|
|
854
|
+
**Why This Matters**:
|
|
855
|
+
|
|
856
|
+
- YAML parsers interpret unquoted "Yes", "No", "True", "False", "On", "Off" as boolean values
|
|
857
|
+
- The CLI expects all answers to be strings (validated with `isinstance(answer, str)`)
|
|
858
|
+
- Boolean values in JSON will cause validation errors: "Answer for Q002 must be a non-empty string"
|
|
859
|
+
- The YAML serializer now automatically quotes boolean-like strings, but JSON parsing must still provide strings
|
|
860
|
+
|
|
861
|
+
**Example JSON file** (`answers.json`):
|
|
862
|
+
|
|
863
|
+
```json
|
|
864
|
+
{
|
|
865
|
+
"Q001": "Developers, DevOps engineers",
|
|
866
|
+
"Q002": "Yes",
|
|
867
|
+
"Q003": "Yes",
|
|
868
|
+
"Q004": "Yes",
|
|
869
|
+
"Q005": "Yes"
|
|
870
|
+
}
|
|
871
|
+
```
|
|
872
|
+
|
|
873
|
+
**Usage**:
|
|
874
|
+
|
|
875
|
+
```bash
|
|
876
|
+
# ✅ RECOMMENDED: Using file path
|
|
877
|
+
specfact plan review --plan <plan_path> --answers answers.json
|
|
878
|
+
|
|
879
|
+
# ⚠️ NOT RECOMMENDED: Using JSON string (only for simple cases)
|
|
880
|
+
specfact plan review --plan <plan_path> --answers '{"Q001": "answer1"}'
|
|
881
|
+
```
|
|
882
|
+
|
|
883
|
+
**Validation After Feeding Answers**:
|
|
884
|
+
|
|
885
|
+
After feeding answers, always verify the plan bundle is valid:
|
|
886
|
+
|
|
887
|
+
```bash
|
|
888
|
+
# Verify plan bundle is valid (should not show validation errors)
|
|
889
|
+
specfact plan review --plan <plan_path> --list-questions --max-questions 1
|
|
890
|
+
```
|
|
891
|
+
|
|
892
|
+
If you see validation errors like "Input should be a valid string", check:
|
|
893
|
+
|
|
894
|
+
1. All answers in JSON file are quoted strings (not booleans)
|
|
895
|
+
2. JSON file syntax is valid (use `python3 -m json.tool answers.json` to validate)
|
|
896
|
+
3. No unquoted boolean-like strings ("Yes", "No", "True", "False")
|
|
897
|
+
|
|
898
|
+
**In Interactive Mode**: The CLI automatically integrates answers after each question.
|
|
899
|
+
|
|
900
|
+
**After CLI processes answers** (Copilot mode), the CLI will:
|
|
901
|
+
|
|
902
|
+
1. **Update plan bundle sections** based on answer:
|
|
903
|
+
- **Functional ambiguity** → `features[].acceptance[]` or `idea.narrative`
|
|
904
|
+
- **Data model** → `features[].constraints[]` or new feature
|
|
905
|
+
- **Non-functional** → `features[].constraints[]` or `idea.constraints[]`
|
|
906
|
+
- **Edge cases** → `features[].acceptance[]` or `stories[].acceptance[]`
|
|
907
|
+
- **Terminology** → Normalize across plan bundle
|
|
908
|
+
|
|
909
|
+
2. **Add clarification to plan bundle**:
|
|
910
|
+
- Ensure `clarifications.sessions[]` exists
|
|
911
|
+
- Create session for today's date if not present
|
|
912
|
+
- Add clarification with:
|
|
913
|
+
- Unique ID (Q001, Q002, etc.)
|
|
914
|
+
- Category
|
|
915
|
+
- Question text
|
|
916
|
+
- Answer
|
|
917
|
+
- Integration points (e.g., `["features.FEATURE-001.acceptance"]`)
|
|
918
|
+
- Timestamp
|
|
919
|
+
|
|
920
|
+
3. **Save plan bundle** (CLI automatically saves after each integration)
|
|
921
|
+
|
|
922
|
+
4. **Validate plan bundle**:
|
|
923
|
+
- Structure is valid
|
|
924
|
+
- No contradictory statements
|
|
925
|
+
- Terminology consistency
|
|
926
|
+
- Clarifications properly formatted
|
|
927
|
+
|
|
928
|
+
**Preserve formatting**: Do not reorder unrelated sections; keep heading hierarchy intact.
|
|
929
|
+
|
|
930
|
+
**Keep each integration minimal and testable** (avoid narrative drift).
|
|
931
|
+
|
|
932
|
+
### 6. Validation
|
|
933
|
+
|
|
934
|
+
**After EACH write plus final pass**:
|
|
935
|
+
|
|
936
|
+
- Clarifications session contains exactly one entry per accepted answer (no duplicates)
|
|
937
|
+
- Total asked (accepted) questions ≤ 5
|
|
938
|
+
- Updated sections contain no lingering vague placeholders
|
|
939
|
+
- No contradictory earlier statement remains
|
|
940
|
+
- Plan bundle structure is valid
|
|
941
|
+
- Terminology consistency: same canonical term used across all updated sections
|
|
942
|
+
|
|
943
|
+
### 7. Report Completion
|
|
944
|
+
|
|
945
|
+
**After questioning loop ends or early termination**:
|
|
946
|
+
|
|
947
|
+
**If questions were asked and answered**:
|
|
948
|
+
|
|
949
|
+
```markdown
|
|
950
|
+
✓ Review complete!
|
|
951
|
+
|
|
952
|
+
**Questions Asked**: 3
|
|
953
|
+
**Plan Bundle**: `.specfact/plans/specfact-import-test.2025-11-17T12-21-48.bundle.yaml`
|
|
954
|
+
**Sections Touched**:
|
|
955
|
+
- `features.FEATURE-001.acceptance`
|
|
956
|
+
- `features.FEATURE-002.constraints`
|
|
957
|
+
- `idea.constraints`
|
|
958
|
+
|
|
959
|
+
**Coverage Summary**:
|
|
960
|
+
|
|
961
|
+
| Category | Status | Notes |
|
|
962
|
+
|----------|--------|-------|
|
|
963
|
+
| Functional Scope | ✅ Clear | Resolved (was Partial, now addressed) |
|
|
964
|
+
| Data Model | ✅ Clear | Already sufficient |
|
|
965
|
+
| Non-Functional | ✅ Clear | Resolved (was Missing, now addressed) |
|
|
966
|
+
| Edge Cases | ⚠️ Partial | Deferred (exceeds question quota, see enrichment guide) |
|
|
967
|
+
| Completion Signals | ⚠️ Partial | Some stories need testable acceptance criteria |
|
|
968
|
+
| Terminology | ✅ Clear | Already sufficient |
|
|
969
|
+
|
|
970
|
+
**Next Steps**:
|
|
971
|
+
- Plan is ready for promotion to `review` stage
|
|
972
|
+
- Run: `/specfact-cli/specfact-plan-promote review`
|
|
973
|
+
- Optional: Enrich Partial categories using CLI commands (see Manual Enrichment section)
|
|
974
|
+
```
|
|
975
|
+
|
|
976
|
+
**If no questions were generated but Partial categories exist**:
|
|
977
|
+
|
|
978
|
+
```markdown
|
|
979
|
+
✓ Review analysis complete!
|
|
980
|
+
|
|
981
|
+
**Plan Bundle**: `.specfact/plans/specfact-import-test.2025-11-17T12-21-48.bundle.yaml`
|
|
982
|
+
**Status**: No critical ambiguities detected (all critical categories are Clear)
|
|
983
|
+
|
|
984
|
+
**Coverage Summary**:
|
|
985
|
+
|
|
986
|
+
| Category | Status | Meaning |
|
|
987
|
+
|----------|--------|---------|
|
|
988
|
+
| Functional Scope | ✅ Clear | No ambiguities detected |
|
|
989
|
+
| Data Model | ⚠️ Partial | Some features mention data but lack constraints (not critical) |
|
|
990
|
+
| Interaction/UX | ⚠️ Partial | Some stories mention UX but lack error handling (not critical) |
|
|
991
|
+
| Integration | ⚠️ Partial | Some features mention integration but lack constraints (not critical) |
|
|
992
|
+
| Edge Cases | ⚠️ Partial | Some stories have <3 acceptance criteria (not critical) |
|
|
993
|
+
| Completion Signals | ⚠️ Partial | Some acceptance criteria are not testable (not critical) |
|
|
994
|
+
| Constraints | ✅ Clear | No ambiguities detected |
|
|
995
|
+
|
|
996
|
+
**Understanding "Partial" Status**:
|
|
997
|
+
|
|
998
|
+
⚠️ **Partial** means the category has some gaps, but they're not high-priority enough to generate questions. The plan can still be promoted, but enrichment would improve quality.
|
|
999
|
+
|
|
1000
|
+
**Enrichment Options**:
|
|
1001
|
+
|
|
1002
|
+
- **Automatic Enrichment** (recommended first step): Use `--auto-enrich` flag to automatically fix vague acceptance criteria, incomplete requirements, and generic tasks
|
|
1003
|
+
|
|
1004
|
+
```bash
|
|
1005
|
+
specfact plan review --auto-enrich --plan <plan_path>
|
|
1006
|
+
```
|
|
1007
|
+
|
|
1008
|
+
- **LLM-Enhanced Refinement** (after auto-enrichment): Use LLM reasoning to:
|
|
1009
|
+
|
|
1010
|
+
- Review auto-enrichment results and verify contextually appropriate improvements
|
|
1011
|
+
- Identify generic improvements that need domain-specific refinement
|
|
1012
|
+
- Suggest specific manual improvements using CLI commands
|
|
1013
|
+
|
|
1014
|
+
- **Manual Enrichment**: Use `specfact plan update-feature` commands to add missing constraints/acceptance criteria with specific details
|
|
1015
|
+
|
|
1016
|
+
- **Defer**: Proceed with promotion and enrich later during implementation (not recommended if Partial categories are high-impact)
|
|
1017
|
+
|
|
1018
|
+
**Next Steps**:
|
|
1019
|
+
|
|
1020
|
+
- Plan can be promoted (no critical blockers)
|
|
1021
|
+
- Optional: Run enrichment to improve Partial categories
|
|
1022
|
+
- Run: `/specfact-cli/specfact-plan-promote review`
|
|
1023
|
+
|
|
1024
|
+
**If Outstanding or Deferred remain**:
|
|
1025
|
+
|
|
1026
|
+
- Recommend whether to proceed to promotion or run review again
|
|
1027
|
+
- Flag high-impact deferred items with rationale
|
|
1028
|
+
- Explain what "Partial" means and when enrichment is recommended vs optional
|
|
1029
|
+
|
|
1030
|
+
## Guidelines
|
|
1031
|
+
|
|
1032
|
+
### Question Quality
|
|
1033
|
+
|
|
1034
|
+
- **High Impact**: Questions that materially affect implementation or validation
|
|
1035
|
+
- **Actionable**: Answers that can be integrated into plan bundle sections
|
|
1036
|
+
- **Focused**: One question per ambiguity, not multiple related questions
|
|
1037
|
+
- **Testable**: Answers that lead to measurable acceptance criteria
|
|
1038
|
+
|
|
1039
|
+
### Integration Quality
|
|
1040
|
+
|
|
1041
|
+
- **Immediate**: Integrate after each answer, don't batch
|
|
1042
|
+
- **Atomic**: Save plan bundle after each integration
|
|
1043
|
+
- **Minimal**: Keep integrations concise and testable
|
|
1044
|
+
- **Consistent**: Use same terminology across all sections
|
|
1045
|
+
|
|
1046
|
+
### Promotion Readiness
|
|
1047
|
+
|
|
1048
|
+
A plan is ready for promotion when:
|
|
1049
|
+
|
|
1050
|
+
- All critical ambiguities resolved
|
|
1051
|
+
- Acceptance criteria are testable
|
|
1052
|
+
- Constraints are explicit and measurable
|
|
1053
|
+
- Terminology is consistent
|
|
1054
|
+
- No contradictory statements remain
|
|
1055
|
+
|
|
1056
|
+
### LLM Reasoning for Continuous Improvement
|
|
1057
|
+
|
|
1058
|
+
**After auto-enrichment, use LLM reasoning to further improve the plan**:
|
|
1059
|
+
|
|
1060
|
+
1. **Analyze Enrichment Quality**:
|
|
1061
|
+
- Review each enhanced acceptance criteria: Is the Given/When/Then scenario specific enough?
|
|
1062
|
+
- Check enhanced requirements: Do they capture the full intent with appropriate verbs?
|
|
1063
|
+
- Evaluate task enhancements: Are file paths and method names accurate for the codebase?
|
|
1064
|
+
|
|
1065
|
+
2. **Identify Generic Patterns**:
|
|
1066
|
+
- Look for placeholder text like "interact with the system" → Suggest specific actions
|
|
1067
|
+
- Find generic file paths like "src/specfact_cli/..." → Research actual codebase structure
|
|
1068
|
+
- Detect vague component names → Suggest specific class/function names from codebase
|
|
1069
|
+
|
|
1070
|
+
3. **Research Codebase Context**:
|
|
1071
|
+
- Search for actual file paths, method names, and component structures
|
|
1072
|
+
- Identify domain-specific patterns and terminology
|
|
1073
|
+
- Understand the actual implementation approach to suggest accurate refinements
|
|
1074
|
+
|
|
1075
|
+
4. **Propose Specific Refinements**:
|
|
1076
|
+
- Use `specfact plan update-feature` to refine generic Given/When/Then scenarios
|
|
1077
|
+
- Add specific file paths, method names, or component references to tasks
|
|
1078
|
+
- Enhance requirements with domain-specific details
|
|
1079
|
+
|
|
1080
|
+
5. **Validate Improvements**:
|
|
1081
|
+
- Ensure all refinements are testable and measurable
|
|
1082
|
+
- Verify terminology consistency across all enhancements
|
|
1083
|
+
- Check that refinements align with codebase structure and patterns
|
|
1084
|
+
|
|
1085
|
+
## Troubleshooting
|
|
1086
|
+
|
|
1087
|
+
### Common Errors and Solutions
|
|
1088
|
+
|
|
1089
|
+
#### Error: "Plan validation failed: Validation error: Input should be a valid string"
|
|
1090
|
+
|
|
1091
|
+
**Cause**: Answers in clarifications section are stored as booleans instead of strings.
|
|
1092
|
+
|
|
1093
|
+
**Symptoms**:
|
|
1094
|
+
|
|
1095
|
+
- Error message: `clarifications.sessions.0.questions.X.answer: Input should be a valid string`
|
|
1096
|
+
- Plan bundle fails to load or validate
|
|
1097
|
+
|
|
1098
|
+
**Solution**:
|
|
1099
|
+
|
|
1100
|
+
1. **Check JSON file format**:
|
|
1101
|
+
|
|
1102
|
+
```bash
|
|
1103
|
+
# Validate JSON syntax
|
|
1104
|
+
python3 -m json.tool answers.json
|
|
1105
|
+
```
|
|
1106
|
+
|
|
1107
|
+
2. **Ensure all answers are quoted strings**:
|
|
1108
|
+
|
|
1109
|
+
```json
|
|
1110
|
+
{
|
|
1111
|
+
"Q001": "Developers, DevOps engineers", // ✅ Quoted string
|
|
1112
|
+
"Q002": "Yes", // ✅ Quoted string (not true or unquoted Yes)
|
|
1113
|
+
"Q003": "No" // ✅ Quoted string (not false or unquoted No)
|
|
1114
|
+
}
|
|
1115
|
+
```
|
|
1116
|
+
|
|
1117
|
+
3. **Fix existing plan bundle** (if already corrupted):
|
|
1118
|
+
|
|
1119
|
+
```bash
|
|
1120
|
+
# Use sed to quote unquoted "Yes" values in YAML
|
|
1121
|
+
sed -i "s/^ answer: Yes$/ answer: 'Yes'/" .specfact/plans/<plan>.bundle.yaml
|
|
1122
|
+
sed -i "s/^ answer: No$/ answer: 'No'/" .specfact/plans/<plan>.bundle.yaml
|
|
1123
|
+
```
|
|
1124
|
+
|
|
1125
|
+
4. **Verify fix**:
|
|
1126
|
+
|
|
1127
|
+
```bash
|
|
1128
|
+
# Check that all answers are strings
|
|
1129
|
+
python3 -c "import yaml; data = yaml.safe_load(open('.specfact/plans/<plan>.bundle.yaml')); print('All strings:', all(isinstance(q['answer'], str) for s in data['clarifications']['sessions'] for q in s['questions']))"
|
|
1130
|
+
```
|
|
1131
|
+
|
|
1132
|
+
#### Error: "Invalid JSON in --answers"
|
|
1133
|
+
|
|
1134
|
+
**Cause**: JSON syntax error in answers file or inline JSON string.
|
|
1135
|
+
|
|
1136
|
+
**Solution**:
|
|
1137
|
+
|
|
1138
|
+
1. **Validate JSON syntax**:
|
|
1139
|
+
|
|
1140
|
+
```bash
|
|
1141
|
+
python3 -m json.tool answers.json
|
|
1142
|
+
```
|
|
1143
|
+
|
|
1144
|
+
2. **Check for common issues**:
|
|
1145
|
+
- Missing quotes around string values
|
|
1146
|
+
- Trailing commas
|
|
1147
|
+
- Unclosed brackets or braces
|
|
1148
|
+
- Special characters not escaped
|
|
1149
|
+
|
|
1150
|
+
3. **Use file path instead of inline JSON** (recommended):
|
|
1151
|
+
|
|
1152
|
+
```bash
|
|
1153
|
+
# ✅ Better: Use file
|
|
1154
|
+
specfact plan review --answers answers.json
|
|
1155
|
+
|
|
1156
|
+
# ⚠️ Avoid: Inline JSON (can have escaping issues)
|
|
1157
|
+
specfact plan review --answers '{"Q001": "answer"}'
|
|
1158
|
+
```
|
|
1159
|
+
|
|
1160
|
+
#### Error: "Answer for Q002 must be a non-empty string"
|
|
1161
|
+
|
|
1162
|
+
**Cause**: Answer value is not a string (e.g., boolean `true`/`false` or `null`).
|
|
1163
|
+
|
|
1164
|
+
**Solution**:
|
|
1165
|
+
|
|
1166
|
+
1. **Ensure all answers are strings in JSON**:
|
|
1167
|
+
|
|
1168
|
+
```json
|
|
1169
|
+
{
|
|
1170
|
+
"Q002": "Yes" // ✅ String
|
|
1171
|
+
}
|
|
1172
|
+
```
|
|
1173
|
+
|
|
1174
|
+
Not:
|
|
1175
|
+
|
|
1176
|
+
```json
|
|
1177
|
+
{
|
|
1178
|
+
"Q002": true // ❌ Boolean
|
|
1179
|
+
"Q002": null // ❌ Null
|
|
1180
|
+
}
|
|
1181
|
+
```
|
|
1182
|
+
|
|
1183
|
+
2. **Validate before feeding to CLI**:
|
|
1184
|
+
|
|
1185
|
+
```bash
|
|
1186
|
+
# Check all values are strings
|
|
1187
|
+
python3 -c "import json; data = json.load(open('answers.json')); print('All strings:', all(isinstance(v, str) for v in data.values()))"
|
|
1188
|
+
```
|
|
1189
|
+
|
|
1190
|
+
#### Error: "Feature 'FEATURE-001' not found in plan"
|
|
1191
|
+
|
|
1192
|
+
**Cause**: Feature key doesn't exist in plan bundle.
|
|
1193
|
+
|
|
1194
|
+
**Solution**:
|
|
1195
|
+
|
|
1196
|
+
1. **List available features**:
|
|
1197
|
+
|
|
1198
|
+
```bash
|
|
1199
|
+
specfact plan select --list-features
|
|
1200
|
+
```
|
|
1201
|
+
|
|
1202
|
+
2. **Use correct feature key** (case-sensitive, exact match required)
|
|
1203
|
+
|
|
1204
|
+
#### Error: "Story 'STORY-001' not found in feature 'FEATURE-001'"
|
|
1205
|
+
|
|
1206
|
+
**Cause**: Story key doesn't exist in the specified feature.
|
|
1207
|
+
|
|
1208
|
+
**Solution**:
|
|
1209
|
+
|
|
1210
|
+
1. **List stories in feature**:
|
|
1211
|
+
|
|
1212
|
+
```bash
|
|
1213
|
+
# Check plan bundle YAML for story keys
|
|
1214
|
+
grep -A 5 "key: FEATURE-001" .specfact/plans/<plan>.bundle.yaml | grep "key: STORY"
|
|
1215
|
+
```
|
|
1216
|
+
|
|
1217
|
+
2. **Use correct story key** (case-sensitive, exact match required)
|
|
1218
|
+
|
|
1219
|
+
### Prevention Checklist
|
|
1220
|
+
|
|
1221
|
+
Before feeding answers to CLI:
|
|
1222
|
+
|
|
1223
|
+
- [ ] **JSON file syntax is valid** (use `python3 -m json.tool` to validate)
|
|
1224
|
+
- [ ] **All answer values are quoted strings** (not booleans, not null)
|
|
1225
|
+
- [ ] **Boolean-like strings are quoted** ("Yes", "No", "True", "False", "On", "Off")
|
|
1226
|
+
- [ ] **Using file path** (not inline JSON string) for complex answers
|
|
1227
|
+
- [ ] **No trailing commas** in JSON
|
|
1228
|
+
- [ ] **All question IDs match** (Q001, Q002, etc. from `--list-questions` output)
|
|
1229
|
+
|
|
1230
|
+
After feeding answers:
|
|
1231
|
+
|
|
1232
|
+
- [ ] **Plan bundle validates** (run `specfact plan review --list-questions --max-questions 1`)
|
|
1233
|
+
- [ ] **No validation errors** in CLI output
|
|
1234
|
+
- [ ] **All clarifications saved** (check `clarifications.sessions` in YAML)
|
|
1235
|
+
|
|
1236
|
+
**Example LLM Reasoning Process**:
|
|
1237
|
+
|
|
1238
|
+
```text
|
|
1239
|
+
1. Auto-enrichment enhanced: "is implemented" → "Given a developer wants to use configure git operations, When they interact with the system, Then configure git operations is functional and verified"
|
|
1240
|
+
|
|
1241
|
+
2. LLM Analysis:
|
|
1242
|
+
|
|
1243
|
+
- ✅ Given clause is contextually appropriate
|
|
1244
|
+
- ⚠️ When clause is too generic ("interact with the system")
|
|
1245
|
+
- ✅ Then clause captures the outcome
|
|
1246
|
+
|
|
1247
|
+
3. Codebase Research:
|
|
1248
|
+
|
|
1249
|
+
- Search for Git operations configuration methods
|
|
1250
|
+
- Find: `src/specfact_cli/utils/git_operations.py` with `configure()` method
|
|
1251
|
+
- Identify: Method signature `configure(repo_path: Path, config: dict) -> bool`
|
|
1252
|
+
|
|
1253
|
+
4. Proposed Refinement:
|
|
1254
|
+
|
|
1255
|
+
- "Given a developer wants to configure Git operations, When they call configure(repo_path, config) with valid parameters, Then the method returns True and configuration is persisted"
|
|
1256
|
+
|
|
1257
|
+
5. Execute Refinement:
|
|
1258
|
+
|
|
1259
|
+
```bash
|
|
1260
|
+
specfact plan update-feature --key FEATURE-001 --acceptance "Given a developer wants to configure Git operations, When they call configure(repo_path, config) with valid parameters, Then the method returns True and configuration is persisted" --plan <path>
|
|
1261
|
+
```
|
|
1262
|
+
|
|
1263
|
+
**Continuous Improvement Loop**:
|
|
1264
|
+
|
|
1265
|
+
1. Run `--auto-enrich` to fix common issues automatically
|
|
1266
|
+
2. Use LLM reasoning to analyze enrichment results
|
|
1267
|
+
3. Research codebase for specific details
|
|
1268
|
+
4. Propose and execute refinements using CLI commands
|
|
1269
|
+
5. Re-run review to verify improvements
|
|
1270
|
+
6. Iterate until plan quality meets promotion standards
|
|
1271
|
+
|
|
1272
|
+
## Context
|
|
1273
|
+
|
|
1274
|
+
{ARGS}
|
|
1275
|
+
|
|
1276
|
+
--- End Command ---
|