fraim-framework 2.0.52 → 2.0.55
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/registry/scripts/profile-server.js +2 -1
- package/dist/src/ai-manager/ai-manager.js +49 -1
- package/dist/src/ai-manager/phase-flow.js +68 -0
- package/dist/src/utils/digest-utils.js +18 -7
- package/dist/tests/test-debug-session.js +6 -2
- package/dist/tests/test-enhanced-session-init.js +6 -2
- package/dist/tests/test-mcp-lifecycle-methods.js +1 -2
- package/dist/tests/test-mcp-template-processing.js +6 -2
- package/dist/tests/test-modular-issue-tracking.js +6 -2
- package/dist/tests/test-node-compatibility.js +4 -2
- package/dist/tests/test-npm-install.js +4 -2
- package/dist/tests/test-productivity-integration.js +157 -0
- package/dist/tests/test-session-rehydration.js +1 -2
- package/dist/tests/test-telemetry.js +1 -2
- package/dist/tests/test-users-to-target-workflow.js +253 -0
- package/index.js +44 -55
- package/package.json +5 -5
- package/registry/agent-guardrails.md +62 -62
- package/registry/scripts/detect-tautological-tests.sh +38 -38
- package/registry/scripts/productivity/build-productivity-csv.mjs +242 -0
- package/registry/scripts/productivity/fetch-pr-details.mjs +144 -0
- package/registry/scripts/productivity/productivity-report.sh +147 -0
- package/registry/scripts/profile-server.ts +1 -1
- package/registry/scripts/validate-openapi-limits.ts +366 -366
- package/registry/scripts/validate-test-coverage.ts +280 -280
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase1-customer-profiling.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase1-survey-scoping.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase2-platform-discovery.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase2-survey-build-linkedin.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase3-prospect-qualification.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase3-survey-build-reddit.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase4-inventory-compilation.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase4-survey-build-x.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase5-survey-build-facebook.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase6-survey-build-custom.md +11 -0
- package/registry/stubs/workflows/customer-development/ai-coach-phases/phase7-survey-dispatch.md +11 -0
- package/registry/stubs/workflows/customer-development/templates/customer-persona-template.md +11 -0
- package/registry/stubs/workflows/customer-development/templates/search-strategy-template.md +11 -0
- package/registry/stubs/workflows/customer-development/user-survey-dispatch.md +11 -0
- package/registry/stubs/workflows/customer-development/users-to-target.md +11 -0
- package/registry/stubs/workflows/productivity-report/productivity-report.md +11 -0
- package/bin/fraim.js +0 -8
- package/dist/registry/ai-manager-rules/design-phases/design.md +0 -108
- package/dist/registry/ai-manager-rules/design-phases/finalize.md +0 -60
- package/dist/registry/ai-manager-rules/design-phases/validate.md +0 -125
- package/dist/registry/ai-manager-rules/design.json +0 -97
- package/dist/registry/ai-manager-rules/implement-phases/code.md +0 -323
- package/dist/registry/ai-manager-rules/implement-phases/completeness-review.md +0 -94
- package/dist/registry/ai-manager-rules/implement-phases/finalize.md +0 -177
- package/dist/registry/ai-manager-rules/implement-phases/quality-review.md +0 -304
- package/dist/registry/ai-manager-rules/implement-phases/regression.md +0 -159
- package/dist/registry/ai-manager-rules/implement-phases/repro.md +0 -101
- package/dist/registry/ai-manager-rules/implement-phases/scoping.md +0 -93
- package/dist/registry/ai-manager-rules/implement-phases/smoke.md +0 -225
- package/dist/registry/ai-manager-rules/implement-phases/spike.md +0 -118
- package/dist/registry/ai-manager-rules/implement-phases/validate.md +0 -347
- package/dist/registry/ai-manager-rules/implement.json +0 -153
- package/dist/registry/ai-manager-rules/shared-phases/finalize.md +0 -169
- package/dist/registry/ai-manager-rules/spec-phases/finalize.md +0 -60
- package/dist/registry/ai-manager-rules/spec-phases/spec.md +0 -102
- package/dist/registry/ai-manager-rules/spec-phases/validate.md +0 -118
- package/dist/registry/ai-manager-rules/spec.json +0 -112
- package/dist/registry/ai-manager-rules/test.json +0 -98
- package/dist/registry/scripts/build-scripts-generator.js +0 -205
- package/dist/registry/scripts/fraim-config.js +0 -61
- package/dist/registry/scripts/generic-issues-api.js +0 -100
- package/dist/registry/scripts/openapi-generator.js +0 -664
- package/dist/registry/scripts/performance/profile-server.js +0 -390
- package/dist/src/ai-manager/evidence-validator.js +0 -309
- package/dist/src/fraim/issue-tracking/ado-provider.js +0 -304
- package/dist/src/fraim/issue-tracking/factory.js +0 -63
- package/dist/src/fraim/issue-tracking/github-provider.js +0 -200
- package/dist/src/fraim/issue-tracking/types.js +0 -7
- package/dist/src/fraim/issue-tracking-config.js +0 -83
- package/dist/src/static-website-middleware.js +0 -75
- package/dist/test-utils.js +0 -96
- package/dist/tests/esm-compat.js +0 -11
- package/dist/tests/test-ai-manager-phase-protocol.js +0 -147
- package/dist/tests/test-ai-manager.js +0 -118
- package/dist/tests/test-chalk-esm-issue.js +0 -159
- package/dist/tests/test-chalk-real-world.js +0 -265
- package/dist/tests/test-chalk-regression.js +0 -377
- package/dist/tests/test-chalk-resolution-issue.js +0 -304
- package/dist/tests/test-evidence-validation.js +0 -221
- package/dist/tests/test-first-run-interactive.js +0 -1
- package/dist/tests/test-fraim-install-chalk-issue.js +0 -254
- package/dist/tests/test-markdown-to-pdf.js +0 -454
- package/dist/tests/test-npm-resolution-diagnostic.js +0 -140
- package/dist/tests/test-pr-review-integration.js +0 -1
- package/dist/website/.nojekyll +0 -0
- package/dist/website/404.html +0 -101
- package/dist/website/CNAME +0 -1
- package/dist/website/README.md +0 -22
- package/dist/website/demo.html +0 -604
- package/dist/website/images/.gitkeep +0 -1
- package/dist/website/images/fraim-logo.png +0 -0
- package/dist/website/index.html +0 -290
- package/dist/website/pricing.html +0 -414
- package/dist/website/script.js +0 -55
- package/dist/website/styles.css +0 -2647
|
@@ -1,225 +0,0 @@
|
|
|
1
|
-
# Phase: Smoke
|
|
2
|
-
|
|
3
|
-
## INTENT
|
|
4
|
-
To run technical health checks ensuring the system is stable: build passes, all tests pass, git status is clean, and core functionality hasn't been broken.
|
|
5
|
-
|
|
6
|
-
## OUTCOME
|
|
7
|
-
All technical checks pass, confirming:
|
|
8
|
-
- Build compiles successfully
|
|
9
|
-
- All tests pass (100% success rate)
|
|
10
|
-
- Git status is clean
|
|
11
|
-
- Core system functionality intact
|
|
12
|
-
- No regressions in critical paths
|
|
13
|
-
|
|
14
|
-
## PRINCIPLES
|
|
15
|
-
- **Zero Tolerance**: All checks must pass
|
|
16
|
-
- **Technical Focus**: Automated verification only
|
|
17
|
-
- **Fix Immediately**: If any check fails, return to implement phase
|
|
18
|
-
- **Fast Feedback**: Quick automated verification
|
|
19
|
-
|
|
20
|
-
## 📋 MANDATORY TECHNICAL CHECKS
|
|
21
|
-
|
|
22
|
-
### Step 1: Build Compilation Check ✅
|
|
23
|
-
|
|
24
|
-
**Requirements**:
|
|
25
|
-
- TypeScript compiles without errors
|
|
26
|
-
- Build process completes successfully
|
|
27
|
-
- No type errors or warnings
|
|
28
|
-
|
|
29
|
-
**Commands to Run**:
|
|
30
|
-
```bash
|
|
31
|
-
# Check TypeScript compilation
|
|
32
|
-
npx tsc --noEmit --skipLibCheck
|
|
33
|
-
|
|
34
|
-
# Run full build (includes validation)
|
|
35
|
-
npm run build
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
**What to Look For**:
|
|
39
|
-
- Exit code 0 (success) for all commands
|
|
40
|
-
- No error messages in output
|
|
41
|
-
- Clean compilation without warnings
|
|
42
|
-
|
|
43
|
-
### Step 2: Complete Test Suite ✅
|
|
44
|
-
|
|
45
|
-
**Requirements**:
|
|
46
|
-
- All tests pass (100% success rate)
|
|
47
|
-
- No timeouts or hanging tests
|
|
48
|
-
- Reasonable test count
|
|
49
|
-
|
|
50
|
-
**Commands to Run**:
|
|
51
|
-
```bash
|
|
52
|
-
# Run all tests
|
|
53
|
-
npm test
|
|
54
|
-
|
|
55
|
-
# Run smoke tests specifically
|
|
56
|
-
npm run test-smoke-ci
|
|
57
|
-
|
|
58
|
-
# Run all tests in CI mode
|
|
59
|
-
npm run test-all-ci
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
**What to Look For**:
|
|
63
|
-
- "All tests passed" or similar success message
|
|
64
|
-
- No "FAILED" or "ERROR" in output
|
|
65
|
-
- No timeout messages
|
|
66
|
-
- Reasonable execution time
|
|
67
|
-
|
|
68
|
-
### Step 3: Git Status Check ✅
|
|
69
|
-
|
|
70
|
-
**Requirements**:
|
|
71
|
-
- Git status is clean
|
|
72
|
-
- Only intended changes present
|
|
73
|
-
- No untracked files (except evidence docs)
|
|
74
|
-
|
|
75
|
-
**Commands to Run**:
|
|
76
|
-
```bash
|
|
77
|
-
# Check git status
|
|
78
|
-
git status
|
|
79
|
-
|
|
80
|
-
# Check recent commits
|
|
81
|
-
git log --oneline -3
|
|
82
|
-
```
|
|
83
|
-
|
|
84
|
-
**What to Look For**:
|
|
85
|
-
- Only intended files are modified
|
|
86
|
-
- No accidentally modified files
|
|
87
|
-
- Clean working directory
|
|
88
|
-
- Meaningful commit messages
|
|
89
|
-
|
|
90
|
-
### Step 4: Code Quality Verification ✅
|
|
91
|
-
|
|
92
|
-
**Requirements**:
|
|
93
|
-
- No type bypassing
|
|
94
|
-
- No debugging code remains
|
|
95
|
-
- No TODO comments in core functionality
|
|
96
|
-
|
|
97
|
-
**Commands to Run**:
|
|
98
|
-
```bash
|
|
99
|
-
# Check for type bypassing
|
|
100
|
-
grep -r "as any" src/ || echo "✅ No type bypassing found"
|
|
101
|
-
|
|
102
|
-
# Look for debugging code
|
|
103
|
-
grep -r "console.log" src/ | grep -v "logger" | grep -v "// OK" || echo "✅ No debug logs found"
|
|
104
|
-
|
|
105
|
-
# Check for TODO comments
|
|
106
|
-
grep -r "TODO" src/ || echo "✅ No TODOs found"
|
|
107
|
-
```
|
|
108
|
-
|
|
109
|
-
**What to Look For**:
|
|
110
|
-
- Zero instances of "as any" (or only justified ones with comments)
|
|
111
|
-
- No console.log statements (except intentional logging)
|
|
112
|
-
- No TODO comments in critical code
|
|
113
|
-
|
|
114
|
-
### Step 5: Registry Validation (If Applicable) ✅
|
|
115
|
-
|
|
116
|
-
**Commands to Run**:
|
|
117
|
-
```bash
|
|
118
|
-
# Validate registry paths
|
|
119
|
-
npm run validate:registry
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
**What to Look For**:
|
|
123
|
-
- No validation errors
|
|
124
|
-
- All registry paths valid
|
|
125
|
-
|
|
126
|
-
## 🛠️ TECHNICAL COMMANDS REFERENCE
|
|
127
|
-
|
|
128
|
-
### Quick Health Check
|
|
129
|
-
```bash
|
|
130
|
-
# One-liner to check basic health
|
|
131
|
-
npx tsc --noEmit --skipLibCheck && npm test && git status
|
|
132
|
-
```
|
|
133
|
-
|
|
134
|
-
### Comprehensive Technical Check
|
|
135
|
-
```bash
|
|
136
|
-
# Full technical validation suite
|
|
137
|
-
npm run build && npm run test-all-ci && git status
|
|
138
|
-
```
|
|
139
|
-
|
|
140
|
-
### Build Verification
|
|
141
|
-
```bash
|
|
142
|
-
# Verify build artifacts are created correctly
|
|
143
|
-
npm run build && ls -la dist/ && echo "✅ Build artifacts present"
|
|
144
|
-
```
|
|
145
|
-
|
|
146
|
-
## VALIDATION
|
|
147
|
-
|
|
148
|
-
### Phase Complete When:
|
|
149
|
-
- ✅ TypeScript compiles (exit code 0)
|
|
150
|
-
- ✅ Build succeeds without errors
|
|
151
|
-
- ✅ All tests pass (100% success rate)
|
|
152
|
-
- ✅ Git status clean (only intended changes)
|
|
153
|
-
- ✅ No "as any" type bypassing
|
|
154
|
-
- ✅ No debugging code remains
|
|
155
|
-
- ✅ No TODO comments in core functionality
|
|
156
|
-
- ✅ Registry validation passes (if applicable)
|
|
157
|
-
- ✅ All technical checks documented
|
|
158
|
-
|
|
159
|
-
### Phase Incomplete If:
|
|
160
|
-
- ❌ TypeScript compilation errors
|
|
161
|
-
- ❌ Build fails
|
|
162
|
-
- ❌ ANY test fails
|
|
163
|
-
- ❌ Git status shows unintended changes
|
|
164
|
-
- ❌ Code quality issues found
|
|
165
|
-
- ❌ Registry validation fails
|
|
166
|
-
|
|
167
|
-
**If ANY check fails, return to implement phase immediately.**
|
|
168
|
-
|
|
169
|
-
## RULES FOR THIS PHASE
|
|
170
|
-
|
|
171
|
-
### Technical Standards
|
|
172
|
-
- Read `.fraim/config.json` for the architecture document path (`customizations.architectureDoc`), then use `get_fraim_file` to read the full technical standards
|
|
173
|
-
- Use Successful Debugging Patterns from `rules/successful-debugging-patterns.md` via `get_fraim_file`
|
|
174
|
-
- When using git commands directly (if MCP tools unavailable), read `registry/rules/git-safe-commands.md` via `get_fraim_file`
|
|
175
|
-
|
|
176
|
-
## SCRIPTS
|
|
177
|
-
|
|
178
|
-
**Run all technical checks:**
|
|
179
|
-
```bash
|
|
180
|
-
npm run build && npm test && git status
|
|
181
|
-
```
|
|
182
|
-
|
|
183
|
-
**Run smoke tests specifically:**
|
|
184
|
-
```bash
|
|
185
|
-
npm run test-smoke-ci
|
|
186
|
-
```
|
|
187
|
-
|
|
188
|
-
### Report Back:
|
|
189
|
-
When you complete this phase, call:
|
|
190
|
-
|
|
191
|
-
```javascript
|
|
192
|
-
seekCoachingOnNextStep({
|
|
193
|
-
workflowType: "implement",
|
|
194
|
-
issueNumber: "{issue_number}",
|
|
195
|
-
currentPhase: "smoke",
|
|
196
|
-
status: "complete",
|
|
197
|
-
evidence: {
|
|
198
|
-
buildPassed: "YES", // Did `npm run build` succeed?
|
|
199
|
-
allTestsPassed: "YES", // Did `npm test` show 100% pass rate?
|
|
200
|
-
gitStatusClean: "YES", // Does `git status` show only intended changes?
|
|
201
|
-
noTypeBypass: "YES", // Did `grep -r "as any" src/` find zero instances?
|
|
202
|
-
noDebugCode: "YES", // Did you remove console.log statements?
|
|
203
|
-
registryValid: "YES", // Did `npm run validate:registry` pass?
|
|
204
|
-
commandsRun: "npm run build && npm test && git status",
|
|
205
|
-
summary: "All technical checks passed. Build succeeds, all tests pass, git status clean."
|
|
206
|
-
}
|
|
207
|
-
})
|
|
208
|
-
```
|
|
209
|
-
|
|
210
|
-
If any technical check fails:
|
|
211
|
-
```javascript
|
|
212
|
-
seekCoachingOnNextStep({
|
|
213
|
-
workflowType: "implement",
|
|
214
|
-
issueNumber: "{issue_number}",
|
|
215
|
-
currentPhase: "smoke",
|
|
216
|
-
status: "incomplete",
|
|
217
|
-
evidence: {
|
|
218
|
-
buildPassed: "NO", // Specify what failed
|
|
219
|
-
allTestsPassed: "NO", // Be specific about failures
|
|
220
|
-
issues: ["Build failed with TypeScript errors", "3 tests failing"],
|
|
221
|
-
commandsRun: "npm run build && npm test",
|
|
222
|
-
nextAction: "Returning to implement phase to fix technical issues"
|
|
223
|
-
}
|
|
224
|
-
})
|
|
225
|
-
```
|
|
@@ -1,118 +0,0 @@
|
|
|
1
|
-
# Phase: Spike (Features Only)
|
|
2
|
-
|
|
3
|
-
## INTENT
|
|
4
|
-
To build a proof-of-concept that validates the technical approach, tests uncertain or risky aspects, and confirms the design is viable before full implementation.
|
|
5
|
-
|
|
6
|
-
## OUTCOME
|
|
7
|
-
A working POC that:
|
|
8
|
-
- Validates key technical assumptions
|
|
9
|
-
- Tests risky or uncertain aspects
|
|
10
|
-
- Gets user approval
|
|
11
|
-
- Informs any necessary design updates
|
|
12
|
-
|
|
13
|
-
## PRINCIPLES
|
|
14
|
-
- **Spike-First Development**: Follow the spike-first rule for unfamiliar technology
|
|
15
|
-
- **Risk Reduction**: Focus on highest-risk or most uncertain aspects
|
|
16
|
-
- **Quick Validation**: Build minimal POC, not production code
|
|
17
|
-
- **User Approval**: Get explicit approval before proceeding
|
|
18
|
-
- **Design Updates**: Update design doc with findings
|
|
19
|
-
|
|
20
|
-
## WORKFLOW
|
|
21
|
-
|
|
22
|
-
### Step 1: Review Design Document
|
|
23
|
-
- Read the technical design/RFC thoroughly
|
|
24
|
-
- Identify technical assumptions
|
|
25
|
-
- Note any unfamiliar technologies or patterns
|
|
26
|
-
- List risky or uncertain aspects
|
|
27
|
-
|
|
28
|
-
### Step 2: Identify Spike Scope
|
|
29
|
-
**Focus on:**
|
|
30
|
-
- Unfamiliar technology or libraries
|
|
31
|
-
- Integration points with external systems
|
|
32
|
-
- Performance-critical components
|
|
33
|
-
- Complex algorithms or logic
|
|
34
|
-
- Uncertain technical feasibility
|
|
35
|
-
|
|
36
|
-
**Do NOT spike:**
|
|
37
|
-
- Well-understood patterns
|
|
38
|
-
- Standard CRUD operations
|
|
39
|
-
- Simple UI changes
|
|
40
|
-
- Routine implementations
|
|
41
|
-
|
|
42
|
-
### Step 3: Build Proof-of-Concept
|
|
43
|
-
- Create minimal code to test assumptions
|
|
44
|
-
- Focus on proving/disproving technical approach
|
|
45
|
-
- Don't worry about production quality
|
|
46
|
-
- Document what you're testing
|
|
47
|
-
|
|
48
|
-
### Step 4: Test Key Assumptions
|
|
49
|
-
- Run the POC
|
|
50
|
-
- Verify it works as expected
|
|
51
|
-
- Test edge cases if relevant
|
|
52
|
-
- Document findings
|
|
53
|
-
|
|
54
|
-
### Step 5: Get User Approval
|
|
55
|
-
- Present POC to user
|
|
56
|
-
- Explain what was validated
|
|
57
|
-
- Share any findings or concerns
|
|
58
|
-
- **Wait for explicit approval** before proceeding
|
|
59
|
-
|
|
60
|
-
### Step 6: Update Design Document
|
|
61
|
-
If POC revealed:
|
|
62
|
-
- Better approaches
|
|
63
|
-
- Technical constraints
|
|
64
|
-
- Performance considerations
|
|
65
|
-
- Integration challenges
|
|
66
|
-
|
|
67
|
-
**Update the design document** with findings
|
|
68
|
-
|
|
69
|
-
## VALIDATION
|
|
70
|
-
|
|
71
|
-
### Phase Complete When:
|
|
72
|
-
- ✅ POC built and tested
|
|
73
|
-
- ✅ Key technical assumptions validated
|
|
74
|
-
- ✅ User has approved the approach
|
|
75
|
-
- ✅ Design document updated (if needed)
|
|
76
|
-
- ✅ Ready to proceed with full implementation
|
|
77
|
-
|
|
78
|
-
### Phase Incomplete If:
|
|
79
|
-
- ❌ POC doesn't work as expected
|
|
80
|
-
- ❌ Technical approach not viable
|
|
81
|
-
- ❌ User hasn't approved
|
|
82
|
-
- ❌ Significant findings not documented
|
|
83
|
-
|
|
84
|
-
## RULES FOR THIS PHASE
|
|
85
|
-
|
|
86
|
-
### Spike-First Development
|
|
87
|
-
Read `registry/rules/spike-first-development.md` via `get_fraim_file` for complete spike methodology.
|
|
88
|
-
|
|
89
|
-
### Architecture Compliance
|
|
90
|
-
Read `.fraim/config.json` for the architecture document path (`customizations.architectureDoc`), then use `get_fraim_file` to read the full architecture guidelines.
|
|
91
|
-
|
|
92
|
-
### Git Operations (if needed)
|
|
93
|
-
When using git commands directly (if MCP tools unavailable), read `registry/rules/git-safe-commands.md` via `get_fraim_file` to avoid interactive commands that hang agents.
|
|
94
|
-
|
|
95
|
-
## SCRIPTS
|
|
96
|
-
Run POC:
|
|
97
|
-
```bash
|
|
98
|
-
npm run spike # or appropriate command
|
|
99
|
-
node spike/your-poc.js
|
|
100
|
-
```
|
|
101
|
-
|
|
102
|
-
### Report Back:
|
|
103
|
-
When you complete this phase, call:
|
|
104
|
-
|
|
105
|
-
```javascript
|
|
106
|
-
seekCoachingOnNextStep({
|
|
107
|
-
workflowType: "implement",
|
|
108
|
-
issueNumber: "{issue_number}",
|
|
109
|
-
currentPhase: "spike",
|
|
110
|
-
status: "complete",
|
|
111
|
-
evidence: {
|
|
112
|
-
pocLocation: "spike/auth-flow.ts",
|
|
113
|
-
validationResults: "OAuth2 integration works as expected",
|
|
114
|
-
userApproval: "User approved approach on [date]",
|
|
115
|
-
designUpdates: "Updated design doc with token refresh findings"
|
|
116
|
-
}
|
|
117
|
-
})
|
|
118
|
-
```
|
|
@@ -1,347 +0,0 @@
|
|
|
1
|
-
# Phase: Validate
|
|
2
|
-
|
|
3
|
-
## INTENT
|
|
4
|
-
To verify that the implementation works correctly by testing it in appropriate ways: browser testing for UI changes, API testing for backend changes, and command-line testing for CLI changes.
|
|
5
|
-
|
|
6
|
-
## OUTCOME
|
|
7
|
-
Confirmation that:
|
|
8
|
-
- Implementation works as expected
|
|
9
|
-
- All acceptance criteria are met
|
|
10
|
-
- Edge cases are handled
|
|
11
|
-
- No obvious bugs or issues
|
|
12
|
-
|
|
13
|
-
## 🎯 VALIDATION MINDSET
|
|
14
|
-
|
|
15
|
-
This is THE critical phase where you prove your implementation actually works. No shortcuts, no assumptions.
|
|
16
|
-
|
|
17
|
-
**Your Mission**: Prove beyond doubt that a real user can successfully use this feature.
|
|
18
|
-
|
|
19
|
-
## RULES FOR THIS PHASE
|
|
20
|
-
|
|
21
|
-
### Success Criteria
|
|
22
|
-
Read `registry/rules/agent-success-criteria.md` via `get_fraim_file` for the complete framework. Focus on:
|
|
23
|
-
- **Integrity** (honest reporting of validation results - never claim something works if you didn't test it)
|
|
24
|
-
- **Correctness** (implementation actually works as expected)
|
|
25
|
-
- **Completeness** (all acceptance criteria validated, edge cases tested)
|
|
26
|
-
|
|
27
|
-
### Simplicity Principles
|
|
28
|
-
Read `registry/rules/simplicity.md` via `get_fraim_file` for complete guidelines. Critical for validation:
|
|
29
|
-
- **Manual Validation Required** - This is THE manual validation phase
|
|
30
|
-
- **Prototype-First** - Validate the working solution before engineering tests
|
|
31
|
-
- **Resource Waste Prevention** - Efficient testing approach
|
|
32
|
-
|
|
33
|
-
### Architecture Compliance
|
|
34
|
-
Read `.fraim/config.json` for the architecture document path (`customizations.architectureDoc`), then use `get_fraim_file` to read the full architecture guidelines. Ensure implementation follows architectural patterns.
|
|
35
|
-
|
|
36
|
-
### Debugging and Git Operations
|
|
37
|
-
- Use Successful Debugging Patterns from `rules/successful-debugging-patterns.md` via `get_fraim_file`
|
|
38
|
-
- When using git commands directly (if MCP tools unavailable), read `registry/rules/git-safe-commands.md` via `get_fraim_file` to avoid interactive commands that hang agents.
|
|
39
|
-
|
|
40
|
-
## PRINCIPLES
|
|
41
|
-
- **Real Testing**: Test actual functionality, not mocks
|
|
42
|
-
- **Appropriate Tools**: Use browser for UI, curl for APIs, CLI for commands
|
|
43
|
-
- **Complete Coverage**: Test all acceptance criteria
|
|
44
|
-
- **Edge Cases**: Test boundary conditions and error scenarios
|
|
45
|
-
- **Evidence**: Document validation results
|
|
46
|
-
|
|
47
|
-
## 📋 MANDATORY VALIDATION STEPS
|
|
48
|
-
|
|
49
|
-
### Step 1: Build and Compilation Check ✅
|
|
50
|
-
|
|
51
|
-
**Requirements**:
|
|
52
|
-
- TypeScript compiles without errors
|
|
53
|
-
- Build process completes successfully
|
|
54
|
-
- No type errors or warnings
|
|
55
|
-
|
|
56
|
-
**Commands to Run**:
|
|
57
|
-
```bash
|
|
58
|
-
# Check TypeScript compilation
|
|
59
|
-
npx tsc --noEmit --skipLibCheck
|
|
60
|
-
|
|
61
|
-
# Run full build (includes validation)
|
|
62
|
-
npm run build
|
|
63
|
-
|
|
64
|
-
# Check for any 'as any' type bypassing
|
|
65
|
-
grep -r "as any" src/ || echo "✅ No type bypassing found"
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
**What to Look For**:
|
|
69
|
-
- Exit code 0 (success) for all commands
|
|
70
|
-
- No error messages in output
|
|
71
|
-
- Clean compilation without warnings
|
|
72
|
-
|
|
73
|
-
### Step 2: Test Suite Validation 📊
|
|
74
|
-
|
|
75
|
-
**Requirements**:
|
|
76
|
-
- All tests pass (100% success rate)
|
|
77
|
-
- No timeouts or hanging tests
|
|
78
|
-
- Meaningful test coverage
|
|
79
|
-
|
|
80
|
-
**Commands to Run**:
|
|
81
|
-
```bash
|
|
82
|
-
# Run all tests
|
|
83
|
-
npm test
|
|
84
|
-
|
|
85
|
-
# Run smoke tests specifically
|
|
86
|
-
npm run test-smoke-ci
|
|
87
|
-
|
|
88
|
-
# Run all tests in CI mode
|
|
89
|
-
npm run test-all-ci
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
**What to Look For**:
|
|
93
|
-
- "All tests passed" or similar success message
|
|
94
|
-
- No "FAILED" or "ERROR" in output
|
|
95
|
-
- Reasonable test count (not just 1-2 tests)
|
|
96
|
-
- No timeout messages
|
|
97
|
-
|
|
98
|
-
### Step 3: Manual Functionality Testing 🧪
|
|
99
|
-
|
|
100
|
-
**Requirements**:
|
|
101
|
-
- Test actual user workflows end-to-end
|
|
102
|
-
- Verify all acceptance criteria manually
|
|
103
|
-
- Test both happy path and error scenarios
|
|
104
|
-
|
|
105
|
-
**For UI Changes**:
|
|
106
|
-
```bash
|
|
107
|
-
# Start development server
|
|
108
|
-
npm run dev
|
|
109
|
-
# Then manually test in browser at http://localhost:3000
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
**For API Changes**:
|
|
113
|
-
```bash
|
|
114
|
-
# Test API endpoints
|
|
115
|
-
curl -X GET http://localhost:3000/health
|
|
116
|
-
curl -X POST http://localhost:3000/api/test -H "Content-Type: application/json" -d '{"test":"data"}'
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
**For CLI Changes**:
|
|
120
|
-
```bash
|
|
121
|
-
# Test CLI commands
|
|
122
|
-
node bin/fraim.js --help
|
|
123
|
-
node bin/fraim.js sync --dry-run
|
|
124
|
-
```
|
|
125
|
-
|
|
126
|
-
**What to Document**:
|
|
127
|
-
- Specific steps you performed
|
|
128
|
-
- What you clicked/typed/tested
|
|
129
|
-
- Results you observed
|
|
130
|
-
- Any errors encountered and how they were handled
|
|
131
|
-
|
|
132
|
-
### Step 4: Git and Code Quality Check 🔍
|
|
133
|
-
|
|
134
|
-
**Requirements**:
|
|
135
|
-
- Git status is clean
|
|
136
|
-
- No debugging code remains
|
|
137
|
-
- Code follows quality standards
|
|
138
|
-
|
|
139
|
-
**Commands to Run**:
|
|
140
|
-
```bash
|
|
141
|
-
# Check git status
|
|
142
|
-
git status
|
|
143
|
-
|
|
144
|
-
# Look for debugging code
|
|
145
|
-
grep -r "console.log" src/ | grep -v "logger" | grep -v "// OK" || echo "✅ No debug logs found"
|
|
146
|
-
|
|
147
|
-
# Check for TODO comments in core functionality
|
|
148
|
-
grep -r "TODO" src/ || echo "✅ No TODOs found"
|
|
149
|
-
|
|
150
|
-
# Validate registry paths (if applicable)
|
|
151
|
-
npm run validate:registry
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
**What to Look For**:
|
|
155
|
-
- Only intended files are modified
|
|
156
|
-
- No console.log statements (except intentional logging)
|
|
157
|
-
- No TODO comments in critical code
|
|
158
|
-
- Clean working directory
|
|
159
|
-
|
|
160
|
-
### Step 5: Bug-Specific Validation (For Bug Fixes) 🐛
|
|
161
|
-
|
|
162
|
-
**Requirements**:
|
|
163
|
-
- Original reproduction test now passes
|
|
164
|
-
- Bug symptoms are gone
|
|
165
|
-
- No regressions introduced
|
|
166
|
-
|
|
167
|
-
**Commands to Run**:
|
|
168
|
-
```bash
|
|
169
|
-
# Run the specific repro test
|
|
170
|
-
npm test -- path/to/repro/test.test.ts
|
|
171
|
-
|
|
172
|
-
# Test the original bug scenario manually
|
|
173
|
-
# (Follow the original reproduction steps)
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
**What to Verify**:
|
|
177
|
-
- Repro test that was failing now passes
|
|
178
|
-
- Original bug behavior is fixed
|
|
179
|
-
- Related functionality still works
|
|
180
|
-
|
|
181
|
-
### Step 6: Feature-Specific Validation (For Features) ✨
|
|
182
|
-
|
|
183
|
-
**Requirements**:
|
|
184
|
-
- New functionality works as specified
|
|
185
|
-
- Integration with existing features works
|
|
186
|
-
- All acceptance criteria met
|
|
187
|
-
|
|
188
|
-
**What to Test**:
|
|
189
|
-
- Main user scenarios from the spec
|
|
190
|
-
- Edge cases and error conditions
|
|
191
|
-
- Integration points with existing code
|
|
192
|
-
- Performance (if applicable)
|
|
193
|
-
|
|
194
|
-
## 🛠️ VALIDATION COMMANDS REFERENCE
|
|
195
|
-
|
|
196
|
-
### Quick Health Check
|
|
197
|
-
```bash
|
|
198
|
-
# One-liner to check basic health
|
|
199
|
-
npx tsc --noEmit --skipLibCheck && npm test && git status
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
### Comprehensive Validation
|
|
203
|
-
```bash
|
|
204
|
-
# Full validation suite
|
|
205
|
-
npm run build && npm run test-all-ci && npm run validate:registry
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
### API Testing
|
|
209
|
-
```bash
|
|
210
|
-
# Test API health and basic functionality
|
|
211
|
-
curl -f http://localhost:3000/health && echo "✅ API healthy"
|
|
212
|
-
```
|
|
213
|
-
|
|
214
|
-
### Build Verification
|
|
215
|
-
```bash
|
|
216
|
-
# Verify build artifacts are created correctly
|
|
217
|
-
npm run build && ls -la dist/ && echo "✅ Build artifacts present"
|
|
218
|
-
```
|
|
219
|
-
|
|
220
|
-
## WORKFLOW
|
|
221
|
-
|
|
222
|
-
### Step 1: Identify Validation Method
|
|
223
|
-
|
|
224
|
-
**For UI Changes:**
|
|
225
|
-
- Use browser to navigate and test
|
|
226
|
-
- Verify visual correctness
|
|
227
|
-
- Test user interactions
|
|
228
|
-
- Check responsive behavior
|
|
229
|
-
|
|
230
|
-
**For API Changes:**
|
|
231
|
-
- Use curl or similar tools
|
|
232
|
-
- Test all endpoints
|
|
233
|
-
- Verify request/response formats
|
|
234
|
-
- Test error handling
|
|
235
|
-
|
|
236
|
-
**For Backend/CLI Changes:**
|
|
237
|
-
- Run relevant commands
|
|
238
|
-
- Test with various inputs
|
|
239
|
-
- Verify output correctness
|
|
240
|
-
- Test error conditions
|
|
241
|
-
|
|
242
|
-
### Step 2: Test All Acceptance Criteria
|
|
243
|
-
- Review acceptance criteria from scoping phase
|
|
244
|
-
- Test each criterion systematically
|
|
245
|
-
- Document results for each
|
|
246
|
-
- Verify all are met
|
|
247
|
-
|
|
248
|
-
### Step 3: Test Edge Cases
|
|
249
|
-
**Common edge cases:**
|
|
250
|
-
- Empty inputs
|
|
251
|
-
- Invalid inputs
|
|
252
|
-
- Boundary values
|
|
253
|
-
- Error conditions
|
|
254
|
-
- Concurrent operations (if applicable)
|
|
255
|
-
|
|
256
|
-
### Step 4: Document Results
|
|
257
|
-
In evidence, include:
|
|
258
|
-
- What was tested
|
|
259
|
-
- How it was tested
|
|
260
|
-
- Results of each test
|
|
261
|
-
- Any issues found and fixed
|
|
262
|
-
- Screenshots/output (if applicable)
|
|
263
|
-
|
|
264
|
-
## VALIDATION
|
|
265
|
-
|
|
266
|
-
### Phase Complete When:
|
|
267
|
-
- ✅ All acceptance criteria tested and passing
|
|
268
|
-
- ✅ Edge cases handled correctly
|
|
269
|
-
- ✅ For bugs: repro test now passes
|
|
270
|
-
- ✅ For features: new functionality works
|
|
271
|
-
- ✅ No obvious bugs or issues
|
|
272
|
-
- ✅ Validation results documented with evidence
|
|
273
|
-
- ✅ All servers running without errors
|
|
274
|
-
- ✅ 100% test success rate
|
|
275
|
-
- ✅ Error scenarios tested and handled
|
|
276
|
-
|
|
277
|
-
### Phase Incomplete If:
|
|
278
|
-
- ❌ Acceptance criteria not met
|
|
279
|
-
- ❌ For bugs: repro test still fails
|
|
280
|
-
- ❌ For features: functionality doesn't work
|
|
281
|
-
- ❌ Edge cases not handled
|
|
282
|
-
- ❌ Obvious bugs found
|
|
283
|
-
- ❌ Missing evidence for claims
|
|
284
|
-
- ❌ Server errors present
|
|
285
|
-
- ❌ Any tests failing
|
|
286
|
-
|
|
287
|
-
### Report Back:
|
|
288
|
-
When you complete this phase, call:
|
|
289
|
-
|
|
290
|
-
```javascript
|
|
291
|
-
seekCoachingOnNextStep({
|
|
292
|
-
workflowType: "implement",
|
|
293
|
-
issueNumber: "{issue_number}",
|
|
294
|
-
currentPhase: "validate",
|
|
295
|
-
status: "complete",
|
|
296
|
-
evidence: {
|
|
297
|
-
buildPassed: "YES", // Did `npm run build` succeed?
|
|
298
|
-
testsPassed: "YES", // Did `npm test` show 100% pass rate?
|
|
299
|
-
manualValidationDone: "YES", // Did you manually test the functionality?
|
|
300
|
-
serversStarted: "YES", // Did servers start without errors?
|
|
301
|
-
reproTestPassed: "YES", // (For bugs) Does the repro test now pass?
|
|
302
|
-
featureWorking: "YES", // (For features) Does the new functionality work?
|
|
303
|
-
commandsRun: "npm run build && npm test", // What validation commands did you run?
|
|
304
|
-
summary: "All validation steps completed successfully. Build passes, all 15 tests pass, manual testing confirmed functionality works as expected."
|
|
305
|
-
}
|
|
306
|
-
})
|
|
307
|
-
```
|
|
308
|
-
|
|
309
|
-
If validation reveals issues, report back with status "incomplete":
|
|
310
|
-
```javascript
|
|
311
|
-
seekCoachingOnNextStep({
|
|
312
|
-
workflowType: "implement",
|
|
313
|
-
issueNumber: "{issue_number}",
|
|
314
|
-
currentPhase: "validate",
|
|
315
|
-
status: "incomplete",
|
|
316
|
-
evidence: {
|
|
317
|
-
buildPassed: "NO", // Specify what failed
|
|
318
|
-
testsPassed: "NO", // Be specific about failures
|
|
319
|
-
issues: ["Build failed with TypeScript errors", "3 tests failing", "Manual testing revealed login button not working"],
|
|
320
|
-
commandsRun: "npm run build && npm test",
|
|
321
|
-
nextAction: "Need to return to implement phase to fix these issues"
|
|
322
|
-
}
|
|
323
|
-
})
|
|
324
|
-
```
|
|
325
|
-
|
|
326
|
-
## SCRIPTS
|
|
327
|
-
|
|
328
|
-
**Browser testing:**
|
|
329
|
-
```bash
|
|
330
|
-
npm run dev # Start dev server
|
|
331
|
-
# Then manually test in browser
|
|
332
|
-
```
|
|
333
|
-
|
|
334
|
-
**API testing:**
|
|
335
|
-
```bash
|
|
336
|
-
curl -X POST http://localhost:3000/api/endpoint -d '{"data": "value"}'
|
|
337
|
-
```
|
|
338
|
-
|
|
339
|
-
**CLI testing:**
|
|
340
|
-
```bash
|
|
341
|
-
npm run cli -- command --flag value
|
|
342
|
-
```
|
|
343
|
-
|
|
344
|
-
**Run all tests:**
|
|
345
|
-
```bash
|
|
346
|
-
npm test
|
|
347
|
-
```
|