@iservu-inc/adf-cli 0.3.0 → 0.4.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.project/chats/{current → complete}/2025-10-03_AGENTS-MD-AND-TOOL-GENERATORS.md +82 -17
- package/.project/chats/complete/2025-10-03_AI-PROVIDER-INTEGRATION.md +568 -0
- package/.project/chats/complete/2025-10-03_FRAMEWORK-UPDATE-SYSTEM.md +497 -0
- package/.project/chats/complete/2025-10-04_CONFIG-COMMAND.md +503 -0
- package/.project/chats/current/2025-10-04_PHASE-4-1-SMART-FILTERING.md +381 -0
- package/.project/chats/current/SESSION-STATUS.md +168 -0
- package/.project/docs/AI-PROVIDER-INTEGRATION.md +600 -0
- package/.project/docs/FRAMEWORK-UPDATE-INTEGRATION.md +421 -0
- package/.project/docs/FRAMEWORK-UPDATE-SYSTEM.md +832 -0
- package/.project/docs/PHASE-4-2-LEARNING-SYSTEM.md +881 -0
- package/.project/docs/PROJECT-STRUCTURE-EXPLANATION.md +500 -0
- package/.project/docs/SMART-FILTERING-SYSTEM.md +385 -0
- package/.project/docs/architecture/SYSTEM-DESIGN.md +122 -1
- package/.project/docs/goals/PROJECT-VISION.md +61 -34
- package/CHANGELOG.md +257 -1
- package/README.md +476 -292
- package/bin/adf.js +7 -0
- package/lib/ai/ai-client.js +328 -0
- package/lib/ai/ai-config.js +398 -0
- package/lib/analyzers/project-analyzer.js +380 -0
- package/lib/commands/config.js +221 -0
- package/lib/commands/init.js +56 -10
- package/lib/filters/question-filter.js +480 -0
- package/lib/frameworks/interviewer.js +271 -12
- package/lib/frameworks/progress-tracker.js +8 -1
- package/lib/learning/learning-manager.js +447 -0
- package/lib/learning/pattern-detector.js +376 -0
- package/lib/learning/rule-generator.js +304 -0
- package/lib/learning/skip-tracker.js +260 -0
- package/lib/learning/storage.js +296 -0
- package/package.json +70 -57
- package/tests/learning-storage.test.js +184 -0
- package/tests/pattern-detector.test.js +297 -0
- package/tests/project-analyzer.test.js +221 -0
- package/tests/question-filter.test.js +297 -0
- package/tests/skip-tracker.test.js +198 -0
|
@@ -0,0 +1,881 @@
|
|
|
1
|
+
# Phase 4.2: Learning System - Project Goals & Feature Details
|
|
2
|
+
|
|
3
|
+
**Phase:** 4.2 - Enhanced Intelligence (Learning System)
|
|
4
|
+
**Status:** 📋 Planning
|
|
5
|
+
**Prerequisites:** Phase 4.1 (Smart Question Filtering) ✅ Complete
|
|
6
|
+
**Target Version:** v0.4.0 or v0.5.0
|
|
7
|
+
**Estimated Effort:** Medium (2-3 sessions)
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## 🎯 Overview
|
|
12
|
+
|
|
13
|
+
Build an intelligent learning system that tracks user behavior during interviews and uses that data to continuously improve question filtering accuracy. The system learns which questions users skip manually, identifies patterns across sessions, and adapts filtering rules to match user preferences and project-specific needs.
|
|
14
|
+
|
|
15
|
+
### Vision Statement
|
|
16
|
+
|
|
17
|
+
"The interview should get smarter with every session, learning from user behavior to skip questions they consistently don't need, while never losing the ability to ask important questions."
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 📊 Problem Statement
|
|
22
|
+
|
|
23
|
+
### Current Limitations (Phase 4.1)
|
|
24
|
+
|
|
25
|
+
**Smart Filtering is Rule-Based:**
|
|
26
|
+
- Fixed rules per project type (CLI skips UI, API skips frontend)
|
|
27
|
+
- No personalization or adaptation
|
|
28
|
+
- Can't learn from user behavior
|
|
29
|
+
- May filter too aggressively or not enough
|
|
30
|
+
- No cross-session memory
|
|
31
|
+
|
|
32
|
+
**User Pain Points:**
|
|
33
|
+
1. Users still manually skip some "relevant" questions that aren't useful for their specific use case
|
|
34
|
+
2. Some questions marked as skipped might actually be important for certain hybrid projects
|
|
35
|
+
3. No way to remember user preferences across sessions
|
|
36
|
+
4. Generic rules don't account for team-specific workflows
|
|
37
|
+
5. Can't learn from patterns (e.g., "User always skips deployment questions for prototypes")
|
|
38
|
+
|
|
39
|
+
### Example Scenarios
|
|
40
|
+
|
|
41
|
+
**Scenario 1: Prototype Projects**
|
|
42
|
+
- User working on quick prototypes consistently skips deployment, testing, CI/CD questions
|
|
43
|
+
- Current system: Asks these questions every time (they're "relevant" for the project type)
|
|
44
|
+
- Desired: Learn that this user/project skips these topics and filter them proactively
|
|
45
|
+
|
|
46
|
+
**Scenario 2: Solo Developer vs Team**
|
|
47
|
+
- Solo developer consistently skips collaboration, code review, team size questions
|
|
48
|
+
- Current system: Asks these every time
|
|
49
|
+
- Desired: Detect solo developer pattern and reduce these questions
|
|
50
|
+
|
|
51
|
+
**Scenario 3: Framework-Specific Workflows**
|
|
52
|
+
- User with Next.js project consistently skips questions about bundlers, routing setup
|
|
53
|
+
- Current system: Asks because they're web app questions
|
|
54
|
+
- Desired: Learn Next.js includes these features, skip the questions
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## 🎯 Goals
|
|
59
|
+
|
|
60
|
+
### Primary Goals
|
|
61
|
+
|
|
62
|
+
1. **Track User Behavior**
|
|
63
|
+
- Record which questions users skip manually
|
|
64
|
+
- Track which filtered questions users view/expand
|
|
65
|
+
- Measure time spent on each question
|
|
66
|
+
- Identify patterns in skip behavior
|
|
67
|
+
|
|
68
|
+
2. **Learn Patterns**
|
|
69
|
+
- Identify questions consistently skipped by user
|
|
70
|
+
- Detect project-type specific patterns
|
|
71
|
+
- Learn framework-specific skip patterns
|
|
72
|
+
- Recognize user preference patterns (e.g., always brief on testing)
|
|
73
|
+
|
|
74
|
+
3. **Improve Filtering**
|
|
75
|
+
- Boost relevance scores for questions user engages with
|
|
76
|
+
- Lower relevance scores for questions user consistently skips
|
|
77
|
+
- Create user-specific filtering rules
|
|
78
|
+
- Create project-specific filtering rules
|
|
79
|
+
|
|
80
|
+
4. **Provide Insights**
|
|
81
|
+
- Show user what the system has learned
|
|
82
|
+
- Allow user to review and adjust learned patterns
|
|
83
|
+
- Explain why certain questions are being filtered
|
|
84
|
+
- Provide confidence scores for learned rules
|
|
85
|
+
|
|
86
|
+
### Secondary Goals
|
|
87
|
+
|
|
88
|
+
1. **Privacy & Control**
|
|
89
|
+
- All learning data stored locally in `.adf/`
|
|
90
|
+
- User can view, edit, or clear learning data
|
|
91
|
+
- Opt-in for cross-project learning
|
|
92
|
+
- Never send learning data externally
|
|
93
|
+
|
|
94
|
+
2. **Transparency**
|
|
95
|
+
- Show "Learned from X sessions" indicators
|
|
96
|
+
- Explain filtering decisions with learning data
|
|
97
|
+
- Allow manual override of learned rules
|
|
98
|
+
- Provide learning statistics dashboard
|
|
99
|
+
|
|
100
|
+
3. **Performance**
|
|
101
|
+
- Learning should not slow down interviews
|
|
102
|
+
- Minimal storage footprint
|
|
103
|
+
- Fast pattern detection
|
|
104
|
+
- Efficient data structure
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
108
|
+
## 🚀 Features
|
|
109
|
+
|
|
110
|
+
### Feature 1: Skip Tracking System
|
|
111
|
+
|
|
112
|
+
**Description:** Track and store every manual skip action during interviews.
|
|
113
|
+
|
|
114
|
+
**Implementation Details:**
|
|
115
|
+
```javascript
|
|
116
|
+
// Data structure for skip tracking
|
|
117
|
+
{
|
|
118
|
+
sessionId: "uuid",
|
|
119
|
+
timestamp: "2025-10-04T10:30:00Z",
|
|
120
|
+
projectType: "cli-tool",
|
|
121
|
+
frameworks: ["commander"],
|
|
122
|
+
question: {
|
|
123
|
+
id: "q_deployment_host",
|
|
124
|
+
text: "Where will you deploy this application?",
|
|
125
|
+
category: "deployment",
|
|
126
|
+
phase: "architecture"
|
|
127
|
+
},
|
|
128
|
+
action: "skipped",
|
|
129
|
+
reason: "manual" | "filtered",
|
|
130
|
+
timeViewed: 2.5 // seconds before skip
|
|
131
|
+
}
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
**User Experience:**
|
|
135
|
+
- Silent tracking (no UI changes)
|
|
136
|
+
- Stored in `.adf/learning/skip-history.json`
|
|
137
|
+
- Option to view skip history: `adf config learning`
|
|
138
|
+
|
|
139
|
+
**Technical Requirements:**
|
|
140
|
+
- Modify interviewer.js to track skip events
|
|
141
|
+
- Create learning/skip-tracker.js module
|
|
142
|
+
- Add storage to .adf/learning/ directory
|
|
143
|
+
- Ensure atomic writes for data integrity
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
### Feature 2: Pattern Detection Engine
|
|
148
|
+
|
|
149
|
+
**Description:** Analyze skip history to identify patterns and generate learned rules.
|
|
150
|
+
|
|
151
|
+
**Pattern Types:**
|
|
152
|
+
|
|
153
|
+
**1. Consistent Skip Pattern**
|
|
154
|
+
```javascript
|
|
155
|
+
// User skips same question 3+ times across different sessions
|
|
156
|
+
{
|
|
157
|
+
pattern: "consistent_skip",
|
|
158
|
+
questionId: "q_deployment_host",
|
|
159
|
+
confidence: 90, // 9 of 10 sessions skipped
|
|
160
|
+
sessions: 10,
|
|
161
|
+
skipped: 9,
|
|
162
|
+
recommendation: "Auto-filter this question"
|
|
163
|
+
}
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
**2. Category Skip Pattern**
|
|
167
|
+
```javascript
|
|
168
|
+
// User skips 80%+ of questions in a category
|
|
169
|
+
{
|
|
170
|
+
pattern: "category_skip",
|
|
171
|
+
category: "deployment",
|
|
172
|
+
confidence: 85, // 85% skip rate
|
|
173
|
+
totalQuestions: 12,
|
|
174
|
+
skipped: 10,
|
|
175
|
+
recommendation: "Reduce relevance score for deployment questions"
|
|
176
|
+
}
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
**3. Project-Type Skip Pattern**
|
|
180
|
+
```javascript
|
|
181
|
+
// CLI projects consistently skip certain questions
|
|
182
|
+
{
|
|
183
|
+
pattern: "project_type_skip",
|
|
184
|
+
projectType: "cli-tool",
|
|
185
|
+
questionId: "q_ui_responsive",
|
|
186
|
+
confidence: 95,
|
|
187
|
+
recommendation: "Already filtered by base rules"
|
|
188
|
+
}
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
**4. Framework-Specific Pattern**
|
|
192
|
+
```javascript
|
|
193
|
+
// Next.js projects skip routing questions
|
|
194
|
+
{
|
|
195
|
+
pattern: "framework_skip",
|
|
196
|
+
framework: "Next.js",
|
|
197
|
+
questionId: "q_routing_setup",
|
|
198
|
+
confidence: 80,
|
|
199
|
+
sessions: 5,
|
|
200
|
+
recommendation: "Create framework-specific rule"
|
|
201
|
+
}
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
**5. User Preference Pattern**
|
|
205
|
+
```javascript
|
|
206
|
+
// User always gives brief answers to testing questions
|
|
207
|
+
{
|
|
208
|
+
pattern: "user_preference",
|
|
209
|
+
category: "testing",
|
|
210
|
+
answerLength: "brief", // avg < 50 words
|
|
211
|
+
confidence: 75,
|
|
212
|
+
recommendation: "Suggest skipping detailed testing questions"
|
|
213
|
+
}
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
**Implementation:**
|
|
217
|
+
```javascript
|
|
218
|
+
// lib/learning/pattern-detector.js
|
|
219
|
+
|
|
220
|
+
class PatternDetector {
|
|
221
|
+
constructor(skipHistory, answerHistory) {
|
|
222
|
+
this.skipHistory = skipHistory;
|
|
223
|
+
this.answerHistory = answerHistory;
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
detectPatterns() {
|
|
227
|
+
return {
|
|
228
|
+
consistentSkips: this.findConsistentSkips(),
|
|
229
|
+
categoryPatterns: this.findCategoryPatterns(),
|
|
230
|
+
frameworkPatterns: this.findFrameworkPatterns(),
|
|
231
|
+
userPreferences: this.findUserPreferences()
|
|
232
|
+
};
|
|
233
|
+
}
|
|
234
|
+
|
|
235
|
+
findConsistentSkips() {
|
|
236
|
+
// Group by questionId, calculate skip rate
|
|
237
|
+
// Return questions with >75% skip rate across 3+ sessions
|
|
238
|
+
}
|
|
239
|
+
|
|
240
|
+
findCategoryPatterns() {
|
|
241
|
+
// Group by category, calculate skip rate per category
|
|
242
|
+
// Return categories with >70% skip rate
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
findFrameworkPatterns() {
|
|
246
|
+
// Group by framework + questionId
|
|
247
|
+
// Find framework-specific skip patterns
|
|
248
|
+
}
|
|
249
|
+
|
|
250
|
+
findUserPreferences() {
|
|
251
|
+
// Analyze answer length, detail level
|
|
252
|
+
// Detect user's typical response style
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
---
|
|
258
|
+
|
|
259
|
+
### Feature 3: Adaptive Filtering
|
|
260
|
+
|
|
261
|
+
**Description:** Apply learned patterns to improve question filtering in real-time.
|
|
262
|
+
|
|
263
|
+
**How It Works:**
|
|
264
|
+
|
|
265
|
+
**Step 1: Load Learning Data**
|
|
266
|
+
```javascript
|
|
267
|
+
// On interview start
|
|
268
|
+
const learningData = await loadLearningData('.adf/learning/');
|
|
269
|
+
const patterns = detectPatterns(learningData);
|
|
270
|
+
```
|
|
271
|
+
|
|
272
|
+
**Step 2: Adjust Relevance Scores**
|
|
273
|
+
```javascript
|
|
274
|
+
function scoreQuestionRelevance(question, projectContext, learningData) {
|
|
275
|
+
let score = baseScore(question, projectContext); // Phase 4.1 scoring
|
|
276
|
+
|
|
277
|
+
// Apply learned patterns
|
|
278
|
+
if (learningData.consistentSkips.includes(question.id)) {
|
|
279
|
+
score -= 30; // Strong signal to skip
|
|
280
|
+
}
|
|
281
|
+
|
|
282
|
+
if (learningData.categorySkipRate[question.category] > 70) {
|
|
283
|
+
score -= 15; // Moderate signal to skip
|
|
284
|
+
}
|
|
285
|
+
|
|
286
|
+
if (learningData.frameworkSkips[framework]?.includes(question.id)) {
|
|
287
|
+
score -= 20; // Framework-specific skip
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
return score;
|
|
291
|
+
}
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
**Step 3: Explain Decisions**
|
|
295
|
+
```javascript
|
|
296
|
+
// When showing filtering summary
|
|
297
|
+
"🎯 Smart Filtering Results (with Learning):
|
|
298
|
+
✓ 10 relevant questions selected
|
|
299
|
+
○ 12 questions skipped
|
|
300
|
+
- 8 skipped by project type rules
|
|
301
|
+
- 4 skipped based on your history (learned from 5 sessions)
|
|
302
|
+
⏱️ Estimated time saved: ~24 minutes"
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
**User Control:**
|
|
306
|
+
```javascript
|
|
307
|
+
// Prompt before applying learned rules
|
|
308
|
+
"? I've learned from your previous 5 sessions. Apply learned preferences? (Y/n)
|
|
309
|
+
- Skip deployment questions (skipped in 4/5 sessions)
|
|
310
|
+
- Skip detailed testing questions (brief answers in 5/5 sessions)
|
|
311
|
+
|
|
312
|
+
You can review and adjust these in: adf config learning"
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
---
|
|
316
|
+
|
|
317
|
+
### Feature 4: Learning Management Interface
|
|
318
|
+
|
|
319
|
+
**Description:** CLI interface to view, manage, and control learning data.
|
|
320
|
+
|
|
321
|
+
**Command:** `adf config learning`
|
|
322
|
+
|
|
323
|
+
**Menu Options:**
|
|
324
|
+
|
|
325
|
+
```
|
|
326
|
+
┌─────────────────────────────────────────────────────┐
|
|
327
|
+
│ Learning System Configuration │
|
|
328
|
+
├─────────────────────────────────────────────────────┤
|
|
329
|
+
│ │
|
|
330
|
+
│ 📊 Learning Statistics │
|
|
331
|
+
│ Sessions analyzed: 12 │
|
|
332
|
+
│ Questions tracked: 240 │
|
|
333
|
+
│ Patterns detected: 8 │
|
|
334
|
+
│ Storage used: 145 KB │
|
|
335
|
+
│ │
|
|
336
|
+
│ 🎯 Detected Patterns │
|
|
337
|
+
│ • Consistently skip deployment questions (9/10) │
|
|
338
|
+
│ • Brief answers on testing (10/10) │
|
|
339
|
+
│ • Skip UI questions for CLI projects (5/5) │
|
|
340
|
+
│ │
|
|
341
|
+
│ ⚙️ Options │
|
|
342
|
+
│ 1. View Skip History │
|
|
343
|
+
│ 2. Review Detected Patterns │
|
|
344
|
+
│ 3. Enable/Disable Learning (✓ Enabled) │
|
|
345
|
+
│ 4. Clear Learning Data │
|
|
346
|
+
│ 5. Export Learning Data │
|
|
347
|
+
│ 6. Back to Main Menu │
|
|
348
|
+
│ │
|
|
349
|
+
└─────────────────────────────────────────────────────┘
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
**Option 1: View Skip History**
|
|
353
|
+
```
|
|
354
|
+
Skip History (Last 20 skips):
|
|
355
|
+
|
|
356
|
+
Session: 2025-10-04 10:30 AM (CLI Tool)
|
|
357
|
+
○ "Where will you deploy?" - deployment
|
|
358
|
+
○ "What CI/CD pipeline?" - deployment
|
|
359
|
+
✓ "What testing framework?" - answered (brief)
|
|
360
|
+
|
|
361
|
+
Session: 2025-10-03 02:15 PM (API Server)
|
|
362
|
+
○ "Responsive design approach?" - design
|
|
363
|
+
○ "Browser compatibility?" - frontend
|
|
364
|
+
✓ "Database choice?" - answered (detailed)
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
**Option 2: Review Detected Patterns**
|
|
368
|
+
```
|
|
369
|
+
Detected Patterns:
|
|
370
|
+
|
|
371
|
+
🔴 High Confidence (>80%)
|
|
372
|
+
• Skip deployment questions
|
|
373
|
+
Confidence: 90% (9 of 10 sessions)
|
|
374
|
+
Recommendation: Auto-filter deployment questions
|
|
375
|
+
[Apply] [Ignore] [Adjust]
|
|
376
|
+
|
|
377
|
+
🟡 Medium Confidence (60-80%)
|
|
378
|
+
• Brief testing answers
|
|
379
|
+
Confidence: 75% (avg 35 words)
|
|
380
|
+
Recommendation: Suggest skipping detailed testing questions
|
|
381
|
+
[Apply] [Ignore] [Adjust]
|
|
382
|
+
|
|
383
|
+
🟢 Low Confidence (<60%)
|
|
384
|
+
• Skip collaboration questions
|
|
385
|
+
Confidence: 50% (3 of 6 sessions)
|
|
386
|
+
Recommendation: Needs more data
|
|
387
|
+
[Track] [Ignore]
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
**Option 3: Enable/Disable Learning**
|
|
391
|
+
```
|
|
392
|
+
Learning System: ✓ Enabled
|
|
393
|
+
|
|
394
|
+
Settings:
|
|
395
|
+
[x] Track skip actions
|
|
396
|
+
[x] Track answer quality
|
|
397
|
+
[x] Detect patterns automatically
|
|
398
|
+
[x] Apply learned filters
|
|
399
|
+
[ ] Share anonymous patterns (opt-in)
|
|
400
|
+
|
|
401
|
+
Minimum sessions for pattern: 3
|
|
402
|
+
Minimum confidence for auto-filter: 75%
|
|
403
|
+
|
|
404
|
+
[Save] [Reset to Defaults] [Cancel]
|
|
405
|
+
```
|
|
406
|
+
|
|
407
|
+
**Option 4: Clear Learning Data**
|
|
408
|
+
```
|
|
409
|
+
⚠️ Clear Learning Data
|
|
410
|
+
|
|
411
|
+
This will delete:
|
|
412
|
+
• Skip history (12 sessions)
|
|
413
|
+
• Detected patterns (8 patterns)
|
|
414
|
+
• User preferences
|
|
415
|
+
|
|
416
|
+
This action cannot be undone.
|
|
417
|
+
|
|
418
|
+
? Are you sure you want to clear all learning data? (y/N)
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
---
|
|
422
|
+
|
|
423
|
+
### Feature 5: Learning Data Storage
|
|
424
|
+
|
|
425
|
+
**Description:** Secure, efficient storage of learning data in `.adf/learning/`
|
|
426
|
+
|
|
427
|
+
**Directory Structure:**
|
|
428
|
+
```
|
|
429
|
+
.adf/
|
|
430
|
+
├── learning/
|
|
431
|
+
│ ├── skip-history.json # All skip events
|
|
432
|
+
│ ├── answer-history.json # Answer metadata (length, quality)
|
|
433
|
+
│ ├── patterns.json # Detected patterns
|
|
434
|
+
│ ├── learned-rules.json # User-approved learned rules
|
|
435
|
+
│ ├── config.json # Learning system settings
|
|
436
|
+
│ └── stats.json # Aggregate statistics
|
|
437
|
+
├── .env # API keys
|
|
438
|
+
└── sessions/ # Session data
|
|
439
|
+
```
|
|
440
|
+
|
|
441
|
+
**skip-history.json:**
|
|
442
|
+
```json
|
|
443
|
+
{
|
|
444
|
+
"version": "1.0",
|
|
445
|
+
"sessions": [
|
|
446
|
+
{
|
|
447
|
+
"sessionId": "550e8400-e29b-41d4-a716-446655440000",
|
|
448
|
+
"timestamp": "2025-10-04T10:30:00Z",
|
|
449
|
+
"projectType": "cli-tool",
|
|
450
|
+
"frameworks": ["commander"],
|
|
451
|
+
"skips": [
|
|
452
|
+
{
|
|
453
|
+
"questionId": "q_deployment_host",
|
|
454
|
+
"text": "Where will you deploy?",
|
|
455
|
+
"category": "deployment",
|
|
456
|
+
"phase": "architecture",
|
|
457
|
+
"action": "skipped",
|
|
458
|
+
"reason": "manual",
|
|
459
|
+
"timeViewed": 2.5
|
|
460
|
+
}
|
|
461
|
+
]
|
|
462
|
+
}
|
|
463
|
+
]
|
|
464
|
+
}
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
**patterns.json:**
|
|
468
|
+
```json
|
|
469
|
+
{
|
|
470
|
+
"version": "1.0",
|
|
471
|
+
"lastUpdated": "2025-10-04T10:35:00Z",
|
|
472
|
+
"patterns": [
|
|
473
|
+
{
|
|
474
|
+
"id": "pattern_001",
|
|
475
|
+
"type": "consistent_skip",
|
|
476
|
+
"questionId": "q_deployment_host",
|
|
477
|
+
"confidence": 90,
|
|
478
|
+
"sessionsAnalyzed": 10,
|
|
479
|
+
"skipCount": 9,
|
|
480
|
+
"status": "active",
|
|
481
|
+
"userApproved": true
|
|
482
|
+
},
|
|
483
|
+
{
|
|
484
|
+
"id": "pattern_002",
|
|
485
|
+
"type": "category_skip",
|
|
486
|
+
"category": "deployment",
|
|
487
|
+
"confidence": 85,
|
|
488
|
+
"totalQuestions": 12,
|
|
489
|
+
"skipCount": 10,
|
|
490
|
+
"status": "active",
|
|
491
|
+
"userApproved": false
|
|
492
|
+
}
|
|
493
|
+
]
|
|
494
|
+
}
|
|
495
|
+
```
|
|
496
|
+
|
|
497
|
+
**learned-rules.json:**
|
|
498
|
+
```json
|
|
499
|
+
{
|
|
500
|
+
"version": "1.0",
|
|
501
|
+
"rules": [
|
|
502
|
+
{
|
|
503
|
+
"id": "rule_001",
|
|
504
|
+
"type": "skip_question",
|
|
505
|
+
"questionId": "q_deployment_host",
|
|
506
|
+
"reason": "User consistently skips (90% confidence)",
|
|
507
|
+
"appliedSince": "2025-10-04T10:35:00Z",
|
|
508
|
+
"enabled": true
|
|
509
|
+
},
|
|
510
|
+
{
|
|
511
|
+
"id": "rule_002",
|
|
512
|
+
"type": "reduce_relevance",
|
|
513
|
+
"category": "deployment",
|
|
514
|
+
"adjustment": -20,
|
|
515
|
+
"reason": "Category skip rate: 85%",
|
|
516
|
+
"appliedSince": "2025-10-04T10:35:00Z",
|
|
517
|
+
"enabled": true
|
|
518
|
+
}
|
|
519
|
+
]
|
|
520
|
+
}
|
|
521
|
+
```
|
|
522
|
+
|
|
523
|
+
---
|
|
524
|
+
|
|
525
|
+
## 🏗️ Technical Architecture
|
|
526
|
+
|
|
527
|
+
### Component Diagram
|
|
528
|
+
|
|
529
|
+
```
|
|
530
|
+
┌─────────────────────────────────────────────────────────┐
|
|
531
|
+
│ Interviewer │
|
|
532
|
+
│ - Runs interview questions │
|
|
533
|
+
│ - Captures user actions │
|
|
534
|
+
└────────────┬────────────────────────────────────────────┘
|
|
535
|
+
│
|
|
536
|
+
│ Events (skip, answer, time)
|
|
537
|
+
│
|
|
538
|
+
▼
|
|
539
|
+
┌─────────────────────────────────────────────────────────┐
|
|
540
|
+
│ Skip Tracker (NEW) │
|
|
541
|
+
│ - Records skip events │
|
|
542
|
+
│ - Records answer metadata │
|
|
543
|
+
│ - Writes to skip-history.json │
|
|
544
|
+
└────────────┬────────────────────────────────────────────┘
|
|
545
|
+
│
|
|
546
|
+
│ History data
|
|
547
|
+
│
|
|
548
|
+
▼
|
|
549
|
+
┌─────────────────────────────────────────────────────────┐
|
|
550
|
+
│ Pattern Detector (NEW) │
|
|
551
|
+
│ - Analyzes skip history │
|
|
552
|
+
│ - Detects consistent patterns │
|
|
553
|
+
│ - Generates learned rules │
|
|
554
|
+
│ - Writes to patterns.json │
|
|
555
|
+
└────────────┬────────────────────────────────────────────┘
|
|
556
|
+
│
|
|
557
|
+
│ Patterns + Rules
|
|
558
|
+
│
|
|
559
|
+
▼
|
|
560
|
+
┌─────────────────────────────────────────────────────────┐
|
|
561
|
+
│ Question Filter (Enhanced) │
|
|
562
|
+
│ - Base relevance scoring (Phase 4.1) │
|
|
563
|
+
│ - Apply learned rules (Phase 4.2) │
|
|
564
|
+
│ - Adjust scores based on patterns │
|
|
565
|
+
└────────────┬────────────────────────────────────────────┘
|
|
566
|
+
│
|
|
567
|
+
│ Filtered questions
|
|
568
|
+
│
|
|
569
|
+
▼
|
|
570
|
+
┌─────────────────────────────────────────────────────────┐
|
|
571
|
+
│ Learning Manager (NEW) │
|
|
572
|
+
│ - CLI interface for learning data │
|
|
573
|
+
│ - View/edit/clear learning data │
|
|
574
|
+
│ - Enable/disable learning │
|
|
575
|
+
└─────────────────────────────────────────────────────────┘
|
|
576
|
+
```
|
|
577
|
+
|
|
578
|
+
### Data Flow
|
|
579
|
+
|
|
580
|
+
**Interview Flow with Learning:**
|
|
581
|
+
```
|
|
582
|
+
1. User starts interview
|
|
583
|
+
↓
|
|
584
|
+
2. Load learning data from .adf/learning/
|
|
585
|
+
↓
|
|
586
|
+
3. Detect patterns from history
|
|
587
|
+
↓
|
|
588
|
+
4. Apply base filtering rules (Phase 4.1)
|
|
589
|
+
↓
|
|
590
|
+
5. Apply learned rules (Phase 4.2)
|
|
591
|
+
↓
|
|
592
|
+
6. Show filtering summary with learning info
|
|
593
|
+
↓
|
|
594
|
+
7. Ask questions
|
|
595
|
+
↓
|
|
596
|
+
8. Track skip events → skip-tracker
|
|
597
|
+
↓
|
|
598
|
+
9. Track answers → answer-tracker
|
|
599
|
+
↓
|
|
600
|
+
10. On session end: analyze patterns
|
|
601
|
+
↓
|
|
602
|
+
11. Update patterns.json
|
|
603
|
+
↓
|
|
604
|
+
12. Apply high-confidence patterns to learned-rules.json
|
|
605
|
+
```
|
|
606
|
+
|
|
607
|
+
---
|
|
608
|
+
|
|
609
|
+
## 📋 Implementation Plan
|
|
610
|
+
|
|
611
|
+
### Phase 4.2.1: Skip Tracking Foundation (Session 1)
|
|
612
|
+
|
|
613
|
+
**Goal:** Build the tracking system to record user behavior
|
|
614
|
+
|
|
615
|
+
**Tasks:**
|
|
616
|
+
1. Create `lib/learning/skip-tracker.js`
|
|
617
|
+
- Track skip events
|
|
618
|
+
- Track answer metadata
|
|
619
|
+
- Write to skip-history.json
|
|
620
|
+
- Atomic file writes
|
|
621
|
+
|
|
622
|
+
2. Create `lib/learning/storage.js`
|
|
623
|
+
- Ensure .adf/learning/ directory
|
|
624
|
+
- Read/write learning data files
|
|
625
|
+
- Handle file locking
|
|
626
|
+
- Data validation
|
|
627
|
+
|
|
628
|
+
3. Modify `lib/frameworks/interviewer.js`
|
|
629
|
+
- Integrate skip-tracker
|
|
630
|
+
- Capture skip events
|
|
631
|
+
- Capture answer events
|
|
632
|
+
- Session metadata
|
|
633
|
+
|
|
634
|
+
4. Create tests
|
|
635
|
+
- Skip tracking tests
|
|
636
|
+
- Storage tests
|
|
637
|
+
- Integration tests
|
|
638
|
+
|
|
639
|
+
**Deliverables:**
|
|
640
|
+
- Skip tracking system functional
|
|
641
|
+
- Data stored in .adf/learning/
|
|
642
|
+
- Tests passing
|
|
643
|
+
|
|
644
|
+
---
|
|
645
|
+
|
|
646
|
+
### Phase 4.2.2: Pattern Detection (Session 2)
|
|
647
|
+
|
|
648
|
+
**Goal:** Analyze skip history to detect patterns
|
|
649
|
+
|
|
650
|
+
**Tasks:**
|
|
651
|
+
1. Create `lib/learning/pattern-detector.js`
|
|
652
|
+
- Consistent skip detection
|
|
653
|
+
- Category pattern detection
|
|
654
|
+
- Framework pattern detection
|
|
655
|
+
- User preference detection
|
|
656
|
+
|
|
657
|
+
2. Create `lib/learning/rule-generator.js`
|
|
658
|
+
- Convert patterns to rules
|
|
659
|
+
- Calculate confidence scores
|
|
660
|
+
- Generate recommendations
|
|
661
|
+
|
|
662
|
+
3. Add pattern analysis to interview end
|
|
663
|
+
- Run pattern detection
|
|
664
|
+
- Update patterns.json
|
|
665
|
+
- Generate learned-rules.json
|
|
666
|
+
|
|
667
|
+
4. Create tests
|
|
668
|
+
- Pattern detection tests
|
|
669
|
+
- Rule generation tests
|
|
670
|
+
- Confidence calculation tests
|
|
671
|
+
|
|
672
|
+
**Deliverables:**
|
|
673
|
+
- Pattern detection working
|
|
674
|
+
- Learned rules generated
|
|
675
|
+
- Tests passing
|
|
676
|
+
|
|
677
|
+
---
|
|
678
|
+
|
|
679
|
+
### Phase 4.2.3: Adaptive Filtering (Session 3)
|
|
680
|
+
|
|
681
|
+
**Goal:** Apply learned rules to improve filtering
|
|
682
|
+
|
|
683
|
+
**Tasks:**
|
|
684
|
+
1. Modify `lib/filters/question-filter.js`
|
|
685
|
+
- Load learned rules
|
|
686
|
+
- Apply learned rules to relevance scoring
|
|
687
|
+
- Add learning-based explanations
|
|
688
|
+
|
|
689
|
+
2. Update filtering summary
|
|
690
|
+
- Show base filtering results
|
|
691
|
+
- Show learning-based filtering
|
|
692
|
+
- Explain learned rules
|
|
693
|
+
|
|
694
|
+
3. Add user prompts
|
|
695
|
+
- "Apply learned preferences?"
|
|
696
|
+
- Show what will be applied
|
|
697
|
+
- Allow opt-out
|
|
698
|
+
|
|
699
|
+
4. Create tests
|
|
700
|
+
- Adaptive filtering tests
|
|
701
|
+
- Score adjustment tests
|
|
702
|
+
- Integration tests
|
|
703
|
+
|
|
704
|
+
**Deliverables:**
|
|
705
|
+
- Learned rules applied to filtering
|
|
706
|
+
- User can opt in/out
|
|
707
|
+
- Tests passing
|
|
708
|
+
|
|
709
|
+
---
|
|
710
|
+
|
|
711
|
+
### Phase 4.2.4: Learning Management UI (Session 4)
|
|
712
|
+
|
|
713
|
+
**Goal:** Build CLI interface to manage learning
|
|
714
|
+
|
|
715
|
+
**Tasks:**
|
|
716
|
+
1. Create `lib/learning/learning-manager.js`
|
|
717
|
+
- View skip history
|
|
718
|
+
- Review patterns
|
|
719
|
+
- Enable/disable learning
|
|
720
|
+
- Clear data
|
|
721
|
+
|
|
722
|
+
2. Add to `adf config` command
|
|
723
|
+
- New "Learning System" menu option
|
|
724
|
+
- Statistics dashboard
|
|
725
|
+
- Pattern review interface
|
|
726
|
+
|
|
727
|
+
3. Add data export/import
|
|
728
|
+
- Export learning data
|
|
729
|
+
- Import learning data
|
|
730
|
+
- Backup/restore
|
|
731
|
+
|
|
732
|
+
4. Create tests
|
|
733
|
+
- Learning manager tests
|
|
734
|
+
- UI interaction tests
|
|
735
|
+
|
|
736
|
+
**Deliverables:**
|
|
737
|
+
- Learning management UI complete
|
|
738
|
+
- User can control all learning features
|
|
739
|
+
- Tests passing
|
|
740
|
+
|
|
741
|
+
---
|
|
742
|
+
|
|
743
|
+
## ✅ Success Criteria
|
|
744
|
+
|
|
745
|
+
### User Experience
|
|
746
|
+
- [ ] Skip events tracked without slowing down interview
|
|
747
|
+
- [ ] Patterns detected after 3+ sessions
|
|
748
|
+
- [ ] Learned rules improve filtering accuracy by 20%+
|
|
749
|
+
- [ ] User can view and manage learning data easily
|
|
750
|
+
- [ ] Clear explanations for all learning-based decisions
|
|
751
|
+
|
|
752
|
+
### Technical
|
|
753
|
+
- [ ] All learning data stored in `.adf/learning/`
|
|
754
|
+
- [ ] Pattern detection runs in <500ms
|
|
755
|
+
- [ ] Learning storage <5MB for 100 sessions
|
|
756
|
+
- [ ] 90%+ test coverage for learning components
|
|
757
|
+
- [ ] No data loss (atomic writes, backups)
|
|
758
|
+
|
|
759
|
+
### Privacy & Control
|
|
760
|
+
- [ ] All data stored locally only
|
|
761
|
+
- [ ] User can disable learning anytime
|
|
762
|
+
- [ ] User can clear all learning data
|
|
763
|
+
- [ ] Transparent explanations for all patterns
|
|
764
|
+
- [ ] Opt-in for any cross-project features
|
|
765
|
+
|
|
766
|
+
---
|
|
767
|
+
|
|
768
|
+
## 🎯 Key Metrics
|
|
769
|
+
|
|
770
|
+
### Learning Effectiveness
|
|
771
|
+
- **Pattern Detection Accuracy:** % of patterns that user approves
|
|
772
|
+
- **Filtering Improvement:** Reduction in manual skips after learning
|
|
773
|
+
- **Time Savings:** Additional time saved beyond Phase 4.1
|
|
774
|
+
- **User Satisfaction:** Do users enable/keep learning enabled?
|
|
775
|
+
|
|
776
|
+
### Performance
|
|
777
|
+
- **Tracking Overhead:** Time added to interview by tracking
|
|
778
|
+
- **Pattern Detection Time:** Time to analyze and detect patterns
|
|
779
|
+
- **Storage Growth:** MB per session
|
|
780
|
+
- **Memory Usage:** Memory overhead during interview
|
|
781
|
+
|
|
782
|
+
---
|
|
783
|
+
|
|
784
|
+
## 🚨 Risks & Mitigation
|
|
785
|
+
|
|
786
|
+
### Risk 1: Over-Filtering
|
|
787
|
+
**Risk:** System learns to skip too many questions
|
|
788
|
+
**Mitigation:**
|
|
789
|
+
- Require high confidence (>75%) for auto-skip
|
|
790
|
+
- User review of patterns before applying
|
|
791
|
+
- Easy override mechanism
|
|
792
|
+
- Periodic re-evaluation of patterns
|
|
793
|
+
|
|
794
|
+
### Risk 2: Privacy Concerns
|
|
795
|
+
**Risk:** Users concerned about data collection
|
|
796
|
+
**Mitigation:**
|
|
797
|
+
- All data stored locally in .adf/
|
|
798
|
+
- Clear documentation of what's tracked
|
|
799
|
+
- Easy disable/clear options
|
|
800
|
+
- Transparent explanations
|
|
801
|
+
|
|
802
|
+
### Risk 3: Storage Growth
|
|
803
|
+
**Risk:** Learning data grows too large
|
|
804
|
+
**Mitigation:**
|
|
805
|
+
- Compress old history (>100 sessions)
|
|
806
|
+
- Archive old sessions
|
|
807
|
+
- Configurable retention period
|
|
808
|
+
- Warn user at 10MB+
|
|
809
|
+
|
|
810
|
+
### Risk 4: Pattern Noise
|
|
811
|
+
**Risk:** False patterns from limited data
|
|
812
|
+
**Mitigation:**
|
|
813
|
+
- Require minimum 3 sessions
|
|
814
|
+
- Confidence thresholds
|
|
815
|
+
- User approval for patterns
|
|
816
|
+
- Decay old patterns over time
|
|
817
|
+
|
|
818
|
+
---
|
|
819
|
+
|
|
820
|
+
## 📚 Dependencies
|
|
821
|
+
|
|
822
|
+
### Required (Phase 4.1)
|
|
823
|
+
- ✅ Project analyzer (Phase 4.1)
|
|
824
|
+
- ✅ Question filter (Phase 4.1)
|
|
825
|
+
- ✅ Interviewer integration (Phase 4.1)
|
|
826
|
+
|
|
827
|
+
### Optional
|
|
828
|
+
- AI provider (for future AI-enhanced pattern detection)
|
|
829
|
+
- Multi-user environments (for shared learning)
|
|
830
|
+
|
|
831
|
+
---
|
|
832
|
+
|
|
833
|
+
## 🔮 Future Enhancements (Post-4.2)
|
|
834
|
+
|
|
835
|
+
### Phase 4.3: AI-Enhanced Learning
|
|
836
|
+
- Use AI to detect subtle patterns
|
|
837
|
+
- AI-powered question relevance scoring
|
|
838
|
+
- Natural language pattern descriptions
|
|
839
|
+
- Smart recommendations based on project goals
|
|
840
|
+
|
|
841
|
+
### Phase 4.4: Cross-Project Learning
|
|
842
|
+
- Learn from patterns across all user projects
|
|
843
|
+
- Detect project archetypes
|
|
844
|
+
- Community-driven patterns (opt-in)
|
|
845
|
+
- Shared best practices
|
|
846
|
+
|
|
847
|
+
### Phase 4.5: Predictive Intelligence
|
|
848
|
+
- Predict likely answers based on history
|
|
849
|
+
- Suggest answer completions
|
|
850
|
+
- Pre-fill common responses
|
|
851
|
+
- Detect answer inconsistencies
|
|
852
|
+
|
|
853
|
+
---
|
|
854
|
+
|
|
855
|
+
## 📖 Documentation Requirements
|
|
856
|
+
|
|
857
|
+
### User Documentation
|
|
858
|
+
- README: Add Learning System feature
|
|
859
|
+
- CHANGELOG: Document Phase 4.2
|
|
860
|
+
- New guide: "Understanding the Learning System"
|
|
861
|
+
- FAQ: Common questions about learning
|
|
862
|
+
|
|
863
|
+
### Technical Documentation
|
|
864
|
+
- Learning System Architecture
|
|
865
|
+
- Data format specifications
|
|
866
|
+
- Pattern detection algorithms
|
|
867
|
+
- Privacy and data handling
|
|
868
|
+
|
|
869
|
+
### Session Documentation
|
|
870
|
+
- Planning doc (this file)
|
|
871
|
+
- Implementation session notes
|
|
872
|
+
- Testing and validation results
|
|
873
|
+
- Lessons learned
|
|
874
|
+
|
|
875
|
+
---
|
|
876
|
+
|
|
877
|
+
**Status:** 📋 Ready for Implementation
|
|
878
|
+
**Next Step:** Create feature branch and begin Phase 4.2.1 (Skip Tracking Foundation)
|
|
879
|
+
**Estimated Timeline:** 2-3 development sessions
|
|
880
|
+
**Complexity:** Medium
|
|
881
|
+
**Value:** High (personalized, adaptive filtering)
|