cursor-recursive-rag 0.2.0-alpha.2 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +179 -203
- package/dist/adapters/llm/anthropic.d.ts +27 -0
- package/dist/adapters/llm/anthropic.d.ts.map +1 -0
- package/dist/adapters/llm/anthropic.js +287 -0
- package/dist/adapters/llm/anthropic.js.map +1 -0
- package/dist/adapters/llm/base.d.ts +62 -0
- package/dist/adapters/llm/base.d.ts.map +1 -0
- package/dist/adapters/llm/base.js +140 -0
- package/dist/adapters/llm/base.js.map +1 -0
- package/dist/adapters/llm/deepseek.d.ts +24 -0
- package/dist/adapters/llm/deepseek.d.ts.map +1 -0
- package/dist/adapters/llm/deepseek.js +228 -0
- package/dist/adapters/llm/deepseek.js.map +1 -0
- package/dist/adapters/llm/groq.d.ts +25 -0
- package/dist/adapters/llm/groq.d.ts.map +1 -0
- package/dist/adapters/llm/groq.js +265 -0
- package/dist/adapters/llm/groq.js.map +1 -0
- package/dist/adapters/llm/index.d.ts +62 -0
- package/dist/adapters/llm/index.d.ts.map +1 -0
- package/dist/adapters/llm/index.js +380 -0
- package/dist/adapters/llm/index.js.map +1 -0
- package/dist/adapters/llm/ollama.d.ts +23 -0
- package/dist/adapters/llm/ollama.d.ts.map +1 -0
- package/dist/adapters/llm/ollama.js +261 -0
- package/dist/adapters/llm/ollama.js.map +1 -0
- package/dist/adapters/llm/openai.d.ts +22 -0
- package/dist/adapters/llm/openai.d.ts.map +1 -0
- package/dist/adapters/llm/openai.js +232 -0
- package/dist/adapters/llm/openai.js.map +1 -0
- package/dist/adapters/llm/openrouter.d.ts +27 -0
- package/dist/adapters/llm/openrouter.d.ts.map +1 -0
- package/dist/adapters/llm/openrouter.js +305 -0
- package/dist/adapters/llm/openrouter.js.map +1 -0
- package/dist/adapters/vector/index.d.ts.map +1 -1
- package/dist/adapters/vector/index.js +8 -0
- package/dist/adapters/vector/index.js.map +1 -1
- package/dist/adapters/vector/redis-native.d.ts +35 -0
- package/dist/adapters/vector/redis-native.d.ts.map +1 -0
- package/dist/adapters/vector/redis-native.js +170 -0
- package/dist/adapters/vector/redis-native.js.map +1 -0
- package/dist/cli/commands/chat.d.ts +4 -0
- package/dist/cli/commands/chat.d.ts.map +1 -0
- package/dist/cli/commands/chat.js +374 -0
- package/dist/cli/commands/chat.js.map +1 -0
- package/dist/cli/commands/maintenance.d.ts +4 -0
- package/dist/cli/commands/maintenance.d.ts.map +1 -0
- package/dist/cli/commands/maintenance.js +237 -0
- package/dist/cli/commands/maintenance.js.map +1 -0
- package/dist/cli/commands/rules.d.ts +9 -0
- package/dist/cli/commands/rules.d.ts.map +1 -0
- package/dist/cli/commands/rules.js +639 -0
- package/dist/cli/commands/rules.js.map +1 -0
- package/dist/cli/commands/setup.js +5 -4
- package/dist/cli/commands/setup.js.map +1 -1
- package/dist/cli/index.js +6 -0
- package/dist/cli/index.js.map +1 -1
- package/dist/config/memoryConfig.d.ts +427 -0
- package/dist/config/memoryConfig.d.ts.map +1 -0
- package/dist/config/memoryConfig.js +258 -0
- package/dist/config/memoryConfig.js.map +1 -0
- package/dist/config/rulesConfig.d.ts +486 -0
- package/dist/config/rulesConfig.d.ts.map +1 -0
- package/dist/config/rulesConfig.js +345 -0
- package/dist/config/rulesConfig.js.map +1 -0
- package/dist/dashboard/coreTools.d.ts +14 -0
- package/dist/dashboard/coreTools.d.ts.map +1 -0
- package/dist/dashboard/coreTools.js +413 -0
- package/dist/dashboard/coreTools.js.map +1 -0
- package/dist/dashboard/public/index.html +1982 -13
- package/dist/dashboard/server.d.ts +1 -8
- package/dist/dashboard/server.d.ts.map +1 -1
- package/dist/dashboard/server.js +846 -13
- package/dist/dashboard/server.js.map +1 -1
- package/dist/dashboard/toolRegistry.d.ts +192 -0
- package/dist/dashboard/toolRegistry.d.ts.map +1 -0
- package/dist/dashboard/toolRegistry.js +322 -0
- package/dist/dashboard/toolRegistry.js.map +1 -0
- package/dist/proxy/index.d.ts +1 -1
- package/dist/proxy/index.d.ts.map +1 -1
- package/dist/proxy/index.js +9 -6
- package/dist/proxy/index.js.map +1 -1
- package/dist/server/index.js +21 -0
- package/dist/server/index.js.map +1 -1
- package/dist/server/tools/crawl.d.ts.map +1 -1
- package/dist/server/tools/crawl.js +8 -0
- package/dist/server/tools/crawl.js.map +1 -1
- package/dist/server/tools/index.d.ts.map +1 -1
- package/dist/server/tools/index.js +19 -1
- package/dist/server/tools/index.js.map +1 -1
- package/dist/server/tools/ingest.d.ts.map +1 -1
- package/dist/server/tools/ingest.js +5 -0
- package/dist/server/tools/ingest.js.map +1 -1
- package/dist/server/tools/memory.d.ts +250 -0
- package/dist/server/tools/memory.d.ts.map +1 -0
- package/dist/server/tools/memory.js +472 -0
- package/dist/server/tools/memory.js.map +1 -0
- package/dist/server/tools/recursive-query.d.ts.map +1 -1
- package/dist/server/tools/recursive-query.js +6 -0
- package/dist/server/tools/recursive-query.js.map +1 -1
- package/dist/server/tools/search.d.ts.map +1 -1
- package/dist/server/tools/search.js +6 -0
- package/dist/server/tools/search.js.map +1 -1
- package/dist/services/activity-log.d.ts +10 -0
- package/dist/services/activity-log.d.ts.map +1 -0
- package/dist/services/activity-log.js +53 -0
- package/dist/services/activity-log.js.map +1 -0
- package/dist/services/categoryManager.d.ts +110 -0
- package/dist/services/categoryManager.d.ts.map +1 -0
- package/dist/services/categoryManager.js +549 -0
- package/dist/services/categoryManager.js.map +1 -0
- package/dist/services/contextEnvironment.d.ts +206 -0
- package/dist/services/contextEnvironment.d.ts.map +1 -0
- package/dist/services/contextEnvironment.js +481 -0
- package/dist/services/contextEnvironment.js.map +1 -0
- package/dist/services/conversationProcessor.d.ts +99 -0
- package/dist/services/conversationProcessor.d.ts.map +1 -0
- package/dist/services/conversationProcessor.js +311 -0
- package/dist/services/conversationProcessor.js.map +1 -0
- package/dist/services/cursorChatReader.d.ts +129 -0
- package/dist/services/cursorChatReader.d.ts.map +1 -0
- package/dist/services/cursorChatReader.js +419 -0
- package/dist/services/cursorChatReader.js.map +1 -0
- package/dist/services/decayCalculator.d.ts +85 -0
- package/dist/services/decayCalculator.d.ts.map +1 -0
- package/dist/services/decayCalculator.js +182 -0
- package/dist/services/decayCalculator.js.map +1 -0
- package/dist/services/enhancedVectorStore.d.ts +102 -0
- package/dist/services/enhancedVectorStore.d.ts.map +1 -0
- package/dist/services/enhancedVectorStore.js +245 -0
- package/dist/services/enhancedVectorStore.js.map +1 -0
- package/dist/services/hybridScorer.d.ts +120 -0
- package/dist/services/hybridScorer.d.ts.map +1 -0
- package/dist/services/hybridScorer.js +334 -0
- package/dist/services/hybridScorer.js.map +1 -0
- package/dist/services/knowledgeExtractor.d.ts +45 -0
- package/dist/services/knowledgeExtractor.d.ts.map +1 -0
- package/dist/services/knowledgeExtractor.js +436 -0
- package/dist/services/knowledgeExtractor.js.map +1 -0
- package/dist/services/knowledgeStorage.d.ts +102 -0
- package/dist/services/knowledgeStorage.d.ts.map +1 -0
- package/dist/services/knowledgeStorage.js +383 -0
- package/dist/services/knowledgeStorage.js.map +1 -0
- package/dist/services/maintenanceScheduler.d.ts +89 -0
- package/dist/services/maintenanceScheduler.d.ts.map +1 -0
- package/dist/services/maintenanceScheduler.js +479 -0
- package/dist/services/maintenanceScheduler.js.map +1 -0
- package/dist/services/memoryMetadataStore.d.ts +62 -0
- package/dist/services/memoryMetadataStore.d.ts.map +1 -0
- package/dist/services/memoryMetadataStore.js +570 -0
- package/dist/services/memoryMetadataStore.js.map +1 -0
- package/dist/services/recursiveRetrieval.d.ts +122 -0
- package/dist/services/recursiveRetrieval.d.ts.map +1 -0
- package/dist/services/recursiveRetrieval.js +443 -0
- package/dist/services/recursiveRetrieval.js.map +1 -0
- package/dist/services/relationshipGraph.d.ts +77 -0
- package/dist/services/relationshipGraph.d.ts.map +1 -0
- package/dist/services/relationshipGraph.js +411 -0
- package/dist/services/relationshipGraph.js.map +1 -0
- package/dist/services/rlmSafeguards.d.ts +273 -0
- package/dist/services/rlmSafeguards.d.ts.map +1 -0
- package/dist/services/rlmSafeguards.js +705 -0
- package/dist/services/rlmSafeguards.js.map +1 -0
- package/dist/services/rulesAnalyzer.d.ts +119 -0
- package/dist/services/rulesAnalyzer.d.ts.map +1 -0
- package/dist/services/rulesAnalyzer.js +768 -0
- package/dist/services/rulesAnalyzer.js.map +1 -0
- package/dist/services/rulesMerger.d.ts +75 -0
- package/dist/services/rulesMerger.d.ts.map +1 -0
- package/dist/services/rulesMerger.js +404 -0
- package/dist/services/rulesMerger.js.map +1 -0
- package/dist/services/rulesParser.d.ts +127 -0
- package/dist/services/rulesParser.d.ts.map +1 -0
- package/dist/services/rulesParser.js +594 -0
- package/dist/services/rulesParser.js.map +1 -0
- package/dist/services/smartChunker.d.ts +110 -0
- package/dist/services/smartChunker.d.ts.map +1 -0
- package/dist/services/smartChunker.js +520 -0
- package/dist/services/smartChunker.js.map +1 -0
- package/dist/types/categories.d.ts +105 -0
- package/dist/types/categories.d.ts.map +1 -0
- package/dist/types/categories.js +108 -0
- package/dist/types/categories.js.map +1 -0
- package/dist/types/extractedKnowledge.d.ts +233 -0
- package/dist/types/extractedKnowledge.d.ts.map +1 -0
- package/dist/types/extractedKnowledge.js +56 -0
- package/dist/types/extractedKnowledge.js.map +1 -0
- package/dist/types/index.d.ts +9 -2
- package/dist/types/index.d.ts.map +1 -1
- package/dist/types/index.js +12 -1
- package/dist/types/index.js.map +1 -1
- package/dist/types/llmProvider.d.ts +282 -0
- package/dist/types/llmProvider.d.ts.map +1 -0
- package/dist/types/llmProvider.js +48 -0
- package/dist/types/llmProvider.js.map +1 -0
- package/dist/types/memory.d.ts +227 -0
- package/dist/types/memory.d.ts.map +1 -0
- package/dist/types/memory.js +76 -0
- package/dist/types/memory.js.map +1 -0
- package/dist/types/relationships.d.ts +167 -0
- package/dist/types/relationships.d.ts.map +1 -0
- package/dist/types/relationships.js +106 -0
- package/dist/types/relationships.js.map +1 -0
- package/dist/types/rulesOptimizer.d.ts +345 -0
- package/dist/types/rulesOptimizer.d.ts.map +1 -0
- package/dist/types/rulesOptimizer.js +22 -0
- package/dist/types/rulesOptimizer.js.map +1 -0
- package/docs/cursor-recursive-rag-memory-spec.md +4569 -0
- package/docs/cursor-recursive-rag-tasks.md +1355 -0
- package/package.json +6 -3
- package/restart-rag.sh +16 -0
|
@@ -0,0 +1,1355 @@
|
|
|
1
|
+
# cursor-recursive-rag Memory Features - Task Breakdown
|
|
2
|
+
|
|
3
|
+
## Overview
|
|
4
|
+
|
|
5
|
+
This document provides a Linear-compatible task breakdown for implementing advanced memory features in cursor-recursive-rag. Each task includes estimates and acceptance criteria.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Epic: Phase 1 - Foundation (Enhanced Schema & Temporal Decay)
|
|
10
|
+
|
|
11
|
+
### CRR-101: Define Enhanced Chunk Interface ✅
|
|
12
|
+
**Estimate**: 1 point
|
|
13
|
+
**Labels**: foundation, types
|
|
14
|
+
**Status**: COMPLETED
|
|
15
|
+
|
|
16
|
+
Create new TypeScript interfaces for enhanced chunks with temporal tracking, importance, decay scores, and entity tags.
|
|
17
|
+
|
|
18
|
+
**File**: `src/types/memory.ts`
|
|
19
|
+
|
|
20
|
+
**Acceptance Criteria**:
|
|
21
|
+
- [x] EnhancedChunk interface defined with all new fields
|
|
22
|
+
- [x] ChunkType enum with all knowledge types
|
|
23
|
+
- [x] EntityTag and EntityType defined
|
|
24
|
+
- [x] Types exported from main index
|
|
25
|
+
- [x] Existing code continues to compile
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
### CRR-102: Create Memory Metadata Store ✅
|
|
30
|
+
**Estimate**: 3 points
|
|
31
|
+
**Labels**: foundation, database
|
|
32
|
+
**Blocks**: CRR-103, CRR-104
|
|
33
|
+
**Status**: COMPLETED
|
|
34
|
+
|
|
35
|
+
Implement SQLite-based metadata store for temporal tracking, relationships, and categories.
|
|
36
|
+
|
|
37
|
+
**File**: `src/services/memoryMetadataStore.ts`
|
|
38
|
+
|
|
39
|
+
**Acceptance Criteria**:
|
|
40
|
+
- [x] SQLite database created on first run
|
|
41
|
+
- [x] Tables: chunks_metadata, relationships, access_log, categories, processed_conversations
|
|
42
|
+
- [x] CRUD operations for all tables
|
|
43
|
+
- [x] Indexes created for performance
|
|
44
|
+
- [x] Access recording updates last_accessed_at and access_count
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
### CRR-103: Implement Decay Score Calculator ✅
|
|
49
|
+
**Estimate**: 2 points
|
|
50
|
+
**Labels**: foundation, algorithm
|
|
51
|
+
**Blocked by**: CRR-102
|
|
52
|
+
**Status**: COMPLETED
|
|
53
|
+
|
|
54
|
+
Create decay calculator with configurable half-life, access factors, and importance weighting.
|
|
55
|
+
|
|
56
|
+
**File**: `src/services/decayCalculator.ts`
|
|
57
|
+
|
|
58
|
+
**Acceptance Criteria**:
|
|
59
|
+
- [x] Decay scores range 0.0 to 1.0
|
|
60
|
+
- [x] New chunks with high importance start high
|
|
61
|
+
- [x] Frequently accessed chunks maintain high scores
|
|
62
|
+
- [x] Old unused chunks decay toward 0
|
|
63
|
+
- [x] Batch update completes in <5s for 10k chunks
|
|
64
|
+
- [x] Configurable weights and half-life
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
### CRR-104: Integrate Metadata Store with Vector Store ✅
|
|
69
|
+
**Estimate**: 3 points
|
|
70
|
+
**Labels**: foundation, integration
|
|
71
|
+
**Blocked by**: CRR-102, CRR-103
|
|
72
|
+
**Status**: COMPLETED
|
|
73
|
+
|
|
74
|
+
Create EnhancedVectorStore wrapper that combines existing vector store with metadata tracking.
|
|
75
|
+
|
|
76
|
+
**File**: `src/services/enhancedVectorStore.ts`
|
|
77
|
+
|
|
78
|
+
**Acceptance Criteria**:
|
|
79
|
+
- [x] All existing tests continue to pass
|
|
80
|
+
- [x] Metadata stored for new chunks
|
|
81
|
+
- [x] Search results include decay scores
|
|
82
|
+
- [x] Access recorded for returned results
|
|
83
|
+
- [x] Re-ranking produces different order than pure similarity
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## Epic: Phase 2 - Cursor Chat History Integration
|
|
88
|
+
|
|
89
|
+
### CRR-201: Implement Cursor Database Reader ✅
|
|
90
|
+
**Estimate**: 3 points
|
|
91
|
+
**Labels**: chat-history, database
|
|
92
|
+
**Status**: COMPLETED
|
|
93
|
+
|
|
94
|
+
Create service to read Cursor's chat history from its SQLite database.
|
|
95
|
+
|
|
96
|
+
**File**: `src/services/cursorChatReader.ts`
|
|
97
|
+
|
|
98
|
+
**Acceptance Criteria**:
|
|
99
|
+
- [x] Correctly locates Cursor DB on macOS, Windows, Linux
|
|
100
|
+
- [x] Reads conversations without corrupting database
|
|
101
|
+
- [x] Handles database being locked (read-only mode)
|
|
102
|
+
- [x] Returns empty array if no conversations
|
|
103
|
+
- [x] Supports filtering by date, project, code presence
|
|
104
|
+
|
|
105
|
+
---
|
|
106
|
+
|
|
107
|
+
### CRR-202: Create Conversation Processor ✅
|
|
108
|
+
**Estimate**: 2 points
|
|
109
|
+
**Labels**: chat-history, processing
|
|
110
|
+
**Blocked by**: CRR-201
|
|
111
|
+
**Status**: COMPLETED
|
|
112
|
+
|
|
113
|
+
Process raw conversations into structured chunks with embeddings.
|
|
114
|
+
|
|
115
|
+
**File**: `src/services/conversationProcessor.ts`
|
|
116
|
+
|
|
117
|
+
**Acceptance Criteria**:
|
|
118
|
+
- [x] Groups messages into logical exchanges
|
|
119
|
+
- [x] Creates embeddings for each chunk
|
|
120
|
+
- [x] Extracts code blocks as separate chunks
|
|
121
|
+
- [x] Calculates reasonable importance scores
|
|
122
|
+
- [x] Extracts basic entities (languages, files)
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
126
|
+
### CRR-203: Implement Chat History Ingestion CLI ✅
|
|
127
|
+
**Estimate**: 2 points
|
|
128
|
+
**Labels**: chat-history, cli
|
|
129
|
+
**Blocked by**: CRR-201, CRR-202
|
|
130
|
+
**Status**: COMPLETED
|
|
131
|
+
|
|
132
|
+
Add CLI commands for chat history ingestion and watching.
|
|
133
|
+
|
|
134
|
+
**File**: `src/cli/commands/chat.ts`
|
|
135
|
+
|
|
136
|
+
**Acceptance Criteria**:
|
|
137
|
+
- [x] `cursor-rag chat ingest` ingests new conversations
|
|
138
|
+
- [x] `cursor-rag chat list` shows available conversations
|
|
139
|
+
- [x] `cursor-rag chat watch` runs in background mode
|
|
140
|
+
- [x] Already-processed conversations are skipped
|
|
141
|
+
- [x] Progress displayed during ingestion
|
|
142
|
+
- [x] `cursor-rag chat stats` shows ingestion statistics
|
|
143
|
+
- [x] `cursor-rag chat reset` resets processing status
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## Epic: Phase 3 - Knowledge Extraction Pipeline
|
|
148
|
+
|
|
149
|
+
### CRR-301: Define Knowledge Extraction Schema ✅
|
|
150
|
+
**Estimate**: 1 point
|
|
151
|
+
**Labels**: extraction, types
|
|
152
|
+
**Status**: COMPLETED
|
|
153
|
+
|
|
154
|
+
Define TypeScript interfaces for extracted knowledge (solutions, patterns, decisions, etc.).
|
|
155
|
+
|
|
156
|
+
**File**: `src/types/extractedKnowledge.ts`
|
|
157
|
+
|
|
158
|
+
**Acceptance Criteria**:
|
|
159
|
+
- [x] All types properly defined and exported
|
|
160
|
+
- [x] Types support JSON serialisation
|
|
161
|
+
- [x] Confidence scores bounded 0-1
|
|
162
|
+
- [x] CodeChange interface for before/after
|
|
163
|
+
|
|
164
|
+
---
|
|
165
|
+
|
|
166
|
+
### CRR-302: Implement LLM Knowledge Extractor ✅
|
|
167
|
+
**Estimate**: 5 points
|
|
168
|
+
**Labels**: extraction, llm
|
|
169
|
+
**Blocked by**: CRR-301
|
|
170
|
+
**Status**: COMPLETED
|
|
171
|
+
|
|
172
|
+
Create service that uses LLM to extract structured knowledge from conversations.
|
|
173
|
+
|
|
174
|
+
**File**: `src/services/knowledgeExtractor.ts`
|
|
175
|
+
|
|
176
|
+
**Acceptance Criteria**:
|
|
177
|
+
- [x] Extracts solutions with problem/solution pairs
|
|
178
|
+
- [x] Extracts patterns with implementation examples
|
|
179
|
+
- [x] Extracts decisions with reasoning
|
|
180
|
+
- [x] Handles LLM response parsing errors gracefully
|
|
181
|
+
- [x] Heuristic fallback when LLM unavailable
|
|
182
|
+
- [x] Low-confidence items filtered out
|
|
183
|
+
- [x] Configurable extraction settings
|
|
184
|
+
|
|
185
|
+
---
|
|
186
|
+
|
|
187
|
+
### CRR-303: Create Knowledge Storage Service ✅
|
|
188
|
+
**Estimate**: 3 points
|
|
189
|
+
**Labels**: extraction, storage
|
|
190
|
+
**Blocked by**: CRR-301, CRR-302, CRR-104
|
|
191
|
+
**Status**: COMPLETED
|
|
192
|
+
|
|
193
|
+
Store extracted knowledge as first-class searchable chunks with relationships.
|
|
194
|
+
|
|
195
|
+
**File**: `src/services/knowledgeStorage.ts`
|
|
196
|
+
|
|
197
|
+
**Acceptance Criteria**:
|
|
198
|
+
- [x] Solutions stored with full problem/solution context
|
|
199
|
+
- [x] Patterns include implementation examples
|
|
200
|
+
- [x] Decisions include reasoning and alternatives
|
|
201
|
+
- [x] Relationships created between related items
|
|
202
|
+
- [x] Appropriate importance scores assigned
|
|
203
|
+
- [x] Integrated with `cursor-rag chat ingest --extract`
|
|
204
|
+
|
|
205
|
+
---
|
|
206
|
+
|
|
207
|
+
## Epic: Phase 4 - Relationship Graph
|
|
208
|
+
|
|
209
|
+
### CRR-401: Define Relationship Types ✅
|
|
210
|
+
**Estimate**: 1 point
|
|
211
|
+
**Labels**: graph, types
|
|
212
|
+
**Status**: COMPLETED
|
|
213
|
+
|
|
214
|
+
Define all relationship types and graph interfaces.
|
|
215
|
+
|
|
216
|
+
**File**: `src/types/relationships.ts`
|
|
217
|
+
|
|
218
|
+
**Acceptance Criteria**:
|
|
219
|
+
- [x] All 19 relationship types defined (extended from spec's 13)
|
|
220
|
+
- [x] RelationshipType enum exported (unified with memory.ts)
|
|
221
|
+
- [x] Relationship interface with strength and metadata
|
|
222
|
+
- [x] GraphTraversalOptions and GraphNode interfaces
|
|
223
|
+
- [x] Helper functions: isBidirectional, getReverseType, getRelationshipsByCategory
|
|
224
|
+
- [x] Relationship categories: semantic, causal, temporal, conflict, preference, structural
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
### CRR-402: Implement Graph Service ✅
|
|
229
|
+
**Estimate**: 4 points
|
|
230
|
+
**Labels**: graph, service
|
|
231
|
+
**Blocked by**: CRR-401, CRR-102
|
|
232
|
+
**Status**: COMPLETED
|
|
233
|
+
|
|
234
|
+
Create graph operations for relationship-based retrieval and contradiction detection.
|
|
235
|
+
|
|
236
|
+
**File**: `src/services/relationshipGraph.ts`
|
|
237
|
+
|
|
238
|
+
**Acceptance Criteria**:
|
|
239
|
+
- [x] All relationship types supported
|
|
240
|
+
- [x] Graph traversal respects depth limits (BFS with configurable maxDepth)
|
|
241
|
+
- [x] Bidirectional relationships create two edges
|
|
242
|
+
- [x] Contradiction detection identifies potential conflicts
|
|
243
|
+
- [x] Traversal filters by type and strength
|
|
244
|
+
- [x] Additional features: cluster detection, supersession chains, graph stats
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
## Epic: Phase 5 - Hierarchical Memory (Categories/Summaries)
|
|
249
|
+
|
|
250
|
+
### CRR-501: Define Category Structure ✅
|
|
251
|
+
**Estimate**: 1 point
|
|
252
|
+
**Labels**: categories, types
|
|
253
|
+
**Status**: COMPLETED
|
|
254
|
+
|
|
255
|
+
Define category types and default categories.
|
|
256
|
+
|
|
257
|
+
**File**: `src/types/categories.ts`
|
|
258
|
+
|
|
259
|
+
**Acceptance Criteria**:
|
|
260
|
+
- [x] Category interface with summary and stats (ExtendedCategory, CategoryWithStats)
|
|
261
|
+
- [x] CategoryItem interface for assignments (re-exported from memory.ts)
|
|
262
|
+
- [x] DEFAULT_CATEGORIES array with 10 categories
|
|
263
|
+
- [x] Types exported
|
|
264
|
+
- [x] Helper functions: findCategoriesByTags, scoreCategoryMatch, getDefaultCategoryNames
|
|
265
|
+
|
|
266
|
+
---
|
|
267
|
+
|
|
268
|
+
### CRR-502: Implement Category Manager ✅
|
|
269
|
+
**Estimate**: 5 points
|
|
270
|
+
**Labels**: categories, service
|
|
271
|
+
**Blocked by**: CRR-501, CRR-302
|
|
272
|
+
**Status**: COMPLETED
|
|
273
|
+
|
|
274
|
+
Create service for category management and summary evolution.
|
|
275
|
+
|
|
276
|
+
**File**: `src/services/categoryManager.ts`
|
|
277
|
+
|
|
278
|
+
**Acceptance Criteria**:
|
|
279
|
+
- [x] Default categories created on first run (initialize method)
|
|
280
|
+
- [x] Chunks classified with relevance scores (heuristic + LLM modes)
|
|
281
|
+
- [x] Summaries evolve as new items added (evolveSummary method)
|
|
282
|
+
- [x] Contradictions handled (LLM detects updates/changes)
|
|
283
|
+
- [x] Category selection uses LLM for relevance (selectRelevantCategories)
|
|
284
|
+
- [x] Additional: keyword matching, tag extraction, custom category creation
|
|
285
|
+
|
|
286
|
+
---
|
|
287
|
+
|
|
288
|
+
## Epic: Phase 6 - Background Maintenance Jobs
|
|
289
|
+
|
|
290
|
+
### CRR-601: Implement Maintenance Scheduler ✅
|
|
291
|
+
**Estimate**: 4 points
|
|
292
|
+
**Labels**: maintenance, background
|
|
293
|
+
**Blocked by**: CRR-103, CRR-502
|
|
294
|
+
|
|
295
|
+
Create scheduled jobs for consolidation, summarisation, and cleanup.
|
|
296
|
+
|
|
297
|
+
**File**: `src/services/maintenanceScheduler.ts`
|
|
298
|
+
|
|
299
|
+
**Acceptance Criteria**:
|
|
300
|
+
- [x] Jobs run on schedule (setInterval-based scheduling with proper cron-like timing)
|
|
301
|
+
- [x] Jobs can be triggered manually (runJob method with 5 job types)
|
|
302
|
+
- [x] Nightly consolidation completes in <5 minutes (decay, duplicates, hot items, archival)
|
|
303
|
+
- [x] Weekly summarisation updates all categories (evolveSummary for each category)
|
|
304
|
+
- [x] Monthly re-index handles large databases (graph analysis, old item archival, vacuum)
|
|
305
|
+
- [x] Proper error handling and logging (MaintenanceJobResult with metrics and errors)
|
|
306
|
+
|
|
307
|
+
---
|
|
308
|
+
|
|
309
|
+
### CRR-602: Add Maintenance CLI Commands ✅
|
|
310
|
+
**Estimate**: 2 points
|
|
311
|
+
**Labels**: maintenance, cli
|
|
312
|
+
**Blocked by**: CRR-601
|
|
313
|
+
|
|
314
|
+
Add CLI commands for maintenance operations.
|
|
315
|
+
|
|
316
|
+
**File**: `src/cli/commands/maintenance.ts`
|
|
317
|
+
|
|
318
|
+
**Acceptance Criteria**:
|
|
319
|
+
- [x] `cursor-rag maintenance run <job>` works (decay, consolidate, summarize, reindex, cleanup)
|
|
320
|
+
- [x] `cursor-rag maintenance start` runs background (with proper scheduling and graceful shutdown)
|
|
321
|
+
- [x] `cursor-rag maintenance stats` shows metrics (memory, graph, scheduler, categories)
|
|
322
|
+
- [x] `cursor-rag maintenance cleanup` safely removes data (with --confirm flag)
|
|
323
|
+
- [x] Dry run mode prevents accidental data loss (--dry-run flag for cleanup)
|
|
324
|
+
|
|
325
|
+
---
|
|
326
|
+
|
|
327
|
+
## Epic: Phase 7 - Enhanced Retrieval Scoring
|
|
328
|
+
|
|
329
|
+
### CRR-701: Implement Hybrid Scorer ✅
|
|
330
|
+
**Estimate**: 4 points
|
|
331
|
+
**Labels**: retrieval, scoring
|
|
332
|
+
**Blocked by**: CRR-402, CRR-502
|
|
333
|
+
|
|
334
|
+
Create hybrid scoring combining similarity, decay, importance, and graph relationships.
|
|
335
|
+
|
|
336
|
+
**File**: `src/services/hybridScorer.ts`
|
|
337
|
+
|
|
338
|
+
**Acceptance Criteria**:
|
|
339
|
+
- [x] Final scores combine all components correctly (weighted combination of 6 factors)
|
|
340
|
+
- [x] Graph boost increases scores for related items (via getGraphContext traversal)
|
|
341
|
+
- [x] Type boost favours solutions and patterns (configurable typeBoosts map)
|
|
342
|
+
- [x] Tiered retrieval tries summaries first (tieredRetrieval method)
|
|
343
|
+
- [x] Recency boost favours recently accessed items (calculateRecencyScore with half-life)
|
|
344
|
+
- [x] Configurable weights (ScoringConfig with DEFAULT_SCORING_CONFIG)
|
|
345
|
+
|
|
346
|
+
---
|
|
347
|
+
|
|
348
|
+
### CRR-702: Add New MCP Tools ✅
|
|
349
|
+
**Estimate**: 3 points
|
|
350
|
+
**Labels**: mcp, tools
|
|
351
|
+
**Blocked by**: CRR-701
|
|
352
|
+
|
|
353
|
+
Add new MCP tools for memory features.
|
|
354
|
+
|
|
355
|
+
**File**: `src/server/tools/memory.ts`
|
|
356
|
+
|
|
357
|
+
**Acceptance Criteria**:
|
|
358
|
+
- [x] search_past_solutions tool working (searches solution chunks with hybrid scoring)
|
|
359
|
+
- [x] find_similar_issues tool working (includes graph traversal for related items)
|
|
360
|
+
- [x] get_project_patterns tool working (filters by category, sorted by importance)
|
|
361
|
+
- [x] recall_decision tool working (searches decision/standard chunks)
|
|
362
|
+
- [x] get_category_summary tool working (returns category summary and metadata)
|
|
363
|
+
- [x] ingest_chat_history tool working (with optional knowledge extraction)
|
|
364
|
+
- [x] memory_stats tool working (comprehensive stats output)
|
|
365
|
+
|
|
366
|
+
---
|
|
367
|
+
|
|
368
|
+
### CRR-703: Create Memory Configuration ✅
|
|
369
|
+
**Estimate**: 1 point
|
|
370
|
+
**Labels**: config
|
|
371
|
+
|
|
372
|
+
Define configuration schema and defaults for all memory features.
|
|
373
|
+
|
|
374
|
+
**File**: `src/config/memoryConfig.ts`
|
|
375
|
+
|
|
376
|
+
**Acceptance Criteria**:
|
|
377
|
+
- [x] MemoryConfig interface complete (Zod schema with 7 config sections)
|
|
378
|
+
- [x] DEFAULT_MEMORY_CONFIG with sensible defaults (all sections have defaults)
|
|
379
|
+
- [x] Config validation (validateMemoryConfig function with weight sum check)
|
|
380
|
+
- [x] Environment variable overrides (getMemoryConfigWithEnvOverrides function)
|
|
381
|
+
|
|
382
|
+
---
|
|
383
|
+
|
|
384
|
+
## Epic: Phase 8 - RLM-Style Recursive Retrieval
|
|
385
|
+
|
|
386
|
+
*Based on the Recursive Language Models paper (Zhang et al., 2024)*
|
|
387
|
+
|
|
388
|
+
### CRR-801: Implement Context Environment ✅
|
|
389
|
+
**Estimate**: 5 points
|
|
390
|
+
**Labels**: rlm, retrieval, core
|
|
391
|
+
|
|
392
|
+
Create sandboxed environment for RLM-style context processing with cost tracking and budget enforcement.
|
|
393
|
+
|
|
394
|
+
**File**: `src/services/contextEnvironment.ts`
|
|
395
|
+
|
|
396
|
+
**Acceptance Criteria**:
|
|
397
|
+
- [x] Context can be loaded as environment variables (loadContext method)
|
|
398
|
+
- [x] State description gives LLM overview without full content (getStateDescription method)
|
|
399
|
+
- [x] Peek allows selective viewing of chunks (peek method with start/end params)
|
|
400
|
+
- [x] Filter supports regex patterns (filter method with RegExp)
|
|
401
|
+
- [x] Sub-queries track cost and enforce budget (subQuery with CostTracker)
|
|
402
|
+
- [x] Async batch queries work with concurrency limit (batchSubQuery with configurable concurrency)
|
|
403
|
+
- [x] Termination conditions enforced (shouldTerminate checks iterations, cost, sub-calls, timeout)
|
|
404
|
+
|
|
405
|
+
---
|
|
406
|
+
|
|
407
|
+
### CRR-802: Implement Recursive Retrieval Controller ✅
|
|
408
|
+
**Estimate**: 5 points
|
|
409
|
+
**Labels**: rlm, retrieval, core
|
|
410
|
+
**Blocked by**: CRR-801, CRR-701
|
|
411
|
+
|
|
412
|
+
Orchestrate iterative retrieval with complexity assessment and action parsing.
|
|
413
|
+
|
|
414
|
+
**File**: `src/services/recursiveRetrieval.ts`
|
|
415
|
+
|
|
416
|
+
**Acceptance Criteria**:
|
|
417
|
+
- [x] Simple queries use direct retrieval (returns direct when complexity === 'simple')
|
|
418
|
+
- [x] Complex queries trigger recursive processing (iterativeProcess method)
|
|
419
|
+
- [x] Complexity assessment considers context size and query type (assessComplexity method)
|
|
420
|
+
- [x] Actions are parsed and executed correctly (parseAction and executeAction methods)
|
|
421
|
+
- [x] Cost tracked across iterations (via ContextEnvironment.getTotalCost)
|
|
422
|
+
- [x] Early termination on budget/iteration limits (handles Budget/Limit/Timeout errors)
|
|
423
|
+
|
|
424
|
+
---
|
|
425
|
+
|
|
426
|
+
### CRR-803: Implement Smart Chunking Strategies ✅
|
|
427
|
+
**Estimate**: 3 points
|
|
428
|
+
**Labels**: rlm, chunking
|
|
429
|
+
**Blocked by**: CRR-801
|
|
430
|
+
|
|
431
|
+
Implement multiple chunking strategies based on RLM paper patterns.
|
|
432
|
+
|
|
433
|
+
**File**: `src/services/smartChunker.ts`
|
|
434
|
+
|
|
435
|
+
**Acceptance Criteria**:
|
|
436
|
+
- [x] Uniform chunking by count/size (uniformChunk, uniformChunkEnhanced methods)
|
|
437
|
+
- [x] Semantic chunking groups similar content (semanticChunk with k-means clustering)
|
|
438
|
+
- [x] Keyword-based chunking filters by patterns (keywordChunk with regex)
|
|
439
|
+
- [x] Structural chunking groups by source file (structuralChunk method)
|
|
440
|
+
- [x] Adaptive chunking chooses strategy based on content/query (adaptiveChunk method)
|
|
441
|
+
|
|
442
|
+
---
|
|
443
|
+
|
|
444
|
+
### CRR-804: Implement Anti-Pattern Mitigations ✅
|
|
445
|
+
**Estimate**: 3 points
|
|
446
|
+
**Labels**: rlm, safety
|
|
447
|
+
**Blocked by**: CRR-801, CRR-802
|
|
448
|
+
|
|
449
|
+
Implement safeguards from RLM paper's Negative Results section.
|
|
450
|
+
|
|
451
|
+
**File**: `src/services/rlmSafeguards.ts`
|
|
452
|
+
|
|
453
|
+
**Acceptance Criteria**:
|
|
454
|
+
- [x] Model-specific configurations (MODEL_CONFIGS map with per-model settings)
|
|
455
|
+
- [x] Capability detection for code execution ability (ModelCapabilities, getModelCapabilities)
|
|
456
|
+
- [x] Token budget management (TokenBudgetManager with reserve ratio)
|
|
457
|
+
- [x] Multi-signal termination detection (TerminationDetector with 4 signals)
|
|
458
|
+
- [x] Sub-call throttling and caching (SubCallThrottler with MD5 cache keys)
|
|
459
|
+
- [x] Circuit breaker for runaway trajectories (CircuitBreaker class)
|
|
460
|
+
- [x] Model prior-based pre-filtering (PriorBasedFilter with keyword extraction)
|
|
461
|
+
|
|
462
|
+
---
|
|
463
|
+
|
|
464
|
+
## Epic: Phase 9 - Dashboard Tools UI
|
|
465
|
+
|
|
466
|
+
*Interactive tool execution from the web dashboard*
|
|
467
|
+
|
|
468
|
+
### CRR-901: Define Tool Registry Interface
|
|
469
|
+
**Estimate**: 2 points
|
|
470
|
+
**Labels**: dashboard, tools, types
|
|
471
|
+
**Status**: ✅ COMPLETED
|
|
472
|
+
|
|
473
|
+
Create a registry system for exposing RAG tools to the dashboard UI.
|
|
474
|
+
|
|
475
|
+
**File**: `src/dashboard/toolRegistry.ts`
|
|
476
|
+
|
|
477
|
+
**Acceptance Criteria**:
|
|
478
|
+
- [x] ToolDefinition interface with name, description, parameters schema
|
|
479
|
+
- [x] ToolParameter interface with type, required, default, validation
|
|
480
|
+
- [x] ToolResult interface with success/error states
|
|
481
|
+
- [x] Registry supports dynamic tool registration
|
|
482
|
+
- [x] JSON Schema generation for parameter forms
|
|
483
|
+
|
|
484
|
+
---
|
|
485
|
+
|
|
486
|
+
### CRR-902: Implement Dashboard Tools API
|
|
487
|
+
**Estimate**: 3 points
|
|
488
|
+
**Labels**: dashboard, api
|
|
489
|
+
**Blocked by**: CRR-901
|
|
490
|
+
**Status**: ✅ COMPLETED
|
|
491
|
+
|
|
492
|
+
Add API endpoints for tool discovery and execution.
|
|
493
|
+
|
|
494
|
+
**File**: `src/dashboard/server.ts`
|
|
495
|
+
|
|
496
|
+
**Acceptance Criteria**:
|
|
497
|
+
- [x] `GET /api/tools` returns list of available tools with schemas
|
|
498
|
+
- [x] `POST /api/tools/:name/execute` runs a tool with parameters
|
|
499
|
+
- [x] `GET /api/tools/:name/status/:jobId` for long-running tools
|
|
500
|
+
- [x] Proper error handling and validation
|
|
501
|
+
- [ ] Rate limiting to prevent abuse (deferred - not critical for MVP)
|
|
502
|
+
|
|
503
|
+
---
|
|
504
|
+
|
|
505
|
+
### CRR-903: Create Tools UI Panel
|
|
506
|
+
**Estimate**: 4 points
|
|
507
|
+
**Labels**: dashboard, ui
|
|
508
|
+
**Blocked by**: CRR-902
|
|
509
|
+
**Status**: ✅ COMPLETED
|
|
510
|
+
|
|
511
|
+
Build interactive tools section in the dashboard.
|
|
512
|
+
|
|
513
|
+
**File**: `src/dashboard/public/index.html`
|
|
514
|
+
|
|
515
|
+
**Acceptance Criteria**:
|
|
516
|
+
- [x] New "Tools" tab in dashboard navigation
|
|
517
|
+
- [x] Tool cards with name, description, and "Run" button
|
|
518
|
+
- [x] Dynamic form generation from parameter schemas
|
|
519
|
+
- [x] Real-time execution status and progress
|
|
520
|
+
- [x] Result display with syntax highlighting for code/JSON
|
|
521
|
+
- [x] Execution history with re-run capability
|
|
522
|
+
- [x] Tool categories/filtering (search, ingest, maintenance, etc.)
|
|
523
|
+
|
|
524
|
+
---
|
|
525
|
+
|
|
526
|
+
### CRR-904: Register Core Tools
|
|
527
|
+
**Estimate**: 2 points
|
|
528
|
+
**Labels**: dashboard, tools
|
|
529
|
+
**Blocked by**: CRR-901, CRR-902
|
|
530
|
+
**Status**: ✅ COMPLETED
|
|
531
|
+
|
|
532
|
+
Register existing RAG tools with the dashboard registry.
|
|
533
|
+
|
|
534
|
+
**File**: `src/dashboard/coreTools.ts`
|
|
535
|
+
|
|
536
|
+
**Acceptance Criteria**:
|
|
537
|
+
- [x] `search` - Search knowledge base with query
|
|
538
|
+
- [x] `ingest_document` - Ingest text/URL into RAG
|
|
539
|
+
- [x] `crawl_and_ingest` - Crawl website and ingest
|
|
540
|
+
- [x] `chat_ingest` - Ingest Cursor chat history
|
|
541
|
+
- [x] `chat_list` - List available conversations
|
|
542
|
+
- [x] `memory_stats` - Show memory statistics
|
|
543
|
+
- [x] `list_sources` - List ingested sources
|
|
544
|
+
- [x] All tools have proper parameter validation
|
|
545
|
+
|
|
546
|
+
---
|
|
547
|
+
|
|
548
|
+
## Epic: Phase 10 - Cursor Rules Optimizer
|
|
549
|
+
|
|
550
|
+
*Intelligent cleanup and optimization of Cursor rules and AGENTS.md files*
|
|
551
|
+
|
|
552
|
+
### CRR-1000: Implement LLM Provider System (Strategy Pattern) ✅
|
|
553
|
+
**Estimate**: 5 points
|
|
554
|
+
**Labels**: llm, infrastructure, core
|
|
555
|
+
**Priority**: HIGH - Required by CRR-1004 and other LLM-dependent features
|
|
556
|
+
**Status**: COMPLETED
|
|
557
|
+
|
|
558
|
+
Create a flexible LLM provider system using the strategy pattern that supports multiple backends.
|
|
559
|
+
|
|
560
|
+
**Files**: `src/types/llmProvider.ts`, `src/adapters/llm/index.ts`, `src/adapters/llm/*.ts`
|
|
561
|
+
|
|
562
|
+
**Acceptance Criteria**:
|
|
563
|
+
- [x] LLMProvider interface with chat/complete methods
|
|
564
|
+
- [x] LLMProviderConfig type with provider-specific options
|
|
565
|
+
- [x] LLMResponse type with content, usage stats, model info
|
|
566
|
+
|
|
567
|
+
**Provider Implementations**:
|
|
568
|
+
- [x] **CursorProvider**: Placeholder (falls back to Ollama)
|
|
569
|
+
- [x] **OpenAIProvider**: OpenAI API (GPT-4o, GPT-4o-mini, o1, etc.)
|
|
570
|
+
- [x] **AnthropicProvider**: Claude API (claude-3.5-sonnet, opus, haiku)
|
|
571
|
+
- [x] **DeepSeekProvider**: DeepSeek API (deepseek-chat, deepseek-coder)
|
|
572
|
+
- [x] **GroqProvider**: Groq API (llama, mixtral models)
|
|
573
|
+
- [x] **OllamaProvider**: Local Ollama models
|
|
574
|
+
- [x] **OpenRouterProvider**: OpenRouter for unified API access
|
|
575
|
+
|
|
576
|
+
**Configuration**:
|
|
577
|
+
- [x] Environment variable support (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)
|
|
578
|
+
- [x] Config file support (~/.cursor-rag/llm-config.json)
|
|
579
|
+
- [x] Auto-detection: env vars → config file → fallback to Ollama
|
|
580
|
+
|
|
581
|
+
**Features**:
|
|
582
|
+
- [x] Retry logic with exponential backoff
|
|
583
|
+
- [x] Rate limiting per provider
|
|
584
|
+
- [x] Cost tracking per request
|
|
585
|
+
- [x] Streaming support where available
|
|
586
|
+
- [x] Model capability detection (context length, vision, JSON mode)
|
|
587
|
+
- [x] Fallback chain (if primary fails, try next)
|
|
588
|
+
|
|
589
|
+
---
|
|
590
|
+
|
|
591
|
+
### CRR-1001: Define Rules Analysis Types ✅
|
|
592
|
+
**Estimate**: 2 points
|
|
593
|
+
**Labels**: rules, types
|
|
594
|
+
**Status**: COMPLETED
|
|
595
|
+
|
|
596
|
+
Define types for rule parsing, analysis, and optimization.
|
|
597
|
+
|
|
598
|
+
**File**: `src/types/rulesOptimizer.ts`
|
|
599
|
+
|
|
600
|
+
**Acceptance Criteria**:
|
|
601
|
+
- [x] RuleFile interface (path, content, format: mdc/md/txt)
|
|
602
|
+
- [x] ParsedRule interface (id, title, content, tags, dependencies)
|
|
603
|
+
- [x] RuleCluster interface for grouping related rules
|
|
604
|
+
- [x] OptimizationReport interface with before/after metrics
|
|
605
|
+
- [x] DuplicateMatch interface with similarity score
|
|
606
|
+
- [x] MergeCandidate interface for rule combinations
|
|
607
|
+
|
|
608
|
+
---
|
|
609
|
+
|
|
610
|
+
### CRR-1002: Implement Rules Parser ✅
|
|
611
|
+
**Estimate**: 3 points
|
|
612
|
+
**Labels**: rules, parser
|
|
613
|
+
**Blocked by**: CRR-1001
|
|
614
|
+
**Status**: COMPLETED
|
|
615
|
+
|
|
616
|
+
Parse various rule file formats into structured representation.
|
|
617
|
+
|
|
618
|
+
**File**: `src/services/rulesParser.ts`
|
|
619
|
+
|
|
620
|
+
**Acceptance Criteria**:
|
|
621
|
+
- [x] Parse `.mdc` files (Cursor rules format)
|
|
622
|
+
- [x] Parse `AGENTS.md` files
|
|
623
|
+
- [x] Parse `.cursorrules` legacy format
|
|
624
|
+
- [x] Extract rule metadata (globs, descriptions, always-apply flags)
|
|
625
|
+
- [x] Handle nested rule structures
|
|
626
|
+
- [x] Preserve original formatting for non-modified rules
|
|
627
|
+
|
|
628
|
+
---
|
|
629
|
+
|
|
630
|
+
### CRR-1003: Implement Rules Analyzer ✅
|
|
631
|
+
**Estimate**: 4 points
|
|
632
|
+
**Labels**: rules, analysis
|
|
633
|
+
**Blocked by**: CRR-1002
|
|
634
|
+
**Status**: COMPLETED
|
|
635
|
+
|
|
636
|
+
Detect duplicate and near-duplicate rules using semantic similarity.
|
|
637
|
+
|
|
638
|
+
**File**: `src/services/rulesAnalyzer.ts`
|
|
639
|
+
|
|
640
|
+
**Acceptance Criteria**:
|
|
641
|
+
- [x] Exact duplicate detection (content hash)
|
|
642
|
+
- [x] Semantic similarity using embeddings (configurable threshold)
|
|
643
|
+
- [x] Detect rules that are subsets of others
|
|
644
|
+
- [x] Identify contradicting rules
|
|
645
|
+
- [x] Group related rules by topic/technology
|
|
646
|
+
- [x] Find outdated rules
|
|
647
|
+
- [x] Generate optimization report
|
|
648
|
+
|
|
649
|
+
---
|
|
650
|
+
|
|
651
|
+
### CRR-1004: Implement Rules Optimizer CLI ✅
|
|
652
|
+
**Estimate**: 3 points
|
|
653
|
+
**Labels**: rules, cli
|
|
654
|
+
**Blocked by**: CRR-1003
|
|
655
|
+
**Status**: COMPLETED
|
|
656
|
+
|
|
657
|
+
CLI commands for rules optimization.
|
|
658
|
+
|
|
659
|
+
**File**: `src/cli/commands/rules.ts`
|
|
660
|
+
|
|
661
|
+
**Acceptance Criteria**:
|
|
662
|
+
- [x] `cursor-rag rules analyze <folder>` - Analyze rules without changes
|
|
663
|
+
- [x] `cursor-rag rules list <folder>` - List all rules
|
|
664
|
+
- [x] `cursor-rag rules duplicates <folder>` - Show duplicates only
|
|
665
|
+
- [x] `cursor-rag rules conflicts <folder>` - Show conflicts only
|
|
666
|
+
- [x] `cursor-rag rules outdated <folder>` - Show outdated rules
|
|
667
|
+
- [x] `cursor-rag rules optimize <folder>` - Run full optimization
|
|
668
|
+
- [x] `--dry-run` flag for preview mode
|
|
669
|
+
- [x] `--json` flag for JSON output
|
|
670
|
+
- [x] Progress display and summary statistics
|
|
671
|
+
|
|
672
|
+
---
|
|
673
|
+
|
|
674
|
+
### CRR-1005: Implement Rules Merger ✅
|
|
675
|
+
**Estimate**: 5 points
|
|
676
|
+
**Labels**: rules, llm
|
|
677
|
+
**Blocked by**: CRR-1000, CRR-1003
|
|
678
|
+
**Status**: COMPLETED
|
|
679
|
+
|
|
680
|
+
Use LLM to intelligently merge and consolidate related rules.
|
|
681
|
+
|
|
682
|
+
**File**: `src/services/rulesMerger.ts`
|
|
683
|
+
|
|
684
|
+
**Acceptance Criteria**:
|
|
685
|
+
- [x] Uses LLMProvider system (CRR-1000) for AI operations
|
|
686
|
+
- [x] Merge duplicate rules preserving all unique information
|
|
687
|
+
- [x] Combine related rules into comprehensive single rules
|
|
688
|
+
- [x] Rewrite verbose rules to be more concise
|
|
689
|
+
- [x] Preserve critical details while reducing token count
|
|
690
|
+
- [x] Maintain rule intent and effectiveness
|
|
691
|
+
- [x] Support dry-run mode with preview
|
|
692
|
+
- [x] Configurable aggressiveness (conservative/balanced/aggressive)
|
|
693
|
+
|
|
694
|
+
**CLI Commands Added**:
|
|
695
|
+
- `cursor-rag rules merge <folder>` - Merge duplicates using LLM
|
|
696
|
+
- `cursor-rag rules rewrite <folder>` - Rewrite verbose rules to be more concise
|
|
697
|
+
|
|
698
|
+
---
|
|
699
|
+
|
|
700
|
+
### CRR-1006: Add Rules Optimizer to Dashboard ✅
|
|
701
|
+
**Estimate**: 3 points
|
|
702
|
+
**Labels**: rules, dashboard
|
|
703
|
+
**Blocked by**: CRR-1003, CRR-903
|
|
704
|
+
**Status**: COMPLETED
|
|
705
|
+
|
|
706
|
+
Add rules optimization UI to dashboard tools.
|
|
707
|
+
|
|
708
|
+
**Acceptance Criteria**:
|
|
709
|
+
- [x] "Rules Optimizer" tool card in Tools section (purple gradient panel)
|
|
710
|
+
- [x] Folder path input with validation (+ server-side folder browser)
|
|
711
|
+
- [x] Analysis results display with duplicate highlighting
|
|
712
|
+
- [x] One-click optimize with confirmation (dry run + apply modes)
|
|
713
|
+
- [x] LLM provider configuration (via Settings tab, not dropdown)
|
|
714
|
+
- [x] Natural language rules support (LLM-interpreted custom rules)
|
|
715
|
+
- [x] In-app modal/toast system (replaced browser alerts)
|
|
716
|
+
- [ ] Before/after comparison view (future enhancement)
|
|
717
|
+
- [ ] Token count savings visualization (future enhancement)
|
|
718
|
+
- [ ] Download optimized rules as zip (future enhancement)
|
|
719
|
+
|
|
720
|
+
**Dashboard Features Added**:
|
|
721
|
+
- Rules Optimizer panel in Tools tab with folder browser
|
|
722
|
+
- Server-side folder navigation API (`/api/system/browse`)
|
|
723
|
+
- Auto-optimize API (`/api/rules/auto-optimize`)
|
|
724
|
+
- Works with or without LLM (pattern matching always available)
|
|
725
|
+
- Automatic backup creation before applying changes
|
|
726
|
+
|
|
727
|
+
---
|
|
728
|
+
|
|
729
|
+
### CRR-1007: Implement Duplicate Detector (Merged into CRR-1003)
|
|
730
|
+
**Estimate**: 4 points
|
|
731
|
+
**Labels**: rules, analysis
|
|
732
|
+
**Blocked by**: CRR-1002
|
|
733
|
+
**Status**: MERGED into CRR-1003
|
|
734
|
+
|
|
735
|
+
Detect duplicate and near-duplicate rules using semantic similarity.
|
|
736
|
+
|
|
737
|
+
**File**: `src/services/rulesDuplicateDetector.ts`
|
|
738
|
+
|
|
739
|
+
**Acceptance Criteria** (merged into rulesAnalyzer.ts):
|
|
740
|
+
- [x] Exact duplicate detection (content hash)
|
|
741
|
+
- [x] Semantic similarity using embeddings (configurable threshold)
|
|
742
|
+
- [x] Detect rules that are subsets of others
|
|
743
|
+
- [x] Identify contradicting rules
|
|
744
|
+
- [x] Group related rules by topic/technology
|
|
745
|
+
- [x] Generate similarity matrix for rule set
|
|
746
|
+
|
|
747
|
+
---
|
|
748
|
+
|
|
749
|
+
### CRR-1008: Add Configurable Rules Analyzer Settings ✅
|
|
750
|
+
**Estimate**: 2 points
|
|
751
|
+
**Labels**: rules, config, dashboard
|
|
752
|
+
**Blocked by**: CRR-1003
|
|
753
|
+
**Status**: COMPLETED
|
|
754
|
+
|
|
755
|
+
Add user-configurable settings for the rules analyzer via config file and dashboard API.
|
|
756
|
+
|
|
757
|
+
**File**: `src/config/rulesConfig.ts`, `src/dashboard/server.ts`
|
|
758
|
+
|
|
759
|
+
**Acceptance Criteria**:
|
|
760
|
+
- [x] RulesAnalyzerConfig schema with Zod validation
|
|
761
|
+
- [x] Analysis settings (duplicateThreshold, maxAgeDays, oldYearThreshold)
|
|
762
|
+
- [x] Custom version check patterns (user-defined technology versions)
|
|
763
|
+
- [x] Custom deprecation patterns (user-defined deprecated code detectors)
|
|
764
|
+
- [x] Custom tag extraction patterns
|
|
765
|
+
- [x] File include/exclude patterns
|
|
766
|
+
- [x] Dashboard API: GET/PUT /api/rules/config
|
|
767
|
+
- [x] Dashboard API: POST /api/rules/validate-pattern
|
|
768
|
+
- [x] Dashboard API: POST /api/rules/test-pattern
|
|
769
|
+
- [x] Dashboard API: CRUD for version-checks and deprecation-patterns
|
|
770
|
+
- [x] Example templates for common technologies (disabled by default)
|
|
771
|
+
|
|
772
|
+
---
|
|
773
|
+
|
|
774
|
+
## Epic: Phase 11 - Comprehensive Test Suite
|
|
775
|
+
|
|
776
|
+
*Unit, integration, and E2E tests for the entire system*
|
|
777
|
+
|
|
778
|
+
### CRR-1101: Test Infrastructure Setup
|
|
779
|
+
**Estimate**: 3 points
|
|
780
|
+
**Labels**: testing, infrastructure
|
|
781
|
+
|
|
782
|
+
Set up testing framework, configuration, and CI integration.
|
|
783
|
+
|
|
784
|
+
**Files**: `vitest.config.ts`, `package.json`, `tests/setup.ts`
|
|
785
|
+
|
|
786
|
+
**Acceptance Criteria**:
|
|
787
|
+
- [ ] Vitest configured with TypeScript support
|
|
788
|
+
- [ ] Test scripts in package.json (`test`, `test:unit`, `test:integration`, `test:e2e`, `test:coverage`)
|
|
789
|
+
- [ ] Coverage thresholds configured (minimum 70%)
|
|
790
|
+
- [ ] Test setup file with common mocks and utilities
|
|
791
|
+
- [ ] SQLite in-memory database for test isolation
|
|
792
|
+
- [ ] Mock embeddings adapter for fast tests
|
|
793
|
+
- [ ] GitHub Actions CI workflow for automated testing
|
|
794
|
+
|
|
795
|
+
---
|
|
796
|
+
|
|
797
|
+
### CRR-1102: Unit Tests - Core Types & Utilities
|
|
798
|
+
**Estimate**: 2 points
|
|
799
|
+
**Labels**: testing, unit
|
|
800
|
+
**Blocked by**: CRR-1101
|
|
801
|
+
|
|
802
|
+
Test type guards, enums, and utility functions.
|
|
803
|
+
|
|
804
|
+
**Files**: `tests/unit/types/*.test.ts`, `tests/unit/utils/*.test.ts`
|
|
805
|
+
|
|
806
|
+
**Acceptance Criteria**:
|
|
807
|
+
- [ ] `memory.ts` types and enums tested
|
|
808
|
+
- [ ] `relationships.ts` helper functions tested (isBidirectional, getReverseType, getRelationshipsByCategory)
|
|
809
|
+
- [ ] `extractedKnowledge.ts` type validation tested
|
|
810
|
+
- [ ] Chunker utility functions tested
|
|
811
|
+
- [ ] Config parsing and validation tested
|
|
812
|
+
|
|
813
|
+
---
|
|
814
|
+
|
|
815
|
+
### CRR-1103: Unit Tests - Services (Phase 1-2)
|
|
816
|
+
**Estimate**: 4 points
|
|
817
|
+
**Labels**: testing, unit
|
|
818
|
+
**Blocked by**: CRR-1101
|
|
819
|
+
|
|
820
|
+
Test foundation and chat history services.
|
|
821
|
+
|
|
822
|
+
**Files**: `tests/unit/services/*.test.ts`
|
|
823
|
+
|
|
824
|
+
**Acceptance Criteria**:
|
|
825
|
+
- [ ] `MemoryMetadataStore` CRUD operations tested
|
|
826
|
+
- [ ] `MemoryMetadataStore` relationship operations tested
|
|
827
|
+
- [ ] `MemoryMetadataStore` category operations tested
|
|
828
|
+
- [ ] `DecayCalculator` scoring logic tested
|
|
829
|
+
- [ ] `DecayCalculator` edge cases (new chunks, old chunks, high access)
|
|
830
|
+
- [ ] `EnhancedVectorStore` wrapper tested with mock vector store
|
|
831
|
+
- [ ] `CursorChatReader` path detection tested (mock filesystem)
|
|
832
|
+
- [ ] `ConversationProcessor` chunking and entity extraction tested
|
|
833
|
+
|
|
834
|
+
---
|
|
835
|
+
|
|
836
|
+
### CRR-1104: Unit Tests - Services (Phase 3-4)
|
|
837
|
+
**Estimate**: 4 points
|
|
838
|
+
**Labels**: testing, unit
|
|
839
|
+
**Blocked by**: CRR-1101
|
|
840
|
+
|
|
841
|
+
Test knowledge extraction and relationship graph services.
|
|
842
|
+
|
|
843
|
+
**Files**: `tests/unit/services/*.test.ts`
|
|
844
|
+
|
|
845
|
+
**Acceptance Criteria**:
|
|
846
|
+
- [ ] `KnowledgeExtractor` heuristic extraction tested
|
|
847
|
+
- [ ] `KnowledgeExtractor` LLM extraction tested (mocked LLM)
|
|
848
|
+
- [ ] `KnowledgeExtractor` confidence filtering tested
|
|
849
|
+
- [ ] `KnowledgeStorage` storage operations tested
|
|
850
|
+
- [ ] `KnowledgeStorage` relationship creation tested
|
|
851
|
+
- [ ] `RelationshipGraph` traversal tested (depth limits, type filtering)
|
|
852
|
+
- [ ] `RelationshipGraph` bidirectional relationships tested
|
|
853
|
+
- [ ] `RelationshipGraph` contradiction detection tested
|
|
854
|
+
- [ ] `RelationshipGraph` cluster finding tested
|
|
855
|
+
|
|
856
|
+
---
|
|
857
|
+
|
|
858
|
+
### CRR-1105: Unit Tests - Adapters
|
|
859
|
+
**Estimate**: 3 points
|
|
860
|
+
**Labels**: testing, unit
|
|
861
|
+
**Blocked by**: CRR-1101
|
|
862
|
+
|
|
863
|
+
Test vector store and embedding adapters.
|
|
864
|
+
|
|
865
|
+
**Files**: `tests/unit/adapters/*.test.ts`
|
|
866
|
+
|
|
867
|
+
**Acceptance Criteria**:
|
|
868
|
+
- [ ] Memory vector store tested (add, search, delete)
|
|
869
|
+
- [ ] Xenova embeddings tested (mocked transformer)
|
|
870
|
+
- [ ] OpenAI embeddings tested (mocked API)
|
|
871
|
+
- [ ] Ollama embeddings tested (mocked API)
|
|
872
|
+
- [ ] Adapter factory functions tested
|
|
873
|
+
- [ ] Error handling for adapter failures tested
|
|
874
|
+
|
|
875
|
+
---
|
|
876
|
+
|
|
877
|
+
### CRR-1106: Integration Tests - Vector Store + Metadata
|
|
878
|
+
**Estimate**: 4 points
|
|
879
|
+
**Labels**: testing, integration
|
|
880
|
+
**Blocked by**: CRR-1103, CRR-1105
|
|
881
|
+
|
|
882
|
+
Test interactions between vector store and metadata store.
|
|
883
|
+
|
|
884
|
+
**Files**: `tests/integration/vector-metadata.test.ts`
|
|
885
|
+
|
|
886
|
+
**Acceptance Criteria**:
|
|
887
|
+
- [ ] EnhancedVectorStore upsert stores in both stores
|
|
888
|
+
- [ ] Search results enriched with metadata correctly
|
|
889
|
+
- [ ] Access recording updates decay scores
|
|
890
|
+
- [ ] Re-ranking with decay scores produces different order
|
|
891
|
+
- [ ] Archived chunks filtered from search results
|
|
892
|
+
- [ ] Relationship data included in search results
|
|
893
|
+
|
|
894
|
+
---
|
|
895
|
+
|
|
896
|
+
### CRR-1107: Integration Tests - Knowledge Pipeline
|
|
897
|
+
**Estimate**: 4 points
|
|
898
|
+
**Labels**: testing, integration
|
|
899
|
+
**Blocked by**: CRR-1104
|
|
900
|
+
|
|
901
|
+
Test the full knowledge extraction and storage pipeline.
|
|
902
|
+
|
|
903
|
+
**Files**: `tests/integration/knowledge-pipeline.test.ts`
|
|
904
|
+
|
|
905
|
+
**Acceptance Criteria**:
|
|
906
|
+
- [ ] Conversation → extraction → storage flow tested
|
|
907
|
+
- [ ] Solutions stored with correct chunk types
|
|
908
|
+
- [ ] Patterns stored with implementation examples
|
|
909
|
+
- [ ] Decisions stored with reasoning
|
|
910
|
+
- [ ] Relationships created between related knowledge
|
|
911
|
+
- [ ] Duplicate conversations not re-processed
|
|
912
|
+
- [ ] Entity extraction from conversations tested
|
|
913
|
+
|
|
914
|
+
---
|
|
915
|
+
|
|
916
|
+
### CRR-1108: Integration Tests - CLI Commands
|
|
917
|
+
**Estimate**: 3 points
|
|
918
|
+
**Labels**: testing, integration
|
|
919
|
+
**Blocked by**: CRR-1103, CRR-1104
|
|
920
|
+
|
|
921
|
+
Test CLI commands end-to-end.
|
|
922
|
+
|
|
923
|
+
**Files**: `tests/integration/cli/*.test.ts`
|
|
924
|
+
|
|
925
|
+
**Acceptance Criteria**:
|
|
926
|
+
- [ ] `cursor-rag ingest` command tested
|
|
927
|
+
- [ ] `cursor-rag search` command tested
|
|
928
|
+
- [ ] `cursor-rag status` command tested
|
|
929
|
+
- [ ] `cursor-rag chat list` command tested
|
|
930
|
+
- [ ] `cursor-rag chat ingest` command tested
|
|
931
|
+
- [ ] `cursor-rag chat stats` command tested
|
|
932
|
+
- [ ] Error handling for invalid inputs tested
|
|
933
|
+
- [ ] Help output validated
|
|
934
|
+
|
|
935
|
+
---
|
|
936
|
+
|
|
937
|
+
### CRR-1109: Integration Tests - MCP Server & Tools
|
|
938
|
+
**Estimate**: 4 points
|
|
939
|
+
**Labels**: testing, integration
|
|
940
|
+
**Blocked by**: CRR-1106
|
|
941
|
+
|
|
942
|
+
Test MCP server protocol and tool execution.
|
|
943
|
+
|
|
944
|
+
**Files**: `tests/integration/mcp/*.test.ts`
|
|
945
|
+
|
|
946
|
+
**Acceptance Criteria**:
|
|
947
|
+
- [ ] MCP server initializes correctly
|
|
948
|
+
- [ ] Tool listing returns all available tools
|
|
949
|
+
- [ ] `search` tool returns relevant results
|
|
950
|
+
- [ ] `ingest` tool processes documents
|
|
951
|
+
- [ ] `crawl` tool handles URLs
|
|
952
|
+
- [ ] `recursive_query` tool performs multi-hop retrieval
|
|
953
|
+
- [ ] `list_sources` tool returns ingested sources
|
|
954
|
+
- [ ] Error responses follow MCP protocol
|
|
955
|
+
|
|
956
|
+
---
|
|
957
|
+
|
|
958
|
+
### CRR-1110: E2E Tests - Dashboard UI
|
|
959
|
+
**Estimate**: 5 points
|
|
960
|
+
**Labels**: testing, e2e
|
|
961
|
+
**Blocked by**: CRR-1101
|
|
962
|
+
|
|
963
|
+
Test dashboard web interface with Playwright.
|
|
964
|
+
|
|
965
|
+
**Files**: `tests/e2e/dashboard/*.test.ts`
|
|
966
|
+
|
|
967
|
+
**Acceptance Criteria**:
|
|
968
|
+
- [ ] Dashboard loads without errors
|
|
969
|
+
- [ ] Search form submits and displays results
|
|
970
|
+
- [ ] Activity log displays recent operations
|
|
971
|
+
- [ ] Statistics cards show correct data
|
|
972
|
+
- [ ] Sources list displays ingested documents
|
|
973
|
+
- [ ] Navigation between tabs works
|
|
974
|
+
- [ ] Dark/light mode toggle works (if implemented)
|
|
975
|
+
- [ ] Responsive layout on mobile viewport
|
|
976
|
+
- [ ] Error states displayed correctly
|
|
977
|
+
|
|
978
|
+
---
|
|
979
|
+
|
|
980
|
+
### CRR-1111: E2E Tests - Full User Flows
|
|
981
|
+
**Estimate**: 5 points
|
|
982
|
+
**Labels**: testing, e2e
|
|
983
|
+
**Blocked by**: CRR-1108, CRR-1109, CRR-1110
|
|
984
|
+
|
|
985
|
+
Test complete user workflows from ingestion to retrieval.
|
|
986
|
+
|
|
987
|
+
**Files**: `tests/e2e/flows/*.test.ts`
|
|
988
|
+
|
|
989
|
+
**Acceptance Criteria**:
|
|
990
|
+
- [ ] Flow: Ingest URL → Search → View results
|
|
991
|
+
- [ ] Flow: Ingest file → Search → Verify content
|
|
992
|
+
- [ ] Flow: Chat ingest → Search past solutions
|
|
993
|
+
- [ ] Flow: MCP search from simulated Cursor request
|
|
994
|
+
- [ ] Flow: Dashboard search → Click result → View details
|
|
995
|
+
- [ ] Performance: Search returns in <500ms for 1000 chunks
|
|
996
|
+
- [ ] Performance: Ingest 100 documents in <30s
|
|
997
|
+
|
|
998
|
+
---
|
|
999
|
+
|
|
1000
|
+
### CRR-1112: Test Fixtures & Factories
|
|
1001
|
+
**Estimate**: 2 points
|
|
1002
|
+
**Labels**: testing, infrastructure
|
|
1003
|
+
**Blocked by**: CRR-1101
|
|
1004
|
+
|
|
1005
|
+
Create reusable test fixtures and data factories.
|
|
1006
|
+
|
|
1007
|
+
**Files**: `tests/fixtures/*.ts`, `tests/factories/*.ts`
|
|
1008
|
+
|
|
1009
|
+
**Acceptance Criteria**:
|
|
1010
|
+
- [ ] Sample conversations fixture (various formats)
|
|
1011
|
+
- [ ] Sample documents fixture (markdown, code, mixed)
|
|
1012
|
+
- [ ] EnhancedChunk factory with sensible defaults
|
|
1013
|
+
- [ ] Conversation factory with customizable messages
|
|
1014
|
+
- [ ] Relationship factory for graph tests
|
|
1015
|
+
- [ ] Category factory for hierarchy tests
|
|
1016
|
+
- [ ] Mock vector store with predictable search results
|
|
1017
|
+
- [ ] Mock LLM with configurable responses
|
|
1018
|
+
|
|
1019
|
+
---
|
|
1020
|
+
|
|
1021
|
+
### CRR-1113: Test Documentation & Coverage Report
|
|
1022
|
+
**Estimate**: 1 point
|
|
1023
|
+
**Labels**: testing, documentation
|
|
1024
|
+
**Blocked by**: CRR-1102 through CRR-1112
|
|
1025
|
+
|
|
1026
|
+
Document testing strategy and generate coverage reports.
|
|
1027
|
+
|
|
1028
|
+
**Files**: `docs/TESTING.md`, `coverage/`
|
|
1029
|
+
|
|
1030
|
+
**Acceptance Criteria**:
|
|
1031
|
+
- [ ] TESTING.md with testing strategy overview
|
|
1032
|
+
- [ ] Instructions for running different test suites
|
|
1033
|
+
- [ ] Coverage report generation configured
|
|
1034
|
+
- [ ] Coverage badges in README
|
|
1035
|
+
- [ ] Test naming conventions documented
|
|
1036
|
+
- [ ] Mock usage guidelines documented
|
|
1037
|
+
|
|
1038
|
+
---
|
|
1039
|
+
|
|
1040
|
+
## Epic: Phase 12 - PageIndex Integration (Vectorless RAG)
|
|
1041
|
+
|
|
1042
|
+
*Complementary retrieval using hierarchical tree indexes based on [PageIndex](https://github.com/VectifyAI/PageIndex)*
|
|
1043
|
+
|
|
1044
|
+
**Key Insight**: Vector RAG and PageIndex solve retrieval differently and work together:
|
|
1045
|
+
- **Vector RAG excels at**: Semantic similarity, cross-document search, finding related concepts
|
|
1046
|
+
- **PageIndex excels at**: Structured documents (PDFs, reports), preserving document hierarchy, explainable retrieval with page references
|
|
1047
|
+
|
|
1048
|
+
### CRR-1201: Define PageIndex Types
|
|
1049
|
+
**Estimate**: 2 points
|
|
1050
|
+
**Labels**: pageindex, types
|
|
1051
|
+
|
|
1052
|
+
Define TypeScript types for PageIndex tree structures and operations.
|
|
1053
|
+
|
|
1054
|
+
**File**: `src/adapters/pageindex/types.ts`
|
|
1055
|
+
|
|
1056
|
+
**Acceptance Criteria**:
|
|
1057
|
+
- [ ] TreeNode interface (title, node_id, start_index, end_index, summary, nodes)
|
|
1058
|
+
- [ ] TreeIndex interface (document metadata + root nodes)
|
|
1059
|
+
- [ ] PageIndexResult interface (node IDs, content, page references)
|
|
1060
|
+
- [ ] PageIndexConfig interface (model, maxPagesPerNode, maxTokensPerNode)
|
|
1061
|
+
- [ ] PageIndexAdapter interface (buildIndex, search, getNodeContent)
|
|
1062
|
+
|
|
1063
|
+
---
|
|
1064
|
+
|
|
1065
|
+
### CRR-1202: Implement Tree Builder Adapter
|
|
1066
|
+
**Estimate**: 4 points
|
|
1067
|
+
**Labels**: pageindex, adapter
|
|
1068
|
+
**Blocked by**: CRR-1201, CRR-1000
|
|
1069
|
+
|
|
1070
|
+
Create adapter that wraps PageIndex Python via child_process.
|
|
1071
|
+
|
|
1072
|
+
**File**: `src/adapters/pageindex/tree-builder.ts`
|
|
1073
|
+
|
|
1074
|
+
**Acceptance Criteria**:
|
|
1075
|
+
- [ ] Spawns Python process with pageindex package
|
|
1076
|
+
- [ ] Passes PDF/markdown path and config options
|
|
1077
|
+
- [ ] Parses JSON tree output
|
|
1078
|
+
- [ ] Handles Python errors gracefully
|
|
1079
|
+
- [ ] Caches built trees to avoid re-processing
|
|
1080
|
+
- [ ] Supports both PDF and markdown input
|
|
1081
|
+
|
|
1082
|
+
---
|
|
1083
|
+
|
|
1084
|
+
### CRR-1203: Implement Tree Storage
|
|
1085
|
+
**Estimate**: 2 points
|
|
1086
|
+
**Labels**: pageindex, storage
|
|
1087
|
+
**Blocked by**: CRR-1201
|
|
1088
|
+
|
|
1089
|
+
Store and retrieve PageIndex tree structures.
|
|
1090
|
+
|
|
1091
|
+
**File**: `src/adapters/pageindex/tree-store.ts`
|
|
1092
|
+
|
|
1093
|
+
**Acceptance Criteria**:
|
|
1094
|
+
- [ ] Save tree JSON to ~/.cursor-rag/pageindex/
|
|
1095
|
+
- [ ] Retrieve tree by document ID/path
|
|
1096
|
+
- [ ] List all indexed documents with tree metadata
|
|
1097
|
+
- [ ] Delete tree when source document removed
|
|
1098
|
+
- [ ] Verify tree freshness against source modification time
|
|
1099
|
+
|
|
1100
|
+
---
|
|
1101
|
+
|
|
1102
|
+
### CRR-1204: Implement Tree Searcher
|
|
1103
|
+
**Estimate**: 4 points
|
|
1104
|
+
**Labels**: pageindex, retrieval
|
|
1105
|
+
**Blocked by**: CRR-1201, CRR-1203, CRR-1000
|
|
1106
|
+
|
|
1107
|
+
LLM-based tree traversal for retrieval.
|
|
1108
|
+
|
|
1109
|
+
**File**: `src/adapters/pageindex/tree-searcher.ts`
|
|
1110
|
+
|
|
1111
|
+
**Acceptance Criteria**:
|
|
1112
|
+
- [ ] Uses LLM to navigate tree hierarchy
|
|
1113
|
+
- [ ] Returns relevant node IDs with reasoning
|
|
1114
|
+
- [ ] Fetches full content for selected nodes
|
|
1115
|
+
- [ ] Includes page references in results
|
|
1116
|
+
- [ ] Supports multi-step tree navigation
|
|
1117
|
+
- [ ] Tracks reasoning chain for explainability
|
|
1118
|
+
|
|
1119
|
+
---
|
|
1120
|
+
|
|
1121
|
+
### CRR-1205: Implement Hybrid Search Merger
|
|
1122
|
+
**Estimate**: 3 points
|
|
1123
|
+
**Labels**: pageindex, retrieval
|
|
1124
|
+
**Blocked by**: CRR-1204
|
|
1125
|
+
|
|
1126
|
+
Merge results from vector search and PageIndex.
|
|
1127
|
+
|
|
1128
|
+
**File**: `src/services/hybridSearchMerger.ts`
|
|
1129
|
+
|
|
1130
|
+
**Acceptance Criteria**:
|
|
1131
|
+
- [ ] Run vector and PageIndex searches in parallel
|
|
1132
|
+
- [ ] Configurable weighting between sources
|
|
1133
|
+
- [ ] Deduplicate overlapping results
|
|
1134
|
+
- [ ] Preserve source attribution (vector vs PageIndex)
|
|
1135
|
+
- [ ] Include page references for PageIndex results
|
|
1136
|
+
- [ ] Score normalization across retrieval methods
|
|
1137
|
+
|
|
1138
|
+
---
|
|
1139
|
+
|
|
1140
|
+
### CRR-1206: Add PageIndex MCP Tools
|
|
1141
|
+
**Estimate**: 3 points
|
|
1142
|
+
**Labels**: pageindex, mcp
|
|
1143
|
+
**Blocked by**: CRR-1202, CRR-1204, CRR-1205
|
|
1144
|
+
|
|
1145
|
+
New MCP tools for PageIndex operations.
|
|
1146
|
+
|
|
1147
|
+
**File**: `src/server/tools/pageindex.ts`
|
|
1148
|
+
|
|
1149
|
+
**Acceptance Criteria**:
|
|
1150
|
+
- [ ] `pageindex_ingest` - Build tree for PDF/markdown
|
|
1151
|
+
- [ ] `pageindex_search` - Query using tree navigation
|
|
1152
|
+
- [ ] `pageindex_list` - List indexed documents with tree info
|
|
1153
|
+
- [ ] `hybrid_search` - Combined vector + PageIndex search
|
|
1154
|
+
- [ ] All tools have proper input validation
|
|
1155
|
+
- [ ] Results include source and page references
|
|
1156
|
+
|
|
1157
|
+
---
|
|
1158
|
+
|
|
1159
|
+
### CRR-1207: Add PageIndex CLI Commands
|
|
1160
|
+
**Estimate**: 2 points
|
|
1161
|
+
**Labels**: pageindex, cli
|
|
1162
|
+
**Blocked by**: CRR-1202, CRR-1203, CRR-1204
|
|
1163
|
+
|
|
1164
|
+
CLI commands for PageIndex management.
|
|
1165
|
+
|
|
1166
|
+
**File**: `src/cli/commands/pageindex.ts`
|
|
1167
|
+
|
|
1168
|
+
**Acceptance Criteria**:
|
|
1169
|
+
- [ ] `cursor-rag pageindex build <path>` - Build tree index
|
|
1170
|
+
- [ ] `cursor-rag pageindex list` - List indexed documents
|
|
1171
|
+
- [ ] `cursor-rag pageindex search <query>` - Search with tree navigation
|
|
1172
|
+
- [ ] `cursor-rag pageindex info <document>` - Show tree structure
|
|
1173
|
+
- [ ] `cursor-rag pageindex remove <document>` - Remove tree index
|
|
1174
|
+
- [ ] Progress display during tree building
|
|
1175
|
+
|
|
1176
|
+
---
|
|
1177
|
+
|
|
1178
|
+
### CRR-1208: Dashboard PageIndex Integration
|
|
1179
|
+
**Estimate**: 3 points
|
|
1180
|
+
**Labels**: pageindex, dashboard
|
|
1181
|
+
**Blocked by**: CRR-1206, CRR-903
|
|
1182
|
+
|
|
1183
|
+
Add PageIndex features to web dashboard.
|
|
1184
|
+
|
|
1185
|
+
**File**: `src/dashboard/public/index.html`, `src/dashboard/server.ts`
|
|
1186
|
+
|
|
1187
|
+
**Acceptance Criteria**:
|
|
1188
|
+
- [ ] PageIndex tab showing indexed documents
|
|
1189
|
+
- [ ] Tree structure visualization
|
|
1190
|
+
- [ ] Search mode toggle (Vector / PageIndex / Hybrid)
|
|
1191
|
+
- [ ] Results show source type and page references
|
|
1192
|
+
- [ ] Build tree from uploaded PDF
|
|
1193
|
+
- [ ] Tree node expansion/collapse
|
|
1194
|
+
|
|
1195
|
+
---
|
|
1196
|
+
|
|
1197
|
+
### CRR-1209: Auto-Detection and Routing
|
|
1198
|
+
**Estimate**: 2 points
|
|
1199
|
+
**Labels**: pageindex, ingest
|
|
1200
|
+
**Blocked by**: CRR-1202, CRR-1205
|
|
1201
|
+
|
|
1202
|
+
Automatically route documents to appropriate pipeline.
|
|
1203
|
+
|
|
1204
|
+
**File**: `src/server/tools/ingest.ts` (modification)
|
|
1205
|
+
|
|
1206
|
+
**Acceptance Criteria**:
|
|
1207
|
+
- [ ] Detect document type (PDF, markdown, text, URL)
|
|
1208
|
+
- [ ] PDFs automatically get PageIndex tree + vector embeddings
|
|
1209
|
+
- [ ] Structured markdown gets PageIndex tree + vector embeddings
|
|
1210
|
+
- [ ] Plain text/URLs only use vector pipeline
|
|
1211
|
+
- [ ] Configuration to override auto-detection
|
|
1212
|
+
- [ ] Progress reporting for dual-pipeline ingestion
|
|
1213
|
+
|
|
1214
|
+
---
|
|
1215
|
+
|
|
1216
|
+
## Summary
|
|
1217
|
+
|
|
1218
|
+
| Epic | Tasks | Total Points |
|
|
1219
|
+
|------|-------|--------------|
|
|
1220
|
+
| Phase 1: Foundation | 4 | 9 |
|
|
1221
|
+
| Phase 2: Chat History | 3 | 7 |
|
|
1222
|
+
| Phase 3: Knowledge Extraction | 3 | 9 |
|
|
1223
|
+
| Phase 4: Relationship Graph | 2 | 5 |
|
|
1224
|
+
| Phase 5: Hierarchical Memory | 2 | 6 |
|
|
1225
|
+
| Phase 6: Maintenance | 2 | 6 |
|
|
1226
|
+
| Phase 7: Enhanced Retrieval | 3 | 8 |
|
|
1227
|
+
| Phase 8: RLM Recursive Retrieval | 4 | 16 |
|
|
1228
|
+
| Phase 9: Dashboard Tools UI | 4 | 11 |
|
|
1229
|
+
| Phase 10: Rules Optimizer | 9 | 30 |
|
|
1230
|
+
| Phase 11: Test Suite | 13 | 44 |
|
|
1231
|
+
| Phase 12: PageIndex Integration | 9 | 25 |
|
|
1232
|
+
| **Total** | **58** | **176** |
|
|
1233
|
+
|
|
1234
|
+
---
|
|
1235
|
+
|
|
1236
|
+
## Suggested Sprint Planning
|
|
1237
|
+
|
|
1238
|
+
### Sprint 1 (Week 1-2): Foundation + Chat History
|
|
1239
|
+
- CRR-101, CRR-102, CRR-103, CRR-104
|
|
1240
|
+
- CRR-201, CRR-202, CRR-203
|
|
1241
|
+
- **Points**: 16
|
|
1242
|
+
|
|
1243
|
+
### Sprint 2 (Week 3-4): Knowledge Extraction + Graph
|
|
1244
|
+
- CRR-301, CRR-302, CRR-303
|
|
1245
|
+
- CRR-401, CRR-402
|
|
1246
|
+
- **Points**: 14
|
|
1247
|
+
|
|
1248
|
+
### Sprint 3 (Week 5-6): Categories + Maintenance + Retrieval
|
|
1249
|
+
- CRR-501, CRR-502
|
|
1250
|
+
- CRR-601, CRR-602
|
|
1251
|
+
- CRR-701, CRR-702, CRR-703
|
|
1252
|
+
- **Points**: 20
|
|
1253
|
+
|
|
1254
|
+
### Sprint 4 (Week 7-8): RLM Recursive Retrieval
|
|
1255
|
+
- CRR-801, CRR-802, CRR-803, CRR-804
|
|
1256
|
+
- **Points**: 16
|
|
1257
|
+
|
|
1258
|
+
### Sprint 5 (Week 9-10): Dashboard Tools + LLM Provider + Rules Start
|
|
1259
|
+
- CRR-901, CRR-902, CRR-903, CRR-904
|
|
1260
|
+
- CRR-1000 (LLM Provider System - enables Phase 10 LLM features)
|
|
1261
|
+
- CRR-1001, CRR-1002
|
|
1262
|
+
- **Points**: 23
|
|
1263
|
+
|
|
1264
|
+
### Sprint 6 (Week 11-12): Rules Optimizer Completion
|
|
1265
|
+
- CRR-1003, CRR-1004, CRR-1005, CRR-1006, CRR-1007
|
|
1266
|
+
- **Points**: 19
|
|
1267
|
+
|
|
1268
|
+
**Total estimated time: 11-12 weeks**
|
|
1269
|
+
|
|
1270
|
+
---
|
|
1271
|
+
|
|
1272
|
+
## Dependencies Graph
|
|
1273
|
+
|
|
1274
|
+
```
|
|
1275
|
+
CRR-101 ──┬── CRR-102 ──┬── CRR-103 ──┬── CRR-104
|
|
1276
|
+
│ │ │
|
|
1277
|
+
│ └── CRR-402 ──┘
|
|
1278
|
+
│
|
|
1279
|
+
└── CRR-301 ──── CRR-302 ──── CRR-303 ──┐
|
|
1280
|
+
│
|
|
1281
|
+
CRR-201 ──── CRR-202 ──── CRR-203 │
|
|
1282
|
+
│
|
|
1283
|
+
CRR-401 ──── CRR-402 ─────────────────────────────┤
|
|
1284
|
+
│
|
|
1285
|
+
CRR-501 ──── CRR-502 ─────────────────────────────┤
|
|
1286
|
+
│
|
|
1287
|
+
CRR-601 ──── CRR-602 │
|
|
1288
|
+
│
|
|
1289
|
+
┌───────────────────────────────────────┘
|
|
1290
|
+
│
|
|
1291
|
+
CRR-701 ──┼── CRR-702
|
|
1292
|
+
│
|
|
1293
|
+
CRR-703 ──┴── CRR-801 ──┬── CRR-802 ──┬── CRR-803
|
|
1294
|
+
│ │
|
|
1295
|
+
└─────────────┴── CRR-804
|
|
1296
|
+
|
|
1297
|
+
Phase 9: Dashboard Tools (can run in parallel)
|
|
1298
|
+
CRR-901 ──── CRR-902 ──── CRR-903
|
|
1299
|
+
└──── CRR-904
|
|
1300
|
+
|
|
1301
|
+
Phase 10: Rules Optimizer (can run in parallel)
|
|
1302
|
+
CRR-1000 (LLM Provider) ──┐
|
|
1303
|
+
├──── CRR-1004 (Rules Merger - needs LLM)
|
|
1304
|
+
CRR-1001 ──── CRR-1002 ──── CRR-1003 ──┘
|
|
1305
|
+
└──── CRR-1005 ──── CRR-1006
|
|
1306
|
+
└──── CRR-1007 (requires CRR-903)
|
|
1307
|
+
|
|
1308
|
+
LLM Provider Priority Order (CRR-1000):
|
|
1309
|
+
1. Cursor AI (via MCP) → if running in Cursor IDE
|
|
1310
|
+
2. Environment vars → OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.
|
|
1311
|
+
3. Config file → ~/.cursor-rag/llm-config.json
|
|
1312
|
+
4. Ollama → local fallback (free, no API key)
|
|
1313
|
+
|
|
1314
|
+
Phase 11: Test Suite (can run in parallel, tests existing features)
|
|
1315
|
+
CRR-1101 (Infrastructure) ──┬── CRR-1102 (Types/Utils)
|
|
1316
|
+
├── CRR-1103 (Services 1-2) ──┬── CRR-1106 (Integration: Vector+Meta)
|
|
1317
|
+
├── CRR-1104 (Services 3-4) ──┴── CRR-1107 (Integration: Knowledge)
|
|
1318
|
+
├── CRR-1105 (Adapters) ──────┘
|
|
1319
|
+
├── CRR-1108 (Integration: CLI)
|
|
1320
|
+
├── CRR-1109 (Integration: MCP)
|
|
1321
|
+
├── CRR-1110 (E2E: Dashboard)
|
|
1322
|
+
└── CRR-1112 (Fixtures)
|
|
1323
|
+
│
|
|
1324
|
+
CRR-1106 + CRR-1107 + CRR-1108 + CRR-1109 + CRR-1110 ──── CRR-1111 (E2E: Flows)
|
|
1325
|
+
│
|
|
1326
|
+
All tests (CRR-1102 through CRR-1112) ──── CRR-1113 (Documentation)
|
|
1327
|
+
|
|
1328
|
+
Phase 12: PageIndex Integration (Vectorless RAG complement)
|
|
1329
|
+
CRR-1201 (Types) ──── CRR-1202 (Tree Builder) ──── CRR-1203 (Tree Storage)
|
|
1330
|
+
│
|
|
1331
|
+
└── CRR-1204 (Tree Searcher) ──── CRR-1205 (Hybrid Merger)
|
|
1332
|
+
│
|
|
1333
|
+
└── CRR-1206 (MCP Tools)
|
|
1334
|
+
│
|
|
1335
|
+
CRR-1000 (LLM Provider) ──────────────────────────────────────────────────────────┘
|
|
1336
|
+
│
|
|
1337
|
+
CRR-1207 (CLI Commands)
|
|
1338
|
+
│
|
|
1339
|
+
CRR-1208 (Dashboard Integration)
|
|
1340
|
+
│
|
|
1341
|
+
CRR-1209 (Auto-Detection)
|
|
1342
|
+
|
|
1343
|
+
Notes:
|
|
1344
|
+
- Phase 8 (RLM) depends on CRR-701 (Hybrid Scorer)
|
|
1345
|
+
- Phase 9 (Dashboard Tools) is independent, can start anytime
|
|
1346
|
+
- Phase 10 (Rules Optimizer) is independent, can start anytime
|
|
1347
|
+
- Phase 11 (Test Suite) is independent, can start anytime - tests existing code
|
|
1348
|
+
- Phase 12 (PageIndex) is independent, can start anytime - complements vector RAG
|
|
1349
|
+
- CRR-1000 (LLM Provider) enables all LLM-dependent features across the system
|
|
1350
|
+
- CRR-1004 depends on CRR-1000 + CRR-1003 for LLM-powered merging
|
|
1351
|
+
- CRR-1007 depends on CRR-903 (Tools UI Panel) for dashboard integration
|
|
1352
|
+
- CRR-1101 (Test Infrastructure) should be done first in Phase 11
|
|
1353
|
+
- CRR-1204 and CRR-1206 depend on CRR-1000 (LLM Provider) for tree traversal
|
|
1354
|
+
- CRR-1208 depends on CRR-903 (Dashboard Tools UI) for integration
|
|
1355
|
+
```
|