@pleaseai/context-please-core 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +74 -46
- package/dist/.tsbuildinfo +1 -1
- package/dist/context.d.ts +12 -12
- package/dist/context.d.ts.map +1 -1
- package/dist/context.js +179 -76
- package/dist/context.js.map +1 -1
- package/dist/embedding/base-embedding.d.ts.map +1 -1
- package/dist/embedding/base-embedding.js +1 -1
- package/dist/embedding/base-embedding.js.map +1 -1
- package/dist/embedding/gemini-embedding.d.ts +2 -1
- package/dist/embedding/gemini-embedding.d.ts.map +1 -1
- package/dist/embedding/gemini-embedding.js +8 -8
- package/dist/embedding/gemini-embedding.js.map +1 -1
- package/dist/embedding/index.d.ts +2 -2
- package/dist/embedding/index.d.ts.map +1 -1
- package/dist/embedding/index.js +2 -2
- package/dist/embedding/index.js.map +1 -1
- package/dist/embedding/ollama-embedding.d.ts +2 -1
- package/dist/embedding/ollama-embedding.d.ts.map +1 -1
- package/dist/embedding/ollama-embedding.js +5 -5
- package/dist/embedding/ollama-embedding.js.map +1 -1
- package/dist/embedding/openai-embedding.d.ts +2 -1
- package/dist/embedding/openai-embedding.d.ts.map +1 -1
- package/dist/embedding/openai-embedding.js +10 -10
- package/dist/embedding/openai-embedding.js.map +1 -1
- package/dist/embedding/voyageai-embedding.d.ts +2 -1
- package/dist/embedding/voyageai-embedding.d.ts.map +1 -1
- package/dist/embedding/voyageai-embedding.js +23 -23
- package/dist/embedding/voyageai-embedding.js.map +1 -1
- package/dist/index.d.ts +4 -4
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +4 -4
- package/dist/index.js.map +1 -1
- package/dist/splitter/ast-splitter.d.ts +1 -1
- package/dist/splitter/ast-splitter.d.ts.map +1 -1
- package/dist/splitter/ast-splitter.js +30 -16
- package/dist/splitter/ast-splitter.js.map +1 -1
- package/dist/splitter/index.d.ts +4 -4
- package/dist/splitter/index.d.ts.map +1 -1
- package/dist/splitter/index.js +1 -1
- package/dist/splitter/index.js.map +1 -1
- package/dist/splitter/langchain-splitter.d.ts +1 -1
- package/dist/splitter/langchain-splitter.d.ts.map +1 -1
- package/dist/splitter/langchain-splitter.js.map +1 -1
- package/dist/sync/merkle.d.ts.map +1 -1
- package/dist/sync/merkle.js +9 -9
- package/dist/sync/merkle.js.map +1 -1
- package/dist/sync/synchronizer.d.ts.map +1 -1
- package/dist/sync/synchronizer.js +15 -15
- package/dist/sync/synchronizer.js.map +1 -1
- package/dist/types.d.ts.map +1 -1
- package/dist/utils/env-manager.d.ts.map +1 -1
- package/dist/utils/env-manager.js +3 -3
- package/dist/utils/env-manager.js.map +1 -1
- package/dist/utils/index.d.ts.map +1 -1
- package/dist/utils/index.js.map +1 -1
- package/dist/vectordb/base/base-vector-database.d.ts +1 -1
- package/dist/vectordb/base/base-vector-database.d.ts.map +1 -1
- package/dist/vectordb/base/base-vector-database.js.map +1 -1
- package/dist/vectordb/factory.d.ts +6 -6
- package/dist/vectordb/factory.d.ts.map +1 -1
- package/dist/vectordb/factory.js +1 -1
- package/dist/vectordb/factory.js.map +1 -1
- package/dist/vectordb/index.d.ts +9 -9
- package/dist/vectordb/index.d.ts.map +1 -1
- package/dist/vectordb/index.js +8 -8
- package/dist/vectordb/index.js.map +1 -1
- package/dist/vectordb/milvus-restful-vectordb.d.ts +6 -5
- package/dist/vectordb/milvus-restful-vectordb.d.ts.map +1 -1
- package/dist/vectordb/milvus-restful-vectordb.js +136 -136
- package/dist/vectordb/milvus-restful-vectordb.js.map +1 -1
- package/dist/vectordb/milvus-vectordb.d.ts +5 -4
- package/dist/vectordb/milvus-vectordb.d.ts.map +1 -1
- package/dist/vectordb/milvus-vectordb.js +31 -31
- package/dist/vectordb/milvus-vectordb.js.map +1 -1
- package/dist/vectordb/qdrant-vectordb.d.ts +28 -3
- package/dist/vectordb/qdrant-vectordb.d.ts.map +1 -1
- package/dist/vectordb/qdrant-vectordb.js +298 -73
- package/dist/vectordb/qdrant-vectordb.js.map +1 -1
- package/dist/vectordb/sparse/index.d.ts +2 -2
- package/dist/vectordb/sparse/index.d.ts.map +1 -1
- package/dist/vectordb/sparse/index.js +4 -4
- package/dist/vectordb/sparse/index.js.map +1 -1
- package/dist/vectordb/sparse/simple-bm25.d.ts +12 -2
- package/dist/vectordb/sparse/simple-bm25.d.ts.map +1 -1
- package/dist/vectordb/sparse/simple-bm25.js +82 -9
- package/dist/vectordb/sparse/simple-bm25.js.map +1 -1
- package/dist/vectordb/sparse/sparse-vector-generator.d.ts +7 -7
- package/dist/vectordb/sparse/sparse-vector-generator.d.ts.map +1 -1
- package/dist/vectordb/sparse/types.d.ts.map +1 -1
- package/dist/vectordb/types.d.ts +12 -12
- package/dist/vectordb/types.d.ts.map +1 -1
- package/dist/vectordb/types.js +1 -1
- package/dist/vectordb/types.js.map +1 -1
- package/dist/vectordb/zilliz-utils.d.ts +10 -10
- package/dist/vectordb/zilliz-utils.d.ts.map +1 -1
- package/dist/vectordb/zilliz-utils.js +16 -17
- package/dist/vectordb/zilliz-utils.js.map +1 -1
- package/package.json +18 -14
package/README.md
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
# @pleaseai/context-please-core
|
|
2
|
+
|
|
2
3
|

|
|
3
4
|
|
|
4
5
|
The core indexing engine for Context Please - a powerful tool for semantic search and analysis of codebases using vector embeddings and AI.
|
|
@@ -17,20 +18,24 @@ npm install @pleaseai/context-please-core
|
|
|
17
18
|
```
|
|
18
19
|
|
|
19
20
|
### Prepare Environment Variables
|
|
21
|
+
|
|
20
22
|
#### OpenAI API key
|
|
23
|
+
|
|
21
24
|
See [OpenAI Documentation](https://platform.openai.com/docs/api-reference) for more details to get your API key.
|
|
25
|
+
|
|
22
26
|
```bash
|
|
23
27
|
OPENAI_API_KEY=your-openai-api-key
|
|
24
28
|
```
|
|
25
29
|
|
|
26
30
|
#### Zilliz Cloud configuration
|
|
27
|
-
|
|
31
|
+
|
|
32
|
+
Get a free Milvus vector database on Zilliz Cloud.
|
|
28
33
|
|
|
29
34
|
Claude Context needs a vector database. You can [sign up](https://cloud.zilliz.com/signup?utm_source=github&utm_medium=referral&utm_campaign=2507-codecontext-readme) on Zilliz Cloud to get a free Serverless cluster.
|
|
30
35
|
|
|
31
36
|

|
|
32
37
|
|
|
33
|
-
After creating your cluster, open your Zilliz Cloud console and copy both the **public endpoint** and your **API key**.
|
|
38
|
+
After creating your cluster, open your Zilliz Cloud console and copy both the **public endpoint** and your **API key**.
|
|
34
39
|
These will be used as `your-zilliz-cloud-public-endpoint` and `your-zilliz-cloud-api-key` in the configuration examples.
|
|
35
40
|
|
|
36
41
|

|
|
@@ -42,56 +47,56 @@ If you need help creating your free vector database or finding these values, see
|
|
|
42
47
|
```bash
|
|
43
48
|
MILVUS_ADDRESS=your-zilliz-cloud-public-endpoint
|
|
44
49
|
MILVUS_TOKEN=your-zilliz-cloud-api-key
|
|
45
|
-
```
|
|
50
|
+
```
|
|
46
51
|
|
|
47
52
|
> 💡 **Tip**: For easier configuration management across different usage scenarios, consider using [global environment variables](../../docs/getting-started/environment-variables.md).
|
|
48
53
|
|
|
49
54
|
## Quick Start
|
|
50
55
|
|
|
51
56
|
```typescript
|
|
52
|
-
import {
|
|
53
|
-
Context,
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
} from '@pleaseai/context-please-core'
|
|
57
|
+
import {
|
|
58
|
+
Context,
|
|
59
|
+
MilvusVectorDatabase,
|
|
60
|
+
OpenAIEmbedding
|
|
61
|
+
} from '@pleaseai/context-please-core'
|
|
57
62
|
|
|
58
63
|
// Initialize embedding provider
|
|
59
64
|
const embedding = new OpenAIEmbedding({
|
|
60
65
|
apiKey: process.env.OPENAI_API_KEY || 'your-openai-api-key',
|
|
61
66
|
model: 'text-embedding-3-small'
|
|
62
|
-
})
|
|
67
|
+
})
|
|
63
68
|
|
|
64
69
|
// Initialize vector database
|
|
65
70
|
const vectorDatabase = new MilvusVectorDatabase({
|
|
66
71
|
address: process.env.MILVUS_ADDRESS || 'localhost:19530',
|
|
67
72
|
token: process.env.MILVUS_TOKEN || ''
|
|
68
|
-
})
|
|
73
|
+
})
|
|
69
74
|
|
|
70
75
|
// Create context instance
|
|
71
76
|
const context = new Context({
|
|
72
77
|
embedding,
|
|
73
78
|
vectorDatabase
|
|
74
|
-
})
|
|
79
|
+
})
|
|
75
80
|
|
|
76
81
|
// Index a codebase
|
|
77
82
|
const stats = await context.indexCodebase('./my-project', (progress) => {
|
|
78
|
-
console.log(`${progress.phase} - ${progress.percentage}%`)
|
|
79
|
-
})
|
|
83
|
+
console.log(`${progress.phase} - ${progress.percentage}%`)
|
|
84
|
+
})
|
|
80
85
|
|
|
81
|
-
console.log(`Indexed ${stats.indexedFiles} files with ${stats.totalChunks} chunks`)
|
|
86
|
+
console.log(`Indexed ${stats.indexedFiles} files with ${stats.totalChunks} chunks`)
|
|
82
87
|
|
|
83
88
|
// Search the codebase
|
|
84
89
|
const results = await context.semanticSearch(
|
|
85
90
|
'./my-project',
|
|
86
91
|
'function that handles user authentication',
|
|
87
92
|
5
|
|
88
|
-
)
|
|
93
|
+
)
|
|
89
94
|
|
|
90
|
-
results.forEach(result => {
|
|
91
|
-
console.log(`${result.relativePath}:${result.startLine}-${result.endLine}`)
|
|
92
|
-
console.log(`Score: ${result.score}`)
|
|
93
|
-
console.log(result.content)
|
|
94
|
-
})
|
|
95
|
+
results.forEach((result) => {
|
|
96
|
+
console.log(`${result.relativePath}:${result.startLine}-${result.endLine}`)
|
|
97
|
+
console.log(`Score: ${result.score}`)
|
|
98
|
+
console.log(result.content)
|
|
99
|
+
})
|
|
95
100
|
```
|
|
96
101
|
|
|
97
102
|
## Features
|
|
@@ -126,13 +131,13 @@ results.forEach(result => {
|
|
|
126
131
|
|
|
127
132
|
```typescript
|
|
128
133
|
interface ContextConfig {
|
|
129
|
-
embedding?: Embedding
|
|
130
|
-
vectorDatabase?: VectorDatabase
|
|
131
|
-
codeSplitter?: Splitter
|
|
132
|
-
supportedExtensions?: string[]
|
|
133
|
-
ignorePatterns?: string[]
|
|
134
|
-
customExtensions?: string[]
|
|
135
|
-
customIgnorePatterns?: string[]
|
|
134
|
+
embedding?: Embedding // Embedding provider
|
|
135
|
+
vectorDatabase?: VectorDatabase // Vector database instance (required)
|
|
136
|
+
codeSplitter?: Splitter // Code splitting strategy
|
|
137
|
+
supportedExtensions?: string[] // File extensions to index
|
|
138
|
+
ignorePatterns?: string[] // Patterns to ignore
|
|
139
|
+
customExtensions?: string[] // Custom extensions from MCP
|
|
140
|
+
customIgnorePatterns?: string[] // Custom ignore patterns from MCP
|
|
136
141
|
}
|
|
137
142
|
```
|
|
138
143
|
|
|
@@ -141,10 +146,30 @@ interface ContextConfig {
|
|
|
141
146
|
```typescript
|
|
142
147
|
[
|
|
143
148
|
// Programming languages
|
|
144
|
-
'.ts',
|
|
145
|
-
'.
|
|
146
|
-
|
|
147
|
-
'.
|
|
149
|
+
'.ts',
|
|
150
|
+
'.tsx',
|
|
151
|
+
'.js',
|
|
152
|
+
'.jsx',
|
|
153
|
+
'.py',
|
|
154
|
+
'.java',
|
|
155
|
+
'.cpp',
|
|
156
|
+
'.c',
|
|
157
|
+
'.h',
|
|
158
|
+
'.hpp',
|
|
159
|
+
'.cs',
|
|
160
|
+
'.go',
|
|
161
|
+
'.rs',
|
|
162
|
+
'.php',
|
|
163
|
+
'.rb',
|
|
164
|
+
'.swift',
|
|
165
|
+
'.kt',
|
|
166
|
+
'.scala',
|
|
167
|
+
'.m',
|
|
168
|
+
'.mm',
|
|
169
|
+
// Text and markup files
|
|
170
|
+
'.md',
|
|
171
|
+
'.markdown',
|
|
172
|
+
'.ipynb'
|
|
148
173
|
]
|
|
149
174
|
```
|
|
150
175
|
|
|
@@ -180,38 +205,37 @@ interface ContextConfig {
|
|
|
180
205
|
|
|
181
206
|
```typescript
|
|
182
207
|
interface SemanticSearchResult {
|
|
183
|
-
content: string
|
|
184
|
-
relativePath: string
|
|
185
|
-
startLine: number
|
|
186
|
-
endLine: number
|
|
187
|
-
language: string
|
|
188
|
-
score: number
|
|
208
|
+
content: string // Code content
|
|
209
|
+
relativePath: string // File path relative to codebase root
|
|
210
|
+
startLine: number // Starting line number
|
|
211
|
+
endLine: number // Ending line number
|
|
212
|
+
language: string // Programming language
|
|
213
|
+
score: number // Similarity score (0-1)
|
|
189
214
|
}
|
|
190
215
|
```
|
|
191
216
|
|
|
192
|
-
|
|
193
217
|
## Examples
|
|
194
218
|
|
|
195
219
|
### Using VoyageAI Embeddings
|
|
196
220
|
|
|
197
221
|
```typescript
|
|
198
|
-
import { Context, MilvusVectorDatabase, VoyageAIEmbedding } from '@pleaseai/context-please-core'
|
|
222
|
+
import { Context, MilvusVectorDatabase, VoyageAIEmbedding } from '@pleaseai/context-please-core'
|
|
199
223
|
|
|
200
224
|
// Initialize with VoyageAI embedding provider
|
|
201
225
|
const embedding = new VoyageAIEmbedding({
|
|
202
226
|
apiKey: process.env.VOYAGEAI_API_KEY || 'your-voyageai-api-key',
|
|
203
227
|
model: 'voyage-code-3'
|
|
204
|
-
})
|
|
228
|
+
})
|
|
205
229
|
|
|
206
230
|
const vectorDatabase = new MilvusVectorDatabase({
|
|
207
231
|
address: process.env.MILVUS_ADDRESS || 'localhost:19530',
|
|
208
232
|
token: process.env.MILVUS_TOKEN || ''
|
|
209
|
-
})
|
|
233
|
+
})
|
|
210
234
|
|
|
211
235
|
const context = new Context({
|
|
212
236
|
embedding,
|
|
213
237
|
vectorDatabase
|
|
214
|
-
})
|
|
238
|
+
})
|
|
215
239
|
```
|
|
216
240
|
|
|
217
241
|
### Custom File Filtering
|
|
@@ -227,7 +251,7 @@ const context = new Context({
|
|
|
227
251
|
'*.spec.ts',
|
|
228
252
|
'*.test.js'
|
|
229
253
|
]
|
|
230
|
-
})
|
|
254
|
+
})
|
|
231
255
|
```
|
|
232
256
|
|
|
233
257
|
## File Synchronization Architecture
|
|
@@ -241,21 +265,25 @@ Claude Context implements an intelligent file synchronization system that effici
|
|
|
241
265
|
The file synchronization system uses a **Merkle tree-based approach** combined with SHA-256 file hashing to detect changes:
|
|
242
266
|
|
|
243
267
|
#### 1. File Hashing
|
|
268
|
+
|
|
244
269
|
- Each file in the codebase is hashed using SHA-256
|
|
245
270
|
- File hashes are computed based on file content, not metadata
|
|
246
271
|
- Hashes are stored with relative file paths for consistency across different environments
|
|
247
272
|
|
|
248
273
|
#### 2. Merkle Tree Construction
|
|
274
|
+
|
|
249
275
|
- All file hashes are organized into a Merkle tree structure
|
|
250
276
|
- The tree provides a single root hash that represents the entire codebase state
|
|
251
277
|
- Any change to any file will cause the root hash to change
|
|
252
278
|
|
|
253
279
|
#### 3. Snapshot Management
|
|
280
|
+
|
|
254
281
|
- File synchronization state is persisted to `~/.context/merkle/` directory
|
|
255
282
|
- Each codebase gets a unique snapshot file based on its absolute path hash
|
|
256
283
|
- Snapshots contain both file hashes and serialized Merkle tree data
|
|
257
284
|
|
|
258
285
|
#### 4. Change Detection Process
|
|
286
|
+
|
|
259
287
|
1. **Quick Check**: Compare current Merkle root hash with stored snapshot
|
|
260
288
|
2. **Detailed Analysis**: If root hashes differ, perform file-by-file comparison
|
|
261
289
|
3. **Change Classification**: Categorize changes into three types:
|
|
@@ -264,15 +292,16 @@ The file synchronization system uses a **Merkle tree-based approach** combined w
|
|
|
264
292
|
- **Removed**: Files that were deleted from the codebase
|
|
265
293
|
|
|
266
294
|
#### 5. Incremental Updates
|
|
295
|
+
|
|
267
296
|
- Only process files that have actually changed
|
|
268
297
|
- Update vector database entries only for modified chunks
|
|
269
298
|
- Remove entries for deleted files
|
|
270
299
|
- Add entries for new files
|
|
271
300
|
|
|
272
|
-
|
|
273
301
|
## Contributing
|
|
274
302
|
|
|
275
303
|
This package is part of the Claude Context monorepo. Please see:
|
|
304
|
+
|
|
276
305
|
- [Main Contributing Guide](../../CONTRIBUTING.md) - General contribution guidelines
|
|
277
306
|
- [Core Package Contributing](CONTRIBUTING.md) - Specific development guide for this package
|
|
278
307
|
|
|
@@ -281,7 +310,6 @@ This package is part of the Claude Context monorepo. Please see:
|
|
|
281
310
|
- **[@claude-context/mcp](../mcp)** - MCP server that uses this core engine
|
|
282
311
|
- **[VSCode Extension](../vscode-extension)** - VSCode extension built on this core
|
|
283
312
|
|
|
284
|
-
|
|
285
313
|
## License
|
|
286
314
|
|
|
287
|
-
MIT - See [LICENSE](../../LICENSE) for details
|
|
315
|
+
MIT - See [LICENSE](../../LICENSE) for details
|