@neural-tools/vector-db 0.1.5 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE.md +21 -80
- package/README.md +421 -0
- package/package.json +2 -2
package/LICENSE.md
CHANGED
|
@@ -1,80 +1,21 @@
|
|
|
1
|
-
#
|
|
2
|
-
|
|
3
|
-
Copyright (c) 2025 Luke Amy
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
### 2. Pro/Enterprise License (Proprietary)
|
|
24
|
-
|
|
25
|
-
The following features require a valid Pro or Enterprise license:
|
|
26
|
-
|
|
27
|
-
**Pro Features:**
|
|
28
|
-
- Vector database integration
|
|
29
|
-
- Semantic caching
|
|
30
|
-
- Fine-tuning workflows
|
|
31
|
-
- Cloud deployment templates (AWS/GCP)
|
|
32
|
-
- Premium templates and examples
|
|
33
|
-
- GitHub automation features
|
|
34
|
-
|
|
35
|
-
**Enterprise Features:**
|
|
36
|
-
- White-label support
|
|
37
|
-
- Custom integrations
|
|
38
|
-
- Priority support
|
|
39
|
-
- SLA guarantees
|
|
40
|
-
- Team collaboration features
|
|
41
|
-
|
|
42
|
-
These features are proprietary and may not be used without a valid license key purchased from neural-tools.dev.
|
|
43
|
-
|
|
44
|
-
### License Terms
|
|
45
|
-
|
|
46
|
-
1. **Free Tier**: You may use the free tier features for any purpose, including commercial use, under the MIT License terms.
|
|
47
|
-
|
|
48
|
-
2. **Pro/Enterprise**: You must purchase a license to access Pro or Enterprise features. Each license is:
|
|
49
|
-
- Per-user for individual licenses
|
|
50
|
-
- Per-organization for team/enterprise licenses
|
|
51
|
-
- Non-transferable without written consent
|
|
52
|
-
- Subject to the terms at neural-tools.dev/terms
|
|
53
|
-
|
|
54
|
-
3. **Source Code**: This repository is private. You may not:
|
|
55
|
-
- Redistribute the source code
|
|
56
|
-
- Create derivative works for redistribution
|
|
57
|
-
- Reverse engineer Pro/Enterprise features
|
|
58
|
-
- Remove or circumvent license checks
|
|
59
|
-
|
|
60
|
-
4. **Support**: Support is provided based on your license tier:
|
|
61
|
-
- Free: Community support only
|
|
62
|
-
- Pro: Email support (48-hour response)
|
|
63
|
-
- Enterprise: Priority support with SLA
|
|
64
|
-
|
|
65
|
-
### Warranty Disclaimer
|
|
66
|
-
|
|
67
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
|
68
|
-
|
|
69
|
-
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
|
70
|
-
|
|
71
|
-
### Contact
|
|
72
|
-
|
|
73
|
-
For licensing inquiries:
|
|
74
|
-
- Email: licensing@neural-tools.dev
|
|
75
|
-
- Website: https://neural-tools.dev/pricing
|
|
76
|
-
- Support: support@neural-tools.dev
|
|
77
|
-
|
|
78
|
-
---
|
|
79
|
-
|
|
80
|
-
**Last Updated:** January 2025
|
|
1
|
+
# MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Luke Amy
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,421 @@
|
|
|
1
|
+
# @neural-tools/vector-db
|
|
2
|
+
|
|
3
|
+
> Vector database abstraction layer for Neural Tools
|
|
4
|
+
|
|
5
|
+
[](https://www.npmjs.com/package/@neural-tools/vector-db)
|
|
6
|
+
[](../../LICENSE.md)
|
|
7
|
+
|
|
8
|
+
Unified interface for working with vector databases. Supports Pinecone, Chroma, Qdrant, and a local in-memory store.
|
|
9
|
+
|
|
10
|
+
## Installation
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
npm install @neural-tools/vector-db
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
### With Pinecone
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
npm install @neural-tools/vector-db @pinecone-database/pinecone
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
### With Chroma
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
npm install @neural-tools/vector-db chromadb
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
### With Qdrant
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
npm install @neural-tools/vector-db @qdrant/js-client-rest
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
## Features
|
|
35
|
+
|
|
36
|
+
- **Unified API** - Same interface for all vector databases
|
|
37
|
+
- **Multiple Providers** - Pinecone, Chroma, Qdrant, local storage
|
|
38
|
+
- **Type-Safe** - Full TypeScript support
|
|
39
|
+
- **Easy Switching** - Change providers without code changes
|
|
40
|
+
- **Local Development** - In-memory store for testing
|
|
41
|
+
|
|
42
|
+
## Quick Start
|
|
43
|
+
|
|
44
|
+
### Using Pinecone
|
|
45
|
+
|
|
46
|
+
```typescript
|
|
47
|
+
import { VectorDB } from '@neural-tools/vector-db';
|
|
48
|
+
|
|
49
|
+
const db = new VectorDB({
|
|
50
|
+
provider: 'pinecone',
|
|
51
|
+
config: {
|
|
52
|
+
apiKey: process.env.PINECONE_API_KEY,
|
|
53
|
+
environment: 'us-west1-gcp',
|
|
54
|
+
indexName: 'my-index'
|
|
55
|
+
}
|
|
56
|
+
});
|
|
57
|
+
|
|
58
|
+
await db.connect();
|
|
59
|
+
|
|
60
|
+
// Insert vectors
|
|
61
|
+
await db.upsert([
|
|
62
|
+
{
|
|
63
|
+
id: '1',
|
|
64
|
+
values: [0.1, 0.2, 0.3, ...],
|
|
65
|
+
metadata: { text: 'Hello world', category: 'greeting' }
|
|
66
|
+
}
|
|
67
|
+
]);
|
|
68
|
+
|
|
69
|
+
// Query
|
|
70
|
+
const results = await db.query({
|
|
71
|
+
vector: [0.1, 0.2, 0.3, ...],
|
|
72
|
+
topK: 5,
|
|
73
|
+
filter: { category: 'greeting' }
|
|
74
|
+
});
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### Using Local Store (Development)
|
|
78
|
+
|
|
79
|
+
```typescript
|
|
80
|
+
import { VectorDB } from '@neural-tools/vector-db';
|
|
81
|
+
|
|
82
|
+
const db = new VectorDB({
|
|
83
|
+
provider: 'local',
|
|
84
|
+
config: {
|
|
85
|
+
dimension: 1536 // Embedding dimension
|
|
86
|
+
}
|
|
87
|
+
});
|
|
88
|
+
|
|
89
|
+
await db.connect();
|
|
90
|
+
|
|
91
|
+
// Same API as other providers
|
|
92
|
+
await db.upsert([...]);
|
|
93
|
+
const results = await db.query({...});
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
### Using Chroma
|
|
97
|
+
|
|
98
|
+
```typescript
|
|
99
|
+
import { VectorDB } from '@neural-tools/vector-db';
|
|
100
|
+
|
|
101
|
+
const db = new VectorDB({
|
|
102
|
+
provider: 'chroma',
|
|
103
|
+
config: {
|
|
104
|
+
url: 'http://localhost:8000',
|
|
105
|
+
collectionName: 'my-collection'
|
|
106
|
+
}
|
|
107
|
+
});
|
|
108
|
+
|
|
109
|
+
await db.connect();
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
### Using Qdrant
|
|
113
|
+
|
|
114
|
+
```typescript
|
|
115
|
+
import { VectorDB } from '@neural-tools/vector-db';
|
|
116
|
+
|
|
117
|
+
const db = new VectorDB({
|
|
118
|
+
provider: 'qdrant',
|
|
119
|
+
config: {
|
|
120
|
+
url: 'http://localhost:6333',
|
|
121
|
+
collectionName: 'my-collection',
|
|
122
|
+
apiKey: process.env.QDRANT_API_KEY // Optional
|
|
123
|
+
}
|
|
124
|
+
});
|
|
125
|
+
|
|
126
|
+
await db.connect();
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
## API Reference
|
|
130
|
+
|
|
131
|
+
### Constructor
|
|
132
|
+
|
|
133
|
+
```typescript
|
|
134
|
+
new VectorDB(options: VectorDBOptions)
|
|
135
|
+
|
|
136
|
+
interface VectorDBOptions {
|
|
137
|
+
provider: 'pinecone' | 'chroma' | 'qdrant' | 'local';
|
|
138
|
+
config: ProviderConfig;
|
|
139
|
+
}
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### Methods
|
|
143
|
+
|
|
144
|
+
#### `connect()`
|
|
145
|
+
|
|
146
|
+
Connect to the vector database.
|
|
147
|
+
|
|
148
|
+
```typescript
|
|
149
|
+
await db.connect();
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
#### `disconnect()`
|
|
153
|
+
|
|
154
|
+
Disconnect from the database.
|
|
155
|
+
|
|
156
|
+
```typescript
|
|
157
|
+
await db.disconnect();
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
#### `upsert(vectors)`
|
|
161
|
+
|
|
162
|
+
Insert or update vectors.
|
|
163
|
+
|
|
164
|
+
```typescript
|
|
165
|
+
await db.upsert([
|
|
166
|
+
{
|
|
167
|
+
id: string;
|
|
168
|
+
values: number[];
|
|
169
|
+
metadata?: Record<string, any>;
|
|
170
|
+
}
|
|
171
|
+
]);
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
#### `query(options)`
|
|
175
|
+
|
|
176
|
+
Search for similar vectors.
|
|
177
|
+
|
|
178
|
+
```typescript
|
|
179
|
+
const results = await db.query({
|
|
180
|
+
vector: number[]; // Query vector
|
|
181
|
+
topK: number; // Number of results
|
|
182
|
+
filter?: object; // Metadata filter
|
|
183
|
+
includeMetadata?: boolean;
|
|
184
|
+
includeValues?: boolean;
|
|
185
|
+
});
|
|
186
|
+
|
|
187
|
+
// Returns
|
|
188
|
+
interface QueryResult {
|
|
189
|
+
id: string;
|
|
190
|
+
score: number;
|
|
191
|
+
values?: number[];
|
|
192
|
+
metadata?: Record<string, any>;
|
|
193
|
+
}
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
#### `delete(ids)`
|
|
197
|
+
|
|
198
|
+
Delete vectors by ID.
|
|
199
|
+
|
|
200
|
+
```typescript
|
|
201
|
+
await db.delete(['id1', 'id2']);
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
#### `fetch(ids)`
|
|
205
|
+
|
|
206
|
+
Retrieve vectors by ID.
|
|
207
|
+
|
|
208
|
+
```typescript
|
|
209
|
+
const vectors = await db.fetch(['id1', 'id2']);
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
## Configuration
|
|
213
|
+
|
|
214
|
+
### Pinecone
|
|
215
|
+
|
|
216
|
+
```typescript
|
|
217
|
+
{
|
|
218
|
+
provider: 'pinecone',
|
|
219
|
+
config: {
|
|
220
|
+
apiKey: string;
|
|
221
|
+
environment: string;
|
|
222
|
+
indexName: string;
|
|
223
|
+
namespace?: string;
|
|
224
|
+
}
|
|
225
|
+
}
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
### Chroma
|
|
229
|
+
|
|
230
|
+
```typescript
|
|
231
|
+
{
|
|
232
|
+
provider: 'chroma',
|
|
233
|
+
config: {
|
|
234
|
+
url: string;
|
|
235
|
+
collectionName: string;
|
|
236
|
+
auth?: {
|
|
237
|
+
provider: string;
|
|
238
|
+
credentials: string;
|
|
239
|
+
};
|
|
240
|
+
}
|
|
241
|
+
}
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
### Qdrant
|
|
245
|
+
|
|
246
|
+
```typescript
|
|
247
|
+
{
|
|
248
|
+
provider: 'qdrant',
|
|
249
|
+
config: {
|
|
250
|
+
url: string;
|
|
251
|
+
collectionName: string;
|
|
252
|
+
apiKey?: string;
|
|
253
|
+
}
|
|
254
|
+
}
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
### Local (In-Memory)
|
|
258
|
+
|
|
259
|
+
```typescript
|
|
260
|
+
{
|
|
261
|
+
provider: 'local',
|
|
262
|
+
config: {
|
|
263
|
+
dimension: number; // Vector dimension
|
|
264
|
+
}
|
|
265
|
+
}
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
## Examples
|
|
269
|
+
|
|
270
|
+
### Semantic Search
|
|
271
|
+
|
|
272
|
+
```typescript
|
|
273
|
+
import { VectorDB } from '@neural-tools/vector-db';
|
|
274
|
+
import { embed } from './embeddings'; // Your embedding function
|
|
275
|
+
|
|
276
|
+
const db = new VectorDB({
|
|
277
|
+
provider: 'pinecone',
|
|
278
|
+
config: { /* ... */ }
|
|
279
|
+
});
|
|
280
|
+
|
|
281
|
+
await db.connect();
|
|
282
|
+
|
|
283
|
+
// Index documents
|
|
284
|
+
const documents = [
|
|
285
|
+
'Neural Tools is amazing',
|
|
286
|
+
'I love building with AI',
|
|
287
|
+
'Vector databases are powerful'
|
|
288
|
+
];
|
|
289
|
+
|
|
290
|
+
for (const [i, doc] of documents.entries()) {
|
|
291
|
+
const embedding = await embed(doc);
|
|
292
|
+
await db.upsert([{
|
|
293
|
+
id: `doc-${i}`,
|
|
294
|
+
values: embedding,
|
|
295
|
+
metadata: { text: doc }
|
|
296
|
+
}]);
|
|
297
|
+
}
|
|
298
|
+
|
|
299
|
+
// Search
|
|
300
|
+
const queryEmbedding = await embed('AI development tools');
|
|
301
|
+
const results = await db.query({
|
|
302
|
+
vector: queryEmbedding,
|
|
303
|
+
topK: 2,
|
|
304
|
+
includeMetadata: true
|
|
305
|
+
});
|
|
306
|
+
|
|
307
|
+
console.log(results);
|
|
308
|
+
// [
|
|
309
|
+
// { id: 'doc-0', score: 0.95, metadata: { text: 'Neural Tools is amazing' } },
|
|
310
|
+
// { id: 'doc-1', score: 0.87, metadata: { text: 'I love building with AI' } }
|
|
311
|
+
// ]
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
### Filtered Search
|
|
315
|
+
|
|
316
|
+
```typescript
|
|
317
|
+
await db.query({
|
|
318
|
+
vector: queryVector,
|
|
319
|
+
topK: 10,
|
|
320
|
+
filter: {
|
|
321
|
+
category: 'documentation',
|
|
322
|
+
published: true,
|
|
323
|
+
date: { $gte: '2024-01-01' }
|
|
324
|
+
}
|
|
325
|
+
});
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
### Batch Operations
|
|
329
|
+
|
|
330
|
+
```typescript
|
|
331
|
+
// Batch insert
|
|
332
|
+
await db.upsert(
|
|
333
|
+
Array.from({ length: 1000 }, (_, i) => ({
|
|
334
|
+
id: `vector-${i}`,
|
|
335
|
+
values: generateVector(),
|
|
336
|
+
metadata: { index: i }
|
|
337
|
+
}))
|
|
338
|
+
);
|
|
339
|
+
|
|
340
|
+
// Batch delete
|
|
341
|
+
await db.delete(
|
|
342
|
+
Array.from({ length: 100 }, (_, i) => `vector-${i}`)
|
|
343
|
+
);
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
## Environment Variables
|
|
347
|
+
|
|
348
|
+
```bash
|
|
349
|
+
# Pinecone
|
|
350
|
+
PINECONE_API_KEY=your-api-key
|
|
351
|
+
PINECONE_ENVIRONMENT=us-west1-gcp
|
|
352
|
+
PINECONE_INDEX=my-index
|
|
353
|
+
|
|
354
|
+
# Chroma
|
|
355
|
+
CHROMA_URL=http://localhost:8000
|
|
356
|
+
CHROMA_COLLECTION=my-collection
|
|
357
|
+
|
|
358
|
+
# Qdrant
|
|
359
|
+
QDRANT_URL=http://localhost:6333
|
|
360
|
+
QDRANT_API_KEY=your-api-key
|
|
361
|
+
QDRANT_COLLECTION=my-collection
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
## Testing
|
|
365
|
+
|
|
366
|
+
The local provider is perfect for testing:
|
|
367
|
+
|
|
368
|
+
```typescript
|
|
369
|
+
import { VectorDB } from '@neural-tools/vector-db';
|
|
370
|
+
|
|
371
|
+
describe('Vector operations', () => {
|
|
372
|
+
let db: VectorDB;
|
|
373
|
+
|
|
374
|
+
beforeEach(async () => {
|
|
375
|
+
db = new VectorDB({
|
|
376
|
+
provider: 'local',
|
|
377
|
+
config: { dimension: 1536 }
|
|
378
|
+
});
|
|
379
|
+
await db.connect();
|
|
380
|
+
});
|
|
381
|
+
|
|
382
|
+
it('should insert and query', async () => {
|
|
383
|
+
await db.upsert([{
|
|
384
|
+
id: '1',
|
|
385
|
+
values: new Array(1536).fill(0.1),
|
|
386
|
+
metadata: { test: true }
|
|
387
|
+
}]);
|
|
388
|
+
|
|
389
|
+
const results = await db.query({
|
|
390
|
+
vector: new Array(1536).fill(0.1),
|
|
391
|
+
topK: 1
|
|
392
|
+
});
|
|
393
|
+
|
|
394
|
+
expect(results[0].id).toBe('1');
|
|
395
|
+
});
|
|
396
|
+
});
|
|
397
|
+
```
|
|
398
|
+
|
|
399
|
+
## Dependencies
|
|
400
|
+
|
|
401
|
+
- [@neural-tools/core](../core) - Core utilities
|
|
402
|
+
|
|
403
|
+
### Peer Dependencies (Optional)
|
|
404
|
+
|
|
405
|
+
- `@pinecone-database/pinecone` - For Pinecone support
|
|
406
|
+
- `chromadb` - For Chroma support
|
|
407
|
+
- `@qdrant/js-client-rest` - For Qdrant support
|
|
408
|
+
|
|
409
|
+
## Contributing
|
|
410
|
+
|
|
411
|
+
Contributions are welcome! See the [main repository](https://github.com/MacLeanLuke/neural-tools) for guidelines.
|
|
412
|
+
|
|
413
|
+
## License
|
|
414
|
+
|
|
415
|
+
MIT - See [LICENSE.md](../../LICENSE.md) for details.
|
|
416
|
+
|
|
417
|
+
## Links
|
|
418
|
+
|
|
419
|
+
- [Documentation](https://neural-tools.com/docs/vector-db.html)
|
|
420
|
+
- [GitHub](https://github.com/MacLeanLuke/neural-tools)
|
|
421
|
+
- [npm](https://www.npmjs.com/package/@neural-tools/vector-db)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@neural-tools/vector-db",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.7",
|
|
4
4
|
"description": "Vector database abstraction layer for Neural Tools",
|
|
5
5
|
"main": "dist/index.js",
|
|
6
6
|
"types": "dist/index.d.ts",
|
|
@@ -27,7 +27,7 @@
|
|
|
27
27
|
"ai"
|
|
28
28
|
],
|
|
29
29
|
"dependencies": {
|
|
30
|
-
"@neural-tools/core": "0.1.
|
|
30
|
+
"@neural-tools/core": "0.1.7"
|
|
31
31
|
},
|
|
32
32
|
"devDependencies": {
|
|
33
33
|
"@types/node": "^20.11.5",
|