@memberjunction/ai-vectordb 2.32.2 → 2.33.0

package/README.md

@@ -0,0 +1,238 @@
+# @memberjunction/ai-vectordb
+
+The MemberJunction AI Vector Database package provides a standardized interface and base classes for working with vector databases in the MemberJunction ecosystem. This package serves as a common abstraction layer that allows different vector database implementations to be used interchangeably.
+
+## Features
+
+- **Abstract Base Class**: Common API for all vector database implementations
+- **Index Management**: Create, list, edit, and delete vector indexes
+- **Record Operations**: Comprehensive CRUD operations for vector records
+- **Query Capabilities**: Flexible vector similarity search with filtering
+- **Type Definitions**: Comprehensive TypeScript type definitions
+- **Provider Agnostic**: Works with any vector database that implements the interface
+- **Standardized Response Format**: Consistent response format across providers
+
+## Installation
+
+```bash
+npm install @memberjunction/ai-vectordb
+```
+
+## Core Components
+
+### VectorDBBase
+
+The abstract base class that all vector database providers must implement:
+
+```typescript
+export abstract class VectorDBBase {
+  constructor(apiKey: string);
+  
+  // Index operations
+  abstract listIndexes(): IndexList | Promise<IndexList>;
+  abstract getIndex(params: BaseRequestParams): BaseResponse | Promise<BaseResponse>;
+  abstract createIndex(params: CreateIndexParams): BaseResponse | Promise<BaseResponse>;
+  abstract deleteIndex(params: BaseRequestParams): BaseResponse | Promise<BaseResponse>;
+  abstract editIndex(params: EditIndexParams): BaseResponse | Promise<BaseResponse>;
+  abstract queryIndex(params: QueryOptions): BaseResponse | Promise<BaseResponse>;
+  
+  // Record operations
+  abstract createRecord(record: VectorRecord): BaseResponse | Promise<BaseResponse>;
+  abstract createRecords(record: VectorRecord[]): BaseResponse | Promise<BaseResponse>;
+  abstract getRecord(param: BaseRequestParams): BaseResponse | Promise<BaseResponse>;
+  abstract getRecords(params: BaseRequestParams): BaseResponse | Promise<BaseResponse>;
+  abstract updateRecord(record: UpdateOptions): BaseResponse | Promise<BaseResponse>;
+  abstract updateRecords(records: UpdateOptions): BaseResponse | Promise<BaseResponse>;
+  abstract deleteRecord(record: VectorRecord): BaseResponse | Promise<BaseResponse>;
+  abstract deleteRecords(records: VectorRecord[]): BaseResponse | Promise<BaseResponse>;
+}
+```
+
+### Key Types and Interfaces
+
+#### Vector Records
+
+```typescript
+export type VectorRecord<T extends RecordMetadata = RecordMetadata> = {
+  id: string;                 // Unique identifier for the record
+  values: number[];           // Vector embedding values (e.g., from an embedding model)
+  sparseValues?: {            // Optional sparse representation for hybrid search
+    indices: number[];
+    values: number[];
+  };
+  metadata?: T;               // Optional metadata for filtering and identification
+};
+```
+
+#### Index Description
+
+```typescript
+export type IndexDescription = {
+  name: string;               // Name of the index
+  dimension: number;          // Vector dimension size
+  metric: IndexModelMetricEnum; // Distance metric: 'cosine', 'euclidean', or 'dotproduct'
+  host: string;               // Host where the index is located
+};
+```
+
+#### Query Options
+
+For similarity search in vector databases:
+
+```typescript
+// Base query parameters
+export type QueryParamsBase = {
+  topK: number;              // Number of results to return
+  includeValues?: boolean;   // Whether to include vector values in results
+  includeMetadata?: boolean; // Whether to include metadata in results
+  filter?: object;           // Metadata filter to apply
+};
+
+// Query by vector values
+export type QueryByVectorValues = QueryParamsBase & {
+  vector: number[];          // The query vector to find similar vectors for
+};
+
+// Query by record ID
+export type QueryByRecordId = QueryParamsBase & {
+  id: string;                // Use an existing record's vector to query
+};
+
+// Combined query options type
+export type QueryOptions = QueryByRecordId | QueryByVectorValues;
+```
+
+#### Query Response
+
+```typescript
+export type QueryResponse<T extends RecordMetadata = RecordMetadata> = {
+  matches: Array<ScoredRecord<T>>; // Results sorted by similarity
+  namespace: string;               // Namespace where query was executed
+  usage?: OperationUsage;          // Usage information
+};
+
+export interface ScoredRecord<T extends RecordMetadata = RecordMetadata> extends VectorRecord<T> {
+  score?: number;                  // Similarity score
+}
+```
+
+## Usage Examples
+
+While this package primarily provides interfaces and base classes, here's how it would be used with a concrete implementation:
+
+### Implementing a Vector Database Provider
+
+Create a provider by implementing the `VectorDBBase` abstract class:
+
+```typescript
+import { VectorDBBase, VectorRecord, BaseResponse, CreateIndexParams } from '@memberjunction/ai-vectordb';
+
+class MyVectorDBProvider extends VectorDBBase {
+  constructor(apiKey: string) {
+    super(apiKey);
+  }
+  
+  async listIndexes() {
+    // Implementation for listing indexes
+    return {
+      indexes: [
+        // Index descriptions
+      ]
+    };
+  }
+  
+  async createIndex(params: CreateIndexParams) {
+    // Implementation for creating an index
+    try {
+      // Provider-specific code
+      return {
+        success: true,
+        message: 'Index created successfully',
+        data: { /* index info */ }
+      };
+    } catch (error) {
+      return {
+        success: false,
+        message: error.message,
+        data: null
+      };
+    }
+  }
+  
+  // Implement remaining methods...
+}
+```
+
+### Using a Vector Database Provider
+
+Once a provider is implemented, you can use it with a consistent API:
+
+```typescript
+import { VectorDBBase, VectorRecord } from '@memberjunction/ai-vectordb';
+import { MyVectorDBProvider } from './my-vector-db-provider';
+
+async function workWithVectors() {
+  // Initialize the provider
+  const vectorDB: VectorDBBase = new MyVectorDBProvider('your-api-key');
+  
+  // Create an index
+  const createResult = await vectorDB.createIndex({
+    id: 'my-index',
+    dimension: 1536,
+    metric: 'cosine'
+  });
+  
+  if (createResult.success) {
+    console.log('Index created:', createResult.data);
+  }
+  
+  // Insert a vector
+  const insertResult = await vectorDB.createRecord({
+    id: 'record-1',
+    values: [0.1, 0.2, 0.3, /* ... */],
+    metadata: {
+      category: 'document',
+      title: 'Sample Document'
+    }
+  });
+  
+  // Query for similar vectors
+  const queryResult = await vectorDB.queryIndex({
+    vector: [0.1, 0.2, 0.3, /* ... */],
+    topK: 5,
+    includeMetadata: true,
+    filter: {
+      category: 'document'
+    }
+  });
+  
+  if (queryResult.success) {
+    console.log('Similar vectors:', queryResult.data.matches);
+  }
+}
+```
+
+## Provider Implementations
+
+MemberJunction provides implementations for popular vector databases:
+
+- `@memberjunction/ai-vectors-pinecone` - Implementation for Pinecone vector database
+
+You can also create your own implementation for any vector database service by extending the `VectorDBBase` class.
+
+## Integration with MemberJunction Ecosystem
+
+This package integrates with the broader MemberJunction AI vector ecosystem:
+
+- `@memberjunction/ai-vectors` - Core vector functionality
+- `@memberjunction/ai-vectors-sync` - Synchronize entity data to vector databases
+- `@memberjunction/ai-vectors-dupe` - Duplicate detection using vector similarity
+
+## Dependencies
+
+- `@memberjunction/core`: MemberJunction core library
+- `@memberjunction/global`: MemberJunction global utilities
+
+## License
+
+ISC

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@memberjunction/ai-vectordb",
-  "version": "2.32.2",
+  "version": "2.33.0",
   "description": "MemberJunction: AI Vector Database Module",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -15,8 +15,8 @@
   "author": "MemberJunction.com",
   "license": "ISC",
   "dependencies": {
-    "@memberjunction/core": "2.32.2",
-    "@memberjunction/global": "2.32.2",
+    "@memberjunction/core": "2.33.0",
+    "@memberjunction/global": "2.33.0",
     "dotenv": "^16.4.1"
   },
   "devDependencies": {