voctar 0.1.0

package/docs/CUSTOM_PROVIDERS.md

@@ -0,0 +1,101 @@
+# Custom Providers
+
+Voctar supports custom providers for embeddings and storage.
+
+## Use Custom Providers
+
+```typescript
+import { Voctar } from 'voctar';
+
+const vector = new Voctar({
+  embedding: {
+    type: 'custom',
+    provider: myEmbeddingProvider,
+  },
+  store: {
+    type: 'custom',
+    provider: myVectorStoreProvider,
+  },
+});
+```
+
+## Custom Embedding Provider
+
+Implement the `EmbeddingProvider` interface:
+
+```typescript
+import type { EmbeddingProvider } from 'voctar';
+
+export class MyEmbeddingProvider implements EmbeddingProvider {
+  async embed(text: string): Promise<number[]> {
+    // Return one embedding vector for one text
+    return [/* ... */];
+  }
+
+  async embedBatch(texts: string[]): Promise<number[][]> {
+    // Return one vector per input text (same order)
+    return texts.map(() => [/* ... */]);
+  }
+
+  getDimension(): number {
+    return 1536;
+  }
+
+  getModelName(): string {
+    return 'my-embedding-model';
+  }
+
+  getTokenLimit(): number {
+    return 8192;
+  }
+}
+```
+
+## Custom Store Provider
+
+Implement the `VectorStoreProvider` interface:
+
+```typescript
+import type {
+  VectorStoreProvider,
+  VectorPoint,
+  SearchOptions,
+  SearchResult,
+  CollectionConfig,
+} from 'voctar';
+
+export class MyVectorStoreProvider implements VectorStoreProvider {
+  async ensureCollection(name: string, dimension: number, config?: CollectionConfig): Promise<void> {
+    // Create collection/index if missing
+  }
+
+  async upsert(collection: string, points: VectorPoint[]): Promise<void> {
+    // Insert or update vectors
+  }
+
+  async search(collection: string, vector: number[], options: SearchOptions): Promise<SearchResult[]> {
+    // Return scored results in descending relevance
+    return [];
+  }
+
+  async delete(collection: string, ids: string[]): Promise<void> {
+    // Delete matching IDs
+  }
+
+  async deleteCollection(collection: string): Promise<void> {
+    // Drop collection/index
+  }
+
+  async getIdsByFilter(collection: string, filter: Record<string, any>, limit?: number): Promise<string[]> {
+    // Return IDs that match filter
+    return [];
+  }
+}
+```
+
+## Integration Tips
+
+- Keep `embedBatch()` order stable with input order.
+- Ensure `getDimension()` matches vectors returned by `embed()`/`embedBatch()`.
+- Normalize errors with useful messages so callers can debug quickly.
+- Implement filter behavior consistently in `search()` and `getIdsByFilter()`.

package/docs/README.md

@@ -0,0 +1,11 @@
+# Documentation Index
+
+The canonical getting-started guide now lives in the root [`README.md`](../README.md).
+
+Use this folder for focused topics:
+
+- [API Reference](./API.md)
+- [Custom Providers](./CUSTOM_PROVIDERS.md)
+- [Storage Backends](./STORAGE_BACKENDS.md)
+- [Chunking](./CHUNKING.md)
+

package/docs/STORAGE_BACKENDS.md

@@ -0,0 +1,189 @@
+# Voctar Storage Backends
+
+This guide covers the available storage backends in Voctar and when to use each one.
+
+Voctar is config-first:
+
+- your app chooses the backend,
+- your app reads env vars (if any),
+- your app passes explicit config to `new Vectar(...)`.
+
+## Available Backends
+
+Voctar supports:
+
+- `sqlite`
+- `qdrant`
+- `memory`
+- `custom`
+
+## Quick Selection Guide
+
+- Use `sqlite` for local dev and simple production workloads.
+- Use `qdrant` for larger datasets, higher throughput, or multi-instance deployments.
+- Use `memory` for tests and short-lived demos only.
+- Use `custom` when integrating an internal or third-party vector store.
+
+## SQLite Backend
+
+Best for:
+
+- local development,
+- prototypes,
+- single-node deployments.
+
+Pros:
+
+- no external service,
+- persistent file-based storage,
+- simplest setup and operations.
+
+Trade-offs:
+
+- limited horizontal scaling,
+- shared file contention under high concurrency.
+
+Example:
+
+```typescript
+import { Voctar } from 'voctar';
+
+const vector = new Voctar({
+  embedding: {
+    type: 'openai',
+    apiKey: process.env.OPENAI_API_KEY!,
+  },
+  store: {
+    type: 'sqlite',
+    path: './data/vector.db',
+  },
+});
+```
+
+In-memory SQLite (testing only):
+
+```typescript
+store: {
+  type: 'sqlite',
+  path: ':memory:',
+  inMemory: true,
+}
+```
+
+## Qdrant Backend
+
+Best for:
+
+- medium and large datasets,
+- high query volume,
+- distributed deployments.
+
+Pros:
+
+- purpose-built vector DB,
+- strong scale characteristics,
+- rich filtering support.
+
+Trade-offs:
+
+- extra service to operate,
+- network hop adds operational complexity.
+
+Example:
+
+```typescript
+import { Voctar } from 'voctar';
+
+const vector = new Voctar({
+  embedding: {
+    type: 'openai',
+    apiKey: process.env.OPENAI_API_KEY!,
+  },
+  store: {
+    type: 'qdrant',
+    url: process.env.QDRANT_URL!,
+    port: process.env.QDRANT_PORT ? Number(process.env.QDRANT_PORT) : 6333,
+    apiKey: process.env.QDRANT_API_KEY || undefined,
+    timeout: 30000,
+    checkCompatibility: false,
+  },
+});
+```
+
+## In-Memory Backend
+
+Best for:
+
+- unit tests,
+- quick local examples.
+
+Trade-offs:
+
+- data is lost on restart,
+- unsuitable for production persistence.
+
+Example:
+
+```typescript
+import { Voctar } from 'voctar';
+
+const vector = new Voctar({
+  embedding: {
+    type: 'openai',
+    apiKey: process.env.OPENAI_API_KEY!,
+  },
+  store: {
+    type: 'memory',
+  },
+});
+```
+
+## Custom Backend
+
+Use this when you need full control over storage behavior.
+
+The provider must implement `VectorStoreProvider`.
+
+Example:
+
+```typescript
+import { Voctar } from 'voctar';
+
+const vector = new Voctar({
+  embedding: {
+    type: 'openai',
+    apiKey: process.env.OPENAI_API_KEY!,
+  },
+  store: {
+    type: 'custom',
+    provider: myVectorStoreProvider,
+  },
+});
+```
+
+See [`CUSTOM_PROVIDERS.md`](./CUSTOM_PROVIDERS.md) for full interface details.
+
+## Environment Variable Pattern (App-Owned)
+
+Voctar does not auto-load env vars, but many apps use a selector like this:
+
+```bash
+VECTOR_STORE=sqlite  # sqlite | qdrant | memory
+SQLITE_PATH=./data/vector.db
+QDRANT_URL=http://localhost
+QDRANT_PORT=6333
+QDRANT_API_KEY=your_api_key
+```
+
+Then in app bootstrap:
+
+```typescript
+const storeType = process.env.VECTOR_STORE ?? 'sqlite';
+```
+
+## Migration and Operations Notes
+
+- Start with `sqlite` if you are early-stage.
+- Move to `qdrant` when dataset size, traffic, or deployment topology requires it.
+- Back up SQLite database files regularly.
+- For Qdrant, use snapshots/backups supported by your Qdrant setup.

package/package.json

@@ -0,0 +1,46 @@
+{
+  "name": "voctar",
+  "version": "0.1.0",
+  "description": "TypeScript library with RAG primitives for vector embeddings, chunking, storing and retrieval.",
+  "main": "dist/index.js",
+  "types": "dist/index.d.ts",
+  "files": [
+    "dist",
+    "README.md",
+    "docs"
+  ],
+  "scripts": {
+    "build": "tsc -p tsconfig.json",
+    "clean": "rm -rf dist",
+    "prepublishOnly": "yarn clean && yarn build"
+  },
+  "keywords": [
+    "rag",
+    "vector",
+    "vector-database",
+    "embeddings",
+    "chunking",
+    "semantic-search",
+    "semantic-retrieval",
+    "typescript",
+    "qdrant",
+    "sqlite"
+  ],
+  "license": "MIT",
+  "engines": {
+    "node": ">=18"
+  },
+  "dependencies": {
+    "@qdrant/js-client-rest": "^1.13.0",
+    "better-sqlite3": "^12.4.1",
+    "openai": "^6.3.0",
+    "tiktoken": "^1.0.22",
+    "uuid": "^9.0.1"
+  },
+  "devDependencies": {
+    "@types/better-sqlite3": "^7.6.13",
+    "@types/node": "^24.0.0",
+    "@types/uuid": "^9.0.8",
+    "typescript": "^5.8.3"
+  }
+}