codebaxing 0.2.44 → 0.2.48

package/README.md

@@ -88,19 +88,40 @@ After installing, AI agents can use these tools:
 
 ## Configuration
 
+### Cloud Embedding (Fastest)
+
+Use OpenAI or Voyage for ~25x faster indexing:
+
+```bash
+# OpenAI (text-embedding-3-small, 384 dims)
+CODEBAXING_EMBEDDING_PROVIDER=openai OPENAI_API_KEY=sk-... npx codebaxing@latest index /path
+
+# Voyage (voyage-code-3, 1024 dims, code-optimized)
+CODEBAXING_EMBEDDING_PROVIDER=voyage VOYAGE_API_KEY=va-... npx codebaxing@latest index /path
+```
+
+> **Note:** Switching between local and cloud providers requires full re-index due to dimension differences.
+
 ### Environment Variables
 
 | Variable | Description | Default |
 |----------|-------------|---------|
 | `CHROMADB_URL` | ChromaDB server URL | `http://localhost:8000` |
-| `CODEBAXING_DEVICE` | Compute: `cpu`, `cuda` | `cpu` |
-| `CODEBAXING_WORKERS` | Worker threads for parallel embedding (0=off, 1-8) | `2` |
+| `CODEBAXING_EMBEDDING_PROVIDER` | Embedding backend: `local`, `openai`, `voyage` | `local` |
+| `CODEBAXING_DEVICE` | Compute device (local only): `cpu`, `cuda` | `cpu` |
+| `CODEBAXING_DTYPE` | Model quantization (local only): `fp32`, `fp16`, `q8`, `q4` | `q8` |
+| `CODEBAXING_WORKERS` | Worker threads for parallel embedding (local only, 0=off) | `2` |
 | `CODEBAXING_MAX_FILE_SIZE` | Max file size in MB | `1` |
-| `CODEBAXING_MAX_CHUNKS` | Max chunks to index | `100000` |
+| `CODEBAXING_MAX_CHUNKS` | Max chunks to index | `500000` |
 | `CODEBAXING_FILES_PER_BATCH` | Files per batch (lower = less RAM) | `100` |
 | `CODEBAXING_PARALLEL_BATCHES` | Concurrent batches | `3` |
 | `CODEBAXING_METADATA_SAVE_INTERVAL` | Save progress every N batches | `10` |
-| `CODEBAXING_MODEL_CACHE` | Directory for embedding model cache | `~/.cache/codebaxing/models` |
+| `CODEBAXING_MODEL_CACHE` | Model cache directory (local only) | `~/.cache/codebaxing/models` |
+| `CODEBAXING_OPENAI_API_KEY` | OpenAI API key (or use `OPENAI_API_KEY`) | - |
+| `CODEBAXING_VOYAGE_API_KEY` | Voyage API key (or use `VOYAGE_API_KEY`) | - |
+| `CODEBAXING_EMBEDDING_MODEL` | Override embedding model name | per-provider default |
+| `CODEBAXING_EMBEDDING_DIMENSIONS` | Override embedding dimensions | per-provider default |
+| `CODEBAXING_EMBEDDING_BASE_URL` | Custom API endpoint for cloud providers | provider default |
 
 ### Manual Editor Config
 
@@ -163,13 +184,16 @@ Python, JavaScript, TypeScript, Go, Rust, Java, C/C++, C#, Ruby, PHP, Kotlin, Sw
 
 | Component | Technology |
 |-----------|------------|
-| Embedding Model | `all-MiniLM-L6-v2` (384 dimensions, ONNX) |
-| Model Cache | `~/.cache/codebaxing/models/` (~90MB, downloaded once) |
+| Local Embedding | `all-MiniLM-L6-v2` (384 dims, ONNX, q8 quantized) |
+| Cloud Embedding | OpenAI `text-embedding-3-small` or Voyage `voyage-code-3` |
+| Model Cache | `~/.cache/codebaxing/models/` (local only, downloaded once) |
 | Vector Database | ChromaDB |
-| Code Parser | Tree-sitter |
+| Code Parser | Tree-sitter (28 languages) |
 | MCP SDK | `@modelcontextprotocol/sdk` |
 
-The embedding model is downloaded from HuggingFace on first run and cached locally at `~/.cache/codebaxing/models/`. Subsequent runs reuse the cached model without network access. To use a custom cache location, set `CODEBAXING_MODEL_CACHE`.
+**Local mode**: The embedding model is downloaded from HuggingFace on first run and cached at `~/.cache/codebaxing/models/`. Uses q8 quantization (~3x faster than fp32). No network access after initial download.
+
+**Cloud mode**: Sends code chunks to OpenAI/Voyage API for embedding. ~25x faster than local CPU. Requires API key.
 
 ## License
 

package/README.vi.md

@@ -88,18 +88,38 @@ Sau khi cài, AI agents có thể dùng các tools:
 
 ## Cấu Hình
 
+### Cloud Embedding (Nhanh nhất)
+
+Dùng OpenAI hoặc Voyage để index nhanh ~25x:
+
+```bash
+# OpenAI (text-embedding-3-small, 384 dims)
+CODEBAXING_EMBEDDING_PROVIDER=openai OPENAI_API_KEY=sk-... npx codebaxing@latest index /path
+
+# Voyage (voyage-code-3, 1024 dims, tối ưu cho code)
+CODEBAXING_EMBEDDING_PROVIDER=voyage VOYAGE_API_KEY=va-... npx codebaxing@latest index /path
+```
+
+> **Lưu ý:** Chuyển đổi giữa local và cloud cần re-index do khác dimension.
+
 ### Biến Môi Trường
 
 | Biến | Mô tả | Mặc định |
 |------|-------|----------|
 | `CHROMADB_URL` | URL ChromaDB server | `http://localhost:8000` |
-| `CODEBAXING_DEVICE` | Compute: `cpu`, `cuda` | `cpu` |
-| `CODEBAXING_WORKERS` | Worker threads cho embedding song song (0=tắt, 1-8) | `2` |
+| `CODEBAXING_EMBEDDING_PROVIDER` | Backend embedding: `local`, `openai`, `voyage` | `local` |
+| `CODEBAXING_DEVICE` | Compute device (chỉ local): `cpu`, `cuda` | `cpu` |
+| `CODEBAXING_DTYPE` | Quantization (chỉ local): `fp32`, `fp16`, `q8`, `q4` | `q8` |
+| `CODEBAXING_WORKERS` | Worker threads cho embedding (chỉ local, 0=tắt) | `2` |
 | `CODEBAXING_MAX_FILE_SIZE` | Kích thước file tối đa (MB) | `1` |
-| `CODEBAXING_MAX_CHUNKS` | Số chunks tối đa | `100000` |
+| `CODEBAXING_MAX_CHUNKS` | Số chunks tối đa | `500000` |
 | `CODEBAXING_FILES_PER_BATCH` | Files mỗi batch (thấp = ít RAM) | `100` |
 | `CODEBAXING_PARALLEL_BATCHES` | Số batches chạy song song | `3` |
 | `CODEBAXING_METADATA_SAVE_INTERVAL` | Lưu tiến trình mỗi N batches | `10` |
+| `CODEBAXING_OPENAI_API_KEY` | OpenAI API key (hoặc dùng `OPENAI_API_KEY`) | - |
+| `CODEBAXING_VOYAGE_API_KEY` | Voyage API key (hoặc dùng `VOYAGE_API_KEY`) | - |
+| `CODEBAXING_EMBEDDING_MODEL` | Override tên model embedding | mặc định theo provider |
+| `CODEBAXING_EMBEDDING_DIMENSIONS` | Override số dimensions | mặc định theo provider |
 
 ### Cấu Hình Editor Thủ Công
 
@@ -162,11 +182,16 @@ Python, JavaScript, TypeScript, Go, Rust, Java, C/C++, C#, Ruby, PHP, Kotlin, Sw
 
 | Component | Công nghệ |
 |-----------|-----------|
-| Embedding Model | `all-MiniLM-L6-v2` (384 dimensions) |
+| Local Embedding | `all-MiniLM-L6-v2` (384 dims, ONNX, q8 quantized) |
+| Cloud Embedding | OpenAI `text-embedding-3-small` hoặc Voyage `voyage-code-3` |
 | Vector Database | ChromaDB |
-| Code Parser | Tree-sitter |
+| Code Parser | Tree-sitter (28 ngôn ngữ) |
 | MCP SDK | `@modelcontextprotocol/sdk` |
 
+**Local mode**: Model tải từ HuggingFace lần đầu, cache tại `~/.cache/codebaxing/models/`. Dùng q8 quantization (~3x nhanh hơn fp32). Không cần mạng sau lần đầu.
+
+**Cloud mode**: Gửi code chunks đến OpenAI/Voyage API. ~25x nhanh hơn local CPU. Cần API key.
+
 ## License
 
 MIT

package/dist/core/interfaces.d.ts

@@ -10,4 +10,12 @@ export interface IParser {
     /** Return list of file extensions this parser supports. */
     getSupportedExtensions(): string[];
 }
+export interface IEmbeddingService {
+    readonly dimensions: number;
+    embed(text: string, isQuery?: boolean): Promise<number[]>;
+    embedBatch(texts: string[], isQuery?: boolean): Promise<number[][]>;
+    getStats(): Record<string, unknown>;
+    clearCache(): void;
+    unload(): Promise<void>;
+}
 //# sourceMappingURL=interfaces.d.ts.map

package/dist/core/interfaces.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"interfaces.d.ts","sourceRoot":"","sources":["../../src/core/interfaces.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAE9C,MAAM,WAAW,OAAO;IACtB,0DAA0D;IAC1D,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC;IAEpC,mDAAmD;IACnD,SAAS,CAAC,QAAQ,EAAE,MAAM,GAAG,UAAU,CAAC;IAExC,2DAA2D;IAC3D,sBAAsB,IAAI,MAAM,EAAE,CAAC;CACpC"}
+{"version":3,"file":"interfaces.d.ts","sourceRoot":"","sources":["../../src/core/interfaces.ts"],"names":[],"mappings":"AAAA;;GAEG;AAEH,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAE9C,MAAM,WAAW,OAAO;IACtB,0DAA0D;IAC1D,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,OAAO,CAAC;IAEpC,mDAAmD;IACnD,SAAS,CAAC,QAAQ,EAAE,MAAM,GAAG,UAAU,CAAC;IAExC,2DAA2D;IAC3D,sBAAsB,IAAI,MAAM,EAAE,CAAC;CACpC;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B,KAAK,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,OAAO,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;IAC1D,UAAU,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,OAAO,CAAC,EAAE,OAAO,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IACpE,QAAQ,IAAI,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACpC,UAAU,IAAI,IAAI,CAAC;IACnB,MAAM,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CACzB"}

package/dist/index.d.ts

@@ -7,7 +7,7 @@
 export declare const VERSION = "0.1.0";
 export { Symbol, SymbolType, ParsedFile, CodebaseIndex } from './core/models.js';
 export { Memory, MemoryType } from './core/models.js';
-export type { IParser } from './core/interfaces.js';
+export type { IParser, IEmbeddingService } from './core/interfaces.js';
 export { CodebaxingError, ParseError, IndexingError, EmbeddingError, SearchError, ConfigurationError, } from './core/exceptions.js';
 export { TreeSitterParser } from './parsers/treesitter-parser.js';
 export { getLanguageForFile, getSupportedExtensions, EXTENSION_MAP } from './parsers/language-configs.js';
@@ -15,4 +15,6 @@ export { SourceRetriever, discoverFiles, loadIgnoreConfig, ensureIgnoreConfig }
 export type { IgnoreConfig } from './indexing/source-retriever.js';
 export { MemoryRetriever } from './indexing/memory-retriever.js';
 export { EmbeddingService, getEmbeddingService } from './indexing/embedding-service.js';
+export { CloudEmbeddingService } from './indexing/cloud-embedding-service.js';
+export { createEmbeddingService, getConfiguredProvider } from './indexing/embedding-factory.js';
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,eAAO,MAAM,OAAO,UAAU,CAAC;AAG/B,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,kBAAkB,CAAC;AACjF,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AACtD,YAAY,EAAE,OAAO,EAAE,MAAM,sBAAsB,CAAC;AAGpD,OAAO,EACL,eAAe,EACf,UAAU,EACV,aAAa,EACb,cAAc,EACd,WAAW,EACX,kBAAkB,GACnB,MAAM,sBAAsB,CAAC;AAG9B,OAAO,EAAE,gBAAgB,EAAE,MAAM,gCAAgC,CAAC;AAClE,OAAO,EAAE,kBAAkB,EAAE,sBAAsB,EAAE,aAAa,EAAE,MAAM,+BAA+B,CAAC;AAG1G,OAAO,EAAE,eAAe,EAAE,aAAa,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,gCAAgC,CAAC;AACtH,YAAY,EAAE,YAAY,EAAE,MAAM,gCAAgC,CAAC;AACnE,OAAO,EAAE,eAAe,EAAE,MAAM,gCAAgC,CAAC;AACjE,OAAO,EAAE,gBAAgB,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,eAAO,MAAM,OAAO,UAAU,CAAC;AAG/B,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,kBAAkB,CAAC;AACjF,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AACtD,YAAY,EAAE,OAAO,EAAE,iBAAiB,EAAE,MAAM,sBAAsB,CAAC;AAGvE,OAAO,EACL,eAAe,EACf,UAAU,EACV,aAAa,EACb,cAAc,EACd,WAAW,EACX,kBAAkB,GACnB,MAAM,sBAAsB,CAAC;AAG9B,OAAO,EAAE,gBAAgB,EAAE,MAAM,gCAAgC,CAAC;AAClE,OAAO,EAAE,kBAAkB,EAAE,sBAAsB,EAAE,aAAa,EAAE,MAAM,+BAA+B,CAAC;AAG1G,OAAO,EAAE,eAAe,EAAE,aAAa,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,gCAAgC,CAAC;AACtH,YAAY,EAAE,YAAY,EAAE,MAAM,gCAAgC,CAAC;AACnE,OAAO,EAAE,eAAe,EAAE,MAAM,gCAAgC,CAAC;AACjE,OAAO,EAAE,gBAAgB,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC;AACxF,OAAO,EAAE,qBAAqB,EAAE,MAAM,uCAAuC,CAAC;AAC9E,OAAO,EAAE,sBAAsB,EAAE,qBAAqB,EAAE,MAAM,iCAAiC,CAAC"}

package/dist/index.js

@@ -17,4 +17,6 @@ export { getLanguageForFile, getSupportedExtensions, EXTENSION_MAP } from './par
 export { SourceRetriever, discoverFiles, loadIgnoreConfig, ensureIgnoreConfig } from './indexing/source-retriever.js';
 export { MemoryRetriever } from './indexing/memory-retriever.js';
 export { EmbeddingService, getEmbeddingService } from './indexing/embedding-service.js';
+export { CloudEmbeddingService } from './indexing/cloud-embedding-service.js';
+export { createEmbeddingService, getConfiguredProvider } from './indexing/embedding-factory.js';
 //# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,MAAM,CAAC,MAAM,OAAO,GAAG,OAAO,CAAC;AAE/B,cAAc;AACd,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,kBAAkB,CAAC;AACjF,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAGtD,aAAa;AACb,OAAO,EACL,eAAe,EACf,UAAU,EACV,aAAa,EACb,cAAc,EACd,WAAW,EACX,kBAAkB,GACnB,MAAM,sBAAsB,CAAC;AAE9B,UAAU;AACV,OAAO,EAAE,gBAAgB,EAAE,MAAM,gCAAgC,CAAC;AAClE,OAAO,EAAE,kBAAkB,EAAE,sBAAsB,EAAE,aAAa,EAAE,MAAM,+BAA+B,CAAC;AAE1G,WAAW;AACX,OAAO,EAAE,eAAe,EAAE,aAAa,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,gCAAgC,CAAC;AAEtH,OAAO,EAAE,eAAe,EAAE,MAAM,gCAAgC,CAAC;AACjE,OAAO,EAAE,gBAAgB,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,MAAM,CAAC,MAAM,OAAO,GAAG,OAAO,CAAC;AAE/B,cAAc;AACd,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,kBAAkB,CAAC;AACjF,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAGtD,aAAa;AACb,OAAO,EACL,eAAe,EACf,UAAU,EACV,aAAa,EACb,cAAc,EACd,WAAW,EACX,kBAAkB,GACnB,MAAM,sBAAsB,CAAC;AAE9B,UAAU;AACV,OAAO,EAAE,gBAAgB,EAAE,MAAM,gCAAgC,CAAC;AAClE,OAAO,EAAE,kBAAkB,EAAE,sBAAsB,EAAE,aAAa,EAAE,MAAM,+BAA+B,CAAC;AAE1G,WAAW;AACX,OAAO,EAAE,eAAe,EAAE,aAAa,EAAE,gBAAgB,EAAE,kBAAkB,EAAE,MAAM,gCAAgC,CAAC;AAEtH,OAAO,EAAE,eAAe,EAAE,MAAM,gCAAgC,CAAC;AACjE,OAAO,EAAE,gBAAgB,EAAE,mBAAmB,EAAE,MAAM,iCAAiC,CAAC;AACxF,OAAO,EAAE,qBAAqB,EAAE,MAAM,uCAAuC,CAAC;AAC9E,OAAO,EAAE,sBAAsB,EAAE,qBAAqB,EAAE,MAAM,iCAAiC,CAAC"}

package/dist/indexing/cloud-embedding-service.d.ts

@@ -0,0 +1,38 @@
+/**
+ * Cloud embedding service using OpenAI or Voyage APIs.
+ *
+ * Sends texts to a cloud embedding API instead of running ONNX locally.
+ * Significantly faster for large codebases (~10,000+ texts/sec vs ~400 on CPU).
+ */
+import type { IEmbeddingService } from '../core/interfaces.js';
+export type CloudProvider = 'openai' | 'voyage';
+export interface CloudEmbeddingConfig {
+    provider: CloudProvider;
+    apiKey: string;
+    model?: string;
+    dimensions?: number;
+    baseUrl?: string;
+    batchSize?: number;
+    concurrency?: number;
+}
+export declare class CloudEmbeddingService implements IEmbeddingService {
+    private provider;
+    private apiKey;
+    private model;
+    private _dimensions;
+    private baseUrl;
+    private batchSize;
+    private concurrency;
+    private cache;
+    private cacheMaxSize;
+    private stats;
+    constructor(config: CloudEmbeddingConfig);
+    get dimensions(): number;
+    embed(text: string, isQuery?: boolean): Promise<number[]>;
+    embedBatch(texts: string[], _isQuery?: boolean): Promise<number[][]>;
+    private callApi;
+    getStats(): Record<string, unknown>;
+    clearCache(): void;
+    unload(): Promise<void>;
+}
+//# sourceMappingURL=cloud-embedding-service.d.ts.map

package/dist/indexing/cloud-embedding-service.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cloud-embedding-service.d.ts","sourceRoot":"","sources":["../../src/indexing/cloud-embedding-service.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,uBAAuB,CAAC;AAK/D,MAAM,MAAM,aAAa,GAAG,QAAQ,GAAG,QAAQ,CAAC;AAEhD,MAAM,WAAW,oBAAoB;IACnC,QAAQ,EAAE,aAAa,CAAC;IACxB,MAAM,EAAE,MAAM,CAAC;IACf,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,WAAW,CAAC,EAAE,MAAM,CAAC;CACtB;AA0BD,qBAAa,qBAAsB,YAAW,iBAAiB;IAC7D,OAAO,CAAC,QAAQ,CAAgB;IAChC,OAAO,CAAC,MAAM,CAAS;IACvB,OAAO,CAAC,KAAK,CAAS;IACtB,OAAO,CAAC,WAAW,CAAS;IAC5B,OAAO,CAAC,OAAO,CAAS;IACxB,OAAO,CAAC,SAAS,CAAS;IAC1B,OAAO,CAAC,WAAW,CAAS;IAG5B,OAAO,CAAC,KAAK,CAAoC;IACjD,OAAO,CAAC,YAAY,CAAQ;IAG5B,OAAO,CAAC,KAAK,CAQX;gBAEU,MAAM,EAAE,oBAAoB;IAgBxC,IAAI,UAAU,IAAI,MAAM,CAEvB;IAEK,KAAK,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,GAAE,OAAe,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IAYhE,UAAU,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,QAAQ,GAAE,OAAe,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC;YA8CnE,OAAO;IAoErB,QAAQ,IAAI,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC;IAWnC,UAAU,IAAI,IAAI;IAMZ,MAAM,IAAI,OAAO,CAAC,IAAI,CAAC;CAG9B"}

package/dist/indexing/cloud-embedding-service.js

@@ -0,0 +1,181 @@
+/**
+ * Cloud embedding service using OpenAI or Voyage APIs.
+ *
+ * Sends texts to a cloud embedding API instead of running ONNX locally.
+ * Significantly faster for large codebases (~10,000+ texts/sec vs ~400 on CPU).
+ */
+import { EmbeddingError } from '../core/exceptions.js';
+const PROVIDER_DEFAULTS = {
+    openai: {
+        model: 'text-embedding-3-small',
+        dimensions: 384,
+        batchSize: 2048,
+        baseUrl: 'https://api.openai.com/v1/embeddings',
+    },
+    voyage: {
+        model: 'voyage-code-3',
+        dimensions: 1024,
+        batchSize: 128,
+        baseUrl: 'https://api.voyageai.com/v1/embeddings',
+    },
+};
+// ─── CloudEmbeddingService ───────────────────────────────────────────────────
+export class CloudEmbeddingService {
+    provider;
+    apiKey;
+    model;
+    _dimensions;
+    baseUrl;
+    batchSize;
+    concurrency;
+    // LRU cache
+    cache = new Map();
+    cacheMaxSize = 5000;
+    // Stats
+    stats = {
+        totalEmbeddings: 0,
+        totalBatches: 0,
+        totalTime: 0,
+        cacheHits: 0,
+        cacheMisses: 0,
+        apiCalls: 0,
+        totalTokens: 0,
+    };
+    constructor(config) {
+        this.provider = config.provider;
+        this.apiKey = config.apiKey;
+        const defaults = PROVIDER_DEFAULTS[config.provider];
+        this.model = config.model ?? defaults.model;
+        this._dimensions = config.dimensions ?? defaults.dimensions;
+        this.baseUrl = config.baseUrl ?? defaults.baseUrl;
+        this.batchSize = config.batchSize ?? defaults.batchSize;
+        this.concurrency = config.concurrency ?? 5;
+        console.error(`[codebaxing] Using cloud embeddings: ${this.provider} (model: ${this.model}, dims: ${this._dimensions})`);
+    }
+    get dimensions() {
+        return this._dimensions;
+    }
+    async embed(text, isQuery = false) {
+        const cacheKey = `${isQuery ? 'q:' : 'd:'}${text}`;
+        const cached = this.cache.get(cacheKey);
+        if (cached) {
+            this.stats.cacheHits++;
+            return cached;
+        }
+        const results = await this.embedBatch([text], isQuery);
+        return results[0];
+    }
+    async embedBatch(texts, _isQuery = false) {
+        if (texts.length === 0)
+            return [];
+        const startTime = performance.now();
+        // Split into API-sized batches
+        const batches = [];
+        for (let i = 0; i < texts.length; i += this.batchSize) {
+            batches.push(texts.slice(i, i + this.batchSize));
+        }
+        // Process with concurrency limit
+        const allEmbeddings = new Array(texts.length);
+        let offset = 0;
+        for (let i = 0; i < batches.length; i += this.concurrency) {
+            const concurrent = batches.slice(i, i + this.concurrency);
+            const promises = concurrent.map(batch => this.callApi(batch));
+            const results = await Promise.all(promises);
+            for (const embeddings of results) {
+                for (const embedding of embeddings) {
+                    allEmbeddings[offset++] = embedding;
+                }
+            }
+        }
+        // Update cache
+        for (let i = 0; i < texts.length; i++) {
+            const cacheKey = `d:${texts[i]}`;
+            this.stats.cacheMisses++;
+            if (this.cache.size >= this.cacheMaxSize) {
+                const firstKey = this.cache.keys().next().value;
+                this.cache.delete(firstKey);
+            }
+            this.cache.set(cacheKey, allEmbeddings[i]);
+        }
+        const elapsed = (performance.now() - startTime) / 1000;
+        this.stats.totalEmbeddings += texts.length;
+        this.stats.totalBatches++;
+        this.stats.totalTime += elapsed;
+        return allEmbeddings;
+    }
+    async callApi(texts, retries = 3) {
+        const body = {
+            model: this.model,
+            input: texts,
+        };
+        // OpenAI supports dimension reduction
+        if (this.provider === 'openai' && this._dimensions) {
+            body.dimensions = this._dimensions;
+        }
+        // Voyage uses input_type
+        if (this.provider === 'voyage') {
+            body.input_type = 'document';
+        }
+        for (let attempt = 0; attempt < retries; attempt++) {
+            try {
+                const response = await fetch(this.baseUrl, {
+                    method: 'POST',
+                    headers: {
+                        'Content-Type': 'application/json',
+                        'Authorization': `Bearer ${this.apiKey}`,
+                    },
+                    body: JSON.stringify(body),
+                });
+                if (response.status === 429 || response.status >= 500) {
+                    const retryAfter = response.headers.get('retry-after');
+                    const waitMs = retryAfter ? parseInt(retryAfter, 10) * 1000 : Math.pow(2, attempt) * 1000;
+                    console.error(`[codebaxing] API rate limited (${response.status}), retrying in ${waitMs}ms...`);
+                    await new Promise(r => setTimeout(r, waitMs));
+                    continue;
+                }
+                if (!response.ok) {
+                    const errorText = await response.text();
+                    throw new EmbeddingError(`${this.provider} API error (${response.status}): ${errorText.slice(0, 200)}`);
+                }
+                const json = await response.json();
+                this.stats.apiCalls++;
+                if (json.usage) {
+                    this.stats.totalTokens += json.usage.total_tokens ?? json.usage.prompt_tokens ?? 0;
+                }
+                // Sort by index to maintain order
+                const sorted = json.data.sort((a, b) => a.index - b.index);
+                return sorted.map(d => d.embedding);
+            }
+            catch (e) {
+                if (e instanceof EmbeddingError)
+                    throw e;
+                if (attempt === retries - 1) {
+                    throw new EmbeddingError(`${this.provider} API call failed: ${e.message}`);
+                }
+                const waitMs = Math.pow(2, attempt) * 1000;
+                console.error(`[codebaxing] API error, retrying in ${waitMs}ms: ${e.message}`);
+                await new Promise(r => setTimeout(r, waitMs));
+            }
+        }
+        throw new EmbeddingError(`${this.provider} API failed after ${retries} retries`);
+    }
+    getStats() {
+        return {
+            ...this.stats,
+            provider: this.provider,
+            model: this.model,
+            embeddingsPerSecond: this.stats.totalTime > 0
+                ? this.stats.totalEmbeddings / this.stats.totalTime
+                : 0,
+        };
+    }
+    clearCache() {
+        this.cache.clear();
+        this.stats.cacheHits = 0;
+        this.stats.cacheMisses = 0;
+    }
+    async unload() {
+        this.cache.clear();
+    }
+}
+//# sourceMappingURL=cloud-embedding-service.js.map

package/dist/indexing/cloud-embedding-service.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"cloud-embedding-service.js","sourceRoot":"","sources":["../../src/indexing/cloud-embedding-service.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAGH,OAAO,EAAE,cAAc,EAAE,MAAM,uBAAuB,CAAC;AAuBvD,MAAM,iBAAiB,GAA4C;IACjE,MAAM,EAAE;QACN,KAAK,EAAE,wBAAwB;QAC/B,UAAU,EAAE,GAAG;QACf,SAAS,EAAE,IAAI;QACf,OAAO,EAAE,sCAAsC;KAChD;IACD,MAAM,EAAE;QACN,KAAK,EAAE,eAAe;QACtB,UAAU,EAAE,IAAI;QAChB,SAAS,EAAE,GAAG;QACd,OAAO,EAAE,wCAAwC;KAClD;CACF,CAAC;AAEF,gFAAgF;AAEhF,MAAM,OAAO,qBAAqB;IACxB,QAAQ,CAAgB;IACxB,MAAM,CAAS;IACf,KAAK,CAAS;IACd,WAAW,CAAS;IACpB,OAAO,CAAS;IAChB,SAAS,CAAS;IAClB,WAAW,CAAS;IAE5B,YAAY;IACJ,KAAK,GAA0B,IAAI,GAAG,EAAE,CAAC;IACzC,YAAY,GAAG,IAAI,CAAC;IAE5B,QAAQ;IACA,KAAK,GAAG;QACd,eAAe,EAAE,CAAC;QAClB,YAAY,EAAE,CAAC;QACf,SAAS,EAAE,CAAC;QACZ,SAAS,EAAE,CAAC;QACZ,WAAW,EAAE,CAAC;QACd,QAAQ,EAAE,CAAC;QACX,WAAW,EAAE,CAAC;KACf,CAAC;IAEF,YAAY,MAA4B;QACtC,IAAI,CAAC,QAAQ,GAAG,MAAM,CAAC,QAAQ,CAAC;QAChC,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC;QAE5B,MAAM,QAAQ,GAAG,iBAAiB,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;QACpD,IAAI,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,IAAI,QAAQ,CAAC,KAAK,CAAC;QAC5C,IAAI,CAAC,WAAW,GAAG,MAAM,CAAC,UAAU,IAAI,QAAQ,CAAC,UAAU,CAAC;QAC5D,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,OAAO,IAAI,QAAQ,CAAC,OAAO,CAAC;QAClD,IAAI,CAAC,SAAS,GAAG,MAAM,CAAC,SAAS,IAAI,QAAQ,CAAC,SAAS,CAAC;QACxD,IAAI,CAAC,WAAW,GAAG,MAAM,CAAC,WAAW,IAAI,CAAC,CAAC;QAE3C,OAAO,CAAC,KAAK,CACX,wCAAwC,IAAI,CAAC,QAAQ,YAAY,IAAI,CAAC,KAAK,WAAW,IAAI,CAAC,WAAW,GAAG,CAC1G,CAAC;IACJ,CAAC;IAED,IAAI,UAAU;QACZ,OAAO,IAAI,CAAC,WAAW,CAAC;IAC1B,CAAC;IAED,KAAK,CAAC,KAAK,CAAC,IAAY,EAAE,UAAmB,KAAK;QAChD,MAAM,QAAQ,GAAG,GAAG,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,GAAG,IAAI,EAAE,CAAC;QACnD,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACxC,IAAI,MAAM,EAAE,CAAC;YACX,IAAI,CAAC,KAAK,CAAC,SAAS,EAAE,CAAC;YACvB,OAAO,MAAM,CAAC;QAChB,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,IAAI,CAAC,UAAU,CAAC,CAAC,IAAI,CAAC,EAAE,OAAO,CAAC,CAAC;QACvD,OAAO,OAAO,CAAC,CAAC,CAAC,CAAC;IACpB,CAAC;IAED,KAAK,CAAC,UAAU,CAAC,KAAe,EAAE,WAAoB,KAAK;QACzD,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,EAAE,CAAC;QAElC,MAAM,SAAS,GAAG,WAAW,CAAC,GAAG,EAAE,CAAC;QAEpC,+BAA+B;QAC/B,MAAM,OAAO,GAAe,EAAE,CAAC;QAC/B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,IAAI,IAAI,CAAC,SAAS,EAAE,CAAC;YACtD,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC;QACnD,CAAC;QAED,iCAAiC;QACjC,MAAM,aAAa,GAAe,IAAI,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;QAC1D,IAAI,MAAM,GAAG,CAAC,CAAC;QAEf,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,IAAI,IAAI,CAAC,WAAW,EAAE,CAAC;YAC1D,MAAM,UAAU,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,WAAW,CAAC,CAAC;YAC1D,MAAM,QAAQ,GAAG,UAAU,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,CAAC,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC;YAC9D,MAAM,OAAO,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;YAE5C,KAAK,MAAM,UAAU,IAAI,OAAO,EAAE,CAAC;gBACjC,KAAK,MAAM,SAAS,IAAI,UAAU,EAAE,CAAC;oBACnC,aAAa,CAAC,MAAM,EAAE,CAAC,GAAG,SAAS,CAAC;gBACtC,CAAC;YACH,CAAC;QACH,CAAC;QAED,eAAe;QACf,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACtC,MAAM,QAAQ,GAAG,KAAK,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC;YACjC,IAAI,CAAC,KAAK,CAAC,WAAW,EAAE,CAAC;YACzB,IAAI,IAAI,CAAC,KAAK,CAAC,IAAI,IAAI,IAAI,CAAC,YAAY,EAAE,CAAC;gBACzC,MAAM,QAAQ,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC,KAAM,CAAC;gBACjD,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAC9B,CAAC;YACD,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,QAAQ,EAAE,aAAa,CAAC,CAAC,CAAC,CAAC,CAAC;QAC7C,CAAC;QAED,MAAM,OAAO,GAAG,CAAC,WAAW,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC,GAAG,IAAI,CAAC;QACvD,IAAI,CAAC,KAAK,CAAC,eAAe,IAAI,KAAK,CAAC,MAAM,CAAC;QAC3C,IAAI,CAAC,KAAK,CAAC,YAAY,EAAE,CAAC;QAC1B,IAAI,CAAC,KAAK,CAAC,SAAS,IAAI,OAAO,CAAC;QAEhC,OAAO,aAAa,CAAC;IACvB,CAAC;IAEO,KAAK,CAAC,OAAO,CAAC,KAAe,EAAE,OAAO,GAAG,CAAC;QAChD,MAAM,IAAI,GAA4B;YACpC,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,KAAK,EAAE,KAAK;SACb,CAAC;QAEF,sCAAsC;QACtC,IAAI,IAAI,CAAC,QAAQ,KAAK,QAAQ,IAAI,IAAI,CAAC,WAAW,EAAE,CAAC;YACnD,IAAI,CAAC,UAAU,GAAG,IAAI,CAAC,WAAW,CAAC;QACrC,CAAC;QACD,yBAAyB;QACzB,IAAI,IAAI,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;YAC/B,IAAI,CAAC,UAAU,GAAG,UAAU,CAAC;QAC/B,CAAC;QAED,KAAK,IAAI,OAAO,GAAG,CAAC,EAAE,OAAO,GAAG,OAAO,EAAE,OAAO,EAAE,EAAE,CAAC;YACnD,IAAI,CAAC;gBACH,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,IAAI,CAAC,OAAO,EAAE;oBACzC,MAAM,EAAE,MAAM;oBACd,OAAO,EAAE;wBACP,cAAc,EAAE,kBAAkB;wBAClC,eAAe,EAAE,UAAU,IAAI,CAAC,MAAM,EAAE;qBACzC;oBACD,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC;iBAC3B,CAAC,CAAC;gBAEH,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG,IAAI,QAAQ,CAAC,MAAM,IAAI,GAAG,EAAE,CAAC;oBACtD,MAAM,UAAU,GAAG,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC,CAAC;oBACvD,MAAM,MAAM,GAAG,UAAU,CAAC,CAAC,CAAC,QAAQ,CAAC,UAAU,EAAE,EAAE,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,OAAO,CAAC,GAAG,IAAI,CAAC;oBAC1F,OAAO,CAAC,KAAK,CAAC,kCAAkC,QAAQ,CAAC,MAAM,kBAAkB,MAAM,OAAO,CAAC,CAAC;oBAChG,MAAM,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,UAAU,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC;oBAC9C,SAAS;gBACX,CAAC;gBAED,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;oBACjB,MAAM,SAAS,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;oBACxC,MAAM,IAAI,cAAc,CACtB,GAAG,IAAI,CAAC,QAAQ,eAAe,QAAQ,CAAC,MAAM,MAAM,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,CAC9E,CAAC;gBACJ,CAAC;gBAED,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAC,IAAI,EAG/B,CAAC;gBAEF,IAAI,CAAC,KAAK,CAAC,QAAQ,EAAE,CAAC;gBACtB,IAAI,IAAI,CAAC,KAAK,EAAE,CAAC;oBACf,IAAI,CAAC,KAAK,CAAC,WAAW,IAAI,IAAI,CAAC,KAAK,CAAC,YAAY,IAAI,IAAI,CAAC,KAAK,CAAC,aAAa,IAAI,CAAC,CAAC;gBACrF,CAAC;gBAED,kCAAkC;gBAClC,MAAM,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;gBAC3D,OAAO,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;YACtC,CAAC;YAAC,OAAO,CAAC,EAAE,CAAC;gBACX,IAAI,CAAC,YAAY,cAAc;oBAAE,MAAM,CAAC,CAAC;gBACzC,IAAI,OAAO,KAAK,OAAO,GAAG,CAAC,EAAE,CAAC;oBAC5B,MAAM,IAAI,cAAc,CAAC,GAAG,IAAI,CAAC,QAAQ,qBAAsB,CAAW,CAAC,OAAO,EAAE,CAAC,CAAC;gBACxF,CAAC;gBACD,MAAM,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,OAAO,CAAC,GAAG,IAAI,CAAC;gBAC3C,OAAO,CAAC,KAAK,CAAC,uCAAuC,MAAM,OAAQ,CAAW,CAAC,OAAO,EAAE,CAAC,CAAC;gBAC1F,MAAM,IAAI,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,UAAU,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC;YAChD,CAAC;QACH,CAAC;QAED,MAAM,IAAI,cAAc,CAAC,GAAG,IAAI,CAAC,QAAQ,qBAAqB,OAAO,UAAU,CAAC,CAAC;IACnF,CAAC;IAED,QAAQ;QACN,OAAO;YACL,GAAG,IAAI,CAAC,KAAK;YACb,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,mBAAmB,EAAE,IAAI,CAAC,KAAK,CAAC,SAAS,GAAG,CAAC;gBAC3C,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,eAAe,GAAG,IAAI,CAAC,KAAK,CAAC,SAAS;gBACnD,CAAC,CAAC,CAAC;SACN,CAAC;IACJ,CAAC;IAED,UAAU;QACR,IAAI,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC;QACnB,IAAI,CAAC,KAAK,CAAC,SAAS,GAAG,CAAC,CAAC;QACzB,IAAI,CAAC,KAAK,CAAC,WAAW,GAAG,CAAC,CAAC;IAC7B,CAAC;IAED,KAAK,CAAC,MAAM;QACV,IAAI,CAAC,KAAK,CAAC,KAAK,EAAE,CAAC;IACrB,CAAC;CACF"}

package/dist/indexing/embedding-factory.d.ts

@@ -0,0 +1,19 @@
+/**
+ * Factory for creating embedding services based on configuration.
+ *
+ * Supports local ONNX inference or cloud APIs (OpenAI, Voyage).
+ * Configured via CODEBAXING_EMBEDDING_PROVIDER environment variable.
+ */
+import type { IEmbeddingService } from '../core/interfaces.js';
+import { type DeviceType, type DType } from './embedding-service.js';
+export type EmbeddingProvider = 'local' | 'openai' | 'voyage';
+export declare function getConfiguredProvider(): EmbeddingProvider;
+export interface CreateEmbeddingServiceOptions {
+    modelName?: string;
+    showProgress?: boolean;
+    device?: DeviceType;
+    dtype?: DType;
+    disableCachePurge?: boolean;
+}
+export declare function createEmbeddingService(options?: CreateEmbeddingServiceOptions): IEmbeddingService;
+//# sourceMappingURL=embedding-factory.d.ts.map

package/dist/indexing/embedding-factory.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"embedding-factory.d.ts","sourceRoot":"","sources":["../../src/indexing/embedding-factory.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,uBAAuB,CAAC;AAE/D,OAAO,EAAmC,KAAK,UAAU,EAAE,KAAK,KAAK,EAAE,MAAM,wBAAwB,CAAC;AAKtG,MAAM,MAAM,iBAAiB,GAAG,OAAO,GAAG,QAAQ,GAAG,QAAQ,CAAC;AAM9D,wBAAgB,qBAAqB,IAAI,iBAAiB,CAMzD;AAID,MAAM,WAAW,6BAA6B;IAC5C,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,YAAY,CAAC,EAAE,OAAO,CAAC;IACvB,MAAM,CAAC,EAAE,UAAU,CAAC;IACpB,KAAK,CAAC,EAAE,KAAK,CAAC;IACd,iBAAiB,CAAC,EAAE,OAAO,CAAC;CAC7B;AAED,wBAAgB,sBAAsB,CACpC,OAAO,GAAE,6BAAkC,GAC1C,iBAAiB,CA0BnB"}

package/dist/indexing/embedding-factory.js

@@ -0,0 +1,62 @@
+/**
+ * Factory for creating embedding services based on configuration.
+ *
+ * Supports local ONNX inference or cloud APIs (OpenAI, Voyage).
+ * Configured via CODEBAXING_EMBEDDING_PROVIDER environment variable.
+ */
+import { EmbeddingError } from '../core/exceptions.js';
+import { EmbeddingService, DEFAULT_MODEL } from './embedding-service.js';
+import { CloudEmbeddingService } from './cloud-embedding-service.js';
+const VALID_PROVIDERS = ['local', 'openai', 'voyage'];
+// ─── Configuration ───────────────────────────────────────────────────────────
+export function getConfiguredProvider() {
+    const envProvider = process.env.CODEBAXING_EMBEDDING_PROVIDER?.toLowerCase();
+    if (envProvider && VALID_PROVIDERS.includes(envProvider)) {
+        return envProvider;
+    }
+    return 'local';
+}
+export function createEmbeddingService(options = {}) {
+    const provider = getConfiguredProvider();
+    if (provider === 'local') {
+        return new EmbeddingService(options.modelName ?? DEFAULT_MODEL, {
+            showProgress: options.showProgress,
+            device: options.device,
+            dtype: options.dtype,
+            disableCachePurge: options.disableCachePurge,
+        });
+    }
+    // Cloud providers
+    const apiKey = getApiKey(provider);
+    const model = process.env.CODEBAXING_EMBEDDING_MODEL;
+    const dimensionsEnv = process.env.CODEBAXING_EMBEDDING_DIMENSIONS;
+    const dimensions = dimensionsEnv ? parseInt(dimensionsEnv, 10) : undefined;
+    const baseUrl = process.env.CODEBAXING_EMBEDDING_BASE_URL;
+    return new CloudEmbeddingService({
+        provider: provider,
+        apiKey,
+        model,
+        dimensions,
+        baseUrl,
+    });
+}
+function getApiKey(provider) {
+    let apiKey;
+    if (provider === 'openai') {
+        apiKey = process.env.CODEBAXING_OPENAI_API_KEY ?? process.env.OPENAI_API_KEY;
+        if (!apiKey) {
+            throw new EmbeddingError('OpenAI API key required. Set CODEBAXING_OPENAI_API_KEY or OPENAI_API_KEY environment variable.');
+        }
+    }
+    else if (provider === 'voyage') {
+        apiKey = process.env.CODEBAXING_VOYAGE_API_KEY ?? process.env.VOYAGE_API_KEY;
+        if (!apiKey) {
+            throw new EmbeddingError('Voyage API key required. Set CODEBAXING_VOYAGE_API_KEY or VOYAGE_API_KEY environment variable.');
+        }
+    }
+    else {
+        throw new EmbeddingError(`Unknown cloud provider: ${provider}`);
+    }
+    return apiKey;
+}
+//# sourceMappingURL=embedding-factory.js.map

package/dist/indexing/embedding-factory.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"embedding-factory.js","sourceRoot":"","sources":["../../src/indexing/embedding-factory.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAGH,OAAO,EAAE,cAAc,EAAE,MAAM,uBAAuB,CAAC;AACvD,OAAO,EAAE,gBAAgB,EAAE,aAAa,EAA+B,MAAM,wBAAwB,CAAC;AACtG,OAAO,EAAE,qBAAqB,EAAsB,MAAM,8BAA8B,CAAC;AAMzF,MAAM,eAAe,GAAwB,CAAC,OAAO,EAAE,QAAQ,EAAE,QAAQ,CAAC,CAAC;AAE3E,gFAAgF;AAEhF,MAAM,UAAU,qBAAqB;IACnC,MAAM,WAAW,GAAG,OAAO,CAAC,GAAG,CAAC,6BAA6B,EAAE,WAAW,EAAE,CAAC;IAC7E,IAAI,WAAW,IAAI,eAAe,CAAC,QAAQ,CAAC,WAAgC,CAAC,EAAE,CAAC;QAC9E,OAAO,WAAgC,CAAC;IAC1C,CAAC;IACD,OAAO,OAAO,CAAC;AACjB,CAAC;AAYD,MAAM,UAAU,sBAAsB,CACpC,UAAyC,EAAE;IAE3C,MAAM,QAAQ,GAAG,qBAAqB,EAAE,CAAC;IAEzC,IAAI,QAAQ,KAAK,OAAO,EAAE,CAAC;QACzB,OAAO,IAAI,gBAAgB,CAAC,OAAO,CAAC,SAAS,IAAI,aAAa,EAAE;YAC9D,YAAY,EAAE,OAAO,CAAC,YAAY;YAClC,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,KAAK,EAAE,OAAO,CAAC,KAAK;YACpB,iBAAiB,EAAE,OAAO,CAAC,iBAAiB;SAC7C,CAAC,CAAC;IACL,CAAC;IAED,kBAAkB;IAClB,MAAM,MAAM,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAC;IACnC,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,0BAA0B,CAAC;IACrD,MAAM,aAAa,GAAG,OAAO,CAAC,GAAG,CAAC,+BAA+B,CAAC;IAClE,MAAM,UAAU,GAAG,aAAa,CAAC,CAAC,CAAC,QAAQ,CAAC,aAAa,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IAC3E,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,6BAA6B,CAAC;IAE1D,OAAO,IAAI,qBAAqB,CAAC;QAC/B,QAAQ,EAAE,QAAyB;QACnC,MAAM;QACN,KAAK;QACL,UAAU;QACV,OAAO;KACR,CAAC,CAAC;AACL,CAAC;AAED,SAAS,SAAS,CAAC,QAA2B;IAC5C,IAAI,MAA0B,CAAC;IAE/B,IAAI,QAAQ,KAAK,QAAQ,EAAE,CAAC;QAC1B,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,yBAAyB,IAAI,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC;QAC7E,IAAI,CAAC,MAAM,EAAE,CAAC;YACZ,MAAM,IAAI,cAAc,CACtB,gGAAgG,CACjG,CAAC;QACJ,CAAC;IACH,CAAC;SAAM,IAAI,QAAQ,KAAK,QAAQ,EAAE,CAAC;QACjC,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,yBAAyB,IAAI,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC;QAC7E,IAAI,CAAC,MAAM,EAAE,CAAC;YACZ,MAAM,IAAI,cAAc,CACtB,gGAAgG,CACjG,CAAC;QACJ,CAAC;IACH,CAAC;SAAM,CAAC;QACN,MAAM,IAAI,cAAc,CAAC,2BAA2B,QAAQ,EAAE,CAAC,CAAC;IAClE,CAAC;IAED,OAAO,MAAM,CAAC;AAChB,CAAC"}

package/dist/indexing/embedding-service.d.ts

@@ -13,12 +13,19 @@
  *
  * Note: macOS does not support CUDA. Use 'webgpu' for GPU acceleration on Mac.
  */
+import type { IEmbeddingService } from '../core/interfaces.js';
 export type DeviceType = 'cpu' | 'cuda' | 'webgpu' | 'auto';
+export type DType = 'fp32' | 'fp16' | 'q8' | 'q4';
 /**
  * Get the configured device from environment variable.
  * Defaults to 'cpu' for maximum compatibility.
  */
 export declare function getConfiguredDevice(): DeviceType;
+/**
+ * Get the configured dtype from environment variable.
+ * Defaults to 'q8' for best speed/quality tradeoff (~3x faster than fp32).
+ */
+export declare function getConfiguredDtype(): DType;
 export interface EmbeddingModelConfig {
     modelId: string;
     dimensions: number;
@@ -28,10 +35,11 @@ export interface EmbeddingModelConfig {
 }
 export declare const EMBEDDING_MODELS: Record<string, EmbeddingModelConfig>;
 export declare const DEFAULT_MODEL = "all-MiniLM-L6-v2";
-export declare class EmbeddingService {
+export declare class EmbeddingService implements IEmbeddingService {
     private modelName;
     private config;
     private device;
+    private dtype;
     private extractor;
     private loading;
     private showProgress;
@@ -48,6 +56,7 @@ export declare class EmbeddingService {
     constructor(modelName?: string, options?: {
         showProgress?: boolean;
         device?: DeviceType;
+        dtype?: DType;
         disableCachePurge?: boolean;
     });
     /** Get the device being used for inference */

package/dist/indexing/embedding-service.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"embedding-service.d.ts","sourceRoot":"","sources":["../../src/indexing/embedding-service.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AASH,MAAM,MAAM,UAAU,GAAG,KAAK,GAAG,MAAM,GAAG,QAAQ,GAAG,MAAM,CAAC;AAI5D;;;GAGG;AACH,wBAAgB,mBAAmB,IAAI,UAAU,CAMhD;AAyCD,MAAM,WAAW,oBAAoB;IACnC,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;IACnB,YAAY,EAAE,MAAM,CAAC;IACrB,WAAW,EAAE,MAAM,CAAC;IACpB,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,eAAO,MAAM,gBAAgB,EAAE,MAAM,CAAC,MAAM,EAAE,oBAAoB,CAejE,CAAC;AAEF,eAAO,MAAM,aAAa,qBAAqB,CAAC;AAIhD,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,SAAS,CAAS;IAC1B,OAAO,CAAC,MAAM,CAAuB;IACrC,OAAO,CAAC,MAAM,CAAa;IAE3B,OAAO,CAAC,SAAS,CAAa;IAC9B,OAAO,CAAC,OAAO,CAA8B;IAC7C,OAAO,CAAC,YAAY,CAAU;IAG9B,OAAO,CAAC,KAAK,CAAoC;IACjD,OAAO,CAAC,YAAY,CAAQ;IAG5B,KAAK;;;;;;MAMH;IAGF,OAAO,CAAC,iBAAiB,CAAU;gBAGjC,SAAS,GAAE,MAAsB,EACjC,OAAO,GAAE;QAAE,YAAY,CAAC,EAAE,OAAO,CAAC;QAAC,MAAM,CAAC,EAAE,UAAU,CAAC;QAAC,iBAAiB,CAAC,EAAE,OAAO,CAAA;KAAO;IAqB5F,8CAA8C;IAC9C,SAAS,IAAI,UAAU;IAIvB,IAAI,UAAU,IAAI,MAAM,CAEvB;YAEa,SAAS;IA+FjB,KAAK,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,GAAE,OAAe,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IAkChE,UAAU,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,OAAO,GAAE,OAAe,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC;IAsEhF,QAAQ,IAAI,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC;IAUnC,UAAU,IAAI,IAAI;IAMlB;;;OAGG;IACH,MAAM,CAAC,eAAe,CAAC,OAAO,EAAE,MAAM,GAAG,OAAO;IAmB1C,MAAM,IAAI,OAAO,CAAC,IAAI,CAAC;CAM9B;AAMD,wBAAgB,mBAAmB,CACjC,SAAS,GAAE,MAAsB,EACjC,OAAO,GAAE;IAAE,YAAY,CAAC,EAAE,OAAO,CAAC;IAAC,MAAM,CAAC,EAAE,UAAU,CAAA;CAAO,GAC5D,gBAAgB,CASlB;AAED,wBAAsB,qBAAqB,CAAC,SAAS,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAa7E"}
+{"version":3,"file":"embedding-service.d.ts","sourceRoot":"","sources":["../../src/indexing/embedding-service.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;GAcG;AAMH,OAAO,KAAK,EAAE,iBAAiB,EAAE,MAAM,uBAAuB,CAAC;AAI/D,MAAM,MAAM,UAAU,GAAG,KAAK,GAAG,MAAM,GAAG,QAAQ,GAAG,MAAM,CAAC;AAC5D,MAAM,MAAM,KAAK,GAAG,MAAM,GAAG,MAAM,GAAG,IAAI,GAAG,IAAI,CAAC;AAKlD;;;GAGG;AACH,wBAAgB,mBAAmB,IAAI,UAAU,CAMhD;AAED;;;GAGG;AACH,wBAAgB,kBAAkB,IAAI,KAAK,CAM1C;AAyCD,MAAM,WAAW,oBAAoB;IACnC,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;IACnB,YAAY,EAAE,MAAM,CAAC;IACrB,WAAW,EAAE,MAAM,CAAC;IACpB,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,eAAO,MAAM,gBAAgB,EAAE,MAAM,CAAC,MAAM,EAAE,oBAAoB,CAejE,CAAC;AAEF,eAAO,MAAM,aAAa,qBAAqB,CAAC;AAIhD,qBAAa,gBAAiB,YAAW,iBAAiB;IACxD,OAAO,CAAC,SAAS,CAAS;IAC1B,OAAO,CAAC,MAAM,CAAuB;IACrC,OAAO,CAAC,MAAM,CAAa;IAC3B,OAAO,CAAC,KAAK,CAAQ;IAErB,OAAO,CAAC,SAAS,CAAa;IAC9B,OAAO,CAAC,OAAO,CAA8B;IAC7C,OAAO,CAAC,YAAY,CAAU;IAG9B,OAAO,CAAC,KAAK,CAAoC;IACjD,OAAO,CAAC,YAAY,CAAQ;IAG5B,KAAK;;;;;;MAMH;IAGF,OAAO,CAAC,iBAAiB,CAAU;gBAGjC,SAAS,GAAE,MAAsB,EACjC,OAAO,GAAE;QAAE,YAAY,CAAC,EAAE,OAAO,CAAC;QAAC,MAAM,CAAC,EAAE,UAAU,CAAC;QAAC,KAAK,CAAC,EAAE,KAAK,CAAC;QAAC,iBAAiB,CAAC,EAAE,OAAO,CAAA;KAAO;IAsB3G,8CAA8C;IAC9C,SAAS,IAAI,UAAU;IAIvB,IAAI,UAAU,IAAI,MAAM,CAEvB;YAEa,SAAS;IAgGjB,KAAK,CAAC,IAAI,EAAE,MAAM,EAAE,OAAO,GAAE,OAAe,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC;IAkChE,UAAU,CAAC,KAAK,EAAE,MAAM,EAAE,EAAE,OAAO,GAAE,OAAe,GAAG,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC;IAsEhF,QAAQ,IAAI,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC;IAUnC,UAAU,IAAI,IAAI;IAMlB;;;OAGG;IACH,MAAM,CAAC,eAAe,CAAC,OAAO,EAAE,MAAM,GAAG,OAAO;IAmB1C,MAAM,IAAI,OAAO,CAAC,IAAI,CAAC;CAM9B;AAMD,wBAAgB,mBAAmB,CACjC,SAAS,GAAE,MAAsB,EACjC,OAAO,GAAE;IAAE,YAAY,CAAC,EAAE,OAAO,CAAC;IAAC,MAAM,CAAC,EAAE,UAAU,CAAA;CAAO,GAC5D,gBAAgB,CASlB;AAED,wBAAsB,qBAAqB,CAAC,SAAS,CAAC,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAa7E"}