claude-eidetic 0.1.0 → 0.1.2

package/README.md

@@ -0,0 +1,333 @@
+# claude-eidetic
+
+
+[![tests](https://img.shields.io/github/actions/workflow/status/eidetics/claude-eidetic/ci.yml?style=flat-square&label=tests)](https://github.com/eidetics/claude-eidetic/actions/workflows/ci.yml)
+[![npm](https://img.shields.io/npm/v/claude-eidetic)](https://www.npmjs.com/package/claude-eidetic)
+[![MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+
+Semantic code search, persistent memory, and session continuity for Claude Code. One plugin, not three.
+
+---
+
+## 🚀 Quick Start
+
+```bash
+claude plugin install eidetics/claude-eidetic
+```
+
+```bash
+export OPENAI_API_KEY=sk-...         # for embeddings (default)
+export ANTHROPIC_API_KEY=sk-ant-...  # for memory extraction (default)
+```
+
+Index your codebase once, then search by meaning:
+
+```
+index_codebase(path="/your/project")
+search_code("how does authentication work")
+```
+
+---
+
+## ✨ Features
+
+### 🔍 Semantic Code Search
+
+**Find code by meaning, not keywords.** Search across your entire codebase with natural language, returning the most relevant functions and classes, not a list of files to read.
+
+```
+search_code("how does the retry logic work")
+search_code("authentication middleware", extensionFilter=[".ts"])
+search_code(project="backend", query="auth flow")
+```
+
+### 🏗️ Architecture at a Glance
+
+**Get every class, function, and method in one call.** `browse_structure` returns a condensed map of your codebase with signatures, grouped by file, replacing a Glob + Read cascade with a single tool call.
+
+```
+browse_structure(path="/my/project", kind="class")
+list_symbols(path="/my/project", nameFilter="handle")
+```
+
+### 📚 Documentation Cache
+
+**Fetch docs once, search them forever.** Cache external documentation as searchable embeddings. Retrieve relevant passages instantly, far cheaper than re-fetching the same page each session.
+
+```
+index_document(content=..., library="react", topic="hooks", source="https://...")
+search_documents("React useCallback dependencies", library="react")
+```
+
+### 🧠 Persistent Memory
+
+**Claude remembers your preferences between sessions.** `add_memory` uses an LLM to extract structured facts from conversation text (coding style, architecture decisions, debugging insights) and deduplicates them semantically. Not a static config file you forget to update.
+
+```
+add_memory("Always use absolute imports, never relative")
+search_memory("how does this team handle errors")
+```
+
+### 🔄 Session Continuity
+
+**Every session picks up where the last one left off.** When a session ends (or context compacts mid-session), Eidetic automatically writes a structured note capturing files changed, tasks, commands, and decisions. `/catchup` at the start of a new session reconstructs exactly where you were. No user action required.
+
+### 👻 Invisible Optimizations
+
+Eight hook events fire automatically, nudging toward cheaper tools, redirecting file reads for 15-20% token savings, tracking changed files, and saving session state on exit.
+
+<details>
+<summary><strong>Hook event details</strong></summary>
+
+| Hook | Trigger | What it does |
+|---|---|---|
+| `SessionStart` | Session opens | Validates config, injects last-session context |
+| `UserPromptSubmit` | Every message | Nudges toward `search_code` over Grep/Explore for conceptual queries |
+| `PreToolUse` (Read) | Before every Read | Blocks Read for text files, redirects to `read_file` for 15-20% token savings |
+| `PreToolUse` (WebFetch / query-docs) | Before doc fetches | Suggests `search_documents` if library is cached (allows fetch either way) |
+| `PostToolUse` (Write / Edit) | After every file write | Tracks changed files in a shadow git index |
+| `Stop` | After Claude responds | Commits shadow index; triggers targeted re-index of changed files only |
+| `PreCompact` | Before context compaction | Captures session state to notes before memory is lost |
+| `SessionEnd` | Session closes | Writes session note (files, tasks, commands); extracts developer memories via LLM |
+
+</details>
+
+---
+
+## 🗺️ When to Use What
+
+| Need | Use | Notes |
+|---|---|---|
+| Find implementations by concept | `search_code` | Semantic, natural language queries |
+| Exact string or regex match | Grep | Grep wins for exact matches |
+| Find file by exact name | Glob | Glob wins for name patterns |
+| Understand module structure | `browse_structure` | One call vs Glob + Read cascade |
+| Read a specific known file | `read_file` | Cheaper than built-in Read for code files |
+| Search cached documentation | `search_documents` | Far cheaper than re-fetching |
+| Recall project conventions | `search_memory` | Global across all projects and sessions |
+
+---
+
+## 📖 Skills Reference
+
+| Skill | What it does |
+|---|---|
+| `/search` | Guided semantic search with best-practice prompts |
+| `/index` | Index or re-index a codebase with dry-run option |
+| `/cache-docs` | Fetch and cache external documentation |
+| `/catchup` | Search session notes and reconstruct where you left off |
+| `/wrapup` | Extract decisions, rationale, open questions, and next actions from the conversation |
+
+---
+
+<details>
+<summary><strong>🔥 Why does this exist? (The Problem)</strong></summary>
+
+Every new Claude Code session starts cold. You re-explain the architecture. You re-fetch the same docs. Claude reads the same files repeatedly, burning tokens just to get back to where you were.
+
+| Task | Without Eidetic | With Eidetic |
+|---|---|---|
+| Find where auth errors are handled | Grep cascade, read 8 files, ~10,700 tokens | `search_code("auth error handling")` ~220 tokens |
+| Resume after context compaction | Re-explain 20 min of context, ~2,000 tokens | `/catchup` ~200 tokens |
+| Look up React hooks docs | Fetch docs page, ~5,000 tokens | `search_documents("React useEffect")` ~20 tokens |
+| Read a 400-line file | Built-in Read with line numbers, ~900 tokens | `read_file(path)` ~740 tokens |
+
+</details>
+
+---
+
+## 📦 Installation
+
+### Plugin (recommended)
+
+```bash
+claude plugin install eidetics/claude-eidetic
+```
+
+The plugin auto-starts the MCP server, installs skills, and configures hooks.
+
+### npx (manual MCP config)
+
+Add to your `.mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "claude-eidetic": {
+      "command": "npx",
+      "args": ["-y", "claude-eidetic"],
+      "env": {
+        "OPENAI_API_KEY": "sk-...",
+        "ANTHROPIC_API_KEY": "sk-ant-..."
+      }
+    }
+  }
+}
+```
+
+`ANTHROPIC_API_KEY` is needed for the memory LLM (default provider). Omit it if using `MEMORY_LLM_PROVIDER=openai` or `ollama`.
+
+### Global install
+
+```bash
+npm install -g claude-eidetic
+```
+
+### From source
+
+```bash
+git clone https://github.com/eidetics/claude-eidetic
+cd claude-eidetic
+npm install && npx tsc && npm start
+```
+
+### Requirements
+
+- Node.js >= 20.0.0
+- An API key (OpenAI for embeddings, Anthropic for memory extraction, or Ollama for both free)
+- Docker (optional): Qdrant auto-provisions via Docker if not already running
+- C/C++ build tools: required by tree-sitter native bindings (`node-gyp`)
+
+---
+
+## ⚙️ Configuration
+
+All configuration is via environment variables. No config files.
+
+### Using Ollama (free, local)
+
+```bash
+export EMBEDDING_PROVIDER=ollama
+export MEMORY_LLM_PROVIDER=ollama
+# No API keys needed
+```
+
+<details>
+<summary><strong>Full configuration reference</strong></summary>
+
+| Variable | Default | Description |
+|---|---|---|
+| `OPENAI_API_KEY` | _(required for openai)_ | OpenAI API key for embeddings and/or memory |
+| `ANTHROPIC_API_KEY` | _(required for anthropic memory)_ | Anthropic API key for memory LLM |
+| `EMBEDDING_PROVIDER` | `openai` | `openai`, `ollama`, or `local` |
+| `EMBEDDING_MODEL` | `text-embedding-3-small` (openai) / `nomic-embed-text` (ollama) | Embedding model name |
+| `EMBEDDING_BATCH_SIZE` | `100` | Batch size for embedding requests (1-2048) |
+| `INDEXING_CONCURRENCY` | `8` | Parallel file indexing workers (1-32) |
+| `OPENAI_BASE_URL` | _(none)_ | Custom OpenAI-compatible endpoint |
+| `OLLAMA_BASE_URL` | `http://localhost:11434/v1` | Ollama server URL |
+| `VECTORDB_PROVIDER` | `qdrant` | `qdrant` or `milvus` |
+| `QDRANT_URL` | `http://localhost:6333` | Qdrant server URL |
+| `QDRANT_API_KEY` | _(none)_ | Qdrant API key (for remote/cloud instances) |
+| `MILVUS_ADDRESS` | `localhost:19530` | Milvus server address |
+| `MILVUS_TOKEN` | _(none)_ | Milvus authentication token |
+| `EIDETIC_DATA_DIR` | `~/.eidetic/` | Data root for snapshots, memory DB, registry |
+| `CUSTOM_EXTENSIONS` | `[]` | JSON array of extra file extensions to index (e.g., `[".dart",".arb"]`) |
+| `CUSTOM_IGNORE_PATTERNS` | `[]` | JSON array of glob patterns to exclude |
+| `MEMORY_LLM_PROVIDER` | `anthropic` | `anthropic`, `openai`, or `ollama` |
+| `MEMORY_LLM_MODEL` | `claude-haiku-4-5-20251001` (anthropic) / `gpt-4o-mini` (openai) / `llama3.2` (ollama) | Model for memory extraction |
+| `MEMORY_LLM_BASE_URL` | _(none)_ | Custom base URL for memory LLM |
+| `MEMORY_LLM_API_KEY` | _(none)_ | API key override for memory LLM |
+
+</details>
+
+---
+
+<details>
+<summary><strong>🔧 Tool Reference</strong></summary>
+
+### 🔍 Code Search
+
+| Tool | Description |
+|---|---|
+| `search_code` | Hybrid semantic search over indexed codebase. Returns compact results by default. |
+| `index_codebase` | Index a directory. Supports `dryRun`, `force`, `customExtensions`, `customIgnorePatterns`. |
+| `list_indexed` | List all indexed codebases with file/chunk counts and status. |
+| `get_indexing_status` | Check indexing progress for a path or project. |
+| `clear_index` | Remove the search index for a codebase. |
+| `cleanup_vectors` | Remove orphaned vectors for deleted files. No re-embedding cost. |
+| `browse_structure` | Condensed structural map: classes, functions, methods with signatures, grouped by file. |
+| `list_symbols` | Compact symbol table with name/kind/file/line. Supports name, kind, and path filters. |
+
+### 📄 File Reading
+
+| Tool | Description |
+|---|---|
+| `read_file` | Read file without line-number overhead. Cheaper than built-in Read for code files. |
+
+### 📚 Documentation Cache
+
+| Tool | Description |
+|---|---|
+| `index_document` | Cache external documentation for semantic search. Supports TTL for staleness tracking. |
+| `search_documents` | Search cached docs. Far cheaper than re-fetching the same page. |
+
+### 🧠 Memory
+
+| Tool | Description |
+|---|---|
+| `add_memory` | LLM-extracted facts from text. Auto-deduplicates. Seven categories. |
+| `search_memory` | Semantic search over stored memories. Filterable by category. |
+| `list_memories` | List all memories, optionally filtered by category. |
+| `delete_memory` | Delete a specific memory by UUID. |
+| `memory_history` | View change history for a memory (additions, updates, deletions). |
+
+</details>
+
+---
+
+## 🌐 Supported Languages
+
+**AST-aware** (functions and classes chunked intact):
+
+<p>
+  <img src="https://img.shields.io/badge/JavaScript-F7DF1E?style=flat-square&logo=javascript&logoColor=black" alt="JavaScript"/>
+  <img src="https://img.shields.io/badge/TypeScript-3178C6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript"/>
+  <img src="https://img.shields.io/badge/React_(JSX/TSX)-61DAFB?style=flat-square&logo=react&logoColor=black" alt="JSX/TSX"/>
+  <img src="https://img.shields.io/badge/Python-3776AB?style=flat-square&logo=python&logoColor=white" alt="Python"/>
+  <img src="https://img.shields.io/badge/Go-00ADD8?style=flat-square&logo=go&logoColor=white" alt="Go"/>
+  <img src="https://img.shields.io/badge/Java-ED8B00?style=flat-square&logo=openjdk&logoColor=white" alt="Java"/>
+  <img src="https://img.shields.io/badge/Rust-000000?style=flat-square&logo=rust&logoColor=white" alt="Rust"/>
+  <img src="https://img.shields.io/badge/C-A8B9CC?style=flat-square&logo=c&logoColor=black" alt="C"/>
+  <img src="https://img.shields.io/badge/C++-00599C?style=flat-square&logo=cplusplus&logoColor=white" alt="C++"/>
+  <img src="https://img.shields.io/badge/C%23-512BD4?style=flat-square&logo=csharp&logoColor=white" alt="C#"/>
+</p>
+
+**Line-based fallback** (sliding window chunking for everything else):
+
+<p>
+  <img src="https://img.shields.io/badge/Markdown-000000?style=flat-square&logo=markdown&logoColor=white" alt="Markdown"/>
+  <img src="https://img.shields.io/badge/YAML-CB171E?style=flat-square&logo=yaml&logoColor=white" alt="YAML"/>
+  <img src="https://img.shields.io/badge/JSON-000000?style=flat-square&logo=json&logoColor=white" alt="JSON"/>
+  <img src="https://img.shields.io/badge/Ruby-CC342D?style=flat-square&logo=ruby&logoColor=white" alt="Ruby"/>
+  <img src="https://img.shields.io/badge/PHP-777BB4?style=flat-square&logo=php&logoColor=white" alt="PHP"/>
+  <img src="https://img.shields.io/badge/Swift-F05138?style=flat-square&logo=swift&logoColor=white" alt="Swift"/>
+  <img src="https://img.shields.io/badge/Kotlin-7F52FF?style=flat-square&logo=kotlin&logoColor=white" alt="Kotlin"/>
+  <img src="https://img.shields.io/badge/and_more...-30363d?style=flat-square" alt="and more"/>
+</p>
+
+---
+
+## 🛠️ Development
+
+```bash
+npm install && npx tsc    # install and build
+npm run dev               # watch mode (tsx)
+npm test                  # unit tests (no external services needed)
+npm run test:integration  # requires Qdrant at localhost:6333 + OPENAI_API_KEY
+```
+
+**Commit format:** `type(scope): description`
+Types: `feat`, `fix`, `docs`, `refactor`, `perf`, `chore`, `test`
+Scopes: `embedding`, `vectordb`, `splitter`, `indexer`, `mcp`, `infra`, `config`
+
+---
+
+## 🙏 Acknowledgements
+
+Heavily inspired by [mem0](https://github.com/mem0ai/mem0), [claude-mem](https://github.com/thedotmack/claude-mem), and [claude-context](https://github.com/zilliztech/claude-context). Documentation retrieval powered by [context7](https://github.com/upstash/context7).
+
+---
+
+## 📄 License
+
+MIT

package/dist/config.d.ts

@@ -15,6 +15,11 @@ declare const configSchema: z.ZodEffects<z.ZodObject<{
     eideticDataDir: z.ZodDefault<z.ZodString>;
     customExtensions: z.ZodEffects<z.ZodDefault<z.ZodArray<z.ZodString, "many">>, string[], unknown>;
     customIgnorePatterns: z.ZodEffects<z.ZodDefault<z.ZodArray<z.ZodString, "many">>, string[], unknown>;
+    memoryLlmProvider: z.ZodDefault<z.ZodEnum<["openai", "ollama", "anthropic"]>>;
+    memoryLlmModel: z.ZodOptional<z.ZodString>;
+    memoryLlmBaseUrl: z.ZodOptional<z.ZodString>;
+    memoryLlmApiKey: z.ZodOptional<z.ZodString>;
+    anthropicApiKey: z.ZodDefault<z.ZodString>;
 }, "strip", z.ZodTypeAny, {
     embeddingProvider: "openai" | "ollama" | "local";
     openaiApiKey: string;
@@ -27,10 +32,15 @@ declare const configSchema: z.ZodEffects<z.ZodObject<{
     eideticDataDir: string;
     customExtensions: string[];
     customIgnorePatterns: string[];
+    memoryLlmProvider: "openai" | "ollama" | "anthropic";
+    anthropicApiKey: string;
     openaiBaseUrl?: string | undefined;
     embeddingModel?: string | undefined;
     qdrantApiKey?: string | undefined;
     milvusToken?: string | undefined;
+    memoryLlmModel?: string | undefined;
+    memoryLlmBaseUrl?: string | undefined;
+    memoryLlmApiKey?: string | undefined;
 }, {
     embeddingProvider?: "openai" | "ollama" | "local" | undefined;
     openaiApiKey?: string | undefined;
@@ -47,8 +57,14 @@ declare const configSchema: z.ZodEffects<z.ZodObject<{
     eideticDataDir?: string | undefined;
     customExtensions?: unknown;
     customIgnorePatterns?: unknown;
+    memoryLlmProvider?: "openai" | "ollama" | "anthropic" | undefined;
+    memoryLlmModel?: string | undefined;
+    memoryLlmBaseUrl?: string | undefined;
+    memoryLlmApiKey?: string | undefined;
+    anthropicApiKey?: string | undefined;
 }>, {
     embeddingModel: string;
+    memoryLlmModel: string;
     embeddingProvider: "openai" | "ollama" | "local";
     openaiApiKey: string;
     ollamaBaseUrl: string;
@@ -60,9 +76,13 @@ declare const configSchema: z.ZodEffects<z.ZodObject<{
     eideticDataDir: string;
     customExtensions: string[];
     customIgnorePatterns: string[];
+    memoryLlmProvider: "openai" | "ollama" | "anthropic";
+    anthropicApiKey: string;
     openaiBaseUrl?: string | undefined;
     qdrantApiKey?: string | undefined;
     milvusToken?: string | undefined;
+    memoryLlmBaseUrl?: string | undefined;
+    memoryLlmApiKey?: string | undefined;
 }, {
     embeddingProvider?: "openai" | "ollama" | "local" | undefined;
     openaiApiKey?: string | undefined;
@@ -79,6 +99,11 @@ declare const configSchema: z.ZodEffects<z.ZodObject<{
     eideticDataDir?: string | undefined;
     customExtensions?: unknown;
     customIgnorePatterns?: unknown;
+    memoryLlmProvider?: "openai" | "ollama" | "anthropic" | undefined;
+    memoryLlmModel?: string | undefined;
+    memoryLlmBaseUrl?: string | undefined;
+    memoryLlmApiKey?: string | undefined;
+    anthropicApiKey?: string | undefined;
 }>;
 export type Config = z.infer<typeof configSchema>;
 export declare function loadConfig(): Config;

package/dist/config.js

@@ -2,7 +2,8 @@ import os from 'node:os';
 import path from 'node:path';
 import { z } from 'zod';
 import { ConfigError } from './errors.js';
-const configSchema = z.object({
+const configSchema = z
+    .object({
     embeddingProvider: z.enum(['openai', 'ollama', 'local']).default('openai'),
     openaiApiKey: z.string().default(''),
     openaiBaseUrl: z.string().optional(),
@@ -16,43 +17,57 @@ const configSchema = z.object({
     milvusAddress: z.string().default('localhost:19530'),
     milvusToken: z.string().optional(),
     eideticDataDir: z.string().default(path.join(os.homedir(), '.eidetic')),
-    customExtensions: z.preprocess((val) => typeof val === 'string' ? JSON.parse(val) : val, z.array(z.string()).default([])),
-    customIgnorePatterns: z.preprocess((val) => typeof val === 'string' ? JSON.parse(val) : val, z.array(z.string()).default([])),
-}).transform((cfg) => ({
+    customExtensions: z.preprocess((val) => (typeof val === 'string' ? JSON.parse(val) : val), z.array(z.string()).default([])),
+    customIgnorePatterns: z.preprocess((val) => (typeof val === 'string' ? JSON.parse(val) : val), z.array(z.string()).default([])),
+    memoryLlmProvider: z.enum(['openai', 'ollama', 'anthropic']).default('anthropic'),
+    memoryLlmModel: z.string().optional(),
+    memoryLlmBaseUrl: z.string().optional(),
+    memoryLlmApiKey: z.string().optional(),
+    anthropicApiKey: z.string().default(''),
+})
+    .transform((cfg) => ({
     ...cfg,
     // Default embedding model depends on provider
-    embeddingModel: cfg.embeddingModel
-        ?? (cfg.embeddingProvider === 'ollama' ? 'nomic-embed-text' : 'text-embedding-3-small'),
+    embeddingModel: cfg.embeddingModel ??
+        (cfg.embeddingProvider === 'ollama' ? 'nomic-embed-text' : 'text-embedding-3-small'),
+    // Default memory LLM model depends on provider
+    memoryLlmModel: cfg.memoryLlmModel ??
+        (cfg.memoryLlmProvider === 'ollama'
+            ? 'llama3.2'
+            : cfg.memoryLlmProvider === 'anthropic'
+                ? 'claude-haiku-4-5-20251001'
+                : 'gpt-4o-mini'),
 }));
 let cachedConfig = null;
 export function loadConfig() {
     const raw = {
         embeddingProvider: process.env.EMBEDDING_PROVIDER,
-        openaiApiKey: process.env.OPENAI_API_KEY ?? '',
+        openaiApiKey: (process.env.OPENAI_API_KEY ?? '').trim().replace(/^["']|["']$/g, ''),
         openaiBaseUrl: process.env.OPENAI_BASE_URL || undefined,
         ollamaBaseUrl: process.env.OLLAMA_BASE_URL,
         embeddingModel: process.env.EMBEDDING_MODEL || undefined,
         embeddingBatchSize: process.env.EMBEDDING_BATCH_SIZE,
         indexingConcurrency: process.env.INDEXING_CONCURRENCY,
         qdrantUrl: process.env.QDRANT_URL,
-        qdrantApiKey: process.env.QDRANT_API_KEY || undefined,
+        qdrantApiKey: process.env.QDRANT_API_KEY?.trim().replace(/^["']|["']$/g, '') || undefined,
         vectordbProvider: process.env.VECTORDB_PROVIDER,
         milvusAddress: process.env.MILVUS_ADDRESS,
-        milvusToken: process.env.MILVUS_TOKEN || undefined,
+        milvusToken: process.env.MILVUS_TOKEN?.trim().replace(/^["']|["']$/g, '') || undefined,
         eideticDataDir: process.env.EIDETIC_DATA_DIR,
         customExtensions: process.env.CUSTOM_EXTENSIONS,
         customIgnorePatterns: process.env.CUSTOM_IGNORE_PATTERNS,
+        memoryLlmProvider: process.env.MEMORY_LLM_PROVIDER,
+        memoryLlmModel: process.env.MEMORY_LLM_MODEL || undefined,
+        memoryLlmBaseUrl: process.env.MEMORY_LLM_BASE_URL || undefined,
+        memoryLlmApiKey: process.env.MEMORY_LLM_API_KEY?.trim().replace(/^["']|["']$/g, '') || undefined,
+        anthropicApiKey: (process.env.ANTHROPIC_API_KEY ?? '').trim().replace(/^["']|["']$/g, ''),
     };
     const result = configSchema.safeParse(raw);
     if (!result.success) {
-        const issues = result.error.issues.map(i => `  ${i.path.join('.')}: ${i.message}`).join('\n');
+        const issues = result.error.issues.map((i) => `  ${i.path.join('.')}: ${i.message}`).join('\n');
         throw new ConfigError(`Invalid configuration:\n${issues}`);
     }
     const config = result.data;
-    if (config.embeddingProvider === 'openai' && !config.openaiApiKey) {
-        throw new ConfigError('OPENAI_API_KEY is required when EMBEDDING_PROVIDER is "openai" (the default). ' +
-            'Set EMBEDDING_PROVIDER=ollama or EMBEDDING_PROVIDER=local to use a local embedding server.');
-    }
     cachedConfig = config;
     return cachedConfig;
 }

package/dist/core/cleanup.d.ts

@@ -0,0 +1,8 @@
+import type { VectorDB } from '../vectordb/types.js';
+export interface CleanupResult {
+    removedFiles: string[];
+    totalRemoved: number;
+    durationMs: number;
+}
+export declare function cleanupVectors(rootPath: string, vectordb: VectorDB, onProgress?: (pct: number, msg: string) => void, customExtensions?: string[], customIgnorePatterns?: string[]): Promise<CleanupResult>;
+//# sourceMappingURL=cleanup.d.ts.map

package/dist/core/cleanup.js

@@ -0,0 +1,41 @@
+import { scanFiles, buildSnapshot, diffSnapshots } from './sync.js';
+import { loadSnapshot, saveSnapshot } from './snapshot-io.js';
+import { pathToCollectionName } from '../paths.js';
+import { IndexingError } from '../errors.js';
+export async function cleanupVectors(rootPath, vectordb, onProgress, customExtensions, customIgnorePatterns) {
+    const startTime = Date.now();
+    const previousSnapshot = loadSnapshot(rootPath);
+    if (!previousSnapshot) {
+        throw new IndexingError(`No snapshot found for ${rootPath}. Index the codebase first before running cleanup.`);
+    }
+    onProgress?.(10, 'Scanning files on disk...');
+    const filePaths = await scanFiles(rootPath, customExtensions ?? [], customIgnorePatterns ?? []);
+    onProgress?.(40, 'Building current snapshot...');
+    const currentSnapshot = buildSnapshot(rootPath, filePaths);
+    onProgress?.(60, 'Diffing snapshots...');
+    const { removed } = diffSnapshots(previousSnapshot, currentSnapshot);
+    if (removed.length === 0) {
+        onProgress?.(100, 'No removed files found.');
+        return { removedFiles: [], totalRemoved: 0, durationMs: Date.now() - startTime };
+    }
+    const collectionName = pathToCollectionName(rootPath);
+    let deletedCount = 0;
+    for (const rel of removed) {
+        onProgress?.(60 + Math.round((deletedCount / removed.length) * 35), `Deleting vectors for ${rel}...`);
+        await vectordb.deleteByPath(collectionName, rel);
+        deletedCount++;
+    }
+    // Save updated snapshot (removes the deleted file entries)
+    const updatedSnapshot = { ...previousSnapshot };
+    for (const rel of removed) {
+        Reflect.deleteProperty(updatedSnapshot, rel);
+    }
+    saveSnapshot(rootPath, updatedSnapshot);
+    onProgress?.(100, 'Cleanup complete.');
+    return {
+        removedFiles: removed,
+        totalRemoved: removed.length,
+        durationMs: Date.now() - startTime,
+    };
+}
+//# sourceMappingURL=cleanup.js.map

package/dist/core/doc-indexer.d.ts

@@ -0,0 +1,13 @@
+import type { Embedding } from '../embedding/types.js';
+import type { VectorDB } from '../vectordb/types.js';
+export interface DocIndexResult {
+    library: string;
+    topic: string;
+    source: string;
+    collectionName: string;
+    totalChunks: number;
+    estimatedTokens: number;
+    durationMs: number;
+}
+export declare function indexDocument(content: string, source: string, library: string, topic: string, embedding: Embedding, vectordb: VectorDB, ttlDays?: number): Promise<DocIndexResult>;
+//# sourceMappingURL=doc-indexer.d.ts.map

package/dist/core/doc-indexer.js

@@ -0,0 +1,76 @@
+import { randomUUID } from 'node:crypto';
+import { LineSplitter } from '../splitter/line.js';
+import { docCollectionName } from '../paths.js';
+import { upsertDocEntry } from '../state/doc-metadata.js';
+import { IndexingError } from '../errors.js';
+import { getConfig } from '../config.js';
+export async function indexDocument(content, source, library, topic, embedding, vectordb, ttlDays = 7) {
+    const start = Date.now();
+    if (!content || content.trim().length === 0) {
+        throw new IndexingError('Document content is empty.');
+    }
+    if (!source)
+        throw new IndexingError('Document source is required.');
+    if (!library)
+        throw new IndexingError('Library name is required.');
+    if (!topic)
+        throw new IndexingError('Topic is required.');
+    const collection = docCollectionName(library);
+    const config = getConfig();
+    const splitter = new LineSplitter();
+    const chunks = splitter.split(content, 'markdown', source);
+    if (chunks.length === 0) {
+        throw new IndexingError('Document produced no chunks after splitting.');
+    }
+    const exists = await vectordb.hasCollection(collection);
+    if (!exists) {
+        await vectordb.createCollection(collection, embedding.dimension);
+    }
+    try {
+        await vectordb.deleteByPath(collection, source);
+    }
+    catch {
+        // collection may be new with no matching docs
+    }
+    const batchSize = config.embeddingBatchSize;
+    let totalChunks = 0;
+    let totalTokens = 0;
+    for (let i = 0; i < chunks.length; i += batchSize) {
+        const batch = chunks.slice(i, i + batchSize);
+        const texts = batch.map((c) => c.content);
+        const estimation = embedding.estimateTokens(texts);
+        totalTokens += estimation.estimatedTokens;
+        const vectors = await embedding.embedBatch(texts);
+        const documents = batch.map((chunk, j) => ({
+            id: randomUUID(),
+            content: chunk.content,
+            vector: vectors[j],
+            relativePath: source,
+            startLine: chunk.startLine,
+            endLine: chunk.endLine,
+            fileExtension: '.md',
+            language: 'markdown',
+        }));
+        await vectordb.insert(collection, documents);
+        totalChunks += batch.length;
+    }
+    upsertDocEntry({
+        library,
+        topic,
+        source,
+        collectionName: collection,
+        indexedAt: new Date().toISOString(),
+        ttlDays,
+        totalChunks,
+    });
+    return {
+        library,
+        topic,
+        source,
+        collectionName: collection,
+        totalChunks,
+        estimatedTokens: totalTokens,
+        durationMs: Date.now() - start,
+    };
+}
+//# sourceMappingURL=doc-indexer.js.map

package/dist/core/doc-searcher.d.ts

@@ -0,0 +1,13 @@
+import type { Embedding } from '../embedding/types.js';
+import type { VectorDB, SearchResult } from '../vectordb/types.js';
+export interface DocSearchResult extends SearchResult {
+    library: string;
+    topic: string;
+    source: string;
+    stale: boolean;
+}
+export declare function searchDocuments(query: string, embedding: Embedding, vectordb: VectorDB, options?: {
+    library?: string;
+    limit?: number;
+}): Promise<DocSearchResult[]>;
+//# sourceMappingURL=doc-searcher.d.ts.map