directory-indexer 0.0.12 → 0.0.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +82 -30
- package/bin/directory-indexer.js +10 -0
- package/dist/cli.js +64 -15
- package/dist/cli.js.map +1 -1
- package/dist/config.js +5 -2
- package/dist/config.js.map +1 -1
- package/dist/embedding.js.map +1 -1
- package/dist/indexing.js +40 -2
- package/dist/indexing.js.map +1 -1
- package/dist/storage.js +116 -6
- package/dist/storage.js.map +1 -1
- package/package.json +6 -4
package/README.md
CHANGED
|
@@ -9,6 +9,55 @@
|
|
|
9
9
|
|
|
10
10
|
Self-hosted semantic search for local files. Enable AI assistants to search your documents using vector embeddings and MCP integration.
|
|
11
11
|
|
|
12
|
+
## Quick Start
|
|
13
|
+
|
|
14
|
+
**Prerequisites:**
|
|
15
|
+
|
|
16
|
+
- **[Docker](https://docs.docker.com/get-docker/)** - For running Qdrant and Ollama _(skip if you already have them running natively)_
|
|
17
|
+
- **[Node.js 18+](https://nodejs.org/en/download/)** - Required for running directory-indexer
|
|
18
|
+
|
|
19
|
+
_Note: For native Qdrant and Ollama installation without Docker, see [Setup section](#setup)._
|
|
20
|
+
|
|
21
|
+
**1. Start Qdrant vector database** _(skip if already running)_
|
|
22
|
+
|
|
23
|
+
```bash
|
|
24
|
+
docker run -d --name qdrant -p 127.0.0.1:6333:6333 -v qdrant_storage:/qdrant/storage qdrant/qdrant
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
**2. Start Ollama embedding service** _(skip if already running)_
|
|
28
|
+
|
|
29
|
+
_Note: Docker Ollama won't use GPU acceleration. For better performance, consider [native installation](#2-embedding-provider)._
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
docker run -d --name ollama -p 127.0.0.1:11434:11434 -v ollama:/root/.ollama ollama/ollama
|
|
33
|
+
|
|
34
|
+
# Pull the embedding model
|
|
35
|
+
docker exec ollama ollama pull nomic-embed-text
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
**3. Index your directories**
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
npx directory-indexer index ~/Documents ~/Projects
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**4. Configure AI assistant** _(Claude Desktop, Cursor, Cline, Roo Code, Zed etc.)_
|
|
45
|
+
|
|
46
|
+
Add to your MCP configuration:
|
|
47
|
+
|
|
48
|
+
```json
|
|
49
|
+
{
|
|
50
|
+
"mcpServers": {
|
|
51
|
+
"directory-indexer": {
|
|
52
|
+
"command": "npx",
|
|
53
|
+
"args": ["directory-indexer", "serve"]
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
}
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Your AI assistant will automatically start the MCP server and can now search your indexed files.
|
|
60
|
+
|
|
12
61
|
## Setup
|
|
13
62
|
|
|
14
63
|
Directory Indexer runs locally on your machine or server. It uses an embedding provider (such as Ollama) to create vector embeddings of your files and stores them in a Qdrant vector database for fast semantic search. Both services can run remotely if needed.
|
|
@@ -62,7 +111,10 @@ export OPENAI_API_KEY="your-api-key-here"
|
|
|
62
111
|
Test your setup:
|
|
63
112
|
|
|
64
113
|
```bash
|
|
65
|
-
# Check Qdrant
|
|
114
|
+
# Check Qdrant Health
|
|
115
|
+
curl http://localhost:6333/healthz
|
|
116
|
+
|
|
117
|
+
# View collections
|
|
66
118
|
curl http://localhost:6333/collections
|
|
67
119
|
|
|
68
120
|
# Check Ollama
|
|
@@ -88,24 +140,42 @@ Configure with AI assistants (Claude Desktop, Cline, etc.) using npx:
|
|
|
88
140
|
}
|
|
89
141
|
```
|
|
90
142
|
|
|
91
|
-
Index your directories
|
|
143
|
+
Index your directories:
|
|
92
144
|
|
|
93
145
|
```bash
|
|
94
146
|
# Index your directories first
|
|
95
147
|
npx directory-indexer index /home/user/projects/docs /home/user/work/reports
|
|
96
|
-
|
|
97
|
-
# Start MCP server (usually done automatically by your AI assistant)
|
|
98
|
-
npx directory-indexer serve
|
|
99
148
|
```
|
|
100
149
|
|
|
150
|
+
**How it works:**
|
|
151
|
+
|
|
152
|
+
1. **MCP server starts automatically** - When your AI assistant connects, it launches the MCP server in the background
|
|
153
|
+
2. **Indexing runs independently** - You can index files before, during, or after MCP setup
|
|
154
|
+
3. **Search immediately available** - Your AI assistant can search files as soon as they're indexed
|
|
155
|
+
|
|
156
|
+
**Key point:** You don't need to wait for indexing to complete before using the MCP server. Index files as needed, and your AI assistant will immediately have access to search them.
|
|
157
|
+
|
|
101
158
|
### Using with AI Assistant
|
|
102
159
|
|
|
103
160
|
Once configured, your AI assistant can search your indexed documents semantically:
|
|
104
161
|
|
|
162
|
+
**Search by concept:**
|
|
163
|
+
|
|
105
164
|
- _"Find API authentication examples"_
|
|
106
|
-
- _"Show me
|
|
165
|
+
- _"Show me error handling patterns"_
|
|
166
|
+
- _"Find configuration for Redis"_
|
|
167
|
+
|
|
168
|
+
**Find similar content:**
|
|
169
|
+
|
|
170
|
+
- _"Show me incidents similar to this outage report"_ _(when you have an incident file open)_
|
|
171
|
+
- _"Find documentation like this API guide"_ _(when viewing an API doc)_
|
|
172
|
+
- _"What files are similar to my deployment script?"_
|
|
173
|
+
|
|
174
|
+
**Troubleshoot issues:**
|
|
175
|
+
|
|
107
176
|
- _"Find troubleshooting guides on SQL deadlocks"_
|
|
108
|
-
- _"
|
|
177
|
+
- _"Show me solutions for timeout errors"_
|
|
178
|
+
- _"Find debugging tips for performance issues"_
|
|
109
179
|
|
|
110
180
|
### Custom Configuration
|
|
111
181
|
|
|
@@ -133,7 +203,7 @@ For advanced users who prefer command-line usage, see [CLI Documentation](./docs
|
|
|
133
203
|
|
|
134
204
|
## Configuration
|
|
135
205
|
|
|
136
|
-
Environment variables
|
|
206
|
+
Environment variables (all optional):
|
|
137
207
|
|
|
138
208
|
```bash
|
|
139
209
|
# Data directory (default: ~/.directory-indexer)
|
|
@@ -142,6 +212,10 @@ export DIRECTORY_INDEXER_DATA_DIR="/opt/ai-knowledge-base"
|
|
|
142
212
|
# Service endpoints (defaults shown)
|
|
143
213
|
export QDRANT_ENDPOINT="http://localhost:6333"
|
|
144
214
|
export OLLAMA_ENDPOINT="http://localhost:11434"
|
|
215
|
+
|
|
216
|
+
# Optional API keys
|
|
217
|
+
export OPENAI_API_KEY="your-key-here"
|
|
218
|
+
export QDRANT_API_KEY="your-key-here"
|
|
145
219
|
```
|
|
146
220
|
|
|
147
221
|
For all configuration options, see [Environment Variables](./docs/design.md#environment-variables).
|
|
@@ -160,28 +234,6 @@ For all configuration options, see [Environment Variables](./docs/design.md#envi
|
|
|
160
234
|
- **[Contributing](docs/CONTRIBUTING.md)**: Development setup and guidelines
|
|
161
235
|
- **[Design](docs/design.md)**: Architecture and technical decisions
|
|
162
236
|
|
|
163
|
-
## Usage Examples
|
|
164
|
-
|
|
165
|
-
Once indexed, try these queries with your AI assistant:
|
|
166
|
-
|
|
167
|
-
**Search by concept:**
|
|
168
|
-
|
|
169
|
-
- _"Find API authentication examples"_
|
|
170
|
-
- _"Show me error handling patterns"_
|
|
171
|
-
- _"Find configuration for Redis"_
|
|
172
|
-
|
|
173
|
-
**Find similar content:**
|
|
174
|
-
|
|
175
|
-
- _"Show me incidents similar to this outage report"_ _(when you have an incident file open)_
|
|
176
|
-
- _"Find documentation like this API guide"_ _(when viewing an API doc)_
|
|
177
|
-
- _"What files are similar to my deployment script?"_
|
|
178
|
-
|
|
179
|
-
**Troubleshoot issues:**
|
|
180
|
-
|
|
181
|
-
- _"Find troubleshooting guides on SQL deadlocks"_
|
|
182
|
-
- _"Show me solutions for timeout errors"_
|
|
183
|
-
- _"Find debugging tips for performance issues"_
|
|
184
|
-
|
|
185
237
|
## License
|
|
186
238
|
|
|
187
239
|
MIT
|
package/dist/cli.js
CHANGED
|
@@ -1,5 +1,8 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
import { Command } from "commander";
|
|
3
|
+
import { fileURLToPath } from "url";
|
|
4
|
+
import { readFileSync } from "fs";
|
|
5
|
+
import { dirname, join } from "path";
|
|
3
6
|
import { indexDirectories } from "./indexing.js";
|
|
4
7
|
import { getFileContent, findSimilarFiles, searchContent } from "./search.js";
|
|
5
8
|
import { loadConfig } from "./config.js";
|
|
@@ -7,6 +10,10 @@ import { getIndexStatus } from "./storage.js";
|
|
|
7
10
|
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
|
|
8
11
|
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
|
9
12
|
import { ListToolsRequestSchema, CallToolRequestSchema } from "@modelcontextprotocol/sdk/types.js";
|
|
13
|
+
const __dirname$1 = dirname(fileURLToPath(import.meta.url));
|
|
14
|
+
const packageJsonPath$1 = join(__dirname$1, "../package.json");
|
|
15
|
+
const packageJson$1 = JSON.parse(readFileSync(packageJsonPath$1, "utf-8"));
|
|
16
|
+
const VERSION$1 = packageJson$1.version;
|
|
10
17
|
const MCP_TOOLS = [
|
|
11
18
|
{
|
|
12
19
|
name: "index",
|
|
@@ -91,7 +98,7 @@ async function startMcpServer(config) {
|
|
|
91
98
|
const server = new Server(
|
|
92
99
|
{
|
|
93
100
|
name: "directory-indexer",
|
|
94
|
-
version:
|
|
101
|
+
version: VERSION$1
|
|
95
102
|
},
|
|
96
103
|
{
|
|
97
104
|
capabilities: {
|
|
@@ -173,7 +180,7 @@ async function startMcpServer(config) {
|
|
|
173
180
|
type: "text",
|
|
174
181
|
text: JSON.stringify({
|
|
175
182
|
name: "directory-indexer",
|
|
176
|
-
version:
|
|
183
|
+
version: VERSION$1,
|
|
177
184
|
status
|
|
178
185
|
}, null, 2)
|
|
179
186
|
}
|
|
@@ -202,9 +209,13 @@ async function startMcpServer(config) {
|
|
|
202
209
|
console.error("MCP server started successfully");
|
|
203
210
|
}
|
|
204
211
|
}
|
|
212
|
+
const __dirname = dirname(fileURLToPath(import.meta.url));
|
|
213
|
+
const packageJsonPath = join(__dirname, "../package.json");
|
|
214
|
+
const packageJson = JSON.parse(readFileSync(packageJsonPath, "utf-8"));
|
|
215
|
+
const VERSION = packageJson.version;
|
|
205
216
|
async function main() {
|
|
206
217
|
const program = new Command();
|
|
207
|
-
program.name("directory-indexer").description("AI-powered directory indexing with semantic search").version(
|
|
218
|
+
program.name("directory-indexer").description("AI-powered directory indexing with semantic search").version(VERSION);
|
|
208
219
|
program.command("index").description("Index directories for semantic search").argument("<paths...>", "Directory paths to index").option("-v, --verbose", "Enable verbose logging").action(async (paths, options) => {
|
|
209
220
|
try {
|
|
210
221
|
const config = await loadConfig({ verbose: options.verbose });
|
|
@@ -285,21 +296,62 @@ async function main() {
|
|
|
285
296
|
try {
|
|
286
297
|
await loadConfig({ verbose: options.verbose });
|
|
287
298
|
const status = await getIndexStatus();
|
|
288
|
-
console.log("
|
|
289
|
-
console.log(
|
|
290
|
-
console.log(
|
|
291
|
-
console.log(
|
|
292
|
-
console.log(`
|
|
293
|
-
console.log(`
|
|
299
|
+
console.log("Directory Indexer Status Report");
|
|
300
|
+
console.log("=====================================");
|
|
301
|
+
console.log("");
|
|
302
|
+
console.log("OVERVIEW:");
|
|
303
|
+
console.log(` • ${status.directoriesIndexed} directories have been indexed`);
|
|
304
|
+
console.log(` • ${status.filesIndexed} files processed for semantic search`);
|
|
305
|
+
console.log(` • ${status.chunksIndexed} text chunks available for AI search`);
|
|
306
|
+
console.log(` • Database storage: ${status.databaseSize}`);
|
|
307
|
+
console.log(` • Most recent indexing: ${status.lastIndexed || "No indexing performed yet"}`);
|
|
294
308
|
if (status.errors.length > 0) {
|
|
295
|
-
console.log(`
|
|
309
|
+
console.log(` • Processing errors encountered: ${status.errors.length}`);
|
|
296
310
|
if (options.verbose) {
|
|
297
|
-
console.log("
|
|
311
|
+
console.log("");
|
|
312
|
+
console.log("RECENT ERRORS:");
|
|
298
313
|
status.errors.slice(0, 5).forEach((error) => {
|
|
299
|
-
console.log(`
|
|
314
|
+
console.log(` - ${error}`);
|
|
300
315
|
});
|
|
301
316
|
}
|
|
302
317
|
}
|
|
318
|
+
console.log("");
|
|
319
|
+
console.log("INDEXED DIRECTORIES:");
|
|
320
|
+
if (status.directories.length === 0) {
|
|
321
|
+
console.log(" No directories have been indexed yet.");
|
|
322
|
+
console.log(' Run "directory-indexer index <path>" to start indexing.');
|
|
323
|
+
} else {
|
|
324
|
+
status.directories.forEach((dir) => {
|
|
325
|
+
console.log("");
|
|
326
|
+
console.log(` Directory: ${dir.path}`);
|
|
327
|
+
console.log(` • Indexing status: ${dir.status}`);
|
|
328
|
+
console.log(` • Files processed: ${dir.filesCount}`);
|
|
329
|
+
console.log(` • Searchable chunks: ${dir.chunksCount}`);
|
|
330
|
+
console.log(` • Last indexed: ${dir.lastIndexed || "Never completed"}`);
|
|
331
|
+
if (dir.errors.length > 0) {
|
|
332
|
+
console.log(` • Files with errors: ${dir.errors.length}`);
|
|
333
|
+
if (options.verbose) {
|
|
334
|
+
console.log(" • Recent errors:");
|
|
335
|
+
dir.errors.slice(0, 3).forEach((error) => {
|
|
336
|
+
console.log(` - ${error}`);
|
|
337
|
+
});
|
|
338
|
+
}
|
|
339
|
+
}
|
|
340
|
+
});
|
|
341
|
+
}
|
|
342
|
+
if (!status.qdrantConsistency.isConsistent) {
|
|
343
|
+
console.log("");
|
|
344
|
+
console.log("SYSTEM STATUS:");
|
|
345
|
+
status.qdrantConsistency.issues.forEach((issue) => {
|
|
346
|
+
console.log(` • ${issue}`);
|
|
347
|
+
});
|
|
348
|
+
console.log("");
|
|
349
|
+
console.log("ℹ️ Note: Status messages above may be normal during setup or active indexing.");
|
|
350
|
+
} else {
|
|
351
|
+
console.log("");
|
|
352
|
+
console.log("SYSTEM STATUS:");
|
|
353
|
+
console.log(" • All systems operational - ready for AI-powered search");
|
|
354
|
+
}
|
|
303
355
|
} catch (error) {
|
|
304
356
|
console.error("Error getting status:", error);
|
|
305
357
|
process.exit(1);
|
|
@@ -307,9 +359,6 @@ async function main() {
|
|
|
307
359
|
});
|
|
308
360
|
await program.parseAsync();
|
|
309
361
|
}
|
|
310
|
-
if (import.meta.url === `file://${process.argv[1]}`) {
|
|
311
|
-
main().catch(console.error);
|
|
312
|
-
}
|
|
313
362
|
export {
|
|
314
363
|
main
|
|
315
364
|
};
|
package/dist/cli.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.js","sources":["../src/mcp.ts","../src/cli.ts"],"sourcesContent":["import { Server } from '@modelcontextprotocol/sdk/server/index.js';\nimport { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';\nimport { \n CallToolRequestSchema, \n ListToolsRequestSchema,\n Tool\n} from '@modelcontextprotocol/sdk/types.js';\nimport { Config } from './config.js';\nimport { indexDirectories } from './indexing.js';\nimport { searchContent, findSimilarFiles, getFileContent } from './search.js';\nimport { getIndexStatus } from './storage.js';\n\nconst MCP_TOOLS: Tool[] = [\n {\n name: 'index',\n description: 'Index directories for semantic search',\n inputSchema: {\n type: 'object',\n properties: {\n directory_path: {\n type: 'string',\n description: 'Comma-separated list of directory paths to index'\n }\n },\n required: ['directory_path']\n }\n },\n {\n name: 'search',\n description: 'Search indexed content semantically',\n inputSchema: {\n type: 'object',\n properties: {\n query: {\n type: 'string',\n description: 'Search query'\n },\n limit: {\n type: 'number',\n description: 'Maximum number of results (default: 10)',\n default: 10\n }\n },\n required: ['query']\n }\n },\n {\n name: 'similar_files',\n description: 'Find files similar to a given file',\n inputSchema: {\n type: 'object',\n properties: {\n file_path: {\n type: 'string',\n description: 'Path to the file to find similar files for'\n },\n limit: {\n type: 'number',\n description: 'Maximum number of results (default: 10)',\n default: 10\n }\n },\n required: ['file_path']\n }\n },\n {\n name: 'get_content',\n description: 'Get file content',\n inputSchema: {\n type: 'object',\n properties: {\n file_path: {\n type: 'string',\n description: 'Path to the file to retrieve'\n },\n chunks: {\n type: 'string',\n description: 'Optional chunk range (e.g., \"2-5\")'\n }\n },\n required: ['file_path']\n }\n },\n {\n name: 'server_info',\n description: 'Get server information and status',\n inputSchema: {\n type: 'object',\n properties: {}\n }\n }\n];\n\nexport async function startMcpServer(config: Config): Promise<void> {\n const server = new Server(\n {\n name: 'directory-indexer',\n version: '0.0.10'\n },\n {\n capabilities: {\n tools: {}\n }\n }\n );\n\n server.setRequestHandler(ListToolsRequestSchema, async () => {\n return {\n tools: MCP_TOOLS\n };\n });\n\n server.setRequestHandler(CallToolRequestSchema, async (request) => {\n const { name, arguments: args } = request.params;\n\n try {\n switch (name) {\n case 'index': {\n if (!args || typeof args.directory_path !== 'string') {\n throw new Error('directory_path is required');\n }\n const paths = args.directory_path.split(',').map((p: string) => p.trim());\n const result = await indexDirectories(paths, config);\n return {\n content: [\n {\n type: 'text',\n text: `Indexed ${result.indexed} files, skipped ${result.skipped} files, ${result.errors.length} errors`\n }\n ]\n };\n }\n\n case 'search': {\n if (!args || typeof args.query !== 'string') {\n throw new Error('query is required');\n }\n const results = await searchContent(args.query, { limit: (args.limit as number) || 10 });\n return {\n content: [\n {\n type: 'text',\n text: JSON.stringify(results, null, 2)\n }\n ]\n };\n }\n\n case 'similar_files': {\n if (!args || typeof args.file_path !== 'string') {\n throw new Error('file_path is required');\n }\n const results = await findSimilarFiles(args.file_path, (args.limit as number) || 10);\n return {\n content: [\n {\n type: 'text',\n text: JSON.stringify(results, null, 2)\n }\n ]\n };\n }\n\n case 'get_content': {\n if (!args || typeof args.file_path !== 'string') {\n throw new Error('file_path is required');\n }\n const content = await getFileContent(args.file_path, args.chunks as string);\n return {\n content: [\n {\n type: 'text',\n text: content\n }\n ]\n };\n }\n\n case 'server_info': {\n const status = await getIndexStatus();\n return {\n content: [\n {\n type: 'text',\n text: JSON.stringify({\n name: 'directory-indexer',\n version: '0.0.10',\n status: status\n }, null, 2)\n }\n ]\n };\n }\n\n default:\n throw new Error(`Unknown tool: ${name}`);\n }\n } catch (error) {\n const errorMessage = error instanceof Error ? error.message : 'Unknown error';\n return {\n content: [\n {\n type: 'text',\n text: `Error: ${errorMessage}`\n }\n ],\n isError: true\n };\n }\n });\n\n const transport = new StdioServerTransport();\n await server.connect(transport);\n \n if (config.verbose) {\n console.error('MCP server started successfully');\n }\n}","#!/usr/bin/env node\n\nimport { Command } from 'commander';\nimport { indexDirectories } from './indexing.js';\nimport { searchContent, findSimilarFiles, getFileContent } from './search.js';\nimport { loadConfig } from './config.js';\nimport { getIndexStatus } from './storage.js';\nimport { startMcpServer } from './mcp.js';\n\nexport async function main() {\n const program = new Command();\n \n program\n .name('directory-indexer')\n .description('AI-powered directory indexing with semantic search')\n .version('0.0.10');\n\n program\n .command('index')\n .description('Index directories for semantic search')\n .argument('<paths...>', 'Directory paths to index')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (paths: string[], options) => {\n try {\n const config = await loadConfig({ verbose: options.verbose });\n console.log(`Indexing ${paths.length} directories...`);\n const result = await indexDirectories(paths, config);\n console.log(`Indexed ${result.indexed} files, skipped ${result.skipped} files, ${result.errors.length} errors`);\n if (result.errors.length > 0 && config.verbose) {\n console.log('Errors:', result.errors);\n }\n } catch (error) {\n console.error('Error indexing directories:', error);\n process.exit(1);\n }\n });\n\n program\n .command('search')\n .description('Search indexed content semantically')\n .argument('<query>', 'Search query')\n .option('-l, --limit <number>', 'Maximum number of results', '10')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (query: string, options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const results = await searchContent(query, { limit: parseInt(options.limit) });\n \n if (results.length === 0) {\n console.log('No results found');\n return;\n }\n\n console.log(`Found ${results.length} results:\\n`);\n results.forEach((result, index) => {\n console.log(`${index + 1}. ${result.filePath}`);\n console.log(` Score: ${result.score.toFixed(3)}`);\n if (result.content) {\n console.log(` Content: ${result.content.substring(0, 150)}...`);\n }\n console.log();\n });\n } catch (error) {\n console.error('Error searching content:', error);\n process.exit(1);\n }\n });\n\n program\n .command('similar')\n .description('Find files similar to a given file')\n .argument('<file>', 'File path to find similar files for')\n .option('-l, --limit <number>', 'Maximum number of results', '10')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (filePath: string, options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const results = await findSimilarFiles(filePath, parseInt(options.limit));\n \n if (results.length === 0) {\n console.log('No similar files found');\n return;\n }\n\n console.log(`Found ${results.length} similar files:\\n`);\n results.forEach((result, index) => {\n console.log(`${index + 1}. ${result.filePath}`);\n console.log(` Similarity: ${result.score.toFixed(3)}`);\n console.log();\n });\n } catch (error) {\n console.error('Error finding similar files:', error);\n process.exit(1);\n }\n });\n\n program\n .command('get')\n .description('Get file content')\n .argument('<file>', 'File path to retrieve')\n .option('-c, --chunks <range>', 'Chunk range (e.g., \"2-5\")')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (filePath: string, options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const content = await getFileContent(filePath, options.chunks);\n console.log(content);\n } catch (error) {\n console.error('Error getting file content:', error);\n process.exit(1);\n }\n });\n\n program\n .command('serve')\n .description('Start MCP server')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (options) => {\n try {\n const config = await loadConfig({ verbose: options.verbose });\n await startMcpServer(config);\n } catch (error) {\n console.error('Error starting MCP server:', error);\n process.exit(1);\n }\n });\n\n program\n .command('status')\n .description('Show indexing status')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const status = await getIndexStatus();\n \n console.log('Indexing Status:');\n console.log(` Directories indexed: ${status.directoriesIndexed}`);\n console.log(` Files indexed: ${status.filesIndexed}`);\n console.log(` Total chunks: ${status.chunksIndexed}`);\n console.log(` Database size: ${status.databaseSize}`);\n console.log(` Last indexed: ${status.lastIndexed || 'Never'}`);\n \n if (status.errors.length > 0) {\n console.log(` Errors: ${status.errors.length}`);\n if (options.verbose) {\n console.log(' Recent errors:');\n status.errors.slice(0, 5).forEach(error => {\n console.log(` - ${error}`);\n });\n }\n }\n } catch (error) {\n console.error('Error getting status:', error);\n process.exit(1);\n }\n });\n\n await program.parseAsync();\n}\n\nif (import.meta.url === `file://${process.argv[1]}`) {\n main().catch(console.error);\n}"],"names":[],"mappings":";;;;;;;;;AAYA,MAAM,YAAoB;AAAA,EACxB;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,gBAAgB;AAAA,UACd,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,MACf;AAAA,MAEF,UAAU,CAAC,gBAAgB;AAAA,IAAA;AAAA,EAC7B;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,OAAO;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,QAEf,OAAO;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,UACb,SAAS;AAAA,QAAA;AAAA,MACX;AAAA,MAEF,UAAU,CAAC,OAAO;AAAA,IAAA;AAAA,EACpB;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,WAAW;AAAA,UACT,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,QAEf,OAAO;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,UACb,SAAS;AAAA,QAAA;AAAA,MACX;AAAA,MAEF,UAAU,CAAC,WAAW;AAAA,IAAA;AAAA,EACxB;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,WAAW;AAAA,UACT,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,QAEf,QAAQ;AAAA,UACN,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,MACf;AAAA,MAEF,UAAU,CAAC,WAAW;AAAA,IAAA;AAAA,EACxB;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY,CAAA;AAAA,IAAC;AAAA,EACf;AAEJ;AAEA,eAAsB,eAAe,QAA+B;AAClE,QAAM,SAAS,IAAI;AAAA,IACjB;AAAA,MACE,MAAM;AAAA,MACN,SAAS;AAAA,IAAA;AAAA,IAEX;AAAA,MACE,cAAc;AAAA,QACZ,OAAO,CAAA;AAAA,MAAC;AAAA,IACV;AAAA,EACF;AAGF,SAAO,kBAAkB,wBAAwB,YAAY;AAC3D,WAAO;AAAA,MACL,OAAO;AAAA,IAAA;AAAA,EAEX,CAAC;AAED,SAAO,kBAAkB,uBAAuB,OAAO,YAAY;AACjE,UAAM,EAAE,MAAM,WAAW,KAAA,IAAS,QAAQ;AAE1C,QAAI;AACF,cAAQ,MAAA;AAAA,QACN,KAAK,SAAS;AACZ,cAAI,CAAC,QAAQ,OAAO,KAAK,mBAAmB,UAAU;AACpD,kBAAM,IAAI,MAAM,4BAA4B;AAAA,UAC9C;AACA,gBAAM,QAAQ,KAAK,eAAe,MAAM,GAAG,EAAE,IAAI,CAAC,MAAc,EAAE,KAAA,CAAM;AACxE,gBAAM,SAAS,MAAM,iBAAiB,OAAO,MAAM;AACnD,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,WAAW,OAAO,OAAO,mBAAmB,OAAO,OAAO,WAAW,OAAO,OAAO,MAAM;AAAA,cAAA;AAAA,YACjG;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,UAAU;AACb,cAAI,CAAC,QAAQ,OAAO,KAAK,UAAU,UAAU;AAC3C,kBAAM,IAAI,MAAM,mBAAmB;AAAA,UACrC;AACA,gBAAM,UAAU,MAAM,cAAc,KAAK,OAAO,EAAE,OAAQ,KAAK,SAAoB,IAAI;AACvF,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,KAAK,UAAU,SAAS,MAAM,CAAC;AAAA,cAAA;AAAA,YACvC;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,iBAAiB;AACpB,cAAI,CAAC,QAAQ,OAAO,KAAK,cAAc,UAAU;AAC/C,kBAAM,IAAI,MAAM,uBAAuB;AAAA,UACzC;AACA,gBAAM,UAAU,MAAM,iBAAiB,KAAK,WAAY,KAAK,SAAoB,EAAE;AACnF,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,KAAK,UAAU,SAAS,MAAM,CAAC;AAAA,cAAA;AAAA,YACvC;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,eAAe;AAClB,cAAI,CAAC,QAAQ,OAAO,KAAK,cAAc,UAAU;AAC/C,kBAAM,IAAI,MAAM,uBAAuB;AAAA,UACzC;AACA,gBAAM,UAAU,MAAM,eAAe,KAAK,WAAW,KAAK,MAAgB;AAC1E,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM;AAAA,cAAA;AAAA,YACR;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,eAAe;AAClB,gBAAM,SAAS,MAAM,eAAA;AACrB,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,KAAK,UAAU;AAAA,kBACnB,MAAM;AAAA,kBACN,SAAS;AAAA,kBACT;AAAA,gBAAA,GACC,MAAM,CAAC;AAAA,cAAA;AAAA,YACZ;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA;AACE,gBAAM,IAAI,MAAM,iBAAiB,IAAI,EAAE;AAAA,MAAA;AAAA,IAE7C,SAAS,OAAO;AACd,YAAM,eAAe,iBAAiB,QAAQ,MAAM,UAAU;AAC9D,aAAO;AAAA,QACL,SAAS;AAAA,UACP;AAAA,YACE,MAAM;AAAA,YACN,MAAM,UAAU,YAAY;AAAA,UAAA;AAAA,QAC9B;AAAA,QAEF,SAAS;AAAA,MAAA;AAAA,IAEb;AAAA,EACF,CAAC;AAED,QAAM,YAAY,IAAI,qBAAA;AACtB,QAAM,OAAO,QAAQ,SAAS;AAE9B,MAAI,OAAO,SAAS;AAClB,YAAQ,MAAM,iCAAiC;AAAA,EACjD;AACF;AChNA,eAAsB,OAAO;AAC3B,QAAM,UAAU,IAAI,QAAA;AAEpB,UACG,KAAK,mBAAmB,EACxB,YAAY,oDAAoD,EAChE,QAAQ,QAAQ;AAEnB,UACG,QAAQ,OAAO,EACf,YAAY,uCAAuC,EACnD,SAAS,cAAc,0BAA0B,EACjD,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,OAAiB,YAAY;AAC1C,QAAI;AACF,YAAM,SAAS,MAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC5D,cAAQ,IAAI,YAAY,MAAM,MAAM,iBAAiB;AACrD,YAAM,SAAS,MAAM,iBAAiB,OAAO,MAAM;AACnD,cAAQ,IAAI,WAAW,OAAO,OAAO,mBAAmB,OAAO,OAAO,WAAW,OAAO,OAAO,MAAM,SAAS;AAC9G,UAAI,OAAO,OAAO,SAAS,KAAK,OAAO,SAAS;AAC9C,gBAAQ,IAAI,WAAW,OAAO,MAAM;AAAA,MACtC;AAAA,IACF,SAAS,OAAO;AACd,cAAQ,MAAM,+BAA+B,KAAK;AAClD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,QAAQ,EAChB,YAAY,qCAAqC,EACjD,SAAS,WAAW,cAAc,EAClC,OAAO,wBAAwB,6BAA6B,IAAI,EAChE,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,OAAe,YAAY;AACxC,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,UAAU,MAAM,cAAc,OAAO,EAAE,OAAO,SAAS,QAAQ,KAAK,GAAG;AAE7E,UAAI,QAAQ,WAAW,GAAG;AACxB,gBAAQ,IAAI,kBAAkB;AAC9B;AAAA,MACF;AAEA,cAAQ,IAAI,SAAS,QAAQ,MAAM;AAAA,CAAa;AAChD,cAAQ,QAAQ,CAAC,QAAQ,UAAU;AACjC,gBAAQ,IAAI,GAAG,QAAQ,CAAC,KAAK,OAAO,QAAQ,EAAE;AAC9C,gBAAQ,IAAI,aAAa,OAAO,MAAM,QAAQ,CAAC,CAAC,EAAE;AAClD,YAAI,OAAO,SAAS;AAClB,kBAAQ,IAAI,eAAe,OAAO,QAAQ,UAAU,GAAG,GAAG,CAAC,KAAK;AAAA,QAClE;AACA,gBAAQ,IAAA;AAAA,MACV,CAAC;AAAA,IACH,SAAS,OAAO;AACd,cAAQ,MAAM,4BAA4B,KAAK;AAC/C,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,SAAS,EACjB,YAAY,oCAAoC,EAChD,SAAS,UAAU,qCAAqC,EACxD,OAAO,wBAAwB,6BAA6B,IAAI,EAChE,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,UAAkB,YAAY;AAC3C,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,UAAU,MAAM,iBAAiB,UAAU,SAAS,QAAQ,KAAK,CAAC;AAExE,UAAI,QAAQ,WAAW,GAAG;AACxB,gBAAQ,IAAI,wBAAwB;AACpC;AAAA,MACF;AAEA,cAAQ,IAAI,SAAS,QAAQ,MAAM;AAAA,CAAmB;AACtD,cAAQ,QAAQ,CAAC,QAAQ,UAAU;AACjC,gBAAQ,IAAI,GAAG,QAAQ,CAAC,KAAK,OAAO,QAAQ,EAAE;AAC9C,gBAAQ,IAAI,kBAAkB,OAAO,MAAM,QAAQ,CAAC,CAAC,EAAE;AACvD,gBAAQ,IAAA;AAAA,MACV,CAAC;AAAA,IACH,SAAS,OAAO;AACd,cAAQ,MAAM,gCAAgC,KAAK;AACnD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,KAAK,EACb,YAAY,kBAAkB,EAC9B,SAAS,UAAU,uBAAuB,EAC1C,OAAO,wBAAwB,2BAA2B,EAC1D,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,UAAkB,YAAY;AAC3C,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,UAAU,MAAM,eAAe,UAAU,QAAQ,MAAM;AAC7D,cAAQ,IAAI,OAAO;AAAA,IACrB,SAAS,OAAO;AACd,cAAQ,MAAM,+BAA+B,KAAK;AAClD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,OAAO,EACf,YAAY,kBAAkB,EAC9B,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,YAAY;AACzB,QAAI;AACF,YAAM,SAAS,MAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC5D,YAAM,eAAe,MAAM;AAAA,IAC7B,SAAS,OAAO;AACd,cAAQ,MAAM,8BAA8B,KAAK;AACjD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,QAAQ,EAChB,YAAY,sBAAsB,EAClC,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,YAAY;AACzB,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,SAAS,MAAM,eAAA;AAErB,cAAQ,IAAI,kBAAkB;AAC9B,cAAQ,IAAI,0BAA0B,OAAO,kBAAkB,EAAE;AACjE,cAAQ,IAAI,oBAAoB,OAAO,YAAY,EAAE;AACrD,cAAQ,IAAI,mBAAmB,OAAO,aAAa,EAAE;AACrD,cAAQ,IAAI,oBAAoB,OAAO,YAAY,EAAE;AACrD,cAAQ,IAAI,mBAAmB,OAAO,eAAe,OAAO,EAAE;AAE9D,UAAI,OAAO,OAAO,SAAS,GAAG;AAC5B,gBAAQ,IAAI,aAAa,OAAO,OAAO,MAAM,EAAE;AAC/C,YAAI,QAAQ,SAAS;AACnB,kBAAQ,IAAI,kBAAkB;AAC9B,iBAAO,OAAO,MAAM,GAAG,CAAC,EAAE,QAAQ,CAAA,UAAS;AACzC,oBAAQ,IAAI,SAAS,KAAK,EAAE;AAAA,UAC9B,CAAC;AAAA,QACH;AAAA,MACF;AAAA,IACF,SAAS,OAAO;AACd,cAAQ,MAAM,yBAAyB,KAAK;AAC5C,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,QAAM,QAAQ,WAAA;AAChB;AAEA,IAAI,YAAY,QAAQ,UAAU,QAAQ,KAAK,CAAC,CAAC,IAAI;AACnD,SAAO,MAAM,QAAQ,KAAK;AAC5B;"}
|
|
1
|
+
{"version":3,"file":"cli.js","sources":["../src/mcp.ts","../src/cli.ts"],"sourcesContent":["import { Server } from '@modelcontextprotocol/sdk/server/index.js';\nimport { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';\nimport { \n CallToolRequestSchema, \n ListToolsRequestSchema,\n Tool\n} from '@modelcontextprotocol/sdk/types.js';\nimport { readFileSync } from 'fs';\nimport { join, dirname } from 'path';\nimport { fileURLToPath } from 'url';\nimport { Config } from './config.js';\nimport { indexDirectories } from './indexing.js';\nimport { searchContent, findSimilarFiles, getFileContent } from './search.js';\nimport { getIndexStatus } from './storage.js';\n\n// Read version from package.json\nconst __dirname = dirname(fileURLToPath(import.meta.url));\nconst packageJsonPath = join(__dirname, '../package.json');\nconst packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));\nconst VERSION = packageJson.version;\n\nconst MCP_TOOLS: Tool[] = [\n {\n name: 'index',\n description: 'Index directories for semantic search',\n inputSchema: {\n type: 'object',\n properties: {\n directory_path: {\n type: 'string',\n description: 'Comma-separated list of directory paths to index'\n }\n },\n required: ['directory_path']\n }\n },\n {\n name: 'search',\n description: 'Search indexed content semantically',\n inputSchema: {\n type: 'object',\n properties: {\n query: {\n type: 'string',\n description: 'Search query'\n },\n limit: {\n type: 'number',\n description: 'Maximum number of results (default: 10)',\n default: 10\n }\n },\n required: ['query']\n }\n },\n {\n name: 'similar_files',\n description: 'Find files similar to a given file',\n inputSchema: {\n type: 'object',\n properties: {\n file_path: {\n type: 'string',\n description: 'Path to the file to find similar files for'\n },\n limit: {\n type: 'number',\n description: 'Maximum number of results (default: 10)',\n default: 10\n }\n },\n required: ['file_path']\n }\n },\n {\n name: 'get_content',\n description: 'Get file content',\n inputSchema: {\n type: 'object',\n properties: {\n file_path: {\n type: 'string',\n description: 'Path to the file to retrieve'\n },\n chunks: {\n type: 'string',\n description: 'Optional chunk range (e.g., \"2-5\")'\n }\n },\n required: ['file_path']\n }\n },\n {\n name: 'server_info',\n description: 'Get server information and status',\n inputSchema: {\n type: 'object',\n properties: {}\n }\n }\n];\n\nexport async function startMcpServer(config: Config): Promise<void> {\n const server = new Server(\n {\n name: 'directory-indexer',\n version: VERSION\n },\n {\n capabilities: {\n tools: {}\n }\n }\n );\n\n server.setRequestHandler(ListToolsRequestSchema, async () => {\n return {\n tools: MCP_TOOLS\n };\n });\n\n server.setRequestHandler(CallToolRequestSchema, async (request) => {\n const { name, arguments: args } = request.params;\n\n try {\n switch (name) {\n case 'index': {\n if (!args || typeof args.directory_path !== 'string') {\n throw new Error('directory_path is required');\n }\n const paths = args.directory_path.split(',').map((p: string) => p.trim());\n const result = await indexDirectories(paths, config);\n return {\n content: [\n {\n type: 'text',\n text: `Indexed ${result.indexed} files, skipped ${result.skipped} files, ${result.errors.length} errors`\n }\n ]\n };\n }\n\n case 'search': {\n if (!args || typeof args.query !== 'string') {\n throw new Error('query is required');\n }\n const results = await searchContent(args.query, { limit: (args.limit as number) || 10 });\n return {\n content: [\n {\n type: 'text',\n text: JSON.stringify(results, null, 2)\n }\n ]\n };\n }\n\n case 'similar_files': {\n if (!args || typeof args.file_path !== 'string') {\n throw new Error('file_path is required');\n }\n const results = await findSimilarFiles(args.file_path, (args.limit as number) || 10);\n return {\n content: [\n {\n type: 'text',\n text: JSON.stringify(results, null, 2)\n }\n ]\n };\n }\n\n case 'get_content': {\n if (!args || typeof args.file_path !== 'string') {\n throw new Error('file_path is required');\n }\n const content = await getFileContent(args.file_path, args.chunks as string);\n return {\n content: [\n {\n type: 'text',\n text: content\n }\n ]\n };\n }\n\n case 'server_info': {\n const status = await getIndexStatus();\n return {\n content: [\n {\n type: 'text',\n text: JSON.stringify({\n name: 'directory-indexer',\n version: VERSION,\n status: status\n }, null, 2)\n }\n ]\n };\n }\n\n default:\n throw new Error(`Unknown tool: ${name}`);\n }\n } catch (error) {\n const errorMessage = error instanceof Error ? error.message : 'Unknown error';\n return {\n content: [\n {\n type: 'text',\n text: `Error: ${errorMessage}`\n }\n ],\n isError: true\n };\n }\n });\n\n const transport = new StdioServerTransport();\n await server.connect(transport);\n \n if (config.verbose) {\n console.error('MCP server started successfully');\n }\n}","#!/usr/bin/env node\n\nimport { Command } from 'commander';\nimport { fileURLToPath } from 'url';\nimport { readFileSync } from 'fs';\nimport { join, dirname } from 'path';\nimport { indexDirectories } from './indexing.js';\nimport { searchContent, findSimilarFiles, getFileContent } from './search.js';\nimport { loadConfig } from './config.js';\nimport { getIndexStatus } from './storage.js';\nimport { startMcpServer } from './mcp.js';\n\n// Read version from package.json\nconst __dirname = dirname(fileURLToPath(import.meta.url));\nconst packageJsonPath = join(__dirname, '../package.json');\nconst packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));\nconst VERSION = packageJson.version;\n\nexport async function main() {\n const program = new Command();\n \n program\n .name('directory-indexer')\n .description('AI-powered directory indexing with semantic search')\n .version(VERSION);\n\n program\n .command('index')\n .description('Index directories for semantic search')\n .argument('<paths...>', 'Directory paths to index')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (paths: string[], options) => {\n try {\n const config = await loadConfig({ verbose: options.verbose });\n console.log(`Indexing ${paths.length} directories...`);\n const result = await indexDirectories(paths, config);\n console.log(`Indexed ${result.indexed} files, skipped ${result.skipped} files, ${result.errors.length} errors`);\n if (result.errors.length > 0 && config.verbose) {\n console.log('Errors:', result.errors);\n }\n } catch (error) {\n console.error('Error indexing directories:', error);\n process.exit(1);\n }\n });\n\n program\n .command('search')\n .description('Search indexed content semantically')\n .argument('<query>', 'Search query')\n .option('-l, --limit <number>', 'Maximum number of results', '10')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (query: string, options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const results = await searchContent(query, { limit: parseInt(options.limit) });\n \n if (results.length === 0) {\n console.log('No results found');\n return;\n }\n\n console.log(`Found ${results.length} results:\\n`);\n results.forEach((result, index) => {\n console.log(`${index + 1}. ${result.filePath}`);\n console.log(` Score: ${result.score.toFixed(3)}`);\n if (result.content) {\n console.log(` Content: ${result.content.substring(0, 150)}...`);\n }\n console.log();\n });\n } catch (error) {\n console.error('Error searching content:', error);\n process.exit(1);\n }\n });\n\n program\n .command('similar')\n .description('Find files similar to a given file')\n .argument('<file>', 'File path to find similar files for')\n .option('-l, --limit <number>', 'Maximum number of results', '10')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (filePath: string, options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const results = await findSimilarFiles(filePath, parseInt(options.limit));\n \n if (results.length === 0) {\n console.log('No similar files found');\n return;\n }\n\n console.log(`Found ${results.length} similar files:\\n`);\n results.forEach((result, index) => {\n console.log(`${index + 1}. ${result.filePath}`);\n console.log(` Similarity: ${result.score.toFixed(3)}`);\n console.log();\n });\n } catch (error) {\n console.error('Error finding similar files:', error);\n process.exit(1);\n }\n });\n\n program\n .command('get')\n .description('Get file content')\n .argument('<file>', 'File path to retrieve')\n .option('-c, --chunks <range>', 'Chunk range (e.g., \"2-5\")')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (filePath: string, options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const content = await getFileContent(filePath, options.chunks);\n console.log(content);\n } catch (error) {\n console.error('Error getting file content:', error);\n process.exit(1);\n }\n });\n\n program\n .command('serve')\n .description('Start MCP server')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (options) => {\n try {\n const config = await loadConfig({ verbose: options.verbose });\n await startMcpServer(config);\n } catch (error) {\n console.error('Error starting MCP server:', error);\n process.exit(1);\n }\n });\n\n program\n .command('status')\n .description('Show indexing status')\n .option('-v, --verbose', 'Enable verbose logging')\n .action(async (options) => {\n try {\n await loadConfig({ verbose: options.verbose });\n const status = await getIndexStatus();\n \n console.log('Directory Indexer Status Report');\n console.log('=====================================');\n console.log('');\n console.log('OVERVIEW:');\n console.log(` • ${status.directoriesIndexed} directories have been indexed`);\n console.log(` • ${status.filesIndexed} files processed for semantic search`);\n console.log(` • ${status.chunksIndexed} text chunks available for AI search`);\n console.log(` • Database storage: ${status.databaseSize}`);\n console.log(` • Most recent indexing: ${status.lastIndexed || 'No indexing performed yet'}`);\n \n if (status.errors.length > 0) {\n console.log(` • Processing errors encountered: ${status.errors.length}`);\n if (options.verbose) {\n console.log('');\n console.log('RECENT ERRORS:');\n status.errors.slice(0, 5).forEach(error => {\n console.log(` - ${error}`);\n });\n }\n }\n \n console.log('');\n console.log('INDEXED DIRECTORIES:');\n if (status.directories.length === 0) {\n console.log(' No directories have been indexed yet.');\n console.log(' Run \"directory-indexer index <path>\" to start indexing.');\n } else {\n status.directories.forEach(dir => {\n console.log('');\n console.log(` Directory: ${dir.path}`);\n console.log(` • Indexing status: ${dir.status}`);\n console.log(` • Files processed: ${dir.filesCount}`);\n console.log(` • Searchable chunks: ${dir.chunksCount}`);\n console.log(` • Last indexed: ${dir.lastIndexed || 'Never completed'}`);\n if (dir.errors.length > 0) {\n console.log(` • Files with errors: ${dir.errors.length}`);\n if (options.verbose) {\n console.log(' • Recent errors:');\n dir.errors.slice(0, 3).forEach(error => {\n console.log(` - ${error}`);\n });\n }\n }\n });\n }\n \n if (!status.qdrantConsistency.isConsistent) {\n console.log('');\n console.log('SYSTEM STATUS:');\n status.qdrantConsistency.issues.forEach(issue => {\n console.log(` • ${issue}`);\n });\n console.log('');\n console.log('ℹ️ Note: Status messages above may be normal during setup or active indexing.');\n } else {\n console.log('');\n console.log('SYSTEM STATUS:');\n console.log(' • All systems operational - ready for AI-powered search');\n }\n } catch (error) {\n console.error('Error getting status:', error);\n process.exit(1);\n }\n });\n\n await program.parseAsync();\n}\n\n// Main function is already exported above"],"names":["__dirname","packageJsonPath","packageJson","VERSION"],"mappings":";;;;;;;;;;;;AAgBA,MAAMA,cAAY,QAAQ,cAAc,YAAY,GAAG,CAAC;AACxD,MAAMC,oBAAkB,KAAKD,aAAW,iBAAiB;AACzD,MAAME,gBAAc,KAAK,MAAM,aAAaD,mBAAiB,OAAO,CAAC;AACrE,MAAME,YAAUD,cAAY;AAE5B,MAAM,YAAoB;AAAA,EACxB;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,gBAAgB;AAAA,UACd,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,MACf;AAAA,MAEF,UAAU,CAAC,gBAAgB;AAAA,IAAA;AAAA,EAC7B;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,OAAO;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,QAEf,OAAO;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,UACb,SAAS;AAAA,QAAA;AAAA,MACX;AAAA,MAEF,UAAU,CAAC,OAAO;AAAA,IAAA;AAAA,EACpB;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,WAAW;AAAA,UACT,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,QAEf,OAAO;AAAA,UACL,MAAM;AAAA,UACN,aAAa;AAAA,UACb,SAAS;AAAA,QAAA;AAAA,MACX;AAAA,MAEF,UAAU,CAAC,WAAW;AAAA,IAAA;AAAA,EACxB;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY;AAAA,QACV,WAAW;AAAA,UACT,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,QAEf,QAAQ;AAAA,UACN,MAAM;AAAA,UACN,aAAa;AAAA,QAAA;AAAA,MACf;AAAA,MAEF,UAAU,CAAC,WAAW;AAAA,IAAA;AAAA,EACxB;AAAA,EAEF;AAAA,IACE,MAAM;AAAA,IACN,aAAa;AAAA,IACb,aAAa;AAAA,MACX,MAAM;AAAA,MACN,YAAY,CAAA;AAAA,IAAC;AAAA,EACf;AAEJ;AAEA,eAAsB,eAAe,QAA+B;AAClE,QAAM,SAAS,IAAI;AAAA,IACjB;AAAA,MACE,MAAM;AAAA,MACN,SAASC;AAAAA,IAAA;AAAA,IAEX;AAAA,MACE,cAAc;AAAA,QACZ,OAAO,CAAA;AAAA,MAAC;AAAA,IACV;AAAA,EACF;AAGF,SAAO,kBAAkB,wBAAwB,YAAY;AAC3D,WAAO;AAAA,MACL,OAAO;AAAA,IAAA;AAAA,EAEX,CAAC;AAED,SAAO,kBAAkB,uBAAuB,OAAO,YAAY;AACjE,UAAM,EAAE,MAAM,WAAW,KAAA,IAAS,QAAQ;AAE1C,QAAI;AACF,cAAQ,MAAA;AAAA,QACN,KAAK,SAAS;AACZ,cAAI,CAAC,QAAQ,OAAO,KAAK,mBAAmB,UAAU;AACpD,kBAAM,IAAI,MAAM,4BAA4B;AAAA,UAC9C;AACA,gBAAM,QAAQ,KAAK,eAAe,MAAM,GAAG,EAAE,IAAI,CAAC,MAAc,EAAE,KAAA,CAAM;AACxE,gBAAM,SAAS,MAAM,iBAAiB,OAAO,MAAM;AACnD,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,WAAW,OAAO,OAAO,mBAAmB,OAAO,OAAO,WAAW,OAAO,OAAO,MAAM;AAAA,cAAA;AAAA,YACjG;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,UAAU;AACb,cAAI,CAAC,QAAQ,OAAO,KAAK,UAAU,UAAU;AAC3C,kBAAM,IAAI,MAAM,mBAAmB;AAAA,UACrC;AACA,gBAAM,UAAU,MAAM,cAAc,KAAK,OAAO,EAAE,OAAQ,KAAK,SAAoB,IAAI;AACvF,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,KAAK,UAAU,SAAS,MAAM,CAAC;AAAA,cAAA;AAAA,YACvC;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,iBAAiB;AACpB,cAAI,CAAC,QAAQ,OAAO,KAAK,cAAc,UAAU;AAC/C,kBAAM,IAAI,MAAM,uBAAuB;AAAA,UACzC;AACA,gBAAM,UAAU,MAAM,iBAAiB,KAAK,WAAY,KAAK,SAAoB,EAAE;AACnF,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,KAAK,UAAU,SAAS,MAAM,CAAC;AAAA,cAAA;AAAA,YACvC;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,eAAe;AAClB,cAAI,CAAC,QAAQ,OAAO,KAAK,cAAc,UAAU;AAC/C,kBAAM,IAAI,MAAM,uBAAuB;AAAA,UACzC;AACA,gBAAM,UAAU,MAAM,eAAe,KAAK,WAAW,KAAK,MAAgB;AAC1E,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM;AAAA,cAAA;AAAA,YACR;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA,KAAK,eAAe;AAClB,gBAAM,SAAS,MAAM,eAAA;AACrB,iBAAO;AAAA,YACL,SAAS;AAAA,cACP;AAAA,gBACE,MAAM;AAAA,gBACN,MAAM,KAAK,UAAU;AAAA,kBACnB,MAAM;AAAA,kBACN,SAASA;AAAAA,kBACT;AAAA,gBAAA,GACC,MAAM,CAAC;AAAA,cAAA;AAAA,YACZ;AAAA,UACF;AAAA,QAEJ;AAAA,QAEA;AACE,gBAAM,IAAI,MAAM,iBAAiB,IAAI,EAAE;AAAA,MAAA;AAAA,IAE7C,SAAS,OAAO;AACd,YAAM,eAAe,iBAAiB,QAAQ,MAAM,UAAU;AAC9D,aAAO;AAAA,QACL,SAAS;AAAA,UACP;AAAA,YACE,MAAM;AAAA,YACN,MAAM,UAAU,YAAY;AAAA,UAAA;AAAA,QAC9B;AAAA,QAEF,SAAS;AAAA,MAAA;AAAA,IAEb;AAAA,EACF,CAAC;AAED,QAAM,YAAY,IAAI,qBAAA;AACtB,QAAM,OAAO,QAAQ,SAAS;AAE9B,MAAI,OAAO,SAAS;AAClB,YAAQ,MAAM,iCAAiC;AAAA,EACjD;AACF;ACrNA,MAAM,YAAY,QAAQ,cAAc,YAAY,GAAG,CAAC;AACxD,MAAM,kBAAkB,KAAK,WAAW,iBAAiB;AACzD,MAAM,cAAc,KAAK,MAAM,aAAa,iBAAiB,OAAO,CAAC;AACrE,MAAM,UAAU,YAAY;AAE5B,eAAsB,OAAO;AAC3B,QAAM,UAAU,IAAI,QAAA;AAEpB,UACG,KAAK,mBAAmB,EACxB,YAAY,oDAAoD,EAChE,QAAQ,OAAO;AAElB,UACG,QAAQ,OAAO,EACf,YAAY,uCAAuC,EACnD,SAAS,cAAc,0BAA0B,EACjD,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,OAAiB,YAAY;AAC1C,QAAI;AACF,YAAM,SAAS,MAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC5D,cAAQ,IAAI,YAAY,MAAM,MAAM,iBAAiB;AACrD,YAAM,SAAS,MAAM,iBAAiB,OAAO,MAAM;AACnD,cAAQ,IAAI,WAAW,OAAO,OAAO,mBAAmB,OAAO,OAAO,WAAW,OAAO,OAAO,MAAM,SAAS;AAC9G,UAAI,OAAO,OAAO,SAAS,KAAK,OAAO,SAAS;AAC9C,gBAAQ,IAAI,WAAW,OAAO,MAAM;AAAA,MACtC;AAAA,IACF,SAAS,OAAO;AACd,cAAQ,MAAM,+BAA+B,KAAK;AAClD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,QAAQ,EAChB,YAAY,qCAAqC,EACjD,SAAS,WAAW,cAAc,EAClC,OAAO,wBAAwB,6BAA6B,IAAI,EAChE,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,OAAe,YAAY;AACxC,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,UAAU,MAAM,cAAc,OAAO,EAAE,OAAO,SAAS,QAAQ,KAAK,GAAG;AAE7E,UAAI,QAAQ,WAAW,GAAG;AACxB,gBAAQ,IAAI,kBAAkB;AAC9B;AAAA,MACF;AAEA,cAAQ,IAAI,SAAS,QAAQ,MAAM;AAAA,CAAa;AAChD,cAAQ,QAAQ,CAAC,QAAQ,UAAU;AACjC,gBAAQ,IAAI,GAAG,QAAQ,CAAC,KAAK,OAAO,QAAQ,EAAE;AAC9C,gBAAQ,IAAI,aAAa,OAAO,MAAM,QAAQ,CAAC,CAAC,EAAE;AAClD,YAAI,OAAO,SAAS;AAClB,kBAAQ,IAAI,eAAe,OAAO,QAAQ,UAAU,GAAG,GAAG,CAAC,KAAK;AAAA,QAClE;AACA,gBAAQ,IAAA;AAAA,MACV,CAAC;AAAA,IACH,SAAS,OAAO;AACd,cAAQ,MAAM,4BAA4B,KAAK;AAC/C,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,SAAS,EACjB,YAAY,oCAAoC,EAChD,SAAS,UAAU,qCAAqC,EACxD,OAAO,wBAAwB,6BAA6B,IAAI,EAChE,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,UAAkB,YAAY;AAC3C,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,UAAU,MAAM,iBAAiB,UAAU,SAAS,QAAQ,KAAK,CAAC;AAExE,UAAI,QAAQ,WAAW,GAAG;AACxB,gBAAQ,IAAI,wBAAwB;AACpC;AAAA,MACF;AAEA,cAAQ,IAAI,SAAS,QAAQ,MAAM;AAAA,CAAmB;AACtD,cAAQ,QAAQ,CAAC,QAAQ,UAAU;AACjC,gBAAQ,IAAI,GAAG,QAAQ,CAAC,KAAK,OAAO,QAAQ,EAAE;AAC9C,gBAAQ,IAAI,kBAAkB,OAAO,MAAM,QAAQ,CAAC,CAAC,EAAE;AACvD,gBAAQ,IAAA;AAAA,MACV,CAAC;AAAA,IACH,SAAS,OAAO;AACd,cAAQ,MAAM,gCAAgC,KAAK;AACnD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,KAAK,EACb,YAAY,kBAAkB,EAC9B,SAAS,UAAU,uBAAuB,EAC1C,OAAO,wBAAwB,2BAA2B,EAC1D,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,UAAkB,YAAY;AAC3C,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,UAAU,MAAM,eAAe,UAAU,QAAQ,MAAM;AAC7D,cAAQ,IAAI,OAAO;AAAA,IACrB,SAAS,OAAO;AACd,cAAQ,MAAM,+BAA+B,KAAK;AAClD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,OAAO,EACf,YAAY,kBAAkB,EAC9B,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,YAAY;AACzB,QAAI;AACF,YAAM,SAAS,MAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC5D,YAAM,eAAe,MAAM;AAAA,IAC7B,SAAS,OAAO;AACd,cAAQ,MAAM,8BAA8B,KAAK;AACjD,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,UACG,QAAQ,QAAQ,EAChB,YAAY,sBAAsB,EAClC,OAAO,iBAAiB,wBAAwB,EAChD,OAAO,OAAO,YAAY;AACzB,QAAI;AACF,YAAM,WAAW,EAAE,SAAS,QAAQ,SAAS;AAC7C,YAAM,SAAS,MAAM,eAAA;AAErB,cAAQ,IAAI,iCAAiC;AAC7C,cAAQ,IAAI,uCAAuC;AACnD,cAAQ,IAAI,EAAE;AACd,cAAQ,IAAI,WAAW;AACvB,cAAQ,IAAI,OAAO,OAAO,kBAAkB,gCAAgC;AAC5E,cAAQ,IAAI,OAAO,OAAO,YAAY,sCAAsC;AAC5E,cAAQ,IAAI,OAAO,OAAO,aAAa,sCAAsC;AAC7E,cAAQ,IAAI,yBAAyB,OAAO,YAAY,EAAE;AAC1D,cAAQ,IAAI,6BAA6B,OAAO,eAAe,2BAA2B,EAAE;AAE5F,UAAI,OAAO,OAAO,SAAS,GAAG;AAC5B,gBAAQ,IAAI,sCAAsC,OAAO,OAAO,MAAM,EAAE;AACxE,YAAI,QAAQ,SAAS;AACnB,kBAAQ,IAAI,EAAE;AACd,kBAAQ,IAAI,gBAAgB;AAC5B,iBAAO,OAAO,MAAM,GAAG,CAAC,EAAE,QAAQ,CAAA,UAAS;AACzC,oBAAQ,IAAI,OAAO,KAAK,EAAE;AAAA,UAC5B,CAAC;AAAA,QACH;AAAA,MACF;AAEA,cAAQ,IAAI,EAAE;AACd,cAAQ,IAAI,sBAAsB;AAClC,UAAI,OAAO,YAAY,WAAW,GAAG;AACnC,gBAAQ,IAAI,yCAAyC;AACrD,gBAAQ,IAAI,2DAA2D;AAAA,MACzE,OAAO;AACL,eAAO,YAAY,QAAQ,CAAA,QAAO;AAChC,kBAAQ,IAAI,EAAE;AACd,kBAAQ,IAAI,gBAAgB,IAAI,IAAI,EAAE;AACtC,kBAAQ,IAAI,0BAA0B,IAAI,MAAM,EAAE;AAClD,kBAAQ,IAAI,0BAA0B,IAAI,UAAU,EAAE;AACtD,kBAAQ,IAAI,4BAA4B,IAAI,WAAW,EAAE;AACzD,kBAAQ,IAAI,uBAAuB,IAAI,eAAe,iBAAiB,EAAE;AACzE,cAAI,IAAI,OAAO,SAAS,GAAG;AACzB,oBAAQ,IAAI,4BAA4B,IAAI,OAAO,MAAM,EAAE;AAC3D,gBAAI,QAAQ,SAAS;AACnB,sBAAQ,IAAI,sBAAsB;AAClC,kBAAI,OAAO,MAAM,GAAG,CAAC,EAAE,QAAQ,CAAA,UAAS;AACtC,wBAAQ,IAAI,WAAW,KAAK,EAAE;AAAA,cAChC,CAAC;AAAA,YACH;AAAA,UACF;AAAA,QACF,CAAC;AAAA,MACH;AAEA,UAAI,CAAC,OAAO,kBAAkB,cAAc;AAC1C,gBAAQ,IAAI,EAAE;AACd,gBAAQ,IAAI,gBAAgB;AAC5B,eAAO,kBAAkB,OAAO,QAAQ,CAAA,UAAS;AAC/C,kBAAQ,IAAI,OAAO,KAAK,EAAE;AAAA,QAC5B,CAAC;AACD,gBAAQ,IAAI,EAAE;AACd,gBAAQ,IAAI,gFAAgF;AAAA,MAC9F,OAAO;AACL,gBAAQ,IAAI,EAAE;AACd,gBAAQ,IAAI,gBAAgB;AAC5B,gBAAQ,IAAI,2DAA2D;AAAA,MACzE;AAAA,IACF,SAAS,OAAO;AACd,cAAQ,MAAM,yBAAyB,KAAK;AAC5C,cAAQ,KAAK,CAAC;AAAA,IAChB;AAAA,EACF,CAAC;AAEH,QAAM,QAAQ,WAAA;AAChB;"}
|
package/dist/config.js
CHANGED
|
@@ -30,11 +30,14 @@ class ConfigError extends Error {
|
|
|
30
30
|
}
|
|
31
31
|
function loadConfig(options = {}) {
|
|
32
32
|
const dataDir = process.env.DIRECTORY_INDEXER_DATA_DIR || join(homedir(), ".directory-indexer");
|
|
33
|
+
const isTest = process.env.NODE_ENV === "test" || process.env.VITEST === "true";
|
|
34
|
+
const dbFileName = isTest ? "test-data.db" : "data.db";
|
|
35
|
+
const defaultCollection = isTest ? "directory-indexer-test" : "directory-indexer";
|
|
33
36
|
const config = {
|
|
34
37
|
storage: {
|
|
35
|
-
sqlitePath: join(dataDir,
|
|
38
|
+
sqlitePath: join(dataDir, dbFileName),
|
|
36
39
|
qdrantEndpoint: process.env.QDRANT_ENDPOINT || "http://localhost:6333",
|
|
37
|
-
qdrantCollection: process.env.DIRECTORY_INDEXER_QDRANT_COLLECTION ||
|
|
40
|
+
qdrantCollection: process.env.DIRECTORY_INDEXER_QDRANT_COLLECTION || defaultCollection
|
|
38
41
|
},
|
|
39
42
|
embedding: {
|
|
40
43
|
provider: process.env.EMBEDDING_PROVIDER || "ollama",
|
package/dist/config.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"config.js","sources":["../src/config.ts"],"sourcesContent":["import { homedir } from 'os';\nimport { join } from 'path';\nimport { z } from 'zod';\n\nconst ConfigSchema = z.object({\n storage: z.object({\n sqlitePath: z.string(),\n qdrantEndpoint: z.string().url(),\n qdrantCollection: z.string(),\n }),\n embedding: z.object({\n provider: z.enum(['ollama', 'openai', 'mock']),\n model: z.string(),\n endpoint: z.string().url(),\n }),\n indexing: z.object({\n chunkSize: z.number().positive(),\n chunkOverlap: z.number().nonnegative(),\n maxFileSize: z.number().positive(),\n ignorePatterns: z.array(z.string()),\n }),\n dataDir: z.string(),\n verbose: z.boolean(),\n});\n\nexport type Config = z.infer<typeof ConfigSchema>;\n\nexport class ConfigError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'ConfigError';\n }\n}\n\nexport function loadConfig(options: { verbose?: boolean } = {}): Config {\n const dataDir = process.env.DIRECTORY_INDEXER_DATA_DIR || join(homedir(), '.directory-indexer');\n \n const config = {\n storage: {\n sqlitePath: join(dataDir,
|
|
1
|
+
{"version":3,"file":"config.js","sources":["../src/config.ts"],"sourcesContent":["import { homedir } from 'os';\nimport { join } from 'path';\nimport { z } from 'zod';\n\nconst ConfigSchema = z.object({\n storage: z.object({\n sqlitePath: z.string(),\n qdrantEndpoint: z.string().url(),\n qdrantCollection: z.string(),\n }),\n embedding: z.object({\n provider: z.enum(['ollama', 'openai', 'mock']),\n model: z.string(),\n endpoint: z.string().url(),\n }),\n indexing: z.object({\n chunkSize: z.number().positive(),\n chunkOverlap: z.number().nonnegative(),\n maxFileSize: z.number().positive(),\n ignorePatterns: z.array(z.string()),\n }),\n dataDir: z.string(),\n verbose: z.boolean(),\n});\n\nexport type Config = z.infer<typeof ConfigSchema>;\n\nexport class ConfigError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'ConfigError';\n }\n}\n\nexport function loadConfig(options: { verbose?: boolean } = {}): Config {\n const dataDir = process.env.DIRECTORY_INDEXER_DATA_DIR || join(homedir(), '.directory-indexer');\n \n // Use separate database file and collection for tests to avoid contaminating main data\n const isTest = process.env.NODE_ENV === 'test' || process.env.VITEST === 'true';\n const dbFileName = isTest ? 'test-data.db' : 'data.db';\n const defaultCollection = isTest ? 'directory-indexer-test' : 'directory-indexer';\n \n const config = {\n storage: {\n sqlitePath: join(dataDir, dbFileName),\n qdrantEndpoint: process.env.QDRANT_ENDPOINT || 'http://localhost:6333',\n qdrantCollection: process.env.DIRECTORY_INDEXER_QDRANT_COLLECTION || defaultCollection,\n },\n embedding: {\n provider: (process.env.EMBEDDING_PROVIDER as Config['embedding']['provider']) || 'ollama',\n model: process.env.EMBEDDING_MODEL || 'nomic-embed-text',\n endpoint: process.env.OLLAMA_ENDPOINT || 'http://localhost:11434',\n },\n indexing: {\n chunkSize: parseInt(process.env.CHUNK_SIZE || '512'),\n chunkOverlap: parseInt(process.env.CHUNK_OVERLAP || '50'),\n maxFileSize: parseInt(process.env.MAX_FILE_SIZE || '10485760'),\n ignorePatterns: ['.git', 'node_modules', 'target', '.DS_Store'],\n },\n dataDir,\n verbose: options.verbose ?? (process.env.VERBOSE === 'true'),\n };\n\n try {\n return ConfigSchema.parse(config);\n } catch (error) {\n if (error instanceof z.ZodError) {\n const messages = error.errors.map(e => `${e.path.join('.')}: ${e.message}`);\n throw new ConfigError(`Configuration validation failed: ${messages.join(', ')}`, error);\n }\n throw new ConfigError('Failed to load configuration', error as Error);\n }\n}"],"names":[],"mappings":";;;AAIA,MAAM,eAAe,EAAE,OAAO;AAAA,EAC5B,SAAS,EAAE,OAAO;AAAA,IAChB,YAAY,EAAE,OAAA;AAAA,IACd,gBAAgB,EAAE,OAAA,EAAS,IAAA;AAAA,IAC3B,kBAAkB,EAAE,OAAA;AAAA,EAAO,CAC5B;AAAA,EACD,WAAW,EAAE,OAAO;AAAA,IAClB,UAAU,EAAE,KAAK,CAAC,UAAU,UAAU,MAAM,CAAC;AAAA,IAC7C,OAAO,EAAE,OAAA;AAAA,IACT,UAAU,EAAE,OAAA,EAAS,IAAA;AAAA,EAAI,CAC1B;AAAA,EACD,UAAU,EAAE,OAAO;AAAA,IACjB,WAAW,EAAE,OAAA,EAAS,SAAA;AAAA,IACtB,cAAc,EAAE,OAAA,EAAS,YAAA;AAAA,IACzB,aAAa,EAAE,OAAA,EAAS,SAAA;AAAA,IACxB,gBAAgB,EAAE,MAAM,EAAE,QAAQ;AAAA,EAAA,CACnC;AAAA,EACD,SAAS,EAAE,OAAA;AAAA,EACX,SAAS,EAAE,QAAA;AACb,CAAC;AAIM,MAAM,oBAAoB,MAAM;AAAA,EACrC,YAAY,SAAiC,OAAe;AAC1D,UAAM,OAAO;AAD8B,SAAA,QAAA;AAE3C,SAAK,OAAO;AAAA,EACd;AACF;AAEO,SAAS,WAAW,UAAiC,IAAY;AACtE,QAAM,UAAU,QAAQ,IAAI,8BAA8B,KAAK,QAAA,GAAW,oBAAoB;AAG9F,QAAM,SAAS,QAAQ,IAAI,aAAa,UAAU,QAAQ,IAAI,WAAW;AACzE,QAAM,aAAa,SAAS,iBAAiB;AAC7C,QAAM,oBAAoB,SAAS,2BAA2B;AAE9D,QAAM,SAAS;AAAA,IACb,SAAS;AAAA,MACP,YAAY,KAAK,SAAS,UAAU;AAAA,MACpC,gBAAgB,QAAQ,IAAI,mBAAmB;AAAA,MAC/C,kBAAkB,QAAQ,IAAI,uCAAuC;AAAA,IAAA;AAAA,IAEvE,WAAW;AAAA,MACT,UAAW,QAAQ,IAAI,sBAA0D;AAAA,MACjF,OAAO,QAAQ,IAAI,mBAAmB;AAAA,MACtC,UAAU,QAAQ,IAAI,mBAAmB;AAAA,IAAA;AAAA,IAE3C,UAAU;AAAA,MACR,WAAW,SAAS,QAAQ,IAAI,cAAc,KAAK;AAAA,MACnD,cAAc,SAAS,QAAQ,IAAI,iBAAiB,IAAI;AAAA,MACxD,aAAa,SAAS,QAAQ,IAAI,iBAAiB,UAAU;AAAA,MAC7D,gBAAgB,CAAC,QAAQ,gBAAgB,UAAU,WAAW;AAAA,IAAA;AAAA,IAEhE;AAAA,IACA,SAAS,QAAQ,WAAY,QAAQ,IAAI,YAAY;AAAA,EAAA;AAGvD,MAAI;AACF,WAAO,aAAa,MAAM,MAAM;AAAA,EAClC,SAAS,OAAO;AACd,QAAI,iBAAiB,EAAE,UAAU;AAC/B,YAAM,WAAW,MAAM,OAAO,IAAI,OAAK,GAAG,EAAE,KAAK,KAAK,GAAG,CAAC,KAAK,EAAE,OAAO,EAAE;AAC1E,YAAM,IAAI,YAAY,oCAAoC,SAAS,KAAK,IAAI,CAAC,IAAI,KAAK;AAAA,IACxF;AACA,UAAM,IAAI,YAAY,gCAAgC,KAAc;AAAA,EACtE;AACF;"}
|
package/dist/embedding.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"embedding.js","sources":["../src/embedding.ts"],"sourcesContent":["import { Config } from './config.js';\n\nexport interface EmbeddingProvider {\n name: string;\n dimensions: number;\n generateEmbedding(text: string): Promise<number[]>;\n generateEmbeddings(texts: string[]): Promise<number[][]>;\n}\n\nexport interface EmbeddingConfig {\n model: string;\n endpoint: string;\n dimensions: number;\n}\n\nexport class EmbeddingError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'EmbeddingError';\n }\n}\n\nclass MockEmbeddingProvider implements EmbeddingProvider {\n name = 'mock';\n \n constructor(private config: EmbeddingConfig) {}\n \n get dimensions(): number {\n return this.config.dimensions;\n }\n \n async generateEmbedding(text: string): Promise<number[]> {\n const hash = this.hashString(text);\n return Array.from({ length: this.dimensions }, (_, i) => \n Math.sin(hash + i) * 0.1\n );\n }\n \n async generateEmbeddings(texts: string[]): Promise<number[][]> {\n return Promise.all(texts.map(text => this.generateEmbedding(text)));\n }\n \n private hashString(str: string): number {\n let hash = 0;\n for (let i = 0; i < str.length; i++) {\n const char = str.charCodeAt(i);\n hash = ((hash << 5) - hash) + char;\n hash = hash & hash;\n }\n return Math.abs(hash);\n }\n}\n\nclass OllamaEmbeddingProvider implements EmbeddingProvider {\n name = 'ollama';\n dimensions = 768;\n \n constructor(private config: EmbeddingConfig) {}\n \n async generateEmbedding(text: string): Promise<number[]> {\n try {\n const response = await fetch(`${this.config.endpoint}/api/embeddings`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n model: this.config.model,\n prompt: text\n })\n });\n \n if (!response.ok) {\n throw new Error(`Ollama API error: ${response.statusText}`);\n }\n \n const data = await response.json();\n return data.embedding;\n } catch (error) {\n throw new EmbeddingError(`Failed to generate Ollama embedding`, error as Error);\n }\n }\n \n async generateEmbeddings(texts: string[]): Promise<number[][]> {\n return Promise.all(texts.map(text => this.generateEmbedding(text)));\n }\n}\n\nclass OpenAIEmbeddingProvider implements EmbeddingProvider {\n name = 'openai';\n dimensions = 1536;\n \n constructor(private config: EmbeddingConfig) {}\n \n async generateEmbedding(text: string): Promise<number[]> {\n try {\n const response = await fetch('https://api.openai.com/v1/embeddings', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`\n },\n body: JSON.stringify({\n model: this.config.model,\n input: text\n })\n });\n \n if (!response.ok) {\n throw new Error(`OpenAI API error: ${response.statusText}`);\n }\n \n const data = await response.json();\n return data.data[0].embedding;\n } catch (error) {\n throw new EmbeddingError(`Failed to generate OpenAI embedding`, error as Error);\n }\n }\n \n async generateEmbeddings(texts: string[]): Promise<number[][]> {\n try {\n const response = await fetch('https://api.openai.com/v1/embeddings', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`\n },\n body: JSON.stringify({\n model: this.config.model,\n input: texts\n })\n });\n \n if (!response.ok) {\n throw new Error(`OpenAI API error: ${response.statusText}`);\n }\n \n const data = await response.json();\n return data.data.map((item:
|
|
1
|
+
{"version":3,"file":"embedding.js","sources":["../src/embedding.ts"],"sourcesContent":["import { Config } from './config.js';\n\nexport interface EmbeddingProvider {\n name: string;\n dimensions: number;\n generateEmbedding(text: string): Promise<number[]>;\n generateEmbeddings(texts: string[]): Promise<number[][]>;\n}\n\nexport interface EmbeddingConfig {\n model: string;\n endpoint: string;\n dimensions: number;\n}\n\nexport class EmbeddingError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'EmbeddingError';\n }\n}\n\nclass MockEmbeddingProvider implements EmbeddingProvider {\n name = 'mock';\n \n constructor(private config: EmbeddingConfig) {}\n \n get dimensions(): number {\n return this.config.dimensions;\n }\n \n async generateEmbedding(text: string): Promise<number[]> {\n const hash = this.hashString(text);\n return Array.from({ length: this.dimensions }, (_, i) => \n Math.sin(hash + i) * 0.1\n );\n }\n \n async generateEmbeddings(texts: string[]): Promise<number[][]> {\n return Promise.all(texts.map(text => this.generateEmbedding(text)));\n }\n \n private hashString(str: string): number {\n let hash = 0;\n for (let i = 0; i < str.length; i++) {\n const char = str.charCodeAt(i);\n hash = ((hash << 5) - hash) + char;\n hash = hash & hash;\n }\n return Math.abs(hash);\n }\n}\n\nclass OllamaEmbeddingProvider implements EmbeddingProvider {\n name = 'ollama';\n dimensions = 768;\n \n constructor(private config: EmbeddingConfig) {}\n \n async generateEmbedding(text: string): Promise<number[]> {\n try {\n const response = await fetch(`${this.config.endpoint}/api/embeddings`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n model: this.config.model,\n prompt: text\n })\n });\n \n if (!response.ok) {\n throw new Error(`Ollama API error: ${response.statusText}`);\n }\n \n const data = await response.json();\n return data.embedding;\n } catch (error) {\n throw new EmbeddingError(`Failed to generate Ollama embedding`, error as Error);\n }\n }\n \n async generateEmbeddings(texts: string[]): Promise<number[][]> {\n return Promise.all(texts.map(text => this.generateEmbedding(text)));\n }\n}\n\nclass OpenAIEmbeddingProvider implements EmbeddingProvider {\n name = 'openai';\n dimensions = 1536;\n \n constructor(private config: EmbeddingConfig) {}\n \n async generateEmbedding(text: string): Promise<number[]> {\n try {\n const response = await fetch('https://api.openai.com/v1/embeddings', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`\n },\n body: JSON.stringify({\n model: this.config.model,\n input: text\n })\n });\n \n if (!response.ok) {\n throw new Error(`OpenAI API error: ${response.statusText}`);\n }\n \n const data = await response.json();\n return data.data[0].embedding;\n } catch (error) {\n throw new EmbeddingError(`Failed to generate OpenAI embedding`, error as Error);\n }\n }\n \n async generateEmbeddings(texts: string[]): Promise<number[][]> {\n try {\n const response = await fetch('https://api.openai.com/v1/embeddings', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`\n },\n body: JSON.stringify({\n model: this.config.model,\n input: texts\n })\n });\n \n if (!response.ok) {\n throw new Error(`OpenAI API error: ${response.statusText}`);\n }\n \n const data = await response.json();\n return data.data.map((item: { embedding: number[] }) => item.embedding);\n } catch (error) {\n throw new EmbeddingError(`Failed to generate OpenAI embeddings`, error as Error);\n }\n }\n}\n\nexport function createEmbeddingProvider(provider: string, config: EmbeddingConfig): EmbeddingProvider {\n switch (provider) {\n case 'mock':\n return new MockEmbeddingProvider(config);\n case 'ollama':\n return new OllamaEmbeddingProvider(config);\n case 'openai':\n return new OpenAIEmbeddingProvider(config);\n default:\n throw new EmbeddingError(`Unknown embedding provider: ${provider}`);\n }\n}\n\nexport async function generateEmbedding(text: string, config: Config): Promise<number[]> {\n const provider = createEmbeddingProvider(config.embedding.provider, {\n model: config.embedding.model,\n endpoint: config.embedding.endpoint,\n dimensions: config.embedding.provider === 'mock' ? 384 : (config.embedding.provider === 'ollama' ? 768 : 1536)\n });\n \n return provider.generateEmbedding(text);\n}"],"names":[],"mappings":"AAeO,MAAM,uBAAuB,MAAM;AAAA,EACxC,YAAY,SAAiC,OAAe;AAC1D,UAAM,OAAO;AAD8B,SAAA,QAAA;AAE3C,SAAK,OAAO;AAAA,EACd;AACF;AAEA,MAAM,sBAAmD;AAAA,EAGvD,YAAoB,QAAyB;AAAzB,SAAA,SAAA;AAAA,EAA0B;AAAA,EAF9C,OAAO;AAAA,EAIP,IAAI,aAAqB;AACvB,WAAO,KAAK,OAAO;AAAA,EACrB;AAAA,EAEA,MAAM,kBAAkB,MAAiC;AACvD,UAAM,OAAO,KAAK,WAAW,IAAI;AACjC,WAAO,MAAM;AAAA,MAAK,EAAE,QAAQ,KAAK,WAAA;AAAA,MAAc,CAAC,GAAG,MACjD,KAAK,IAAI,OAAO,CAAC,IAAI;AAAA,IAAA;AAAA,EAEzB;AAAA,EAEA,MAAM,mBAAmB,OAAsC;AAC7D,WAAO,QAAQ,IAAI,MAAM,IAAI,UAAQ,KAAK,kBAAkB,IAAI,CAAC,CAAC;AAAA,EACpE;AAAA,EAEQ,WAAW,KAAqB;AACtC,QAAI,OAAO;AACX,aAAS,IAAI,GAAG,IAAI,IAAI,QAAQ,KAAK;AACnC,YAAM,OAAO,IAAI,WAAW,CAAC;AAC7B,cAAS,QAAQ,KAAK,OAAQ;AAC9B,aAAO,OAAO;AAAA,IAChB;AACA,WAAO,KAAK,IAAI,IAAI;AAAA,EACtB;AACF;AAEA,MAAM,wBAAqD;AAAA,EAIzD,YAAoB,QAAyB;AAAzB,SAAA,SAAA;AAAA,EAA0B;AAAA,EAH9C,OAAO;AAAA,EACP,aAAa;AAAA,EAIb,MAAM,kBAAkB,MAAiC;AACvD,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,mBAAmB;AAAA,QACrE,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU;AAAA,UACnB,OAAO,KAAK,OAAO;AAAA,UACnB,QAAQ;AAAA,QAAA,CACT;AAAA,MAAA,CACF;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,qBAAqB,SAAS,UAAU,EAAE;AAAA,MAC5D;AAEA,YAAM,OAAO,MAAM,SAAS,KAAA;AAC5B,aAAO,KAAK;AAAA,IACd,SAAS,OAAO;AACd,YAAM,IAAI,eAAe,uCAAuC,KAAc;AAAA,IAChF;AAAA,EACF;AAAA,EAEA,MAAM,mBAAmB,OAAsC;AAC7D,WAAO,QAAQ,IAAI,MAAM,IAAI,UAAQ,KAAK,kBAAkB,IAAI,CAAC,CAAC;AAAA,EACpE;AACF;AAEA,MAAM,wBAAqD;AAAA,EAIzD,YAAoB,QAAyB;AAAzB,SAAA,SAAA;AAAA,EAA0B;AAAA,EAH9C,OAAO;AAAA,EACP,aAAa;AAAA,EAIb,MAAM,kBAAkB,MAAiC;AACvD,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,wCAAwC;AAAA,QACnE,QAAQ;AAAA,QACR,SAAS;AAAA,UACP,gBAAgB;AAAA,UAChB,iBAAiB,UAAU,QAAQ,IAAI,cAAc;AAAA,QAAA;AAAA,QAEvD,MAAM,KAAK,UAAU;AAAA,UACnB,OAAO,KAAK,OAAO;AAAA,UACnB,OAAO;AAAA,QAAA,CACR;AAAA,MAAA,CACF;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,qBAAqB,SAAS,UAAU,EAAE;AAAA,MAC5D;AAEA,YAAM,OAAO,MAAM,SAAS,KAAA;AAC5B,aAAO,KAAK,KAAK,CAAC,EAAE;AAAA,IACtB,SAAS,OAAO;AACd,YAAM,IAAI,eAAe,uCAAuC,KAAc;AAAA,IAChF;AAAA,EACF;AAAA,EAEA,MAAM,mBAAmB,OAAsC;AAC7D,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,wCAAwC;AAAA,QACnE,QAAQ;AAAA,QACR,SAAS;AAAA,UACP,gBAAgB;AAAA,UAChB,iBAAiB,UAAU,QAAQ,IAAI,cAAc;AAAA,QAAA;AAAA,QAEvD,MAAM,KAAK,UAAU;AAAA,UACnB,OAAO,KAAK,OAAO;AAAA,UACnB,OAAO;AAAA,QAAA,CACR;AAAA,MAAA,CACF;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,qBAAqB,SAAS,UAAU,EAAE;AAAA,MAC5D;AAEA,YAAM,OAAO,MAAM,SAAS,KAAA;AAC5B,aAAO,KAAK,KAAK,IAAI,CAAC,SAAkC,KAAK,SAAS;AAAA,IACxE,SAAS,OAAO;AACd,YAAM,IAAI,eAAe,wCAAwC,KAAc;AAAA,IACjF;AAAA,EACF;AACF;AAEO,SAAS,wBAAwB,UAAkB,QAA4C;AACpG,UAAQ,UAAA;AAAA,IACN,KAAK;AACH,aAAO,IAAI,sBAAsB,MAAM;AAAA,IACzC,KAAK;AACH,aAAO,IAAI,wBAAwB,MAAM;AAAA,IAC3C,KAAK;AACH,aAAO,IAAI,wBAAwB,MAAM;AAAA,IAC3C;AACE,YAAM,IAAI,eAAe,+BAA+B,QAAQ,EAAE;AAAA,EAAA;AAExE;AAEA,eAAsB,kBAAkB,MAAc,QAAmC;AACvF,QAAM,WAAW,wBAAwB,OAAO,UAAU,UAAU;AAAA,IAClE,OAAO,OAAO,UAAU;AAAA,IACxB,UAAU,OAAO,UAAU;AAAA,IAC3B,YAAY,OAAO,UAAU,aAAa,SAAS,MAAO,OAAO,UAAU,aAAa,WAAW,MAAM;AAAA,EAAA,CAC1G;AAED,SAAO,SAAS,kBAAkB,IAAI;AACxC;"}
|
package/dist/indexing.js
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { promises } from "fs";
|
|
2
2
|
import { join } from "path";
|
|
3
|
-
import { normalizePath, shouldIgnoreFile, isDirectory, isFile, isSupportedFileType
|
|
3
|
+
import { getFileInfo, normalizePath, shouldIgnoreFile, isDirectory, isFile, isSupportedFileType } from "./utils.js";
|
|
4
4
|
import { generateEmbedding } from "./embedding.js";
|
|
5
5
|
import { initializeStorage } from "./storage.js";
|
|
6
6
|
class IndexingError extends Error {
|
|
@@ -86,6 +86,31 @@ async function getFileMetadata(filePath) {
|
|
|
86
86
|
throw new IndexingError(`Failed to get file metadata`, error);
|
|
87
87
|
}
|
|
88
88
|
}
|
|
89
|
+
async function shouldReprocessFile(filePath, existingRecord, config) {
|
|
90
|
+
try {
|
|
91
|
+
const fs2 = await import("fs/promises");
|
|
92
|
+
const currentStats = await fs2.stat(filePath);
|
|
93
|
+
const existingModTime = new Date(existingRecord.modifiedTime);
|
|
94
|
+
if (currentStats.mtime <= existingModTime) {
|
|
95
|
+
return false;
|
|
96
|
+
}
|
|
97
|
+
const currentFileInfo = await getFileInfo(filePath);
|
|
98
|
+
return currentFileInfo.hash !== existingRecord.hash;
|
|
99
|
+
} catch (modtimeError) {
|
|
100
|
+
if (config.verbose) {
|
|
101
|
+
console.log(`Warning: Could not check modification time for ${filePath}:`, modtimeError);
|
|
102
|
+
}
|
|
103
|
+
try {
|
|
104
|
+
const currentFileInfo = await getFileInfo(filePath);
|
|
105
|
+
return currentFileInfo.hash !== existingRecord.hash;
|
|
106
|
+
} catch (hashError) {
|
|
107
|
+
if (config.verbose) {
|
|
108
|
+
console.log(`Warning: Could not compute hash for ${filePath}:`, hashError);
|
|
109
|
+
}
|
|
110
|
+
return true;
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
}
|
|
89
114
|
async function indexDirectories(paths, config) {
|
|
90
115
|
let indexed = 0;
|
|
91
116
|
let skipped = 0;
|
|
@@ -97,9 +122,19 @@ async function indexDirectories(paths, config) {
|
|
|
97
122
|
const { sqlite, qdrant } = await initializeStorage(config);
|
|
98
123
|
for (const path of paths) {
|
|
99
124
|
try {
|
|
125
|
+
await sqlite.upsertDirectory(path, "indexing");
|
|
100
126
|
const files = await scanDirectory(path, scanOptions);
|
|
101
127
|
for (const file of files) {
|
|
102
128
|
try {
|
|
129
|
+
const existingFile = await sqlite.getFile(file.path);
|
|
130
|
+
if (existingFile) {
|
|
131
|
+
const needsReprocessing = await shouldReprocessFile(file.path, existingFile, config);
|
|
132
|
+
if (!needsReprocessing) {
|
|
133
|
+
skipped++;
|
|
134
|
+
continue;
|
|
135
|
+
}
|
|
136
|
+
await qdrant.deletePointsByFileHash(existingFile.hash);
|
|
137
|
+
}
|
|
103
138
|
const content = await promises.readFile(file.path, "utf-8");
|
|
104
139
|
const chunks = chunkText(content, config.indexing.chunkSize, config.indexing.chunkOverlap);
|
|
105
140
|
await sqlite.upsertFile(file, chunks);
|
|
@@ -123,13 +158,16 @@ async function indexDirectories(paths, config) {
|
|
|
123
158
|
}
|
|
124
159
|
indexed++;
|
|
125
160
|
} catch (error) {
|
|
126
|
-
skipped++;
|
|
127
161
|
const errorMessage = error instanceof Error ? error.message : String(error);
|
|
128
162
|
const causeMessage = error instanceof Error && error.cause ? `: ${error.cause.message}` : "";
|
|
129
163
|
errors.push(`Failed to process ${file.path}: ${errorMessage}${causeMessage}`);
|
|
130
164
|
}
|
|
131
165
|
}
|
|
166
|
+
const directoryErrors = errors.filter((err) => err.includes(path));
|
|
167
|
+
const directoryStatus = directoryErrors.length > 0 ? "failed" : "completed";
|
|
168
|
+
await sqlite.upsertDirectory(path, directoryStatus);
|
|
132
169
|
} catch (error) {
|
|
170
|
+
await sqlite.upsertDirectory(path, "failed");
|
|
133
171
|
errors.push(`Failed to scan directory ${path}: ${error.message}`);
|
|
134
172
|
}
|
|
135
173
|
}
|
package/dist/indexing.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"indexing.js","sources":["../src/indexing.ts"],"sourcesContent":["import { promises as fs } from 'fs';\nimport { join } from 'path';\nimport { Config } from './config.js';\nimport { \n FileInfo, \n ChunkInfo, \n normalizePath, \n getFileInfo, \n shouldIgnoreFile, \n isSupportedFileType,\n isDirectory,\n isFile\n} from './utils.js';\nimport { generateEmbedding } from './embedding.js';\nimport { initializeStorage } from './storage.js';\n\nexport interface ScanOptions {\n ignorePatterns: string[];\n maxFileSize: number;\n}\n\nexport interface IndexResult {\n indexed: number;\n skipped: number;\n errors: string[];\n}\n\nexport class IndexingError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'IndexingError';\n }\n}\n\nexport function chunkText(content: string, chunkSize: number, overlap: number): ChunkInfo[] {\n if (content.length <= chunkSize) {\n return [{\n id: '0',\n content,\n startIndex: 0,\n endIndex: content.length\n }];\n }\n \n const chunks: ChunkInfo[] = [];\n let startIndex = 0;\n let chunkId = 0;\n \n while (startIndex < content.length) {\n const endIndex = Math.min(startIndex + chunkSize, content.length);\n const chunkContent = content.slice(startIndex, endIndex);\n \n chunks.push({\n id: chunkId.toString(),\n content: chunkContent,\n startIndex,\n endIndex\n });\n \n chunkId++;\n const nextStart = endIndex - overlap;\n \n if (nextStart <= startIndex) {\n startIndex = startIndex + Math.max(1, chunkSize - overlap);\n } else {\n startIndex = nextStart;\n }\n \n if (startIndex >= content.length) break;\n }\n \n return chunks;\n}\n\nexport async function scanDirectory(dirPath: string, options: ScanOptions): Promise<FileInfo[]> {\n const files: FileInfo[] = [];\n const visited = new Set<string>();\n \n async function walkDirectory(currentPath: string): Promise<void> {\n const normalizedPath = normalizePath(currentPath);\n \n if (visited.has(normalizedPath)) {\n return;\n }\n visited.add(normalizedPath);\n \n try {\n if (shouldIgnoreFile(normalizedPath, options.ignorePatterns)) {\n return;\n }\n \n if (await isDirectory(normalizedPath)) {\n const entries = await fs.readdir(normalizedPath);\n \n for (const entry of entries) {\n const fullPath = join(normalizedPath, entry);\n await walkDirectory(fullPath);\n }\n } else if (await isFile(normalizedPath)) {\n if (!isSupportedFileType(normalizedPath)) {\n return;\n }\n \n const stats = await fs.stat(normalizedPath);\n if (stats.size > options.maxFileSize) {\n return;\n }\n \n const fileInfo = await getFileInfo(normalizedPath);\n files.push(fileInfo);\n }\n } catch (error) {\n throw new IndexingError(`Failed to scan directory: ${normalizedPath}`, error as Error);\n }\n }\n \n await walkDirectory(dirPath);\n return files;\n}\n\nexport async function getFileMetadata(filePath: string): Promise<FileInfo> {\n try {\n return await getFileInfo(filePath);\n } catch (error) {\n throw new IndexingError(`Failed to get file metadata`, error as Error);\n }\n}\n\nexport async function indexDirectories(paths: string[], config: Config): Promise<IndexResult> {\n let indexed = 0;\n let skipped = 0;\n const errors: string[] = [];\n \n const scanOptions: ScanOptions = {\n ignorePatterns: config.indexing.ignorePatterns,\n maxFileSize: config.indexing.maxFileSize\n };\n \n // Initialize storage\n const { sqlite, qdrant } = await initializeStorage(config);\n \n for (const path of paths) {\n try {\n const files = await scanDirectory(path, scanOptions);\n \n for (const file of files) {\n try {\n const content = await fs.readFile(file.path, 'utf-8');\n const chunks = chunkText(content, config.indexing.chunkSize, config.indexing.chunkOverlap);\n \n // Store file metadata in SQLite\n await sqlite.upsertFile(file, chunks);\n \n // Generate embeddings and store in Qdrant\n for (let i = 0; i < chunks.length; i++) {\n const chunk = chunks[i];\n const embedding = await generateEmbedding(chunk.content, config);\n // Generate a unique integer ID by combining hash and chunk index\n const hashNum = parseInt(file.hash.slice(0, 8), 16);\n const pointId = (hashNum % 1000000) * 1000 + parseInt(chunk.id);\n const point = {\n id: pointId,\n vector: embedding,\n payload: {\n filePath: file.path,\n chunkId: chunk.id,\n fileHash: file.hash,\n content: chunk.content,\n parentDirectories: file.parentDirs\n }\n };\n await qdrant.upsertPoints([point]);\n }\n \n indexed++;\n } catch (error) {\n skipped++;\n const errorMessage = error instanceof Error ? error.message : String(error);\n const causeMessage = error instanceof Error && error.cause ? `: ${(error.cause as Error).message}` : '';\n errors.push(`Failed to process ${file.path}: ${errorMessage}${causeMessage}`);\n }\n }\n } catch (error) {\n errors.push(`Failed to scan directory ${path}: ${(error as Error).message}`);\n }\n }\n \n return { indexed, skipped, errors };\n}"],"names":["fs"],"mappings":";;;;;AA2BO,MAAM,sBAAsB,MAAM;AAAA,EACvC,YAAY,SAAiC,OAAe;AAC1D,UAAM,OAAO;AAD8B,SAAA,QAAA;AAE3C,SAAK,OAAO;AAAA,EACd;AACF;AAEO,SAAS,UAAU,SAAiB,WAAmB,SAA8B;AAC1F,MAAI,QAAQ,UAAU,WAAW;AAC/B,WAAO,CAAC;AAAA,MACN,IAAI;AAAA,MACJ;AAAA,MACA,YAAY;AAAA,MACZ,UAAU,QAAQ;AAAA,IAAA,CACnB;AAAA,EACH;AAEA,QAAM,SAAsB,CAAA;AAC5B,MAAI,aAAa;AACjB,MAAI,UAAU;AAEd,SAAO,aAAa,QAAQ,QAAQ;AAClC,UAAM,WAAW,KAAK,IAAI,aAAa,WAAW,QAAQ,MAAM;AAChE,UAAM,eAAe,QAAQ,MAAM,YAAY,QAAQ;AAEvD,WAAO,KAAK;AAAA,MACV,IAAI,QAAQ,SAAA;AAAA,MACZ,SAAS;AAAA,MACT;AAAA,MACA;AAAA,IAAA,CACD;AAED;AACA,UAAM,YAAY,WAAW;AAE7B,QAAI,aAAa,YAAY;AAC3B,mBAAa,aAAa,KAAK,IAAI,GAAG,YAAY,OAAO;AAAA,IAC3D,OAAO;AACL,mBAAa;AAAA,IACf;AAEA,QAAI,cAAc,QAAQ,OAAQ;AAAA,EACpC;AAEA,SAAO;AACT;AAEA,eAAsB,cAAc,SAAiB,SAA2C;AAC9F,QAAM,QAAoB,CAAA;AAC1B,QAAM,8BAAc,IAAA;AAEpB,iBAAe,cAAc,aAAoC;AAC/D,UAAM,iBAAiB,cAAc,WAAW;AAEhD,QAAI,QAAQ,IAAI,cAAc,GAAG;AAC/B;AAAA,IACF;AACA,YAAQ,IAAI,cAAc;AAE1B,QAAI;AACF,UAAI,iBAAiB,gBAAgB,QAAQ,cAAc,GAAG;AAC5D;AAAA,MACF;AAEA,UAAI,MAAM,YAAY,cAAc,GAAG;AACrC,cAAM,UAAU,MAAMA,SAAG,QAAQ,cAAc;AAE/C,mBAAW,SAAS,SAAS;AAC3B,gBAAM,WAAW,KAAK,gBAAgB,KAAK;AAC3C,gBAAM,cAAc,QAAQ;AAAA,QAC9B;AAAA,MACF,WAAW,MAAM,OAAO,cAAc,GAAG;AACvC,YAAI,CAAC,oBAAoB,cAAc,GAAG;AACxC;AAAA,QACF;AAEA,cAAM,QAAQ,MAAMA,SAAG,KAAK,cAAc;AAC1C,YAAI,MAAM,OAAO,QAAQ,aAAa;AACpC;AAAA,QACF;AAEA,cAAM,WAAW,MAAM,YAAY,cAAc;AACjD,cAAM,KAAK,QAAQ;AAAA,MACrB;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,cAAc,6BAA6B,cAAc,IAAI,KAAc;AAAA,IACvF;AAAA,EACF;AAEA,QAAM,cAAc,OAAO;AAC3B,SAAO;AACT;AAEA,eAAsB,gBAAgB,UAAqC;AACzE,MAAI;AACF,WAAO,MAAM,YAAY,QAAQ;AAAA,EACnC,SAAS,OAAO;AACd,UAAM,IAAI,cAAc,+BAA+B,KAAc;AAAA,EACvE;AACF;AAEA,eAAsB,iBAAiB,OAAiB,QAAsC;AAC5F,MAAI,UAAU;AACd,MAAI,UAAU;AACd,QAAM,SAAmB,CAAA;AAEzB,QAAM,cAA2B;AAAA,IAC/B,gBAAgB,OAAO,SAAS;AAAA,IAChC,aAAa,OAAO,SAAS;AAAA,EAAA;AAI/B,QAAM,EAAE,QAAQ,OAAA,IAAW,MAAM,kBAAkB,MAAM;AAEzD,aAAW,QAAQ,OAAO;AACxB,QAAI;AACF,YAAM,QAAQ,MAAM,cAAc,MAAM,WAAW;AAEnD,iBAAW,QAAQ,OAAO;AACxB,YAAI;AACF,gBAAM,UAAU,MAAMA,SAAG,SAAS,KAAK,MAAM,OAAO;AACpD,gBAAM,SAAS,UAAU,SAAS,OAAO,SAAS,WAAW,OAAO,SAAS,YAAY;AAGzF,gBAAM,OAAO,WAAW,MAAM,MAAM;AAGpC,mBAAS,IAAI,GAAG,IAAI,OAAO,QAAQ,KAAK;AACtC,kBAAM,QAAQ,OAAO,CAAC;AACtB,kBAAM,YAAY,MAAM,kBAAkB,MAAM,SAAS,MAAM;AAE/D,kBAAM,UAAU,SAAS,KAAK,KAAK,MAAM,GAAG,CAAC,GAAG,EAAE;AAClD,kBAAM,UAAW,UAAU,MAAW,MAAO,SAAS,MAAM,EAAE;AAC9D,kBAAM,QAAQ;AAAA,cACZ,IAAI;AAAA,cACJ,QAAQ;AAAA,cACR,SAAS;AAAA,gBACP,UAAU,KAAK;AAAA,gBACf,SAAS,MAAM;AAAA,gBACf,UAAU,KAAK;AAAA,gBACf,SAAS,MAAM;AAAA,gBACf,mBAAmB,KAAK;AAAA,cAAA;AAAA,YAC1B;AAEF,kBAAM,OAAO,aAAa,CAAC,KAAK,CAAC;AAAA,UACnC;AAEA;AAAA,QACF,SAAS,OAAO;AACd;AACA,gBAAM,eAAe,iBAAiB,QAAQ,MAAM,UAAU,OAAO,KAAK;AAC1E,gBAAM,eAAe,iBAAiB,SAAS,MAAM,QAAQ,KAAM,MAAM,MAAgB,OAAO,KAAK;AACrG,iBAAO,KAAK,qBAAqB,KAAK,IAAI,KAAK,YAAY,GAAG,YAAY,EAAE;AAAA,QAC9E;AAAA,MACF;AAAA,IACF,SAAS,OAAO;AACd,aAAO,KAAK,4BAA4B,IAAI,KAAM,MAAgB,OAAO,EAAE;AAAA,IAC7E;AAAA,EACF;AAEA,SAAO,EAAE,SAAS,SAAS,OAAA;AAC7B;"}
|
|
1
|
+
{"version":3,"file":"indexing.js","sources":["../src/indexing.ts"],"sourcesContent":["import { promises as fs } from 'fs';\nimport { join } from 'path';\nimport { Config } from './config.js';\nimport { \n FileInfo, \n ChunkInfo, \n normalizePath, \n getFileInfo, \n shouldIgnoreFile, \n isSupportedFileType,\n isDirectory,\n isFile\n} from './utils.js';\nimport { generateEmbedding } from './embedding.js';\nimport { initializeStorage, FileRecord } from './storage.js';\n\nexport interface ScanOptions {\n ignorePatterns: string[];\n maxFileSize: number;\n}\n\nexport interface IndexResult {\n indexed: number;\n skipped: number;\n errors: string[];\n}\n\nexport class IndexingError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'IndexingError';\n }\n}\n\nexport function chunkText(content: string, chunkSize: number, overlap: number): ChunkInfo[] {\n if (content.length <= chunkSize) {\n return [{\n id: '0',\n content,\n startIndex: 0,\n endIndex: content.length\n }];\n }\n \n const chunks: ChunkInfo[] = [];\n let startIndex = 0;\n let chunkId = 0;\n \n while (startIndex < content.length) {\n const endIndex = Math.min(startIndex + chunkSize, content.length);\n const chunkContent = content.slice(startIndex, endIndex);\n \n chunks.push({\n id: chunkId.toString(),\n content: chunkContent,\n startIndex,\n endIndex\n });\n \n chunkId++;\n const nextStart = endIndex - overlap;\n \n if (nextStart <= startIndex) {\n startIndex = startIndex + Math.max(1, chunkSize - overlap);\n } else {\n startIndex = nextStart;\n }\n \n if (startIndex >= content.length) break;\n }\n \n return chunks;\n}\n\nexport async function scanDirectory(dirPath: string, options: ScanOptions): Promise<FileInfo[]> {\n const files: FileInfo[] = [];\n const visited = new Set<string>();\n \n async function walkDirectory(currentPath: string): Promise<void> {\n const normalizedPath = normalizePath(currentPath);\n \n if (visited.has(normalizedPath)) {\n return;\n }\n visited.add(normalizedPath);\n \n try {\n if (shouldIgnoreFile(normalizedPath, options.ignorePatterns)) {\n return;\n }\n \n if (await isDirectory(normalizedPath)) {\n const entries = await fs.readdir(normalizedPath);\n \n for (const entry of entries) {\n const fullPath = join(normalizedPath, entry);\n await walkDirectory(fullPath);\n }\n } else if (await isFile(normalizedPath)) {\n if (!isSupportedFileType(normalizedPath)) {\n return;\n }\n \n const stats = await fs.stat(normalizedPath);\n if (stats.size > options.maxFileSize) {\n return;\n }\n \n const fileInfo = await getFileInfo(normalizedPath);\n files.push(fileInfo);\n }\n } catch (error) {\n throw new IndexingError(`Failed to scan directory: ${normalizedPath}`, error as Error);\n }\n }\n \n await walkDirectory(dirPath);\n return files;\n}\n\nexport async function getFileMetadata(filePath: string): Promise<FileInfo> {\n try {\n return await getFileInfo(filePath);\n } catch (error) {\n throw new IndexingError(`Failed to get file metadata`, error as Error);\n }\n}\n\nasync function shouldReprocessFile(filePath: string, existingRecord: FileRecord, config: Config): Promise<boolean> {\n try {\n const fs = await import('fs/promises');\n \n // Try modtime check first (fast path)\n const currentStats = await fs.stat(filePath);\n const existingModTime = new Date(existingRecord.modifiedTime);\n \n // If modtime is clearly older, likely unchanged\n if (currentStats.mtime <= existingModTime) {\n return false; // Skip processing\n }\n \n // If modtime suggests change, verify with hash\n const currentFileInfo = await getFileInfo(filePath);\n return currentFileInfo.hash !== existingRecord.hash;\n \n } catch (modtimeError) {\n // Graceful fallback: skip modtime, use hash only\n if (config.verbose) {\n console.log(`Warning: Could not check modification time for ${filePath}:`, modtimeError);\n }\n try {\n const currentFileInfo = await getFileInfo(filePath);\n return currentFileInfo.hash !== existingRecord.hash;\n } catch (hashError) {\n // If we can't hash either, assume changed to be safe\n if (config.verbose) {\n console.log(`Warning: Could not compute hash for ${filePath}:`, hashError);\n }\n return true;\n }\n }\n}\n\nexport async function indexDirectories(paths: string[], config: Config): Promise<IndexResult> {\n let indexed = 0;\n let skipped = 0;\n const errors: string[] = [];\n \n const scanOptions: ScanOptions = {\n ignorePatterns: config.indexing.ignorePatterns,\n maxFileSize: config.indexing.maxFileSize\n };\n \n // Initialize storage\n const { sqlite, qdrant } = await initializeStorage(config);\n \n for (const path of paths) {\n try {\n // Mark directory as indexing\n await sqlite.upsertDirectory(path, 'indexing');\n \n const files = await scanDirectory(path, scanOptions);\n \n for (const file of files) {\n try {\n // Check if file already exists and needs reprocessing\n const existingFile = await sqlite.getFile(file.path);\n \n if (existingFile) {\n const needsReprocessing = await shouldReprocessFile(file.path, existingFile, config);\n if (!needsReprocessing) {\n skipped++;\n continue; // Skip unchanged file\n }\n \n // File changed - clean up old vectors first\n await qdrant.deletePointsByFileHash(existingFile.hash);\n }\n \n const content = await fs.readFile(file.path, 'utf-8');\n const chunks = chunkText(content, config.indexing.chunkSize, config.indexing.chunkOverlap);\n \n // Store file metadata in SQLite\n await sqlite.upsertFile(file, chunks);\n \n // Generate embeddings and store in Qdrant\n for (let i = 0; i < chunks.length; i++) {\n const chunk = chunks[i];\n const embedding = await generateEmbedding(chunk.content, config);\n // Generate a unique integer ID by combining hash and chunk index\n const hashNum = parseInt(file.hash.slice(0, 8), 16);\n const pointId = (hashNum % 1000000) * 1000 + parseInt(chunk.id);\n const point = {\n id: pointId,\n vector: embedding,\n payload: {\n filePath: file.path,\n chunkId: chunk.id,\n fileHash: file.hash,\n content: chunk.content,\n parentDirectories: file.parentDirs\n }\n };\n await qdrant.upsertPoints([point]);\n }\n \n indexed++;\n } catch (error) {\n const errorMessage = error instanceof Error ? error.message : String(error);\n const causeMessage = error instanceof Error && error.cause ? `: ${(error.cause as Error).message}` : '';\n errors.push(`Failed to process ${file.path}: ${errorMessage}${causeMessage}`);\n }\n }\n \n // Mark directory as completed if no errors for this directory\n const directoryErrors = errors.filter(err => err.includes(path));\n const directoryStatus = directoryErrors.length > 0 ? 'failed' : 'completed';\n await sqlite.upsertDirectory(path, directoryStatus);\n \n } catch (error) {\n await sqlite.upsertDirectory(path, 'failed');\n errors.push(`Failed to scan directory ${path}: ${(error as Error).message}`);\n }\n }\n \n return { indexed, skipped, errors };\n}"],"names":["fs"],"mappings":";;;;;AA2BO,MAAM,sBAAsB,MAAM;AAAA,EACvC,YAAY,SAAiC,OAAe;AAC1D,UAAM,OAAO;AAD8B,SAAA,QAAA;AAE3C,SAAK,OAAO;AAAA,EACd;AACF;AAEO,SAAS,UAAU,SAAiB,WAAmB,SAA8B;AAC1F,MAAI,QAAQ,UAAU,WAAW;AAC/B,WAAO,CAAC;AAAA,MACN,IAAI;AAAA,MACJ;AAAA,MACA,YAAY;AAAA,MACZ,UAAU,QAAQ;AAAA,IAAA,CACnB;AAAA,EACH;AAEA,QAAM,SAAsB,CAAA;AAC5B,MAAI,aAAa;AACjB,MAAI,UAAU;AAEd,SAAO,aAAa,QAAQ,QAAQ;AAClC,UAAM,WAAW,KAAK,IAAI,aAAa,WAAW,QAAQ,MAAM;AAChE,UAAM,eAAe,QAAQ,MAAM,YAAY,QAAQ;AAEvD,WAAO,KAAK;AAAA,MACV,IAAI,QAAQ,SAAA;AAAA,MACZ,SAAS;AAAA,MACT;AAAA,MACA;AAAA,IAAA,CACD;AAED;AACA,UAAM,YAAY,WAAW;AAE7B,QAAI,aAAa,YAAY;AAC3B,mBAAa,aAAa,KAAK,IAAI,GAAG,YAAY,OAAO;AAAA,IAC3D,OAAO;AACL,mBAAa;AAAA,IACf;AAEA,QAAI,cAAc,QAAQ,OAAQ;AAAA,EACpC;AAEA,SAAO;AACT;AAEA,eAAsB,cAAc,SAAiB,SAA2C;AAC9F,QAAM,QAAoB,CAAA;AAC1B,QAAM,8BAAc,IAAA;AAEpB,iBAAe,cAAc,aAAoC;AAC/D,UAAM,iBAAiB,cAAc,WAAW;AAEhD,QAAI,QAAQ,IAAI,cAAc,GAAG;AAC/B;AAAA,IACF;AACA,YAAQ,IAAI,cAAc;AAE1B,QAAI;AACF,UAAI,iBAAiB,gBAAgB,QAAQ,cAAc,GAAG;AAC5D;AAAA,MACF;AAEA,UAAI,MAAM,YAAY,cAAc,GAAG;AACrC,cAAM,UAAU,MAAMA,SAAG,QAAQ,cAAc;AAE/C,mBAAW,SAAS,SAAS;AAC3B,gBAAM,WAAW,KAAK,gBAAgB,KAAK;AAC3C,gBAAM,cAAc,QAAQ;AAAA,QAC9B;AAAA,MACF,WAAW,MAAM,OAAO,cAAc,GAAG;AACvC,YAAI,CAAC,oBAAoB,cAAc,GAAG;AACxC;AAAA,QACF;AAEA,cAAM,QAAQ,MAAMA,SAAG,KAAK,cAAc;AAC1C,YAAI,MAAM,OAAO,QAAQ,aAAa;AACpC;AAAA,QACF;AAEA,cAAM,WAAW,MAAM,YAAY,cAAc;AACjD,cAAM,KAAK,QAAQ;AAAA,MACrB;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,cAAc,6BAA6B,cAAc,IAAI,KAAc;AAAA,IACvF;AAAA,EACF;AAEA,QAAM,cAAc,OAAO;AAC3B,SAAO;AACT;AAEA,eAAsB,gBAAgB,UAAqC;AACzE,MAAI;AACF,WAAO,MAAM,YAAY,QAAQ;AAAA,EACnC,SAAS,OAAO;AACd,UAAM,IAAI,cAAc,+BAA+B,KAAc;AAAA,EACvE;AACF;AAEA,eAAe,oBAAoB,UAAkB,gBAA4B,QAAkC;AACjH,MAAI;AACF,UAAMA,MAAK,MAAM,OAAO,aAAa;AAGrC,UAAM,eAAe,MAAMA,IAAG,KAAK,QAAQ;AAC3C,UAAM,kBAAkB,IAAI,KAAK,eAAe,YAAY;AAG5D,QAAI,aAAa,SAAS,iBAAiB;AACzC,aAAO;AAAA,IACT;AAGA,UAAM,kBAAkB,MAAM,YAAY,QAAQ;AAClD,WAAO,gBAAgB,SAAS,eAAe;AAAA,EAEjD,SAAS,cAAc;AAErB,QAAI,OAAO,SAAS;AAClB,cAAQ,IAAI,kDAAkD,QAAQ,KAAK,YAAY;AAAA,IACzF;AACA,QAAI;AACF,YAAM,kBAAkB,MAAM,YAAY,QAAQ;AAClD,aAAO,gBAAgB,SAAS,eAAe;AAAA,IACjD,SAAS,WAAW;AAElB,UAAI,OAAO,SAAS;AAClB,gBAAQ,IAAI,uCAAuC,QAAQ,KAAK,SAAS;AAAA,MAC3E;AACA,aAAO;AAAA,IACT;AAAA,EACF;AACF;AAEA,eAAsB,iBAAiB,OAAiB,QAAsC;AAC5F,MAAI,UAAU;AACd,MAAI,UAAU;AACd,QAAM,SAAmB,CAAA;AAEzB,QAAM,cAA2B;AAAA,IAC/B,gBAAgB,OAAO,SAAS;AAAA,IAChC,aAAa,OAAO,SAAS;AAAA,EAAA;AAI/B,QAAM,EAAE,QAAQ,OAAA,IAAW,MAAM,kBAAkB,MAAM;AAEzD,aAAW,QAAQ,OAAO;AACxB,QAAI;AAEF,YAAM,OAAO,gBAAgB,MAAM,UAAU;AAE7C,YAAM,QAAQ,MAAM,cAAc,MAAM,WAAW;AAEnD,iBAAW,QAAQ,OAAO;AACxB,YAAI;AAEF,gBAAM,eAAe,MAAM,OAAO,QAAQ,KAAK,IAAI;AAEnD,cAAI,cAAc;AAChB,kBAAM,oBAAoB,MAAM,oBAAoB,KAAK,MAAM,cAAc,MAAM;AACnF,gBAAI,CAAC,mBAAmB;AACtB;AACA;AAAA,YACF;AAGA,kBAAM,OAAO,uBAAuB,aAAa,IAAI;AAAA,UACvD;AAEA,gBAAM,UAAU,MAAMA,SAAG,SAAS,KAAK,MAAM,OAAO;AACpD,gBAAM,SAAS,UAAU,SAAS,OAAO,SAAS,WAAW,OAAO,SAAS,YAAY;AAGzF,gBAAM,OAAO,WAAW,MAAM,MAAM;AAGpC,mBAAS,IAAI,GAAG,IAAI,OAAO,QAAQ,KAAK;AACtC,kBAAM,QAAQ,OAAO,CAAC;AACtB,kBAAM,YAAY,MAAM,kBAAkB,MAAM,SAAS,MAAM;AAE/D,kBAAM,UAAU,SAAS,KAAK,KAAK,MAAM,GAAG,CAAC,GAAG,EAAE;AAClD,kBAAM,UAAW,UAAU,MAAW,MAAO,SAAS,MAAM,EAAE;AAC9D,kBAAM,QAAQ;AAAA,cACZ,IAAI;AAAA,cACJ,QAAQ;AAAA,cACR,SAAS;AAAA,gBACP,UAAU,KAAK;AAAA,gBACf,SAAS,MAAM;AAAA,gBACf,UAAU,KAAK;AAAA,gBACf,SAAS,MAAM;AAAA,gBACf,mBAAmB,KAAK;AAAA,cAAA;AAAA,YAC1B;AAEF,kBAAM,OAAO,aAAa,CAAC,KAAK,CAAC;AAAA,UACnC;AAEA;AAAA,QACF,SAAS,OAAO;AACd,gBAAM,eAAe,iBAAiB,QAAQ,MAAM,UAAU,OAAO,KAAK;AAC1E,gBAAM,eAAe,iBAAiB,SAAS,MAAM,QAAQ,KAAM,MAAM,MAAgB,OAAO,KAAK;AACrG,iBAAO,KAAK,qBAAqB,KAAK,IAAI,KAAK,YAAY,GAAG,YAAY,EAAE;AAAA,QAC9E;AAAA,MACF;AAGA,YAAM,kBAAkB,OAAO,OAAO,SAAO,IAAI,SAAS,IAAI,CAAC;AAC/D,YAAM,kBAAkB,gBAAgB,SAAS,IAAI,WAAW;AAChE,YAAM,OAAO,gBAAgB,MAAM,eAAe;AAAA,IAEpD,SAAS,OAAO;AACd,YAAM,OAAO,gBAAgB,MAAM,QAAQ;AAC3C,aAAO,KAAK,4BAA4B,IAAI,KAAM,MAAgB,OAAO,EAAE;AAAA,IAC7E;AAAA,EACF;AAEA,SAAO,EAAE,SAAS,SAAS,OAAA;AAC7B;"}
|
package/dist/storage.js
CHANGED
|
@@ -14,7 +14,7 @@ class QdrantClient {
|
|
|
14
14
|
}
|
|
15
15
|
async healthCheck() {
|
|
16
16
|
try {
|
|
17
|
-
const response = await fetch(`${this.config.storage.qdrantEndpoint}/
|
|
17
|
+
const response = await fetch(`${this.config.storage.qdrantEndpoint}/healthz`);
|
|
18
18
|
return response.ok;
|
|
19
19
|
} catch {
|
|
20
20
|
return false;
|
|
@@ -53,7 +53,8 @@ class QdrantClient {
|
|
|
53
53
|
body: JSON.stringify({ points })
|
|
54
54
|
});
|
|
55
55
|
if (!response.ok) {
|
|
56
|
-
|
|
56
|
+
const errorText = await response.text();
|
|
57
|
+
throw new Error(`Failed to upsert points: ${response.status} ${response.statusText} - ${errorText}`);
|
|
57
58
|
}
|
|
58
59
|
} catch (error) {
|
|
59
60
|
throw new StorageError(`Failed to upsert points to Qdrant`, error);
|
|
@@ -100,6 +101,31 @@ class QdrantClient {
|
|
|
100
101
|
throw new StorageError(`Failed to delete points from Qdrant`, error);
|
|
101
102
|
}
|
|
102
103
|
}
|
|
104
|
+
async deletePointsByFileHash(fileHash) {
|
|
105
|
+
const collectionName = this.config.storage.qdrantCollection;
|
|
106
|
+
try {
|
|
107
|
+
const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points/delete`, {
|
|
108
|
+
method: "POST",
|
|
109
|
+
headers: { "Content-Type": "application/json" },
|
|
110
|
+
body: JSON.stringify({
|
|
111
|
+
filter: {
|
|
112
|
+
must: [
|
|
113
|
+
{
|
|
114
|
+
key: "fileHash",
|
|
115
|
+
match: { value: fileHash }
|
|
116
|
+
}
|
|
117
|
+
]
|
|
118
|
+
}
|
|
119
|
+
})
|
|
120
|
+
});
|
|
121
|
+
if (!response.ok) {
|
|
122
|
+
const errorText = await response.text();
|
|
123
|
+
throw new Error(`Failed to delete points by file hash: ${response.status} ${response.statusText} - ${errorText}`);
|
|
124
|
+
}
|
|
125
|
+
} catch (error) {
|
|
126
|
+
throw new StorageError(`Failed to delete points by file hash from Qdrant`, error);
|
|
127
|
+
}
|
|
128
|
+
}
|
|
103
129
|
}
|
|
104
130
|
class SQLiteStorage {
|
|
105
131
|
constructor(config) {
|
|
@@ -174,7 +200,7 @@ class SQLiteStorage {
|
|
|
174
200
|
id: row.id,
|
|
175
201
|
path: row.path,
|
|
176
202
|
size: row.size,
|
|
177
|
-
modifiedTime: new Date(row.modified_time),
|
|
203
|
+
modifiedTime: new Date(row.modified_time * 1e3),
|
|
178
204
|
hash: row.hash,
|
|
179
205
|
parentDirs: JSON.parse(row.parent_dirs),
|
|
180
206
|
chunks: row.chunks_json ? JSON.parse(row.chunks_json) : [],
|
|
@@ -219,7 +245,7 @@ class SQLiteStorage {
|
|
|
219
245
|
id: row.id,
|
|
220
246
|
path: row.path,
|
|
221
247
|
size: row.size,
|
|
222
|
-
modifiedTime: new Date(row.modified_time),
|
|
248
|
+
modifiedTime: new Date(row.modified_time * 1e3),
|
|
223
249
|
hash: row.hash,
|
|
224
250
|
parentDirs: JSON.parse(row.parent_dirs),
|
|
225
251
|
chunks: row.chunks_json ? JSON.parse(row.chunks_json) : [],
|
|
@@ -242,6 +268,50 @@ async function initializeStorage(config) {
|
|
|
242
268
|
async function initDatabase(dbPath) {
|
|
243
269
|
return new Database(dbPath);
|
|
244
270
|
}
|
|
271
|
+
async function checkQdrantConsistency(sqlite, config) {
|
|
272
|
+
const issues = [];
|
|
273
|
+
try {
|
|
274
|
+
const qdrant = new QdrantClient(config);
|
|
275
|
+
const isHealthy = await qdrant.healthCheck();
|
|
276
|
+
if (!isHealthy) {
|
|
277
|
+
issues.push("Qdrant vector database is not running or accessible");
|
|
278
|
+
return { isConsistent: false, issues };
|
|
279
|
+
}
|
|
280
|
+
const filesWithChunksStmt = sqlite.db.prepare("SELECT COUNT(*) as count FROM files WHERE chunks_json IS NOT NULL");
|
|
281
|
+
const filesWithChunks = filesWithChunksStmt.get();
|
|
282
|
+
const totalChunksStmt = sqlite.db.prepare("SELECT SUM(json_array_length(chunks_json)) as count FROM files WHERE chunks_json IS NOT NULL");
|
|
283
|
+
const totalChunks = totalChunksStmt.get();
|
|
284
|
+
if (filesWithChunks.count > 0 && (totalChunks.count || 0) === 0) {
|
|
285
|
+
issues.push("Files exist but no chunks found - possible data corruption");
|
|
286
|
+
}
|
|
287
|
+
const collectionName = config.storage.qdrantCollection;
|
|
288
|
+
try {
|
|
289
|
+
const response = await fetch(`${config.storage.qdrantEndpoint}/collections/${collectionName}`);
|
|
290
|
+
if (!response.ok) {
|
|
291
|
+
issues.push(`Vector collection '${collectionName}' not found (normal during first-time setup)`);
|
|
292
|
+
return { isConsistent: false, issues };
|
|
293
|
+
}
|
|
294
|
+
const collectionInfo = await response.json();
|
|
295
|
+
const qdrantPointCount = collectionInfo.result?.points_count || 0;
|
|
296
|
+
const sqliteChunkCount = totalChunks.count || 0;
|
|
297
|
+
if (Math.abs(qdrantPointCount - sqliteChunkCount) > 0) {
|
|
298
|
+
if (qdrantPointCount > sqliteChunkCount) {
|
|
299
|
+
issues.push(`Extra vectors in database: ${qdrantPointCount} vectors vs ${sqliteChunkCount} indexed chunks (normal during cleanup)`);
|
|
300
|
+
} else {
|
|
301
|
+
issues.push(`Missing vectors: ${sqliteChunkCount} indexed chunks vs ${qdrantPointCount} vectors (normal during indexing)`);
|
|
302
|
+
}
|
|
303
|
+
}
|
|
304
|
+
} catch (error) {
|
|
305
|
+
issues.push(`Cannot verify vector database status: ${error}`);
|
|
306
|
+
}
|
|
307
|
+
} catch (error) {
|
|
308
|
+
issues.push(`Database status check failed: ${error}`);
|
|
309
|
+
}
|
|
310
|
+
return {
|
|
311
|
+
isConsistent: issues.length === 0,
|
|
312
|
+
issues
|
|
313
|
+
};
|
|
314
|
+
}
|
|
245
315
|
async function getIndexStatus() {
|
|
246
316
|
const config = await import("./config.js").then((m) => m.loadConfig());
|
|
247
317
|
const sqlite = new SQLiteStorage(config);
|
|
@@ -252,7 +322,7 @@ async function getIndexStatus() {
|
|
|
252
322
|
const filesCount = filesStmt.get();
|
|
253
323
|
const chunksStmt = sqlite.db.prepare("SELECT SUM(json_array_length(chunks_json)) as count FROM files WHERE chunks_json IS NOT NULL");
|
|
254
324
|
const chunksCount = chunksStmt.get();
|
|
255
|
-
const lastIndexedStmt = sqlite.db.prepare("SELECT MAX(indexed_at) as last_indexed FROM directories");
|
|
325
|
+
const lastIndexedStmt = sqlite.db.prepare("SELECT MAX(indexed_at) as last_indexed FROM directories WHERE indexed_at > 0");
|
|
256
326
|
const lastIndexedResult = lastIndexedStmt.get();
|
|
257
327
|
const errorsStmt = sqlite.db.prepare("SELECT errors_json FROM files WHERE errors_json IS NOT NULL");
|
|
258
328
|
const errorRows = errorsStmt.all();
|
|
@@ -265,6 +335,44 @@ async function getIndexStatus() {
|
|
|
265
335
|
allErrors.push("Failed to parse error JSON");
|
|
266
336
|
}
|
|
267
337
|
});
|
|
338
|
+
const directoriesDetailStmt = sqlite.db.prepare(`
|
|
339
|
+
SELECT
|
|
340
|
+
d.path,
|
|
341
|
+
d.status,
|
|
342
|
+
d.indexed_at,
|
|
343
|
+
COUNT(f.id) as files_count,
|
|
344
|
+
COALESCE(SUM(json_array_length(f.chunks_json)), 0) as chunks_count
|
|
345
|
+
FROM directories d
|
|
346
|
+
LEFT JOIN files f ON f.parent_dirs LIKE '%' || d.path || '%'
|
|
347
|
+
GROUP BY d.id, d.path, d.status, d.indexed_at
|
|
348
|
+
ORDER BY d.indexed_at DESC
|
|
349
|
+
`);
|
|
350
|
+
const directoryDetails = directoriesDetailStmt.all();
|
|
351
|
+
const directories = directoryDetails.map((row) => {
|
|
352
|
+
const errorsByDirStmt = sqlite.db.prepare(`
|
|
353
|
+
SELECT errors_json FROM files
|
|
354
|
+
WHERE parent_dirs LIKE '%' || ? || '%' AND errors_json IS NOT NULL
|
|
355
|
+
`);
|
|
356
|
+
const dirErrors = errorsByDirStmt.all(row.path);
|
|
357
|
+
const dirErrorsList = [];
|
|
358
|
+
dirErrors.forEach((errorRow) => {
|
|
359
|
+
try {
|
|
360
|
+
const errors = JSON.parse(errorRow.errors_json);
|
|
361
|
+
dirErrorsList.push(...errors);
|
|
362
|
+
} catch {
|
|
363
|
+
dirErrorsList.push("Failed to parse error JSON");
|
|
364
|
+
}
|
|
365
|
+
});
|
|
366
|
+
return {
|
|
367
|
+
path: row.path,
|
|
368
|
+
status: row.status,
|
|
369
|
+
filesCount: row.files_count,
|
|
370
|
+
chunksCount: row.chunks_count,
|
|
371
|
+
lastIndexed: row.indexed_at && row.indexed_at > 0 ? new Date(row.indexed_at).toISOString() : null,
|
|
372
|
+
errors: dirErrorsList
|
|
373
|
+
};
|
|
374
|
+
});
|
|
375
|
+
const qdrantConsistency = await checkQdrantConsistency(sqlite, config);
|
|
268
376
|
const fs = await import("fs");
|
|
269
377
|
let databaseSize = "0 KB";
|
|
270
378
|
try {
|
|
@@ -286,7 +394,9 @@ async function getIndexStatus() {
|
|
|
286
394
|
chunksIndexed: chunksCount.count || 0,
|
|
287
395
|
databaseSize,
|
|
288
396
|
lastIndexed: lastIndexedResult.last_indexed ? new Date(lastIndexedResult.last_indexed).toISOString() : null,
|
|
289
|
-
errors: allErrors
|
|
397
|
+
errors: allErrors,
|
|
398
|
+
directories,
|
|
399
|
+
qdrantConsistency
|
|
290
400
|
};
|
|
291
401
|
} finally {
|
|
292
402
|
sqlite.close();
|
package/dist/storage.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"storage.js","sources":["../src/storage.ts"],"sourcesContent":["import Database from 'better-sqlite3';\nimport { Config } from './config.js';\nimport { FileInfo, ChunkInfo, ensureDirectory } from './utils.js';\nimport { dirname } from 'path';\n\nexport interface DirectoryRecord {\n id: number;\n path: string;\n status: 'pending' | 'indexing' | 'completed' | 'failed';\n indexedAt: Date;\n}\n\nexport interface FileRecord {\n id: number;\n path: string;\n size: number;\n modifiedTime: Date;\n hash: string;\n parentDirs: string[];\n chunks: ChunkInfo[];\n errors?: string[];\n}\n\nexport interface QdrantPoint {\n id: string | number;\n vector: number[];\n payload: {\n filePath: string;\n chunkId: string;\n parentDirectories: string[];\n };\n score?: number;\n}\n\nexport class StorageError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'StorageError';\n }\n}\n\nexport class QdrantClient {\n constructor(private config: Config) {}\n\n async healthCheck(): Promise<boolean> {\n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/health`);\n return response.ok;\n } catch {\n return false;\n }\n }\n\n async createCollection(): Promise<void> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const checkResponse = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}`);\n if (checkResponse.ok) {\n return;\n }\n\n const createResponse = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}`, {\n method: 'PUT',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n vectors: {\n size: 768,\n distance: 'Cosine'\n }\n })\n });\n\n if (!createResponse.ok) {\n throw new Error(`Failed to create collection: ${createResponse.statusText}`);\n }\n } catch (error) {\n throw new StorageError(`Failed to create Qdrant collection`, error as Error);\n }\n }\n\n async upsertPoints(points: QdrantPoint[]): Promise<void> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points`, {\n method: 'PUT',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({ points })\n });\n\n if (!response.ok) {\n throw new Error(`Failed to upsert points: ${response.statusText}`);\n }\n } catch (error) {\n throw new StorageError(`Failed to upsert points to Qdrant`, error as Error);\n }\n }\n\n async searchPoints(vector: number[], limit: number = 10): Promise<QdrantPoint[]> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points/search`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n vector,\n limit,\n with_payload: true\n })\n });\n\n if (!response.ok) {\n throw new Error(`Failed to search points: ${response.statusText}`);\n }\n\n const data = await response.json();\n return data.result.map((item: any) => ({\n id: item.id,\n vector: item.vector,\n payload: item.payload,\n score: item.score\n }));\n } catch (error) {\n throw new StorageError(`Failed to search points in Qdrant`, error as Error);\n }\n }\n\n async deletePoints(ids: string[]): Promise<void> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points/delete`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({ points: ids })\n });\n\n if (!response.ok) {\n throw new Error(`Failed to delete points: ${response.statusText}`);\n }\n } catch (error) {\n throw new StorageError(`Failed to delete points from Qdrant`, error as Error);\n }\n }\n}\n\nexport class SQLiteStorage {\n public db: Database.Database;\n\n constructor(private config: Config) {\n this.db = this.initializeDatabase();\n }\n\n private initializeDatabase(): Database.Database {\n try {\n ensureDirectory(dirname(this.config.storage.sqlitePath));\n \n const db = new Database(this.config.storage.sqlitePath);\n \n db.exec(`\n CREATE TABLE IF NOT EXISTS directories (\n id INTEGER PRIMARY KEY,\n path TEXT UNIQUE NOT NULL,\n status TEXT DEFAULT 'pending',\n indexed_at INTEGER DEFAULT 0\n );\n\n CREATE TABLE IF NOT EXISTS files (\n id INTEGER PRIMARY KEY,\n path TEXT UNIQUE NOT NULL,\n size INTEGER NOT NULL,\n modified_time INTEGER NOT NULL,\n hash TEXT NOT NULL,\n parent_dirs TEXT NOT NULL,\n chunks_json TEXT,\n errors_json TEXT\n );\n\n CREATE INDEX IF NOT EXISTS idx_files_path ON files(path);\n CREATE INDEX IF NOT EXISTS idx_files_hash ON files(hash);\n CREATE INDEX IF NOT EXISTS idx_directories_path ON directories(path);\n `);\n\n return db;\n } catch (error) {\n throw new StorageError(`Failed to initialize SQLite database`, error as Error);\n }\n }\n\n async getDirectory(path: string): Promise<DirectoryRecord | null> {\n try {\n const stmt = this.db.prepare('SELECT * FROM directories WHERE path = ?');\n const row = stmt.get(path) as any;\n \n if (!row) return null;\n \n return {\n id: row.id,\n path: row.path,\n status: row.status,\n indexedAt: new Date(row.indexed_at)\n };\n } catch (error) {\n throw new StorageError(`Failed to get directory record`, error as Error);\n }\n }\n\n async upsertDirectory(path: string, status: DirectoryRecord['status']): Promise<void> {\n try {\n const stmt = this.db.prepare(`\n INSERT OR REPLACE INTO directories (path, status, indexed_at)\n VALUES (?, ?, ?)\n `);\n \n stmt.run(path, status, Date.now());\n } catch (error) {\n throw new StorageError(`Failed to upsert directory record`, error as Error);\n }\n }\n\n async getFile(path: string): Promise<FileRecord | null> {\n try {\n const stmt = this.db.prepare('SELECT * FROM files WHERE path = ?');\n const row = stmt.get(path) as any;\n \n if (!row) return null;\n \n return {\n id: row.id,\n path: row.path,\n size: row.size,\n modifiedTime: new Date(row.modified_time),\n hash: row.hash,\n parentDirs: JSON.parse(row.parent_dirs),\n chunks: row.chunks_json ? JSON.parse(row.chunks_json) : [],\n errors: row.errors_json ? JSON.parse(row.errors_json) : undefined\n };\n } catch (error) {\n throw new StorageError(`Failed to get file record`, error as Error);\n }\n }\n\n async upsertFile(fileInfo: FileInfo, chunks: ChunkInfo[] = [], errors: string[] = []): Promise<void> {\n try {\n const stmt = this.db.prepare(`\n INSERT OR REPLACE INTO files (path, size, modified_time, hash, parent_dirs, chunks_json, errors_json)\n VALUES (?, ?, ?, ?, ?, ?, ?)\n `);\n \n stmt.run(\n fileInfo.path,\n fileInfo.size,\n fileInfo.modifiedTime.getTime(),\n fileInfo.hash,\n JSON.stringify(fileInfo.parentDirs),\n chunks.length > 0 ? JSON.stringify(chunks) : null,\n errors.length > 0 ? JSON.stringify(errors) : null\n );\n } catch (error) {\n throw new StorageError(`Failed to upsert file record`, error as Error);\n }\n }\n\n async deleteFile(path: string): Promise<void> {\n try {\n const stmt = this.db.prepare('DELETE FROM files WHERE path = ?');\n stmt.run(path);\n } catch (error) {\n throw new StorageError(`Failed to delete file record`, error as Error);\n }\n }\n\n async getFilesByDirectory(directoryPath: string): Promise<FileRecord[]> {\n try {\n const stmt = this.db.prepare('SELECT * FROM files WHERE path LIKE ?');\n const rows = stmt.all(`${directoryPath}%`) as any[];\n \n return rows.map(row => ({\n id: row.id,\n path: row.path,\n size: row.size,\n modifiedTime: new Date(row.modified_time),\n hash: row.hash,\n parentDirs: JSON.parse(row.parent_dirs),\n chunks: row.chunks_json ? JSON.parse(row.chunks_json) : [],\n errors: row.errors_json ? JSON.parse(row.errors_json) : undefined\n }));\n } catch (error) {\n throw new StorageError(`Failed to get files by directory`, error as Error);\n }\n }\n\n close(): void {\n this.db.close();\n }\n}\n\nexport async function initializeStorage(config: Config): Promise<{ sqlite: SQLiteStorage; qdrant: QdrantClient }> {\n const sqlite = new SQLiteStorage(config);\n const qdrant = new QdrantClient(config);\n \n await qdrant.createCollection();\n \n return { sqlite, qdrant };\n}\n\nexport async function initDatabase(dbPath: string): Promise<Database.Database> {\n return new Database(dbPath);\n}\n\nexport interface IndexStatus {\n directoriesIndexed: number;\n filesIndexed: number;\n chunksIndexed: number;\n databaseSize: string;\n lastIndexed: string | null;\n errors: string[];\n}\n\nexport async function getIndexStatus(): Promise<IndexStatus> {\n const config = await import('./config.js').then(m => m.loadConfig());\n const sqlite = new SQLiteStorage(config);\n \n try {\n const directoriesStmt = sqlite.db.prepare('SELECT COUNT(*) as count FROM directories WHERE status = ?');\n const directoriesCount = directoriesStmt.get('completed') as { count: number };\n \n const filesStmt = sqlite.db.prepare('SELECT COUNT(*) as count FROM files');\n const filesCount = filesStmt.get() as { count: number };\n \n const chunksStmt = sqlite.db.prepare('SELECT SUM(json_array_length(chunks_json)) as count FROM files WHERE chunks_json IS NOT NULL');\n const chunksCount = chunksStmt.get() as { count: number | null };\n \n const lastIndexedStmt = sqlite.db.prepare('SELECT MAX(indexed_at) as last_indexed FROM directories');\n const lastIndexedResult = lastIndexedStmt.get() as { last_indexed: number | null };\n \n const errorsStmt = sqlite.db.prepare('SELECT errors_json FROM files WHERE errors_json IS NOT NULL');\n const errorRows = errorsStmt.all() as { errors_json: string }[];\n \n const allErrors: string[] = [];\n errorRows.forEach(row => {\n try {\n const errors = JSON.parse(row.errors_json);\n allErrors.push(...errors);\n } catch {\n allErrors.push('Failed to parse error JSON');\n }\n });\n \n const fs = await import('fs');\n let databaseSize = '0 KB';\n try {\n const stats = fs.statSync(config.storage.sqlitePath);\n const sizeInBytes = stats.size;\n if (sizeInBytes > 1024 * 1024) {\n databaseSize = `${(sizeInBytes / (1024 * 1024)).toFixed(2)} MB`;\n } else if (sizeInBytes > 1024) {\n databaseSize = `${(sizeInBytes / 1024).toFixed(2)} KB`;\n } else {\n databaseSize = `${sizeInBytes} bytes`;\n }\n } catch {\n databaseSize = 'Unknown';\n }\n \n return {\n directoriesIndexed: directoriesCount.count,\n filesIndexed: filesCount.count,\n chunksIndexed: chunksCount.count || 0,\n databaseSize,\n lastIndexed: lastIndexedResult.last_indexed ? new Date(lastIndexedResult.last_indexed).toISOString() : null,\n errors: allErrors\n };\n } finally {\n sqlite.close();\n }\n}"],"names":[],"mappings":";;;AAkCO,MAAM,qBAAqB,MAAM;AAAA,EACtC,YAAY,SAAiC,OAAe;AAC1D,UAAM,OAAO;AAD8B,SAAA,QAAA;AAE3C,SAAK,OAAO;AAAA,EACd;AACF;AAEO,MAAM,aAAa;AAAA,EACxB,YAAoB,QAAgB;AAAhB,SAAA,SAAA;AAAA,EAAiB;AAAA,EAErC,MAAM,cAAgC;AACpC,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,SAAS;AAC3E,aAAO,SAAS;AAAA,IAClB,QAAQ;AACN,aAAO;AAAA,IACT;AAAA,EACF;AAAA,EAEA,MAAM,mBAAkC;AACtC,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,gBAAgB,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,EAAE;AACvG,UAAI,cAAc,IAAI;AACpB;AAAA,MACF;AAEA,YAAM,iBAAiB,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,IAAI;AAAA,QACxG,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU;AAAA,UACnB,SAAS;AAAA,YACP,MAAM;AAAA,YACN,UAAU;AAAA,UAAA;AAAA,QACZ,CACD;AAAA,MAAA,CACF;AAED,UAAI,CAAC,eAAe,IAAI;AACtB,cAAM,IAAI,MAAM,gCAAgC,eAAe,UAAU,EAAE;AAAA,MAC7E;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,sCAAsC,KAAc;AAAA,IAC7E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,QAAsC;AACvD,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,WAAW;AAAA,QACzG,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU,EAAE,QAAQ;AAAA,MAAA,CAChC;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,4BAA4B,SAAS,UAAU,EAAE;AAAA,MACnE;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,qCAAqC,KAAc;AAAA,IAC5E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,QAAkB,QAAgB,IAA4B;AAC/E,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,kBAAkB;AAAA,QAChH,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU;AAAA,UACnB;AAAA,UACA;AAAA,UACA,cAAc;AAAA,QAAA,CACf;AAAA,MAAA,CACF;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,4BAA4B,SAAS,UAAU,EAAE;AAAA,MACnE;AAEA,YAAM,OAAO,MAAM,SAAS,KAAA;AAC5B,aAAO,KAAK,OAAO,IAAI,CAAC,UAAe;AAAA,QACrC,IAAI,KAAK;AAAA,QACT,QAAQ,KAAK;AAAA,QACb,SAAS,KAAK;AAAA,QACd,OAAO,KAAK;AAAA,MAAA,EACZ;AAAA,IACJ,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,qCAAqC,KAAc;AAAA,IAC5E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,KAA8B;AAC/C,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,kBAAkB;AAAA,QAChH,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU,EAAE,QAAQ,KAAK;AAAA,MAAA,CACrC;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,4BAA4B,SAAS,UAAU,EAAE;AAAA,MACnE;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,uCAAuC,KAAc;AAAA,IAC9E;AAAA,EACF;AACF;AAEO,MAAM,cAAc;AAAA,EAGzB,YAAoB,QAAgB;AAAhB,SAAA,SAAA;AAClB,SAAK,KAAK,KAAK,mBAAA;AAAA,EACjB;AAAA,EAJO;AAAA,EAMC,qBAAwC;AAC9C,QAAI;AACF,sBAAgB,QAAQ,KAAK,OAAO,QAAQ,UAAU,CAAC;AAEvD,YAAM,KAAK,IAAI,SAAS,KAAK,OAAO,QAAQ,UAAU;AAEtD,SAAG,KAAK;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,OAsBP;AAED,aAAO;AAAA,IACT,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,wCAAwC,KAAc;AAAA,IAC/E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,MAA+C;AAChE,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,0CAA0C;AACvE,YAAM,MAAM,KAAK,IAAI,IAAI;AAEzB,UAAI,CAAC,IAAK,QAAO;AAEjB,aAAO;AAAA,QACL,IAAI,IAAI;AAAA,QACR,MAAM,IAAI;AAAA,QACV,QAAQ,IAAI;AAAA,QACZ,WAAW,IAAI,KAAK,IAAI,UAAU;AAAA,MAAA;AAAA,IAEtC,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,kCAAkC,KAAc;AAAA,IACzE;AAAA,EACF;AAAA,EAEA,MAAM,gBAAgB,MAAc,QAAkD;AACpF,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ;AAAA;AAAA;AAAA,OAG5B;AAED,WAAK,IAAI,MAAM,QAAQ,KAAK,KAAK;AAAA,IACnC,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,qCAAqC,KAAc;AAAA,IAC5E;AAAA,EACF;AAAA,EAEA,MAAM,QAAQ,MAA0C;AACtD,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,oCAAoC;AACjE,YAAM,MAAM,KAAK,IAAI,IAAI;AAEzB,UAAI,CAAC,IAAK,QAAO;AAEjB,aAAO;AAAA,QACL,IAAI,IAAI;AAAA,QACR,MAAM,IAAI;AAAA,QACV,MAAM,IAAI;AAAA,QACV,cAAc,IAAI,KAAK,IAAI,aAAa;AAAA,QACxC,MAAM,IAAI;AAAA,QACV,YAAY,KAAK,MAAM,IAAI,WAAW;AAAA,QACtC,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI,CAAA;AAAA,QACxD,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI;AAAA,MAAA;AAAA,IAE5D,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,6BAA6B,KAAc;AAAA,IACpE;AAAA,EACF;AAAA,EAEA,MAAM,WAAW,UAAoB,SAAsB,CAAA,GAAI,SAAmB,CAAA,GAAmB;AACnG,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ;AAAA;AAAA;AAAA,OAG5B;AAED,WAAK;AAAA,QACH,SAAS;AAAA,QACT,SAAS;AAAA,QACT,SAAS,aAAa,QAAA;AAAA,QACtB,SAAS;AAAA,QACT,KAAK,UAAU,SAAS,UAAU;AAAA,QAClC,OAAO,SAAS,IAAI,KAAK,UAAU,MAAM,IAAI;AAAA,QAC7C,OAAO,SAAS,IAAI,KAAK,UAAU,MAAM,IAAI;AAAA,MAAA;AAAA,IAEjD,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,gCAAgC,KAAc;AAAA,IACvE;AAAA,EACF;AAAA,EAEA,MAAM,WAAW,MAA6B;AAC5C,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,kCAAkC;AAC/D,WAAK,IAAI,IAAI;AAAA,IACf,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,gCAAgC,KAAc;AAAA,IACvE;AAAA,EACF;AAAA,EAEA,MAAM,oBAAoB,eAA8C;AACtE,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,uCAAuC;AACpE,YAAM,OAAO,KAAK,IAAI,GAAG,aAAa,GAAG;AAEzC,aAAO,KAAK,IAAI,CAAA,SAAQ;AAAA,QACtB,IAAI,IAAI;AAAA,QACR,MAAM,IAAI;AAAA,QACV,MAAM,IAAI;AAAA,QACV,cAAc,IAAI,KAAK,IAAI,aAAa;AAAA,QACxC,MAAM,IAAI;AAAA,QACV,YAAY,KAAK,MAAM,IAAI,WAAW;AAAA,QACtC,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI,CAAA;AAAA,QACxD,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI;AAAA,MAAA,EACxD;AAAA,IACJ,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,oCAAoC,KAAc;AAAA,IAC3E;AAAA,EACF;AAAA,EAEA,QAAc;AACZ,SAAK,GAAG,MAAA;AAAA,EACV;AACF;AAEA,eAAsB,kBAAkB,QAA0E;AAChH,QAAM,SAAS,IAAI,cAAc,MAAM;AACvC,QAAM,SAAS,IAAI,aAAa,MAAM;AAEtC,QAAM,OAAO,iBAAA;AAEb,SAAO,EAAE,QAAQ,OAAA;AACnB;AAEA,eAAsB,aAAa,QAA4C;AAC7E,SAAO,IAAI,SAAS,MAAM;AAC5B;AAWA,eAAsB,iBAAuC;AAC3D,QAAM,SAAS,MAAM,OAAO,aAAa,EAAE,KAAK,CAAA,MAAK,EAAE,YAAY;AACnE,QAAM,SAAS,IAAI,cAAc,MAAM;AAEvC,MAAI;AACF,UAAM,kBAAkB,OAAO,GAAG,QAAQ,4DAA4D;AACtG,UAAM,mBAAmB,gBAAgB,IAAI,WAAW;AAExD,UAAM,YAAY,OAAO,GAAG,QAAQ,qCAAqC;AACzE,UAAM,aAAa,UAAU,IAAA;AAE7B,UAAM,aAAa,OAAO,GAAG,QAAQ,8FAA8F;AACnI,UAAM,cAAc,WAAW,IAAA;AAE/B,UAAM,kBAAkB,OAAO,GAAG,QAAQ,yDAAyD;AACnG,UAAM,oBAAoB,gBAAgB,IAAA;AAE1C,UAAM,aAAa,OAAO,GAAG,QAAQ,6DAA6D;AAClG,UAAM,YAAY,WAAW,IAAA;AAE7B,UAAM,YAAsB,CAAA;AAC5B,cAAU,QAAQ,CAAA,QAAO;AACvB,UAAI;AACF,cAAM,SAAS,KAAK,MAAM,IAAI,WAAW;AACzC,kBAAU,KAAK,GAAG,MAAM;AAAA,MAC1B,QAAQ;AACN,kBAAU,KAAK,4BAA4B;AAAA,MAC7C;AAAA,IACF,CAAC;AAED,UAAM,KAAK,MAAM,OAAO,IAAI;AAC5B,QAAI,eAAe;AACnB,QAAI;AACF,YAAM,QAAQ,GAAG,SAAS,OAAO,QAAQ,UAAU;AACnD,YAAM,cAAc,MAAM;AAC1B,UAAI,cAAc,OAAO,MAAM;AAC7B,uBAAe,IAAI,eAAe,OAAO,OAAO,QAAQ,CAAC,CAAC;AAAA,MAC5D,WAAW,cAAc,MAAM;AAC7B,uBAAe,IAAI,cAAc,MAAM,QAAQ,CAAC,CAAC;AAAA,MACnD,OAAO;AACL,uBAAe,GAAG,WAAW;AAAA,MAC/B;AAAA,IACF,QAAQ;AACN,qBAAe;AAAA,IACjB;AAEA,WAAO;AAAA,MACL,oBAAoB,iBAAiB;AAAA,MACrC,cAAc,WAAW;AAAA,MACzB,eAAe,YAAY,SAAS;AAAA,MACpC;AAAA,MACA,aAAa,kBAAkB,eAAe,IAAI,KAAK,kBAAkB,YAAY,EAAE,YAAA,IAAgB;AAAA,MACvG,QAAQ;AAAA,IAAA;AAAA,EAEZ,UAAA;AACE,WAAO,MAAA;AAAA,EACT;AACF;"}
|
|
1
|
+
{"version":3,"file":"storage.js","sources":["../src/storage.ts"],"sourcesContent":["import Database from 'better-sqlite3';\nimport { Config } from './config.js';\nimport { FileInfo, ChunkInfo, ensureDirectory } from './utils.js';\nimport { dirname } from 'path';\n\nexport interface DirectoryRecord {\n id: number;\n path: string;\n status: 'pending' | 'indexing' | 'completed' | 'failed';\n indexedAt: Date;\n}\n\nexport interface FileRecord {\n id: number;\n path: string;\n size: number;\n modifiedTime: Date;\n hash: string;\n parentDirs: string[];\n chunks: ChunkInfo[];\n errors?: string[];\n}\n\nexport interface QdrantPoint {\n id: string | number;\n vector: number[];\n payload: {\n filePath: string;\n chunkId: string;\n parentDirectories: string[];\n };\n score?: number;\n}\n\nexport class StorageError extends Error {\n constructor(message: string, public override cause?: Error) {\n super(message);\n this.name = 'StorageError';\n }\n}\n\nexport class QdrantClient {\n constructor(private config: Config) {}\n\n async healthCheck(): Promise<boolean> {\n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/healthz`);\n return response.ok;\n } catch {\n return false;\n }\n }\n\n async createCollection(): Promise<void> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const checkResponse = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}`);\n if (checkResponse.ok) {\n return;\n }\n\n const createResponse = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}`, {\n method: 'PUT',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n vectors: {\n size: 768,\n distance: 'Cosine'\n }\n })\n });\n\n if (!createResponse.ok) {\n throw new Error(`Failed to create collection: ${createResponse.statusText}`);\n }\n } catch (error) {\n throw new StorageError(`Failed to create Qdrant collection`, error as Error);\n }\n }\n\n async upsertPoints(points: QdrantPoint[]): Promise<void> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points`, {\n method: 'PUT',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({ points })\n });\n\n if (!response.ok) {\n const errorText = await response.text();\n throw new Error(`Failed to upsert points: ${response.status} ${response.statusText} - ${errorText}`);\n }\n } catch (error) {\n throw new StorageError(`Failed to upsert points to Qdrant`, error as Error);\n }\n }\n\n async searchPoints(vector: number[], limit: number = 10): Promise<QdrantPoint[]> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points/search`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n vector,\n limit,\n with_payload: true\n })\n });\n\n if (!response.ok) {\n throw new Error(`Failed to search points: ${response.statusText}`);\n }\n\n const data = await response.json();\n return data.result.map((item: { id: string | number; vector: number[]; payload: Record<string, unknown>; score: number }) => ({\n id: item.id,\n vector: item.vector,\n payload: item.payload,\n score: item.score\n }));\n } catch (error) {\n throw new StorageError(`Failed to search points in Qdrant`, error as Error);\n }\n }\n\n async deletePoints(ids: (string | number)[]): Promise<void> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points/delete`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({ points: ids })\n });\n\n if (!response.ok) {\n throw new Error(`Failed to delete points: ${response.statusText}`);\n }\n } catch (error) {\n throw new StorageError(`Failed to delete points from Qdrant`, error as Error);\n }\n }\n\n async deletePointsByFileHash(fileHash: string): Promise<void> {\n const collectionName = this.config.storage.qdrantCollection;\n \n try {\n const response = await fetch(`${this.config.storage.qdrantEndpoint}/collections/${collectionName}/points/delete`, {\n method: 'POST',\n headers: { 'Content-Type': 'application/json' },\n body: JSON.stringify({\n filter: {\n must: [\n {\n key: 'fileHash',\n match: { value: fileHash }\n }\n ]\n }\n })\n });\n\n if (!response.ok) {\n const errorText = await response.text();\n throw new Error(`Failed to delete points by file hash: ${response.status} ${response.statusText} - ${errorText}`);\n }\n } catch (error) {\n throw new StorageError(`Failed to delete points by file hash from Qdrant`, error as Error);\n }\n }\n}\n\nexport class SQLiteStorage {\n public db: Database.Database;\n\n constructor(private config: Config) {\n this.db = this.initializeDatabase();\n }\n\n private initializeDatabase(): Database.Database {\n try {\n ensureDirectory(dirname(this.config.storage.sqlitePath));\n \n const db = new Database(this.config.storage.sqlitePath);\n \n db.exec(`\n CREATE TABLE IF NOT EXISTS directories (\n id INTEGER PRIMARY KEY,\n path TEXT UNIQUE NOT NULL,\n status TEXT DEFAULT 'pending',\n indexed_at INTEGER DEFAULT 0\n );\n\n CREATE TABLE IF NOT EXISTS files (\n id INTEGER PRIMARY KEY,\n path TEXT UNIQUE NOT NULL,\n size INTEGER NOT NULL,\n modified_time INTEGER NOT NULL,\n hash TEXT NOT NULL,\n parent_dirs TEXT NOT NULL,\n chunks_json TEXT,\n errors_json TEXT\n );\n\n CREATE INDEX IF NOT EXISTS idx_files_path ON files(path);\n CREATE INDEX IF NOT EXISTS idx_files_hash ON files(hash);\n CREATE INDEX IF NOT EXISTS idx_directories_path ON directories(path);\n `);\n\n return db;\n } catch (error) {\n throw new StorageError(`Failed to initialize SQLite database`, error as Error);\n }\n }\n\n async getDirectory(path: string): Promise<DirectoryRecord | null> {\n try {\n const stmt = this.db.prepare('SELECT * FROM directories WHERE path = ?');\n const row = stmt.get(path) as { id: number; path: string; status: 'pending' | 'indexing' | 'completed' | 'failed'; indexed_at: number } | undefined;\n \n if (!row) return null;\n \n return {\n id: row.id,\n path: row.path,\n status: row.status,\n indexedAt: new Date(row.indexed_at)\n };\n } catch (error) {\n throw new StorageError(`Failed to get directory record`, error as Error);\n }\n }\n\n async upsertDirectory(path: string, status: DirectoryRecord['status']): Promise<void> {\n try {\n const stmt = this.db.prepare(`\n INSERT OR REPLACE INTO directories (path, status, indexed_at)\n VALUES (?, ?, ?)\n `);\n \n stmt.run(path, status, Date.now());\n } catch (error) {\n throw new StorageError(`Failed to upsert directory record`, error as Error);\n }\n }\n\n async getFile(path: string): Promise<FileRecord | null> {\n try {\n const stmt = this.db.prepare('SELECT * FROM files WHERE path = ?');\n const row = stmt.get(path) as { id: number; path: string; size: number; modified_time: number; hash: string; parent_dirs: string; chunks_json: string | null; errors_json: string | null } | undefined;\n \n if (!row) return null;\n \n return {\n id: row.id,\n path: row.path,\n size: row.size,\n modifiedTime: new Date(row.modified_time * 1000),\n hash: row.hash,\n parentDirs: JSON.parse(row.parent_dirs),\n chunks: row.chunks_json ? JSON.parse(row.chunks_json) : [],\n errors: row.errors_json ? JSON.parse(row.errors_json) : undefined\n };\n } catch (error) {\n throw new StorageError(`Failed to get file record`, error as Error);\n }\n }\n\n async upsertFile(fileInfo: FileInfo, chunks: ChunkInfo[] = [], errors: string[] = []): Promise<void> {\n try {\n const stmt = this.db.prepare(`\n INSERT OR REPLACE INTO files (path, size, modified_time, hash, parent_dirs, chunks_json, errors_json)\n VALUES (?, ?, ?, ?, ?, ?, ?)\n `);\n \n stmt.run(\n fileInfo.path,\n fileInfo.size,\n fileInfo.modifiedTime.getTime(),\n fileInfo.hash,\n JSON.stringify(fileInfo.parentDirs),\n chunks.length > 0 ? JSON.stringify(chunks) : null,\n errors.length > 0 ? JSON.stringify(errors) : null\n );\n } catch (error) {\n throw new StorageError(`Failed to upsert file record`, error as Error);\n }\n }\n\n async deleteFile(path: string): Promise<void> {\n try {\n const stmt = this.db.prepare('DELETE FROM files WHERE path = ?');\n stmt.run(path);\n } catch (error) {\n throw new StorageError(`Failed to delete file record`, error as Error);\n }\n }\n\n async getFilesByDirectory(directoryPath: string): Promise<FileRecord[]> {\n try {\n const stmt = this.db.prepare('SELECT * FROM files WHERE path LIKE ?');\n const rows = stmt.all(`${directoryPath}%`) as { id: number; path: string; size: number; modified_time: number; hash: string; parent_dirs: string; chunks_json: string | null; errors_json: string | null }[];\n \n return rows.map(row => ({\n id: row.id,\n path: row.path,\n size: row.size,\n modifiedTime: new Date(row.modified_time * 1000),\n hash: row.hash,\n parentDirs: JSON.parse(row.parent_dirs),\n chunks: row.chunks_json ? JSON.parse(row.chunks_json) : [],\n errors: row.errors_json ? JSON.parse(row.errors_json) : undefined\n }));\n } catch (error) {\n throw new StorageError(`Failed to get files by directory`, error as Error);\n }\n }\n\n close(): void {\n this.db.close();\n }\n}\n\nexport async function initializeStorage(config: Config): Promise<{ sqlite: SQLiteStorage; qdrant: QdrantClient }> {\n const sqlite = new SQLiteStorage(config);\n const qdrant = new QdrantClient(config);\n \n await qdrant.createCollection();\n \n return { sqlite, qdrant };\n}\n\nexport async function initDatabase(dbPath: string): Promise<Database.Database> {\n return new Database(dbPath);\n}\n\nexport interface DirectoryStatus {\n path: string;\n status: string;\n filesCount: number;\n chunksCount: number;\n lastIndexed: string | null;\n errors: string[];\n}\n\nexport interface IndexStatus {\n directoriesIndexed: number;\n filesIndexed: number;\n chunksIndexed: number;\n databaseSize: string;\n lastIndexed: string | null;\n errors: string[];\n directories: DirectoryStatus[];\n qdrantConsistency: {\n isConsistent: boolean;\n issues: string[];\n };\n}\n\nasync function checkQdrantConsistency(sqlite: SQLiteStorage, config: Config): Promise<{ isConsistent: boolean; issues: string[] }> {\n const issues: string[] = [];\n \n try {\n const qdrant = new QdrantClient(config);\n const isHealthy = await qdrant.healthCheck();\n \n if (!isHealthy) {\n issues.push('Qdrant vector database is not running or accessible');\n return { isConsistent: false, issues };\n }\n \n const filesWithChunksStmt = sqlite.db.prepare('SELECT COUNT(*) as count FROM files WHERE chunks_json IS NOT NULL');\n const filesWithChunks = filesWithChunksStmt.get() as { count: number };\n \n const totalChunksStmt = sqlite.db.prepare('SELECT SUM(json_array_length(chunks_json)) as count FROM files WHERE chunks_json IS NOT NULL');\n const totalChunks = totalChunksStmt.get() as { count: number | null };\n \n if (filesWithChunks.count > 0 && (totalChunks.count || 0) === 0) {\n issues.push('Files exist but no chunks found - possible data corruption');\n }\n \n const collectionName = config.storage.qdrantCollection;\n try {\n const response = await fetch(`${config.storage.qdrantEndpoint}/collections/${collectionName}`);\n if (!response.ok) {\n issues.push(`Vector collection '${collectionName}' not found (normal during first-time setup)`);\n return { isConsistent: false, issues };\n }\n \n const collectionInfo = await response.json();\n const qdrantPointCount = collectionInfo.result?.points_count || 0;\n const sqliteChunkCount = totalChunks.count || 0;\n \n if (Math.abs(qdrantPointCount - sqliteChunkCount) > 0) {\n if (qdrantPointCount > sqliteChunkCount) {\n issues.push(`Extra vectors in database: ${qdrantPointCount} vectors vs ${sqliteChunkCount} indexed chunks (normal during cleanup)`);\n } else {\n issues.push(`Missing vectors: ${sqliteChunkCount} indexed chunks vs ${qdrantPointCount} vectors (normal during indexing)`);\n }\n }\n } catch (error) {\n issues.push(`Cannot verify vector database status: ${error}`);\n }\n \n } catch (error) {\n issues.push(`Database status check failed: ${error}`);\n }\n \n return {\n isConsistent: issues.length === 0,\n issues\n };\n}\n\nexport async function getIndexStatus(): Promise<IndexStatus> {\n const config = await import('./config.js').then(m => m.loadConfig());\n const sqlite = new SQLiteStorage(config);\n \n try {\n const directoriesStmt = sqlite.db.prepare('SELECT COUNT(*) as count FROM directories WHERE status = ?');\n const directoriesCount = directoriesStmt.get('completed') as { count: number };\n \n const filesStmt = sqlite.db.prepare('SELECT COUNT(*) as count FROM files');\n const filesCount = filesStmt.get() as { count: number };\n \n const chunksStmt = sqlite.db.prepare('SELECT SUM(json_array_length(chunks_json)) as count FROM files WHERE chunks_json IS NOT NULL');\n const chunksCount = chunksStmt.get() as { count: number | null };\n \n const lastIndexedStmt = sqlite.db.prepare('SELECT MAX(indexed_at) as last_indexed FROM directories WHERE indexed_at > 0');\n const lastIndexedResult = lastIndexedStmt.get() as { last_indexed: number | null };\n \n const errorsStmt = sqlite.db.prepare('SELECT errors_json FROM files WHERE errors_json IS NOT NULL');\n const errorRows = errorsStmt.all() as { errors_json: string }[];\n \n const allErrors: string[] = [];\n errorRows.forEach(row => {\n try {\n const errors = JSON.parse(row.errors_json);\n allErrors.push(...errors);\n } catch {\n allErrors.push('Failed to parse error JSON');\n }\n });\n \n const directoriesDetailStmt = sqlite.db.prepare(`\n SELECT \n d.path,\n d.status,\n d.indexed_at,\n COUNT(f.id) as files_count,\n COALESCE(SUM(json_array_length(f.chunks_json)), 0) as chunks_count\n FROM directories d\n LEFT JOIN files f ON f.parent_dirs LIKE '%' || d.path || '%'\n GROUP BY d.id, d.path, d.status, d.indexed_at\n ORDER BY d.indexed_at DESC\n `);\n const directoryDetails = directoriesDetailStmt.all() as { id: number; path: string; status: 'pending' | 'indexing' | 'completed' | 'failed'; indexed_at: number; files_count: number; chunks_count: number }[];\n \n const directories: DirectoryStatus[] = directoryDetails.map(row => {\n const errorsByDirStmt = sqlite.db.prepare(`\n SELECT errors_json FROM files \n WHERE parent_dirs LIKE '%' || ? || '%' AND errors_json IS NOT NULL\n `);\n const dirErrors = errorsByDirStmt.all(row.path) as { errors_json: string }[];\n \n const dirErrorsList: string[] = [];\n dirErrors.forEach(errorRow => {\n try {\n const errors = JSON.parse(errorRow.errors_json);\n dirErrorsList.push(...errors);\n } catch {\n dirErrorsList.push('Failed to parse error JSON');\n }\n });\n \n return {\n path: row.path,\n status: row.status,\n filesCount: row.files_count,\n chunksCount: row.chunks_count,\n lastIndexed: row.indexed_at && row.indexed_at > 0 ? new Date(row.indexed_at).toISOString() : null,\n errors: dirErrorsList\n };\n });\n \n const qdrantConsistency = await checkQdrantConsistency(sqlite, config);\n \n const fs = await import('fs');\n let databaseSize = '0 KB';\n try {\n const stats = fs.statSync(config.storage.sqlitePath);\n const sizeInBytes = stats.size;\n if (sizeInBytes > 1024 * 1024) {\n databaseSize = `${(sizeInBytes / (1024 * 1024)).toFixed(2)} MB`;\n } else if (sizeInBytes > 1024) {\n databaseSize = `${(sizeInBytes / 1024).toFixed(2)} KB`;\n } else {\n databaseSize = `${sizeInBytes} bytes`;\n }\n } catch {\n databaseSize = 'Unknown';\n }\n \n return {\n directoriesIndexed: directoriesCount.count,\n filesIndexed: filesCount.count,\n chunksIndexed: chunksCount.count || 0,\n databaseSize,\n lastIndexed: lastIndexedResult.last_indexed ? new Date(lastIndexedResult.last_indexed).toISOString() : null,\n errors: allErrors,\n directories,\n qdrantConsistency\n };\n } finally {\n sqlite.close();\n }\n}"],"names":[],"mappings":";;;AAkCO,MAAM,qBAAqB,MAAM;AAAA,EACtC,YAAY,SAAiC,OAAe;AAC1D,UAAM,OAAO;AAD8B,SAAA,QAAA;AAE3C,SAAK,OAAO;AAAA,EACd;AACF;AAEO,MAAM,aAAa;AAAA,EACxB,YAAoB,QAAgB;AAAhB,SAAA,SAAA;AAAA,EAAiB;AAAA,EAErC,MAAM,cAAgC;AACpC,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,UAAU;AAC5E,aAAO,SAAS;AAAA,IAClB,QAAQ;AACN,aAAO;AAAA,IACT;AAAA,EACF;AAAA,EAEA,MAAM,mBAAkC;AACtC,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,gBAAgB,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,EAAE;AACvG,UAAI,cAAc,IAAI;AACpB;AAAA,MACF;AAEA,YAAM,iBAAiB,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,IAAI;AAAA,QACxG,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU;AAAA,UACnB,SAAS;AAAA,YACP,MAAM;AAAA,YACN,UAAU;AAAA,UAAA;AAAA,QACZ,CACD;AAAA,MAAA,CACF;AAED,UAAI,CAAC,eAAe,IAAI;AACtB,cAAM,IAAI,MAAM,gCAAgC,eAAe,UAAU,EAAE;AAAA,MAC7E;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,sCAAsC,KAAc;AAAA,IAC7E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,QAAsC;AACvD,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,WAAW;AAAA,QACzG,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU,EAAE,QAAQ;AAAA,MAAA,CAChC;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,YAAY,MAAM,SAAS,KAAA;AACjC,cAAM,IAAI,MAAM,4BAA4B,SAAS,MAAM,IAAI,SAAS,UAAU,MAAM,SAAS,EAAE;AAAA,MACrG;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,qCAAqC,KAAc;AAAA,IAC5E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,QAAkB,QAAgB,IAA4B;AAC/E,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,kBAAkB;AAAA,QAChH,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU;AAAA,UACnB;AAAA,UACA;AAAA,UACA,cAAc;AAAA,QAAA,CACf;AAAA,MAAA,CACF;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,4BAA4B,SAAS,UAAU,EAAE;AAAA,MACnE;AAEA,YAAM,OAAO,MAAM,SAAS,KAAA;AAC5B,aAAO,KAAK,OAAO,IAAI,CAAC,UAAsG;AAAA,QAC5H,IAAI,KAAK;AAAA,QACT,QAAQ,KAAK;AAAA,QACb,SAAS,KAAK;AAAA,QACd,OAAO,KAAK;AAAA,MAAA,EACZ;AAAA,IACJ,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,qCAAqC,KAAc;AAAA,IAC5E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,KAAyC;AAC1D,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,kBAAkB;AAAA,QAChH,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU,EAAE,QAAQ,KAAK;AAAA,MAAA,CACrC;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,IAAI,MAAM,4BAA4B,SAAS,UAAU,EAAE;AAAA,MACnE;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,uCAAuC,KAAc;AAAA,IAC9E;AAAA,EACF;AAAA,EAEA,MAAM,uBAAuB,UAAiC;AAC5D,UAAM,iBAAiB,KAAK,OAAO,QAAQ;AAE3C,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,KAAK,OAAO,QAAQ,cAAc,gBAAgB,cAAc,kBAAkB;AAAA,QAChH,QAAQ;AAAA,QACR,SAAS,EAAE,gBAAgB,mBAAA;AAAA,QAC3B,MAAM,KAAK,UAAU;AAAA,UACnB,QAAQ;AAAA,YACN,MAAM;AAAA,cACJ;AAAA,gBACE,KAAK;AAAA,gBACL,OAAO,EAAE,OAAO,SAAA;AAAA,cAAS;AAAA,YAC3B;AAAA,UACF;AAAA,QACF,CACD;AAAA,MAAA,CACF;AAED,UAAI,CAAC,SAAS,IAAI;AAChB,cAAM,YAAY,MAAM,SAAS,KAAA;AACjC,cAAM,IAAI,MAAM,yCAAyC,SAAS,MAAM,IAAI,SAAS,UAAU,MAAM,SAAS,EAAE;AAAA,MAClH;AAAA,IACF,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,oDAAoD,KAAc;AAAA,IAC3F;AAAA,EACF;AACF;AAEO,MAAM,cAAc;AAAA,EAGzB,YAAoB,QAAgB;AAAhB,SAAA,SAAA;AAClB,SAAK,KAAK,KAAK,mBAAA;AAAA,EACjB;AAAA,EAJO;AAAA,EAMC,qBAAwC;AAC9C,QAAI;AACF,sBAAgB,QAAQ,KAAK,OAAO,QAAQ,UAAU,CAAC;AAEvD,YAAM,KAAK,IAAI,SAAS,KAAK,OAAO,QAAQ,UAAU;AAEtD,SAAG,KAAK;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,OAsBP;AAED,aAAO;AAAA,IACT,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,wCAAwC,KAAc;AAAA,IAC/E;AAAA,EACF;AAAA,EAEA,MAAM,aAAa,MAA+C;AAChE,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,0CAA0C;AACvE,YAAM,MAAM,KAAK,IAAI,IAAI;AAEzB,UAAI,CAAC,IAAK,QAAO;AAEjB,aAAO;AAAA,QACL,IAAI,IAAI;AAAA,QACR,MAAM,IAAI;AAAA,QACV,QAAQ,IAAI;AAAA,QACZ,WAAW,IAAI,KAAK,IAAI,UAAU;AAAA,MAAA;AAAA,IAEtC,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,kCAAkC,KAAc;AAAA,IACzE;AAAA,EACF;AAAA,EAEA,MAAM,gBAAgB,MAAc,QAAkD;AACpF,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ;AAAA;AAAA;AAAA,OAG5B;AAED,WAAK,IAAI,MAAM,QAAQ,KAAK,KAAK;AAAA,IACnC,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,qCAAqC,KAAc;AAAA,IAC5E;AAAA,EACF;AAAA,EAEA,MAAM,QAAQ,MAA0C;AACtD,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,oCAAoC;AACjE,YAAM,MAAM,KAAK,IAAI,IAAI;AAEzB,UAAI,CAAC,IAAK,QAAO;AAEjB,aAAO;AAAA,QACL,IAAI,IAAI;AAAA,QACR,MAAM,IAAI;AAAA,QACV,MAAM,IAAI;AAAA,QACV,cAAc,IAAI,KAAK,IAAI,gBAAgB,GAAI;AAAA,QAC/C,MAAM,IAAI;AAAA,QACV,YAAY,KAAK,MAAM,IAAI,WAAW;AAAA,QACtC,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI,CAAA;AAAA,QACxD,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI;AAAA,MAAA;AAAA,IAE5D,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,6BAA6B,KAAc;AAAA,IACpE;AAAA,EACF;AAAA,EAEA,MAAM,WAAW,UAAoB,SAAsB,CAAA,GAAI,SAAmB,CAAA,GAAmB;AACnG,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ;AAAA;AAAA;AAAA,OAG5B;AAED,WAAK;AAAA,QACH,SAAS;AAAA,QACT,SAAS;AAAA,QACT,SAAS,aAAa,QAAA;AAAA,QACtB,SAAS;AAAA,QACT,KAAK,UAAU,SAAS,UAAU;AAAA,QAClC,OAAO,SAAS,IAAI,KAAK,UAAU,MAAM,IAAI;AAAA,QAC7C,OAAO,SAAS,IAAI,KAAK,UAAU,MAAM,IAAI;AAAA,MAAA;AAAA,IAEjD,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,gCAAgC,KAAc;AAAA,IACvE;AAAA,EACF;AAAA,EAEA,MAAM,WAAW,MAA6B;AAC5C,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,kCAAkC;AAC/D,WAAK,IAAI,IAAI;AAAA,IACf,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,gCAAgC,KAAc;AAAA,IACvE;AAAA,EACF;AAAA,EAEA,MAAM,oBAAoB,eAA8C;AACtE,QAAI;AACF,YAAM,OAAO,KAAK,GAAG,QAAQ,uCAAuC;AACpE,YAAM,OAAO,KAAK,IAAI,GAAG,aAAa,GAAG;AAEzC,aAAO,KAAK,IAAI,CAAA,SAAQ;AAAA,QACtB,IAAI,IAAI;AAAA,QACR,MAAM,IAAI;AAAA,QACV,MAAM,IAAI;AAAA,QACV,cAAc,IAAI,KAAK,IAAI,gBAAgB,GAAI;AAAA,QAC/C,MAAM,IAAI;AAAA,QACV,YAAY,KAAK,MAAM,IAAI,WAAW;AAAA,QACtC,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI,CAAA;AAAA,QACxD,QAAQ,IAAI,cAAc,KAAK,MAAM,IAAI,WAAW,IAAI;AAAA,MAAA,EACxD;AAAA,IACJ,SAAS,OAAO;AACd,YAAM,IAAI,aAAa,oCAAoC,KAAc;AAAA,IAC3E;AAAA,EACF;AAAA,EAEA,QAAc;AACZ,SAAK,GAAG,MAAA;AAAA,EACV;AACF;AAEA,eAAsB,kBAAkB,QAA0E;AAChH,QAAM,SAAS,IAAI,cAAc,MAAM;AACvC,QAAM,SAAS,IAAI,aAAa,MAAM;AAEtC,QAAM,OAAO,iBAAA;AAEb,SAAO,EAAE,QAAQ,OAAA;AACnB;AAEA,eAAsB,aAAa,QAA4C;AAC7E,SAAO,IAAI,SAAS,MAAM;AAC5B;AAyBA,eAAe,uBAAuB,QAAuB,QAAsE;AACjI,QAAM,SAAmB,CAAA;AAEzB,MAAI;AACF,UAAM,SAAS,IAAI,aAAa,MAAM;AACtC,UAAM,YAAY,MAAM,OAAO,YAAA;AAE/B,QAAI,CAAC,WAAW;AACd,aAAO,KAAK,qDAAqD;AACjE,aAAO,EAAE,cAAc,OAAO,OAAA;AAAA,IAChC;AAEA,UAAM,sBAAsB,OAAO,GAAG,QAAQ,mEAAmE;AACjH,UAAM,kBAAkB,oBAAoB,IAAA;AAE5C,UAAM,kBAAkB,OAAO,GAAG,QAAQ,8FAA8F;AACxI,UAAM,cAAc,gBAAgB,IAAA;AAEpC,QAAI,gBAAgB,QAAQ,MAAM,YAAY,SAAS,OAAO,GAAG;AAC/D,aAAO,KAAK,4DAA4D;AAAA,IAC1E;AAEA,UAAM,iBAAiB,OAAO,QAAQ;AACtC,QAAI;AACF,YAAM,WAAW,MAAM,MAAM,GAAG,OAAO,QAAQ,cAAc,gBAAgB,cAAc,EAAE;AAC7F,UAAI,CAAC,SAAS,IAAI;AAChB,eAAO,KAAK,sBAAsB,cAAc,8CAA8C;AAC9F,eAAO,EAAE,cAAc,OAAO,OAAA;AAAA,MAChC;AAEA,YAAM,iBAAiB,MAAM,SAAS,KAAA;AACtC,YAAM,mBAAmB,eAAe,QAAQ,gBAAgB;AAChE,YAAM,mBAAmB,YAAY,SAAS;AAE9C,UAAI,KAAK,IAAI,mBAAmB,gBAAgB,IAAI,GAAG;AACrD,YAAI,mBAAmB,kBAAkB;AACvC,iBAAO,KAAK,8BAA8B,gBAAgB,eAAe,gBAAgB,yCAAyC;AAAA,QACpI,OAAO;AACL,iBAAO,KAAK,oBAAoB,gBAAgB,sBAAsB,gBAAgB,mCAAmC;AAAA,QAC3H;AAAA,MACF;AAAA,IACF,SAAS,OAAO;AACd,aAAO,KAAK,yCAAyC,KAAK,EAAE;AAAA,IAC9D;AAAA,EAEF,SAAS,OAAO;AACd,WAAO,KAAK,iCAAiC,KAAK,EAAE;AAAA,EACtD;AAEA,SAAO;AAAA,IACL,cAAc,OAAO,WAAW;AAAA,IAChC;AAAA,EAAA;AAEJ;AAEA,eAAsB,iBAAuC;AAC3D,QAAM,SAAS,MAAM,OAAO,aAAa,EAAE,KAAK,CAAA,MAAK,EAAE,YAAY;AACnE,QAAM,SAAS,IAAI,cAAc,MAAM;AAEvC,MAAI;AACF,UAAM,kBAAkB,OAAO,GAAG,QAAQ,4DAA4D;AACtG,UAAM,mBAAmB,gBAAgB,IAAI,WAAW;AAExD,UAAM,YAAY,OAAO,GAAG,QAAQ,qCAAqC;AACzE,UAAM,aAAa,UAAU,IAAA;AAE7B,UAAM,aAAa,OAAO,GAAG,QAAQ,8FAA8F;AACnI,UAAM,cAAc,WAAW,IAAA;AAE/B,UAAM,kBAAkB,OAAO,GAAG,QAAQ,8EAA8E;AACxH,UAAM,oBAAoB,gBAAgB,IAAA;AAE1C,UAAM,aAAa,OAAO,GAAG,QAAQ,6DAA6D;AAClG,UAAM,YAAY,WAAW,IAAA;AAE7B,UAAM,YAAsB,CAAA;AAC5B,cAAU,QAAQ,CAAA,QAAO;AACvB,UAAI;AACF,cAAM,SAAS,KAAK,MAAM,IAAI,WAAW;AACzC,kBAAU,KAAK,GAAG,MAAM;AAAA,MAC1B,QAAQ;AACN,kBAAU,KAAK,4BAA4B;AAAA,MAC7C;AAAA,IACF,CAAC;AAED,UAAM,wBAAwB,OAAO,GAAG,QAAQ;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,KAW/C;AACD,UAAM,mBAAmB,sBAAsB,IAAA;AAE/C,UAAM,cAAiC,iBAAiB,IAAI,CAAA,QAAO;AACjE,YAAM,kBAAkB,OAAO,GAAG,QAAQ;AAAA;AAAA;AAAA,OAGzC;AACD,YAAM,YAAY,gBAAgB,IAAI,IAAI,IAAI;AAE9C,YAAM,gBAA0B,CAAA;AAChC,gBAAU,QAAQ,CAAA,aAAY;AAC5B,YAAI;AACF,gBAAM,SAAS,KAAK,MAAM,SAAS,WAAW;AAC9C,wBAAc,KAAK,GAAG,MAAM;AAAA,QAC9B,QAAQ;AACN,wBAAc,KAAK,4BAA4B;AAAA,QACjD;AAAA,MACF,CAAC;AAED,aAAO;AAAA,QACL,MAAM,IAAI;AAAA,QACV,QAAQ,IAAI;AAAA,QACZ,YAAY,IAAI;AAAA,QAChB,aAAa,IAAI;AAAA,QACjB,aAAa,IAAI,cAAc,IAAI,aAAa,IAAI,IAAI,KAAK,IAAI,UAAU,EAAE,YAAA,IAAgB;AAAA,QAC7F,QAAQ;AAAA,MAAA;AAAA,IAEZ,CAAC;AAED,UAAM,oBAAoB,MAAM,uBAAuB,QAAQ,MAAM;AAErE,UAAM,KAAK,MAAM,OAAO,IAAI;AAC5B,QAAI,eAAe;AACnB,QAAI;AACF,YAAM,QAAQ,GAAG,SAAS,OAAO,QAAQ,UAAU;AACnD,YAAM,cAAc,MAAM;AAC1B,UAAI,cAAc,OAAO,MAAM;AAC7B,uBAAe,IAAI,eAAe,OAAO,OAAO,QAAQ,CAAC,CAAC;AAAA,MAC5D,WAAW,cAAc,MAAM;AAC7B,uBAAe,IAAI,cAAc,MAAM,QAAQ,CAAC,CAAC;AAAA,MACnD,OAAO;AACL,uBAAe,GAAG,WAAW;AAAA,MAC/B;AAAA,IACF,QAAQ;AACN,qBAAe;AAAA,IACjB;AAEA,WAAO;AAAA,MACL,oBAAoB,iBAAiB;AAAA,MACrC,cAAc,WAAW;AAAA,MACzB,eAAe,YAAY,SAAS;AAAA,MACpC;AAAA,MACA,aAAa,kBAAkB,eAAe,IAAI,KAAK,kBAAkB,YAAY,EAAE,YAAA,IAAgB;AAAA,MACvG,QAAQ;AAAA,MACR;AAAA,MACA;AAAA,IAAA;AAAA,EAEJ,UAAA;AACE,WAAO,MAAA;AAAA,EACT;AACF;"}
|
package/package.json
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "directory-indexer",
|
|
3
|
-
"version": "0.0.
|
|
3
|
+
"version": "0.0.14",
|
|
4
4
|
"description": "AI-powered directory indexing with semantic search for MCP servers",
|
|
5
5
|
"main": "dist/cli.js",
|
|
6
6
|
"bin": {
|
|
7
|
-
"directory-indexer": "
|
|
7
|
+
"directory-indexer": "bin/directory-indexer.js"
|
|
8
8
|
},
|
|
9
9
|
"type": "module",
|
|
10
10
|
"scripts": {
|
|
@@ -18,7 +18,8 @@
|
|
|
18
18
|
"lint": "eslint src tests --ext .ts",
|
|
19
19
|
"typecheck": "tsc --noEmit",
|
|
20
20
|
"clean": "rimraf dist",
|
|
21
|
-
"prepare": "npm run build"
|
|
21
|
+
"prepare": "npm run build",
|
|
22
|
+
"cli": "node bin/directory-indexer.js"
|
|
22
23
|
},
|
|
23
24
|
"keywords": [
|
|
24
25
|
"mcp",
|
|
@@ -42,7 +43,8 @@
|
|
|
42
43
|
"node": ">=18.0.0"
|
|
43
44
|
},
|
|
44
45
|
"files": [
|
|
45
|
-
"dist/"
|
|
46
|
+
"dist/",
|
|
47
|
+
"bin/"
|
|
46
48
|
],
|
|
47
49
|
"dependencies": {
|
|
48
50
|
"@modelcontextprotocol/sdk": "^0.6.0",
|