directory-indexer 0.0.1 → 0.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +89 -144
- package/bin/directory-indexer.js +70 -0
- package/binaries/directory-indexer-darwin-arm64 +0 -0
- package/binaries/directory-indexer-darwin-x64 +0 -0
- package/binaries/directory-indexer-linux-x64 +0 -0
- package/binaries/directory-indexer.exe-win32-x64 +0 -0
- package/package.json +33 -21
- package/scripts/build-all.js +87 -0
- package/scripts/cleanup-act.sh +44 -0
- package/scripts/postinstall.js +47 -0
- package/scripts/pre-push +40 -0
- package/scripts/start-dev-services.sh +108 -0
- package/scripts/stop-dev-services.sh +42 -0
- package/bin/cli.js +0 -17
- package/index.js +0 -13
package/README.md
CHANGED
|
@@ -1,199 +1,144 @@
|
|
|
1
1
|
# Directory Indexer
|
|
2
2
|
|
|
3
|
-
**Turn your directories into an AI-powered knowledge base
|
|
3
|
+
**Turn your directories into an AI-powered knowledge base.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
[](https://npmjs.com/package/directory-indexer)
|
|
6
|
+
[](https://crates.io/crates/directory-indexer)
|
|
7
|
+
[](https://opensource.org/licenses/MIT)
|
|
8
|
+
[](https://github.com/peteretelej/directory-indexer/actions)
|
|
6
9
|
|
|
7
|
-
|
|
10
|
+
Give AI assistants semantic search across your local files. Find relevant documents based on meaning, not just filenames.
|
|
8
11
|
|
|
9
|
-
|
|
12
|
+
## Quick Start
|
|
10
13
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
- Configuration files and deployment guides
|
|
15
|
-
|
|
16
|
-
Finding relevant files requires knowing exactly what to search for:
|
|
17
|
-
|
|
18
|
-
- Filename search needs exact words from the filename
|
|
19
|
-
- Text search needs specific terms that appear in the content
|
|
20
|
-
- No way to find conceptually related files or similar solutions to problems
|
|
21
|
-
|
|
22
|
-
## Solution
|
|
23
|
-
|
|
24
|
-
[directory-indexer](https://github.com/peteretelej/directory-indexer) creates vector embeddings of your files and provides semantic search through MCP tools. AI assistants can find relevant documents based on meaning and context.
|
|
14
|
+
```bash
|
|
15
|
+
# Install via npm
|
|
16
|
+
npm install -g directory-indexer
|
|
25
17
|
|
|
26
|
-
|
|
18
|
+
# Index your directories
|
|
19
|
+
directory-indexer index ~/Documents ~/work/docs
|
|
27
20
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
4. AI searches embeddings, finds relevant files, reads content, provides context
|
|
21
|
+
# Start MCP server for AI assistants
|
|
22
|
+
directory-indexer serve
|
|
23
|
+
```
|
|
32
24
|
|
|
33
|
-
|
|
25
|
+
**Configure with Claude Desktop:**
|
|
34
26
|
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
│ Monitor │
|
|
45
|
-
└──────────────────┘
|
|
27
|
+
```json
|
|
28
|
+
{
|
|
29
|
+
"mcpServers": {
|
|
30
|
+
"directory-indexer": {
|
|
31
|
+
"command": "directory-indexer",
|
|
32
|
+
"args": ["serve"]
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
}
|
|
46
36
|
```
|
|
47
37
|
|
|
48
|
-
|
|
38
|
+
Now ask Claude: _"Find files similar to my Redis incident reports"_ and it will search your indexed documents semantically.
|
|
49
39
|
|
|
50
|
-
|
|
51
|
-
- **File Processor**: Converts various formats (PDF, DOCX, etc.) to text (initially unimplemented)
|
|
52
|
-
- **Indexing Engine**: Extracts content, creates embeddings, handles updates
|
|
53
|
-
- **Vector Storage**: Qdrant for fast similarity search
|
|
54
|
-
- **File Monitor**: Watches for changes, incremental updates
|
|
40
|
+
## Requirements
|
|
55
41
|
|
|
56
|
-
|
|
42
|
+
- **Qdrant**: Vector database for semantic search
|
|
57
43
|
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
-
|
|
61
|
-
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
## CLI Commands
|
|
44
|
+
```bash
|
|
45
|
+
docker run -d --name qdrant \
|
|
46
|
+
-p 127.0.0.1:6333:6333 \
|
|
47
|
+
-v qdrant_storage:/qdrant/storage \
|
|
48
|
+
qdrant/qdrant
|
|
49
|
+
```
|
|
65
50
|
|
|
66
|
-
|
|
67
|
-
- `directory-indexer search <query> [path]` - Search indexed content
|
|
68
|
-
- `directory-indexer similar <file> [--limit N]` - Find similar files
|
|
69
|
-
- `directory-indexer get <file> [--chunks N-M]` - Get file content
|
|
70
|
-
- `directory-indexer serve` - Start MCP server
|
|
71
|
-
- `directory-indexer status` - Show indexing status
|
|
51
|
+
**Embedding Provider** (choose one):
|
|
72
52
|
|
|
73
|
-
|
|
53
|
+
- **Ollama** (recommended): Self-hosted embeddings
|
|
74
54
|
|
|
75
55
|
```bash
|
|
76
|
-
# Install
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
56
|
+
# Install Ollama natively for GPU support (recommended)
|
|
57
|
+
curl -fsSL https://ollama.ai/install.sh | sh
|
|
58
|
+
# Visit https://ollama.ai for Windows installer
|
|
59
|
+
ollama pull nomic-embed-text
|
|
60
|
+
|
|
61
|
+
# OR use Docker (CPU-only, slower)
|
|
62
|
+
docker run -d --name ollama \
|
|
63
|
+
-p 127.0.0.1:11434:11434 \
|
|
64
|
+
-v ollama_data:/root/.ollama \
|
|
65
|
+
ollama/ollama
|
|
66
|
+
|
|
67
|
+
docker exec ollama ollama pull nomic-embed-text
|
|
81
68
|
```
|
|
82
69
|
|
|
83
|
-
**
|
|
70
|
+
- **OpenAI**: Requires API key
|
|
84
71
|
|
|
85
|
-
|
|
86
|
-
- Embedding provider:
|
|
87
|
-
- Local: Ollama (free)
|
|
88
|
-
- Remote: OpenAI, Voyage AI, OpenRouter, or compatible API (requires API key)
|
|
89
|
-
|
|
90
|
-
## Usage
|
|
72
|
+
## Configuration
|
|
91
73
|
|
|
92
|
-
|
|
74
|
+
Directory Indexer uses environment variables for configuration. Set these if your services run on different ports or require API keys:
|
|
93
75
|
|
|
94
76
|
```bash
|
|
95
|
-
|
|
96
|
-
|
|
77
|
+
# Service endpoints (defaults shown)
|
|
78
|
+
export QDRANT_ENDPOINT="http://localhost:6333"
|
|
79
|
+
export OLLAMA_ENDPOINT="http://localhost:11434"
|
|
97
80
|
|
|
98
|
-
|
|
81
|
+
# Optional database path (default: ~/.directory-indexer/data.db)
|
|
82
|
+
export DIRECTORY_INDEXER_DB="/path/to/your/database.db"
|
|
99
83
|
|
|
100
|
-
|
|
101
|
-
#
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
# Find similar files
|
|
105
|
-
directory-indexer similar ~/work/incidents/icm-2024-0123.md
|
|
84
|
+
# Optional Qdrant collection name (default: directory-indexer)
|
|
85
|
+
# Note: Setting to "test" enables auto-cleanup for testing
|
|
86
|
+
export DIRECTORY_INDEXER_QDRANT_COLLECTION="my-custom-collection"
|
|
106
87
|
|
|
107
|
-
#
|
|
108
|
-
|
|
88
|
+
# Optional API keys
|
|
89
|
+
export QDRANT_API_KEY="your-qdrant-key"
|
|
90
|
+
export OLLAMA_API_KEY="your-ollama-key" # if using hosted Ollama
|
|
109
91
|
```
|
|
110
92
|
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
Add to Cline, Claude Desktop, or Cursor config:
|
|
93
|
+
**For MCP clients** (like Claude Desktop), configure with environment variables:
|
|
114
94
|
|
|
115
95
|
```json
|
|
116
96
|
{
|
|
117
97
|
"mcpServers": {
|
|
118
98
|
"directory-indexer": {
|
|
119
99
|
"command": "directory-indexer",
|
|
120
|
-
"args": ["serve"]
|
|
100
|
+
"args": ["serve"],
|
|
101
|
+
"env": {
|
|
102
|
+
"QDRANT_ENDPOINT": "http://localhost:6333",
|
|
103
|
+
"OLLAMA_ENDPOINT": "http://localhost:11434",
|
|
104
|
+
"DIRECTORY_INDEXER_DB": "/path/to/your/database.db"
|
|
105
|
+
}
|
|
121
106
|
}
|
|
122
107
|
}
|
|
123
108
|
}
|
|
124
109
|
```
|
|
125
110
|
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
- _"Find documentation about our Redis setup"_
|
|
129
|
-
- _"What incidents have we had with database timeouts?"_
|
|
130
|
-
- _"Show me examples of error handling in our Go services"_
|
|
111
|
+
## CLI Usage
|
|
131
112
|
|
|
132
|
-
|
|
113
|
+
```bash
|
|
114
|
+
# Index your directories
|
|
115
|
+
directory-indexer index ~/Documents ~/work/docs
|
|
133
116
|
|
|
134
|
-
|
|
117
|
+
# Search semantically
|
|
118
|
+
directory-indexer search "database timeout errors"
|
|
135
119
|
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
"embedding": {
|
|
139
|
-
"provider": "ollama",
|
|
140
|
-
"model": "nomic-embed-text",
|
|
141
|
-
"endpoint": "http://localhost:11434"
|
|
142
|
-
},
|
|
143
|
-
"storage": {
|
|
144
|
-
"type": "qdrant",
|
|
145
|
-
"endpoint": "http://localhost:6333",
|
|
146
|
-
"collection": "documents"
|
|
147
|
-
},
|
|
148
|
-
"indexing": {
|
|
149
|
-
"chunk_size": 512,
|
|
150
|
-
"overlap": 50,
|
|
151
|
-
"max_file_size": 10485760,
|
|
152
|
-
"ignore_patterns": [".git", "node_modules", "target"]
|
|
153
|
-
},
|
|
154
|
-
"monitoring": {
|
|
155
|
-
"enabled": true,
|
|
156
|
-
"batch_size": 100
|
|
157
|
-
}
|
|
158
|
-
}
|
|
159
|
-
```
|
|
120
|
+
# Find similar files
|
|
121
|
+
directory-indexer similar ~/incidents/redis-outage.md
|
|
160
122
|
|
|
161
|
-
|
|
123
|
+
# Get file content
|
|
124
|
+
directory-indexer get ~/docs/api-guide.md
|
|
162
125
|
|
|
163
|
-
|
|
164
|
-
-
|
|
165
|
-
|
|
126
|
+
# Show status
|
|
127
|
+
directory-indexer status
|
|
128
|
+
```
|
|
166
129
|
|
|
167
|
-
## Supported
|
|
130
|
+
## Supported Files
|
|
168
131
|
|
|
169
|
-
- **Text**: `.md`, `.txt
|
|
170
|
-
- **Code**:
|
|
132
|
+
- **Text**: `.md`, `.txt`
|
|
133
|
+
- **Code**: `.rs`, `.py`, `.js`, `.ts`, `.go`, `.java`, etc.
|
|
171
134
|
- **Data**: `.json`, `.yaml`, `.csv`, `.toml`
|
|
172
135
|
- **Config**: `.env`, `.conf`, `.ini`
|
|
173
|
-
- **Web**: `.html`, `.xml`
|
|
174
|
-
|
|
175
|
-
## Planned Support
|
|
176
|
-
|
|
177
|
-
- **Documents**: `.pdf`, `.docx`, `.pptx` (requires file conversion preprocessing)
|
|
178
|
-
- **Spreadsheets**: `.xlsx`, `.ods`
|
|
179
|
-
- **Archives**: Extract and index contents of `.zip`, `.tar.gz`
|
|
180
|
-
|
|
181
|
-
## Development
|
|
182
|
-
|
|
183
|
-
**Tech stack:**
|
|
184
|
-
|
|
185
|
-
- Rust (tokio async runtime)
|
|
186
|
-
- Qdrant (vector database)
|
|
187
|
-
- Ollama or OpenAI-compatible embedding API
|
|
188
|
-
- Model Context Protocol
|
|
189
|
-
- Platform-native file watching
|
|
190
136
|
|
|
191
|
-
|
|
137
|
+
## Documentation
|
|
192
138
|
|
|
193
|
-
-
|
|
194
|
-
-
|
|
195
|
-
-
|
|
196
|
-
- Local-first with optional remote embedding APIs
|
|
139
|
+
- **[API Reference](docs/designs/API.md)**: Complete CLI and MCP tool documentation
|
|
140
|
+
- **[Contributing](docs/CONTRIBUTING.md)**: Development setup and guidelines
|
|
141
|
+
- **[Design](docs/design.md)**: Architecture and technical decisions
|
|
197
142
|
|
|
198
143
|
## License
|
|
199
144
|
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
const { spawn } = require('child_process');
|
|
4
|
+
const path = require('path');
|
|
5
|
+
const fs = require('fs');
|
|
6
|
+
|
|
7
|
+
function getBinaryPath() {
|
|
8
|
+
const platform = process.platform;
|
|
9
|
+
const arch = process.arch;
|
|
10
|
+
|
|
11
|
+
let binaryName = 'directory-indexer';
|
|
12
|
+
let platformDir = '';
|
|
13
|
+
|
|
14
|
+
if (platform === 'win32') {
|
|
15
|
+
binaryName += '.exe';
|
|
16
|
+
platformDir = 'win32-x64';
|
|
17
|
+
} else if (platform === 'darwin') {
|
|
18
|
+
platformDir = arch === 'arm64' ? 'darwin-arm64' : 'darwin-x64';
|
|
19
|
+
} else if (platform === 'linux') {
|
|
20
|
+
platformDir = 'linux-x64';
|
|
21
|
+
} else {
|
|
22
|
+
console.error(`Unsupported platform: ${platform}-${arch}`);
|
|
23
|
+
process.exit(1);
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
// Try different locations for the binary
|
|
27
|
+
const possiblePaths = [
|
|
28
|
+
// Pre-built binary from npm package
|
|
29
|
+
path.join(__dirname, '..', 'binaries', `${binaryName}-${platformDir}`),
|
|
30
|
+
// Development build
|
|
31
|
+
path.join(__dirname, '..', 'target', 'release', binaryName),
|
|
32
|
+
path.join(__dirname, '..', 'target', 'debug', binaryName),
|
|
33
|
+
];
|
|
34
|
+
|
|
35
|
+
for (const binaryPath of possiblePaths) {
|
|
36
|
+
if (fs.existsSync(binaryPath)) {
|
|
37
|
+
return binaryPath;
|
|
38
|
+
}
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
console.error(`Binary not found for ${platform}-${arch}`);
|
|
42
|
+
console.error('Tried paths:', possiblePaths);
|
|
43
|
+
console.error('Run "cargo build --release" to build the binary');
|
|
44
|
+
process.exit(1);
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
function main() {
|
|
48
|
+
const binaryPath = getBinaryPath();
|
|
49
|
+
const args = process.argv.slice(2);
|
|
50
|
+
|
|
51
|
+
const child = spawn(binaryPath, args, {
|
|
52
|
+
stdio: 'inherit',
|
|
53
|
+
windowsHide: false,
|
|
54
|
+
});
|
|
55
|
+
|
|
56
|
+
child.on('error', (error) => {
|
|
57
|
+
console.error('Failed to start directory-indexer:', error.message);
|
|
58
|
+
process.exit(1);
|
|
59
|
+
});
|
|
60
|
+
|
|
61
|
+
child.on('close', (code) => {
|
|
62
|
+
process.exit(code);
|
|
63
|
+
});
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
if (require.main === module) {
|
|
67
|
+
main();
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
module.exports = { getBinaryPath };
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/package.json
CHANGED
|
@@ -1,28 +1,27 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "directory-indexer",
|
|
3
|
-
"version": "0.0.
|
|
4
|
-
"description": "
|
|
5
|
-
"main": "
|
|
3
|
+
"version": "0.0.5",
|
|
4
|
+
"description": "AI-powered directory indexing with semantic search for MCP servers",
|
|
5
|
+
"main": "bin/directory-indexer.js",
|
|
6
6
|
"bin": {
|
|
7
|
-
"directory-indexer": "
|
|
8
|
-
},
|
|
9
|
-
"directories": {
|
|
10
|
-
"doc": "docs"
|
|
7
|
+
"directory-indexer": "bin/directory-indexer.js"
|
|
11
8
|
},
|
|
12
9
|
"scripts": {
|
|
13
|
-
"
|
|
14
|
-
"
|
|
10
|
+
"build": "cargo build --release",
|
|
11
|
+
"build-all": "node scripts/build-all.js",
|
|
12
|
+
"test": "npm run unit-test",
|
|
13
|
+
"unit-test": "cargo test --lib && cargo test --test storage_tests",
|
|
14
|
+
"integration-test": "cargo test",
|
|
15
|
+
"lint": "cargo fmt --all -- --check && cargo clippy --all-targets --all-features -- -D warnings",
|
|
16
|
+
"postinstall": "node scripts/postinstall.js"
|
|
15
17
|
},
|
|
16
18
|
"keywords": [
|
|
17
19
|
"mcp",
|
|
18
|
-
"
|
|
19
|
-
"vector-database",
|
|
20
|
-
"ai",
|
|
20
|
+
"search",
|
|
21
21
|
"indexing",
|
|
22
|
-
"
|
|
23
|
-
"
|
|
24
|
-
"
|
|
25
|
-
"search"
|
|
22
|
+
"ai",
|
|
23
|
+
"semantic",
|
|
24
|
+
"cli"
|
|
26
25
|
],
|
|
27
26
|
"author": "Peter Etelej <peter@etelej.com>",
|
|
28
27
|
"license": "MIT",
|
|
@@ -30,17 +29,30 @@
|
|
|
30
29
|
"type": "git",
|
|
31
30
|
"url": "https://github.com/peteretelej/directory-indexer.git"
|
|
32
31
|
},
|
|
33
|
-
"homepage": "https://github.com/peteretelej/directory-indexer
|
|
32
|
+
"homepage": "https://github.com/peteretelej/directory-indexer",
|
|
34
33
|
"bugs": {
|
|
35
34
|
"url": "https://github.com/peteretelej/directory-indexer/issues"
|
|
36
35
|
},
|
|
37
36
|
"engines": {
|
|
38
|
-
"node": ">=
|
|
37
|
+
"node": ">=16.0.0"
|
|
39
38
|
},
|
|
39
|
+
"os": [
|
|
40
|
+
"linux",
|
|
41
|
+
"darwin",
|
|
42
|
+
"win32"
|
|
43
|
+
],
|
|
44
|
+
"cpu": [
|
|
45
|
+
"x64",
|
|
46
|
+
"arm64"
|
|
47
|
+
],
|
|
40
48
|
"files": [
|
|
41
|
-
"index.js",
|
|
42
49
|
"bin/",
|
|
50
|
+
"scripts/",
|
|
51
|
+
"binaries/",
|
|
43
52
|
"README.md",
|
|
44
53
|
"LICENSE"
|
|
45
|
-
]
|
|
46
|
-
|
|
54
|
+
],
|
|
55
|
+
"devDependencies": {
|
|
56
|
+
"cross-env": "^7.0.3"
|
|
57
|
+
}
|
|
58
|
+
}
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
const { spawn } = require('child_process');
|
|
4
|
+
const fs = require('fs');
|
|
5
|
+
const path = require('path');
|
|
6
|
+
|
|
7
|
+
const targets = [
|
|
8
|
+
{ target: 'x86_64-unknown-linux-gnu', platform: 'linux', arch: 'x64' },
|
|
9
|
+
{ target: 'x86_64-apple-darwin', platform: 'darwin', arch: 'x64' },
|
|
10
|
+
{ target: 'aarch64-apple-darwin', platform: 'darwin', arch: 'arm64' },
|
|
11
|
+
{ target: 'x86_64-pc-windows-gnu', platform: 'win32', arch: 'x64' },
|
|
12
|
+
];
|
|
13
|
+
|
|
14
|
+
function runCommand(command, args = [], options = {}) {
|
|
15
|
+
return new Promise((resolve, reject) => {
|
|
16
|
+
console.log(`Running: ${command} ${args.join(' ')}`);
|
|
17
|
+
const child = spawn(command, args, { stdio: 'inherit', ...options });
|
|
18
|
+
|
|
19
|
+
child.on('close', (code) => {
|
|
20
|
+
if (code === 0) {
|
|
21
|
+
resolve();
|
|
22
|
+
} else {
|
|
23
|
+
reject(new Error(`Command failed with exit code ${code}`));
|
|
24
|
+
}
|
|
25
|
+
});
|
|
26
|
+
|
|
27
|
+
child.on('error', reject);
|
|
28
|
+
});
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
async function buildTarget(targetInfo) {
|
|
32
|
+
const { target, platform, arch } = targetInfo;
|
|
33
|
+
|
|
34
|
+
console.log(`\n🏗️ Building for ${platform}-${arch} (${target})`);
|
|
35
|
+
|
|
36
|
+
try {
|
|
37
|
+
// Add the target if not already installed
|
|
38
|
+
await runCommand('rustup', ['target', 'add', target]);
|
|
39
|
+
|
|
40
|
+
// Build for the target
|
|
41
|
+
await runCommand('cargo', ['build', '--release', '--target', target]);
|
|
42
|
+
|
|
43
|
+
// Copy binary to binaries directory
|
|
44
|
+
const binariesDir = path.join(__dirname, '..', 'binaries');
|
|
45
|
+
if (!fs.existsSync(binariesDir)) {
|
|
46
|
+
fs.mkdirSync(binariesDir, { recursive: true });
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
const extension = platform === 'win32' ? '.exe' : '';
|
|
50
|
+
const sourcePath = path.join(__dirname, '..', 'target', target, 'release', `directory-indexer${extension}`);
|
|
51
|
+
const destPath = path.join(binariesDir, `directory-indexer-${platform}-${arch}${extension}`);
|
|
52
|
+
|
|
53
|
+
if (fs.existsSync(sourcePath)) {
|
|
54
|
+
fs.copyFileSync(sourcePath, destPath);
|
|
55
|
+
fs.chmodSync(destPath, 0o755);
|
|
56
|
+
console.log(`✅ Built ${destPath}`);
|
|
57
|
+
} else {
|
|
58
|
+
throw new Error(`Binary not found at ${sourcePath}`);
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
} catch (error) {
|
|
62
|
+
console.error(`❌ Failed to build ${platform}-${arch}:`, error.message);
|
|
63
|
+
throw error;
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
|
|
67
|
+
async function main() {
|
|
68
|
+
console.log('🚀 Building directory-indexer for all platforms...\n');
|
|
69
|
+
|
|
70
|
+
// Build for all targets
|
|
71
|
+
for (const target of targets) {
|
|
72
|
+
try {
|
|
73
|
+
await buildTarget(target);
|
|
74
|
+
} catch (error) {
|
|
75
|
+
console.error(`Failed to build ${target.platform}-${target.arch}, continuing...`);
|
|
76
|
+
}
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
console.log('\n✨ Build complete! Binaries are in the binaries/ directory.');
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
if (require.main === module) {
|
|
83
|
+
main().catch((error) => {
|
|
84
|
+
console.error('Build failed:', error);
|
|
85
|
+
process.exit(1);
|
|
86
|
+
});
|
|
87
|
+
}
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
|
|
3
|
+
# Cleanup script for act containers and networks
|
|
4
|
+
# Act doesn't clean up after itself, causing port conflicts
|
|
5
|
+
|
|
6
|
+
echo "🧹 Cleaning up act containers and networks..."
|
|
7
|
+
|
|
8
|
+
# Colors for output
|
|
9
|
+
RED='\033[0;31m'
|
|
10
|
+
GREEN='\033[0;32m'
|
|
11
|
+
YELLOW='\033[1;33m'
|
|
12
|
+
NC='\033[0m' # No Color
|
|
13
|
+
|
|
14
|
+
# Count act containers
|
|
15
|
+
ACT_CONTAINERS=$(docker ps -aq --filter "name=act-" | wc -l)
|
|
16
|
+
ACT_NETWORKS=$(docker network ls | grep act | wc -l)
|
|
17
|
+
|
|
18
|
+
if [ "$ACT_CONTAINERS" -eq 0 ] && [ "$ACT_NETWORKS" -eq 0 ]; then
|
|
19
|
+
echo -e "${GREEN}✅ No act containers or networks to clean up${NC}"
|
|
20
|
+
exit 0
|
|
21
|
+
fi
|
|
22
|
+
|
|
23
|
+
echo -e "${YELLOW}Found $ACT_CONTAINERS act containers and $ACT_NETWORKS act networks${NC}"
|
|
24
|
+
|
|
25
|
+
# Stop act containers
|
|
26
|
+
if [ "$ACT_CONTAINERS" -gt 0 ]; then
|
|
27
|
+
echo -e "${YELLOW}Stopping act containers...${NC}"
|
|
28
|
+
docker stop $(docker ps -q --filter "name=act-") 2>/dev/null || true
|
|
29
|
+
|
|
30
|
+
echo -e "${YELLOW}Removing act containers...${NC}"
|
|
31
|
+
docker rm $(docker ps -aq --filter "name=act-") 2>/dev/null || true
|
|
32
|
+
fi
|
|
33
|
+
|
|
34
|
+
# Remove act networks
|
|
35
|
+
if [ "$ACT_NETWORKS" -gt 0 ]; then
|
|
36
|
+
echo -e "${YELLOW}Removing act networks...${NC}"
|
|
37
|
+
docker network ls | grep act | awk '{print $1}' | xargs -r docker network rm 2>/dev/null || true
|
|
38
|
+
fi
|
|
39
|
+
|
|
40
|
+
echo -e "${GREEN}🎉 Act cleanup completed!${NC}"
|
|
41
|
+
|
|
42
|
+
# Show what's still running
|
|
43
|
+
echo -e "${YELLOW}Currently running containers:${NC}"
|
|
44
|
+
docker ps --format "table {{.Names}}\t{{.Ports}}" | head -10
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
const fs = require('fs');
|
|
4
|
+
const path = require('path');
|
|
5
|
+
|
|
6
|
+
function makeExecutable(filePath) {
|
|
7
|
+
if (fs.existsSync(filePath)) {
|
|
8
|
+
try {
|
|
9
|
+
fs.chmodSync(filePath, 0o755);
|
|
10
|
+
console.log(`Made ${filePath} executable`);
|
|
11
|
+
} catch (error) {
|
|
12
|
+
console.warn(`Failed to make ${filePath} executable:`, error.message);
|
|
13
|
+
}
|
|
14
|
+
}
|
|
15
|
+
}
|
|
16
|
+
|
|
17
|
+
function main() {
|
|
18
|
+
const platform = process.platform;
|
|
19
|
+
const arch = process.arch;
|
|
20
|
+
|
|
21
|
+
// Make the wrapper script executable
|
|
22
|
+
const wrapperPath = path.join(__dirname, '..', 'bin', 'directory-indexer.js');
|
|
23
|
+
makeExecutable(wrapperPath);
|
|
24
|
+
|
|
25
|
+
// Make the platform-specific binary executable
|
|
26
|
+
let binaryName, platformSuffix;
|
|
27
|
+
|
|
28
|
+
if (platform === 'win32') {
|
|
29
|
+
binaryName = 'directory-indexer.exe-win32-x64';
|
|
30
|
+
} else if (platform === 'darwin') {
|
|
31
|
+
binaryName = arch === 'arm64' ? 'directory-indexer-darwin-arm64' : 'directory-indexer-darwin-x64';
|
|
32
|
+
} else if (platform === 'linux') {
|
|
33
|
+
binaryName = 'directory-indexer-linux-x64';
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
if (binaryName) {
|
|
37
|
+
const binaryPath = path.join(__dirname, '..', 'binaries', binaryName);
|
|
38
|
+
makeExecutable(binaryPath);
|
|
39
|
+
}
|
|
40
|
+
|
|
41
|
+
console.log('directory-indexer installed successfully!');
|
|
42
|
+
console.log('Run "directory-indexer --help" to get started.');
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
if (require.main === module) {
|
|
46
|
+
main();
|
|
47
|
+
}
|
package/scripts/pre-push
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
|
|
3
|
+
# Pre-push hook for directory-indexer
|
|
4
|
+
# Run essential quality checks before pushing code
|
|
5
|
+
|
|
6
|
+
set -e
|
|
7
|
+
|
|
8
|
+
echo "Running pre-push checks..."
|
|
9
|
+
|
|
10
|
+
# Check if we're in a Rust project
|
|
11
|
+
if [ ! -f "Cargo.toml" ]; then
|
|
12
|
+
echo "ERROR: Cargo.toml not found. This script should be run from the project root."
|
|
13
|
+
exit 1
|
|
14
|
+
fi
|
|
15
|
+
|
|
16
|
+
echo "Running clippy (linter)..."
|
|
17
|
+
# Check main library code with strict linting
|
|
18
|
+
cargo clippy --lib --all-features -- -D warnings -D clippy::uninlined_format_args
|
|
19
|
+
# Check other targets with standard linting (tests can be less strict)
|
|
20
|
+
cargo clippy --bins --all-features -- -D warnings
|
|
21
|
+
|
|
22
|
+
echo "Checking code formatting..."
|
|
23
|
+
cargo fmt --check
|
|
24
|
+
|
|
25
|
+
echo "Running essential tests..."
|
|
26
|
+
# Only run fast, essential tests - skip the slow integration tests
|
|
27
|
+
cargo test --lib
|
|
28
|
+
cargo test --test error_scenarios_tests
|
|
29
|
+
cargo test --test storage_tests
|
|
30
|
+
|
|
31
|
+
echo "Checking for security vulnerabilities..."
|
|
32
|
+
if command -v cargo-audit >/dev/null 2>&1; then
|
|
33
|
+
cargo audit
|
|
34
|
+
else
|
|
35
|
+
echo "Installing cargo-audit..."
|
|
36
|
+
cargo install cargo-audit
|
|
37
|
+
cargo audit
|
|
38
|
+
fi
|
|
39
|
+
|
|
40
|
+
echo "All pre-push checks passed!"
|
|
@@ -0,0 +1,108 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
|
|
3
|
+
# Development services startup script
|
|
4
|
+
# Sets up isolated development environment using environment variables
|
|
5
|
+
|
|
6
|
+
set -e
|
|
7
|
+
|
|
8
|
+
echo "Starting development services for directory-indexer..."
|
|
9
|
+
|
|
10
|
+
# Colors for output
|
|
11
|
+
RED='\033[0;31m'
|
|
12
|
+
GREEN='\033[0;32m'
|
|
13
|
+
YELLOW='\033[1;33m'
|
|
14
|
+
NC='\033[0m' # No Color
|
|
15
|
+
|
|
16
|
+
# Standard ports for development
|
|
17
|
+
DEV_QDRANT_PORT=6333
|
|
18
|
+
DEV_OLLAMA_PORT=11434
|
|
19
|
+
|
|
20
|
+
# Check if Docker is running
|
|
21
|
+
if ! docker info >/dev/null 2>&1; then
|
|
22
|
+
echo -e "${RED}Docker is not running. Please start Docker first.${NC}"
|
|
23
|
+
exit 1
|
|
24
|
+
fi
|
|
25
|
+
|
|
26
|
+
# Start Qdrant on standard port
|
|
27
|
+
echo -e "${YELLOW}Starting Qdrant on port $DEV_QDRANT_PORT...${NC}"
|
|
28
|
+
docker stop qdrant-dev || true
|
|
29
|
+
docker rm qdrant-dev || true
|
|
30
|
+
|
|
31
|
+
docker run -d \
|
|
32
|
+
--name qdrant-dev \
|
|
33
|
+
-p $DEV_QDRANT_PORT:6333 \
|
|
34
|
+
-v qdrant_dev_storage:/qdrant/storage \
|
|
35
|
+
qdrant/qdrant
|
|
36
|
+
|
|
37
|
+
# Wait for Qdrant to be ready
|
|
38
|
+
echo -e "${YELLOW}Waiting for Qdrant to be ready...${NC}"
|
|
39
|
+
for i in {1..30}; do
|
|
40
|
+
if curl -s http://localhost:$DEV_QDRANT_PORT/ >/dev/null 2>&1; then
|
|
41
|
+
echo -e "${GREEN}Qdrant is ready on http://localhost:$DEV_QDRANT_PORT${NC}"
|
|
42
|
+
break
|
|
43
|
+
fi
|
|
44
|
+
if [ $i -eq 30 ]; then
|
|
45
|
+
echo -e "${RED}Qdrant failed to start after 30 seconds${NC}"
|
|
46
|
+
exit 1
|
|
47
|
+
fi
|
|
48
|
+
sleep 1
|
|
49
|
+
done
|
|
50
|
+
|
|
51
|
+
# Start Ollama on standard port
|
|
52
|
+
echo -e "${YELLOW}Starting Ollama on port $DEV_OLLAMA_PORT...${NC}"
|
|
53
|
+
docker stop ollama-dev || true
|
|
54
|
+
docker rm ollama-dev || true
|
|
55
|
+
|
|
56
|
+
docker run -d \
|
|
57
|
+
--name ollama-dev \
|
|
58
|
+
-p $DEV_OLLAMA_PORT:11434 \
|
|
59
|
+
-v ollama_dev_data:/root/.ollama \
|
|
60
|
+
ollama/ollama
|
|
61
|
+
|
|
62
|
+
# Wait for Ollama to be ready
|
|
63
|
+
echo -e "${YELLOW}Waiting for Ollama to be ready...${NC}"
|
|
64
|
+
for i in {1..30}; do
|
|
65
|
+
if curl -s http://localhost:$DEV_OLLAMA_PORT/api/tags >/dev/null 2>&1; then
|
|
66
|
+
echo -e "${GREEN}Ollama is ready on http://localhost:$DEV_OLLAMA_PORT${NC}"
|
|
67
|
+
break
|
|
68
|
+
fi
|
|
69
|
+
if [ $i -eq 30 ]; then
|
|
70
|
+
echo -e "${RED}Ollama failed to start after 30 seconds${NC}"
|
|
71
|
+
exit 1
|
|
72
|
+
fi
|
|
73
|
+
sleep 1
|
|
74
|
+
done
|
|
75
|
+
|
|
76
|
+
# Pull required embedding model
|
|
77
|
+
echo -e "${YELLOW}Ensuring nomic-embed-text model is available...${NC}"
|
|
78
|
+
if docker exec ollama-dev ollama list | grep -q "nomic-embed-text"; then
|
|
79
|
+
echo -e "${GREEN}nomic-embed-text model already available${NC}"
|
|
80
|
+
else
|
|
81
|
+
echo -e "${YELLOW}Pulling nomic-embed-text model (this may take a while)...${NC}"
|
|
82
|
+
docker exec ollama-dev ollama pull nomic-embed-text
|
|
83
|
+
echo -e "${GREEN}nomic-embed-text model ready${NC}"
|
|
84
|
+
fi
|
|
85
|
+
|
|
86
|
+
# Set environment variables for development
|
|
87
|
+
export QDRANT_ENDPOINT="http://localhost:$DEV_QDRANT_PORT"
|
|
88
|
+
export OLLAMA_ENDPOINT="http://localhost:$DEV_OLLAMA_PORT"
|
|
89
|
+
|
|
90
|
+
echo -e "${GREEN}Development services are ready!${NC}"
|
|
91
|
+
echo
|
|
92
|
+
echo -e "${YELLOW}Services:${NC}"
|
|
93
|
+
echo " Qdrant: http://localhost:$DEV_QDRANT_PORT"
|
|
94
|
+
echo " Ollama: http://localhost:$DEV_OLLAMA_PORT"
|
|
95
|
+
echo
|
|
96
|
+
echo -e "${YELLOW}Environment variables set:${NC}"
|
|
97
|
+
echo " QDRANT_ENDPOINT=$QDRANT_ENDPOINT"
|
|
98
|
+
echo " OLLAMA_ENDPOINT=$OLLAMA_ENDPOINT"
|
|
99
|
+
echo
|
|
100
|
+
echo -e "${YELLOW}To use these services in your current shell:${NC}"
|
|
101
|
+
echo " export QDRANT_ENDPOINT=$QDRANT_ENDPOINT"
|
|
102
|
+
echo " export OLLAMA_ENDPOINT=$OLLAMA_ENDPOINT"
|
|
103
|
+
echo
|
|
104
|
+
echo -e "${YELLOW}To stop services:${NC}"
|
|
105
|
+
echo " ./scripts/stop-dev-services.sh"
|
|
106
|
+
echo
|
|
107
|
+
echo -e "${YELLOW}To run tests with dev services:${NC}"
|
|
108
|
+
echo " QDRANT_ENDPOINT=$QDRANT_ENDPOINT OLLAMA_ENDPOINT=$OLLAMA_ENDPOINT cargo test --test connectivity_tests"
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
|
|
3
|
+
# Development services stop script
|
|
4
|
+
|
|
5
|
+
set -e
|
|
6
|
+
|
|
7
|
+
echo "Stopping development services..."
|
|
8
|
+
|
|
9
|
+
# Colors for output
|
|
10
|
+
RED='\033[0;31m'
|
|
11
|
+
GREEN='\033[0;32m'
|
|
12
|
+
YELLOW='\033[1;33m'
|
|
13
|
+
NC='\033[0m' # No Color
|
|
14
|
+
|
|
15
|
+
# Stop and remove Qdrant dev container
|
|
16
|
+
if docker ps -q --filter "name=qdrant-dev" | grep -q .; then
|
|
17
|
+
echo -e "${YELLOW}Stopping Qdrant development container...${NC}"
|
|
18
|
+
docker stop qdrant-dev
|
|
19
|
+
docker rm qdrant-dev
|
|
20
|
+
echo -e "${GREEN}Qdrant development container stopped${NC}"
|
|
21
|
+
else
|
|
22
|
+
echo -e "${YELLOW}Qdrant development container not running${NC}"
|
|
23
|
+
fi
|
|
24
|
+
|
|
25
|
+
# Stop and remove Ollama dev container
|
|
26
|
+
if docker ps -q --filter "name=ollama-dev" | grep -q .; then
|
|
27
|
+
echo -e "${YELLOW}Stopping Ollama development container...${NC}"
|
|
28
|
+
docker stop ollama-dev
|
|
29
|
+
docker rm ollama-dev
|
|
30
|
+
echo -e "${GREEN}Ollama development container stopped${NC}"
|
|
31
|
+
else
|
|
32
|
+
echo -e "${YELLOW}Ollama development container not running${NC}"
|
|
33
|
+
fi
|
|
34
|
+
|
|
35
|
+
echo -e "${GREEN}Development services stopped!${NC}"
|
|
36
|
+
echo
|
|
37
|
+
echo -e "${YELLOW}Note:${NC} Data volumes are preserved:"
|
|
38
|
+
echo " qdrant_dev_storage"
|
|
39
|
+
echo " ollama_dev_data"
|
|
40
|
+
echo
|
|
41
|
+
echo -e "${YELLOW}To completely remove data volumes:${NC}"
|
|
42
|
+
echo " docker volume rm qdrant_dev_storage ollama_dev_data"
|
package/bin/cli.js
DELETED
|
@@ -1,17 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env node
|
|
2
|
-
|
|
3
|
-
const packageJson = require('../package.json');
|
|
4
|
-
|
|
5
|
-
console.log(`${packageJson.name} v${packageJson.version}`);
|
|
6
|
-
console.log('This is a placeholder package. Full implementation coming soon!');
|
|
7
|
-
console.log('');
|
|
8
|
-
console.log('Planned commands:');
|
|
9
|
-
console.log(' directory-indexer index <paths...> - Index directories');
|
|
10
|
-
console.log(' directory-indexer search <query> - Search indexed content');
|
|
11
|
-
console.log(' directory-indexer similar <file> - Find similar files');
|
|
12
|
-
console.log(' directory-indexer serve - Start MCP server');
|
|
13
|
-
console.log(' directory-indexer status - Show indexing status');
|
|
14
|
-
console.log('');
|
|
15
|
-
console.log('GitHub: https://github.com/peteretelej/directory-indexer');
|
|
16
|
-
|
|
17
|
-
process.exit(0);
|
package/index.js
DELETED
|
@@ -1,13 +0,0 @@
|
|
|
1
|
-
#!/usr/bin/env node
|
|
2
|
-
|
|
3
|
-
console.log('Directory Indexer v0.0.1');
|
|
4
|
-
console.log('This is a placeholder package. Full implementation coming soon!');
|
|
5
|
-
console.log('');
|
|
6
|
-
console.log('Stay tuned for:');
|
|
7
|
-
console.log('- MCP server for semantic directory indexing');
|
|
8
|
-
console.log('- Vector-based file search with AI assistants');
|
|
9
|
-
console.log('- Support for multiple file formats');
|
|
10
|
-
console.log('');
|
|
11
|
-
console.log('GitHub: https://github.com/peteretelej/directory-indexer');
|
|
12
|
-
|
|
13
|
-
process.exit(0);
|