directory-indexer 0.0.6 → 0.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -4,48 +4,25 @@
|
|
|
4
4
|
|
|
5
5
|
[](https://npmjs.com/package/directory-indexer)
|
|
6
6
|
[](https://crates.io/crates/directory-indexer)
|
|
7
|
+
[](https://codecov.io/gh/peteretelej/directory-indexer)
|
|
8
|
+
[](https://www.rust-lang.org)
|
|
7
9
|
[](https://opensource.org/licenses/MIT)
|
|
8
10
|
[](https://github.com/peteretelej/directory-indexer/actions)
|
|
9
11
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
## Quick Start
|
|
13
|
-
|
|
14
|
-
```bash
|
|
15
|
-
# Install via npm
|
|
16
|
-
npm install -g directory-indexer
|
|
17
|
-
|
|
18
|
-
# Index your directories
|
|
19
|
-
directory-indexer index ~/Documents ~/work/docs
|
|
20
|
-
|
|
21
|
-
# Start MCP server for AI assistants
|
|
22
|
-
directory-indexer serve
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
**Configure with Claude Desktop:**
|
|
26
|
-
|
|
27
|
-
```json
|
|
28
|
-
{
|
|
29
|
-
"mcpServers": {
|
|
30
|
-
"directory-indexer": {
|
|
31
|
-
"command": "directory-indexer",
|
|
32
|
-
"args": ["serve"]
|
|
33
|
-
}
|
|
34
|
-
}
|
|
35
|
-
}
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
Now ask Claude: _"Find files similar to my Redis incident reports"_ and it will search your indexed documents semantically.
|
|
12
|
+
Self-hosted semantic search for local files. Enable AI assistants to search your documents using vector embeddings and MCP integration.
|
|
39
13
|
|
|
40
14
|
## Setup
|
|
41
15
|
|
|
42
|
-
|
|
16
|
+
Directory Indexer runs locally on your machine or server. It uses an embedding provider (such as Ollama) to create vector embeddings of your files and stores them in a Qdrant vector database for fast semantic search. Both services can run remotely if needed.
|
|
17
|
+
|
|
18
|
+
Setup requires two services:
|
|
43
19
|
|
|
44
20
|
### 1. Qdrant Vector Database
|
|
45
21
|
|
|
46
22
|
Choose one option:
|
|
47
23
|
|
|
48
24
|
**Docker (recommended for most users):**
|
|
25
|
+
|
|
49
26
|
```bash
|
|
50
27
|
docker run -d --name qdrant \
|
|
51
28
|
-p 127.0.0.1:6333:6333 \
|
|
@@ -53,6 +30,9 @@ docker run -d --name qdrant \
|
|
|
53
30
|
qdrant/qdrant
|
|
54
31
|
```
|
|
55
32
|
|
|
33
|
+
- This option requires [Docker](https://docs.docker.com/get-docker/)
|
|
34
|
+
- Runs Qdrant on docker container, uses a named volume `qdrant_storage` for persistent storage.
|
|
35
|
+
|
|
56
36
|
**Alternative:** Install natively from [qdrant.tech](https://qdrant.tech/documentation/guides/installation/)
|
|
57
37
|
|
|
58
38
|
### 2. Embedding Provider
|
|
@@ -60,6 +40,7 @@ docker run -d --name qdrant \
|
|
|
60
40
|
Choose one option:
|
|
61
41
|
|
|
62
42
|
**Option A: Ollama (recommended - free, runs locally)**
|
|
43
|
+
|
|
63
44
|
```bash
|
|
64
45
|
# Install Ollama
|
|
65
46
|
curl -fsSL https://ollama.ai/install.sh | sh # Linux/macOS
|
|
@@ -69,7 +50,11 @@ curl -fsSL https://ollama.ai/install.sh | sh # Linux/macOS
|
|
|
69
50
|
ollama pull nomic-embed-text
|
|
70
51
|
```
|
|
71
52
|
|
|
53
|
+
- You can also [run Ollama via Docker](https://ollama.com/blog/ollama-is-now-available-as-an-official-docker-image)
|
|
54
|
+
- GPU support may require additional configuration
|
|
55
|
+
|
|
72
56
|
**Option B: OpenAI (requires paid API key)**
|
|
57
|
+
|
|
73
58
|
```bash
|
|
74
59
|
export OPENAI_API_KEY="your-api-key-here"
|
|
75
60
|
```
|
|
@@ -77,16 +62,73 @@ export OPENAI_API_KEY="your-api-key-here"
|
|
|
77
62
|
### Quick Verification
|
|
78
63
|
|
|
79
64
|
Test your setup:
|
|
65
|
+
|
|
80
66
|
```bash
|
|
81
67
|
# Check Qdrant
|
|
82
68
|
curl http://localhost:6333/collections
|
|
83
69
|
|
|
84
|
-
# Check Ollama
|
|
70
|
+
# Check Ollama
|
|
85
71
|
curl http://localhost:11434/api/tags
|
|
86
72
|
```
|
|
87
73
|
|
|
88
74
|
If either fails, directory-indexer will show a helpful error with setup guidance.
|
|
89
75
|
|
|
76
|
+
## Installation
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
npm install -g directory-indexer
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## Usage
|
|
83
|
+
|
|
84
|
+
### MCP Integration
|
|
85
|
+
|
|
86
|
+
Configure with Claude Desktop:
|
|
87
|
+
|
|
88
|
+
```json
|
|
89
|
+
{
|
|
90
|
+
"mcpServers": {
|
|
91
|
+
"directory-indexer": {
|
|
92
|
+
"command": "directory-indexer",
|
|
93
|
+
"args": ["serve"]
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Start the MCP server:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
directory-indexer serve
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Your AI assistant (Claude, Cline, Copilot, etc.) can now search your indexed documents semantically. Ask: _"Find API authentication examples"_, _"Show me incidents similar to this"_, or _"Find troubleshooting guides on SQL deadlocks"_.
|
|
106
|
+
|
|
107
|
+
### CLI Commands
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
# Index your directories
|
|
111
|
+
# Linux/macOS
|
|
112
|
+
directory-indexer index /home/user/projects/api-docs /mnt/work/incident-reports
|
|
113
|
+
# Windows
|
|
114
|
+
directory-indexer index "C:\work\documentation" "D:\projects\my-app\docs"
|
|
115
|
+
|
|
116
|
+
# Search semantically
|
|
117
|
+
directory-indexer search "database timeout errors"
|
|
118
|
+
|
|
119
|
+
# Find similar files
|
|
120
|
+
# Linux/macOS
|
|
121
|
+
directory-indexer similar /mnt/work/incident-reports/redis-outage.md
|
|
122
|
+
# Windows
|
|
123
|
+
directory-indexer similar "C:\work\incidents\redis-outage.md"
|
|
124
|
+
|
|
125
|
+
# Get file content
|
|
126
|
+
directory-indexer get /home/user/projects/api-docs/auth-guide.md
|
|
127
|
+
|
|
128
|
+
# Show status
|
|
129
|
+
directory-indexer status
|
|
130
|
+
```
|
|
131
|
+
|
|
90
132
|
## Configuration
|
|
91
133
|
|
|
92
134
|
Directory Indexer uses environment variables for configuration. Set these if your services run on different ports or require API keys:
|
|
@@ -97,7 +139,10 @@ export QDRANT_ENDPOINT="http://localhost:6333"
|
|
|
97
139
|
export OLLAMA_ENDPOINT="http://localhost:11434"
|
|
98
140
|
|
|
99
141
|
# Optional data directory (default: ~/.directory-indexer)
|
|
100
|
-
|
|
142
|
+
# Linux/macOS
|
|
143
|
+
export DIRECTORY_INDEXER_DATA_DIR="/opt/directory-indexer-data"
|
|
144
|
+
# Windows
|
|
145
|
+
set DIRECTORY_INDEXER_DATA_DIR=D:\data\directory-indexer
|
|
101
146
|
|
|
102
147
|
# Optional Qdrant collection name (default: directory-indexer)
|
|
103
148
|
# Note: Setting to "test" enables auto-cleanup for testing
|
|
@@ -119,32 +164,13 @@ export OLLAMA_API_KEY="your-ollama-key" # if using hosted Ollama
|
|
|
119
164
|
"env": {
|
|
120
165
|
"QDRANT_ENDPOINT": "http://localhost:6333",
|
|
121
166
|
"OLLAMA_ENDPOINT": "http://localhost:11434",
|
|
122
|
-
"DIRECTORY_INDEXER_DATA_DIR": "/
|
|
167
|
+
"DIRECTORY_INDEXER_DATA_DIR": "/opt/directory-indexer-data"
|
|
123
168
|
}
|
|
124
169
|
}
|
|
125
170
|
}
|
|
126
171
|
}
|
|
127
172
|
```
|
|
128
173
|
|
|
129
|
-
## CLI Usage
|
|
130
|
-
|
|
131
|
-
```bash
|
|
132
|
-
# Index your directories
|
|
133
|
-
directory-indexer index ~/Documents ~/work/docs
|
|
134
|
-
|
|
135
|
-
# Search semantically
|
|
136
|
-
directory-indexer search "database timeout errors"
|
|
137
|
-
|
|
138
|
-
# Find similar files
|
|
139
|
-
directory-indexer similar ~/incidents/redis-outage.md
|
|
140
|
-
|
|
141
|
-
# Get file content
|
|
142
|
-
directory-indexer get ~/docs/api-guide.md
|
|
143
|
-
|
|
144
|
-
# Show status
|
|
145
|
-
directory-indexer status
|
|
146
|
-
```
|
|
147
|
-
|
|
148
174
|
## Supported Files
|
|
149
175
|
|
|
150
176
|
- **Text**: `.md`, `.txt`
|
|
@@ -158,6 +184,25 @@ directory-indexer status
|
|
|
158
184
|
- **[Contributing](docs/CONTRIBUTING.md)**: Development setup and guidelines
|
|
159
185
|
- **[Design](docs/design.md)**: Architecture and technical decisions
|
|
160
186
|
|
|
187
|
+
## Usage Examples
|
|
188
|
+
|
|
189
|
+
Once indexed, try these queries with your AI assistant:
|
|
190
|
+
|
|
191
|
+
**Search by concept:**
|
|
192
|
+
- _"Find API authentication examples"_
|
|
193
|
+
- _"Show me error handling patterns"_
|
|
194
|
+
- _"Find configuration for Redis"_
|
|
195
|
+
|
|
196
|
+
**Find similar content:**
|
|
197
|
+
- _"Show me incidents similar to this outage report"_ *(when you have an incident file open)*
|
|
198
|
+
- _"Find documentation like this API guide"_ *(when viewing an API doc)*
|
|
199
|
+
- _"What files are similar to my deployment script?"_
|
|
200
|
+
|
|
201
|
+
**Troubleshoot issues:**
|
|
202
|
+
- _"Find troubleshooting guides on SQL deadlocks"_
|
|
203
|
+
- _"Show me solutions for timeout errors"_
|
|
204
|
+
- _"Find debugging tips for performance issues"_
|
|
205
|
+
|
|
161
206
|
## License
|
|
162
207
|
|
|
163
208
|
MIT
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/package.json
CHANGED
package/scripts/pre-push
CHANGED
|
@@ -13,6 +13,21 @@ if [ ! -f "Cargo.toml" ]; then
|
|
|
13
13
|
exit 1
|
|
14
14
|
fi
|
|
15
15
|
|
|
16
|
+
echo "Checking version synchronization..."
|
|
17
|
+
# Extract version from Cargo.toml
|
|
18
|
+
CARGO_VERSION=$(grep '^version = ' Cargo.toml | head -1 | sed 's/version = "\(.*\)"/\1/')
|
|
19
|
+
|
|
20
|
+
# Extract version from package.json
|
|
21
|
+
NPM_VERSION=$(grep '"version":' package.json | head -1 | sed 's/.*"version": "\(.*\)".*/\1/')
|
|
22
|
+
|
|
23
|
+
if [ "$CARGO_VERSION" != "$NPM_VERSION" ]; then
|
|
24
|
+
echo "ERROR: Version mismatch detected!"
|
|
25
|
+
echo " Cargo.toml version: $CARGO_VERSION"
|
|
26
|
+
echo " package.json version: $NPM_VERSION"
|
|
27
|
+
echo " Please ensure both files have the same version."
|
|
28
|
+
exit 1
|
|
29
|
+
fi
|
|
30
|
+
|
|
16
31
|
echo "Running clippy (linter)..."
|
|
17
32
|
# Check main library code with strict linting
|
|
18
33
|
cargo clippy --lib --all-features -- -D warnings -D clippy::uninlined_format_args
|
|
@@ -22,11 +37,12 @@ cargo clippy --bins --all-features -- -D warnings
|
|
|
22
37
|
echo "Checking code formatting..."
|
|
23
38
|
cargo fmt --check
|
|
24
39
|
|
|
25
|
-
echo "Running
|
|
26
|
-
|
|
40
|
+
echo "Running tests..."
|
|
41
|
+
export DIRECTORY_INDEXER_DATA_DIR=/tmp/directory-indexer-test
|
|
42
|
+
export DIRECTORY_INDEXER_QDRANT_COLLECTION=directory-indexer-test
|
|
43
|
+
|
|
27
44
|
cargo test --lib
|
|
28
|
-
cargo test
|
|
29
|
-
cargo test --test storage_tests
|
|
45
|
+
cargo test tests
|
|
30
46
|
|
|
31
47
|
echo "Checking for security vulnerabilities..."
|
|
32
48
|
if command -v cargo-audit >/dev/null 2>&1; then
|