directory-indexer 0.0.5 → 0.0.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
CHANGED
|
@@ -4,25 +4,86 @@
|
|
|
4
4
|
|
|
5
5
|
[](https://npmjs.com/package/directory-indexer)
|
|
6
6
|
[](https://crates.io/crates/directory-indexer)
|
|
7
|
+
[](https://codecov.io/gh/peteretelej/directory-indexer)
|
|
8
|
+
[](https://www.rust-lang.org)
|
|
7
9
|
[](https://opensource.org/licenses/MIT)
|
|
8
10
|
[](https://github.com/peteretelej/directory-indexer/actions)
|
|
9
11
|
|
|
10
|
-
|
|
12
|
+
Self-hosted semantic search for local files. Enable AI assistants to search your documents using vector embeddings and MCP integration.
|
|
11
13
|
|
|
12
|
-
##
|
|
14
|
+
## Setup
|
|
15
|
+
|
|
16
|
+
Directory Indexer runs locally on your machine or server. It uses an embedding provider (such as Ollama) to create vector embeddings of your files and stores them in a Qdrant vector database for fast semantic search. Both services can run remotely if needed.
|
|
17
|
+
|
|
18
|
+
Setup requires two services:
|
|
19
|
+
|
|
20
|
+
### 1. Qdrant Vector Database
|
|
21
|
+
|
|
22
|
+
Choose one option:
|
|
23
|
+
|
|
24
|
+
**Docker (recommended for most users):**
|
|
13
25
|
|
|
14
26
|
```bash
|
|
15
|
-
|
|
16
|
-
|
|
27
|
+
docker run -d --name qdrant \
|
|
28
|
+
-p 127.0.0.1:6333:6333 \
|
|
29
|
+
-v qdrant_storage:/qdrant/storage \
|
|
30
|
+
qdrant/qdrant
|
|
31
|
+
```
|
|
17
32
|
|
|
18
|
-
|
|
19
|
-
|
|
33
|
+
- This option requires [Docker](https://docs.docker.com/get-docker/)
|
|
34
|
+
- Runs Qdrant on docker container, uses a named volume `qdrant_storage` for persistent storage.
|
|
20
35
|
|
|
21
|
-
|
|
22
|
-
|
|
36
|
+
**Alternative:** Install natively from [qdrant.tech](https://qdrant.tech/documentation/guides/installation/)
|
|
37
|
+
|
|
38
|
+
### 2. Embedding Provider
|
|
39
|
+
|
|
40
|
+
Choose one option:
|
|
41
|
+
|
|
42
|
+
**Option A: Ollama (recommended - free, runs locally)**
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
# Install Ollama
|
|
46
|
+
curl -fsSL https://ollama.ai/install.sh | sh # Linux/macOS
|
|
47
|
+
# For Windows: Download from https://ollama.ai
|
|
48
|
+
|
|
49
|
+
# Pull the embedding model
|
|
50
|
+
ollama pull nomic-embed-text
|
|
23
51
|
```
|
|
24
52
|
|
|
25
|
-
|
|
53
|
+
- You can also [run Ollama via Docker](https://ollama.com/blog/ollama-is-now-available-as-an-official-docker-image)
|
|
54
|
+
- GPU support may require additional configuration
|
|
55
|
+
|
|
56
|
+
**Option B: OpenAI (requires paid API key)**
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
export OPENAI_API_KEY="your-api-key-here"
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Quick Verification
|
|
63
|
+
|
|
64
|
+
Test your setup:
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
# Check Qdrant
|
|
68
|
+
curl http://localhost:6333/collections
|
|
69
|
+
|
|
70
|
+
# Check Ollama
|
|
71
|
+
curl http://localhost:11434/api/tags
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
If either fails, directory-indexer will show a helpful error with setup guidance.
|
|
75
|
+
|
|
76
|
+
## Installation
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
npm install -g directory-indexer
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## Usage
|
|
83
|
+
|
|
84
|
+
### MCP Integration
|
|
85
|
+
|
|
86
|
+
Configure with Claude Desktop:
|
|
26
87
|
|
|
27
88
|
```json
|
|
28
89
|
{
|
|
@@ -35,39 +96,38 @@ directory-indexer serve
|
|
|
35
96
|
}
|
|
36
97
|
```
|
|
37
98
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
## Requirements
|
|
41
|
-
|
|
42
|
-
- **Qdrant**: Vector database for semantic search
|
|
99
|
+
Start the MCP server:
|
|
43
100
|
|
|
44
101
|
```bash
|
|
45
|
-
|
|
46
|
-
-p 127.0.0.1:6333:6333 \
|
|
47
|
-
-v qdrant_storage:/qdrant/storage \
|
|
48
|
-
qdrant/qdrant
|
|
102
|
+
directory-indexer serve
|
|
49
103
|
```
|
|
50
104
|
|
|
51
|
-
|
|
105
|
+
Your AI assistant (Claude, Cline, Copilot, etc.) can now search your indexed documents semantically. Ask: _"Find API authentication examples"_, _"Show me incidents similar to this"_, or _"Find troubleshooting guides on SQL deadlocks"_.
|
|
52
106
|
|
|
53
|
-
|
|
107
|
+
### CLI Commands
|
|
54
108
|
|
|
55
109
|
```bash
|
|
56
|
-
#
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
110
|
+
# Index your directories
|
|
111
|
+
# Linux/macOS
|
|
112
|
+
directory-indexer index /home/user/projects/api-docs /mnt/work/incident-reports
|
|
113
|
+
# Windows
|
|
114
|
+
directory-indexer index "C:\work\documentation" "D:\projects\my-app\docs"
|
|
60
115
|
|
|
61
|
-
#
|
|
62
|
-
|
|
63
|
-
-p 127.0.0.1:11434:11434 \
|
|
64
|
-
-v ollama_data:/root/.ollama \
|
|
65
|
-
ollama/ollama
|
|
116
|
+
# Search semantically
|
|
117
|
+
directory-indexer search "database timeout errors"
|
|
66
118
|
|
|
67
|
-
|
|
68
|
-
|
|
119
|
+
# Find similar files
|
|
120
|
+
# Linux/macOS
|
|
121
|
+
directory-indexer similar /mnt/work/incident-reports/redis-outage.md
|
|
122
|
+
# Windows
|
|
123
|
+
directory-indexer similar "C:\work\incidents\redis-outage.md"
|
|
69
124
|
|
|
70
|
-
|
|
125
|
+
# Get file content
|
|
126
|
+
directory-indexer get /home/user/projects/api-docs/auth-guide.md
|
|
127
|
+
|
|
128
|
+
# Show status
|
|
129
|
+
directory-indexer status
|
|
130
|
+
```
|
|
71
131
|
|
|
72
132
|
## Configuration
|
|
73
133
|
|
|
@@ -78,8 +138,11 @@ Directory Indexer uses environment variables for configuration. Set these if you
|
|
|
78
138
|
export QDRANT_ENDPOINT="http://localhost:6333"
|
|
79
139
|
export OLLAMA_ENDPOINT="http://localhost:11434"
|
|
80
140
|
|
|
81
|
-
# Optional
|
|
82
|
-
|
|
141
|
+
# Optional data directory (default: ~/.directory-indexer)
|
|
142
|
+
# Linux/macOS
|
|
143
|
+
export DIRECTORY_INDEXER_DATA_DIR="/opt/directory-indexer-data"
|
|
144
|
+
# Windows
|
|
145
|
+
set DIRECTORY_INDEXER_DATA_DIR=D:\data\directory-indexer
|
|
83
146
|
|
|
84
147
|
# Optional Qdrant collection name (default: directory-indexer)
|
|
85
148
|
# Note: Setting to "test" enables auto-cleanup for testing
|
|
@@ -101,32 +164,13 @@ export OLLAMA_API_KEY="your-ollama-key" # if using hosted Ollama
|
|
|
101
164
|
"env": {
|
|
102
165
|
"QDRANT_ENDPOINT": "http://localhost:6333",
|
|
103
166
|
"OLLAMA_ENDPOINT": "http://localhost:11434",
|
|
104
|
-
"
|
|
167
|
+
"DIRECTORY_INDEXER_DATA_DIR": "/opt/directory-indexer-data"
|
|
105
168
|
}
|
|
106
169
|
}
|
|
107
170
|
}
|
|
108
171
|
}
|
|
109
172
|
```
|
|
110
173
|
|
|
111
|
-
## CLI Usage
|
|
112
|
-
|
|
113
|
-
```bash
|
|
114
|
-
# Index your directories
|
|
115
|
-
directory-indexer index ~/Documents ~/work/docs
|
|
116
|
-
|
|
117
|
-
# Search semantically
|
|
118
|
-
directory-indexer search "database timeout errors"
|
|
119
|
-
|
|
120
|
-
# Find similar files
|
|
121
|
-
directory-indexer similar ~/incidents/redis-outage.md
|
|
122
|
-
|
|
123
|
-
# Get file content
|
|
124
|
-
directory-indexer get ~/docs/api-guide.md
|
|
125
|
-
|
|
126
|
-
# Show status
|
|
127
|
-
directory-indexer status
|
|
128
|
-
```
|
|
129
|
-
|
|
130
174
|
## Supported Files
|
|
131
175
|
|
|
132
176
|
- **Text**: `.md`, `.txt`
|
|
@@ -140,6 +184,25 @@ directory-indexer status
|
|
|
140
184
|
- **[Contributing](docs/CONTRIBUTING.md)**: Development setup and guidelines
|
|
141
185
|
- **[Design](docs/design.md)**: Architecture and technical decisions
|
|
142
186
|
|
|
187
|
+
## Usage Examples
|
|
188
|
+
|
|
189
|
+
Once indexed, try these queries with your AI assistant:
|
|
190
|
+
|
|
191
|
+
**Search by concept:**
|
|
192
|
+
- _"Find API authentication examples"_
|
|
193
|
+
- _"Show me error handling patterns"_
|
|
194
|
+
- _"Find configuration for Redis"_
|
|
195
|
+
|
|
196
|
+
**Find similar content:**
|
|
197
|
+
- _"Show me incidents similar to this outage report"_ *(when you have an incident file open)*
|
|
198
|
+
- _"Find documentation like this API guide"_ *(when viewing an API doc)*
|
|
199
|
+
- _"What files are similar to my deployment script?"_
|
|
200
|
+
|
|
201
|
+
**Troubleshoot issues:**
|
|
202
|
+
- _"Find troubleshooting guides on SQL deadlocks"_
|
|
203
|
+
- _"Show me solutions for timeout errors"_
|
|
204
|
+
- _"Find debugging tips for performance issues"_
|
|
205
|
+
|
|
143
206
|
## License
|
|
144
207
|
|
|
145
208
|
MIT
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/package.json
CHANGED
package/scripts/pre-push
CHANGED
|
@@ -13,6 +13,21 @@ if [ ! -f "Cargo.toml" ]; then
|
|
|
13
13
|
exit 1
|
|
14
14
|
fi
|
|
15
15
|
|
|
16
|
+
echo "Checking version synchronization..."
|
|
17
|
+
# Extract version from Cargo.toml
|
|
18
|
+
CARGO_VERSION=$(grep '^version = ' Cargo.toml | head -1 | sed 's/version = "\(.*\)"/\1/')
|
|
19
|
+
|
|
20
|
+
# Extract version from package.json
|
|
21
|
+
NPM_VERSION=$(grep '"version":' package.json | head -1 | sed 's/.*"version": "\(.*\)".*/\1/')
|
|
22
|
+
|
|
23
|
+
if [ "$CARGO_VERSION" != "$NPM_VERSION" ]; then
|
|
24
|
+
echo "ERROR: Version mismatch detected!"
|
|
25
|
+
echo " Cargo.toml version: $CARGO_VERSION"
|
|
26
|
+
echo " package.json version: $NPM_VERSION"
|
|
27
|
+
echo " Please ensure both files have the same version."
|
|
28
|
+
exit 1
|
|
29
|
+
fi
|
|
30
|
+
|
|
16
31
|
echo "Running clippy (linter)..."
|
|
17
32
|
# Check main library code with strict linting
|
|
18
33
|
cargo clippy --lib --all-features -- -D warnings -D clippy::uninlined_format_args
|
|
@@ -22,11 +37,12 @@ cargo clippy --bins --all-features -- -D warnings
|
|
|
22
37
|
echo "Checking code formatting..."
|
|
23
38
|
cargo fmt --check
|
|
24
39
|
|
|
25
|
-
echo "Running
|
|
26
|
-
|
|
40
|
+
echo "Running tests..."
|
|
41
|
+
export DIRECTORY_INDEXER_DATA_DIR=/tmp/directory-indexer-test
|
|
42
|
+
export DIRECTORY_INDEXER_QDRANT_COLLECTION=directory-indexer-test
|
|
43
|
+
|
|
27
44
|
cargo test --lib
|
|
28
|
-
cargo test
|
|
29
|
-
cargo test --test storage_tests
|
|
45
|
+
cargo test tests
|
|
30
46
|
|
|
31
47
|
echo "Checking for security vulnerabilities..."
|
|
32
48
|
if command -v cargo-audit >/dev/null 2>&1; then
|