@autodev/codebase 0.0.3 → 0.0.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (4) hide show
  1. package/README.md +172 -63
  2. package/dist/index.js +2492 -2133
  3. package/dist/index.js.map +1 -1
  4. package/package.json +5 -4
package/README.md CHANGED
@@ -3,9 +3,8 @@
3
3
  # @autodev/codebase
4
4
 
5
5
  <div align="center">
6
- <img src="src/images/image1.png" width="300" alt="Image 1" style="margin: 0 10px;" />
7
- <img src="src/images/image2.png" width="200" alt="Image 2" style="margin: 0 10px;" />
8
- <img src="src/images/image3.png" height="150" alt="Image 3" style="margin: 0 10px;" />
6
+ <img src="src/images/image2.png" alt="Image 2" style="display: inline-block; width: 350px; margin: 0 10px;" />
7
+ <img src="src/images/image3.png" alt="Image 3" style="display: inline-block; width: 200px; margin: 0 10px;" />
9
8
  </div>
10
9
 
11
10
  <br />
@@ -33,10 +32,25 @@ brew install ollama
33
32
  ollama serve
34
33
 
35
34
  # In a new terminal, pull the embedding model
36
- ollama pull nomic-embed-text
35
+ ollama pull dengcao/Qwen3-Embedding-0.6B:Q8_0
37
36
  ```
38
37
 
39
- ### 2. Install and Start Qdrant
38
+ ### 2. Install ripgrep
39
+
40
+ `ripgrep` is required for fast codebase indexing. Install it with:
41
+
42
+ ```bash
43
+ # Install ripgrep (macOS)
44
+ brew install ripgrep
45
+
46
+ # Or on Ubuntu/Debian
47
+ sudo apt-get install ripgrep
48
+
49
+ # Or on Arch Linux
50
+ sudo pacman -S ripgrep
51
+ ```
52
+
53
+ ### 3. Install and Start Qdrant
40
54
 
41
55
  Start Qdrant using Docker:
42
56
 
@@ -54,7 +68,7 @@ tar -xzf qdrant-x86_64-unknown-linux-gnu.tar.gz
54
68
  ./qdrant
55
69
  ```
56
70
 
57
- ### 3. Verify Services Are Running
71
+ ### 4. Verify Services Are Running
58
72
 
59
73
  ```bash
60
74
  # Check Ollama
@@ -63,8 +77,13 @@ curl http://localhost:11434/api/tags
63
77
  # Check Qdrant
64
78
  curl http://localhost:6333/collections
65
79
  ```
66
- ### 4. Install project locally
80
+ ### 5. Install Autodev-codebase
81
+
82
+ ```bash
83
+ npm install -g @autodev/codebase
84
+ ```
67
85
 
86
+ Alternatively, you can install it locally:
68
87
  ```
69
88
  git clone https://github.com/anrgct/autodev-codebase
70
89
  cd autodev-codebase
@@ -102,53 +121,89 @@ codebase mcp-server --port=3001 --host=localhost
102
121
  codebase mcp-server --path=/workspace --port=3002
103
122
  ```
104
123
 
105
- ### IDE Integration (Cursor/Claude)
106
124
 
107
- Configure your IDE to connect to the MCP server:
125
+ ## ⚙️ Configuration
126
+
127
+ ### Configuration Files & Priority
128
+
129
+ The library uses a layered configuration system, allowing you to customize settings at different levels. The priority order (highest to lowest) is:
130
+
131
+ 1. **CLI Parameters** (e.g., `--model`, `--ollama-url`, `--qdrant-url`, `--config`, etc.)
132
+ 2. **Project Config File** (`./autodev-config.json`)
133
+ 3. **Global Config File** (`~/.autodev-cache/autodev-config.json`)
134
+ 4. **Built-in Defaults**
135
+
136
+ Settings specified at a higher level override those at lower levels. This lets you tailor the behavior for your environment or project as needed.
137
+
138
+ **Config file locations:**
139
+ - Global: `~/.autodev-cache/autodev-config.json`
140
+ - Project: `./autodev-config.json`
141
+ - CLI: Pass parameters directly when running commands
142
+
143
+
144
+ #### Global Configuration
145
+
146
+ Create a global configuration file at `~/.autodev-cache/autodev-config.json`:
108
147
 
109
148
  ```json
110
149
  {
111
- "mcpServers": {
112
- "codebase": {
113
- "url": "http://localhost:3001/sse"
114
- }
115
- }
150
+ "isEnabled": true,
151
+ "embedder": {
152
+ "provider": "ollama",
153
+ "model": "dengcao/Qwen3-Embedding-0.6B:Q8_0",
154
+ "dimension": 1024,
155
+ "baseUrl": "http://localhost:11434"
156
+ },
157
+ "qdrantUrl": "http://localhost:6333",
158
+ "qdrantApiKey": "your-api-key-if-needed",
159
+ "searchMinScore": 0.4
116
160
  }
117
161
  ```
118
162
 
119
- For clients that do not support SSE MCP, you can use the following configuration:
163
+ #### Project Configuration
164
+
165
+ Create a project-specific configuration file at `./autodev-config.json`:
120
166
 
121
167
  ```json
122
168
  {
123
- "mcpServers": {
124
- "codebase": {
125
- "command": "codebase",
126
- "args": [
127
- "stdio-adapter",
128
- "--server-url=http://localhost:3001/sse"
129
- ]
130
- }
131
- }
169
+ "embedder": {
170
+ "provider": "openai-compatible",
171
+ "apiKey": "sk-xxxxx",
172
+ "baseUrl": "http://localhost:2302/v1",
173
+ "model": "openai/text-embedding-3-smallnpm",
174
+ "dimension": 1536,
175
+ },
176
+ "qdrantUrl": "http://localhost:6334"
132
177
  }
133
178
  ```
134
179
 
135
- ### Library Usage
180
+ #### Configuration Options
181
+
182
+ | Option | Type | Description | Default |
183
+ |--------|------|-------------|---------|
184
+ | `isEnabled` | boolean | Enable/disable code indexing feature | `true` |
185
+ | `embedder.provider` | string | Embedding provider (`ollama`, `openai`, `openai-compatible`) | `ollama` |
186
+ | `embedder.model` | string | Embedding model name | `dengcao/Qwen3-Embedding-0.6B:Q8_0` |
187
+ | `embedder.dimension` | number | Vector dimension size | `1024` |
188
+ | `embedder.baseUrl` | string | Provider API base URL | `http://localhost:11434` |
189
+ | `embedder.apiKey` | string | API key (for OpenAI/compatible providers) | - |
190
+ | `qdrantUrl` | string | Qdrant vector database URL | `http://localhost:6333` |
191
+ | `qdrantApiKey` | string | Qdrant API key (if authentication enabled) | - |
192
+ | `searchMinScore` | number | Minimum similarity score for search results | `0.4` |
193
+
194
+ **Note**: The `isConfigured` field is automatically calculated based on the completeness of your configuration and should not be set manually. The system will determine if the configuration is valid based on the required fields for your chosen provider.
195
+
196
+ #### Configuration Priority Examples
136
197
 
137
- #### Node.js Usage
138
- ```typescript
139
- import { createNodeDependencies } from '@autodev/codebase/adapters/nodejs'
140
- import { CodeIndexManager } from '@autodev/codebase'
198
+ ```bash
199
+ # Use global config defaults
200
+ codebase
141
201
 
142
- const deps = createNodeDependencies({
143
- workspacePath: '/path/to/project',
144
- storageOptions: { /* ... */ },
145
- loggerOptions: { /* ... */ },
146
- configOptions: { /* ... */ }
147
- })
202
+ # Override model via CLI (highest priority)
203
+ codebase --model="custom-model"
148
204
 
149
- const manager = CodeIndexManager.getInstance(deps)
150
- await manager.initialize()
151
- await manager.startIndexing()
205
+ # Use project config with CLI overrides
206
+ codebase --config=./my-config.json --qdrant-url=http://remote:6333
152
207
  ```
153
208
 
154
209
  ## 🔧 CLI Options
@@ -156,6 +211,7 @@ await manager.startIndexing()
156
211
  ### Global Options
157
212
  - `--path=<path>` - Workspace path (default: current directory)
158
213
  - `--demo` - Create demo files in workspace
214
+ - `--force` - ignore cache force re-index
159
215
  - `--ollama-url=<url>` - Ollama API URL (default: http://localhost:11434)
160
216
  - `--qdrant-url=<url>` - Qdrant vector DB URL (default: http://localhost:6333)
161
217
  - `--model=<model>` - Embedding model (default: nomic-embed-text)
@@ -163,12 +219,43 @@ await manager.startIndexing()
163
219
  - `--storage=<path>` - Storage directory path
164
220
  - `--cache=<path>` - Cache directory path
165
221
  - `--log-level=<level>` - Log level: error|warn|info|debug (default: error)
222
+ - `--log-level=<level>` - Log level: error|warn|info|debug (default: error)
166
223
  - `--help, -h` - Show help
167
224
 
168
225
  ### MCP Server Options
169
226
  - `--port=<port>` - HTTP server port (default: 3001)
170
227
  - `--host=<host>` - HTTP server host (default: localhost)
171
228
 
229
+
230
+ ### IDE Integration (Cursor/Claude)
231
+
232
+ Configure your IDE to connect to the MCP server:
233
+
234
+ ```json
235
+ {
236
+ "mcpServers": {
237
+ "codebase": {
238
+ "url": "http://localhost:3001/sse"
239
+ }
240
+ }
241
+ }
242
+ ```
243
+
244
+ For clients that do not support SSE MCP, you can use the following configuration:
245
+
246
+ ```json
247
+ {
248
+ "mcpServers": {
249
+ "codebase": {
250
+ "command": "codebase",
251
+ "args": [
252
+ "stdio-adapter",
253
+ "--server-url=http://localhost:3001/sse"
254
+ ]
255
+ }
256
+ }
257
+ }
258
+ ```
172
259
  ## 🌐 MCP Server Features
173
260
 
174
261
  ### Web Interface
@@ -180,8 +267,7 @@ await manager.startIndexing()
180
267
  - **`search_codebase`** - Semantic search through your codebase
181
268
  - Parameters: `query` (string), `limit` (number), `filters` (object)
182
269
  - Returns: Formatted search results with file paths, scores, and code blocks
183
- - **`get_search_stats`** - Get indexing status and statistics
184
- - **`configure_search`** - Configure search parameters at runtime
270
+
185
271
 
186
272
 
187
273
  ### Scripts
@@ -202,26 +288,49 @@ npm run demo-tui
202
288
  npm run mcp-server
203
289
  ```
204
290
 
205
- ## 💡 Why Use MCP Server Mode?
206
-
207
- ### Problems Solved
208
- - **❌ Repeated Indexing**: Every IDE connection re-indexes, wasting time and resources
209
- - **❌ Complex Configuration**: Each project needs different path parameters in IDE
210
- - **❌ Resource Waste**: Multiple IDE windows start multiple server instances
211
-
212
- ### Benefits
213
- - **✅ One-time Indexing**: Server runs long-term, index persists
214
- - **✅ Simplified Configuration**: Universal IDE configuration, no project-specific paths
215
- - **✅ Resource Efficiency**: One server instance per project
216
- - **✅ Better Developer Experience**: Start server in project directory intuitively
217
- - **✅ Backward Compatible**: Still supports traditional per-connection mode
218
- - **✅ Web Interface**: Status monitoring and configuration help
219
- - **✅ Dual Mode**: Can run both TUI and MCP server simultaneously
220
-
221
-
222
- This is a platform-agnostic library extracted from the roo-code VSCode plugin.
223
- ## 📚 Examples
224
-
225
- See the `examples/` directory for complete usage examples:
226
- - `nodejs-usage.ts` - Node.js integration examples
227
- - `run-demo-tui.tsx` - TUI demo application
291
+ ## Embedding Models PK
292
+
293
+ **Mainstream Embedding Models Performance**
294
+
295
+ | Model | Dimension | Avg Precision@3 | Avg Precision@5 | Good Queries (≥66.7%) | Failed Queries (0%) |
296
+ | ------------------------------------------------ | --------- | --------------- | --------------- | --------------------- | ------------------- |
297
+ | siliconflow/Qwen/Qwen3-Embedding-8B | 4096 | **76.7%** | 66.0% | 5/10 | 0/10 |
298
+ | siliconflow/Qwen/Qwen3-Embedding-4B | 2560 | **73.3%** | 54.0% | 5/10 | 1/10 |
299
+ | voyage/voyage-code-3 | 1024 | **73.3%** | 52.0% | 6/10 | 1/10 |
300
+ | siliconflow/Qwen/Qwen3-Embedding-0.6B | 1024 | **63.3%** | 42.0% | 4/10 | 1/10 |
301
+ | morph-embedding-v2 | 1536 | **56.7%** | 44.0% | 3/10 | 1/10 |
302
+ | openai/text-embedding-ada-002 | 1536 | **53.3%** | 38.0% | 2/10 | 1/10 |
303
+ | voyage/voyage-3-large | 1024 | **53.3%** | 42.0% | 3/10 | 2/10 |
304
+ | openai/text-embedding-3-large | 3072 | **46.7%** | 38.0% | 1/10 | 3/10 |
305
+ | voyage/voyage-3.5 | 1024 | **43.3%** | 38.0% | 1/10 | 2/10 |
306
+ | voyage/voyage-3.5-lite | 1024 | **36.7%** | 28.0% | 1/10 | 2/10 |
307
+ | openai/text-embedding-3-small | 1536 | **33.3%** | 28.0% | 1/10 | 4/10 |
308
+ | siliconflow/BAAI/bge-large-en-v1.5 | 1024 | **30.0%** | 28.0% | 0/10 | 3/10 |
309
+ | siliconflow/Pro/BAAI/bge-m3 | 1024 | **26.7%** | 24.0% | 0/10 | 2/10 |
310
+ | ollama/nomic-embed-text | 768 | **16.7%** | 18.0% | 0/10 | 6/10 |
311
+ | siliconflow/netease-youdao/bce-embedding-base_v1 | 1024 | **13.3%** | 16.0% | 0/10 | 6/10 |
312
+
313
+ ------
314
+
315
+ **Ollama-based Embedding Models Performance**
316
+
317
+ | Model | Dimension | Precision@3 | Precision@5 | Good Queries (≥66.7%) | Failed Queries (0%) |
318
+ | -------------------------------------------------------- | --------- | ----------- | ----------- | --------------------- | ------------------- |
319
+ | ollama/dengcao/Qwen3-Embedding-4B:Q4_K_M | 2560 | 66.7% | 48.0% | 4/10 | 1/10 |
320
+ | ollama/dengcao/Qwen3-Embedding-0.6B:f16 | 1024 | 63.3% | 44.0% | 3/10 | 0/10 |
321
+ | ollama/dengcao/Qwen3-Embedding-0.6B:Q8_0 | 1024 | 63.3% | 44.0% | 3/10 | 0/10 |
322
+ | ollama/dengcao/Qwen3-Embedding-4B:Q8_0 | 2560 | 60.0% | 48.0% | 3/10 | 1/10 |
323
+ | lmstudio/taylor-jones/bge-code-v1-Q8_0-GGUF | 1536 | 60.0% | 54.0% | 4/10 | 1/10 |
324
+ | ollama/dengcao/Qwen3-Embedding-8B:Q4_K_M | 4096 | 56.7% | 42.0% | 2/10 | 2/10 |
325
+ | ollama/hf.co/nomic-ai/nomic-embed-code-GGUF:Q4_K_M | 3584 | 53.3% | 44.0% | 2/10 | 0/10 |
326
+ | ollama/bge-m3:f16 | 1024 | 26.7% | 24.0% | 0/10 | 2/10 |
327
+ | ollama/hf.co/nomic-ai/nomic-embed-text-v2-moe-GGUF:f16 | 768 | 26.7% | 20.0% | 0/10 | 2/10 |
328
+ | ollama/granite-embedding:278m-fp16 | 768 | 23.3% | 18.0% | 0/10 | 4/10 |
329
+ | ollama/unclemusclez/jina-embeddings-v2-base-code:f16 | 768 | 23.3% | 16.0% | 0/10 | 5/10 |
330
+ | lmstudio/awhiteside/CodeRankEmbed-Q8_0-GGUF | 768 | 23.3% | 16.0% | 0/10 | 5/10 |
331
+ | lmstudio/wsxiaoys/jina-embeddings-v2-base-code-Q8_0-GGUF | 768 | 23.3% | 16.0% | 0/10 | 5/10 |
332
+ | ollama/dengcao/Dmeta-embedding-zh:F16 | 768 | 20.0% | 20.0% | 0/10 | 6/10 |
333
+ | ollama/znbang/bge:small-en-v1.5-q8_0 | 384 | 16.7% | 16.0% | 0/10 | 6/10 |
334
+ | lmstudio/nomic-ai/nomic-embed-text-v1.5-GGUF@Q4_K_M | 768 | 16.7% | 14.0% | 0/10 | 6/10 |
335
+ | ollama/nomic-embed-text:f16 | 768 | 16.7% | 18.0% | 0/10 | 6/10 |
336
+ | ollama/snowflake-arctic-embed2:568m:f16 | 1024 | 16.7% | 18.0% | 0/10 | 5/10 |