@softerist/heuristic-mcp 3.0.10 → 3.0.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (107) hide show
  1. package/README.md +345 -349
  2. package/config.jsonc +723 -723
  3. package/index.js +739 -611
  4. package/lib/cache-utils.js +3 -2
  5. package/lib/cache.js +296 -39
  6. package/lib/cli.js +166 -164
  7. package/lib/config.js +945 -894
  8. package/lib/constants.js +414 -0
  9. package/lib/embed-query-process.js +227 -0
  10. package/lib/embedding-process.js +355 -268
  11. package/lib/embedding-worker.js +867 -772
  12. package/lib/logging.js +74 -0
  13. package/lib/path-utils.js +40 -0
  14. package/lib/server-lifecycle.js +35 -2
  15. package/lib/slice-normalize.js +41 -0
  16. package/lib/tokenizer.js +9 -4
  17. package/lib/utils.js +76 -25
  18. package/lib/vector-store-binary.js +11 -8
  19. package/lib/vector-store-sqlite.js +135 -105
  20. package/package.json +75 -75
  21. package/specs/plan.md +23 -0
  22. package/.agent/workflows/code-review.md +0 -71
  23. package/ARCHITECTURE.md +0 -227
  24. package/CONTRIBUTING.md +0 -0
  25. package/features/ann-config.js +0 -131
  26. package/features/clear-cache.js +0 -84
  27. package/features/find-similar-code.js +0 -245
  28. package/features/hybrid-search.js +0 -407
  29. package/features/index-codebase.js +0 -2678
  30. package/features/lifecycle.js +0 -915
  31. package/features/package-version.js +0 -300
  32. package/features/register.js +0 -237
  33. package/features/set-workspace.js +0 -206
  34. package/scripts/clear-cache.js +0 -20
  35. package/scripts/download-model.js +0 -43
  36. package/scripts/mcp-launcher.js +0 -35
  37. package/scripts/postinstall.js +0 -12
  38. package/test/ann-config.test.js +0 -179
  39. package/test/ann-fallback.test.js +0 -68
  40. package/test/binary-store.test.js +0 -69
  41. package/test/boost.test.js +0 -281
  42. package/test/cache-branches.test.js +0 -120
  43. package/test/cache-errors.test.js +0 -268
  44. package/test/cache-extended.test.js +0 -302
  45. package/test/cache-helpers.test.js +0 -205
  46. package/test/cache-hnsw-failure.test.js +0 -40
  47. package/test/cache-json-worker.test.js +0 -190
  48. package/test/cache-utils.test.js +0 -202
  49. package/test/cache-worker.test.js +0 -104
  50. package/test/cache.test.js +0 -527
  51. package/test/call-graph.test.js +0 -241
  52. package/test/clear-cache.test.js +0 -286
  53. package/test/code-review-workflow.test.js +0 -50
  54. package/test/config.test.js +0 -425
  55. package/test/coverage-gap.test.js +0 -497
  56. package/test/coverage-maximizer.test.js +0 -238
  57. package/test/debug-analysis.js +0 -106
  58. package/test/embedding-model.test.js +0 -312
  59. package/test/embedding-worker-extended.test.js +0 -276
  60. package/test/embedding-worker.test.js +0 -158
  61. package/test/extended.test.js +0 -95
  62. package/test/features.test.js +0 -141
  63. package/test/find-similar-code.test.js +0 -193
  64. package/test/fixes-verifier.test.js +0 -55
  65. package/test/fixtures/mcp-workspace/config.jsonc +0 -12
  66. package/test/fixtures/mcp-workspace/example.js +0 -8
  67. package/test/helpers.js +0 -266
  68. package/test/helpers.test.js +0 -46
  69. package/test/hybrid-search-basic.test.js +0 -62
  70. package/test/hybrid-search-branch.test.js +0 -201
  71. package/test/hybrid-search-callgraph.test.js +0 -230
  72. package/test/hybrid-search-extra.test.js +0 -81
  73. package/test/hybrid-search.test.js +0 -656
  74. package/test/index-cli.test.js +0 -531
  75. package/test/index-codebase-batch.test.js +0 -117
  76. package/test/index-codebase-branches.test.js +0 -580
  77. package/test/index-codebase-core.test.js +0 -1039
  78. package/test/index-codebase-edge-cases.test.js +0 -251
  79. package/test/index-codebase-errors.test.js +0 -132
  80. package/test/index-codebase-extended.test.js +0 -261
  81. package/test/index-codebase-gap.test.js +0 -251
  82. package/test/index-codebase-lines.test.js +0 -151
  83. package/test/index-codebase-watcher.test.js +0 -369
  84. package/test/index-codebase.test.js +0 -541
  85. package/test/index-memory.test.js +0 -231
  86. package/test/indexer-extended.test.js +0 -176
  87. package/test/integration.test.js +0 -279
  88. package/test/json-worker.test.js +0 -50
  89. package/test/lifecycle.test.js +0 -543
  90. package/test/master.test.js +0 -198
  91. package/test/mcp-search.test.js +0 -78
  92. package/test/perfection.test.js +0 -349
  93. package/test/polish.test.js +0 -189
  94. package/test/project-detector.test.js +0 -65
  95. package/test/register.test.js +0 -262
  96. package/test/stress.js +0 -1086
  97. package/test/tokenizer.test.js +0 -187
  98. package/test/ultra-maximizer.test.js +0 -116
  99. package/test/utils-branches.test.js +0 -160
  100. package/test/utils-extra.test.js +0 -116
  101. package/test/utils.test.js +0 -131
  102. package/test/worker-errors.test.js +0 -98
  103. package/test/worker-init.test.js +0 -102
  104. package/test/worker_throttling.test.js +0 -91
  105. package/tools/scripts/benchmark-search.js +0 -95
  106. package/tools/scripts/cache-stats.js +0 -71
  107. package/tools/scripts/manual-search.js +0 -70
package/README.md CHANGED
@@ -1,349 +1,345 @@
1
- # Heuristic MCP Server
2
-
3
- An enhanced MCP server for your codebase. It provides intelligent semantic search, find-similar-code, recency-aware ranking, call-graph proximity boosts, and smart chunking. Optimized for Antigravity, Cursor, Claude Desktop, and VS Code.
4
-
5
- ---
6
-
7
- ## Key Features
8
-
9
- - Zero-touch setup: postinstall auto-registers the MCP server with supported IDEs when possible.
10
- - Smart indexing: detects project type and applies smart ignore patterns on top of your excludes.
11
- - Semantic search: find code by meaning, not just keywords.
12
- - Find similar code: locate near-duplicate or related patterns from a snippet.
13
- - Package version lookup: check latest versions from npm, PyPI, crates.io, Maven, and more.
14
- - Workspace switching: change workspace at runtime without restarting the server.
15
- - Recency ranking and call-graph boosting: surfaces fresh and related code.
16
- - Optional ANN index: faster candidate retrieval for large codebases.
17
- - Optional binary vector store: mmap-friendly cache format for large repos.
18
- - Flexible embedding dimensions: MRL-compatible dimension reduction (64-768d) for speed/quality tradeoffs.
19
-
20
- ---
21
-
22
- ## Installation
23
-
24
- Install globally (recommended):
25
-
26
- ```bash
27
- npm install -g @softerist/heuristic-mcp
28
- ```
29
-
30
- What happens during install:
31
-
32
- - Registration runs automatically (`scripts/postinstall.js`).
33
- - Model pre-download is attempted (`scripts/download-model.js`). If offline, it will be skipped and downloaded on first run.
34
-
35
- If auto-registration did not update your IDE config, run:
36
-
37
- ```bash
38
- heuristic-mcp --register
39
- ```
40
-
41
- ---
42
-
43
- ## CLI Commands
44
-
45
- The `heuristic-mcp` binary manages the server lifecycle.
46
-
47
- ### Status
48
-
49
- ```bash
50
- heuristic-mcp --status
51
- ```
52
-
53
- Shows server PID(s) and cache stats.
54
-
55
- ### Logs
56
-
57
- ```bash
58
- heuristic-mcp --logs
59
- ```
60
-
61
- Tails the server log for the current workspace (defaults to last 200 lines and follows).
62
-
63
- Optional flags:
64
-
65
- ```bash
66
- heuristic-mcp --logs --tail 100
67
- heuristic-mcp --logs --no-follow
68
- ```
69
-
70
- ### Version
71
-
72
- ```bash
73
- heuristic-mcp --version
74
- ```
75
-
76
- ### Register (manual)
77
-
78
- ```bash
79
- heuristic-mcp --register
80
- heuristic-mcp --register antigravity
81
- heuristic-mcp --register cursor
82
- heuristic-mcp --register "Claude Desktop"
83
- ```
84
-
85
- ### Start/Stop
86
-
87
- ```bash
88
- heuristic-mcp --start
89
- heuristic-mcp --stop
90
- ```
91
-
92
- `--stop` also disables the MCP server entry in supported IDE configs so the IDE won't immediately respawn it. `--start` re-enables it (restart/reload the IDE to launch).
93
-
94
- ### Clear Cache
95
-
96
- ```bash
97
- heuristic-mcp --clear-cache
98
- ```
99
-
100
- Clears the cache for the current working directory (or `--workspace` if provided) and removes stale cache directories without metadata.
101
-
102
- ---
103
-
104
- ## Configuration (`config.jsonc`)
105
-
106
- Configuration is loaded from your workspace root when the server runs with `--workspace` (this is how IDEs launch it). In server mode, it falls back to the package `config.jsonc` (or `config.json`) and then your current working directory.
107
-
108
- Example `config.jsonc`:
109
-
110
- ```json
111
- {
112
- "excludePatterns": ["**/legacy-code/**", "**/*.test.ts"],
113
- "fileNames": ["Dockerfile", ".env.example", "Makefile"],
114
- "smartIndexing": true,
115
- "embeddingModel": "jinaai/jina-embeddings-v2-base-code",
116
- "workerThreads": 0,
117
- "embeddingBatchSize": null,
118
- "embeddingProcessNumThreads": 8,
119
- "enableExplicitGc": false,
120
- "recencyBoost": 0.1,
121
- "recencyDecayDays": 30,
122
- "callGraphEnabled": true,
123
- "callGraphBoost": 0.15,
124
- "annEnabled": true,
125
- "vectorStoreFormat": "binary",
126
- "vectorStoreContentMode": "external",
127
- "vectorStoreLoadMode": "disk",
128
- "contentCacheEntries": 256,
129
- "vectorCacheEntries": 64,
130
- "clearCacheAfterIndex": true
131
- }
132
- ```
133
-
134
- ### Embedding Model & Dimension Options
135
-
136
- **Default model:** `jinaai/jina-embeddings-v2-base-code` (768 dimensions)
137
-
138
- > **Important:** The default Jina model was **not** trained with Matryoshka Representation Learning (MRL). Dimension reduction (`embeddingDimension`) will significantly degrade search quality with this model. Only use dimension reduction with MRL-trained models.
139
-
140
- For faster search with smaller embeddings, switch to an MRL-compatible model:
141
-
142
- ```json
143
- {
144
- "embeddingModel": "nomic-ai/nomic-embed-text-v1.5",
145
- "embeddingDimension": 128
146
- }
147
- ```
148
-
149
- **MRL-compatible models:**
150
- - `nomic-ai/nomic-embed-text-v1.5` — recommended for 128d/256d
151
- - Other models explicitly trained with Matryoshka loss
152
-
153
- **embeddingDimension values:** `64 | 128 | 256 | 512 | 768 | null` (null = full dimensions)
154
-
155
- Cache location:
156
-
157
- - By default, the cache is stored in a global OS cache directory under `heuristic-mcp/<hash>`.
158
- - You can override with `cacheDirectory` in your config file.
159
-
160
- ### Environment Variables
161
-
162
- Selected overrides (prefix `SMART_CODING_`):
163
-
164
- - `SMART_CODING_VERBOSE=true|false` — enable detailed logging.
165
- - `SMART_CODING_WORKER_THREADS=auto|N` — worker thread count.
166
- - `SMART_CODING_BATCH_SIZE=100` — files per indexing batch.
167
- - `SMART_CODING_CHUNK_SIZE=25` — lines per chunk.
168
- - `SMART_CODING_MAX_RESULTS=5` — max search results.
169
- - `SMART_CODING_EMBEDDING_BATCH_SIZE=64` — embedding batch size (1–256, overrides auto).
170
- - `SMART_CODING_EMBEDDING_THREADS=8` — ONNX threads for the embedding child process.
171
- - `SMART_CODING_RECENCY_BOOST=0.1` — boost for recently edited files.
172
- - `SMART_CODING_RECENCY_DECAY_DAYS=30` — days until recency boost decays to 0.
173
- - `SMART_CODING_ANN_ENABLED=true|false` — enable ANN index.
174
- - `SMART_CODING_ANN_EF_SEARCH=64` — ANN search quality/speed tradeoff.
175
- - `SMART_CODING_VECTOR_STORE_FORMAT=json|binary|sqlite` — on-disk vector store format.
176
- - `SMART_CODING_VECTOR_STORE_CONTENT_MODE=external|inline` — where content is stored for binary format.
177
- - `SMART_CODING_VECTOR_STORE_LOAD_MODE=memory|disk` — vector loading strategy.
178
- - `SMART_CODING_CONTENT_CACHE_ENTRIES=256` — LRU entries for decoded content.
179
- - `SMART_CODING_VECTOR_CACHE_ENTRIES=64` — LRU entries for vectors (disk mode).
180
- - `SMART_CODING_CLEAR_CACHE_AFTER_INDEX=true|false` drop in-memory vectors after indexing.
181
- - `SMART_CODING_EXPLICIT_GC=true|false` — opt-in to explicit GC (requires `--expose-gc`).
182
- - `SMART_CODING_INCREMENTAL_GC_THRESHOLD_MB=2048` RSS threshold for running incremental GC after watcher updates (requires explicit GC).
183
- - `SMART_CODING_EMBEDDING_DIMENSION=64|128|256|512|768` — MRL dimension reduction (only for MRL-trained models).
184
-
185
- See `lib/config.js` for the full list.
186
-
187
- ### Binary Vector Store
188
-
189
- Set `vectorStoreFormat` to `binary` to use the on-disk binary cache. This keeps vectors and content out of JS heap
190
- and reads on demand. Recommended for large repos.
191
-
192
- - `vectorStoreContentMode=external` keeps content in the binary file and only loads for top-N results.
193
- - `contentCacheEntries` controls the small in-memory LRU for decoded content strings.
194
- - `vectorStoreLoadMode=disk` streams vectors from disk to reduce memory usage.
195
- - `vectorCacheEntries` controls the small in-memory LRU for vectors when using disk mode.
196
- - `clearCacheAfterIndex=true` drops in-memory vectors after indexing and reloads lazily on next query.
197
- - Note: `annEnabled=true` with `vectorStoreLoadMode=disk` can increase disk reads during ANN rebuilds on large indexes.
198
-
199
- ### SQLite Vector Store
200
-
201
- Set `vectorStoreFormat` to `sqlite` to use SQLite for persistence. This provides:
202
-
203
- - ACID transactions for reliable writes
204
- - Simpler concurrent access
205
- - Standard database format for inspection
206
-
207
- ```json
208
- {
209
- "vectorStoreFormat": "sqlite"
210
- }
211
- ```
212
-
213
- The vectors and content are stored in `vectors.sqlite` in your cache directory. You can inspect it with any SQLite browser.
214
- `vectorStoreContentMode` and `vectorStoreLoadMode` are respected for SQLite (use `vectorStoreLoadMode=disk` to avoid loading vectors into memory).
215
-
216
- **Tradeoffs vs Binary:**
217
- - Slightly higher read overhead (SQL queries vs direct memory access)
218
- - Better write reliability (transactions)
219
- - Easier debugging (standard SQLite file)
220
-
221
- ### Benchmarking Search
222
-
223
- Use the built-in script to compare memory vs latency tradeoffs:
224
-
225
- ```bash
226
- node tools/scripts/benchmark-search.js --query "database connection" --runs 10
227
- ```
228
-
229
- Compare modes quickly:
230
-
231
- ```bash
232
- SMART_CODING_VECTOR_STORE_LOAD_MODE=memory node tools/scripts/benchmark-search.js --runs 10
233
- SMART_CODING_VECTOR_STORE_LOAD_MODE=disk node tools/scripts/benchmark-search.js --runs 10
234
- SMART_CODING_VECTOR_STORE_FORMAT=binary SMART_CODING_VECTOR_STORE_LOAD_MODE=disk node tools/scripts/benchmark-search.js --runs 10
235
- ```
236
-
237
- Note: On small repos, disk mode may be slightly slower and show noisy RSS deltas; benefits are clearer on large indexes with a small `vectorCacheEntries`.
238
-
239
- ---
240
-
241
- ## MCP Tools Reference
242
-
243
- ### `a_semantic_search`
244
- Find code by meaning. Ideal for natural language queries like "authentication logic" or "database queries".
245
-
246
- ### `b_index_codebase`
247
- Manually trigger a full reindex. Useful after large code changes.
248
-
249
- ### `c_clear_cache`
250
- Clear the embeddings cache and force reindex.
251
-
252
- ### `d_ann_config`
253
- Configure the ANN (Approximate Nearest Neighbor) index. Actions: `stats`, `set_ef_search`, `rebuild`.
254
-
255
- ### `d_find_similar_code`
256
- Find similar code patterns given a snippet. Useful for finding duplicates or refactoring opportunities.
257
-
258
- ### `e_check_package_version`
259
- Fetch the latest version of a package from its official registry.
260
-
261
- **Supported registries:**
262
- - **npm** (default): `lodash`, `@types/node`
263
- - **PyPI**: `pip:requests`, `pypi:django`
264
- - **crates.io**: `cargo:serde`, `rust:tokio`
265
- - **Maven**: `maven:org.springframework:spring-core`
266
- - **Go**: `go:github.com/gin-gonic/gin`
267
- - **RubyGems**: `gem:rails`
268
- - **NuGet**: `nuget:Newtonsoft.Json`
269
- - **Packagist**: `composer:laravel/framework`
270
- - **Hex**: `hex:phoenix`
271
- - **pub.dev**: `pub:flutter`
272
- - **Homebrew**: `brew:node`
273
- - **Conda**: `conda:numpy`
274
-
275
- ### `f_set_workspace`
276
- Change the workspace directory at runtime. Updates search directory, cache location, and optionally triggers reindex.
277
-
278
- **Parameters:**
279
- - `workspacePath` (required): Absolute path to the new workspace
280
- - `reindex` (optional, default: `true`): Whether to trigger a full reindex
281
-
282
- ---
283
-
284
- ## Troubleshooting
285
-
286
- **Server isn't starting**
287
-
288
- 1. Run `heuristic-mcp --status` to check config and cache status.
289
- 2. Run `heuristic-mcp --logs` to see startup errors.
290
-
291
- **Native ONNX backend unavailable (falls back to WASM)**
292
-
293
- If you see log lines like:
294
-
295
- ```
296
- Native ONNX backend unavailable: The operating system cannot run %1.
297
- ...onnxruntime_binding.node. Falling back to WASM.
298
- ```
299
-
300
- The server will automatically disable workers and force `embeddingProcessPerBatch` to reduce memory spikes, but you
301
- should fix the native binding to restore stable memory usage:
302
-
303
- - Ensure you are running **64-bit Node.js** (`node -p "process.arch"` should be `x64`).
304
- - Install **Microsoft Visual C++ 2015–2022 Redistributable (x64)**.
305
- - Reinstall dependencies (clears locked native binaries):
306
-
307
- ```bash
308
- Remove-Item -Recurse -Force node_modules\\onnxruntime-node, node_modules\\.onnxruntime-node-* -ErrorAction SilentlyContinue
309
- npm install
310
- ```
311
-
312
- If you see a warning about **version mismatch** (e.g. "onnxruntime-node 1.23.x incompatible with transformers.js
313
- expectation 1.14.x"), install the matching version:
314
-
315
- ```bash
316
- npm install onnxruntime-node@1.14.0
317
- ```
318
-
319
- **Search returns no results**
320
-
321
- - Check `heuristic-mcp --status` for indexing progress.
322
- - If indexing shows zero files, review `excludePatterns` and `fileExtensions`.
323
-
324
- **Model download fails**
325
-
326
- - The install step tries to pre-download the model, but it can be skipped offline.
327
- - The server will download on first run; ensure network access at least once.
328
-
329
- **Clear cache**
330
-
331
- - Use the MCP tool `c_clear_cache`, run `heuristic-mcp --clear-cache`, or delete the cache directory. For local dev, run `npm run clean`.
332
-
333
- **Inspect cache**
334
-
335
- ```bash
336
- node tools/scripts/cache-stats.js --workspace <path>
337
- ```
338
-
339
- **Stop doesn't stick**
340
-
341
- - The IDE will auto-restart the server if it's still enabled in its config. `--stop` now disables the server entry for Antigravity, Cursor, Claude Desktop, and VS Code (when using common MCP settings keys). Restart the IDE after `--start` to re-enable.
342
-
343
- ---
344
-
345
- ## Contributing
346
-
347
- See `CONTRIBUTING.md` for guidelines.
348
-
349
- License: MIT
1
+ # Heuristic MCP Server
2
+
3
+ An enhanced MCP server for your codebase. It provides intelligent semantic search, find-similar-code, recency-aware ranking, call-graph proximity boosts, and smart chunking. Optimized for Antigravity, Cursor, Claude Desktop, and VS Code.
4
+
5
+ ---
6
+
7
+ ## Key Features
8
+
9
+ - Zero-touch setup: postinstall auto-registers the MCP server with supported IDEs when possible.
10
+ - Smart indexing: detects project type and applies smart ignore patterns on top of your excludes.
11
+ - Semantic search: find code by meaning, not just keywords.
12
+ - Find similar code: locate near-duplicate or related patterns from a snippet.
13
+ - Package version lookup: check latest versions from npm, PyPI, crates.io, Maven, and more.
14
+ - Workspace switching: change workspace at runtime without restarting the server.
15
+ - Recency ranking and call-graph boosting: surfaces fresh and related code.
16
+ - Optional ANN index: faster candidate retrieval for large codebases.
17
+ - Optional binary vector store: mmap-friendly cache format for large repos.
18
+ - Flexible embedding dimensions: MRL-compatible dimension reduction (64-768d) for speed/quality tradeoffs.
19
+
20
+ ---
21
+
22
+ ## Installation
23
+
24
+ Install globally (recommended):
25
+
26
+ ```bash
27
+ npm install -g @softerist/heuristic-mcp
28
+ ```
29
+
30
+ What happens during install:
31
+
32
+ - Registration runs automatically (`scripts/postinstall.js`).
33
+ - Model pre-download is attempted (`scripts/download-model.js`). If offline, it will be skipped and downloaded on first run.
34
+
35
+ If auto-registration did not update your IDE config, run:
36
+
37
+ ```bash
38
+ heuristic-mcp --start
39
+ ```
40
+
41
+ ---
42
+
43
+ ## CLI Commands
44
+
45
+ The `heuristic-mcp` binary manages the server lifecycle.
46
+
47
+ ### Status
48
+
49
+ ```bash
50
+ heuristic-mcp --status
51
+ ```
52
+
53
+ Shows server PID(s) and cache stats.
54
+
55
+ ### Logs
56
+
57
+ ```bash
58
+ heuristic-mcp --logs
59
+ ```
60
+
61
+ Tails the server log for the current workspace (defaults to last 200 lines and follows).
62
+
63
+ Optional flags:
64
+
65
+ ```bash
66
+ heuristic-mcp --logs --tail 100
67
+ heuristic-mcp --logs --no-follow
68
+ ```
69
+
70
+ ### Version
71
+
72
+ ```bash
73
+ heuristic-mcp --version
74
+ ```
75
+
76
+ ### Start/Stop
77
+
78
+ ```bash
79
+ heuristic-mcp --start
80
+ heuristic-mcp --start antigravity
81
+ heuristic-mcp --start cursor
82
+ heuristic-mcp --start "Claude Desktop"
83
+ heuristic-mcp --stop
84
+ ```
85
+
86
+ `--start` registers (if needed) and enables the MCP server entry. `--stop` disables it so the IDE won't immediately respawn it. Restart/reload the IDE after `--start` to launch.
87
+
88
+ ### Clear Cache
89
+
90
+ ```bash
91
+ heuristic-mcp --clear-cache
92
+ ```
93
+
94
+ Clears the cache for the current working directory (or `--workspace` if provided) and removes stale cache directories without metadata.
95
+
96
+ ---
97
+
98
+ ## Configuration (`config.jsonc`)
99
+
100
+ Configuration is loaded from your workspace root when the server runs with `--workspace` (this is how IDEs launch it). In server mode, it falls back to the package `config.jsonc` (or `config.json`) and then your current working directory.
101
+
102
+ Example `config.jsonc`:
103
+
104
+ ```json
105
+ {
106
+ "excludePatterns": ["**/legacy-code/**", "**/*.test.ts"],
107
+ "fileNames": ["Dockerfile", ".env.example", "Makefile"],
108
+ "smartIndexing": true,
109
+ "embeddingModel": "jinaai/jina-embeddings-v2-base-code",
110
+ "workerThreads": 0,
111
+ "embeddingBatchSize": null,
112
+ "embeddingProcessNumThreads": 8,
113
+ "enableExplicitGc": false,
114
+ "recencyBoost": 0.1,
115
+ "recencyDecayDays": 30,
116
+ "callGraphEnabled": true,
117
+ "callGraphBoost": 0.15,
118
+ "annEnabled": true,
119
+ "vectorStoreFormat": "binary",
120
+ "vectorStoreContentMode": "external",
121
+ "vectorStoreLoadMode": "disk",
122
+ "contentCacheEntries": 256,
123
+ "vectorCacheEntries": 64,
124
+ "clearCacheAfterIndex": true
125
+ }
126
+ ```
127
+
128
+ ### Embedding Model & Dimension Options
129
+
130
+ **Default model:** `jinaai/jina-embeddings-v2-base-code` (768 dimensions)
131
+
132
+ > **Important:** The default Jina model was **not** trained with Matryoshka Representation Learning (MRL). Dimension reduction (`embeddingDimension`) will significantly degrade search quality with this model. Only use dimension reduction with MRL-trained models.
133
+
134
+ For faster search with smaller embeddings, switch to an MRL-compatible model:
135
+
136
+ ```json
137
+ {
138
+ "embeddingModel": "nomic-ai/nomic-embed-text-v1.5",
139
+ "embeddingDimension": 128
140
+ }
141
+ ```
142
+
143
+ **MRL-compatible models:**
144
+ - `nomic-ai/nomic-embed-text-v1.5` — recommended for 128d/256d
145
+ - Other models explicitly trained with Matryoshka loss
146
+
147
+ **embeddingDimension values:** `64 | 128 | 256 | 512 | 768 | null` (null = full dimensions)
148
+
149
+ Cache location:
150
+
151
+ - By default, the cache is stored in a global OS cache directory under `heuristic-mcp/<hash>`.
152
+ - You can override with `cacheDirectory` in your config file.
153
+
154
+ ### Environment Variables
155
+
156
+ Selected overrides (prefix `SMART_CODING_`):
157
+
158
+ - `SMART_CODING_VERBOSE=true|false` enable detailed logging.
159
+ - `SMART_CODING_WORKER_THREADS=auto|N` — worker thread count.
160
+ - `SMART_CODING_BATCH_SIZE=100` — files per indexing batch.
161
+ - `SMART_CODING_CHUNK_SIZE=25` — lines per chunk.
162
+ - `SMART_CODING_MAX_RESULTS=5` max search results.
163
+ - `SMART_CODING_EMBEDDING_BATCH_SIZE=64` — embedding batch size (1–256, overrides auto).
164
+ - `SMART_CODING_EMBEDDING_THREADS=8` — ONNX threads for the embedding child process.
165
+ - `SMART_CODING_RECENCY_BOOST=0.1` — boost for recently edited files.
166
+ - `SMART_CODING_RECENCY_DECAY_DAYS=30` — days until recency boost decays to 0.
167
+ - `SMART_CODING_ANN_ENABLED=true|false` — enable ANN index.
168
+ - `SMART_CODING_ANN_EF_SEARCH=64` — ANN search quality/speed tradeoff.
169
+ - `SMART_CODING_VECTOR_STORE_FORMAT=json|binary|sqlite` — on-disk vector store format.
170
+ - `SMART_CODING_VECTOR_STORE_CONTENT_MODE=external|inline` — where content is stored for binary format.
171
+ - `SMART_CODING_VECTOR_STORE_LOAD_MODE=memory|disk` — vector loading strategy.
172
+ - `SMART_CODING_CONTENT_CACHE_ENTRIES=256` — LRU entries for decoded content.
173
+ - `SMART_CODING_VECTOR_CACHE_ENTRIES=64` — LRU entries for vectors (disk mode).
174
+ - `SMART_CODING_CLEAR_CACHE_AFTER_INDEX=true|false` — drop in-memory vectors after indexing.
175
+ - `SMART_CODING_UNLOAD_MODEL_AFTER_INDEX=true|false` — unload embedding model after indexing to free RAM (~500MB-1GB).
176
+ - `SMART_CODING_EXPLICIT_GC=true|false` — opt-in to explicit GC (requires `--expose-gc`).
177
+ - `SMART_CODING_INCREMENTAL_GC_THRESHOLD_MB=2048` — RSS threshold for running incremental GC after watcher updates (requires explicit GC).
178
+ - `SMART_CODING_EMBEDDING_DIMENSION=64|128|256|512|768` — MRL dimension reduction (only for MRL-trained models).
179
+
180
+ See `lib/config.js` for the full list.
181
+
182
+ ### Binary Vector Store
183
+
184
+ Set `vectorStoreFormat` to `binary` to use the on-disk binary cache. This keeps vectors and content out of JS heap
185
+ and reads on demand. Recommended for large repos.
186
+
187
+ - `vectorStoreContentMode=external` keeps content in the binary file and only loads for top-N results.
188
+ - `contentCacheEntries` controls the small in-memory LRU for decoded content strings.
189
+ - `vectorStoreLoadMode=disk` streams vectors from disk to reduce memory usage.
190
+ - `vectorCacheEntries` controls the small in-memory LRU for vectors when using disk mode.
191
+ - `clearCacheAfterIndex=true` drops in-memory vectors after indexing and reloads lazily on next query.
192
+ - `unloadModelAfterIndex=true` (default) unloads the embedding model after indexing to free ~500MB-1GB of RAM; the model will reload on the next search query.
193
+ - Note: `annEnabled=true` with `vectorStoreLoadMode=disk` can increase disk reads during ANN rebuilds on large indexes.
194
+
195
+ ### SQLite Vector Store
196
+
197
+ Set `vectorStoreFormat` to `sqlite` to use SQLite for persistence. This provides:
198
+
199
+ - ACID transactions for reliable writes
200
+ - Simpler concurrent access
201
+ - Standard database format for inspection
202
+
203
+ ```json
204
+ {
205
+ "vectorStoreFormat": "sqlite"
206
+ }
207
+ ```
208
+
209
+ The vectors and content are stored in `vectors.sqlite` in your cache directory. You can inspect it with any SQLite browser.
210
+ `vectorStoreContentMode` and `vectorStoreLoadMode` are respected for SQLite (use `vectorStoreLoadMode=disk` to avoid loading vectors into memory).
211
+
212
+ **Tradeoffs vs Binary:**
213
+ - Slightly higher read overhead (SQL queries vs direct memory access)
214
+ - Better write reliability (transactions)
215
+ - Easier debugging (standard SQLite file)
216
+
217
+ ### Benchmarking Search
218
+
219
+ Use the built-in script to compare memory vs latency tradeoffs:
220
+
221
+ ```bash
222
+ node tools/scripts/benchmark-search.js --query "database connection" --runs 10
223
+ ```
224
+
225
+ Compare modes quickly:
226
+
227
+ ```bash
228
+ SMART_CODING_VECTOR_STORE_LOAD_MODE=memory node tools/scripts/benchmark-search.js --runs 10
229
+ SMART_CODING_VECTOR_STORE_LOAD_MODE=disk node tools/scripts/benchmark-search.js --runs 10
230
+ SMART_CODING_VECTOR_STORE_FORMAT=binary SMART_CODING_VECTOR_STORE_LOAD_MODE=disk node tools/scripts/benchmark-search.js --runs 10
231
+ ```
232
+
233
+ Note: On small repos, disk mode may be slightly slower and show noisy RSS deltas; benefits are clearer on large indexes with a small `vectorCacheEntries`.
234
+
235
+ ---
236
+
237
+ ## MCP Tools Reference
238
+
239
+ ### `a_semantic_search`
240
+ Find code by meaning. Ideal for natural language queries like "authentication logic" or "database queries".
241
+
242
+ ### `b_index_codebase`
243
+ Manually trigger a full reindex. Useful after large code changes.
244
+
245
+ ### `c_clear_cache`
246
+ Clear the embeddings cache and force reindex.
247
+
248
+ ### `d_ann_config`
249
+ Configure the ANN (Approximate Nearest Neighbor) index. Actions: `stats`, `set_ef_search`, `rebuild`.
250
+
251
+ ### `d_find_similar_code`
252
+ Find similar code patterns given a snippet. Useful for finding duplicates or refactoring opportunities.
253
+
254
+ ### `e_check_package_version`
255
+ Fetch the latest version of a package from its official registry.
256
+
257
+ **Supported registries:**
258
+ - **npm** (default): `lodash`, `@types/node`
259
+ - **PyPI**: `pip:requests`, `pypi:django`
260
+ - **crates.io**: `cargo:serde`, `rust:tokio`
261
+ - **Maven**: `maven:org.springframework:spring-core`
262
+ - **Go**: `go:github.com/gin-gonic/gin`
263
+ - **RubyGems**: `gem:rails`
264
+ - **NuGet**: `nuget:Newtonsoft.Json`
265
+ - **Packagist**: `composer:laravel/framework`
266
+ - **Hex**: `hex:phoenix`
267
+ - **pub.dev**: `pub:flutter`
268
+ - **Homebrew**: `brew:node`
269
+ - **Conda**: `conda:numpy`
270
+
271
+ ### `f_set_workspace`
272
+ Change the workspace directory at runtime. Updates search directory, cache location, and optionally triggers reindex.
273
+
274
+ **Parameters:**
275
+ - `workspacePath` (required): Absolute path to the new workspace
276
+ - `reindex` (optional, default: `true`): Whether to trigger a full reindex
277
+
278
+ ---
279
+
280
+ ## Troubleshooting
281
+
282
+ **Server isn't starting**
283
+
284
+ 1. Run `heuristic-mcp --status` to check config and cache status.
285
+ 2. Run `heuristic-mcp --logs` to see startup errors.
286
+
287
+ **Native ONNX backend unavailable (falls back to WASM)**
288
+
289
+ If you see log lines like:
290
+
291
+ ```
292
+ Native ONNX backend unavailable: The operating system cannot run %1.
293
+ ...onnxruntime_binding.node. Falling back to WASM.
294
+ ```
295
+
296
+ The server will automatically disable workers and force `embeddingProcessPerBatch` to reduce memory spikes, but you
297
+ should fix the native binding to restore stable memory usage:
298
+
299
+ - Ensure you are running **64-bit Node.js** (`node -p "process.arch"` should be `x64`).
300
+ - Install **Microsoft Visual C++ 2015–2022 Redistributable (x64)**.
301
+ - Reinstall dependencies (clears locked native binaries):
302
+
303
+ ```bash
304
+ Remove-Item -Recurse -Force node_modules\\onnxruntime-node, node_modules\\.onnxruntime-node-* -ErrorAction SilentlyContinue
305
+ npm install
306
+ ```
307
+
308
+ If you see a warning about **version mismatch** (e.g. "onnxruntime-node 1.23.x incompatible with transformers.js
309
+ expectation 1.14.x"), install the matching version:
310
+
311
+ ```bash
312
+ npm install onnxruntime-node@1.14.0
313
+ ```
314
+
315
+ **Search returns no results**
316
+
317
+ - Check `heuristic-mcp --status` for indexing progress.
318
+ - If indexing shows zero files, review `excludePatterns` and `fileExtensions`.
319
+
320
+ **Model download fails**
321
+
322
+ - The install step tries to pre-download the model, but it can be skipped offline.
323
+ - The server will download on first run; ensure network access at least once.
324
+
325
+ **Clear cache**
326
+
327
+ - Use the MCP tool `c_clear_cache`, run `heuristic-mcp --clear-cache`, or delete the cache directory. For local dev, run `npm run clean`.
328
+
329
+ **Inspect cache**
330
+
331
+ ```bash
332
+ node tools/scripts/cache-stats.js --workspace <path>
333
+ ```
334
+
335
+ **Stop doesn't stick**
336
+
337
+ - The IDE will auto-restart the server if it's still enabled in its config. `--stop` now disables the server entry for Antigravity, Cursor, Claude Desktop, and VS Code (when using common MCP settings keys). Restart the IDE after `--start` to re-enable.
338
+
339
+ ---
340
+
341
+ ## Contributing
342
+
343
+ See `CONTRIBUTING.md` for guidelines.
344
+
345
+ License: MIT