codesearch-mcp 0.3.0-alpha → 0.4.0-alpha
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +59 -6
- package/LICENSE +661 -21
- package/README.md +33 -17
- package/dist/app.js +13 -20
- package/dist/src/ast/CallExtractor.d.ts +8 -0
- package/dist/src/ast/CallExtractor.js +68 -0
- package/dist/src/ast/GraphqlNameExtractor.d.ts +22 -0
- package/dist/src/ast/GraphqlNameExtractor.js +240 -0
- package/dist/src/ast/ImportEdgeExtractor.d.ts +11 -0
- package/dist/src/ast/ImportEdgeExtractor.js +129 -0
- package/dist/src/ast/IndexedNodeResolver.d.ts +14 -0
- package/dist/src/ast/IndexedNodeResolver.js +217 -0
- package/dist/src/ast/MemberExtractor.d.ts +8 -0
- package/dist/src/ast/MemberExtractor.js +129 -0
- package/dist/src/ast/TopLevelSymbolExtractor.d.ts +5 -0
- package/dist/src/ast/TopLevelSymbolExtractor.js +86 -0
- package/dist/src/ast/graphql-grammar.d.ts +24 -0
- package/dist/src/ast/graphql-grammar.js +93 -0
- package/dist/src/ast/node-helpers.d.ts +11 -0
- package/dist/src/ast/node-helpers.js +55 -0
- package/dist/src/commands/EvalCommand.d.ts +7 -3
- package/dist/src/commands/EvalCommand.js +36 -158
- package/dist/src/commands/IndexCommand.d.ts +10 -4
- package/dist/src/commands/IndexCommand.js +52 -124
- package/dist/src/commands/McpCommand.d.ts +6 -3
- package/dist/src/commands/McpCommand.js +19 -14
- package/dist/src/commands/WebCommand.d.ts +9 -3
- package/dist/src/commands/WebCommand.js +17 -8
- package/dist/src/commands/eval/baseline-comparison.d.ts +6 -0
- package/dist/src/commands/eval/baseline-comparison.js +54 -0
- package/dist/src/commands/eval/eval-args.d.ts +6 -0
- package/dist/src/commands/eval/eval-args.js +71 -0
- package/dist/src/commands/eval/eval-corpora.d.ts +6 -0
- package/dist/src/commands/eval/eval-corpora.js +76 -0
- package/dist/src/commands/eval/eval-metadata.d.ts +17 -0
- package/dist/src/commands/eval/eval-metadata.js +75 -0
- package/dist/src/commands/eval/index-inspector.d.ts +3 -0
- package/dist/src/commands/eval/index-inspector.js +21 -0
- package/dist/src/commands/eval/staged-search.d.ts +3 -0
- package/dist/src/commands/eval/staged-search.js +20 -0
- package/dist/src/commands/eval/suite-loader.d.ts +8 -0
- package/dist/src/commands/eval/suite-loader.js +52 -0
- package/dist/src/commands/index/backend-probe.d.ts +2 -0
- package/dist/src/commands/index/backend-probe.js +15 -0
- package/dist/src/commands/index/embedding-setup.d.ts +3 -0
- package/dist/src/commands/index/embedding-setup.js +16 -0
- package/dist/src/commands/index/index-summary.d.ts +2 -0
- package/dist/src/commands/index/index-summary.js +22 -0
- package/dist/src/commands/index/legacy-ollama-guard.d.ts +3 -0
- package/dist/src/commands/index/legacy-ollama-guard.js +23 -0
- package/dist/src/commands/index/prune-step.d.ts +3 -0
- package/dist/src/commands/index/prune-step.js +22 -0
- package/dist/src/commands/index/scan-targets.d.ts +8 -0
- package/dist/src/commands/index/scan-targets.js +107 -0
- package/dist/src/config-store.d.ts +0 -1
- package/dist/src/config-store.js +3 -4
- package/dist/src/config.d.ts +1 -1
- package/dist/src/config.js +1 -1
- package/dist/src/constants.d.ts +15 -0
- package/dist/src/constants.js +34 -6
- package/dist/src/container.d.ts +4 -2
- package/dist/src/container.js +7 -5
- package/dist/src/db/drizzle-types.d.ts +7 -0
- package/dist/src/db/index.d.ts +1 -0
- package/dist/src/db/schema-steps.d.ts +8 -0
- package/dist/src/db/schema-steps.js +105 -0
- package/dist/src/declarations/DeclarationTree.d.ts +1 -1
- package/dist/src/eval-results.js +16 -2
- package/dist/src/eval-vector-index.d.ts +5 -0
- package/dist/src/eval-vector-index.js +60 -0
- package/dist/src/help.js +9 -1
- package/dist/src/index-args.d.ts +13 -0
- package/dist/src/index-args.js +48 -0
- package/dist/src/indexing/AnalysisBatcher.d.ts +12 -0
- package/dist/src/indexing/AnalysisBatcher.js +28 -0
- package/dist/src/indexing/ChunkEmbedder.d.ts +24 -0
- package/dist/src/indexing/ChunkEmbedder.js +37 -0
- package/dist/src/indexing/ConsoleScanReporter.d.ts +10 -0
- package/dist/src/indexing/ConsoleScanReporter.js +73 -0
- package/dist/src/indexing/EmbeddingSchemaPreparer.d.ts +14 -0
- package/dist/src/indexing/EmbeddingSchemaPreparer.js +66 -0
- package/dist/src/indexing/FileIngestor.d.ts +22 -0
- package/dist/src/indexing/FileIngestor.js +116 -0
- package/dist/src/indexing/IndexStore.d.ts +24 -0
- package/dist/src/indexing/IndexStore.js +69 -0
- package/dist/src/indexing/StaleFilePruner.d.ts +8 -0
- package/dist/src/indexing/StaleFilePruner.js +55 -0
- package/dist/src/indexing/analysis-rows.d.ts +4 -0
- package/dist/src/indexing/analysis-rows.js +65 -0
- package/dist/src/indexing/project-notices.d.ts +2 -0
- package/dist/src/indexing/project-notices.js +63 -0
- package/dist/src/interfaces/IAstParser.d.ts +2 -1
- package/dist/src/interfaces/ICallGraph.d.ts +28 -0
- package/dist/src/interfaces/ICallGraph.js +2 -0
- package/dist/src/interfaces/IChunkSearch.d.ts +42 -0
- package/dist/src/interfaces/IChunkSearch.js +2 -0
- package/dist/src/interfaces/IDatabase.d.ts +6 -2
- package/dist/src/interfaces/IDependencyGraph.d.ts +24 -0
- package/dist/src/interfaces/IDependencyGraph.js +2 -0
- package/dist/src/interfaces/IIndexIdentityGuard.d.ts +3 -0
- package/dist/src/interfaces/IIndexIdentityGuard.js +2 -0
- package/dist/src/interfaces/IIndexedNodeResolver.d.ts +10 -0
- package/dist/src/interfaces/IIndexedNodeResolver.js +2 -0
- package/dist/src/interfaces/IMcpReadiness.d.ts +12 -0
- package/dist/src/interfaces/IMcpReadiness.js +2 -0
- package/dist/src/interfaces/IProjectCatalog.d.ts +13 -0
- package/dist/src/interfaces/IProjectCatalog.js +2 -0
- package/dist/src/interfaces/IProjectRoots.d.ts +3 -0
- package/dist/src/interfaces/IProjectRoots.js +2 -0
- package/dist/src/interfaces/IRetrievalEvaluator.d.ts +21 -2
- package/dist/src/interfaces/IScanReporter.d.ts +13 -0
- package/dist/src/interfaces/IScanReporter.js +2 -0
- package/dist/src/interfaces/ISymbolLookup.d.ts +49 -0
- package/dist/src/interfaces/ISymbolLookup.js +2 -0
- package/dist/src/interfaces/IToolPaths.d.ts +62 -0
- package/dist/src/interfaces/IToolPaths.js +2 -0
- package/dist/src/interfaces/IWorkspaceFiles.d.ts +28 -0
- package/dist/src/interfaces/IWorkspaceFiles.js +2 -0
- package/dist/src/interfaces/SymbolKind.d.ts +1 -0
- package/dist/src/interfaces/SymbolKind.js +2 -0
- package/dist/src/interfaces/index.d.ts +12 -1
- package/dist/src/project-scope.d.ts +1 -0
- package/dist/src/project-scope.js +5 -0
- package/dist/src/retrieval-eval.d.ts +2 -1
- package/dist/src/retrieval-eval.js +45 -2
- package/dist/src/search/CallGraph.d.ts +9 -0
- package/dist/src/search/CallGraph.js +84 -0
- package/dist/src/search/ChunkSearch.d.ts +23 -0
- package/dist/src/search/ChunkSearch.js +115 -0
- package/dist/src/search/DependencyGraph.d.ts +19 -0
- package/dist/src/search/DependencyGraph.js +180 -0
- package/dist/src/search/IndexIdentityGuard.d.ts +11 -0
- package/dist/src/search/IndexIdentityGuard.js +37 -0
- package/dist/src/search/ProjectCatalog.d.ts +11 -0
- package/dist/src/search/ProjectCatalog.js +38 -0
- package/dist/src/search/SymbolLookup.d.ts +24 -0
- package/dist/src/search/SymbolLookup.js +97 -0
- package/dist/src/search/chunks/CandidateFetcher.d.ts +19 -0
- package/dist/src/search/chunks/CandidateFetcher.js +40 -0
- package/dist/src/search/chunks/IterativeScanGate.d.ts +10 -0
- package/dist/src/search/chunks/IterativeScanGate.js +42 -0
- package/dist/src/search/chunks/RerankStage.d.ts +15 -0
- package/dist/src/search/chunks/RerankStage.js +42 -0
- package/dist/src/search/chunks/search-metrics.d.ts +18 -0
- package/dist/src/search/chunks/search-metrics.js +32 -0
- package/dist/src/search/chunks/search-plan.d.ts +18 -0
- package/dist/src/search/chunks/search-plan.js +19 -0
- package/dist/src/search/create-search-services.d.ts +26 -0
- package/dist/src/search/create-search-services.js +29 -0
- package/dist/src/search/index-paths.d.ts +1 -0
- package/dist/src/search/index-paths.js +5 -0
- package/dist/src/search/symbol-sql.d.ts +13 -0
- package/dist/src/search/symbol-sql.js +66 -0
- package/dist/src/search/symbols/SymbolGrepFallback.d.ts +19 -0
- package/dist/src/search/symbols/SymbolGrepFallback.js +106 -0
- package/dist/src/search/symbols/SymbolIndexQuery.d.ts +14 -0
- package/dist/src/search/symbols/SymbolIndexQuery.js +52 -0
- package/dist/src/search/symbols/SymbolTextSearch.d.ts +13 -0
- package/dist/src/search/symbols/SymbolTextSearch.js +84 -0
- package/dist/src/search/symbols/TargetColumnGate.d.ts +7 -0
- package/dist/src/search/symbols/TargetColumnGate.js +23 -0
- package/dist/src/search/symbols/find-symbol-constants.d.ts +4 -0
- package/dist/src/search/symbols/find-symbol-constants.js +8 -0
- package/dist/src/search/symbols/grep-args.d.ts +3 -0
- package/dist/src/search/symbols/grep-args.js +7 -0
- package/dist/src/search/symbols/grep-patterns.d.ts +4 -0
- package/dist/src/search/symbols/grep-patterns.js +60 -0
- package/dist/src/services/AstParserService.d.ts +8 -44
- package/dist/src/services/AstParserService.js +19 -938
- package/dist/src/services/DatabaseService.d.ts +8 -10
- package/dist/src/services/DatabaseService.js +29 -86
- package/dist/src/services/IndexerService.d.ts +7 -10
- package/dist/src/services/IndexerService.js +36 -255
- package/dist/src/services/McpReadiness.d.ts +18 -0
- package/dist/src/services/McpReadiness.js +73 -0
- package/dist/src/services/McpServer.d.ts +32 -9
- package/dist/src/services/McpServer.js +78 -41
- package/dist/src/services/WebServerService.d.ts +4 -3
- package/dist/src/services/WebServerService.js +16 -8
- package/dist/src/setup.d.ts +16 -4
- package/dist/src/setup.js +115 -40
- package/dist/src/symbol-name.d.ts +1 -0
- package/dist/src/symbol-name.js +16 -12
- package/dist/src/tools/FindSymbolTool.d.ts +17 -7
- package/dist/src/tools/FindSymbolTool.js +54 -35
- package/dist/src/tools/GetCallersTool.d.ts +16 -8
- package/dist/src/tools/GetCallersTool.js +65 -35
- package/dist/src/tools/GetDependenciesTool.d.ts +16 -8
- package/dist/src/tools/GetDependenciesTool.js +35 -28
- package/dist/src/tools/GetDependentsTool.d.ts +20 -8
- package/dist/src/tools/GetDependentsTool.js +47 -33
- package/dist/src/tools/ListFilesTool.d.ts +16 -8
- package/dist/src/tools/ListFilesTool.js +20 -26
- package/dist/src/tools/ListProjectsTool.d.ts +11 -7
- package/dist/src/tools/ListProjectsTool.js +13 -11
- package/dist/src/tools/McpTool.d.ts +17 -0
- package/dist/src/tools/McpTool.js +28 -0
- package/dist/src/tools/ReadFileRangeTool.d.ts +16 -8
- package/dist/src/tools/ReadFileRangeTool.js +23 -25
- package/dist/src/tools/ReadFileTool.d.ts +15 -8
- package/dist/src/tools/ReadFileTool.js +19 -24
- package/dist/src/tools/SearchCodebaseTool.d.ts +22 -10
- package/dist/src/tools/SearchCodebaseTool.js +69 -51
- package/dist/src/tools/format-helpers.d.ts +9 -3
- package/dist/src/tools/format-helpers.js +59 -14
- package/dist/src/tools/tool-args.d.ts +38 -0
- package/dist/src/tools/tool-args.js +95 -0
- package/dist/src/workspace/GrepRunner.d.ts +15 -0
- package/dist/src/workspace/GrepRunner.js +104 -0
- package/dist/src/workspace/ProjectRoots.d.ts +22 -0
- package/dist/src/workspace/ProjectRoots.js +102 -0
- package/dist/src/workspace/ToolPaths.d.ts +31 -0
- package/dist/src/workspace/ToolPaths.js +312 -0
- package/dist/src/workspace/WorkspaceFiles.d.ts +19 -0
- package/dist/src/workspace/WorkspaceFiles.js +285 -0
- package/dist/src/workspace/fs-access.d.ts +9 -0
- package/dist/src/workspace/fs-access.js +47 -0
- package/dist/src/workspace/path-helpers.d.ts +3 -0
- package/dist/src/workspace/path-helpers.js +43 -0
- package/package.json +4 -3
- package/dist/src/interfaces/ISearcher.d.ts +0 -46
- package/dist/src/services/SearcherService.d.ts +0 -235
- package/dist/src/services/SearcherService.js +0 -1118
- /package/dist/src/{interfaces/ISearcher.js → db/drizzle-types.js} +0 -0
package/CHANGELOG.md
CHANGED
|
@@ -3,14 +3,67 @@
|
|
|
3
3
|
All notable changes to this project will be documented in this file.
|
|
4
4
|
Format based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
5
5
|
|
|
6
|
+
## [0.4.0-alpha] - 2026-11-14
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
|
|
10
|
+
- **`rag-index --project <name> --require-cwd`** — refuses the run, with exit code 1 and before any database or embedding work, unless the current directory is the directory of the project or unit that name resolves to. A linked worktree, a second clone or a subdirectory is refused, naming both directories, instead of indexing the main checkout and exiting 0. Directories are compared by real path. Without a `--project` name the flag is a usage error and no prompt opens. A relative `workspaceRoot` is refused. In a root-level Nx workspace with no discovered project at its root, no name matches that root, so every run from there is refused.
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- **License: `AGPL-3.0-only`** — from this version on the project is licensed under the GNU Affero General Public License v3.0 only instead of MIT. `0.3.0-alpha` and earlier releases remain available under the MIT License.
|
|
15
|
+
- **The vector index is built with more links per chunk** — the HNSW index on `code_chunk.embedding` is created with `m = 32` and `ef_construction = 200` instead of pgvector's defaults (16 and 64), so a search reaches the closest chunks more often: on 500 sampled chunks of a 77 k-chunk index, a search for a chunk's own vector with the `hnsw.ef_search` a default search sets (81) missed it in 0.2% of cases instead of 1.8%. The index is larger (about 600 MB for 77 k chunks of 1024 dimensions) and takes longer to build. An existing index keeps its parameters: to rebuild it, run `DROP INDEX idx_code_chunk_embedding;` and run `rag-index`, which recreates it. No re-index.
|
|
16
|
+
- **`rag-index eval` records the vector index it searched** _(internal development tool)_ — every result file and report names each index on `code_chunk.embedding` with its operator class and build parameters (`m`, `ef_construction`, or `lists`, with pgvector's defaults for what `WITH` left out), the pgvector version and the server's `hnsw.max_scan_tuples` and `hnsw.scan_mem_multiplier`, read before the first search (`**Vector index:** idx_code_chunk_embedding hnsw (vector_cosine_ops, m=32, ef_construction=200) | pgvector 0.8.2 | hnsw.max_scan_tuples 20000 | hnsw.scan_mem_multiplier 1`); if the read fails, the run stops before searching. `--baseline` warns when any of them differs, and when either file records no vector index (`baseline records no vector index`).
|
|
17
|
+
- **Log messages of search, path and file reading are prefixed with the class that writes them** — `ChunkSearch:`, `IterativeScanGate:`, `RerankStage:`, `SymbolLookup:`, `SymbolTextSearch:`, `SymbolGrepFallback:`, `DependencyGraph:`, `CallGraph:`, `ToolPaths:`, `WorkspaceFiles:` or `GrepRunner:` instead of `SearcherService:`, e.g. `GrepRunner: grep timed out`. The text after the prefix, the level and the fields are unchanged; a log filter on `SearcherService:` must use the class names.
|
|
18
|
+
- **BREAKING (behavior): `rag-index` exits non-zero when a file fails to index** — in every mode. The run prints the summary, and stderr carries `Indexing finished with <n> of <total> file(s) failed.` after it; unless interrupted, the run also still indexes the remaining files and projects and writes the project graph. It exits with the new code `2` when the rest of the index is usable, and `1` when the run was interrupted, something else already failed (such as a prune), no file was skipped and the run stored no chunks, or a final check of the database and the embedding backend fails (`Indexing backend unavailable: <cause>`). A script that relied on exit 0 after failed files now sees a failure. An interrupted run with no failed file still exits 0. The next run retries a failed file, and `rag-index --help` lists the exit codes.
|
|
19
|
+
- **BREAKING (behavior): `rag-index` refuses a second `--project`** — `rag-index --project a --project b` prints `Unexpected argument: --project` and the help, and exits 1 without indexing; it used to scan `a`. `rag-index --project --project b` is refused the same way instead of opening the prompt.
|
|
20
|
+
- **BREAKING (MCP tool arguments): invalid values are errors, not silent defaults** — every tool now checks its arguments before running, so a bad call runs no query, grep or embedding request. A missing or blank required text argument answers `Missing <argument>`: `search_codebase` with `query: ""` no longer reaches the embedding API, without `query` it no longer searches for the word `undefined`, and `get_callers` without `project` no longer answers about an empty project name; `find_symbol` and `get_callers` answer `Missing name` / `Missing symbol` instead of `Invalid symbol name` for an empty value. A value of the wrong type answers `Invalid <argument>: expected a string` / `a number` / `a boolean`, and a value outside an enum answers `Invalid <argument>: expected one of …`, instead of being replaced: an unknown `kind` no longer means `all`, and a non-numeric `threshold`, `limit`, `per_file`, `max_lines`, `depth`, `start_line` or `end_line`, or a non-boolean `rerank`, no longer falls back to the default. A non-boolean `transitive`, `include_neighbors` or `include_graph_context` used to be read by truthiness, so `1` and `"false"` meant true: `"true"` and `"false"` now read by value, and any other non-boolean is refused. `file_path: 123` answers `Invalid file_path: expected a string` instead of being read as the path `123`. `limit: 0` means 1 instead of the default, and so does a value below 1 for `max_lines`, `depth` or `per_file`, negatives included. Integer arguments drop their fraction (`threshold` does not), and `end_line: 0` answers `end_line 0 is before start_line …`. Numeric strings are accepted, `null` or a blank optional value means omitted (for the optional path `callee_file`, only `null` or `""`), unknown arguments stay ignored, a number above its maximum is still clamped, and path refusals are unchanged. Tool names, argument names and the advertised schemas do not change; no re-index.
|
|
21
|
+
- **BREAKING (`search_codebase` output): `**Relevance:**` is now `**Distance:**`** — the number is unchanged: it is the cosine distance, so a lower value is a closer match. A response with results starts with one line saying that, and that the results may be reranked, so their order need not follow the distance. The server instructions section `## Relevance scores` is now `## Distance scores`. Clients get the new text after reconnecting; no re-index.
|
|
22
|
+
- **An empty `search_codebase` response says why** — after `No relevant results found.` it says the project has no indexed chunks (check the name with `list_projects`), the index is empty, or no chunk came within the distance threshold, which does not show the code is absent, followed by what to try next.
|
|
23
|
+
- **An empty `get_callers` result names its cause** — instead of advising `rag-index --reindex`, the message says the project has no call graph (the only case that advises indexing, with `rag-index --project <name>`), or no such symbol is indexed (with up to 5 similar names, and a note that the index may be behind the source, so Grep should confirm), or the symbol is defined but no calls are recorded (with up to 5 definitions, the possible reasons, and how to continue with `get_dependents`, `list_projects` and Grep). With `callee_file` it also suggests retrying without it. If the cause cannot be looked up, it says so.
|
|
24
|
+
- **BREAKING (behavior): `search.similarityThreshold` follows the embedding model unless set** — the key has no fixed default any more. The effective threshold is the call's `threshold`, else the key, else the recommended value of `embedding.model` (`0.95` for `voyage-code-4`, measured on the frozen eval corpora, and inherited unmeasured by `voyage-4`, `voyage-4-large` and `voyage-4-lite`; `0.6` for `jinaai/jina-embeddings-v2-base-code`), else `1.0`, which cuts only chunks at cosine distance 1 or more. With a reranker active, `1.0` sends the full candidate pool (80 by default) to it on every reranked search in a scope holding that many chunks, which costs latency on a local reranker and per-document fees on a cloud one; set the key to avoid that. A `jinaai/jina-embeddings-v2-base-code` install without the key behaves as before; any other model that relied on the implicit `0.6` now uses its own value or `1.0`. An explicit value more than `0.1` from the recommendation is reported, without stopping anything, in the MCP server instructions and the log (`warn`), and on the console at the start of `rag-index eval`; remove the key to follow the model, or retune it. No re-index. To restore the previous behaviour, set the key to the previous value. An existing `config.schema.json` keeps the old description until the next config write or `rag-config edit`.
|
|
25
|
+
- **`search_codebase` guidance states no fixed distances** — the MCP server instructions and the `threshold` parameter description show the effective default for the current model and describe adjustments relative to it. The advice for an empty result is to rephrase, use `find_symbol`, drop `project` or raise the threshold, never to tighten it.
|
|
26
|
+
- **`rag-setup` asks for the similarity threshold** — after the embedding and reranker sections it shows the chosen model's recommended value. Enter leaves the key out, so the threshold keeps following the model; a number from 0 to 1 is written as `search.similarityThreshold`.
|
|
27
|
+
- **The dashboard shows the threshold resolved for the saved model** — an unset `search.similarityThreshold` renders empty, with that value as the placeholder and the `using default` hint.
|
|
28
|
+
- **`rag-index eval` records where the threshold came from** _(internal development tool)_ — result files hold the distinct `{ value, source }` pairs the searches reported (`threshold 0.95 (model), 0.8 (call)`). An older result file holding `similarityThreshold` loads with the source `not recorded`, and `--baseline` warns on a differing pair, source included.
|
|
29
|
+
- **BREAKING (runtime): Node.js `^22.13.0` or `>=23.5.0` is required** — `engines` and the README said `>=18`. Node 18 and 20 are end-of-life and no longer supported; 22.0–22.12 and 23.0–23.4 are excluded because `@inquirer/prompts` does not support them.
|
|
30
|
+
- **BREAKING (wizard only): `rag-setup` has a short Voyage path** — after choosing `voyage`, the wizard asks for the API key first (required; a blank answer is re-asked) and stores it once as `VOYAGE_API_KEY`; it no longer asks for a base URL or writes `embedding.baseURL` (the default `https://api.voyageai.com/v1` applies). The embedding model is picked from a numbered list (`voyage-code-4` default, `voyage-4-large`, `voyage-4`, `voyage-4-lite`): a number, Enter for the default, or any typed model name. The reranker is one question, `Enable Voyage reranker? (y/n) (y)`: `y` picks `rerank-3` or `rerank-3-lite` the same way and writes `reranker.apiKey: env:VOYAGE_API_KEY` — no second key prompt; `n` writes `reranker: { enabled: false }`. With a Voyage embedder the wizard no longer offers an `openai-compatible` reranker or a separate `VOYAGE_RERANK_API_KEY`; both remain configurable via `rag-config set` / `rag-config edit` / the dashboard. Existing configs and the `infinity` / `openai-compatible` wizard paths are unchanged.
|
|
31
|
+
- **BREAKING (behavior): `rag-mcp` starts not ready when a startup check fails, and recovers by itself** — when the database connection, the PostgreSQL probe or the index/model check (`Index was built with … Run \`rag-index --reindex\``) fails, `rag-mcp` writes one line `codesearch-mcp: <cause>` to stderr (whatever `logging.level` / `logging.dir` say), then still answers the MCP handshake, so `/mcp` shows `codesearch-mcp` as connected. Its server instructions begin with a `## codesearch-mcp is not ready` section holding the cause, quoted on one line in a fenced block so that text from the database cannot add headings or instructions of its own. `tools/list` lists every tool; each tool call first repeats the startup check, and while it fails answers `Tool error: codesearch-mcp is not ready: <cause>` with a hint on reconnecting. Once a cause outside the config is fixed (PostgreSQL started, `rag-index --reindex`), the next tool call passes the check and the server serves normally, with no reconnect. The re-check uses the config loaded at start, so after editing the config reconnect with `/mcp`. Only a failure to start the MCP server itself still exits with code 1. The "MCP server is unavailable" tool response — a server that connected but refused every call until restarted — is removed.
|
|
32
|
+
- **`rag-mcp` gives up on a database connection after 5 seconds** — with a database host that drops packets, `rag-mcp` now starts not ready in about 5 s (stderr line `codesearch-mcp: Connection terminated due to connection timeout`, and the not-ready section in its instructions) instead of hanging, and each re-check on a tool call is bounded the same way; a startup check or re-check whose index/model check stalls after the connection succeeded fails after 10 s (`startup check did not finish within 10 s`). The same bound applies to connections opened during a session and to a tool call waiting for a free pooled connection (`Tool error: timeout exceeded when trying to connect`). `rag-index`, `rag-index eval` and `rag-web` connect without a bound, as before. No config key.
|
|
33
|
+
- **`Tool error:` names the cause of an empty-message error** — a handler failing with a message-less `AggregateError` (PostgreSQL refusing both `::1` and `127.0.0.1`) answers `Tool error: connect ECONNREFUSED ::1:5432; connect ECONNREFUSED 127.0.0.1:5432` instead of `Tool error: `.
|
|
34
|
+
|
|
35
|
+
- **`search_codebase` without reranking fetches a wider pool when the per-file limit is active** — with `per_file` below `limit` (the default 2 is), a search without reranking now fetches the same candidate pool as a reranked one, `min(max(limit × reranker.overfetch, reranker.candidateFloor), 200)` (80 at the defaults), instead of `min(limit × 3, 200)`. One file holding the nearest chunks could fill the old pool, so the search returned fewer than `limit` results and missed other files. Results stay in cosine order. `reranker.overfetch` and `reranker.candidateFloor` now also size this pool; no re-index.
|
|
36
|
+
- **`read_file` and `read_file_range` refuse binary files** — a file with a NUL byte in its first 8 KiB answers the error `Binary file: <path>` instead of printing its bytes. A UTF-16 text file counts as binary.
|
|
37
|
+
- **`get_dependencies` and `get_dependents` name a `symbol` filter that removed every direct row** — instead of `No dependencies found` / `No direct import dependents found` for a file that has imports, the answer says how many files the unfiltered result holds, which names they import, how many use a default, namespace, side-effect or `export *` import the filter cannot match, and to retry without `symbol`. `get_dependents` still runs its text-search fallback after that text. With `transitive: true` the transitive heading says that `symbol` does not filter it, and the `symbol` descriptions of both tools state what the filter sees.
|
|
38
|
+
- **An empty `find_symbol` result lists similar indexed names** — up to 5 names starting with the last segment, ignoring case and kind, as `get_callers` does, so a name in the wrong case or of the wrong `kind` gets a suggestion, with advice to check with Grep.
|
|
39
|
+
- **`Did you mean:` offers near names** — a path whose last segment is within two edits of an entry of its parent directory, and fewer edits than half its length (counted in characters), gets that entry: `RAG/src/constant.ts` suggests `RAG/src/constants.ts`, and `list_files` with `RAG/srcc` suggests `RAG/src`.
|
|
40
|
+
- **`list_files` with a `pattern` prints only what matches** — a file is printed only when its name contains the pattern, and a directory only when a match or an unreadable directory lies below it. With no match the answer is `No files matching "<pattern>" found within depth <n>.`
|
|
41
|
+
- **Neighbour chunks show their `Symbol` line** — the chunks `include_neighbors` adds now carry the owner breadcrumb, as results do.
|
|
42
|
+
|
|
43
|
+
### Fixed
|
|
44
|
+
|
|
45
|
+
- **`find_symbol` says why grep found nothing** — a `project` with no directory in the workspace answers ``Project `x` has no directory in the workspace, so source files were not searched. Check the name with `list_projects`.`` instead of `(grep fallback failed — see log)`, at a log level that wrote no line. A project on disk without index rows is still searched. A real grep failure names its cause, `(grep fallback failed: <cause>)`: grep's first stderr line, its exit status or signal, the timeout, or the spawn error code. A grep that failed after printing matches shows them under `*(partial — some files could not be searched)*` instead of discarding them.
|
|
46
|
+
- **`find_symbol` no longer runs grep outside the workspace** — a `project` such as `../..`, or a name that is a symlink out of the workspace, is answered as a project with no directory.
|
|
47
|
+
- **`find_symbol` no longer drops a container that does not exist** — `Wrong.resolveToolPath` used to report the indexed `ToolPaths.resolveToolPath` as a grep hit `not in index`. A qualified name whose last segment is indexed, but not under that container, now answers so without grep and lists the similar names; a grep hit for a qualified name says its container was not checked.
|
|
48
|
+
- **`get_dependents` no longer passes `symbol` to grep as an option** — in the text-search fallback, a value starting with `-`, such as `--file=<path>`, made grep read that path as its pattern list. It is now searched for as literal text. A value holding a newline, which used to list unrelated lines as usages, answers `No results.`.
|
|
49
|
+
- **A grep that cannot start answers as a grep failure** — a NUL byte in `find_symbol`'s `project`, or a `project` over the system limit, made `find_symbol` answer the raw spawn error, and running out of file descriptors (`EMFILE`, `ENFILE`) ended the process. Such a `project` now answers that it has no directory in the workspace, without running grep; a grep that cannot start answers `(grep fallback failed: <error code>)` in `find_symbol`, and `get_dependents` ends its fallback section in ` failed`.
|
|
50
|
+
- **A grep ended by a signal is a failure, not an empty result** — a killed grep used to read as a clean search, so `find_symbol` and `get_dependents` could report no usages although the search never finished. They now answer as for any failed grep, and `get_dependents` marks matches printed before the kill as partial.
|
|
51
|
+
- **A grep can no longer run without limit** — a grep still running after 60 seconds is killed and answers as a failed grep, and one still running when the process exits normally, for example after its client closes stdin, is killed with it. A server ended by a signal cannot do this, and its grep runs until it ends by itself. `find_symbol` refuses a bare or qualified name, and `get_dependents` a `symbol`, over 256 characters with `Invalid symbol name`, since a long repetitive value could hold a CPU for minutes.
|
|
52
|
+
- **A very long line no longer ends the process** — `read_file` and `read_file_range` answer `Cannot read: <path> (ELINETOOLONG)` for a line over 64 MiB, counted in file bytes between `\n` and `\r`. A line over about 512 MB used to crash the server, and a line from 64 MiB up to that size used to be returned. A range or `max_lines` ending before the line still returns its lines.
|
|
53
|
+
- **A very long path answers at once** — a path argument of 4096 bytes or more, or one that reaches 4096 bytes together with the workspace root, answers `Cannot read: <path> (ENAMETOOLONG)` with no search for its existing ancestors. A path of many missing segments used to keep one call busy for minutes or hours, past a cancel, and over about 120000 segments answered `EIO`; a path of many `..` segments stopped the whole server for seconds.
|
|
54
|
+
- **Grep no longer loses matches** — `find_symbol` and the `get_dependents` text search list a match in a file grep takes for binary, which used to answer as if nothing matched, and `find_symbol` lists a match on a line ending in CRLF or holding U+2028 or U+2029, which it used to skip.
|
|
55
|
+
- **A very long `get_callers` symbol is read like any other** — a `symbol` of over about 2 million `.`-separated segments answered `Tool error: Maximum call stack size exceeded`. It now answers `Invalid symbol name` when malformed, and is looked up when well formed.
|
|
56
|
+
- **`get_callers` rejects a malformed `symbol` first** — with a `callee_file`, it used to query the index and could answer a path error; it now answers `Invalid symbol name` with no query.
|
|
57
|
+
- **A dropped idle database connection no longer crashes the process** — when PostgreSQL restarts or drops an idle pooled connection, the process keeps running in every mode and logs `warn` `DatabaseService: idle client error` (`rag-setup` has no log, so there the line is not written). A connection lost while a client is checked out by an indexing transaction still ends the process. In `rag-mcp`, while the database is down, tools that query it answer `Tool error: …` (`find_symbol` falls back to grep, as before), and they succeed again once it is back, without reconnecting the server. `get_callers` now answers a database error as `Tool error: …` too, and a database timeout during the index/model check is no longer reported as a legacy `ollama`/`bge-m3` index.
|
|
58
|
+
|
|
6
59
|
## [0.3.0-alpha] - 2026-10-31
|
|
7
60
|
|
|
8
61
|
### Added
|
|
9
62
|
|
|
10
63
|
- **One `tool call` log line per MCP tool call** — at `logging.level` `info` (or `debug`) the MCP server writes one line per completed call, whatever the outcome (result, handler error, unknown tool, refused after a failed startup, cancelled by the client): `tool`, `args` (strings cut at 500 characters, arrays at 20 items), `isError`, `responseChars` (the response size — never its content), `ms`, and the caller: `client` (`{name, version}` from `initialize`), `connectionId` (random, one per server process), `session` (`CLAUDE_CODE_SESSION_ID`, or `null`), `meta` (the request's `_meta` without `progressToken`), `callId` (a per-process counter) and, for Claude Code, `toolUseId`; `cancelled: true` marks a call the client cancelled. Find them with `jq 'select(.msg=="tool call")'`.
|
|
11
64
|
- **Call context on every MCP server log line** — every line of the MCP server carries its process's `connectionId`, and every line written during a tool call, at any level and from any service (a search debug trace, a `rerank failed` warning, a handler error), also carries that call's `callId` and `toolUseId`. `rag-index`, `rag-web` and eval logs are unchanged.
|
|
12
|
-
- **`rag-index eval` reports each search stage separately** _(internal development tool: the eval suites
|
|
13
|
-
- **`rag-index eval` runs on frozen corpora** _(internal development tool: the eval suites
|
|
65
|
+
- **`rag-index eval` reports each search stage separately** _(internal development tool: the eval suites and corpora are not shipped in the npm package)_ — every case is searched once and scored at the `vector` stage (the candidate pool's cosine order as the reranker receives it, cut to 10 — not the same as a `rerank: false` search, which fetches a smaller pool) and, when a reranker is configured, the `reranked` stage. Each case also records where its answer sits anywhere in the candidate pool and the pool's size; the run reports pool recall and how many cases got a pool shorter than the size their search requested. Within a run, the `vector` → `reranked` change is reported per category as improved / worsened / unchanged cases, the cases that gained or lost hit@5, and an exact two-sided sign-test p-value. A failed `rerank()` is still scored on the vector-order fallback `search_codebase` returns, but counted and the `reranked` stage flagged as degraded. Result files record the search conditions (similarity threshold, `reranker.overfetch`, `reranker.candidateFloor`, and the pool sizes and HNSW settings the searches reported). `--baseline` compares per stage with the same paired counts, compares pool recall and short pools, names stages present on one side only, and warns on any differing condition or rerank failures. Result files written before this change load as a baseline in the stage their recorded reranker implies, with "no pool data" and a missing-conditions warning. Each category row shows the share of one case, so a single-case move is visible as such. No change to `search_codebase` results.
|
|
66
|
+
- **`rag-index eval` runs on frozen corpora** _(internal development tool: the eval suites and corpora are not shipped in the npm package)_ — the default suite directory `benchmarks/eval/` now holds suites written against two pinned repositories, `twenty` (Nx: NestJS + GraphQL server, React front, small packages) and `gamma-backend` (Express + Apollo, schema-first GraphQL), declared in each suite as `corpus: { root, commit }`. Before any search, eval checks that each corpus is checked out at that commit with no change to tracked files, no untracked file the indexer would collect, and untracked `.ragignore` files equal to the copies in `benchmarks/corpus/`; otherwise it exits 1 without searching or writing a result file, naming every difference. Result files record per corpus the commit, its indexed projects, chunk count and a fingerprint of the indexed chunks (positions, text, owners, embeddings), plus the number of live cases; `--baseline` warns when a corpus is on one side only, its commit or fingerprint differs, or either run includes live cases. Metrics, pool recall and stage change are also reported per corpus. The former suites on `RAG` and `acme-backend` moved to `benchmarks/eval-live/` (`--suite benchmarks/eval-live`) as smoke tests. Setup: keep the corpus checkouts at their commits and `rag-index --project <p> --reindex` every corpus project. While it runs, eval prints one progress line per case to stderr (`[12/153] T_ID_012 vector 3 → reranked 1 (4.2s)`).
|
|
14
67
|
- **Voyage reranker** — `reranker.provider: voyage` reranks through Voyage AI (`rerank-3` by default, `rerank-3-lite` as the cheaper alternative) with any embedder, including a local Infinity one. Its defaults are its own: `reranker.baseURL` `https://api.voyageai.com/v1`, endpoint `/v1/rerank`. The key comes only from `reranker.apiKey` — never from `embedding.apiKey` — and a Voyage reranker without one stays disabled, with a warning in the log at `logging.level` `warn` or lower; startup is unaffected.
|
|
15
68
|
- **`rag-setup` asks for a reranker for every embedding provider** — `voyage`, `openai-compatible` (base URL, model, optional key stored as `RERANKER_API_KEY`) or `none`, plus `infinity` (same container) for an Infinity embedder; the default follows the embedding provider (`infinity` → `infinity`, `voyage` → `voyage`, `openai-compatible` → `none`). The Voyage reranker key is always asked separately and stored as `VOYAGE_RERANK_API_KEY`, so it never overwrites the embedding key; a blank key offers to disable the reranker. The wizard always writes an explicit `reranker` block.
|
|
16
69
|
- **Current Voyage embedding models** — `rag-setup` suggests `voyage-code-4`, and the model registry and dashboard suggestions list `voyage-code-4`, `voyage-4-large`, `voyage-4` and `voyage-4-lite` (1024 dimensions by default, up to 2048). `voyage-code-3`, `voyage-3` and `voyage-3-lite` are no longer listed but still work — their dimension is probed at index time. The dashboard suggests `rerank-3` / `rerank-3-lite` for `reranker.model` when the saved `reranker.provider` is `voyage`.
|
|
@@ -43,18 +96,18 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
|
43
96
|
|
|
44
97
|
### Added
|
|
45
98
|
|
|
46
|
-
- **`rag-index --project [name]` scopes a run to one project or monorepo unit** — matched exactly against the same names `list_projects` prints; a unit name (e.g. `
|
|
99
|
+
- **`rag-index --project [name]` scopes a run to one project or monorepo unit** — matched exactly against the same names `list_projects` prints; a unit name (e.g. `mu-monorepo`) expands to all of its children. Bypasses the `.ragprojects`/`.ragskip` selector and project pruning entirely, leaving every other indexed project untouched. Composes with `--reindex` to force a full reindex of just that project. Omitting the name opens a one-off interactive single-choice prompt (distinct from the `--select` multi-select checkbox). An unresolved name is reported with the full list of available projects and exits non-zero before any database or embedding-backend work. The post-index project graph (`project-graph.html`) now always reflects the full discovered workspace regardless of `--select`/`--project` scoping.
|
|
47
100
|
- **`search_codebase` gains a per-call `rerank` override** — pass `rerank: false` to skip reranking regardless of `reranker.enabled` (uses `fetchLimit = limit`, no candidate floor), or `rerank: true` to force it on when a reranker is configured (no-op otherwise). A 41-case / 82-query benchmark across both projects' `benchmarks/eval/*.json` suites found `rerank: false` performs same-or-better in a strong majority of identifier/behaviour/concept cases at `threshold=0.6`/`limit=5`; guidance in `McpServer.ts` and `CLAUDE.md` recommends `false` for "what does X do" / "where is X" lookups described without an exact name, and points exact-name lookups at `find_symbol` instead of `search_codebase` (the rerank setting is moot there — `find_symbol` resolved every sampled identifier exactly at rank 1).
|
|
48
101
|
- **`find_symbol` finds NestJS GraphQL operations, resolve fields and types by their GraphQL name** — `@Query`/`@Mutation`/`@Subscription`/`@ResolveField`/`@ObjectType`/`@InputType`/`@InterfaceType` decorators imported from `@nestjs/graphql` get an extra `code_symbol` row named by the GraphQL name (e.g. `adminPendingSubscriptionOrders`, not the method's TypeScript name), with new kinds `graphql_query`/`graphql_mutation`/`graphql_subscription`/`graphql_field`/`graphql_type` and a ` → \`<target>\`` in the output naming the TypeScript declaration. These rows never affect retrieval, chunk ownership or the call graph. Run `rag-index --reindex` to get the new rows for a project already indexed.
|
|
49
102
|
|
|
50
103
|
### Changed (breaking)
|
|
51
104
|
|
|
52
|
-
- **One path form for every MCP tool: workspace-relative** — the form `search_codebase`, `find_symbol` and `get_callers` already print (`my-app/src/index.ts`; for a nested-monorepo project the on-disk path, `
|
|
105
|
+
- **One path form for every MCP tool: workspace-relative** — the form `search_codebase`, `find_symbol` and `get_callers` already print (`my-app/src/index.ts`; for a nested-monorepo project the on-disk path, `mu-monorepo/apps/account/...`). `read_file` takes it instead of an absolute path; `list_files` takes a workspace directory (`.` for the root) and loses its `project` argument; `get_dependencies` / `get_dependents` lose their `project` argument (a passed `project` is ignored). An absolute path is refused with its workspace-relative equivalent, so update prompts or skills that pass absolute paths or `project` to these tools.
|
|
53
106
|
|
|
54
107
|
### Changed
|
|
55
108
|
|
|
56
109
|
- **A path that names nothing is an error, not an empty result** — every path-taking tool answers `Not found:` with the expected form (example `my-app/src` for `list_files`, `my-app/src/index.ts` elsewhere) and up to three `Did you mean:` candidates, `Not indexed:` (with the reason — excluded file type, ignored directory, `.ragignore`, unselected project, or not yet indexed), `Deleted from disk:`, `Not a file:` / `Not a directory:` or `Cannot read:`, all as MCP error responses. A database or discovery failure is an error response, never "not found".
|
|
57
|
-
- **`Did you mean:` resolves nested-monorepo project names** — a path that starts with a project's name instead of its directory (`
|
|
110
|
+
- **`Did you mean:` resolves nested-monorepo project names** — a path that starts with a project's name instead of its directory (`mu-monorepo/account/src/main.ts`) is offered at its on-disk location (`mu-monorepo/apps/account/src/main.ts`) first, when that file exists; `list_files` given a project name is pointed at the project's directory.
|
|
58
111
|
- **Graph tools span projects** — `get_dependencies`, `get_dependents` and `include_graph_context` list edges into and out of other projects; an edge stored under two overlapping projects prints once; import targets outside the workspace (`../…`) are never printed.
|
|
59
112
|
- **`read_file_range` reads unindexed files** — both read tools read any file under the workspace (symlinks resolved before the boundary check; a symlinked file already in the index stays readable). `start_line` past the end and `end_line` before `start_line` are errors; an unreadable file is `Cannot read:`.
|
|
60
113
|
- **`list_projects` prints each project's directory** as a `path:` line, marked `(not on disk)` when it is gone. `list_files` prints a `**Directory:**` header and marks unreadable subdirectories `(unreadable)`.
|
|
@@ -147,7 +200,7 @@ Format based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
|
|
147
200
|
- **`IMonorepoResolver` interface + `NxMonorepoResolver`** — resolver abstraction for monorepo tooling. `NxMonorepoResolver` detects `nx.json`, discovers projects via `workspace.json` / `angular.json` or an `apps/`/`libs/`/`packages/` scan for `project.json` / `package.json`, and resolves the root tsconfig (`tsconfig.base.json`, falling back to `tsconfig.json`).
|
|
148
201
|
- **`MonorepoAwareDiscoveryService`** — implements `IProjectDiscovery` and is wired as the default discovery in the container. Uses the resolver when a monorepo is detected; otherwise delegates to `ProjectDiscoveryService` (flat `package.json` scan). Exposes `analyzeWorkspace()` for full workspace metadata.
|
|
149
202
|
- **`DiscoveredProject` metadata** — discovery now carries optional `packageName` (npm scope + name), `type` (`app` | `lib`), and `rootTsConfigPath` per project.
|
|
150
|
-
- **Nested Nx monorepo under a flat workspace** — when `workspaceRoot` is a flat directory but one of its top-level subdirectories is itself an Nx repo, that repo is discovered as a single selectable unit whose sub-projects (`<repo>/<project>`, e.g. `
|
|
203
|
+
- **Nested Nx monorepo under a flat workspace** — when `workspaceRoot` is a flat directory but one of its top-level subdirectories is itself an Nx repo, that repo is discovered as a single selectable unit whose sub-projects (`<repo>/<project>`, e.g. `mu-monorepo/account`) are expanded and indexed individually. `DiscoveredProject` gains an optional `children` field; the exported `toLeafProjects()` helper expands units into their leaf projects and is shared by `IndexCommand` (indexing loop) and `SearcherService` (name → path resolution). Each sub-project carries the repo's `tsconfig.base.json` as `rootTsConfigPath` for cross-project alias resolution.
|
|
151
204
|
- **Root-level Nx libs via `tsconfig.base.json` paths** — `NxMonorepoResolver` now also discovers projects referenced only through `compilerOptions.paths` aliases (e.g. `contracts/`, `interfaces/` living at the repo root), which the `apps/`/`libs/`/`packages/` scan misses. Only targets that resolve to a real project (with `project.json` / `package.json`) are added; the `tsconfig` reader tolerates JSONC (comments, trailing commas).
|
|
152
205
|
- **Tabbed project graph** — `project-graph.html` and the dashboard's `/graph` now have inner tabs: a **Workspace** tab (flat projects + each nested Nx repo collapsed to one node, double-click to open) and **one tab per Nx repo** showing its sub-projects with real import edges between them. Intra-repo edges come from `SearcherService.getProjectEdges()`, which aggregates `code_dependency` (file → file) into project → project links (most-specific owner wins). `getProjectEdges` is exposed on `ISearcher`.
|
|
153
206
|
|