codebase-brain-mcp 0.1.0__tar.gz → 0.2.0__tar.gz

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codebase-brain-mcp
-Version: 0.1.0
+Version: 0.2.0
 Summary: An MCP server that gives AI agents a cited, searchable memory of a codebase.
 Keywords: mcp,model-context-protocol,rag,code-search,ai-agents,claude,git
 Author: Noman Nazir
@@ -18,6 +18,9 @@ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Requires-Dist: fastembed>=0.9.1
 Requires-Dist: mcp>=2.3.0
 Requires-Dist: numpy>=2.5.4
+Requires-Dist: tree-sitter>=0.25.2,<0.26
+Requires-Dist: tree-sitter-javascript>=0.25
+Requires-Dist: tree-sitter-typescript>=0.23.2
 Requires-Python: >=3.12
 Project-URL: Homepage, https://github.com/Nomannazir/codebase-brain-mcp
 Project-URL: Repository, https://github.com/Nomannazir/codebase-brain-mcp
@@ -73,8 +76,10 @@ A plug-in for any AI assistant that turns a codebase into something it can actua
 
 ## Why it's different (technical)
 
-- **Code-aware chunking.** Python is split by function, class and method using the real syntax tree, Markdown by
-  heading. Each chunk knows its symbol (`TimestampSigner.unsign`) and exact lines, so citations point at real code.
+- **Code-aware chunking.** Python (with `ast`) and JavaScript / TypeScript / JSX / TSX (with tree-sitter) are split
+  by function, class, method, arrow function, interface, type and enum, with their doc comments. Markdown is split
+  by heading. Each chunk knows its symbol (`TimestampSigner.unsign`, `Ky.#retry`) and exact lines, so citations
+  point at real code. Other languages are indexed in overlapping line windows.
 - **Hybrid retrieval.** SQLite FTS5 BM25 (with `camelCase` / `snake_case` identifier splitting) plus local
   embeddings, merged with reciprocal-rank fusion. Exact names and fuzzy questions both work.
 - **Local and free.** Embeddings run on your machine (`fastembed`, ONNX). No API keys, and no code leaves your
@@ -85,7 +90,8 @@ A plug-in for any AI assistant that turns a codebase into something it can actua
   the current lines, and links the pull requests behind each change. So agents can answer *"why is it built this
   way?"*, not just *"what does it do?"*.
 - **Impact analysis.** `impact_of_change` lists everything that uses a function, directly and transitively, plus the
-  tests that exercise it, before an agent changes anything.
+  tests that exercise it, before an agent changes anything. It works across Python, JavaScript and TypeScript files,
+  and follows a constructor to the code that builds its class.
 - **Doc-drift detection.** `find_doc_drift` checks Markdown and reStructuredText docs against the code. It finds
   files, functions, settings and flags that don't exist; examples passing parameters a function doesn't take; imports
   that would fail; and sections last edited long before the code they describe changed. Each finding cites the doc
@@ -209,10 +215,14 @@ because they're supposed to mention removed things.
 ## Evaluation
 
 Does search actually put the right code in front of the model? `codebase-brain eval` answers that with a benchmark.
-The benchmark has 32 questions about [pallets/itsdangerous](https://github.com/pallets/itsdangerous), pinned to
-commit `672971d`, each with the code or docs a good answer must come from. The questions are in
-[`evals/datasets/itsdangerous.toml`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/datasets/itsdangerous.toml). They were written by reading the code
-before any results were measured, and haven't been changed since.
+There are two, each with 32 questions pinned to one commit, and each question lists the code or docs a good answer
+must come from. The questions were written by reading the code before any results were measured, and haven't been
+changed since.
+
+### Python: pallets/itsdangerous
+
+32 questions about [pallets/itsdangerous](https://github.com/pallets/itsdangerous) at commit `672971d`, in
+[`evals/datasets/itsdangerous.toml`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/datasets/itsdangerous.toml).
 
 | Mode | Hit@1 | Hit@3 | Hit@5 | Hit@10 | MRR@10 (95% CI) | nDCG@10 | p50 latency |
 |---|---|---|---|---|---|---|---|
@@ -236,10 +246,40 @@ before any results were measured, and haven't been changed since.
   the top 5. They're listed in [`evals/results/itsdangerous.md`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/results/itsdangerous.md) and are the next
   thing to improve. A larger, held-out question set will come first, so any tuning isn't fitted to these 32.
 
-Reproduce it (the first run downloads the repo and the embedding model):
+### TypeScript: sindresorhus/ky
+
+32 questions about [sindresorhus/ky](https://github.com/sindresorhus/ky) at commit `3541888`, in
+[`evals/datasets/ky.toml`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/datasets/ky.toml).
+The readme is one 2,000-line file, so docs questions must find the right *section*, not just the file.
+
+| Mode | Hit@1 | Hit@3 | Hit@5 | Hit@10 | MRR@10 (95% CI) | nDCG@10 | p50 latency |
+|---|---|---|---|---|---|---|---|
+| **hybrid** (default) | 47% | 66% | 69% | 78% | 0.563 (0.41–0.71) | 0.576 | 6 ms |
+| keyword only (BM25) | 38% | 44% | 53% | 62% | 0.436 (0.29–0.59) | 0.455 | 1 ms |
+| vector only (bge-small) | 47% | **69%** | **75%** | 78% | **0.594** (0.46–0.74) | **0.604** | 5 ms |
+
+| Hit@5 by question type | describes behaviour (14) | about the docs (7) | names the code (11) |
+|---|---|---|---|
+| hybrid | 36% | **100%** | 91% |
+| keyword only | 7% | 86% | 91% |
+| vector only | **43%** | **100%** | **100%** |
+
+**What this shows:**
+- **Naming the code or asking about the docs works well:** 91–100% in the top 5.
+- **Describing behaviour in plain words is the weak spot (36%).** ky has about 25,000 lines of tests next to
+  5,000 lines of source, and the tests describe the same behaviour in plain words, so they outrank the source.
+  Every miss is listed in [`evals/results/ky.md`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/results/ky.md).
+- **Hybrid is not better than vector-only here.** The intervals overlap heavily (hybrid beats vector on 5
+  questions and loses on 9), so neither is proven better. Keyword-only is clearly worst: hybrid beats it on 12
+  questions and loses on none.
+- These results are published as measured. Tuning (for example, ranking source above tests) will be checked
+  against a new held-out set, so it isn't fitted to these questions.
+
+Reproduce either benchmark (the first run downloads the repo and the embedding model):
 
 ```bash
 uv run codebase-brain eval evals/datasets/itsdangerous.toml --out evals/results
+uv run codebase-brain eval evals/datasets/ky.toml --out evals/results
 ```
 
 How it's measured:
@@ -255,7 +295,7 @@ How it's measured:
 
 ```
 source ─► resolve (local dir / shallow https clone) ─► list files (.gitignore-aware, skips binaries and lockfiles)
-       ─► chunk (ast / headings / line windows) ─► embed (fastembed) ─► SQLite: chunks + vectors + FTS5
+       ─► chunk (ast / tree-sitter / headings / windows) ─► embed (fastembed) ─► SQLite: chunks + vectors + FTS5
 query  ─► BM25 top 50 ┐
        ─► vector top 50 ┴─► reciprocal-rank fusion ─► filters ─► cited sources
 ```
@@ -285,9 +325,11 @@ uv run pyright          # types
 - [x] Git history and PR discussions: `explain_history`, `search_history`
 - [x] Symbol references and transitive dependents: `impact_of_change`
 - [x] `find_doc_drift`: Markdown and reStructuredText, signatures, imports, settings, flags, staleness
-- [ ] tree-sitter chunking for more languages
+- [x] JavaScript and TypeScript (tree-sitter): chunking, references, impact analysis
+- [ ] More languages (Go, Rust, Java) with tree-sitter
 - [x] Retrieval evals with confidence intervals (`codebase-brain eval`)
 - [ ] Larger held-out eval set, then retrieval tuning (reranking, query expansion)
 - [x] Streamable HTTP with bearer-token auth, Docker image
-- [ ] Hosted demo, PyPI package, MCP Registry listing
+- [x] PyPI package, MCP Registry listing
+- [ ] Hosted demo
 - [ ] OpenTelemetry tracing

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/README.md

@@ -47,8 +47,10 @@ A plug-in for any AI assistant that turns a codebase into something it can actua
 
 ## Why it's different (technical)
 
-- **Code-aware chunking.** Python is split by function, class and method using the real syntax tree, Markdown by
-  heading. Each chunk knows its symbol (`TimestampSigner.unsign`) and exact lines, so citations point at real code.
+- **Code-aware chunking.** Python (with `ast`) and JavaScript / TypeScript / JSX / TSX (with tree-sitter) are split
+  by function, class, method, arrow function, interface, type and enum, with their doc comments. Markdown is split
+  by heading. Each chunk knows its symbol (`TimestampSigner.unsign`, `Ky.#retry`) and exact lines, so citations
+  point at real code. Other languages are indexed in overlapping line windows.
 - **Hybrid retrieval.** SQLite FTS5 BM25 (with `camelCase` / `snake_case` identifier splitting) plus local
   embeddings, merged with reciprocal-rank fusion. Exact names and fuzzy questions both work.
 - **Local and free.** Embeddings run on your machine (`fastembed`, ONNX). No API keys, and no code leaves your
@@ -59,7 +61,8 @@ A plug-in for any AI assistant that turns a codebase into something it can actua
   the current lines, and links the pull requests behind each change. So agents can answer *"why is it built this
   way?"*, not just *"what does it do?"*.
 - **Impact analysis.** `impact_of_change` lists everything that uses a function, directly and transitively, plus the
-  tests that exercise it, before an agent changes anything.
+  tests that exercise it, before an agent changes anything. It works across Python, JavaScript and TypeScript files,
+  and follows a constructor to the code that builds its class.
 - **Doc-drift detection.** `find_doc_drift` checks Markdown and reStructuredText docs against the code. It finds
   files, functions, settings and flags that don't exist; examples passing parameters a function doesn't take; imports
   that would fail; and sections last edited long before the code they describe changed. Each finding cites the doc
@@ -183,10 +186,14 @@ because they're supposed to mention removed things.
 ## Evaluation
 
 Does search actually put the right code in front of the model? `codebase-brain eval` answers that with a benchmark.
-The benchmark has 32 questions about [pallets/itsdangerous](https://github.com/pallets/itsdangerous), pinned to
-commit `672971d`, each with the code or docs a good answer must come from. The questions are in
-[`evals/datasets/itsdangerous.toml`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/datasets/itsdangerous.toml). They were written by reading the code
-before any results were measured, and haven't been changed since.
+There are two, each with 32 questions pinned to one commit, and each question lists the code or docs a good answer
+must come from. The questions were written by reading the code before any results were measured, and haven't been
+changed since.
+
+### Python: pallets/itsdangerous
+
+32 questions about [pallets/itsdangerous](https://github.com/pallets/itsdangerous) at commit `672971d`, in
+[`evals/datasets/itsdangerous.toml`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/datasets/itsdangerous.toml).
 
 | Mode | Hit@1 | Hit@3 | Hit@5 | Hit@10 | MRR@10 (95% CI) | nDCG@10 | p50 latency |
 |---|---|---|---|---|---|---|---|
@@ -210,10 +217,40 @@ before any results were measured, and haven't been changed since.
   the top 5. They're listed in [`evals/results/itsdangerous.md`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/results/itsdangerous.md) and are the next
   thing to improve. A larger, held-out question set will come first, so any tuning isn't fitted to these 32.
 
-Reproduce it (the first run downloads the repo and the embedding model):
+### TypeScript: sindresorhus/ky
+
+32 questions about [sindresorhus/ky](https://github.com/sindresorhus/ky) at commit `3541888`, in
+[`evals/datasets/ky.toml`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/datasets/ky.toml).
+The readme is one 2,000-line file, so docs questions must find the right *section*, not just the file.
+
+| Mode | Hit@1 | Hit@3 | Hit@5 | Hit@10 | MRR@10 (95% CI) | nDCG@10 | p50 latency |
+|---|---|---|---|---|---|---|---|
+| **hybrid** (default) | 47% | 66% | 69% | 78% | 0.563 (0.41–0.71) | 0.576 | 6 ms |
+| keyword only (BM25) | 38% | 44% | 53% | 62% | 0.436 (0.29–0.59) | 0.455 | 1 ms |
+| vector only (bge-small) | 47% | **69%** | **75%** | 78% | **0.594** (0.46–0.74) | **0.604** | 5 ms |
+
+| Hit@5 by question type | describes behaviour (14) | about the docs (7) | names the code (11) |
+|---|---|---|---|
+| hybrid | 36% | **100%** | 91% |
+| keyword only | 7% | 86% | 91% |
+| vector only | **43%** | **100%** | **100%** |
+
+**What this shows:**
+- **Naming the code or asking about the docs works well:** 91–100% in the top 5.
+- **Describing behaviour in plain words is the weak spot (36%).** ky has about 25,000 lines of tests next to
+  5,000 lines of source, and the tests describe the same behaviour in plain words, so they outrank the source.
+  Every miss is listed in [`evals/results/ky.md`](https://github.com/Nomannazir/codebase-brain-mcp/blob/main/evals/results/ky.md).
+- **Hybrid is not better than vector-only here.** The intervals overlap heavily (hybrid beats vector on 5
+  questions and loses on 9), so neither is proven better. Keyword-only is clearly worst: hybrid beats it on 12
+  questions and loses on none.
+- These results are published as measured. Tuning (for example, ranking source above tests) will be checked
+  against a new held-out set, so it isn't fitted to these questions.
+
+Reproduce either benchmark (the first run downloads the repo and the embedding model):
 
 ```bash
 uv run codebase-brain eval evals/datasets/itsdangerous.toml --out evals/results
+uv run codebase-brain eval evals/datasets/ky.toml --out evals/results
 ```
 
 How it's measured:
@@ -229,7 +266,7 @@ How it's measured:
 
 ```
 source ─► resolve (local dir / shallow https clone) ─► list files (.gitignore-aware, skips binaries and lockfiles)
-       ─► chunk (ast / headings / line windows) ─► embed (fastembed) ─► SQLite: chunks + vectors + FTS5
+       ─► chunk (ast / tree-sitter / headings / windows) ─► embed (fastembed) ─► SQLite: chunks + vectors + FTS5
 query  ─► BM25 top 50 ┐
        ─► vector top 50 ┴─► reciprocal-rank fusion ─► filters ─► cited sources
 ```
@@ -259,9 +296,11 @@ uv run pyright          # types
 - [x] Git history and PR discussions: `explain_history`, `search_history`
 - [x] Symbol references and transitive dependents: `impact_of_change`
 - [x] `find_doc_drift`: Markdown and reStructuredText, signatures, imports, settings, flags, staleness
-- [ ] tree-sitter chunking for more languages
+- [x] JavaScript and TypeScript (tree-sitter): chunking, references, impact analysis
+- [ ] More languages (Go, Rust, Java) with tree-sitter
 - [x] Retrieval evals with confidence intervals (`codebase-brain eval`)
 - [ ] Larger held-out eval set, then retrieval tuning (reranking, query expansion)
 - [x] Streamable HTTP with bearer-token auth, Docker image
-- [ ] Hosted demo, PyPI package, MCP Registry listing
+- [x] PyPI package, MCP Registry listing
+- [ ] Hosted demo
 - [ ] OpenTelemetry tracing

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "codebase-brain-mcp"
-version = "0.1.0"
+version = "0.2.0"
 description = "An MCP server that gives AI agents a cited, searchable memory of a codebase."
 readme = "README.md"
 license = "MIT"
@@ -29,6 +29,9 @@ dependencies = [
     "fastembed>=0.9.1",
     "mcp>=2.3.0",
     "numpy>=2.5.4",
+    "tree-sitter>=0.25.2,<0.26",
+    "tree-sitter-javascript>=0.25",
+    "tree-sitter-typescript>=0.23.2",
 ]
 
 [[project.authors]]

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/pyproject.toml.orig

@@ -1,6 +1,6 @@
 [project]
 name = "codebase-brain-mcp"
-version = "0.1.0"
+version = "0.2.0"
 description = "An MCP server that gives AI agents a cited, searchable memory of a codebase."
 readme = "README.md"
 authors = [
@@ -24,6 +24,9 @@ dependencies = [
     "fastembed>=0.9.1",
     "mcp>=2.3.0",
     "numpy>=2.5.4",
+    "tree-sitter>=0.25.2,<0.26",  # 0.26.0 crashes the process (SIGBUS) while walking big files
+    "tree-sitter-javascript>=0.25",
+    "tree-sitter-typescript>=0.23.2",
 ]
 
 [project.urls]

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/src/codebase_brain_mcp/analysis.py

@@ -13,10 +13,12 @@ from datetime import datetime, timedelta
 from pathlib import Path
 
 from . import history
+from .chunking import DEFINITION_KINDS
 from .sources import SourceError
 from .store import Store, StoredChunk
 
 MAX_PER_LEVEL = 60
+STRUCTURED_SUFFIXES = (".py", ".pyi", ".js", ".jsx", ".mjs", ".cjs", ".ts", ".tsx")
 STALE_AFTER = timedelta(days=180)
 MAX_STALENESS_CHECKS = 40
 _FENCE = re.compile(r"^(`{3,}|~{3,})\s*([\w+-]*)[^\n]*\n(.*?)^\1\s*$", re.MULTILINE | re.DOTALL)
@@ -31,7 +33,7 @@ _PATH = re.compile(r"^\.?/?[\w.-]+(/[\w.-]+)+/?$")
 _ENV = re.compile(r"^[A-Z][A-Z0-9]*(_[A-Z0-9]+)+$")
 _FLAG = re.compile(r"^(--[a-z][a-z0-9-]+)(=\S*)?$")
 _SYMBOL = re.compile(r"^[A-Za-z_]\w*(\.[A-Za-z_]\w*)*(\(\))?$")
-_TEST_PATH = re.compile(r"(^|/)(tests?|testing)/|(^|/)test_[^/]+$|_test\.\w+$")
+_TEST_PATH = re.compile(r"(^|/)(tests?|testing|__tests__)/|(^|/)test_[^/]+$|_test\.\w+$|\.(test|spec)\.[jt]sx?$")
 _COMMON = set(dir(builtins)) | set(keyword.kwlist) | {"true", "false", "null", "none", "self", "cls"}
 _SEVERITY_ORDER = {"high": 0, "medium": 1, "low": 2}
 _CHANGELOG = re.compile(
@@ -57,6 +59,8 @@ def definition_span(matches: list[StoredChunk]) -> tuple[str, int, int, str | No
 
 # ---- impact_of_change -------------------------------------------------------------------------
 
+_CONSTRUCTORS = {"__init__", "__new__", "constructor"}
+
 
 @dataclass(frozen=True)
 class Dependent:
@@ -79,13 +83,21 @@ class ImpactReport:
     notes: list[str] = field(default_factory=list)
 
 
+def _used_as(symbol: str) -> str:
+    """The name other code uses to reach ``symbol``: a constructor is reached through its class."""
+    owner, _, short = symbol.rpartition(".")
+    if short in _CONSTRUCTORS and owner:
+        return owner.rsplit(".", 1)[-1]
+    return short
+
+
 def impact_of_change(store: Store, target: StoredChunk, span: tuple[int, int], depth: int = 2) -> ImpactReport:
     """Everything that uses ``target`` by name, directly and (up to ``depth``) transitively."""
     symbol = target.symbol or ""
     short = symbol.rsplit(".", 1)[-1]
     excluded = {c.id for c in store.find_symbol(symbol, target.path)}
     dependents: list[Dependent] = []
-    frontier = [short]
+    frontier = [_used_as(symbol)]
     for level in range(1, max(1, min(depth, 3)) + 1):
         next_frontier: list[str] = []
         for name in frontier:
@@ -97,8 +109,8 @@ def impact_of_change(store: Store, target: StoredChunk, span: tuple[int, int], d
                 dependents.append(
                     Dependent(chunk=chunk, via=name, level=level, is_test=bool(_TEST_PATH.search(chunk.path)))
                 )
-                if chunk.symbol and chunk.kind in ("function", "method", "class"):
-                    next_frontier.append(chunk.symbol.rsplit(".", 1)[-1])
+                if chunk.symbol and chunk.kind in DEFINITION_KINDS:
+                    next_frontier.append(_used_as(chunk.symbol))
         frontier = list(dict.fromkeys(next_frontier))
         if not frontier:
             break
@@ -114,8 +126,8 @@ def impact_of_change(store: Store, target: StoredChunk, span: tuple[int, int], d
             f"{len(twins)} other definition(s) share the name {short!r}; some dependents may use those "
             "instead. Check the import or receiver before changing anything."
         )
-    if not target.path.endswith((".py", ".pyi")):
-        notes.append("Only Python is analysed structurally; for other languages, also run search for the name.")
+    if not target.path.endswith(STRUCTURED_SUFFIXES):
+        notes.append("Only Python, JS and TS are analysed structurally; for other languages, also search the name.")
     notes.append("Found by name, so dynamic calls (getattr, string dispatch, callbacks) aren't included.")
     return ImpactReport(
         symbol=symbol,

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/src/codebase_brain_mcp/chunking.py

@@ -1,7 +1,8 @@
 """Split files into chunks that follow the code's own structure.
 
-Python is split by function and class (with exact line numbers, from ``ast``), Markdown by heading,
-and everything else into overlapping line windows. Chunks that are still too long are windowed
+Python is split by function and class (with exact line numbers, from ``ast``), JavaScript and
+TypeScript the same way (with tree-sitter), Markdown by heading, and everything else into overlapping
+line windows. Chunks that are still too long are windowed
 again but keep their symbol, so a citation always says which function it came from.
 """
 
@@ -12,11 +13,16 @@ import builtins
 import re
 from dataclasses import dataclass, replace
 
+from . import chunking_js
+
 WINDOW_LINES = 60
 WINDOW_OVERLAP = 10
 MAX_CHUNK_LINES = 120
 _HEADING = re.compile(r"^(#{1,6})\s+(.*?)\s*#*\s*$")
 _IGNORED_NAMES = set(dir(builtins)) | {"self", "cls", "super", "typing", "t", "_"}
+DEFINITION_KINDS = ("function", "method", "class", "interface", "type", "enum")
+"""Chunk kinds that define a named thing other code can use."""
+Signatures = dict[int, tuple[tuple[str, ...], bool]]
 
 
 @dataclass(frozen=True)
@@ -27,11 +33,11 @@ class Chunk:
     end_line: int
     """1-based, inclusive."""
     kind: str
-    """function, class, method, module, section, or code."""
+    """function, class, method, interface, type, enum, module, section, or code."""
     symbol: str | None
     text: str
     refs: tuple[str, ...] = ()
-    """Names this chunk uses: calls, attributes and imports (Python only). Feeds impact_of_change."""
+    """Names this chunk uses: calls, attributes and imports (Python, JS and TS). Feeds impact_of_change."""
     params: tuple[str, ...] | None = None
     """Parameter names when the chunk is a function or method; None otherwise. Feeds find_doc_drift."""
     accepts_any_keyword: bool = False
@@ -52,7 +58,9 @@ def chunk_file(path: str, language: str, text: str) -> list[Chunk]:
         except (SyntaxError, ValueError):
             return _windows(path, lines, 1, len(lines), "code", None)
         chunks = [piece for chunk in _python_chunks(path, lines, tree) for piece in _split_long(chunk, lines)]
-        return _with_python_facts(chunks, tree)
+        return _with_facts(chunks, *_python_facts(tree))
+    if language in ("javascript", "typescript"):
+        return _js_chunks(path, language, lines, text)
     if language == "markdown":
         chunks = _markdown_chunks(path, lines)
     else:
@@ -72,7 +80,22 @@ def _python_chunks(path: str, lines: list[str], tree: ast.Module) -> list[Chunk]
             chunks.extend(_class_chunks(path, lines, node, start, end))
         else:
             chunks.append(_make(path, lines, start, end, "function", node.name))
-    # Whatever sits between definitions (imports, constants, module docstring) is one "module" chunk each.
+    return _with_module_gaps(path, lines, chunks, covered)
+
+
+def _js_chunks(path: str, language: str, lines: list[str], text: str) -> list[Chunk]:
+    found = chunking_js.outline(path, language, text, MAX_CHUNK_LINES)
+    if found is None:
+        return _windows(path, lines, 1, len(lines), "code", None)
+    chunks = [_make(path, lines, d.start_line, d.end_line, d.kind, d.symbol) for d in found.definitions]
+    chunks = _with_module_gaps(path, lines, chunks, found.covered)
+    pieces = [piece for chunk in chunks for piece in _split_long(chunk, lines)]
+    return _with_facts(pieces, found.uses, found.signatures, chunking_js.IGNORED_NAMES)
+
+
+def _with_module_gaps(path: str, lines: list[str], chunks: list[Chunk], covered: list[tuple[int, int]]) -> list[Chunk]:
+    """Whatever sits between definitions (imports, constants, module docstring) is one "module" chunk each."""
+    result = list(chunks)
     cursor = 1
     for start, end in [*sorted(covered), (len(lines) + 1, len(lines) + 1)]:
         gap_start, gap_end = cursor, start - 1
@@ -81,15 +104,15 @@ def _python_chunks(path: str, lines: list[str], tree: ast.Module) -> list[Chunk]
         while gap_end >= gap_start and not lines[gap_end - 1].strip():
             gap_end -= 1
         if gap_start <= gap_end:
-            chunks.append(_make(path, lines, gap_start, gap_end, "module", None))
+            result.append(_make(path, lines, gap_start, gap_end, "module", None))
         cursor = max(cursor, end + 1)
-    return sorted(chunks, key=lambda c: c.start_line)
+    return sorted(result, key=lambda c: c.start_line)
 
 
-def _with_python_facts(chunks: list[Chunk], tree: ast.Module) -> list[Chunk]:
-    """Attach the names each chunk uses, and the parameters of each function chunk."""
+def _python_facts(tree: ast.Module) -> tuple[list[tuple[int, str]], Signatures, set[str]]:
+    """The names used on each line, and each function's parameters keyed by its first line."""
     uses: list[tuple[int, str]] = []
-    signatures: dict[int, tuple[tuple[str, ...], bool]] = {}
+    signatures: Signatures = {}
     for node in ast.walk(tree):
         if isinstance(node, ast.Name) and isinstance(node.ctx, ast.Load):
             uses.append((node.lineno, node.id))
@@ -105,6 +128,13 @@ def _with_python_facts(chunks: list[Chunk], tree: ast.Module) -> list[Chunk]:
             args = node.args
             names = [a.arg for a in (*args.posonlyargs, *args.args, *args.kwonlyargs)]
             signatures[_span(node)[0]] = (tuple(n for n in names if n not in ("self", "cls")), args.kwarg is not None)
+    return uses, signatures, _IGNORED_NAMES
+
+
+def _with_facts(
+    chunks: list[Chunk], uses: list[tuple[int, str]], signatures: Signatures, ignored: set[str]
+) -> list[Chunk]:
+    """Attach the names each chunk uses, and the parameters of each function chunk."""
     result = []
     for chunk in chunks:
         own = chunk.symbol.rsplit(".", 1)[-1] if chunk.symbol else None
@@ -112,10 +142,7 @@ def _with_python_facts(chunks: list[Chunk], tree: ast.Module) -> list[Chunk]:
             {
                 name
                 for line, name in uses
-                if chunk.start_line <= line <= chunk.end_line
-                and name not in _IGNORED_NAMES
-                and name != own
-                and len(name) > 2
+                if chunk.start_line <= line <= chunk.end_line and name not in ignored and name != own and len(name) > 1
             }
         )
         signature = signatures.get(chunk.start_line) if chunk.kind in ("function", "method") else None
@@ -151,7 +178,8 @@ def _span(node: ast.FunctionDef | ast.AsyncFunctionDef | ast.ClassDef) -> tuple[
 
 def _markdown_chunks(path: str, lines: list[str]) -> list[Chunk]:
     chunks: list[Chunk] = []
-    trail: list[str] = []
+    trail: list[tuple[int, str]] = []
+    """(level, heading) of each open heading; a new heading closes every one at its level or deeper."""
     section_start, section_symbol = 1, None
     in_fence = False
     for number, line in enumerate(lines, start=1):
@@ -163,8 +191,8 @@ def _markdown_chunks(path: str, lines: list[str]) -> list[Chunk]:
         if number > section_start:
             chunks.append(_make(path, lines, section_start, number - 1, "section", section_symbol))
         level = len(match.group(1))
-        trail = [*trail[: level - 1], match.group(2)]
-        section_start, section_symbol = number, " > ".join(trail)
+        trail = [*(open_ for open_ in trail if open_[0] < level), (level, match.group(2))]
+        section_start, section_symbol = number, " > ".join(heading for _, heading in trail)
     chunks.append(_make(path, lines, section_start, len(lines), "section", section_symbol))
     return [c for c in chunks if c.text.strip()]
 

codebase_brain_mcp-0.2.0/src/codebase_brain_mcp/chunking_js.py

@@ -0,0 +1,255 @@
+"""Find the definitions in JavaScript and TypeScript files with tree-sitter.
+
+Returns plain line spans plus the facts ``chunking`` attaches to chunks: which names each line uses
+(calls, properties, types, imports) and each function's parameters. Comment blocks directly above a
+definition belong to it, so a JSDoc comment is searched together with its function.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from functools import cache
+from typing import Literal
+
+import tree_sitter_javascript
+import tree_sitter_typescript
+from tree_sitter import Language, Node, Parser
+
+Grammar = Literal["javascript", "typescript", "tsx"]
+
+FUNCTION_VALUES = {"arrow_function", "function_expression", "function", "generator_function"}
+CLASS_VALUES = {"class", "class_expression"}
+CLASS_DECLARATIONS = {"class_declaration", "abstract_class_declaration"}
+TYPE_DECLARATIONS = {"interface_declaration": "interface", "type_alias_declaration": "type", "enum_declaration": "enum"}
+FUNCTION_DECLARATIONS = {"function_declaration", "generator_function_declaration", "function_signature"}
+METHOD_NODES = {"method_definition", "public_field_definition", "field_definition", "abstract_method_signature"}
+NAME_NODES = {
+    "identifier", "type_identifier", "property_identifier", "private_property_identifier",
+    "shorthand_property_identifier",
+}  # fmt: skip
+DECLARING_PARENTS = {
+    *FUNCTION_DECLARATIONS, *CLASS_DECLARATIONS, *TYPE_DECLARATIONS, *METHOD_NODES,
+    "variable_declarator", "formal_parameters", "required_parameter", "optional_parameter", "property_signature",
+}  # fmt: skip
+"""A name directly under one of these (as its name or parameter) is being declared, not used."""
+IGNORED_NAMES = {
+    "Array", "Boolean", "Date", "Error", "JSON", "Map", "Math", "Number", "Object", "Promise", "RegExp", "Set",
+    "String", "Symbol", "console", "exports", "length", "module", "require", "undefined", "window", "document",
+}  # fmt: skip
+
+
+@dataclass(frozen=True)
+class Definition:
+    start_line: int
+    """1-based, inclusive."""
+    end_line: int
+    """1-based, inclusive."""
+    kind: str
+    symbol: str
+
+
+@dataclass(frozen=True)
+class Outline:
+    definitions: list[Definition]
+    covered: list[tuple[int, int]]
+    """Line spans owned by top-level definitions; the rest of the file becomes module chunks."""
+    uses: list[tuple[int, str]]
+    signatures: dict[int, tuple[tuple[str, ...], bool]]
+    """Definition start line -> (parameter names, takes a rest parameter)."""
+
+
+def grammar_for(path: str, language: str) -> Grammar:
+    if language == "typescript":
+        return "tsx" if path.endswith(".tsx") else "typescript"
+    return "javascript"  # this grammar also understands JSX
+
+
+@cache
+def _parser(grammar: Grammar) -> Parser:
+    if grammar == "javascript":
+        handle = tree_sitter_javascript.language()
+    elif grammar == "tsx":
+        handle = tree_sitter_typescript.language_tsx()
+    else:
+        handle = tree_sitter_typescript.language_typescript()
+    return Parser(Language(handle))
+
+
+def outline(path: str, language: str, text: str, max_class_lines: int) -> Outline | None:
+    """The file's definitions, or None when it doesn't parse (the caller then falls back to windows)."""
+    tree = _parser(grammar_for(path, language)).parse(text.encode())
+    root = tree.root_node
+    if root.has_error:
+        return None
+    collector = _Collector(max_class_lines)
+    for statement in root.named_children:
+        collector.top_level(statement)
+    return Outline(
+        definitions=collector.definitions,
+        covered=collector.covered,
+        uses=_uses(root),
+        signatures=collector.signatures,
+    )
+
+
+class _Collector:
+    def __init__(self, max_class_lines: int) -> None:
+        self.max_class_lines = max_class_lines
+        self.definitions: list[Definition] = []
+        self.covered: list[tuple[int, int]] = []
+        self.signatures: dict[int, tuple[tuple[str, ...], bool]] = {}
+
+    def top_level(self, statement: Node) -> None:
+        found = _declared(statement)
+        if found is None:
+            return
+        kind, name, node = found
+        start, end = _with_leading_comments(statement), statement.end_point.row + 1
+        self.covered.append((start, end))
+        if kind == "class":
+            self._class(node, name, start, end)
+            return
+        self.definitions.append(Definition(start, end, kind, name))
+        if kind == "function":
+            self.signatures[start] = _parameters(node)
+
+    def _class(self, node: Node, name: str, start: int, end: int) -> None:
+        body = node.child_by_field_name("body")
+        methods = [m for m in body.named_children if m.type in METHOD_NODES and _is_function(m)] if body else []
+        if end - start + 1 <= self.max_class_lines or not methods:
+            self.definitions.append(Definition(start, end, "class", name))
+            return
+        # A big class: its header (up to the first method) plus one chunk per method, like Python.
+        first = _with_leading_comments(methods[0])
+        self.definitions.append(Definition(start, max(start, first - 1), "class", name))
+        for method in methods:
+            m_start = _with_leading_comments(method)
+            symbol = f"{name}.{_text(method.child_by_field_name('name')) or 'anonymous'}"
+            self.definitions.append(Definition(m_start, method.end_point.row + 1, "method", symbol))
+            self.signatures[m_start] = _parameters(_function_of(method))
+
+
+def _declared(statement: Node) -> tuple[str, str, Node] | None:
+    """(kind, name, node) when a top-level statement defines one named thing."""
+    if statement.type == "export_statement":
+        inner = statement.child_by_field_name("declaration") or statement.child_by_field_name("value")
+        if inner is None:
+            return None
+        if inner.type in FUNCTION_VALUES:
+            return "function", "default", inner  # export default function () {}
+        if inner.type in CLASS_VALUES:
+            return "class", "default", inner
+        statement = inner
+    kind = statement.type
+    name = _text(statement.child_by_field_name("name"))
+    if kind in FUNCTION_DECLARATIONS and name:
+        return "function", name, statement
+    if kind in CLASS_DECLARATIONS and name:
+        return "class", name, statement
+    if kind in TYPE_DECLARATIONS and name:
+        return TYPE_DECLARATIONS[kind], name, statement
+    if kind in ("lexical_declaration", "variable_declaration"):
+        return _bound_value(statement)
+    if kind == "expression_statement":
+        return _assigned_value(statement)
+    return None
+
+
+def _bound_value(statement: Node) -> tuple[str, str, Node] | None:
+    """``const name = () => ...`` or ``const Name = class {...}``, when it is the only binding."""
+    declarators = [d for d in statement.named_children if d.type == "variable_declarator"]
+    if len(declarators) != 1:
+        return None
+    return _named_value(declarators[0].child_by_field_name("name"), declarators[0].child_by_field_name("value"))
+
+
+def _assigned_value(statement: Node) -> tuple[str, str, Node] | None:
+    """CommonJS and prototype style: ``module.exports.name = function () {...}``."""
+    expression = statement.named_children[0] if statement.named_children else None
+    if expression is None or expression.type != "assignment_expression":
+        return None
+    left = expression.child_by_field_name("left")
+    if left is not None and left.type == "member_expression":
+        left = left.child_by_field_name("property")
+    return _named_value(left, expression.child_by_field_name("right"))
+
+
+def _named_value(name_node: Node | None, value: Node | None) -> tuple[str, str, Node] | None:
+    name = _text(name_node)
+    if not name or value is None or name_node is None or name_node.type not in NAME_NODES:
+        return None
+    if value.type in FUNCTION_VALUES:
+        return "function", name, value
+    if value.type in CLASS_VALUES:
+        return "class", name, value
+    return None
+
+
+def _is_function(member: Node) -> bool:
+    return member.type in ("method_definition", "abstract_method_signature") or _function_of(member) is not member
+
+
+def _function_of(member: Node) -> Node:
+    """The function node holding a class member's parameters (a field may hold an arrow function)."""
+    value = member.child_by_field_name("value")
+    return value if value is not None and value.type in FUNCTION_VALUES else member
+
+
+def _parameters(function: Node) -> tuple[tuple[str, ...], bool]:
+    single = function.child_by_field_name("parameter")  # x => x
+    if single is not None:
+        return (_text(single),), False
+    params = function.child_by_field_name("parameters")
+    names: list[str] = []
+    takes_rest = False
+    for param in params.named_children if params else []:
+        pattern = param.child_by_field_name("pattern") or param
+        if pattern.type == "assignment_pattern":
+            pattern = pattern.child_by_field_name("left") or pattern
+        if pattern.type == "rest_pattern":
+            takes_rest = True
+            pattern = pattern.named_children[0] if pattern.named_children else pattern
+        if pattern.type == "identifier":
+            names.append(_text(pattern))
+    return tuple(names), takes_rest
+
+
+def _with_leading_comments(node: Node) -> int:
+    """The 1-based start line, moved up over comments that sit directly above the node."""
+    start = node.start_point.row
+    previous = node.prev_named_sibling
+    while previous is not None and previous.type == "comment" and previous.end_point.row >= start - 1:
+        start = previous.start_point.row
+        previous = previous.prev_named_sibling
+    return start + 1
+
+
+def _uses(root: Node) -> list[tuple[int, str]]:
+    """Every name the file mentions, by line: identifiers, properties, types and imported names.
+
+    Walks with a cursor, so a big file doesn't create a Python object for every node at once.
+    """
+    uses: list[tuple[int, str]] = []
+    cursor = root.walk()
+    while True:
+        node = cursor.node
+        if node is not None and node.type in NAME_NODES and not _is_declared(node, cursor.field_name):
+            name = _text(node)
+            if name not in IGNORED_NAMES:
+                uses.append((node.start_point.row + 1, name))
+        if cursor.goto_first_child():
+            continue
+        while not cursor.goto_next_sibling():
+            if not cursor.goto_parent():
+                return uses
+
+
+def _is_declared(node: Node, field: str | None) -> bool:
+    parent = node.parent
+    if parent is None or parent.type not in DECLARING_PARENTS:
+        return False
+    return field in ("name", "pattern") or parent.type == "formal_parameters"
+
+
+def _text(node: Node | None) -> str:
+    return node.text.decode(errors="replace") if node is not None and node.text is not None else ""

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/src/codebase_brain_mcp/evals.py

@@ -19,6 +19,7 @@ from pathlib import Path
 
 from . import sources
 from .brain import SEARCH_MODES, Brain, SearchMode
+from .chunking import DEFINITION_KINDS
 from .store import StoredChunk
 
 CUTOFFS = (1, 3, 5, 10)
@@ -42,7 +43,9 @@ class Target:
             return False
         if self.symbol is None:
             return True
-        return chunk.symbol == self.symbol or (chunk.symbol or "").startswith(f"{self.symbol}.")
+        # A method belongs to its class ("Signer.sign"), a subsection to its section ("API > retry").
+        symbol = chunk.symbol or ""
+        return symbol == self.symbol or symbol.startswith((f"{self.symbol}.", f"{self.symbol} > "))
 
 
 @dataclass(frozen=True)
@@ -151,7 +154,7 @@ def validate_targets(brain: Brain, repo: str, dataset: Dataset) -> None:
     """Fail loudly if the dataset names files or symbols that aren't in the index (dataset rot)."""
     store = brain.store_for(repo)
     files = store.file_paths()
-    symbols = set(store.symbols())
+    symbols = set(store.symbols((*DEFINITION_KINDS, "section")))
     missing = []
     for query in dataset.queries:
         for target in query.targets:

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/src/codebase_brain_mcp/server.py

@@ -187,7 +187,7 @@ def build_server(brain: Brain | None = None) -> MCPServer:
         query: str, repo: str | None = None, k: int = 8, path_prefix: str | None = None, kinds: list[str] | None = None
     ) -> SearchResult:
         """Hybrid search (BM25 keywords + embeddings, fused by rank). Filter by `path_prefix`
-        (e.g. "src/auth/") or `kinds` (function, class, method, module, section, code)."""
+        (e.g. "src/auth/") or `kinds` (function, class, method, interface, type, enum, module, section, code)."""
         hits = await _search(brain, query, repo, k, path_prefix, kinds)
         return SearchResult(repo=brain.resolve_repo(repo), query=query, results=[_source(h) for h in hits])
 
@@ -232,7 +232,7 @@ def build_server(brain: Brain | None = None) -> MCPServer:
         symbol: str, repo: str | None = None, path: str | None = None, depth: int = 2
     ) -> ImpactResult:
         """What breaks if `symbol` changes: code that uses it directly, code that uses that code
-        (up to `depth`, max 3), and the tests involved. Python is analysed structurally."""
+        (up to `depth`, max 3), and the tests involved. Python, JavaScript and TypeScript are analysed structurally."""
         report = await _call(lambda: brain.impact_of_change(repo, symbol, path, depth))
         return _impact(report)
 

{codebase_brain_mcp-0.1.0 → codebase_brain_mcp-0.2.0}/src/codebase_brain_mcp/store.py

@@ -6,12 +6,13 @@ import json
 import re
 import sqlite3
 import threading
+from collections.abc import Sequence
 from dataclasses import dataclass
 from pathlib import Path
 
 import numpy as np
 
-from .chunking import Chunk
+from .chunking import DEFINITION_KINDS, Chunk
 from .history import Commit, PullRequest
 
 SCHEMA = """
@@ -46,7 +47,7 @@ CREATE TABLE IF NOT EXISTS commits (
 CREATE VIRTUAL TABLE IF NOT EXISTS commits_fts USING fts5 (subject, body, sha UNINDEXED, tokenize = 'porter unicode61');
 CREATE TABLE IF NOT EXISTS pull_requests (number INTEGER PRIMARY KEY, data TEXT NOT NULL);
 """
-SCHEMA_VERSION = "3"
+SCHEMA_VERSION = "4"
 """Bump when the index layout changes; older indexes are rebuilt automatically."""
 _WORD = re.compile(r"[A-Za-z0-9_]+")
 _CAMEL = re.compile(r"[A-Z]+(?![a-z])|[A-Z]?[a-z]+|\d+")
@@ -221,19 +222,22 @@ class Store:
         )
         return [StoredChunk(**dict(row)) for row in rows]
 
-    def symbols(self) -> list[tuple[str, str]]:
-        """Every defined (symbol, path) pair."""
+    def symbols(self, kinds: Sequence[str] = DEFINITION_KINDS) -> list[tuple[str, str]]:
+        """Every (symbol, path) pair of the given chunk kinds; by default, everything code defines."""
+        marks = ", ".join("?" * len(kinds))
         rows = self._db.execute(
-            "SELECT DISTINCT symbol, path FROM chunks WHERE symbol IS NOT NULL"
-            " AND kind IN ('function', 'class', 'method')"
+            f"SELECT DISTINCT symbol, path FROM chunks WHERE symbol IS NOT NULL AND kind IN ({marks})",  # noqa: S608 (only "?" placeholders)
+            tuple(kinds),
         )
         return [(row["symbol"], row["path"]) for row in rows]
 
     def signatures(self, short_name: str) -> list[tuple[str, str, list[str], bool]]:
-        """(symbol, path, params, accepts_any_keyword) for functions named ``short_name``."""
+        """(symbol, path, params, accepts_any_keyword) for Python functions named ``short_name``.
+
+        Python only, because the docs' code examples that drift checks against them are Python."""
         rows = self._db.execute(
-            "SELECT symbol, path, params, accepts_any_keyword FROM chunks WHERE params IS NOT NULL"
-            " AND (symbol = ? OR symbol GLOB ?)",
+            "SELECT c.symbol, c.path, c.params, c.accepts_any_keyword FROM chunks c JOIN files f ON f.path = c.path"
+            " WHERE c.params IS NOT NULL AND f.language = 'python' AND (c.symbol = ? OR c.symbol GLOB ?)",
             (short_name, f"*.{_glob_literal(short_name)}"),
         )
         return [(r["symbol"], r["path"], json.loads(r["params"]), bool(r["accepts_any_keyword"])) for r in rows]