codecompass-mcp 5.0.0__tar.gz → 5.2.0__tar.gz

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codecompass-mcp
-Version: 5.0.0
+Version: 5.2.0
 Summary: Structural code context for AI coding agents — a local code knowledge graph for blast radius, impact, deps, dead code, and flow tracing
 Author: CodeCompass contributors
 License: MIT
@@ -32,13 +32,16 @@ Requires-Dist: watchdog>=4.0.0
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0.0; extra == "dev"
 Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
+Provides-Extra: search
+Requires-Dist: lancedb>=0.13.0; extra == "search"
+Requires-Dist: fastembed>=0.3.0; extra == "search"
 Dynamic: license-file
 
 # CodeCompass
 
-A local code knowledge graph that gives AI agents a map of your codebase — so they navigate by structure instead of grepping blind, and know what's connected before they edit.
+A local code knowledge graph that gives AI agents a map of your codebase — so they navigate by structure instead of grepping blind, and know what's connected before they edit. It learns as agents use it: parser misses they record are preserved across re-indexes, and an optional local vector index adds semantic search over everything in the graph.
 
-No database. No cloud. One JSON file per repo. Python, JavaScript/TypeScript, PHP, HTML/CSS.
+No cloud. No API keys for core queries. One JSON graph per repo, plus an optional LanceDB vector index. Python, JavaScript/TypeScript, PHP, HTML/CSS.
 
 ---
 
@@ -68,6 +71,7 @@ The graph turns navigation into a cheap, deterministic loop:
    | You have… | Use |
    |---|---|
    | a concept, name, or pattern | `grep` (regex over graph entities) |
+   | an idea, not a name ("where does caching go?") | `search` (semantic vector search) |
    | the full layout | `tree` |
 
 2. **Trace** — a relationship around a known symbol/file:
@@ -137,6 +141,21 @@ no agent-facing query CLI. `grep`, `impact`, `blast_radius`, `deps`, `flow`,
 `flow_summary`, `dead_code`, `tree` and friends are all MCP tools; pass
 `hops` for traversal depth (start at 1 and follow the one path you need).
 
+### Semantic search
+
+`grep` finds symbols when you know the name; `search` finds them when you only
+have the idea. It embeds every entity's name/kind/file/description into a local
+LanceDB index (`.codecompass/vectors.lance`) using fastembed's BGE-small model
+— ONNX, CPU-only, no API keys. Opt in with:
+
+```bash
+pip install 'codecompass-mcp[search]'
+```
+
+The index follows the graph's lifecycle: wiped and rebuilt at the end of every
+`ingest`, so parser nodes *and* agent-recorded ones are searchable. Without the
+extra, ingest simply skips the vector step.
+
 ### Enrichment (agent-in-the-loop)
 
 Via the MCP tools: `enrich` stages entities for an agent swarm (one-line
@@ -161,6 +180,7 @@ with use. Ambiguous call targets are skipped, never guessed.
 | Tool | Returns |
 |---|---|
 | `grep(pattern, field, ignore_case)` | Regex search over graph entities |
+| `search(query, limit)` | Semantic vector search over entity names/kinds/files/descriptions |
 | `impact(symbol, hops)` | Callers/importers, disambiguated, with `resolved` + `line` |
 | `blast_radius(target, hops)` | Files reachable from a file or symbol |
 | `batch_impact(targets, hops)` | Union of blast radii for a multi-file change |
@@ -226,25 +246,31 @@ Source files
    ▼  enricher            agent-in-the-loop: enrich batches (descriptions +
                           missing calls) and opportunistic add_entity/add_call
                           writes — agent_inferred, preserved across re-ingest
+   ▼  vector_store        optional: entity embeddings in vectors.lance (LanceDB
+                          + fastembed), wiped & rebuilt on every ingest
 ```
 
-Everything runs locally, in-process. No network, no database, no API keys.
+Everything runs locally, in-process. Core queries need no network, no database,
+no API keys; semantic search adds a local vector DB and a one-time model
+download.
 
 Inside each indexed project:
 
 ```
 your-project/
-├── .codecompass/graph.json   the code knowledge graph (auto-generated)
-└── AGENTS.md                 discovery guide for agents (auto-updated)
+├── .codecompass/graph.json      the code knowledge graph (auto-generated)
+├── .codecompass/vectors.lance/  semantic search index (optional, rebuilt on ingest)
+└── AGENTS.md                    discovery guide for agents (auto-updated)
 ```
 
 ---
 
 ## Limitations
 
-- **Structure first, semantics optional** — the parser knows what calls what,
-  not what it means. `enrich` closes that gap with agent-written descriptions
-  and missed edges, marked `agent_inferred`.
+- **Structure first, semantics layered on** — the parser knows what calls what,
+  not what it means. Two optional layers close that gap: `enrich` (agent-written
+  descriptions and missed edges, marked `agent_inferred`) and `search`
+  (semantic vector search over those descriptions and names).
 - **Static analysis** — dynamic dispatch, reflection, and string-based invocation
   can't be fully resolved. `impact` surfaces those flagged `resolved: false`, and
   `dead_code` results are always candidates to verify.

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/README.md

@@ -1,8 +1,8 @@
 # CodeCompass
 
-A local code knowledge graph that gives AI agents a map of your codebase — so they navigate by structure instead of grepping blind, and know what's connected before they edit.
+A local code knowledge graph that gives AI agents a map of your codebase — so they navigate by structure instead of grepping blind, and know what's connected before they edit. It learns as agents use it: parser misses they record are preserved across re-indexes, and an optional local vector index adds semantic search over everything in the graph.
 
-No database. No cloud. One JSON file per repo. Python, JavaScript/TypeScript, PHP, HTML/CSS.
+No cloud. No API keys for core queries. One JSON graph per repo, plus an optional LanceDB vector index. Python, JavaScript/TypeScript, PHP, HTML/CSS.
 
 ---
 
@@ -32,6 +32,7 @@ The graph turns navigation into a cheap, deterministic loop:
    | You have… | Use |
    |---|---|
    | a concept, name, or pattern | `grep` (regex over graph entities) |
+   | an idea, not a name ("where does caching go?") | `search` (semantic vector search) |
    | the full layout | `tree` |
 
 2. **Trace** — a relationship around a known symbol/file:
@@ -101,6 +102,21 @@ no agent-facing query CLI. `grep`, `impact`, `blast_radius`, `deps`, `flow`,
 `flow_summary`, `dead_code`, `tree` and friends are all MCP tools; pass
 `hops` for traversal depth (start at 1 and follow the one path you need).
 
+### Semantic search
+
+`grep` finds symbols when you know the name; `search` finds them when you only
+have the idea. It embeds every entity's name/kind/file/description into a local
+LanceDB index (`.codecompass/vectors.lance`) using fastembed's BGE-small model
+— ONNX, CPU-only, no API keys. Opt in with:
+
+```bash
+pip install 'codecompass-mcp[search]'
+```
+
+The index follows the graph's lifecycle: wiped and rebuilt at the end of every
+`ingest`, so parser nodes *and* agent-recorded ones are searchable. Without the
+extra, ingest simply skips the vector step.
+
 ### Enrichment (agent-in-the-loop)
 
 Via the MCP tools: `enrich` stages entities for an agent swarm (one-line
@@ -125,6 +141,7 @@ with use. Ambiguous call targets are skipped, never guessed.
 | Tool | Returns |
 |---|---|
 | `grep(pattern, field, ignore_case)` | Regex search over graph entities |
+| `search(query, limit)` | Semantic vector search over entity names/kinds/files/descriptions |
 | `impact(symbol, hops)` | Callers/importers, disambiguated, with `resolved` + `line` |
 | `blast_radius(target, hops)` | Files reachable from a file or symbol |
 | `batch_impact(targets, hops)` | Union of blast radii for a multi-file change |
@@ -190,25 +207,31 @@ Source files
    ▼  enricher            agent-in-the-loop: enrich batches (descriptions +
                           missing calls) and opportunistic add_entity/add_call
                           writes — agent_inferred, preserved across re-ingest
+   ▼  vector_store        optional: entity embeddings in vectors.lance (LanceDB
+                          + fastembed), wiped & rebuilt on every ingest
 ```
 
-Everything runs locally, in-process. No network, no database, no API keys.
+Everything runs locally, in-process. Core queries need no network, no database,
+no API keys; semantic search adds a local vector DB and a one-time model
+download.
 
 Inside each indexed project:
 
 ```
 your-project/
-├── .codecompass/graph.json   the code knowledge graph (auto-generated)
-└── AGENTS.md                 discovery guide for agents (auto-updated)
+├── .codecompass/graph.json      the code knowledge graph (auto-generated)
+├── .codecompass/vectors.lance/  semantic search index (optional, rebuilt on ingest)
+└── AGENTS.md                    discovery guide for agents (auto-updated)
 ```
 
 ---
 
 ## Limitations
 
-- **Structure first, semantics optional** — the parser knows what calls what,
-  not what it means. `enrich` closes that gap with agent-written descriptions
-  and missed edges, marked `agent_inferred`.
+- **Structure first, semantics layered on** — the parser knows what calls what,
+  not what it means. Two optional layers close that gap: `enrich` (agent-written
+  descriptions and missed edges, marked `agent_inferred`) and `search`
+  (semantic vector search over those descriptions and names).
 - **Static analysis** — dynamic dispatch, reflection, and string-based invocation
   can't be fully resolved. `impact` surfaces those flagged `resolved: false`, and
   `dead_code` results are always candidates to verify.

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/codecompass_mcp.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: codecompass-mcp
-Version: 5.0.0
+Version: 5.2.0
 Summary: Structural code context for AI coding agents — a local code knowledge graph for blast radius, impact, deps, dead code, and flow tracing
 Author: CodeCompass contributors
 License: MIT
@@ -32,13 +32,16 @@ Requires-Dist: watchdog>=4.0.0
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0.0; extra == "dev"
 Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
+Provides-Extra: search
+Requires-Dist: lancedb>=0.13.0; extra == "search"
+Requires-Dist: fastembed>=0.3.0; extra == "search"
 Dynamic: license-file
 
 # CodeCompass
 
-A local code knowledge graph that gives AI agents a map of your codebase — so they navigate by structure instead of grepping blind, and know what's connected before they edit.
+A local code knowledge graph that gives AI agents a map of your codebase — so they navigate by structure instead of grepping blind, and know what's connected before they edit. It learns as agents use it: parser misses they record are preserved across re-indexes, and an optional local vector index adds semantic search over everything in the graph.
 
-No database. No cloud. One JSON file per repo. Python, JavaScript/TypeScript, PHP, HTML/CSS.
+No cloud. No API keys for core queries. One JSON graph per repo, plus an optional LanceDB vector index. Python, JavaScript/TypeScript, PHP, HTML/CSS.
 
 ---
 
@@ -68,6 +71,7 @@ The graph turns navigation into a cheap, deterministic loop:
    | You have… | Use |
    |---|---|
    | a concept, name, or pattern | `grep` (regex over graph entities) |
+   | an idea, not a name ("where does caching go?") | `search` (semantic vector search) |
    | the full layout | `tree` |
 
 2. **Trace** — a relationship around a known symbol/file:
@@ -137,6 +141,21 @@ no agent-facing query CLI. `grep`, `impact`, `blast_radius`, `deps`, `flow`,
 `flow_summary`, `dead_code`, `tree` and friends are all MCP tools; pass
 `hops` for traversal depth (start at 1 and follow the one path you need).
 
+### Semantic search
+
+`grep` finds symbols when you know the name; `search` finds them when you only
+have the idea. It embeds every entity's name/kind/file/description into a local
+LanceDB index (`.codecompass/vectors.lance`) using fastembed's BGE-small model
+— ONNX, CPU-only, no API keys. Opt in with:
+
+```bash
+pip install 'codecompass-mcp[search]'
+```
+
+The index follows the graph's lifecycle: wiped and rebuilt at the end of every
+`ingest`, so parser nodes *and* agent-recorded ones are searchable. Without the
+extra, ingest simply skips the vector step.
+
 ### Enrichment (agent-in-the-loop)
 
 Via the MCP tools: `enrich` stages entities for an agent swarm (one-line
@@ -161,6 +180,7 @@ with use. Ambiguous call targets are skipped, never guessed.
 | Tool | Returns |
 |---|---|
 | `grep(pattern, field, ignore_case)` | Regex search over graph entities |
+| `search(query, limit)` | Semantic vector search over entity names/kinds/files/descriptions |
 | `impact(symbol, hops)` | Callers/importers, disambiguated, with `resolved` + `line` |
 | `blast_radius(target, hops)` | Files reachable from a file or symbol |
 | `batch_impact(targets, hops)` | Union of blast radii for a multi-file change |
@@ -226,25 +246,31 @@ Source files
    ▼  enricher            agent-in-the-loop: enrich batches (descriptions +
                           missing calls) and opportunistic add_entity/add_call
                           writes — agent_inferred, preserved across re-ingest
+   ▼  vector_store        optional: entity embeddings in vectors.lance (LanceDB
+                          + fastembed), wiped & rebuilt on every ingest
 ```
 
-Everything runs locally, in-process. No network, no database, no API keys.
+Everything runs locally, in-process. Core queries need no network, no database,
+no API keys; semantic search adds a local vector DB and a one-time model
+download.
 
 Inside each indexed project:
 
 ```
 your-project/
-├── .codecompass/graph.json   the code knowledge graph (auto-generated)
-└── AGENTS.md                 discovery guide for agents (auto-updated)
+├── .codecompass/graph.json      the code knowledge graph (auto-generated)
+├── .codecompass/vectors.lance/  semantic search index (optional, rebuilt on ingest)
+└── AGENTS.md                    discovery guide for agents (auto-updated)
 ```
 
 ---
 
 ## Limitations
 
-- **Structure first, semantics optional** — the parser knows what calls what,
-  not what it means. `enrich` closes that gap with agent-written descriptions
-  and missed edges, marked `agent_inferred`.
+- **Structure first, semantics layered on** — the parser knows what calls what,
+  not what it means. Two optional layers close that gap: `enrich` (agent-written
+  descriptions and missed edges, marked `agent_inferred`) and `search`
+  (semantic vector search over those descriptions and names).
 - **Static analysis** — dynamic dispatch, reflection, and string-based invocation
   can't be fully resolved. `impact` surfaces those flagged `resolved: false`, and
   `dead_code` results are always candidates to verify.

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/codecompass_mcp.egg-info/SOURCES.txt

@@ -14,6 +14,7 @@ codecompass_mcp.egg-info/top_level.txt
 graph/__init__.py
 graph/code_graph_client.py
 graph/code_queries.py
+graph/vector_store.py
 ingestion/__init__.py
 ingestion/code_normalizer.py
 ingestion/code_parser.py
@@ -28,7 +29,9 @@ tests/test_block_hook.py
 tests/test_code_parser.py
 tests/test_enrich_preserve.py
 tests/test_enricher.py
+tests/test_ingest_progress.py
 tests/test_init.py
 tests/test_lit_css_parser.py
 tests/test_mcp_server.py
-tests/test_php_parser.py
+tests/test_php_parser.py
+tests/test_vector_store.py

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/codecompass_mcp.egg-info/requires.txt

@@ -14,3 +14,7 @@ watchdog>=4.0.0
 [dev]
 pytest>=7.0.0
 pytest-asyncio>=0.23.0
+
+[search]
+lancedb>=0.13.0
+fastembed>=0.3.0

codecompass_mcp-5.2.0/graph/vector_store.py

@@ -0,0 +1,110 @@
+"""Local vector search over graph entities — LanceDB + fastembed.
+
+Optional feature: needs `pip install codecompass-mcp[search]`.
+The index lives at `.codecompass/vectors.lance` and follows the graph's
+lifecycle: wiped and rebuilt wholesale at the end of every ingest, from
+whatever the graph contains at that point (parser + agent-inferred nodes).
+
+ponytail: full rebuild each ingest (no per-file incremental updates) — a few
+thousand short embeddings take seconds; add incremental indexing only if
+ingest time actually hurts.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+from functools import lru_cache
+
+DB_DIRNAME = "vectors.lance"
+TABLE = "entities"
+MODEL = "BAAI/bge-small-en-v1.5"  # 384-dim, ONNX, CPU — no torch, no network calls
+
+
+class VectorDepsMissing(RuntimeError):
+    """Raised when lancedb/fastembed aren't installed."""
+
+
+def _deps():
+    try:
+        import lancedb
+        from fastembed import TextEmbedding
+        return lancedb, TextEmbedding
+    except ImportError as exc:
+        raise VectorDepsMissing(
+            "Vector search needs the optional deps: "
+            "pip install 'codecompass-mcp[search]'"
+        ) from exc
+
+
+@lru_cache(maxsize=1)
+def _embedder():
+    _, TextEmbedding = _deps()
+    return TextEmbedding(model_name=MODEL)
+
+
+def _db_path(repo_path: str) -> str:
+    return os.path.join(repo_path, ".codecompass", DB_DIRNAME)
+
+
+def _entity_text(a: dict) -> str:
+    """The string that gets embedded for one entity node."""
+    parts = [a.get("kind") or "", a.get("name") or "", a.get("file") or "",
+             a.get("description") or ""]
+    return " ".join(p for p in parts if p)
+
+
+def index_entities(repo_path: str) -> int:
+    """Wipe and rebuild the vector index from graph.json. Returns rows indexed."""
+    lancedb, _ = _deps()
+    graph_path = os.path.join(repo_path, ".codecompass", "graph.json")
+    with open(graph_path) as f:
+        nodes = json.load(f).get("nodes", [])
+
+    rows = []
+    for a in nodes:
+        if a.get("type") != "Entity":
+            continue
+        rows.append({
+            "id": a.get("id") or "",
+            "name": a.get("name") or "",
+            "kind": a.get("kind") or "",
+            "file": a.get("file") or "",
+            "line": a.get("line") or 0,
+            "description": a.get("description") or "",
+            "text": _entity_text(a),
+        })
+    if not rows:
+        return 0
+
+    vecs = list(_embedder().embed([r["text"] for r in rows]))
+    for r, v in zip(rows, vecs):
+        r["vector"] = v
+
+    db = lancedb.connect(_db_path(repo_path))
+    db.create_table(TABLE, rows, mode="overwrite")
+    return len(rows)
+
+
+def search_entities(repo_path: str, query: str, limit: int = 10) -> dict:
+    """Semantic search over entity names/kinds/files/descriptions."""
+    lancedb, _ = _deps()
+    db_path = _db_path(repo_path)
+    if not os.path.exists(db_path):
+        return {"query": query, "matches": [], "count": 0,
+                "hint": "No vector index yet — run ingest to build it."}
+
+    vec = list(_embedder().embed([query]))[0]
+    db = lancedb.connect(db_path)
+    tbl = db.open_table(TABLE)
+    hits = tbl.search(vec).limit(limit).to_list()
+
+    matches = [{
+        "name": h["name"],
+        "kind": h["kind"],
+        "file": h["file"],
+        "line": h["line"],
+        "description": h["description"],
+        "distance": round(h["_distance"], 4),
+    } for h in hits]
+    return {"query": query, "matches": matches, "count": len(matches)}

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/ingestion/code_parser.py

@@ -1753,6 +1753,7 @@ def parse_directory(
     project_root: str,
     skip_dirs: set[str] | None = None,
     progress: bool = False,
+    on_progress=None,
 ) -> list[CodeTriple]:
     """Recursively parse all supported files under project_root.
 
@@ -1760,6 +1761,8 @@ def parse_directory(
         project_root: Absolute path to the repo root.
         skip_dirs: Directory names to skip (merged with defaults).
         progress: Show a tqdm progress bar while parsing.
+        on_progress: Called as on_progress(files_done, files_total) after each
+            file — for callers with no console (the MCP server).
     """
     default_skip = {
         ".git", "node_modules", "__pycache__", ".venv", "venv",
@@ -1779,8 +1782,10 @@ def parse_directory(
     source_files_iter = track(source_files, description="Parsing files") if progress else source_files
 
     triples: list[CodeTriple] = []
-    for full_path in source_files_iter:
+    for done, full_path in enumerate(source_files_iter, 1):
         triples.extend(parse_file(full_path, project_root))
+        if on_progress:
+            on_progress(done, len(source_files))
 
     return triples
 

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/ingestion/enricher.py

@@ -180,19 +180,28 @@ def prepare_enrich_batches(
     return result
 
 
-def _resolve_callee(client, project: str, name: str) -> str | None:
-    """Node id of the single in-project entity called `name`, else None.
+def _resolve_callee(client, project: str, name: str,
+                    allow_external: bool = False) -> str | None:
+    """Node id of the single entity called `name`, else None.
+
+    In-project entities (those with a file) only, unless allow_external — the
+    parser also emits file-less nodes for imported stdlib/third-party modules,
+    and an IMPORTS target may legitimately be one of those.
 
     Ambiguous names (multiple definitions) are never guessed — the parser's
     whole unresolved-receiver problem lives here, and a wrong edge is worse
     than a missing one.
     """
-    matches = [
+    matches = _match_ids(client, project, name, allow_external)
+    return matches[0] if len(matches) == 1 else None
+
+
+def _match_ids(client, project: str, name: str, allow_external: bool = False) -> list:
+    return [
         nid for nid, a in client.graph.nodes(data=True)
         if a.get("type") == "Entity" and a.get("project") == project
-        and a.get("file") and a.get("name") == name
+        and (a.get("file") or allow_external) and a.get("name") == name
     ]
-    return matches[0] if len(matches) == 1 else None
 
 
 def add_entity(repo_path: str, name: str, kind: str = "function",
@@ -247,30 +256,55 @@ def add_entity(repo_path: str, name: str, kind: str = "function",
         client.close()
 
 
+# Edge types an agent may record. CONTAINS/DEFINED_IN are structural — the
+# parser owns the hierarchy and a guessed one corrupts it.
+AGENT_RELATIONS = ("CALLS", "IMPORTS", "INHERITS")
+
+
 def add_call(repo_path: str, caller: str, callee: str,
-             line: int | None = None) -> dict[str, str]:
-    """Record a CALLS edge the agent spotted in source but the parser missed.
-    Both ends must resolve unambiguously — ambiguous names are skipped, never
-    guessed. Idempotent: an existing CALLS edge is left alone."""
+             line: int | None = None, relation: str = "CALLS") -> dict[str, str]:
+    """Record an edge the agent spotted in source but the parser missed.
+    relation is one of AGENT_RELATIONS (CALLS by default). Both ends must
+    resolve unambiguously — ambiguous names are skipped, never guessed.
+    Idempotent: an existing edge of the same type is left alone."""
+    relation = relation.upper()
+    if relation not in AGENT_RELATIONS:
+        return {"status": "skipped",
+                "reason": f"relation {relation!r} not one of {', '.join(AGENT_RELATIONS)}"}
     repo_path = os.path.abspath(repo_path)
     project = os.path.basename(repo_path)
     client = get_client(repo_path)
     try:
+        imports = relation == "IMPORTS"
         from_id = _resolve_callee(client, project, caller)
-        to_id = _resolve_callee(client, project, callee)
+        # An import target may be a stdlib/third-party module, which lives in
+        # the graph as a file-less node — or not at all, the first time anyone
+        # records it.
+        to_id = _resolve_callee(client, project, callee, allow_external=imports)
         if from_id is None:
             return {"status": "skipped", "reason": f"caller {caller!r} not found or ambiguous"}
+        # Only when nothing of that name exists — an ambiguous name still skips.
+        if (to_id is None and imports
+                and not _match_ids(client, project, callee, allow_external=True)):
+            to_id = f"{project}:{callee}"
+            language = client.graph.nodes[from_id].get("language", "")
+            client.graph.add_node(
+                to_id, type="Entity", name=callee, project=project,
+                entity_type="module", kind=f"module:{language}" if language else "module",
+                language=language, description=f"{language} module".strip(),
+                agent_inferred=True, agent_created=True,
+            )
         if to_id is None:
             return {"status": "skipped", "reason": f"callee {callee!r} not found or ambiguous"}
         already = any(
-            e.get("type") == "CALLS"
+            e.get("type") == relation
             for e in client.graph.get_edge_data(from_id, to_id, default={}).values()
         )
         if already:
             return {"status": "exists", "from": from_id, "to": to_id}
         client.graph.add_edge(
             from_id, to_id,
-            type="CALLS",
+            type=relation,
             source_file=client.graph.nodes[from_id].get("file", ""),
             line=line,
             resolved=False,

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/main.py

@@ -118,6 +118,7 @@ def init_project(repo_path: str) -> None:
 
 _GITIGNORE_ENTRIES = [
     (".codecompass/enrich/", "# CodeCompass transient enrich-swarm staging dir"),
+    (".codecompass/vectors.lance/", "# CodeCompass vector index (rebuilt on ingest)"),
 ]
 
 
@@ -194,8 +195,14 @@ _REGISTRY = os.environ.get(
 _BLOCKED_TOOLS = {"Grep", "Glob"}
 # Word-boundary match anywhere in the command: catches `grep foo`,
 # `git grep foo`, `sudo cat f`, `xargs rg` — not just command position.
-# (?![\w-]) avoids false positives like `git cat-file`.
-_BLOCKED_SHELL_RE = re.compile(r"\b(?:grep|rg|cat)\b(?![\w-])")
+# (?![\w-]) keeps the bare-`cat` rule off hyphenated names; git's own
+# subcommands are matched by the `\bgit\b ...` alternatives below.
+_BLOCKED_SHELL_RE = re.compile(
+    r"\b(?:grep|rg|cat)\b(?![\w-])"
+    # git's own search/dump: `git grep`, `git log -S/-G`, `git ls-files`, `git cat-file`
+    r"|\bgit\b[^|;&]*?\s(?:grep|ls-files|cat-file)\b"
+    r"|\bgit\b[^|;&]*?\slog\b[^|;&]*?\s-[SG]"
+)
 
 
 def _repos() -> list:
@@ -261,11 +268,11 @@ def main() -> None:
                 saw_path = True
                 repo = _repo_containing(p)
                 if repo:
-                    _block("grep/rg/cat")
+                    _block("grep/rg/cat/git grep")
             if not saw_path:  # unparseable — decide by where the agent stands
                 repo = _repo_containing(os.path.realpath(cwd))
                 if repo:
-                    _block("grep/rg/cat")
+                    _block("grep/rg/cat/git grep")
             # every named path is outside all codecompass repos — allow
 
     sys.exit(0)
@@ -374,10 +381,12 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 // Word-boundary match anywhere in the command: catches `grep foo`,
 // `git grep foo`, `sudo cat f`, `xargs rg` — not just command position.
 // (?![\w-]) avoids false positives like `git cat-file`.
-const BLOCKED_SHELL_RE = /\b(?:grep|rg|cat)\b(?![\w-])/;
+// git's own search/dump is blocked too: `git grep`, `git log -S/-G`, `git ls-files`, `git cat-file`.
+const BLOCKED_SHELL_RE =
+  /\b(?:grep|rg|cat)\b(?![\w-])|\bgit\b[^|;&]*?\s(?:grep|ls-files|cat-file)\b|\bgit\b[^|;&]*?\slog\b[^|;&]*?\s-[SG]/;
 
 const REASON =
-  "Don't grep/cat/rg the repo. Discover through the codecompass MCP tools — " +
+  "Don't grep/cat/rg (or `git grep`) the repo. Discover through the codecompass MCP tools — " +
   "`grep` to find what's relevant, then `flow`/`impact`/`deps` to trace — " +
   "then read the specific slice with the Read tool (or sed -n/head/tail), " +
   "not a whole-file dump.";
@@ -436,16 +445,24 @@ def _ensure_pi_agents_md(repo_path: str) -> None:
         )
 
 
-def ingest_code(repo_path: str, normalize: bool = False, dump_triples: str | None = None) -> None:
+def ingest_code(repo_path: str, normalize: bool = False, dump_triples: str | None = None,
+                on_progress=None) -> None:
     """Ingest a codebase into the local code knowledge graph.
 
     Phase 1: Walk the repo and write the Project → Folder → File skeleton.
     Phase 2: Parse every source file with tree-sitter into CodeTriples.
     Phase 3: Normalize entity names via Haiku (only if --normalize is passed).
     Phase 4: Write all triples to the local graph.json.
+
+    on_progress: called as on_progress(percent, message) — for callers with no
+    console (the MCP server). Percent is 0-100 across all phases.
     """
     import json
 
+    def _report(pct: int, message: str) -> None:
+        if on_progress:
+            on_progress(pct, message)
+
     repo_path = os.path.abspath(repo_path)
     project_name = os.path.basename(repo_path)
     console.print(f"[bold blue]Ingesting codebase:[/] {repo_path}")
@@ -473,11 +490,24 @@ def ingest_code(repo_path: str, normalize: bool = False, dump_triples: str | Non
     client.graph.clear()
 
     console.print("[dim]Phase 1/4 — Building hierarchy…[/]")
+    _report(2, "Building hierarchy…")
     file_id_map = build_hierarchy(repo_path, project_name, client)
     console.print(f"[dim]  {len(file_id_map)} source files indexed[/]")
 
     console.print("[dim]Phase 2/4 — Parsing source files…[/]")
-    raw_triples = parse_directory(repo_path, progress=True)
+    # Parsing dominates the wall clock, so it owns 5-70% of the bar.
+    _report(5, f"Parsing {len(file_id_map)} source files…")
+    last_pct = 5
+
+    def _parsed(done: int, total: int) -> None:
+        nonlocal last_pct
+        pct = 5 + int(65 * done / max(total, 1))
+        if pct > last_pct:  # one notification per percent, not per file
+            last_pct = pct
+            _report(pct, f"Parsing files ({done}/{total})…")
+
+    raw_triples = parse_directory(repo_path, progress=True,
+                                  on_progress=_parsed if on_progress else None)
     console.print(f"[dim]  {len(raw_triples)} raw triples extracted[/]")
 
     if not raw_triples:
@@ -507,6 +537,7 @@ def ingest_code(repo_path: str, normalize: bool = False, dump_triples: str | Non
     if normalize:
         from ingestion.code_normalizer import normalize_triples
         console.print("[dim]Phase 3/4 — Normalizing triples via Haiku…[/]")
+        _report(72, "Normalizing triples via Haiku…")
         triples = normalize_triples(raw_triples, progress=True)
         console.print(f"[dim]  {len(triples)} triples after normalization[/]")
     else:
@@ -514,6 +545,7 @@ def ingest_code(repo_path: str, normalize: bool = False, dump_triples: str | Non
         triples = raw_triples
 
     console.print("[dim]Phase 4/4 — Writing to local graph…[/]")
+    _report(85, f"Writing {len(triples)} triples to the graph…")
     written = client.write_code_triples_batch(triples, file_id_map, project_name)
 
     # Restore agent-authored data onto the new graph.
@@ -537,6 +569,17 @@ def ingest_code(repo_path: str, normalize: bool = False, dump_triples: str | Non
     total_nodes = client.node_count()
     client.close()
 
+    # Phase 5 — rebuild the vector index from the final graph (parser nodes
+    # plus restored agent-inferred ones). Wipes and rewrites like the graph.
+    try:
+        from graph.vector_store import index_entities
+        _report(95, "Rebuilding vector index…")
+        n = index_entities(repo_path)
+        console.print(f"[dim]Phase 5/5 — Vector index rebuilt ({n} entities embedded)[/]")
+    except Exception as exc:
+        console.print(f"[dim]Phase 5/5 — Vector index skipped ({exc})[/]")
+
+    _report(100, f"Done — {written} triples, {total_nodes} nodes.")
     console.print(
         f"[bold green]Done.[/] Wrote {written} triples. "
         f"Graph now has {total_nodes} nodes."
@@ -571,6 +614,7 @@ it at another repo.
    | You have… | Use | Example |
    |---|---|---|
    | a concept, name, or pattern | `grep` | `grep(pattern="^Session")`, `grep(pattern=".*Adapter$")` |
+   | an idea, not a name ("where does caching go?") | `search` | `search(query="session timeout")` — semantic vector search over entity names/kinds/files/descriptions (needs the optional `search` extra + an ingest to build the index) |
    | the full layout | `tree` | (large — read it in slices) |
 
 2. **Trace** — understand relationships around a known symbol/file:
@@ -678,14 +722,26 @@ doc → `memory.md`; code-comment warning to the next person → `learnings.md`.
 - After major refactors (moved functions, renamed classes)
 - If query results look stale or incomplete
 
-### The graph improves with use — record what it missed
-
-While reading code you may find entities, calls, or important variables the
-parser didn't capture. Record them immediately with the MCP tools
-`add_entity(name, kind, file, line, description)` and
-`add_call(caller, callee, line)`. Both mark entries `agent_inferred` and skip
-anything ambiguous rather than guess. Small opportunistic writes keep the
-graph accurate between full `enrich` runs.
+### The graph improves with use — record what it missed (NOT optional)
+
+Exploring the code is how you find what the parser missed, so write it back as
+you go — every session, not only when asked:
+
+- Read a function/class/constant the graph doesn't have (or `grep`/`search`
+  found nothing for) → `add_entity(name, kind, file, line, description)`.
+- Read a call the graph doesn't show — dynamic dispatch, a callback, a handler
+  wired up at runtime → `add_call(caller, callee, line)`. Same tool for a
+  missed import or base class: `add_call(a, b, relation="IMPORTS")` /
+  `relation="INHERITS"`. IMPORTS targets may be stdlib or third-party
+  modules (`add_call("main", "pathlib", relation="IMPORTS")`).
+- Worked out what an entity actually does → `add_entity` again with the real
+  one-line description; it overwrites the placeholder.
+
+Record it in the same turn you learned it, before you answer the user — a fact
+you postpone is a fact the next session re-derives. Both tools mark entries
+`agent_inferred` and skip anything ambiguous rather than guess, so a wrong
+guess costs nothing but a `skipped` status. Small opportunistic writes keep
+the graph accurate between full `enrich` runs.
 
 ### Enrichment — user-triggered ONLY
 

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/mcp_server.py

@@ -15,10 +15,13 @@ agent says "use this repo" or "cd to this repo" to point it elsewhere.
 
 from __future__ import annotations
 
+import functools
 import os
 from pathlib import Path
 
-from fastmcp import FastMCP
+import anyio.from_thread
+import anyio.to_thread
+from fastmcp import Context, FastMCP
 
 from graph.code_queries import (
     DEFAULT_HOPS,
@@ -95,11 +98,25 @@ def _active_repo() -> str:
     return _REPO_PATH
 
 
+# Attached to every read result: exploring is when the agent learns what the
+# parser missed, so the reminder to write it back rides along with the answer.
+_RECORD_NUDGE = (
+    "Before moving on: anything you read that the graph is missing or "
+    "describes badly — record it now. add_entity(name, kind, file, line, "
+    "description) for entities, add_call(caller, callee, line) for calls. "
+    "Ambiguous entries are skipped, not guessed; agent-recorded data survives "
+    "re-ingest."
+)
+
+
 def _tool(fetch_fn, *args, **kwargs):
     """Resolve the active repo, ensure it is initialized, and call a fetch helper."""
     repo = _active_repo()
     _ensure_initialized(repo)
-    return fetch_fn(*args, repo, os.path.basename(repo), **kwargs)
+    result = fetch_fn(*args, repo, os.path.basename(repo), **kwargs)
+    if isinstance(result, dict):
+        result.setdefault("next", _RECORD_NUDGE)
+    return result
 
 
 @mcp.tool()
@@ -165,6 +182,25 @@ def grep(pattern: str, field: str = "all", ignore_case: bool = True, limit: int
     return _tool(fetch_grep, pattern, field=field, ignore_case=ignore_case, limit=limit)
 
 
+@mcp.tool()
+def search(query: str, limit: int = 10) -> dict:
+    """Semantic vector search over entities — for concepts that aren't symbol names.
+
+    Matches the query against embedded entity name/kind/file/description, so
+    it finds things like "session timeout" or "caching" even when no symbol
+    carries those words. Use grep for exact names/patterns; use this when you
+    have an idea, not a name. Requires the optional vector deps
+    (`pip install 'codecompass-mcp[search]'`) and an ingest to build the index.
+    """
+    repo = _active_repo()
+    _ensure_initialized(repo)
+    from graph.vector_store import search_entities
+    result = search_entities(repo, query, limit=limit)
+    if isinstance(result, dict):
+        result.setdefault("next", _RECORD_NUDGE)
+    return result
+
+
 @mcp.tool()
 def flow(
     entry_symbol: str,
@@ -231,7 +267,8 @@ def init() -> dict:
 
 
 @mcp.tool()
-def ingest(normalize: bool = False, dump_triples: str | None = None) -> dict:
+async def ingest(ctx: Context, normalize: bool = False,
+                 dump_triples: str | None = None) -> dict:
     """Re-index the currently configured repo and rebuild the code knowledge graph.
 
     normalize: normalize entity names via Haiku (slower, needs an API key).
@@ -239,7 +276,15 @@ def ingest(normalize: bool = False, dump_triples: str | None = None) -> dict:
     loading them into the graph (debugging the parser)."""
     repo = _active_repo()
     _ensure_initialized(repo)
-    ingest_code(repo, normalize=normalize, dump_triples=dump_triples)
+
+    # ingest_code is blocking, so it runs in a worker thread; progress hops back
+    # to the event loop from there so notifications flush while it works.
+    def on_progress(pct: int, message: str) -> None:
+        anyio.from_thread.run(ctx.report_progress, pct, 100, message)
+
+    await anyio.to_thread.run_sync(
+        functools.partial(ingest_code, repo, normalize=normalize,
+                          dump_triples=dump_triples, on_progress=on_progress))
     return {"status": "ok", "repo": repo, "project": os.path.basename(repo),
             "normalize": normalize, "dump_triples": dump_triples,
             "next": "Record anything the parser missed while you were reading "
@@ -286,16 +331,24 @@ def add_entity(name: str, kind: str = "function", file: str = "",
 
 
 @mcp.tool()
-def add_call(caller: str, callee: str, line: int | None = None) -> dict:
-    """Record a CALLS edge you spotted in source that the parser missed
-    (dynamic dispatch, callbacks, string-based lookup). Both names must
+def add_call(caller: str, callee: str, line: int | None = None,
+             relation: str = "CALLS") -> dict:
+    """Record an edge you spotted in source that the parser missed — a call via
+    dynamic dispatch, a callback, string-based lookup, a conditional import, a
+    runtime-registered base class.
+
+    relation is CALLS (default), IMPORTS, or INHERITS. Structural edges
+    (CONTAINS/DEFINED_IN) are parser-owned and cannot be added. Both names must
     resolve unambiguously — ambiguous targets are skipped, never guessed.
-    Idempotent: existing edges are left alone.
+    IMPORTS may target a stdlib or third-party module (`add_call("main",
+    "pathlib", relation="IMPORTS")`); the module node is created if the graph
+    has never seen it. Idempotent: existing edges of the same type are left
+    alone.
     """
     repo = _active_repo()
     _ensure_initialized(repo)
     from ingestion.enricher import add_call as _add
-    return _add(repo, caller, callee, line=line)
+    return _add(repo, caller, callee, line=line, relation=relation)
 
 
 def main() -> None:

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "codecompass-mcp"
-version = "5.0.0"
+version = "5.2.0"
 description = "Structural code context for AI coding agents — a local code knowledge graph for blast radius, impact, deps, dead code, and flow tracing"
 readme = "README.md"
 license = {text = "MIT"}
@@ -39,6 +39,10 @@ dev = [
     "pytest>=7.0.0",
     "pytest-asyncio>=0.23.0",
 ]
+search = [
+    "lancedb>=0.13.0",
+    "fastembed>=0.3.0",
+]
 
 [project.scripts]
 codecompass = "main:main"

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/tests/test_block_hook.py

@@ -56,11 +56,20 @@ def test_hook_blocks_inside_repo_allows_outside(tmp_path):
                     "cwd": str(repo)}, env)
     assert r.returncode == 2
 
-    # git cat-file is not a search -> allowed
-    r = _run(hook, {"tool_name": "Bash",
-                    "tool_input": {"command": "git cat-file -p HEAD"},
-                    "cwd": str(repo)}, env)
-    assert r.returncode == 0
+    # git's other content searches -> blocked
+    for cmd in ("git ls-files", "git log -S needle", "git -C . grep foo",
+                "git cat-file -p HEAD"):
+        r = _run(hook, {"tool_name": "Bash",
+                        "tool_input": {"command": cmd},
+                        "cwd": str(repo)}, env)
+        assert r.returncode == 2, cmd
+
+    # ordinary git history/read commands -> allowed
+    for cmd in ("git log --oneline", "git status", "git show HEAD"):
+        r = _run(hook, {"tool_name": "Bash",
+                        "tool_input": {"command": cmd},
+                        "cwd": str(repo)}, env)
+        assert r.returncode == 0, cmd
 
     # Grep tool defaulting to cwd (inside the repo) -> blocked
     r = _run(hook, {"tool_name": "Grep",

{codecompass_mcp-5.0.0 → codecompass_mcp-5.2.0}/tests/test_enricher.py

@@ -109,12 +109,38 @@ def test_add_entity_and_add_call(tmp_path):
     assert add_call(str(repo_path), "caller", "shared")["status"] == "skipped"
     assert add_call(str(repo_path), "ghost", "callee")["status"] == "skipped"
 
+    # Non-CALLS relations ride the same tool; CALLS on the same pair already
+    # exists, so a distinct type must still be added.
+    assert add_call(str(repo_path), "caller", "callee",
+                    relation="imports")["status"] == "added"
+    assert add_call(str(repo_path), "caller", "callee",
+                    relation="IMPORTS")["status"] == "exists"
+    # Structural edges stay parser-owned
+    assert add_call(str(repo_path), "caller", "callee",
+                    relation="CONTAINS")["status"] == "skipped"
+
+    # IMPORTS of a module the graph has never seen creates the external node;
+    # a CALLS to the same unknown name still skips (only imports go external).
+    assert add_call(str(repo_path), "caller", "pathlib",
+                    relation="IMPORTS")["status"] == "added"
+    assert add_call(str(repo_path), "caller", "pathlib")["status"] == "skipped"
+    # Ambiguity still wins over node creation
+    assert add_call(str(repo_path), "caller", "shared",
+                    relation="IMPORTS")["status"] == "skipped"
+
     client = get_client(str(repo_path))
     node = next(a for _, a in client.graph.nodes(data=True) if a.get("name") == "helper")
     assert node["description"] == "Async helper." and node["agent_inferred"] is True
     edge = next(e for e in client.graph.get_edge_data(
         f"{project}:caller", f"{project}:callee").values() if e.get("type") == "CALLS")
     assert edge["agent_inferred"] is True
+    imports = next(e for e in client.graph.get_edge_data(
+        f"{project}:caller", f"{project}:callee").values() if e.get("type") == "IMPORTS")
+    assert imports["agent_inferred"] is True
+    # The stdlib module node the agent created: file-less, like the parser's own
+    stdlib = client.graph.nodes[f"{project}:pathlib"]
+    assert stdlib["kind"] == "module:python" and not stdlib.get("file")
+    assert stdlib["agent_created"] is True
     client.close()
 
 
@@ -136,6 +162,8 @@ def test_agent_data_survives_reingest(tmp_path):
 
     add_entity(str(repo_path), "helper", file="src/a.py", description="Async helper.")
     assert add_call(str(repo_path), "caller", "target", line=2)["status"] == "added"
+    assert add_call(str(repo_path), "caller", "pathlib",
+                    relation="IMPORTS")["status"] == "added"
 
     cc_main.ingest_code(str(repo_path))  # full rebuild
 
@@ -147,4 +175,8 @@ def test_agent_data_survives_reingest(tmp_path):
     assert len(agent_edges) == 1
     names = {client.graph.nodes[n].get("name") for n in agent_edges[0]}
     assert names == {"caller", "target"}
+    # The file-less stdlib node and its IMPORTS edge survive too
+    assert client.graph.nodes[f"{project}:pathlib"]["agent_created"] is True
+    assert any(e.get("type") == "IMPORTS" and e.get("agent_inferred")
+               for _, _, e in client.graph.edges(data=True))
     client.close()

codecompass_mcp-5.2.0/tests/test_ingest_progress.py

@@ -0,0 +1,43 @@
+"""The MCP ingest tool streams progress notifications while it works."""
+
+from __future__ import annotations
+
+import os
+import sys
+
+import pytest
+
+sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
+
+
+@pytest.mark.anyio
+async def test_ingest_reports_progress(tmp_path, monkeypatch):
+    from fastmcp import Client
+
+    import mcp_server
+
+    repo = tmp_path / "repo"
+    (repo / "pkg").mkdir(parents=True)
+    for i in range(5):
+        (repo / "pkg" / f"m{i}.py").write_text(f"def f{i}():\n    return {i}\n")
+
+    monkeypatch.setattr(mcp_server, "_REPO_PATH", str(repo))
+    seen = []
+
+    async def handler(progress: float, total: float | None, message: str | None):
+        seen.append((progress, total, message))
+
+    async with Client(mcp_server.mcp, progress_handler=handler) as client:
+        result = await client.call_tool("ingest", {})
+
+    assert result.data["status"] == "ok"
+    percents = [p for p, _, _ in seen]
+    assert percents == sorted(percents)  # monotonic
+    assert percents[0] < 100 and percents[-1] == 100
+    assert all(t == 100 for _, t, _ in seen)
+    assert any("Parsing" in (m or "") for _, _, m in seen)
+
+
+@pytest.fixture
+def anyio_backend():
+    return "asyncio"

codecompass_mcp-5.2.0/tests/test_vector_store.py

@@ -0,0 +1,65 @@
+"""Vector store: wipe-rebuild on index + semantic search returns the right entity.
+
+Uses real LanceDB in a tmp dir; the embedding model is faked with a
+bag-of-words hash so the test needs no download. Skips when the optional
+`search` deps aren't installed.
+"""
+
+import json
+
+import pytest
+
+pytest.importorskip("lancedb")
+pytest.importorskip("fastembed")
+
+from graph import vector_store
+
+
+class _FakeEmbedder:
+    def embed(self, texts):
+        vecs = []
+        for t in texts:
+            v = [0.0] * 384
+            for word in t.lower().split():
+                v[hash(word) % 384] += 1.0
+            vecs.append(v)
+        return iter(vecs)
+
+
+@pytest.fixture
+def repo(tmp_path, monkeypatch):
+    monkeypatch.setattr(vector_store, "_embedder", lambda: _FakeEmbedder())
+    cc = tmp_path / ".codecompass"
+    cc.mkdir()
+    graph = {
+        "nodes": [
+            {"type": "Entity", "id": "p:a.py:session_handler", "name": "session_handler",
+             "kind": "function:python", "file": "a.py", "line": 3,
+             "description": "handles session timeout"},
+            {"type": "Entity", "id": "p:b.py:render_button", "name": "render_button",
+             "kind": "function:python", "file": "b.py", "line": 1,
+             "description": ""},
+            {"type": "File", "id": "p:a.py", "path": "a.py"},  # not indexed
+        ],
+        "links": [],
+    }
+    (cc / "graph.json").write_text(json.dumps(graph))
+    return str(tmp_path)
+
+
+def test_index_skips_non_entities(repo):
+    assert vector_store.index_entities(repo) == 2
+
+
+def test_search_finds_by_description(repo):
+    vector_store.index_entities(repo)
+    hits = vector_store.search_entities(repo, "session timeout")
+    assert hits["count"] >= 1
+    assert hits["matches"][0]["name"] == "session_handler"
+
+
+def test_search_without_index_reports_hint(tmp_path):
+    (tmp_path / ".codecompass").mkdir()
+    out = vector_store.search_entities(str(tmp_path), "anything")
+    assert out["count"] == 0
+    assert "hint" in out