codespine 0.5.3__tar.gz → 0.5.4__tar.gz

codespine-0.5.4/PKG-INFO

@@ -0,0 +1,270 @@
+Metadata-Version: 2.4
+Name: codespine
+Version: 0.5.4
+Summary: Local Java code intelligence indexer backed by a graph database
+Author: CodeSpine contributors
+License: MIT License
+        
+        Copyright (c) 2026 CodeSpine contributors
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Homepage, https://github.com/vinayak3022/codeSpine
+Project-URL: Repository, https://github.com/vinayak3022/codeSpine
+Project-URL: Issues, https://github.com/vinayak3022/codeSpine/issues
+Keywords: java,code-indexing,graph,kuzu,mcp
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click
+Requires-Dist: kuzu
+Requires-Dist: tree-sitter
+Requires-Dist: tree-sitter-java
+Requires-Dist: fastmcp>=2.3.0
+Requires-Dist: psutil
+Requires-Dist: watchfiles
+Provides-Extra: ml
+Requires-Dist: sentence-transformers; extra == "ml"
+Requires-Dist: numpy; extra == "ml"
+Provides-Extra: community
+Requires-Dist: igraph; extra == "community"
+Requires-Dist: leidenalg; extra == "community"
+Provides-Extra: full
+Requires-Dist: sentence-transformers; extra == "full"
+Requires-Dist: numpy; extra == "full"
+Requires-Dist: igraph; extra == "full"
+Requires-Dist: leidenalg; extra == "full"
+Dynamic: license-file
+
+# CodeSpine
+
+CodeSpine cuts token burn for coding agents working on Java codebases.
+
+Instead of having an agent open dozens of `.java` files to answer one question, CodeSpine indexes the codebase once and serves the structure over MCP. The agent asks for symbols, callers, impact, flows, dead code, and module boundaries directly, which means fewer file reads, fewer wasted context windows, and fewer hallucinated code paths.
+
+It indexes classes, methods, calls, type relationships, cross-module links, git coupling, dead-code candidates, and execution flows so agents can work from graph answers first and source files second.
+
+## Why It Saves Tokens
+
+- One MCP call can replace many file opens. `get_symbol_context("PaymentService")` returns a resolved neighborhood instead of forcing the agent to read every caller and callee file manually.
+- Search is structure-aware. Agents can ask for a symbol, concept, impact radius, or dead-code candidate without scanning entire packages.
+- Multi-module repos stay scoped. Project-aware IDs and `project=` parameters reduce noise from unrelated modules and workspaces.
+- Repeat sessions get cheaper. Once indexed, the agent reuses the graph instead of re-discovering the same relationships every turn.
+
+## Install
+
+```bash
+pip install codespine
+```
+
+Optional semantic search:
+
+```bash
+pip install "codespine[ml]"
+```
+
+## What It Does
+
+- Hybrid search: BM25 + fuzzy by default, semantic vector search with `--embed`
+- Impact analysis: callers, dependencies, and confidence-scored edges
+- Dead code detection: Java-aware exemptions for tests, framework hooks, contracts, and common DI patterns
+- Execution flows: traces from entry points through the call graph
+- Community detection: structural clusters for architectural context
+- Change coupling: git-history-based file relationships
+- Multi-project and multi-module indexing: workspaces, Maven modules, Gradle subprojects
+- MCP server: structured tools for Claude, Cursor, Cline, Copilot, and similar clients
+
+## Quick Start
+
+Index a repo:
+
+```bash
+codespine analyse /path/to/project
+```
+
+Run a deeper pass:
+
+```bash
+codespine analyse /path/to/project --deep
+```
+
+Add embeddings for semantic search:
+
+```bash
+codespine analyse /path/to/project --embed
+```
+
+Typical output:
+
+```text
+$ codespine analyse .
+Walking files...               142 files found
+Index mode...                  incremental (8 files to index, 0 deleted)
+Parsing code...                8/8
+Tracing calls...               847 calls resolved
+Analyzing types...             234 type relationships
+Cross-module linking...        skipped (single module)
+Detecting communities...       8 clusters found
+Detecting execution flows...   34 processes found
+Finding dead code...           12 unreachable symbols
+Analyzing git history...       18 coupled file pairs
+Generating embeddings...       0 vectors stored
+
+Done in 4.2s - 623 symbols, 1847 edges, 8 clusters, 34 flows (no embeddings; rerun with --embed for semantic search)
+```
+
+Search the index:
+
+```bash
+codespine search "retry payment"
+codespine context "PaymentService"
+codespine impact "com.example.PaymentService#charge(java.lang.String)"
+codespine stats
+```
+
+## MCP
+
+Foreground MCP server:
+
+```bash
+codespine mcp
+```
+
+Minimal MCP config:
+
+```json
+{
+  "mcpServers": {
+    "codespine": {
+      "command": "codespine",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+If the client launches the wrong Python environment, use the absolute binary path instead:
+
+```json
+{
+  "mcpServers": {
+    "codespine": {
+      "command": "/absolute/path/to/codespine",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+Common MCP tools:
+
+- `search_hybrid(query, k, project)`
+- `find_symbol(name, kind, project, limit)`
+- `get_symbol_context(query, max_depth, project)`
+- `get_impact(symbol, max_depth, project)`
+- `detect_dead_code(limit, project, strict)`
+- `trace_execution_flows(entry_symbol, max_depth, project)`
+- `get_symbol_community(symbol)`
+- `get_change_coupling(months, min_strength, min_cochanges, project)`
+- `compare_branches(base_ref, head_ref)`
+- `get_codebase_stats()`
+
+## CLI
+
+Core commands:
+
+```bash
+codespine analyse <path>
+codespine analyse <path> --full
+codespine analyse <path> --deep
+codespine analyse <path> --embed
+codespine watch --path .
+codespine search "query"
+codespine context "symbol"
+codespine impact "symbol"
+codespine deadcode
+codespine flow
+codespine community
+codespine coupling
+codespine diff main..feature
+codespine stats
+codespine list
+codespine clear-project <project_id>
+codespine clear-index
+```
+
+`analyse` defaults to incremental mode. Repeat runs are designed to be fast when files have not changed.
+
+## Workspace And Module Detection
+
+CodeSpine can index:
+
+- a single Java repo
+- a multi-module Maven or Gradle repo
+- a workspace directory containing multiple repos
+
+Project IDs are:
+
+- single-module repo: `payments-service`
+- multi-module repo: `payments-service::core`, `payments-service::api`
+
+That same project ID can be passed into MCP tools and CLI analysis calls that support project scoping.
+
+## Deep Analysis Trade-Offs
+
+`--deep` enables the expensive graph-wide passes:
+
+- communities
+- execution flows
+- dead code
+- git coupling
+
+Use it when you want architecture-level context. Skip it when you just need the graph refreshed for search, context, and impact.
+
+`--embed` is also optional. Without it, CodeSpine still supports exact, keyword, and fuzzy search. Add embeddings when you need concept-level retrieval.
+
+## Runtime Files
+
+- `~/.codespine_db` - graph database
+- `~/.codespine.pid` - MCP background server PID
+- `~/.codespine.log` - server log
+- `~/.codespine_embedding_cache.json` - embedding cache
+- `~/.codespine_index_meta/` - incremental file metadata cache
+
+## Notes
+
+- `codespine start` launches a background MCP server. Most IDE MCP clients should use `codespine mcp` instead and manage the process themselves.
+- `codespine clear-index` rebuilds the local index database from scratch.
+- For large Spring or JPA-heavy repos, dead-code results should still be reviewed before deletion. The tool is conservative, not authoritative.
+
+## Project Docs
+
+- [Contributing](.github/CONTRIBUTING.md)
+- [Security](.github/SECURITY.md)
+- [Code of Conduct](.github/CODE_OF_CONDUCT.md)

codespine-0.5.4/README.md

@@ -0,0 +1,206 @@
+# CodeSpine
+
+CodeSpine cuts token burn for coding agents working on Java codebases.
+
+Instead of having an agent open dozens of `.java` files to answer one question, CodeSpine indexes the codebase once and serves the structure over MCP. The agent asks for symbols, callers, impact, flows, dead code, and module boundaries directly, which means fewer file reads, fewer wasted context windows, and fewer hallucinated code paths.
+
+It indexes classes, methods, calls, type relationships, cross-module links, git coupling, dead-code candidates, and execution flows so agents can work from graph answers first and source files second.
+
+## Why It Saves Tokens
+
+- One MCP call can replace many file opens. `get_symbol_context("PaymentService")` returns a resolved neighborhood instead of forcing the agent to read every caller and callee file manually.
+- Search is structure-aware. Agents can ask for a symbol, concept, impact radius, or dead-code candidate without scanning entire packages.
+- Multi-module repos stay scoped. Project-aware IDs and `project=` parameters reduce noise from unrelated modules and workspaces.
+- Repeat sessions get cheaper. Once indexed, the agent reuses the graph instead of re-discovering the same relationships every turn.
+
+## Install
+
+```bash
+pip install codespine
+```
+
+Optional semantic search:
+
+```bash
+pip install "codespine[ml]"
+```
+
+## What It Does
+
+- Hybrid search: BM25 + fuzzy by default, semantic vector search with `--embed`
+- Impact analysis: callers, dependencies, and confidence-scored edges
+- Dead code detection: Java-aware exemptions for tests, framework hooks, contracts, and common DI patterns
+- Execution flows: traces from entry points through the call graph
+- Community detection: structural clusters for architectural context
+- Change coupling: git-history-based file relationships
+- Multi-project and multi-module indexing: workspaces, Maven modules, Gradle subprojects
+- MCP server: structured tools for Claude, Cursor, Cline, Copilot, and similar clients
+
+## Quick Start
+
+Index a repo:
+
+```bash
+codespine analyse /path/to/project
+```
+
+Run a deeper pass:
+
+```bash
+codespine analyse /path/to/project --deep
+```
+
+Add embeddings for semantic search:
+
+```bash
+codespine analyse /path/to/project --embed
+```
+
+Typical output:
+
+```text
+$ codespine analyse .
+Walking files...               142 files found
+Index mode...                  incremental (8 files to index, 0 deleted)
+Parsing code...                8/8
+Tracing calls...               847 calls resolved
+Analyzing types...             234 type relationships
+Cross-module linking...        skipped (single module)
+Detecting communities...       8 clusters found
+Detecting execution flows...   34 processes found
+Finding dead code...           12 unreachable symbols
+Analyzing git history...       18 coupled file pairs
+Generating embeddings...       0 vectors stored
+
+Done in 4.2s - 623 symbols, 1847 edges, 8 clusters, 34 flows (no embeddings; rerun with --embed for semantic search)
+```
+
+Search the index:
+
+```bash
+codespine search "retry payment"
+codespine context "PaymentService"
+codespine impact "com.example.PaymentService#charge(java.lang.String)"
+codespine stats
+```
+
+## MCP
+
+Foreground MCP server:
+
+```bash
+codespine mcp
+```
+
+Minimal MCP config:
+
+```json
+{
+  "mcpServers": {
+    "codespine": {
+      "command": "codespine",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+If the client launches the wrong Python environment, use the absolute binary path instead:
+
+```json
+{
+  "mcpServers": {
+    "codespine": {
+      "command": "/absolute/path/to/codespine",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+Common MCP tools:
+
+- `search_hybrid(query, k, project)`
+- `find_symbol(name, kind, project, limit)`
+- `get_symbol_context(query, max_depth, project)`
+- `get_impact(symbol, max_depth, project)`
+- `detect_dead_code(limit, project, strict)`
+- `trace_execution_flows(entry_symbol, max_depth, project)`
+- `get_symbol_community(symbol)`
+- `get_change_coupling(months, min_strength, min_cochanges, project)`
+- `compare_branches(base_ref, head_ref)`
+- `get_codebase_stats()`
+
+## CLI
+
+Core commands:
+
+```bash
+codespine analyse <path>
+codespine analyse <path> --full
+codespine analyse <path> --deep
+codespine analyse <path> --embed
+codespine watch --path .
+codespine search "query"
+codespine context "symbol"
+codespine impact "symbol"
+codespine deadcode
+codespine flow
+codespine community
+codespine coupling
+codespine diff main..feature
+codespine stats
+codespine list
+codespine clear-project <project_id>
+codespine clear-index
+```
+
+`analyse` defaults to incremental mode. Repeat runs are designed to be fast when files have not changed.
+
+## Workspace And Module Detection
+
+CodeSpine can index:
+
+- a single Java repo
+- a multi-module Maven or Gradle repo
+- a workspace directory containing multiple repos
+
+Project IDs are:
+
+- single-module repo: `payments-service`
+- multi-module repo: `payments-service::core`, `payments-service::api`
+
+That same project ID can be passed into MCP tools and CLI analysis calls that support project scoping.
+
+## Deep Analysis Trade-Offs
+
+`--deep` enables the expensive graph-wide passes:
+
+- communities
+- execution flows
+- dead code
+- git coupling
+
+Use it when you want architecture-level context. Skip it when you just need the graph refreshed for search, context, and impact.
+
+`--embed` is also optional. Without it, CodeSpine still supports exact, keyword, and fuzzy search. Add embeddings when you need concept-level retrieval.
+
+## Runtime Files
+
+- `~/.codespine_db` - graph database
+- `~/.codespine.pid` - MCP background server PID
+- `~/.codespine.log` - server log
+- `~/.codespine_embedding_cache.json` - embedding cache
+- `~/.codespine_index_meta/` - incremental file metadata cache
+
+## Notes
+
+- `codespine start` launches a background MCP server. Most IDE MCP clients should use `codespine mcp` instead and manage the process themselves.
+- `codespine clear-index` rebuilds the local index database from scratch.
+- For large Spring or JPA-heavy repos, dead-code results should still be reviewed before deletion. The tool is conservative, not authoritative.
+
+## Project Docs
+
+- [Contributing](.github/CONTRIBUTING.md)
+- [Security](.github/SECURITY.md)
+- [Code of Conduct](.github/CODE_OF_CONDUCT.md)

{codespine-0.5.3 → codespine-0.5.4}/codespine/__init__.py

@@ -1,4 +1,4 @@
 """CodeSpine package."""
 
 __all__ = ["__version__"]
-__version__ = "0.5.3"
+__version__ = "0.5.4"

codespine-0.5.4/codespine/analysis/community.py

@@ -0,0 +1,182 @@
+from __future__ import annotations
+
+from collections import Counter, defaultdict
+
+MAX_LEIDEN_SYMBOLS = 12000
+MIN_COMMUNITY_SIZE = 2
+PACKAGE_BUCKET_DEPTH = 5
+
+
+def _package_bucket(fqname: str) -> str:
+    base = (fqname or "").split("#", 1)[0]
+    parts = [p for p in base.split(".") if p]
+    if len(parts) <= 2:
+        return base or "default"
+    package_parts = parts[:-1] if len(parts) > 1 else parts
+    depth = min(PACKAGE_BUCKET_DEPTH, len(package_parts))
+    return ".".join(package_parts[:depth]) or base or "default"
+
+
+def _community_label(symbol_ids: list[str], symbol_meta: dict[str, dict]) -> str:
+    bucket_counts = Counter(_package_bucket(symbol_meta[sid].get("fqname", "")) for sid in symbol_ids if sid in symbol_meta)
+    if bucket_counts:
+        return bucket_counts.most_common(1)[0][0]
+    return "community"
+
+
+def _call_graph_communities(symbol_meta: dict[str, dict], method_edges: list[tuple[str, str]], progress=None) -> dict[str, int]:
+    def _ping(msg: str) -> None:
+        if progress:
+            progress(msg)
+
+    graph_nodes = sorted({sid for edge in method_edges for sid in edge})
+    if not graph_nodes:
+        return {}
+
+    if len(graph_nodes) > MAX_LEIDEN_SYMBOLS:
+        _ping(f"graph too large for leiden ({len(graph_nodes)} symbols), using package fallback")
+        return {}
+
+    index_of = {sid: i for i, sid in enumerate(graph_nodes)}
+    membership: dict[str, int] = {}
+    try:
+        import igraph as ig
+        import leidenalg
+
+        _ping(f"{len(graph_nodes)} connected symbols, running leiden")
+        g = ig.Graph(directed=False)
+        g.add_vertices(len(graph_nodes))
+        g.add_edges([(index_of[src], index_of[dst]) for src, dst in method_edges if src in index_of and dst in index_of])
+        part = leidenalg.find_partition(g, leidenalg.ModularityVertexPartition)
+        for idx, cid in enumerate(part.membership):
+            membership[graph_nodes[idx]] = int(cid)
+    except Exception:
+        _ping("leiden unavailable, using package fallback")
+        return {}
+    return membership
+
+
+def detect_communities(store, progress=None) -> list[dict]:
+    def _ping(msg: str) -> None:
+        if progress:
+            progress(msg)
+
+    _ping("loading symbols")
+    symbols = store.query_records(
+        """
+        MATCH (s:Symbol)
+        RETURN s.id as id, s.kind as kind, s.fqname as fqname, s.file_id as file_id
+        """
+    )
+    if not symbols:
+        return []
+
+    symbol_meta = {s["id"]: s for s in symbols}
+    method_symbols_by_key: dict[tuple[str, str], str] = {}
+    class_symbols_by_key: dict[tuple[str, str], str] = {}
+    for symbol in symbols:
+        key = (symbol.get("file_id", ""), symbol.get("fqname", ""))
+        if symbol.get("kind") == "method":
+            method_symbols_by_key[key] = symbol["id"]
+        elif symbol.get("kind") == "class":
+            class_symbols_by_key[key] = symbol["id"]
+
+    _ping("loading methods")
+    method_rows = store.query_records(
+        """
+        MATCH (m:Method), (c:Class)
+        WHERE m.class_id = c.id
+        RETURN m.id as method_id, c.file_id as file_id, c.fqcn as class_fqcn, m.signature as signature
+        """
+    )
+    method_symbol_ids: dict[str, str] = {}
+    graph_edges: set[tuple[str, str]] = set()
+    for row in method_rows:
+        file_id = row.get("file_id", "")
+        fqname = f"{row.get('class_fqcn', '')}#{row.get('signature', '')}"
+        method_symbol_id = method_symbols_by_key.get((file_id, fqname))
+        if not method_symbol_id:
+            continue
+        method_symbol_ids[row["method_id"]] = method_symbol_id
+        class_symbol_id = class_symbols_by_key.get((file_id, row.get("class_fqcn", "")))
+        if class_symbol_id and class_symbol_id != method_symbol_id:
+            graph_edges.add(tuple(sorted((method_symbol_id, class_symbol_id))))
+
+    _ping("loading call edges")
+    call_rows = store.query_records(
+        """
+        MATCH (a:Method)-[:CALLS]->(b:Method)
+        RETURN a.id as src, b.id as dst
+        """
+    )
+    for row in call_rows:
+        src = method_symbol_ids.get(row.get("src", ""))
+        dst = method_symbol_ids.get(row.get("dst", ""))
+        if src and dst and src != dst:
+            graph_edges.add(tuple(sorted((src, dst))))
+
+    _ping(f"{len(symbols)} symbols, {len(graph_edges)} structural edges")
+    membership = _call_graph_communities(symbol_meta, sorted(graph_edges), progress=progress)
+
+    grouped: dict[str, list[str]] = defaultdict(list)
+    next_fallback_id = 1000000
+
+    # Keep only meaningful graph communities; tiny ones get merged by package bucket below.
+    temp_grouped: dict[int, list[str]] = defaultdict(list)
+    for sid, cid in membership.items():
+        temp_grouped[cid].append(sid)
+
+    for cid, members in temp_grouped.items():
+        if len(members) >= MIN_COMMUNITY_SIZE:
+            grouped[f"graph:{cid}"].extend(members)
+        else:
+            for sid in members:
+                grouped[f"pkg:{_package_bucket(symbol_meta[sid].get('fqname', ''))}"].append(sid)
+
+    for sid, meta in symbol_meta.items():
+        if sid in membership:
+            continue
+        grouped[f"pkg:{_package_bucket(meta.get('fqname', ''))}"].append(sid)
+
+    # Filter out residual singletons. They are not useful architectural communities.
+    filtered = {cid: members for cid, members in grouped.items() if len(members) >= MIN_COMMUNITY_SIZE}
+    if not filtered:
+        # Last resort: put everything into one broad bucket so callers still get context.
+        cid = f"fallback:{next_fallback_id}"
+        filtered[cid] = list(symbol_meta.keys())
+
+    _ping(f"{len(filtered)} clusters, replacing previous communities")
+    store.clear_communities()
+
+    communities: list[dict] = []
+    total_clusters = len(filtered)
+    for idx, (cid, symbol_ids) in enumerate(sorted(filtered.items()), start=1):
+        label = _community_label(symbol_ids, symbol_meta)
+        cohesion = min(1.0, len(symbol_ids) / max(len(symbol_meta), 1))
+        store.set_community(cid, label, cohesion, symbol_ids)
+        if idx % 100 == 0 or idx == total_clusters:
+            _ping(f"persisting {idx}/{total_clusters} clusters")
+        communities.append(
+            {
+                "community_id": cid,
+                "label": label,
+                "cohesion": cohesion,
+                "size": len(symbol_ids),
+            }
+        )
+
+    communities.sort(key=lambda c: (c["size"], c["label"]), reverse=True)
+    return communities
+
+
+def symbol_community(store, symbol_query: str) -> dict:
+    recs = store.query_records(
+        """
+        MATCH (s:Symbol)-[:IN_COMMUNITY]->(c:Community)
+        WHERE s.id = $q OR lower(s.fqname) = lower($q) OR lower(s.name) = lower($q)
+        RETURN s.id as symbol_id, s.fqname as fqname, c.id as community_id, c.label as label, c.cohesion as cohesion
+        LIMIT 20
+        """,
+        {"q": symbol_query},
+    )
+    return {"query": symbol_query, "matches": recs}