code-graph-rag 0.0.345__tar.gz → 0.0.421__tar.gz

{code_graph_rag-0.0.345/code_graph_rag.egg-info → code_graph_rag-0.0.421}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: code-graph-rag
-Version: 0.0.345
+Version: 0.0.421
 Summary: The ultimate RAG for your monorepo. Query, understand, and edit multi-language codebases with the power of AI and knowledge graphs
 License-Expression: MIT
 Keywords: rag,retrieval-augmented-generation,knowledge-graph,code-analysis,tree-sitter,mcp,mcp-server,llm,graph-database,semantic-search,codebase,memgraph,developer-tools,monorepo
@@ -19,7 +19,7 @@ Requires-Python: >=3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: loguru>=0.7.3
-Requires-Dist: mcp>=1.25.0
+Requires-Dist: mcp>=1.28.1
 Requires-Dist: pydantic-ai>=1.102.0
 Requires-Dist: pydantic-settings>=2.12.0
 Requires-Dist: pymgclient>=1.5.1
@@ -47,6 +47,8 @@ Requires-Dist: pytest-xdist>=3.8.0; extra == "test"
 Requires-Dist: testcontainers>=4.9.0; extra == "test"
 Requires-Dist: libclang>=18.1.1; extra == "test"
 Requires-Dist: filelock>=3.12; extra == "test"
+Requires-Dist: ast-grep-py>=0.44.0; extra == "test"
+Requires-Dist: pyyaml>=6.0; extra == "test"
 Provides-Extra: treesitter-full
 Requires-Dist: tree-sitter-python>=0.23.6; extra == "treesitter-full"
 Requires-Dist: tree-sitter-javascript>=0.23.1; extra == "treesitter-full"
@@ -65,11 +67,16 @@ Provides-Extra: semantic
 Requires-Dist: qdrant-client>=1.9.0; extra == "semantic"
 Requires-Dist: torch>=2.6.0; extra == "semantic"
 Requires-Dist: transformers>=4.0.0; extra == "semantic"
+Provides-Extra: milvus
+Requires-Dist: pymilvus[milvus-lite]>=3.0.0; extra == "milvus"
+Provides-Extra: ast-grep
+Requires-Dist: ast-grep-py>=0.44.0; extra == "ast-grep"
+Requires-Dist: pyyaml>=6.0; extra == "ast-grep"
 Dynamic: license-file
 
 # Code-Graph-RAG
 
-A graph-based RAG system that parses multi-language codebases with Tree-sitter, builds knowledge graphs in Memgraph, and enables natural language querying, editing, and optimization.
+A graph-based RAG system that parses multi-language codebases with Tree-sitter, builds knowledge graphs in Memgraph, and enables natural language querying, editing, and optimisation.
 
 ## Install
 
@@ -89,6 +96,16 @@ With semantic code search (UniXcoder embeddings):
 pip install 'code-graph-rag[semantic]'
 ```
 
+Qdrant is the default vector store for semantic search. To use Milvus Lite,
+install `code-graph-rag[semantic,milvus]`, then set
+`CGR_VECTOR_STORE_BACKEND=milvus` and `MILVUS_URI=./.milvus_code_embeddings.db`
+before indexing.
+
+To compute embeddings on an OpenAI-compatible endpoint (OpenAI, Ollama, vLLM)
+instead of locally, set `CGR_EMBEDDING_PROVIDER=openai` with
+`OPENAI_EMBEDDING_BASE_URL` and `OPENAI_EMBEDDING_MODEL`; torch and
+transformers are then not required locally.
+
 ### Prerequisites
 
 - Python 3.12+
@@ -120,7 +137,7 @@ cgr index -o ./index-output --repo-path ./my-project
 cgr export -o graph.json
 ```
 
-**AI-guided optimization:**
+**AI-guided optimisation:**
 
 ```bash
 cgr optimize python --repo-path ./my-project

{code_graph_rag-0.0.345 → code_graph_rag-0.0.421}/PYPI_README.md

@@ -1,6 +1,6 @@
 # Code-Graph-RAG
 
-A graph-based RAG system that parses multi-language codebases with Tree-sitter, builds knowledge graphs in Memgraph, and enables natural language querying, editing, and optimization.
+A graph-based RAG system that parses multi-language codebases with Tree-sitter, builds knowledge graphs in Memgraph, and enables natural language querying, editing, and optimisation.
 
 ## Install
 
@@ -20,6 +20,16 @@ With semantic code search (UniXcoder embeddings):
 pip install 'code-graph-rag[semantic]'
 ```
 
+Qdrant is the default vector store for semantic search. To use Milvus Lite,
+install `code-graph-rag[semantic,milvus]`, then set
+`CGR_VECTOR_STORE_BACKEND=milvus` and `MILVUS_URI=./.milvus_code_embeddings.db`
+before indexing.
+
+To compute embeddings on an OpenAI-compatible endpoint (OpenAI, Ollama, vLLM)
+instead of locally, set `CGR_EMBEDDING_PROVIDER=openai` with
+`OPENAI_EMBEDDING_BASE_URL` and `OPENAI_EMBEDDING_MODEL`; torch and
+transformers are then not required locally.
+
 ### Prerequisites
 
 - Python 3.12+
@@ -51,7 +61,7 @@ cgr index -o ./index-output --repo-path ./my-project
 cgr export -o graph.json
 ```
 
-**AI-guided optimization:**
+**AI-guided optimisation:**
 
 ```bash
 cgr optimize python --repo-path ./my-project

code_graph_rag-0.0.421/README.md

@@ -0,0 +1,189 @@
+<div align="center">
+  <!-- Bitbucket strips <picture>/<source> tags, so we use a single light-mode <img>. Restore the theme-aware <picture> block below when the GitHub account is reinstated:
+  <picture>
+    <source srcset="assets/logo-dark-any.png" media="(prefers-color-scheme: dark)">
+    <source srcset="assets/logo-light-any.png" media="(prefers-color-scheme: light)">
+    <img src="assets/logo-dark-any.png" alt="Code-Graph-RAG Logo" width="480">
+  </picture>
+  -->
+  <img src="assets/logo-light-any.png" alt="Code-Graph-RAG Logo" width="480">
+
+  <p>
+  <!-- Badges below are commented out while the GitHub account is suspended. Restore them when the account is reinstated.
+       Stars/Forks: shields.io hits GitHub's API (returns "repo not found" for suspended accounts).
+       gitcgr: indexes from GitHub (shows "not indexed" while unavailable).
+       MseeP.ai: badge PNG ignores inline height on Bitbucket and renders as a full-size tile.
+  <a href="https://github.com/vitali87/code-graph-rag/stargazers">
+    <img src="https://img.shields.io/github/stars/vitali87/code-graph-rag?style=social" alt="GitHub stars" />
+  </a>
+  <a href="https://github.com/vitali87/code-graph-rag/network/members">
+    <img src="https://img.shields.io/github/forks/vitali87/code-graph-rag?style=social" alt="GitHub forks" />
+  </a>
+  -->
+  <!-- Codecov, SonarCloud, and OpenSSF Scorecard badges are fed by GitHub-side CI. Uncomment once that CI is confirmed to publish to these services again.
+  <a href="https://codecov.io/gh/vitali87/code-graph-rag">
+    <img src="https://codecov.io/gh/vitali87/code-graph-rag/graph/badge.svg" alt="Codecov" />
+  </a>
+  <a href="https://sonarcloud.io/summary/overall?id=vitali87_code-graph-rag">
+    <img src="https://sonarcloud.io/api/project_badges/measure?project=vitali87_code-graph-rag&metric=alert_status" alt="Quality Gate Status" />
+  </a>
+  -->
+  <!--
+  <a href="https://mseep.ai/app/vitali87-code-graph-rag">
+    <img src="https://mseep.net/pr/vitali87-code-graph-rag-badge.png" alt="MseeP.ai Security Assessment" height="20" />
+  </a>
+  -->
+  <a href="https://code-graph-rag.com">
+    <img src="https://img.shields.io/badge/Enterprise-Support%20%26%20Services-6366f1" alt="Enterprise Support" />
+  </a>
+  <a href="https://pepy.tech/projects/code-graph-rag">
+    <img src="https://static.pepy.tech/personalized-badge/code-graph-rag?period=total&units=INTERNATIONAL_SYSTEM&left_color=BLACK&right_color=GREEN&left_text=downloads" alt="PyPI Downloads" />
+  </a>
+  <!-- OpenSSF Scorecard only tracks GitHub-hosted repos. Uncomment once the Scorecard workflow is confirmed running on the authoritative GitHub repo.
+  <a href="https://scorecard.dev/viewer/?uri=github.com/vitali87/code-graph-rag">
+    <img src="https://api.scorecard.dev/projects/github.com/vitali87/code-graph-rag/badge" alt="OpenSSF Scorecard" />
+  </a>
+  -->
+  <!--
+  <a href="https://gitcgr.com/vitali87/code-graph-rag">
+    <img src="https://gitcgr.com/badge/vitali87/code-graph-rag.svg" alt="gitcgr" />
+  </a>
+  -->
+</p>
+</div>
+
+# Code-Graph-RAG
+
+Code-Graph-RAG parses a multi-language codebase with Tree-sitter, builds a knowledge graph of its structure in Memgraph, and lets you query, edit, and optimise that code in plain English. It works across a monorepo of mixed languages under one unified graph schema.
+
+<p align="center">
+  <img src="./assets/demo.gif" alt="demo">
+</p>
+
+## Latest News 🔥
+
+<!-- SECTION:latest_news -->
+- **Ruby Support**: Ruby joins the graph through a new pluggable ast-grep tier that adds a language from a single YAML pattern file, emitting `Module`, `Function`, and `Class` nodes plus import edges without a hand-written parser.
+- **Structural Search & Replace**: Find and rewrite code by AST pattern with ast-grep, exposed as agent tools so you can match and transform structure across the whole codebase instead of relying on text or regex.
+- **Data-Flow Tracing**: New `FLOWS_TO` taint edges follow values through assignments, function calls, and I/O sinks, with coverage across C#, Java, C, and Go.
+<!-- /SECTION:latest_news -->
+
+See [NEWS.md](NEWS.md) for the full history.
+
+## What It Does
+
+Point Code-Graph-RAG at a repository and it reads every source file, extracts functions, classes, methods, modules, and the relationships between them, and stores the result as an interconnected graph. Once the graph exists you can:
+
+- Ask questions about the codebase in natural language and get answers grounded in the real structure.
+- Retrieve the actual source of any function, class, or method by name or by intent.
+- Edit code through the agent with AST-based surgical patching and a diff preview before anything changes.
+- Optimise code against language best practices or your own coding standards.
+- Find dead code by walking call and reference edges from entry points.
+- Search and rewrite structurally by AST pattern with ast-grep.
+
+## How It Works
+
+The system has two components:
+
+1. **Multi-language parser.** A Tree-sitter based parser reads the codebase and ingests functions, classes, methods, modules, and their relationships into Memgraph under a single language-agnostic schema.
+2. **RAG system** (`codebase_rag/`). An interactive CLI that turns natural language into Cypher queries, retrieves matching code, and drives AI-powered editing and optimisation.
+
+```
+Source Code -> Tree-sitter Parser -> AST Analysis -> Memgraph Knowledge Graph
+                                                             |
+User Query -> AI Model (Cypher Gen) -> Cypher Query -> Graph Results -> Response
+```
+
+See the [Architecture Overview](docs/architecture/overview.md) and [Graph Schema](docs/architecture/graph-schema.md) for the full picture.
+
+## Supported Languages
+
+Python, TypeScript, TSX, JavaScript, Rust, Go, Java, C, C++, C#, PHP, Lua, and Dart are fully supported. Scala is in development. See the [Language Support](docs/architecture/language-support.md) matrix for per-language capabilities.
+
+## Installation
+
+`cgr` is published to PyPI. Install it system-wide with the `treesitter-full` (all languages) and `semantic` (vector search) extras:
+
+```bash
+# with uv (recommended)
+uv tool install "code-graph-rag[treesitter-full,semantic]"
+
+# or with pipx
+pipx install "code-graph-rag[treesitter-full,semantic]"
+```
+
+You also need Docker (for Memgraph), `cmake`, and `ripgrep`. Full prerequisites, source installs, and environment setup are in the [Installation](docs/getting-started/installation.md) guide.
+
+## Quick Start
+
+```bash
+# Start the packaged Memgraph + Qdrant stack (no compose file needed)
+cgr daemon up
+
+# Parse a repository into the graph, then query it
+cgr start --repo-path /path/to/repo --update-graph --clean
+cgr start --repo-path /path/to/repo
+```
+
+The [Quick Start](docs/getting-started/quickstart.md) guide walks through parsing, querying, and exporting in five minutes.
+
+## MCP Server
+
+Code-Graph-RAG runs as an [MCP](https://modelcontextprotocol.io) server so Claude Code and other MCP clients can query and edit your codebase directly. See the [MCP Server](docs/guide/mcp-server.md) guide for setup.
+
+## Documentation
+
+**Getting Started**
+- [Installation](docs/getting-started/installation.md)
+- [Quick Start](docs/getting-started/quickstart.md)
+- [Configuration](docs/getting-started/configuration.md)
+
+**User Guide**
+- [CLI Reference](docs/guide/cli-reference.md)
+- [Interactive Querying](docs/guide/interactive-querying.md)
+- [Code Optimisation](docs/guide/code-optimization.md)
+- [Dead Code Detection](docs/guide/dead-code.md)
+- [Graph Export](docs/guide/graph-export.md)
+- [Real-Time Updates](docs/guide/realtime-updates.md)
+- [MCP Server](docs/guide/mcp-server.md)
+
+**Architecture**
+- [Overview](docs/architecture/overview.md)
+- [Graph Schema](docs/architecture/graph-schema.md)
+- [Language Support](docs/architecture/language-support.md)
+- [Data-Flow Edges](docs/architecture/data-flow-edges.md)
+
+**Python SDK**
+- [Overview](docs/sdk/overview.md)
+- [Graph Loader](docs/sdk/graph-loader.md)
+- [Cypher Generator](docs/sdk/cypher-generator.md)
+- [Semantic Search](docs/sdk/semantic-search.md)
+
+**Advanced**
+- [Adding Languages](docs/advanced/adding-languages.md)
+- [Ignore Patterns](docs/advanced/ignore-patterns.md)
+- [Building Binaries](docs/advanced/building-binaries.md)
+- [Troubleshooting](docs/advanced/troubleshooting.md)
+
+## Enterprise Services
+
+Code-Graph-RAG is open source and free to use. For organisations that need more, we offer **fully managed cloud-hosted solutions** and **on-premise deployments**:
+
+- **Cloud-Hosted Deployment**: Managed cloud infrastructure for both the graph database and the AI agent connection. Zero infrastructure overhead, so we handle scaling, updates, and availability while your team focuses on building.
+- **On-Premise & Air-Gapped Deployment**: Deploy Code-Graph-RAG entirely within your own environment, including air-gapped networks. Full data sovereignty for regulated industries and security-sensitive organisations.
+
+We also offer custom development, integration consulting, technical support contracts, and team training.
+
+**[View plans & pricing at code-graph-rag.com](https://code-graph-rag.com/enterprise)**
+
+## Contributing
+
+Please see [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines. Good first PRs come from the TODO issues.
+
+## Support
+
+For issues or questions, check the [Troubleshooting](docs/advanced/troubleshooting.md) guide first, then open an issue.
+
+## License
+
+MIT. See [LICENSE](LICENSE).

{code_graph_rag-0.0.345 → code_graph_rag-0.0.421/code_graph_rag.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: code-graph-rag
-Version: 0.0.345
+Version: 0.0.421
 Summary: The ultimate RAG for your monorepo. Query, understand, and edit multi-language codebases with the power of AI and knowledge graphs
 License-Expression: MIT
 Keywords: rag,retrieval-augmented-generation,knowledge-graph,code-analysis,tree-sitter,mcp,mcp-server,llm,graph-database,semantic-search,codebase,memgraph,developer-tools,monorepo
@@ -19,7 +19,7 @@ Requires-Python: >=3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: loguru>=0.7.3
-Requires-Dist: mcp>=1.25.0
+Requires-Dist: mcp>=1.28.1
 Requires-Dist: pydantic-ai>=1.102.0
 Requires-Dist: pydantic-settings>=2.12.0
 Requires-Dist: pymgclient>=1.5.1
@@ -47,6 +47,8 @@ Requires-Dist: pytest-xdist>=3.8.0; extra == "test"
 Requires-Dist: testcontainers>=4.9.0; extra == "test"
 Requires-Dist: libclang>=18.1.1; extra == "test"
 Requires-Dist: filelock>=3.12; extra == "test"
+Requires-Dist: ast-grep-py>=0.44.0; extra == "test"
+Requires-Dist: pyyaml>=6.0; extra == "test"
 Provides-Extra: treesitter-full
 Requires-Dist: tree-sitter-python>=0.23.6; extra == "treesitter-full"
 Requires-Dist: tree-sitter-javascript>=0.23.1; extra == "treesitter-full"
@@ -65,11 +67,16 @@ Provides-Extra: semantic
 Requires-Dist: qdrant-client>=1.9.0; extra == "semantic"
 Requires-Dist: torch>=2.6.0; extra == "semantic"
 Requires-Dist: transformers>=4.0.0; extra == "semantic"
+Provides-Extra: milvus
+Requires-Dist: pymilvus[milvus-lite]>=3.0.0; extra == "milvus"
+Provides-Extra: ast-grep
+Requires-Dist: ast-grep-py>=0.44.0; extra == "ast-grep"
+Requires-Dist: pyyaml>=6.0; extra == "ast-grep"
 Dynamic: license-file
 
 # Code-Graph-RAG
 
-A graph-based RAG system that parses multi-language codebases with Tree-sitter, builds knowledge graphs in Memgraph, and enables natural language querying, editing, and optimization.
+A graph-based RAG system that parses multi-language codebases with Tree-sitter, builds knowledge graphs in Memgraph, and enables natural language querying, editing, and optimisation.
 
 ## Install
 
@@ -89,6 +96,16 @@ With semantic code search (UniXcoder embeddings):
 pip install 'code-graph-rag[semantic]'
 ```
 
+Qdrant is the default vector store for semantic search. To use Milvus Lite,
+install `code-graph-rag[semantic,milvus]`, then set
+`CGR_VECTOR_STORE_BACKEND=milvus` and `MILVUS_URI=./.milvus_code_embeddings.db`
+before indexing.
+
+To compute embeddings on an OpenAI-compatible endpoint (OpenAI, Ollama, vLLM)
+instead of locally, set `CGR_EMBEDDING_PROVIDER=openai` with
+`OPENAI_EMBEDDING_BASE_URL` and `OPENAI_EMBEDDING_MODEL`; torch and
+transformers are then not required locally.
+
 ### Prerequisites
 
 - Python 3.12+
@@ -120,7 +137,7 @@ cgr index -o ./index-output --repo-path ./my-project
 cgr export -o graph.json
 ```
 
-**AI-guided optimization:**
+**AI-guided optimisation:**
 
 ```bash
 cgr optimize python --repo-path ./my-project

{code_graph_rag-0.0.345 → code_graph_rag-0.0.421}/code_graph_rag.egg-info/SOURCES.txt

@@ -38,6 +38,20 @@ codebase_rag/tool_errors.py
 codebase_rag/types_defs.py
 codebase_rag/unixcoder.py
 codebase_rag/vector_store.py
+codebase_rag/analyzers/__init__.py
+codebase_rag/analyzers/ast_grep_analyzer.py
+codebase_rag/analyzers/ast_grep_rules/patterns/javascript.yaml
+codebase_rag/analyzers/ast_grep_rules/patterns/python.yaml
+codebase_rag/analyzers/ast_grep_rules/patterns/tsx.yaml
+codebase_rag/analyzers/ast_grep_rules/patterns/typescript.yaml
+codebase_rag/analyzers/ast_grep_rules/security/javascript.yaml
+codebase_rag/analyzers/ast_grep_rules/security/python.yaml
+codebase_rag/analyzers/ast_grep_rules/security/tsx.yaml
+codebase_rag/analyzers/ast_grep_rules/security/typescript.yaml
+codebase_rag/analyzers/ast_grep_rules/smells/javascript.yaml
+codebase_rag/analyzers/ast_grep_rules/smells/python.yaml
+codebase_rag/analyzers/ast_grep_rules/smells/tsx.yaml
+codebase_rag/analyzers/ast_grep_rules/smells/typescript.yaml
 codebase_rag/constants/__init__.py
 codebase_rag/constants/ast_cpp.py
 codebase_rag/constants/ast_csharp.py
@@ -65,11 +79,13 @@ codebase_rag/constants/mcp.py
 codebase_rag/constants/providers.py
 codebase_rag/constants/security.py
 codebase_rag/constants/stdlib_types.py
+codebase_rag/constants/structural.py
 codebase_rag/mcp/__init__.py
 codebase_rag/mcp/client.py
 codebase_rag/mcp/server.py
 codebase_rag/mcp/tools.py
 codebase_rag/parsers/__init__.py
+codebase_rag/parsers/ast_grep_tier.py
 codebase_rag/parsers/call_processor.py
 codebase_rag/parsers/call_resolver.py
 codebase_rag/parsers/definition_processor.py
@@ -83,6 +99,7 @@ codebase_rag/parsers/stdlib_extractor.py
 codebase_rag/parsers/structure_processor.py
 codebase_rag/parsers/type_inference.py
 codebase_rag/parsers/utils.py
+codebase_rag/parsers/ast_grep_patterns/ruby.yaml
 codebase_rag/parsers/class_ingest/__init__.py
 codebase_rag/parsers/class_ingest/cpp_modules.py
 codebase_rag/parsers/class_ingest/identity.py
@@ -103,7 +120,13 @@ codebase_rag/parsers/cpp_frontend/qn.py
 codebase_rag/parsers/csharp/__init__.py
 codebase_rag/parsers/csharp/type_inference.py
 codebase_rag/parsers/csharp/utils.py
+codebase_rag/parsers/csharp_frontend/__init__.py
+codebase_rag/parsers/csharp_frontend/frontend.py
+codebase_rag/parsers/csharp_frontend/roslyn/Frontend.cs
+codebase_rag/parsers/csharp_frontend/roslyn/Frontend.csproj
+codebase_rag/parsers/csharp_frontend/roslyn/Program.cs
 codebase_rag/parsers/dart/__init__.py
+codebase_rag/parsers/dart/type_inference.py
 codebase_rag/parsers/dart/utils.py
 codebase_rag/parsers/flow_access/__init__.py
 codebase_rag/parsers/flow_access/constants.py
@@ -168,6 +191,7 @@ codebase_rag/stack/constants.py
 codebase_rag/stack/health.py
 codebase_rag/stack/manager.py
 codebase_rag/tools/__init__.py
+codebase_rag/tools/ast_grep_service.py
 codebase_rag/tools/code_retrieval.py
 codebase_rag/tools/codebase_query.py
 codebase_rag/tools/directory_lister.py
@@ -178,6 +202,8 @@ codebase_rag/tools/health_checker.py
 codebase_rag/tools/language.py
 codebase_rag/tools/semantic_search.py
 codebase_rag/tools/shell_command.py
+codebase_rag/tools/structural_editor.py
+codebase_rag/tools/structural_search.py
 codebase_rag/tools/tool_descriptions.py
 codebase_rag/utils/__init__.py
 codebase_rag/utils/dependencies.py

{code_graph_rag-0.0.345 → code_graph_rag-0.0.421}/code_graph_rag.egg-info/requires.txt

@@ -1,5 +1,5 @@
 loguru>=0.7.3
-mcp>=1.25.0
+mcp>=1.28.1
 pydantic-ai>=1.102.0
 pydantic-settings>=2.12.0
 pymgclient>=1.5.1
@@ -20,6 +20,13 @@ huggingface-hub[hf-xet]>=1.7.2
 griffe<2,>=1.0
 pathspec>=1.0.4
 
+[ast-grep]
+ast-grep-py>=0.44.0
+pyyaml>=6.0
+
+[milvus]
+pymilvus[milvus-lite]>=3.0.0
+
 [semantic]
 qdrant-client>=1.9.0
 torch>=2.6.0
@@ -33,6 +40,8 @@ pytest-xdist>=3.8.0
 testcontainers>=4.9.0
 libclang>=18.1.1
 filelock>=3.12
+ast-grep-py>=0.44.0
+pyyaml>=6.0
 
 [treesitter-full]
 tree-sitter-python>=0.23.6

code_graph_rag-0.0.421/codebase_rag/analyzers/__init__.py

@@ -0,0 +1,4 @@
+# ast-grep finding analyzers (issue #413).
+from .ast_grep_analyzer import FindingAnalyzer, load_finding_rules
+
+__all__ = ["FindingAnalyzer", "load_finding_rules"]

code_graph_rag-0.0.421/codebase_rag/analyzers/ast_grep_analyzer.py

@@ -0,0 +1,201 @@
+# ast-grep finding analyzer (issue #413). A post-pass over indexed source
+# files that runs categorized ast-grep YAML rules and emits
+# Pattern/CodeSmell/SecurityIssue nodes linked to each file's Module. Rules
+# live in ast_grep_rules/{patterns,smells,security}/<lang>.yaml; a new rule is
+# a YAML entry, no code. The FINDINGS capture group is opt-in, so the analyzer
+# no-ops entirely unless a finding relationship is enabled. Findings link to
+# the Module (not the enclosing Class/Function); the finding node carries the
+# line so the site is still locatable. Symbol-level linkage is a follow-up.
+from __future__ import annotations
+
+import logging
+from dataclasses import dataclass
+from pathlib import Path
+from typing import TYPE_CHECKING, Any
+
+from .. import constants as cs
+from ..utils.path_utils import cached_relative_path
+
+if TYPE_CHECKING:
+    from ..capture import CaptureSelection
+    from ..services import IngestorProtocol
+
+logger = logging.getLogger(__name__)
+
+_RULES_DIR = Path(__file__).parent / "ast_grep_rules"
+_SNIPPET_MAX = 200
+
+# rule-category directory name -> (finding node label, relationship type).
+_CATEGORY_MAP: dict[str, tuple[cs.NodeLabel, cs.RelationshipType]] = {
+    "patterns": (cs.NodeLabel.PATTERN, cs.RelationshipType.IMPLEMENTS_PATTERN),
+    "smells": (cs.NodeLabel.CODE_SMELL, cs.RelationshipType.HAS_SMELL),
+    "security": (cs.NodeLabel.SECURITY_ISSUE, cs.RelationshipType.HAS_VULNERABILITY),
+}
+
+
+@dataclass(frozen=True)
+class _Rule:
+    rule_id: str
+    message: str
+    body: dict[str, Any]  # ast-grep rule body, splatted into find_all(**body)
+    node_label: cs.NodeLabel
+    rel_type: cs.RelationshipType
+
+
+@dataclass(frozen=True)
+class _LangRules:
+    ast_grep_id: str
+    rules: tuple[_Rule, ...]
+
+
+def load_finding_rules() -> dict[str, _LangRules]:
+    """Load ast_grep_rules/<category>/*.yaml, merged per file extension."""
+    lang_id_by_ext: dict[str, str] = {}
+    rules_by_ext: dict[str, list[_Rule]] = {}
+    for category_dir in sorted(_RULES_DIR.iterdir()):
+        mapping = _CATEGORY_MAP.get(category_dir.name)
+        if not category_dir.is_dir() or mapping is None:
+            continue
+        node_label, rel_type = mapping
+        for path in sorted(category_dir.glob("*.yaml")):
+            ast_grep_id, extensions, rules = _parse_rule_file(
+                path, node_label, rel_type
+            )
+            for ext in extensions:
+                lang_id_by_ext[ext] = ast_grep_id
+                rules_by_ext.setdefault(ext, []).extend(rules)
+    return {
+        ext: _LangRules(lang_id_by_ext[ext], tuple(rules))
+        for ext, rules in rules_by_ext.items()
+    }
+
+
+def _parse_rule_file(
+    path: Path, node_label: cs.NodeLabel, rel_type: cs.RelationshipType
+) -> tuple[str, list[str], list[_Rule]]:
+    import yaml
+
+    data = yaml.safe_load(path.read_text(encoding="utf-8")) or {}
+    ast_grep_id = data.get("ast_grep_id")
+    extensions = data.get("extensions")
+    if isinstance(extensions, str):
+        extensions = [ext.strip() for ext in extensions.split(",") if ext.strip()]
+    if not ast_grep_id or not extensions:
+        raise ValueError(f"{path.name}: 'ast_grep_id' and 'extensions' are required")
+    rules = [
+        _Rule(
+            rule_id=str(entry["id"]),
+            message=str(entry.get("message", entry["id"])),
+            body=entry["rule"],
+            node_label=node_label,
+            rel_type=rel_type,
+        )
+        for entry in (data.get("rules") or [])
+    ]
+    return str(ast_grep_id), list(extensions), rules
+
+
+class FindingAnalyzer:
+    """Runs ast-grep finding rules over indexed files and emits finding nodes."""
+
+    __slots__ = ("_ingestor", "_repo_path", "_enabled", "_rules")
+
+    def __init__(
+        self,
+        ingestor: IngestorProtocol,
+        repo_path: Path,
+        selection: CaptureSelection,
+    ) -> None:
+        self._ingestor = ingestor
+        self._repo_path = repo_path
+        finding_rels = cs.CAPTURE_GROUP_RELS[cs.CaptureGroup.FINDINGS]
+        self._enabled = any(selection.rel_enabled(rel) for rel in finding_rels)
+        self._rules: dict[str, _LangRules] = {}
+        if not self._enabled:
+            return
+        try:
+            import ast_grep_py  # noqa: F401
+
+            self._rules = load_finding_rules()
+        except ImportError:
+            # ast-grep/pyyaml are the [ast-grep] extra; no-op if absent.
+            logger.warning("ast-grep-py unavailable; finding analyzer disabled")
+        except Exception as exc:  # noqa: BLE001
+            # a malformed shipped rule file must not crash indexing.
+            logger.warning("ast-grep finding analyzer disabled: %s", exc)
+
+    def analyze(self, module_qn_to_file_path: dict[str, Path]) -> None:
+        if not self._enabled or not self._rules:
+            return
+        from ast_grep_py import SgRoot
+
+        for module_qn, file_path in module_qn_to_file_path.items():
+            lang_rules = self._rules.get(file_path.suffix)
+            if lang_rules is None:
+                continue
+            try:
+                raw = file_path.read_bytes()
+            except OSError:
+                continue
+            # decode with replacement so a few malformed bytes don't skip the
+            # whole file; ast-grep tolerates U+FFFD in the source.
+            source = raw.decode("utf-8", errors="replace")
+            try:
+                root = SgRoot(source, lang_rules.ast_grep_id).root()
+            except (RuntimeError, ValueError) as exc:
+                logger.warning("ast-grep failed to parse %s: %s", file_path, exc)
+                continue
+            relative_path = cached_relative_path(file_path, self._repo_path).as_posix()
+            for rule in lang_rules.rules:
+                self._emit_rule(root, rule, module_qn, relative_path, file_path)
+
+    def _emit_rule(
+        self,
+        root: Any,
+        rule: _Rule,
+        module_qn: str,
+        relative_path: str,
+        file_path: Path,
+    ) -> None:
+        try:
+            matches = root.find_all(**rule.body)
+        except (RuntimeError, TypeError, ValueError) as exc:
+            logger.warning(
+                "bad ast-grep rule %r for %s: %s", rule.rule_id, file_path, exc
+            )
+            return
+        for node in matches:
+            self._emit_finding(rule, node, module_qn, relative_path)
+
+    def _emit_finding(
+        self, rule: _Rule, node: Any, module_qn: str, relative_path: str
+    ) -> None:
+        node_range = node.range()
+        start_line = node_range.start.line + 1
+        end_line = node_range.end.line + 1
+        # qn scopes the finding to file+line+column+rule so two matches of one
+        # rule on the same line stay distinct, while re-indexing merges the
+        # same site idempotently.
+        qualified_name = cs.SEPARATOR_DOT.join(
+            [module_qn, str(start_line), str(node_range.start.column), rule.rule_id]
+        )
+        snippet = node.text()
+        if len(snippet) > _SNIPPET_MAX:
+            snippet = snippet[:_SNIPPET_MAX]
+        self._ingestor.ensure_node_batch(
+            rule.node_label,
+            {
+                cs.KEY_QUALIFIED_NAME: qualified_name,
+                cs.KEY_NAME: rule.rule_id,
+                cs.KEY_MESSAGE: rule.message,
+                cs.KEY_START_LINE: start_line,
+                cs.KEY_END_LINE: end_line,
+                cs.KEY_PATH: relative_path,
+                cs.KEY_SNIPPET: snippet,
+            },
+        )
+        self._ingestor.ensure_relationship_batch(
+            (cs.NodeLabel.MODULE, cs.KEY_QUALIFIED_NAME, module_qn),
+            rule.rel_type,
+            (rule.node_label, cs.KEY_QUALIFIED_NAME, qualified_name),
+        )