code-explore-by-sql 0.1.0__tar.gz

code_explore_by_sql-0.1.0/.gitignore

@@ -0,0 +1,11 @@
+.venv/
+__pycache__/
+.pytest_cache/
+.ruff_cache/
+*.pyc
+*.db
+*.db-shm
+*.db-wal
+.env
+.DS_Store
+.mcp.json

code_explore_by_sql-0.1.0/AGENTS.md

@@ -0,0 +1,82 @@
+# AGENTS.md
+
+This repository provides a local MCP server for source code retrieval using **SQLite FTS5** (trigram tokenizer) with a **bracket skeleton structural index**.
+
+## Architecture
+
+One file = one row in `source_files`. FTS5 `snippet()` extracts relevant code fragments via a **two-step deferred query** (rank first, snippet only for top-N, truncated to 300 chars), producing compact ~2,600 token responses for 20 results (95% token reduction vs full snippets).
+
+**Bracket skeleton index**: A 6-state FSM scans C/C++ source tracking brace pairs while ignoring braces in comments, strings, and raw string literals. Each brace pair records depth, open/close line, block type, block name, and `parent_id` (hierarchical parent-child relationships). This provides structural context without AST parsing — robust against macros and incomplete syntax.
+
+**Symbol references**: Pre-computed identifier references tracked in `symbol_references` table. Supports fast lookup of which files reference a given symbol, with enclosing block context.
+
+**Include dependency graph**: O(1) basename hash lookup resolves 96.5% of include paths. Supports upstream/downstream traversal with configurable recursion depth.
+
+**History-as-ranking-signal**: Past search feedback adjusts result ranking but never filters out results. This prevents confirmation bias while still accelerating relevant results.
+
+### Token cost quick reference
+
+| Operation | ~Tokens | Note |
+|-----------|---------|------|
+| `search_code_source` (20 results) | ~2,600 | Compact 300-char snippets |
+| `get_file_content(anchor=...)` | ~125 | **Always prefer** over full read |
+| `get_file_content` (full file) | ~45,000 | Avoid — use anchor or line range |
+| `find_references` (pre-computed) | 50–500 | **Try first** for symbol lookup |
+| `find_callers` (specific symbol) | 127–3,000 | Use `scope` for common symbols |
+| `find_include_graph` | 50–2,100 | Cheap — use freely |
+
+## Tools (7)
+
+1. **`search_code_source`** — FTS5 search with history ranking, scope filtering, compact snippets.
+   - Simple: `query="GetGBuffer"`
+   - Advanced: `raw_query='"GetGBuffer" AND "Emissive"'`
+   - `scope_filter` is a **dict**: `{"block_type": "function"}` (no JSON string needed)
+   - `module="Renderer"` — filter by module name
+
+2. **`get_file_content`** — Read file content. Prefer **anchor mode** for efficiency.
+   - Anchor: `anchor="Render", context_chars=500` (~125 tokens)
+   - Line range: `start_line=100, end_line=200`
+   - Auto-records feedback from search results
+
+3. **`log_code_query`** — Record explicit feedback (optional, only to correct automatic feedback)
+
+4. **`find_include_graph`** — Include dependency graph (upstream/downstream, recursive, depth control)
+
+5. **`find_callers`** — Dynamic caller lookup via FTS5 + full file scan + bracket matching.
+   - Returns exact `caller_line` per call site
+   - **Always use `scope`** for common symbols like "Render" (500+ callers otherwise)
+
+6. **`find_references`** — Pre-computed symbol references from `symbol_references` table.
+   - Fast lookup: `find_references("BeginPlay", limit=100)`
+   - Returns file paths and enclosing block info
+   - Falls back to empty list if table is not yet populated
+
+7. **`get_directory_structure`** — Discover valid module names and directory layout.
+   - Returns `modules` (top 30 by file count), `top_dirs`, `total_files`
+   - Use `module_name` as `module` param in `search_code_source` or `scope` in `find_callers`
+
+## Recommended flow
+
+1. `search_code_source` → compact snippets (~2,600 tok)
+2. `get_file_content(anchor=...)` → deep context (~125 tok each)
+3. `find_callers` / `find_include_graph` → structural exploration
+4. `log_code_query` → only to correct feedback
+
+## FTS5 Query Syntax (for raw_query)
+
+| Operator | Example |
+|----------|---------|
+| AND | `'"A" AND "B"'` |
+| OR | `'"A" OR "B"'` |
+| NOT | `'"A" NOT "B"'` |
+| Grouping | `'("A" OR "B") AND "C"'` |
+| Column filter | `'file_path : "BasePass"'` |
+
+All terms must be 3+ characters. NEAR and prefix (`*`) do NOT work with trigram tokenizer.
+
+## Guidance
+
+- Use the `code-source-lookup` skill for detailed tool documentation and search strategy
+- Avoid full file reads — anchor mode is 358x cheaper in tokens
+- History feedback is automatic — no need to manually log unless correcting
+- If the database has not been built yet, guide the user toward indexing first

code_explore_by_sql-0.1.0/CLAUDE.md

@@ -0,0 +1,15 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Working Rules
+
+### Plan Writing
+
+- Plans must be **complete and directly executable**. Every step must include concrete code, file paths, and parameter values. No TODO, TBD, or vague placeholders allowed.
+- Break tasks into **minimum independently executable steps**. Each step does exactly one thing, with clear structure and explicit ordering.
+
+### Code Refactoring
+
+- Refactored code must preserve **identical observable behavior** to the original. No changes to external API, output, or side effects.
+- Before and after refactoring, run the same tests or verification steps to confirm behavioral equivalence.

code_explore_by_sql-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 yanwei
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

code_explore_by_sql-0.1.0/MANIFEST.in

@@ -0,0 +1,2 @@
+include LICENSE
+include README.md

code_explore_by_sql-0.1.0/PKG-INFO

@@ -0,0 +1,205 @@
+Metadata-Version: 2.4
+Name: code-explore-by-sql
+Version: 0.1.0
+Summary: SQLite FTS5 (trigram) MCP server for code source search.
+Project-URL: Repository, https://github.com/didi514354875/code-explore-by-sql
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Code Generators
+Classifier: Topic :: Text Processing :: Indexing
+Requires-Python: >=3.10
+Requires-Dist: mcp[cli]>=1.2.0
+Requires-Dist: pydantic>=2.0
+Description-Content-Type: text/markdown
+
+# code-explore-by-sql
+
+Local stdio MCP server for fast source code navigation using **SQLite FTS5** (trigram tokenizer) + **bracket skeleton indexing**.
+
+## Features
+
+- **Full-text search**: FTS5 with trigram tokenizer for code-symbol-precise search (`GetGBuffer`, `FMaterial`, `UE_LOG`)
+- **Symbol lookup**: read code by qualified name with fuzzy matching (`Jump` → `ACharacter::Jump`)
+- **Bracket skeleton index**: lightweight structural indexing via FSM brace matching (no AST parser needed)
+- **12 language support**: C, C++, C#, Go, HLSL, GLSL, Java, JavaScript, Kotlin, Python, Rust, Swift
+- **Multi-database**: query multiple codebases simultaneously via `CODE_SOURCE_DBS`
+- **Token-efficient responses**: compact snippets (~2,600 tokens/20 results, 95% reduction vs full file reads)
+
+## Installation
+
+### From PyPI (recommended)
+
+```bash
+# Run the MCP server directly (no clone needed)
+uvx code-explore-by-sql
+
+# Or install persistently
+pip install code-explore-by-sql
+```
+
+### Build a database
+
+```bash
+# Build index for your codebase
+uvx code-source-sql-build-db /path/to/source /path/to/output.db
+
+# Smoke test with limited files
+uvx code-source-sql-build-db /path/to/source /path/to/output.db --limit 1000
+```
+
+Performance: ~84,700 files indexed in ~3.3 minutes on a 2-core machine.
+
+### Configure in MCP clients
+
+**Claude Code** (`.claude/mcp.json`):
+```json
+{
+  "mcpServers": {
+    "code-source-sql": {
+      "command": "uvx",
+      "args": ["code-explore-by-sql"],
+      "env": {
+        "CODE_SOURCE_DB": "/path/to/your/code.db",
+        "CODE_SOURCE_DBS": "/path/to/your/code.db:/path/to/another.db"
+      }
+    }
+  }
+}
+```
+
+**VS Code** (`.vscode/mcp.json`):
+```json
+{
+  "servers": {
+    "code-source-sql": {
+      "type": "stdio",
+      "command": "uvx",
+      "args": ["code-explore-by-sql"],
+      "env": {
+        "CODE_SOURCE_DB": "/path/to/your/code.db"
+      }
+    }
+  }
+}
+```
+
+**OpenAI Codex** (`~/.codex/config.toml`):
+```toml
+[mcp_servers.code-source-sql]
+command = "uvx"
+args = ["code-explore-by-sql"]
+
+[mcp_servers.code-source-sql.env]
+CODE_SOURCE_DB = "/path/to/your/code.db"
+```
+
+**Hermes Agent** (`~/.hermes/config.yaml`):
+```yaml
+mcp_servers:
+  code-source-sql:
+    command: uvx
+    args:
+      - code-explore-by-sql
+    env:
+      CODE_SOURCE_DB: /path/to/your/code.db
+```
+
+## Tools (5)
+
+| Tool | Purpose |
+|------|---------|
+| `list_databases` | Discover available databases with stats |
+| `search_fts_tool` | FTS5 search — locate code blocks by keyword or raw FTS5 query |
+| `read_symbol` | Read symbol code by qualified name (exact or fuzzy) |
+| `read_file_range` | Read source code by file path and line range |
+| `get_directory_structure` | Module/file counts overview |
+
+### Multi-database
+
+Each tool accepts an optional `db` parameter to select a database by alias. Aliases are derived from database filenames (`unreal.db` → `"unreal"`). Use `list_databases` to discover available aliases. Default (`db=""`) uses the primary database (`CODE_SOURCE_DB`).
+
+### Search query modes
+
+**Simple mode** (`keyword`):
+```
+keyword="GetGBuffer"
+keyword="FMaterial Render"
+```
+
+**Advanced mode** (`raw_query`) — full FTS5 boolean:
+```
+raw_query='"GetGBuffer" AND "Emissive"'
+raw_query='"Material" NOT "hlsl"'
+raw_query='(file_path : "BasePass") AND "roughness"'
+raw_query='(module_name : "Renderer") AND "VirtualTexture"'
+```
+
+### Three-level funnel
+
+1. **`search_fts_tool(keyword)`** → file candidates + block QNs
+2. **`search_fts_tool(raw_query, file_path filter)`** → precise block in target file
+3. **`read_symbol(block QN)`** or **`read_file_range(file, line)`** → full code
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────────────────────┐
+│                    MCP Server (FastMCP)                       │
+├──────────┬──────────┬──────────┬──────────┬──────────────────┤
+│ search   │ read     │ read     │ get_dir  │ list             │
+│ fts_tool │ _symbol  │ _file    │ _struct  │ _databases       │
+│          │          │ _range   │          │                  │
+├──────────┴──────────┴──────────┴──────────┴──────────────────┤
+│                     Query Pipeline                            │
+│  FTS5 trigram → Symbol match → Edge extraction               │
+├──────────────────────────────────────────────────────────────┤
+│                    SQLite Database                            │
+│  file_content + FTS5 │ symbol_index │ strict_edges           │
+└──────────────────────────────────────────────────────────────┘
+```
+
+### Bracket skeleton index
+
+A 6-state finite state machine (CODE, LINE_COMMENT, BLOCK_COMMENT, STRING, CHAR_LITERAL, RAW_STRING) scans source code tracking brace pairs while correctly ignoring braces in comments and string literals. Each matched pair records `open_line`, `close_line`, `depth`, and `is_complete`.
+
+Top-level blocks are classified by a **symbol analyzer** producing `block_type` (namespace/class/enum/function/macro) and `block_name` (qualified name).
+
+### Multi-database registry
+
+Databases are registered via environment variables at server startup:
+- `CODE_SOURCE_DB` — primary database (default when `db` is omitted)
+- `CODE_SOURCE_DBS` — colon-separated list of additional databases
+
+Aliases are auto-derived from filename stems. Connections are cached with health checks.
+
+## Environment Variables
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `CODE_SOURCE_DB` | Yes | Path to primary SQLite database |
+| `CODE_SOURCE_DBS` | No | Colon-separated paths to additional databases |
+
+## Development
+
+```bash
+# Clone and setup
+git clone https://github.com/didi514354875/code-explore-by-sql.git
+cd code-explore-by-sql
+uv sync --dev
+
+# Run tests
+uv run pytest
+uv run ruff check .
+```
+
+## License
+
+MIT

code_explore_by_sql-0.1.0/README.md

@@ -0,0 +1,183 @@
+# code-explore-by-sql
+
+Local stdio MCP server for fast source code navigation using **SQLite FTS5** (trigram tokenizer) + **bracket skeleton indexing**.
+
+## Features
+
+- **Full-text search**: FTS5 with trigram tokenizer for code-symbol-precise search (`GetGBuffer`, `FMaterial`, `UE_LOG`)
+- **Symbol lookup**: read code by qualified name with fuzzy matching (`Jump` → `ACharacter::Jump`)
+- **Bracket skeleton index**: lightweight structural indexing via FSM brace matching (no AST parser needed)
+- **12 language support**: C, C++, C#, Go, HLSL, GLSL, Java, JavaScript, Kotlin, Python, Rust, Swift
+- **Multi-database**: query multiple codebases simultaneously via `CODE_SOURCE_DBS`
+- **Token-efficient responses**: compact snippets (~2,600 tokens/20 results, 95% reduction vs full file reads)
+
+## Installation
+
+### From PyPI (recommended)
+
+```bash
+# Run the MCP server directly (no clone needed)
+uvx code-explore-by-sql
+
+# Or install persistently
+pip install code-explore-by-sql
+```
+
+### Build a database
+
+```bash
+# Build index for your codebase
+uvx code-source-sql-build-db /path/to/source /path/to/output.db
+
+# Smoke test with limited files
+uvx code-source-sql-build-db /path/to/source /path/to/output.db --limit 1000
+```
+
+Performance: ~84,700 files indexed in ~3.3 minutes on a 2-core machine.
+
+### Configure in MCP clients
+
+**Claude Code** (`.claude/mcp.json`):
+```json
+{
+  "mcpServers": {
+    "code-source-sql": {
+      "command": "uvx",
+      "args": ["code-explore-by-sql"],
+      "env": {
+        "CODE_SOURCE_DB": "/path/to/your/code.db",
+        "CODE_SOURCE_DBS": "/path/to/your/code.db:/path/to/another.db"
+      }
+    }
+  }
+}
+```
+
+**VS Code** (`.vscode/mcp.json`):
+```json
+{
+  "servers": {
+    "code-source-sql": {
+      "type": "stdio",
+      "command": "uvx",
+      "args": ["code-explore-by-sql"],
+      "env": {
+        "CODE_SOURCE_DB": "/path/to/your/code.db"
+      }
+    }
+  }
+}
+```
+
+**OpenAI Codex** (`~/.codex/config.toml`):
+```toml
+[mcp_servers.code-source-sql]
+command = "uvx"
+args = ["code-explore-by-sql"]
+
+[mcp_servers.code-source-sql.env]
+CODE_SOURCE_DB = "/path/to/your/code.db"
+```
+
+**Hermes Agent** (`~/.hermes/config.yaml`):
+```yaml
+mcp_servers:
+  code-source-sql:
+    command: uvx
+    args:
+      - code-explore-by-sql
+    env:
+      CODE_SOURCE_DB: /path/to/your/code.db
+```
+
+## Tools (5)
+
+| Tool | Purpose |
+|------|---------|
+| `list_databases` | Discover available databases with stats |
+| `search_fts_tool` | FTS5 search — locate code blocks by keyword or raw FTS5 query |
+| `read_symbol` | Read symbol code by qualified name (exact or fuzzy) |
+| `read_file_range` | Read source code by file path and line range |
+| `get_directory_structure` | Module/file counts overview |
+
+### Multi-database
+
+Each tool accepts an optional `db` parameter to select a database by alias. Aliases are derived from database filenames (`unreal.db` → `"unreal"`). Use `list_databases` to discover available aliases. Default (`db=""`) uses the primary database (`CODE_SOURCE_DB`).
+
+### Search query modes
+
+**Simple mode** (`keyword`):
+```
+keyword="GetGBuffer"
+keyword="FMaterial Render"
+```
+
+**Advanced mode** (`raw_query`) — full FTS5 boolean:
+```
+raw_query='"GetGBuffer" AND "Emissive"'
+raw_query='"Material" NOT "hlsl"'
+raw_query='(file_path : "BasePass") AND "roughness"'
+raw_query='(module_name : "Renderer") AND "VirtualTexture"'
+```
+
+### Three-level funnel
+
+1. **`search_fts_tool(keyword)`** → file candidates + block QNs
+2. **`search_fts_tool(raw_query, file_path filter)`** → precise block in target file
+3. **`read_symbol(block QN)`** or **`read_file_range(file, line)`** → full code
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────────────────────┐
+│                    MCP Server (FastMCP)                       │
+├──────────┬──────────┬──────────┬──────────┬──────────────────┤
+│ search   │ read     │ read     │ get_dir  │ list             │
+│ fts_tool │ _symbol  │ _file    │ _struct  │ _databases       │
+│          │          │ _range   │          │                  │
+├──────────┴──────────┴──────────┴──────────┴──────────────────┤
+│                     Query Pipeline                            │
+│  FTS5 trigram → Symbol match → Edge extraction               │
+├──────────────────────────────────────────────────────────────┤
+│                    SQLite Database                            │
+│  file_content + FTS5 │ symbol_index │ strict_edges           │
+└──────────────────────────────────────────────────────────────┘
+```
+
+### Bracket skeleton index
+
+A 6-state finite state machine (CODE, LINE_COMMENT, BLOCK_COMMENT, STRING, CHAR_LITERAL, RAW_STRING) scans source code tracking brace pairs while correctly ignoring braces in comments and string literals. Each matched pair records `open_line`, `close_line`, `depth`, and `is_complete`.
+
+Top-level blocks are classified by a **symbol analyzer** producing `block_type` (namespace/class/enum/function/macro) and `block_name` (qualified name).
+
+### Multi-database registry
+
+Databases are registered via environment variables at server startup:
+- `CODE_SOURCE_DB` — primary database (default when `db` is omitted)
+- `CODE_SOURCE_DBS` — colon-separated list of additional databases
+
+Aliases are auto-derived from filename stems. Connections are cached with health checks.
+
+## Environment Variables
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `CODE_SOURCE_DB` | Yes | Path to primary SQLite database |
+| `CODE_SOURCE_DBS` | No | Colon-separated paths to additional databases |
+
+## Development
+
+```bash
+# Clone and setup
+git clone https://github.com/didi514354875/code-explore-by-sql.git
+cd code-explore-by-sql
+uv sync --dev
+
+# Run tests
+uv run pytest
+uv run ruff check .
+```
+
+## License
+
+MIT

code_explore_by_sql-0.1.0/pyproject.toml

@@ -0,0 +1,81 @@
+[project]
+name = "code-explore-by-sql"
+version = "0.1.0"
+description = "SQLite FTS5 (trigram) MCP server for code source search."
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.10"
+dependencies = [
+    "mcp[cli]>=1.2.0",
+    "pydantic>=2.0",
+]
+
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Software Development :: Code Generators",
+    "Topic :: Text Processing :: Indexing",
+]
+
+[project.urls]
+Repository = "https://github.com/didi514354875/code-explore-by-sql"
+
+[project.scripts]
+code-source-sql = "code_source_sql:main"
+code-source-sql-build-db = "code_source_sql.build_db:main"
+
+[tool.hatch.build.targets.wheel]
+packages = [
+    "src/code_source_sql",
+]
+
+[tool.hatch.build.targets.sdist]
+exclude = [
+    "*.db",
+    "*.db-shm",
+    "*.db-wal",
+    "*.db.bak*",
+    "*.log",
+    ".venv/",
+    "uv.lock",
+    "tests/",
+    "scripts/",
+    "ref/",
+    "references/",
+    ".claude/",
+    ".claude-plugin/",
+    ".github/",
+    ".vscode/",
+    "*.txt",
+    "simplePlan.md",
+    "problemArch.md",
+    "SKILL copy.md",
+]
+
+[tool.pytest.ini_options]
+pythonpath = ["src"]
+testpaths = ["tests"]
+
+[tool.ruff]
+line-length = 120
+src = ["src", "tests"]
+target-version = "py310"
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B"]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[dependency-groups]
+dev = [
+    "pytest>=9.0.3",
+    "ruff>=0.15.13",
+]

code_explore_by_sql-0.1.0/src/code_source_sql/__init__.py

@@ -0,0 +1,9 @@
+"""UE Semantic Search — plan.md implementation.
+
+Three-table architecture:
+  file_content (FTS5) -> symbol_index (QN + UE meta) -> strict_edges (4 types)
+"""
+
+from .server import main
+
+__all__ = ["main"]

code_explore_by_sql-0.1.0/src/code_source_sql/__main__.py

@@ -0,0 +1,5 @@
+"""Allow running as: python -m code_source_sql"""
+from .server import main
+
+if __name__ == "__main__":
+    main()