graphnav 0.1.0__tar.gz

graphnav-0.1.0/PKG-INFO

@@ -0,0 +1,9 @@
+Metadata-Version: 2.4
+Name: graphnav
+Version: 0.1.0
+Summary: Knowledge-graph context injection for AI coding agents in monorepos
+Requires-Python: >=3.11
+Requires-Dist: graphifyy>=0.8
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: anthropic>=0.40; extra == "dev"

graphnav-0.1.0/README.md

@@ -0,0 +1,259 @@
+# codex-graph
+
+**Token-cheap AI coding for monorepos.** Builds a graphify knowledge graph of your codebase, then gives every AI coding agent (GitHub Copilot, Claude Code, OpenAI Codex) a minimal, targeted context pack instead of the whole repo.
+
+---
+
+## The problem
+
+AI coding agents default to exploring the filesystem with `find`/`ls`/`cat`, reading entire files, and burning tokens on irrelevant code. In a monorepo this compounds: every request pulls context from every service.
+
+codex-graph solves this by:
+
+1. Extracting a knowledge graph (symbols, call edges, cross-service links) once, up front
+2. Giving agents a **one-command retrieval path** that returns only the files and `file:line` locations relevant to the current task
+3. Writing instruction files that explicitly direct agents to use retrieval first — and ban raw filesystem exploration
+
+---
+
+## Install
+
+```bash
+pip install git+https://github.com/Amogh887/leveraging-graphify.git
+```
+
+Requires Python ≥ 3.11. Pulls `graphifyy` (the `graphify` binary) automatically.
+
+**API key:** Place a `.env` file anywhere up the directory tree from your project (or inside any service subfolder). codex-graph walks up and down to find it:
+
+```
+ANTHROPIC_KEY=sk-ant-...
+```
+
+---
+
+## Quickstart
+
+```bash
+# In your monorepo root — detects services, builds graphs, writes agent instructions
+codex-graph map
+
+# Get a context pack for a task (free, no LLM, ~instant)
+codex-graph context "add a critique scoring function to the coach"
+
+# Keep graphs live as you edit
+codex-graph watch
+```
+
+After `map`, every AI agent in the repo has access to:
+
+- **`CLAUDE.md`** — picked up by Claude Code
+- **`AGENTS.md`** — picked up by OpenAI Codex CLI
+- **`.github/copilot-instructions.md`** — picked up by GitHub Copilot
+- **`<service>/graphify-out/SYMBOLS.md`** — symbol→`file:line` index per service
+- **`<service>/graphify-out/BRIDGES.md`** — exact cross-service call sites with line numbers
+- **`graphify-out/MONOREPO_MAP.md`** — overview of all services and their connections
+
+---
+
+## Commands
+
+### `codex-graph map`
+
+Builds the knowledge graph and generates all agent instruction files.
+
+```
+codex-graph map [--root PATH] [--backend BACKEND] [--dry-run]
+```
+
+| Flag | Default | Description |
+|---|---|---|
+| `--root` | `.` | Monorepo root directory |
+| `--backend` | `claude` | LLM backend for extraction: `claude`, `openai`, `gemini`, `deepseek`, `ollama` |
+| `--dry-run` | off | Detect services and print the plan without calling graphify |
+
+What it does:
+1. Auto-detects service boundaries (by marker files **and** by source code presence)
+2. Extracts a single overarching knowledge graph of the whole repo via graphify
+3. Partitions it into per-service local graphs
+4. Analyzes cross-service edges and writes `BRIDGES.md` per service
+5. Writes `SYMBOLS.md`, `MONOREPO_MAP.md`, and the coding playbook to `CLAUDE.md`, `AGENTS.md`, `.github/copilot-instructions.md`
+
+---
+
+### `codex-graph context`
+
+Prints a token-budgeted context pack for a coding task. **No LLM call — free and instant.**
+
+```
+codex-graph context "<task>" [--root PATH] [--budget N] [--files N]
+```
+
+| Flag | Default | Description |
+|---|---|---|
+| `--budget` | `2000` | Approximate token budget for the output |
+| `--files` | `8` | Max number of files to include |
+| `--root` | `.` | Repo root |
+
+Example output:
+
+```
+# Context for: add a critique scoring function to the coach
+
+## Open only these files
+- backend/coach.py — generate_response() L145, practice_critique() L326
+- eval/run_eval.py — run_prompts_on_dataset() L78, judge_responses() L119
+
+## Cross-service impact
+- eval/run_eval.py:run_prompts_on_dataset() L78 --calls--> backend/coach.py:generate_response() L145
+
+## Next
+Read only the file:line regions above. Before changing a symbol under
+Cross-service impact, run `graphify affected "<symbol>"`. Then run the tests.
+```
+
+Works on single-service repos too (the Cross-service section is omitted).
+
+---
+
+### `codex-graph watch`
+
+Long-running daemon. Watches the repo for file changes and keeps all graphs, symbol maps, bridge notes, and agent instructions up to date.
+
+```
+codex-graph watch [--root PATH] [--backend BACKEND]
+```
+
+Press `Ctrl-C` to stop cleanly.
+
+---
+
+### `codex-graph` (no subcommand)
+
+If run with no arguments in a monorepo root, auto-detects services and runs `map` automatically. If a prompt is given, falls through to the context-injection path for the Codex CLI.
+
+---
+
+## Service detection
+
+codex-graph detects a subdirectory as a service if it contains:
+
+- A marker file: `package.json`, `pyproject.toml`, `requirements.txt`, `go.mod`, `Cargo.toml`, `tsconfig.json`, `Gemfile`, and more, **or**
+- Any source code files (`.py`, `.ts`, `.tsx`, `.js`, `.go`, `.rs`, `.java`, etc.)
+
+Skipped automatically: `node_modules`, `dist`, `build`, `graphify-out`, `__pycache__`, `.git`, dotdirs, and other non-source directories.
+
+---
+
+## Generated files
+
+### `CLAUDE.md` / `AGENTS.md` / `.github/copilot-instructions.md`
+
+All three contain the same managed block — the coding playbook. Content is written between `<!-- codex-graph:start -->` / `<!-- codex-graph:end -->` markers so re-running `map` updates only the block and preserves any hand-written content outside it.
+
+The playbook instructs agents to:
+
+1. Read `MONOREPO_MAP.md` first for any non-trivial task
+2. Run `codex-graph context "<task>"` instead of exploring with `find`/`ls`/`cat`
+3. Open only the returned `file:line` regions
+4. Check `graphify affected` before changing cross-service symbols
+5. Skip all of the above for single-line edits
+
+### `<service>/graphify-out/SYMBOLS.md`
+
+Compact symbol index for the service. Example:
+
+```
+# Symbols: backend
+
+## coach.py
+- generate_response() — L145
+- practice_critique() — L326
+- _parse_json_response() — L79
+```
+
+Much smaller than the raw `graph.json` (tens of bytes per symbol vs. kilobytes per node).
+
+### `<service>/graphify-out/BRIDGES.md`
+
+Cross-service call sites with exact line numbers on both sides.
+
+```
+| Local File | Symbol | Loc | Relation | → Service | Remote File | Remote Symbol | Loc |
+|---|---|---|---|---|---|---|---|
+| run_eval.py | run_prompts_on_dataset() | L78 | calls | backend | backend/coach.py | generate_response() | L145 |
+```
+
+Includes a note to run `graphify affected "<symbol>"` before editing any listed symbol.
+
+### `graphify-out/MONOREPO_MAP.md`
+
+Overview of all services and which services each connects to.
+
+```
+| Service | Graph | Bridges To |
+|---|---|---|
+| api | api/graphify-out/graph.json | _none_ |
+| backend | backend/graphify-out/graph.json | api |
+| eval | eval/graphify-out/graph.json | backend |
+```
+
+---
+
+## Configuration
+
+Place a `config.toml` in the project root (or pass `--config PATH`):
+
+```toml
+[mono]
+graphify_backend = "claude"        # LLM backend for extraction
+watch_poll_interval = 3.0          # seconds between mtime checks in watch mode
+context_budget_tokens = 2000       # token budget for codex-graph context output
+context_top_files = 8              # max files returned by context command
+
+[graph]
+skip_patterns = ["node_modules", ".git", "graphify-out", "playwright-report"]
+```
+
+---
+
+## How cross-service bridges work
+
+codex-graph extracts **one overarching graph** of the whole repo (not one per service). This means graphify's AST and semantic extraction can find call edges that cross service boundaries — something a per-service extraction followed by a union merge can never do.
+
+The overarching graph is then partitioned into per-service local graphs for navigation. Bridges are derived from the overarching graph where an edge's endpoints belong to different services.
+
+**Note:** Bridges only appear for direct code references (imports, function calls). Services that communicate over HTTP (e.g. a React frontend calling a Python backend via `fetch`) will correctly show zero bridges — the connection exists at the protocol level, not the code level.
+
+---
+
+## Team setup
+
+Every team member runs one command after cloning:
+
+```bash
+pip install git+https://github.com/Amogh887/leveraging-graphify.git
+```
+
+Drop a `.env` with your API key anywhere in or above the repo:
+
+```
+ANTHROPIC_KEY=sk-ant-...
+```
+
+Then:
+
+```bash
+codex-graph map          # one-time setup, or re-run after large refactors
+codex-graph watch        # optional: keep graphs live during active development
+```
+
+The generated `CLAUDE.md`, `AGENTS.md`, and `.github/copilot-instructions.md` can be committed to the repo so teammates get the agent instructions without needing to re-run `map`.
+
+---
+
+## Requirements
+
+- Python ≥ 3.11
+- `graphifyy` ≥ 0.8 (installed automatically)
+- An API key for your chosen LLM backend (only needed for `map` / `watch`; `context` is free)

graphnav-0.1.0/codex_graph/__init__.py

@@ -0,0 +1,10 @@
+class GraphNotFoundError(Exception):
+    pass
+
+
+class CodexNotFoundError(Exception):
+    pass
+
+
+class CodexTimeoutError(Exception):
+    pass

graphnav-0.1.0/codex_graph/cli.py

@@ -0,0 +1,238 @@
+from __future__ import annotations
+
+import argparse
+import os
+import sys
+
+from codex_graph import CodexNotFoundError, CodexTimeoutError, GraphNotFoundError
+from codex_graph.config import load_config
+from codex_graph.graph_query import load_index, query_files
+from codex_graph import runner
+
+
+def _run_mono_command(cmd: str, argv: list[str]) -> None:
+    from codex_graph import multirepo
+
+    parser = argparse.ArgumentParser(
+        prog=f"codex-graph {cmd}",
+        description={
+            "map": "Build per-service graphs and cross-service bridge notes for a monorepo",
+            "watch": "Watch for file changes and keep per-service graphs and bridge notes up-to-date",
+        }[cmd],
+    )
+    parser.add_argument("--root", default=".", metavar="PATH", help="Monorepo root directory (default: current directory)")
+    parser.add_argument("--backend", default=None, metavar="BACKEND", help="graphify LLM backend (claude|openai|gemini|deepseek|ollama)")
+    parser.add_argument("--config", default=None, metavar="PATH", help="Path to config.toml")
+    if cmd == "map":
+        parser.add_argument("--dry-run", action="store_true", help="Detect services and print the plan without invoking graphify")
+
+    args = parser.parse_args(argv)
+    cfg = load_config(args.config)
+
+    if cmd == "map":
+        rc = multirepo.run_map(
+            root=args.root,
+            mono_cfg=cfg.mono,
+            backend_override=args.backend,
+            dry_run=args.dry_run,
+        )
+    else:
+        rc = multirepo.run_watch(
+            root=args.root,
+            mono_cfg=cfg.mono,
+            backend_override=args.backend,
+        )
+    sys.exit(rc)
+
+
+def _auto_map_if_needed(cfg_path: str | None) -> None:
+    from codex_graph import multirepo
+    from codex_graph.config import load_config
+
+    cfg = load_config(cfg_path)
+    root = os.path.abspath(".")
+    services = multirepo.detect_services(root, cfg.mono.marker_files)
+    if not services:
+        return
+
+    names = ", ".join(s.name for s in services)
+    print(f"[codex-graph] Detected {len(services)} service(s): {names}")
+    print(f"[codex-graph] Running 'codex-graph map' to build knowledge graphs ...", file=sys.stderr)
+    rc = multirepo.run_map(root=root, mono_cfg=cfg.mono)
+    sys.exit(rc)
+
+
+def _run_context_command(argv: list[str]) -> None:
+    from codex_graph import multirepo
+
+    parser = argparse.ArgumentParser(
+        prog="codex-graph context",
+        description="Print a token-budgeted context pack (files + symbol locations + cross-service impact) for a coding task",
+    )
+    parser.add_argument("task", nargs="?", help="The coding task, in natural language")
+    parser.add_argument("--root", default=".", metavar="PATH", help="Repo root (default: current directory)")
+    parser.add_argument("--budget", type=int, default=None, metavar="N", help="Approx token budget for the pack")
+    parser.add_argument("--files", type=int, default=None, metavar="N", help="Max number of files to include")
+    parser.add_argument("--config", default=None, metavar="PATH", help="Path to config.toml")
+    args = parser.parse_args(argv)
+
+    task = args.task
+    if not task and not sys.stdin.isatty():
+        task = sys.stdin.read().strip()
+    if not task:
+        parser.print_help()
+        sys.exit(1)
+
+    cfg = load_config(args.config)
+    pack = multirepo.build_context_pack(
+        root=args.root,
+        task=task,
+        top_files=args.files if args.files is not None else cfg.mono.context_top_files,
+        budget_tokens=args.budget if args.budget is not None else cfg.mono.context_budget_tokens,
+        skip_patterns=cfg.graph.skip_patterns,
+    )
+    print(pack)
+    sys.exit(0)
+
+
+def _run_graph_query_command(kind: str, argv: list[str]) -> None:
+    from codex_graph import multirepo
+    from codex_graph.graph_nav import GraphNav
+
+    parser = argparse.ArgumentParser(prog=f"codex-graph {kind}")
+    parser.add_argument("term", nargs="?", help="query (find) or symbol (neighbors)")
+    parser.add_argument("--root", default=".", metavar="PATH")
+    parser.add_argument("--config", default=None, metavar="PATH")
+    args = parser.parse_args(argv)
+    if not args.term:
+        parser.print_help()
+        sys.exit(1)
+
+    cfg = load_config(args.config)
+    graph_path = multirepo._overarching_graph_path(os.path.abspath(args.root))
+    if not os.path.exists(graph_path):
+        print(f"Error: no knowledge graph at {graph_path}. Run `codex-graph map` first.", file=sys.stderr)
+        sys.exit(2)
+    nav = GraphNav(graph_path, cfg.graph.skip_patterns)
+
+    if kind == "find":
+        hits = nav.find_symbols(args.term, k=10)
+        if not hits:
+            print("(no matches)")
+        for h in hits:
+            print(f"{h['symbol']} — {h['file']}:{h['loc']}")
+    else:
+        r = nav.neighbors(args.term)
+        if not r.get("found", True):
+            print("(symbol not found)")
+            sys.exit(0)
+        print(f"{r['symbol']} defined at {r['defined_at']}")
+        if r.get("callers"):
+            print("callers:")
+            for c in r["callers"]:
+                print("  " + c)
+        if r.get("callees"):
+            print("calls:")
+            for c in r["callees"]:
+                print("  " + c)
+    sys.exit(0)
+
+
+def main() -> None:
+    if len(sys.argv) > 1 and sys.argv[1] in ("map", "watch"):
+        _run_mono_command(sys.argv[1], sys.argv[2:])
+        return
+    if len(sys.argv) > 1 and sys.argv[1] == "context":
+        _run_context_command(sys.argv[2:])
+        return
+    if len(sys.argv) > 1 and sys.argv[1] in ("find", "neighbors"):
+        _run_graph_query_command(sys.argv[1], sys.argv[2:])
+        return
+
+    parser = argparse.ArgumentParser(
+        prog="codex-graph",
+        description=(
+            "Codex CLI with knowledge-graph context injection for monorepos.\n\n"
+            "First-run (after pip install): just run 'codex-graph' or 'codex-graph map'\n"
+            "in your monorepo root — services are auto-detected and graphs are built.\n\n"
+            "Subcommands:\n"
+            "  map    Build per-service graphs and cross-service bridge notes\n"
+            "  watch  Keep graphs and bridge notes up-to-date as files change"
+        ),
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    parser.add_argument("prompt", nargs="?", help="Natural language task prompt")
+    parser.add_argument("--config", default=None, metavar="PATH", help="Path to config.toml")
+    parser.add_argument("--top-k", type=int, default=None, metavar="N", help="Number of files to inject as context")
+    parser.add_argument("--graph", default=None, metavar="PATH", help="Path to graph.json")
+    parser.add_argument("--dry-run", action="store_true", help="Print enriched prompt without calling codex")
+    parser.add_argument("--list-files", action="store_true", help="Print ranked files and scores, then exit")
+    parser.add_argument("--no-context", action="store_true", help="Pass prompt to codex with no graph context")
+
+    args = parser.parse_args()
+
+    prompt = args.prompt
+    if not prompt:
+        if sys.stdin.isatty():
+            _auto_map_if_needed(args.config)
+            parser.print_help()
+            sys.exit(1)
+        prompt = sys.stdin.read().strip()
+    if not prompt:
+        parser.print_help()
+        sys.exit(1)
+
+    cfg = load_config(args.config)
+
+    if args.top_k is not None:
+        cfg.query.top_k = args.top_k
+    if args.graph is not None:
+        cfg.graph.path = args.graph
+
+    project_root = os.path.abspath(cfg.graph.project_root)
+    graph_path = (
+        cfg.graph.path
+        if os.path.isabs(cfg.graph.path)
+        else os.path.join(os.getcwd(), cfg.graph.path)
+    )
+
+    if args.no_context:
+        ranked = []
+    else:
+        try:
+            index = load_index(graph_path, cfg.graph.skip_patterns)
+        except GraphNotFoundError as e:
+            print(f"Error: {e}", file=sys.stderr)
+            sys.exit(2)
+
+        ranked = query_files(
+            prompt,
+            index,
+            cfg.query.top_k,
+            cfg.query.community_boost_weight,
+            cfg.query.bm25_k1,
+            cfg.query.bm25_b,
+        )
+
+    if args.list_files:
+        for rf in ranked:
+            print(f"{rf.score:.3f}  {rf.source_file}")
+        sys.exit(0)
+
+    if args.dry_run:
+        print(runner.build_prompt(prompt, ranked, cfg, project_root))
+        sys.exit(0)
+
+    try:
+        exit_code = runner.run(prompt, ranked, cfg, project_root)
+        sys.exit(exit_code)
+    except CodexNotFoundError as e:
+        print(f"Error: {e}", file=sys.stderr)
+        sys.exit(127)
+    except CodexTimeoutError as e:
+        print(f"Error: {e}", file=sys.stderr)
+        sys.exit(124)
+
+
+if __name__ == "__main__":
+    main()

graphnav-0.1.0/codex_graph/config.py

@@ -0,0 +1,127 @@
+from __future__ import annotations
+
+import os
+import tomllib
+from dataclasses import dataclass, field
+
+
+@dataclass
+class GraphConfig:
+    path: str = "graphify-out/graph.json"
+    project_root: str = "."
+    skip_patterns: list[str] = field(default_factory=lambda: ["playwright-report", "node_modules", ".git"])
+
+
+@dataclass
+class QueryConfig:
+    top_k: int = 5
+    community_boost_weight: float = 2.0
+    bm25_k1: float = 1.5
+    bm25_b: float = 0.75
+
+
+@dataclass
+class ContextConfig:
+    max_file_chars: int = 8000
+    show_scores: bool = False
+
+
+@dataclass
+class CodexConfig:
+    command: str = "codex"
+    subcommand: str = "exec"
+    extra_args: list[str] = field(default_factory=list)
+    inject_via: str = "stdin"
+    timeout_seconds: int = 300
+
+
+@dataclass
+class MonoConfig:
+    marker_files: list[str] = field(default_factory=lambda: [
+        "package.json", "pyproject.toml", "go.mod", "Cargo.toml",
+        "pom.xml", "build.gradle", "setup.py", "setup.cfg",
+        "requirements.txt", "Gemfile", "composer.json", "tsconfig.json",
+    ])
+    graphify_backend: str = "claude"
+    watch_poll_interval: float = 3.0
+    context_budget_tokens: int = 2000
+    context_top_files: int = 8
+
+
+@dataclass
+class Config:
+    graph: GraphConfig = field(default_factory=GraphConfig)
+    query: QueryConfig = field(default_factory=QueryConfig)
+    context: ContextConfig = field(default_factory=ContextConfig)
+    codex: CodexConfig = field(default_factory=CodexConfig)
+    mono: MonoConfig = field(default_factory=MonoConfig)
+
+
+def _apply_toml(cfg: Config, data: dict) -> Config:
+    if "graph" in data:
+        g = data["graph"]
+        cfg.graph = GraphConfig(
+            path=g.get("path", cfg.graph.path),
+            project_root=g.get("project_root", cfg.graph.project_root),
+            skip_patterns=g.get("skip_patterns", cfg.graph.skip_patterns),
+        )
+    if "query" in data:
+        q = data["query"]
+        cfg.query = QueryConfig(
+            top_k=q.get("top_k", cfg.query.top_k),
+            community_boost_weight=q.get("community_boost_weight", cfg.query.community_boost_weight),
+            bm25_k1=q.get("bm25_k1", cfg.query.bm25_k1),
+            bm25_b=q.get("bm25_b", cfg.query.bm25_b),
+        )
+    if "context" in data:
+        c = data["context"]
+        cfg.context = ContextConfig(
+            max_file_chars=c.get("max_file_chars", cfg.context.max_file_chars),
+            show_scores=c.get("show_scores", cfg.context.show_scores),
+        )
+    if "codex" in data:
+        cx = data["codex"]
+        cfg.codex = CodexConfig(
+            command=cx.get("command", cfg.codex.command),
+            subcommand=cx.get("subcommand", cfg.codex.subcommand),
+            extra_args=cx.get("extra_args", cfg.codex.extra_args),
+            inject_via=cx.get("inject_via", cfg.codex.inject_via),
+            timeout_seconds=cx.get("timeout_seconds", cfg.codex.timeout_seconds),
+        )
+    if "mono" in data:
+        m = data["mono"]
+        cfg.mono = MonoConfig(
+            marker_files=m.get("marker_files", cfg.mono.marker_files),
+            graphify_backend=m.get("graphify_backend", cfg.mono.graphify_backend),
+            watch_poll_interval=m.get("watch_poll_interval", cfg.mono.watch_poll_interval),
+            context_budget_tokens=m.get("context_budget_tokens", cfg.mono.context_budget_tokens),
+            context_top_files=m.get("context_top_files", cfg.mono.context_top_files),
+        )
+    return cfg
+
+
+def load_config(explicit_path: str | None = None) -> Config:
+    cfg = Config()
+
+    candidates: list[str] = []
+    if explicit_path:
+        candidates = [explicit_path]
+    else:
+        env_path = os.environ.get("CODEX_GRAPH_CONFIG")
+        if env_path:
+            candidates.append(env_path)
+        candidates.append(os.path.join(os.getcwd(), "config.toml"))
+        candidates.append(os.path.expanduser("~/.codex-graph/config.toml"))
+
+    for path in candidates:
+        if os.path.exists(path):
+            with open(path, "rb") as f:
+                data = tomllib.load(f)
+            cfg = _apply_toml(cfg, data)
+            break
+    else:
+        if explicit_path:
+            import sys
+            print(f"Warning: config file not found: {explicit_path}", file=sys.stderr)
+
+    return cfg