repokg 0.1.0__tar.gz

repokg-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nihar Shah
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

repokg-0.1.0/PKG-INFO

@@ -0,0 +1,160 @@
+Metadata-Version: 2.4
+Name: repokg
+Version: 0.1.0
+Summary: Generate an AI-ready knowledge graph (KNOWLEDGE_GRAPH.md + kg.json) of any codebase: modules, dependency edges, branches, PRs, ops surface.
+Author: Nihar Shah
+License: MIT
+Project-URL: Homepage, https://github.com/NehharShah/repokg
+Keywords: knowledge-graph,codebase,ai,agents,documentation,repo-map
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Documentation
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# repokg
+
+Generate an **AI-ready knowledge graph** of any codebase — so an AI agent (or a new
+developer) can read one file and start building immediately.
+
+`repokg` extracts everything that can be known *deterministically* about a repo —
+module inventory, internal import graph, every branch classified against every PR
+(merged / squash-merged / abandoned / stale), contributor stats, CI/Docker/Helm/Make
+surface — and renders it as:
+
+- **`KNOWLEDGE_GRAPH.md`** — a single human/AI-readable document with a mermaid architecture
+  graph, module tables, branch & PR catalog, timeline, and ops inventory.
+- **`.repokg/kg.json`** — the same graph, machine-readable.
+
+The semantic layer (module purposes, data-flow narratives, project eras, gotchas)
+can't be produced by static analysis without guessing — so repokg is
+**agent-first**: it emits `.repokg/prompts/enrich.md`, a rigorous prompt any AI coding
+agent (Claude Code, Cursor, Copilot Workspace…) executes to verify-and-fill the
+narrative sections, writing `.repokg/narratives.json`. Re-render and the knowledge graph is
+complete. No API keys, no LLM dependency in the tool itself.
+
+## Install
+
+```sh
+pipx install repokg        # or: pip install repokg
+# from source:
+pipx install git+https://github.com/NehharShah/repokg
+```
+
+Requirements: Python ≥ 3.9, `git`. Optional: [`gh`](https://cli.github.com) (logged
+in) for the PR/branch cross-reference — without it the knowledge graph still builds, minus PR data.
+
+## Usage
+
+```sh
+cd your-repo
+repokg                      # = generate: scan + prompts + render
+```
+
+Output:
+
+```
+.repokg/kg.json               # machine-readable knowledge graph
+.repokg/prompts/enrich.md        # hand this to your AI agent
+KNOWLEDGE_GRAPH.md                        # the knowledge graph document
+```
+
+Then, in your AI agent of choice:
+
+> Follow the instructions in .repokg/prompts/enrich.md
+
+The agent explores the code, writes `.repokg/narratives.json`, and runs
+`repokg render` — KNOWLEDGE_GRAPH.md now carries verified purposes, data flows,
+timeline eras, and gotchas alongside the deterministic structure.
+
+### Commands
+
+| Command | Effect |
+|---|---|
+| `repokg scan [path]` | Extract structure → `.repokg/kg.json` |
+| `repokg prompts [path]` | Write the enrichment prompt |
+| `repokg render [path]` | `kg.json` (+ `narratives.json`) → `KNOWLEDGE_GRAPH.md` |
+| `repokg generate [path]` | All three (default) |
+| `repokg inject [path]` | Wire the knowledge graph into `CLAUDE.md` / `AGENTS.md` / Cursor rules |
+| `repokg check [path]` | Exit 1 if the knowledge graph is stale vs `HEAD` (CI-friendly) |
+
+Flags: `--out DIR` (default `<repo>/.repokg`), `--md FILE` (default `<repo>/KNOWLEDGE_GRAPH.md`),
+`--no-github`, `--pr-limit N`.
+
+## Agent integration
+
+`repokg inject` adds a **managed block** (delimited by
+`<!-- repokg:begin/end -->`, idempotent, never touches your hand-written
+content) pointing agents at KNOWLEDGE_GRAPH.md:
+
+- **`CLAUDE.md`** (Claude Code) — updated if present
+- **`AGENTS.md`** (the cross-tool agent standard) — updated if present, created if
+  no agent file exists at all
+- **`.github/copilot-instructions.md`** (Copilot) — updated if present
+- **`.cursor/rules/repokg.mdc`** (Cursor, with `alwaysApply: true`) — created
+  if `.cursor/rules/` exists; falls back to legacy `.cursorrules`
+
+Keep it fresh in CI:
+
+```yaml
+- run: pipx run repokg check . || echo "::warning::KNOWLEDGE_GRAPH.md is stale"
+```
+
+KNOWLEDGE_GRAPH.md itself also lists any agent-context files it found, so an agent landing
+on the knowledge graph discovers your rules — and vice versa.
+
+## What gets extracted (all verified, never guessed)
+
+| Area | How |
+|---|---|
+| Branch classification | `git for-each-ref` + `--merged` ancestry vs the integration branch (auto-detects `staging`/`develop`), cross-referenced with every PR's head ref via `gh` — distinguishes true merges from squash-merges from abandoned work |
+| PR catalog | `gh pr list --state all` — open / merged / closed-unmerged, full appendix table |
+| Module inventory | Filesystem walk with LOC per directory, language detection, generated-code flagging |
+| Import graph | Go: `import` blocks resolved against `go.mod` module paths · Python: stdlib `ast` incl. relative imports · JS/TS: relative `import`/`require` resolution. Directory→directory edges with counts |
+| Ops surface | CI workflow names, Dockerfiles, compose files, Helm charts, Makefile targets, config/docs/test/migration dirs |
+| Timeline | Merged PRs grouped by month with conventional-commit scope frequencies (replaced by agent-written eras after enrichment) |
+
+## Why agent-first instead of calling an LLM API?
+
+Because the enrichment quality depends on *reading the code*, and your coding agent
+already has the repo open, tools to search it, and your permission model. A prompt it
+can execute beats a second LLM integration with its own keys, costs, and context limits.
+The contract between tool and agent is one JSON file (`narratives.json`) with a fixed
+schema — everything else stays deterministic and reproducible.
+
+## Known limitations
+
+- **JS/TS**: only relative imports are resolved; alias imports (`@/…`,
+  tsconfig `paths`) are ignored.
+- **Fork PRs**: a fork PR whose head branch name matches a local branch will be
+  linked to it (GitHub's API reports bare head refs).
+- **Python**: packages are discovered at the repo root and under `src/`;
+  deeper monorepo layouts (`packages/*/src/…`) get file-level edges only.
+- Branch `ahead` counts use one batched git call on git ≥ 2.41, with a
+  per-branch fallback on older git.
+
+## Roadmap
+
+- [ ] Rust / Java / Kotlin import graphs
+- [ ] `--exclude` glob patterns
+- [ ] `llms.txt` emission alongside KNOWLEDGE_GRAPH.md
+- [ ] tsconfig `paths` alias resolution
+- [ ] PyPI release + prebuilt GitHub Action
+
+## Development
+
+```sh
+pip install -e .
+python -m unittest discover -s tests -v
+```
+
+No runtime dependencies — stdlib only.
+
+## License
+
+MIT

repokg-0.1.0/README.md

@@ -0,0 +1,141 @@
+# repokg
+
+Generate an **AI-ready knowledge graph** of any codebase — so an AI agent (or a new
+developer) can read one file and start building immediately.
+
+`repokg` extracts everything that can be known *deterministically* about a repo —
+module inventory, internal import graph, every branch classified against every PR
+(merged / squash-merged / abandoned / stale), contributor stats, CI/Docker/Helm/Make
+surface — and renders it as:
+
+- **`KNOWLEDGE_GRAPH.md`** — a single human/AI-readable document with a mermaid architecture
+  graph, module tables, branch & PR catalog, timeline, and ops inventory.
+- **`.repokg/kg.json`** — the same graph, machine-readable.
+
+The semantic layer (module purposes, data-flow narratives, project eras, gotchas)
+can't be produced by static analysis without guessing — so repokg is
+**agent-first**: it emits `.repokg/prompts/enrich.md`, a rigorous prompt any AI coding
+agent (Claude Code, Cursor, Copilot Workspace…) executes to verify-and-fill the
+narrative sections, writing `.repokg/narratives.json`. Re-render and the knowledge graph is
+complete. No API keys, no LLM dependency in the tool itself.
+
+## Install
+
+```sh
+pipx install repokg        # or: pip install repokg
+# from source:
+pipx install git+https://github.com/NehharShah/repokg
+```
+
+Requirements: Python ≥ 3.9, `git`. Optional: [`gh`](https://cli.github.com) (logged
+in) for the PR/branch cross-reference — without it the knowledge graph still builds, minus PR data.
+
+## Usage
+
+```sh
+cd your-repo
+repokg                      # = generate: scan + prompts + render
+```
+
+Output:
+
+```
+.repokg/kg.json               # machine-readable knowledge graph
+.repokg/prompts/enrich.md        # hand this to your AI agent
+KNOWLEDGE_GRAPH.md                        # the knowledge graph document
+```
+
+Then, in your AI agent of choice:
+
+> Follow the instructions in .repokg/prompts/enrich.md
+
+The agent explores the code, writes `.repokg/narratives.json`, and runs
+`repokg render` — KNOWLEDGE_GRAPH.md now carries verified purposes, data flows,
+timeline eras, and gotchas alongside the deterministic structure.
+
+### Commands
+
+| Command | Effect |
+|---|---|
+| `repokg scan [path]` | Extract structure → `.repokg/kg.json` |
+| `repokg prompts [path]` | Write the enrichment prompt |
+| `repokg render [path]` | `kg.json` (+ `narratives.json`) → `KNOWLEDGE_GRAPH.md` |
+| `repokg generate [path]` | All three (default) |
+| `repokg inject [path]` | Wire the knowledge graph into `CLAUDE.md` / `AGENTS.md` / Cursor rules |
+| `repokg check [path]` | Exit 1 if the knowledge graph is stale vs `HEAD` (CI-friendly) |
+
+Flags: `--out DIR` (default `<repo>/.repokg`), `--md FILE` (default `<repo>/KNOWLEDGE_GRAPH.md`),
+`--no-github`, `--pr-limit N`.
+
+## Agent integration
+
+`repokg inject` adds a **managed block** (delimited by
+`<!-- repokg:begin/end -->`, idempotent, never touches your hand-written
+content) pointing agents at KNOWLEDGE_GRAPH.md:
+
+- **`CLAUDE.md`** (Claude Code) — updated if present
+- **`AGENTS.md`** (the cross-tool agent standard) — updated if present, created if
+  no agent file exists at all
+- **`.github/copilot-instructions.md`** (Copilot) — updated if present
+- **`.cursor/rules/repokg.mdc`** (Cursor, with `alwaysApply: true`) — created
+  if `.cursor/rules/` exists; falls back to legacy `.cursorrules`
+
+Keep it fresh in CI:
+
+```yaml
+- run: pipx run repokg check . || echo "::warning::KNOWLEDGE_GRAPH.md is stale"
+```
+
+KNOWLEDGE_GRAPH.md itself also lists any agent-context files it found, so an agent landing
+on the knowledge graph discovers your rules — and vice versa.
+
+## What gets extracted (all verified, never guessed)
+
+| Area | How |
+|---|---|
+| Branch classification | `git for-each-ref` + `--merged` ancestry vs the integration branch (auto-detects `staging`/`develop`), cross-referenced with every PR's head ref via `gh` — distinguishes true merges from squash-merges from abandoned work |
+| PR catalog | `gh pr list --state all` — open / merged / closed-unmerged, full appendix table |
+| Module inventory | Filesystem walk with LOC per directory, language detection, generated-code flagging |
+| Import graph | Go: `import` blocks resolved against `go.mod` module paths · Python: stdlib `ast` incl. relative imports · JS/TS: relative `import`/`require` resolution. Directory→directory edges with counts |
+| Ops surface | CI workflow names, Dockerfiles, compose files, Helm charts, Makefile targets, config/docs/test/migration dirs |
+| Timeline | Merged PRs grouped by month with conventional-commit scope frequencies (replaced by agent-written eras after enrichment) |
+
+## Why agent-first instead of calling an LLM API?
+
+Because the enrichment quality depends on *reading the code*, and your coding agent
+already has the repo open, tools to search it, and your permission model. A prompt it
+can execute beats a second LLM integration with its own keys, costs, and context limits.
+The contract between tool and agent is one JSON file (`narratives.json`) with a fixed
+schema — everything else stays deterministic and reproducible.
+
+## Known limitations
+
+- **JS/TS**: only relative imports are resolved; alias imports (`@/…`,
+  tsconfig `paths`) are ignored.
+- **Fork PRs**: a fork PR whose head branch name matches a local branch will be
+  linked to it (GitHub's API reports bare head refs).
+- **Python**: packages are discovered at the repo root and under `src/`;
+  deeper monorepo layouts (`packages/*/src/…`) get file-level edges only.
+- Branch `ahead` counts use one batched git call on git ≥ 2.41, with a
+  per-branch fallback on older git.
+
+## Roadmap
+
+- [ ] Rust / Java / Kotlin import graphs
+- [ ] `--exclude` glob patterns
+- [ ] `llms.txt` emission alongside KNOWLEDGE_GRAPH.md
+- [ ] tsconfig `paths` alias resolution
+- [ ] PyPI release + prebuilt GitHub Action
+
+## Development
+
+```sh
+pip install -e .
+python -m unittest discover -s tests -v
+```
+
+No runtime dependencies — stdlib only.
+
+## License
+
+MIT

repokg-0.1.0/pyproject.toml

@@ -0,0 +1,30 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "repokg"
+version = "0.1.0"
+description = "Generate an AI-ready knowledge graph (KNOWLEDGE_GRAPH.md + kg.json) of any codebase: modules, dependency edges, branches, PRs, ops surface."
+readme = "README.md"
+license = { text = "MIT" }
+authors = [{ name = "Nihar Shah" }]
+requires-python = ">=3.9"
+keywords = ["knowledge-graph", "codebase", "ai", "agents", "documentation", "repo-map"]
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Environment :: Console",
+  "Intended Audience :: Developers",
+  "License :: OSI Approved :: MIT License",
+  "Programming Language :: Python :: 3",
+  "Topic :: Software Development :: Documentation",
+]
+
+[project.urls]
+Homepage = "https://github.com/NehharShah/repokg"
+
+[project.scripts]
+repokg = "repokg.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]

repokg-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

repokg-0.1.0/src/repokg/__init__.py

@@ -0,0 +1,3 @@
+"""repokg: generate an AI-ready knowledge graph of any codebase."""
+
+__version__ = "0.1.0"

repokg-0.1.0/src/repokg/__main__.py

@@ -0,0 +1,6 @@
+import sys
+
+from .cli import main
+
+if __name__ == "__main__":
+    sys.exit(main())

repokg-0.1.0/src/repokg/cli.py

@@ -0,0 +1,144 @@
+"""repokg CLI: scan | prompts | render | generate."""
+
+import argparse
+import datetime
+import json
+import os
+import sys
+
+from . import __version__, code, deps, github, gitinfo, inject, markdown, ops, prompts
+
+
+def scan(repo, out, no_github, pr_limit):
+    info, branches = gitinfo.collect(repo)
+    if no_github:
+        prs, note = [], "GitHub lookup disabled (--no-github)"
+    else:
+        prs, note = github.collect(repo, pr_limit)
+    github.classify(branches, prs, info["trunk"], info["integration"])
+    tree = dict(code.walk(repo))  # single filesystem walk, shared by all collectors
+    languages, modules = code.collect(repo, tree)
+    kg = {
+        "repokg_version": 1,
+        "generated_at": datetime.date.today().isoformat(),
+        "repo": info,
+        "languages": languages,
+        "modules": modules,
+        "edges": deps.collect(repo, tree),
+        "branches": branches,
+        "prs": prs,
+        "github_note": note,
+        "ops": ops.collect(repo, tree),
+    }
+    os.makedirs(out, exist_ok=True)
+    path = os.path.join(out, "kg.json")
+    with open(path, "w", encoding="utf-8") as f:
+        json.dump(kg, f, indent=1)
+    print("wrote %s (%d modules, %d edges, %d branches, %d PRs)" %
+          (path, len(modules), len(kg["edges"]), len(branches), len(prs)))
+    return kg
+
+
+def write_prompts(repo, out, md):
+    pdir = os.path.join(out, "prompts")
+    os.makedirs(pdir, exist_ok=True)
+    path = os.path.join(pdir, "enrich.md")
+    with open(path, "w", encoding="utf-8") as f:
+        f.write(prompts.render(repo, out, md))
+    print("wrote %s (hand this to your AI agent)" % path)
+
+
+def render(out, md):
+    with open(os.path.join(out, "kg.json"), encoding="utf-8") as f:
+        kg = json.load(f)
+    narratives = {}
+    npath = os.path.join(out, "narratives.json")
+    if os.path.isfile(npath):
+        with open(npath, encoding="utf-8") as f:
+            narratives = json.load(f)
+    doc = markdown.render(kg, narratives)
+    with open(md, "w", encoding="utf-8") as f:
+        f.write(doc)
+    state = "enriched" if narratives else "structure-only; run .repokg/prompts/enrich.md to enrich"
+    print("wrote %s (%s)" % (md, state))
+
+
+def do_inject(repo, md):
+    for path, status in inject.run(repo, md).items():
+        print("%s: %s" % (path, status))
+
+
+def check(repo, out, md):
+    """Exit 0 if the knowledge graph matches HEAD, 1 if stale/missing. CI-friendly."""
+    apath = os.path.join(out, "kg.json")
+    if not os.path.isfile(apath) or not os.path.isfile(md):
+        print("stale: knowledge graph not generated (run `repokg generate`)")
+        return 1
+    with open(apath, encoding="utf-8") as f:
+        stored = json.load(f).get("repo", {}).get("head", "")
+    head = gitinfo.try_run(repo, "rev-parse", "HEAD")
+    if stored and head and stored != head:
+        print("stale: knowledge graph at %s, HEAD is %s (run `repokg generate`)"
+              % (stored[:12], head[:12]))
+        return 1
+    print("fresh: knowledge graph matches HEAD %s" % (head[:12] or "(unknown)"))
+    return 0
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(
+        prog="repokg",
+        description="Generate an AI-ready knowledge graph of a codebase.")
+    ap.add_argument("command", nargs="?", default="generate",
+                    choices=["scan", "prompts", "render", "generate", "inject",
+                             "check", "version"],
+                    help="scan: extract structure to .repokg/kg.json | "
+                         "prompts: write the AI enrichment prompt | "
+                         "render: kg.json (+narratives.json) -> KNOWLEDGE_GRAPH.md | "
+                         "generate: scan + prompts + render (default) | "
+                         "inject: add knowledge-graph pointer to CLAUDE.md/AGENTS.md/cursor rules | "
+                         "check: exit 1 if knowledge graph is stale vs HEAD")
+    ap.add_argument("path", nargs="?", default=".", help="repository path (default: .)")
+    ap.add_argument("--out", default=None, help="output dir (default: <repo>/.repokg)")
+    ap.add_argument("--md", default=None, help="markdown output (default: <repo>/KNOWLEDGE_GRAPH.md)")
+    ap.add_argument("--no-github", action="store_true", help="skip gh PR lookup")
+    ap.add_argument("--pr-limit", type=int, default=1000, help="max PRs to fetch (default 1000)")
+    args = ap.parse_args(argv)
+
+    if args.command == "version":
+        print("repokg %s" % __version__)
+        return 0
+
+    repo = os.path.abspath(args.path)
+    if not os.path.isdir(repo):
+        print("error: %s is not a directory" % repo, file=sys.stderr)
+        return 2
+    out = args.out or os.path.join(repo, ".repokg")
+    md = args.md or os.path.join(repo, "KNOWLEDGE_GRAPH.md")
+
+    try:
+        if args.command == "scan":
+            scan(repo, out, args.no_github, args.pr_limit)
+        elif args.command == "prompts":
+            write_prompts(repo, out, md)
+        elif args.command == "render":
+            render(out, md)
+        elif args.command == "inject":
+            do_inject(repo, md)
+        elif args.command == "check":
+            return check(repo, out, md)
+        else:  # generate
+            scan(repo, out, args.no_github, args.pr_limit)
+            write_prompts(repo, out, md)
+            render(out, md)
+    except RuntimeError as e:
+        print("error: %s" % e, file=sys.stderr)
+        return 1
+    except FileNotFoundError as e:
+        print("error: %s (run `repokg scan` first?)" % e, file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

repokg-0.1.0/src/repokg/code.py

@@ -0,0 +1,104 @@
+"""Code collectors: language breakdown, module discovery, LOC."""
+
+import os
+import re
+
+SKIP_DIRS = {
+    ".git", "node_modules", "vendor", "dist", "build", "out", "target", ".next",
+    ".venv", "venv", "__pycache__", ".idea", ".vscode", ".repokg", "coverage",
+    ".terraform", "third_party", ".tox", ".mypy_cache", ".ruff_cache",
+    "site-packages", ".pytest_cache", ".cache",
+}
+
+LANG_BY_EXT = {
+    ".go": "Go", ".py": "Python", ".ts": "TypeScript", ".tsx": "TypeScript",
+    ".js": "JavaScript", ".jsx": "JavaScript", ".mjs": "JavaScript",
+    ".rs": "Rust", ".java": "Java", ".kt": "Kotlin", ".rb": "Ruby",
+    ".php": "PHP", ".cs": "C#", ".c": "C", ".cpp": "C++", ".cc": "C++",
+    ".h": "C/C++", ".hpp": "C++", ".sol": "Solidity", ".swift": "Swift",
+    ".scala": "Scala", ".ex": "Elixir", ".exs": "Elixir", ".zig": "Zig",
+    ".lua": "Lua", ".dart": "Dart", ".vue": "Vue", ".svelte": "Svelte",
+    ".sql": "SQL", ".sh": "Shell", ".proto": "Protobuf", ".tf": "Terraform",
+    ".yaml": "YAML", ".yml": "YAML", ".html": "HTML", ".css": "CSS",
+}
+
+# Languages whose presence makes a directory a "module" worth graphing.
+MODULE_LANGS = {
+    "Go", "Python", "TypeScript", "JavaScript", "Rust", "Java", "Kotlin",
+    "Ruby", "PHP", "C#", "C", "C++", "Solidity", "Swift", "Scala", "Elixir",
+    "Zig", "Lua", "Dart", "Vue", "Svelte",
+}
+
+ROOT_MARKERS = {
+    "go.mod", "package.json", "pyproject.toml", "setup.py", "Cargo.toml",
+    "pom.xml", "build.gradle", "composer.json", "Gemfile",
+}
+
+GENERATED_RE = re.compile(r"(generated|sqlcgen|_pb2|\.pb\.|/pb$|/pb/|bindings)")
+MAX_FILE_BYTES = 2_000_000
+
+
+def walk(repo):
+    """os.walk with skip-dir pruning; yields (rel_dir, filenames)."""
+    for dirpath, dirnames, filenames in os.walk(repo):
+        dirnames[:] = sorted(
+            d for d in dirnames
+            if d not in SKIP_DIRS and not (d.startswith(".") and d != ".github")
+        )
+        rel = os.path.relpath(dirpath, repo)
+        yield ("" if rel == "." else rel.replace(os.sep, "/")), filenames
+
+
+def count_loc(path):
+    try:
+        if os.path.getsize(path) > MAX_FILE_BYTES:
+            return 0
+        with open(path, "rb") as f:
+            data = f.read()
+        if b"\0" in data[:1024]:  # binary
+            return 0
+        return data.count(b"\n") + (1 if data and not data.endswith(b"\n") else 0)
+    except OSError:
+        return 0
+
+
+def collect(repo, tree=None):
+    """Return (languages, modules).
+
+    languages: [{lang, files, loc}] sorted by loc desc
+    modules:   [{path, lang, files, loc, root, generated}] one per dir containing code
+    tree: optional pre-built {rel_dir: filenames} to avoid re-walking the repo.
+    """
+    lang_stats = {}
+    modules = []
+    for rel, files in (tree.items() if tree is not None else walk(repo)):
+        if rel.startswith(".github"):
+            continue
+        per_lang = {}
+        is_root = any(f in ROOT_MARKERS for f in files)
+        for f in files:
+            ext = os.path.splitext(f)[1].lower()
+            lang = LANG_BY_EXT.get(ext)
+            if not lang:
+                continue
+            loc = count_loc(os.path.join(repo, rel, f))
+            ls = lang_stats.setdefault(lang, {"lang": lang, "files": 0, "loc": 0})
+            ls["files"] += 1
+            ls["loc"] += loc
+            if lang in MODULE_LANGS:
+                pl = per_lang.setdefault(lang, [0, 0])
+                pl[0] += 1
+                pl[1] += loc
+        if per_lang:
+            dominant = max(per_lang.items(), key=lambda kv: kv[1][1])
+            modules.append({
+                "path": rel or "(root)",
+                "lang": dominant[0],
+                "files": sum(v[0] for v in per_lang.values()),
+                "loc": sum(v[1] for v in per_lang.values()),
+                "root": is_root,
+                "generated": bool(GENERATED_RE.search("/" + rel)),
+            })
+    languages = sorted(lang_stats.values(), key=lambda x: -x["loc"])
+    modules.sort(key=lambda m: -m["loc"])
+    return languages, modules