savetoken 0.1.0__tar.gz

savetoken-0.1.0/PKG-INFO

@@ -0,0 +1,164 @@
+Metadata-Version: 2.4
+Name: savetoken
+Version: 0.1.0
+Summary: Convert Repomix/repominify output into compact AI-friendly codebase summaries using free LLMs.
+License: MIT
+Project-URL: Homepage, https://github.com/yourname/savetoken
+Project-URL: Issues, https://github.com/yourname/savetoken/issues
+Keywords: ai,llm,codebase,repomix,prompt,tokens,summarizer
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Documentation
+Classifier: Topic :: Utilities
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: mypy; extra == "dev"
+
+# savetoken 🪙
+
+> Convert your Repomix codebase dump into a **compact semantic summary** — optimized for AI prompts, with zero raw code.
+
+Instead of pasting 50,000 tokens of source code into ChatGPT or Claude, `savetoken` uses a **free LLM** to read the code once and produce a structured `codebrief.md` you reuse forever.
+
+---
+
+## How it fits in your workflow
+
+```
+Your project
+    ↓
+npx repomix              # packs codebase → repomix-output.xml
+    ↓
+savetoken summarize repomix-output.xml   # translates → codebrief.md  ← you are here
+    ↓
+Paste codebrief.md into your AI prompt   # ~500 tokens instead of 50k+
+```
+
+---
+
+## Install
+
+```bash
+pip install savetoken
+```
+
+Requires Python 3.10+. Zero dependencies beyond stdlib.
+
+---
+
+## Quick start
+
+```bash
+# 1. Pack your codebase with Repomix
+npx repomix
+
+# 2. Summarize with Gemini Flash (free tier)
+export GEMINI_API_KEY=your_key_here
+savetoken summarize repomix-output.xml
+
+# Output: codebrief.md — paste it into any AI prompt!
+```
+
+### Python API
+
+```python
+from savetoken import SaveToken
+
+st = SaveToken(provider="gemini", api_key="...")
+summary = st.summarize("repomix-output.xml")
+st.save(summary, "codebrief.md")
+```
+
+---
+
+## Providers
+
+| Provider | Free tier | Set env var |
+|---|---|---|
+| **Gemini Flash 2.0** ⭐ | ✅ Generous | `GEMINI_API_KEY` |
+| **Groq (Llama 3.3 70B)** | ✅ Fast | `GROQ_API_KEY` |
+| **Mistral Small** | ✅ Available | `MISTRAL_API_KEY` |
+| **OpenAI / DeepSeek** | Paid | `OPENAI_API_KEY` |
+
+Gemini is recommended — 1M token context window handles large codebases.
+
+---
+
+## CLI
+
+```bash
+savetoken summarize repomix-output.xml
+savetoken summarize repomix-output.xml --provider groq --output brief.md --lang pt
+savetoken summarize repomix-output.xml --force   # ignore cache, regenerate all
+savetoken clear-cache
+```
+
+| Flag | Default | Description |
+|---|---|---|
+| `--provider` | `gemini` | LLM provider |
+| `--output` | `codebrief.md` | Output file |
+| `--format` | `markdown` | `markdown` or `json` |
+| `--lang` | `en` | Description language |
+| `--force` | off | Skip cache |
+| `--cache-dir` | `.savetoken_cache` | Cache location |
+
+---
+
+## Caching
+
+`savetoken` hashes each file's content. On re-runs, **only changed files are re-summarized**. The cache lives in `.savetoken_cache/` — commit it to git to share across your team.
+
+---
+
+## VS Code Extension
+
+Install the extension and use:
+- **Command Palette → SaveToken: Summarize Codebase**
+- Right-click `repomix-output.xml` → **SaveToken: Summarize**
+- Configure provider and API key in Settings → SaveToken
+
+---
+
+## Example output (`codebrief.md`)
+
+```markdown
+# MyShop — savetoken summary
+
+E-commerce backend built with FastAPI and PostgreSQL.
+
+**Stack:** FastAPI, PostgreSQL, SQLAlchemy, Redis
+**Architecture:** Layered MVC
+**Entry points:** main.py
+
+## Directory Map
+- `src/api` — HTTP route handlers (REST)
+- `src/services` — Business logic layer
+- `src/models` — SQLAlchemy ORM entities
+
+## Files
+
+### `src/services/order.py`
+Manages the full lifecycle of customer orders.
+
+**Entities:**
+- Order(id, user_id, total, status: enum[pending, paid, cancelled])
+
+**Functions:**
+- create_order(user_id, items) → Order: validates stock and creates order
+- cancel_order(order_id) → bool: cancels if status allows
+- calculate_total(items) → Decimal: applies discounts and rounds
+```
+
+---
+
+## License
+
+MIT

savetoken-0.1.0/README.md

@@ -0,0 +1,140 @@
+# savetoken 🪙
+
+> Convert your Repomix codebase dump into a **compact semantic summary** — optimized for AI prompts, with zero raw code.
+
+Instead of pasting 50,000 tokens of source code into ChatGPT or Claude, `savetoken` uses a **free LLM** to read the code once and produce a structured `codebrief.md` you reuse forever.
+
+---
+
+## How it fits in your workflow
+
+```
+Your project
+    ↓
+npx repomix              # packs codebase → repomix-output.xml
+    ↓
+savetoken summarize repomix-output.xml   # translates → codebrief.md  ← you are here
+    ↓
+Paste codebrief.md into your AI prompt   # ~500 tokens instead of 50k+
+```
+
+---
+
+## Install
+
+```bash
+pip install savetoken
+```
+
+Requires Python 3.10+. Zero dependencies beyond stdlib.
+
+---
+
+## Quick start
+
+```bash
+# 1. Pack your codebase with Repomix
+npx repomix
+
+# 2. Summarize with Gemini Flash (free tier)
+export GEMINI_API_KEY=your_key_here
+savetoken summarize repomix-output.xml
+
+# Output: codebrief.md — paste it into any AI prompt!
+```
+
+### Python API
+
+```python
+from savetoken import SaveToken
+
+st = SaveToken(provider="gemini", api_key="...")
+summary = st.summarize("repomix-output.xml")
+st.save(summary, "codebrief.md")
+```
+
+---
+
+## Providers
+
+| Provider | Free tier | Set env var |
+|---|---|---|
+| **Gemini Flash 2.0** ⭐ | ✅ Generous | `GEMINI_API_KEY` |
+| **Groq (Llama 3.3 70B)** | ✅ Fast | `GROQ_API_KEY` |
+| **Mistral Small** | ✅ Available | `MISTRAL_API_KEY` |
+| **OpenAI / DeepSeek** | Paid | `OPENAI_API_KEY` |
+
+Gemini is recommended — 1M token context window handles large codebases.
+
+---
+
+## CLI
+
+```bash
+savetoken summarize repomix-output.xml
+savetoken summarize repomix-output.xml --provider groq --output brief.md --lang pt
+savetoken summarize repomix-output.xml --force   # ignore cache, regenerate all
+savetoken clear-cache
+```
+
+| Flag | Default | Description |
+|---|---|---|
+| `--provider` | `gemini` | LLM provider |
+| `--output` | `codebrief.md` | Output file |
+| `--format` | `markdown` | `markdown` or `json` |
+| `--lang` | `en` | Description language |
+| `--force` | off | Skip cache |
+| `--cache-dir` | `.savetoken_cache` | Cache location |
+
+---
+
+## Caching
+
+`savetoken` hashes each file's content. On re-runs, **only changed files are re-summarized**. The cache lives in `.savetoken_cache/` — commit it to git to share across your team.
+
+---
+
+## VS Code Extension
+
+Install the extension and use:
+- **Command Palette → SaveToken: Summarize Codebase**
+- Right-click `repomix-output.xml` → **SaveToken: Summarize**
+- Configure provider and API key in Settings → SaveToken
+
+---
+
+## Example output (`codebrief.md`)
+
+```markdown
+# MyShop — savetoken summary
+
+E-commerce backend built with FastAPI and PostgreSQL.
+
+**Stack:** FastAPI, PostgreSQL, SQLAlchemy, Redis
+**Architecture:** Layered MVC
+**Entry points:** main.py
+
+## Directory Map
+- `src/api` — HTTP route handlers (REST)
+- `src/services` — Business logic layer
+- `src/models` — SQLAlchemy ORM entities
+
+## Files
+
+### `src/services/order.py`
+Manages the full lifecycle of customer orders.
+
+**Entities:**
+- Order(id, user_id, total, status: enum[pending, paid, cancelled])
+
+**Functions:**
+- create_order(user_id, items) → Order: validates stock and creates order
+- cancel_order(order_id) → bool: cancels if status allows
+- calculate_total(items) → Decimal: applies discounts and rounds
+```
+
+---
+
+## License
+
+MIT

savetoken-0.1.0/pyproject.toml

@@ -0,0 +1,46 @@
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "savetoken"
+version = "0.1.0"
+description = "Convert Repomix/repominify output into compact AI-friendly codebase summaries using free LLMs."
+readme = "README.md"
+license = { text = "MIT" }
+requires-python = ">=3.10"
+keywords = ["ai", "llm", "codebase", "repomix", "prompt", "tokens", "summarizer"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Software Development :: Documentation",
+    "Topic :: Utilities",
+]
+# Zero required dependencies — uses only stdlib (urllib, json, hashlib, re, ast)
+dependencies = []
+
+[project.optional-dependencies]
+dev = ["pytest>=7", "pytest-cov", "ruff", "mypy"]
+
+[project.scripts]
+savetoken = "savetoken.cli:main"
+
+[project.urls]
+Homepage = "https://github.com/yourname/savetoken"
+Issues   = "https://github.com/yourname/savetoken/issues"
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["savetoken*"]
+
+[tool.ruff]
+line-length = 100
+target-version = "py310"
+
+[tool.mypy]
+python_version = "3.10"
+strict = false

savetoken-0.1.0/savetoken.egg-info/PKG-INFO

@@ -0,0 +1,164 @@
+Metadata-Version: 2.4
+Name: savetoken
+Version: 0.1.0
+Summary: Convert Repomix/repominify output into compact AI-friendly codebase summaries using free LLMs.
+License: MIT
+Project-URL: Homepage, https://github.com/yourname/savetoken
+Project-URL: Issues, https://github.com/yourname/savetoken/issues
+Keywords: ai,llm,codebase,repomix,prompt,tokens,summarizer
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Documentation
+Classifier: Topic :: Utilities
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+Requires-Dist: pytest>=7; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+Requires-Dist: ruff; extra == "dev"
+Requires-Dist: mypy; extra == "dev"
+
+# savetoken 🪙
+
+> Convert your Repomix codebase dump into a **compact semantic summary** — optimized for AI prompts, with zero raw code.
+
+Instead of pasting 50,000 tokens of source code into ChatGPT or Claude, `savetoken` uses a **free LLM** to read the code once and produce a structured `codebrief.md` you reuse forever.
+
+---
+
+## How it fits in your workflow
+
+```
+Your project
+    ↓
+npx repomix              # packs codebase → repomix-output.xml
+    ↓
+savetoken summarize repomix-output.xml   # translates → codebrief.md  ← you are here
+    ↓
+Paste codebrief.md into your AI prompt   # ~500 tokens instead of 50k+
+```
+
+---
+
+## Install
+
+```bash
+pip install savetoken
+```
+
+Requires Python 3.10+. Zero dependencies beyond stdlib.
+
+---
+
+## Quick start
+
+```bash
+# 1. Pack your codebase with Repomix
+npx repomix
+
+# 2. Summarize with Gemini Flash (free tier)
+export GEMINI_API_KEY=your_key_here
+savetoken summarize repomix-output.xml
+
+# Output: codebrief.md — paste it into any AI prompt!
+```
+
+### Python API
+
+```python
+from savetoken import SaveToken
+
+st = SaveToken(provider="gemini", api_key="...")
+summary = st.summarize("repomix-output.xml")
+st.save(summary, "codebrief.md")
+```
+
+---
+
+## Providers
+
+| Provider | Free tier | Set env var |
+|---|---|---|
+| **Gemini Flash 2.0** ⭐ | ✅ Generous | `GEMINI_API_KEY` |
+| **Groq (Llama 3.3 70B)** | ✅ Fast | `GROQ_API_KEY` |
+| **Mistral Small** | ✅ Available | `MISTRAL_API_KEY` |
+| **OpenAI / DeepSeek** | Paid | `OPENAI_API_KEY` |
+
+Gemini is recommended — 1M token context window handles large codebases.
+
+---
+
+## CLI
+
+```bash
+savetoken summarize repomix-output.xml
+savetoken summarize repomix-output.xml --provider groq --output brief.md --lang pt
+savetoken summarize repomix-output.xml --force   # ignore cache, regenerate all
+savetoken clear-cache
+```
+
+| Flag | Default | Description |
+|---|---|---|
+| `--provider` | `gemini` | LLM provider |
+| `--output` | `codebrief.md` | Output file |
+| `--format` | `markdown` | `markdown` or `json` |
+| `--lang` | `en` | Description language |
+| `--force` | off | Skip cache |
+| `--cache-dir` | `.savetoken_cache` | Cache location |
+
+---
+
+## Caching
+
+`savetoken` hashes each file's content. On re-runs, **only changed files are re-summarized**. The cache lives in `.savetoken_cache/` — commit it to git to share across your team.
+
+---
+
+## VS Code Extension
+
+Install the extension and use:
+- **Command Palette → SaveToken: Summarize Codebase**
+- Right-click `repomix-output.xml` → **SaveToken: Summarize**
+- Configure provider and API key in Settings → SaveToken
+
+---
+
+## Example output (`codebrief.md`)
+
+```markdown
+# MyShop — savetoken summary
+
+E-commerce backend built with FastAPI and PostgreSQL.
+
+**Stack:** FastAPI, PostgreSQL, SQLAlchemy, Redis
+**Architecture:** Layered MVC
+**Entry points:** main.py
+
+## Directory Map
+- `src/api` — HTTP route handlers (REST)
+- `src/services` — Business logic layer
+- `src/models` — SQLAlchemy ORM entities
+
+## Files
+
+### `src/services/order.py`
+Manages the full lifecycle of customer orders.
+
+**Entities:**
+- Order(id, user_id, total, status: enum[pending, paid, cancelled])
+
+**Functions:**
+- create_order(user_id, items) → Order: validates stock and creates order
+- cancel_order(order_id) → bool: cancels if status allows
+- calculate_total(items) → Decimal: applies discounts and rounds
+```
+
+---
+
+## License
+
+MIT

savetoken-0.1.0/savetoken.egg-info/SOURCES.txt

@@ -0,0 +1,9 @@
+README.md
+pyproject.toml
+savetoken.egg-info/PKG-INFO
+savetoken.egg-info/SOURCES.txt
+savetoken.egg-info/dependency_links.txt
+savetoken.egg-info/entry_points.txt
+savetoken.egg-info/requires.txt
+savetoken.egg-info/top_level.txt
+tests/test_savetoken.py

savetoken-0.1.0/savetoken.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

savetoken-0.1.0/savetoken.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+savetoken = savetoken.cli:main

savetoken-0.1.0/savetoken.egg-info/requires.txt

@@ -0,0 +1,6 @@
+
+[dev]
+pytest>=7
+pytest-cov
+ruff
+mypy

savetoken-0.1.0/savetoken.egg-info/top_level.txt

@@ -0,0 +1 @@
+

savetoken-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

savetoken-0.1.0/tests/test_savetoken.py

@@ -0,0 +1,741 @@
+"""Tests for savetoken v0.4"""
+import json, pytest
+from savetoken.core import _parse_repomix, _hash, _format_sigs, _compact_sig
+from savetoken.output import render_markdown, render_json, _e
+from savetoken.models import *
+from savetoken.exceptions import ParseError, InputError
+from savetoken.providers.base import _parse_json, _safe_truncate, _ensure_list
+
+REPOMIX_XML = """
+<file path="src/main.py">
+def main():
+    print("hello")
+</file>
+<file path="src/orders.py">
+def create_order(user_id, items):
+    return reserve(items)
+def reserve(items):
+    return items
+</file>
+"""
+
+# ══════════════════════════════════════════════════════════════════════
+# Parser
+# ══════════════════════════════════════════════════════════════════════
+def test_parse_xml():      assert len(_parse_repomix(REPOMIX_XML)) == 2
+def test_parse_plain():    assert len(_parse_repomix("===== a.py =====\ncode\n===== b.py =====\nmore\n")) == 2
+def test_parse_empty():    assert _parse_repomix("   ") == []
+def test_parse_no_empty(): assert len(_parse_repomix('<file path="e.py"></file><file path="r.py">x</file>')) == 1
+def test_hash_stable():    assert _hash("x") == _hash("x") != _hash("y")
+
+# ══════════════════════════════════════════════════════════════════════
+# Classifier
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.classifier import classify, extract_critical_comments
+def test_cls_entry():      assert classify("main.py","") == FileRole.STRATEGIC
+def test_cls_schema():     assert classify("src/schemas/order.py","") == FileRole.STRATEGIC
+def test_cls_test():       assert classify("tests/test_orders.py","") == FileRole.STRATEGIC
+def test_cls_config():     assert classify("settings.py","") == FileRole.STRATEGIC
+def test_cls_hack():       assert classify("util.py","# HACK: do not remove") == FileRole.STRATEGIC
+def test_cls_pydantic():   assert classify("dto.py","class OrderIn(BaseModel):\n    id: int") == FileRole.STRATEGIC
+def test_cls_proto():      assert classify("order.proto","syntax='proto3'") == FileRole.STRATEGIC
+def test_cls_adr():        assert classify("docs/adr/001.md","# ADR") == FileRole.STRATEGIC
+def test_cls_exception():  assert classify("src/exceptions/base.py","") == FileRole.STRATEGIC
+def test_cls_skip_pyc():   assert classify("a.cpython-312.pyc","") == FileRole.SKIP
+def test_cls_skip_mig():   assert classify("migrations/0001_init.py","") == FileRole.SKIP
+def test_cls_skip_lock():  assert classify("package-lock.json","{}") == FileRole.SKIP
+def test_cls_regular():    assert classify("src/services/billing.py","def charge():\n    pass\n" * 30) == FileRole.REGULAR
+def test_cls_small_file(): assert classify("src/tiny.py","x=1\ny=2") == FileRole.STRATEGIC
+def test_extract_crit():
+    c = extract_critical_comments("x=1\n# HACK: bug\ny=2\n# FIXME: broken\n# HACK: bug")
+    assert len(c) == 2
+
+# ══════════════════════════════════════════════════════════════════════
+# AST types
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.ast_types import extract_signatures
+
+def test_ast_basic_sig():
+    chunks = [{"path":"a.py","content":"def foo(x: int, y: str) -> bool:\n    return True\n"}]
+    sigs = extract_signatures(chunks)
+    assert "a.py" in sigs
+    fns = sigs["a.py"].functions
+    assert fns[0].name == "foo"
+    assert fns[0].return_type == "bool"
+    assert fns[0].params[0].annotation == "int"
+
+def test_ast_async():
+    chunks = [{"path":"a.py","content":"async def bar() -> None:\n    pass\n"}]
+    sigs = extract_signatures(chunks)
+    assert sigs["a.py"].functions[0].is_async
+
+def test_ast_raises():
+    chunks = [{"path":"a.py","content":"def f():\n    raise ValueError('x')\n"}]
+    sigs = extract_signatures(chunks)
+    assert "ValueError" in sigs["a.py"].functions[0].raises
+
+def test_ast_module_doc():
+    chunks = [{"path":"a.py","content":'"""Order service module."""\ndef f(): pass\n'}]
+    sigs = extract_signatures(chunks)
+    assert "Order service" in sigs["a.py"].module_doc
+
+def test_ast_class_methods():
+    chunks = [{"path":"a.py","content":"class Svc:\n    def run(self, x: int): pass\n"}]
+    sigs = extract_signatures(chunks)
+    assert "Svc" in sigs["a.py"].classes
+    assert sigs["a.py"].classes["Svc"][0].name == "run"
+
+def test_ast_non_python():
+    sigs = extract_signatures([{"path":"style.css","content":"body{}"}])
+    assert sigs == {}
+
+def test_ast_compact():
+    from savetoken.ast_types import FunctionSignature, Param
+    sig = FunctionSignature("create_order",[Param("uid","UUID"),Param("req","CreateOrderRequest")],"OrderResponse",["PaymentError"],True)
+    c = sig.compact()
+    assert "async" in c
+    assert "UUID" in c
+    assert "OrderResponse" in c
+    assert "PaymentError" in c
+
+def test_format_sigs_roundtrip():
+    chunks = [{"path":"a.py","content":"def foo(x: int) -> bool:\n    pass\n"}]
+    sigs = extract_signatures(chunks)
+    from dataclasses import asdict
+    sigs_dict = asdict(sigs["a.py"])
+    result = _format_sigs(sigs_dict)
+    assert "foo" in result and "int" in result and "bool" in result
+
+# ══════════════════════════════════════════════════════════════════════
+# Analyzer (prod/test split)
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.analyzer import build_graph
+def test_graph_cross_file():
+    g = build_graph([{"path":"a.py","content":"def foo():\n    bar()\n"},
+                     {"path":"b.py","content":"def bar():\n    pass\n"}])
+    assert any(e.caller_file=="a.py" and e.callee_fn=="bar" for e in g.calls)
+
+def test_graph_test_separation():
+    g = build_graph([{"path":"test_x.py","content":"def test_foo():\n    create()\n"},
+                     {"path":"service.py","content":"def create():\n    pass\n"}])
+    assert all(e.caller_file != "test_x.py" for e in g.calls)
+    assert any(e.caller_file == "test_x.py" for e in g.test_calls)
+
+def test_graph_inheritance():
+    g = build_graph([{"path":"base.py","content":"class Animal:\n    pass\n"},
+                     {"path":"dog.py","content":"class Dog(Animal):\n    pass\n"}])
+    assert any(e.child_class=="Dog" for e in g.inheritance)
+
+def test_graph_no_self_calls():
+    g = build_graph([{"path":"a.py","content":"def foo():\n    foo()\n"}])
+    assert all(e.caller_file != e.callee_file for e in g.calls)
+
+def test_graph_syntax_error():
+    g = build_graph([{"path":"bad.py","content":"def (x:"}])
+    assert g.calls == []
+
+# ══════════════════════════════════════════════════════════════════════
+# Ranker
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.ranker import rank_functions
+def test_ranker_basic():
+    edges = [CallEdge("a.py","foo","b.py","bar"), CallEdge("c.py","baz","b.py","bar")]
+    chunks = [{"path":"b.py","content":"def bar():\n    return 1\n"}]
+    hot = rank_functions(edges, chunks, top_n=5)
+    assert any(h.name=="bar" for h in hot)
+
+def test_ranker_empty(): assert rank_functions([],[],top_n=5) == []
+
+def test_ranker_budget():
+    edges = [CallEdge("a.py","foo","b.py","bar")]
+    chunks = [{"path":"b.py","content":"def bar():\n    " + "x=1\n" * 1000}]
+    assert rank_functions(edges, chunks, top_n=5, token_budget=5) == []
+
+# ══════════════════════════════════════════════════════════════════════
+# Env extractor
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.env_extractor import extract_env_vars
+def test_env_getenv():
+    evs = extract_env_vars([{"path":"a.py","content":"import os\nX=os.getenv('DB_URL','sqlite')\n"}])
+    assert any(e.name=="DB_URL" and e.default=="sqlite" for e in evs)
+
+def test_env_environ():
+    evs = extract_env_vars([{"path":"a.py","content":"import os\nX=os.environ['SECRET']\n"}])
+    assert any(e.name=="SECRET" and e.required for e in evs)
+
+def test_env_dotenv():
+    evs = extract_env_vars([{"path":".env.example","content":"DB=postgres\nDEBUG=false\n"}])
+    assert {e.name for e in evs} >= {"DB","DEBUG"}
+
+def test_env_dedup():
+    chunks=[{"path":"a.py","content":"import os\nX=os.getenv('V')\n"},
+            {"path":"b.py","content":"import os\nY=os.getenv('V','d')\n"}]
+    matches=[e for e in extract_env_vars(chunks) if e.name=="V"]
+    assert len(matches)==1 and len(matches[0].files)==2
+
+# ══════════════════════════════════════════════════════════════════════
+# SQL schema extractor
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.schema_sql import extract_sql_schema
+def test_sql_raw():
+    tables = extract_sql_schema([{"path":"s.sql","content":"CREATE TABLE orders (id INT, total FLOAT);\n"}])
+    assert any(t.name=="orders" for t in tables)
+
+def test_sql_fk():
+    sql = "CREATE TABLE orders (id INT, user_id INT, FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE);"
+    tables = extract_sql_schema([{"path":"s.sql","content":sql}])
+    t = next(t for t in tables if t.name=="orders")
+    assert any("users" in fk for fk in t.foreign_keys)
+
+def test_sql_django():
+    content = "from django.db import models\nclass Order(models.Model):\n    total = models.DecimalField(max_length=10)\n"
+    tables = extract_sql_schema([{"path":"models.py","content":content}])
+    assert any(t.name=="order" for t in tables)
+
+def test_sql_sqla():
+    content = "from sqlalchemy import Column,Integer\nclass Product(Base):\n    __tablename__='products'\n    id=Column(Integer,nullable=False)\n"
+    tables = extract_sql_schema([{"path":"m.py","content":content}])
+    assert any(t.name=="products" for t in tables)
+
+def test_sql_empty():
+    assert extract_sql_schema([{"path":"a.py","content":"x=1"}]) == []
+
+# ══════════════════════════════════════════════════════════════════════
+# Critical context
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.critical_context import extract_critical_blocks
+def test_crit_window():
+    lines = [f"line{i}" for i in range(30)]
+    lines[15] = "# HACK: workaround"
+    blocks = extract_critical_blocks("f.py","\n".join(lines))
+    assert len(blocks)==1 and "HACK" in blocks[0].marker and "line5" in blocks[0].context
+
+def test_crit_no_overlap():
+    lines = ["# HACK: a" if i==5 else "# HACK: b" if i==6 else f"l{i}" for i in range(30)]
+    assert len(extract_critical_blocks("f.py","\n".join(lines)))==1
+
+def test_crit_empty():
+    assert extract_critical_blocks("f.py","x=1") == []
+
+# ══════════════════════════════════════════════════════════════════════
+# Budget
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.budget import allocate_budget, compute_total_budget, estimate_tokens, trim_to_budget
+
+def test_budget_all_fit():
+    chunks = [{"path":"a.py","content":"x=1"},{"path":"b.py","content":"y=2"}]
+    r = allocate_budget(chunks, sum(estimate_tokens(c["content"]) for c in chunks), 2.0)
+    assert set(r.included) == {"a.py","b.py"} and r.excluded == []
+
+def test_budget_drop_tests_first():
+    big = "x=1\n" * 5000
+    chunks = [{"path":"tests/test_x.py","content":big},
+              {"path":"src/schemas.py","content":"class A: pass"}]
+    total = estimate_tokens(big) + estimate_tokens("class A: pass")
+    # budget just enough for schemas but not tests
+    r = allocate_budget(chunks, total, 0.15)
+    assert "src/schemas.py" in r.included
+    assert "tests/test_x.py" in r.excluded
+
+def test_budget_no_truncate():
+    # Even if over budget, included files are whole (not truncated)
+    chunks = [{"path":"settings.py","content":"X=1\nY=2\nZ=3\n"}]
+    r = allocate_budget(chunks, estimate_tokens(chunks[0]["content"]) * 10, 2.0)
+    assert "settings.py" in r.included
+
+def test_compute_total_budget():
+    assert compute_total_budget(10_000, 0.5) == 5_000
+    assert compute_total_budget(100, 0.5) == 2_000   # minimum floor
+
+def test_trim_to_budget():
+    r = trim_to_budget("line1\nline2\n"+"x"*1000, 5)
+    assert "line1" in r and "truncated" in r
+
+# ══════════════════════════════════════════════════════════════════════
+# Deduplicator
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.deduplicator import deduplicate
+
+def _make_dup_summary() -> CodebaseSummary:
+    return CodebaseSummary(
+        files=[
+            FileSummary("src/schemas.py","Schemas",FileRole.STRATEGIC.value,verbatim="class A: pass"),
+            FileSummary("src/orders.py","Orders",FileRole.REGULAR.value,
+                        functions=["create_order()->Order"]),
+        ],
+        overview=ProjectOverview("P","Desc"),
+        graph=CodebaseGraph(
+            calls=[CallEdge("src/orders.py","create","src/schemas.py","A")],
+            inheritance=[
+                InheritanceEdge("src/schemas.py","A","","BaseModel"),  # external
+                InheritanceEdge("src/orders.py","OrderPaid","src/orders.py","Order"),  # internal
+            ],
+            test_calls=[CallEdge("tests/test_x.py","test_a","src/orders.py","create")],
+        ),
+        hot_functions=[
+            HotFunction("src/schemas.py","A",0.9,"def A(): pass",3),  # verbatim file
+            HotFunction("src/orders.py","create",0.5,"def create(): pass",1),
+        ],
+        sql_tables=[
+            SqlTable("schemas_a","","src/schemas.py"),   # verbatim source
+            SqlTable("orders","","src/orders.py"),        # non-verbatim
+        ],
+        critical_blocks=[
+            CriticalBlock("src/schemas.py",1,"HACK","ctx"),  # in verbatim
+            CriticalBlock("src/orders.py",5,"FIXME","ctx"),  # not verbatim
+        ],
+    )
+
+def test_dedup_removes_sql_from_verbatim():
+    s = deduplicate(_make_dup_summary())
+    assert all(t.source != "src/schemas.py" for t in s.sql_tables)
+    assert any(t.name == "orders" for t in s.sql_tables)
+
+def test_dedup_removes_external_inheritance():
+    s = deduplicate(_make_dup_summary())
+    assert all(e.parent_class != "BaseModel" for e in s.graph.inheritance)
+
+def test_dedup_keeps_internal_inheritance():
+    s = deduplicate(_make_dup_summary())
+    assert any(e.child_class == "OrderPaid" for e in s.graph.inheritance)
+
+def test_dedup_removes_hot_from_verbatim():
+    s = deduplicate(_make_dup_summary())
+    assert all(h.file != "src/schemas.py" for h in s.hot_functions)
+    assert any(h.name == "create" for h in s.hot_functions)
+
+def test_dedup_removes_crit_from_verbatim():
+    s = deduplicate(_make_dup_summary())
+    assert all(b.file != "src/schemas.py" for b in s.critical_blocks)
+    assert any(b.file == "src/orders.py" for b in s.critical_blocks)
+
+def test_dedup_file_summaries_regular_only():
+    s = deduplicate(_make_dup_summary())
+    assert all(f.role == FileRole.REGULAR.value for f in s._dedup_file_summaries)
+
+# ══════════════════════════════════════════════════════════════════════
+# Provider base
+# ══════════════════════════════════════════════════════════════════════
+def test_parse_json_clean():   assert _parse_json('{"a":1}') == {"a":1}
+def test_parse_json_fences():  assert _parse_json("```json\n{\"a\":1}\n```") == {"a":1}
+def test_parse_json_raises():
+    with pytest.raises(ParseError): _parse_json("nope","test")
+def test_safe_truncate():
+    assert _safe_truncate("short",100) == "short"
+    assert "truncated" in _safe_truncate("a\nb\n"+"x"*500, 10)
+def test_ensure_list():
+    assert _ensure_list(["a"]) == ["a"]
+    assert _ensure_list(None)  == []
+    assert _ensure_list("x")   == ["x"]
+
+# ══════════════════════════════════════════════════════════════════════
+# Output renderer
+# ══════════════════════════════════════════════════════════════════════
+
+def _full_summary() -> CodebaseSummary:
+    s = CodebaseSummary(
+        files=[
+            FileSummary("src/orders.py","Manages orders",FileRole.REGULAR.value,
+                        functions=["create_order(uid,items)->Order"],
+                        entities=["Order(id,total,status)"],
+                        critical_comments=["# HACK: ORM bug"],
+                        module_doc="Order lifecycle management."),
+            FileSummary("src/schemas.py","Schemas",FileRole.STRATEGIC.value,
+                        verbatim="class OrderIn(BaseModel):\n    id: int",
+                        module_doc="Pydantic schemas."),
+        ],
+        overview=ProjectOverview("Shop","E-commerce",["FastAPI","PostgreSQL"],
+                                 "Layered MVC",["main.py"],{"src":"code"},
+                                 {"fastapi":"0.100.0","sqlalchemy":"1.4.0"}),
+        graph=CodebaseGraph(
+            calls=[CallEdge("src/orders.py","create_order","src/inv.py","reserve")],
+            inheritance=[InheritanceEdge("src/orders.py","PaidOrder","src/orders.py","Order")],
+            test_calls=[CallEdge("tests/t.py","test_create","src/orders.py","create_order")],
+        ),
+        flows=[Flow("POST order","main.py","post_order",
+                    [FlowStep("main.py","post_order"),FlowStep("src/orders.py","create_order")],
+                    "Creates an order.")],
+        hot_functions=[HotFunction("src/orders.py","create_order",0.9,
+                                   "def create_order(uid,items):\n    return items",2)],
+        env_vars=[EnvVar("DATABASE_URL","postgresql://localhost",["src/db.py"],True)],
+        sql_tables=[SqlTable("orders",["id INT","total FLOAT"],"src/models.py",
+                             foreign_keys=["user_id → users.id ON DELETE CASCADE"],
+                             indexes=["(user_id, created_at)"])],
+        critical_blocks=[CriticalBlock("src/orders.py",42,"HACK","line41\n>>> # HACK\nline43")],
+    )
+    s._dedup_file_summaries = [s.files[0]]
+    return s
+
+def test_md_has_toc():
+    md = render_markdown(_full_summary())
+    assert "## Contents" in md
+
+def test_md_overview():
+    md = render_markdown(_full_summary())
+    assert "Shop" in md and "FastAPI" in md
+
+def test_md_dep_versions():
+    md = render_markdown(_full_summary())
+    assert "fastapi==0.100.0" in md
+
+def test_md_env_table():
+    md = render_markdown(_full_summary())
+    assert "DATABASE_URL" in md
+
+def test_md_sql_fk():
+    md = render_markdown(_full_summary())
+    assert "Database Schema" in md
+    assert "users.id" in md
+    assert "(user_id, created_at)" in md
+
+def test_md_flows_compact():
+    md = render_markdown(_full_summary())
+    assert "Execution Flows" in md
+    assert "post_order" in md
+
+def test_md_call_graph_compressed():
+    md = render_markdown(_full_summary())
+    assert "Call Graph" in md
+    assert "reserve" in md
+
+def test_md_test_coverage_line():
+    md = render_markdown(_full_summary())
+    assert "test_create" in md or "Tests cover" in md
+
+def test_md_hot_functions():
+    md = render_markdown(_full_summary())
+    assert "Hot Functions" in md
+
+def test_md_verbatim_files():
+    md = render_markdown(_full_summary())
+    assert "Verbatim Files" in md
+    assert "OrderIn" in md
+    assert "Pydantic schemas." in md   # module_doc
+
+def test_md_critical_blocks():
+    md = render_markdown(_full_summary())
+    assert "Critical Blocks" in md
+
+def test_md_file_summaries_module_doc():
+    md = render_markdown(_full_summary())
+    assert "Order lifecycle management." in md
+
+def test_md_source_tags():
+    md = render_markdown(_full_summary())
+    assert "[ast+verbatim]" in md or "[verbatim]" in md or "[llm]" in md
+
+def test_json_valid():
+    data = json.loads(render_json(_full_summary()))
+    assert all(k in data for k in ("overview","files","graph","hot_functions",
+                                   "env_vars","sql_tables","critical_blocks"))
+    assert "_dedup_file_summaries" not in data   # stripped from JSON
+
+def test_escape(): assert _e("a`b") == "a'b" and _e("a|b") == "a-b"
+
+# ══════════════════════════════════════════════════════════════════════
+# Cache
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.cache import CacheManager
+def test_cache_roundtrip(tmp_path):
+    cm = CacheManager(tmp_path/"c"); cm.set("k",{"x":1}); assert cm.get("k")=={"x":1}
+def test_cache_miss(tmp_path):     assert CacheManager(tmp_path/"c").get("x") is None
+def test_cache_corrupt(tmp_path):
+    cm = CacheManager(tmp_path/"c"); (tmp_path/"c"/"bad.json").write_text("bad"); assert cm.get("bad") is None
+def test_cache_clear(tmp_path):
+    cm = CacheManager(tmp_path/"c"); cm.set("k",{}); cm.clear(); assert cm.get("k") is None
+def test_cache_stats(tmp_path):
+    cm = CacheManager(tmp_path/"c"); cm.set("a",{}); cm.set("b",{}); assert cm.stats()["entries"]==2
+
+# ══════════════════════════════════════════════════════════════════════
+# Core integration with mock provider
+# ══════════════════════════════════════════════════════════════════════
+class MockProvider:
+    name = "mock"
+    def summarize_file(self,path,content,language="en",signatures=""):
+        return FileSummary(path=path,purpose=f"Mock:{path}")
+    def summarize_project(self,files,language="en"):
+        return ProjectOverview(name="MockProject",description="Mocked.",entry_points=["main.py"])
+    def describe_flow(self,flow,language="en"):
+        return f"Flow:{flow.name}"
+
+def _st(tmp_path):
+    from savetoken.core import SaveToken
+    st = SaveToken.__new__(SaveToken)
+    from savetoken.cache import CacheManager
+    st.provider     = MockProvider()
+    st.cache        = CacheManager(tmp_path/"cache")
+    st.language     = "en"
+    st.flow_depth   = 3
+    st.target_ratio = 0.5
+    return st
+
+def test_core_xml(tmp_path):
+    src = tmp_path/"r.xml"; src.write_text(REPOMIX_XML)
+    s = _st(tmp_path).summarize(src)
+    assert len(s.files)==2 and s.overview.name=="MockProject"
+
+def test_core_text(tmp_path):
+    s = _st(tmp_path).summarize_text(REPOMIX_XML)
+    assert len(s.files)==2
+
+def test_core_strategic_verbatim(tmp_path):
+    # settings.py → strategic, included verbatim (pad to ensure budget)
+    padding = "".join(f'<file path="s{i}.py">{"x=1\n"*100}</file>' for i in range(20))
+    xml = '<file path="settings.py">DEBUG=True\nSECRET=x\n</file>' + padding
+    s = _st(tmp_path).summarize_text(xml)
+    strat = [f for f in s.files if f.role==FileRole.STRATEGIC.value and f.verbatim]
+    assert any(f.path=="settings.py" for f in strat)
+
+def test_core_skips_lockfile(tmp_path):
+    xml = '<file path="package-lock.json">{}</file><file path="app.py">x=1</file>'
+    s = _st(tmp_path).summarize_text(xml)
+    assert all(f.path!="package-lock.json" for f in s.files)
+
+def test_core_env_vars(tmp_path):
+    xml = '<file path="a.py">import os\nX=os.getenv("MY_VAR","d")\n</file>'
+    s = _st(tmp_path).summarize_text(xml)
+    assert isinstance(s.env_vars, list)
+
+def test_core_sql_tables(tmp_path):
+    xml = '<file path="s.sql">CREATE TABLE items (id INT);</file>'
+    s = _st(tmp_path).summarize_text(xml)
+    assert isinstance(s.sql_tables, list)
+
+def test_core_graph(tmp_path):
+    xml = '<file path="a.py">def foo():\n    bar()\n</file><file path="b.py">def bar():\n    pass\n</file>'
+    s = _st(tmp_path).summarize_text(xml)
+    assert isinstance(s.graph, CodebaseGraph)
+
+def test_core_dedup_applied(tmp_path):
+    # After dedup, _dedup_file_summaries should only contain REGULAR files
+    s = _st(tmp_path).summarize_text(REPOMIX_XML)
+    for f in s._dedup_file_summaries:
+        assert f.role == FileRole.REGULAR.value
+
+def test_core_cache(tmp_path):
+    st = _st(tmp_path)
+    src = tmp_path/"r.xml"; src.write_text(REPOMIX_XML)
+    st.summarize(src)
+    class Fail:
+        name="fail"
+        def summarize_file(self,*a,**k): raise AssertionError("cache miss")
+        def summarize_project(self,*a,**k): raise AssertionError("cache miss")
+        def describe_flow(self,*a,**k): raise AssertionError("cache miss")
+    st.provider=Fail(); st.summarize(src)  # must not raise
+
+def test_core_missing(tmp_path):
+    with pytest.raises(InputError): _st(tmp_path).summarize(tmp_path/"nope.xml")
+
+def test_core_empty(tmp_path):
+    src=tmp_path/"e.xml"; src.write_text("   ")
+    with pytest.raises(InputError): _st(tmp_path).summarize(src)
+
+def test_core_save_md(tmp_path):
+    s = _st(tmp_path).summarize_text(REPOMIX_XML)
+    out = _st(tmp_path).save(s, tmp_path/"b.md")
+    assert "MockProject" in out.read_text()
+
+def test_core_save_json(tmp_path):
+    s = _st(tmp_path).summarize_text(REPOMIX_XML)
+    out = _st(tmp_path).save(s, tmp_path/"b.json", fmt="json")
+    data = json.loads(out.read_text())
+    assert all(k in data for k in ("overview","files","graph","hot_functions","env_vars","sql_tables"))
+    assert "_dedup_file_summaries" not in data
+
+def test_core_progress(tmp_path):
+    calls=[]
+    _st(tmp_path).summarize_text(REPOMIX_XML,on_progress=lambda c,t,p:calls.append(c))
+    assert len(calls)>=2
+
+def test_flows(tmp_path):
+    from savetoken.flows import trace_flows
+    g = CodebaseGraph(calls=[
+        CallEdge("main.py","post_order","orders.py","create_order"),
+        CallEdge("orders.py","create_order","inv.py","reserve"),
+    ])
+    flows = trace_flows(g, entry_file_hints=["main.py"])
+    assert any(s.fn=="post_order" for s in flows[0].steps)
+
+
+# ══════════════════════════════════════════════════════════════════════
+# Docstring extractor
+# ══════════════════════════════════════════════════════════════════════
+from savetoken.docstring_extractor import (
+    extract_file_docs, render_file_doc, FileDoc, FnDoc, ClassDoc, ConstantDoc
+)
+
+_RICH_PY = '''\
+"""Order service — manages the full lifecycle of customer orders."""
+from decimal import Decimal
+
+MAX_ITEMS = 50
+SUPPORTED_CURRENCIES = ("BRL", "USD", "EUR")
+
+class OrderService:
+    """Orchestrates order creation, payment, and cancellation."""
+
+    def create_order(self, user_id: str, items: list) -> dict:
+        """Create a new order, reserve inventory, and charge payment."""
+        pass
+
+    def cancel_order(self, order_id: str) -> bool:
+        """Cancel an order if status allows. Returns True on success."""
+        raise ValueError("not cancellable")
+
+    def _internal(self):
+        """Private helper — should not appear in output."""
+        pass
+
+def calculate_discount(total: Decimal, code: str) -> Decimal:
+    """Apply coupon code discount to total. Returns discounted amount."""
+    pass
+'''
+
+def test_docext_module_doc():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    assert "Order service" in docs["a.py"].module_doc
+
+def test_docext_constants():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    names = [c.name for c in docs["a.py"].constants]
+    assert "MAX_ITEMS" in names
+    assert "SUPPORTED_CURRENCIES" in names
+
+def test_docext_class_doc():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    cls = docs["a.py"].classes[0]
+    assert cls.name == "OrderService"
+    assert "Orchestrates" in cls.doc
+
+def test_docext_method_docs():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    cls = docs["a.py"].classes[0]
+    method_names = [m.name for m in cls.methods]
+    assert "create_order" in method_names
+    assert "cancel_order" in method_names
+
+def test_docext_method_raises():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    cls = docs["a.py"].classes[0]
+    cancel = next(m for m in cls.methods if m.name == "cancel_order")
+    assert "ValueError" in cancel.raises
+
+def test_docext_function_doc():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    fns = {f.name: f for f in docs["a.py"].functions}
+    assert "calculate_discount" in fns
+    assert "coupon" in fns["calculate_discount"].doc
+
+def test_docext_private_excluded_from_render():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    rendered = render_file_doc(docs["a.py"])
+    assert "_internal" not in rendered
+
+def test_docext_signature_types():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    cls = docs["a.py"].classes[0]
+    create = next(m for m in cls.methods if m.name == "create_order")
+    assert "str" in create.signature
+    assert "list" in create.signature
+
+def test_docext_render_has_all_sections():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    rendered = render_file_doc(docs["a.py"])
+    assert "Order service" in rendered        # module doc
+    assert "MAX_ITEMS" in rendered            # constant
+    assert "OrderService" in rendered         # class
+    assert "create_order" in rendered         # method
+    assert "calculate_discount" in rendered   # function
+
+def test_docext_non_python_skipped():
+    docs = extract_file_docs([{"path": "style.css", "content": "body{}"}])
+    assert docs == {}
+
+def test_docext_syntax_error_skipped():
+    docs = extract_file_docs([{"path": "bad.py", "content": "def (x:"}])
+    assert docs == {}
+
+def test_docext_empty_file():
+    docs = extract_file_docs([{"path": "empty.py", "content": ""}])
+    fd = docs.get("empty.py")
+    assert fd is not None
+    assert fd.module_doc == ""
+    assert fd.constants == []
+
+def test_docext_token_estimate():
+    docs = extract_file_docs([{"path": "a.py", "content": _RICH_PY}])
+    tok = docs["a.py"].token_estimate()
+    assert tok > 0
+
+# ══════════════════════════════════════════════════════════════════════
+# Core integration — AST role
+# ══════════════════════════════════════════════════════════════════════
+
+def test_core_regular_python_gets_ast_role(tmp_path):
+    """Regular Python files should get FileRole.AST, not REGULAR."""
+    # A file big enough to be REGULAR (>25 lines, has external imports)
+    big_py = "import requests\n" + "def fn():\n    pass\n" * 40
+    xml = f'<file path="service.py">{big_py}</file>'
+    s = _st(tmp_path).summarize_text(xml)
+    service = next((f for f in s.files if f.path == "service.py"), None)
+    assert service is not None
+    assert service.role == FileRole.AST.value
+
+def test_core_ast_file_has_ast_doc(tmp_path):
+    """AST files must have ast_doc populated."""
+    big_py = '"""My service module."""\nimport requests\n' + "def fn():\n    '''Does nothing.'''\n    pass\n" * 40
+    xml = f'<file path="service.py">{big_py}</file>'
+    s = _st(tmp_path).summarize_text(xml)
+    service = next(f for f in s.files if f.path == "service.py")
+    assert service.ast_doc  # non-empty
+
+def test_core_ast_file_no_llm_called(tmp_path):
+    """AST files must NOT trigger LLM calls."""
+    big_py = "import requests\n" + "def fn(): pass\n" * 40
+    xml = f'<file path="service.py">{big_py}</file>'
+
+    call_count = {"n": 0}
+    class TrackingProvider:
+        name = "tracking"
+        def summarize_file(self, *a, **kw):
+            call_count["n"] += 1
+            return FileSummary(path=a[0], purpose="mock")
+        def summarize_project(self, *a, **kw):
+            return ProjectOverview(name="P", description="D")
+        def describe_flow(self, *a, **kw): return ""
+
+    st = _st(tmp_path)
+    st.provider = TrackingProvider()
+    st.summarize_text(xml)
+    # summarize_file should NOT have been called for the .py file
+    assert call_count["n"] == 0
+
+def test_core_ast_section_in_markdown(tmp_path):
+    """Rendered output must contain the File Docs [ast] section."""
+    big_py = '"""My module."""\nimport requests\n' + "def fn(): pass\n" * 40
+    xml = f'<file path="service.py">{big_py}</file>'
+    s = _st(tmp_path).summarize_text(xml)
+    out = _st(tmp_path).save(s, tmp_path / "brief.md")
+    md = out.read_text()
+    assert "File Docs" in md
+    assert "My module" in md
+
+def test_core_non_python_regular_still_uses_llm(tmp_path):
+    """Non-Python regular files should still use LLM summary."""
+    call_count = {"n": 0}
+    class TrackingProvider:
+        name = "tracking"
+        def summarize_file(self, *a, **kw):
+            call_count["n"] += 1
+            return FileSummary(path=a[0], purpose="mock")
+        def summarize_project(self, *a, **kw):
+            return ProjectOverview(name="P", description="D")
+        def describe_flow(self, *a, **kw): return ""
+
+    # A non-Python file (YAML config large enough to not be strategic)
+    big_yaml = "key: value\n" * 200
+    xml = f'<file path="config.yml">{big_yaml}</file>'
+    st = _st(tmp_path)
+    st.provider = TrackingProvider()
+    st.summarize_text(xml)
+    assert call_count["n"] >= 1