eval-doctor 0.1.0__tar.gz

eval_doctor-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Sneha Upadhyaula
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

eval_doctor-0.1.0/PKG-INFO

@@ -0,0 +1,131 @@
+Metadata-Version: 2.4
+Name: eval-doctor
+Version: 0.1.0
+Summary: Know what you are not testing before you ship - an eval coverage auditor for RAG and agent systems.
+Author: Sneha Upadhyaula
+License: MIT
+Keywords: evals,llm,rag,agents,testing,evaluation
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Software Development :: Quality Assurance
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: typer>=0.9
+Requires-Dist: pydantic>=2.0
+Requires-Dist: rich>=13.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: tomli>=2.0; python_version < "3.11"
+Provides-Extra: anthropic
+Requires-Dist: anthropic>=0.40; extra == "anthropic"
+Provides-Extra: openai
+Requires-Dist: openai>=1.60; extra == "openai"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: ruff>=0.15; extra == "dev"
+Dynamic: license-file
+
+# Eval Doctor
+
+> Know what you are not testing before you ship.
+
+Eval Doctor is a CLI that helps AI builders find blind spots in their eval suite — a linter for AI evals.
+
+Most AI projects start with architecture and implementation. Evals come later, once the system already "kind of works." But for RAG systems and agents, the hardest part is not writing test cases — it's knowing **what failure modes matter**. Eval Doctor scans your repo, detects your AI system's capabilities, maps them to expected evaluation dimensions, and tells you what your current eval suite is missing.
+
+```text
+You are testing faithfulness and answer relevance.
+
+You are not testing:
+- empty retrieval
+- stale documents
+- tool timeout
+- hallucinated tool arguments
+- prompt injection
+- ambiguous user requests
+```
+
+## Install
+
+```bash
+pip install eval-doctor               # static analysis, no API keys needed
+pip install "eval-doctor[anthropic]"  # + optional LLM-powered capability detection (Claude API)
+pip install "eval-doctor[openai]"     # + optional LLM-powered capability detection (OpenAI API)
+```
+
+Already have [Claude Code](https://claude.com/claude-code) installed? No extra or API key needed — `eval-doctor audit --llm` uses your Claude subscription through the `claude` CLI.
+
+## Usage
+
+```bash
+cd your-ai-project
+eval-doctor audit
+```
+
+You get a console report plus two files:
+
+- `eval_doctor_report.md` — human-readable coverage report
+- `eval_doctor_findings.json` — machine-readable findings for automation
+
+Options:
+
+```bash
+eval-doctor audit --repo path/to/repo      # audit a different directory
+eval-doctor audit --evals path/to/evals    # eval files living outside the repo
+eval-doctor audit --output-dir reports/    # where to write report files
+eval-doctor audit --llm                    # force LLM enrichment (also enables the claude CLI path)
+eval-doctor audit --no-llm                 # force pure static analysis
+eval-doctor audit --force                  # overwrite existing report files
+eval-doctor interview                      # short risk interview; audit picks up the answers
+eval-doctor show-taxonomy                  # print the built-in taxonomy
+```
+
+### Risk interview
+
+Severity defaults are generic; your risks are not. `eval-doctor interview` asks a few multiple-choice questions keyed to what the scanner actually found in your repo — retrieval, web grounding, tool calling, SQL, memory — plus where the system will run, and writes `eval_doctor_risk.json`. The next `eval-doctor audit` picks it up automatically (or point at one with `--risk path/to/eval_doctor_risk.json`) and elevates the severity of the dimensions you said would hurt most, re-ordering the report. Answers only ever raise severity — the interview never argues a risk away.
+
+## How it works
+
+```text
+Repo Scanner ──► Capability Extractor ──► Coverage Engine ──► Report
+(deterministic)  (static rules,           (expected vs
+                  optional LLM enrich)     observed dimensions)
+```
+
+1. **Scan** — a deterministic scanner walks your repo and detects signals: frameworks (LangChain, LlamaIndex, CrewAI, ...), vector DBs, embeddings, chunking, reranking, tool definitions, SQL, browser automation, memory, prompt templates, and existing eval/test files. Nothing leaves your machine.
+2. **Extract** — signals are mapped to a capability profile (`rag`, `tool_calling_agent`, `sql_agent`, ...).
+3. **Compare** — the profile selects applicable dimensions from a built-in taxonomy of 40 eval dimensions across RAG, tool calling, SQL agents, memory, and general AI risks. Your existing eval files are checked for evidence of each.
+4. **Report** — every applicable dimension is rated `covered`, `weak`, or `missing`, with a severity and a concrete recommendation.
+
+### Optional LLM enhancement
+
+Static analysis sees imports and patterns, not intent. With an LLM available, Eval Doctor sends its collected evidence snippets (never your whole repo) to a model to correct and complete the capability profile — catching things like web-search-grounded generation that deserves faithfulness evals, hand-rolled RAG with no framework, and pruning false positives like an sqlite cache being flagged as a SQL agent. Two ways to enable it:
+
+- **API key** — set `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` and install the matching extra. Used automatically when present.
+- **Claude subscription** — have [Claude Code](https://claude.com/claude-code) installed and signed in, then pass `--llm`. Eval Doctor shells out to `claude` on your machine; no API key or extra needed. This path is opt-in only — an installed CLI is never used unless you ask.
+
+Without either, everything still works; the report just notes it ran in static-only mode.
+
+Additional providers are welcome contributions: implement the two-method `LLMProvider` protocol in [`eval_doctor/llm.py`](eval_doctor/llm.py) — the existing providers are small, self-contained classes.
+
+## Example
+
+See [`examples/sample_report.md`](examples/sample_report.md) for the report produced by auditing a small RAG + tool-calling fixture project.
+
+## Roadmap
+
+- **Candidate eval generation** — generate golden eval cases for your missing dimensions
+- **Coverage diff** — `eval-doctor diff` to score an eval suite against recommended coverage
+- **CI / PR mode** — comment on PRs that add AI capabilities without corresponding evals
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT

eval_doctor-0.1.0/README.md

@@ -0,0 +1,103 @@
+# Eval Doctor
+
+> Know what you are not testing before you ship.
+
+Eval Doctor is a CLI that helps AI builders find blind spots in their eval suite — a linter for AI evals.
+
+Most AI projects start with architecture and implementation. Evals come later, once the system already "kind of works." But for RAG systems and agents, the hardest part is not writing test cases — it's knowing **what failure modes matter**. Eval Doctor scans your repo, detects your AI system's capabilities, maps them to expected evaluation dimensions, and tells you what your current eval suite is missing.
+
+```text
+You are testing faithfulness and answer relevance.
+
+You are not testing:
+- empty retrieval
+- stale documents
+- tool timeout
+- hallucinated tool arguments
+- prompt injection
+- ambiguous user requests
+```
+
+## Install
+
+```bash
+pip install eval-doctor               # static analysis, no API keys needed
+pip install "eval-doctor[anthropic]"  # + optional LLM-powered capability detection (Claude API)
+pip install "eval-doctor[openai]"     # + optional LLM-powered capability detection (OpenAI API)
+```
+
+Already have [Claude Code](https://claude.com/claude-code) installed? No extra or API key needed — `eval-doctor audit --llm` uses your Claude subscription through the `claude` CLI.
+
+## Usage
+
+```bash
+cd your-ai-project
+eval-doctor audit
+```
+
+You get a console report plus two files:
+
+- `eval_doctor_report.md` — human-readable coverage report
+- `eval_doctor_findings.json` — machine-readable findings for automation
+
+Options:
+
+```bash
+eval-doctor audit --repo path/to/repo      # audit a different directory
+eval-doctor audit --evals path/to/evals    # eval files living outside the repo
+eval-doctor audit --output-dir reports/    # where to write report files
+eval-doctor audit --llm                    # force LLM enrichment (also enables the claude CLI path)
+eval-doctor audit --no-llm                 # force pure static analysis
+eval-doctor audit --force                  # overwrite existing report files
+eval-doctor interview                      # short risk interview; audit picks up the answers
+eval-doctor show-taxonomy                  # print the built-in taxonomy
+```
+
+### Risk interview
+
+Severity defaults are generic; your risks are not. `eval-doctor interview` asks a few multiple-choice questions keyed to what the scanner actually found in your repo — retrieval, web grounding, tool calling, SQL, memory — plus where the system will run, and writes `eval_doctor_risk.json`. The next `eval-doctor audit` picks it up automatically (or point at one with `--risk path/to/eval_doctor_risk.json`) and elevates the severity of the dimensions you said would hurt most, re-ordering the report. Answers only ever raise severity — the interview never argues a risk away.
+
+## How it works
+
+```text
+Repo Scanner ──► Capability Extractor ──► Coverage Engine ──► Report
+(deterministic)  (static rules,           (expected vs
+                  optional LLM enrich)     observed dimensions)
+```
+
+1. **Scan** — a deterministic scanner walks your repo and detects signals: frameworks (LangChain, LlamaIndex, CrewAI, ...), vector DBs, embeddings, chunking, reranking, tool definitions, SQL, browser automation, memory, prompt templates, and existing eval/test files. Nothing leaves your machine.
+2. **Extract** — signals are mapped to a capability profile (`rag`, `tool_calling_agent`, `sql_agent`, ...).
+3. **Compare** — the profile selects applicable dimensions from a built-in taxonomy of 40 eval dimensions across RAG, tool calling, SQL agents, memory, and general AI risks. Your existing eval files are checked for evidence of each.
+4. **Report** — every applicable dimension is rated `covered`, `weak`, or `missing`, with a severity and a concrete recommendation.
+
+### Optional LLM enhancement
+
+Static analysis sees imports and patterns, not intent. With an LLM available, Eval Doctor sends its collected evidence snippets (never your whole repo) to a model to correct and complete the capability profile — catching things like web-search-grounded generation that deserves faithfulness evals, hand-rolled RAG with no framework, and pruning false positives like an sqlite cache being flagged as a SQL agent. Two ways to enable it:
+
+- **API key** — set `ANTHROPIC_API_KEY` or `OPENAI_API_KEY` and install the matching extra. Used automatically when present.
+- **Claude subscription** — have [Claude Code](https://claude.com/claude-code) installed and signed in, then pass `--llm`. Eval Doctor shells out to `claude` on your machine; no API key or extra needed. This path is opt-in only — an installed CLI is never used unless you ask.
+
+Without either, everything still works; the report just notes it ran in static-only mode.
+
+Additional providers are welcome contributions: implement the two-method `LLMProvider` protocol in [`eval_doctor/llm.py`](eval_doctor/llm.py) — the existing providers are small, self-contained classes.
+
+## Example
+
+See [`examples/sample_report.md`](examples/sample_report.md) for the report produced by auditing a small RAG + tool-calling fixture project.
+
+## Roadmap
+
+- **Candidate eval generation** — generate golden eval cases for your missing dimensions
+- **Coverage diff** — `eval-doctor diff` to score an eval suite against recommended coverage
+- **CI / PR mode** — comment on PRs that add AI capabilities without corresponding evals
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT

eval_doctor-0.1.0/eval_doctor/__init__.py

@@ -0,0 +1,8 @@
+"""Eval Doctor - an eval coverage auditor for RAG and agent systems."""
+
+from importlib.metadata import PackageNotFoundError, version
+
+try:
+    __version__ = version("eval-doctor")
+except PackageNotFoundError:  # running from a source tree without install
+    __version__ = "0.0.0.dev0"

eval_doctor-0.1.0/eval_doctor/capability_extractor.py

@@ -0,0 +1,120 @@
+"""Map scanner signals to a CapabilityProfile.
+
+The static pass always runs and needs no API key. An optional LLM provider
+(see llm.py) can enrich the static profile using the collected evidence.
+"""
+
+from __future__ import annotations
+
+from .schemas import CapabilityProfile, Evidence, ScanResult
+
+_RETRIEVAL_SIGNALS = {
+    "retrieval",
+    "embedding",
+    "chunking",
+    "query_rewriting",
+    "reranking",
+}
+_TOOL_SIGNALS = {"tool_calling", "http_tool"}
+
+# Which scanner signals substantiate each capability tag (used to attach
+# repo evidence to the taxonomy dimensions selected by that tag).
+_TAG_SIGNALS: dict[str, set[str]] = {
+    "retrieval": _RETRIEVAL_SIGNALS,
+    "tool_calling": _TOOL_SIGNALS,
+    "sql": {"sql", "sql_query_text"},
+    "browser": {"browser"},
+    "web_grounding": {"web_search"},
+    "memory": {"memory"},
+    "embedding": {"embedding"},
+    "chunking": {"chunking"},
+    "query_rewriting": {"query_rewriting"},
+    "reranking": {"reranking"},
+    "prompting": {"prompt_template"},
+}
+_TAG_SIGNAL_PREFIXES: dict[str, tuple[str, ...]] = {"retrieval": ("vector_db:",)}
+
+
+def _has_prefix(signals: list[str], prefix: str) -> bool:
+    return any(s.startswith(prefix) for s in signals)
+
+
+def extract_static(scan: ScanResult) -> CapabilityProfile:
+    """Rule-based capability extraction from scanner signals."""
+    signals = scan.signals
+    profile = CapabilityProfile(source="static")
+
+    profile.retrieval = bool(_RETRIEVAL_SIGNALS & set(signals)) or _has_prefix(
+        signals, "vector_db:"
+    )
+    profile.browser = "browser" in signals
+    profile.memory = "memory" in signals
+    # Only real tool-wiring patterns (@tool, tools=[, input_schema, ...) make a
+    # tool_calling agent. http_tool alone is not enough: shikhu's requests.post
+    # calls its LLM provider (the model call itself, not a tool the model
+    # invokes), yet it profiled as a tool_calling_agent with 9 tool dims.
+    tool_calling = "tool_calling" in signals
+    # SQL counts as a capability only when it can be model-facing (wired as a
+    # tool). Standalone sqlite/SQL is app storage: shikhu's coverage.db got 8
+    # SQL-agent dims (4 critical) that hijacked the top recommendation.
+    profile.sql = "sql" in signals and tool_calling
+
+    if profile.retrieval:
+        profile.capabilities.append("retrieval")
+        profile.system_type.append("rag")
+    for cap in ("query_rewriting", "reranking", "chunking", "embedding"):
+        if cap in signals:
+            profile.capabilities.append(cap)
+    if tool_calling:
+        profile.capabilities.append("tool_calling")
+        profile.system_type.append("tool_calling_agent")
+    if profile.sql:
+        profile.capabilities.append("sql")
+        profile.tools.append("sql_tool")
+    if profile.browser:
+        profile.capabilities.append("browser")
+        profile.tools.append("browser_tool")
+    if "web_search" in signals:
+        profile.capabilities.append("web_grounding")
+        profile.tools.append("web_search_tool")
+    if tool_calling and "http_tool" in signals:
+        profile.tools.append("http_tool")
+    if profile.memory:
+        profile.capabilities.append("memory")
+    if "prompt_template" in signals:
+        profile.capabilities.append("prompting")
+    if not profile.system_type and (
+        _has_prefix(signals, "sdk:") or _has_prefix(signals, "framework:")
+    ):
+        profile.system_type.append("llm_pipeline")
+
+    profile.eval_files_detected = list(scan.eval_files)
+    return profile
+
+
+def evidence_for_tags(evidence: list[Evidence], tags: list[str]) -> list[Evidence]:
+    """Scanner evidence that substantiates any of the given capability tags."""
+    signals: set[str] = set()
+    prefixes: tuple[str, ...] = ()
+    for tag in tags:
+        signals |= _TAG_SIGNALS.get(tag, set())
+        prefixes += _TAG_SIGNAL_PREFIXES.get(tag, ())
+    return [
+        e
+        for e in evidence
+        if e.snippet and (e.signal in signals or e.signal.startswith(prefixes))
+    ]
+
+
+def capability_tags(profile: CapabilityProfile) -> set[str]:
+    """Tags used to select applicable taxonomy dimensions."""
+    tags = set(profile.capabilities)
+    if profile.retrieval:
+        tags.add("retrieval")
+    if profile.sql:
+        tags.add("sql")
+    if profile.browser:
+        tags.add("browser")
+    if profile.memory:
+        tags.add("memory")
+    return tags

eval_doctor-0.1.0/eval_doctor/cli.py

@@ -0,0 +1,249 @@
+"""Eval Doctor CLI."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import typer
+from rich.console import Console
+from rich.table import Table
+
+from . import __version__
+from .capability_extractor import capability_tags, evidence_for_tags, extract_static
+from .coverage_engine import analyze_coverage, apply_coverage_judgments
+from .interview import RISK_JSON, build_risk_profile, questions_for
+from .llm import enrich_profile, get_default_provider
+from .report_generator import print_console_report, write_reports
+from .scanner import scan_repo
+from .schemas import RiskProfile
+from .taxonomy import TAXONOMY
+
+app = typer.Typer(
+    name="eval-doctor",
+    help="Know what you are not testing before you ship - an eval coverage auditor for RAG and agent systems.",
+    no_args_is_help=True,
+)
+console = Console()
+
+
+@app.command()
+def audit(
+    repo: Path = typer.Option(Path("."), "--repo", help="Path to the repo to audit."),
+    evals: Path | None = typer.Option(
+        None, "--evals", help="Extra path to eval files outside the repo."
+    ),
+    output_dir: Path = typer.Option(
+        Path("."), "--output-dir", help="Where to write report files."
+    ),
+    llm: bool | None = typer.Option(
+        None,
+        "--llm/--no-llm",
+        help="Force LLM enrichment on/off. Default: use it when an API key is available.",
+    ),
+    risk: Path | None = typer.Option(
+        None,
+        "--risk",
+        help=f"Risk profile from `eval-doctor interview`. Default: {RISK_JSON} in --output-dir, if present.",
+    ),
+    force: bool = typer.Option(
+        False, "--force", help="Overwrite existing report files."
+    ),
+):
+    """Scan a repo and produce an eval coverage report."""
+    try:
+        scan = scan_repo(repo, evals_path=evals)
+    except FileNotFoundError as exc:
+        console.print(f"[red]error:[/red] {exc}")
+        raise typer.Exit(code=1) from None
+
+    risk_path = risk if risk is not None else output_dir / RISK_JSON
+    risk_profile = None
+    if risk_path.exists():
+        try:
+            risk_profile = RiskProfile.model_validate_json(
+                risk_path.read_text(encoding="utf-8")
+            )
+        except ValueError as exc:
+            console.print(f"[red]error:[/red] invalid risk profile {risk_path}: {exc}")
+            raise typer.Exit(code=1) from None
+    elif risk is not None:
+        console.print(f"[red]error:[/red] risk profile not found: {risk}")
+        raise typer.Exit(code=1)
+
+    profile = extract_static(scan)
+
+    provider = None
+    if llm is not False:
+        # The claude CLI (subscription, no keys) is opt-in via --llm only.
+        provider = get_default_provider(include_claude_cli=llm is True)
+        if llm is True and provider is None:
+            console.print(
+                "[red]error:[/red] --llm requested but no provider is available. "
+                "Set ANTHROPIC_API_KEY or OPENAI_API_KEY (and install "
+                "eval-doctor\\[anthropic] or eval-doctor\\[openai]), or install "
+                "Claude Code so the `claude` CLI can be used."
+            )
+            raise typer.Exit(code=1)
+    if provider is not None:
+        try:
+            with console.status(f"Enriching capability profile via {provider.name}..."):
+                profile = enrich_profile(provider, scan.evidence, profile)
+        except Exception as exc:  # degrade, don't crash the audit
+            console.print(
+                f"[yellow]warning:[/yellow] LLM enrichment failed ({exc}); using static analysis only."
+            )
+    elif llm is None:
+        console.print(
+            "[dim]Static analysis only - set ANTHROPIC_API_KEY or OPENAI_API_KEY "
+            "(and pip install 'eval-doctor\\[anthropic]' or 'eval-doctor\\[openai]'), "
+            "or pass --llm to use an installed Claude Code (`claude` CLI), for "
+            "deeper capability detection and eval-quality judgment.[/dim]"
+        )
+
+    report = analyze_coverage(scan, profile, risk=risk_profile)
+
+    if not report.findings and not profile.system_type:
+        console.print(
+            f"No AI system detected in {scan.repo_path} "
+            f"({scan.files_scanned} files scanned) - nothing to audit."
+        )
+        raise typer.Exit()
+
+    if provider is not None and scan.eval_files:
+        try:
+            with console.status(f"Judging eval coverage via {provider.name}..."):
+                judgments = provider.judge_coverage(
+                    scan.eval_file_contents, report.findings
+                )
+            apply_coverage_judgments(report, judgments)
+        except Exception as exc:  # degrade, don't crash the audit
+            console.print(
+                f"[yellow]warning:[/yellow] LLM coverage judgment failed ({exc}); "
+                "keeping keyword-based coverage."
+            )
+
+    print_console_report(report, console)
+
+    try:
+        paths = write_reports(report, output_dir, force=force)
+    except FileExistsError as exc:
+        console.print(f"[red]error:[/red] {exc}")
+        raise typer.Exit(code=1) from None
+    console.print(f"Wrote {', '.join(str(p) for p in paths)}")
+
+
+@app.command()
+def interview(
+    repo: Path = typer.Option(
+        Path("."), "--repo", help="Path to the repo the interview is about."
+    ),
+    output_dir: Path = typer.Option(
+        Path("."), "--output-dir", help=f"Where to write {RISK_JSON}."
+    ),
+    force: bool = typer.Option(
+        False, "--force", help="Overwrite an existing risk profile."
+    ),
+):
+    """Short risk interview based on what the scanner found; feeds `audit`."""
+    try:
+        scan = scan_repo(repo)
+    except FileNotFoundError as exc:
+        console.print(f"[red]error:[/red] {exc}")
+        raise typer.Exit(code=1) from None
+
+    profile = extract_static(scan)
+    tags = capability_tags(profile)
+    if not profile.system_type and not tags:
+        console.print(
+            f"No AI system detected in {scan.repo_path} "
+            f"({scan.files_scanned} files scanned) - nothing to interview."
+        )
+        raise typer.Exit()
+
+    risk_path = Path(output_dir) / RISK_JSON
+    if risk_path.exists() and not force:
+        console.print(
+            f"[red]error:[/red] Refusing to overwrite {risk_path} (use --force)."
+        )
+        raise typer.Exit(code=1)
+
+    questions = questions_for(tags)
+    console.print("\n[bold]Eval Doctor Risk Interview[/bold]")
+    console.print(f"[dim]{scan.repo_path}[/dim]")
+    console.print(
+        f"Detected system type: [bold]{' + '.join(profile.system_type)}[/bold]\n"
+    )
+
+    answered = []
+    for number, question in enumerate(questions, 1):
+        anchors = evidence_for_tags(scan.evidence, question.applies_when)
+        location = f" [dim]({anchors[0].path})[/dim]" if anchors else ""
+        console.print(
+            f"[bold]\\[{number}/{len(questions)}][/bold] {question.context}{location}"
+        )
+        console.print(question.prompt)
+        for n, option in enumerate(question.options, 1):
+            console.print(f"  {n}. {option.label}")
+        while True:
+            choice = typer.prompt("Answer", type=int)
+            if 1 <= choice <= len(question.options):
+                break
+            console.print(f"Enter a number between 1 and {len(question.options)}.")
+        answered.append((question, choice - 1))
+        console.print()
+
+    risk = build_risk_profile(scan.repo_path, answered)
+    Path(output_dir).mkdir(parents=True, exist_ok=True)
+    risk_path.write_text(risk.model_dump_json(indent=2), encoding="utf-8")
+
+    names = {d.id: d.name for d in TAXONOMY}
+    if risk.elevated_dimensions:
+        elevated = ", ".join(names.get(d, d) for d in risk.elevated_dimensions)
+        console.print(f"Elevated severity for: [bold]{elevated}[/bold]")
+    else:
+        console.print("No dimensions elevated - default severities stand.")
+    console.print(
+        f"Wrote {risk_path}. Run [bold]eval-doctor audit --repo {repo}[/bold] "
+        "to see the re-prioritized report."
+    )
+
+
+@app.command("show-taxonomy")
+def show_taxonomy():
+    """Print the built-in eval coverage taxonomy."""
+    table = Table(
+        title=f"Eval Doctor taxonomy ({len(TAXONOMY)} dimensions)", show_lines=False
+    )
+    table.add_column("Category", style="bold")
+    table.add_column("Dimension")
+    table.add_column("Applies when")
+    table.add_column("Severity")
+    table.add_column("Description", max_width=60)
+    for dim in TAXONOMY:
+        table.add_row(
+            dim.category,
+            dim.name,
+            ", ".join(dim.applies_when),
+            dim.severity_default,
+            dim.description,
+        )
+    console.print(table)
+
+
+def _print_version(value: bool):
+    if value:
+        console.print(f"eval-doctor {__version__}")
+        raise typer.Exit()
+
+
+@app.callback()
+def _main(
+    version: bool = typer.Option(
+        False,
+        "--version",
+        help="Show version and exit.",
+        callback=_print_version,
+        is_eager=True,
+    ),
+):
+    pass