fastevals 0.1.0__tar.gz

fastevals-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Dmitry Semenov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fastevals-0.1.0/PKG-INFO

@@ -0,0 +1,244 @@
+Metadata-Version: 2.4
+Name: fastevals
+Version: 0.1.0
+Summary: Small, provider-agnostic evaluation runner for LLM applications.
+Author: Dmitry Semenov
+License: MIT
+Keywords: llm,evaluation,benchmark,litellm,mcp
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: jsonschema>=4.23
+Provides-Extra: native
+Requires-Dist: litellm>=1.74; extra == "native"
+Provides-Extra: mcp
+Requires-Dist: mcp>=1.2; extra == "mcp"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.24; extra == "dev"
+Requires-Dist: ruff>=0.6; extra == "dev"
+Requires-Dist: mypy>=1.11; extra == "dev"
+Requires-Dist: types-jsonschema>=4.23; extra == "dev"
+Requires-Dist: mcp>=1.2; extra == "dev"
+Dynamic: license-file
+
+# fastevals
+
+**Evaluation tooling your AI agents can drive.**
+
+fastevals is a small, provider-agnostic evaluation runner for LLM
+applications. Run one prompt — or a whole dataset — across a matrix of
+models, reasoning efforts and providers, save every response, and get a
+readable standalone HTML comparison report with cost, latency and token
+metrics.
+
+It ships as an **MCP server**, so Claude Desktop, Claude Code or any other
+MCP client can run evaluations as a native tool: your agent decides *what*
+to test, fastevals answers *which model does it best*.
+
+[![CI](https://github.com/semenovdv/fastevals/actions/workflows/ci.yml/badge.svg)](https://github.com/semenovdv/fastevals/actions/workflows/ci.yml)
+![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue)
+![Coverage](https://img.shields.io/badge/coverage-89%25-brightgreen)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+![mypy](https://img.shields.io/badge/mypy-strict-blue)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+<p align="center">
+  <img src="docs/assets/report.png" alt="fastevals HTML report" width="820">
+</p>
+
+## Drive it from Claude (MCP)
+
+Install the server extras and register the entry point with any MCP client:
+
+```bash
+python3 -m pip install 'fastevals[mcp,native]'
+claude mcp add fastevals -- fastevals-mcp        # Claude Code
+```
+
+Claude Desktop (`claude_desktop_config.json`):
+
+```json
+{
+  "mcpServers": { "fastevals": { "command": "fastevals-mcp" } }
+}
+```
+
+Exposed tools:
+
+| Tool | Purpose |
+|---|---|
+| `run_evaluation` | Run a prompt or dataset across providers; returns JSON summary + report paths |
+| `list_models` | Registry inspector: models, reasoning efforts, pricing |
+| `get_run` | Summarize a saved run: pass rate, errors, total cost |
+
+Example agent prompts that now just work:
+
+> Use fastevals to compare gpt-5.6-luna at reasoning low and high on "Summarize
+> this contract in 5 bullets" — which one is cheaper per correct answer?
+
+> List my registered models, then evaluate cases.jsonl on terra and report
+> the pass rate per effort level.
+
+Because the CLI is fully non-interactive and returns structured JSON, agents
+can also drive evaluations through plain shell execution without MCP.
+
+## Why fastevals
+
+- **Structured output that verifies** — compact schema syntax compiles to JSON Schema, is sent to the provider, and every response is validated locally before it reaches `run.json`.
+- **Honest metrics** — disjoint token buckets (input / output / reasoning / cached), per-bucket pricing from your registry, no fake TTFT without streaming.
+- **Real evaluation loop** — JSONL/CSV datasets, deterministic evaluators (`exact_match`, `contains`, `json_valid`, `regex`), repeated runs for stability.
+- **Boring engineering** — strict typing, ~90% branch coverage, ruff + mypy + coverage gates in CI, single-file reports with zero telemetry.
+
+## Install
+
+```bash
+python3 -m pip install 'fastevals[native]'    # from PyPI once released
+# or from source:
+git clone https://github.com/semenovdv/fastevals && python3 -m pip install -e '.[native]'
+```
+
+## CLI quick start
+
+```bash
+export OPENAI_API_KEY=...                  # keys live in the environment only
+fastevals --prompt "Explain evaluation in three bullets" \
+          --providers openai --out runs
+```
+
+Every run writes a timestamped directory under `--out` containing
+`run.json` (machine-readable) and `report.html` (a standalone dashboard you
+can open or send to anyone). Exit codes: `0` when every model completed,
+`1` otherwise — easy to script.
+
+### Models and reasoning efforts
+
+Entries in `config/models.toml` become cells in the matrix:
+
+```toml
+["openai:gpt-5.6-luna"]
+provider = "openai"
+model = "gpt-5.6-luna"
+api_key_env = "OPENAI_API_KEY"
+reasoning_efforts = "none|low"          # expands into two runs
+input_cost_usd_per_mtok = 1.0           # USD per 1M tokens
+output_cost_usd_per_mtok = 6.0
+```
+
+Providers are validated against the registry; unknown names fail fast with a
+helpful message. API keys are read from environment variables only — never
+from the registry, never logged, and scrubbed from error messages.
+
+### Structured output
+
+```bash
+fastevals \
+  --prompt "Extract all relevant invoice fields" \
+  --structured-output 'invoice_number:str("Unique identifier"),total:float("Amount incl. tax"),line_items:str[]("Items"),notes:str?' \
+  --providers openai --out runs/invoice
+```
+
+`?` marks optional fields, `[]` arrays, `"..."` descriptions passed to the
+model (`str|int|float|bool` with aliases supported).
+
+### Files and images
+
+Images become vision parts, PDFs OpenAI-style file parts, text files inline:
+
+```bash
+fastevals --image screenshot.png --structured-output 'x:int,y:int,width:int,height:int' \
+  --prompt "Bounding box of the main widget" --providers openai --out runs/image
+```
+
+### Datasets, evaluators, consistency
+
+```jsonl
+{"id": "capital-france", "prompt": "Capital of France? City name only.", "expected": "Paris", "evaluator": "exact_match"}
+{"id": "json-output", "prompt": "Return {\"status\": \"ok\"} as JSON.", "evaluator": "json_valid"}
+```
+
+```bash
+fastevals --dataset cases.jsonl --nruns 3 --providers openai --out runs/dataset
+```
+
+Reports aggregate pass rates, latency and cost per model across all attempts.
+
+## The report
+
+Each `report.html` is a self-contained dashboard (Chart.js from CDN, no
+build step, no telemetry): summary cards with fastest / cheapest /
+top-throughput runs, sortable and filterable comparison table with CSV and
+Markdown export, latency / throughput / token / cost charts, detailed result
+cards, per-model aggregates for datasets.
+
+## Python API
+
+```python
+import asyncio
+from fastevals import RunConfig, run, save_report
+
+config = RunConfig(prompt="Summarize eval best practices", providers=frozenset({"openai"}))
+results = asyncio.run(run(config))
+save_report(config, results, "runs")
+print(results[0].output, results[0].latency_ms, results[0].total_cost_usd)
+```
+
+## Architecture
+
+```mermaid
+flowchart LR
+    CLI["cli.py"] --> RC["RunConfig"]
+    RC --> Runner["runner.py"]
+    DS["dataset.py"] --> Runner
+    EV["evaluators.py"] --> Runner
+    Runner --> Reg["registry.py"]
+    Reg --> Specs["ModelSpec"]
+    Runner --> Prov["providers.py<br/>LiteLLM adapter"]
+    Prov --> ST["structured.py<br/>schema · validation"]
+    Runner --> PR["pricing.py"]
+    Runner --> Rep["report.py<br/>single-file HTML"]
+    Rep --> Out["run.json + report.html"]
+
+    MCP["mcp_server.py"] --> Runner
+```
+
+Adding a provider means implementing the single `call_model` contract in
+`providers.py`; adding a model means adding five lines to the TOML registry.
+No other layers need to change.
+
+## Development
+
+```bash
+make dev        # install with dev tooling
+make check      # ruff + mypy --strict + tests with an 85% coverage floor
+make format     # auto-fix style
+```
+
+The test suite is fully offline: provider calls are replaced by a recorded
+stub at the LiteLLM boundary; live API calls never run in CI.
+
+## Limitations (by design)
+
+- No streaming yet — TTFT is reported as unavailable rather than faked; latency and throughput are end-to-end.
+- One prompt template per case; no few-shot templating or conversation history.
+- Evaluators are deterministic heuristics; LLM-as-judge scoring is not included.
+- Pricing comes from your registry, not a live price feed — keep it current.
+
+See [`docs/ROADMAP.md`](docs/ROADMAP.md) for where this is heading.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

fastevals-0.1.0/README.md

@@ -0,0 +1,207 @@
+# fastevals
+
+**Evaluation tooling your AI agents can drive.**
+
+fastevals is a small, provider-agnostic evaluation runner for LLM
+applications. Run one prompt — or a whole dataset — across a matrix of
+models, reasoning efforts and providers, save every response, and get a
+readable standalone HTML comparison report with cost, latency and token
+metrics.
+
+It ships as an **MCP server**, so Claude Desktop, Claude Code or any other
+MCP client can run evaluations as a native tool: your agent decides *what*
+to test, fastevals answers *which model does it best*.
+
+[![CI](https://github.com/semenovdv/fastevals/actions/workflows/ci.yml/badge.svg)](https://github.com/semenovdv/fastevals/actions/workflows/ci.yml)
+![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue)
+![Coverage](https://img.shields.io/badge/coverage-89%25-brightgreen)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+![mypy](https://img.shields.io/badge/mypy-strict-blue)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+<p align="center">
+  <img src="docs/assets/report.png" alt="fastevals HTML report" width="820">
+</p>
+
+## Drive it from Claude (MCP)
+
+Install the server extras and register the entry point with any MCP client:
+
+```bash
+python3 -m pip install 'fastevals[mcp,native]'
+claude mcp add fastevals -- fastevals-mcp        # Claude Code
+```
+
+Claude Desktop (`claude_desktop_config.json`):
+
+```json
+{
+  "mcpServers": { "fastevals": { "command": "fastevals-mcp" } }
+}
+```
+
+Exposed tools:
+
+| Tool | Purpose |
+|---|---|
+| `run_evaluation` | Run a prompt or dataset across providers; returns JSON summary + report paths |
+| `list_models` | Registry inspector: models, reasoning efforts, pricing |
+| `get_run` | Summarize a saved run: pass rate, errors, total cost |
+
+Example agent prompts that now just work:
+
+> Use fastevals to compare gpt-5.6-luna at reasoning low and high on "Summarize
+> this contract in 5 bullets" — which one is cheaper per correct answer?
+
+> List my registered models, then evaluate cases.jsonl on terra and report
+> the pass rate per effort level.
+
+Because the CLI is fully non-interactive and returns structured JSON, agents
+can also drive evaluations through plain shell execution without MCP.
+
+## Why fastevals
+
+- **Structured output that verifies** — compact schema syntax compiles to JSON Schema, is sent to the provider, and every response is validated locally before it reaches `run.json`.
+- **Honest metrics** — disjoint token buckets (input / output / reasoning / cached), per-bucket pricing from your registry, no fake TTFT without streaming.
+- **Real evaluation loop** — JSONL/CSV datasets, deterministic evaluators (`exact_match`, `contains`, `json_valid`, `regex`), repeated runs for stability.
+- **Boring engineering** — strict typing, ~90% branch coverage, ruff + mypy + coverage gates in CI, single-file reports with zero telemetry.
+
+## Install
+
+```bash
+python3 -m pip install 'fastevals[native]'    # from PyPI once released
+# or from source:
+git clone https://github.com/semenovdv/fastevals && python3 -m pip install -e '.[native]'
+```
+
+## CLI quick start
+
+```bash
+export OPENAI_API_KEY=...                  # keys live in the environment only
+fastevals --prompt "Explain evaluation in three bullets" \
+          --providers openai --out runs
+```
+
+Every run writes a timestamped directory under `--out` containing
+`run.json` (machine-readable) and `report.html` (a standalone dashboard you
+can open or send to anyone). Exit codes: `0` when every model completed,
+`1` otherwise — easy to script.
+
+### Models and reasoning efforts
+
+Entries in `config/models.toml` become cells in the matrix:
+
+```toml
+["openai:gpt-5.6-luna"]
+provider = "openai"
+model = "gpt-5.6-luna"
+api_key_env = "OPENAI_API_KEY"
+reasoning_efforts = "none|low"          # expands into two runs
+input_cost_usd_per_mtok = 1.0           # USD per 1M tokens
+output_cost_usd_per_mtok = 6.0
+```
+
+Providers are validated against the registry; unknown names fail fast with a
+helpful message. API keys are read from environment variables only — never
+from the registry, never logged, and scrubbed from error messages.
+
+### Structured output
+
+```bash
+fastevals \
+  --prompt "Extract all relevant invoice fields" \
+  --structured-output 'invoice_number:str("Unique identifier"),total:float("Amount incl. tax"),line_items:str[]("Items"),notes:str?' \
+  --providers openai --out runs/invoice
+```
+
+`?` marks optional fields, `[]` arrays, `"..."` descriptions passed to the
+model (`str|int|float|bool` with aliases supported).
+
+### Files and images
+
+Images become vision parts, PDFs OpenAI-style file parts, text files inline:
+
+```bash
+fastevals --image screenshot.png --structured-output 'x:int,y:int,width:int,height:int' \
+  --prompt "Bounding box of the main widget" --providers openai --out runs/image
+```
+
+### Datasets, evaluators, consistency
+
+```jsonl
+{"id": "capital-france", "prompt": "Capital of France? City name only.", "expected": "Paris", "evaluator": "exact_match"}
+{"id": "json-output", "prompt": "Return {\"status\": \"ok\"} as JSON.", "evaluator": "json_valid"}
+```
+
+```bash
+fastevals --dataset cases.jsonl --nruns 3 --providers openai --out runs/dataset
+```
+
+Reports aggregate pass rates, latency and cost per model across all attempts.
+
+## The report
+
+Each `report.html` is a self-contained dashboard (Chart.js from CDN, no
+build step, no telemetry): summary cards with fastest / cheapest /
+top-throughput runs, sortable and filterable comparison table with CSV and
+Markdown export, latency / throughput / token / cost charts, detailed result
+cards, per-model aggregates for datasets.
+
+## Python API
+
+```python
+import asyncio
+from fastevals import RunConfig, run, save_report
+
+config = RunConfig(prompt="Summarize eval best practices", providers=frozenset({"openai"}))
+results = asyncio.run(run(config))
+save_report(config, results, "runs")
+print(results[0].output, results[0].latency_ms, results[0].total_cost_usd)
+```
+
+## Architecture
+
+```mermaid
+flowchart LR
+    CLI["cli.py"] --> RC["RunConfig"]
+    RC --> Runner["runner.py"]
+    DS["dataset.py"] --> Runner
+    EV["evaluators.py"] --> Runner
+    Runner --> Reg["registry.py"]
+    Reg --> Specs["ModelSpec"]
+    Runner --> Prov["providers.py<br/>LiteLLM adapter"]
+    Prov --> ST["structured.py<br/>schema · validation"]
+    Runner --> PR["pricing.py"]
+    Runner --> Rep["report.py<br/>single-file HTML"]
+    Rep --> Out["run.json + report.html"]
+
+    MCP["mcp_server.py"] --> Runner
+```
+
+Adding a provider means implementing the single `call_model` contract in
+`providers.py`; adding a model means adding five lines to the TOML registry.
+No other layers need to change.
+
+## Development
+
+```bash
+make dev        # install with dev tooling
+make check      # ruff + mypy --strict + tests with an 85% coverage floor
+make format     # auto-fix style
+```
+
+The test suite is fully offline: provider calls are replaced by a recorded
+stub at the LiteLLM boundary; live API calls never run in CI.
+
+## Limitations (by design)
+
+- No streaming yet — TTFT is reported as unavailable rather than faked; latency and throughput are end-to-end.
+- One prompt template per case; no few-shot templating or conversation history.
+- Evaluators are deterministic heuristics; LLM-as-judge scoring is not included.
+- Pricing comes from your registry, not a live price feed — keep it current.
+
+See [`docs/ROADMAP.md`](docs/ROADMAP.md) for where this is heading.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

fastevals-0.1.0/fastevals/__init__.py

@@ -0,0 +1,26 @@
+"""Fast, provider-agnostic LLM evaluation toolkit."""
+
+from .config import SUPPORTED_PROVIDERS, ModelSpec, RunConfig
+from .exceptions import ConfigError, FastEvalError, ProviderError, StructuredOutputError
+from .models import ModelResponse, RunResult
+from .registry import load_registry
+from .report import save_report
+from .runner import run
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "SUPPORTED_PROVIDERS",
+    "ConfigError",
+    "FastEvalError",
+    "ModelResponse",
+    "ModelSpec",
+    "ProviderError",
+    "RunConfig",
+    "RunResult",
+    "StructuredOutputError",
+    "__version__",
+    "load_registry",
+    "run",
+    "save_report",
+]

fastevals-0.1.0/fastevals/cli.py

@@ -0,0 +1,125 @@
+"""Command-line interface for fastevals."""
+
+import argparse
+import asyncio
+import json
+import os
+from pathlib import Path
+
+from .config import DEFAULT_MAX_CONCURRENCY, SUPPORTED_PROVIDERS, RunConfig
+from .exceptions import FastEvalError
+from .report import save_report
+from .runner import run
+from .structured import shorthand_to_schema
+
+ALL_PROVIDERS = "all"
+
+
+def _dotenv_candidates() -> list[Path]:
+    return [Path.cwd() / ".env", Path(__file__).resolve().parents[1] / ".env"]
+
+
+def _load_dotenv() -> None:
+    """Load simple KEY=VALUE entries from a project .env if present."""
+    for env_path in _dotenv_candidates():
+        if not env_path.exists():
+            continue
+        for raw_line in env_path.read_text().splitlines():
+            line = raw_line.strip()
+            if not line or line.startswith("#") or "=" not in line:
+                continue
+            key, value = line.split("=", 1)
+            key = key.strip()
+            value = value.strip().strip('"').strip("'")
+            if key and value and key not in os.environ:
+                os.environ[key] = value
+
+
+def _parse_providers(raw: str) -> frozenset[str]:
+    providers = {item.strip().lower() for item in raw.split("|") if item.strip()}
+    unknown = sorted(providers - set(SUPPORTED_PROVIDERS) - {ALL_PROVIDERS})
+    if unknown:
+        supported = ", ".join((*SUPPORTED_PROVIDERS, ALL_PROVIDERS))
+        raise argparse.ArgumentTypeError(f"Unknown provider(s): {', '.join(unknown)}. Supported: {supported}")
+    return frozenset(providers)
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="fastevals",
+        description="Compare one task results across LLM providers and models.",
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+        epilog="""Examples:
+  fastevals --prompt \"Summarize this\" --providers \"openai|gemini\" --out runs
+  fastevals --image image.png --prompt \"Find widget bboxes\" \\
+    --structured-output \"x:int(X coord),y:int(Y coord),width:int(Width),height:int(Height)\" \\
+    --providers openai
+  fastevals --dataset cases.jsonl --nruns 3 --providers openai --out runs/dataset
+
+""",
+    )
+    parser.add_argument("-p", "--prompt", help="Task prompt (omit when --dataset provides the prompts)")
+    parser.add_argument("-s", "--structured-output", help="Structured output compact schema for the response")
+    parser.add_argument("-f", "--file", type=Path, help="Input document (sent to the model as an attachment)")
+    parser.add_argument("-i", "--image", type=Path, help="Input image")
+    parser.add_argument(
+        "-pr",
+        "--providers",
+        type=_parse_providers,
+        default=frozenset({ALL_PROVIDERS}),
+        help=f"Pipe-separated providers: {'|'.join(SUPPORTED_PROVIDERS)}|all (default: all)",
+    )
+    parser.add_argument(
+        "-r", "--registry", type=Path, help="Path to the model registry TOML (default: config/models.toml)"
+    )
+    parser.add_argument(
+        "-d",
+        "--dataset",
+        type=Path,
+        help="JSONL or CSV file with evaluation cases (columns: prompt, expected, evaluator, pattern)",
+    )
+    parser.add_argument(
+        "-n", "--nruns", type=int, default=1, help="Repeat every case this many times for consistency checks"
+    )
+    parser.add_argument(
+        "-c", "--concurrency", type=int, default=DEFAULT_MAX_CONCURRENCY, help="Max parallel model calls"
+    )
+    parser.add_argument("-o", "--out", type=Path, default=Path("runs"), help="Output directory")
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    _load_dotenv()
+    args = build_parser().parse_args(argv)
+
+    try:
+        config = RunConfig(
+            prompt=args.prompt or "",
+            providers=args.providers,
+            file=str(args.file) if args.file else None,
+            image=str(args.image) if args.image else None,
+            structured_output=shorthand_to_schema(args.structured_output) if args.structured_output else None,
+            dataset=str(args.dataset) if args.dataset else None,
+            nruns=max(1, args.nruns),
+            registry=str(args.registry) if args.registry else None,
+            max_concurrency=max(1, args.concurrency),
+            out=str(args.out),
+        )
+        results = asyncio.run(run(config))
+    except (FastEvalError, ValueError) as exc:
+        print(json.dumps({"ok": False, "error": str(exc), "results": []}, ensure_ascii=False))
+        return 1
+
+    json_path, html_path = save_report(config, results, args.out)
+    payload = {
+        "ok": all(row.ok for row in results),
+        "json_path": str(json_path),
+        "html_path": str(html_path) if html_path else None,
+        "results": [row.as_dict() for row in results],
+    }
+    print(json.dumps(payload, ensure_ascii=False, indent=2))
+    return 0 if payload["ok"] else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())