remuda 0.1.0__tar.gz

remuda-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 David Marsa / Neomanex
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

remuda-0.1.0/PKG-INFO

@@ -0,0 +1,221 @@
+Metadata-Version: 2.4
+Name: remuda
+Version: 0.1.0
+Summary: Bulk LLM inference over pools of free/cheap models: ride one until it tires, swap to the next
+License-Expression: MIT
+License-File: LICENSE
+Keywords: llm,inference,bulk,openrouter,classification,csv,cli
+Author: David Marsa
+Author-email: david.marsa@neomanex.com
+Requires-Python: >=3.12,<3.14
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Dist: httpx (>=0.28,<0.29)
+Requires-Dist: jinja2 (>=3.1,<4.0)
+Requires-Dist: pydantic (>=2.10,<3.0)
+Requires-Dist: pyyaml (>=6.0,<7.0)
+Requires-Dist: rich (>=13.9,<15.0)
+Requires-Dist: typer (>=0.15,<0.20)
+Project-URL: Changelog, https://github.com/daviunx/remuda/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/daviunx/remuda/issues
+Project-URL: Repository, https://github.com/daviunx/remuda
+Description-Content-Type: text/markdown
+
+# remuda
+
+Bulk LLM inference over pools of free/cheap models. Ride one until it tires, swap to the next.
+
+[![Test](https://github.com/daviunx/remuda/actions/workflows/test.yml/badge.svg)](https://github.com/daviunx/remuda/actions/workflows/test.yml)
+[![PyPI](https://img.shields.io/pypi/v/remuda)](https://pypi.org/project/remuda/)
+[![Python](https://img.shields.io/pypi/pyversions/remuda)](https://pypi.org/project/remuda/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+A *remuda* is the herd of spare horses a working cowboy draws from: when one
+tires, you saddle the next and keep moving. This tool does that with language
+models. Point it at a pool of free or cheap models, hand it a thousand rows,
+and it classifies, extracts, scores or generates a column for every one:
+validating every answer, retrying with feedback, rotating past rate limits and
+broken models, and never reporting a partial result as success.
+
+## Why
+
+Free-tier models are individually unreliable and collectively excellent. On
+any given day a quarter of a free pool answers well, another quarter is rate
+limited, and the rest return markdown essays when you asked for one word.
+remuda's job is to make that mess dependable: a validation ladder per answer,
+rotation across the pool, a crash-safe ledger, and resume that retries only
+what failed, at no repeat cost.
+
+## Install
+
+```bash
+pipx install remuda        # recommended for the CLI
+pip install remuda         # or as a library
+```
+
+## Quick start
+
+With `OPENROUTER_API_KEY` exported and no configuration at all, remuda
+self-configures: an implicit `openrouter` provider and a `free` pool
+discovered from its free tier. A local model server answering on
+`127.0.0.1:11434` (Ollama, for instance) adds an implicit `local` provider.
+Every implicit resolution is announced on stderr, an explicit registry entry
+of the same name always wins, and `--no-bootstrap` (or `REMUDA_NO_BOOTSTRAP=1`)
+turns it off.
+
+```bash
+# one question: the answer is all that reaches stdout
+remuda run "Name the capital of Spain" -p free
+
+# inline bulk: CSV in, the same CSV out with one new column
+remuda run "Severity of this complaint: {{ complaint }}" \
+    -i complaints.csv --field severity --vocab high,low -p free -o enriched.csv
+
+# a repeatable job directory
+remuda init jobs/my-job          # scaffold a commented job directory
+remuda check jobs/my-job         # lint the spec, its input file and its pools
+remuda preview jobs/my-job -n 3  # see the exact prompts, zero model calls
+remuda run jobs/my-job           # derive every field for every row
+remuda render .runs/my-job/<ts> --enrich -o out.csv   # input + new columns
+```
+
+All three shapes are one engine. A run resumes by default: re-invoke the same
+command and only what is left is computed. Any key the pool could not answer
+makes the run exit non-zero. A partial result is never reported as success.
+
+## The ladder
+
+An answer that fails validation is retried on the same model with feedback
+describing what was wrong, then the chunk rotates to the next model in the
+pool, then to the pool's mop-up model, and only then is the key recorded as
+failed with its last reason. Within a packed call, valid answers are banked
+immediately; only the keys still invalid re-enter the ladder. Rate limits and
+transport failures cool a model down and redistribute its work without
+consuming a validation attempt.
+
+Each chunk prints a progress line to stderr as it lands, naming the model that
+answered. Everything a run decides is appended to
+`.runs/<job>/<timestamp>/ledger.jsonl` as it happens, so a killed run loses
+nothing.
+
+## What the models actually did
+
+```bash
+remuda runs list --runs-dir jobs/my-job/.runs
+remuda stats    --runs-dir jobs/my-job/.runs   # success rate, latency, cost
+remuda runs prune --keep 5 --runs-dir jobs/my-job/.runs   # dry; --write deletes
+```
+
+`stats` ranks models by how they have really performed, so YOU can reorder a
+pool. remuda never reorders one itself: a tool that rewrote its own
+configuration from yesterday's latency would be impossible to reason about.
+
+## Embedding
+
+```python
+from remuda import Job, Registry, run_sync
+
+report = run_sync(job, rows, registry, sink=collect, only=["severity"])
+```
+
+Rows are plain mappings from any source, results arrive per completed row, and
+the `Report` is the same object the CLI persists. No files are required. The
+registry is also constructible entirely in code; configuration files are one
+loader over it, never the only way in.
+
+## Job directory
+
+```
+my-job/
+├── job.yaml            # name, input, fields
+├── input.csv           # rows (CSV, JSONL or a JSON array)
+└── vocab/labels.txt    # closed vocabulary for a `classify` field
+```
+
+Field kinds: `classify` (closed vocabulary), `extract` (declared shape),
+`generate` (free text under constraints), `score` (number in a range), and
+`map` (deterministic lookup, no model involved). A field may declare a `when:`
+precondition and may `depends_on` one previously derived field (one level, no
+chains).
+
+A prompt template can only read the columns listed in `prompt.inputs`. Any
+other column on the row is unreachable from the template, and a template
+variable outside that allowlist is refused at load time.
+
+## Registry
+
+Endpoints, credentials and model quirks are named once, never inside a job:
+
+```
+~/.config/remuda/       # user level
+./.remuda/              # project level, overrides the user level by name
+├── providers.yaml
+├── models.yaml
+└── pools.yaml
+```
+
+Credentials are never written in a file: a provider names the environment
+variable its key is read from (`key_env`). Pools are ordered model references
+with a `scatter` or `waterfall` strategy, an optional paid mop-up model, and
+optional `discover:` queries resolved against a provider's live catalog when a
+run starts.
+
+### Discovering models
+
+A pool entry can be a query against a provider's live catalog instead of a
+model name:
+
+```yaml
+free-big:
+  strategy: scatter
+  entries:
+    - discover: {provider: openrouter, free: true, min_context: 32000,
+                 sort: context, take: 4}
+```
+
+Each provider declares which catalog shape it serves (`openrouter`,
+`openai_compat`, `ollama`). A filter the catalog cannot answer (free-tier
+filtering against a bare `/v1/models` list, for instance) is **refused**
+naming the filter and the provider, never silently ignored.
+
+Queries resolve when a run starts, and the resolved membership is snapshotted
+into the run directory: a resumed run reuses exactly the models the first
+attempt used, because free-tier membership churns week to week.
+
+```bash
+remuda pools show free-big      # materialized membership
+remuda pools check free-big     # one minimal request per member
+remuda models list openrouter --free
+```
+
+### Transports
+
+| Provider kind | Runs where |
+|---|---|
+| `openai_compat` (OpenRouter, Ollama, vLLM, NIM) | anywhere |
+| `opencode` (`opencode run -m <model>`) | the operator's machine only |
+
+The opencode transport builds its command as an argument list and never
+invokes a shell, so row data reaching the prompt cannot become a command. Its
+version is probed once before a run that uses it.
+
+## Documentation
+
+The full CLI surface is documented in [docs/usage.md](docs/usage.md), and
+recorded design exceptions in [docs/decisions.md](docs/decisions.md).
+
+## Development
+
+```bash
+poetry install
+poetry run pytest -q
+poetry run ruff check . && poetry run ruff format --check .
+poetry run mypy .
+```
+
+## License
+
+[MIT](LICENSE)
+

remuda-0.1.0/README.md

@@ -0,0 +1,195 @@
+# remuda
+
+Bulk LLM inference over pools of free/cheap models. Ride one until it tires, swap to the next.
+
+[![Test](https://github.com/daviunx/remuda/actions/workflows/test.yml/badge.svg)](https://github.com/daviunx/remuda/actions/workflows/test.yml)
+[![PyPI](https://img.shields.io/pypi/v/remuda)](https://pypi.org/project/remuda/)
+[![Python](https://img.shields.io/pypi/pyversions/remuda)](https://pypi.org/project/remuda/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+A *remuda* is the herd of spare horses a working cowboy draws from: when one
+tires, you saddle the next and keep moving. This tool does that with language
+models. Point it at a pool of free or cheap models, hand it a thousand rows,
+and it classifies, extracts, scores or generates a column for every one:
+validating every answer, retrying with feedback, rotating past rate limits and
+broken models, and never reporting a partial result as success.
+
+## Why
+
+Free-tier models are individually unreliable and collectively excellent. On
+any given day a quarter of a free pool answers well, another quarter is rate
+limited, and the rest return markdown essays when you asked for one word.
+remuda's job is to make that mess dependable: a validation ladder per answer,
+rotation across the pool, a crash-safe ledger, and resume that retries only
+what failed, at no repeat cost.
+
+## Install
+
+```bash
+pipx install remuda        # recommended for the CLI
+pip install remuda         # or as a library
+```
+
+## Quick start
+
+With `OPENROUTER_API_KEY` exported and no configuration at all, remuda
+self-configures: an implicit `openrouter` provider and a `free` pool
+discovered from its free tier. A local model server answering on
+`127.0.0.1:11434` (Ollama, for instance) adds an implicit `local` provider.
+Every implicit resolution is announced on stderr, an explicit registry entry
+of the same name always wins, and `--no-bootstrap` (or `REMUDA_NO_BOOTSTRAP=1`)
+turns it off.
+
+```bash
+# one question: the answer is all that reaches stdout
+remuda run "Name the capital of Spain" -p free
+
+# inline bulk: CSV in, the same CSV out with one new column
+remuda run "Severity of this complaint: {{ complaint }}" \
+    -i complaints.csv --field severity --vocab high,low -p free -o enriched.csv
+
+# a repeatable job directory
+remuda init jobs/my-job          # scaffold a commented job directory
+remuda check jobs/my-job         # lint the spec, its input file and its pools
+remuda preview jobs/my-job -n 3  # see the exact prompts, zero model calls
+remuda run jobs/my-job           # derive every field for every row
+remuda render .runs/my-job/<ts> --enrich -o out.csv   # input + new columns
+```
+
+All three shapes are one engine. A run resumes by default: re-invoke the same
+command and only what is left is computed. Any key the pool could not answer
+makes the run exit non-zero. A partial result is never reported as success.
+
+## The ladder
+
+An answer that fails validation is retried on the same model with feedback
+describing what was wrong, then the chunk rotates to the next model in the
+pool, then to the pool's mop-up model, and only then is the key recorded as
+failed with its last reason. Within a packed call, valid answers are banked
+immediately; only the keys still invalid re-enter the ladder. Rate limits and
+transport failures cool a model down and redistribute its work without
+consuming a validation attempt.
+
+Each chunk prints a progress line to stderr as it lands, naming the model that
+answered. Everything a run decides is appended to
+`.runs/<job>/<timestamp>/ledger.jsonl` as it happens, so a killed run loses
+nothing.
+
+## What the models actually did
+
+```bash
+remuda runs list --runs-dir jobs/my-job/.runs
+remuda stats    --runs-dir jobs/my-job/.runs   # success rate, latency, cost
+remuda runs prune --keep 5 --runs-dir jobs/my-job/.runs   # dry; --write deletes
+```
+
+`stats` ranks models by how they have really performed, so YOU can reorder a
+pool. remuda never reorders one itself: a tool that rewrote its own
+configuration from yesterday's latency would be impossible to reason about.
+
+## Embedding
+
+```python
+from remuda import Job, Registry, run_sync
+
+report = run_sync(job, rows, registry, sink=collect, only=["severity"])
+```
+
+Rows are plain mappings from any source, results arrive per completed row, and
+the `Report` is the same object the CLI persists. No files are required. The
+registry is also constructible entirely in code; configuration files are one
+loader over it, never the only way in.
+
+## Job directory
+
+```
+my-job/
+├── job.yaml            # name, input, fields
+├── input.csv           # rows (CSV, JSONL or a JSON array)
+└── vocab/labels.txt    # closed vocabulary for a `classify` field
+```
+
+Field kinds: `classify` (closed vocabulary), `extract` (declared shape),
+`generate` (free text under constraints), `score` (number in a range), and
+`map` (deterministic lookup, no model involved). A field may declare a `when:`
+precondition and may `depends_on` one previously derived field (one level, no
+chains).
+
+A prompt template can only read the columns listed in `prompt.inputs`. Any
+other column on the row is unreachable from the template, and a template
+variable outside that allowlist is refused at load time.
+
+## Registry
+
+Endpoints, credentials and model quirks are named once, never inside a job:
+
+```
+~/.config/remuda/       # user level
+./.remuda/              # project level, overrides the user level by name
+├── providers.yaml
+├── models.yaml
+└── pools.yaml
+```
+
+Credentials are never written in a file: a provider names the environment
+variable its key is read from (`key_env`). Pools are ordered model references
+with a `scatter` or `waterfall` strategy, an optional paid mop-up model, and
+optional `discover:` queries resolved against a provider's live catalog when a
+run starts.
+
+### Discovering models
+
+A pool entry can be a query against a provider's live catalog instead of a
+model name:
+
+```yaml
+free-big:
+  strategy: scatter
+  entries:
+    - discover: {provider: openrouter, free: true, min_context: 32000,
+                 sort: context, take: 4}
+```
+
+Each provider declares which catalog shape it serves (`openrouter`,
+`openai_compat`, `ollama`). A filter the catalog cannot answer (free-tier
+filtering against a bare `/v1/models` list, for instance) is **refused**
+naming the filter and the provider, never silently ignored.
+
+Queries resolve when a run starts, and the resolved membership is snapshotted
+into the run directory: a resumed run reuses exactly the models the first
+attempt used, because free-tier membership churns week to week.
+
+```bash
+remuda pools show free-big      # materialized membership
+remuda pools check free-big     # one minimal request per member
+remuda models list openrouter --free
+```
+
+### Transports
+
+| Provider kind | Runs where |
+|---|---|
+| `openai_compat` (OpenRouter, Ollama, vLLM, NIM) | anywhere |
+| `opencode` (`opencode run -m <model>`) | the operator's machine only |
+
+The opencode transport builds its command as an argument list and never
+invokes a shell, so row data reaching the prompt cannot become a command. Its
+version is probed once before a run that uses it.
+
+## Documentation
+
+The full CLI surface is documented in [docs/usage.md](docs/usage.md), and
+recorded design exceptions in [docs/decisions.md](docs/decisions.md).
+
+## Development
+
+```bash
+poetry install
+poetry run pytest -q
+poetry run ruff check . && poetry run ruff format --check .
+poetry run mypy .
+```
+
+## License
+
+[MIT](LICENSE)

remuda-0.1.0/pyproject.toml

@@ -0,0 +1,135 @@
+[project]
+name = "remuda"
+version = "0.1.0"
+description = "Bulk LLM inference over pools of free/cheap models: ride one until it tires, swap to the next"
+authors = [{name = "David Marsa", email = "david.marsa@neomanex.com"}]
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.12,<3.14"
+keywords = ["llm", "inference", "bulk", "openrouter", "classification", "csv", "cli"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+dependencies = [
+    "pydantic (>=2.10,<3.0)",
+    "pyyaml (>=6.0,<7.0)",
+    "jinja2 (>=3.1,<4.0)",
+    "typer (>=0.15,<0.20)",
+    "rich (>=13.9,<15.0)",
+    "httpx (>=0.28,<0.29)",
+]
+
+[project.urls]
+Repository = "https://github.com/daviunx/remuda"
+Issues = "https://github.com/daviunx/remuda/issues"
+Changelog = "https://github.com/daviunx/remuda/blob/main/CHANGELOG.md"
+
+[project.scripts]
+remuda = "remuda.cli.app:main"
+
+[dependency-groups]
+dev = [
+    "pytest (>=8.3.4,<10.0.0)",
+    "pytest-asyncio (>=1.1.0,<2.0.0)",
+    "pytest-cov (>=6.0.0,<7.0.0)",
+    "mypy (>=1.14,<2.0)",
+    "types-PyYAML (>=6.0,<7.0)",
+    # Pinned in the manifest and installed from it by CI. A gate that resolves
+    # its linter at run time reddens on someone else's release date
+    # (documentation/standards/python/03-tooling.md).
+    "ruff (>=0.16.0,<0.17.0)",
+]
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"
+
+[tool.ruff]
+line-length = 88
+target-version = "py312"
+exclude = [".git", ".venv", "__pycache__"]
+
+[tool.ruff.lint]
+# EXPLICIT, never inherited — a tool's default rule set is not a stable contract
+# across versions (documentation/standards/python/03-tooling.md).
+select = [
+    "E",    # pycodestyle errors
+    "W",    # pycodestyle warnings
+    "F",    # Pyflakes
+    "I",    # isort
+    "B",    # flake8-bugbear
+    "C4",   # flake8-comprehensions
+    "UP",   # pyupgrade
+    "ARG",  # flake8-unused-arguments
+    "SIM",  # flake8-simplify
+    "TC",   # flake8-type-checking
+    "PTH",  # flake8-use-pathlib
+    "ERA",  # eradicate (commented code)
+    "PL",   # Pylint
+    "RUF",  # Ruff-specific rules
+]
+ignore = [
+    "E501",    # Line too long (handled by formatter)
+    "PLR0913", # Too many arguments
+    "PLR2004", # Magic value comparison
+]
+
+[tool.ruff.lint.per-file-ignores]
+"tests/*" = ["ARG", "PLR2004"]
+
+[tool.ruff.lint.isort]
+known-first-party = ["remuda"]
+
+[tool.ruff.format]
+quote-style = "double"
+indent-style = "space"
+skip-magic-trailing-comma = false
+line-ending = "auto"
+
+[tool.mypy]
+python_version = "3.12"
+strict = true
+warn_return_any = true
+warn_unused_ignores = true
+disallow_untyped_defs = true
+disallow_incomplete_defs = true
+check_untyped_defs = true
+disallow_untyped_decorators = true
+no_implicit_optional = true
+warn_redundant_casts = true
+warn_unused_configs = true
+plugins = ["pydantic.mypy"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+asyncio_mode = "auto"
+asyncio_default_fixture_loop_scope = "function"
+addopts = ["-v", "--tb=short", "--strict-markers"]
+markers = [
+    "slow: marks tests as slow",
+    "integration: marks tests as integration tests",
+]
+
+[tool.coverage.run]
+source = ["remuda"]
+branch = true
+# The engine appends the ledger through asyncio.to_thread, so lines execute on
+# worker threads; without this, coverage loses track of them
+# (documentation/standards/testing/overview.md → Coverage Enforcement).
+# `greenlet` is deliberately absent — remuda has no greenlet dependency and
+# coverage errors out on a concurrency mode whose module is not installed.
+concurrency = ["thread"]
+omit = ["*/tests/*", "*/__pycache__/*"]
+
+[tool.coverage.report]
+exclude_lines = [
+    "pragma: no cover",
+    "def __repr__",
+    "raise NotImplementedError",
+    "if TYPE_CHECKING:",
+    "if __name__ == .__main__.:",
+]
+fail_under = 80

remuda-0.1.0/remuda/__init__.py

@@ -0,0 +1,128 @@
+"""remuda — bulk LLM inference over pools of free/cheap models.
+
+The herd of spare horses on a cattle drive: ride one until it tires, swap to
+the next. Public API surface lives here; internal module paths are not a
+contract.
+"""
+
+from remuda.api import JobRunOutcome, run, run_job_dir, run_sync
+from remuda.engine.plan import EngineError
+from remuda.engine.progress import ProgressEvent
+from remuda.engine.runner import RowResult, Runner
+from remuda.errors import RemudaError
+from remuda.inspection import (
+    CheckReport,
+    PreviewReport,
+    RenderedPrompt,
+    check_job_dir,
+    preview_job,
+)
+from remuda.ledger import (
+    LedgerEntry,
+    RenderError,
+    RenderRequest,
+    Report,
+    ResumeRefusedError,
+    RunStore,
+    SpecLock,
+    render,
+)
+from remuda.registry import (
+    DiscoverQuery,
+    ModelConfig,
+    Pool,
+    PoolEntry,
+    Provider,
+    Registry,
+    RegistryError,
+    RegistryValidationError,
+    default_config_dirs,
+    load_registry,
+)
+from remuda.rows import RowSourceError, read_columns, read_rows
+from remuda.spec import (
+    ExtractSchema,
+    FieldSpec,
+    GenerateConstraints,
+    InputSpec,
+    Job,
+    MapTable,
+    Precondition,
+    PromptRenderError,
+    PromptSpec,
+    ScoreRange,
+    SpecValidationError,
+)
+from remuda.transport import (
+    CompletionRequest,
+    CompletionResult,
+    PermanentTransportError,
+    RateLimitedError,
+    TransientTransportError,
+    Transport,
+    TransportError,
+    Usage,
+)
+from remuda.validate import Outcome, Verdict, validate_answer
+from remuda.version import __version__
+
+__all__ = [
+    "CheckReport",
+    "CompletionRequest",
+    "CompletionResult",
+    "DiscoverQuery",
+    "EngineError",
+    "ExtractSchema",
+    "FieldSpec",
+    "GenerateConstraints",
+    "InputSpec",
+    "Job",
+    "JobRunOutcome",
+    "LedgerEntry",
+    "MapTable",
+    "ModelConfig",
+    "Outcome",
+    "PermanentTransportError",
+    "Pool",
+    "PoolEntry",
+    "Precondition",
+    "PreviewReport",
+    "ProgressEvent",
+    "PromptRenderError",
+    "PromptSpec",
+    "Provider",
+    "RateLimitedError",
+    "Registry",
+    "RegistryError",
+    "RegistryValidationError",
+    "RemudaError",
+    "RenderError",
+    "RenderRequest",
+    "RenderedPrompt",
+    "Report",
+    "ResumeRefusedError",
+    "RowResult",
+    "RowSourceError",
+    "RunStore",
+    "Runner",
+    "ScoreRange",
+    "SpecLock",
+    "SpecValidationError",
+    "TransientTransportError",
+    "Transport",
+    "TransportError",
+    "Usage",
+    "Verdict",
+    "__version__",
+    "check_job_dir",
+    "default_config_dirs",
+    "load_registry",
+    "preview_job",
+    "read_columns",
+    "read_rows",
+    "render",
+    "run",
+    "run_job_dir",
+    "run_sync",
+    "validate_answer",
+]