ebaml 0.1.0__tar.gz

ebaml-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026, Johnny Godoy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ebaml-0.1.0/PKG-INFO

@@ -0,0 +1,450 @@
+Metadata-Version: 2.4
+Name: ebaml
+Version: 0.1.0
+Summary: BAML for Explainable Boosting Machines: talk to your EBM.
+Keywords: baml,llm,explainable-ai,explainable-boosting-machine,interpretml
+Author: Johnny Godoy
+Author-email: Johnny Godoy <johnny.godoy@ing.uchile.cl>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Dist: attrs>=26.1.0
+Requires-Dist: baml-bridge>=0.23.0
+Requires-Dist: beartype>=0.23.1
+Requires-Dist: interpret>=0.7.8
+Requires-Dist: orjson>=3.13.0
+Requires-Dist: structlog>=26.1.0
+Requires-Python: >=3.13
+Project-URL: Documentation, https://johnny-godoy.github.io/EBaMl/
+Project-URL: Source, https://github.com/johnny-godoy/EBaMl
+Project-URL: Issues, https://github.com/johnny-godoy/EBaMl/issues
+Project-URL: Changelog, https://github.com/johnny-godoy/EBaMl/blob/main/CHANGELOG.md
+Description-Content-Type: text/markdown
+
+# EBaMl
+
+<!-- --8<-- [start:intro] -->
+BAML for Explainable Boosting Machines. Python extracts the graphs from an EBM, and BAML holds the prompts, the LLM
+clients and the chain-of-thought pipelines.
+
+EBaMl is based on [TalkToEBM](https://github.com/interpretml/TalkToEBM) and adds local
+explanations (why did the model predict this for this input?) computed exactly and put in words for a layperson or a
+data scientist, a benchmark that scores answers automatically, a configurable LLM provider, token budgets and typed,
+structured logging.
+
+LLM calls go through [OpenRouter](https://openrouter.ai)'s free router (`openrouter/free`), so no paid API key is needed.
+
+<!-- --8<-- [end:intro] -->
+## Supported models
+
+<!-- --8<-- [start:supported-models] -->
+| Feature | Binary classifier | Regressor | Multiclass classifier | Interactions |
+| --- | :---: | :---: | :---: | :---: |
+| Graph descriptions (`describe_graph`) | ✅ | ✅ | ❌ | ⚠️ |
+| Model summary (`describe_ebm`) | ✅ | ✅ | ❌ | ⚠️ |
+| Graph text (`EBMGraph`) | ✅ | ✅ | ❌ | ⚠️ |
+| Local explanations (`explain_instance`) | ✅ | ❌ | ❌ | ❌ |
+
+✅ supported. ⚠️ partly supported: the main effects are described, and the interaction terms are skipped without a
+warning. ❌ not supported yet. Full support is on the
+[roadmap](https://github.com/johnny-godoy/EBaMl/blob/main/ROADMAP.md).
+<!-- --8<-- [end:supported-models] -->
+## Setup
+
+<!-- --8<-- [start:setup] -->
+### Requirements
+
+- Python 3.13 or later and [uv](https://docs.astral.sh/uv/getting-started/installation/).
+- The BAML toolchain 0.23 or later (step 1).
+- An API key for an LLM provider. The default is [OpenRouter](https://openrouter.ai)'s free router, which needs no
+  paid plan. You can use another provider instead, see "Choosing the model".
+
+### 1. Install the BAML toolchain
+
+```bash
+curl -fsSL https://pkg.boundaryml.com/install.sh | sh -s
+```
+
+The installer prints a line to add to your shell profile. Add it, then open a new shell:
+
+```bash
+. "$HOME/.baml/env"
+```
+
+Check it:
+
+```bash
+baml --version   # toolchain 0.23 or later
+```
+
+If `baml` resolves to an older install (Homebrew `baml`, or `baml-py` in a virtualenv), remove it or put `~/.baml/bin`
+first on `PATH`. The old CLIs have no `describe`, `generate add` or `agent` commands.
+
+If a command stops with "the BAML agent skill is required", run `baml agent install` and retry.
+
+### 2. Get the code and install the Python dependencies
+
+```bash
+git clone <repository-url> EBaMl
+cd EBaMl
+uv sync
+```
+
+### 3. Set the API key of your LLM provider
+
+For the default provider, OpenRouter:
+
+1. Create an account at <https://openrouter.ai> and open <https://openrouter.ai/settings/keys>.
+2. Create a key. Keys start with `sk-or-`.
+3. Export it in the shell you run the code from:
+
+```bash
+export OPENROUTER_API_KEY=sk-or-...
+```
+
+The key is read from the environment when a request is sent. It is never written to a `.baml` file.
+
+Using another provider? Set its variable instead (`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, ...) and pass `model=` as
+described in "Choosing the model". Local models (Ollama) need no key.
+
+### 4. Generate the Python SDK
+
+```bash
+baml generate
+```
+
+This writes the `src/baml_sdk/` package from `baml_src/`, next to `ebaml`, so it imports wherever the project is
+installed. Rerun it every time a `.baml` file changes, including when you change the provider.
+
+If `src/baml_sdk/` does not exist, `ebaml.llm.descriptors` runs `baml generate` when you import it. If the package was
+generated by another BAML toolchain than the one that `baml-bridge` expects (a version skew), the import runs
+`baml toolchain use <version>` and `baml generate`. If the agent skill check stops `baml generate`, the import runs it
+again with `--agent-skill-check off`. If none of this works, `SDKUnavailableError` lists the steps that fix the cause.
+
+### 5. Check the setup
+
+```bash
+OPENROUTER_API_KEY=unused baml test -x root::Live   # offline tests on a scripted client, no network
+baml run main                         # prints "hello from baml"
+```
+
+These two do not call an LLM, so they pass without a working key or provider. They check the toolchain and the SDK.
+
+### 6. Live test (optional)
+
+Makes real requests to your provider (9 on the OpenRouter free router, so it can be slow or rate limited). It does
+nothing without `OPENROUTER_API_KEY`, so with another provider run the example under "Usage" instead.
+
+```bash
+OPENROUTER_API_KEY=sk-or-... uv run python scripts/live_openrouter.py
+```
+
+It fits a small EBM, runs the three-turn chain directly and checks that the provider reports token usage, then runs
+`describe_graph` and `describe_ebm`. It exits non-zero on the first failed assertion.
+
+### Troubleshooting
+
+| Symptom | Cause and fix |
+| --- | --- |
+| `baml: command not found` | The profile line from step 1 is not loaded. Open a new shell or run `. "$HOME/.baml/env"`. |
+| `unknown command: describe` (or `generate add`, `agent`) | An older `baml` is first on `PATH`. See step 1. |
+| `SDKUnavailableError`, or `ModuleNotFoundError: baml_sdk` | The SDK is not generated or has the wrong version, and `ebaml` could not fix it. The message of the error lists the steps. Run `baml generate` (step 4). |
+| `MissingAPIKeyError` | The key of the provider is not exported in this shell. The message names the variable. See "Choosing the model" for the variable of each provider. |
+| `APIKeyRejectedError` | The provider rejected the key. Check the key, its account and its credit. |
+| `RateLimitedError` | The provider limited the rate. Lower the concurrency, `describe_ebm(ebm, max_concurrency=2)`, or pin a paid model. |
+| `ProviderUnreachableError` | A request did not reach the provider. Check the network, `HTTPS_PROXY` and `base_url`. |
+
+<!-- --8<-- [end:setup] -->
+## Usage
+
+<!-- --8<-- [start:usage] -->
+```python
+from interpret.glassbox import ExplainableBoostingClassifier
+
+from ebaml.llm.descriptors import describe_ebm, describe_graph
+from ebaml.llm.options import PromptOptions
+
+ebm = ExplainableBoostingClassifier().fit(X, y)
+
+print(describe_graph(ebm, feature_index=0))
+print(describe_ebm(ebm, num_sentences=30, prompt=PromptOptions(dataset_description="...")))
+```
+
+A graph can be turned into text with `EBMGraph.to_text` and back with `EBMGraph.from_text`, which also reads a graph
+embedded in a longer prompt. It needs the confidence bounds, and it only reads continuous features.
+
+Three keyword-only arguments set the options. `graph=GraphOptions(...)` sets the options of `EBMGraph.from_ebm` and
+`EBMGraph.to_text`, for example `max_tokens`. `prompt=PromptOptions(...)` sets the descriptions that the prompts use
+(`expert_description`, `graph_description`, `dataset_description` and `task_description`). `model=` selects the LLM, see
+"Choosing the model". The classes are in `ebaml.llm.options`. Token usage reported by the provider is logged at `INFO` on the `ebaml.llm.descriptors` logger.
+
+### Explaining one prediction
+
+```python
+from ebaml.llm.descriptors import describe_ebm, explain_instance
+
+ebm = ExplainableBoostingClassifier(interactions=0).fit(X, y)  # binary classifier, no interactions
+global_explanation = describe_ebm(ebm)  # computed once, prepended to every local prompt
+
+explanation = explain_instance(ebm, X.iloc[0].to_dict(), global_explanation, audience="layperson")
+print(explanation.counterfactual)      # what would have to change to flip the prediction
+print(explanation.sufficient_reason)   # which inputs settle the prediction on their own
+print(explanation.completions)         # how certain the outcome is as inputs become known
+```
+
+The facts are computed exactly from the scores and confidence intervals of the EBM (`ebaml.local`); the LLM only puts
+them in words. They are the three queries of [arXiv:2010.12265](https://arxiv.org/abs/2010.12265): minimum change
+required, minimum sufficient reason and count completions. `audience="layperson"` (default) gives no model numbers;
+`audience="data_scientist"` quotes scores, probabilities and confidence intervals. It supports binary classifiers
+without interactions, and `explanation.facts` holds the numbers behind the text. The "Local explanations" page of the documentation explains what each explanation means, the options and the limits,
+and `notebooks/local_explanations.py` walks through them on the Titanic data.
+
+### Errors
+
+Every error that `ebaml` raises on purpose derives from `ebaml.EBaMlError`. Use `except EBaMlError` to catch the errors
+of `ebaml` apart from the errors of other code. Each error for a bad value also derives from `ValueError`, so an
+`except ValueError` catches it too.
+
+| Error | `ValueError` | Raised when |
+| --- | --- | --- |
+| `InvalidOptionError` | yes | An option is not one of its valid values, for example an unknown `Normalization`. |
+| `OptionRangeError` | yes | A number option is outside its range, for example `confidence_level=1`. |
+| `InvalidInstanceError` | yes | The instance to explain lacks a value, or has a category that the EBM did not see. |
+| `UnsupportedModelError` | yes | A local explanation gets an EBM with interactions, or one that is not a binary classifier. |
+| `GraphTextError` | yes | `EBMGraph.from_text` cannot read the text of a graph. |
+| `EBMGraphError` | no | The base class of the errors of a graph. |
+| `EBMGraphShapeError` | yes | The bins of a graph do not fit its scores, or a boolean format gets more than two bins. |
+| `EBMGraphTokenLimitError` | no | The text of a graph is over the token budget, and its format cannot be simplified. |
+| `EBMGraphSimplificationError` | no | The text of a graph is still over the token budget at the largest simplification. |
+| `SDKUnavailableError` | no | `baml_sdk` is missing, and `ebaml` cannot run `baml generate`. The error also derives from `ImportError`. |
+| `LLMRequestError` | no | The base class of the errors of a provider. Its message lists the steps that fix the cause. |
+| `MissingAPIKeyError` | no | A provider needs an API key and the environment has none. |
+| `APIKeyRejectedError` | no | A provider answers a request with "unauthorized" or "forbidden". |
+| `RateLimitedError` | no | A provider limits the rate of the requests. |
+| `ProviderUnreachableError` | no | A request does not reach the provider. |
+
+`EBMGraphShapeError`, `EBMGraphTokenLimitError` and `EBMGraphSimplificationError` derive from `EBMGraphError`. The four provider errors derive from `LLMRequestError`. All classes are in `ebaml.core.exceptions`, and `ebaml` exports
+them too.
+
+<!-- --8<-- [end:usage] -->
+## How it works
+
+| Where | What |
+| --- | --- |
+| `src/ebaml/core/` | Shared kernel: types and errors. It imports no other part of `ebaml`. |
+| `src/ebaml/graphs/` | Graph extraction and simplification (numpy, `interpret`), and the text the LLM reads. |
+| `src/ebaml/llm/` | The edge: the calls into BAML (`descriptors.py`), their options and the retry after a cut-off reply. |
+| `baml_src/talk_to_ebm.baml` | Prompts, the default LLM clients, the three-turn description chain, the summary, `describe_ebm`, token accounting. |
+| `baml_src/model_spec.baml` | The model of a call: `ModelSpec` and one client builder for each provider. |
+| `src/ebaml/local/`, `baml_src/local_explanations.baml` | Local explanations: the facts of one prediction computed deterministically, their rendering as JSON texts, and the prompts that put them in words. |
+| `baml_src/talk_to_ebm_test.baml`, `baml_src/model_spec_test.baml` | Offline tests on a scripted client. |
+| `baml.toml` | Generator config for the Python SDK. |
+| `src/ebaml/benchmark.py`, `benchmark/` | The benchmark: loading, answer parsing and scoring, and the data and runner (see [Benchmark](#benchmark)). |
+| `notebooks/` | Example notebooks, written with marimo. |
+| `docs/`, `zensical.toml` | The documentation, built with Zensical (see [Development](#development)). |
+| `tests/unit/` | Python unit tests (100% coverage required). `scripts/live_openrouter.py` is the optional live test. |
+| `scripts/` | Maintenance scripts, see [scripts/README.md](scripts/README.md). |
+| `.github/` | CI workflow and the scripts that turn failures into reports. |
+| `src/baml_sdk/` | Generated. Do not edit. |
+
+<!-- --8<-- [start:behaviour] -->
+- **Model:** `openrouter/free` picks a free model for each request, so the model can change between calls and between
+  the turns of one conversation. Quality and speed vary with it.
+- **Retries:** transient failures (rate limits, network errors, 5xx) are retried up to 3 times. Requests the provider
+  rejects are not retried.
+- **Concurrency:** `describe_ebm` describes features concurrently, at most `max_concurrency` at a time (default 8).
+  Free models are rate limited, so lower it (`describe_ebm(ebm, max_concurrency=2)`) if you see rate-limit errors.
+- **Token budget:** there is no tokenizer before a call, so the check uses a rough estimate (about 3 characters per
+  token). Going a little over `max_tokens` only warns. Going more than `overbudget_percent` (default 50) over makes
+  `to_text` simplify the graph, or raise if it cannot. The provider's own error is the real limit. The functions in
+  `ebaml.llm.descriptors` pass the BAML check. A direct call of `EBMGraph.to_text` checks nothing, unless you pass
+  `check_budget`, for example `baml_sdk.check_token_budget`.
+
+<!-- --8<-- [end:behaviour] -->
+## Notebook
+
+[`notebooks/titanic.py`](notebooks/titanic.py) is a [marimo](https://marimo.io) notebook that ports TalkToEBM's example.
+It trains an EBM on the Titanic dataset (downloaded from OpenML), asks the LLM to describe a graph and the whole model,
+and shows the graph and text API (`from_ebm`, `simplify`, `to_text`, `from_text`). The LLM cells run only when you press
+their button. They use the default model, which needs `OPENROUTER_API_KEY`. See "Choosing the model" for other models.
+
+```bash
+uv run marimo edit notebooks/titanic.py   # interactive
+uv run marimo run notebooks/titanic.py    # as an app
+```
+
+[`notebooks/local_explanations.py`](notebooks/local_explanations.py) is the place to learn the local explanations: pick a
+passenger, see the facts with no API key, move the confidence slider, check the guarantees against random completions,
+compare what the LLM reads for a layperson and a data scientist, and (with a key) get the explanations in words.
+
+```bash
+uv run marimo edit notebooks/local_explanations.py
+```
+
+## Benchmark
+
+The benchmark follows [TalkToEBM's benchmark](https://github.com/interpretml/TalkToEBM/tree/main/benchmark) that checks how well a
+model reads EBM graphs: function recognition (easy and hard), the largest jump, monotonicity, reading a value and the
+widest confidence interval. It makes real requests, and it asks two LLM turns per question (6 tasks, 480 questions in
+all). The default model needs `OPENROUTER_API_KEY`. The flag `--model provider/model` selects another model.
+
+```bash
+uv run python benchmark/run.py --task monotonicity --limit 10 --output results.json
+```
+
+See [benchmark/README.md](benchmark/README.md) for the tasks, the scoring and what it leaves out.
+
+## Choosing the model
+
+<!-- --8<-- [start:changing-model] -->
+The default model is OpenRouter's free router, `openrouter/free`. It needs `OPENROUTER_API_KEY`. The `model` argument
+of `describe_graph`, `describe_ebm` and `explain_instance` selects another model for one call. EBaMl sends every LLM
+request of that call to that model. You do not change the package.
+
+### Quick start
+
+Pass the shorthand `"provider/model"`:
+
+```python
+from ebaml.llm.descriptors import describe_ebm
+
+print(describe_ebm(ebm, model="anthropic/claude-haiku-4-5"))
+```
+
+Pass `ModelOptions` to set more:
+
+```python
+from ebaml.llm.options import ModelOptions
+
+model = ModelOptions(provider="anthropic", model="claude-haiku-4-5", temperature=0.2, max_output_tokens=8000)
+print(describe_ebm(ebm, model=model))
+```
+
+EBaMl splits the shorthand at the first `/`. The text `"openrouter/openai/gpt-4o-mini"` selects the provider
+`openrouter` and the model `openai/gpt-4o-mini`. EBaMl checks the provider before the first paid request. It raises
+`InvalidOptionError` if the provider is not in the table below.
+
+### Providers
+
+The provider is a BAML provider prefix. Set the environment variable of the key in the shell that runs Python.
+
+| Provider | Default key variable | Notes |
+| --- | --- | --- |
+| `openrouter` | `OPENROUTER_API_KEY` | The model name is the OpenRouter name, for example `openai/gpt-4o-mini`. |
+| `openai` | `OPENAI_API_KEY` | |
+| `anthropic` | `ANTHROPIC_API_KEY` | |
+| `google` | `GOOGLE_API_KEY` | The Gemini API. |
+| `ollama` | None | A local server. `base_url` defaults to `http://localhost:11434/v1`. |
+| `openai-generic` | None | Any OpenAI-compatible server. `base_url` is required. Set `api_key_env` if the server needs a key. |
+| `azure` | `AZURE_OPENAI_API_KEY` | The model name is the name of the deployment. Set `base_url` to the deployment, or set `AZURE_OPENAI_ENDPOINT`. |
+| `vertex` | None | Google Cloud credentials (`GOOGLE_APPLICATION_CREDENTIALS` or `gcloud auth`). The project comes from `GOOGLE_CLOUD_PROJECT`. |
+| `bedrock` | None | The AWS credentials of the machine. The region comes from `AWS_REGION`. `api_key_env` has no effect. |
+
+Run `baml describe <client>.new`, for example `baml describe aws.BedrockClient.new`, to see all settings of a client.
+
+### Options
+
+| Field | Meaning | Default |
+| --- | --- | --- |
+| `provider` | The BAML provider prefix from the table. | Required. |
+| `model` | The name of the model at the provider. | Required. |
+| `api_key_env` | The name of the environment variable that holds the key. | The variable of the provider. |
+| `base_url` | The address of the server. | The address of the provider. |
+| `temperature` | The sampling temperature. | 0.7 (0.0 in the benchmark). |
+| `max_output_tokens` | The most tokens of one reply. | See below. |
+| `headers` | Headers to add to each request. | None. |
+
+A field that you set wins over the default of the call. The output limit has two defaults:
+
+| Calls | `max_output_tokens` |
+| --- | --- |
+| Turn 1 of a graph description, the draft of the summary, and the two turns of the benchmark | 8000 |
+| All other calls, including the local explanations | 6000 |
+
+### Examples
+
+```python
+# OpenRouter, a pinned model (the free router picks a new model for each request)
+describe_ebm(ebm, model="openrouter/openai/gpt-4o-mini")
+
+# OpenAI
+describe_ebm(ebm, model="openai/gpt-4o-mini")
+
+# Google Gemini
+describe_ebm(ebm, model="google/gemini-2.5-flash")
+
+# A local model with Ollama. It needs no key.
+describe_ebm(ebm, model="ollama/llama3.1")
+
+# A company gateway with an OpenAI-compatible API
+gateway = ModelOptions(
+    provider="openai-generic",
+    model="my-model",
+    base_url="https://llm.example.com/v1",
+    api_key_env="MY_GATEWAY_KEY",
+    headers={"x-team": "ebm"},
+)
+describe_ebm(ebm, model=gateway)
+```
+
+The benchmark has the flag `--model`:
+
+```bash
+uv run python benchmark/run.py --model anthropic/claude-haiku-4-5 --limit 10
+```
+
+### Keys
+
+Put the key in an environment variable, in the shell or with `os.environ`. `ModelOptions` holds the name of the
+variable, never the key, so the key does not appear in a `repr`, a log line or a traceback. EBaMl reads the variable
+only when it sends a request.
+
+### Limits
+
+- One call uses one model. The turns of a call cannot use different models.
+- Python cannot set up fallback chains or round robin. A BAML client object cannot cross from Python to BAML, so
+  Python sends plain data and BAML builds the client. A BAML client declaration can use `ai.clients.Fallback`: see
+  "Change the default model" in `CONTRIBUTING.md`.
+- A reasoning model counts its hidden tokens in `max_output_tokens`. Keep the limit large, or the reply can end before
+  the answer.
+- Quality, speed and cost depend on the model. Small models can ignore the number of sentences that you ask for.
+
+Every client has the retry policy of the default clients: EBaMl retries a transient failure up to 3 times.
+<!-- --8<-- [end:changing-model] -->
+
+## Development
+
+<!-- --8<-- [start:development] -->
+
+```bash
+baml fmt          # format .baml files
+baml check        # compile check
+baml test -x root::Live   # run the offline BAML tests
+baml generate     # regenerate src/baml_sdk/
+uv run pytest     # Python unit tests, fails under 100% coverage
+uv run pytest README.md docs src --no-cov   # the code examples of the docs and the docstrings
+prek run --all-files                       # every hook: formatters, ruff, vulture, pyrefly, baml fmt, ...
+uv run python scripts/mutation.py          # mutation testing of all modules, fails if any mutant survives
+uv run python scripts/mutation.py --base origin/main   # the same, but only for modules changed since main
+uv run python scripts/mutation.py --module src/ebaml/local/facts.py   # the same, but only for one module
+uv run python -m scripts.check_ignores     # every lint/type/test workaround needs a reason written next to it
+uv run python -m scripts.compile_ignores   # all workarounds and their reasons as one Markdown report to audit
+uv run --group docs python -m scripts.build_docs         # build the docs; fails on a broken link or a docstring warning
+uv run --group docs python -m scripts.build_docs serve   # preview the docs (exports the notebooks, so the first run is slow)
+```
+
+CI (`.github/workflows/ci.yml`) runs prek, secret and security scans (trufflehog, semgrep, trivy), `baml check` and
+`baml test`, pytest with 100% coverage, the code examples of the documentation, and mutation testing of the changed modules with a 100% score. A weekly run tests all modules, one job for each module. Lint and type workarounds
+(`# noqa`, `# type: ignore`, skipped tests, ...) must carry a written reason next to them.
+
+The `python` code blocks of `README.md` and `docs/*.md` run in order, and the blocks of one file share their variables.
+The `Examples` sections of the docstrings run as doctests. `tests/docs/conftest.py` gives each Markdown file a small
+data set as `X` and `y`, and it replaces the LLM calls with fakes, so the examples send no request. Put setup that the
+reader does not need in an `<!-- invisible-code-block: python -->` comment. To skip a block, put
+`<!-- skip: next -->` above it, and write the reason in the text.
+<!-- --8<-- [end:development] -->