fitlab 0.1.0__tar.gz

fitlab-0.1.0/.gitignore

@@ -0,0 +1,16 @@
+# Environments and caches
+.venv/
+__pycache__/
+.ruff_cache/
+*.egg-info/
+
+# Build output
+build/
+dist/
+
+# Local run output
+fitlab-results/
+kernel/
+out/
+results/
+this-week-pins.txt

fitlab-0.1.0/LICENSE

@@ -0,0 +1,3 @@
+Code: Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0)
+Data in /data and published registries: CC-BY-4.0 (https://creativecommons.org/licenses/by/4.0/)
+Benchmark results are point-in-time measurements. Models referenced belong to their respective authors and licenses.

fitlab-0.1.0/PKG-INFO

@@ -0,0 +1,478 @@
+Metadata-Version: 2.5
+Name: fitlab
+Version: 0.1.0
+Summary: Will it run on my setup? Fit-check + benchmark wizard for local LLMs on your GPU — weekly-updated model registry.
+Project-URL: Homepage, https://github.com/ruslanmv/fitlab
+Project-URL: Source, https://github.com/ruslanmv/fitlab
+Project-URL: Issues, https://github.com/ruslanmv/fitlab/issues
+Author: Ruslan Magana Vsevolodovna
+License: Apache-2.0
+License-File: LICENSE
+Keywords: benchmark,colab,gpu,llm,local-ai,ollama,vram
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: rich>=13
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: jsonschema>=4.21; extra == 'dev'
+Requires-Dist: pyyaml>=6.0; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Requires-Dist: twine>=5.1; extra == 'dev'
+Provides-Extra: maintainer
+Requires-Dist: jsonschema>=4.21; extra == 'maintainer'
+Requires-Dist: pyyaml>=6.0; extra == 'maintainer'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="site/logo.svg" width="96" alt="FitLab logo" />
+</p>
+
+<h1 align="center">LLM FitLab</h1>
+
+<p align="center">
+  <strong>An evidence-based compatibility radar for running open LLMs on the hardware you actually have.</strong>
+</p>
+
+<p align="center">
+  <a href="https://github.com/ruslanmv/fitlab/actions/workflows/weekly-sync.yml"><img alt="Weekly sync" src="https://github.com/ruslanmv/fitlab/actions/workflows/weekly-sync.yml/badge.svg" /></a>
+  <a href="https://github.com/ruslanmv/fitlab/actions/workflows/weekly-benchmark.yml"><img alt="Weekly benchmark" src="https://github.com/ruslanmv/fitlab/actions/workflows/weekly-benchmark.yml/badge.svg" /></a>
+  <img alt="Python" src="https://img.shields.io/badge/python-3.10%2B-blue" />
+  <img alt="Code license" src="https://img.shields.io/badge/code-Apache--2.0-green" />
+  <img alt="Data license" src="https://img.shields.io/badge/data-CC--BY--4.0-green" />
+  <img alt="Status" src="https://img.shields.io/badge/status-beta-orange" />
+</p>
+
+---
+
+FitLab answers the three questions every engineer asks before pulling a local model — and it
+answers them with **regenerated data, not a hand-maintained list**. Every claim on the page is
+produced by a script, validated against a JSON Schema, and merged through a reviewed pull request.
+
+| Lane | Question it answers | Method | Confidence |
+|---|---|---|---|
+| **FITS** | Will this model fit my GPU? | Deterministic VRAM arithmetic from the model's own `config.json` — weights + KV cache + runtime overhead, per quantization and context length | Computed, reproducible |
+| **RUNS** | How fast is it, really? | A real benchmark on a free T4 16 GB GPU every week, plus community-submitted runs on real consumer hardware | Measured, with raw data retained |
+| **PLUGS** | Does it work with my stack? | Weekly end-to-end probes installing the **latest released** Ollama, [OllaBridge](https://github.com/ruslanmv/ollabridge), LangGraph, CrewAI, Langflow, Langfuse and DeepAgents, then issuing a genuine request through the whole chain | Verified, with exact versions recorded |
+
+**Source of truth:** [`data/registry.json`](data/) · **Design rationale:** [`ARCHITECTURE.md`](ARCHITECTURE.md)
+
+## Table of contents
+
+- [Why FitLab exists](#why-fitlab-exists)
+- [Installation](#installation)
+- [Quick start](#quick-start)
+- [CLI reference](#cli-reference)
+- [Configuration](#configuration)
+- [Running FitLab inside an organization](#running-fitlab-inside-an-organization)
+- [Data contract](#data-contract)
+- [How the numbers are produced](#how-the-numbers-are-produced)
+- [Automation](#automation)
+- [Development](#development)
+- [Contributing](#contributing)
+- [Security](#security)
+- [Support and versioning](#support-and-versioning)
+- [Related projects](#related-projects)
+- [License](#license)
+
+## Why FitLab exists
+
+Curated "best model for X GB" lists go stale within a quarter. FitLab is built so that the
+**data outlives any individual model**:
+
+1. **Nothing is hand-ranked.** A weekly job pulls the leading models per Hugging Face pipeline tag
+   through the public API, runs each one through the FITS engine, scores them with a versioned
+   formula, and opens a review PR. Humans review; automation refreshes.
+2. **Claims are tested, not copied.** One model per week receives a real benchmark on a free
+   T4 16 GB — the same GPU class the Colab free tier allocates — so the numbers transfer directly.
+3. **The community is the sensor network.** Anyone can run the
+   [one-click Colab notebook](colab/benchmark_colab.ipynb) or the local benchmark script; the result
+   is schema-validated and sanity-bounded by CI before it is merged, with attribution.
+4. **Writes are gated.** Every JSON is validated against [`data/schema/`](data/schema/), and all
+   automation writes through pull requests — never directly to the default branch.
+5. **Compatibility is re-verified, not assumed.** A green PLUGS cell means the integration worked
+   *this week*, at version pins recorded alongside the result.
+
+## Installation
+
+**Requirements:** Python 3.10 or newer. [Ollama](https://ollama.com) is required only for the
+`bench` command and the wizard's benchmarking step — fit checks need no GPU and no runtime.
+
+FitLab is published to PyPI from tagged GitHub releases via
+[Trusted Publishing](https://docs.pypi.org/trusted-publishers/):
+
+```bash
+pip install fitlab
+```
+
+Until the first release lands on PyPI, install from source. The Makefile bootstraps
+[uv](https://docs.astral.sh/uv/), creates `./.venv` and installs everything:
+
+```bash
+git clone https://github.com/ruslanmv/fitlab && cd fitlab
+make install
+make run
+```
+
+Or without make, using any tool that reads `pyproject.toml`:
+
+```bash
+pip install .                     # or: uv sync
+fitlab --version
+```
+
+The model list refreshes itself at runtime: the CLI fetches the weekly `registry.json`, caches it
+for 24 hours, and falls back to a bundled seed when offline. Upgrading the package is not required
+to see new models.
+
+## Quick start
+
+Run the wizard with no arguments — it detects your hardware, shows what fits, and offers to
+benchmark your selection:
+
+```bash
+fitlab
+```
+
+```text
+// llm fitlab — fit-check console · v0.1.0
+your rig: NVIDIA GeForce RTX 3060 · 12.0 GB VRAM · ~360 GB/s
+registry 2026-08-24 · 22 models · live (weekly)
+
+  1 colab-free — Top 10 — Colab free / Kaggle (T4 16 GB)
+  2 local-12gb — Top 10 — local inference, low-end GPU (12 GB)
+  3 multimodal — Top multimodal (vision-language) for 12–16 GB
+  4 all        — everything, deduplicated
+? Category 1-4 (1)
+```
+
+Or check a specific GPU profile without running anything:
+
+```bash
+fitlab check --gpu rtx3060-12 --category local-12gb --limit 6
+```
+
+```text
+                  fit check · RTX 3060 12GB · Q4_K_M · 8K ctx
+┏━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
+┃ # ┃ model             ┃ params ┃ est GB ┃ verdict ┃ ~tok/s ┃ ollama tag      ┃
+┡━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
+│ 1 │ Qwen/Qwen3-8B     │ 8.2B   │ 7.2    │ FITS    │ ~44.8  │ qwen3:8b        │
+│ 2 │ ibm-granite/gran… │ 8.1B   │ 7.3    │ FITS    │ ~45.4  │ granite3.3:8b   │
+│ 3 │ meta-llama/Llama… │ 8.0B   │ 7.0    │ FITS    │ ~45.9  │ llama3.1:8b     │
+│ 4 │ mistralai/Minist… │ 8.0B   │ 7.1    │ FITS    │ ~45.9  │ ministral-8b    │
+│ 5 │ Qwen/Qwen3-14B    │ 14.8B  │ 11.5   │ TIGHT   │ ~24.8  │ qwen3:14b       │
+│ 6 │ deepseek-ai/Deep… │ 14.8B  │ 11.8   │ TIGHT   │ ~24.8  │ deepseek-r1:14b │
+└───┴───────────────────┴────────┴────────┴─────────┴────────┴─────────────────┘
+```
+
+After a benchmark, the CLI prints a prefilled submission link so your result can join the public
+dataset. Submitting is entirely optional; nothing is transmitted automatically.
+
+## CLI reference
+
+| Command | Purpose |
+|---|---|
+| `fitlab` | Interactive wizard: detect → fit table → select → benchmark |
+| `fitlab detect` | Print detected hardware (NVIDIA via `nvidia-smi`, Apple Silicon via `sysctl`, CPU fallback) |
+| `fitlab check` | Fit table for a GPU profile. No model download, no GPU required |
+| `fitlab bench` | Benchmark one model non-interactively; writes a schema-valid JSON result |
+| `fitlab update` | Force a registry refresh, bypassing the 24-hour cache |
+| `fitlab --version` | Print the installed version |
+
+**`fitlab check`**
+
+| Flag | Default | Values |
+|---|---|---|
+| `--gpu` | `auto` | `auto`, `rtx3060-12`, `t4-16`, `rtx4070-12`, `rtx4060ti-16`, `apple-m-16`, `cpu-only` |
+| `--quant` | `Q4_K_M` | `Q2_K`, `Q3_K_M`, `Q4_0`, `Q4_K_M`, `Q5_K_M`, `Q6_K`, `Q8_0`, `MXFP4`, `F16` |
+| `--ctx` | `8192` | `4096`, `8192`, `16384`, `32768` |
+| `--category` | `all` | A registry category id, or `all` |
+| `--limit` | unset | Maximum rows to print |
+
+**`fitlab bench`**
+
+| Flag | Default | Notes |
+|---|---|---|
+| `--model` | *required* | Ollama tag (`qwen3:8b`) or registry slug (`qwen3-8b`) |
+| `--reps` | `3` | Repetitions; `gen_tps` is reported as the median |
+| `--context` | `4096` | Context length for the run |
+| `--out` | `fitlab-results` | Output directory for the result JSON |
+| `--source` | `community-local` | `ci-kaggle`, `ci-actions-cpu`, `community-colab`, `community-local`, `maintainer` |
+| `--submitter` | empty | GitHub handle recorded for attribution |
+| `--dry-run` | off | Print the fit verdict and execution plan without running |
+
+Benchmarks are seeded and fixed-length (`fitlab-v1-256` prompt, `num_predict 256`, `seed 42`) so
+runs are comparable across machines and across time.
+
+## Configuration
+
+All configuration is by environment variable — no config file, no implicit state beyond the cache.
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `FITLAB_REGISTRY_URL` | Public `registry.json` on the default branch | Point the CLI at a different registry — an internal mirror, a pinned commit, or a `file://` path |
+| `FITLAB_CACHE` | `~/.cache/fitlab` | Cache directory for the fetched registry (24-hour TTL) |
+| `FITLAB_REPO` | `OWNER/llm-fitlab` | Repository used to build benchmark submission links |
+| `OLLABRIDGE_URL` | `http://127.0.0.1:11435/v1` | Gateway endpoint used by the PLUGS compatibility probes |
+| `KAGGLE_USERNAME`, `KAGGLE_KEY` | unset | Required only by the weekly GPU benchmark workflow |
+
+Network behaviour is deliberately narrow and predictable:
+
+- `check`, `detect` and the wizard's fit stage perform **one** HTTPS GET for the registry.
+- Nothing is uploaded. Benchmark results are written to local disk; sharing is a manual step.
+- With no network, the CLI degrades in a defined order: fresh cache → stale cache → bundled seed,
+  and the active source is printed on every run.
+
+## Running FitLab inside an organization
+
+**Internal registry mirror.** Fit verdicts and rankings are just data. To decouple from the public
+repository — for change control, or to publish an approved-model list — host your own
+`registry.json` and point clients at it:
+
+```bash
+export FITLAB_REGISTRY_URL="https://models.internal.example.com/fitlab/registry.json"
+fitlab check --gpu rtx4060ti-16
+```
+
+**Pinned, reproducible data.** Because every registry carries `schema_version`, `ranking_version`
+and `generated_at`, you can pin clients to a specific commit and get byte-identical results:
+
+```bash
+export FITLAB_REGISTRY_URL="https://raw.githubusercontent.com/ruslanmv/fitlab/<commit-sha>/data/registry.json"
+```
+
+**Air-gapped use.** Ship a registry file with your image and reference it directly; no egress is
+required for fit checks:
+
+```bash
+export FITLAB_REGISTRY_URL="file:///opt/fitlab/registry.json"
+```
+
+**Custom hardware fleet.** Add your GPUs to [`data/hardware.yaml`](data/hardware.yaml) — including
+capability flags such as `bf16`, `flash_attn2` and `marlin` — and regenerate the registry to get a
+fit matrix for your own fleet rather than the public reference profiles.
+
+**Extending the compatibility matrix.** Add a component to [`data/stack.yaml`](data/stack.yaml)
+with a `probe` that exercises it end to end, and it will be verified on the same weekly cadence as
+the rest of the stack.
+
+## Data contract
+
+Everything downstream depends on `data/`, and only on `data/`. The site is a renderer; the CLI is a
+client. Both read the same documents.
+
+```
+data/
+  registry.json              generated — the published source of truth (do not hand-edit)
+  models.seed.yaml           human-curated seeds and editorial notes
+  hardware.yaml              GPU profiles: VRAM, bandwidth, architecture capability flags
+  stack.yaml                 ecosystem components and version-discovery rules
+  benchmarks/                append-only measured results, one JSON per run
+  schema/
+    model.schema.json        registry model entries
+    benchmark.schema.json    benchmark result submissions
+```
+
+`registry.json` top-level keys: `schema_version`, `generated_at`, `ranking_version`, `models`,
+`categories`, `benchmarks_latest`, `compat`.
+
+Stability guarantees:
+
+- Every document carries `schema_version`, so historical data stays parseable indefinitely.
+- `data/benchmarks/` is append-only; results are never rewritten or deleted.
+- Models withdrawn from Hugging Face are marked `stale` and hidden after 60 days — never removed.
+- Fit values are labeled `estimated` until a real benchmark upgrades them to `measured`.
+
+## How the numbers are produced
+
+**FITS** is arithmetic, not opinion:
+
+```
+est_vram = weights(params × bpw / 8 × 1.03)
+         + kv_cache(2 × layers × kv_heads × head_dim × context × 2 bytes)
+         + runtime_overhead
+```
+
+Verdicts are thresholds against the profile's VRAM: `fits` at ≤ 92%, `tight` at ≤ 100%,
+`offload` at ≤ 160% (partial GPU offload viable), otherwise `no`. Mixture-of-experts models are
+sized by *total* parameters for memory and by *active* parameters for speed; the registry stores
+both.
+
+**RUNS** measures load time, time-to-first-token, generation tokens/second and peak VRAM through
+Ollama's `/api/generate` timing fields — chosen because Ollama is what the target audience actually
+runs. The weekly rotation is `week_number % queue_length`, so every model is measured and
+periodically re-measured.
+
+**Ranking** is a transparent, versioned scored sort (currently `ranking_version` 1.1):
+
+```
+score = 0.35·fit + 0.25·hf_momentum + 0.25·speed + 0.15·plugs
+```
+
+where `speed` uses a measured value when one exists and a bandwidth-model estimate discounted by
+0.5× otherwise. Changing the weights requires a version bump, making every rank shift visible in
+the pull request diff.
+
+## Automation
+
+| Workflow | Schedule | What it does |
+|---|---|---|
+| [`weekly-sync`](.github/workflows/weekly-sync.yml) | Wed 04:41 UTC | Refreshes the registry from Hugging Face, re-runs the FITS engine, validates against the schema, opens a review PR |
+| [`weekly-benchmark`](.github/workflows/weekly-benchmark.yml) | Mon 03:17 UTC | Benchmarks the rotation model on a free T4 via the Kaggle Kernels API, with a CPU lane as fallback, and runs the PLUGS probes |
+| [`community-submission`](.github/workflows/community-submission.yml) | On labeled issues | Extracts, schema-validates and sanity-bounds a submitted result, then merges it via PR with attribution |
+| [`publish-pypi`](.github/workflows/publish-pypi.yml) | On release | Builds, smoke-tests and publishes the package using PyPI Trusted Publishing (OIDC, no long-lived token) |
+
+Operational properties: workflows use `concurrency` groups to prevent overlapping writes, every
+script is idempotent and safe to re-run, and degradation is graceful — if Hugging Face is
+unreachable the sync PR is simply not opened and the last-good registry stands; if Kaggle is
+unavailable the CPU lane still lands a datapoint.
+
+## Development
+
+The repository is driven by a Makefile backed by [uv](https://docs.astral.sh/uv/). From a clean
+clone, two commands give you a working environment and a running CLI — uv is bootstrapped
+automatically if it is not already installed:
+
+```bash
+git clone https://github.com/ruslanmv/fitlab && cd fitlab
+make install          # creates ./.venv and installs fitlab + all tooling
+make run              # launches the fit-check wizard
+```
+
+`make run` depends on `make install`, so `make run` alone is enough on a fresh clone. Run
+`make help` to list every target.
+
+| Target | What it does |
+|---|---|
+| `make install` | Create `./.venv` and install the package plus the `dev` extra (`uv sync --extra dev`) |
+| `make run` | Launch the interactive wizard |
+| `make detect` / `make check` / `make bench` / `make update` | The corresponding `fitlab` subcommands |
+| `make registry` | Rebuild `data/registry.json` from Hugging Face (FITS + sync) |
+| `make estimate` | Full FITS verdict matrix across seed models × reference GPUs × quantizations |
+| `make probe` | Run the PLUGS end-to-end stack probes |
+| `make validate` | Schema-validate every document in `data/` |
+| `make lint` / `make format` | Ruff check / safe auto-fix |
+| `make smoke` | Exercise the CLI end to end without downloading a model |
+| `make test` | `lint` + `validate` + `smoke` |
+| `make build` | Build sdist and wheel into `dist/`, then `twine check` them |
+| `make site` | Serve the static site on `http://127.0.0.1:8000` |
+| `make clean` / `make distclean` | Remove artifacts / also remove `./.venv` |
+
+Targets accept overrides on the command line:
+
+```bash
+make check GPU=t4-16 CATEGORY=colab-free LIMIT=5
+make bench MODEL=qwen3:14b REPS=5
+```
+
+Available variables: `GPU`, `QUANT`, `CTX`, `CATEGORY`, `LIMIT`, `MODEL`, `REPS`, `OUT`.
+
+In a checkout, the Makefile points `FITLAB_REGISTRY_URL` at the repository's own
+`data/registry.json`, so every target reads the registry you are editing and works offline.
+Override it to exercise a remote fetch:
+
+```bash
+make run FITLAB_REGISTRY_URL=https://example.com/fitlab/registry.json
+```
+
+Dependency groups live in `pyproject.toml`: the `maintainer` extra carries what the `scripts/`
+engines need (`pyyaml`, `jsonschema`), and `dev` adds `ruff`, `build` and `twine` on top. `uv.lock`
+is committed, so `make install` resolves to identical versions on every machine.
+
+The engines can also be invoked directly, without make:
+
+```bash
+# FITS — verdict for one model on one GPU
+python scripts/estimate_vram.py --model Qwen/Qwen3-14B --gpu rtx3060-12 --quant Q4_K_M
+
+# Registry refresh from Hugging Face (writes data/registry.json locally)
+python scripts/sync_hf.py
+
+# RUNS — benchmark on whatever hardware you are on (local, Colab, or Kaggle)
+python scripts/bench_gguf.py --model qwen3:8b --out data/benchmarks/
+
+# PLUGS — end-to-end stack probes (CPU is sufficient)
+python scripts/compat_probe.py --report data/compat_report.json
+
+# Schema validation
+python scripts/validate_data.py
+```
+
+`compat_probe.py` and `validate_data.py` both exit with the number of failures, so CI fails
+honestly rather than reporting a green matrix built on skipped probes.
+
+There is no unit-test suite yet; `make test` runs linting, schema validation and a CLI smoke test.
+
+The published site is static and reads `data/*.json` directly; serve `site/` through GitHub Pages
+or any static host.
+
+## Contributing
+
+Benchmark results are the most valuable contribution and take about three minutes on free
+infrastructure:
+
+1. Open [`colab/benchmark_colab.ipynb`](colab/benchmark_colab.ipynb) in Colab, select a T4 runtime,
+   and run all cells — or run `python scripts/bench_gguf.py --model <tag> --source community-local`
+   on your own machine.
+2. The final cell prints your result and a link that opens a prefilled GitHub issue.
+3. CI validates the JSON and merges it; your GPU and handle appear on the page.
+
+Model suggestions belong in `data/models.seed.yaml` only — never in the generated `registry.json`.
+Ranks are computed rather than negotiated: to change a rank, change the formula in
+`scripts/sync_hf.py` and bump `ranking_version`. Full guidelines are in
+[`CONTRIBUTING.md`](CONTRIBUTING.md).
+
+## Security
+
+- **Community submissions are data, never code.** Submitted JSON is parsed and validated; it is
+  never executed.
+- **Untrusted code paths are isolated.** The weekly stack probes install unpinned latest releases
+  and run inside ephemeral CI runners, optionally under
+  [MatrixLab](https://github.com/agent-matrix/matrixlab) Docker isolation for local maintainer runs
+  (see [`docker-compose.matrixlab.yml`](docker-compose.matrixlab.yml)).
+- **Secrets never reach fork-triggered workflows.** `pull_request_target` is deliberately avoided.
+- **Publishing uses OIDC.** PyPI Trusted Publishing means no long-lived API token exists to leak.
+
+Please report suspected vulnerabilities privately through
+[GitHub Security Advisories](https://github.com/ruslanmv/fitlab/security/advisories/new) rather
+than in a public issue.
+
+## Support and versioning
+
+This is a community open-source project maintained on a best-effort basis; no service level is
+implied. Questions and bug reports belong in
+[GitHub Issues](https://github.com/ruslanmv/fitlab/issues).
+
+Three version numbers move independently, and only the first is a package concern:
+
+- **Package version** (`fitlab`) follows semantic versioning.
+- **`schema_version`** changes only for breaking data-format changes; consumers should pin it.
+- **`ranking_version`** changes whenever scoring weights change, so ranking shifts are auditable.
+
+Benchmark results are point-in-time measurements on specific hardware and driver versions. They are
+a comparative signal, not a guarantee of what your machine will do.
+
+## Related projects
+
+- [OllaBridge](https://github.com/ruslanmv/ollabridge) — one OpenAI-compatible URL for all your local models
+- [HomePilot](https://github.com/ruslanmv/HomePilot) — local-first chat, image and video studio
+- [GitPilot](https://github.com/ruslanmv/gitpilot) — Git workflow automation
+- [MatrixLab](https://github.com/agent-matrix/matrixlab) — Docker isolation sandbox used for untrusted probes
+- [Best-of-the-Best](https://github.com/ruslanmv/Best-of-the-Best) — the auto-curation pattern this repository generalizes
+
+## License
+
+**Code** is licensed under [Apache-2.0](LICENSE). **Data** in `data/` and all published registries
+are licensed under [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/).
+
+Benchmark results are point-in-time measurements. Models referenced remain the property of their
+respective authors and are subject to their own licenses.