horde-engine 2.22.2__tar.gz → 3.1.0__tar.gz

{horde_engine-2.22.2 → horde_engine-3.1.0}/.gitignore

@@ -83,7 +83,7 @@ profile_default/
 ipython_config.py
 
 # pyenv
-.python-version
+# .python-version # this competes the with uv convention for specifying the python version
 
 # pipenv
 #   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.

{horde_engine-2.22.2 → horde_engine-3.1.0}/PKG-INFO

@@ -1,7 +1,10 @@
 Metadata-Version: 2.4
 Name: horde-engine
-Version: 2.22.2
+Version: 3.1.0
 Summary: A wrapper around ComfyUI to allow use by AI Horde.
+Project-URL: Homepage, https://github.com/Haidra-Org/hordelib
+Project-URL: Bug Tracker, https://github.com/Haidra-Org/hordelib/issues
+Project-URL: Changelog, https://github.com/Haidra-Org/hordelib/releases
 Author-email: tazlin <tazlin.on.github@gmail.com>, db0 <mail@dbzer0.com>, Jug <jugdev@proton.me>
 License:                     GNU AFFERO GENERAL PUBLIC LICENSE
                                Version 3, 19 November 2007
@@ -664,78 +667,87 @@ License:                     GNU AFFERO GENERAL PUBLIC LICENSE
         if any, to sign a "copyright disclaimer" for the program, if necessary.
         For more information on this, and how to apply and follow the GNU AGPL, see
         <https://www.gnu.org/licenses/>.
-        
-Project-URL: Homepage, https://github.com/Haidra-Org/hordelib
-Project-URL: Bug Tracker, https://github.com/Haidra-Org/hordelib/issues
-Project-URL: Changelog, https://github.com/Haidra-Org/hordelib/blob/releases/CHANGELOG.md
-Classifier: Programming Language :: Python :: 3
+License-File: LICENSE
+Classifier: Development Status :: 5 - Production/Stable
 Classifier: License :: OSI Approved :: GNU Affero General Public License v3
 Classifier: Operating System :: OS Independent
-Classifier: Development Status :: 5 - Production/Stable
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-License-File: CHANGELOG.md
-Requires-Dist: horde_sdk>=0.15.0
-Requires-Dist: horde_model_reference>=0.9.1
-Requires-Dist: numpy==1.26.4
-Requires-Dist: torch>=2.4.1
-Requires-Dist: torchvision
-Requires-Dist: torchdiffeq
-Requires-Dist: torchsde
-Requires-Dist: einops
-Requires-Dist: open-clip-torch
-Requires-Dist: transformers>=4.37.2
-Requires-Dist: tokenizers>=0.13.3
-Requires-Dist: sentencepiece
-Requires-Dist: safetensors>=0.4.2
-Requires-Dist: pytorch_lightning
-Requires-Dist: pynvml
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.12
+Requires-Dist: accelerate>=1.11.0
+Requires-Dist: addict>=2.4.0
 Requires-Dist: aiohttp>=3.11.8
-Requires-Dist: accelerate
-Requires-Dist: pyyaml
-Requires-Dist: pillow
-Requires-Dist: loguru
-Requires-Dist: GitPython
-Requires-Dist: clip-anytorch
+Requires-Dist: alembic
+Requires-Dist: av>=16.0.0
+Requires-Dist: blake3
+Requires-Dist: clip-anytorch>=2.6.0
+Requires-Dist: comfy-aimdo==0.4.9
+Requires-Dist: comfy-kitchen==0.2.10
+Requires-Dist: comfyui-embedded-docs==0.5.3
+Requires-Dist: comfyui-frontend-package==1.45.15
+Requires-Dist: comfyui-workflow-templates==0.9.98
 Requires-Dist: diffusers>=0.25.0
-Requires-Dist: omegaconf
+Requires-Dist: distro>=1.9.0
+Requires-Dist: einops
+Requires-Dist: fairscale>=0.4.13
+Requires-Dist: filelock
+Requires-Dist: fuzzywuzzy>=0.18.0
+Requires-Dist: gitpython>=3.1.45
+Requires-Dist: horde-model-reference<6.0.0,>=5.1.1
+Requires-Dist: horde-sdk>=0.20.1
+Requires-Dist: kornia>=0.7.1
+Requires-Dist: logfire[system-metrics]==4.14.2
+Requires-Dist: loguru>=0.7.3
+Requires-Dist: lpips>=0.1.4
+Requires-Dist: matplotlib>=3.10.9
+Requires-Dist: numpy>=1.25.0
+Requires-Dist: omegaconf>=2.3.0
+Requires-Dist: open-clip-torch>=2.32.0
+Requires-Dist: opencv-contrib-python>=4.11.0.86
+Requires-Dist: opencv-python>=4.11.0.86
+Requires-Dist: opentelemetry-instrumentation-aiohttp-client>=0.59b0
+Requires-Dist: opentelemetry-instrumentation-requests>=0.59b0
+Requires-Dist: pillow
 Requires-Dist: psutil
-Requires-Dist: typing-extensions
-Requires-Dist: distro
-Requires-Dist: python-dotenv
-Requires-Dist: strenum
-Requires-Dist: rembg[cpu]
-Requires-Dist: opencv-python
-Requires-Dist: opencv-contrib-python
-Requires-Dist: opencv-python
-Requires-Dist: timm==0.9.16
-Requires-Dist: torchvision
-Requires-Dist: scipy
-Requires-Dist: addict
-Requires-Dist: fairscale
-Requires-Dist: scikit-image
-Requires-Dist: mediapipe==0.10.21
-Requires-Dist: unidecode
-Requires-Dist: fuzzywuzzy
-Requires-Dist: strenum
-Requires-Dist: kornia
-Requires-Dist: qrcode==7.4.2
-Requires-Dist: spandrel
-Requires-Dist: spandrel_extra_arches
-Requires-Dist: soundfile
-Requires-Dist: lpips
-Requires-Dist: av>=14.2.0
-Requires-Dist: pydantic~=2.0
 Requires-Dist: pydantic-settings~=2.0
-Requires-Dist: SQLAlchemy
-Requires-Dist: alembic
+Requires-Dist: pydantic~=2.0
+Requires-Dist: pynvml>=13.0.1
+Requires-Dist: python-dotenv>=1.2.1
+Requires-Dist: pytorch-lightning>=2.5.6
+Requires-Dist: pyyaml
+Requires-Dist: qrcode==7.4.2
+Requires-Dist: rembg[cpu]>=2.0.67
+Requires-Dist: requests
+Requires-Dist: safetensors>=0.4.2
+Requires-Dist: scikit-image>=0.25.2
+Requires-Dist: scipy
+Requires-Dist: sentencepiece
+Requires-Dist: simpleeval>=1.0.0
+Requires-Dist: soundfile>=0.13.1
+Requires-Dist: spandrel-extra-arches>=0.2.0
+Requires-Dist: spandrel>=0.4.1
+Requires-Dist: sqlalchemy>=2.0.0
+Requires-Dist: strenum>=0.4.15
+Requires-Dist: timm==0.9.16
+Requires-Dist: tokenizers>=0.13.3
+Requires-Dist: torchaudio<2.13,>=2.9.0
+Requires-Dist: torchdiffeq>=0.2.5
+Requires-Dist: torchsde
+Requires-Dist: torchvision<0.28,>=0.24.0
 Requires-Dist: tqdm
+Requires-Dist: transformers>=4.50.3
+Requires-Dist: typing-extensions>=4.15.0
+Requires-Dist: unidecode>=1.4.0
+Requires-Dist: yapf>=0.43.0
 Requires-Dist: yarl>=1.18.0
-Requires-Dist: comfyui-frontend-package==1.25.11
-Requires-Dist: comfyui-workflow-templates==0.1.75
-Requires-Dist: comfyui-embedded-docs==0.2.6
-Dynamic: license-file
+Provides-Extra: cpu
+Requires-Dist: torch; extra == 'cpu'
+Provides-Extra: cu126
+Requires-Dist: torch; extra == 'cu126'
+Provides-Extra: cu130
+Requires-Dist: torch; extra == 'cu130'
+Provides-Extra: cu132
+Requires-Dist: torch; extra == 'cu132'
+Description-Content-Type: text/markdown
 
 # horde-engine
 
@@ -749,268 +761,283 @@ Dynamic: license-file
 [![All Models][all-model-images]][all-model-url]
 [![Release Changelog][changelog-image]][changelog-url]
 
-> Note: This project was formerly known as `hordelib`. The project namespace will be changed in the near future to reflect this change.
-
-`horde-engine` is a wrapper around [ComfyUI](https://github.com/comfyanonymous/ComfyUI) primarily to enable the [AI Horde](https://aihorde.net/) to run inference pipelines designed visually in the ComfyUI GUI.
-
-The developers of `horde-engine` can be found in the AI Horde Discord server: [https://discord.gg/3DxrhksKzn](https://discord.gg/3DxrhksKzn)
-
-Note that `horde-engine` (previously known as `hordelib`) has been the default inference backend library of the [AI Horde](https://aihorde.net/) since `hordelib` v1.0.0.
-
-## Purpose
-
-The goal here is to be able to design inference pipelines in the excellent ComfyUI, and then call those inference pipelines programmatically. Whilst providing features that maintain compatibility with the existing horde implementation.
-
-## Installation
-
-If being installed from pypi, use a requirements file of the form:
-```
---extra-index-url https://download.pytorch.org/whl/cu121
-hordelib
+`horde-engine` wraps a pinned commit of [ComfyUI](https://github.com/comfyanonymous/ComfyUI) so the [AI Horde](https://aihorde.net/) can run ComfyUI-designed inference pipelines programmatically. It is the default inference backend for AI Horde since v1.0.0.
 
-...your other dependencies...
-```
+Python `>=3.12`. GPU/CUDA required. The package name on PyPI is `horde-engine`; the Python module is `hordelib`.
 
-#### Linux Installation
+Community: [AI Horde Discord](https://discord.gg/3DxrhksKzn).
 
-On Linux you will need to install the Nvidia CUDA Toolkit. Linux installers are provided by Nvidia at https://developer.nvidia.com/cuda-downloads
+---
 
-Note if you only have 16GB of RAM and a default /tmp on tmpfs, you will likely need to increase the size of your temporary space to install the CUDA Toolkit or it may fail to extract the archive. One way to do that is just before installing the CUDA Toolkit:
-```bash
-sudo mount -o remount,size=16G /tmp
-```
-If you only have 16GB of RAM you will also need swap space. So if you typically run without swap, add some. You won't be able to run this library without it.
+## Quick start
 
-## Usage
+1. Install `horde-engine` together with a PyTorch build for your hardware (see
+   [Choosing a PyTorch build](#choosing-a-pytorch-build)). For NVIDIA:
+   ```
+   # CUDA 13+ driver -> torch 2.12.0 (the default line):
+   pip install --extra-index-url https://download.pytorch.org/whl/cu130 horde-engine
+   # CUDA 12.x driver -> the newest CUDA 12.8 build is torch 2.11.0:
+   pip install --extra-index-url https://download.pytorch.org/whl/cu128 horde-engine "torch==2.11.0"
+   ```
 
-Horde payloads can be processed simply with (for example):
+2. Set `AIWORKER_CACHE_HOME` to a model directory (see [Model directory layout](#model-directory-layout)).
 
-```python
-# import os
-# Wherever your models are
-# os.environ["AIWORKER_CACHE_HOME"] = "f:/ai/models" # Defaults to `models/` in the current working directory
+3. Call `hordelib.initialise()` once per process, then use the public API:
+   ```python
+   import hordelib
+   hordelib.initialise()
 
-import hordelib
+   from hordelib.api import HordeLib, ImageGenPayload
 
-hordelib.initialise()  # This must be called before any other hordelib functions
+   horde = HordeLib()
+   payload = ImageGenPayload(
+       prompt="an ancient llamia monster",
+       model="Deliberate",
+       sampler_name="k_dpmpp_2m",
+       cfg_scale=7.5,
+       ddim_steps=25,
+       width=512,
+       height=512,
+       seed=123456789,
+   )
+   result = horde.basic_inference_single_image(payload)
+   result.image.save("test.png")
+   ```
 
-from hordelib.horde import HordeLib
-from hordelib.shared_model_manager import SharedModelManager
+`hordelib.initialise()` wipes `sys.argv` -- parse command-line arguments first. It also forbids dev-mode paths containing spaces (do not clone the repo to a path with spaces) and raises if you try to import any ComfyUI-touching code before calling it.
 
-generate = HordeLib()
+For more usage patterns, see the [public API docs](docs/public_api.md) and the example scripts in `examples/`. For complete API surface reference, consult the test suite under `tests/`.
 
-if SharedModelManager.manager.compvis is None:
-    raise Exception("Failed to load compvis model manager")
+---
 
-SharedModelManager.manager.compvis.download_model("Deliberate")
-SharedModelManager.manager.compvis.validate_model("Deliberate")
+## Installation
 
+Requires an NVIDIA GPU with CUDA. On Linux, install the [NVIDIA CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) first. Systems with 16 GB RAM or less need swap space and may need an enlarged `/tmp`:
+```bash
+sudo mount -o remount,size=16G /tmp
+```
 
-data = {
-    "sampler_name": "k_dpmpp_2m",
-    "cfg_scale": 7.5,
-    "denoising_strength": 1.0,
-    "seed": 123456789,
-    "height": 512,
-    "width": 512,
-    "karras": False,
-    "tiling": False,
-    "hires_fix": False,
-    "clip_skip": 1,
-    "control_type": None,
-    "image_is_control": False,
-    "return_control_map": False,
-    "prompt": "an ancient llamia monster",
-    "ddim_steps": 25,
-    "n_iter": 1,
-    "model": "Deliberate",
-}
-pil_image = generate.basic_inference_single_image(data).image
+### From PyPI
 
-if pil_image is None:
-    raise Exception("Failed to generate image")
+```
+# CUDA 13+ driver -> torch 2.12.0 (the default line):
+pip install --extra-index-url https://download.pytorch.org/whl/cu130 horde-engine
+# CUDA 12.x driver -> the newest CUDA 12.8 build is torch 2.11.0:
+pip install --extra-index-url https://download.pytorch.org/whl/cu128 horde-engine "torch==2.11.0"
+```
 
-pil_image.save("test.png")
+### From source (development)
 
+```bash
+git clone https://github.com/Haidra-Org/hordelib.git
+cd hordelib
+uv sync   # installs the latest torch (2.12.0, the CUDA 12.6 build) -- see "Choosing a PyTorch build"
 ```
 
-Note that `hordelib.initialise()` will erase all command line arguments from argv. So make sure you parse them before you call that.
-
-See `tests/run_*.py` for more standalone examples.
+### Choosing a PyTorch build
 
-### Logging
+`horde-engine` does not pin `torch`: it declares `torchvision`/`torchaudio` as version *ranges*, and
+torchvision pins the exact matching torch in its metadata, so installing the library pulls the latest
+supported torch (**2.12.0** -> torchvision 0.27.0 / torchaudio 2.11.0; torchaudio has no 2.12 release).
+You only pick the *build* (which CUDA/ROCm wheels) for your hardware.
 
-If you don't want `hordelib` to setup and control the logging configuration (we use [loguru](https://loguru.readthedocs.io/en/stable/)) initialise with:
+For development, `uv sync` installs the **CUDA 12.6** build of torch 2.12.0 (the index hordelib routes
+to in `[tool.uv.sources]`). It runs on any CUDA 12.6+ driver and, via NVIDIA driver
+backward-compatibility, on CUDA 13 drivers too -- the broadest single build. To use a different build,
+override it ad-hoc after syncing:
 
-```python
-import hordelib
-hordelib.initialise(setup_logging=False)
+```bash
+uv sync
+# then, only if you want a build other than the cu126 default:
+UV_TORCH_BACKEND=auto uv pip install torch torchvision torchaudio                # auto-detect your GPU
+uv pip install torch --extra-index-url https://download.pytorch.org/whl/cu130    # exact CUDA 13 build
 ```
 
-## Acknowledgments
-
-`hordelib` depends on a large number of open source projects, and most of these dependencies are automatically downloaded and installed when you install `hordelib`. Due to the nature and purpose of `hordelib` some dependencies are bundled directly _inside_ `hordelib` itself.
+With `pip`, select the index with `--extra-index-url`:
 
-### [ComfyUI](https://github.com/comfyanonymous/ComfyUI)
-A powerful and modular stable diffusion GUI with a graph/nodes interface. Licensed under the terms of the GNU General Public License v3.0.
+| Hardware | index | Notes |
+|---|---|---|
+| NVIDIA, CUDA 13+ driver | `cu130` | torch 2.12.0; a 13.0 build covers any 13.x driver (incl. 13.2). |
+| NVIDIA, CUDA 13.2 (exact) | `cu132` | torch 2.12.0; `cu130` also runs on a 13.2 driver. |
+| NVIDIA, CUDA 12.6+ driver | `cu126` | torch 2.12.0; the only CUDA-12 build of 2.12.0. |
+| NVIDIA, CUDA 12.x, older torch | `cu128` | No 2.12.0 wheel; caps at torch **2.11.0** (add `"torch==2.11.0"`). |
+| AMD (Linux only) | `rocm6.4` | torch **2.9.1 only** (see note below). |
+| No GPU | `cpu` | ~100x slower; testing only. |
 
-The entire purpose of `hordelib` is to access the power of ComfyUI.
+> **ROCm:** only torch 2.9.1 (ROCm 6.4) is currently installable. PyTorch publishes torch 2.10-2.12 on
+> rocm7.x but not the matching `pytorch-triton-rocm` (3.6.0/3.7.0) those wheels hard-depend on, so they
+> cannot be resolved yet. Install it ad-hoc: `uv pip install torch==2.9.1 torchvision torchaudio
+> pytorch-triton-rocm --extra-index-url https://download.pytorch.org/whl/rocm6.4`.
 
-### [Controlnet Preprocessors for ComfyUI](https://github.com/Fannovel16/comfy_controlnet_preprocessors)
-Custom nodes for ComfyUI providing Controlnet preprocessing capability. Licened under the terms of the Apache License 2.0.
+---
 
-### [ComfyUI Face Restore Node](https://civitai.com/models/24690/comfyui-facerestore-node)
+## Architecture (v3)
 
-Custom nodes for ComfyUI providing face restoration.
+The v3 refactor replaced the legacy dict-heavy pipeline machinery with a typed, testable, registry-driven layer. The key pieces:
 
-### [ComfyQR](https://gitlab.com/sofuego-comfy-nodes/ComfyQR)
-
-Nodes for generating QR codes
-
-## Development
+| Layer | Location | Role |
+|---|---|---|
+| **Public API** | `hordelib/api.py` | Declared consumer surface. Workers import from here only. |
+| **Bootstrap** | `hordelib/initialisation.py` | `initialise()` -- manifest installer, path patching, ComfyUI import, monkeypatches. |
+| **Typed pipeline** | `hordelib/pipeline/` | `ImageGenPayload` (pydantic, clamp-dont-reject), `PipelineTemplate`/`ParamBinding`, priority-ordered `PipelineRegistry`, pure graph patch functions, IO/model resolution. All GPU-free and unit-tested. |
+| **Execution bridge** | `hordelib/execution/` | `ExecutionBackend` protocol, in-process ComfyUI backend, VRAM monkeypatches, pure graph utilities. |
+| **Environment manifest** | `hordelib/installation/` | `manifest.json` pins ComfyUI commit + external custom nodes. `EnvironmentInstaller` checks out/clones idempotently. |
+| **ComfyUI bridge** | `hordelib/comfy_horde.py` | `Comfy_Horde` -- the actual ComfyUI process. `do_comfy_import()` applies monkeypatches. |
+| **Model managers** | `hordelib/model_manager/` | Per-category model records (pydantic), download/validate/lookup, LoRA/TI adhoc cache. |
+| **Pipeline JSON** | `hordelib/pipelines/` | Backend-format pipeline files executed at runtime. |
+| **Pipeline designs** | `hordelib/pipeline_designs/` | ComfyUI-app-editable source workflows (convert to `pipelines/`). |
+| **Custom nodes** | `hordelib/nodes/` | First-party Horde nodes (`HordeCheckpointLoader`, `HordeImageOutput`) and vendored compat nodes. External nodes (controlnet aux, QR) are git-pinned in the manifest. |
 
-Requirements:
-- `git` (install git)
-- `tox` (`pip install tox`)
-- Set the environmental variable `AIWORKER_CACHE_HOME` to point to a model directory.
+### Pipeline flow
 
-Note the model directory must currently be in the original AI Horde directory structure:
 ```
-<AIWORKER_CACHE_HOME>\
-    clip\
-    codeformer\
-    compvis\
-        Deliberate.ckpt
-        ...etc...
-    controlnet\
-    embeds\
-    esrgan\
-    gfpgan\
-    safety_checker\
+dict payload --> normalize (horde_compat) --> ImageGenPayload
+  --> resolve_context (model files, loras, TIs)
+  --> registry.select (predicate + priority --> PipelineTemplate)
+  --> materialize (bindings + patch steps --> ComfyGraph)
+  --> backend.run_pipeline --> ResultingImageReturn
 ```
 
-### Running the Tests
-
-Simply execute: `tox` (or `tox -q` for less noisy output)
-
-This will take a while the first time as it installs all the dependencies.
-
-If the tests run successfully images will be produced in the `images/` folder.
-
-#### Running a specific test file
+### Key invariants
 
-`tox -- -k <filename>` for example `tox -- -k test_initialisation`
+- **`initialise()` before everything.** Importing `HordeLib` or any ComfyUI-touching code before `initialise()` raises `RuntimeError`. The test suite enforces this.
+- **Public API only.** Consumers import from `hordelib.api`. Anything not re-exported there is internal and may change without notice. Enforced by `tests/meta/test_public_api_contract.py`.
+- **Clamp, don't reject.** Payload validation coerces bad values to defaults rather than raising -- the Horde contract requires tolerance.
+- **Node titles are the parameter namespace.** A KSampler titled `sampler` produces dotted parameters `sampler.cfg`, `sampler.seed`, etc. Renaming a title silently breaks parameter mapping.
+- **Pipeline selection is registry-driven.** Predicates + explicit priorities replace the old if/elif tree. Adding a new baseline requires one family module, one template, one register call, and one inference test. See [Adding a baseline](docs/adding-a-baseline.md).
 
-#### Running a specific predefined test suite
+---
 
-`tox list`
+## Model directory layout
 
-This will list all groups of tests which are involved in either the development, build or CI proccess. Tests which have the word 'fix' in them will automatically apply changes when run, such as to linting or formatting. You can do this by running:
+`AIWORKER_CACHE_HOME` must follow the AI Horde directory structure:
 
-`tox -e [test_suite_name_here]`
-
-### Directory Structure
-
-`hordelib/pipeline_designs/`
-Contains ComfyUI pipelines in a format that can be opened by the ComfyUI web app. These saved directly from the web app.
-
-`hordelib/pipelines/`
-Contains the above pipeline JSON files converted to the format required by the backend pipeline processor. These are converted from the web app, see _Converting ComfyUI pipelines_ below.
-
-`hordelib/nodes/` These are the custom ComfyUI nodes we use for `hordelib` specific processing.
-
-### Running ComfyUI Web Application
+```
+<AIWORKER_CACHE_HOME>/
+    clip/
+    codeformer/
+    compvis/
+        Deliberate.ckpt
+        ...
+    controlnet/
+    embeds/
+    esrgan/
+    gfpgan/
+    safety_checker/
+```
 
-In this example we install the dependencies in the OS default environment. When using the git version of `hordelib`, from the project root:
+Models are managed through `SharedModelManager` (download, validate, list available). See `examples/download_all_sd_models.py` for bulk downloading.
 
-`pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu118 --upgrade`
+---
 
-Ensure ComfyUI is installed, one way is running the tests:
+## Development
 
-`tox -- -k test_comfy_install`
+Requirements: `git`, [`uv`](https://docs.astral.sh/uv/), `AIWORKER_CACHE_HOME` set to a model directory, and a CUDA GPU.
 
-From then on to run ComfyUI:
+### Build, lint, test
 
-`cd ComfyUI`
-`python main.py`
+The whole toolchain lives in one `uv`-managed `.venv`. Sync it once (installs the CUDA 12.6 build of
+torch 2.12.0 via `[tool.uv.sources]`), then drive tools with `uv run --no-sync`:
 
-Then open a browser at: http://127.0.0.1:8188/
+```bash
+uv sync                          # one-time, and after dependency changes
 
-### Designing ComfyUI Pipelines
+# Full test suite. The import guard runs first in a clean process (it asserts no ComfyUI
+# import happens before initialise()), then the rest of the suite runs:
+uv run --no-sync pytest -x tests/meta -k test_no_initialise_comfy_horde
+uv run --no-sync pytest -x tests --cov --ignore=tests/meta --durations=20
 
-Use the standard ComfyUI web app. Use the "title" attribute to name the nodes, these names become parameter names in the `hordelib`. For example, a KSampler with the "title" of "sampler2" would become a parameter `sampler2.seed`, `sampler2.cfg`, etc. Load the pipeline `hordelib/pipeline_designs/pipeline_stable_diffusion.json` in the ComfyUI web app for an example.
+uv run --no-sync pytest -k <pattern>           # run a subset
+uv run --no-sync pre-commit run --all-files    # lint + format gate (black + ruff + mypy)
+uv build                                       # build sdist + wheel
+```
 
-Save any new pipeline in `hordelib/pipeline_designs` using the naming convention "pipeline_\<name\>.json".
+(Equivalently, activate `.venv` and call `pytest` / `pre-commit` directly.)
 
-Convert the JSON for the model (see _Converting ComfyUI pipelines_ below) and save the resulting JSON in `hordelib/pipelines` using the same filename as the previous JSON file.
+The first test run downloads many multi-GB models and is very slow. Most CI/agent sandboxes
+cannot run the full inference suite; prefer `uv run --no-sync pre-commit run --all-files` for
+quick feedback.
 
-That is all. This can then be called from `hordelib` using the `run_image_pipeline()` method in `hordelib.comfy.Comfy()`
+Tests require a CUDA GPU, `AIWORKER_CACHE_HOME`, `CIVIT_API_TOKEN`, and the committed `images_expected/` reference images. Image-output tests compare generated images against references via cosine/histogram similarity. Set `HORDELIB_SKIP_SIMILARITY_FAIL=1` to downgrade similarity failures to skips on different hardware.
 
-### Converting ComfyUI pipelines
+### Key test areas
 
-In addition to the design file saved from the UI, we need to save the pipeline file in the backend format. This file is created in the `hordelib` project root named `comfy-prompt.json` automatically if you run a pipeline through the ComfyUI version embedded in `hordelib`. Running ComfyUI with `tox -e comfyui` automatically patches ComfyUI so this JSON file is saved.
+| Path | What it covers |
+|---|---|
+| `tests/pipeline/` | Payload models, graph operations, template integrity, registry selection -- all GPU-free |
+| `tests/execution/` | Graph utilities, modality seams -- GPU-free |
+| `tests/meta/` | Public API contract, `initialise()`-before-use guard |
+| `tests/test_horde_inference_*.py` | Full GPU inference with image-similarity oracles |
+| `tests/installation/` | Manifest round-trip, installer idempotency |
 
-### Build Configuration
+### Designing and converting pipelines
 
-The main config files for the project are: `pyproject.toml`, `tox.ini` and `requirements.txt`
+Design pipelines in the ComfyUI web app (`cd ComfyUI && python main.py`, then http://127.0.0.1:8188/). Assign stable `title` attributes to nodes -- these become the dotted parameter names. Save the design in `hordelib/pipeline_designs/` as `pipeline_<name>.json`.
 
-### PyPi Publishing
+Convert to backend format by running the pipeline through the embedded ComfyUI (it auto-saves `comfy-prompt.json`), then save the result in `hordelib/pipelines/` with the same filename.
 
-Pypi publishing is automatic all from the GitHub website.
+Do not hand-edit `pipelines/*.json` if a `pipeline_designs/` source exists.
 
-1. Create a PR from `main` to `releases`
-1. Label the PR with "release:patch" (0.0.1) or "release:minor" (0.1.0)
-1. Merge the PR with a standard merge commit (not squash)
+### Updating ComfyUI or external custom nodes
 
+Change the relevant commit SHA in `hordelib/installation/manifest.json`:
+- `comfyui_ref` for ComfyUI itself
+- `ref` for each external custom node
 
-### Standalone "clean" environment test from Pypi
+Full 40-character SHAs only. The next `hordelib.initialise()` (or `python -m hordelib.installation`) applies the update. Run the full test suite afterwards.
 
-Here's an example:
+### Local wheel build
 
-Start in a new empty directory. Create requirements.txt:
-```
---extra-index-url https://download.pytorch.org/whl/cu118
-hordelib
+```bash
+uv build        # writes dist/horde_engine-<version>.tar.gz and -<version>-py3-none-any.whl
 ```
 
-Create the directory `images/` and copy the `test_db0.jpg` into it.
+The version is derived from git tags by `hatch-vcs`: a clean checkout sitting exactly on a
+`vX.Y.Z` tag builds `X.Y.Z`; anything else builds a dev version (e.g. `1.2.3.devN`). ComfyUI is
+not bundled into the wheel -- it is fetched per `hordelib/installation/manifest.json` on first
+`initialise()`. To smoke-test the wheel away from the source tree (so the repo's `hordelib/`
+cannot shadow the installed package), install it into a throwaway venv created outside the repo.
 
-Copy `run_controlnet.py` from the `hordelib/tests/` directory.
+### Release process
 
-Build a venv:
-```
-python -m venv venv
-.\venv\Scripts\activate
-pip install -r requirements.txt
-```
+Releases are cut by pushing a version tag. The tag **is** the version (`hatch-vcs`), and the
+[release workflow](.github/workflows/release.yml) builds with `uv build` and publishes to PyPI
+via trusted publishing (then creates a GitHub Release for the tag).
 
-Run the test we copied:
+```bash
+git checkout main && git pull
+git tag vX.Y.Z          # must be vX.Y.Z and sit on the commit you want released
+git push origin vX.Y.Z
 ```
-python run_controlnet.py
 
-The `images/` directory should have our test images.
-```
+---
 
-### Creating a local build of hordelib
+## Documentation
 
-This is useful when testing new nodes  via the horde-reGen-worker etc
+| Document | Content |
+|---|---|
+| [docs/public_api.md](docs/public_api.md) | Declared consumer surface and key patterns |
+| [docs/adding-a-baseline.md](docs/adding-a-baseline.md) | Checklist for supporting a new model architecture |
+| [docs/modality-readiness.md](docs/modality-readiness.md) | Audit of audio/video modality seams |
+| [docs/plans/major-overhaul-1.md](docs/plans/major-overhaul-1.md) | v3 phased refactor plan and status |
+| [AGENTS.md](AGENTS.md) | Comprehensive codebase map, conventions, and sensitive areas |
 
-```bash
-python build_helper.py
-python -m build --sdist --wheel --outdir dist/ .
-python build_helper.py --fix
-```
+---
 
-On the venv where you want to install th new version
+## License
 
-```bash
-python -m pip install /path/to/hordelib/dist/horde_engine-*.whl
-```
+[GNU Affero General Public License v3.0](LICENSE)
+
+## Acknowledgments
 
-### Updating the embedded version of ComfyUI
+`horde-engine` builds on a large number of open-source projects:
 
-- Change the value in `consts.py` to the desired ComfyUI version.
-- Run the test suite via `tox`
+- [ComfyUI](https://github.com/comfyanonymous/ComfyUI) (GPLv3) -- the graph/nodes inference engine this project wraps.
+- [comfyui_controlnet_aux](https://github.com/Fannovel16/comfy_controlnet_preprocessors) (Apache 2.0) -- ControlNet preprocessors, installed via the manifest.
+- [ComfyUI-layerdiffuse](https://github.com/huchenlei/ComfyUI-layerdiffuse) -- layer diffusion nodes (vendored with compat fixes).
+- [facerestore_cf](https://civitai.com/models/24690/comfyui-facerestore-node) -- face restoration nodes (vendored with compat fixes).
+- [ComfyQR](https://gitlab.com/sofuego-comfy-nodes/ComfyQR) -- QR code generation, installed via the manifest.
+- [horde-sdk](https://github.com/Haidra-Org/horde-sdk) and [horde-model-reference](https://github.com/Haidra-Org/horde-model-reference) -- payload types and model metadata consumed by this library.
 
 
 
@@ -1030,4 +1057,4 @@ python -m pip install /path/to/hordelib/dist/horde_engine-*.whl
 [pr-test-url]: https://tests.hordelib.org/unstable/index.html
 [all-model-url]: https://tests.hordelib.org/all_models/
 [changelog-image]: https://img.shields.io/badge/Release-Changelog-yellow
-[changelog-url]: https://github.com/Haidra-Org/hordelib/blob/releases/CHANGELOG.md
+[changelog-url]: https://github.com/Haidra-Org/hordelib/releases