inline-core 1.1.1__tar.gz → 1.2.0__tar.gz

{inline_core-1.1.1 → inline_core-1.2.0}/.gitignore

@@ -1,4 +1,4 @@
-# Model weights (gigabytes). The engine's models root — `models_dir()` defaults to ./models, and
+# Model weights (gigabytes). The engine's models root - `models_dir()` defaults to ./models, and
 # runners scan ./models/<category>/ (e.g. diffusion_models/). Leading slash anchors this to the repo
 # root ONLY, so the source package src/inline_core/models/ (code) stays tracked.
 /models/

{inline_core-1.1.1 → inline_core-1.2.0}/CLAUDE.md

@@ -1,8 +1,8 @@
-# Inline Core — Engineering Guide
+# Inline Core - Engineering Guide
 
 Inline Core is the **generation engine behind Inline** (the Storyline / Inline Studio UI client). It
 takes a **typed node graph (JSON)** and returns immutable renders ("takes"), running image and video
-models across macOS, Windows, and Linux — from CPU-only boxes and low-VRAM laptops up to multi-GPU
+models across macOS, Windows, and Linux - from CPU-only boxes and low-VRAM laptops up to multi-GPU
 machines that split a single image's sampling across GPUs (via xDiT). **It is the render backend that
 replaces ComfyUI for Inline.**
 
@@ -19,20 +19,20 @@ replaces ComfyUI for Inline.**
 Graph (typed nodes + edges)  →  Run  →  Take[]  (immutable renders)
 ```
 
-- **Graph** — a JSON DAG of typed nodes. Edges are type-checked (`model`, `vae`, `conditioning`,
+- **Graph** - a JSON DAG of typed nodes. Edges are type-checked (`model`, `vae`, `conditioning`,
   `latent`, media) **before** the run, so a bad graph is rejected at submit (422), never mid-denoise.
-- **Run** — one execution of a target node's upstream closure. Durable (survives a restart) and
+- **Run** - one execution of a target node's upstream closure. Durable (survives a restart) and
   pollable; progress streams over a websocket.
-- **Take** — one immutable output. Regenerating adds a take; **nothing is overwritten** (this mirrors
-  Inline Studio's frame/take model — the take history is the core value Comfy lacks).
-- **Node** — has a **descriptor** (the data half: ports, params, file pickers — served at
+- **Take** - one immutable output. Regenerating adds a take; **nothing is overwritten** (this mirrors
+  Inline Studio's frame/take model - the take history is the core value Comfy lacks).
+- **Node** - has a **descriptor** (the data half: ports, params, file pickers - served at
   `/v1/models`) and a **runner** (the behavior half). A descriptor with no runner is served and
   type-checked but cannot execute yet.
 
 ### The two boundaries that matter most (why this isn't ComfyUI)
 
 1. **Graph orchestration is decoupled from GPU work.** The executor runs cheap orchestration inline
-   and never runs the denoise loop itself — a model runner submits a `SampleJob` through the
+   and never runs the denoise loop itself - a model runner submits a `SampleJob` through the
    **batched-sampler seam** (`sampling/batch.py`). The graph is the unit of caching; the sampler is
    the unit of batching; the multi-GPU split routes through that same seam.
 2. **The device policy is the single owner of placement.** No node or component ever picks a device,
@@ -40,7 +40,7 @@ Graph (typed nodes + edges)  →  Run  →  Take[]  (immutable renders)
    6 GB laptop, pure CPU, or split across several GPUs, without touching the graph.
 
 If you find yourself hardcoding a device in a component, or running a denoise loop inside the
-executor, stop — you're breaking one of the two boundaries the whole design exists to keep.
+executor, stop - you're breaking one of the two boundaries the whole design exists to keep.
 
 ## Architecture
 
@@ -54,53 +54,53 @@ HTTP/WS  →  server/app.py  →  RunManager  →  Executor  →  Registry (desc
                                             SampleJob  →  BatchedSampler (inline | xDiT worker group)
 ```
 
-- **`server/`** — the `/v1` API. `app.py` (routes), `manager.py` (validate → queue → run on a worker
+- **`server/`** - the `/v1` API. `app.py` (routes), `manager.py` (validate → queue → run on a worker
   thread → fan out events), `run_store.py` (SQLite durability), `bootstrap.py` (best-effort model
   registration), `serialize.py` (contract JSON), `assets.py` (content-addressed upload).
-- **`graph/`** — the engine core. `schema.py` (typed `Graph`/`Node`/`Edge` + JSON parser),
+- **`graph/`** - the engine core. `schema.py` (typed `Graph`/`Node`/`Edge` + JSON parser),
   `descriptor.py` (node data half), `runners.py` (node behavior half + source nodes), `registry.py`
   (descriptors + runners), `validate.py` + `topo.py` (type-check + order), `executor.py` (lazy
   closure execution, node cache), `primitives.py` (the low-level node vocabulary), `cache.py`.
-- **`components/`** — the five device-agnostic component interfaces (`TextEncoder`, `Scheduler`,
+- **`components/`** - the five device-agnostic component interfaces (`TextEncoder`, `Scheduler`,
   `Denoiser`, `Sampler`, `VAE`) plus opaque `Conditioning`/`Latents`. Placement comes from the ctx.
-- **`sampling/`** — `batch.py`: the graph/GPU boundary. `SampleJob` → `BatchedSampler`
+- **`sampling/`** - `batch.py`: the graph/GPU boundary. `SampleJob` → `BatchedSampler`
   (`InlineBatchedSampler` today; `XFuserBatchedSampler` routes a parallel placement to the worker
   group). Keep this module torch-free and mockable.
-- **`device/`** — the policy. `policy.py` (interface: `Placement`, `Profile`, quant, attention),
+- **`device/`** - the policy. `policy.py` (interface: `Placement`, `Profile`, quant, attention),
   `memory.py` (`MemoryPolicy`), `detect.py` / `auto.py` (enumerate GPUs, NVLink vs PCIe), `types.py`.
-- **`parallel/`** — the xDiT (xfuser) worker group: one process per GPU via `torchrun`, talking over
+- **`parallel/`** - the xDiT (xfuser) worker group: one process per GPU via `torchrun`, talking over
   local IPC behind the sampler seam. `launch.py`, `worker.py`, `group.py`, `registry.py`, `config.py`,
   `protocol.py`. The HTTP server, DB, and graph stay single-process; only the denoise distributes.
-- **`models/`** — `catalog.py` (scans the models root, feeds `options_from` selects + the registry
+- **`models/`** - `catalog.py` (scans the models root, feeds `options_from` selects + the registry
   version) and the **model-runner subpackages** (e.g. `zimage/`), imported best-effort by
   `server/bootstrap.py` so a torch-less install still boots.
-- **`runtime/`** — `context.py` (`ExecutionContext`, `CancelToken`), `run.py` (`RunState`),
+- **`runtime/`** - `context.py` (`ExecutionContext`, `CancelToken`), `run.py` (`RunState`),
   `progress.py` (events + emitters), `store.py` / `file_store.py` (`TakeStore`: owns take bytes/hash/
   uri).
-- **`importer/`** — `comfy.py`: best-effort ComfyUI-workflow → Inline-Core-graph mapping. All ComfyUI
+- **`importer/`** - `comfy.py`: best-effort ComfyUI-workflow → Inline-Core-graph mapping. All ComfyUI
   format knowledge lives here.
-- **`config.py`** — all env config, small and explicit. **`takes.py`**, **`media.py`**, **`errors.py`**
-  — domain primitives (`Take`/`AssetRef`, `MediaKind`, the error hierarchy).
+- **`config.py`** - all env config, small and explicit. **`takes.py`**, **`media.py`**, **`errors.py`**
+  - domain primitives (`Take`/`AssetRef`, `MediaKind`, the error hierarchy).
 
 ### Node vocabularies (three, and their status)
 
-- **Source nodes** (`graph/runners.py`) — `input/text`, `input/image`. Runners exist; pure, no takes.
+- **Source nodes** (`graph/runners.py`) - `input/text`, `input/image`. Runners exist; pure, no takes.
   These are the closure boundary: the UI feeds curated inputs in as source nodes so nothing upstream
   is recomputed.
-- **Low-level primitives** (`graph/primitives.py`) — `load/diffusion-model`, `load/vae`,
+- **Low-level primitives** (`graph/primitives.py`) - `load/diffusion-model`, `load/vae`,
   `load/text-encoder`, `encode/text`, `latent/empty`, `sample`, `vae/decode`, `vae/encode`. These are
   the ComfyUI-equivalent decomposed graph and the intended long-term surface. **Descriptor-only
-  today — their runners land in C2.** A graph built from them validates and type-checks but raises
+  today - their runners land in C2.** A graph built from them validates and type-checks but raises
   `No runner registered` at execution.
-- **Model runners** (`models/<name>/`) — high-level single generation nodes. **`alibaba/z-image-turbo`
+- **Model runners** (`models/<name>/`) - high-level single generation nodes. **`alibaba/z-image-turbo`
   (`models/zimage/`) is the one runnable generation path today** (prompt + optional image → one take,
   backed by diffusers' `ZImagePipeline`/`ZImageImg2ImgPipeline`). This is the "Z-Image pipeline that
-  already works" — the primitives will reach parity in C2. It loads from a **single diffusion
-  `.safetensors`** (drop one file in `diffusion_models/`, ComfyUI-style — no repo folder to set up):
+  already works" - the primitives will reach parity in C2. It loads from a **single diffusion
+  `.safetensors`** (drop one file in `diffusion_models/`, ComfyUI-style - no repo folder to set up):
   the runner loads the transformer via `from_single_file` and pulls the VAE / text-encoder / tokenizer
   from the reference repo behind the scenes, so the user only ever handles one model file.
 - **Low-level primitives and source nodes are `hidden`** (`NodeDescriptor.hidden`): they are served for
-  validation/execution but never offered in the UI's add-node menu. Generation stays one-click — the
+  validation/execution but never offered in the UI's add-node menu. Generation stays one-click - the
   user sees only high-level model nodes; loaders/VAE/encoders are wired up behind them.
 
 Only media-output nodes (`vae/decode`, a model runner) become Frames with take history; the engine
@@ -109,21 +109,38 @@ between nodes and are never takes.
 
 ### Storage & configuration (all env, see `config.py`)
 
-- **Models root** — `INLINE_MODELS_DIR`, else `./models`. **Bring your own weights; nothing is
+- **Models root** - `INLINE_MODELS_DIR`, else `./models`. **Bring your own weights; nothing is
   downloaded.** ComfyUI-style category subfolders (`diffusion_models/`, `vae/`, `text_encoders/`,
   `loras/`, `controlnet/`, `checkpoints/`, `clip_vision/`, `upscale_models/`, `embeddings/`). The
   catalog scans this on start; a file dropped in bumps the registry version so clients refetch
   `/v1/models`. A model may be a single weight file or a folder (e.g. a diffusers snapshot or a
   sharded text encoder).
-- **Data dir** — `INLINE_DATA_DIR`, else `./.inline`. Engine-owned working data: `runs.db` (durable
+- **Data dir** - `INLINE_DATA_DIR`, else `./.inline`. Engine-owned working data: `runs.db` (durable
   runs) and `takes/` (output bytes).
-- **Server bind** — `INLINE_HOST` (default `127.0.0.1`), `INLINE_PORT` (default `8848`).
-- **Model overrides** — e.g. `INLINE_ZIMAGE_MODEL` (a single `.safetensors` file path, a local
+- **Server bind** - `INLINE_HOST` (default `127.0.0.1`), `INLINE_PORT` (default `8848`).
+- **Model overrides** - e.g. `INLINE_ZIMAGE_MODEL` (a single `.safetensors` file path, a local
   diffusers dir, or a HF repo id for Z-Image). Auto-resolved from `diffusion_models/` when unset.
-- **Memory** — always prefer the GPU: even under the `lowvram` profile, weights stay resident on the
-  GPU (VAE tiling/slicing + attention slicing + int8 do the saving); we do **not** auto-offload to CPU.
-  `INLINE_ALLOW_CPU_OFFLOAD=1` opts back into `enable_model_cpu_offload()` for extreme cases.
-- **Multi-GPU** — `INLINE_PARALLEL` (e.g. `pipefusion=2`, `pipefusion=2,ulysses=2`); degrees multiply
+- **Memory** - by default prefer the GPU, and **auto-fit the model to it**. When a runner hands the
+  policy the model's on-disk sizes (`DevicePolicy.set_footprint`, a `ModelFootprint`), the policy sizes
+  the weights against total VRAM (minus an activation headroom) and picks the lightest plan that fits:
+  full-precision **resident** → else int8 **resident** (torchao halves the transformer + text encoder,
+  no CPU offload - int8 + accelerate's `enable_model_cpu_offload` deadlock together) → else unquantized
+  `SEQUENTIAL` submodule streaming. So a card that can't hold Z-Image full-precision (a T4)
+  **auto-int8s with no flag**; `set_footprint(None)` / an unsizable whole-pipeline folder falls back to
+  the coarse total-VRAM buckets. Capacity is TOTAL VRAM (a fixed device property), not live-free, so the
+  plan - and the pipeline cache key it feeds - is stable across runs. The runner does a **pre-flight
+  check** (`DevicePolicy.fit_estimate`): a model too big for VRAM+RAM fails with a clean node error
+  before any load, instead of a host-RAM OOM-kill that would take the shared server down. Weights stream
+  straight to the GPU on load (`device=`/`device_map` in `models/loaders.py`) so peak host RAM ≈ one
+  tensor, and switching checkpoints **evicts** the previous model (`loaders.unload_components`,
+  `_evict_stale`) rather than stacking. `webui.sh` always sets `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`
+  to cut fragmentation OOMs. **Smart memory** (`INLINE_SMART_MEMORY=1`, `--smart-memory`) and
+  `INLINE_ALLOW_CPU_OFFLOAD=1` remain as explicit overrides when no footprint is set.
+- **Compute dtype** - bf16 on the GPU by default, but **fp16 on cards without bf16 acceleration**
+  (Turing/Volta - compute capability < 8.0, e.g. a T4): same footprint, but it uses the fp16 tensor
+  cores instead of bf16's slow path. The **VAE stays upcast** (bf16, or fp32 when the denoiser is
+  fp16) because fp16 VAE decode can overflow to black/NaN images (`_compute_dtype` in `device/`).
+- **Multi-GPU** - `INLINE_PARALLEL` (e.g. `pipefusion=2`, `pipefusion=2,ulysses=2`); degrees multiply
   to the world size, which must equal the GPU count.
 
 ### The `/v1` API (the contract with Inline Studio)
@@ -136,17 +153,17 @@ between nodes and are never takes.
 - `GET /v1/models/{type}`, `GET /v1/takes/{id}`, `GET /v1/takes/{id}/bytes`, `POST /v1/assets`,
   `GET /v1/health`.
 
-Errors are `{error: {code, message, nodeId?}}` with the right HTTP status — they never leak a raw
+Errors are `{error: {code, message, nodeId?}}` with the right HTTP status - they never leak a raw
 traceback. The JSON shapes live in `server/serialize.py`.
 
 ## Multi-GPU (xDiT): split one image across GPUs
 
-One image's **denoise loop** (the expensive part) runs collectively across GPUs — not "one image per
+One image's **denoise loop** (the expensive part) runs collectively across GPUs - not "one image per
 GPU". It's done with xfuser in an isolated worker group (one process per GPU via `torchrun`, over
 local IPC) behind the `XFuserBatchedSampler` seam. Single-GPU/CPU runs take the in-process path and
 pay no overhead. Split method is chosen from the detected interconnect: **PipeFusion** (default, PCIe)
 or **Ulysses** (NVLink). The policy and IPC round-trip are in place and tested with a stub worker; the
-real xfuser denoise lands with the GPU-side runner (C2). Keep `sampling/batch.py` torch-free — the
+real xfuser denoise lands with the GPU-side runner (C2). Keep `sampling/batch.py` torch-free - the
 real codec that moves tensors lives with the model runner.
 
 ## Code standards (non-negotiable)
@@ -158,7 +175,7 @@ real codec that moves tensors lives with the model runner.
   (`graph/validate.py` + `port_satisfies`) rejects bad wiring at submit. New port kinds go in
   `schema.py`.
 - **Device policy owns placement.** Components and runners **never** self-assign a device/dtype/
-  offload — they call `ctx.policy.placement(role)` (`text_encoder`, `denoiser`, `vae`, …). This is the
+  offload - they call `ctx.policy.placement(role)` (`text_encoder`, `denoiser`, `vae`, …). This is the
   rule that keeps one graph portable across GPU / low-VRAM / CPU / multi-GPU. The policy **prefers the
   GPU**: a low-VRAM GPU keeps weights resident (tiling/slicing/int8 do the saving) and does not
   auto-offload to CPU (`placement.offload` defaults False; opt in with `INLINE_ALLOW_CPU_OFFLOAD`).
@@ -166,7 +183,9 @@ real codec that moves tensors lives with the model runner.
   `SampleJob` through the batched-sampler seam. The executor orchestrates; it does not sample.
 - **Immutable takes.** The `TakeStore` owns bytes/hash/uri; regenerating adds a take. Never overwrite.
 - **Engine deps are optional and import-guarded.** Heavy deps (torch, diffusers, xfuser) live in
-  `[project.optional-dependencies]` extras (`zimage`, `server`, `parallel`, `dev`). Model-runner
+  `[project.optional-dependencies]` extras (`runtime`, `server`, `parallel`, `dev`). **`runtime` is
+  the single shared ML stack - a new model must reuse it, never declare its own torch/diffusers
+  block.** Model-runner
   subpackages import torch/diffusers at module top **on purpose**: an absent extra makes the import
   raise, and `server/bootstrap.py` skips that model best-effort so a core install still boots and
   serves source nodes. Never import a heavy dep at package top level outside a runner subpackage.
@@ -183,14 +202,14 @@ real codec that moves tensors lives with the model runner.
 
 ```
 uv venv                                   # create ./.venv
-uv pip install -e ".[dev]"                # engine + server + test tooling
-uv pip install -e ".[zimage]"             # + torch, diffusers, transformers (real generation)
-uv pip install -e ".[parallel]"           # + xfuser, for multi-GPU denoise (2+ GPUs)
+uv pip install -e ".[server,dev]"         # engine + server + test tooling
+uv pip install -e ".[runtime]"            # + torch, diffusers, transformers (real generation)
+uv pip install -e ".[runtime,parallel]"   # + xfuser, for multi-GPU denoise (2+ GPUs)
 
 ./webui.sh                                # run (loopback:8848); friendly flags → INLINE_* env
 ./webui.sh --listen --port 9000           # bind all interfaces
 ./webui.sh --lowvram                      # tight-VRAM profile
-./webui.sh --install --extra zimage       # set up ./.venv with the Z-Image runtime, then exit
+./webui.sh --install --extra runtime      # set up ./.venv with the model runtime, then exit
 python -m inline_core.server              # run the server directly (INLINE_HOST / INLINE_PORT)
 
 ruff check .                              # lint (zero warnings)
@@ -202,7 +221,7 @@ uv run pytest -q                          # tests (no GPU; model code is import-
 - **New model runner** → a subpackage `models/<name>/` with `runner.py` (a `NodeDescriptor` + a
   `NodeRunner` + `register_<name>(registry, store, policy)`) and an `__init__.py` re-exporting it;
   add a `try/except ImportError` block in `server/bootstrap.py`; add an optional-deps extra in
-  `pyproject.toml`. Copy `models/zimage/` — it's the reference.
+  `pyproject.toml`. Copy `models/zimage/` - it's the reference.
 - **New low-level primitive** → descriptor in `graph/primitives.py`; its runner lands with the C2 work
   (build a component in `components/`, wire it through `encode`/`sample`/`vae` and the sampler seam).
 - **New `/v1` route** → add it in `server/app.py`, shape the JSON in `server/serialize.py`, keep errors

inline_core-1.2.0/PKG-INFO

@@ -0,0 +1,247 @@
+Metadata-Version: 2.4
+Name: inline-core
+Version: 1.2.0
+Summary: The generation engine behind Inline Studio.
+Requires-Python: >=3.11
+Requires-Dist: numpy>=1.26
+Requires-Dist: psutil>=5.9
+Provides-Extra: all
+Requires-Dist: accelerate>=0.30; extra == 'all'
+Requires-Dist: diffusers>=0.36; extra == 'all'
+Requires-Dist: fastapi>=0.110; extra == 'all'
+Requires-Dist: huggingface-hub>=0.23; extra == 'all'
+Requires-Dist: imageio-ffmpeg>=0.4; extra == 'all'
+Requires-Dist: safetensors>=0.4; extra == 'all'
+Requires-Dist: scipy>=1.11; extra == 'all'
+Requires-Dist: torch>=2.2; extra == 'all'
+Requires-Dist: torchao>=0.14; extra == 'all'
+Requires-Dist: transformers>=4.44; extra == 'all'
+Requires-Dist: uvicorn[standard]>=0.29; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: httpx>=0.27; extra == 'dev'
+Requires-Dist: pyright>=1.1; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Provides-Extra: parallel
+Requires-Dist: nvidia-ml-py>=12; extra == 'parallel'
+Requires-Dist: xfuser>=0.4; extra == 'parallel'
+Provides-Extra: runtime
+Requires-Dist: accelerate>=0.30; extra == 'runtime'
+Requires-Dist: diffusers>=0.36; extra == 'runtime'
+Requires-Dist: huggingface-hub>=0.23; extra == 'runtime'
+Requires-Dist: safetensors>=0.4; extra == 'runtime'
+Requires-Dist: scipy>=1.11; extra == 'runtime'
+Requires-Dist: torch>=2.2; extra == 'runtime'
+Requires-Dist: torchao>=0.14; extra == 'runtime'
+Requires-Dist: transformers>=4.44; extra == 'runtime'
+Provides-Extra: server
+Requires-Dist: fastapi>=0.110; extra == 'server'
+Requires-Dist: imageio-ffmpeg>=0.4; extra == 'server'
+Requires-Dist: uvicorn[standard]>=0.29; extra == 'server'
+Description-Content-Type: text/markdown
+
+# Inline Core
+
+The generation engine behind Inline. It takes a typed node graph (JSON) and returns immutable renders
+("takes"), running image and video models on macOS, Windows, and Linux, from CPU-only boxes and
+low-VRAM laptops up to multi-GPU machines that split a single image's sampling across GPUs (via
+xDiT). It is Inline Studio's built-in render backend.
+
+First model: Z-Image (Alibaba Tongyi), a 6B rectified-flow diffusion transformer (and a model xDiT
+already supports, so the multi-GPU split works on it from the start).
+
+> Status: early, and running end to end against a stub engine. In place and tested: the graph engine,
+> the typed `/v1` HTTP + websocket API (durable runs, streamed progress, coalescing), the model-dir
+> scan, the device + memory policy (profiles, dtype, offload, int8), the low-level primitive node
+> vocabulary, and a ComfyUI workflow importer. The Z-Image loader is written and validates on a GPU.
+> Cross-request batching, single-image multi-GPU (an xDiT worker group behind the sampler seam, with
+> the policy and IPC round-trip tested), and out-of-process custom nodes are built as seams but not
+> yet running on real hardware.
+
+## Engine design
+
+The two boundaries that matter most: graph orchestration is decoupled from GPU batching (graphs are
+the unit of caching, the sampler is the unit of batching, and the multi-GPU split routes through that
+same seam), and the device policy is the single owner of placement, so the same graph runs on a 4090,
+a 6 GB laptop, pure CPU, or split across several GPUs, without touching the graph.
+
+|              |                                                                                                                                                  |
+| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
+| Graph vs GPU | orchestration (cheap, per request) is separate from a batched sampler that groups compatible jobs across requests                                |
+| Schema       | typed graph, named params, edges type-checked before the run (a bad graph is rejected at submit)                                                 |
+| Devices      | one device/memory policy owns dtype, placement, offload, and attention; no node hardcodes a device, so one graph runs GPU, low-VRAM, or pure CPU |
+| Multi-GPU    | one image's denoise can split across GPUs via xDiT (PipeFusion on PCIe, Ulysses on NVLink), in an isolated worker group behind the sampler seam  |
+| Custom nodes | run out of process, each pack in its own venv, behind a semver SDK                                                                               |
+| Interface    | a headless `/v1` HTTP + websocket API; runs are durable and survive a restart                                                                    |
+| Outputs      | immutable takes; regenerating adds a take, never overwrites                                                                                      |
+| Models       | drop-in `models/` layout (bring your own, no downloads); a typed catalog feeds versioned node descriptors the UI renders generically             |
+
+## Install
+
+Requires Python 3.11+ and [uv](https://docs.astral.sh/uv/).
+
+```
+uv venv
+uv pip install -e ".[server]"     # engine + HTTP/websocket API
+uv pip install -e ".[runtime]"    # + torch, diffusers, transformers (for real generation)
+uv pip install -e ".[runtime,parallel]"  # + xfuser, to split one image across GPUs (2+ GPUs)
+```
+
+## Models
+
+Drop files into the models dir (default `./models`, override with `INLINE_MODELS_DIR`), ComfyUI-style,
+by category:
+
+```
+models/
+  diffusion_models/   z_image_turbo_bf16.safetensors   <- the one file you need for Z-Image
+  vae/                ae.safetensors                   <- optional (see below)
+  text_encoders/      qwen/   (an HF-format folder: config + tokenizer + weights)  <- optional
+  loras/  controlnet/  checkpoints/  ...
+```
+
+**Z-Image loads from a single diffusion `.safetensors`.** Drop that one ComfyUI-style file in
+`diffusion_models/`; the runner loads the transformer via diffusers `from_single_file`. It also needs
+a VAE + text-encoder + tokenizer, resolved from **local files** - `vae/*.safetensors` or a dir, and an
+HF-format `text_encoders/` dir.
+
+**Nothing is ever downloaded as a side effect of a render.** Every load runs `local_files_only=True`,
+so a missing model is a clear, fast error, not a silent multi-GB fetch. Models arrive by exactly two
+paths:
+
+1. **You place files** under `models/` (bring-your-own / fully offline); or
+2. **You download them explicitly** from the Z-Image node's model popup in the UI - it lists the
+   diffusion model, VAE, and text-encoder, shows which are missing, and downloads them **into
+   `models/`** (never the hidden HF cache) with visible progress.
+
+Override the diffusion source with `INLINE_ZIMAGE_MODEL` (a file or a diffusers dir), and the supporting
+components with `INLINE_ZIMAGE_VAE` / `INLINE_ZIMAGE_TEXT_ENCODER`. The engine scans the models dir on
+start; a node's model pickers list what is present.
+
+## Nodes
+
+`/v1/models` serves each node's descriptor (ports, params, file pickers), so the Inline Studio canvas
+renders any node generically - adding a node type needs no UI release.
+
+**High-level model nodes are what the user sees.** Generation is one-click: you drop a single
+**Z-Image Turbo** node, wire a Prompt into it, and hit Run. The node hooks up the diffusion model, VAE,
+and text-encoder behind the scenes - no loader/sampler wiring.
+
+Underneath, a **low-level primitive vocabulary** exists (`load/diffusion-model`, `load/vae`,
+`load/text-encoder`, `encode/text`, `latent/empty`, `sample`, `vae/decode`, `vae/encode`) - the
+ComfyUI-equivalent decomposed graph, kept for validation/execution - but these are marked **`hidden`**
+and never appear in the add-node menu. Engine handles (`model`, `vae`, `text-encoder`, `conditioning`,
+`latent`) are typed sockets between nodes; only media outputs become Frames with take history. A
+best-effort ComfyUI importer maps existing workflows onto the primitives.
+
+## Multi-GPU: split one image across GPUs
+
+Cut a single image's latency by running its denoise across several GPUs. The denoise loop (the
+iterative sampling step) is the expensive part of a render; with two or more GPUs, Inline Core runs
+each step's transformer forward collectively across them so one image finishes faster. This is not
+"one image per GPU" (independent renders); it is one image whose sampling is shared by all the GPUs.
+
+It is done with xDiT (`xfuser`), which parallelizes diffusion-transformer inference. xfuser runs in
+an isolated worker group the engine spawns (one process per GPU via `torchrun`) and talks to over
+local IPC, so the HTTP server, database, and graph stay single-process and only the denoise is
+distributed. It sits behind the sampler seam (`XFuserBatchedSampler`), so a single-GPU or CPU run
+takes the in-process path and pays no overhead.
+
+Two split methods, chosen from the interconnect the engine detects:
+
+- **PipeFusion (default, PCIe):** shards the transformer into a displaced patch pipeline with low,
+  depth-independent communication. It needs no NVLink and works over plain PCIe (or Ethernet across
+  nodes), so it is the default on a typical multi-GPU box.
+- **Ulysses (NVLink):** sequence-parallel attention, used when NVLink is present because it wants
+  the higher interconnect bandwidth.
+
+Enabling it:
+
+1. **Install the extra** and have 2+ CUDA GPUs on one machine:
+   ```
+   uv pip install -e ".[runtime,parallel]"  # xfuser + nvidia-ml-py; torchrun ships with torch
+   ```
+2. **Run normally.** On the first denoise, the device policy enumerates the GPUs, detects NVLink vs
+   PCIe (via `nvidia-ml-py`), and returns a parallel placement when there is more than one GPU. The
+   engine then spawns the xfuser worker group (lazily, once, then reuses it) and splits the sampling
+   across the GPUs. No graph, API, or per-request change is needed.
+3. **Override the split** if you want to pick it by hand, with `INLINE_PARALLEL`:
+   ```
+   INLINE_PARALLEL=pipefusion=2              # 2 GPUs, PipeFusion
+   INLINE_PARALLEL=pipefusion=2,ulysses=2    # 4 GPUs, PipeFusion x Ulysses
+   ```
+   The degrees multiply to the world size, which must equal the number of GPUs.
+
+The device policy and the worker-group IPC are in place and tested with a stub worker; the real
+xfuser denoise lands with the GPU-side Z-Image runner.
+
+## Memory: fit a big model onto a small GPU
+
+When a model is too large to hold full-precision on your GPU - or larger than your system RAM - Inline
+Core fits it to the machine automatically, **with no flags**. This is the low-VRAM path, and it aims to
+"just run" instead of making you tune profiles.
+
+- **Weights stream straight to the GPU.** Each component loads directly onto the GPU from its
+  memory-mapped `.safetensors` - the engine never materializes a full copy in system RAM first. So you
+  do **not** need roughly 2× the model size in RAM to load it, and a load can no longer exhaust host RAM
+  and take the server down.
+- **Auto-fit picks the lightest plan that fits.** The policy sizes the model against your VRAM and
+  chooses, in order: full-precision **resident** → **int8 resident** (weight-only quantization halves the
+  big weights so they fit on the GPU) → **CPU-offload streaming** (submodules move on/off the GPU). A card
+  that can't hold the model full-precision quantizes to int8 on its own - no need to know a flag.
+- **Load-time fit check.** A model too large for VRAM + RAM fails fast with a clear message on the node,
+  instead of crashing mid-load. The model popup also shows the estimate up front (fits / will run int8 /
+  won't fit).
+- **Switching models frees the old one.** Loading a different checkpoint evicts the previous weights
+  rather than stacking them, so you can move between models without accumulating VRAM.
+- **Generation fits too, not just loading.** Prompts encode under `no_grad` so the text-encoder forward
+  never retains its activation graph, and the encoder is parked on the CPU during denoise to free its
+  VRAM; large images (1024² and up) decode the VAE in tiles instead of one full-frame pass. So the
+  memory-heavy steps - encode and decode - stay bounded rather than spiking host RAM or VRAM and taking
+  the server down at high resolution.
+- **Fragmentation-resistant allocator.** `webui.sh` runs with PyTorch expandable CUDA segments, which
+  avoids the "allocation failed with VRAM still free" fragmentation OOMs common on low-VRAM cards.
+
+You can still steer it by hand: `--lowvram` for the tight-VRAM profile, `--smart-memory` to force the
+offload machinery, `--vram-budget GB` / `INLINE_VRAM_BUDGET_GB` to cap the assumed VRAM, or
+`INLINE_PROFILE` to pin `gpu-max | lowvram | cpu`. `GET /v1/health` reports the live VRAM/RAM budget.
+
+## Run
+
+The easy path is `webui.sh`, which maps friendly flags onto the engine's `INLINE_*` env knobs:
+
+```
+./webui.sh                            # loopback, port 8848
+./webui.sh --listen --port 9000       # bind all interfaces on 9000
+./webui.sh --multi-gpu                # split one image across GPUs (auto with 2+ GPUs)
+./webui.sh --lowvram                  # tight-VRAM profile
+./webui.sh --install --extra zimage   # set up ./.venv with the Z-Image runtime, then exit
+```
+
+`./webui.sh --help` lists every flag (networking, multi-GPU, device/memory profile, paths). The same
+flags are available on the Python entrypoint - `python main.py --help` - which is the dev path (it also
+takes `--front-end-root DIR` to serve a local SPA build). Or run the server module directly:
+
+```
+python main.py --listen --port 9000   # same flags as webui.sh; --front-end-root for a local UI build
+python -m inline_core.server          # bare server (INLINE_HOST / INLINE_PORT from the env)
+```
+
+Working data (the run database and takes) lives in `INLINE_DATA_DIR` (default `./.inline`).
+
+## API (v1)
+
+- `POST /v1/runs {graph, target}` returns `{runId}` (validated up front; 422 on a bad graph)
+- `GET /v1/runs/{id}` returns run state (durable; survives a restart)
+- `GET /v1/runs/{id}/events` (websocket): a snapshot, then `progress` / `node_done` / `run_done`
+- `DELETE /v1/runs/{id}` cancels
+- `GET /v1/models` returns node descriptors + `registryVersion` (ETag-aware)
+- `GET /v1/takes/{id}` and `/v1/takes/{id}/bytes`
+- `POST /v1/assets` (content-addressed upload) and `GET /v1/health`
+
+## Development
+
+```
+ruff check .        # lint
+uv run pytest -q    # tests (no GPU needed; model code is import-guarded)
+```