vhotseat 0.1.0__tar.gz

vhotseat-0.1.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.py[cod]
+.venv/
+dist/
+build/
+*.egg-info/
+.pytest_cache/
+.ruff_cache/
+uv.lock

vhotseat-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 wandermyz
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

vhotseat-0.1.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.5
+Name: vhotseat
+Version: 0.1.0
+Summary: Take turns on one GPU: hand it between a local LLM (e.g. Strata, driven by an agent like pi), ComfyUI and any other workload, never interrupting running work.
+Project-URL: Homepage, https://github.com/wandermyz/vhotseat
+Project-URL: Issues, https://github.com/wandermyz/vhotseat/issues
+Author: wandermyz
+License-Expression: MIT
+License-File: LICENSE
+Keywords: comfyui,gpu,llm,local-llm,minimax-h3,pi-agent,scheduler,strata,vram
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: System :: Hardware
+Requires-Python: >=3.10
+Requires-Dist: pyyaml>=6
+Description-Content-Type: text/markdown
+
+# vhotseat
+
+[![CI](https://github.com/wandermyz/vhotseat/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/wandermyz/vhotseat/actions/workflows/ci.yml)
+[![Release](https://img.shields.io/github/v/release/wandermyz/vhotseat?sort=semver)](https://github.com/wandermyz/vhotseat/releases/latest)
+
+**Want your local LLM agent to drive ComfyUI on the same consumer graphics
+card?** Without coordination, the first video render lands on top of the
+loaded LLM, and something runs out of memory. **Boom, OOM!**
+
+vhotseat makes [Strata](https://github.com/Niko1221/Strata) and
+[ComfyUI](https://github.com/comfyanonymous/ComfyUI) take turns on VRAM *and*
+RAM: when the agent queues a render, the LLM is unloaded; when it asks the LLM
+its next question, ComfyUI is freed. Neither ever loads on top of the other,
+and **running work is never interrupted**: not an answer the LLM is writing,
+not a render in progress.
+
+> v-hotseat: VRAM hotseat. Hot-seat multiplayer: one device, players take
+> turns, nobody is kicked out mid-turn.
+
+![vhotseat: one GPU, local agentic GenAI. pi drives, ComfyUI renders, Strata reviews and composites.](https://raw.githubusercontent.com/wandermyz/vhotseat/a619e2e66e284521f3a8f66e2892790a562403df/branding/social-preview.png)
+
+**Tested on:**
+
+| Part | Tested with |
+| --- | --- |
+| GPU / RAM | RTX 5070 Ti (16 GB VRAM), 64 GB RAM |
+| LLM | Strata serving Qwen3.8-Flash-Next IQ3_XXS (128K context) |
+| Agent | [pi](https://github.com/earendil-works/pi) |
+| ComfyUI | MiniMax H3 video (int8 `fl2va` / `ref2va`) |
+
+With this setup, pi drove long-running video workflows unattended, over
+several rounds of render → review → render, overnight.
+
+Status: Linux, NVIDIA. Windows is designed for and planned. MIT licence.
+
+**→ [Quick start for this exact setup](https://github.com/wandermyz/vhotseat/blob/main/docs/guides/quickstart-strata-pi-comfyui.md)**:
+a step-by-step guide that you, or your coding agent, can follow. Or just
+prompt your coding agent to set up the whole workflow:
+
+> Set up vhotseat so my local LLM agent can drive ComfyUI on this GPU without
+> running out of memory. Follow
+> https://github.com/wandermyz/vhotseat/blob/main/docs/guides/quickstart-strata-pi-comfyui.md,
+> adapting it to the servers and paths on this machine, and run each step's
+> check before going on.
+
+## How it works
+
+The servers stay as they are, and every client is covered: the agent, its
+scripts, a web UI, curl. Nobody calls vhotseat; each workload asks for the GPU
+through a **gate** right before it loads, and vhotseat makes room by releasing
+the others:
+
+- **The LLM** runs `vhotseat gate` from its pre-load hook. While ComfyUI has
+  work queued the load is refused (a 503 the agent's harness retries).
+- **ComfyUI** runs `vhotseat switch` before every job through a small custom
+  node, the guard. It waits for any answer the LLM is writing, then unloads it.
+- **Admission** checks there's enough free VRAM and RAM before a load, and
+  restarts an idle ComfyUI if its leftover RAM is what's in the way.
+- **[comfy-render](https://github.com/wandermyz/vhotseat/blob/main/kit/README.md#comfy-render)** lets the agent run a batch of
+  workflows in one call, so a batch costs two handovers instead of two per job.
+
+Strata and ComfyUI are just the built-in examples: anything that uses the GPU
+(another LLM server, a TTS server, a training run) can join through a config
+entry or a small driver, with a hook, plugin, proxy or `hold` as its gate.
+
+**How it differs.** [llama-swap](https://github.com/mostlygeek/llama-swap) and
+Ollama's scheduler swap models between LLM servers they control; they don't
+take turns with a non-LLM app like ComfyUI, and don't wait for a running job.
+vhotseat arbitrates between *different* servers, of any kind, and can sit on
+top of either (they can be one of its workloads).
+
+## Install
+
+```sh
+uv tool install vhotseat      # or: pipx install vhotseat
+```
+
+That installs two commands, `vhotseat` and `comfy-render`, with one dependency
+(PyYAML). From a checkout, `./vhotseat` and `./comfy-render` run as they are;
+symlink them into `~/.local/bin`.
+
+Then set it up for your servers: follow the
+[quick start](https://github.com/wandermyz/vhotseat/blob/main/docs/guides/quickstart-strata-pi-comfyui.md), or let a coding
+agent do it with the [vhotseat-setup skill](https://github.com/wandermyz/vhotseat/blob/main/kit/skills/vhotseat-setup/SKILL.md).
+[config.example.yaml](https://github.com/wandermyz/vhotseat/blob/main/config.example.yaml) is commented key by key, and
+[examples/](https://github.com/wandermyz/vhotseat/blob/main/examples/) has configs for other servers.
+
+```sh
+vhotseat doctor     # check the config, the GPU, every workload and gate
+vhotseat status     # who owns the GPU and what each workload holds
+vhotseat web        # dashboard: RAM and VRAM over time, handovers, events, logs
+```
+
+![The dashboard on an RTX 5070 Ti: Strata hands the GPU to ComfyUI at 01:04, and a render is running.](https://raw.githubusercontent.com/wandermyz/vhotseat/24999a8e62e3a6342e9af6c3bc70042e77765c60/web-dashboard/desktop.png)
+
+## Documentation
+
+- [Quick start](https://github.com/wandermyz/vhotseat/blob/main/docs/guides/quickstart-strata-pi-comfyui.md): Strata + pi +
+  ComfyUI on one 16 GB card, step by step.
+- [How it works](https://github.com/wandermyz/vhotseat/blob/main/docs/how-it-works.md): a full agent → ComfyUI → agent flow,
+  the races and deadlocks, and how often the GPU changes hands.
+- [Reference](https://github.com/wandermyz/vhotseat/blob/main/docs/reference.md): commands, gates, admission, the ComfyUI
+  guard, `comfy-render`, the dashboard and telemetry.
+- [Agent kit](https://github.com/wandermyz/vhotseat/blob/main/kit/README.md): `comfy-render` and
+  [Agent Skills](https://agentskills.io) for the agent that renders, and for
+  the coding agent that sets vhotseat up and keeps it working.
+- [Writing a driver](https://github.com/wandermyz/vhotseat/blob/main/docs/guides/driver-authoring.md) for another server, and
+  [building a proxy gate](https://github.com/wandermyz/vhotseat/blob/main/docs/guides/proxy-gate.md) for one with no pre-load
+  hook.
+- [After a ComfyUI update](https://github.com/wandermyz/vhotseat/blob/main/docs/comfyui-updates.md): checking the guard still
+  works. [Fixing a VRAM leak](https://github.com/wandermyz/vhotseat/blob/main/docs/examples/vram-leak-tts-audio-suite.md) in a
+  custom node is a worked example.
+
+## Development
+
+```sh
+uv run --group dev pytest       # ~2 min; fake LLM and ComfyUI servers, no GPU needed
+uvx ruff check .
+```
+
+The tests run the real CLI and arbiter against fake servers over HTTP on
+localhost; the config they use is [config.example.yaml](https://github.com/wandermyz/vhotseat/blob/main/config.example.yaml),
+and every file in [examples/](https://github.com/wandermyz/vhotseat/blob/main/examples/) is validated. CI runs both on Python
+3.10 to 3.13.
+
+## Licence
+
+MIT. See [LICENSE](https://github.com/wandermyz/vhotseat/blob/main/LICENSE).

vhotseat-0.1.0/README.md

@@ -0,0 +1,128 @@
+# vhotseat
+
+[![CI](https://github.com/wandermyz/vhotseat/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/wandermyz/vhotseat/actions/workflows/ci.yml)
+[![Release](https://img.shields.io/github/v/release/wandermyz/vhotseat?sort=semver)](https://github.com/wandermyz/vhotseat/releases/latest)
+
+**Want your local LLM agent to drive ComfyUI on the same consumer graphics
+card?** Without coordination, the first video render lands on top of the
+loaded LLM, and something runs out of memory. **Boom, OOM!**
+
+vhotseat makes [Strata](https://github.com/Niko1221/Strata) and
+[ComfyUI](https://github.com/comfyanonymous/ComfyUI) take turns on VRAM *and*
+RAM: when the agent queues a render, the LLM is unloaded; when it asks the LLM
+its next question, ComfyUI is freed. Neither ever loads on top of the other,
+and **running work is never interrupted**: not an answer the LLM is writing,
+not a render in progress.
+
+> v-hotseat: VRAM hotseat. Hot-seat multiplayer: one device, players take
+> turns, nobody is kicked out mid-turn.
+
+![vhotseat: one GPU, local agentic GenAI. pi drives, ComfyUI renders, Strata reviews and composites.](https://raw.githubusercontent.com/wandermyz/vhotseat/a619e2e66e284521f3a8f66e2892790a562403df/branding/social-preview.png)
+
+**Tested on:**
+
+| Part | Tested with |
+| --- | --- |
+| GPU / RAM | RTX 5070 Ti (16 GB VRAM), 64 GB RAM |
+| LLM | Strata serving Qwen3.8-Flash-Next IQ3_XXS (128K context) |
+| Agent | [pi](https://github.com/earendil-works/pi) |
+| ComfyUI | MiniMax H3 video (int8 `fl2va` / `ref2va`) |
+
+With this setup, pi drove long-running video workflows unattended, over
+several rounds of render → review → render, overnight.
+
+Status: Linux, NVIDIA. Windows is designed for and planned. MIT licence.
+
+**→ [Quick start for this exact setup](docs/guides/quickstart-strata-pi-comfyui.md)**:
+a step-by-step guide that you, or your coding agent, can follow. Or just
+prompt your coding agent to set up the whole workflow:
+
+> Set up vhotseat so my local LLM agent can drive ComfyUI on this GPU without
+> running out of memory. Follow
+> https://github.com/wandermyz/vhotseat/blob/main/docs/guides/quickstart-strata-pi-comfyui.md,
+> adapting it to the servers and paths on this machine, and run each step's
+> check before going on.
+
+## How it works
+
+The servers stay as they are, and every client is covered: the agent, its
+scripts, a web UI, curl. Nobody calls vhotseat; each workload asks for the GPU
+through a **gate** right before it loads, and vhotseat makes room by releasing
+the others:
+
+- **The LLM** runs `vhotseat gate` from its pre-load hook. While ComfyUI has
+  work queued the load is refused (a 503 the agent's harness retries).
+- **ComfyUI** runs `vhotseat switch` before every job through a small custom
+  node, the guard. It waits for any answer the LLM is writing, then unloads it.
+- **Admission** checks there's enough free VRAM and RAM before a load, and
+  restarts an idle ComfyUI if its leftover RAM is what's in the way.
+- **[comfy-render](kit/README.md#comfy-render)** lets the agent run a batch of
+  workflows in one call, so a batch costs two handovers instead of two per job.
+
+Strata and ComfyUI are just the built-in examples: anything that uses the GPU
+(another LLM server, a TTS server, a training run) can join through a config
+entry or a small driver, with a hook, plugin, proxy or `hold` as its gate.
+
+**How it differs.** [llama-swap](https://github.com/mostlygeek/llama-swap) and
+Ollama's scheduler swap models between LLM servers they control; they don't
+take turns with a non-LLM app like ComfyUI, and don't wait for a running job.
+vhotseat arbitrates between *different* servers, of any kind, and can sit on
+top of either (they can be one of its workloads).
+
+## Install
+
+```sh
+uv tool install vhotseat      # or: pipx install vhotseat
+```
+
+That installs two commands, `vhotseat` and `comfy-render`, with one dependency
+(PyYAML). From a checkout, `./vhotseat` and `./comfy-render` run as they are;
+symlink them into `~/.local/bin`.
+
+Then set it up for your servers: follow the
+[quick start](docs/guides/quickstart-strata-pi-comfyui.md), or let a coding
+agent do it with the [vhotseat-setup skill](kit/skills/vhotseat-setup/SKILL.md).
+[config.example.yaml](config.example.yaml) is commented key by key, and
+[examples/](examples/) has configs for other servers.
+
+```sh
+vhotseat doctor     # check the config, the GPU, every workload and gate
+vhotseat status     # who owns the GPU and what each workload holds
+vhotseat web        # dashboard: RAM and VRAM over time, handovers, events, logs
+```
+
+![The dashboard on an RTX 5070 Ti: Strata hands the GPU to ComfyUI at 01:04, and a render is running.](https://raw.githubusercontent.com/wandermyz/vhotseat/24999a8e62e3a6342e9af6c3bc70042e77765c60/web-dashboard/desktop.png)
+
+## Documentation
+
+- [Quick start](docs/guides/quickstart-strata-pi-comfyui.md): Strata + pi +
+  ComfyUI on one 16 GB card, step by step.
+- [How it works](docs/how-it-works.md): a full agent → ComfyUI → agent flow,
+  the races and deadlocks, and how often the GPU changes hands.
+- [Reference](docs/reference.md): commands, gates, admission, the ComfyUI
+  guard, `comfy-render`, the dashboard and telemetry.
+- [Agent kit](kit/README.md): `comfy-render` and
+  [Agent Skills](https://agentskills.io) for the agent that renders, and for
+  the coding agent that sets vhotseat up and keeps it working.
+- [Writing a driver](docs/guides/driver-authoring.md) for another server, and
+  [building a proxy gate](docs/guides/proxy-gate.md) for one with no pre-load
+  hook.
+- [After a ComfyUI update](docs/comfyui-updates.md): checking the guard still
+  works. [Fixing a VRAM leak](docs/examples/vram-leak-tts-audio-suite.md) in a
+  custom node is a worked example.
+
+## Development
+
+```sh
+uv run --group dev pytest       # ~2 min; fake LLM and ComfyUI servers, no GPU needed
+uvx ruff check .
+```
+
+The tests run the real CLI and arbiter against fake servers over HTTP on
+localhost; the config they use is [config.example.yaml](config.example.yaml),
+and every file in [examples/](examples/) is validated. CI runs both on Python
+3.10 to 3.13.
+
+## Licence
+
+MIT. See [LICENSE](LICENSE).

vhotseat-0.1.0/comfy-render

@@ -0,0 +1,12 @@
+#!/usr/bin/env python3
+"""comfy-render from a checkout (installed as a symlink, like ./vhotseat). See src/vhotseat/kit/comfy_render.py."""
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent / "src"))
+
+from vhotseat.kit.comfy_render import main  # noqa: E402
+
+if __name__ == "__main__":
+    sys.exit(main())

vhotseat-0.1.0/config.example.yaml

@@ -0,0 +1,101 @@
+# vhotseat config: ~/.config/vhotseat/config.yaml (or $VHOTSEAT_CONFIG).
+# Two workloads on one 16 GB card: Strata (a local LLM server) and ComfyUI.
+# Every key is commented. Change the paths, units and figures to your machine,
+# then run `vhotseat doctor`. examples/ has one snippet per workload.
+
+gpu:
+  index: 0
+  backend: auto        # nvml (no subprocess per read), nvidia-smi, or auto (nvml, else nvidia-smi)
+
+workloads:
+  # A workload is named here; that name is what `vhotseat switch NAME` takes.
+  strata:
+    driver: http       # described entirely by HTTP requests below
+    label: Strata
+    url: http://127.0.0.1:2335
+    service: {systemd-user: strata}   # started when stopped; restarted when allow_restart
+    # Its VRAM is found by the systemd unit its processes run in. Without
+    # `vram:`, the service's unit is used; {process: SUBSTRING} matches the
+    # command line instead.
+    resident: {get: /health, path: loaded}   # true while its model is in VRAM
+    release:
+      post: /unload?wait={wait}   # {wait}: seconds to let a running answer finish
+      always: true                # ask even when it looks unloaded: a load may be under way
+      busy_when: [{status: 409}, {path: status, in: [busy]}]
+      ok_when: [{status: 200, path: status, in: [unloaded, not loaded]}]
+      # timeout: 30      # seconds the request may take on top of {wait} (default 30)
+      # pre: {post: …}   # a request first, e.g. a proxy's drain (docs/guides/proxy-gate.md)
+    confirm: {vram_at_most: 0}    # released once its VRAM is down to this
+    load: {post: /load, timeout: 300}   # `switch strata` leaves it loaded
+    model: {get: /health, path: model}  # which model it serves; picks the admission below
+    gate:
+      kind: hook       # Strata runs `vhotseat gate strata` before every load
+      config: ~/Strata/strata.json   # its config file, checked by doctor
+      timeout: 90      # Strata's before_load_timeout_s: the hook must fit in it
+    admission:
+      # RAM the machine must be able to give Strata before it may load: its
+      # experts plus working memory, with some headroom. Compared with
+      # *available* RAM (MemAvailable), which counts page cache the kernel can
+      # drop on demand; *free* RAM is always small on a busy machine. When
+      # other workloads still hold too much after releasing their models, idle
+      # ones with allow_restart are restarted; if that still isn't enough, the
+      # load is refused and the event log says RAM was short.
+      # This default is for a model not listed under by_model (or when Strata
+      # doesn't say which it serves): the largest one, to stay safe.
+      min_available_ram_mib: 46000
+      # VRAM that must be free on the card before Strata may load. Other
+      # workloads always release the GPU first, whatever this says; this
+      # catches what vhotseat doesn't manage (the desktop, a game, a release
+      # that left VRAM behind).
+      min_free_vram_mib: 14000
+      # Per model, as /health reports it. For example, loading from idle,
+      #   IQ3_XXS took 44.5 GiB of available RAM (42.5 GiB resident, 56 GiB at the peak)
+      #   IQ2_XS  took 38.2 GiB (35.6 GiB resident, 47 GiB at the peak)
+      # Both use ~14.7 GB of VRAM. To measure a model: load it (`vhotseat switch
+      # strata` from idle), note how much MemAvailable dropped, add ~1.5 GiB.
+      by_model:
+        qwen3.8-flash-next-iq3_xxs: {min_available_ram_mib: 46000}
+        qwen3.8-flash-next-iq2_xs: {min_available_ram_mib: 40000}
+    allow_restart: true
+
+  comfy:
+    driver: comfyui    # built-in: /queue, /free, restart when /free isn't enough
+    label: ComfyUI
+    url: http://127.0.0.1:8188
+    service: {systemd-user: comfyui}
+    idle_vram_mib: 1500   # its CUDA context and caches; above this it holds models
+    gate:
+      kind: plugin     # the guard node takes the GPU before every job
+      custom_nodes: ~/ComfyUI/custom_nodes   # where the guard is installed; checked by doctor
+      timeout: 300     # how long a job waits for an answer Strata is writing
+    # No admission: what ComfyUI needs depends on the workflow, and it manages
+    # its own memory. Other workloads still release the GPU before every job.
+    # Restarted when its release leaves VRAM behind, or when another
+    # workload's admission needs RAM its idle process still holds (6-15 GiB
+    # after a video render). Only when its queue is empty and its last result
+    # finished at least history_settle_s ago (a restart wipes /history).
+    allow_restart: true
+
+# Local patches to other projects' code that a handover depends on. An update
+# of that project drops a patch silently, and the symptom shows up far away
+# (a release that leaves VRAM behind, so every handover is refused). `vhotseat
+# doctor` checks each marker is still in its file, and says when the installed
+# version has the fix upstream so the patch can go. An entry looks like:
+#
+#   - name: TTS Audio Suite IndexTTS unload
+#     file: ~/ComfyUI/custom_nodes/TTS-Audio-Suite/engines/index_tts/indextts/infer_v2_5.py
+#     marker: "LOCAL PATCH (vhotseat)"           # a comment you put next to your change
+#     doc: ~/.config/vhotseat/patches/tts-audio-suite.md   # optional: what and why
+#     fixed_upstream_in: {version: 5.10.4, version_file: ~/ComfyUI/custom_nodes/TTS-Audio-Suite/pyproject.toml}
+patches: []
+
+events:
+  keep: 500            # ~/.local/state/vhotseat/events.jsonl
+
+# RAM and VRAM over time, events and log lines, for `vhotseat web`. Every
+# vhotseat command records as it runs (~/.local/state/vhotseat/telemetry.db);
+# `vhotseat record` (or `vhotseat web`) samples between commands too.
+telemetry:
+  # enabled: true
+  retention_hours: 24  # ~6 MiB on disk with two workloads; ~0.8 MiB a day more per further workload
+  interval_s: 5        # between samples while something samples continuously

vhotseat-0.1.0/docs/comfyui-updates.md

@@ -0,0 +1,99 @@
+# Handling ComfyUI updates
+
+The guard node, `integrations/comfyui/vhotseat_guard`, makes each ComfyUI job
+take the GPU from the other workloads before it runs. It does that by wrapping
+a **private** ComfyUI method, `execution.PromptExecutor.execute_async`.
+ComfyUI doesn't promise to keep that method stable, so an update can break the
+guard. The `vhotseat-comfyui-update` skill walks an agent through this page.
+
+Below, `$COMFY` is ComfyUI's folder and the service is a systemd user unit
+named `comfyui`; adjust both to your setup.
+
+## What breaks, and how you'd notice
+
+| Change in ComfyUI | What happens | Visible as |
+| --- | --- | --- |
+| `execution.PromptExecutor` or `execute_async` renamed or removed | Guard turns itself **off**; ComfyUI keeps working, unguarded | `vhotseat_guard: OFF, ComfyUI internals changed` in the log at startup |
+| `execute_async` gets new parameters | Guard is on, but every job fails with a `TypeError` | Every job errors right after you queue it |
+| The worker stops calling `execute_async` once per job | Guard is on but never runs | Jobs run while the LLM is loaded; most likely an out-of-memory error |
+| Fields read by `_fail_job` change (`status_messages`, `history_result`, `add_message`) | Only the failure path breaks: a refused job crashes instead of failing cleanly | Test 4 below |
+
+The first row is the safe one. The others are why you should check after every
+update rather than wait to notice.
+
+## After every ComfyUI update
+
+1. **Note the current commit, update, then restart:**
+   ```sh
+   cd $COMFY && git rev-parse --short HEAD    # to roll back to, if needed
+   git pull && systemctl --user restart comfyui
+   ```
+
+2. **Confirm the guard loaded and is on:**
+   ```sh
+   journalctl --user -u comfyui --since "-5min" | grep vhotseat_guard
+   ```
+   Expect `vhotseat_guard: on (…/vhotseat, wait 300s)`. If you see `OFF` or
+   an import error, stop here and go to *Fixing it* below.
+
+3. **Check that the method's signature hasn't changed.** The guard mirrors it:
+   ```sh
+   grep -n "async def execute_async" $COMFY/execution.py
+   ```
+   Expect `async def execute_async(self, prompt, prompt_id, extra_data={}, execute_outputs=[]):`.
+   Also check that the worker still calls it once per job: `main.py` →
+   `prompt_worker` → `e.execute(...)` → `asyncio.run(self.execute_async(...))`.
+
+4. **Run the smoke tests.** These take a couple of minutes and need a few GB
+   of VRAM. `integrations/comfyui/smoke_job.py` queues a 1-step SDXL job that
+   ends in `PreviewImage`, so nothing is written to `output/`; pass a
+   checkpoint you have with `--ckpt NAME.safetensors`. It prints
+   `success in 9.0s`, or `error in 2.5s: GPU busy, job not run: …`. The tests
+   use Strata as the LLM; for another LLM, send it the same kind of requests.
+
+   | # | Do | Expect |
+   | --- | --- | --- |
+   | 1 | `vhotseat switch strata`, then queue a job | Job succeeds; `vhotseat status` shows `comfy`, Strata unloaded |
+   | 2 | Start a long Strata answer, queue a job 2–3 s later | Strata's answer completes; job runs after it and succeeds |
+   | 3 | Queue a job, send a Strata request, queue another | Both jobs succeed; each Strata request frees ComfyUI first |
+   | 4 | `systemctl --user set-environment VHOTSEAT_GUARD_WAIT_S=2`, restart ComfyUI, start a long Strata answer, queue a job | Job ends `status: error` with `GPU busy, job not run: …` in `/history/<id>`; Strata's answer completes. Then `systemctl --user unset-environment VHOTSEAT_GUARD_WAIT_S` and restart ComfyUI |
+   | 5 | `comfy-render` a batch of 2–3 small jobs (a `{"base": …, "jobs": [seeds]}` file ending in `SaveImage`) | Progress lines while it runs, one `ok` line per job with its file, exit code 0 |
+
+   For a long Strata answer:
+   ```sh
+   curl -s localhost:2335/v1/chat/completions -H 'Content-Type: application/json' \
+     -d '{"messages":[{"role":"user","content":"Count from 1 to 400."}],"max_tokens":1500}' >/dev/null &
+   ```
+
+   Test 5 checks comfy-render, which reads ComfyUI's public `/history` and
+   websocket formats rather than private methods, so it breaks less often than
+   the guard, but it does depend on them.
+
+5. **Record the commit** you verified (step 1's `git rev-parse` after the
+   pull), so the next rollback has a known-good target.
+
+## Fixing it
+
+- **Method renamed:** find what `prompt_worker` in `main.py` now calls per job,
+  and wrap that instead. The requirements are:
+  - it runs on the worker thread (blocking is fine there; the web server's
+    thread must never block);
+  - it runs once per job, before any model loads;
+  - you can still report a failure that shows up in `/history`.
+- **Do not move the guard into `PromptServer.add_on_prompt_handler`.** That
+  runs at submit time, on the web server's event loop. Blocking there for up
+  to 300 s would freeze ComfyUI's UI and API.
+- **To run unguarded until it's fixed:** set `VHOTSEAT_GUARD=0` in the
+  unit's environment (`systemctl --user edit comfyui`). Then queue ComfyUI
+  work only after running `vhotseat switch comfy` by hand.
+
+## Rolling back ComfyUI
+
+When a fix isn't quick, go back to the last commit the guard was verified on:
+
+```sh
+cd $COMFY && git log --oneline -5      # find the previous commit
+git checkout <commit> && systemctl --user restart comfyui
+```
+
+The guard was last verified by the vhotseat project on ComfyUI v0.39.0.

vhotseat-0.1.0/docs/examples/vram-leak-tts-audio-suite.md

@@ -0,0 +1,80 @@
+# Example: a VRAM leak in TTS Audio Suite, from symptom to upstream fix
+
+A worked example of the `vhotseat-vram-leak` skill. The machine shared one
+16 GB GPU between a local LLM (Strata) and ComfyUI, with the
+[TTS Audio Suite](https://github.com/diodiogod/TTS-Audio-Suite) custom node
+pack rendering speech with IndexTTS-2.5. Fixed upstream in **5.10.4**
+([PR #373](https://github.com/diodiogod/TTS-Audio-Suite/pull/373)); suite
+versions up to 5.10.3 have the leak.
+
+## Symptom
+
+After an IndexTTS render, ComfyUI's `POST /free` reported "freed 4804MB", but
+ComfyUI still held about 2 GB, above its `idle_vram_mib` of 1500. Every LLM
+load after a TTS render was refused:
+
+```
+strata-load (allowed=False, reason=could not free ComfyUI's 2066 MiB:
+  timed out after 20s waiting for ComfyUI to release VRAM (below 1500 MiB))
+```
+
+With `allow_restart: true` vhotseat restarts ComfyUI instead, which works but
+adds ~20 s to that handover, and the leak still costs ComfyUI 2 GB whenever its
+own model manager swaps the TTS model out for another one.
+
+## Diagnosis
+
+Reproduced standalone: load IndexTTS2 the way the suite does, render once
+under `torch.inference_mode()` (as ComfyUI does), call the suite's unload,
+then list the `gc.get_objects()` tensors still `is_cuda`.
+
+The suite wraps the IndexTTS2 engine (`engines/index_tts/indextts/infer_v2_5.py`)
+for ComfyUI's model manager. On unload, the wrapper calls `model.to("cpu")` if
+the object has a `to()`, and otherwise moves each `nn.Module` attribute one by
+one. `IndexTTS2` had no `to()`, so only those modules moved. Left on the GPU:
+
+| Left behind | Size |
+| --- | --- |
+| `qwen_emo` (QwenEmotion, a Qwen 0.6B text-to-emotion model held in a plain attribute, so never moved) | 1137 MiB |
+| Plain tensor attributes inside modules, which `Module.to()` skips: old-style `weight_norm` weights, a `causal_mask` (64 MiB), `freqs_cis` | ~180 MiB |
+| Cached reference-voice conditioning, emotion and speaker matrices | ~8 MiB |
+| Allocator cache held because of the above | the rest |
+
+## Fix
+
+A `to(device)` method on `IndexTTS2`, which the wrapper then uses for both
+unload and reload. It moves every model module plus the plain tensor
+attributes inside them, QwenEmotion if loaded, and the emotion and speaker
+matrices, and drops the cached reference-voice conditioning (rebuilt on the
+next render).
+
+| After a render and unload | ComfyUI VRAM |
+| --- | --- |
+| Before | 2066 MiB: LLM refused |
+| **Patched** | **~340 MiB** standalone, ~500 MiB in ComfyUI |
+| Floor: any process that has used CUDA | ~270 MiB |
+
+## Recording it until upstream shipped it
+
+While the fix was local, every changed spot carried `LOCAL PATCH (vhotseat)`,
+and the config had:
+
+```yaml
+patches:
+  - name: TTS Audio Suite IndexTTS unload
+    file: <ComfyUI>/custom_nodes/TTS-Audio-Suite/engines/index_tts/indextts/infer_v2_5.py
+    marker: "LOCAL PATCH (vhotseat)"
+    doc: ~/.config/vhotseat/patches/tts-audio-suite.md
+    fixed_upstream_in: {version: 5.10.4, version_file: <ComfyUI>/custom_nodes/TTS-Audio-Suite/pyproject.toml}
+```
+
+When the suite updated to 5.10.4, `vhotseat doctor` reported that the
+installed version had the fix upstream and the patch could go, before anyone
+looked; the entry was then removed.
+
+## If it comes back
+
+Other engines in the suite are wrapped the same way, so each needs a `to()`
+that moves everything. The symptom is the same: ComfyUI above `idle_vram_mib`
+after `/free`, or the event log showing ComfyUI restarted because it kept VRAM
+after `/free`, right after renders with that engine. Diagnose it the same way.