rag-wright 0.6.0__tar.gz → 0.7.0__tar.gz

{rag_wright-0.6.0 → rag_wright-0.7.0}/.claude/skills/qwen-vllm-modal/SKILL.md

@@ -18,7 +18,7 @@ This skill is the operational checklist that ties them together.
 
 ## The LOCKED production config (ADR-0110) — do not re-litigate
 **Config B: `Qwen/Qwen3.8-27B-FP8` weights + `--kv-cache-dtype fp8`, on 1× A100-80GB, TP=1, `--max-model-len 16384`, `--gpu-memory-utilization 0.95`.**
-- Config B (FP8 weights) is the **locked deploy** — 52.71× concurrency @16K (ADR-0110 rounds it to 53×), ~23% higher throughput, ~27 GB weights;
+- Config B (FP8 weights) is the **locked deploy** — 52.71× concurrency @16K (ADR-0110 rounds it to 53×), ~23% higher throughput, ~31 GB weights (30.9 GB on disk, 28.9 GiB in VRAM);
   accuracy measured **identical to bf16** on the real compliance workload (no FP8 penalty).
 - **Config A** (`Qwen/Qwen3.8-27B`, **bf16 weights** + FP8 KV) is the **documented FALLBACK only** — 25.47× concurrency,
   "zero weight-loss" — use it *only* if a later clean check ever shows an FP8-weight regression. Don't default to A.
@@ -29,8 +29,8 @@ This skill is the operational checklist that ties them together.
 **From the engine (PS-19): deploy, then use the server like any model.** `ModelServerSpec`'s defaults ARE the
 locked config below (FP8 weights + FP8 KV, one A100-80GB, TP 1, 16K, 0.95, served `Qwen/Qwen3.8-27B`, hermes +
 qwen3 parsers, image builder 2025.06); state only the concurrency target, which ADR-0110 leaves to you.
-`adeploy_model_server` runs the deploy below with those settings, waits for `/health` (with progress; a cold start is
-~8 min), and registers a model profile with the server's own endpoint, so no `VLLM_BASE_URL` is needed:
+`adeploy_model_server` runs the deploy below with those settings, waits for `/health` (with progress; a restore from
+the snapshot is seconds, a snapshot-building boot 4-9 min, see Cold start below), and registers a model profile with the server's own endpoint, so no `VLLM_BASE_URL` is needed:
 
 ```python
 from rag_wright.api import EngineConfig, ModelServerSpec, adeploy_model_server, stop_model_server
@@ -62,7 +62,7 @@ The script refuses to deploy without `VLLM_API_KEY`: Modal's web endpoint is pub
 `uv run --no-sync modal app stop rw-qwen3-modal --yes` (A100 is expensive).
 
 **The engine does NOT reach Modal by default.** Every role's default is `_PRODUCT_LLM = "qwen3.8-27b-modal-or"`,
-the same model routed through OpenRouter (the Modal cold start is unsolved). To send calls to the Modal server, set
+the same model routed through OpenRouter (a Modal cold start can still take minutes when Modal builds a snapshot). To send calls to the Modal server, set
 `RAG_MODEL_ALL=qwen3.8-27b-modal` (all roles), a per-role `RAG_MODEL_<ROLE>`, or `EngineConfig.models`. For bulk
 closed-vocab labeling, the reasoning-off profile `qwen3.8-27b-modal-nothink` exists (opt-in; see the reasoning
 caveat below). `VISION_OCR` now defaults to the product LLM too, but image inputs through the Modal vLLM server are
@@ -98,17 +98,31 @@ the OFFICIAL prebuilt vLLM image** rather than pip-building vLLM:
    reasoning calls take ~50–60 s; the seam's default 60 s timeout kills the ones just over the line → **retry pileup →
    requests pile past `max_inputs` → instant `APIConnectionError` (0.0 s) cascade**. FIX: the seam reads
    `RAG_STRUCTURED_TIMEOUT_S` (default 60) — raise it (e.g. `150`) for reasoning-ON bulk work. Server logs confirm the
-   calls actually complete 200 OK (~50 s); it was the client giving up.
+   calls actually complete 200 OK (~50 s); it was the client giving up. The total per call (all tries) follows it;
+   `RAG_MODEL_DEADLINE_S` overrides that (ADR-0133). Reasoning loops end at the profile's 8,192-token structured cap,
+   not at the deadline; keep temperature 0 (Qwen's recommended sampling did not remove them).
 3. **Pick client concurrency empirically, below the ceiling.** 16 was stable; 20 (== `max-num-seqs`) sat at the exact
    ceiling and risked queueing. `Running: N, Waiting: 0` + zero cancellations = healthy.
 4. **Reasoning ON vs OFF for bulk classification:** reasoning-OFF is ~30× faster but **changes the labels**
    (over-keeps / different distribution) — do NOT swap it in for work that must match the reasoning-teacher's gold
    (e.g. distillation silver). Keep reasoning ON when fidelity to the gold teacher matters; accept the cost.
+5. **Never expose vLLM's port directly (PS-20).** Sleep mode needs `VLLM_SERVER_DEV_MODE=1`, which adds
+   unauthenticated `/sleep`, `/wake_up` and `/collective_rpc` routes (`--api-key` guards only `/v1`, `/v2`,
+   `/inference`, `/cohere`). The script binds vLLM to 127.0.0.1:8001 and serves `rw_gate.py` on the public port,
+   forwarding only `/v1/*` and `/health`; keep `rw_gate.py` next to the script.
 
 ## Cold start & cost
-- Cold start ~440–480 s (CPU-bound engine init + a 51-graph CUDA-graph capture + first-time weight download), NOT
-  weight I/O — no load trick alone fixes it. GPU-snapshot / sleep-mode is a DEFERRED todo (the one attempt failed
-  because it was CPU-only; GPU retry not done). Run long deploys detached; poll `/health` for readiness.
+- **The server is a class restored from a Modal GPU memory snapshot (PS-20, ADR-0132).** A boot without a snapshot
+  starts vLLM with `--enable-sleep-mode`, warms it, sleeps it at level 1 (weights to host RAM, KV discarded) and
+  Modal snapshots the container; a restore only wakes vLLM (~3.5 s) and is ready ~13 s after its container starts.
+  Modal builds the first few snapshots after a deploy (4-9 min boots; 3-4 in our runs) and now and then another
+  (a new host, a preemption): budget for an occasional slow cold start. Redeploying keeps the snapshots; **`modal app
+  stop` discards them**. Snapshots need TP=1 (the script boots without one at TP>1).
+- Without a snapshot a cold start is ~2-4 min on Config B (114-245 s measured): about 30 s of imports in EACH of
+  vLLM's two processes (transformers' `importlib.metadata.packages_distributions()` stats every installed file),
+  a weight read of 12-154 s from the Volume (9P), then compile-cache load + CUDA-graph capture. Measured and NOT
+  worth adopting (ADR-0109 addendum): `--load-format runai_streamer`, precompiled `.pyc`, `HF_HUB_OFFLINE=1` (it
+  changes vLLM's compile-cache key every container, so each start recompiles). Poll `/health` for readiness.
 - **A100 is billed while up — `modal app stop <app> --yes` the moment you're done.** Verify with `modal app list`
   (state `stopped`, 0 tasks) + endpoint 404.
 - **Do NOT warm the GPU before the consumer is ready, and do NOT trust auto-scaledown to protect billing** (burned

rag_wright-0.7.0/.claude/skills/qwen-vllm-modal/scripts/modal_qwen3_vllm_server.py

@@ -0,0 +1,164 @@
+"""Qwen3.8-27B as an OpenAI-compatible vLLM web server on Modal A100-80GB. Two uses, one script:
+
+(1) QUANTIZATION ACCURACY EVAL (ADR-0110): serve one config under a stable served-name ("qwen3-eval").
+  MODEL=Qwen/Qwen3.8-27B     KV_CACHE_DTYPE=auto uv run --no-sync modal deploy scripts/modal_qwen3_vllm_server.py
+  MODEL=Qwen/Qwen3.8-27B-FP8 KV_CACHE_DTYPE=fp8  uv run --no-sync modal deploy scripts/modal_qwen3_vllm_server.py
+
+(2) 0048 part-2 JOINT RUN -- the UNIFIED endpoint both the product's agent-llm (tool-calling) and the engine
+    drive: Config B (FP8), ALL capabilities on (tool calls + reasoning + guided decoding), served under the
+    name the engine profile `qwen3.8-27b-modal` expects ("Qwen/Qwen3.8-27B"), sized for the measured ~16 peak
+    concurrency of unbounded reasoning outputs (2 GPUs to keep decode from saturating).
+  APP_NAME=rw-qwen3-modal MODEL=Qwen/Qwen3.8-27B-FP8 KV_CACHE_DTYPE=fp8 \
+    SERVED_NAME=Qwen/Qwen3.8-27B TOOL_PARSER=hermes REASONING_PARSER=qwen3 \
+    GPU=A100-80GB:2 TP=2 MAX_NUM_SEQS=32 \
+    uv run --no-sync modal deploy scripts/modal_qwen3_vllm_server.py
+  uv run --no-sync modal app stop <APP_NAME>   # tear down (stop billing)
+
+URL: https://<workspace>--<APP_NAME>-serve.modal.run  (+ /v1 for the OpenAI base). Weights + the serve-path
+torch.compile cache reuse the persistent Volumes (fast warm cold-start where a matching cache exists).
+
+Cold start (PS-20, ADR-0132): at TP=1 the server is restored from a Modal GPU memory snapshot. A boot without one
+starts vLLM, warms it, puts it to sleep (weights parked in host RAM) and Modal snapshots the container; a restore
+only wakes vLLM (~13 s from container start). Modal builds the first 2-3 snapshots per GPU type, and now and then
+another, on ordinary boots; stopping the app discards them. vLLM listens on 127.0.0.1 only: sleep mode needs its
+unauthenticated dev routes, so the public port is `rw_gate.py`, which forwards only /v1/* and /health.
+"""
+
+import os
+import subprocess
+import time
+import urllib.request
+from pathlib import Path
+
+import modal
+
+MODEL = os.environ.get("MODEL", "Qwen/Qwen3.8-27B")          # ...-FP8 for the quantized-weights config
+KV_DTYPE = os.environ.get("KV_CACHE_DTYPE", "auto")          # "auto" (bf16 KV, the reference) | "fp8"
+GPU = os.environ.get("GPU", "A100-80GB:1")
+TP = int(os.environ.get("TP", "1"))
+MAX_LEN = int(os.environ.get("MAX_LEN", "16384"))            # the real workload's context (9K+ prompts)
+GPU_UTIL = os.environ.get("GPU_UTIL", "0.95")
+MAX_NUM_SEQS = os.environ.get("MAX_NUM_SEQS", "256")         # concurrency ceiling; size to the eval's ~16 peak
+SERVED = os.environ.get("SERVED_NAME", "qwen3-eval")         # served-model-name clients ask for (profile match)
+TOOL_PARSER = os.environ.get("TOOL_PARSER", "")             # "" off | "hermes" -> tool calling (the product agent)
+REASONING_PARSER = os.environ.get("REASONING_PARSER", "")   # "" off | "qwen3" -> reasoning coexists with a tool call
+APP_NAME = os.environ.get("APP_NAME", "rw-qwen3-vllm-eval")  # distinct app (=distinct URL) per purpose
+API_KEY = os.environ.get("VLLM_API_KEY")  # vLLM's --api-key: the only gate on the server's public URL
+if not API_KEY:  # no default key (PS-21): a default would be public, and would gate a billing GPU
+    raise SystemExit("set VLLM_API_KEY to the server's API key (a long random string) before deploying")
+SNAPSHOT = TP == 1  # Modal: GPU memory snapshots do not work across several GPUs; TP>1 boots without one
+VLLM_PORT, PUBLIC_PORT = 8001, 8000  # vLLM on loopback; the gate on the public port
+GATE = "/root/rw_gate.py"
+
+HF_CACHE = "/root/.cache/huggingface"
+VLLM_CACHE = "/root/.cache/vllm"
+hf_vol = modal.Volume.from_name("rw-hf-cache", create_if_missing=True)
+vllm_cache_vol = modal.Volume.from_name("rw-vllm-serve-compile-cache", create_if_missing=True)
+
+app = modal.App(APP_NAME)
+vllm_image = (
+    # Base off the OFFICIAL prebuilt vLLM image: vllm + torch + xformers + flashinfer already installed as matched
+    # wheels, so a fresh build on a clean account compiles NOTHING (the xformers source-build / resolver failures
+    # only happen when pip/uv try to assemble that stack themselves). Latest/unpinned by design — `:latest` tracks
+    # newest vLLM, which is all we need since we only consume it. add_python omitted (the image ships Python).
+    # The vLLM image ships `python3` but no `python`; Modal's OWN pip bootstrap runs right after FROM (before any
+    # later layer) as `python -m pip …` -> exit 127. setup_dockerfile_commands runs BEFORE that bootstrap, so the
+    # symlink is in place when Modal bootstraps.
+    modal.Image.from_registry(
+        "vllm/vllm-openai:latest",
+        setup_dockerfile_commands=["RUN ln -sf $(command -v python3) /usr/local/bin/python"],
+    )
+    .entrypoint([])  # the image's ENTRYPOINT (the api_server) would hijack Modal's runtime; clear it
+    .pip_install("hf_transfer")  # enable fast HF weight downloads (HF_HUB_ENABLE_HF_TRANSFER below)
+    # Bake the runtime knobs into the image env: a deployed function reads the CONTAINER's env, not the deploying
+    # shell's, so an override must live here to reach the container (same lesson as the bench script). Xet disabled
+    # so a fresh weight download's open log handle can't block a Volume commit.
+    .env({"HF_HUB_ENABLE_HF_TRANSFER": "1", "HF_HUB_DISABLE_XET": "1", "HF_HOME": HF_CACHE,
+          "MODEL": MODEL, "KV_CACHE_DTYPE": KV_DTYPE, "TP": str(TP), "MAX_LEN": str(MAX_LEN),
+          "GPU_UTIL": GPU_UTIL, "MAX_NUM_SEQS": str(MAX_NUM_SEQS), "SERVED_NAME": SERVED,
+          "TOOL_PARSER": TOOL_PARSER, "REASONING_PARSER": REASONING_PARSER, "VLLM_API_KEY": API_KEY,
+          # Sleep mode's /sleep and /wake_up exist only in dev mode; they stay on loopback (see the gate).
+          "VLLM_SERVER_DEV_MODE": "1", "RW_VLLM_PORT": str(VLLM_PORT), "RW_PUBLIC_PORT": str(PUBLIC_PORT),
+          "TORCHINDUCTOR_COMPILE_THREADS": "1"})  # Modal: torch.compile can otherwise break snapshot creation
+    .add_local_file(Path(__file__).resolve().parent / "rw_gate.py", GATE)  # resolve(): scripts/ holds a symlink
+)
+
+
+def _vllm(path: str, method: str = "GET", body: bytes | None = None, timeout: float = 600) -> int:
+    request = urllib.request.Request(f"http://127.0.0.1:{VLLM_PORT}{path}", data=body, method=method, headers={
+        "Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"})
+    with urllib.request.urlopen(request, timeout=timeout) as response:
+        return response.status
+
+
+def _wait_healthy(proc: subprocess.Popen) -> None:
+    while True:
+        if proc.poll() is not None:
+            raise RuntimeError(f"vLLM exited with {proc.returncode}")
+        try:
+            if _vllm("/health", timeout=5) == 200:
+                return
+        except OSError:
+            pass
+        time.sleep(1)
+
+
+@app.cls(image=vllm_image, gpu=GPU, volumes={HF_CACHE: hf_vol, VLLM_CACHE: vllm_cache_vol},
+         timeout=3600, max_containers=1, scaledown_window=600,
+         memory=65536,  # host RAM for the sleeping weights (~29 GiB for Config B), which the snapshot captures
+         enable_memory_snapshot=SNAPSHOT, experimental_options={"enable_gpu_snapshot": SNAPSHOT})
+# Without this Modal feeds the container ONE HTTP request at a time (Running:1 + "cancellation signal" floods,
+# /health blocked) — vLLM's continuous batching never gets fed. Let many requests hit the one container; vLLM
+# batches them internally up to --max-num-seqs.
+@modal.concurrent(max_inputs=int(MAX_NUM_SEQS))
+class QwenServer:
+    """vLLM's OpenAI-compatible server for the configured Qwen config. Reuses the persisted weights + serve-path
+    torch.compile cache; compiled (production-faithful), optional FP8 KV, optional tool-calling + reasoning parsers so
+    ONE endpoint serves free-text, guided/JSON, tool calls, and reasoning together."""
+
+    @modal.enter(snap=True)
+    def start(self) -> None:
+        """Start vLLM, warm it, and put it to sleep: the snapshot is taken after this (when SNAPSHOT is on)."""
+        hf_vol.reload()
+        vllm_cache_vol.reload()
+        args = [
+            "vllm", "serve", MODEL, "--host", "127.0.0.1", "--port", str(VLLM_PORT), "--served-model-name", SERVED,
+            "--tensor-parallel-size", str(TP), "--dtype", "bfloat16", "--max-model-len", str(MAX_LEN),
+            "--gpu-memory-utilization", GPU_UTIL, "--max-num-seqs", str(MAX_NUM_SEQS),
+            "--api-key", API_KEY, "--trust-remote-code", "--enable-sleep-mode",
+        ]
+        if KV_DTYPE != "auto":
+            args += ["--kv-cache-dtype", KV_DTYPE]
+        if TOOL_PARSER:  # the product's deep-agent orchestrator is a tool-caller (RuleWright Q1/Q3)
+            args += ["--enable-auto-tool-choice", "--tool-call-parser", TOOL_PARSER]
+        if REASONING_PARSER:  # so reasoning coexists with a forced tool/schema instead of colliding
+            args += ["--reasoning-parser", REASONING_PARSER]
+        print(f"[qwen] serving {MODEL} as '{SERVED}' | TP={TP} max_len={MAX_LEN} util={GPU_UTIL} kv={KV_DTYPE} "
+              f"max_seqs={MAX_NUM_SEQS} tool={TOOL_PARSER or '-'} reasoning={REASONING_PARSER or '-'} "
+              f"snapshot={SNAPSHOT}", flush=True)
+        self.vllm = subprocess.Popen(args)
+        _wait_healthy(self.vllm)
+        warm = (b'{"model": "%s", "messages": [{"role": "user", "content": "Say OK."}], "max_tokens": 8, '
+                b'"chat_template_kwargs": {"enable_thinking": false}}' % SERVED.encode())
+        for _ in range(2):
+            _vllm("/v1/chat/completions", "POST", warm)
+        _vllm("/sleep?level=1", "POST", b"")  # level 1: weights to host RAM (in the snapshot), KV cache discarded
+        print("[qwen] warmed and asleep; snapshot next", flush=True)
+
+    @modal.enter(snap=False)
+    def wake(self) -> None:
+        """After a restore (or straight after `start` without one): wake vLLM, then open the public port."""
+        _vllm("/wake_up", "POST", b"")
+        _wait_healthy(self.vllm)
+        self.gate = subprocess.Popen(["python", GATE])
+        print("[qwen] awake; gate open", flush=True)
+
+    @modal.web_server(port=PUBLIC_PORT, startup_timeout=1200, label=f"{APP_NAME}-serve")
+    def serve(self) -> None:
+        """The public URL: the gate, which forwards /v1/* and /health to vLLM."""
+
+    @modal.exit()
+    def stop(self) -> None:
+        for proc in (getattr(self, "gate", None), getattr(self, "vllm", None)):
+            if proc:
+                proc.terminate()

rag_wright-0.7.0/.claude/skills/qwen-vllm-modal/scripts/rw_gate.py

@@ -0,0 +1,58 @@
+"""The model server's public front (PS-20): forwards only the OpenAI API (`/v1/...`) and `/health` to vLLM.
+
+vLLM's `--api-key` guards only `/v1`, `/v2`, `/inference` and `/cohere`. Sleep mode, which the GPU snapshot needs,
+requires `VLLM_SERVER_DEV_MODE=1`, and that adds unauthenticated `/sleep`, `/wake_up` and `/collective_rpc` routes. So
+vLLM listens on 127.0.0.1 only and this gate is what the public URL reaches. It runs inside the server container
+(`modal_qwen3_vllm_server.py` copies it into the image); responses stream through as they arrive.
+"""
+import os
+
+UPSTREAM = f"http://127.0.0.1:{os.environ.get('RW_VLLM_PORT', '8001')}"
+_HOP_BY_HOP = {"connection", "keep-alive", "transfer-encoding", "upgrade", "proxy-connection", "te", "trailer",
+               "host", "content-length"}
+
+
+def allowed(path: str) -> bool:
+    """The routes the public URL may reach: the OpenAI API and the health check."""
+    return path == "/health" or path.startswith("/v1/")
+
+
+async def _forward(request):
+    from aiohttp import web
+
+    if not allowed(request.path):
+        return web.json_response({"error": "Not Found"}, status=404)
+    headers = {k: v for k, v in request.headers.items() if k.lower() not in _HOP_BY_HOP}
+    async with request.app["session"].request(request.method, UPSTREAM + request.path_qs, headers=headers,
+                                              data=await request.read()) as upstream:
+        response = web.StreamResponse(status=upstream.status, headers={
+            k: v for k, v in upstream.headers.items() if k.lower() not in _HOP_BY_HOP})
+        await response.prepare(request)
+        async for chunk in upstream.content.iter_any():  # streamed completions pass through as they arrive
+            await response.write(chunk)
+        await response.write_eof()
+        return response
+
+
+async def _app():
+    from aiohttp import ClientSession, ClientTimeout, TCPConnector, web
+
+    app = web.Application(client_max_size=64 * 1024 * 1024)
+
+    async def open_session(app):
+        app["session"] = ClientSession(connector=TCPConnector(limit=0), timeout=ClientTimeout(total=None))
+
+    async def close_session(app):
+        await app["session"].close()
+
+    app.on_startup.append(open_session)
+    app.on_cleanup.append(close_session)
+    app.router.add_route("*", "/{tail:.*}", _forward)
+    return app
+
+
+if __name__ == "__main__":
+    from aiohttp import web
+
+    web.run_app(_app(), host=os.environ.get("RW_GATE_HOST", "0.0.0.0"),
+                port=int(os.environ.get("RW_PUBLIC_PORT", "8000")), print=None)

rag_wright-0.7.0/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "0.7.0"
+}

{rag_wright-0.6.0 → rag_wright-0.7.0}/CHANGELOG.md

@@ -4,6 +4,14 @@ All notable changes to RAG_Wright are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project aims to follow
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.7.0](https://github.com/fzaidi-dai/RAG_Wright/compare/0.6.0...0.7.0) (2026-10-11)
+
+
+### Features
+
+* **PS-20:** the model server cold-starts from a Modal GPU memory snapshot behind a public-route gate (ADR-0132) ([1305237](https://github.com/fzaidi-dai/RAG_Wright/commit/13052378c57f31d126d281cda49c614ca9e7207b))
+* **PS-22:** the model-call deadline follows the configured timeout; a profile caps thinking models' structured output (ADR-0133) ([d327813](https://github.com/fzaidi-dai/RAG_Wright/commit/d3278130aaae10e7af3c326e503612337e7336cb))
+
 ## [0.6.0](https://github.com/fzaidi-dai/RAG_Wright/compare/0.5.1...0.6.0) (2026-10-10)
 
 

{rag_wright-0.6.0 → rag_wright-0.7.0}/CLAUDE.md

@@ -55,7 +55,7 @@ The failure mode to avoid, which has happened before: a command fails with "modu
 
 ## Start of every session
 
-> **CURRENT STATE (2026-10-09): the public-surface workstream is COMPLETE and released as `rag-wright` 0.5.0** (ledger
+> **CURRENT STATE (2026-10-11): the public-surface workstream is COMPLETE and released as `rag-wright` 0.6.0** (ledger
 > `docs/specs/public-surface/plan.md`, ADR-0128: two public tiers, `rag_wright.api` for a product's seam and
 > `rag_wright.pack_sdk` for a pack's code, enforced by import contracts and a domain-vocabulary guard). Next:
 > RuleWright moves from `docs/specs/public-surface/rulewright-migration-0.3.0.md` then `rulewright-migration-0.4.0.md`, `-0.5.0.md` and `-0.6.0.md`; PS-R4 (the unit-labelling eval) is done, unreleased. Before it,

{rag_wright-0.6.0 → rag_wright-0.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: rag-wright
-Version: 0.6.0
+Version: 0.7.0
 Summary: A domain-retargetable, open-core engine for hybrid retrieval + knowledge-graph RAG: parsing, chunking, embedding, hybrid search, reranking, graph extraction, entity resolution, ontology-driven knowledge, and a capability runtime.
 Project-URL: Homepage, https://github.com/fzaidi-dai/RAG_Wright
 Project-URL: Repository, https://github.com/fzaidi-dai/RAG_Wright

{rag_wright-0.6.0 → rag_wright-0.7.0}/docs/adr/0109-vllm-cold-start-reduction.md

@@ -45,6 +45,44 @@ Plausible target after the snapshot is built: **~10–20s cold restart**, well u
 - **Fallback if the alpha proves unstable:** the §4 supporting trims alone do NOT reach sub-60s (per the reframe), so the fallback for a hard sub-60s SLA reverts to keep-one-warm (the ruled-out cost) until the snapshot path matures.
 - No engine code changes here; this is deployment/serving infrastructure for the product substrate (ADR-0039). This ADR sets direction and diagnosis; a follow-up will record the prototype's measured result.
 
+## Addendum (2026-10-11): `runai_streamer` measured on Modal, not adopted
+
+The §4 trim `--load-format runai_streamer` was measured on the locked Config B (ADR-0110: `Qwen/Qwen3.8-27B-FP8`,
+fp8 KV, 1×A100-80GB, TP=1, 16K) with the weights cached on the `rw-hf-cache` Volume and the AOT compile cache hit in
+every run. It is already in the `vllm/vllm-openai` image (vLLM's Dockerfile installs `runai-model-streamer`), and it
+streams from the Hugging Face cache when given a repo id. Seven cold starts, timed from container start:
+
+| Loader | Weight read (s) | "Model loading" (s) | Container start → ready (s) |
+|---|---|---|---|
+| default | 40.6 / 26.5 / 11.6 | 46.8 / 36.8 / 17.0 | 146 / 245 / 114 |
+| `runai_streamer`, concurrency 16 | 30 / 31 / 56 | 36.9 / 61.9 / 64.8 | 154 / 167 / 185 |
+| `runai_streamer`, concurrency 32 | 45 | 57.0 | 218 |
+
+**Decision: keep the default loader.** Means: weight read 26 s (default) against 39 s (streamer at 16); start to
+ready 168 s against 169 s. On a Modal Volume the read rate swings 0.76 to 2.7 GB/s with the default loader alone,
+depending on whether the files are already cached, which is larger than any loader effect; the streamer's
+published gains (2.4× on local NVMe, up to 6× from object storage) do not transfer. The streamer's "Model loading"
+also carries ~30 s beyond its read. The weight read is 10-30% of a cold start; the remainder is process start
+(~70-120 s before the load begins) and engine init (~30-50 s), which confirms the reframe above: the cold-start
+work is the snapshot path, not the loader.
+
+Two corrections to figures above. Config B's weights are **30.9 GB on disk (28.9 GiB in VRAM)**, not ~27 GB. The
+current cold start with weights and compile cache warm is **~2-4 minutes** (114-245 s here), not the ~481 s
+measured for the bf16 model in September.
+
+## Addendum (2026-10-11): where the non-snapshot cold start goes, and the outcome
+
+Timed from the server function's start in three default-loader cold starts (146 / 245 / 114 s): the API server
+process reaching its arguments 30 / 80 / 31 s; the engine process initialised 31 / 70 / 29 s later; model loading
+47 / 37 / 17 s; compile-cache load and CUDA-graph capture 31 / 45 / 31 s. The first two are imports: on a CPU
+container the API server's import takes 37 s cold and 28-29 s warm, the same with every module precompiled to
+bytecode (so not compilation). The largest single cost is transformers' `import_utils`, whose import calls
+`importlib.metadata.packages_distributions()` (it stats every file of every installed package) and reads its own
+model modules (21,413 `stat` calls in one `import transformers`). `HF_HUB_OFFLINE=1` removes the Hub lookups but
+resolves the model to a local snapshot path, which changed vLLM's compile-cache key in each container (a 73-104 s
+recompile per start), so it is not adopted. **Outcome: the snapshot path (decision 2-3 above) is adopted; see
+ADR-0132.**
+
 ### Sources
 - Modal GPU snapshots (alpha, what's captured): https://modal.com/blog/gpu-mem-snapshots ; caveats (multi-GPU incompat, storage-bound caveat, torch.compile): https://modal.com/docs/guide/memory-snapshots ; vLLM example: https://modal.com/docs/examples/lfm_snapshot , https://modal.com/docs/examples/gpu_snapshot
 - Cold-start is CPU-bound; Tensorizer fastest: https://arxiv.org/abs/2606.07362 ; 35B breakdown: https://route179.dev/2026/06/30/optimizing-vllm-cold-start-with-model-streaming-and-compile-caching/
@@ -52,3 +90,4 @@ Plausible target after the snapshot is built: **~10–20s cold restart**, well u
 - vLLM loaders: runai_streamer https://docs.vllm.ai/en/stable/models/extensions/runai_model_streamer/ ; Tensorizer https://docs.vllm.ai/en/v0.10.1/models/extensions/tensorizer.html
 - vLLM CUDA graphs / capture cost: https://docs.vllm.ai/en/stable/design/cuda_graphs/
 - Modal Volume 1–2 GB/s, keep-warm knobs: https://modal.com/docs/guide/high-performance-llm-inference , https://modal.com/docs/guide/cold-start , https://modal.com/docs/guide/scale
+- vLLM loaders (cont.): runai_streamer source https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/model_loader/runai_streamer_loader.py ; route179 35B breakdown (2.4× on NVMe) above

rag_wright-0.7.0/docs/adr/0132-model-server-cold-start-from-a-gpu-snapshot.md

@@ -0,0 +1,45 @@
+# ADR-0132: The model server cold-starts from a Modal GPU memory snapshot, behind a public-route gate
+
+**Status:** accepted · **Date:** 2026-10-11 · **Related:** ADR-0109 (cold-start reduction: direction and the
+measurements behind it), ADR-0110 (Config B), ADR-0130/0131 (the model-server API), PS-20
+
+## Context
+
+A cold start of the self-hosted Qwen server (Config B on one A100) took 114-245 s, sometimes more, every time a
+scaled-down server got a request. Measured on 2026-10-11 (ADR-0109 addenda): about 30 s of imports in each of vLLM's
+two processes, a 12-154 s weight read from the Volume, then compile-cache load and CUDA-graph capture. The levers
+that do not remove startup were measured and rejected: the Run:ai streamer, precompiled bytecode, and
+`HF_HUB_OFFLINE=1` (it changes vLLM's compile-cache key in every container, so each start recompiles).
+
+## Decision
+
+1. **The server is a Modal class restored from a GPU memory snapshot** (`enable_memory_snapshot` +
+   `experimental_options={"enable_gpu_snapshot": True}`, alpha). `@modal.enter(snap=True)` starts vLLM with
+   `--enable-sleep-mode`, warms it with two requests and sleeps it at level 1 (weights to host RAM, ~29 GiB, KV
+   discarded; the class requests 64 GiB of memory); Modal snapshots the container. `@modal.enter(snap=False)` wakes
+   vLLM. `TORCHINDUCTOR_COMPILE_THREADS=1` as Modal requires. Snapshots only at TP=1 (Modal: incompatible with
+   several GPUs); TP>1 boots without one.
+2. **vLLM listens on 127.0.0.1 only; the public port is `rw_gate.py`**, which forwards `/v1/*` and `/health` and
+   returns 404 for everything else. Sleep mode needs `VLLM_SERVER_DEV_MODE=1`, which adds unauthenticated `/sleep`,
+   `/wake_up` and `/collective_rpc` routes, and vLLM's `--api-key` guards only `/v1`, `/v2`, `/inference` and
+   `/cohere` (v0.30.0 `serve/middleware/authenticate.py`). Modal's published vLLM snapshot examples expose those
+   routes publicly; we do not.
+3. **The URL does not change**: `@modal.web_server(..., label=f"{APP_NAME}-serve")` keeps
+   `https://<workspace>--<app>-serve.modal.run`. The engine looks the server up as
+   `modal.Cls.from_name(name, "QwenServer")().serve` (the URL and the container count; Modal's public
+   `Function.from_name` refuses a class's `<Class>.*` name). The public API is unchanged.
+
+## Consequences
+
+- **A restore is ready ~13 s after its container starts** (wake ~3.5 s); 21-67 s end to end in our runs, including
+  getting a container. One restore took ~112 s (Modal loading the snapshot slowly).
+- **Modal decides per container whether to restore or build**, and a build is a full boot plus the snapshot: 4-9
+  minutes. It built 3-4 snapshots after each deploy in our runs (one after a GPU preemption) and another later in
+  one run. So a scaled-to-zero server usually answers in seconds and occasionally makes a request wait minutes; a
+  product that cannot accept that keeps the server warm during working hours.
+- **Redeploying keeps the snapshots; stopping the app discards them**, so the next deploy pays the builds again.
+- **Alpha-feature risk** stays (Modal labels GPU snapshots alpha). If a restore fails, Modal boots the container in
+  full, which is the pre-snapshot behaviour.
+- Live gate (engine API only): deploy, a question metered on the server's model id, status while up, public
+  `/sleep` and `/collective_rpc` 404, three cold starts, a redeploy then a restored cold start (67 s), stop, status
+  not deployed. Prototype runs: restores in 4 of 11 cold starts across two deployments, all after the first builds.

rag_wright-0.7.0/docs/adr/0133-model-call-deadline-follows-the-timeout-and-a-thinking-cap.md

@@ -0,0 +1,40 @@
+# ADR-0133: The model-call deadline follows the configured timeout, and a thinking model's structured output is capped by its profile
+
+**Status:** accepted · **Date:** 2026-10-11 · **Related:** ADR-0056/0057 (one bounded retry layer; the async
+wall-clock deadline), ADR-0110 (the timeout trap on reasoning-on calls), PS-22
+
+## Context
+
+The 2026-10-11 sampling A/B (ContractNLI, 150 pairs, thinking on, Qwen on our A100) found two things:
+
+1. The async seam cancelled every logical call at a fixed 180 s (`_MODEL_DEADLINE_S`), whatever
+   `RAG_STRUCTURED_TIMEOUT_S` said. The docs and the qwen skill tell people to raise that per-try timeout for
+   reasoning-on work, and past 180 s in total it did nothing. A loop and a slow healthy call both ended as
+   `ModelCallTimeout`, and the ContractNLI eval scored that as "neutral".
+2. Reasoning loops are real and random: one pair ran to 8,192 tokens in 1 of 3 tries at Qwen's recommended
+   sampling, and correct verdicts took up to ~7k tokens. Qwen's recommended sampling did not remove loops (greedy
+   stays the default).
+
+## Decision
+
+1. **`model_deadline_s()`** is `RAG_MODEL_DEADLINE_S` when set; otherwise the per-try timeout times the tries plus
+   the largest backoffs, never under 180 s. Every async seam call runs under it, and `ModelCallTimeout` names both
+   settings.
+2. **`ModelProfile.structured_max_tokens`** is the default `max_tokens` for a forced structured call (a caller's own
+   value wins). The four thinking-on Qwen profiles set 8,192, so a loop ends at `finish_reason=length`, a parse
+   failure the bounded retry handles, rather than at the deadline. A server deployed from the engine inherits it.
+3. **The cap never fails a call that worked without it.** vLLM rejects a request whose prompt plus `max_tokens`
+   exceeds the context with a 400 naming `input_tokens` (`vllm/renderers/params.py`), so that error is answered by
+   sending the call once more without the cap. A caller's own `max_tokens` keeps today's behaviour (no retry).
+4. `eval/contractnli_judge.py` reports failed calls as errors, never as a verdict.
+
+Rejected: vLLM's `thinking_token_budget` (needs `--reasoning-config` on the server and is not supported by vLLM's V2
+model runner yet, and it is vLLM-only), and higher sampling temperature as a loop fix (measured, above).
+
+## Consequences
+
+- Raising `RAG_STRUCTURED_TIMEOUT_S` now raises the total budget too; the default is ~182 s (was 180 s).
+- A looping thinking call costs at most 8,192 output tokens per try instead of running until cancelled.
+- Free-text calls are not capped by the profile (thinking is off on that path in every Qwen profile).
+- RuleWright: callers that turn a failed call into a result should treat `ModelCallTimeout` as an error, not a
+  verdict (`rulewright-migration-0.7.0.md`).

{rag_wright-0.6.0 → rag_wright-0.7.0}/docs/adr/README.md

@@ -1,7 +1,7 @@
 # Architecture Decision Records — index
 
-129 ADR files (0001–0119, plus 0120 — the former duplicate `0015`, renumbered 2026-10-05 to repair the collision —
-and 0121–0131). ADRs are an
+131 ADR files (0001–0119, plus 0120 — the former duplicate `0015`, renumbered 2026-10-05 to repair the collision —
+and 0121–0133). ADRs are an
 immutable, cross-referenced decision log: nothing here is moved or renumbered — legacy records carry a `Status:`
 banner pointing to what replaced them. This index groups them so a newcomer isn't drowned.
 
@@ -187,4 +187,6 @@ framing that still appears in older ADRs and the root `SPEC.md`/`plan.md`.
 0128 Two public tiers: `rag_wright.api` for products, `rag_wright.pack_sdk` for packs ·
 0129 The shipped Laya temperatures stay as fitted; scikit-learn floor at 1.9.1 ·
 0130 A product deploys the self-hosted model server from the engine and uses it like any model ·
-0131 No default server key anywhere; a model's concurrency ceiling travels with its id
+0131 No default server key anywhere; a model's concurrency ceiling travels with its id ·
+0132 The model server cold-starts from a Modal GPU memory snapshot, behind a public-route gate ·
+0133 The model-call deadline follows the configured timeout; a thinking model's structured output is capped by its profile

{rag_wright-0.6.0 → rag_wright-0.7.0}/docs/api/README.md

@@ -46,7 +46,7 @@ No deployed model server has this name (never deployed, or stopped).
 
 ### `ModelServerStatus(name: 'str', deployed: 'bool', running: 'bool', healthy: 'bool', weights_cached: 'bool') -> None`
 
-What `amodel_server_status` found. `deployed`: the Modal app exists. `running`: a container is up (the GPU is billing); False when scaled to zero after idling. `healthy`: a running container answers `/health` (False while it starts). `weights_cached`: the spec's model weights are in the weights Volume, so a deploy or cold start skips the ~27 GB download (an empty `modal app list` says nothing about this).
+What `amodel_server_status` found. `deployed`: the Modal app exists. `running`: a container is up (the GPU is billing); False when scaled to zero after idling. `healthy`: a running container answers `/health` (False while it starts). `weights_cached`: the spec's model weights are in the weights Volume, so a deploy or cold start skips the ~31 GB download (an empty `modal app list` says nothing about this).
 
 ### `ModelRole(*values)`
 
@@ -260,7 +260,7 @@ Resolve every model role through `ws`'s `EngineConfig.models` inside the block (
 
 ### `adeploy_model_server(spec: 'ModelServerSpec', *, wait: 'bool' = True, timeout_s: 'float' = 1500, poll_s: 'float' = 15, progress: 'Callable[[str], object]' = print) -> 'ModelServer'`
 
-Deploy (or redeploy) the server described by `spec` on Modal, register its model profile, and (with `wait`) wait for it to be ready. Returns the `ModelServer`; use `server.model_id` as the model. Raises `RuntimeError` when the API-key variable is unset (nothing is deployed). The server runs (and bills) until `stop_model_server`, or until Modal scales it down after 10 idle minutes.
+Deploy (or redeploy) the server described by `spec` on Modal, register its model profile, and (with `wait`) wait for it to be ready. Returns the `ModelServer`; use `server.model_id` as the model. Raises `RuntimeError` when the API-key variable is unset (nothing is deployed). The server runs (and bills) until `stop_model_server`, or until Modal scales it down after 10 idle minutes. Stopping discards the server's snapshots, so the next deploy pays snapshot-building boots again (ADR-0132).
 
 ### `amodel_server(spec: 'ModelServerSpec') -> 'ModelServer'`
 
@@ -268,7 +268,7 @@ Reconnect to a server deployed earlier (by this process or another) without rede
 
 ### `await_model_server(server: 'ModelServer', *, timeout_s: 'float' = 1500, poll_s: 'float' = 15, progress: 'Callable[[str], object]' = print) -> 'ModelServer'`
 
-Wait until the server answers `/health` (a cold start is about 8 minutes, ADR-0109); progress as `i/N`. Raises `TimeoutError` after `timeout_s`.
+Wait until the server answers `/health`; progress as `i/N`. A cold start restored from the server's snapshot is ready ~13 s after its container starts; a boot that builds a snapshot (the first few, and now and then another) takes 4-9 minutes (ADR-0132). Raises `TimeoutError` after `timeout_s`.
 
 ### `stop_model_server(name: 'str') -> 'None'`
 

{rag_wright-0.6.0 → rag_wright-0.7.0}/docs/api/pack_sdk.md

@@ -70,7 +70,7 @@ Self-loop is rejected here only at the ref level (the same mention as both sourc
 
 ### `ModelCallTimeout`
 
-A logical model call exceeded the total wall-clock deadline (`_MODEL_DEADLINE_S`) and was truly cancelled (socket torn down). TERMINAL: a stalling peer is not a transient worth re-hitting, so this is deliberately NOT in `_STRUCTURED_RETRY_ON` and must be kept out of any pregel `retry_on` -- the caller degrades or dead-letters on it. It is the async fix a per-socket-op timeout cannot be (engine issue 0003 / ADR-0057).
+A logical model call exceeded the total wall-clock deadline (`model_deadline_s()`) and was truly cancelled (socket torn down). TERMINAL: a stalling peer is not a transient worth re-hitting, so this is deliberately NOT in `_STRUCTURED_RETRY_ON` and must be kept out of any pregel `retry_on` -- the caller degrades or dead-letters on it. It is the async fix a per-socket-op timeout cannot be (engine issue 0003 / ADR-0057).
 
 ### `TransientExtraction`
 
@@ -310,7 +310,7 @@ A structured-output runnable for `model_id`, driven by its profile.
 
 The profile supplies the method and the optional structured-only `extra_body`; the `extra_body` is bound to this forced structured call only. This is the sole path to `with_structured_output`. The runnable is wrapped in a SINGLE bounded retry layer (`_with_bounded_retry`); the SDK's own retry loop is disabled here (max_retries=0) so the two do not stack into a ~36 min worst case (engine issue 0003 / ADR-0056).
 
-`temperature` defaults to 0 (deterministic-intent); a caller doing best-of-N self-consistency raises it to sample GENUINELY diverse structured completions (the base client's temperature, not a provider flag). `max_tokens` caps the completion length -- a safety net against a model that runs away to the context limit under a schema constraint (observed on self-hosted Gemma-4 with a mis-set chat template).
+`temperature` defaults to 0 (deterministic-intent); a caller doing best-of-N self-consistency raises it to sample GENUINELY diverse structured completions (the base client's temperature, not a provider flag). `max_tokens` caps the completion length -- a safety net against a model that runs away to the context limit under a schema constraint (observed on self-hosted Gemma-4 with a mis-set chat template). Left None, the profile's `structured_max_tokens` applies (PS-22: a thinking model's reasoning-loop guard), and a prompt too long to leave room for that cap is sent again without it.
 
 ### `build_tag_structured(model_id: 'str', schema: 'type[BaseModel]', *, include_raw: 'bool' = False, temperature: 'float' = 0.0, max_tokens: 'int | None' = 2048, retries: 'int' = 1, label: 'str | None' = None, fields: 'set[str] | None' = None) -> '_TagStructuredRunnable'`
 
@@ -332,6 +332,8 @@ ADR-0100: resolve a model STRING to its access (backend + base_url + key + the i
 
 PS-8b: the total wall-clock deadline (seconds) for one logical model call, across its bounded retries -- the value every seam call runs under (read at call time, so a test or configuration override is honoured).
 
+PS-22: `RAG_MODEL_DEADLINE_S` when set; otherwise long enough for every try at the per-try timeout plus the largest backoffs between them, and never under 180 s.
+
 ### `call_description(model_id: 'str', label: 'str | None') -> 'str'`
 
 The model-call description used in the deadline/retry warnings + the timeout message. ADR-0058 side-fix (issue 0004): include the STAGE/call-site (`label`) when the caller supplies it, so a timeout names WHICH stage was cancelled (e.g. `granite-4.2-8b for semantic_chunking.discover`), not just the model.

{rag_wright-0.6.0 → rag_wright-0.7.0}/docs/configuration.md

@@ -175,7 +175,7 @@ so the GPU is billing), `healthy` and `weights_cached`. It asks `/health` only o
 request to a scaled-down server starts a cold start. `weights_cached` is what decides a deploy's cost: the weights
 live in the Modal Volume `rw-hf-cache` (the vLLM compile cache in `rw-vllm-serve-compile-cache`), and stay there when
 the app is stopped, so an empty `modal app list` does not mean a cold cache. With the weights cached, a deploy is a
-short image build and a cold start; without them, it also downloads about 27 GB. `astop_model_server` is
+short image build and a cold start; without them, it also downloads about 31 GB. `astop_model_server` is
 `stop_model_server` for async code.
 
 **Concurrency: the engine's fan-out multiplies with yours.** `max_num_seqs` is the most requests the server batches
@@ -194,13 +194,26 @@ model the workspace resolves. It cannot see your other parallel runs: keep their
 
 **What to know before you rely on it.**
 
-- **Cold start.** A new or scaled-down server takes minutes to become ready (226 s in the last live run; up to about
-  8 minutes, ADR-0109). After 10 idle minutes Modal scales the container to zero (no GPU billing) but keeps the app
-  deployed; the next request waits through a cold start, so call `await_model_server` before sending traffic.
-- **Cost.** An A100 bills while a container runs. `stop_model_server("acme-qwen")` removes the app entirely; deploy
-  again to bring it back.
+- **Cold start.** After 10 idle minutes Modal scales the container to zero (no GPU billing) but keeps the app
+  deployed; the next request waits through a cold start, so call `await_model_server` before sending traffic. The
+  server is restored from a Modal GPU memory snapshot (ADR-0132): a restore is ready about 13 s after its container
+  starts (67 s end to end in the live run, including getting a container). Modal decides per container whether to
+  restore or to boot in full and build a new snapshot (4 to 9 minutes): it builds several after a deploy (four in a
+  row in the live run, one of them after a GPU preemption), and now and then another later. Redeploying a server
+  keeps its snapshots; stopping it discards them. So a scaled-down server usually answers in seconds but can make
+  a request wait minutes; where that is not acceptable, keep the server busy or warm rather than relying on the
+  snapshot.
+- **Cost.** An A100 bills while a container runs. `stop_model_server("acme-qwen")` removes the app entirely, and its
+  snapshots with it; deploy again to bring it back (and pay the snapshot-building boots again).
 - **Timeouts.** Reasoning-on structured calls under full load can take about a minute; raise
-  `RAG_STRUCTURED_TIMEOUT_S` (default 60) for bulk work, or calls time out and retry.
+  `RAG_STRUCTURED_TIMEOUT_S` (default 60) for bulk work, or calls time out and retry. The total time for one call,
+  across its retries, follows that setting (three tries plus backoff, never under 180 s); set
+  `RAG_MODEL_DEADLINE_S` to choose it yourself. A call that runs past it raises `ModelCallTimeout`: count it as a
+  failed call, not as an answer.
+- **Reasoning loops.** A thinking model now and then reasons in circles. Its profile caps a structured call's
+  output (`ModelProfile.structured_max_tokens`, 8,192 on the thinking Qwen profiles), so a loop ends as a failed
+  parse that the seam retries; a prompt too long to leave room for the cap is sent again without it. A caller's
+  own `max_tokens` takes precedence. Keep temperature 0: Qwen's recommended sampling did not remove loops (ADR-0133).
 - **Process-wide.** A registered server model id is known to the whole process, like every model profile; after a
   restart, call `amodel_server` again.
 
@@ -226,6 +239,7 @@ Prefer config; use env for secrets and for non-API callers. The common ones the
 | `CLASSIFY_CONCURRENCY` / `CLAUSE_CONCURRENCY` / `RAG_INGEST_AFFILIATIONS` / `RAG_FUNCTION_CLASSIFIER` | contracts-pack ingest knobs (mirror `ContractIngestOptions`) |
 | `RAG_INGEST_LIST_MODEL` / `RAG_INGEST_CLAUSE_SAMPLES` / `RAG_INGEST_CLAUSE_EXTRACTOR` | read only by the contracts pack's legacy tag-parse clause extractor, which the default pipeline does not use (it is classifier-first); its second list model runs only when `RAG_INGEST_LIST_MODEL` names one (no default model) |
 | `RAG_STRUCTURED_TIMEOUT_S` / `RAG_JEV_TIMEOUT_S` / `RAG_JUDGE_TIMEOUT_S` / `RAG_RELEVANCE_TIMEOUT_S` | call timeouts (raise for reasoning-ON bulk work) |
+| `RAG_MODEL_DEADLINE_S` | total wall-clock limit for one model call across its retries (default: follows `RAG_STRUCTURED_TIMEOUT_S`, never under 180 s) |
 | `RAG_SPACY_MODEL` | the spaCy model name for the optional NER extra |
 | `RAG_MODELS_DIR` | the models root every trained classifier loads its weights from (default: the engine checkout's `data/models` when it exists, else `./data/models`); fill it with `scripts/fetch_reference_models.py` for the reference pack |
 | `EMBED_DEVICE` / `RAG_SETFIT_DEVICE` / `RAG_SETFIT_CLAUSE_DIR` / `RAG_SETFIT_THRESHOLD` / `RAG_SETFIT_TOPK` | embedder / classifier device + fleet knobs (`RAG_SETFIT_CLAUSE_DIR` overrides `<models root>/setfit_clause`) |