lobes-cli 0.89.2__tar.gz → 0.90.0__tar.gz

lobes_cli-0.90.0/.devague/deliveries/vllm-0-31-0-fleet.json

@@ -0,0 +1,100 @@
+{
+  "plan_slug": "vllm-0-31-0-fleet",
+  "schema_version": 2,
+  "created": "2026-10-17T20:53:11Z",
+  "updated": "2026-10-18T04:52:16Z",
+  "deviations": [
+    {
+      "id": "d1",
+      "what": "Thor worker: recreate once with --mm-encoder-attn-backend=TORCH_SDPA (a serve flag the plan never named) to test whether the vision regression on vLLM 0.31.0 / sm_110 is the FLASH_ATTN ViT path; if it fixes the colour controls, the flag becomes a Thor worker knob carried into the template (R2); if not, fall back to rolling back to the nightly or withdrawing the image advert",
+      "task_ref": "t8",
+      "reason": "t8 acceptance found the worker on 0.31.0 misreading solid-colour images (red/blue -> Black/White) through the gateway and on the lane, while the same checkpoint was correct on the nightly 2026-09-10; worker log shows FLASH_ATTN chosen for the ViT on sm_110 (docs/evidence/2026-10-17-accept-vllm-031-thor.txt)",
+      "affects": [
+        "t8",
+        "t14",
+        "t17"
+      ],
+      "origin": "user",
+      "status": "approved",
+      "classification": "risky",
+      "seq": 1
+    },
+    {
+      "id": "d2",
+      "what": "Graded governor on the Spark cortex: the target container gains explicit GPU device nodes (devices: /dev/nvidia0, /dev/nvidiactl, /dev/nvidia-modeset, /dev/nvidia-uvm, /dev/nvidia-uvm-tools) in docker-compose.override.yml so docker update no longer revokes GPU access, and the governor runs with bands raised for this Spark's acpitz zones (cap 12 cpus at >=92 C, 8 at >=95 C, 4 at >=98 C, pause at >=101 C) instead of the 84/88/91/93.5 defaults",
+      "task_ref": "t12",
+      "reason": "The governor's first docker update killed the cortex lane's first boot with NVMLError_Unknown (systemd cgroup driver revokes NEW GPU access on docker update; reproduced on a throwaway container; explicit --device entries verified to survive docker update --cpus 8/4/12/16), and this Spark's acpitz zones idle ~88 C and reach 92 C under one benchmark stream, so the default bands would throttle at idle (docs/evidence/2026-10-17-accept-vllm-031-spark-cortex.txt)",
+      "affects": [
+        "t12",
+        "t4",
+        "t14"
+      ],
+      "origin": "user",
+      "status": "approved",
+      "classification": "risky",
+      "seq": 2
+    },
+    {
+      "id": "d3",
+      "what": "Split engine pin: Thor and Orin stay on vllm/vllm-openai:v0.31.0-ubuntu2404; the Spark's cortex is tested on v0.30.0-ubuntu2404 (index sha256:439c19d48db36401abc914b9842d060fe610a54bc3ac29bb02505f0b69af1baf) and spark2 does not move to 0.31.0 until strict+thinking is clean there",
+      "task_ref": "t12",
+      "reason": "On 0.31.0 (xgrammar 0.2.7, both the plain and -ubuntu2404 variants) a strict tool call WITH thinking on the cortex lane emits 78 repeated calls until max_tokens; the nightly (xgrammar 0.2.3) emits one call. The structural-tag grammar is identical across engines, so the runtime enforcement regressed. Only the cortex plugin's reasoning-aware grammar path is affected: the Thor worker (plain qwen3_coder) returned one call, and the Orin has no tool-calling lane",
+      "affects": [
+        "t12",
+        "t13",
+        "t14",
+        "t17"
+      ],
+      "origin": "user",
+      "status": "approved",
+      "classification": "risky",
+      "seq": 3
+    },
+    {
+      "id": "d4",
+      "what": "The Spark's cortex is tested on the GB10-native third-party build ghcr.io/spark-arena/dgx-vllm-eugr-nightly@sha256:cc47adc1b210066dca421bdcd7ca6f16680b6579533f348c0d8c072da194cf98 (vLLM 0.32.0rc1.dev301+g1ac942555, TORCH_CUDA_ARCH_LIST=12.1a, built 2026-10-16), the engine sparkrun's official Qwen3.8-27B recipe uses; if strict+thinking is clean there, the Spark runs it as a recorded exception to the one-pin rule",
+      "task_ref": "t12",
+      "reason": "strict tool calls WITH thinking repeat 78 times until max_tokens on BOTH official v0.30.0-ubuntu2404 and v0.31.0-ubuntu2404 (3/3 runs each), while the nightly 0.26.1rc1.dev942 makes one call; the grammar and its xgrammar enforcement are identical across all three (offline matcher test), so the regression is elsewhere in vLLM between 0.26.1rc1 and 0.30.0",
+      "affects": [
+        "t12",
+        "t13",
+        "t14"
+      ],
+      "origin": "user",
+      "status": "approved",
+      "classification": "risky",
+      "seq": 4
+    },
+    {
+      "id": "d5",
+      "what": "qwen3_coder_thinking plugin: on engines that hold the structured-output grammar until reasoning ends (vLLM >= 0.30.0; seen on 0.30.0, 0.31.0 and eugr 0.32.0rc1) the plugin must pass reasoning=False, because its thinking-prefixed grammar otherwise waits for a second </think> and masks EOS, so strict+thinking repeats the tool call until max_tokens; the old nightly keeps the per-request derivation. Fixed engine-aware in R2 with tests for both engine kinds; until then the Spark runs a hand-patched, backed-up copy (reasoning=False)",
+      "task_ref": "t12",
+      "reason": "root cause of the strict+thinking regression found 2026-10-18: offline matcher started after </think> masks EOS on every grammar/xgrammar combination; live on eugr, reasoning=False gives 1 call / 45 tokens (65 with DSpark) with reasoning kept, and maxLength=3 still enforced (strict 'the' vs non-strict full string); DSpark ruled out (78 calls with it off)",
+      "affects": [
+        "t14",
+        "t17"
+      ],
+      "origin": "user",
+      "status": "approved",
+      "classification": "needs-follow-up",
+      "seq": 5
+    },
+    {
+      "id": "d6",
+      "what": "d3 (split pin) and d4 (eugr exception) withdrawn: with the d5 plugin fix, strict+thinking is clean on official v0.31.0-ubuntu2404, so the Sparks run the same official 0.31.0 pin as Thor and Orin; spark2 follows in t13 on 0.31.0; eugr stays a measured, unadopted candidate (17.2 tok/s vs 15.9 on 0.31.0, cortex with DSpark)",
+      "task_ref": "t12",
+      "reason": "operator decision 2026-10-18 after measuring both engines under the d5 fix: one official pin over a ~7-8% decode gain from a third-party rc build",
+      "affects": [
+        "t13",
+        "t14"
+      ],
+      "origin": "user",
+      "status": "approved",
+      "classification": "acceptable",
+      "seq": 6
+    }
+  ],
+  "evidence": [],
+  "deltas": [],
+  "supersessions": []
+}

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/.devague/frames/vllm-0-31-0-fleet.json

@@ -4,7 +4,7 @@
   "schema_version": 6,
   "status": "exported",
   "created": "2026-10-17T16:54:43Z",
-  "updated": "2026-10-17T19:47:33Z",
+  "updated": "2026-10-18T04:20:39Z",
   "claims": [
     {
       "id": "c1",
@@ -955,6 +955,30 @@
       "links": [],
       "instruction": "",
       "revisions": []
+    },
+    {
+      "id": "c54",
+      "kind": "decision",
+      "text": "GB10 gateways keep thermal shedding ON through the rollout: LOBES_THERMAL_SHED_C=91 / RESUME_C=86 from the spark profile (operator, 2026-10-17); the graded governor throttles the cortex container below that",
+      "origin": "user",
+      "status": "confirmed",
+      "honesty_conditions": [],
+      "hard_questions": [],
+      "links": [],
+      "instruction": "",
+      "revisions": []
+    },
+    {
+      "id": "c55",
+      "kind": "decision",
+      "text": "Spark gateway thermal shed raised to 98 C (resume 94 C) after measuring its acpitz zones at ~88 C idle and 92 C under one benchmark stream; spark2 keeps the profile's 91/86 (it peaked at 67.7 C). Supersedes c54 for the Spark (operator, 2026-10-18)",
+      "origin": "user",
+      "status": "confirmed",
+      "honesty_conditions": [],
+      "hard_questions": [],
+      "links": [],
+      "instruction": "",
+      "revisions": []
     }
   ],
   "open_vagueness": [
@@ -963,8 +987,8 @@
       "text": "The spark-lobe DSpark transcript (docs/evidence/2026-08-24-spike-dspark-cortex-spark.txt:30-31,80-81) logs the running primary's digest as sha256:49d2eb65\u2026, not 8bd082c2, while reporting version 0.26.1rc1.dev942 \u2014 the digest\u2192version mapping of the incumbent baseline is unreconciled (possibly a per-arch child digest)",
       "kind": "unknown_nonblocking",
       "claim_id": null,
-      "resolved": false,
-      "resolution": "",
+      "resolved": true,
+      "resolution": "49d2eb65 is the local image id of vllm/vllm-openai@sha256:8bd082c2 (arm64), confirmed on spark2 2026-10-17 (docs/evidence/2026-10-17-baseline-nightly-spark2.txt); the 2026-08-24 DSpark baseline ran on the nightly (operator approved closing, 2026-10-17)",
       "resolution_claim_id": null
     },
     {
@@ -1403,6 +1427,54 @@
       ],
       "origin": "llm",
       "status": "proposed"
+    },
+    {
+      "id": "l8",
+      "code": "n-below-claim",
+      "what": "t8 throughput comparison: lobes measure is one tiny batch per run (embedder 1.9-98 docs/s, reranker 3.6-65.5 docs/s across runs) against a ONE-run nightly baseline per role; the instrument's spread exceeds any difference, so no regression/speedup is claimed \u2014 a proper same-harness throughput run is still missing",
+      "skipped_check": "a multi-run, warmed throughput harness for pooling lanes",
+      "refs": [
+        "t7",
+        "t8"
+      ],
+      "origin": "llm",
+      "status": "proposed"
+    },
+    {
+      "id": "l9",
+      "code": "control-absent",
+      "what": "t3 declared 'docker update --cpus is SAFE on a running vLLM container' from a test that only exercised an already-initialised server; it never re-initialised the GPU after an update. On this host (systemd cgroup driver, cgroup v2) docker update revokes NEW GPU access in the container: nvidia-smi 'Failed to initialize NVML: Unknown Error', torch.cuda.is_available() False (reproduced 2026-10-17 22:17Z with a throwaway container). In t12 the governor's first docker update (22:11:37Z) killed the Spark cortex lane's first boot with NVMLError_Unknown (22:11:38Z); the restart policy recovered it",
+      "skipped_check": "re-initialise the GPU (nvidia-smi / a new CUDA context / a booting engine) after docker update",
+      "refs": [
+        "t3",
+        "t4",
+        "t12"
+      ],
+      "origin": "llm",
+      "status": "proposed"
+    },
+    {
+      "id": "l10",
+      "code": "assumption-for-measurement",
+      "what": "Told the operator the strict+thinking regression 'narrows to an xgrammar runtime regression (0.2.3 -> 0.2.7)' from version correlation alone; an offline matcher test then showed nightly, 0.30.0 and 0.31.0 enforce the identical grammar identically (after one call both EOS and another <tool_call> are allowed), falsifying it. The cause is elsewhere in 0.31.0's generation under the grammar (DSpark interplay untested)",
+      "skipped_check": "test the grammar matcher per xgrammar version before naming a cause",
+      "refs": [
+        "t12",
+        "d3"
+      ],
+      "origin": "llm",
+      "status": "proposed"
+    },
+    {
+      "id": "l11",
+      "code": "assumption-for-measurement",
+      "what": "t12 handoff stated the strict+thinking grammar was identical across nightly, 0.30.0 and 0.31.0; the saved dumps differ (the thinking block's any_text gained excludes [<think>, </think>]), and the offline matcher test only fed the full thinking text, never the post-</think> start the newer engines actually use \u2014 so it could not detect the real cause (the plugin's thinking prefix under engine-side reasoning gating)",
+      "skipped_check": "diff the grammar dumps field by field; probe the matcher from the engine's real start state",
+      "refs": [
+        "t12"
+      ],
+      "origin": "llm",
+      "status": "proposed"
     }
   ],
   "obligations": []

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/CHANGELOG.md

@@ -4,6 +4,22 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
 adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.90.0] - 2026-10-18
+
+### Added
+
+- `WORKER_MM_ENCODER_ATTN_BACKEND` (worker knob `mm_encoder_attn_backend`): renders `--mm-encoder-attn-backend=<value>` on the worker lane when set, nothing when empty; the `thor-worker` shape sets `TORCH_SDPA`, which fixes the worker's image intake on vLLM 0.31.0 / sm_110 (#310 deviation d1).
+
+### Changed
+
+- The fleet's vLLM pin moves from the nightly `vllm/vllm-openai@sha256:8bd082c2...` (0.26.1rc1.dev942) to official vLLM 0.31.0 `v0.31.0-ubuntu2404` (index `sha256:3fb023cc...`): every vLLM image line in the fleet templates and `env.example`, plus the `Dockerfile.embed-st`, `Dockerfile.decider` and `Dockerfile.decider-vllm` bases (#310 R2, t14). `Dockerfile.vllm-gemma4` is unchanged. Existing deployments keep their own `.env` pin until it is changed there.
+- Values measured on the nightly are relabelled DECLARED/UNVALIDATED (#108) unless a 0.31.0 transcript re-measures them; the decider's agreement is re-promoted to 0.9928 (415/418) on `-ubuntu2404` (t15).
+- `deployments/` re-captured from the live Thor (thor-worker) and Orin (orin-embed) on 0.31.0 (t16).
+
+### Fixed
+
+- `scripts/scan_deployment_secrets.py`: a fully quoted list item such as `- "HF_TOKEN=${HF_TOKEN:-}"` is a pure variable reference, not a literal; a quoted literal or non-empty default is still flagged.
+
 ## [0.89.2] - 2026-10-18
 
 ### Fixed

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/CLAUDE.md

@@ -52,6 +52,17 @@ re-render force-writes the key, so changing it means selecting a different
 shape or forking the shape file (issue #204). Measured KV pool at the adopted pair:
 760,806 tokens = **2.90× concurrency at 262144**.
 
+**Engine relabel (#310 t15).** The DSpark-vs-MTP figures and the 760,806-token
+KV pool above were measured on the nightly 0.26.1rc1.dev942 and are
+DECLARED/UNVALIDATED (#108) as vLLM 0.31.0 values; they stay on record. The
+same cortex + DSpark n=7 / 262144 / util 0.58 pairing has since been booted on
+0.31.0 on the Spark (KV 754,621 tokens, `lobes benchmark` 15.9 tok/s,
+`docs/evidence/2026-10-18-accept-vllm-031-spark-cortex.txt`) and spark2 (KV
+751,086; same-box nightly -> 0.31.0 code 37.1 -> 41.5, reasoning 33.9 -> 31.5,
+prose 15.4 -> 14.5 tok/s, `lobes benchmark` 17.0/16.9 -> 19.2/19.3,
+`docs/evidence/2026-10-18-accept-vllm-031-spark2.txt`) — a different harness
+from the spike, so the MTP-n2 comparison is not re-measured.
+
 That shape previously declared a YaRN-extended **1M window**
 (`max_model_len=1048576`), MEASURED live 2026-08-19
 (`docs/evidence/2026-08-19-accept-qwen38-1m-spark.txt`): same `gpu_mem_util=0.58`
@@ -87,7 +98,10 @@ for `worker` + `hand` + fine-tuning headroom. The Thor now serves
 OFF** (this checkpoint's GDN decode carries an MTP variant with no sm_110
 kernel image on the fleet's current nightly — plain non-MTP GDN decode
 works fine), measured **12.1 tok/s** single-stream decode, TTFT ~300 ms —
-see `docs/evidence/2026-08-20-accept-cortex-local-thor.txt`. See
+see `docs/evidence/2026-08-20-accept-cortex-local-thor.txt`. (#310 t15: the
+1M window, KV pool, 12.1 tok/s and the missing sm_110 MTP kernel are nightly
+0.26.1rc1 measurements, DECLARED/UNVALIDATED on 0.31.0 — no Thor cortex lane
+has been booted on 0.31.0.) See
 `docs/qwen3.8-27b-nvfp4.md` and the `worker`/`muse` paragraphs
 below for the rest of the d1 topology.
 lobes runs it; the `acp` `vllm-local` provider connects the lobes agent to it.
@@ -281,7 +295,11 @@ probability for every option of every question; the domain is chosen by the
 forwarded byte-identical, and the path is never shed. Served by per-domain
 decider sidecars (`lobes/decider_sidecar/`). The default checkpoint is our own
 `culture-dev/clef-27b-GPTQ-W4A16-g64` (Clef 27B, weight-only 4-bit for the Orin's
-Ampere), MEASURED 0.988 top-1 agreement with bf16 on 418 labelled questions; the
+Ampere), MEASURED 0.9928 agreement with bf16 (415/418, top-1 0.9378) on 418
+labelled questions on the vLLM 0.31.0 `-ubuntu2404` base
+(`docs/evidence/2026-10-17-accept-vllm-031-orin.txt`; the earlier 0.988 was
+measured 2026-10-14 on plain `v0.31.0`, not `-ubuntu2404`, and is kept on
+record in `docs/decider.md`); the
 `orin-decider` shape renders it beside `orin-embed` (render DECLARED/UNVALIDATED
 until its live acceptance, #108). Responsibilities include `final_decision`;
 forbidden `repo_action`, `security_decision`. See `docs/decider.md`.
@@ -371,7 +389,11 @@ proxy (2026-09-11).** The Thor serves the lane locally through the
 DFlash speculation, and `WORKER_MAX_NUM_SEQS=1`. One request at a time is
 **by design** and is still to be tested under a real multi-agent load. See
 `docs/nvidia-qwen3.6-35b-a3b-nvfp4.md` for the recipe and the 196.6 tok/s
-measurement. The Spark does not host it: `WORKER_FEASIBLE=false` plus
+measurement (nightly 0.26.1rc1; on vLLM 0.31.0 the lane boots at the same
+budget with KV 768,707 tokens = 2.93x, and image intake needs
+`--mm-encoder-attn-backend=TORCH_SDPA` on sm_110 — deviation d1,
+`docs/evidence/2026-10-17-accept-vllm-031-thor.txt`; 196.6 tok/s and the other
+nightly figures are DECLARED/UNVALIDATED there, #310 t15). The Spark does not host it: `WORKER_FEASIBLE=false` plus
 `WORKER_PEER_ORIGIN` / `WORKER_PEER_PROXY` forward `model=worker` (and the
 raw id) to the Thor, with `X-Lobes-Proxied-By` on every answer. This was
 validated live on 0.75.1 gateways

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: lobes-cli
-Version: 0.89.2
+Version: 0.90.0
 Summary: lobes — run, assess, and switch the local vLLM model.
 Project-URL: Homepage, https://github.com/agentculture/lobes-cli
 Project-URL: Issues, https://github.com/agentculture/lobes-cli/issues

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/deployments/jetson-agx-orin__orin-embed/VARIATION.md

@@ -27,9 +27,26 @@ The files here are the live deployment's, verbatim.
   `docker-compose.shape.yml` and `docker-compose.override.yml`. It runs
   without the audio overlay, which this board never ran.
 
+**Re-captured from the LIVE box on 2026-10-18 at 0.89.0** (#310 t16, after the
+fleet moved to vLLM 0.31.0 `v0.31.0-ubuntu2404`). The files are what the Orin
+holds now (fetched read-only over ssh; `.env` was read for the allowlist only
+and is not committed). The box now also runs a `decider-general` sidecar
+(`COMPOSE_PROFILES=decider,gemma2-embed`), so `Dockerfile.decider-vllm` joins
+the captured files and the `[env]` table gained the `DECIDER_DOMAIN*` keys,
+`LOBES_BOOT_ORDER` and `LOBES_THERMAL_ZONES`; `GEMMA2_EMBED_MEM_LIMIT` moved
+from 6g to 8g. The engine pin is a `.env` key the lock's allowlist does not
+record, so it is not in this directory.
+
 ## Measured result
 
-Measured: `docs/evidence/2026-10-07-accept-orin-embed.txt` (mesh-wide
+Measured: `docs/evidence/2026-10-17-accept-vllm-031-orin.txt` covers the
+state captured here: embed, rerank, gemma2-embed and decider-general all
+serving on vLLM 0.31.0 (sm_87), with probes passing, the embedder measured at
+roughly 13% lower throughput than the nightly (recorded there as a named
+cost), and the decider re-measured at 0.9928 agreement with bf16.
+
+Earlier measurements of the pre-0.31.0 engine:
+`docs/evidence/2026-10-07-accept-orin-embed.txt` (mesh-wide
 acceptance), with the lane footprint in
 `docs/evidence/2026-10-07-spike-embeddinggemma2-sidecar-orin.txt` and the
 code head-to-head in `docs/evidence/2026-10-07-h2h-code-retrieval-orin.txt`.
@@ -50,12 +67,3 @@ code head-to-head in `docs/evidence/2026-10-07-h2h-code-retrieval-orin.txt`.
 - **No capture verb exists.** This lock was written by calling
   `lobes.runtime._lock.capture_lock` / `write_lock` directly, over the live
   `.env` (read only, never committed).
-
-**Predates the lane-safety change (2026-10-17).** The captured
-`mg-logwrap.sh` and compose files predate the lane-safety work (boot hold,
-start counter, memory gate, compile-cache mounts, wrapped services; plan
-`docs/plans/2026-10-17-lane-safety-and-quality-304-307-308-309.md`, claim c7).
-A box restored from, or compared against, this lock will therefore show
-`lock_drift` for `mg-logwrap.sh` and `docker-compose*.yml` until the lock is
-re-captured. That drift is expected; no captured file was rewritten from a
-live box for this note.

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/deployments/jetson-agx-orin__orin-embed/deployment.lock.toml

@@ -9,8 +9,8 @@ schema_version = 1
 id = "jetson-agx-orin"
 profile = "orin"
 shape = "orin-embed"
-lobes_version = "0.84.0.dev619"
-evidence = "docs/evidence/2026-10-07-accept-orin-embed.txt"
+lobes_version = "0.89.0"
+evidence = "docs/evidence/2026-10-17-accept-vllm-031-orin.txt"
 
 [env]
 ASSOCIATE_FEASIBLE = "false"
@@ -23,13 +23,16 @@ ASSOCIATE_MODEL = "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4"
 ASSOCIATE_QUANTIZATION = "modelopt"
 ASSOCIATE_SERVED_NAME = "nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4"
 ASSOCIATE_SPECULATIVE_CONFIG = "\"'--speculative-config={\\\"method\\\": \\\"dspark\\\", \\\"model\\\": \\\"nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark\\\", \\\"num_speculative_tokens\\\": 5, \\\"kv_cache_dtype\\\": \\\"bfloat16\\\"}'\""
+DECIDER_DOMAINS = "general=http://decider-general:8000"
+DECIDER_DOMAIN_CONTEXTS = "general=16384"
+DECIDER_DOMAIN_MODELS = "general=culture-dev/clef-27b-GPTQ-W4A16-g64@4daaa224c6dc553b13120bb10414a7e7de7a5a95"
 EMBED_ATTENTION_BACKEND = "TRITON_ATTN"
 EMBED_GPU_MEM_UTIL = "0.06"
 EMBED_MAX_MODEL_LEN = "8192"
 EMBED_MODEL = "Qwen/Qwen3-Embedding-0.6B"
 EMBED_SERVED_NAME = "Qwen/Qwen3-Embedding-0.6B"
 GATEWAY_READ_TIMEOUT = "7200"
-GEMMA2_EMBED_MEM_LIMIT = "6g"
+GEMMA2_EMBED_MEM_LIMIT = "8g"
 GEMMA2_EMBED_TESTED_ON = "jetson-agx-orin 2026-10-07"
 HAND_ATTENTION_BACKEND = "TRITON_ATTN"
 HAND_FEASIBLE = "false"
@@ -38,7 +41,9 @@ HAND_MAX_MODEL_LEN = "32768"
 HAND_MODEL = "LiquidAI/LFM2.5-1.2B-Instruct"
 HAND_SERVED_NAME = "LiquidAI/LFM2.5-1.2B-Instruct"
 INNEREYE_DECLARED_PEAK_GIB = "31.42"
+LOBES_BOOT_ORDER = "multimodal,embed,rerank"
 LOBES_IOWAIT_DEGRADED_THRESHOLD = "100"
+LOBES_THERMAL_ZONES = "tj-thermal"
 MULTIMODAL_ATTENTION_BACKEND = "TRITON_ATTN"
 MULTIMODAL_FEASIBLE = "false"
 MULTIMODAL_GPU_MEM_UTIL = "0.45"
@@ -68,13 +73,14 @@ RERANK_SERVED_NAME = "Qwen/Qwen3-Reranker-0.6B"
 WORKER_FEASIBLE = "false"
 
 [files]
+"Dockerfile.decider-vllm" = "sha256:4a8a2227f2ff619a768c8542b531e31bb54b4d8b78cb97ef51611258cb6c60e9"
 "Dockerfile.embed-st" = "sha256:49980e4645ebf973f2a5a4d5312bf4eea0e988aa03276e69cfd44f7b4b570c75"
 "Dockerfile.gateway" = "sha256:7c3d685df77bba1894ccc4ce46aca2b8da716072d8857a210294cc0dc002eebd"
 "docker-compose.embed.yml" = "sha256:5a513fa9f87476405fb9ddf12b9f5e03e4cfd3be5406003ecaa604f722985646"
 "docker-compose.gpu.yml" = "sha256:9a725dc59144280c882d73ab4eeab86ad3acf7bf0cd37c82f83d64748282cbd2"
-"docker-compose.override.yml" = "sha256:6a93a0570994feaba56f63457ea86313dcf228df73a4c9363b3bdc2f4d716b1b"
-"docker-compose.shape.yml" = "sha256:142e558ed662d552e009d773b41af0c838f60cb443258bb38cc6ac70261c4dc0"
-"docker-compose.yml" = "sha256:9f429dc24787663cfa45507efa5aa9b2c79e49f05ef20eab8c7056ceddc9e8f5"
-"mg-logwrap.sh" = "sha256:fd9dfb7d30cee5d8e3045201ecb9c99ee3b1e82c5d397a1b993c99f0006e0ee0"
+"docker-compose.override.yml" = "sha256:44d518c191afb0efc91c8a3ee7aca030619d17a9d4a55f71896be32e7b60a0d9"
+"docker-compose.shape.yml" = "sha256:06e1a95896f4916c9c702a303256ede7cd2e4183e8652d928dac46802fffff26"
+"docker-compose.yml" = "sha256:741c671f2247775921528ef49df8fd04fb59b5ec11c19c6374b92ccf55748f15"
+"mg-logwrap.sh" = "sha256:421c1268e5371f9db5440ef1d5284adeb6d00c0865f271380d20a53b4405ffc3"
 "qwen3_reranker.jinja" = "sha256:e1ee98e69aab7b2da366edf1c50efcef37e34b4a0c50fb816336213e68d9047a"
-"qwen3_thinking_tool_parser.py" = "sha256:22a030552d033b897316ca2d0aa75ccac53dc955fe9ddd6c614b75c684338130"
+"qwen3_thinking_tool_parser.py" = "sha256:36a95d103704249fce76bf564442afb0746641bb125d9a9b31f711ff9333e591"

lobes_cli-0.90.0/deployments/jetson-agx-orin__orin-embed/docker-compose.override.yml

@@ -0,0 +1,63 @@
+services:
+  gateway:
+    environment:
+      - LOBES_MESH_KEY=${LOBES_MESH_KEY:-}
+      - LOBES_MESH_NAME=${LOBES_MESH_NAME:-}
+      - LOBES_MESH_SEEDS=${LOBES_MESH_SEEDS:-}
+      - LOBES_MESH_HEARTBEAT_S=${LOBES_MESH_HEARTBEAT_S:-60}
+      - LOBES_MESH_MISSED_MAX=${LOBES_MESH_MISSED_MAX:-3}
+      - LOBES_MESH_LEDGER_PATH=${LOBES_MESH_LEDGER_PATH:-/home/gateway/mesh/ledger.json}
+      - GATEWAY_SELF_ORIGIN=${GATEWAY_SELF_ORIGIN:-}
+      # -- lane safety (#304/#307/#308, R1 0.89.0) - #310 t9, 2026-10-17 --
+      - MG_LOG_DIR=/logs/model-gear
+      - MG_LOG_NAME=gateway
+      - LOBES_BOOT_DELAY_S=${LOBES_BOOT_DELAY_S:-120}
+      - LOBES_BOOT_ORDER=${LOBES_BOOT_ORDER:-}
+      - LOBES_BOOT_STAGGER_S=${LOBES_BOOT_STAGGER_S:-20}
+      - LOBES_THERMAL_ZONES=${LOBES_THERMAL_ZONES:-}
+      - LOBES_THERMAL_SHED_C=${LOBES_THERMAL_SHED_C:-}
+      - LOBES_THERMAL_RESUME_C=${LOBES_THERMAL_RESUME_C:-}
+    volumes:
+      - ${LOBES_MESH_DIR:-./mesh}:/home/gateway/mesh
+      - ${MODEL_GEAR_LOG_DIR:-./logs}:/logs/model-gear
+      - ./mg-logwrap.sh:/usr/local/bin/mg-logwrap:ro
+    # gateway under the wrapper (boot hold); image ENTRYPOINT is python -m lobes.gateway, no CMD
+    entrypoint: ["bash", "/usr/local/bin/mg-logwrap"]
+    command: ["python", "-m", "lobes.gateway"]
+    healthcheck:
+      start_period: 200s
+  # lane-safety H6 (2026-10-17): per-lane vLLM compile cache (lobes PR #315, #309)
+  vllm-embed:
+    environment:
+      - VLLM_CACHE_ROOT=/cache/vllm
+      - FLASHINFER_WORKSPACE_BASE=${LOBES_FLASHINFER_WORKSPACE_BASE:-/cache/flashinfer}
+    volumes:
+      - ./compile-cache/embed:/cache/vllm
+      - ${LOBES_COMPILE_CACHE_DIR:-./compile-cache}/embed-flashinfer:/cache/flashinfer
+    healthcheck:
+      start_period: 780s
+  # -- lane safety (#304/#307/#308, R1 0.89.0) - #310 t9, 2026-10-17 --
+  vllm-rerank:
+    environment:
+      - VLLM_CACHE_ROOT=${LOBES_VLLM_CACHE_ROOT:-/cache/vllm}
+      - FLASHINFER_WORKSPACE_BASE=${LOBES_FLASHINFER_WORKSPACE_BASE:-/cache/flashinfer}
+    volumes:
+      - ${LOBES_COMPILE_CACHE_DIR:-./compile-cache}/rerank:/cache/vllm
+      - ${LOBES_COMPILE_CACHE_DIR:-./compile-cache}/rerank-flashinfer:/cache/flashinfer
+    healthcheck:
+      start_period: 780s
+  # -- #310 t10, 2026-10-17: sidecars rebuilt on the fleet pin (v0.31.0-ubuntu2404) --
+  decider-general:
+    build:
+      args:
+        DECIDER_VLLM_BASE_IMAGE: vllm/vllm-openai@sha256:3fb023cc987cc7c2bca7ca14c1e0aff5ae06bd9431bf07773731ce7e8a682460
+    environment:
+      - VLLM_CACHE_ROOT=${LOBES_VLLM_CACHE_ROOT:-/cache/vllm}
+      - FLASHINFER_WORKSPACE_BASE=${LOBES_FLASHINFER_WORKSPACE_BASE:-/cache/flashinfer}
+    volumes:
+      - ${LOBES_COMPILE_CACHE_DIR:-./compile-cache}/decider-general:/cache/vllm
+      - ${LOBES_COMPILE_CACHE_DIR:-./compile-cache}/decider-general-flashinfer:/cache/flashinfer
+  embed-gemma2-embed:
+    build:
+      args:
+        ST_BASE_IMAGE: vllm/vllm-openai@sha256:3fb023cc987cc7c2bca7ca14c1e0aff5ae06bd9431bf07773731ce7e8a682460

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/deployments/jetson-agx-orin__orin-embed/docker-compose.shape.yml

@@ -17,6 +17,10 @@
 # started every heavy lane, including one this box declares infeasible — so
 # there is nothing left here to reset.
 #
+# It is also where a decider-hosting shape's per-DOMAIN decider-<name>
+# services are declared: the domain list is shape data, so those services
+# cannot live in the static template (orin-decision-models plan, t8).
+#
 # This file is also where the deployment's OWN opt-in compose edits are
 # rendered — today exactly one, the INNEREYE_UI_PORT publication of
 # ComfyUI's web UI (see the block below when present). Compose cannot
@@ -32,3 +36,45 @@ services:
     profiles: ["shape-dropped"]
   vllm-primary:
     profiles: ["shape-dropped"]
+  # Decider domains (orin-decision-models): one sidecar per domain, built from
+  # Dockerfile.decider. NO host port (`expose` only — the sidecar has no auth,
+  # the gateway is the sole way in), no start-order edge either way (issue #222), and
+  # a per-domain mem_limit (Jetson GPU memory counts against the cgroup). The
+  # gateway reaches them through DECIDER_DOMAINS in .env.
+  # clef-vllm domains build from Dockerfile.decider-vllm instead.
+  decider-general:
+    build:
+      context: .
+      dockerfile: Dockerfile.decider-vllm
+      args:
+        MODEL_GEAR_VERSION: ${MODEL_GEAR_VERSION:-}
+        LOBES_DEV_INDEX_URL: ${GATEWAY_PIP_EXTRA_INDEX_URL:-}
+    container_name: model-gear-decider-general
+    profiles: ["decider"]
+    restart: unless-stopped
+    mem_limit: "32g"
+    runtime: nvidia
+    ipc: host
+    environment:
+      - "DECIDER_BACKEND=clef-vllm"
+      - "DECIDER_MODEL=culture-dev/clef-27b-GPTQ-W4A16-g64"
+      - "DECIDER_REVISION=4daaa224c6dc553b13120bb10414a7e7de7a5a95"
+      - "DECIDER_GPU_MEM_UTIL=0.40"
+      - "DECIDER_MAX_MODEL_LEN=16384"
+      - "DECIDER_KV_CACHE_MEMORY_BYTES=4294967296"
+      - "DECIDER_PORT=8000"
+      - "HF_HOME=/root/.cache/huggingface"
+      - "HF_TOKEN=${HF_TOKEN:-}"
+      - "HF_HUB_DISABLE_XET=1"
+      - "NVIDIA_VISIBLE_DEVICES=all"
+      - "NVIDIA_DISABLE_REQUIRE=1"
+    volumes:
+      - ${HF_CACHE:-${HOME:-/root}/.cache/huggingface}:/root/.cache/huggingface
+    expose:
+      - "8000"
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 5
+      start_period: 900s

{lobes_cli-0.89.2 → lobes_cli-0.90.0}/deployments/jetson-agx-orin__orin-embed/docker-compose.yml

@@ -1995,6 +1995,17 @@ services:
       # `model=associate` 404s role_infeasible, never a silent fallback.
       - ASSOCIATE_BASE_URL=${ASSOCIATE_BASE_URL:-}
       - ASSOCIATE_SERVED_NAME=${ASSOCIATE_SERVED_NAME:-}
+      # The opt-in decider role (schema-scored decisions): wired only when a
+      # decider-hosting shape renders DECIDER_BASE_URL; unwired it is
+      # infeasible. Gateway passthrough only; the lane service lands later.
+      - DECIDER_BASE_URL=${DECIDER_BASE_URL:-}
+      - DECIDER_SERVED_NAME=${DECIDER_SERVED_NAME:-}
+      - DECIDER_DOMAINS=${DECIDER_DOMAINS:-}
+      # Each domain's checkpoint (`name=model[@revision]`), rendered by the
+      # shape beside DECIDER_DOMAINS: the /capabilities `decider.domains`
+      # advert and the per-domain mesh fingerprint read it (odm t7).
+      - DECIDER_DOMAIN_MODELS=${DECIDER_DOMAIN_MODELS:-}
+      - DECIDER_DOMAIN_CONTEXTS=${DECIDER_DOMAIN_CONTEXTS:-}
       # The opt-in `innereye` RENDER tenant (the eleventh Colleague role —
       # ComfyUI behind the gateway's /v1/render facade, issue #82). Same
       # opt-in mechanics as muse/worker/associate above: an innereye-hosting
@@ -2090,6 +2101,7 @@ services:
       - MUSE_FEASIBLE=${MUSE_FEASIBLE:-}
       - WORKER_FEASIBLE=${WORKER_FEASIBLE:-}
       - ASSOCIATE_FEASIBLE=${ASSOCIATE_FEASIBLE:-}
+      - DECIDER_FEASIBLE=${DECIDER_FEASIBLE:-}
       # hand rides the same feasibility channel as every other role since
       # 2026-08-20 (d1 reversal — see lobes/gateway/_config.py's
       # NEVER_PROXIED_BACKENDS): a box whose card cannot serve LFM2.5
@@ -2138,6 +2150,7 @@ services:
       - MUSE_MAX_ACTIVE=${MUSE_MAX_ACTIVE:-}
       - WORKER_MAX_ACTIVE=${WORKER_MAX_ACTIVE:-}
       - ASSOCIATE_MAX_ACTIVE=${ASSOCIATE_MAX_ACTIVE:-}
+      - DECIDER_MAX_ACTIVE=${DECIDER_MAX_ACTIVE:-}
       - HAND_MAX_ACTIVE=${HAND_MAX_ACTIVE:-}
       - EMBED_MAX_ACTIVE=${EMBED_MAX_ACTIVE:-}
       - RERANK_MAX_ACTIVE=${RERANK_MAX_ACTIVE:-}
@@ -2229,6 +2242,11 @@ services:
       - ASSOCIATE_REASONING_PARSER=${ASSOCIATE_REASONING_PARSER:-}
       - ASSOCIATE_TOOL_CALL_PARSER=${ASSOCIATE_TOOL_CALL_PARSER:-}
       - ASSOCIATE_SPECULATIVE_CONFIG=${ASSOCIATE_SPECULATIVE_CONFIG-}
+      - DECIDER_QUANTIZATION=${DECIDER_QUANTIZATION:-}
+      - DECIDER_KV_CACHE_DTYPE=${DECIDER_KV_CACHE_DTYPE:-}
+      - DECIDER_REASONING_PARSER=${DECIDER_REASONING_PARSER:-}
+      - DECIDER_TOOL_CALL_PARSER=${DECIDER_TOOL_CALL_PARSER:-}
+      - DECIDER_SPECULATIVE_CONFIG=${DECIDER_SPECULATIVE_CONFIG-}
       # HAND_QUANTIZATION / HAND_KV_CACHE_DTYPE / HAND_REASONING_PARSER /
       # HAND_SPECULATIVE_CONFIG: no such lane knobs exist — vllm-hand is
       # bf16 (deliberately NO --quantization flag), has no thinking mode