lobes-cli 0.45.0__tar.gz → 0.45.2__tar.gz

{lobes_cli-0.45.0 → lobes_cli-0.45.2}/.github/workflows/publish.yml

@@ -23,6 +23,9 @@ jobs:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
       - uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a # v4
+        with:
+          version: "0.11.29"
+          enable-cache: true
 
       - run: uv python install 3.12
 
@@ -44,6 +47,9 @@ jobs:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
       - uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a # v4
+        with:
+          version: "0.11.29"
+          enable-cache: true
 
       - run: uv python install 3.12
 
@@ -79,6 +85,9 @@ jobs:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
       - uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a # v4
+        with:
+          version: "0.11.29"
+          enable-cache: true
 
       - run: uv python install 3.12
 
@@ -101,6 +110,9 @@ jobs:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
       - uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a # v4
+        with:
+          version: "0.11.29"
+          enable-cache: true
 
       - run: uv python install 3.12
 

{lobes_cli-0.45.0 → lobes_cli-0.45.2}/.github/workflows/tests.yml

@@ -24,6 +24,9 @@ jobs:
           fetch-depth: 0
 
       - uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a # v4
+        with:
+          version: "0.11.29"
+          enable-cache: true
 
       - run: uv python install 3.12
 
@@ -49,6 +52,9 @@ jobs:
           node-version: '20'
 
       - uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a # v4
+        with:
+          version: "0.11.29"
+          enable-cache: true
 
       - run: uv python install 3.12
 

{lobes_cli-0.45.0 → lobes_cli-0.45.2}/CHANGELOG.md

@@ -4,6 +4,40 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
 adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.45.2] - 2026-07-17
+
+### Added
+
+- `docs/orin-profiles.md` — Jetson AGX Orin 64GB live-validation evidence
+  (2026-07-16/17, issue #127 mesh work): the operator profile serving Gemma
+  `senses` at its full 128K context on Ampere sm_87 (measured `gpu_mem_util`
+  0.45; 0.30 refused with 2.25 GiB KV vs the 3.08 GiB that 131072 needs; KV
+  pool 802,644 tokens / 6.12x concurrency), embedder/reranker probe results,
+  why `cortex` (modelopt NVFP4 W4A4) is architecturally infeasible on Ampere,
+  three Jetson/sm_87 divergences found live (csv-mode GPU access needs
+  `runtime: nvidia`; the Parakeet base image is Spark-only — no sm_87
+  kernels; unified-memory use far exceeds the util sum), the validated #127
+  cortex proxy wiring, and the gateway→gateway audio-chaining limitation
+  (readiness probes `/v1/health/ready`, which a peer gateway 401s/404s —
+  first-class audio referral knobs are the phase-2 candidate).
+
+### Changed
+
+- `docs/machine-profiles.md` cross-links the Orin worked example from the
+  custom-profile section.
+
+### Fixed
+
+- `docs/machine-profiles.md` documented `lobes init --machine` for selecting
+  a profile; `init`'s actual flag is `--profile` (`--machine` belongs to
+  `switch`). The wrong flag made the documented custom-profile flow error out.
+
+## [0.45.1] - 2026-07-17
+
+### Changed
+
+- CI hardening: pin `astral-sh/setup-uv`'s `version` to `0.11.29` (was `latest`) and turn on `enable-cache: true` across both workflows (`tests.yml`, `publish.yml`, 6 usages total), so a transient GitHub release-CDN outage can no longer take down every job by failing uv's "resolve latest" step; two identical CI failures on PR #132 (2026-07-16 22:39Z/22:43Z, GitHub 503 HTML from the setup-uv download) prompted the change. The action's SHA pin, tokens, and cache-dependency-glob are unchanged
+
 ## [0.45.0] - 2026-07-16
 
 ### Added

{lobes_cli-0.45.0 → lobes_cli-0.45.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: lobes-cli
-Version: 0.45.0
+Version: 0.45.2
 Summary: lobes — run, assess, and switch the local vLLM model.
 Project-URL: Homepage, https://github.com/agentculture/lobes-cli
 Project-URL: Issues, https://github.com/agentculture/lobes-cli/issues

{lobes_cli-0.45.0 → lobes_cli-0.45.2}/docs/machine-profiles.md

@@ -16,8 +16,21 @@ reference.
 
 ## How detection works
 
-When `lobes init` runs (or `lobes serve` / `lobes status` without an explicit
-`--machine`), the following steps resolve the machine name:
+The card-detection flow below runs only during **`lobes init`** — either
+auto-detected from the card, or forced with an explicit `--profile <name>` —
+to resolve the per-role `Profile` this document describes and render it into
+`.env`. **`lobes serve` and `lobes status` run no detection at all** and
+accept neither `--profile` nor `--machine`: they only read the deployment's
+already-rendered `.env` (including the persisted `LOBES_PROFILE`) — profile
+selection happens once, at `init` time, not on every `serve`/`status` call.
+`lobes switch` has its own, separate `--machine` flag — a *different*,
+legacy single-model machine-profile knob (`lobes/profiles/__init__.py`'s
+`MachineProfile` / `resolve_machine()`, tuning the single-model template's
+GPU/context defaults), not the fleet `Profile` system covered here; left at
+its default (`auto`), it re-detects the GPU name via `nvidia-smi -L` on its
+own code path, not `lobes/runtime/_detect.py::detect_card()`.
+
+The following steps resolve the machine name at `lobes init` time:
 
 1. **Gather raw facts** (from `lobes/runtime/_detect.py::detect_card()`):
    - **Device name**: from `nvidia-smi --query-gpu=name,compute_cap`
@@ -51,8 +64,8 @@ When `lobes init` runs (or `lobes serve` / `lobes status` without an explicit
 
 ## How a profile is chosen
 
-Once detection (or an explicit `--machine` flag) resolves the card name, a
-profile is looked up via `lobes/profiles/loader.py::resolve_profile()`:
+Once detection (or an explicit `--profile` flag on `lobes init`) resolves the
+card name, a profile is looked up via `lobes/profiles/loader.py::resolve_profile()`:
 
 1. **Explicit always wins**: if `--profile <name>` was given (or a
    `LOBES_PROFILE` env var), that profile is used, even if it diverges from
@@ -294,7 +307,8 @@ latency (higher tail latency per request).
 
 Operator-defined profiles go in `<deployment-dir>/profiles/<name>.toml` (the
 file's stem is the profile name). They are discovered automatically by
-`lobes init` / `lobes serve` and override any built-in with the same name.
+`lobes init` (the only verb that resolves a `Profile` — see "How detection
+works" above) and override any built-in with the same name.
 
 A profile is a self-contained TOML declaration. Example:
 
@@ -334,7 +348,8 @@ max_model_len = 8192
 **Rules:**
 
 - **File name is the profile name.** `profiles/my-custom-box.toml` → profile
-  name is `my-custom-box`. Use it as `lobes init --machine my-custom-box --apply`.
+  name is `my-custom-box`. Use it as `lobes init --profile my-custom-box --apply`
+  (the flag is `--profile` on `init`, `--machine` on `switch`).
 - **Inline `name` field must match the file stem.** If the file declares
   `name = "something-else"`, profile loading fails with an error (this prevents
   confusion from stale copies).
@@ -355,8 +370,8 @@ max_model_len = 8192
 To use your profile:
 
 ```bash
-# Explicit --machine or LOBES_PROFILE env var:
-lobes init --machine my-custom-box --apply
+# Explicit flag (--profile on init, --machine on switch) or LOBES_PROFILE env var:
+lobes init --profile my-custom-box --apply
 lobes switch my-model --machine my-custom-box --apply
 
 # Automatic if the detected machine name matches:
@@ -369,6 +384,11 @@ Auto-detection only works if the *detected machine name* (from
 profile to be auto-selected on a new card, add a new `CardStrategy` module to
 `lobes/machines/` (following the `spark.py` / `thor.py` pattern).
 
+For a complete worked example — a live-validated operator profile for the
+Jetson AGX Orin 64GB (Gemma `senses` at its full 128K context on Ampere sm_87,
+with measured knob values and the Jetson divergences found on the way) — see
+[`orin-profiles.md`](orin-profiles.md).
+
 ## The goldens contract
 
 CI guards against cross-machine breakage via golden `.env` files — one per
@@ -395,14 +415,21 @@ built-in defaults (when no profile is loaded) render as expected.
 
 lobes is deployed and validated on the following hardware, with the stated
 caveat profile as the truthful, tested configuration. Aspirational entries are
-marked clearly as **unvalidated**.
+marked clearly as **unvalidated**. **This table's scope is BUILT-IN
+profiles/shapes** (`lobes/profiles/builtin/*.toml`,
+`lobes/profiles/builtin_shapes/*.toml`) — the #108 rule is that a built-in
+earns "validated" status only via a repo-recorded boot of that *exact*
+built-in, so a validated operator-defined profile (`<deploy-dir>/profiles/
+<name>.toml`) does not by itself validate a built-in for the same card. The
+Orin row below carries both facts side by side so they read as complementary,
+not contradictory.
 
 | card | machine | profile | status | validation |
 |---|---|---|---|---|
 | **DGX Spark** (Grace Blackwell, 128 GB unified) | `spark` | `spark.toml` | load-tested | 2026-06-03 — `lobes benchmark` ran at ~7.8–8.0 tok/s decode (27B primary, util 0.30, single-stream) on the fleet duo (cortex 128K, senses 32K) with FlashInfer attention. **The correctness probes postdate that run and have not been executed on the GB10** — rerank ordering there is explicitly unverified (issue #106). See `docs/tuning-profiles.md` and the Spark run on `docs/qwen3.6-27b-text-nvfp4-mtp.md`. |
 | **Jetson AGX Thor** (Blackwell-class sm_110, 128 GB unified) | `thor` | `thor.toml` | load-tested | 2026-07-13 — the three correctness probes (cortex known-answer, embed paraphrase-ranking, rerank ordering) all pass with the profile's four divergences (`cortex kv_cache_dtype=auto`, `embedder TRITON_ATTN`, `reranker TRITON_ATTN + enforce_eager=true`); senses' health was not confirmed in that run (it boots last and the concurrent-boot race, caveat 4 below, can catch it). |
 | **unknown card** (UNKNOWN) | `generic` | `base.toml` | conservative fallback | — — no hardware beyond Spark/Thor is load-tested; UNKNOWN cards are served a small 4B model, no 27B, and no multimodal (`senses` disabled). Avoids OOM crashes on first boot. See issue #107 for broader "small default on every card" work. |
-| Jetson AGX Orin / Orin Nano Super | `(not yet)` | (not yet) | **unvalidated** | Not deployed or tested. Named in the mesh topology for future integration but no profile yet. Do not claim Orin support. |
+| Jetson AGX Orin / Orin Nano Super | `(not yet)` | (not yet) | **no built-in profile — unvalidated at that scope; operator-validated separately** | **Built-ins:** no built-in `orin` profile exists, and the built-in `orin-small` *shape* remains DECLARED/UNVALIDATED (#108 — an unbooted built-in stays unvalidated regardless of operator activity elsewhere). Do not claim built-in Orin support from this row. **Operator deployment:** a physical Jetson AGX Orin 64GB WAS operator-validated 2026-07-16/17, using a hand-written **operator profile** (`<deploy-dir>/profiles/orin.toml` — `senses`/`embedder`/`reranker`, `senses` at its full native 131072 context, measured `gpu_mem_util=0.45`) composed with the built-in `thor-lobe` *shape* (not `orin-small`). See [`orin-profiles.md`](orin-profiles.md) for the measured knobs and the Jetson/sm_87 divergences found live. |
 
 ## Thor caveats and validation facts
 

lobes_cli-0.45.2/docs/orin-profiles.md

@@ -0,0 +1,183 @@
+# Jetson AGX Orin 64GB — live-validated senses / embedder / reranker profile
+
+**Status: operator-validated on physical hardware, 2026-07-16/17** (issue #127
+mesh work). This page records what a real Jetson AGX Orin 64GB Developer Kit
+(Ampere **sm_87**, 61.3 GiB unified memory, CUDA 13.2 / driver 595.78) actually
+serves, with measured numbers — the #108 discipline: a card earns its tuning
+data by booting it, and this boot is what earned Orin's. The repo's built-in
+`orin-small` *shape* remains DECLARED/UNVALIDATED (this deployment used
+`thor-lobe`, not `orin-small` — see "Shape choice" below).
+
+## What this box serves
+
+| role | model | context | gpu_mem_util | result |
+|---|---|---|---|---|
+| `senses` | `coolthor/gemma-4-12B-it-NVFP4A16` | **131072 (full native 128K)** | **0.45** | KV pool **802,644 tokens**, **6.12×** concurrency at 131,072/request |
+| `embedder` | `Qwen/Qwen3-Embedding-0.6B` | 8192 | 0.06 | healthy; 1024-dim embeddings |
+| `reranker` | `Qwen/Qwen3-Reranker-0.6B` | 8192 | 0.06 | healthy; **rerank ordering probe passes** |
+| `cortex` | — | — | — | `feasible=false` (see "Why no cortex"), referred/proxied to the Spark |
+| `stt`/`tts` | — | — | — | dropped locally; `/v1/audio/*` forwards to a peer gateway via `AUDIO_URL` |
+
+Probes passed (2026-07-17, via the gateway on `:8000`): senses known-answer
+(text, `model=senses` alias), senses **vision** intake (solid-color image →
+correct one-word answer), embeddings (1024 dims), rerank ordering (relevant
+document top-ranked, score 0.82). The three core-role correctness probes that
+`lobes assess` encodes are therefore all exercised on sm_87.
+
+## The operator profile
+
+There is deliberately no built-in `orin` profile in `lobes/profiles/builtin/`
+(#108: no invented knobs for an unbooted card). This deployment declares an
+**operator profile** at `<deploy-dir>/profiles/orin.toml`, selected with
+`lobes init --profile orin`:
+
+```toml
+name = "orin"
+summary = "Jetson AGX Orin 64GB (Ampere sm_87, 64 GB unified) — operator profile: senses + pooling gears, no cortex"
+
+[roles.cortex]
+feasible = false
+
+[roles.senses]
+feasible = true
+model = "coolthor/gemma-4-12B-it-NVFP4A16"
+gpu_mem_util = 0.45
+max_model_len = 131072
+quantization = "compressed-tensors"
+attention_backend = "TRITON_ATTN"
+
+[roles.embedder]
+feasible = true
+model = "Qwen/Qwen3-Embedding-0.6B"
+gpu_mem_util = 0.06
+max_model_len = 8192
+attention_backend = "TRITON_ATTN"
+
+[roles.reranker]
+feasible = true
+model = "Qwen/Qwen3-Reranker-0.6B"
+gpu_mem_util = 0.06
+max_model_len = 8192
+attention_backend = "TRITON_ATTN"
+enforce_eager = true
+```
+
+### Why these knobs
+
+- **`senses` runs on Ampere because the checkpoint is weight-only FP4.**
+  `coolthor/gemma-4-12B-it-NVFP4A16` is compressed-tensors **W4A16** — weights
+  dequantise through vLLM's Marlin/FP4 path (compute capability ≥ 8.0), so
+  sm_87 qualifies. Activations stay 16-bit; no Blackwell FP4 tensor cores
+  needed. The full stack — Marlin GEMM, torch.compile/inductor, the MTP
+  (`eagle_head`) draft, TRITON_ATTN — ran clean on sm_87 with the same pinned
+  vLLM nightly digest the Spark/Thor fleets use (`vllm/vllm-openai@sha256:7c5a…`,
+  vLLM 0.23.1rc1.dev672). vLLM's own arm64 release builds list 8.7 as an
+  aarch64-only target, which is why the multi-arch image carries the kernels.
+- **`gpu_mem_util = 0.45` is MEASURED, not computed** (the same discipline as
+  thor-lobe's 0.30). The first boot at util 0.30 (18.4 GiB of 61.3) was refused
+  by vLLM: after weights (~7.7 GiB), the MTP draft head, CUDA graphs
+  (0.75 GiB) and runtime overhead, only **2.25 GiB** KV remained where
+  `max_model_len=131072` needs **3.08 GiB** (vLLM's estimated max at 0.30 was
+  76,352). At 0.45 (~27.6 GiB) the same boot reports **18.86 GiB KV =
+  802,644 tokens = 6.12×** concurrency at full context. Do not copy Thor's
+  0.30 onto a 64 GB board for 128K.
+- **`max_model_len = 131072`** — the checkpoint's full native context. It fits;
+  no 64K/32K trim was needed on 64 GB once util was raised.
+- **`cortex feasible = false` (why no cortex).** The 27B primary
+  (`sakamakismile/Qwen3.6-27B-Text-NVFP4-MTP`) is **modelopt NVFP4 W4A4** —
+  its quantised *activations* need Blackwell FP4 tensor cores; sm_87 is
+  Ampere. This is a hard architecture line, not a memory tradeoff.
+- **Pooling gears mirror Thor's Jetson divergences as the conservative first
+  boot** (`TRITON_ATTN` both, reranker `enforce_eager`). Thor's FLASH_ATTN
+  pooling hang is sm_110-specific and was NOT re-tested on sm_87 — these are
+  safe-boot choices, not proven-necessary ones. Relaxing them (and measuring)
+  is open follow-up work.
+
+## Shape choice: `thor-lobe`, not `orin-small`
+
+The built-in `orin-small` reference shape (minor + pooling + audio, both heavy
+lobes dropped) was NOT what this deployment wanted — the assignment here is
+**Orin hosts the Gemma senses lobe** (most important role for this box, user
+decision on #127). `thor-lobe` hosts exactly `senses + embedder + reranker +
+stt + tts` and drops `cortex`, so it renders the right deployment on this box
+when composed over the operator profile:
+
+```bash
+lobes init --profile orin --shape thor-lobe --apply
+```
+
+Note the shape's own senses overrides (`gpu_mem_util=0.30`,
+`max_model_len=131072` — Thor-measured values) overlay the profile at render
+time; on this box the rendered `.env` needed `MULTIMODAL_GPU_MEM_UTIL=0.45`
+(the Orin-measured value above). `orin-small` itself remains unbooted and
+therefore still DECLARED/UNVALIDATED.
+
+## Jetson/sm_87 divergences found live (upstreaming candidates)
+
+1. **csv-mode GPU access.** This Orin's NVIDIA container toolkit (1.19.1,
+   `mode = "auto"`) resolves to legacy **csv** mode, where the compose
+   template's `deploy.resources.reservations.devices` (`--gpus`-style) request
+   fails at container create: *"invoking the NVIDIA Container Runtime Hook
+   directly … is not supported. Please use the NVIDIA Container Runtime"*.
+   Fix applied on-box: every GPU service's `deploy:` stanza in the deployed
+   `docker-compose.yml`/`docker-compose.audio.yml` replaced with
+   `runtime: nvidia`. **A re-init reverts this hand edit** — a template knob or
+   machine-strategy overlay is the real fix. (Thor presumably works because
+   JetPack 7 ships CDI mode.)
+2. **The Parakeet STT base image is Spark-specific.** `Dockerfile.parakeet`'s
+   base `scitrera/dgx-spark-vllm:0.16.0-t4` ships a torch with no sm_87
+   kernels — NeMo dies at model load with `CUDA error: no kernel image is
+   available for execution on the device` (observed live: 8 container
+   restarts). Working fix (staged on-box before local audio was dropped):
+   rebase the Dockerfile onto the fleet's own pinned vLLM nightly digest
+   (whose arm64 torch is sm_87-validated by senses itself) and add
+   `ENTRYPOINT []` so its CMD survives the base image's entrypoint. The same
+   mismatch plausibly explains stt/tts `ready=false` on the sm_110 Thor.
+3. **Unified-memory budgeting is tighter than the util sum.** Each vLLM engine
+   consumes CPU-side memory beyond `gpu_mem_util × total` (host buffers plus
+   the default 4 GiB `swap_space` per engine). With senses 0.45 + two 0.06
+   gears the box sat at **54/61 GiB used with zero swap configured** — the
+   util sum (0.57 ≈ 35 GiB) undercounts by ~19 GiB. Leave real headroom, or
+   audio sidecars/host workloads will OOM-race the fleet.
+
+## Mesh wiring (#127) from this box
+
+- `PRIMARY_PEER_ORIGIN=http://spark.tail0be7e0.ts.net:8001` — cortex referral
+  (the same operator-typed value the Thor uses). `PRIMARY_PEER_PROXY=true` +
+  `PRIMARY_PEER_API_KEY=<the Spark's inbound GATEWAY_API_KEY>` arm the
+  follow-the-referral forward. **VALIDATED LIVE 2026-07-17**: a
+  `model=cortex` chat request from this box answers from the Spark's 27B
+  (`X-Lobes-Proxied-By: http://spark.tail0be7e0.ts.net:8001`, HTTP 200),
+  caller's own credential stripped per the #127 contract.
+- `AUDIO_URL=http://100.127.105.72:8001` — `/v1/audio/*` is *configured* to
+  forward to the Spark's gateway instead of local sidecars, but **audio
+  cannot chain gateway→gateway as of 0.45.0** (found live, 2026-07-17): the
+  audio readiness probe GETs `<AUDIO_URL>/v1/health/ready` unauthenticated,
+  which a peer *gateway* 401s (its GET /v1/* namespace is bearer-gated) and
+  would 404 anyway (that endpoint belongs to the realtime *bridge*, and
+  gateways only path-route `/v1/audio/*`) — the probe returns False forever
+  and this box answers a permanent 503. Working peer-audio today means
+  pointing `AUDIO_URL` at the peer's realtime **bridge** directly (publish
+  its port, e.g. `8080`, on the peer's tailnet — the bridge is keyless and
+  serves `/v1/health/ready` itself). Audio is also **outside the #127
+  pairwise-auth contract** (four core roles only): the forward passes the
+  caller's own `Authorization` through verbatim, attaches no per-peer key,
+  and `stt`/`tts` get no `hosted_by` referral annotation. First-class audio
+  referral/proxy knobs are the phase-2 candidate this measures out.
+
+## Reproducing on another Orin
+
+```bash
+uv tool install lobes-cli               # or: uv sync in a checkout
+mkdir -p ~/.lobes/profiles              # then write profiles/orin.toml (above)
+lobes init --profile orin --shape thor-lobe --apply
+# csv-mode boards: swap the deploy.resources GPU stanzas for `runtime: nvidia`
+# (divergence 1 above) until a template knob lands.
+lobes up senses --apply                 # boot order: senses first, then gears
+lobes up embedder --apply && lobes up reranker --apply
+docker compose -f docker-compose.yml -f docker-compose.shape.yml up -d gateway
+```
+
+Boot sequentially — the machine-profiles boot-ordering caveat (concurrent
+first boots race vLLM's free-memory measurement on unified boards) applies to
+this card too.

{lobes_cli-0.45.0 → lobes_cli-0.45.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "lobes-cli"
-version = "0.45.0"
+version = "0.45.2"
 description = "lobes — run, assess, and switch the local vLLM model."
 readme = "README.md"
 license = "Apache-2.0"

{lobes_cli-0.45.0 → lobes_cli-0.45.2}/uv.lock

@@ -440,7 +440,7 @@ wheels = [
 
 [[package]]
 name = "lobes-cli"
-version = "0.45.0"
+version = "0.45.2"
 source = { editable = "." }
 
 [package.optional-dependencies]