@miller-tech/uap 2.39.1 → 2.44.1

package/config/capacity-policy.json

@@ -72,6 +72,31 @@
         "gpuMinFreeMiB": 200,
         "_headroom_comment": "200, not the 600 the llama.cpp-era entries use: strata deliberately fills the card (--vram-reserve-mib 1200 is its OWN internal headroom, reported reserve-aware at /metrics), and the measured steady state with the desktop sharing the GPU is ~245 MiB free. A 600 floor would hold the doctor permanently RED at the operator's chosen operating point and train everyone to ignore it. 200 still alerts when something NEW eats the card or strata loses its reserve."
       }
+    },
+    {
+      "name": "edge0-145b-server",
+      "systemd": {
+        "unit": "uap-edge0-145b-server.service",
+        "scope": "user"
+      },
+      "budget": {
+        "vramMiB": 23376,
+        "note": "Edge0 145B latent-MoE on the edge0-strata engine since 2026-11-13: uap-edge0-145b-server.service runs edge0-infer serve, which delegates to edge0-strata (build/strata + .venv python) and proxies :8080 (--port 8080, registered as --alias edge0_145b). The engine launch template (edge0-strata/edge0/models/edge0-145b.json) sets expert cache auto, prefill auto, spec 4 / min-p 0.5, MTP, int8 KV and 131k context; the unit adds only --vram-reserve-mib 1200, the same desktop-co-tenancy reserve as the strata-server stack, so vramMiB is the card minus that reserve (24576 - 1200). The 51.2 GiB PLE table is ple-io=direct (read from disk, not resident); 40.1 GiB experts SSD-streamed with hot experts in VRAM. execStartMustContain pins the load-bearing words of the note against the unit's own ExecStart — the command lives inline in the unit, not in a wrapper script, exactly so this check has something to check. Dormant (inactive) reads DARK like every switched-away :8080 backend.",
+        "execStartMustContain": [
+          "--model /home/cogtek/models/Edge0-145B-q3k",
+          "--port 8080",
+          "--alias edge0_145b",
+          "--vram-reserve-mib",
+          "1200"
+        ]
+      },
+      "headroom": {
+        "gpuMinFreeMiB": 200
+      },
+      "restartBudget": {
+        "knownRestarts": 0,
+        "allowedNew": 0
+      }
     }
   ]
 }

package/config/model-profiles/qwen38.json

@@ -1,10 +1,10 @@
 {
   "_profile": "qwen38",
   "_description": "Qwen3.8 Flash-Next — local Strata serve-layer deployment for agentic tool-calling workloads. Successor to qwen36 and to the 2026-09-20 GSQ-RCO llama.cpp deployment; the default local executor in UAP routing. Routes through the anthropic-proxy for the tool/finalize guardrails, NOT the inference server directly.",
-  "_engine": "strata serve layer 0.1.44.1 (build_info 'Strata 0.1.44.1', compiled locally with the user-space CUDA 13.4 toolkit) on :8080, launched by the systemd user unit uap-strata-server.service with /home/cogtek/dev/strata/strata-swift-iq3_s.json since the 2026-10-17 Swift 1.5 A/B trial (revert: repoint the unit's --config at strata-iq3_s.json, daemon-reload, restart; the unit file carries the dated note). Serves llama.cpp-shaped /props, /slots and /v1, plus a JSON /metrics (engine/live/requests — NOT Prometheus text; `uap inference health` auto-detects it). Weights: strata-data pack swift-iq3_s (Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S, the UkisAI reasoning-efficient post-train, since the 2026-10-17 trial; predecessor base Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S since the 2026-10-07 swap per the IQ3_S speed≈IQ3_XXS reports; earlier predecessors Swift 1.5 IQ3_XXS 2026-10-06, Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS, native experts + PLE shard) with MTP speculative drafts (spec 8 since the 2026-10-06 tuning; spec-min-p 0.70 after the 0.1.44.1-era calibration, previously 0.8, before that 0.5; previously spec 4, mtp_max 4) and an auto-sized expert cache. Model id advertised at /v1/models: swift-1.5-iq3_s; the server ignores requested model ids entirely, which is why stale client pins (e.g. a resumed session still naming qwen36 or qwen3.8-flash-next ids) keep working.",
+  "_engine": "strata serve layer 0.1.59.1 on :8080, launched by the systemd user unit uap-strata-abliterated-server.service with /home/cogtek/dev/strata/strata-abliterated-iq2_xs.json (the operator switched the fleet back from the v31 arm on 2026-11-13; the v31 default had run since 2026-11-11, and the panel switch log carries both moves — switch/revert: ~/.config/uap/model-switch.sh strata|strata-swift|strata-abliterated). Serves llama.cpp-shaped /props, /slots and /v1, plus a JSON /metrics (engine/live/requests — NOT Prometheus text; `uap inference health` auto-detects it). Weights: strata-data pack abliterated-iq2_xs (DavidAU abliterated Flash-Next IQ2_XS, full expert pinning; the pristine Unsloth Dynamic v3.1 UD-IQ4_XS arm 2026-11-11 -> 2026-11-13 and the Swift 1.5 IQ3_S arm 2026-10-17 -> 2026-10-28, both kept switchable) with MTP speculative drafts (spec 12, spec-min-p 0.7, mtp_max 8 — live /metrics 2026-11-13: spec window 12, mtp_max 8, spec_min_p 0.70, lookup 3), int8 KV resident 32768. Model id advertised at /v1/models: abliterated-iq2_xs; the server ignores requested model ids entirely, which is why stale client pins (e.g. a resumed session still naming qwen36 or qwen3.8-flash-next ids) keep working.",
   "_engine_history": "SUPERSEDED 2026-10-04: this profile described buun-llama-cpp behind uap-gsq-rco-server.service (2 rails sharing a 229376-cell VBR pool, DFlash2 draft). That unit is now INACTIVE — the box serves the Strata layer on :8080 instead (tuned 2026-10-03: +15% over the llama.cpp fork on agent workloads). The old description was itself a 2026-09-20 correction of a ninfer-serve description. Verify against /metrics before trusting engine claims here; the dead unit's ExecStart still reports -c 229376 to systemctl, and on 2026-10-04 `uap inference health` printed that stale pool size for the live (131072) backend.",
   "model": "abliterated-iq2_xs",
-  "_model_comment": "The alias the live backend advertises at /v1/models; the rails suite drift-checks this field against the serving process (test/model-profiles-qwen38-rails.test.ts). 2026-10-17: the Swift 1.5 IQ3_S A/B trial swapped the unit from the base pack, which advertised qwen3.8-flash-next-iq3_s. The server ignores requested model ids, so clients pinning the old id are unaffected; on revert, restore this field alongside the unit's --config. 2026-10-28: the unit default moved to the DavidAU abliterated IQ2_XS arm (uap-strata-server -> strata-abliterated-iq2_xs.json), so the live alias is abliterated-iq2_xs; updated here the same day the rails drift-check caught it (the _engine description above still documents the Swift 1.5 A/B trial era).",
+  "_model_comment": "The alias the live backend advertises at /v1/models; the rails suite drift-checks this field against the serving process (test/model-profiles-qwen38-rails.test.ts). 2026-10-17: the Swift 1.5 IQ3_S A/B trial swapped the unit from the base pack, which advertised qwen3.8-flash-next-iq3_s. The server ignores requested model ids, so clients pinning the old id are unaffected; on revert, restore this field alongside the unit's --config. 2026-10-28: the unit default moved to the DavidAU abliterated IQ2_XS arm (uap-strata-server -> strata-abliterated-iq2_xs.json), so the live alias is abliterated-iq2_xs; updated here the same day the rails drift-check caught it. 2026-11-11: the default arm moved to the pristine Unsloth Dynamic v3.1 UD-IQ4_XS base quant (uap-strata-server -> strata-unsloth-ud-iq4_xs-v31.json), so the live alias was qwen3.8-flash-next-unsloth-ud-iq4_xs-v31. 2026-11-13: the operator switched the fleet back to the abliterated arm (panel switch to uap-strata-abliterated-server.service), so the live alias is abliterated-iq2_xs again; updated here the same day the rails drift-check caught it.",
   "max_tokens": 57344,
   "_max_tokens_comment": "Deliberately generous: 50% of the 114688 window, the same ratio the previous 65536/131072 pairing used. The engine returns chain-of-thought in a SEPARATE reasoning_content field, so reasoning spends completion budget without ever appearing in content. A tight cap therefore does not produce a short answer, it produces an EMPTY one with finish_reason=length — measured previously as ~90% first-turn decode failure that read as bad output format and was really budget truncation. Do not lower this without re-measuring decode compliance. (The proxy additionally clamps tool turns to PROXY_TOOL_TURN_MAX_TOKENS=32768 regardless.)",
   "temperature": 0.3,
@@ -12,7 +12,7 @@
   "top_k": 20,
   "min_p": 0.05,
   "repetition_penalty": 1.0,
-  "_sampling_comment": "Carried over from qwen36 unchanged: these are UAP's tool-call-tuned values, not the vendor's chat defaults. They were re-measured only against the llama.cpp GSQ-RCO engine, never against strata — strata's own defaults (spec 4, spec-min-p 0.5) affect drafting, not base sampling, so the values are carried as-is. Re-tune with `uap tune` before changing them.",
+  "_sampling_comment": "Carried over from qwen36 unchanged: these are UAP's tool-call-tuned values, not the vendor's chat defaults. They were re-measured only against the llama.cpp GSQ-RCO engine, never against strata — strata's own defaults (spec 12, spec-min-p 0.5) affect drafting, not base sampling, so the values are carried as-is. Re-tune with `uap tune` before changing them.",
   "stop_sequences": [
     "<|im_end|>"
   ],
@@ -51,14 +51,15 @@
     "speculative_decoding": {
       "enabled": true,
       "type": "mtp",
-      "draft_model": "strata-data/mtp/rt (MTP head, not a separate draft model)",
-      "draft_max": 4,
-      "draft_min_p": 0.5,
-      "_comment": "Strata's native MTP speculative path (--spec 4 --spec-min-p 0.5, mtp_max 4, plus a lookup-3 secondary). Replaces the DFlash2 draft model of the llama.cpp era: observed 2026-10-04 at 84-89% acceptance on agent workloads (vs 62% on one 311-token outlier), which is what carries decode to the observed 46-52 tok/s."
+      "draft_model": "/home/cogtek/models/strata-data/mtp/rt (the model's own draft layer, shared with the other strata arms)",
+      "draft_max": 12,
+      "draft_conf": 0.7,
+      "_comment": "2026-11-13: the serving arm is again strata-abliterated-iq2_xs.json, which drafts with the model's own MTP layer, pinned by \"drafting\": \"mtp\" in its config — live /metrics: spec window 12, mtp_max 8, spec_min_p 0.70, lookup 3 (the v31 arm that served 2026-11-11 -> 2026-11-12 drafted at spec 12 / min-p 0.50 / mtp_max 4). The DFlash2 flip did NOT hold anywhere: the auto plan for this model class (spec 6 / conf 0.7, the tuned strata1 pack) OOM-killed the engine at start twice on 2026-11-11 and the serve wrapper's DIDNT_START fallback landed on MTP; sm_86 is also the card class where upstream measured DFlash2 slower than MTP (strata_control/drafting.py _SLOWER) and this box's own 2026-10-28 A/B rejected the stock drafter (-5.5% mean decode, bench/results/2026-10-28-dflash2-vs-mtp). A DFlash2 retry needs a smaller RAM budget and a fresh measured A/B — it stays documentation, never a live value here. The 2026-10-04 acceptance figure in `measured` predates every drafter here: re-measure on real agent traffic before quoting a number."
     },
     "_comment": "These are the flags run-iq3_xxs.sh actually runs strata with, recorded so a rebuild reproduces the measured behaviour. The load-bearing pair (kv int8, max-context 131072) is pinned in config/capacity-policy.json under the strata-server http entry, where `uap doctor` verifies them against /metrics."
   },
   "measured": {
+    "_arm_comment": "The figures below were measured 2026-10-04 on the pre-arm-swap strata base pack (IQ3_S-era weights). The 2026-11-11 move to the unsloth UD-IQ4_XS v3.1 arm and the 2026-11-13 switch back to the abliterated arm both keep the engine, the 131072 pool and the int8 KV tier, so they carry as estimates; the drafting numbers differ per arm (v31: spec 12 / 0.50 / mtp_max 4; abliterated: spec 12 / 0.70 / mtp_max 8 — recorded in server_optimization), and decode/acceptance on THIS arm were never re-measured (uap tune / scripts/bench-decode-reps.sh) before quoting them as its figures.",
     "date": "2026-10-04",
     "gpu": "NVIDIA GeForce RTX 3090, 24576 MiB",
     "_gpu_comment": "Strata's own reserve-aware /metrics reported ~950 MiB free under live load (--vram-reserve-mib 1200 already held out; arena 40925 MiB across host+device, expert cache ~14.5 GiB). Host-wide free via nvidia-smi is the doctor's number; engine.vram_free_mib is strata's view, and both are checked now.",