@miller-tech/uap 2.48.1 → 2.48.3

package/config/model-registry.json

@@ -14,7 +14,6 @@
     "qwen38-gsq-rco-27b",
     "qwen3.8-flash-next-iq3_s",
     "swift-1.5-iq3_s",
-    "Qwen3.8-27B",
     "Qwen3.8-27B-EXL3-3.5bpw"
   ],
   "models": {

package/docs/plans/droid-client-performance-options.md

@@ -306,3 +306,164 @@ Finding 1.
   "Opus 5.5 / Fable 5.2 levels" lever on this box today.
 - **R2-D — Re-measure after R2-A:** the post-client-update window is the
   honest post-fix read; era B's zeros become testable at population scale.
+
+---
+
+## Monitoring round 3 (2026-11-14 late) — the post-fix read at scale, and what the instrument was actually measuring
+
+Post-state: client 2.48.2 global (O-C hints + readGate + turnpack live),
+proxy 2.48.x with the O-D shim, baseModelId shipped (Droid treats the local
+arm as the qwen3.8-max built-in class). The first window with BOTH layers
+fixed: 916 calls, 09:50–11:15 UTC, 13 runs.
+
+### The fixes hold at scale — the first true post-fix read
+
+| Class | Era A (pre, n=19,794) | Era C (post, n=916) |
+|---|---:|---:|
+| ok | 75.5% | **80.2%** |
+| edit-miss | 3.2% | **0** |
+| path-not-found | 2.6% | **0** |
+| stub-refusal | 5.4% | 7.6% (all write_file) |
+| refused | 2.2% | 3.3% (all run_bash) |
+
+**edit-miss and path-not-found — the two classes O-C and turnpack were
+built for — are zero at n=916**, including adversarial fixtures designed
+to trigger them. ok-rate +4.7 points. The lever that remains un-shrunk is
+stub-refusal.
+
+### The instrument finding: era B and era C are both mostly TEST traffic
+
+The era-C stub-refusals all carry fixture paths (`src/big.ts`, `big.js`,
+`js/player.js` — deliberate gutting attempts at turn=1 in deliver-gate
+test tasks, i.e. the guard PASSING). Top era-C paths are fixtures
+(`a.js` 380, `b.js`/`big.js` 70 each); the real-path remainder is small.
+Era B's round-2 samples were the same fixtures. **The tool-calls DB pools
+benchmark-harness traffic with live-client traffic**, so corpus-level
+"class rates" blend a guard's self-tests with real client behavior —
+future monitoring must segment first (the working recipe: filter on the
+fixture path set + `.uap-deliver/verify.sh`, or segment by `task_id`).
+Era-C's honest split: fixture traffic shows the fixed layer passing its
+own adversarial tests; the live-client signal is thin (the kubectl
+deliver contributed 24 kubectl/doctl-pathed calls).
+
+### The live client cost evidence
+
+The kubectl deliver (the live interactive client, floor arm via
+local-auto, `--max-turns 20`) is **2h29m in** on a credential-permission
+ops task. The O-B reference: 6 gate-verified tasks in 4.7 min on glm-5.3.
+Arm + arm-tier remains the dominant cost factor.
+
+### The levers now (R3)
+
+- **R3-A — Frontier-class sessions route by tier (the literal ask):** the
+  Factory catalog now carries `claude-opus-5-5` (1.6x) and
+  `claude-fable-5.2` (3.2x) as built-ins — "Opus 5.5 / Fable 5.2 levels"
+  is a MODEL ROUTING choice, one flag per session. Measured reference
+  (glm-5.3 on the O-B suite): identical gates at 1/24 the tool volume and
+  1/10 the wall time. Suggested split: glm-5.3 default (measured, cheap),
+  opus-5-5/fable-5.2 for the hardest tier, local arm for
+  cost-insensitive/background work.
+- **R3-B — v31 arm switch (was O-A/R2-B, still unexecuted):** iq2_xs
+  still serves all client work; the strata unit additionally sits in the
+  panel-race latch state (failed+disabled with a live engine process
+  holding :8080). Recovery + switch is the documented one-liner
+  (`systemctl --user reset-failed` + `model-switch.sh strata`) — operator
+  action (infra-protect).
+- **R3-C — baseModelId seam (shipped, unmeasured):** local-arm Droid
+  sessions now inherit qwen3.8-max-class prompts, tool conventions,
+  reasoning levels, and Script-mode eligibility. The first local-arm
+  Droid sessions post-change (a live three.js client was streaming
+  through the proxy at 22:31) are the watch window.
+- **R3-D — telemetry run-kind tagging:** the era-B/C misread argues for a
+  `run_kind` tag (test vs live) at the classifier, or at minimum the
+  fixture-path segmentation recipe documented here. Small change, next
+  tools round.
+- **R3-E — stub-refusal remains the top live class:** in live traffic it
+  is the gutting guard + readGate doing their jobs; the open question is
+  whether read-after-refusal recovery got FASTER (429 successful reads in
+  era C vs the era-A re-send loops). Needs a live-session segment large
+  enough to read.
+
+---
+
+## Monitoring round 4 (2026-11-15 early) — the per-turn budget is the wall-clock bottleneck the arm ladder alone can't fix
+
+Post-state: client 2.48.2 global (O-C + turnpack live in every new
+session), proxy 2.48.x with the O-D shim, baseModelId live, and the GPU
+operating point shipped by the infra round (bf16 unlocked on the fleet
+binaries, 370 W power limit, SM clock locked at 1800 MHz — idle-verified
+188 W/1800 MHz/65 °C; boot-default confirmation pending the next reboot).
+
+### Finding 1 — the live client's real-path calls went 34/34 ok (first clean live run)
+
+The kubectl deliver (floor arm, `--max-turns 20`) ended at ~2h45m with
+**every real-path tool call ok** — 34/34 kubectl/doctl-pathed calls, max
+turn 12, and its final 14 calls (11:42–11:46 UTC) all ok (8 read_file,
+4 list_dir, 1 edit_file, 1 write_file). The fixed tool layer's first
+complete live run shows zero failures on real paths; the failure classes
+still visible in the corpus (round 3) are the guards' own fixtures. The
+deliver exited without writing its `[deliver]` lesson (no memory entry, no
+task record, no OOM, no reboot — cause unknown; the telemetry DB is its
+only record).
+
+### Finding 2 — the per-turn generation budget, now measured
+
+A live session on the floor arm was streaming with the engine capping
+`thinking limited to 24576 of max_tokens 32768` — 21 capped generations
+in ~17 minutes, i.e. routine thinking-heavy turns near the 32k budget.
+At the measured deep-context decode rate (126 tok/s) a full-budget turn
+is a ~4.3-minute floor **before tool volume even starts**. The O-B suite
+hit the 480 s ceiling on 5/6 local tasks after the file work was done,
+and the kubectl deliver had a 45-minute stretch with zero tool calls
+(one long turn). Tool volume (O-B's ~24x) and per-turn budget multiply
+into the ~10x wall gap.
+
+### Finding 3 — the GPU operating point is measurable on the serving arm
+
+Inference health on the live arm post-operating-point: decode
+113 → 126 tok/s in the >30k-depth bucket (n=6/6), speculative drafts
+accepted 67%, prompt-cache reuse 56% (~17,560 tok re-read per turn),
+vram 3,046 MiB free. The arm tier (R3-B) and the operating point stack.
+The box is otherwise loaded (100/121 GB used, load 13–22), so idle-window
+reruns will read cleaner than this sample.
+
+### The levers now (R4)
+
+- **R4-A — Thinking-budget discipline (new, the biggest untried
+  wall-clock lever):** routine tool turns do not need 24.5k thinking
+  tokens. The engine reserves 8k for the answer at max_tokens 32768
+  (the serve log names the knob: `"answer_room": false` turns the cap
+  off); a lower thinking budget — or a lower client max_tokens for
+  tool-heavy turns — cuts the per-turn floor from ~4.3 min toward
+  ~1.5–2 min (12k thinking ≈ 95 s at 126 tok/s). Verify gate parity on
+  the R4-B rerun. Knobs: the strata engine config (operator) or request
+  shaping at the proxy (UAP, a tools-round change).
+- **R4-B — Arm tier + operating point, measured together (R3-B + the
+  GPU round):** rerun the archived O-B paired suite on v31 under the
+  new operating point against the archived iq2_xs baseline — one
+  repeatable run quantifies the stacked local-arm gain (tier + 11%
+  decode + bf16). Still gated on the operator arm switch (the strata
+  unit's latch-state recovery).
+- **R4-C — Draft acceptance 67% → higher:** each acceptance point is
+  free decode; the speculative drafter's spec config (topK/epsilon) is
+  tunable on the strata engine.
+- **R4-D — Cache reuse 56%:** 17.5k tok/turn re-read at depth; the
+  health tool found no prefill bucket with enough samples to split, so
+  measure prefill in the R4-B rerun (the Sharp template + turnpack
+  context discipline target this).
+- **R4-E — Deliver-lesson gap:** abnormal deliver termination writes no
+  `[deliver]` lesson, so the kubectl run's outcome exists only in
+  telemetry. Write the lesson on every exit path — small harness change,
+  next tools round.
+
+Routing unchanged from R3-A: frontier-class sessions route by tier
+(glm-5.3 measured default; opus-5-5 / fable-5.2 for the hardest tier).
+Outcome equivalence is already demonstrated (O-B gates TIED 13/13);
+R4 is about closing the wall-clock gap on the local arm.
+
+**Caveats (labeled):** the decode 113→126 comparison is bucket-vs-bucket
+in the engine's ring log (no timestamps; `--since` could not be applied),
+n=6/6 directional. The live-cadence count (21 caps/~17 min) is journal
+grep on the serve log, one session. The 34/34 ok count covers real-path
+calls only; the deliver's gate outcome is unknown (Finding 1's lesson
+gap).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@miller-tech/uap",
-  "version": "2.48.1",
+  "version": "2.48.3",
   "description": "Autonomous AI agent memory system with CLAUDE.md protocol enforcement",
   "type": "module",
   "main": "dist/index.js",

package/scripts/lib/droid-sync-contracts.sh

@@ -66,6 +66,33 @@ _UAP_DROID_SYNC_CONTRACTS_LOADED=1
 # local model id — it must never carry a qwen id.
 : "${DROID_ANTHROPIC_BASE_MODEL:=claude-sonnet-4-6}"
 
+# DROID_CHAT_BASE_MODEL — the Factory BUILT-IN catalog id written into
+# `baseModelId` of the OpenAI-shaped local entries (the proxy chat /v1 route
+# and the direct Responses :8080 route). qwen3.8-max is the choice because the
+# served stack IS the Qwen3.8 family (Flash-Next 125B arms and its Edge0-145B
+# latent-MoE derivative): the entry inherits the built-in's Qwen3.8-class
+# prompts, tool-call conventions, reasoning levels, and Script-mode
+# eligibility instead of tripping the "does not map to a built-in model"
+# warning. Same family, same catalog — this is a BUILT-IN id, never a local
+# wire id: it must never be the served alias, a retired arm, or anything the
+# registry's advertises lists carry. (The prefix-era guard "never a qwen id"
+# predates the Factory catalog gaining Qwen3.8 built-ins; the exact local-id
+# check lives in the sandbox suite.)
+: "${DROID_CHAT_BASE_MODEL:=qwen3.8-max}"
+
+# DROID_LOCAL_27B_PIN / DROID_LOCAL_27B_DISPLAY — the SECOND local family the
+# Droid pickers list alongside the live Flash-Next alias: the registry's
+# qwen3.8-27b family (the llama.cpp / yskn / vLLM 27B arms — same :4000/:8080
+# endpoints, the engine serves whichever arm is switched live, so the entry
+# is a FAMILY LISTING, not a route switch; when placement turns on, the pin
+# is registry-matchable). The pin must be an id that family advertises —
+# checked against config/model-registry.json at sync time, fail-closed to
+# skip-adding. A camelCase entry carrying the 27B marker id
+# (custom:Qwen…-27B-UAP-local-N) is a DELIBERATE listing: the convergence
+# loop never repoints or renames it, it only stamps baseModelId when absent.
+: "${DROID_LOCAL_27B_PIN:=Qwen3.8-27B}"
+: "${DROID_LOCAL_27B_DISPLAY:=Qwen3.8 27B (UAP local)}"
+
 # DROID_SYNC_LOCK — lock file serialising concurrent Droid config writes
 # (session-start hook + manual installer run + a second Droid window).
 # Without it two writers read-modify-write the same JSON and one set of
@@ -95,19 +122,37 @@ fi
 #            in place or left alone. "0" = patch only when present.
 #   Reads from the caller's scope: MODEL_ALIAS, PROXY_URL, DIRECT_URL,
 #   CTX_SESSION, CTX_POOL, MAX_OUTPUT, DROID_ANTHROPIC_BASE_MODEL,
+#   DROID_CHAT_BASE_MODEL, DROID_LOCAL_27B_PIN, DROID_LOCAL_27B_DISPLAY,
 #   PROXY_TOKEN (via UAP_SYNC_TOKEN env), DRY_RUN.
 #   Migration rules (idempotent; no change => file left byte-identical):
-#     * entries whose model is a RETIRED local id — or already the live
-#       alias, or model-less but pointed at :4000/:8080 — converge onto
-#       MODEL_ALIAS;
+#     * entries whose model is a RETIRED local id — already the live alias,
+#       a FLEET wire id (any registry advertises entry) pointed at this
+#       box's :4000/:8080, or model-less but pointed at :4000/:8080 —
+#       converge onto MODEL_ALIAS (fleet ids are gated on a local base:
+#       the compat names collide with real cloud ids, so a cloud-router
+#       entry carrying one survives untouched);
+#     * a camelCase entry whose id carries the 27B marker
+#       (custom:Qwen…-27B-UAP-local-N) is a DELIBERATE family listing:
+#       it is skipped by the convergence loop (only baseModelId is
+#       stamped when absent) and is preferred over non-marker members
+#       in its dedupe group;
 #     * route by existing shape: baseUrl with :8080 -> direct strata route
 #       (pool-sized); :4000 WITHOUT /v1 -> Anthropic Messages root (proxy
-#       root + baseModelId + apiKey only if absent, session-sized); anything
-#       else local -> proxy chat route ($PROXY_URL/v1 + token);
+#       root + baseModelId=DROID_ANTHROPIC_BASE_MODEL + apiKey only if
+#       absent, session-sized); anything else local -> proxy chat route
+#       ($PROXY_URL/v1 + token); every camelCase local route also gains
+#       baseModelId (anthropic shape -> DROID_ANTHROPIC_BASE_MODEL, OpenAI
+#       shapes -> DROID_CHAT_BASE_MODEL) when absent, so Droid inherits the
+#       built-in's prompts/tools/reasoning instead of warning the entry
+#       "does not map to a built-in model";
 #     * cloud models (anything else, including the :8317 router entries) are
 #       deliberate user choices and are never repointed;
 #     * a settings.json with NO local entry after migration gains the
-#       canonical proxy entry (add-if-missing);
+#       canonical proxy entry (add-if-missing), and gains the 27B family
+#       entry (model=DROID_LOCAL_27B_PIN on the proxy chat route, marker
+#       id, baseModelId) when the pin is an id the registry's qwen3.8-27b
+#       family advertises — the two-family listing: Flash-Next live alias
+#       + the 27B family, side by side in every picker;
 #     * the 192.168.1.165:8317 split-brain host is fixed everywhere;
 #     * writes are atomic (tmp file in the same directory + os.replace),
 #       under the flock, with backups taken by the caller's backup().