@miller-tech/uap 2.44.1 → 2.45.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/config/model-profiles/qwen38.json +5 -5
- package/config/model-registry.json +28 -1
- package/docs/benchmarks/PROMPT_UPLIFT_BLENDER.md +334 -56
- package/docs/guides/LOCAL_MODELS.md +9 -1
- package/package.json +1 -1
- package/src/policies/enforcers/__pycache__/_common.cpython-311.pyc +0 -0
- package/src/policies/enforcers/__pycache__/expert_review_required.cpython-311.pyc +0 -0
- package/src/policies/enforcers/__pycache__/quality_metrics_gate.cpython-311.pyc +0 -0
- package/templates/hooks/__pycache__/deliver_autoroute.cpython-311.pyc +0 -0
- package/tools/agents/scripts/__pycache__/toolcall_path_normalizer.cpython-311.pyc +0 -0
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_profile": "qwen38",
|
|
3
3
|
"_description": "Qwen3.8 Flash-Next — local Strata serve-layer deployment for agentic tool-calling workloads. Successor to qwen36 and to the 2026-09-20 GSQ-RCO llama.cpp deployment; the default local executor in UAP routing. Routes through the anthropic-proxy for the tool/finalize guardrails, NOT the inference server directly.",
|
|
4
|
-
"_engine": "strata serve layer 0.1.59.1 on :8080, launched by the systemd user unit uap-strata-
|
|
4
|
+
"_engine": "strata serve layer 0.1.59.1 on :8080, launched by the systemd user unit uap-strata-server.service with /home/cogtek/dev/strata/strata-unsloth-ud-iq4_xs-v31.json (the v31 default has run since 2026-11-11, except an abliterated excursion the operator reverted on 2026-11-14 — the abliterated unit's last active window was 2026-11-13 -> 2026-11-14, and the panel switch log carries both moves; switch/revert: ~/.config/uap/model-switch.sh strata|strata-swift|strata-abliterated). Serves llama.cpp-shaped /props, /slots and /v1, plus a JSON /metrics (engine/live/requests — NOT Prometheus text; `uap inference health` auto-detects it). Weights: strata-data pack unsloth-UD-IQ4_XS-v31 (the pristine Unsloth Dynamic v3.1 UD-IQ4_XS arm, the :8080 default since 2026-11-11; the DavidAU abliterated Flash-Next IQ2_XS arm 2026-11-13 -> 2026-11-14 and the Swift 1.5 IQ3_S arm 2026-10-17 -> 2026-10-28, both kept switchable) with MTP speculative drafts (spec 12, spec-min-p 0.5, mtp_max 4 — live /metrics 2026-11-14: spec window 12, mtp_max 4, spec_min_p 0.50), int8 KV resident 32768. Model id advertised at /v1/models: qwen3.8-flash-next-unsloth-ud-iq4_xs-v31; the server ignores requested model ids entirely, which is why stale client pins (e.g. a resumed session still naming qwen36 or qwen3.8-flash-next ids) keep working.",
|
|
5
5
|
"_engine_history": "SUPERSEDED 2026-10-04: this profile described buun-llama-cpp behind uap-gsq-rco-server.service (2 rails sharing a 229376-cell VBR pool, DFlash2 draft). That unit is now INACTIVE — the box serves the Strata layer on :8080 instead (tuned 2026-10-03: +15% over the llama.cpp fork on agent workloads). The old description was itself a 2026-09-20 correction of a ninfer-serve description. Verify against /metrics before trusting engine claims here; the dead unit's ExecStart still reports -c 229376 to systemctl, and on 2026-10-04 `uap inference health` printed that stale pool size for the live (131072) backend.",
|
|
6
|
-
"model": "
|
|
7
|
-
"_model_comment": "The alias the live backend advertises at /v1/models; the rails suite drift-checks this field against the serving process (test/model-profiles-qwen38-rails.test.ts). 2026-10-17: the Swift 1.5 IQ3_S A/B trial swapped the unit from the base pack, which advertised qwen3.8-flash-next-iq3_s. The server ignores requested model ids, so clients pinning the old id are unaffected; on revert, restore this field alongside the unit's --config. 2026-10-28: the unit default moved to the DavidAU abliterated IQ2_XS arm (uap-strata-server -> strata-abliterated-iq2_xs.json), so the live alias is abliterated-iq2_xs; updated here the same day the rails drift-check caught it. 2026-11-11: the default arm moved to the pristine Unsloth Dynamic v3.1 UD-IQ4_XS base quant (uap-strata-server -> strata-unsloth-ud-iq4_xs-v31.json), so the live alias was qwen3.8-flash-next-unsloth-ud-iq4_xs-v31. 2026-11-13: the operator switched the fleet
|
|
6
|
+
"model": "qwen3.8-flash-next-unsloth-ud-iq4_xs-v31",
|
|
7
|
+
"_model_comment": "The alias the live backend advertises at /v1/models; the rails suite drift-checks this field against the serving process (test/model-profiles-qwen38-rails.test.ts). 2026-10-17: the Swift 1.5 IQ3_S A/B trial swapped the unit from the base pack, which advertised qwen3.8-flash-next-iq3_s. The server ignores requested model ids, so clients pinning the old id are unaffected; on revert, restore this field alongside the unit's --config. 2026-10-28: the unit default moved to the DavidAU abliterated IQ2_XS arm (uap-strata-server -> strata-abliterated-iq2_xs.json), so the live alias is abliterated-iq2_xs; updated here the same day the rails drift-check caught it. 2026-11-11: the default arm moved to the pristine Unsloth Dynamic v3.1 UD-IQ4_XS base quant (uap-strata-server -> strata-unsloth-ud-iq4_xs-v31.json), so the live alias was qwen3.8-flash-next-unsloth-ud-iq4_xs-v31. 2026-11-13: the operator switched the fleet to the abliterated arm (panel switch to uap-strata-abliterated-server.service), so the live alias was abliterated-iq2_xs; updated here the same day the rails drift-check caught it. 2026-11-14: the operator reverted to the v31 arm (uap-strata-server.service active again, the abliterated unit dead; live /v1/models advertises qwen3.8-flash-next-unsloth-ud-iq4_xs-v31), so the live alias is back to the v31 id; updated here the same day the rails drift-check caught it.",
|
|
8
8
|
"max_tokens": 57344,
|
|
9
9
|
"_max_tokens_comment": "Deliberately generous: 50% of the 114688 window, the same ratio the previous 65536/131072 pairing used. The engine returns chain-of-thought in a SEPARATE reasoning_content field, so reasoning spends completion budget without ever appearing in content. A tight cap therefore does not produce a short answer, it produces an EMPTY one with finish_reason=length — measured previously as ~90% first-turn decode failure that read as bad output format and was really budget truncation. Do not lower this without re-measuring decode compliance. (The proxy additionally clamps tool turns to PROXY_TOOL_TURN_MAX_TOKENS=32768 regardless.)",
|
|
10
10
|
"temperature": 0.3,
|
|
@@ -54,12 +54,12 @@
|
|
|
54
54
|
"draft_model": "/home/cogtek/models/strata-data/mtp/rt (the model's own draft layer, shared with the other strata arms)",
|
|
55
55
|
"draft_max": 12,
|
|
56
56
|
"draft_conf": 0.7,
|
|
57
|
-
"_comment": "2026-11-
|
|
57
|
+
"_comment": "2026-11-14: the serving arm is again the v31 pack (strata-unsloth-ud-iq4_xs-v31.json), which drafts with the model's own MTP layer, pinned by \"drafting\": \"mtp\" in its config — live /metrics 2026-11-14: spec window 12, mtp_max 4, spec_min_p 0.50 (the abliterated arm that served 2026-11-13 -> 2026-11-14 drafted at spec 12 / min-p 0.70 / mtp_max 8). The DFlash2 flip did NOT hold anywhere: the auto plan for this model class (spec 6 / conf 0.7, the tuned strata1 pack) OOM-killed the engine at start twice on 2026-11-11 and the serve wrapper's DIDNT_START fallback landed on MTP; sm_86 is also the card class where upstream measured DFlash2 slower than MTP (strata_control/drafting.py _SLOWER) and this box's own 2026-10-28 A/B rejected the stock drafter (-5.5% mean decode, bench/results/2026-10-28-dflash2-vs-mtp). A DFlash2 retry needs a smaller RAM budget and a fresh measured A/B — it stays documentation, never a live value here. The 2026-10-04 acceptance figure in `measured` predates every drafter here: re-measure on real agent traffic before quoting a number."
|
|
58
58
|
},
|
|
59
59
|
"_comment": "These are the flags run-iq3_xxs.sh actually runs strata with, recorded so a rebuild reproduces the measured behaviour. The load-bearing pair (kv int8, max-context 131072) is pinned in config/capacity-policy.json under the strata-server http entry, where `uap doctor` verifies them against /metrics."
|
|
60
60
|
},
|
|
61
61
|
"measured": {
|
|
62
|
-
"_arm_comment": "The figures below were measured 2026-10-04 on the pre-arm-swap strata base pack (IQ3_S-era weights). The 2026-11-11 move to the unsloth UD-IQ4_XS v3.1 arm
|
|
62
|
+
"_arm_comment": "The figures below were measured 2026-10-04 on the pre-arm-swap strata base pack (IQ3_S-era weights). The 2026-11-11 move to the unsloth UD-IQ4_XS v3.1 arm, the 2026-11-13 switch to the abliterated arm and the 2026-11-14 revert to the v31 arm all keep the engine, the 131072 pool and the int8 KV tier, so they carry as estimates; the drafting numbers differ per arm (v31: spec 12 / 0.50 / mtp_max 4; abliterated: spec 12 / 0.70 / mtp_max 8 — recorded in server_optimization), and decode/acceptance on THIS arm were never re-measured (uap tune / scripts/bench-decode-reps.sh) before quoting them as its figures.",
|
|
63
63
|
"date": "2026-10-04",
|
|
64
64
|
"gpu": "NVIDIA GeForce RTX 3090, 24576 MiB",
|
|
65
65
|
"_gpu_comment": "Strata's own reserve-aware /metrics reported ~950 MiB free under live load (--vram-reserve-mib 1200 already held out; arena 40925 MiB across host+device, expert cache ~14.5 GiB). Host-wide free via nvidia-smi is the doctor's number; engine.vram_free_mib is strata's view, and both are checked now.",
|
|
@@ -13,7 +13,9 @@
|
|
|
13
13
|
"qwen35-a3b-iq4xs",
|
|
14
14
|
"qwen38-gsq-rco-27b",
|
|
15
15
|
"qwen3.8-flash-next-iq3_s",
|
|
16
|
-
"swift-1.5-iq3_s"
|
|
16
|
+
"swift-1.5-iq3_s",
|
|
17
|
+
"Qwen3.8-27B",
|
|
18
|
+
"Qwen3.8-27B-EXL3-3.5bpw"
|
|
17
19
|
],
|
|
18
20
|
"models": {
|
|
19
21
|
"qwen3.6-27b": {
|
|
@@ -93,6 +95,30 @@
|
|
|
93
95
|
}
|
|
94
96
|
]
|
|
95
97
|
},
|
|
98
|
+
"strata-abliterated-iq3_s": {
|
|
99
|
+
"status": "unmeasured",
|
|
100
|
+
"engine": "strata",
|
|
101
|
+
"unit": "uap-strata-abliterated-iq3_s-server",
|
|
102
|
+
"note": "The DavidAU abliterated Flash-Next IQ3_S tier — the :8080 default 2026-10-23 -> 2026-10-28 (adopted over the Swift 1.5 arm on the same-day A/B), moved to the IQ2_XS tier in the 2026-10-28 kernel-OOM fix (full-pin 53.7 GB anon RSS; the low-RAM --resident-budget-gib 20 posture ships in strata-abliterated-iq3_s.json, and engine 0.1.59.1 supports the flag), re-staged 2026-11-14 as a switchable panel arm. Config strata-abliterated-iq3_s.json, unit uap-strata-abliterated-iq3_s-server.service, pack strata-data/packs/abliterated-iq3_s; advertises abliterated-iq3_s at /v1/models; switch: ~/.config/uap/model-switch.sh strata-abliterated-iq3_s (revert: strata | strata-abliterated). Run `uap models measure` against it before placement loads it (fail-closed). Raw GGUF shards declared here: the pack is derived from them by tools/iq_pack.py and rebuilds from these files; shard 2 is byte-identical across the release's tiers (the tier-shared digest also pinned in strata-abliterated-iq2_xs). Provenance is pinned (repo + rev, from feature/386): bytes + sha256 are the HF LFS oids, verified against the local pair at the 2026-10-23 adoption and re-verified against the repo digests 2026-11-14, so `uap models ensure` can re-fetch and verify from this pin.",
|
|
103
|
+
"files": [
|
|
104
|
+
{
|
|
105
|
+
"path": "/home/cogtek/models/strata-data/models/abliterated-IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf",
|
|
106
|
+
"repo": "DavidAU/Qwen3.8-Flash-Next-Abliterated-Uncensored-Turbo-Brilliance-50x-Reasoning-Instruct-GSQ-RCO-GGUF",
|
|
107
|
+
"rev": "623b419bd443e6e76ec51ca4b95d951da93c6969",
|
|
108
|
+
"file": "Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00001-of-00002.gguf",
|
|
109
|
+
"bytes": 55063664928,
|
|
110
|
+
"sha256": "78706315c2fe912777c006b4ae18be1024c9f4353bd25d33b8a306c9c55f9e34"
|
|
111
|
+
},
|
|
112
|
+
{
|
|
113
|
+
"path": "/home/cogtek/models/strata-data/models/abliterated-IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00002-of-00002.gguf",
|
|
114
|
+
"repo": "DavidAU/Qwen3.8-Flash-Next-Abliterated-Uncensored-Turbo-Brilliance-50x-Reasoning-Instruct-GSQ-RCO-GGUF",
|
|
115
|
+
"rev": "623b419bd443e6e76ec51ca4b95d951da93c6969",
|
|
116
|
+
"file": "Qwen3.8-Flash-Next-GSQ-RCO-abliterated-IQ3_S-00002-of-00002.gguf",
|
|
117
|
+
"bytes": 28800138432,
|
|
118
|
+
"sha256": "316b46f3a2dbd68c900f43136ab9449f9dcc3725dfd8c794847c204bc161e113"
|
|
119
|
+
}
|
|
120
|
+
]
|
|
121
|
+
},
|
|
96
122
|
"strata-unsloth-ud-iq4_xs-v31": {
|
|
97
123
|
"status": "unmeasured",
|
|
98
124
|
"engine": "strata",
|
|
@@ -137,6 +163,7 @@
|
|
|
137
163
|
"advertises": [
|
|
138
164
|
"qwen3.8-flash-next-unsloth-ud-iq4_xs-v31",
|
|
139
165
|
"abliterated-iq2_xs",
|
|
166
|
+
"abliterated-iq3_s",
|
|
140
167
|
"qwen3.8-flash-next-iq3_s",
|
|
141
168
|
"swift-1.5-iq3_s",
|
|
142
169
|
"gpt-5.1-codex"
|
|
@@ -1,9 +1,9 @@
|
|
|
1
1
|
# Prompt Uplift Field Study: a 2-bit Model Passes a Pixel Gate (2026-11)
|
|
2
2
|
|
|
3
|
-
**Date:** 2026-11-12 · **Subject:** `abliterated-iq2_xs` on a
|
|
4
|
-
contract, driven by bespoke harness tooling (the repo's UAP
|
|
5
|
-
supplies the memory store and worktree workflow here, not the
|
|
6
|
-
uplift subsystem to attribute a result to)
|
|
3
|
+
**Date:** 2026-11-12, revised 2026-11-13 · **Subject:** `abliterated-iq2_xs` on a
|
|
4
|
+
frozen Blender build contract, driven by bespoke harness tooling (the repo's UAP
|
|
5
|
+
install is 2.39.1; UAP supplies the memory store and worktree workflow here, not the
|
|
6
|
+
harness — there is no uplift subsystem to attribute a result to)
|
|
7
7
|
**Scope:** one measured prompt-uplift run against a frozen multi-gate acceptance
|
|
8
8
|
contract, and the four transferable lessons it produced. This is a dated study, not
|
|
9
9
|
a feature doc: the harness lives in the experiment repo, not here. It is an
|
|
@@ -14,6 +14,10 @@ causal claims. For the evergreen mechanism the study informs, see
|
|
|
14
14
|
[guides/DELIVER.md](../guides/DELIVER.md) (iterate against real completion gates) and
|
|
15
15
|
[guides/LOCAL_MODELS.md](../guides/LOCAL_MODELS.md).
|
|
16
16
|
|
|
17
|
+
The revision quarantines 15 of the 124 scored attempts: the `B-visual-v4` and
|
|
18
|
+
`B-visual-v5` window was served by a different model than the study recorded. See
|
|
19
|
+
"An arm switch went unnoticed for two stages".
|
|
20
|
+
|
|
17
21
|
## The experiment
|
|
18
22
|
|
|
19
23
|
Task: write a Blender Python script that builds a rigged, animated, game-ready
|
|
@@ -35,25 +39,114 @@ Model: `abliterated-iq2_xs` (a ~2-bit quantized small model), temp 0, **first at
|
|
|
35
39
|
only, no repair turns**. Each stage is one prompt revision, scored by `prompt_sha`, so
|
|
36
40
|
a score is attributable to an exact prompt.
|
|
37
41
|
|
|
38
|
-
`n` counts scored attempts —
|
|
39
|
-
`--dry-prompt` rows
|
|
40
|
-
scored 0, not excluded. `12/12` is the structural gate; `both gates` means 12/12
|
|
41
|
-
8/8 on the same first attempt.
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
|
46
|
-
|
|
47
|
-
|
|
|
48
|
-
|
|
|
49
|
-
| B-visual
|
|
50
|
-
| B-visual-
|
|
51
|
-
| B-visual-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
(
|
|
42
|
+
`n` counts scored attempts — 124 of the 130 rows in `.uap/uplift/results.jsonl`; the
|
|
43
|
+
six `--dry-prompt` rows print the prompt and record no score. A build that crashes
|
|
44
|
+
is scored 0, not excluded. `12/12` is the structural gate; `both gates` means 12/12
|
|
45
|
+
*and* 8/8 on the same first attempt. The `arm` column is the model that actually
|
|
46
|
+
served the attempt, established after the fact (see the quarantine section); the
|
|
47
|
+
runner now records `served_model` and scores 0 on `arm_mismatch`.
|
|
48
|
+
|
|
49
|
+
| stage | prompt change | n | arm | 12/12 | both gates | `sys` crashes |
|
|
50
|
+
|-------|---------------|---|-----|-------|------------|---------------|
|
|
51
|
+
| bare | raw prompt | 5 | IQ2_XS | 0 | n/a (no visual gate) | 0 |
|
|
52
|
+
| A-acceptance | 12-check contract + API crib sheet | 38 | IQ2_XS | 23 | n/a (no visual gate) | 0 |
|
|
53
|
+
| B-visual | + render/visual gate | 3 | IQ2_XS | 3 | 0 (visual 3/8–4/8) | 0 |
|
|
54
|
+
| B-visual-v2 | visual rules rewritten | 3 | IQ2_XS | 0 (all three builds crashed) | 0 | 0 |
|
|
55
|
+
| B-visual-v3 | tightened geometry rules | 3 | IQ2_XS | 2 | 0 (visual 6/8 twice) | 0 |
|
|
56
|
+
| B-visual-v4 | vertical limb chains, blade inside hand | 9 | **IQ4_XS v31** | 1 | 1 | 4 |
|
|
57
|
+
| B-visual-v4 | *same prompt*, re-run on the restored IQ2_XS arm (`--tag arm-rerun`) | 9 | IQ2_XS | 8 | 3 | 0 |
|
|
58
|
+
| B-visual-v5 | verbatim import header + copied fix loop | 6 | **IQ4_XS v31** | 6 | 4 (67%) | 0 |
|
|
59
|
+
| B-visual-v5 | *same prompt*, re-run on IQ2_XS | 6 | IQ2_XS | 5 | 2 | 0 |
|
|
60
|
+
| B-visual-v6 | read back the keyed fcurves before saving | 9 | IQ2_XS | 0 (all nine died in the build) | 0 | 0 |
|
|
61
|
+
| B-visual-v6 | `channelbag(create=)` → positional `ensure` | 3 | IQ2_XS | 0 (all three died on that kwarg) | 0 | 0 |
|
|
62
|
+
| B-visual-v6 | pose tables keyed to the 19 `bone_defs` names, appended self-check block at column 0, animation-amplitude rungs | 6 | IQ2_XS | 5 | 5 (83%) | 0 |
|
|
63
|
+
| **B-visual-v6** | *same prompt*, plus one rule: every pose-table row has exactly three elements | 6 | IQ2_XS | **5** | **5 (83%)** | 0 |
|
|
64
|
+
| **B-visual-v6** | *same prompt*, plus a rule banning assignment expressions inside pose tables | 6 | IQ2_XS | **4** | **4 (67%)** | 0 |
|
|
65
|
+
| **B-visual-v6** | *same prompt*, plus a named angle block, a 3-tuple row rule, and a row-shape self-check | 6 | IQ2_XS | **5** | **1 (17%)** | 0 |
|
|
66
|
+
| **B-visual-v6** | *same prompt*, plus the first-slot rule and a `SELFCHECK slot0=` self-check | 6 | IQ2_XS | **5** | **5 (83%)** | 0 |
|
|
67
|
+
|
|
68
|
+
First-attempt rate at *both* gates, measured on IQ2_XS alone, went 0/3 → 3/9 → 2/6 →
|
|
69
|
+
5/6 → 5/6 → 4/6 → 1/6 → 5/6 across the prompt revisions that survive the quarantine.
|
|
70
|
+
The 5/6 rate held across two consecutive prompt revisions; the revision after them
|
|
71
|
+
regressed to 4/6 with two failure modes that neither of its predecessors had shown. The
|
|
72
|
+
revision after *that* closed both of those modes and regressed further, to 1/6, on a
|
|
73
|
+
mode no earlier revision had produced. The revision after *that* — the same prompt plus
|
|
74
|
+
one rule about which Euler slot carries the animated angle — restored 5/6, so three
|
|
75
|
+
prompts now measure 5/6 and only the last of them closes the failure family its
|
|
76
|
+
predecessor exposed. The series reads as a record of which rung falls next rather than
|
|
77
|
+
of monotone improvement: each revision closed the family that was visible and left a
|
|
78
|
+
different one open. v4's nine IQ4-arm runs split four ways: 4 `sys` crashes, 3
|
|
79
|
+
reasoning truncations, 1 built-but-incomplete (11/12, 6/8), 1 clean pass.
|
|
80
|
+
|
|
81
|
+
## An arm switch went unnoticed for two stages
|
|
82
|
+
|
|
83
|
+
`B-visual-v4` and `B-visual-v5` — 15 attempts, the two stages that produced every
|
|
84
|
+
recorded both-gates pass in the original write-up — were not served by the model the
|
|
85
|
+
stage rows say they were. Both arms declare port 8080, so only one could own the
|
|
86
|
+
endpoint, and the runner recorded the *configured* model (`MODEL`,
|
|
87
|
+
`abliterated-iq2_xs`) rather than the served one; `served_model` and the
|
|
88
|
+
`arm_mismatch` guard were added afterwards.
|
|
89
|
+
|
|
90
|
+
Prompt-token counts are a fingerprint, but a weak one inside this window: the same
|
|
91
|
+
prompt file produces the same prompt for either arm, so `13868` and `14828` appear in
|
|
92
|
+
both engine logs. The completion-token count is per-attempt and separates them
|
|
93
|
+
cleanly — each window's counts appear in one log, in attempt order, and not in the
|
|
94
|
+
other at all.
|
|
95
|
+
|
|
96
|
+
| window | prompt tokens | `usage.completion_tokens` | IQ2_XS log | IQ4_XS v31 log |
|
|
97
|
+
|--------|--------------:|---------------------------|-----------:|---------------:|
|
|
98
|
+
| B-visual (n=3) | 8851 | — | 3 lines | 0 |
|
|
99
|
+
| B-visual-v2 (n=3) | 10389 | — | 3 | 0 |
|
|
100
|
+
| B-visual-v3 (n=3) | 10075 | — | 3 | 0 |
|
|
101
|
+
| B-visual-v4 originals (n=9) | 13908 ×3, 13868 ×6 | 5798, 5706, 5680, 5356, 5945, 5597 (6 untruncated) | 2 unrelated lines\* | 3 × 13908 + 6 × 13868 |
|
|
102
|
+
| B-visual-v4 reruns (n=9) | 13868 | 7311, 6741, 6830, 7126, 6163, 7482, 7471, 6927, 7085 | 9 | 0 |
|
|
103
|
+
| B-visual-v5 originals (n=6) | 14828 | 6597, 6585, 6472, 5225, 5189, 6211 | 0 | 6 |
|
|
104
|
+
| B-visual-v5 reruns (n=6) | 14828 | 6858, 5932, 7096, 6064, 6102, 6490 | 6 | 0 |
|
|
105
|
+
| B-visual-v6 (n=42, seven prompt revisions) | 15962 ×9, 16140 ×3, 16517 ×6, 16762 ×6, 16961 ×6, 17629 ×6, 18163 ×6 | 6496, 6208, 5844, 6397, 6791, 6385 (pose-table prompt); 7194, 6738, 6460, 6589, 6764, 6664 (pose-table + row-arity prompt); 6814, 6566, 5774, 6420, 6430, 6316 (pose-table + no-assignment-expression prompt); 6900, 6513, 6415, 7181, 6486, 6832 (pose-table + named angle block); 7363, 6683, 6636, 6743, 7075, 7003 (pose-table + first-slot rule) | 9, 3, 6\*, 6, 6, 6, 6 | 0 |
|
|
106
|
+
|
|
107
|
+
\* Two `13868` lines and one `16517` line in the IQ2_XS log belong to unrelated work:
|
|
108
|
+
they generated 634, 178 and 110 tokens rather than any of the attempt counts above,
|
|
109
|
+
and one of them reused 16,512 of 16,517 tokens. Excluding them, every prompt-token
|
|
110
|
+
line in the window belongs to the arm the completion-token column says it does. The
|
|
111
|
+
last four windows' twenty-four completion counts each appear only in the IQ2_XS log, in
|
|
112
|
+
attempt order: the `16762` window at lines 272807–272924, the `16961` window at lines
|
|
113
|
+
276731–276971, the `17629` window at lines 283670–283950, the `18163` window at lines
|
|
114
|
+
284394, 284446, 284452, 284478, 284512 and 284562, every one of them at its window's
|
|
115
|
+
prompt-token count. The `18163` prompt count appears in neither the IQ4_XS v31 log nor
|
|
116
|
+
the swift-1.5-iq3_s log at all, and that window's six completion counts appear in
|
|
117
|
+
neither. The `16961` window has exactly seven such lines in the
|
|
118
|
+
IQ2_XS log — the six attempts plus one unrelated 19-token generation at line 104791 —
|
|
119
|
+
and one line in the IQ4_XS log at an unrelated 1,378-token generation. The `16762`
|
|
120
|
+
window's `6738` also shows up once in that log at an unrelated 42,964-token prompt, and
|
|
121
|
+
in the IQ4_XS log not at all. The `17629` prompt count appears once in the IQ4_XS log,
|
|
122
|
+
at line 17123 with 237 generated tokens; none of that window's six completion counts
|
|
123
|
+
appear in that log at all.
|
|
124
|
+
|
|
125
|
+
The same log lines carry generation speed, which corroborates: ~150–170 tok/s on
|
|
126
|
+
IQ2_XS against ~80 tok/s on IQ4_XS v31, and the reruns' recorded `gen_elapsed`
|
|
127
|
+
implies IQ2_XS speed. Port 8080 declared `abliterated-iq2_xs` (`GET /v1/models`)
|
|
128
|
+
throughout the rerun windows — it has since been restored to the fleet default
|
|
129
|
+
`strata` — and every `B-visual-v6` row carries
|
|
130
|
+
`served_model=abliterated-iq2_xs` with no `arm_mismatch`.
|
|
131
|
+
|
|
132
|
+
Consequences:
|
|
133
|
+
|
|
134
|
+
- The published `0/3 → 1/9 → 4/6` headline spanned two arms and is replaced above.
|
|
135
|
+
- Every both-gates pass in the original table came from the IQ4_XS arm, and so did
|
|
136
|
+
v4's four `sys` `NameError` crashes, its three reasoning truncations, and the
|
|
137
|
+
17 px walk-cycle failure. Those are not 2-bit-model behaviour.
|
|
138
|
+
- Re-running the same two prompts on the restored IQ2_XS arm still clears the pixel
|
|
139
|
+
gate: 3/9 at v4 and 2/6 at v5, so the claim survives — at a lower rate, and with
|
|
140
|
+
the improvement continuing to 5/6 at v6 for two consecutive prompt revisions, each
|
|
141
|
+
of which cleared both gates five of six attempts. The third revision in that series
|
|
142
|
+
regressed to 4/6 and the fourth to 1/6, so the step to 5/6 is real but neither later
|
|
143
|
+
revision is the one to keep. The fifth revision — the same prompt plus a rule naming
|
|
144
|
+
which Euler slot carries the animated angle — restored 5/6 and closed the family the
|
|
145
|
+
fourth had exposed, so it is the one to keep.
|
|
146
|
+
- The mechanism is lesson 1 with the verifier swapped for the model: integrity
|
|
147
|
+
established after the fact instead of enforced at run time. One line in the
|
|
148
|
+
runner — compare `served_model` to `MODEL` before scoring — would have caught the
|
|
149
|
+
switch on the first affected attempt.
|
|
57
150
|
|
|
58
151
|
## Four lessons
|
|
59
152
|
|
|
@@ -67,8 +160,25 @@ out of the way, but nothing enforces the header's sentence at runtime: the model
|
|
|
67
160
|
script runs as ordinary Blender Python with write access to its workspace, and the
|
|
68
161
|
workspace sits inside the same repo as the originals. Integrity was therefore
|
|
69
162
|
established *after* the fact: pinned hash matches live, and every workspace carried
|
|
70
|
-
byte-identical gate copies
|
|
71
|
-
visual-gate copies
|
|
163
|
+
byte-identical gate copies — one distinct hash across all 117 validator copies, all 78
|
|
164
|
+
visual-gate copies, all 78 render-script copies and all 117 `gates.md` copies, covering
|
|
165
|
+
the 124 scored runs.
|
|
166
|
+
|
|
167
|
+
That audit has one weakness worth naming. `make_workspace` builds
|
|
168
|
+
`runs/<stage>/attempt-N` and `--tag` is **not** in the path, so a rerun `rmtree`s and
|
|
169
|
+
rewrites the directory its predecessor used. The 124 scored attempts land in 102 attempt
|
|
170
|
+
directories, 19 of which were written more than once — 18 pairs (the nine
|
|
171
|
+
`B-visual-v4` and six `B-visual-v5` originals plus their `arm-rerun` twins, and the
|
|
172
|
+
first three `B-visual-v6` attempts plus their `arm-probe` twins) and `bare` attempt 1
|
|
173
|
+
five times, 22 extra writes in total — and the hash audit can only see the **last
|
|
174
|
+
writer** of each. The 15 IQ4-arm workspaces survive only because the quarantine copied
|
|
175
|
+
them to
|
|
176
|
+
`runs/B-visual-v{4,5}-arm-iq4_xs-v31/` before the rerun; those 15 archived directories
|
|
177
|
+
have no rows in `results.jsonl`, so the 117 directories on disk outnumber the 102 that
|
|
178
|
+
carry scores. So "no tampering was found" is a statement about the surviving directory
|
|
179
|
+
contents, not about every attempt that ever ran: the same-hash-per-copy result holds
|
|
180
|
+
for every directory on disk, and the attempt that matters for a pass claim is the one
|
|
181
|
+
whose score row and workspace agree on the same writer.
|
|
72
182
|
|
|
73
183
|
Be precise about the strength of that. The runner copies the gates but does not
|
|
74
184
|
re-verify the hashes *inside* the run, and the workspace copy is writable, so the
|
|
@@ -79,7 +189,7 @@ Closing that gap is cheap: recompute the workspace gate hashes after the model's
|
|
|
79
189
|
script has written and before the gates execute, and score the attempt 0 on mismatch.
|
|
80
190
|
Four `hashlib` calls in the runner turn "no tampering was found afterwards" into
|
|
81
191
|
"tampering could not have helped". The transferable part stands either way: without a
|
|
82
|
-
pinned verifier, a
|
|
192
|
+
pinned verifier, a pass rate is unfalsifiable, because the cheapest path to 12/12
|
|
83
193
|
is editing the checker. This is the
|
|
84
194
|
same `enforce, don't suggest` stance as everywhere else in UAP, applied to the verifier
|
|
85
195
|
rather than the producer.
|
|
@@ -88,10 +198,38 @@ rather than the producer.
|
|
|
88
198
|
|
|
89
199
|
Every run records `prompt_sha` alongside its score, and prompt files compose with
|
|
90
200
|
`#extends <stage>` so a revision is a delta, not a rewrite. That is what made v4
|
|
91
|
-
diagnosable: the geometry rules demonstrably worked
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
201
|
+
diagnosable: the geometry rules demonstrably worked — attempt 5 hit both gates, and
|
|
202
|
+
its static numbers matched the offline geometry test that had been run before the
|
|
203
|
+
stage shipped: 842 tris, feet 0.0000 m, grip 0.0000 m, predicted island ratio 0.088
|
|
204
|
+
against a 0.30 limit and measured 0.087. Its clip diffs came out smaller than the
|
|
205
|
+
offline prediction (predicted 0.020/0.080/0.029/0.161, measured 0.021/0.068/0.027/
|
|
206
|
+
0.084) but every one cleared the movement threshold, so the pass was not a fluke.
|
|
207
|
+
Without per-revision attribution, "v4 is worse than v3" would have buried a working
|
|
208
|
+
fix under an unrelated crash.
|
|
209
|
+
|
|
210
|
+
That pass was an IQ4_XS-served attempt, so the attribution needed the arm column too:
|
|
211
|
+
the same prompt on IQ2_XS cleared both gates 3 of 9 times, which confirms the geometry
|
|
212
|
+
rules rather than the arm. The isolation came from the appended `sys` crash and the
|
|
213
|
+
reasoning truncations, both of which the quarantine section shows were IQ4-side.
|
|
214
|
+
|
|
215
|
+
The same attribution is what made the arm switch recoverable rather than fatal: the
|
|
216
|
+
rerun rows carry the identical `prompt_sha` as the originals and differ only by
|
|
217
|
+
`--tag`, so the two arms can be separated attempt by attempt instead of discarded.
|
|
218
|
+
It also paid for itself forward: v6's five pose-table revisions are a demonstrable
|
|
219
|
+
uplift step, not a lucky streak. The three rungs that failed in the v4 and v5 IQ2_XS
|
|
220
|
+
reruns — the attack clip losing all its weapon pixels (`gap 999`, four times at v4,
|
|
221
|
+
twice at v5), a walk cycle lifting feet past the limit, and jump endpoints leaving the
|
|
222
|
+
root off the floor — each got a named rule, and the next six attempts cleared all eight
|
|
223
|
+
rungs five times. The revision after that, which added one rule and changed the prompt
|
|
224
|
+
hash, cleared them five times again in a second six-attempt window, so the step did not
|
|
225
|
+
rest on one prompt's luck. A third revision, banning assignment expressions inside pose
|
|
226
|
+
tables, cleared them only four times in its own six-attempt window and introduced two
|
|
227
|
+
failure modes its predecessors had not seen; a fourth, which closed both of those,
|
|
228
|
+
cleared them once; and a fifth, which added one rule about which Euler slot carries the
|
|
229
|
+
animated angle, cleared them five times again and closed the family the fourth had
|
|
230
|
+
exposed — which is why the honest headline is 5/6 three times, not 5/6 and rising, and
|
|
231
|
+
why the improvement claim belongs to the prompts that measured 5/6 rather than to the
|
|
232
|
+
series of edits.
|
|
95
233
|
|
|
96
234
|
### 3. A regression can be a missing word, not a wrong idea
|
|
97
235
|
|
|
@@ -110,52 +248,192 @@ runs each happened to write `import bpy, sys, os`, while v4's four crashing runs
|
|
|
110
248
|
instruction, different coin flip. v5 states the header as four lines to copy verbatim,
|
|
111
249
|
`import sys` included, and the crash count went to 0/6 with the validator at 6/6.
|
|
112
250
|
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
251
|
+
State the arm caveat: those four crashes were IQ4_XS runs. On IQ2_XS the v4 rerun had
|
|
252
|
+
0 `sys` crashes and 8/9 at 12/12, so the crash lesson cannot be separated from the
|
|
253
|
+
arm switch by this log alone — but v5's IQ2_XS rerun also had 0 `sys` crashes, and
|
|
254
|
+
its one build failure was an `IndentationError`, a different mode. The instruction
|
|
255
|
+
that matters is the one the failure names: when a prompt asks for a construct, state
|
|
256
|
+
the import or declaration that construct needs, as text to copy rather than prose to
|
|
257
|
+
understand. Small models rewrite headers; they do not reliably infer them.
|
|
258
|
+
|
|
259
|
+
The same shape showed up twice more in v6, both as prompt-text defects rather than
|
|
260
|
+
model defects. The appended post-save self-check block sits indented 4 spaces in the
|
|
261
|
+
prompt file for display, and one v6 run pasted it verbatim into a top-level script and
|
|
262
|
+
died on `IndentationError: unexpected indent` — fixed by a rule that says to paste it
|
|
263
|
+
at column 0; an `IndentationError` also ended one v5 IQ2_XS rerun. And `Action.fcurves`
|
|
264
|
+
does not exist in Blender 5.2: seven v6 runs died on that `AttributeError` — four in
|
|
265
|
+
the original window, three in the arm probe — *after* printing correct self-check
|
|
266
|
+
values and *after* saving the file, so the build exited 0 while the attempt scored 0.
|
|
267
|
+
Every scored row in the log reports `build_rc` 0; a crash is caught from the build
|
|
268
|
+
tail, never from the return code. That is why the prompt
|
|
269
|
+
now ships a measured, verified read path (`act.layers → strips → channelbags → fcurves`)
|
|
270
|
+
and an explicit ban list instead of letting the model invent one.
|
|
116
271
|
|
|
117
272
|
### 4. Reasoning budget is a hard constraint, not a tuning knob
|
|
118
273
|
|
|
119
274
|
Three v4 runs left reasoning on: the model spent 17k–27k reasoning characters before
|
|
120
275
|
writing any code and hit `finish_reason=length` mid-table, at both 8k and 12k max
|
|
121
|
-
tokens, producing an unterminated tuple and a `SyntaxError`. All three scored 0.
|
|
276
|
+
tokens, producing an unterminated tuple and a `SyntaxError`. All three scored 0. Those
|
|
277
|
+
three were IQ4_XS runs.
|
|
122
278
|
|
|
123
|
-
The pattern holds across the whole log, not just v4. Of
|
|
124
|
-
reasoning on and
|
|
279
|
+
The pattern holds across the whole log, not just v4. Of 124 scored runs, 15 had
|
|
280
|
+
reasoning on and 109 had it off. Every one of the 109 reasoning-off runs finished with
|
|
125
281
|
`finish=stop` — none truncated. All 8 truncated runs in the log had reasoning on, and
|
|
126
282
|
all 8 scored 0. Reasoning on did not become safe at a bigger budget either: the 7
|
|
127
283
|
reasoning-on runs that were given 32k–64k tokens did finish, and their best score was
|
|
128
|
-
11/12 — no reasoning-on run in the log ever cleared the structural gate.
|
|
129
|
-
|
|
130
|
-
|
|
284
|
+
11/12 — no reasoning-on run in the log ever cleared the structural gate. 12 of the 15
|
|
285
|
+
reasoning-on runs predate the arm switch (the `bare` and `A-acceptance` stages, recorded
|
|
286
|
+
as IQ2_XS), and the conclusion is unchanged on that subset alone: 5 truncated, best
|
|
287
|
+
score 11/12, no 12/12. Meanwhile v5, run with
|
|
288
|
+
`--reasoning-effort none --max-tokens 12000`, scored 12/12 six of six on the IQ4 arm
|
|
289
|
+
and five of six on IQ2_XS, and v6's five pose-table revisions scored 12/12 five, five,
|
|
290
|
+
four, five and five of six on IQ2_XS.
|
|
291
|
+
A truncated run is a budget misconfiguration, not a model failure, and the budget
|
|
131
292
|
must be pinned in the invocation for scores to stay comparable across stages.
|
|
132
293
|
|
|
133
|
-
## Where the
|
|
294
|
+
## Where the remaining failures sit
|
|
134
295
|
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
296
|
+
On the IQ2_XS arm the pixel gate's two animation rungs were the last thing standing for
|
|
297
|
+
most of the study, and both were amplitude, not geometry:
|
|
298
|
+
|
|
299
|
+
- **`vis-weapon-at-hand`** — 4 of 9 v4 reruns and 2 of 6 v5 reruns. Every failure
|
|
300
|
+
reported `gaps {'attack.mid': 999}` or `{'attack.end': 999}`: `999` means *no
|
|
301
|
+
weapon pixels found in that frame*, not "drifted a little". The attack pose rotated
|
|
302
|
+
the blade edge-on out of the camera plane.
|
|
303
|
+
- **`vis-feet-on-floor`** — 1 v4 rerun (13 px at `jump.end`, limit 12) and 1 v5 rerun
|
|
304
|
+
(26 px at `walk.end`).
|
|
139
305
|
|
|
140
306
|
The structural gate measures the rest pose, so both builds pass 12/12 and fail the
|
|
141
|
-
pixel gate. Its `feet-on-floor` rung
|
|
142
|
-
|
|
143
|
-
every rendered frame. That gap, a structural check
|
|
144
|
-
state, is the argument for the visual gate existing
|
|
307
|
+
pixel gate. Its `feet-on-floor` rung measures the lowest world-Z of any `Foot*` mesh
|
|
308
|
+
vertex in the rest pose against a 0.005 m tolerance — a static measurement — while the
|
|
309
|
+
visual gate measures the lift in every rendered frame. That gap, a structural check
|
|
310
|
+
that never exercises the animated state, is the argument for the visual gate existing
|
|
311
|
+
at all.
|
|
312
|
+
|
|
313
|
+
For scale, the logged margins say the improvement was in headroom, not only in the
|
|
314
|
+
pass rate. All 45 runs that cleared the island rung measured 0.083–0.089 of the bbox
|
|
315
|
+
diagonal against a 0.30 limit except `rule-f` attempt 16 at 0.113, still well under the
|
|
316
|
+
limit, while the five runs that failed it measured 3.142–3.338
|
|
317
|
+
at `B-visual` and 4.504 and 6.393 at `B-visual-v3`; all five `rule-i` clears measured
|
|
318
|
+
0.087. On the floor rung, eight of the
|
|
319
|
+
eleven IQ2_XS runs that cleared it at v4 or v5 reported a max lift of exactly 12 px —
|
|
320
|
+
the limit itself, at `jump.start` — so "cleared" there was a coin flip against the
|
|
321
|
+
next frame. v6's twenty-four clearing runs reported 1–9 px, none at the limit, and all
|
|
322
|
+
five `rule-i` clears reported 9 px at `walk.start` with the feet touching the floor line
|
|
323
|
+
at 1 px on `idle.start`. The rules moved the margin.
|
|
324
|
+
|
|
325
|
+
v6's pose-table prompt is what closed the two rungs: the attack clip keys `upper_arm_R`
|
|
326
|
+
only (0 → −70° → 0, forearm and hand never keyed, so the blade stays face-on to the
|
|
327
|
+
camera), the walk cycle keys both thighs ±20° with shins capped at ±12°, and the jump
|
|
328
|
+
holds root-z 0 at both endpoints with endpoint thigh keys at 0. Its failures after that
|
|
329
|
+
came in two families: names and expressions written inside pose-table rows, and, once
|
|
330
|
+
those were gone, which Euler slot carries the animation angle. Each revision closed one
|
|
331
|
+
family while the model found the other:
|
|
145
332
|
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
333
|
+
- attempt 10 wrote a pose-table row with two tuple elements where `build_clip`
|
|
334
|
+
unpacked three (`ValueError: not enough values to unpack (expected 3, got 2)`), and
|
|
335
|
+
the prompt's existing "one tuple shape for every row" rule covers the *parts* table,
|
|
336
|
+
not the pose tables.
|
|
337
|
+
- That rule shipped in the revision after it, which scored 5/6 again. Its one failure
|
|
338
|
+
was a second member: attempt 19 put a walrus target that is not a name inside a
|
|
339
|
+
pose-table row (`SyntaxError: cannot use assignment expressions with expression`, at
|
|
340
|
+
`(-d30 := math.radians(-30.0), d0, d0)`), so the arity rule fixed the shape of the
|
|
341
|
+
row without constraining the expressions inside it.
|
|
342
|
+
- The revision after that banned assignment expressions inside pose tables outright.
|
|
343
|
+
It scored 4/6, and its two failures were two more members of the same family rather
|
|
344
|
+
than a repeat of either: attempt 24 mixed types in the rotation slot — `d0` is the
|
|
345
|
+
3-tuple `(0.0, 0.0, 0.0)` while `d10, d20, dm10, dm20, dm70` are scalar
|
|
346
|
+
`math.radians(…)` values, and rows pass those bare names straight into
|
|
347
|
+
`Euler(rot, 'XYZ')` — so `Euler()` received a scalar where it wanted a triple and
|
|
348
|
+
died with the bare message `TypeError: mathutils.Euler()`; re-executing its tables in
|
|
349
|
+
order, the first failing call is `Idle / spine_03 / frame 24` with `rot = 0.1745`,
|
|
350
|
+
five rows *upstream* of the walrus row that first drew attention to this attempt.
|
|
351
|
+
Attempt 25 referenced `dm30` in a row without ever assigning it (`NameError: name
|
|
352
|
+
'dm30' is not defined. Did you mean: 'dm10'?`) — it obeyed the ban on `:=` and the
|
|
353
|
+
arity rule, then used a name the prompt's own crib sheet does not define.
|
|
354
|
+
|
|
355
|
+
Banning one syntax did not make the rows plain. The next revision shipped exactly that
|
|
356
|
+
stronger rule: a named angle block above the tables (`d0, d10, dm10, d20, dm20, dm30,
|
|
357
|
+
dm70`, all scalar `math.radians(…)`), a requirement that every row be `(int frame,
|
|
358
|
+
3-tuple location, 3-tuple rotation)`, an explicit ban on bare scalars and inline
|
|
359
|
+
`math.radians()` in the rotation slot, and a pre-keyframe self-check printing
|
|
360
|
+
`SELFCHECK pose_rows=<count> all_rows_3=True rot_all_triples=True`. It closed the
|
|
361
|
+
family: all six attempts used only the seven listed angle names, none used `:=`, five
|
|
362
|
+
printed the self-check line, and the structural rate rose to 5/6. It also regressed the
|
|
363
|
+
both-gates rate to 1/6, on a failure no earlier revision had produced.
|
|
364
|
+
|
|
365
|
+
Four of those five structural passes failed `vis-clips-move` with identical numbers:
|
|
366
|
+
`walk` diff 0.0037, `attack` diff 0.0007, only `idle` and `jump` moving against a
|
|
367
|
+
"3 of 4" requirement. The cause is which Euler slot carries the angle. All four keyed
|
|
368
|
+
the attack as `(d0, dm70, d0)`; the one pass keyed it `(dm70, d0, d0)`. Rebuilding the
|
|
369
|
+
same script in a scratch workspace with nothing changed but that slot ordering moved the
|
|
370
|
+
attack diff from 0.0007 to 0.0296 and the visual score from 7/8 to 8/8; moving the walk
|
|
371
|
+
thigh angles to the first slot as well took walk from 0.0037 to 0.0768. The limb bones
|
|
372
|
+
run along their local −Z, so a Y-slot rotation mostly foreshortens a limb instead of
|
|
373
|
+
sweeping it across the camera planes, and the gate's pixel-difference test never sees it.
|
|
374
|
+
|
|
375
|
+
That rule shipped in the fifth revision, which scored 5/6 again. It changed RULE H's
|
|
376
|
+
worked example from `(d0, dm20, d0)` to `(dm20, d0, d0)` and added RULE I: the measured
|
|
377
|
+
failure, the mechanism, the requirement that the animated angle go in the *first* slot
|
|
378
|
+
of the rotation triple, and a self-check line `SELFCHECK slot0=<count>`. All six
|
|
379
|
+
attempts reported the same 18,163-token prompt, five cleared both gates, and an
|
|
380
|
+
independent scan of the scripts' syntax trees found non-zero rotation slots only in the
|
|
381
|
+
first position — eight or ten animated rows per script, and zero second-slot patterns,
|
|
382
|
+
where every rule-h failure had *only* second-slot patterns. RULE H's family stayed
|
|
383
|
+
closed: no `Euler()` type errors, no undefined angle names, no `:=`. The one failure was
|
|
384
|
+
a returning family, not a new one: attempt 39 wrote a **7-element** row into the
|
|
385
|
+
parts table at line 96 (`Forearm_R`, with the location tuple duplicated into the size
|
|
386
|
+
slot) against the 6-name unpack at line 148, and the build died on `ValueError: too many
|
|
387
|
+
values to unpack (expected 6)` before it reached keyframing or the save at lines 377 and
|
|
388
|
+
401, so `robot.blend` was never written and the runner scored 0 before validating or
|
|
389
|
+
rendering. `build_rc` is 0 because Blender 5.2 exits 0 on an uncaught script exception
|
|
390
|
+
under `-b -P`; the traceback is in the row's build tail, and the workspace holds the
|
|
391
|
+
18,155-character script but no `.blend`. Every self-check the earlier revisions added
|
|
392
|
+
lives *below* the crash (line 263 onward), so none of them ran for that attempt. It is
|
|
393
|
+
the fourth unpack-arity failure in the log — `A-acceptance` attempt 9 died on `bone_defs`
|
|
394
|
+
(`expected 4, got 3`), `B-visual-v2` attempt 1 on `parts` (`too many values … expected
|
|
395
|
+
5`), `prompt-fix-rerun` attempt 10 on the pose keys (`expected 3, got 2`) — and the
|
|
396
|
+
second on the parts table. RULES F, G, H and I pin the *pose* tables' row shape; the
|
|
397
|
+
prompt shows a parts table to copy but never states its field count or asks for a
|
|
398
|
+
self-check on it.
|
|
399
|
+
|
|
400
|
+
So the same lesson as the structural rungs, one level down: the rung that falls next is
|
|
401
|
+
the one the prompt has not described yet, and a rule that fixes types can unmask a rule
|
|
402
|
+
about axes. The slot rule is now measured rather than recommended, and it restored 5/6
|
|
403
|
+
while closing the family its predecessor exposed. Two rungs remain open, both below the
|
|
404
|
+
tables the prompt already pins. First, state the parts table's field count and order and
|
|
405
|
+
require a printed count of rows that match it, before any mesh is built. Second, the
|
|
406
|
+
artifact itself: require the script to save, then assert the saved file exists and is
|
|
407
|
+
non-empty, and print its path and size, so a clean exit that wrote nothing cannot score
|
|
408
|
+
as a build.
|
|
151
409
|
|
|
152
410
|
## Reproduce
|
|
153
411
|
|
|
154
412
|
Harness, prompts, findings, and per-attempt workspaces live in the experiment repo
|
|
155
413
|
(`blender-test-1`): `tooling/uplift_runner.py`, `.uap/uplift/prompts/`,
|
|
156
|
-
`.uap/uplift/FINDINGS-B-visual-v{4,5}.md`,
|
|
414
|
+
`.uap/uplift/FINDINGS-B-visual-v{4,5,6}.md`,
|
|
415
|
+
`.uap/uplift/FINDINGS-arm-contamination.md`, `.uap/uplift/runs/`.
|
|
157
416
|
|
|
158
417
|
```
|
|
159
|
-
python3 tooling/uplift_runner.py --stage B-visual-
|
|
160
|
-
--reasoning-effort none --max-tokens 12000
|
|
418
|
+
python3 tooling/uplift_runner.py --stage B-visual-v6 --attempt N \
|
|
419
|
+
--reasoning-effort none --max-tokens 12000 --tag rule-i-rerun
|
|
161
420
|
```
|
|
421
|
+
|
|
422
|
+
Pin `--reasoning-effort none` and `--max-tokens 12000` (lesson 4), keep `--tag` so an
|
|
423
|
+
attempt served by a different arm stays separable in `results.jsonl` (the quarantine
|
|
424
|
+
section) — the tagged rerun windows used `--tag arm-rerun`, `api-fix-rerun`,
|
|
425
|
+
`prompt-fix-rerun`, `rule-f-rerun`, `rule-g-rerun`, `rule-h-rerun` and `rule-i-rerun`,
|
|
426
|
+
and `.uap/uplift/rerun-arm.sh`, `.uap/uplift/rerun-rule-g.sh`,
|
|
427
|
+
`.uap/uplift/rerun-rule-h.sh` and `.uap/uplift/rerun-rule-i.sh` drive whole windows —
|
|
428
|
+
and confirm the endpoint before scoring: `GET /v1/models` on port 8080 must return
|
|
429
|
+
`abliterated-iq2_xs`. Note that
|
|
430
|
+
`--tag` does **not** change the workspace path, so a rerun overwrites the attempt
|
|
431
|
+
directory of any earlier run with the same `--stage` and `--attempt`; archive the
|
|
432
|
+
directory first if the original run's artifacts still matter, as the quarantine did.
|
|
433
|
+
The rule windows therefore used fresh attempt numbers — 10–15, 16–21, 22–27, 28–33 and
|
|
434
|
+
34–39 — so no earlier workspace was overwritten. As of this revision the on-disk
|
|
435
|
+
`B-visual-v6` prompt carries the first-slot rule (RULE I) on top of the named-angle
|
|
436
|
+
block (RULE H): composed prompt sha `e6733cb043d58a8f`, 18,163 prompt tokens, attempts
|
|
437
|
+
34–39.
|
|
438
|
+
`tooling/uplift_runner.py` also now hard-fails on a fifth chained `#extends` link or a
|
|
439
|
+
cycle (`EXTENDS_MAX_DEPTH`), where it used to silently truncate the composition.
|
|
@@ -194,7 +194,15 @@ truth the rails suite drift-checks against the live `/props` on :8080), then:
|
|
|
194
194
|
- pins `baseModelId` on Anthropic-protocol entries (default
|
|
195
195
|
`claude-sonnet-4-6`, override `DROID_ANTHROPIC_BASE_MODEL`) so the entry
|
|
196
196
|
inherits model-specific prompts, tools, and Script-mode eligibility instead
|
|
197
|
-
of the "custom model does not map to a built-in" fallback behavior
|
|
197
|
+
of the "custom model does not map to a built-in" fallback behavior;
|
|
198
|
+
- collapses duplicate picker lines: entries sharing `(model, baseUrl,
|
|
199
|
+
displayName)` are one line rendered twice — Droid re-copies legacy
|
|
200
|
+
`config.json` entries into `settings.json` under fresh ids each copy era,
|
|
201
|
+
so each era leaves a full extra set; a group keeps the canonical entry,
|
|
202
|
+
else its newest member;
|
|
203
|
+
- repoints dead `modelFavorites` pins from the local-entry id family onto the
|
|
204
|
+
surviving local entry, so the favorites bar keeps selecting the local stack
|
|
205
|
+
across arm swaps instead of rendering a dead line.
|
|
198
206
|
|
|
199
207
|
For "at any given time" freshness, run it with `MODELS_ONLY=1`: that mode
|
|
200
208
|
converges **only** the `.factory` custom-model lists, takes the alias from the
|
package/package.json
CHANGED
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|