omnilane 0.42.2 → 0.42.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +2 -2
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +76 -1
- package/README.ja.md +62 -0
- package/README.ko.md +62 -0
- package/README.md +65 -0
- package/README.zh-CN.md +59 -0
- package/README.zh-TW.md +59 -0
- package/VERSION +1 -1
- package/docs/release-notes-0.42.5.md +99 -0
- package/package.json +2 -2
- package/plugin.json +1 -1
- package/scripts/doctor.sh +26 -0
- package/scripts/lib/aa_policy.py +29 -2
- package/scripts/lib/build_overlay.py +189 -0
- package/scripts/lib/overlay_health.py +91 -0
- package/scripts/lib/probe.py +184 -0
- package/skills/omnilane/SKILL.md +90 -1
- package/docs/release-notes-0.42.2.md +0 -31
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Omnilane 0.42.5
|
|
2
|
+
|
|
3
|
+
This release stops one vendor's CLI upgrade from refusing every vendor's dispatch,
|
|
4
|
+
makes that failure visible in `omnilane doctor`, and stops the probe harness from
|
|
5
|
+
signing evidence it never read. Routing, scores, and the frozen AA registry are
|
|
6
|
+
unchanged.
|
|
7
|
+
|
|
8
|
+
## Why
|
|
9
|
+
|
|
10
|
+
On 2026-09-09 `agy` was upgraded from 1.1.27 to 1.1.28. That changed its sha256,
|
|
11
|
+
and `apply_transport_overlay` compared every evidence hash in one loop and raised
|
|
12
|
+
on the first mismatch, so `load_registry` failed and **every** dispatch was
|
|
13
|
+
refused:
|
|
14
|
+
|
|
15
|
+
```json
|
|
16
|
+
{"allowed":false,"code":"invalid-policy-input","message":"transport contract evidence changed"}
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Only the seven gemini mappings depended on that binary; the other forty-two were
|
|
20
|
+
collateral. `omnilane doctor` reported 19 passed, 0 failed throughout, because no
|
|
21
|
+
check ever loaded the overlay.
|
|
22
|
+
|
|
23
|
+
Re-probing showed the upgrade had changed nothing observable: all six agy
|
|
24
|
+
selectors still answered correctly. The outage was entirely the blast radius of a
|
|
25
|
+
config linter that had been given the authority of a security gate.
|
|
26
|
+
|
|
27
|
+
The same investigation found six `claude-fable-5-1` configurations that had been
|
|
28
|
+
unusable since 2026-09-07. Their probes had failed with a quota refusal, and
|
|
29
|
+
`build_overlay.py` simply omitted them — no record, no warning, nothing for an
|
|
30
|
+
operator to notice.
|
|
31
|
+
|
|
32
|
+
## Changes
|
|
33
|
+
|
|
34
|
+
### Evidence staleness is per vendor
|
|
35
|
+
|
|
36
|
+
- Overlay `evidence[]` entries take an optional `vendor` tag. A tagged entry whose
|
|
37
|
+
hash drifts, or whose file no longer exists, marks that vendor stale and skips
|
|
38
|
+
the `verified` upgrade for its mappings; `decide()` then returns
|
|
39
|
+
`unknown-target-runtime` for that vendor alone.
|
|
40
|
+
- Untagged entries keep the previous global fail-closed behaviour, so overlays
|
|
41
|
+
built before the tags exist are unaffected.
|
|
42
|
+
- Missing files are treated as staleness, not corruption: the codex and claude
|
|
43
|
+
evidence paths embed version directories, so their upgrades delete the file
|
|
44
|
+
rather than change its digest.
|
|
45
|
+
- Structural overlay checks — schema, snapshot, host, exact identity, selector
|
|
46
|
+
type, effort alignment — remain hard failures.
|
|
47
|
+
|
|
48
|
+
### Doctor loads the overlay
|
|
49
|
+
|
|
50
|
+
- A new `transport-overlay` check loads the configured overlay through
|
|
51
|
+
`aa_policy.load_registry` and names the offending file and vendor on failure.
|
|
52
|
+
It reports per-vendor verified counts on success and warns when a vendor has
|
|
53
|
+
degraded. `live-capable` is unchanged: it answers whether a CLI supports a live
|
|
54
|
+
session, which stays true while the gate refuses the vendor.
|
|
55
|
+
|
|
56
|
+
### Probes record a verdict
|
|
57
|
+
|
|
58
|
+
- `probe.py` now derives a `verdict`, `verdict_reason`, `observed_model`, and
|
|
59
|
+
`probed_at` from the raw evidence through a pure function, per vendor:
|
|
60
|
+
- Claude responses are judged on `modelUsage` — the billed model the CLI
|
|
61
|
+
reports — plus `is_error`, and are failed when stderr shows the CLI silently
|
|
62
|
+
substituted the default effort for an unknown `--effort`. Effort is half of a
|
|
63
|
+
scored identity, so that path would otherwise certify a mapping at the wrong
|
|
64
|
+
tier while exit status, `is_error`, `modelUsage`, and the expected token all
|
|
65
|
+
look correct.
|
|
66
|
+
- grok and agy reject invalid input outright, so exit status and a clean stderr
|
|
67
|
+
are sufficient.
|
|
68
|
+
- codex prints a banner to stderr on every run, so stderr is recorded for review
|
|
69
|
+
rather than treated as failure.
|
|
70
|
+
- `build_overlay.py` refuses to sign a non-passing probe and records it in a new
|
|
71
|
+
`unproven[]` block with its reason, so a quota refusal is visible instead of
|
|
72
|
+
silently dropping configurations. Evidence predating the verdict field is
|
|
73
|
+
still signed, with a warning naming each legacy entry.
|
|
74
|
+
- The six `claude-fable-5-1` and `claude-fable-5` configurations are listed in
|
|
75
|
+
the probe table so their failures reach `unproven[]`. Re-probing them on
|
|
76
|
+
2026-09-09 returned the same quota refusal as on 09-07, so they remain
|
|
77
|
+
unusable — but `omnilane doctor` now says so rather than leaving an operator
|
|
78
|
+
to discover it from a refused dispatch.
|
|
79
|
+
- `build_overlay.py` and `probe.py` moved from an untracked `.rollback` sweep
|
|
80
|
+
directory into `scripts/lib/`, and take `--root`; overlay rebuilds no longer
|
|
81
|
+
depend on a directory that a cleanup can delete.
|
|
82
|
+
|
|
83
|
+
## Verification boundary
|
|
84
|
+
|
|
85
|
+
A passing probe proves the CLI accepted the selector and, for Claude, that the
|
|
86
|
+
billed model matches the request. It still does not certify upstream provider
|
|
87
|
+
identity, and `upstream_identity_verified` remains `false`. grok, agy, and codex
|
|
88
|
+
expose no equivalent of `modelUsage` in the evidence captured so far.
|
|
89
|
+
|
|
90
|
+
The frozen AA registry and its approved SHA are unchanged. Coverage remains 78
|
|
91
|
+
scored targets, and the host overlay still verifies 49 mappings — codex 26,
|
|
92
|
+
claude 11, gemini 7, grok 5.
|
|
93
|
+
|
|
94
|
+
## Upgrade
|
|
95
|
+
|
|
96
|
+
Existing overlays keep working untouched. To gain per-vendor degradation, rebuild
|
|
97
|
+
the overlay with the new `build_overlay.py` so its evidence carries vendor tags.
|
|
98
|
+
Run `omnilane doctor` afterwards and confirm the `transport-overlay` check reports
|
|
99
|
+
the vendor counts you expect.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "omnilane",
|
|
3
|
-
"version": "0.42.
|
|
3
|
+
"version": "0.42.5",
|
|
4
4
|
"description": "One routing table, every harness — classify subtasks into lanes and delegate each lane through a compatible caller-owned native agent or vendor CLI.",
|
|
5
5
|
"bin": {
|
|
6
6
|
"omnilane": "bin/omnilane"
|
|
@@ -29,7 +29,7 @@
|
|
|
29
29
|
"docs/model-capabilities-2026-09.md",
|
|
30
30
|
"docs/native-executor.md",
|
|
31
31
|
"docs/completion-wakeup.md",
|
|
32
|
-
"docs/release-notes-0.42.
|
|
32
|
+
"docs/release-notes-0.42.5.md",
|
|
33
33
|
"hooks/",
|
|
34
34
|
"skills/",
|
|
35
35
|
".claude-plugin/",
|
package/plugin.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://antigravity.google/schemas/v1/plugin.json",
|
|
3
3
|
"name": "omnilane",
|
|
4
|
-
"version": "0.42.
|
|
4
|
+
"version": "0.42.5",
|
|
5
5
|
"description": "One routing table, every harness: classify subtasks into lanes and delegate through compatible caller-owned native agents or vendor CLIs with exact-AA downward policy and supervised jobs."
|
|
6
6
|
}
|
package/scripts/doctor.sh
CHANGED
|
@@ -33,6 +33,7 @@ REPO="${OMNILANE_DOCTOR_REPO:-$SCRIPT_ROOT}"
|
|
|
33
33
|
OMNILANE_HOME="${OMNILANE_HOME:-$HOME/.omnilane}"
|
|
34
34
|
PROBE_SCRIPT="${OMNILANE_PROVIDER_PROBE_SCRIPT:-$REPO/scripts/provider-probe.sh}"
|
|
35
35
|
GOAL_LOOP="${OMNILANE_DOCTOR_GOAL_LOOP:-$REPO/scripts/lib/goal-loop.sh}"
|
|
36
|
+
OVERLAY_HEALTH="${OMNILANE_DOCTOR_OVERLAY_HEALTH:-$REPO/scripts/lib/overlay_health.py}"
|
|
36
37
|
# shellcheck disable=SC1091
|
|
37
38
|
source "$SCRIPT_ROOT/scripts/lib/live-protocol.sh"
|
|
38
39
|
PASS_COUNT=0
|
|
@@ -409,6 +410,31 @@ else
|
|
|
409
410
|
report PASS live-unavailable "none"
|
|
410
411
|
fi
|
|
411
412
|
|
|
413
|
+
# live-capable above answers "does this CLI support a live session", which stays
|
|
414
|
+
# true while the AA gate refuses every dispatch. Nothing else loads the overlay,
|
|
415
|
+
# so one drifted evidence hash used to go unreported by an all-green doctor.
|
|
416
|
+
overlay_path="$(
|
|
417
|
+
set +u
|
|
418
|
+
[[ -f "$OMNILANE_HOME/local.sh" ]] && . "$OMNILANE_HOME/local.sh" 2>/dev/null
|
|
419
|
+
printf '%s' "${OMNILANE_AA_TRANSPORT_OVERLAY:-}"
|
|
420
|
+
)"
|
|
421
|
+
if [[ -z "$overlay_path" ]]; then
|
|
422
|
+
report PASS transport-overlay "no overlay configured; every runtime mapping stays unverified"
|
|
423
|
+
elif ! command -v python3 >/dev/null 2>&1; then
|
|
424
|
+
report WARN transport-overlay "python3 is absent; cannot load the AA transport overlay"
|
|
425
|
+
elif [[ ! -r "$OVERLAY_HEALTH" ]]; then
|
|
426
|
+
report WARN transport-overlay "$OVERLAY_HEALTH is missing"
|
|
427
|
+
else
|
|
428
|
+
overlay_line="$(OMNILANE_AA_TRANSPORT_OVERLAY="$overlay_path" \
|
|
429
|
+
python3 "$OVERLAY_HEALTH" "$REPO" 2>&1)"
|
|
430
|
+
overlay_level="${overlay_line%% *}"
|
|
431
|
+
overlay_message="${overlay_line#* }"
|
|
432
|
+
case "$overlay_level" in
|
|
433
|
+
PASS|WARN|FAIL) report "$overlay_level" transport-overlay "$overlay_message" ;;
|
|
434
|
+
*) report WARN transport-overlay "unreadable overlay health output: $overlay_line" ;;
|
|
435
|
+
esac
|
|
436
|
+
fi
|
|
437
|
+
|
|
412
438
|
if [[ -n "$PROBE_VENDOR" ]]; then
|
|
413
439
|
if [[ ! -x "$PROBE_SCRIPT" ]]; then
|
|
414
440
|
report FAIL provider-probe "probe runner is unavailable"
|
package/scripts/lib/aa_policy.py
CHANGED
|
@@ -28,6 +28,7 @@ MAX_BYTES = 1_048_576
|
|
|
28
28
|
APPROVED_REGISTRY_SHA256 = "0782c87de123c02738c3ff60e4bc3c1cc10d110113e872b8f8627212861cdaab"
|
|
29
29
|
|
|
30
30
|
IDENTITY_FIELDS = ("vendor", "model", "effort", "reasoning", "fallback")
|
|
31
|
+
TRANSPORT_EVIDENCE_VENDORS = frozenset(("codex", "claude", "grok", "gemini"))
|
|
31
32
|
IDENTIFIER = re.compile(r"[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}\Z")
|
|
32
33
|
|
|
33
34
|
|
|
@@ -147,14 +148,35 @@ def apply_transport_overlay(registry: dict[str, Any]) -> None:
|
|
|
147
148
|
_check(overlay.get("schema_version") == 1, "unsupported transport overlay")
|
|
148
149
|
_check(overlay.get("snapshot_id") == registry["snapshot"]["id"], "transport overlay snapshot mismatch")
|
|
149
150
|
_check(overlay.get("host") == socket.gethostname(), "transport overlay host mismatch")
|
|
151
|
+
stale_vendors: set[str] = set()
|
|
150
152
|
for evidence in overlay.get("evidence", []):
|
|
151
|
-
|
|
153
|
+
_check(isinstance(evidence, dict), "invalid transport evidence")
|
|
154
|
+
vendor = evidence.get("vendor")
|
|
155
|
+
_check(
|
|
156
|
+
"vendor" not in evidence
|
|
157
|
+
or type(vendor) is str and vendor in TRANSPORT_EVIDENCE_VENDORS,
|
|
158
|
+
"invalid transport evidence vendor",
|
|
159
|
+
)
|
|
160
|
+
evidence_path = evidence["path"]
|
|
161
|
+
evidence_sha256 = evidence["sha256"]
|
|
162
|
+
_check(type(evidence_path) is str, "invalid transport evidence path")
|
|
163
|
+
_check(type(evidence_sha256) is str, "invalid transport evidence digest")
|
|
164
|
+
try:
|
|
165
|
+
fd = os.open(evidence_path, os.O_RDONLY | os.O_NOFOLLOW | os.O_NONBLOCK)
|
|
166
|
+
except FileNotFoundError:
|
|
167
|
+
if vendor is None:
|
|
168
|
+
raise
|
|
169
|
+
stale_vendors.add(vendor)
|
|
170
|
+
continue
|
|
152
171
|
with os.fdopen(fd, "rb") as stream:
|
|
153
172
|
_check(stat.S_ISREG(os.fstat(stream.fileno()).st_mode), "invalid transport evidence file")
|
|
154
173
|
digest_file = hashlib.sha256()
|
|
155
174
|
for block in iter(lambda: stream.read(1024 * 1024), b""):
|
|
156
175
|
digest_file.update(block)
|
|
157
|
-
|
|
176
|
+
if digest_file.hexdigest() != evidence_sha256:
|
|
177
|
+
if vendor is None:
|
|
178
|
+
_check(False, "transport contract evidence changed")
|
|
179
|
+
stale_vendors.add(vendor)
|
|
158
180
|
_check(bool(overlay.get("evidence")), "transport overlay requires local evidence")
|
|
159
181
|
for mapping in overlay.get("mappings", []):
|
|
160
182
|
rows = [row for row in registry["scored_configs"] if row["id"] == mapping.get("config_id")]
|
|
@@ -175,6 +197,8 @@ def apply_transport_overlay(registry: dict[str, Any]) -> None:
|
|
|
175
197
|
_check(mapping["runtime_model"].endswith("-" + row["effort"]), "encoded effort does not match exact tuple")
|
|
176
198
|
else:
|
|
177
199
|
_check(mapping.get("runtime_model") == row["model"], "overlay model mismatch")
|
|
200
|
+
if row["vendor"] in stale_vendors:
|
|
201
|
+
continue
|
|
178
202
|
row["transport_mapping"].update(
|
|
179
203
|
status="verified", runtime_verified=True,
|
|
180
204
|
runtime_model=mapping["runtime_model"], runtime_effort=mapping["runtime_effort"],
|
|
@@ -183,6 +207,7 @@ def apply_transport_overlay(registry: dict[str, Any]) -> None:
|
|
|
183
207
|
verification="request-selector-contract", upstream_identity_verified=False,
|
|
184
208
|
overlay_sha256=digest, overlay_host=overlay["host"],
|
|
185
209
|
)
|
|
210
|
+
registry["_stale_transport_vendors"] = sorted(stale_vendors)
|
|
186
211
|
|
|
187
212
|
|
|
188
213
|
def load_registry(path: str | Path, expected_sha256: str | None = None) -> tuple[dict[str, Any], str]:
|
|
@@ -233,6 +258,8 @@ def _runtime_target(registry: dict[str, Any], vendor: str, model: str,
|
|
|
233
258
|
exact_id = [row for row in registry["scored_configs"] if row["id"] == target_config] if target_config else registry["scored_configs"]
|
|
234
259
|
if target_config and not exact_id:
|
|
235
260
|
return None, "unknown-target-config", {"target_config": target_config}
|
|
261
|
+
if vendor in registry.get("_stale_transport_vendors", []):
|
|
262
|
+
return None, "unknown-target-runtime", {"vendor": vendor, "model": model, "effort": effort}
|
|
236
263
|
vendor_rows = [row for row in exact_id if row["vendor"] == vendor]
|
|
237
264
|
candidates: list[dict[str, Any]] = []
|
|
238
265
|
unresolved: list[str] = []
|
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Build the merged omnilane AA transport overlay from verified probe evidence.
|
|
3
|
+
|
|
4
|
+
Only (config_id, selector) pairs listed in PROVEN are written. Every entry here
|
|
5
|
+
corresponds to a probe run under the selected evidence root whose raw
|
|
6
|
+
stdout/stderr is hashed into the manifest, so the overlay's evidence[] anchors
|
|
7
|
+
the whole set.
|
|
8
|
+
"""
|
|
9
|
+
import argparse
|
|
10
|
+
import hashlib
|
|
11
|
+
import json
|
|
12
|
+
import os
|
|
13
|
+
import socket
|
|
14
|
+
from datetime import datetime, timezone
|
|
15
|
+
from pathlib import Path
|
|
16
|
+
|
|
17
|
+
REPO = Path(os.environ.get("OMNILANE_REPO", "/Users/vincentw/dev/omnilane"))
|
|
18
|
+
HOME = Path.home()
|
|
19
|
+
SWEEP_ID = os.environ.get("OMNILANE_TRANSPORT_SWEEP_ID", "overlay-reprobe-20260909")
|
|
20
|
+
DEFAULT_ROOT = HOME / ".omnilane" / "transport-evidence" / SWEEP_ID
|
|
21
|
+
REGISTRY = json.loads((REPO / "config/aa-model-policy.json").read_text())
|
|
22
|
+
ROWS = {r["id"]: r for r in REGISTRY["scored_configs"]}
|
|
23
|
+
IDENTITY_FIELDS = ("vendor", "model", "effort", "reasoning", "fallback")
|
|
24
|
+
|
|
25
|
+
# config_id -> (selector_type, runtime_model, probe evidence basename)
|
|
26
|
+
PROVEN: dict[str, tuple[str, str, str]] = {}
|
|
27
|
+
|
|
28
|
+
for model, slug in [("gpt-6-astra", "gpt-6-astra"), ("gpt-5.6-sol", "gpt-5_6-sol"),
|
|
29
|
+
("gpt-5.6-luna", "gpt-5_6-luna"), ("gpt-5.6-terra", "gpt-5_6-terra")]:
|
|
30
|
+
base = model.replace(".", "-").replace("gpt-", "gpt-")
|
|
31
|
+
for effort in ["max", "xhigh", "high", "medium", "low"]:
|
|
32
|
+
cid = f"codex/{model.replace('.', '-')}" + ("" if effort == "max" else f"-{effort}")
|
|
33
|
+
ev = f"cx-avail-{model.replace('.', '_')}" if effort == "high" else f"cx-{model.replace('.', '_')}-{effort}"
|
|
34
|
+
PROVEN[cid] = ("model_and_effort", model, ev)
|
|
35
|
+
|
|
36
|
+
for effort in ["xhigh", "medium"]:
|
|
37
|
+
PROVEN[f"codex/gpt-5-4-mini" + ("" if effort == "xhigh" else f"-{effort}")] = (
|
|
38
|
+
"model_and_effort", "gpt-5.4-mini", f"cx-gpt-5_4-mini-{effort}")
|
|
39
|
+
|
|
40
|
+
PROVEN["grok/grok-4-6"] = ("cli_reasoning_effort", "grok-4.6", "PRIOR:grok-effort-2026-09-07")
|
|
41
|
+
for effort in ["xhigh", "medium", "low"]:
|
|
42
|
+
PROVEN[f"grok/grok-4-6-{effort}"] = ("cli_reasoning_effort", "grok-4.6", f"gk-grok-4_6-{effort}")
|
|
43
|
+
PROVEN["grok/grok-4-5"] = ("cli_reasoning_effort", "grok-4.5", "gk-grok-4_5-high")
|
|
44
|
+
|
|
45
|
+
for cid, rid, ev in [
|
|
46
|
+
("gemini/gemini-3-8-flash", "gemini-3.8-flash-high", "PRIOR:gemini-flash-high"),
|
|
47
|
+
("gemini/gemini-3-8-flash-medium", "gemini-3.8-flash-medium", "agy-gemini-3_8-flash-medium"),
|
|
48
|
+
("gemini/gemini-3-8-flash-low", "gemini-3.8-flash-low", "agy-gemini-3_8-flash-low"),
|
|
49
|
+
("gemini/gemini-3-7-flash", "gemini-3.7-flash-high", "agy-gemini-3_7-flash-high"),
|
|
50
|
+
("gemini/gemini-3-7-flash-medium", "gemini-3.7-flash-medium", "agy-gemini-3_7-flash-medium"),
|
|
51
|
+
("gemini/gemini-3-7-flash-low", "gemini-3.7-flash-low", "agy-gemini-3_7-flash-low"),
|
|
52
|
+
("gemini/gemini-3-6-flash", "gemini-3.6-flash-high", "agy-gemini-3_6-flash-high"),
|
|
53
|
+
]:
|
|
54
|
+
PROVEN[cid] = ("model_id_encoded_effort", rid, ev)
|
|
55
|
+
|
|
56
|
+
for effort in ["max", "xhigh", "high", "medium", "low"]:
|
|
57
|
+
cid = "claude/claude-opus-5" + ("" if effort == "max" else f"-{effort}")
|
|
58
|
+
PROVEN[cid] = ("model_and_effort", "claude-opus-5", f"cl-claude-opus-5-{effort}")
|
|
59
|
+
# gpt-6-astra rejects effort "none" upstream ("Unsupported value: 'none' is not
|
|
60
|
+
# supported with the 'gpt-6-astra' model"), so it has no non-reasoning selector.
|
|
61
|
+
for model in ["gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna", "gpt-5.4-mini"]:
|
|
62
|
+
PROVEN[f"codex/{model.replace('.', '-')}-non-reasoning"] = (
|
|
63
|
+
"model_and_effort", model, f"cx-{model.replace('.', '_')}-none")
|
|
64
|
+
|
|
65
|
+
# Default (no --effort) Haiku 4.5 spent 124 thinking tokens, so the selector
|
|
66
|
+
# lands on the reasoning row rather than its non-reasoning sibling.
|
|
67
|
+
PROVEN["claude/claude-4-5-haiku-reasoning"] = (
|
|
68
|
+
"model_and_effort", "claude-haiku-4-5", "cl-rmode-claude-haiku-4-5-noeffort")
|
|
69
|
+
|
|
70
|
+
for cid, model in [("claude/claude-sonnet-5", "claude-sonnet-5"),
|
|
71
|
+
("claude/claude-opus-4-8", "claude-opus-4-8"),
|
|
72
|
+
("claude/claude-opus-4-7", "claude-opus-4-7"),
|
|
73
|
+
("claude/claude-opus-4-6-adaptive", "claude-opus-4-6"),
|
|
74
|
+
("claude/claude-sonnet-4-6-adaptive", "claude-sonnet-4-6")]:
|
|
75
|
+
PROVEN[cid] = ("model_and_effort", model, f"cl-{model}-max")
|
|
76
|
+
|
|
77
|
+
# Fable is listed so its failures reach unproven[] rather than vanishing. Its
|
|
78
|
+
# probes were refused for quota on 2026-09-07 and again on 2026-09-09; the
|
|
79
|
+
# verdict decides whether these rows become mappings or stay visible failures.
|
|
80
|
+
for effort in ["max", "xhigh", "high", "medium", "low"]:
|
|
81
|
+
cid = "claude/claude-fable-5-1" + ("" if effort == "max" else f"-{effort}")
|
|
82
|
+
PROVEN[cid] = ("model_and_effort", "claude-fable-5-1", f"cl-claude-fable-5-1-{effort}")
|
|
83
|
+
PROVEN["claude/claude-fable-5"] = (
|
|
84
|
+
"model_and_effort", "claude-fable-5", "cl-claude-fable-5-max")
|
|
85
|
+
|
|
86
|
+
CORE_EVIDENCE = [
|
|
87
|
+
(HOME / ".grok/downloads/grok-1.0.13-macos-aarch64", "grok"),
|
|
88
|
+
(REPO / "scripts/runners/run-grok.sh", "grok"),
|
|
89
|
+
(HOME / ".codex/packages/standalone/releases/0.153.4-aarch64-apple-darwin/bin/codex", "codex"),
|
|
90
|
+
(REPO / "scripts/runners/run-codex.sh", "codex"),
|
|
91
|
+
(HOME / ".local/share/claude/versions/2.1.263", "claude"),
|
|
92
|
+
(REPO / "scripts/runners/run-claude.sh", "claude"),
|
|
93
|
+
(HOME / ".local/bin/agy", "gemini"),
|
|
94
|
+
(REPO / "scripts/runners/run-gemini.sh", "gemini"),
|
|
95
|
+
]
|
|
96
|
+
|
|
97
|
+
|
|
98
|
+
def sha256(path: Path) -> str:
|
|
99
|
+
digest = hashlib.sha256()
|
|
100
|
+
with open(path, "rb") as stream:
|
|
101
|
+
for block in iter(lambda: stream.read(1 << 20), b""):
|
|
102
|
+
digest.update(block)
|
|
103
|
+
return digest.hexdigest()
|
|
104
|
+
|
|
105
|
+
|
|
106
|
+
def main(argv: list[str] | None = None) -> None:
|
|
107
|
+
parser = argparse.ArgumentParser(description=__doc__)
|
|
108
|
+
parser.add_argument(
|
|
109
|
+
"--root",
|
|
110
|
+
type=Path,
|
|
111
|
+
default=DEFAULT_ROOT,
|
|
112
|
+
help=f"probe sweep root (default: {DEFAULT_ROOT})",
|
|
113
|
+
)
|
|
114
|
+
args = parser.parse_args(argv)
|
|
115
|
+
root = args.root.expanduser()
|
|
116
|
+
|
|
117
|
+
manifest = {"probe_runs": {}}
|
|
118
|
+
unproven = []
|
|
119
|
+
for cid, (_, _, ev) in sorted(PROVEN.items()):
|
|
120
|
+
if ev.startswith("PRIOR:"):
|
|
121
|
+
manifest["probe_runs"][cid] = {"source": ev, "note": "verified in the 2026-09-07 Codex run"}
|
|
122
|
+
continue
|
|
123
|
+
entry = {}
|
|
124
|
+
for suffix in ("json", "stdout", "stderr"):
|
|
125
|
+
path = root / "evidence" / f"{ev}.{suffix}"
|
|
126
|
+
if path.exists():
|
|
127
|
+
entry[suffix] = {"path": str(path), "sha256": sha256(path)}
|
|
128
|
+
if "json" not in entry:
|
|
129
|
+
raise SystemExit(f"missing probe evidence for {cid}: {ev}")
|
|
130
|
+
descriptor_path = Path(entry["json"]["path"])
|
|
131
|
+
descriptor = json.loads(descriptor_path.read_text())
|
|
132
|
+
if not isinstance(descriptor, dict):
|
|
133
|
+
raise SystemExit(f"invalid probe descriptor for {cid}: {ev}")
|
|
134
|
+
if "verdict" not in descriptor:
|
|
135
|
+
print(f"warning: legacy evidence (verdict=unknown): {cid}: {ev}")
|
|
136
|
+
elif descriptor["verdict"] != "pass":
|
|
137
|
+
unproven.append({
|
|
138
|
+
"config_id": cid,
|
|
139
|
+
"verdict_reason": descriptor.get("verdict_reason") or f"verdict: {descriptor['verdict']}",
|
|
140
|
+
"observed_model": descriptor.get("observed_model"),
|
|
141
|
+
"probed_at": descriptor.get("probed_at") or datetime.fromtimestamp(
|
|
142
|
+
descriptor_path.stat().st_mtime, timezone.utc).isoformat(),
|
|
143
|
+
})
|
|
144
|
+
# Visibility only: failed evidence must not enter the signed manifest.
|
|
145
|
+
continue
|
|
146
|
+
manifest["probe_runs"][cid] = entry
|
|
147
|
+
manifest_path = root / "probe-manifest.json"
|
|
148
|
+
manifest_path.write_text(json.dumps(manifest, indent=2, sort_keys=True) + "\n")
|
|
149
|
+
|
|
150
|
+
mappings = []
|
|
151
|
+
for cid, (selector, runtime_model, _) in sorted(PROVEN.items()):
|
|
152
|
+
if cid not in manifest["probe_runs"]:
|
|
153
|
+
continue
|
|
154
|
+
row = ROWS[cid]
|
|
155
|
+
mapping = {
|
|
156
|
+
"config_id": cid,
|
|
157
|
+
"identity": {key: row[key] for key in IDENTITY_FIELDS},
|
|
158
|
+
"runtime_model": runtime_model,
|
|
159
|
+
"runtime_effort": row["effort"],
|
|
160
|
+
"selector_type": selector,
|
|
161
|
+
"verification": "request-selector-contract",
|
|
162
|
+
}
|
|
163
|
+
if selector == "cli_reasoning_effort":
|
|
164
|
+
mapping["cli_flag"] = "--reasoning-effort"
|
|
165
|
+
mappings.append(mapping)
|
|
166
|
+
|
|
167
|
+
evidence = [
|
|
168
|
+
{"path": str(path), "sha256": sha256(path), "vendor": vendor}
|
|
169
|
+
for path, vendor in CORE_EVIDENCE
|
|
170
|
+
]
|
|
171
|
+
evidence.append({"path": str(manifest_path), "sha256": sha256(manifest_path)})
|
|
172
|
+
|
|
173
|
+
overlay = {
|
|
174
|
+
"schema_version": 1,
|
|
175
|
+
"snapshot_id": REGISTRY["snapshot"]["id"],
|
|
176
|
+
"host": socket.gethostname(),
|
|
177
|
+
"source": ("claude-code / MacStudio / operator-directed full sweep 2026-09-07; "
|
|
178
|
+
"gemini selectors re-probed 2026-09-09 after agy 1.1.27 -> 1.1.28"),
|
|
179
|
+
"evidence": evidence,
|
|
180
|
+
"mappings": mappings,
|
|
181
|
+
"unproven": unproven,
|
|
182
|
+
}
|
|
183
|
+
out = root / "transport-contracts.local.json"
|
|
184
|
+
out.write_text(json.dumps(overlay, indent=2, ensure_ascii=False) + "\n")
|
|
185
|
+
print(f"wrote {out} with {len(mappings)} mappings and {len(evidence)} evidence anchors")
|
|
186
|
+
|
|
187
|
+
|
|
188
|
+
if __name__ == "__main__":
|
|
189
|
+
main()
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Report whether the configured AA transport overlay still loads.
|
|
3
|
+
|
|
4
|
+
Prints one `LEVEL<TAB>message` line for `omnilane doctor`. Always exits 0; the
|
|
5
|
+
caller decides how to grade the level. Nothing here contacts a provider.
|
|
6
|
+
|
|
7
|
+
Exists because no other doctor check observes the AA gate: every vendor CLI can
|
|
8
|
+
be reachable and every lane resolvable while `load_registry` refuses the whole
|
|
9
|
+
registry over one drifted evidence hash.
|
|
10
|
+
"""
|
|
11
|
+
import hashlib
|
|
12
|
+
import os
|
|
13
|
+
import sys
|
|
14
|
+
from collections import Counter
|
|
15
|
+
from pathlib import Path
|
|
16
|
+
|
|
17
|
+
REPO = Path(sys.argv[1]) if len(sys.argv) > 1 else Path(__file__).resolve().parents[1]
|
|
18
|
+
sys.path.insert(0, str(REPO / "scripts" / "lib"))
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
def emit(level: str, message: str) -> None:
|
|
22
|
+
print(f"{level}\t{message}")
|
|
23
|
+
raise SystemExit(0)
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
def digest(path: Path) -> str:
|
|
27
|
+
value = hashlib.sha256()
|
|
28
|
+
with open(path, "rb") as stream:
|
|
29
|
+
for block in iter(lambda: stream.read(1024 * 1024), b""):
|
|
30
|
+
value.update(block)
|
|
31
|
+
return value.hexdigest()
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def offenders(overlay_path: Path) -> list[str]:
|
|
35
|
+
"""Name the evidence entries that no longer match, for an actionable report."""
|
|
36
|
+
import json
|
|
37
|
+
|
|
38
|
+
try:
|
|
39
|
+
overlay = json.loads(overlay_path.read_text())
|
|
40
|
+
except (OSError, ValueError):
|
|
41
|
+
return []
|
|
42
|
+
found = []
|
|
43
|
+
for entry in overlay.get("evidence", []):
|
|
44
|
+
path = Path(entry.get("path", ""))
|
|
45
|
+
tag = entry.get("vendor") or "untagged"
|
|
46
|
+
if not path.exists():
|
|
47
|
+
found.append(f"{tag}:missing {path}")
|
|
48
|
+
elif digest(path) != entry.get("sha256"):
|
|
49
|
+
found.append(f"{tag}:hash drift {path}")
|
|
50
|
+
return found
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
def main() -> None:
|
|
54
|
+
overlay_path = os.environ.get("OMNILANE_AA_TRANSPORT_OVERLAY", "")
|
|
55
|
+
if not overlay_path:
|
|
56
|
+
emit("PASS", "no overlay configured; every runtime mapping stays unverified")
|
|
57
|
+
if not Path(overlay_path).exists():
|
|
58
|
+
emit("FAIL", f"configured overlay is missing: {overlay_path}")
|
|
59
|
+
|
|
60
|
+
try:
|
|
61
|
+
import aa_policy
|
|
62
|
+
except ImportError as error:
|
|
63
|
+
emit("WARN", f"cannot import aa_policy: {error}")
|
|
64
|
+
|
|
65
|
+
try:
|
|
66
|
+
registry, _ = aa_policy.load_registry(str(REPO / "config" / "aa-model-policy.json"))
|
|
67
|
+
except Exception as error: # PolicyError, OSError, and anything else fails the gate
|
|
68
|
+
detail = "; ".join(offenders(Path(overlay_path))) or str(error)
|
|
69
|
+
emit("FAIL", f"overlay rejected, every dispatch is refused: {error} ({detail})")
|
|
70
|
+
|
|
71
|
+
verified = Counter()
|
|
72
|
+
for row in registry["scored_configs"]:
|
|
73
|
+
if row["transport_mapping"].get("runtime_verified") is True:
|
|
74
|
+
verified[row["vendor"]] += 1
|
|
75
|
+
summary = ", ".join(f"{v} {n}" for v, n in sorted(verified.items())) or "none"
|
|
76
|
+
|
|
77
|
+
import json
|
|
78
|
+
|
|
79
|
+
overlay = json.loads(Path(overlay_path).read_text())
|
|
80
|
+
unproven = overlay.get("unproven", [])
|
|
81
|
+
extra = f"; {len(unproven)} config(s) recorded unproven" if unproven else ""
|
|
82
|
+
|
|
83
|
+
stale = registry.get("_stale_transport_vendors", [])
|
|
84
|
+
if stale:
|
|
85
|
+
detail = "; ".join(o for o in offenders(Path(overlay_path))) or "unknown cause"
|
|
86
|
+
emit("WARN", f"stale vendor(s) {', '.join(stale)} degraded to unverified "
|
|
87
|
+
f"({detail}); still verified: {summary}{extra}")
|
|
88
|
+
emit("PASS", f"verified mappings: {summary}{extra}")
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
main()
|