@softspark/ai-toolkit 4.18.0 → 4.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,279 +0,0 @@
1
- ---
2
- title: "SOP: rtk Upstream Sync"
3
- category: procedures
4
- service: ai-toolkit
5
- tags: [sop, rtk, rtk-pack, upstream, cross-build, telemetry, checksum, port-validation, advisory]
6
- version: "1.0.0"
7
- created: "2026-07-26"
8
- last_updated: "2026-07-26"
9
- description: "Procedure for moving rtk-pack to a newer upstream rtk release: detect the tag, review the files we depend on, re-validate the Python port that every coverage number rests on, rebuild five targets with telemetry undefined, verify silence, publish to our own release namespace, and bump the pack. Written after the v0.43.0 to v0.44.0 bump, which changed every file this SOP names."
10
- ---
11
-
12
- # SOP: rtk Upstream Sync
13
-
14
- Moves `rtk-pack` from one pinned upstream tag to the next.
15
-
16
- Currently pinned: **v0.44.0**, shipped as
17
- `softspark-rtk-v0.44.0-1`. The pin lives in
18
- `app/plugins/rtk-pack/plugin.json` under `upstream.version`.
19
-
20
- Upstream ships stable tags roughly every two to four weeks behind a long
21
- release-candidate train (300+ RCs preceded v0.44.0). Do not track RCs.
22
-
23
- Background and the measured numbers: `kb/history/completed/rtk-pack-integration-20260726.md`.
24
-
25
- ## Why this SOP is not "just rebuild"
26
-
27
- The v0.43.0 to v0.44.0 bump was 200 commits and touched **every file listed in
28
- Phase 2 below**. It also inverted pipeline rewriting: v0.43.0 rewrote the first
29
- stage of a pipeline, v0.44.0 rewrites the last. A rebuild without the review
30
- step would have shipped that silently, and the coverage numbers quoted to users
31
- would have described a version we no longer ship.
32
-
33
- ## Phase 1: Detect
34
-
35
- ```bash
36
- gh api repos/rtk-ai/rtk/releases --paginate \
37
- --jq '.[] | select(.tag_name | test("^v[0-9]")) | "\(.tag_name)\t\(.published_at)"' | head -5
38
- ```
39
-
40
- Compare against `upstream.version` in `app/plugins/rtk-pack/plugin.json`.
41
-
42
- Cadence: on demand, plus a check folded into release preparation.
43
-
44
- ## Phase 2: Review before building
45
-
46
- Fetch the diff for the areas the pack depends on:
47
-
48
- ```bash
49
- gh api repos/rtk-ai/rtk/compare/<pinned>...<new> \
50
- --jq '{ahead: .ahead_by, files: [.files[] | {f: .filename, add: .additions, del: .deletions}]}'
51
- ```
52
-
53
- Read the changelog, then diff these specifically:
54
-
55
- | File | Why it matters |
56
- |---|---|
57
- | `src/discover/registry.rs` | rewrite eligibility, pipeline handling, the TOML call sites |
58
- | `src/discover/rules.rs` | the rule table and `IGNORED_PREFIXES` |
59
- | `src/discover/lexer.rs` | tokenisation and `contains_unattestable_construct` |
60
- | `src/core/toml_filter.rs` | the filter DSL, which affects users who write their own filters |
61
- | `src/hooks/trust.rs` | the trust gate and its paths |
62
- | `src/hooks/hook_cmd.rs` | the Claude hook contract and permission handling |
63
- | `src/core/telemetry.rs`, `src/core/telemetry_cmd.rs` | the compile-time endpoint gate |
64
- | `Cargo.toml`, `Cargo.lock` | native deps, the MSRV, and new advisories |
65
-
66
- **A change to the DSL, to trust handling, or to the permission flow is a
67
- stop-and-think, not a rebuild.** In particular:
68
-
69
- - The pack ships no filters of its own, so a DSL change cannot break us. It can
70
- still break a **user's** `filters.toml`, which upstream then skips silently
71
- (`toml_filter.rs:220-221`). Worth a release-note line, not a code change.
72
- - If `hook_cmd.rs` changes when `permissionDecision` is emitted, the trust
73
- boundary documented in the pack README changes with it.
74
- - If `IGNORED_PREFIXES` or the `pipeline_final_safe` rule set moves, every
75
- coverage number is stale.
76
-
77
- ## Phase 3: Re-validate the port
78
-
79
- Every coverage and saving figure the pack quotes comes from `rtk_port.py`, a
80
- Python model of rtk's rewrite pipeline. **Anything short of full agreement
81
- invalidates those numbers until the port is fixed.** Tooling lives in
82
- `~/rtk-measurement-archive/`.
83
-
84
- ```bash
85
- git clone --depth 1 --branch <new-tag> https://github.com/rtk-ai/rtk.git /tmp/rtk-new
86
- cd ~/rtk-measurement-archive
87
-
88
- # Rewrite assertions from the tag's own test block.
89
- python3 extract_cases.py /tmp/rtk-new/src/discover/registry.rs cases.json
90
- python3 validate_port.py cases.json
91
-
92
- # The entry gate both real hook paths apply before rewrite_command.
93
- # Its assertions live in lexer.rs; re-extract if that block moved.
94
- python3 -c "import entry_gate; print('gate import ok')"
95
- ```
96
-
97
- Baseline at v0.44.0: **203/203** rewrite assertions, **35/35** gate assertions.
98
-
99
- If the port diverges, fix the port first, then re-measure:
100
-
101
- ```bash
102
- python3 measure_gated.py gated 0:1224
103
- ```
104
-
105
- Measure over the whole transcript pool, never the default 134-file window: at
106
- that size the projection swings 8.8x on an unchanged mechanism.
107
-
108
- ### 3.1 Re-measure, do not just re-validate
109
-
110
- Port agreement proves rtk still rewrites the same commands. It says nothing
111
- about how much each rewrite saves, and that is where the value actually sits.
112
-
113
- **`rtk grep` carries 4.04 MB of the 5.66 MB measurable saving: 71% of the total
114
- rests on one family.** If upstream changes that one filter, the headline number
115
- moves even with the port at full agreement. Measured effectiveness has already
116
- diverged from expectation in both directions once: `rtk grep` measured 22.3%
117
- against 9.0% modelled, `rtk rg` 7.0% against 30% modelled.
118
-
119
- So on every bump, after the port passes, replay against the newly built binary:
120
-
121
- ```bash
122
- python3 replay_rtk.py --pool 1300 --rtk <path-to-the-new-binary>
123
- ```
124
-
125
- Compare per family against the recorded baseline:
126
-
127
- | Family | Measured at v0.44.0 | Share of total saving |
128
- |---|---:|---:|
129
- | `rtk grep` | 22.3% | 71% |
130
- | `rtk git` | 33.1% | 21% |
131
- | `rtk find` | 35.6% | 8% |
132
-
133
- **Act on the result, do not just record it.** If the total lands below the
134
- published kill number of **0.05% of input tokens**, the pack has stopped earning
135
- its supply-chain surface and retiring it is the correct outcome, exactly as the
136
- in-house filter was retired. The margin at v0.44.0 is 0.0615%, which is 1.23x
137
- the kill number, so a single-family regression is enough to cross it.
138
-
139
- ## Phase 4: Rebuild
140
-
141
- ```bash
142
- gh workflow run rtk-build.yml --ref main \
143
- -f upstream_tag=<new-tag> -f build_revision=1 -f rust_version=<pinned> -f publish=false
144
- ```
145
-
146
- Five targets: `x86_64-apple-darwin`, `aarch64-apple-darwin`,
147
- `x86_64-unknown-linux-musl`, `aarch64-unknown-linux-gnu`,
148
- `x86_64-pc-windows-msvc`. A target that will not build is dropped, not faked.
149
-
150
- Pin `rust_version` explicitly. Upstream uses unpinned `stable` with
151
- `warnings = "deny"`, so a new rustc lint can turn the build red with no change
152
- on either side.
153
-
154
- **Advisories.** The `audit` job blocks. Re-derive the disposition rather than
155
- carrying the previous one forward:
156
-
157
- ```bash
158
- # Cross-reference the new lockfile against OSV without waiting for CI.
159
- python3 - <<'PY'
160
- import json, re, urllib.request, pathlib
161
- lock = pathlib.Path("/tmp/rtk-new/Cargo.lock").read_text()
162
- pkgs = [(re.search(r'^name = "([^"]+)"', b, re.M).group(1),
163
- re.search(r'^version = "([^"]+)"', b, re.M).group(1))
164
- for b in lock.split("[[package]]")[1:]
165
- if re.search(r'^name = ', b, re.M) and re.search(r'^version = ', b, re.M)]
166
- q = [{"package": {"name": n, "ecosystem": "crates.io"}, "version": v} for n, v in pkgs]
167
- req = urllib.request.Request("https://api.osv.dev/v1/querybatch",
168
- data=json.dumps({"queries": q}).encode(), headers={"Content-Type": "application/json"})
169
- res = json.load(urllib.request.urlopen(req, timeout=60))
170
- for (n, v), r in zip(pkgs, res["results"]):
171
- if r.get("vulns"):
172
- print(n, v, [x["id"] for x in r["vulns"]])
173
- PY
174
- ```
175
-
176
- For each advisory decide, and record the reason in the workflow:
177
-
178
- - **In-range fix** (`cargo update -p <crate>` works): add the crate to
179
- `RTK_CARGO_UPDATES` in `.github/workflows/rtk-build.yml`. Both the audit job
180
- and every build job apply it, so we audit what we ship.
181
- - **Needs a `Cargo.toml` change**: that is a source modification and breaks the
182
- NOTICE claim. Ignore with a written reason, or escalate.
183
- - **Carried-forward ignores**: re-check every `--ignore` still applies. The
184
- quick-xml pair exists only because upstream pins `"0.37"`; **delete both the
185
- moment upstream moves to 0.41 or later** rather than carrying them.
186
-
187
- ## Phase 5: Verify
188
-
189
- CI asserts this per target and the run fails on any `fail` verdict:
190
-
191
- 1. `RTK_TELEMETRY_URL` and `RTK_TELEMETRY_TOKEN` unset at build time
192
- 2. the artifact starts and reports the expected version
193
- 3. no telemetry state written into a sandboxed home
194
- 4. on Linux, identical behaviour with no network route
195
- 5. every archive holds exactly one flat entry
196
-
197
- Then check by hand:
198
-
199
- ```bash
200
- gh run download <run-id> --dir /tmp/rtk-verify
201
- cd /tmp/rtk-verify && shasum -a 256 -c checksums.txt
202
- ```
203
-
204
- Three things CI cannot tell you:
205
-
206
- - **`strings` markers.** Compare `tls_markers_present` per target against the
207
- previous build. Their absence is an LLVM outcome under LTO, not a guarantee,
208
- so a sudden appearance means the telemetry stack survived and is worth
209
- understanding before shipping.
210
- - **Reproducibility.** Four of five targets are bit-reproducible; compare
211
- extracted binaries, never the tarballs, because gzip records a timestamp.
212
- `x86_64-pc-windows-msvc` differs by 24 bytes per link (MSVC timestamp plus a
213
- CodeView GUID), so a changed Windows digest proves nothing on its own.
214
- - **Upstream has no test asserting network silence.** That property can regress
215
- on any bump without turning their CI red. We own it.
216
-
217
- ## Phase 6: Publish
218
-
219
- ```bash
220
- gh workflow run rtk-build.yml --ref main \
221
- -f upstream_tag=<new-tag> -f build_revision=1 -f rust_version=<pinned> -f publish=true
222
- ```
223
-
224
- Creates `softspark-rtk-<upstream>-<revision>`. Bump the revision, not the
225
- upstream part, when rebuilding the same upstream tag.
226
-
227
- ## Phase 7: Bump the pack
228
-
229
- In `app/plugins/rtk-pack/plugin.json`:
230
-
231
- - `upstream.version` to the new tag
232
- - `binary.release_tag` to the new release
233
- - every `assets.*.sha256` from the published `checksums.txt`
234
- - `version` (the pack's own) — bump it whether or not the upstream tag moved,
235
- because `plugin update` skips a pack whose recorded version still matches, so
236
- an unbumped pack never reaches installed users. The two fields are separate so
237
- a pack-only fix does not pretend to be an upstream bump
238
-
239
- Then:
240
-
241
- ```bash
242
- npm test # tests/test_rtk_pack.bats asserts digest shape and layout
243
- python3 scripts/validate.py --strict
244
- python3 scripts/audit_skills.py --ci
245
- shellcheck --severity=warning app/hooks/*.sh app/plugins/*/hooks/*.sh
246
- ```
247
-
248
- Verify a real install end to end, against the published release rather than a
249
- mirror:
250
-
251
- ```bash
252
- H=$(mktemp -d)
253
- AI_TOOLKIT_DATA_DIR="$H" python3 app/plugins/rtk-pack/scripts/init.py
254
- AI_TOOLKIT_DATA_DIR="$H" python3 app/plugins/rtk-pack/scripts/status.py
255
- ```
256
-
257
- ## Phase 8: Ship
258
-
259
- The pack version bump reaches installed users through `ai-toolkit update`,
260
- which runs `plugin update --editor all --all`. A pack whose recorded version
261
- matches its manifest is skipped silently, so the bump in Phase 7 is what makes
262
- the update fire at all. Forgetting it means nobody gets the new binary.
263
-
264
- ## Phase 9: Record the licence position
265
-
266
- If upstream relicenses, adds a `NOTICE`, or changes `DISCLAIMER.md`, update the
267
- NOTICE generated in `.github/workflows/rtk-build.yml`. It currently records two
268
- build-time differences: the undefined telemetry endpoint, and in-range lockfile
269
- security updates. If Phase 4 changes that set, the NOTICE text changes with it.
270
-
271
- `DISCLAIMER.md` is deliberately not redistributed because it states telemetry is
272
- collected by default, which our builds contradict. Re-check that this is still
273
- the reason before changing the decision.
274
-
275
- ## Related
276
-
277
- - `kb/history/completed/rtk-pack-integration-20260726.md` — decisions, measurements, pre-mortem
278
- - `kb/history/completed/output-filter-retirement-20260726.md` — why premise validation comes first
279
- - `app/plugins/rtk-pack/README.md` — the user-facing trust boundary
@@ -1,335 +0,0 @@
1
- #!/usr/bin/env python3
2
- """Verify a cross-built rtk binary is silent, and fingerprint it for drift.
3
-
4
- Phase 1 of kb/history/completed/rtk-pack-integration-20260726.md. "No telemetry symbols in
5
- the binary" is NOT a usable acceptance test: the guard is a runtime branch on a
6
- const (`telemetry.rs:23-26`), not a `#[cfg]`, and `Cargo.toml:51` sets
7
- `strip = true`, so a symbol check passes for the wrong reason. This script
8
- asserts what is actually checkable instead:
9
-
10
- build-gate RTK_TELEMETRY_URL / RTK_TELEMETRY_TOKEN unset at build time
11
- runs the binary starts and reports its version
12
- no-state a sandboxed run creates no telemetry state on disk
13
- offline a run with no network route behaves identically (Linux only)
14
- fingerprint size, digest and TLS-marker scan, recorded for drift detection
15
-
16
- Stdlib only. Emits JSON to stdout; exits non-zero if any assertion fails.
17
- """
18
- from __future__ import annotations
19
-
20
- import argparse
21
- import hashlib
22
- import json
23
- import os
24
- import platform
25
- import re
26
- import shutil
27
- import subprocess
28
- import sys
29
- import tempfile
30
- from pathlib import Path
31
-
32
- # Compile-time telemetry inputs. Upstream injects these in its own release
33
- # workflow (release.yml:85-86); our builds must leave both undefined.
34
- TELEMETRY_BUILD_VARS = ("RTK_TELEMETRY_URL", "RTK_TELEMETRY_TOKEN")
35
-
36
- # Telemetry state rtk would create if it ever ran the ping path. Names come from
37
- # src/core/telemetry.rs (salt file) and the tracking DB under the data dir.
38
- TELEMETRY_STATE_GLOBS = ("**/rtk/*salt*", "**/rtk/telemetry*", "**/rtk/*consent*")
39
-
40
- # webpki-roots embeds CA subjects as readable DER. Their presence means rustls
41
- # survived LTO, which is advisory rather than a failure: it says the HTTP stack
42
- # was linked in, not that anything is sent. Recorded so drift is visible.
43
- TLS_MARKERS = (b"ISRG Root X1", b"DigiCert", b"Baltimore CyberTrust", b"GlobalSign")
44
-
45
- RUNNABLE_HERE = {
46
- ("Darwin", "arm64"): {"aarch64-apple-darwin"},
47
- ("Darwin", "x86_64"): {"x86_64-apple-darwin"},
48
- ("Linux", "x86_64"): {"x86_64-unknown-linux-gnu", "x86_64-unknown-linux-musl"},
49
- ("Linux", "aarch64"): {"aarch64-unknown-linux-gnu", "aarch64-unknown-linux-musl"},
50
- ("Windows", "AMD64"): {"x86_64-pc-windows-msvc"},
51
- }
52
-
53
- # qemu-user turns the one genuinely cross-built target into a verifiable one.
54
- # Without it every assertion below reports "skipped" and the artifact ships
55
- # having been started exactly zero times.
56
- QEMU_FOR = {
57
- "aarch64-unknown-linux-gnu": ("qemu-aarch64-static", "qemu-aarch64"),
58
- "aarch64-unknown-linux-musl": ("qemu-aarch64-static", "qemu-aarch64"),
59
- }
60
-
61
- # Rosetta 2 does the same job on Apple silicon, which is what lets us verify an
62
- # x86_64 artifact built on an arm64 runner. Probed, never assumed: the image can
63
- # ship without it.
64
- ROSETTA_FOR = {"x86_64-apple-darwin": ("Darwin", "arm64")}
65
-
66
-
67
- class Failure(Exception):
68
- pass
69
-
70
-
71
- def can_run_natively(target: str) -> bool:
72
- return target in RUNNABLE_HERE.get((platform.system(), platform.machine()), set())
73
-
74
-
75
- def emulator_for(target: str) -> list | None:
76
- """qemu invocation for this target, with the sysroot passed as a flag.
77
-
78
- The prefix goes through `-L` rather than QEMU_LD_PREFIX because the offline
79
- check runs under sudo, and sudo's env_reset strips the variable. A gnu
80
- target is dynamically linked, so losing it means qemu cannot find
81
- ld-linux-aarch64.so.1 and the process dies with 255 before main.
82
- """
83
- if can_run_natively(target) or platform.system() != "Linux":
84
- return None
85
- for candidate in QEMU_FOR.get(target, ()):
86
- found = shutil.which(candidate)
87
- if not found:
88
- continue
89
- prefix = os.environ.get("QEMU_LD_PREFIX", "")
90
- return [found, "-L", prefix] if prefix else [found]
91
- return None
92
-
93
-
94
- def rosetta_for(target: str) -> list | None:
95
- if ROSETTA_FOR.get(target) != (platform.system(), platform.machine()):
96
- return None
97
- if shutil.which("arch") is None:
98
- return None
99
- try:
100
- probe = subprocess.run(
101
- ["arch", "-x86_64", "/usr/bin/true"], capture_output=True, timeout=30
102
- )
103
- except (OSError, subprocess.SubprocessError):
104
- return None
105
- return ["arch", "-x86_64"] if probe.returncode == 0 else None
106
-
107
-
108
- def launcher(target: str):
109
- """(wrapper, how) for running this target here, or (None, reason)."""
110
- if can_run_natively(target):
111
- return [], "native"
112
- emu = emulator_for(target)
113
- if emu:
114
- return emu, f"emulated via {Path(emu[0]).name}"
115
- rosetta = rosetta_for(target)
116
- if rosetta:
117
- return rosetta, "translated via Rosetta 2"
118
- return None, f"{target} is not runnable on {platform.system()}/{platform.machine()} and no emulator is installed"
119
-
120
-
121
- def sandbox_env(root: Path) -> dict:
122
- """An environment whose config/data/cache all resolve inside root."""
123
- env = dict(os.environ)
124
- for var in TELEMETRY_BUILD_VARS:
125
- env.pop(var, None)
126
- env["HOME"] = str(root)
127
- env["USERPROFILE"] = str(root)
128
- env["XDG_CONFIG_HOME"] = str(root / "config")
129
- env["XDG_DATA_HOME"] = str(root / "data")
130
- env["XDG_CACHE_HOME"] = str(root / "cache")
131
- env["APPDATA"] = str(root / "AppData" / "Roaming")
132
- env["LOCALAPPDATA"] = str(root / "AppData" / "Local")
133
- return env
134
-
135
-
136
- def run(binary: Path, args: list, env: dict, wrapper: list | None = None):
137
- cmd = (wrapper or []) + [str(binary)] + args
138
- proc = subprocess.run(cmd, env=env, capture_output=True, timeout=120)
139
- return proc.returncode, proc.stdout.decode("utf-8", "replace"), proc.stderr.decode("utf-8", "replace")
140
-
141
-
142
- def check_build_gate() -> dict:
143
- """The variables must be undefined in this environment too.
144
-
145
- The workflow asserts this before `cargo build`; re-asserting here catches a
146
- verification job that was handed a binary from a contaminated build.
147
- """
148
- leaked = [v for v in TELEMETRY_BUILD_VARS if os.environ.get(v)]
149
- if leaked:
150
- raise Failure(f"telemetry build variables are set: {', '.join(leaked)}")
151
- return {"pass": True, "checked": list(TELEMETRY_BUILD_VARS)}
152
-
153
-
154
- def check_runs(binary: Path, target: str, upstream_tag: str) -> dict:
155
- """Starts, identifies itself as rtk, and reports the version we asked for.
156
-
157
- Without the identity assertion this check passes for any binary that exits
158
- 0 on an unknown flag, `/bin/echo` included.
159
- """
160
- wrapper, how = launcher(target)
161
- if wrapper is None:
162
- return {"pass": None, "skipped": how}
163
- with tempfile.TemporaryDirectory() as tmp:
164
- code, out, err = run(binary, ["--version"], sandbox_env(Path(tmp)), wrapper=wrapper)
165
- if code != 0:
166
- raise Failure(f"`rtk --version` exited {code}: {err.strip()[:200]}")
167
- version = (out.strip() or err.strip())
168
- if "rtk" not in version.lower():
169
- raise Failure(f"`--version` output does not identify rtk: {version[:120]!r}")
170
- expected = upstream_tag.lstrip("v")
171
- if expected and expected not in version:
172
- raise Failure(f"version {version[:120]!r} does not match upstream tag {upstream_tag}")
173
- return {"pass": True, "version": version, "how": how}
174
-
175
-
176
- def check_no_state(binary: Path, target: str) -> dict:
177
- """A real command must not leave telemetry state behind."""
178
- wrapper, how = launcher(target)
179
- if wrapper is None:
180
- return {"pass": None, "skipped": how}
181
- with tempfile.TemporaryDirectory() as tmp:
182
- root = Path(tmp)
183
- env = sandbox_env(root)
184
- for args in (["--version"], ["--help"], ["git", "status"]):
185
- run(binary, args, env, wrapper=wrapper)
186
- found = sorted(
187
- str(p.relative_to(root))
188
- for pattern in TELEMETRY_STATE_GLOBS
189
- for p in root.glob(pattern)
190
- if p.is_file()
191
- )
192
- if found:
193
- raise Failure(f"telemetry state created: {found}")
194
- return {"pass": True, "sandbox_clean": True}
195
-
196
-
197
- def check_offline(binary: Path, target: str) -> dict:
198
- """Run with no network route and assert identical behaviour.
199
-
200
- `unshare -rn` gives a network namespace with only a down loopback, so any
201
- outbound connection fails immediately. There is no equivalent that works
202
- unprivileged on macOS or Windows runners, so this assertion is Linux-only
203
- and reports itself skipped elsewhere rather than pretending to pass.
204
- """
205
- wrapper, how = launcher(target)
206
- if wrapper is None:
207
- return {"pass": None, "skipped": how}
208
- if platform.system() != "Linux" or shutil.which("unshare") is None:
209
- return {"pass": None, "skipped": "unshare(1) network namespaces are Linux-only"}
210
-
211
- # Ubuntu 24.04 sets kernel.apparmor_restrict_unprivileged_userns=1, so the
212
- # unprivileged form is refused on GitHub runners and this assertion silently
213
- # became a skip. Fall back to passwordless sudo, which runners have.
214
- #
215
- # Each entry is (baseline, isolated): identical except for the network
216
- # namespace. Comparing against a plain run instead would confound the
217
- # network with sudo's env_reset, and the difference would be read as
218
- # evidence about the binary when it is evidence about the harness.
219
- isolators = [
220
- (["unshare", "-r"], ["unshare", "-rn"]),
221
- (["sudo", "-n", "unshare", "-r"], ["sudo", "-n", "unshare", "-rn"]),
222
- ]
223
- baseline = isolated = None
224
- for base, iso in isolators:
225
- if subprocess.run(iso + ["true"], capture_output=True, timeout=30).returncode == 0:
226
- baseline, isolated = base, iso
227
- break
228
- if isolated is None:
229
- return {"pass": None, "skipped": "no usable network namespace: unprivileged userns refused and sudo unavailable"}
230
-
231
- with tempfile.TemporaryDirectory() as tmp:
232
- env = sandbox_env(Path(tmp))
233
- online_code, _, online_err = run(binary, ["--version"], env, wrapper=baseline + wrapper)
234
- offline_code, _, offline_err = run(binary, ["--version"], env, wrapper=isolated + wrapper)
235
- if online_code != 0:
236
- # The baseline could not start inside the namespace, so the comparison
237
- # says nothing about the binary. That is a harness limitation, not a
238
- # defect in the artifact, and reporting it as a failure would blame the
239
- # thing being measured for the measurement not working. Skip instead,
240
- # and carry the reason so it is visible rather than silent.
241
- return {
242
- "pass": None,
243
- "skipped": (
244
- f"baseline run under {' '.join(baseline)} exited {online_code}, "
245
- f"so the network comparison proves nothing"
246
- ),
247
- "detail": online_err.strip()[:200],
248
- }
249
- if offline_code != online_code:
250
- raise Failure(
251
- f"behaviour differs without a network route: online exit {online_code}, offline exit {offline_code}"
252
- )
253
- return {
254
- "pass": True,
255
- "exit_code": offline_code,
256
- "stderr_empty": not offline_err.strip(),
257
- "isolator": " ".join(isolated),
258
- }
259
-
260
-
261
- def fingerprint(binary: Path) -> dict:
262
- data = binary.read_bytes()
263
- markers = sorted(m.decode() for m in TLS_MARKERS if m in data)
264
- printable = re.findall(rb"[\x20-\x7e]{8,}", data)
265
- return {
266
- "size_bytes": len(data),
267
- "sha256": hashlib.sha256(data).hexdigest(),
268
- "tls_markers_present": markers,
269
- "printable_string_count": len(printable),
270
- "strings_digest": hashlib.sha256(b"\n".join(sorted(set(printable)))).hexdigest(),
271
- }
272
-
273
-
274
- def main() -> int:
275
- ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
276
- ap.add_argument("--binary", required=True, type=Path)
277
- ap.add_argument("--target", required=True)
278
- ap.add_argument("--upstream-tag", default="", help="recorded in the manifest for traceability")
279
- ap.add_argument("--out", type=Path, help="write the manifest here as well as stdout")
280
- args = ap.parse_args()
281
-
282
- if not args.binary.is_file():
283
- print(json.dumps({"error": f"no such binary: {args.binary}"}), file=sys.stderr)
284
- return 2
285
-
286
- checks = {}
287
- failures = []
288
- for name, fn in (
289
- ("build_gate", lambda: check_build_gate()),
290
- ("runs", lambda: check_runs(args.binary, args.target, args.upstream_tag)),
291
- ("no_state", lambda: check_no_state(args.binary, args.target)),
292
- ("offline", lambda: check_offline(args.binary, args.target)),
293
- ):
294
- try:
295
- checks[name] = fn()
296
- except Failure as exc:
297
- checks[name] = {"pass": False, "reason": str(exc)}
298
- failures.append(f"{name}: {exc}")
299
- except (OSError, subprocess.SubprocessError) as exc:
300
- checks[name] = {"pass": False, "reason": f"{type(exc).__name__}: {exc}"}
301
- failures.append(f"{name}: {exc}")
302
-
303
- # A target nobody could start here must not report "pass". Every runtime
304
- # assertion would have been skipped, and a green tick on an artifact that
305
- # was never executed is the same silent-pass trap this plan keeps finding.
306
- ran_anything = checks.get("runs", {}).get("pass") is True
307
- if failures:
308
- verdict = "fail"
309
- elif ran_anything:
310
- verdict = "pass"
311
- else:
312
- verdict = "inconclusive"
313
-
314
- manifest = {
315
- "target": args.target,
316
- "upstream_tag": args.upstream_tag,
317
- "host": f"{platform.system()}/{platform.machine()}",
318
- # Recorded because the emulated path only works when the workflow
319
- # supplies the cross sysroot; a manifest that does not say so cannot be
320
- # audited later.
321
- "qemu_ld_prefix": os.environ.get("QEMU_LD_PREFIX", ""),
322
- "checks": checks,
323
- "fingerprint": fingerprint(args.binary),
324
- "verdict": verdict,
325
- "failures": failures,
326
- }
327
- text = json.dumps(manifest, indent=2, sort_keys=True)
328
- print(text)
329
- if args.out:
330
- args.out.write_text(text + "\n", encoding="utf-8")
331
- return 1 if failures else 0
332
-
333
-
334
- if __name__ == "__main__":
335
- sys.exit(main())