@softspark/ai-toolkit 4.17.0 → 4.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/llms-full.txt CHANGED
@@ -21,6 +21,7 @@
21
21
  - [Plan: Offline-First SLM Profile — Lightweight Mode for Local Models](kb/history/completed/offline-slm-profile-plan-20260411.md)
22
22
  - [Retirement: Native Tool-Output Filter](kb/history/completed/output-filter-retirement-20260726.md)
23
23
  - [Plan: Output & Token Discipline](kb/history/completed/output-token-discipline-plan-20260504.md)
24
+ - [rtk Pack Integration](kb/history/completed/rtk-pack-integration-20260726.md)
24
25
  - [How-To Guides](kb/howto/README.md)
25
26
  - [Plan: Cloud Security Pack — Multi-Cloud Audit](kb/planning/cloud-security-pack-plan.md)
26
27
  - [Plan: Drop Cascade hooks after 2026-07-01 sunset](kb/planning/drop-cascade-hooks-after-sunset.md)
@@ -29,6 +30,7 @@
29
30
  - [SOP: AI Toolkit Maintenance](kb/procedures/maintenance-sop.md)
30
31
  - [SOP: Release Preparation](kb/procedures/release-preparation-sop.md)
31
32
  - [SOP: Release Verification](kb/procedures/release-verification-sop.md)
33
+ - [SOP: rtk Upstream Sync](kb/procedures/rtk-upstream-sync-sop.md)
32
34
  - [Agents Catalog](kb/reference/agents-catalog.md)
33
35
  - [Anti-Pattern Registry Format](kb/reference/anti-pattern-registry-format.md)
34
36
  - [AI Toolkit Architecture](kb/reference/architecture-overview.md)
@@ -3447,6 +3449,721 @@ Updated: `README.md`, `kb/reference/architecture-overview.md`, `kb/reference/ski
3447
3449
 
3448
3450
  ---
3449
3451
 
3452
+ ## kb/history/completed/rtk-pack-integration-20260726.md
3453
+
3454
+ ---
3455
+ title: "Plan: rtk Pack Integration"
3456
+ category: planning
3457
+ service: ai-toolkit
3458
+ tags:
3459
+ - rtk
3460
+ - plugin-pack
3461
+ - token-reduction
3462
+ - vendored-binaries
3463
+ - cross-build
3464
+ - upstream-sync
3465
+ doc_type: postmortem
3466
+ status: completed
3467
+ created: "2026-07-26"
3468
+ last_updated: "2026-07-26"
3469
+ completion: "100% — phases 0, 1, 2, 4, 5 delivered; phase 3 cut on the measurement"
3470
+ shipped_in: "v4.18.0"
3471
+ pinned_upstream: "v0.44.0"
3472
+ description: "Integrate rtk as an opt-in ai-toolkit plugin pack: binaries cross-built from source in our CI with telemetry disabled, hosted on our own GitHub Release, fetched and checksum-verified at pack install, auto-update on ai-toolkit update, and an SOP for tracking upstream releases. Custom filter presets were cut on the measurement (0.008% of input tokens). Phase 0 validated the premise against v0.44.0 on the full transcript corpus before any build work."
3473
+ ---
3474
+
3475
+ # rtk Pack Integration
3476
+
3477
+ **Completed 2026-07-26, shipped in v4.18.0.** Kept whole rather than summarised,
3478
+ because the value here is the measurements and the things that turned out to be
3479
+ false, not the plan structure. Live procedure:
3480
+ `kb/procedures/rtk-upstream-sync-sop.md`.
3481
+
3482
+ Headline: rtk-pack ships opt-in, built from source with telemetry compiled out,
3483
+ and saves a **measured 0.0615% of input tokens** against a kill number of 0.05%
3484
+ published before the measurement. Phase 3 (our own filter presets) was cut at
3485
+ 0.008%. The projection this plan published for itself was about twice too
3486
+ optimistic, and section 10.1 says why.
3487
+
3488
+ ## 1. Decision
3489
+
3490
+ Ship `rtk` to ai-toolkit users as an **opt-in plugin pack**, with the binary
3491
+ supply chain owned end to end by us.
3492
+
3493
+ Five choices, made and locked:
3494
+
3495
+ | Question | Decision |
3496
+ |---|---|
3497
+ | Delivery | Plugin pack (`rtk-pack`), never a native feature |
3498
+ | Binary hosting | Our own GitHub Release, fetched and SHA-256 verified at pack install |
3499
+ | Binary provenance | Cross-built from upstream source in our CI, `RTK_TELEMETRY_URL` unset |
3500
+ | Pinned upstream tag | **v0.44.0** (2026-07-26). v0.43.0 was ruled out on evidence, see section 2 |
3501
+ | Upstream tracking | An SOP that rebuilds, verifies, and re-publishes when rtk ships a release |
3502
+
3503
+ Upstream: https://github.com/rtk-ai/rtk, Apache-2.0, Rust 1.91 plus a C
3504
+ toolchain (rusqlite `bundled` compiles SQLite from source).
3505
+
3506
+ ## 2. Phase 0: premise validated before building anything
3507
+
3508
+ The retired output filter validated its design across five phases and its
3509
+ premise not at all until the fifth. This plan runs premise validation first.
3510
+ Everything below is measured, not estimated.
3511
+
3512
+ ### 2.1 Why v0.43.0 was ruled out
3513
+
3514
+ Two structural facts, read from the source of both tags:
3515
+
3516
+ - **Custom TOML filters are not wired into the rewrite path in v0.43.0.**
3517
+ `src/discover/registry.rs` at v0.43.0 contains zero references to
3518
+ `toml_filter`. In v0.44.0 they appear at `registry.rs:1047,1055,1058`.
3519
+ *This reason no longer applies:* section 7 cut our own presets, so nothing we
3520
+ ship depends on that wiring. It is recorded because it was one of the two
3521
+ arguments made at pin time, and the pin now rests on the second alone.
3522
+ - **`pipeline_final_safe` does not exist in v0.43.0.** It arrives in v0.44.0
3523
+ (`rules.rs:7,115,124`, `registry.rs:605,659,864,1069`). v0.43.0 rewrote the
3524
+ first pipeline stage; v0.44.0 rewrites the last. On this traffic 56.5% of
3525
+ `rtk grep` hits arrive through the pipeline-final path, so they do not exist
3526
+ at all on v0.43.0.
3527
+
3528
+ ### 2.2 The port models v0.44.0, not v0.43.0
3529
+
3530
+ `rtk_port.py` is re-validated by extracting the `assert_eq!` assertions from
3531
+ each tag's `registry.rs` `#[cfg(test)]` block and replaying them:
3532
+
3533
+ | Port checked against | Assertions | Exact-string agreement |
3534
+ |---|---:|---:|
3535
+ | **v0.44.0** | 203 | **203/203** |
3536
+ | v0.43.0 | 160 | 147/160 |
3537
+
3538
+ The 13 v0.43.0 failures are all the same pipeline inversion. The port also
3539
+ passes the 15 assertions v0.44.0 added after the port was written. Pinning
3540
+ v0.44.0 makes the existing measurement correct rather than requiring rework.
3541
+
3542
+ `registry.rs:590-592` returns `Some(unchanged)` for a simple already-`rtk`
3543
+ command; the port's equivalent bail must be mapped to that, not to `None`.
3544
+
3545
+ ### 2.3 The recorded numbers came from an 11% sample
3546
+
3547
+ `measure_rtk.py` sets `N_FILES = 134` and takes the most recently modified
3548
+ transcripts. The available pool is **1224 transcripts**. At n=134 the projected
3549
+ saving swings **8.8x** across windows (0.071% to 0.628%); at n>=408 it converges
3550
+ to within **1.05x**. The spread recorded in earlier revisions of this plan was
3551
+ sampling noise.
3552
+
3553
+ ### 2.4 The port over-counted, and by how much
3554
+
3555
+ Both real entry points call `contains_unattestable_construct` **before**
3556
+ `registry::rewrite_command`: `hook_cmd.rs:149-151` (Defer) and
3557
+ `rewrite_cmd.rs:54-56` (Passthrough). Any command carrying a command
3558
+ substitution, a process substitution, or a redirect with a file target is
3559
+ rejected whole. Upstream's rewrite tests call `rewrite_command` directly, so a
3560
+ port validated only against them passes while still over-counting.
3561
+
3562
+ The gate is ported from `lexer.rs:295-347` and validated **35/35** against
3563
+ upstream's own assertions at `lexer.rs:1186-1276`. Applying it costs 3.1%
3564
+ relative coverage.
3565
+
3566
+ ### 2.5 Corrected numbers
3567
+
3568
+ Full corpus, 1224 transcripts, 552 MB, 106 MB of tool results, 26.6 MB of
3569
+ successful Bash, entry gate applied:
3570
+
3571
+ | Metric | Earlier revision (n=134, ungated) | Phase 0 (n=1224, gated) |
3572
+ |---|---:|---:|
3573
+ | Coverage of successful Bash bytes | 31.5% | **35.17%** |
3574
+ | Coverage of all tool-result bytes | 9.69% | **8.77%** |
3575
+ | Ceiling: tool results as share of input volume | 5.52% | **4.54%** |
3576
+ | Projection at rtk's 60-90% claim | 0.32-0.48% | **0.239-0.359%** |
3577
+ | Projection, mechanism modelled | 0.15-0.21% | **0.117-0.164%** |
3578
+ | Reach of our own TOML presets (the number that cut section 7) | 1.91% of Bash | **1.08% of Bash** |
3579
+ | `Read` share of tool-result bytes | 53.8% | **62.8%** |
3580
+
3581
+ Modelled effectiveness per family against the claim: `rtk grep` 75% claimed and
3582
+ **9.0%** modelled; `rtk read` 60% claimed and **0.0%** modelled; `rtk rg` 75%
3583
+ and 30%; `rtk make` 65% and 16%; `rtk jq` 74%.
3584
+
3585
+ Largest unaddressed buckets, as a share of successful Bash bytes: pipeline not a
3586
+ rewritable final stage **39.78%**, ignored by rtk **10.72%**, entry gate
3587
+ **8.29%**, multiline script **4.71%**.
3588
+
3589
+ ### 2.6 Open question 3 answered: no
3590
+
3591
+ Patching `pipeline_final_safe` for `head` and `tail` unlocks 10.56 MB, 39.96% of
3592
+ Bash bytes. **88.3% of that (9.33 MB) routes to `rtk read`, measured at 0.00%
3593
+ saving** on this traffic because it returns files verbatim at the default
3594
+ `--level none`. The largest reachable gap is reachable and worthless. The
3595
+ argument for maintaining a fork does not survive its own measurement.
3596
+
3597
+ ## 3. Why a pack rather than a native feature
3598
+
3599
+ - **Opt-in is structural, not a policy.** Plugin packs must not be
3600
+ auto-installed [PATH: kb/reference/plugin-pack-conventions.md]. The retired
3601
+ output filter shipped `off` by default and still had to be wired into every
3602
+ install path, which is why v4.17.0 needed a bespoke migration cleanup.
3603
+ - **Removal is already solved.** `plugin remove` strips hook entries from
3604
+ `settings.json`, deletes owned `plugin-<pack>-*` assets, and leaves core and
3605
+ user files alone.
3606
+ - **The core package stays lean.** ai-toolkit is 4.4 MB unpacked; the platform
3607
+ artifacts are ~19.5 MB compressed. They never enter the base package.
3608
+ - **There is a working precedent.** `memory-pack` ships hooks, a shared script
3609
+ directory, an install-time init script, its own data with retention, and a
3610
+ clean uninstall. `rtk-pack` is the same shape with a binary instead of a
3611
+ database.
3612
+
3613
+ ## 4. Architecture
3614
+
3615
+ ```text
3616
+ app/plugins/rtk-pack/
3617
+ ├── plugin.json # pinned rtk version + per-platform SHA-256
3618
+ ├── README.md
3619
+ ├── hooks/
3620
+ │ └── rewrite.sh # PreToolUse; delegates to the fetched binary
3621
+ └── scripts/
3622
+ ├── init.py # platform detect → fetch → verify → install
3623
+ └── status.py # reports binary, digest and hook wiring
3624
+ ```
3625
+
3626
+ Runtime layout on a user machine:
3627
+
3628
+ ```text
3629
+ ~/.softspark/ai-toolkit/
3630
+ ├── hooks/plugin-rtk-pack-rewrite.sh # installed hook
3631
+ └── plugin-scripts/rtk-pack/
3632
+ ├── bin/rtk # verified binary, 0755
3633
+ └── version.json # pinned version + recorded digest
3634
+ ```
3635
+
3636
+ Everything the pack writes lives under `plugin-scripts/rtk-pack/`, which
3637
+ `plugin remove` deletes wholesale (`plugin.py:500-503`). The pack owns no state
3638
+ outside the toolkit data directory: no `filters.toml`, no entry in rtk's trust
3639
+ store, nothing in an OS config directory. That is a consequence of cutting
3640
+ section 7 and it is what keeps removal complete.
3641
+
3642
+ Binary supply chain:
3643
+
3644
+ ```text
3645
+ upstream tag → our CI cross-build (RTK_TELEMETRY_URL unset)
3646
+ → our GitHub Release softspark-rtk-<upstream>-<build>
3647
+ → SHA-256 recorded in plugin.json
3648
+ → plugin install: detect platform, fetch, verify, chmod
3649
+ ```
3650
+
3651
+ ## 5. Phase 1: build pipeline and binary release
3652
+
3653
+ **Outcome:** we can produce, from an upstream tag, a set of binaries that
3654
+ provably make no network calls of their own.
3655
+
3656
+ **Targets: five, not six.** `x86_64-apple-darwin`, `aarch64-apple-darwin`,
3657
+ `x86_64-unknown-linux-musl`, `aarch64-unknown-linux-gnu`,
3658
+ `x86_64-pc-windows-msvc`. Earlier revisions added `x86_64-unknown-linux-gnu` on
3659
+ the reasoning that upstream does not ship it. It is dropped: upstream's own
3660
+ Homebrew formula routes Linux x86_64 to the **musl** tarball
3661
+ (`release.yml:314-316`), a static musl binary runs on glibc, and upstream has
3662
+ never validated a gnu x86_64 artifact, so we would be first to ship one and
3663
+ would own all its breakage plus an uncontrolled glibc floor inherited from the
3664
+ runner image.
3665
+
3666
+ Build constraints, all read from the pinned checkout:
3667
+
3668
+ - **Pin the toolchain.** Upstream uses `dtolnay/rust-toolchain@stable`
3669
+ (`release.yml:65,116`) with `warnings = "deny"` (`Cargo.toml:70-72`) and ships
3670
+ no `rust-toolchain` file. A future rustc lint turns our builds red with no
3671
+ change on either side. We pin an exact version at or above 1.91.
3672
+ - **Windows must build natively.** `build.rs:6-13` emits
3673
+ `cargo:rustc-link-arg=/STACK:8388608` under `#[cfg(windows)]`, which in a
3674
+ build script is a **host** predicate. Cross-building Windows from Linux
3675
+ silently drops the 8 MiB stack reservation that upstream's own comment says is
3676
+ what makes `rtk.exe --version`, `--help`, and hook entry points start
3677
+ reliably. Build on `windows-latest`, or pass the link-arg explicitly.
3678
+ - **Both darwin targets build on `macos-latest` (arm64).** `macos-13` is the
3679
+ last x86_64 macOS image and GitHub is winding it down: on runs 30212577757 and
3680
+ 30214341444 that job sat queued indefinitely while every other target
3681
+ finished. Upstream builds both on arm64 too, but never runs its x86_64
3682
+ artifact. We do: the verifier detects Rosetta 2 and executes it, and reports
3683
+ `inconclusive` rather than `pass` if it cannot.
3684
+ - **Every target needs a target-capable C compiler.** rusqlite `bundled`
3685
+ (`Cargo.toml:26`) is not switchable off; there is no `[features]` table and no
3686
+ `cfg(feature` in `src/`. There is no pure-Rust escape route.
3687
+ - **aarch64-linux is the hard case, but upstream already solves it** with
3688
+ `cross: true` on `ubuntu-latest` (`release.yml:51-54`). Copy that. Upstream
3689
+ sets only the linker env var and lets the `cc` crate guess the C compiler;
3690
+ export `CC_aarch64_unknown_linux_gnu` and `AR_aarch64_unknown_linux_gnu`
3691
+ explicitly so a runner image change surfaces as a clear error.
3692
+ - **Run our own blocking `cargo audit`.** Upstream's is advisory: `ci.yml:88-96`
3693
+ swallows failures into a warning. Rebuilding from a tag inherits that
3694
+ lockfile, and on v0.44.0 it inherits four advisories across three of its 203
3695
+ crates. The audit and the build jobs apply the same remediation, so we audit
3696
+ the dependency set we ship rather than a different one:
3697
+
3698
+ | Crate | Advisory | Disposition |
3699
+ |---|---|---|
3700
+ | `anyhow` 1.0.102 | RUSTSEC-2026-0190 | `cargo update` to 1.0.103, in range |
3701
+ | `crossbeam-epoch` 0.9.18 | RUSTSEC-2026-0204 | `cargo update` to 0.9.20, in range |
3702
+ | `quick-xml` 0.37.5 | RUSTSEC-2026-0194, RUSTSEC-2026-0195 | ignored with reasons |
3703
+
3704
+ The quick-xml fix lands only in 0.41.0 while rtk pins `"0.37"` as a direct
3705
+ dependency (`Cargo.toml:34`), so taking it means editing `Cargo.toml` and the
3706
+ calling code in `src/cmds/dotnet/dotnet_trx.rs`. That would break the promise
3707
+ that the only difference from an upstream build is the undefined telemetry
3708
+ endpoint. The reachable surface is narrow: quick-xml parses .NET TRX test
3709
+ output only, the input is a report produced locally by the user's own test
3710
+ run, and both advisories are availability-only (CVSS `C:N/I:N/A:H`). The SOP
3711
+ re-checks this on every sync and deletes the ignores once upstream moves to
3712
+ quick-xml 0.41 or later.
3713
+ - A target that will not build is dropped, not faked.
3714
+
3715
+ ### 5.1 Proving the binary is silent
3716
+
3717
+ The endpoint is compile-time only: `option_env!("RTK_TELEMETRY_URL")` at
3718
+ `telemetry.rs:16` and `telemetry_cmd.rs:176`, with the token at
3719
+ `telemetry.rs:17` and `telemetry_cmd.rs:189`. Nothing supplies a default:
3720
+ `build.rs` emits no `cargo:rustc-env`, `TelemetryConfig` has no URL field, and
3721
+ no runtime `std::env::var` resolves the endpoint. Upstream injects it only in
3722
+ `release.yml:85-86,124-125,151-152`. Building without it yields a binary whose
3723
+ telemetry destination cannot be re-enabled by any env var or config at runtime.
3724
+
3725
+ **The acceptance criterion in earlier revisions was untestable and is replaced.**
3726
+ "No telemetry symbols in the binary" cannot be checked: the guard is a runtime
3727
+ `if TELEMETRY_URL.is_none()` on a const (`telemetry.rs:23-26`), not a `#[cfg]`,
3728
+ so the code compiles in and is only eliminated by LLVM as an optimisation; and
3729
+ `Cargo.toml:51` sets `strip = true`, which removes the symbols regardless of
3730
+ whether the code is present. A symbol check would pass for the wrong reason.
3731
+
3732
+ What we assert instead:
3733
+
3734
+ 0. **The artifact was actually started.** Every other assertion is worthless on
3735
+ a binary nobody executed, and two of the five targets are not native to their
3736
+ runner. `aarch64-unknown-linux-gnu` runs under `qemu-user` with the cross
3737
+ sysroot passed as `-L`, and `x86_64-apple-darwin` runs under Rosetta 2 on the
3738
+ arm64 runner. A target that cannot be started reports `inconclusive`, never
3739
+ `pass`.
3740
+ 1. **Build gate.** `RTK_TELEMETRY_URL` and `RTK_TELEMETRY_TOKEN` are unset in
3741
+ the build environment, asserted in CI before `cargo build`, with a clean
3742
+ target directory per build.
3743
+ 2. **Offline smoke run.** Each artifact runs its real command surface with no
3744
+ network route available and makes zero outbound connections. This is
3745
+ Linux-only: `unshare -rn` has no unprivileged equivalent on macOS or Windows
3746
+ runners, and Ubuntu 24.04's
3747
+ `kernel.apparmor_restrict_unprivileged_userns=1` means even there it needs
3748
+ `sudo`. On the first run this assertion silently degraded to a skip on every
3749
+ target while the verdict still read `pass`, so the manifest now records which
3750
+ isolator was used and a target that could not be started at all reports
3751
+ `inconclusive` rather than `pass`.
3752
+ 3. **Filesystem assertion.** No telemetry state is created under the resolved
3753
+ data directory.
3754
+ 4. **Drift detection.** Record artifact size and a string-allowlist hash per
3755
+ target and fail the build on unexplained drift, since whether `ureq`,
3756
+ `rustls`, `ring` and `webpki-roots` are actually eliminated is an LLVM
3757
+ outcome under `lto = true`, not a guarantee.
3758
+
3759
+ Three findings that must reach the pack README:
3760
+
3761
+ - **`rtk cc` shells out to `npx --yes ccusage`** when `ccusage` is not on PATH
3762
+ (`ccusage.rs:104-119`). That is a runtime npm fetch and third-party code
3763
+ execution. It is not telemetry and not automatic, but it is outbound network
3764
+ from a binary we tell users makes no network calls.
3765
+ - **`RTK_TELEMETRY_DISABLED` only accepts the exact string `1`**
3766
+ (`telemetry_cmd.rs:31-33`, locked by the test at `:222-229`). `true` and `yes`
3767
+ are silent no-ops. Never write anything else anywhere in the pack.
3768
+ - **`rtk telemetry forget` on an endpoint-free build prints a misleading
3769
+ failure** telling the user to email upstream to complete erasure
3770
+ (`telemetry_cmd.rs:158-169`), when nothing was ever sent.
3771
+
3772
+ Apache-2.0 obligations ship alongside: upstream `LICENSE`, a `NOTICE` file
3773
+ (upstream has none), and a statement that the only build-time change is leaving
3774
+ the telemetry endpoint undefined. **Do not redistribute upstream
3775
+ `DISCLAIMER.md` verbatim**: `DISCLAIMER.md:25` states telemetry is collected by
3776
+ default, which the code contradicts and which a compliance reviewer would read
3777
+ as a reason to block the pack. `docs/TELEMETRY.md:180` calls the mechanism
3778
+ "compile-time gating" where "all telemetry code is dead", which is imprecise for
3779
+ the same reason our own criterion changed.
3780
+
3781
+ **Success criteria:** five artifacts build from the pinned tag; each runs
3782
+ `rtk --version` on its target; the silence assertions pass or say plainly that
3783
+ they did not run; checksums published.
3784
+
3785
+ ### 5.2 Result, run 30214882862
3786
+
3787
+ All five targets build and verify. Every artifact was actually started, none
3788
+ shipped on a skipped assertion.
3789
+
3790
+ | Target | `runs` | `offline` |
3791
+ |---|---|---|
3792
+ | `aarch64-apple-darwin` | native | not applicable |
3793
+ | `x86_64-apple-darwin` | translated via Rosetta 2 | not applicable |
3794
+ | `x86_64-unknown-linux-musl` | native | passed under `sudo -n unshare -rn` |
3795
+ | `aarch64-unknown-linux-gnu` | emulated via `qemu-aarch64-static` | passed under `sudo -n unshare -rn` |
3796
+ | `x86_64-pc-windows-msvc` | native | not applicable |
3797
+
3798
+ No TLS markers in any artifact, so `ureq`, `rustls`, `ring` and `webpki-roots`
3799
+ appear to be eliminated under LTO once the endpoint const is `None`. That is an
3800
+ optimisation outcome rather than a guarantee, which is why the fingerprint is
3801
+ recorded per target and drift fails the build.
3802
+
3803
+ **Four of five targets are bit-reproducible.** Across independent runs with
3804
+ identical inputs, `aarch64-apple-darwin`, `x86_64-apple-darwin`,
3805
+ `aarch64-unknown-linux-gnu` and `x86_64-unknown-linux-musl` produced
3806
+ byte-identical binaries. The concern about unset `trim-paths` embedding registry
3807
+ paths does not materialise on these runners.
3808
+
3809
+ `x86_64-pc-windows-msvc` did not, and the cause was bounded: **24 bytes out of
3810
+ 8,018,432 differed**, in five regions. Four were the same two-byte value at
3811
+ `0x100` and in the debug directory, which is the MSVC link timestamp; the fifth
3812
+ was a 16-byte CodeView GUID. The code was identical; only the link stamp varied.
3813
+
3814
+ `-C link-arg=/Brepro` is now set for the Windows target, which makes the linker
3815
+ derive that timestamp from content rather than the clock. **The shipped
3816
+ `softspark-rtk-v0.44.0-1` artifacts predate that change**, so the Windows digest
3817
+ in `plugin.json` is from a non-deterministic link; the flag takes effect on the
3818
+ next rebuild, which will produce a different Windows digest for the same source.
3819
+
3820
+ This matters for the sync SOP: a changed digest means changed input rather than
3821
+ build noise. That inference holds on four targets today and on all five from the
3822
+ next rebuild onward.
3823
+
3824
+ Note that the **archives** are not reproducible even where the binaries are,
3825
+ because gzip records a timestamp. Compare extracted binaries, never the
3826
+ tarballs.
3827
+
3828
+ Three defects the run surfaced, all now fixed and guarded:
3829
+
3830
+ - The gnu target is dynamically linked, so `qemu-user` needs the cross sysroot.
3831
+ It is passed as `-L` rather than `QEMU_LD_PREFIX`, because the offline check
3832
+ runs under `sudo` and `env_reset` drops the variable.
3833
+ - The offline check compared a plain run against a namespaced one, so `sudo`'s
3834
+ environment handling was confounded with the network variable and a harness
3835
+ failure was reported as a behaviour difference in the binary. Both sides now
3836
+ run in a namespace and differ only in `-n`.
3837
+ - `7z` stored the full relative path, so the Windows zip held
3838
+ `rtk-src/target/<triple>/release/rtk.exe` while every tar.gz held a bare
3839
+ `rtk`. The pack installer extracts all five the same way, so this was a
3840
+ silent install failure on one platform. A `collect` step now asserts every
3841
+ archive holds exactly one flat entry.
3842
+
3843
+ ## 6. Phase 2: the pack
3844
+
3845
+ **Outcome:** `ai-toolkit plugin install rtk-pack` produces a working setup and
3846
+ `plugin remove` leaves nothing behind.
3847
+
3848
+ - `scripts/init.py` detects platform and libc, fetches the matching artifact,
3849
+ verifies its SHA-256 against `plugin.json`, and installs to
3850
+ `plugin-scripts/rtk-pack/bin/rtk`. A mismatch aborts and removes the partial
3851
+ download. No network at runtime, only at install.
3852
+ - Failure to fetch is not an install failure: the pack degrades to inert and
3853
+ says so, matching how the core behaves when `jq` is missing.
3854
+ - The hook is wired at `PreToolUse` through the existing pack hook mechanism, so
3855
+ `plugin remove` strips it via the `_source` marker.
3856
+ - `plugin status` dispatches to the pack's own `scripts/status.py`, which is
3857
+ generic rather than another hardcoded `if name == ...` branch. It reports the
3858
+ binary, the install record, the live version, and hook wiring, and announces a
3859
+ missing binary as inert rather than letting a green install imply it works.
3860
+
3861
+ ### 6.1 No trust step, because the pack ships no filters
3862
+
3863
+ Section 7 is cut, so the pack writes no `filters.toml` and takes no entry in
3864
+ rtk's trust store. That removes the single highest-ranked risk this plan
3865
+ carried: an untrusted or content-changed preset produces **zero output on the
3866
+ command path with no warning, no stderr line, and no non-zero exit**
3867
+ (`toml_filter.rs:220-221`, `:450-458`) — installed, apparently fine, doing
3868
+ nothing, which is exactly the shape of failure that retired the in-house filter.
3869
+
3870
+ The pack instead reports whether it is actually working. `scripts/status.py`
3871
+ distinguishes installed from functioning: a missing binary is announced as
3872
+ `MISSING — the hook is inert and every command runs unchanged` rather than
3873
+ inferred from a green install.
3874
+
3875
+ **If presets are ever revisited, these are the constraints, kept because they
3876
+ were expensive to establish:**
3877
+
3878
+ - Trust is a byte-exact SHA-256 (`trust.rs:142-163`), so every write, upgrade,
3879
+ repair, trailing newline, or git CRLF normalisation invalidates it.
3880
+ - `rtk trust --yes` is indiscriminate: it trusts every gated file that exists,
3881
+ including a `.rtk/filters.toml` an attacker committed into the CWD
3882
+ (`trust.rs:262-305`, project path is CWD-relative at `:207`).
3883
+ - It exits 1 both when there is nothing to trust and when the file is invalid
3884
+ TOML (`trust.rs:307-312`), so `set -e` aborts on the benign case.
3885
+ - There is no `rtk doctor`, and `rtk trust --list` never re-hashes
3886
+ (`trust.rs:242-256`). Verification means reading the store and recomputing.
3887
+ - The global path is not `~/.config/rtk/filters.toml` everywhere: it comes from
3888
+ `dirs::config_dir()` (`trust.rs:206-216`), which on macOS is
3889
+ `~/Library/Application Support`. rtk's own error text hardcodes the Linux path
3890
+ and is wrong there.
3891
+ - One global file, no include mechanism, so injection would have to follow the
3892
+ merge-friendly install model [PATH: kb/reference/merge-friendly-install-model.md]
3893
+ with comment markers rather than owning the file.
3894
+
3895
+ **Success criteria:** install, status, update, remove, and re-install are
3896
+ idempotent; a machine without the pack behaves exactly as today; uninstalling
3897
+ ai-toolkit removes every pack artifact.
3898
+
3899
+ ## 7. Phase 3: cut
3900
+
3901
+ **Cut on the measurement, 2026-07-26.** Custom TOML presets reach **1.08% of
3902
+ Bash bytes**, which is 0.27% of tool-result bytes. At a generous 60% reduction
3903
+ that is **0.008% of input tokens**, roughly one part in twelve thousand.
3904
+
3905
+ Against that: presets would have required a trust step that fails silently, a
3906
+ doctor check that recomputes digests, marker injection into a file the user may
3907
+ also edit, re-trusting after every write, and a per-upstream-release review of a
3908
+ DSL whose `deny_unknown_fields` rejects the entire file on one unknown key. The
3909
+ maintenance surface is large, the payoff is not measurable, and every item on
3910
+ that list is a way for the pack to look installed while doing nothing.
3911
+
3912
+ The pack therefore ships upstream rtk's built-in filters and nothing of our own.
3913
+ The 35.2% coverage quoted in section 2.5 is already what those built-ins
3914
+ deliver; it does not shrink as a result of this cut.
3915
+
3916
+ Two consequences recorded so they are not rediscovered:
3917
+
3918
+ - The pack owns no state outside `~/.softspark/ai-toolkit/plugin-scripts/rtk-pack/`,
3919
+ which is what makes `plugin remove` complete.
3920
+ - One of the two arguments for pinning v0.44.0 was that custom filters are only
3921
+ wired into the rewrite path there. That argument is now moot; the pin rests on
3922
+ `pipeline_final_safe`, which carries 56.5% of `rtk grep` hits on this traffic
3923
+ and does not exist in v0.43.0. See section 2.1.
3924
+
3925
+ Reopening this is a plan change, not a task: it needs a workload where rtk's
3926
+ built-ins measurably under-perform and a preset that measurably closes the gap,
3927
+ demonstrated by replay before any code is written.
3928
+
3929
+ ## 8. Phase 4: auto-update on `ai-toolkit update`
3930
+
3931
+ **Outcome:** a user who installed the pack gets the new binary by running the
3932
+ update they already run.
3933
+
3934
+ Verified gap: `handleUpdate` in `bin/ai-toolkit.js` reads installed modules from
3935
+ `state.json` and does not touch `plugins.json`. Nothing propagates to packs
3936
+ today, so this is new wiring, not a configuration change.
3937
+
3938
+ **Done.** `ai-toolkit update` now calls `plugin update --editor all --all`
3939
+ after the core update. Two things had to change first, both generic rather than
3940
+ rtk-specific:
3941
+
3942
+ - `update_pack` was an unconditional remove-then-reinstall, so wiring it into
3943
+ the core update would have refetched the binary on every run. `plugins.json`
3944
+ now records the pack version installed per editor, and an update whose
3945
+ manifest version matches is a silent no-op. `--force` overrides.
3946
+ - State written before versions were tracked has no `versions` map, so every
3947
+ pack reads as stale exactly once and is updated once. That is the intended
3948
+ migration, not a bug.
3949
+
3950
+ `--local` leaves packs alone: they live in `~/.softspark/ai-toolkit` and are
3951
+ global, while `--local` is project-local config only.
3952
+
3953
+ - After the core update completes, `ai-toolkit update` reads `plugins.json` and
3954
+ runs the equivalent of `plugin update` for every **currently installed** pack.
3955
+ Packs that are not installed stay untouched, which preserves adoption rule 1.
3956
+ - The rtk-pack update path is: compare the manifest's pinned version against the
3957
+ recorded pack version; if they differ, fetch and verify the new binary and
3958
+ re-install the hook.
3959
+ - `--dry-run` reports what would change per pack.
3960
+ - A pack update failure warns and continues; it never fails the core update.
3961
+ - Bats coverage in `tests/test_plugin_update.bats`: stale version replaced;
3962
+ current version untouched and silent; pack absent means no work; `--dry-run`
3963
+ reports without acting; legacy state with no version map updates once; a
3964
+ corrupt `plugins.json` does not crash the run.
3965
+
3966
+ **Success criteria:** the wiring is generic across packs, not rtk-specific;
3967
+ `update` remains idempotent; a failed pack update never leaves a half-installed
3968
+ binary.
3969
+
3970
+ ## 9. Phase 5: upstream sync SOP
3971
+
3972
+ **Outcome:** a written procedure so tracking upstream is routine rather than a
3973
+ research project each time. **Written**, at
3974
+ `kb/procedures/rtk-upstream-sync-sop.md`, modelled on the existing
3975
+ ecosystem-sync SOP.
3976
+
3977
+ The 0.43.0 to 0.44.0 bump is a worked example of why the review step exists: 200
3978
+ commits, and every file the SOP names changed, including a semantic inversion in
3979
+ pipeline rewriting and the arrival of the trust gate.
3980
+
3981
+ Steps the SOP must cover:
3982
+
3983
+ 1. **Detect.** Check the upstream releases feed for a tag newer than the pinned
3984
+ one. Their stable cadence is roughly two to four weeks, behind a long
3985
+ release-candidate train. Cadence: on demand, plus a check folded into release
3986
+ preparation.
3987
+ 2. **Review before building.** Read the changelog and diff `src/discover/rules.rs`,
3988
+ `src/discover/registry.rs`, `src/discover/lexer.rs`, `IGNORED_PREFIXES`, the
3989
+ TOML DSL structs in `src/core/toml_filter.rs`, `src/hooks/trust.rs`, and
3990
+ anything touching telemetry. A change to the DSL or to trust handling is a
3991
+ stop-and-think, not a rebuild.
3992
+ 3. **Re-validate the port.** Re-extract the `rewrite_command` assertions from the
3993
+ new tag's `registry.rs` test block and the gate assertions from `lexer.rs`,
3994
+ and replay both. Anything short of full agreement invalidates every coverage
3995
+ number until the port is fixed.
3996
+ 4. **Rebuild** all five targets from the new tag with telemetry unset.
3997
+ 5. **Verify.** Binary runs on each target; the silence assertions pass; the
3998
+ pack's fixtures still produce the expected decisions.
3999
+ 6. **Publish** a new release in our namespace and record the new digests.
4000
+ 7. **Bump** `rtk-pack` version in `plugin.json` and note the upstream version it
4001
+ tracks. The bump is what makes `plugin update` fire at all, since a pack whose
4002
+ recorded version still matches is skipped silently.
4003
+ 8. **Ship** in the next ai-toolkit release; installed packs pick it up through
4004
+ Phase 4.
4005
+ 9. **Record** the licence position if upstream relicenses or adds a NOTICE.
4006
+
4007
+ There is no upstream test asserting network silence, so that property can
4008
+ regress on any bump without turning their CI red. We own that test and re-run it
4009
+ every time.
4010
+
4011
+ **Success criteria:** a maintainer who has never done it can follow the SOP end
4012
+ to end; the review step names specific files rather than saying "check for
4013
+ breaking changes".
4014
+
4015
+ ## 10. Verification
4016
+
4017
+ The pack must be able to answer "did this help?" with a number.
4018
+
4019
+ **The before-and-after design is replaced.** Phase 0 measured its noise floor:
4020
+ on a 134-transcript window the projected saving varies by 8.8x while the
4021
+ mechanism is unchanged. A before-and-after comparison cannot detect an effect of
4022
+ 0.117% to 0.164% against that. Two changes make the measurement possible:
4023
+
4024
+ - **Measure over the whole transcript pool, not a recency window.** At n>=408
4025
+ the same metric converges to within 1.05x.
4026
+ - **Measure by replay, not by elapsed calendar time.** Run real captured tool
4027
+ output through the built binary and compare byte counts directly. That is the
4028
+ method that produced the honest 0% which retired the in-house filter, and it
4029
+ removes the confound of what work the user happened to do that week.
4030
+
4031
+ The harness is `replay_rtk.py` in the measurement archive, not a script inside
4032
+ the pack. It pairs each successful Bash result with the command that produced
4033
+ it, keeps the ones the validated port says rtk would rewrite, and pipes the
4034
+ captured bytes through the shipped binary via `rtk pipe -f <filter>`.
4035
+
4036
+ Two limits, stated because the number is meaningless without them:
4037
+
4038
+ - `rtk pipe` runs the same filter code as the command path but knows less than
4039
+ it does: no result caps, no exit code, no file set. For search families the
4040
+ replay figure is a **lower bound** on what the command path would save.
4041
+ - Families with no pipe filter (`rtk read`, `rtk ls`, `rtk wc`) cannot be
4042
+ measured this way at all. They are reported as **unmeasurable**, never folded
4043
+ in as zero, because a zero that is really an absence is how the previous
4044
+ effort talked itself into shipping.
4045
+
4046
+ Session-level token accounting, when it is wanted, comes from the session JSONL
4047
+ [PATH: scripts/session_token_stats.py], summing all four usage fields: in a
4048
+ cached session most context tokens land in the cache fields rather than
4049
+ `input_tokens`.
4050
+
4051
+ **Kill number, published before the measurement rather than argued after it:**
4052
+ if replay on the full corpus shows the shipped binary saving less than 0.05% of
4053
+ input tokens, the pack is not worth its maintenance and supply-chain surface,
4054
+ and it is retired the way the output filter was.
4055
+
4056
+ ### 10.1 Result: measured 0.0615%, kill number survived by 23%
4057
+
4058
+ Full pool, 28.56 MB of successful Bash output, replayed through the published
4059
+ `softspark-rtk-v0.44.0-1` binary:
4060
+
4061
+ | | |
4062
+ |---|---:|
4063
+ | Addressed by rtk | 9.99 MB, **35.00%** of Bash bytes |
4064
+ | Measurable through `rtk pipe` | 5.66 MB |
4065
+ | After filtering | 4.22 MB |
4066
+ | **Measured saving** | **1.44 MB = 360,529 tokens** |
4067
+ | As a share of tool-result bytes | 1.35% |
4068
+ | **As a share of input tokens** | **0.0615%** |
4069
+
4070
+ Against a kill number of 0.05%, the pack survives by a factor of 1.23. That is a
4071
+ pass, not a vindication.
4072
+
4073
+ **The projection published in section 2.5 was roughly twice too optimistic.**
4074
+ It said 0.117% to 0.164%; the measurement says 0.0615%. The error has a single
4075
+ identifiable cause: the model credited families it could not simulate with
4076
+ rtk's own claimed 60-90%. Replayed, the families that can be measured deliver
4077
+ **25.5%** in aggregate.
4078
+
4079
+ Per family, measured against modelled and against upstream's claim:
4080
+
4081
+ | Family | Measured | Modelled | Claimed |
4082
+ |---|---:|---:|---:|
4083
+ | `rtk grep` | **22.3%** | 9.0% | 75% |
4084
+ | `rtk git` | **33.1%** | — | 70% |
4085
+ | `rtk find` | **35.6%** | — | 70% |
4086
+ | `rtk rg` | **7.0%** | 30% | 75% |
4087
+ | `rtk ruff` | **0.0%** | — | — |
4088
+
4089
+ The model **understated** grep by 2.5x and the total still came out high, so the
4090
+ two errors are unrelated: coverage modelling was sound, per-family effectiveness
4091
+ was guesswork wherever the source was not read closely.
4092
+
4093
+ **What the replay does not settle.** 43.3% of addressed bytes (4.33 MB) have no
4094
+ `rtk pipe` filter and are unmeasurable, the largest being `rtk read` at 1.55 MB.
4095
+ That family is known from source to return files verbatim at the default
4096
+ `--level none`, so the headroom is smaller than 43.3% suggests. No extrapolation
4097
+ is applied: a zero that is really an absence of measurement is how the previous
4098
+ effort talked itself into shipping. For search families the pipe path also lacks
4099
+ the command path's result caps, making these figures a lower bound.
4100
+
4101
+ **One independent confirmation.** The replay measured coverage at 35.00% of Bash
4102
+ bytes; the gated port measured 35.17% on the same pool by a different method.
4103
+ The port is right about *which* commands rtk touches even where it was wrong
4104
+ about how much each saves.
4105
+
4106
+ Reproduce with `replay_rtk.py --pool 1300 --rtk <binary>` in the measurement
4107
+ archive.
4108
+
4109
+ ## 11. Licence and security obligations
4110
+
4111
+ - **Apache-2.0.** Ship upstream `LICENSE` with the binaries, add a `NOTICE`, and
4112
+ state the build-time change. Never relabel any upstream file MIT. Do not ship
4113
+ `DISCLAIMER.md` verbatim, see section 5.1.
4114
+ - **Telemetry.** Proven absent per build by the four assertions in section 5.1,
4115
+ re-proven on every upstream sync.
4116
+ - **Supply chain.** Digests pinned in `plugin.json`, verified on fetch,
4117
+ re-verified by `doctor`. This mirrors the existing checksum-pin discipline for
4118
+ URL-sourced rules and hooks. Run a blocking `cargo audit` on the pinned
4119
+ lockfile, because upstream's is advisory.
4120
+ - **Trust boundary.** rtk rewrites commands before execution, which the retired
4121
+ in-house contract explicitly forbade for itself. Adopting it is a conscious
4122
+ reversal of that constraint and its own threat surface: what runs is not what
4123
+ the model asked for. The pack's README must say this plainly, and
4124
+ `plugin install` must not be silent about it. The README must also disclose
4125
+ the `rtk cc` npx path.
4126
+
4127
+ The sharp edge is more specific than "the command changes". rtk evaluates the
4128
+ host's permission rules against the **original** command and applies the
4129
+ verdict to the **rewritten** one: `decide_hook_action(cmd, ...)` takes the
4130
+ command the model asked for, and `permissionDecision: allow` is then emitted
4131
+ for the substituted `rtk …` form (`hook_cmd.rs:405-436`). An allowlist entry
4132
+ for `git status` therefore authorises `rtk git status`, which the user never
4133
+ wrote a rule for. Upstream documents the absence case in its own test
4134
+ (`hook_cmd.rs:1272-1283`): with no matching allow rule, no decision is emitted
4135
+ and the normal prompt stands. Both halves belong in the README.
4136
+
4137
+ ## 12. Pre-mortem
4138
+
4139
+ | Rank | Failure mode | Probability | Impact | Mitigation |
4140
+ |---:|---|:---:|:---:|---|
4141
+ | — | ~~Presets install but are untrusted, so they silently do nothing~~ | — | — | **Eliminated**, not mitigated: section 7 is cut, so the pack ships no filters and takes no trust-store entry |
4142
+ | 2 | Cross-building with bundled SQLite is fragile in CI | Medium | Medium | Copy upstream's `cross` setup for aarch64-linux; five targets not six; a target that will not build is dropped, not faked |
4143
+ | 3 | Pre-execution rewriting changes command semantics | Medium | High | Opt-in pack, documented one-flag disable, upstream's own review process, our integration tests on real commands |
4144
+ | 4 | Windows binary cross-built without the 8 MiB stack reservation | Medium | High | Build natively on `windows-latest`, or pass the link-arg explicitly. `build.rs:6-13` |
4145
+ | 5 | Upstream velocity breaks the rewrite logic the coverage numbers model | Medium | Medium | Pinned version, SOP review step naming exact files, port re-validation as a gate, fork option preserved by Apache-2.0 |
4146
+ | 6 | Unpinned toolchain plus `warnings = "deny"` turns builds red with no change | Medium | Low | Pin an exact rustc version; treat bumps as deliberate |
4147
+ | 7 | A fetch failure leaves a half-installed pack | Medium | Medium | Verify-then-install, abort and clean on digest mismatch, `doctor` detects drift |
4148
+ | 8 | Live saving lands near the projection, not the claim | High | Medium | Section 10 measures it by replay either way, against a published kill number |
4149
+ | 9 | The auto-update wiring makes `update` slower or flakier | Low | Medium | Skip when versions match, warn-and-continue on failure, `--dry-run` coverage |
4150
+ | 10 | The pack drifts into being installed by default | Low | High | Adoption rule 1 plus a test asserting `install` never pulls it in |
4151
+
4152
+ ## 13. Open questions
4153
+
4154
+ 1. ~~Which upstream tag do we pin first?~~ **Answered: v0.44.0.** See section 2.1.
4155
+ 2. ~~User scope only, or a project-scoped preset variant too?~~ **Moot:** section 7 is cut, so the pack installs no presets at any scope.
4156
+ 3. ~~Is patching `pipeline_final_safe` for `head`/`tail` in scope later?~~
4157
+ **Answered: no.** See section 2.6.
4158
+ 4. ~~Does Phase 3 survive its own measurement at 0.008% of input tokens?~~
4159
+ **Answered: no, cut 2026-07-26.** See section 7.
4160
+ 5. ~~How does the pack share rtk's single global `filters.toml`?~~ **Moot:** it
4161
+ writes no filters at all. The marker-injection design and the trust
4162
+ constraints that would have applied are kept in section 6.1 in case presets
4163
+ are ever revisited.
4164
+
4165
+ ---
4166
+
3450
4167
  ## kb/howto/README.md
3451
4168
 
3452
4169
  ---
@@ -5453,7 +6170,7 @@ python3 scripts/validate.py --strict
5453
6170
  ai-toolkit plugin list # show available packs
5454
6171
  ai-toolkit plugin install --editor claude <name> # install for Claude Code global target
5455
6172
  ai-toolkit plugin install --editor codex <name> # install for Codex global target
5456
- ai-toolkit plugin install --editor all --all # install all 11 packs for both runtimes
6173
+ ai-toolkit plugin install --editor all --all # install all 12 packs for both runtimes
5457
6174
  ai-toolkit plugin update --editor all --all # re-apply all installed packs after toolkit updates
5458
6175
  ai-toolkit plugin clean <name> # prune data older than 90 days
5459
6176
  ai-toolkit plugin clean <name> --days 30 # custom retention
@@ -5904,12 +6621,17 @@ Add entry at the top of `CHANGELOG.md` (after the header, before previous releas
5904
6621
 
5905
6622
  ## Phase 4: Regenerate Artifacts
5906
6623
 
6624
+ Use the npm scripts, not the generators directly:
6625
+
5907
6626
  ```bash
5908
- python3 scripts/generate_agents_md.py > AGENTS.md
5909
- python3 scripts/generate_llms_txt.py > llms.txt
5910
- python3 scripts/generate_llms_txt.py --full > llms-full.txt
6627
+ npm run generate:agents # AI_TOOLKIT_NO_CUSTOM_RULES=1 python3 scripts/generate_agents_md.py > AGENTS.md
6628
+ npm run generate:llms # llms.txt + llms-full.txt
5911
6629
  ```
5912
6630
 
6631
+ `generate:agents` sets `AI_TOOLKIT_NO_CUSTOM_RULES=1`. Running
6632
+ `generate_agents_md.py` bare picks up whatever is registered in the maintainer's
6633
+ own `~/.softspark/ai-toolkit/rules/`, which then ships inside `AGENTS.md`.
6634
+
5913
6635
  Check if anything actually changed:
5914
6636
 
5915
6637
  ```bash
@@ -6618,6 +7340,290 @@ ai-toolkit eject /tmp/test # retry
6618
7340
 
6619
7341
  ---
6620
7342
 
7343
+ ## kb/procedures/rtk-upstream-sync-sop.md
7344
+
7345
+ ---
7346
+ title: "SOP: rtk Upstream Sync"
7347
+ category: procedures
7348
+ service: ai-toolkit
7349
+ tags: [sop, rtk, rtk-pack, upstream, cross-build, telemetry, checksum, port-validation, advisory]
7350
+ version: "1.0.0"
7351
+ created: "2026-07-26"
7352
+ last_updated: "2026-07-26"
7353
+ description: "Procedure for moving rtk-pack to a newer upstream rtk release: detect the tag, review the files we depend on, re-validate the Python port that every coverage number rests on, rebuild five targets with telemetry undefined, verify silence, publish to our own release namespace, and bump the pack. Written after the v0.43.0 to v0.44.0 bump, which changed every file this SOP names."
7354
+ ---
7355
+
7356
+ # SOP: rtk Upstream Sync
7357
+
7358
+ Moves `rtk-pack` from one pinned upstream tag to the next.
7359
+
7360
+ Currently pinned: **v0.44.0**, shipped as
7361
+ `softspark-rtk-v0.44.0-1`. The pin lives in
7362
+ `app/plugins/rtk-pack/plugin.json` under `upstream.version`.
7363
+
7364
+ Upstream ships stable tags roughly every two to four weeks behind a long
7365
+ release-candidate train (300+ RCs preceded v0.44.0). Do not track RCs.
7366
+
7367
+ Background and the measured numbers: `kb/history/completed/rtk-pack-integration-20260726.md`.
7368
+
7369
+ ## Why this SOP is not "just rebuild"
7370
+
7371
+ The v0.43.0 to v0.44.0 bump was 200 commits and touched **every file listed in
7372
+ Phase 2 below**. It also inverted pipeline rewriting: v0.43.0 rewrote the first
7373
+ stage of a pipeline, v0.44.0 rewrites the last. A rebuild without the review
7374
+ step would have shipped that silently, and the coverage numbers quoted to users
7375
+ would have described a version we no longer ship.
7376
+
7377
+ ## Phase 1: Detect
7378
+
7379
+ ```bash
7380
+ gh api repos/rtk-ai/rtk/releases --paginate \
7381
+ --jq '.[] | select(.tag_name | test("^v[0-9]")) | "\(.tag_name)\t\(.published_at)"' | head -5
7382
+ ```
7383
+
7384
+ Compare against `upstream.version` in `app/plugins/rtk-pack/plugin.json`.
7385
+
7386
+ Cadence: on demand, plus a check folded into release preparation.
7387
+
7388
+ ## Phase 2: Review before building
7389
+
7390
+ Fetch the diff for the areas the pack depends on:
7391
+
7392
+ ```bash
7393
+ gh api repos/rtk-ai/rtk/compare/<pinned>...<new> \
7394
+ --jq '{ahead: .ahead_by, files: [.files[] | {f: .filename, add: .additions, del: .deletions}]}'
7395
+ ```
7396
+
7397
+ Read the changelog, then diff these specifically:
7398
+
7399
+ | File | Why it matters |
7400
+ |---|---|
7401
+ | `src/discover/registry.rs` | rewrite eligibility, pipeline handling, the TOML call sites |
7402
+ | `src/discover/rules.rs` | the rule table and `IGNORED_PREFIXES` |
7403
+ | `src/discover/lexer.rs` | tokenisation and `contains_unattestable_construct` |
7404
+ | `src/core/toml_filter.rs` | the filter DSL, which affects users who write their own filters |
7405
+ | `src/hooks/trust.rs` | the trust gate and its paths |
7406
+ | `src/hooks/hook_cmd.rs` | the Claude hook contract and permission handling |
7407
+ | `src/core/telemetry.rs`, `src/core/telemetry_cmd.rs` | the compile-time endpoint gate |
7408
+ | `Cargo.toml`, `Cargo.lock` | native deps, the MSRV, and new advisories |
7409
+
7410
+ **A change to the DSL, to trust handling, or to the permission flow is a
7411
+ stop-and-think, not a rebuild.** In particular:
7412
+
7413
+ - The pack ships no filters of its own, so a DSL change cannot break us. It can
7414
+ still break a **user's** `filters.toml`, which upstream then skips silently
7415
+ (`toml_filter.rs:220-221`). Worth a release-note line, not a code change.
7416
+ - If `hook_cmd.rs` changes when `permissionDecision` is emitted, the trust
7417
+ boundary documented in the pack README changes with it.
7418
+ - If `IGNORED_PREFIXES` or the `pipeline_final_safe` rule set moves, every
7419
+ coverage number is stale.
7420
+
7421
+ ## Phase 3: Re-validate the port
7422
+
7423
+ Every coverage and saving figure the pack quotes comes from `rtk_port.py`, a
7424
+ Python model of rtk's rewrite pipeline. **Anything short of full agreement
7425
+ invalidates those numbers until the port is fixed.** Tooling lives in
7426
+ `~/rtk-measurement-archive/`.
7427
+
7428
+ ```bash
7429
+ git clone --depth 1 --branch <new-tag> https://github.com/rtk-ai/rtk.git /tmp/rtk-new
7430
+ cd ~/rtk-measurement-archive
7431
+
7432
+ # Rewrite assertions from the tag's own test block.
7433
+ python3 extract_cases.py /tmp/rtk-new/src/discover/registry.rs cases.json
7434
+ python3 validate_port.py cases.json
7435
+
7436
+ # The entry gate both real hook paths apply before rewrite_command.
7437
+ # Its assertions live in lexer.rs; re-extract if that block moved.
7438
+ python3 -c "import entry_gate; print('gate import ok')"
7439
+ ```
7440
+
7441
+ Baseline at v0.44.0: **203/203** rewrite assertions, **35/35** gate assertions.
7442
+
7443
+ If the port diverges, fix the port first, then re-measure:
7444
+
7445
+ ```bash
7446
+ python3 measure_gated.py gated 0:1224
7447
+ ```
7448
+
7449
+ Measure over the whole transcript pool, never the default 134-file window: at
7450
+ that size the projection swings 8.8x on an unchanged mechanism.
7451
+
7452
+ ### 3.1 Re-measure, do not just re-validate
7453
+
7454
+ Port agreement proves rtk still rewrites the same commands. It says nothing
7455
+ about how much each rewrite saves, and that is where the value actually sits.
7456
+
7457
+ **`rtk grep` carries 4.04 MB of the 5.66 MB measurable saving: 71% of the total
7458
+ rests on one family.** If upstream changes that one filter, the headline number
7459
+ moves even with the port at full agreement. Measured effectiveness has already
7460
+ diverged from expectation in both directions once: `rtk grep` measured 22.3%
7461
+ against 9.0% modelled, `rtk rg` 7.0% against 30% modelled.
7462
+
7463
+ So on every bump, after the port passes, replay against the newly built binary:
7464
+
7465
+ ```bash
7466
+ python3 replay_rtk.py --pool 1300 --rtk <path-to-the-new-binary>
7467
+ ```
7468
+
7469
+ Compare per family against the recorded baseline:
7470
+
7471
+ | Family | Measured at v0.44.0 | Share of total saving |
7472
+ |---|---:|---:|
7473
+ | `rtk grep` | 22.3% | 71% |
7474
+ | `rtk git` | 33.1% | 21% |
7475
+ | `rtk find` | 35.6% | 8% |
7476
+
7477
+ **Act on the result, do not just record it.** If the total lands below the
7478
+ published kill number of **0.05% of input tokens**, the pack has stopped earning
7479
+ its supply-chain surface and retiring it is the correct outcome, exactly as the
7480
+ in-house filter was retired. The margin at v0.44.0 is 0.0615%, which is 1.23x
7481
+ the kill number, so a single-family regression is enough to cross it.
7482
+
7483
+ ## Phase 4: Rebuild
7484
+
7485
+ ```bash
7486
+ gh workflow run rtk-build.yml --ref main \
7487
+ -f upstream_tag=<new-tag> -f build_revision=1 -f rust_version=<pinned> -f publish=false
7488
+ ```
7489
+
7490
+ Five targets: `x86_64-apple-darwin`, `aarch64-apple-darwin`,
7491
+ `x86_64-unknown-linux-musl`, `aarch64-unknown-linux-gnu`,
7492
+ `x86_64-pc-windows-msvc`. A target that will not build is dropped, not faked.
7493
+
7494
+ Pin `rust_version` explicitly. Upstream uses unpinned `stable` with
7495
+ `warnings = "deny"`, so a new rustc lint can turn the build red with no change
7496
+ on either side.
7497
+
7498
+ **Advisories.** The `audit` job blocks. Re-derive the disposition rather than
7499
+ carrying the previous one forward:
7500
+
7501
+ ```bash
7502
+ # Cross-reference the new lockfile against OSV without waiting for CI.
7503
+ python3 - <<'PY'
7504
+ import json, re, urllib.request, pathlib
7505
+ lock = pathlib.Path("/tmp/rtk-new/Cargo.lock").read_text()
7506
+ pkgs = [(re.search(r'^name = "([^"]+)"', b, re.M).group(1),
7507
+ re.search(r'^version = "([^"]+)"', b, re.M).group(1))
7508
+ for b in lock.split("[[package]]")[1:]
7509
+ if re.search(r'^name = ', b, re.M) and re.search(r'^version = ', b, re.M)]
7510
+ q = [{"package": {"name": n, "ecosystem": "crates.io"}, "version": v} for n, v in pkgs]
7511
+ req = urllib.request.Request("https://api.osv.dev/v1/querybatch",
7512
+ data=json.dumps({"queries": q}).encode(), headers={"Content-Type": "application/json"})
7513
+ res = json.load(urllib.request.urlopen(req, timeout=60))
7514
+ for (n, v), r in zip(pkgs, res["results"]):
7515
+ if r.get("vulns"):
7516
+ print(n, v, [x["id"] for x in r["vulns"]])
7517
+ PY
7518
+ ```
7519
+
7520
+ For each advisory decide, and record the reason in the workflow:
7521
+
7522
+ - **In-range fix** (`cargo update -p <crate>` works): add the crate to
7523
+ `RTK_CARGO_UPDATES` in `.github/workflows/rtk-build.yml`. Both the audit job
7524
+ and every build job apply it, so we audit what we ship.
7525
+ - **Needs a `Cargo.toml` change**: that is a source modification and breaks the
7526
+ NOTICE claim. Ignore with a written reason, or escalate.
7527
+ - **Carried-forward ignores**: re-check every `--ignore` still applies. The
7528
+ quick-xml pair exists only because upstream pins `"0.37"`; **delete both the
7529
+ moment upstream moves to 0.41 or later** rather than carrying them.
7530
+
7531
+ ## Phase 5: Verify
7532
+
7533
+ CI asserts this per target and the run fails on any `fail` verdict:
7534
+
7535
+ 1. `RTK_TELEMETRY_URL` and `RTK_TELEMETRY_TOKEN` unset at build time
7536
+ 2. the artifact starts and reports the expected version
7537
+ 3. no telemetry state written into a sandboxed home
7538
+ 4. on Linux, identical behaviour with no network route
7539
+ 5. every archive holds exactly one flat entry
7540
+
7541
+ Then check by hand:
7542
+
7543
+ ```bash
7544
+ gh run download <run-id> --dir /tmp/rtk-verify
7545
+ cd /tmp/rtk-verify && shasum -a 256 -c checksums.txt
7546
+ ```
7547
+
7548
+ Three things CI cannot tell you:
7549
+
7550
+ - **`strings` markers.** Compare `tls_markers_present` per target against the
7551
+ previous build. Their absence is an LLVM outcome under LTO, not a guarantee,
7552
+ so a sudden appearance means the telemetry stack survived and is worth
7553
+ understanding before shipping.
7554
+ - **Reproducibility.** Four of five targets are bit-reproducible; compare
7555
+ extracted binaries, never the tarballs, because gzip records a timestamp.
7556
+ `x86_64-pc-windows-msvc` differs by 24 bytes per link (MSVC timestamp plus a
7557
+ CodeView GUID), so a changed Windows digest proves nothing on its own.
7558
+ - **Upstream has no test asserting network silence.** That property can regress
7559
+ on any bump without turning their CI red. We own it.
7560
+
7561
+ ## Phase 6: Publish
7562
+
7563
+ ```bash
7564
+ gh workflow run rtk-build.yml --ref main \
7565
+ -f upstream_tag=<new-tag> -f build_revision=1 -f rust_version=<pinned> -f publish=true
7566
+ ```
7567
+
7568
+ Creates `softspark-rtk-<upstream>-<revision>`. Bump the revision, not the
7569
+ upstream part, when rebuilding the same upstream tag.
7570
+
7571
+ ## Phase 7: Bump the pack
7572
+
7573
+ In `app/plugins/rtk-pack/plugin.json`:
7574
+
7575
+ - `upstream.version` to the new tag
7576
+ - `binary.release_tag` to the new release
7577
+ - every `assets.*.sha256` from the published `checksums.txt`
7578
+ - `version` (the pack's own) — bump it whether or not the upstream tag moved,
7579
+ because `plugin update` skips a pack whose recorded version still matches, so
7580
+ an unbumped pack never reaches installed users. The two fields are separate so
7581
+ a pack-only fix does not pretend to be an upstream bump
7582
+
7583
+ Then:
7584
+
7585
+ ```bash
7586
+ npm test # tests/test_rtk_pack.bats asserts digest shape and layout
7587
+ python3 scripts/validate.py --strict
7588
+ python3 scripts/audit_skills.py --ci
7589
+ shellcheck --severity=warning app/hooks/*.sh app/plugins/*/hooks/*.sh
7590
+ ```
7591
+
7592
+ Verify a real install end to end, against the published release rather than a
7593
+ mirror:
7594
+
7595
+ ```bash
7596
+ H=$(mktemp -d)
7597
+ AI_TOOLKIT_DATA_DIR="$H" python3 app/plugins/rtk-pack/scripts/init.py
7598
+ AI_TOOLKIT_DATA_DIR="$H" python3 app/plugins/rtk-pack/scripts/status.py
7599
+ ```
7600
+
7601
+ ## Phase 8: Ship
7602
+
7603
+ The pack version bump reaches installed users through `ai-toolkit update`,
7604
+ which runs `plugin update --editor all --all`. A pack whose recorded version
7605
+ matches its manifest is skipped silently, so the bump in Phase 7 is what makes
7606
+ the update fire at all. Forgetting it means nobody gets the new binary.
7607
+
7608
+ ## Phase 9: Record the licence position
7609
+
7610
+ If upstream relicenses, adds a `NOTICE`, or changes `DISCLAIMER.md`, update the
7611
+ NOTICE generated in `.github/workflows/rtk-build.yml`. It currently records two
7612
+ build-time differences: the undefined telemetry endpoint, and in-range lockfile
7613
+ security updates. If Phase 4 changes that set, the NOTICE text changes with it.
7614
+
7615
+ `DISCLAIMER.md` is deliberately not redistributed because it states telemetry is
7616
+ collected by default, which our builds contradict. Re-check that this is still
7617
+ the reason before changing the decision.
7618
+
7619
+ ## Related
7620
+
7621
+ - `kb/history/completed/rtk-pack-integration-20260726.md` — decisions, measurements, pre-mortem
7622
+ - `kb/history/completed/output-filter-retirement-20260726.md` — why premise validation comes first
7623
+ - `app/plugins/rtk-pack/README.md` — the user-facing trust boundary
7624
+
7625
+ ---
7626
+
6621
7627
  ## kb/reference/agents-catalog.md
6622
7628
 
6623
7629
  ---
@@ -7746,7 +8752,7 @@ Usage: ai-toolkit <command> [options]
7746
8752
  |---------|-------------|
7747
8753
  | `plugin list` | Show available plugin packs with install status |
7748
8754
  | `plugin install <name> [--editor claude\|codex\|all]` | Install a plugin pack for Claude Code and/or Codex (`claude` means Claude Code, not the Claude app) |
7749
- | `plugin install --all [--editor claude\|codex\|all]` | Install all 11 plugin packs |
8755
+ | `plugin install --all [--editor claude\|codex\|all]` | Install all 12 plugin packs |
7750
8756
  | `plugin update <name> [--editor claude\|codex\|all]` | Update a plugin pack (remove + reinstall, preserves data) |
7751
8757
  | `plugin update --all [--editor claude\|codex\|all]` | Update all installed plugin packs |
7752
8758
  | `plugin clean <name> [--days N]` | Prune old plugin data (default: 90 days) |
@@ -12491,10 +13497,10 @@ Required keys:
12491
13497
  ## CLI Management
12492
13498
 
12493
13499
  ```bash
12494
- ai-toolkit plugin list # show all 11 packs with install status
13500
+ ai-toolkit plugin list # show all 12 packs with install status
12495
13501
  ai-toolkit plugin install --editor claude <name> # Claude Code global target
12496
13502
  ai-toolkit plugin install --editor codex <name> # Codex global target
12497
- ai-toolkit plugin install --editor all --all # install all 11 packs for both runtimes
13503
+ ai-toolkit plugin install --editor all --all # install all 12 packs for both runtimes
12498
13504
  ai-toolkit plugin update --editor all --all # update all installed packs
12499
13505
  ai-toolkit plugin clean <name> # prune data older than 90 days (default)
12500
13506
  ai-toolkit plugin clean <name> --days 30 # prune data older than 30 days
@@ -12557,6 +13563,21 @@ ai-toolkit plugin status --editor all # show installed packs with r
12557
13563
  | `kotlin-pack` | kotlin | 0 | 1 | 0 | Kotlin patterns |
12558
13564
  | `swift-pack` | swift | 0 | 1 | 0 | Swift patterns |
12559
13565
  | `ruby-pack` | ruby | 0 | 1 | 0 | Ruby patterns |
13566
+ | `rtk-pack` | token-reduction | 0 | 0 | 1 | Command rewriting via a checksum-pinned rtk binary fetched at install |
13567
+
13568
+ `rtk-pack` is the first pack to break three assumptions the others share, so it
13569
+ is the one to read when extending the contract:
13570
+
13571
+ - **It fetches from the network at install time.** `scripts/init.py` downloads a
13572
+ platform-specific artifact and verifies its SHA-256 against `plugin.json`
13573
+ before installing anything. A mismatch aborts and leaves nothing behind.
13574
+ - **It declares platform assets and digests in `plugin.json`.** The manifest
13575
+ schema tolerates extra keys, so `upstream` and `binary` are additive; nothing
13576
+ validates them, which means a malformed block fails at install rather than in
13577
+ `validate.py --strict`.
13578
+ - **It reports its own health.** `scripts/status.py` is picked up generically by
13579
+ `plugin status`, replacing what used to be a hardcoded `if name == "memory-pack"`
13580
+ branch. Any pack can now ship one.
12560
13581
 
12561
13582
  ## Optional Hook Modules
12562
13583