@astrofoundry/pi-astro 0.34.0 → 0.35.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -38,7 +38,7 @@ Wrappers ship as TypeScript and run under Node 24 type stripping: erasable synta
38
38
  - Config only through `readConfig(service, shape)`; every field required, no fallbacks, no env vars.
39
39
  - Exit codes: `UsageError` 2, `ServiceError` 1, success 0. Diagnostics on stderr, results on stdout, JSON where possible.
40
40
  - Output filters: Arcane responses pass `stripKeys(ARCANE_DENIED_KEYS)`; Zitadel responses pass `redactSecrets`. Add keys there rather than in a wrapper.
41
- - Fixed remote commands: `DMZ_COMMANDS`, `PULSAR_COMMANDS`, `VPS_COMMANDS`, `OBS_COMMANDS`, `BACKUP_COMMANDS`, `HOST_COMMANDS`, and `INFERENCE_COMMANDS` must match the entry scripts in `entry/remote/` and their tracked copies in the homelab repository (`02-pulsar-proxmox/dmz/system/`, `02-pulsar-proxmox/pulsar/system/`, `00-frontdoor-vps/system/`, `02-pulsar-proxmox/observability/system/`, `02-pulsar-proxmox/hermes/system/`, `04-nexus-macstudio/system/`, and `host/` of the `astronaute77/arcane` repository). Pulsar's root `authorized_keys` carries one forced key per specialist (network, backup, proxmox, inference), each pinned to its own entry script. Change both sides in the same commit and redeploy the remote script. A new privileged step on the VPS also needs its exact line in `spc-edge-frontdoor-sudoers`.
41
+ - Fixed remote commands: `DMZ_COMMANDS`, `PULSAR_COMMANDS`, `VPS_COMMANDS`, `OBS_COMMANDS`, `BACKUP_COMMANDS`, `HOST_COMMANDS`, and `INFERENCE_COMMANDS` must match the entry scripts in `entry/remote/` and their tracked copies in the homelab repository (`02-pulsar-proxmox/dmz/system/`, `02-pulsar-proxmox/pulsar/system/`, `00-frontdoor-vps/system/`, `02-pulsar-proxmox/observability/system/`, `04-nexus-macstudio/system/`, and `host/` of the `astronaute77/arcane` repository). Pulsar's root `authorized_keys` carries one forced key per specialist (network, backup, proxmox), each pinned to its own entry script. Change both sides in the same commit and redeploy the remote script. A new privileged step on the VPS also needs its exact line in `spc-edge-frontdoor-sudoers`.
42
42
  - Repository access is `lib/repo.ts` (`repoCommand`, `RepoSpec`, `trackedFile`): one clone per specialist under `~/.specialists/work/<checkoutName>` (`homelab-edge`, `homelab-identity`; never share a checkout between specialists), a write deploy key, and an optional `writeRoot` that limits `git write`. `edge` (`deploy-nginx`) and `identity` (`dmz deploy-config`) deploy only what origin has: `trackedFile` refuses a dirty, ahead, or behind checkout. The DMZ entry keeps the previous template and restores it when render or restart fails.
43
43
  - `network` firewall policy writes are `policyWrite`: JSON body from the agent, checked for the documented required fields, sent as `POST`, `PUT`, or `DELETE` through the same pinned request; UniFi validates the rest.
44
44
  - Pi executes sibling tool calls in parallel, so two wrapper processes of the same specialist can run at once. `lib/lock.ts` (`withLock`, a directory lock under `~/.specialists/work/locks`, stale after 10 minutes) serialises shared resources; `edge` holds `edge-tunnel` around the fixed IAP port and the gcloud configuration directory. `repo.ts` checkouts are not locked yet.
@@ -47,16 +47,16 @@ Wrappers ship as TypeScript and run under Node 24 type stripping: erasable synta
47
47
  - `security` parses Wazuh alert lines locally (`summariseWazuh`, `summariseVulns`) so the guest never needs `jq`; `wazuh-vulns` is a local view over the same `wazuh-alerts` dump, not a guest command.
48
48
  - TLS pinning for self-signed consoles is `lib/pinned.ts` (`pinnedRequest`); `network` and `proxmox` use it with a `*CertSha256` config value obtained once through the wrapper's `fingerprint` command.
49
49
  - `proxmox` writes are typed commands, never a generic POST: power actions and snapshot changes go through `runTask`, which polls `/nodes/<node>/tasks/<upid>/status` until `stopped` and fails on any `exitstatus` but `OK`. `--confirm` is stripped by `needsConfirm` for every interrupting action. `SET_KEYS` mirrors the privileges of the `SpecialistGuest` role; the API refuses anything the role lacks, the wrapper refuses earlier.
50
- - `backup` and `inference` are pure forced-command wrappers with per-command timeouts (`timeoutMs` in the tables) because runs such as `vzdump-run` or `hermes-upgrade` take many minutes; the gate's `timeoutMs` in `specialists.json` must exceed the longest of them.
51
- - `inference` reads upstream release notes from the GitHub releases API (`hermes-releases`); the skill makes reading them a precondition of `hermes-upgrade`. The Europa check is HTTP only until Windows OpenSSH forced commands are verified.
50
+ - `backup` and `inference` are pure forced-command wrappers with per-command timeouts (`timeoutMs` in the tables) because runs such as `vzdump-run` take many minutes; the gate's `timeoutMs` in `specialists.json` must exceed the longest of them.
51
+ - `inference` reaches Nexus through one forced-command key. The Europa check is HTTP only until Windows OpenSSH forced commands are verified.
52
52
  - `REFUSED_PREFIXES` protects only the wrapper itself (`config`, `auth`, `self-update`, `completion`). The GitOps-first rule for managed projects lives in the skill and the agent prompt, not in code, because unmanaged projects and emergencies need the direct CLI.
53
53
  - The wrapper's repository commands are the `git` group; `gitops` must stay free because it is an `arcane-cli` command group (server-side syncs).
54
54
  - Every passthrough call writes a private per-call copy of `arcanecli.yml` with `jwt_token` set, because some `arcane-cli` commands validate the config file before reading `ARCANE_TOKEN`. The copy is removed after the call.
55
55
  - Every wrapper answers `--help` without reading config, so agents can discover subcommands before anything else is set up.
56
- - `entry/specialist-install.sh` runs as root, so it runs only `/usr/bin`, `/bin`, `/usr/sbin`, `/sbin` and the Node it installed, never Homebrew or anything else the main user can write; it checks that `/usr/local/libexec/specialist-node` and its parents belong to root and are writable by nobody else. Arguments are checked before the root test (tests exercise that part). Downloads go to a root-only temporary directory and are verified before use: Node against `SHASUMS256.txt` and the LTS flag in `index.json`, npm tarballs against the registry's `sha512` integrity, arcane-cli against the release's checksums file. Inside `/Users/specialist` it works only as `specialist` (`sudo -u specialist -H`), from a staging directory root hands over, so a link the specialist plants cannot redirect a root write. Each component switches by rename after a check (every installed wrapper's `--help`, `--version` of the new binary) and restores the previous copy when the check fails. Older versions are refused; `rollback` swaps current and previous. Homebrew's `bw` and `jq` (provision) and `gcloud` (edge) still run as `specialist`; that is a separate issue.
56
+ - `entry/specialist-install.sh` runs as root, so it runs only `/usr/bin`, `/bin`, `/usr/sbin`, `/sbin` and the Node it installed, never Homebrew or anything else the main user can write; it checks that `/usr/local/libexec/specialist-node` (and each `specialist-<tool>` folder) and its parents belong to root and are writable by nobody else. Arguments are checked before the root test (tests exercise that part). Downloads go to a root-only temporary directory and are verified before use: Node against `SHASUMS256.txt` and the LTS flag in `index.json`, npm tarballs against the registry's `sha512` integrity, arcane-cli against the release's checksums file. Inside `/Users/specialist` it works only as `specialist` (`sudo -u specialist -H`), from a staging directory root hands over, so a link the specialist plants cannot redirect a root write. Each component switches by rename after a check (every installed wrapper's `--help`, `--version` of the new binary) and restores the previous copy when the check fails. Older versions are refused; `rollback` swaps current and previous. `python`, `gcloud`, `bw`, and `jq` follow the same rules: version folders under `/usr/local/libexec/specialist-<tool>/` behind the links `current` and `previous`, sealed `root:wheel` and not writable by others, smoke-tested as `specialist` before `current` moves; `python` refuses a switch that breaks the current gcloud. Consumers use the `current` paths: `provision.sh` for `bw` and `jq`, `edge.json` for `gcloud` and `gcloudPython`. Nothing a wrapper runs with secrets may come from Homebrew.
57
57
  - `main()` waits for stdout and stderr to flush before `process.exit`: on a pipe, exiting right after a write drops everything past the 64 KiB buffer, which cut large answers such as `projects list --all` until 2026-09-25 (`lib/flush.test.ts`).
58
58
  - `arcane update-report [--all]` is `lib/image-updates.ts` over `lib/registry.ts`: a fresh `images updates check --all`, then `projects list --all` for the stack and service of each image reference, then the registries' tag lists and manifest digests (anonymous bearer tokens from each registry's challenge; digests by HEAD, which Docker Hub does not count as a pull; lscr.io is read from ghcr.io). A floating or major-series tag is resolved to the release tag whose digest matches the running and the latest build, searching the newest tags of the three commonest release shapes; a fixed tag is compared with the newest tag of the same shape; a digest pin with the newest release. Checks older than one minute before the run count as stale and show as unknown. Private images (anonymous access refused with 401 or 403) are excluded from rows, counts, and table, listed in `privateImages`, and counted in the summary line. The table is fixed-width text for a code block, because Discord renders no Markdown tables.
59
- - `arcane host <command>` runs over the forced-command key of user `spc-arcane` on VM 100. `HOST_COMMANDS` in `lib/arcane-host.ts` must list exactly the operations of `entry/remote/spc-arcane-op.py` (a test compares them); the tracked copies live in the arcane repository's `host/` (`spc-arcane-host-entry`, `spc-arcane-op`, `spc-arcane-sudoers`, `spc-arcane-install.sh`) with the probes in `scripts/arcane_ops/`. The entry only splits words (letters, digits, space, `._:/+-`) and hands them to `sudo -n /usr/local/sbin/spc-arcane-op`, the one command its sudoers line allows; the op checks the name and every argument and runs the reviewed arcane-ops probes, which print only allowlisted fields. Manager steps (`manager-*`) are internal to `manager-upgrade`/`manager-rollback` and refused through `host`. Nothing on the host deletes data: a manager rollback restores the image and Compose file, not the data volume (R3 stays manual), `db-restore` only fills an empty database, and dumps are never pruned by the tool. `host-audit` compares the host's hashes with `host/<name>` of the `arcane-gitops` checkout after a pull.
59
+ - `arcane host <command>` runs over the forced-command key of user `spc-arcane` on VM 100. `HOST_COMMANDS` in `lib/arcane-host.ts` must list exactly the operations of `entry/remote/spc-arcane-op.py` (a test compares them); the tracked copies live in the arcane repository's `host/` (`spc-arcane-host-entry`, `spc-arcane-op`, `spc-arcane-sudoers`, `spc-arcane-install.sh`) with the probes in `host/spc-arcane-probes/`. The entry only splits words (letters, digits, space, `._:/+-`) and hands them to `sudo -n /usr/local/sbin/spc-arcane-op`, the one command its sudoers line allows; the op checks the name and every argument and runs the reviewed arcane-ops probes, which print only allowlisted fields. Manager steps (`manager-*`) are internal to `manager-upgrade`/`manager-rollback` and refused through `host`. Nothing on the host deletes data: a manager rollback restores the image and Compose file, not the data volume (R3 stays manual), `db-restore` only fills an empty database, and dumps are never pruned by the tool. `host-audit` compares the host's hashes with `host/<name>` of the `arcane-gitops` checkout after a pull.
60
60
  - `arcane manager-upgrade <version>` (`lib/arcane-manager.ts`) follows arcane/AGENTS.md "Manager upgrades": the official image's revision label must equal the release tag's commit (GitHub API), `manager-prepare` records `compose.before.yaml` and every container before the pin commit, the pin goes through the `arcane-gitops` checkout (`git` is exported from `lib/repo.ts` for this), the wait polls `gitops list` and the deployed Compose hash every 30 s for up to 30 minutes, then pull, cold backup, `up -d --no-deps --pull never`, and checks. Any failure before the restart reverts the pin (`stopped`); a failed check after it reverts the pin and runs `manager-rollback` (`rolled back`). The answer always exits 0 with `status`, so the bridge and the agent both see it.
61
61
  - `media` runs trendarr unchanged: `trendarrCli` in `media.json` is its `dist/cli.js`, started with `process.execPath` because sudo's PATH has no node. Each call gets `HOME` set to a fresh `0700` directory under `~/.specialists/work/media/` that holds only `.trendarr/config.json` (`0600`, trendarr's own schema, built from the note fields and `media.json`) and is removed in `finally`; trendarr reads no environment variables and has no config flag, and Node's `os.homedir()` follows `$HOME` on POSIX. Every call passes `--json`; `init` is refused. An add prints trendarr's per-ID `results` and exits 0 even when trendarr exits 1 for a failed ID, because the gate shows only stderr on failure; other failures carry trendarr's stderr without its `warning:` lines, which go into the JSON as `warnings`. The bridge's slash commands and Add buttons call the same wrapper through the entry point (`extensions/astro-discord/media.ts`).
62
62
  - `arcane release-notes <stack>[/<service>]` is `lib/releases.ts`: a single-image check and `resolveRef` give the running and latest versions, `HttpRegistry.labels` reads `org.opencontainers.image.source` from the latest image (index to linux/amd64 manifest to config blob), and the anonymous GitHub releases API (60 requests an hour, up to 3 pages of 100) gives the releases in between, oldest first. `condenseNotes` keeps each release under 12,000 characters by section priority; `capTotal` drops the oldest releases past 40,000 characters in total.
@@ -12,7 +12,7 @@ A specialist is a Pi subagent that is the only way to operate one area of the ho
12
12
  | `astro.security` | Wazuh, CrowdSec, rsyslog archive on VM 106 | Read-only: `status`, `attention`, `agents`, `alerts`, `decisions`, `bouncers`, `metrics`, `wazuh-alerts`, `wazuh-vulns`, `wazuh-log`, `remote-hosts`, `remote-log`, `journal` |
13
13
  | `astro.backup` | vzdump, the shared Restic repository (Pulsar host config, Arcane app backups), GCS offsite from LXC 102 | Status, journals, and manual runs of every layer; `restic-snapshots|stats|ls`, `restic-restore` into `/var/tmp/spc-restore` on Pulsar, `restore-clean` |
14
14
  | `astro.proxmox` | Proxmox VE on Pulsar | API token: `guests`, `guest`, `get <path>`, `tasks`; `start`, `shutdown|reboot|stop --confirm`; snapshots create, delete, rollback; `set` of CPU, memory, options, network. Pulsar key: `host-status`, `host-journal`, `updates`, `guest-exec <vmid> <status|journal|df|updates>` |
15
- | `astro.inference` | llama.cpp on Nexus, Hermes on LXC 101, Europa health | `nexus-status|start|stop|restart|log|disk`; `hermes-status|journal|errors|restart|api-health|version|guest-status|snapshots`, `hermes-releases`, `hermes-upgrade --confirm`; `europa-health` |
15
+ | `astro.inference` | llama.cpp on Nexus, Europa health | `nexus-status|start|stop|restart|log|disk`; `europa-health` |
16
16
  | `astro.media` | trendarr over TMDB, OMDb, and the `media` project on VM 100 (Radarr, Sonarr, SABnzbd) | `trending`, `search`, `add-movies`, `add-series` (at most 20 IMDb IDs), `sabnzbd status|pause|resume`; always trendarr's JSON plus its warnings |
17
17
 
18
18
  Call one with `/run astro.<name> -- <task>` or through the `subagent` tool (both from the bundled `astro-subagents` extension). `/run` saves the specialist's session under `~/.pi/agent/subagent-sessions/<agent>/`; `/run astro.<name> --continue -- <task>` resumes the last one started from the same directory, so the specialist keeps what it saw. Sessions older than 7 days are deleted. Specialists may call each other once (`maxSubagentDepth: 1`). They run on `openai-codex/gpt-6-astra` with `xhigh` thinking, set by `model` and `thinking` in each `agents/<name>.md`; an agent without these fields inherits the caller's model and thinking. Each tool takes `args`, an array of strings; `["--help"]` lists the subcommands.
@@ -21,7 +21,7 @@ Call one with `/run astro.<name> -- <task>` or through the `subagent` tool (both
21
21
 
22
22
  - Credentials and CLIs belong to a dedicated macOS user `specialist` on the host. Agents run as the main user and reach the wrappers only through `sudo -n -u specialist -H /usr/local/bin/specialist-cli`, the single passwordless sudo rule. General sudo asks for a password.
23
23
  - Credentials come from Vaultwarden (`homelab` / `specialists`, one Secure Note per specialist) through `specialist-cli provision`, into `0600` files. Wrappers never call `bw`.
24
- - The wrappers run on a root-owned Node in `/usr/local/libexec/specialist-node/` (latest LTS from nodejs.org), not on Homebrew's, because the main user owns all of `/opt/homebrew` and could otherwise swap the interpreter that reads every secret. `/usr/local/sbin/specialist-install` (root, one passwordless sudo rule) installs that Node, the wrappers, arcane-cli, and trendarr from downloads it checks against the publisher's checksums; it refuses older versions and keeps the previous one for `rollback`. Still open: `provision` runs Homebrew's `bw` and `jq`, and `edge` runs Homebrew's `gcloud`, both as `specialist`.
24
+ - The wrappers run on a root-owned Node in `/usr/local/libexec/specialist-node/` (latest LTS from nodejs.org), not on Homebrew's, because the main user owns all of `/opt/homebrew` and could otherwise swap the interpreter that reads every secret. `/usr/local/sbin/specialist-install` (root, one passwordless sudo rule) installs that Node, the wrappers, arcane-cli, trendarr, and the tools the wrappers run with secrets (`bw` and `jq` for `provision`, `gcloud` and its Python for `edge`) from downloads it checks against the publisher's checksums; it refuses older versions and keeps the previous one for `rollback`.
25
25
  - Arcane is reached with a Zitadel service user: a fresh token per call, exchanged for a 300 s Arcane token. Other services use API keys, a PAT, tokens, or SSH keys that the targets pin to forced commands. The VPS is reached through `gcloud compute start-iap-tunnel` with a service account that may only open IAP tunnels and read instances; its gcloud state lives in `/Users/specialist/.specialists/work/gcloud`.
26
26
  - Every call is logged in `/Users/specialist/.specialists/audit.log`.
27
27
  - The `specialist-gate` extension registers the tool only inside the matching specialist child and blocks other agents from calling the CLIs or reading `/Users/specialist`. `security-guard` carries the same blocks.
@@ -30,7 +30,7 @@ Limits: `network` writes only firewall policies on UniFi. `security` is read-onl
30
30
 
31
31
  ## Enable specialists on a host
32
32
 
33
- Host: Cortex (macOS on Apple silicon; Homebrew `bw`, `jq`, `gcloud-cli` with `python@3.14`, git; the specialists' Node and trendarr come from `specialist-install`). Steps 1 to 3 are one-time on the services; 4 to 8 are per host.
33
+ Host: Cortex (macOS on Apple silicon; Homebrew git; the specialists' Node, trendarr, `bw`, `jq`, `gcloud`, and gcloud's Python come from `specialist-install`). Steps 1 to 3 are one-time on the services; 4 to 8 are per host.
34
34
 
35
35
  1. **Vaultwarden.** Account `specialists@monadeo.com` with **Can view** on collection `specialists` in organization `homelab`. Secure Notes with Hidden fields (`_B64` fields hold `base64 -i <keyfile> | tr -d '\n'`):
36
36
  - `arcane`: `ZITADEL_KEY_JSON`, `GITOPS_DEPLOY_KEY_B64`, `ZITADEL_USER_ID` (plain), `SSH_KEY_ARCANE_HOST_B64` (user `spc-arcane` on VM 100, for `host`, `manager-*`), and optionally `MEALIE_API_KEY` (only `host mealie-api` reads it).
@@ -41,7 +41,7 @@ Host: Cortex (macOS on Apple silicon; Homebrew `bw`, `jq`, `gcloud-cli` with `py
41
41
  - `security`: `SSH_KEY_OBS_B64`.
42
42
  - `backup`: `SSH_KEY_PULSAR_B64`, `SSH_KEY_ARCANE_B64`.
43
43
  - `proxmox`: `PVE_TOKEN_ID` (plain, `spc-proxmox@pve!cortex`), `PVE_TOKEN_SECRET`, `SSH_KEY_PULSAR_B64`.
44
- - `inference`: `SSH_KEY_NEXUS_B64`, `SSH_KEY_HERMES_B64`, `SSH_KEY_PULSAR_B64`.
44
+ - `inference`: `SSH_KEY_NEXUS_B64`.
45
45
  - `media`: `TMDB_ACCESS_TOKEN` (TMDB API Read Access Token), `OMDB_API_KEY`, `RADARR_API_KEY`, `SONARR_API_KEY` (Settings, General, Security; each app has one), `SABNZBD_API_KEY` (the API key, not the NZB key, which can only add NZBs and cannot read or pause the queue).
46
46
  2. **Identities.** Zitadel (`https://auth.37pla.net/ui/console`): service user `spc-arcane` with Access Token Type JWT and a JSON key; service user `spc-identity` with a Personal Access Token and the Instance Owner role. Technitium (both resolvers): a user with Zones View/Modify/Delete, Dashboard View, DNS Client View, and an API token each. Cloudflare: an account-owned API token with `Zone:Read` and `DNS:Edit` on both zones. GCP project `frontdoor-1337`: service account `spc-edge` with `roles/iap.tunnelResourceAccessor` and `roles/compute.viewer`, JSON key. Proxmox (`ssh pulsar`): user `spc-proxmox@pve`, role `SpecialistGuest` = `VM.Audit VM.PowerMgmt VM.Snapshot VM.Snapshot.Rollback VM.Config.CPU VM.Config.Memory VM.Config.Options VM.Config.Network VM.Config.CDROM VM.GuestAgent.Audit`, ACLs `PVEAuditor` on `/` and `SpecialistGuest` on `/vms` for both the user and the token `spc-proxmox@pve!cortex` (`-privsep 1`).
47
47
  3. **Services.** Arcane (Settings, Authentication, Federated Credentials): Issuer `https://auth.37pla.net`, Audience `387825541182390275`, Subject exact = `spc-arcane` user ID, Role Admin, Global, lifetime 300. UniFi: API key (Settings, API Keys) and policies allowing Cortex `10.0.40.37` to TCP 22 on `10.0.20.10`, `10.0.10.37`, `10.0.90.10`, TCP 5380 on `10.0.40.53`, `10.0.40.54`, TCP 8006 on `10.0.10.37`, and TCP 8081 on `10.0.60.5`. GitHub: `spc-arcane-gitops` as a write deploy key on `astronaute77/arcane`, `spc-edge-homelab` and `spc-identity-homelab` as write deploy keys on `astronaute77/homelab`.
@@ -57,7 +57,10 @@ Host: Cortex (macOS on Apple silicon; Homebrew `bw`, `jq`, `gcloud-cli` with `py
57
57
  printf 'cortex ALL=(root) NOPASSWD: /usr/local/sbin/specialist-install\n' > /tmp/specialist-install.sudoers && sudo visudo -cf /tmp/specialist-install.sudoers && sudo install -m 0440 -o root -g wheel /tmp/specialist-install.sudoers /etc/sudoers.d/specialist-install && rm /tmp/specialist-install.sudoers
58
58
  sudo /usr/local/sbin/specialist-install node 24.22.0
59
59
  sudo install -m 0755 -o root -g wheel <package>/specialists/entry/specialist-cli.sh /usr/local/bin/specialist-cli
60
- brew install --cask gcloud-cli
60
+ sudo /usr/local/sbin/specialist-install jq <version>
61
+ sudo /usr/local/sbin/specialist-install bw <version>
62
+ sudo /usr/local/sbin/specialist-install python <x.y.z+YYYYMMDD>
63
+ sudo /usr/local/sbin/specialist-install gcloud <version>
61
64
  ```
62
65
  `specialist-cli` runs the wrappers with the Node that `specialist-install node` put in place, so that Node comes first. Add `cortex ALL=(specialist) NOPASSWD: ALL` as `/etc/sudoers.d/specialists-setup` (mode `0440`) for the duration of steps 5 to 7, then delete it. `specialist-cli` and `specialist-install` hold the same list of services; after an update that adds one (such as `media`), install both again with the same `sudo install` lines.
63
66
  5. **Bootstrap**, as `specialist` (`sudo -u specialist -H zsh`, `umask 077`): `~/.config/specialists/client.env` with `BW_SERVER=https://vault.37pla.net`, `BW_CLIENTID`, `BW_CLIENTSECRET`; `~/.config/specialists/master-password`. Both `0600`.
@@ -70,15 +73,14 @@ Host: Cortex (macOS on Apple silicon; Homebrew `bw`, `jq`, `gcloud-cli` with `py
70
73
  "homelabRemote": "git@github.com:astronaute77/homelab.git", "homelabBranch": "main", "repoArea": "02-pulsar-proxmox/dmz", "pomeriumConfig": "02-pulsar-proxmox/dmz/config.yaml" }
71
74
  { "unifiBaseUrl": "https://10.0.10.1/proxy/network/integration", "unifiCertSha256": "<from: specialist-cli network -- unifi fingerprint>", "pulsarHost": "10.0.10.37", "pulsarUser": "root" }
72
75
  { "technitiumPrimaryUrl": "http://10.0.40.53:5380", "technitiumSecondaryUrl": "http://10.0.40.54:5380", "technitiumCatalog": "catalog.invalid" }
73
- { "gcloud": "/opt/homebrew/bin/gcloud", "gcloudPython": "/opt/homebrew/opt/python@3.14/bin/python3.14",
76
+ { "gcloud": "/usr/local/libexec/specialist-gcloud/current/bin/gcloud", "gcloudPython": "/usr/local/libexec/specialist-python/current/bin/python3",
74
77
  "project": "frontdoor-1337", "zone": "us-east1-c", "instance": "frontdoor-1337", "serviceAccount": "spc-edge@frontdoor-1337.iam.gserviceaccount.com",
75
78
  "iapLocalPort": 22337, "sshUser": "spc-edge", "publicIp": "35.237.66.101", "probeHost": "auth.37pla.net",
76
79
  "homelabRemote": "git@github.com:astronaute77/homelab.git", "homelabBranch": "main", "repoArea": "00-frontdoor-vps", "nginxConf": "00-frontdoor-vps/nginx.conf" }
77
80
  { "obsHost": "10.0.90.10", "obsUser": "observability" }
78
81
  { "pulsarHost": "10.0.10.37", "pulsarUser": "root", "arcaneHost": "10.0.40.30", "arcaneUser": "arcane" }
79
82
  { "apiUrl": "https://10.0.10.37:8006", "certSha256": "<from: specialist-cli proxmox -- fingerprint>", "node": "pulsar", "pulsarHost": "10.0.10.37", "pulsarUser": "root" }
80
- { "nexusHost": "10.0.40.38", "nexusUser": "nexus", "hermesHost": "10.0.40.31", "hermesUser": "hermes", "pulsarHost": "10.0.10.37", "pulsarUser": "root",
81
- "europaHealthUrl": "http://10.0.60.5:8081/health", "hermesRepo": "NousResearch/hermes-agent" }
83
+ { "nexusHost": "10.0.40.38", "nexusUser": "nexus", "europaHealthUrl": "http://10.0.60.5:8081/health" }
82
84
  { "trendarrCli": "/Users/specialist/.specialists/bin/trendarr/node_modules/@astrofoundry/trendarr/dist/cli.js",
83
85
  "radarrUrl": "http://10.0.40.30:7878", "radarrQualityProfileId": 7, "radarrRootFolderPath": "/media/movies",
84
86
  "sonarrUrl": "http://10.0.40.30:8989", "sonarrQualityProfileId": 7, "sonarrRootFolderPath": "/media/tv", "sonarrMonitor": "all",
@@ -93,12 +95,17 @@ Host: Cortex (macOS on Apple silicon; Homebrew `bw`, `jq`, `gcloud-cli` with `py
93
95
  sudo /usr/local/sbin/specialist-install wrappers <pi-astro version>
94
96
  sudo /usr/local/sbin/specialist-install arcane-cli <Arcane server version>
95
97
  sudo /usr/local/sbin/specialist-install trendarr <version>
96
- sudo /usr/local/sbin/specialist-install rollback node|wrappers|arcane-cli|trendarr
98
+ sudo /usr/local/sbin/specialist-install jq|bw|gcloud <version>
99
+ sudo /usr/local/sbin/specialist-install python <x.y.z+YYYYMMDD>
100
+ sudo /usr/local/sbin/specialist-install rollback node|wrappers|arcane-cli|trendarr|python|gcloud|bw|jq
97
101
  sudo /usr/local/sbin/specialist-install status
98
102
  ```
99
103
  - `node` accepts only an LTS release listed in `https://nodejs.org/dist/index.json`, checks the tarball against `SHASUMS256.txt`, and switches `current` only after every installed wrapper answers `--help` with it; `previous` keeps the last one.
100
104
  - `wrappers` takes the pi-astro tarball from the npm registry, checks its `sha512` integrity, copies what `install.sh` copies, writes `~/.specialists/bin/VERSION`, and restores the old files if any wrapper fails `--help`. `install.sh` stays the first-time installer (it also writes the default config files).
101
105
  - `arcane-cli` checks the release tarball against `arcane_<version>_checksums.txt`; it follows the Arcane server's version.
106
+ - `python` installs a python-build-standalone `install_only_stripped` build (astral-sh), checked against the release's `SHA256SUMS`; before the switch it imports `ssl` and `sqlite3` and runs the current gcloud with the new Python.
107
+ - `gcloud` assembles the SDK from Google's component snapshot `components-v<version>.json` (core, core-nix, gcloud-deps, and their darwin-arm parts), each tarball checked against the snapshot's sha256, because Google lists the full archive's sha256 only in the table on its versioned-archives page, for the current release; it needs the installed Python and answers `gcloud version` before the switch.
108
+ - `bw` takes `bw-macos-arm64-<version>.zip` from the bitwarden/clients release `cli-v<version>`, checked against the asset digest from the GitHub API; `jq` takes `jq-macos-arm64` from the jqlang/jq release, checked against its `sha256sum.txt`.
102
109
  - `trendarr` installs with the root-owned Node's npm, without install scripts, into `trendarr-<version>`.
103
110
  - A version older than the installed one is refused; `rollback` swaps back to the kept previous copy. `status` prints the installed versions as JSON.
104
111
  7. **SSH targets.** Each target gets an entry script from `entry/remote/` as `/usr/local/sbin/<entry>` (`0755`, root) and the public key in the right `authorized_keys` as
@@ -107,11 +114,10 @@ Host: Cortex (macOS on Apple silicon; Homebrew `bw`, `jq`, `gcloud-cli` with `py
107
114
  - Pulsar, root: `spc-network-pulsar-entry.sh`.
108
115
  - VM 106, user `observability`: `spc-security-obs-entry.sh` (tracked in the homelab repository, installed by `02-pulsar-proxmox/observability/deploy.sh`).
109
116
  - Frontdoor VPS: `spc-edge-frontdoor-install.sh` creates user `spc-edge` (group `systemd-journal`), installs `spc-edge-frontdoor-entry.sh` and the sudoers file `spc-edge-frontdoor-sudoers`, and writes `authorized_keys`. Copy the three files and the public key to `/tmp` with `gcloud compute scp --tunnel-through-iap`, then run it as root.
110
- - Pulsar, root, three more keys: `spc-backup-pulsar-entry.sh`, `spc-proxmox-pulsar-entry.sh`, `spc-inference-pulsar-entry.sh` (tracked in `02-pulsar-proxmox/pulsar/system/`).
117
+ - Pulsar, root, two more keys: `spc-backup-pulsar-entry.sh`, `spc-proxmox-pulsar-entry.sh` (tracked in `02-pulsar-proxmox/pulsar/system/`).
111
118
  - VM 100, user `arcane`: `spc-backup-arcane-entry.sh` (tracked as `host/spc-backup-arcane-entry` in the `astronaute77/arcane` repository, next to the host backup scripts).
112
- - VM 100, user `spc-arcane` (no docker group): `spc-arcane-host-entry.sh` hands each command to the root-owned `spc-arcane-op.py` (installed as `/usr/local/sbin/spc-arcane-op`), the only command `spc-arcane-sudoers` allows; it runs the reviewed arcane-ops probes from `/usr/local/lib/spc-arcane` and the fixed manager and database steps. `spc-arcane-install.sh` creates the user and installs all of it from `/tmp/spc-arcane`; the files are tracked under `host/` in the arcane repository, the probes under `scripts/arcane_ops/`. Add `"hostAddress": "10.0.40.30", "hostUser": "spc-arcane"` to `arcane.json`; other arcane commands do not read them.
119
+ - VM 100, user `spc-arcane` (no docker group): `spc-arcane-host-entry.sh` hands each command to the root-owned `spc-arcane-op.py` (installed as `/usr/local/sbin/spc-arcane-op`), the only command `spc-arcane-sudoers` allows; it runs the reviewed probes from `/usr/local/lib/spc-arcane` and the fixed manager and database steps. `spc-arcane-install.sh` creates the user and installs all of it from `/tmp/spc-arcane`; the files are tracked under `host/` in the arcane repository, the probes under `host/spc-arcane-probes/` (moved from the arcane-ops CLI, retired on 2026-10-19). Add `"hostAddress": "10.0.40.30", "hostUser": "spc-arcane"` to `arcane.json`; other arcane commands do not read them.
113
120
  - Nexus, user `nexus`: `spc-inference-nexus-entry.sh` as `/usr/local/sbin/spc-inference-nexus-entry` (tracked in `04-nexus-macstudio/system/`).
114
- - LXC 101, user `hermes`: `spc-inference-hermes-entry.sh` (tracked in `02-pulsar-proxmox/hermes/system/`).
115
121
  8. **Pi**, as the main user: `~/.pi/agent/specialists.json` (below) and the specialist blocks in `~/.pi/agent/security-guard.json` (`specialist-cli`, `arcane-cli`, `-u specialist` in `operations`; `/Users/specialist/**` in `reads` and `writes`; keep the default rules next to them).
116
122
  ```json
117
123
  { "entry": "/usr/local/bin/specialist-cli", "user": "specialist",
@@ -144,6 +150,6 @@ The `astro-discord` extension gives Discord users access to the specialists from
144
150
 
145
151
  ## Rotation and operations
146
152
 
147
- Rotation steps for every field live in the homelab repository, `03-cortex-macmini/ROTATION.md`. After any change in Vaultwarden run `specialist-cli provision`. Host state and the sudo rules are documented in `03-cortex-macmini/README.md`; the forced-command scripts are tracked under `02-pulsar-proxmox/dmz/system/`, `02-pulsar-proxmox/pulsar/system/`, `02-pulsar-proxmox/observability/system/`, `02-pulsar-proxmox/hermes/system/`, `host/` of the `astronaute77/arcane` repository, `04-nexus-macstudio/system/`, and `00-frontdoor-vps/system/`.
153
+ Rotation steps for every field live in the homelab repository, `03-cortex-macmini/ROTATION.md`. After any change in Vaultwarden run `specialist-cli provision`. Host state and the sudo rules are documented in `03-cortex-macmini/README.md`; the forced-command scripts are tracked under `02-pulsar-proxmox/dmz/system/`, `02-pulsar-proxmox/pulsar/system/`, `02-pulsar-proxmox/observability/system/`, `host/` of the `astronaute77/arcane` repository, `04-nexus-macstudio/system/`, and `00-frontdoor-vps/system/`.
148
154
 
149
155
  All nine planned specialists exist. See `AGENTS.md` for how to add one.
@@ -15,7 +15,7 @@ const SERVICE = "edge";
15
15
 
16
16
  interface EdgeConfig extends Record<string, string | number> {
17
17
  gcloud: string;
18
- /** Python 3.10+ interpreter for gcloud; the Homebrew cask installs none of its own. */
18
+ /** Python 3.10+ interpreter for gcloud: the installer's root-owned Python, never one the main user can replace. */
19
19
  gcloudPython: string;
20
20
  project: string;
21
21
  zone: string;
@@ -78,17 +78,17 @@ const PACKAGE = /^[a-z0-9][a-z0-9+.-]{0,63}$/;
78
78
  * public DNS the way users reach them: status code only, no redirect
79
79
  * followed, no credentials. `expect` holds the codes a healthy route returns
80
80
  * (the Supabase settings endpoint wants an API key, so 401; Studio redirects
81
- * to the Pomerium sign-in; the rest answer 200, as arcane-ops judged them).
81
+ * to the Pomerium sign-in; the rest answer 200).
82
82
  */
83
83
  export const PUBLIC_ROUTES: Readonly<Record<string, { url: string; expect: readonly number[] }>> = {
84
84
  "supabase-auth-settings": { url: "https://supabase.37pla.net/auth/v1/settings", expect: [401] },
85
85
  "supabase-jwks": { url: "https://supabase.37pla.net/auth/v1/.well-known/jwks.json", expect: [200] },
86
86
  "supabase-openid": { url: "https://supabase.37pla.net/auth/v1/.well-known/openid-configuration", expect: [200] },
87
- "supabase-oauth-discovery": { url: "https://supabase.37pla.net/auth/v1/.well-known/oauth-authorization-server", expect: [200] },
87
+ "supabase-oauth-discovery": { url: "https://supabase.37pla.net/auth/v1/.well-known/oauth-authorization-server", expect: [404] },
88
88
  "supabase-studio": { url: "https://supabase-studio.37pla.net/", expect: [301, 302, 303, 307, 308] },
89
89
  "grimoire-public": { url: "https://cli.grimoire.monadeo.com/v1/auth/config", expect: [200] },
90
90
  "grimoire-public-health": { url: "https://cli.grimoire.monadeo.com/healthz", expect: [200] },
91
- "mealie-public": { url: "https://mealie.37pla.net/api/recipes?page=1&perPage=1", expect: [200] },
91
+ "mealie-public": { url: "https://mealie.37pla.net/api/recipes?page=1&perPage=1", expect: [301, 302, 303, 307, 308] },
92
92
  };
93
93
 
94
94
  const HELP = `edge specialist (Frontdoor VPS through an IAP tunnel)
@@ -1,8 +1,8 @@
1
1
  #!/bin/sh
2
2
  # One-time setup of the spc-arcane user on VM 100 for the arcane specialist.
3
3
  # Run as root with these files in /tmp/spc-arcane: spc-arcane-host-entry,
4
- # spc-arcane-op, spc-arcane-sudoers, spc-arcane-host.pub, and the arcane-ops
5
- # probes host_probe.py, resource_probe.py, incident_probe.py, kernel_probe.py,
4
+ # spc-arcane-op, spc-arcane-sudoers, spc-arcane-host.pub, and the
5
+ # probes from host/spc-arcane-probes/: host_probe.py, resource_probe.py, incident_probe.py, kernel_probe.py,
6
6
  # storage_probe.py, queue_probe.py, mealie_probe.py, radarr_probe.py.
7
7
  # Idempotent; reinstalling updates the files and keeps the user.
8
8
  set -eu
@@ -68,6 +68,9 @@ HOST_FILES = (
68
68
  ("spc-arcane-host-entry", "/usr/local/sbin/spc-arcane-host-entry"),
69
69
  ("spc-arcane-op", "/usr/local/sbin/spc-arcane-op"),
70
70
  ("spc-arcane-sudoers", "/etc/sudoers.d/spc-arcane"),
71
+ ) + tuple(
72
+ (f"spc-arcane-probes/{name}.py", f"{PROBES}/{name}.py")
73
+ for name in ("host_probe", "resource_probe", "incident_probe", "kernel_probe", "storage_probe", "queue_probe", "mealie_probe", "radarr_probe")
71
74
  )
72
75
  UNITS = tuple(name for name, _ in HOST_FILES if name.endswith((".service", ".timer")))
73
76
  LOOPBACK_CHECKS = {
@@ -1,14 +1,16 @@
1
1
  #!/bin/sh
2
2
  # /usr/local/sbin/specialist-install: installs the specialist user's Node, wrappers,
3
- # arcane-cli, and trendarr from downloads it checks itself. Installed root-owned 0755.
3
+ # arcane-cli, trendarr, Python, gcloud, bw, and jq from downloads it checks itself.
4
+ # Installed root-owned 0755.
4
5
  # Runs as root through
5
6
  # cortex ALL=(root) NOPASSWD: /usr/local/sbin/specialist-install
6
7
  # It runs only system binaries and the Node it installed, never anything the cortex
7
8
  # user can write (all of Homebrew), and it touches the specialist's home only as that
8
9
  # user, so links the specialist plants there cannot redirect a root write.
9
10
  #
10
- # specialist-install node|wrappers|arcane-cli|trendarr <version>
11
- # specialist-install rollback node|wrappers|arcane-cli|trendarr
11
+ # specialist-install node|wrappers|arcane-cli|trendarr|gcloud|bw|jq <version>
12
+ # specialist-install python <x.y.z+YYYYMMDD>
13
+ # specialist-install rollback node|wrappers|arcane-cli|trendarr|python|gcloud|bw|jq
12
14
  # specialist-install status
13
15
  set -eu
14
16
  PATH=/usr/bin:/bin:/usr/sbin:/sbin
@@ -18,11 +20,18 @@ cd /
18
20
 
19
21
  SPECIALIST=specialist
20
22
  NODE_BASE=/usr/local/libexec/specialist-node
23
+ PYTHON_BASE=/usr/local/libexec/specialist-python
24
+ GCLOUD_BASE=/usr/local/libexec/specialist-gcloud
25
+ BW_BASE=/usr/local/libexec/specialist-bw
26
+ JQ_BASE=/usr/local/libexec/specialist-jq
27
+ # Components of the official darwin-arm gcloud archive that gcloud itself needs (bq,
28
+ # gsutil, and the empty macOS bundled-python component are left out).
29
+ GCLOUD_COMPONENTS="core core-nix gcloud-deps gcloud-deps-darwin-arm gcloud-crc32c-darwin-arm"
21
30
  # Same list as the allowlist in specialist-cli; a new specialist needs both reinstalled.
22
31
  SERVICES="arcane identity network dns edge security backup proxmox inference media"
23
32
 
24
33
  usage() {
25
- echo "usage: specialist-install node|wrappers|arcane-cli|trendarr <version> | rollback node|wrappers|arcane-cli|trendarr | status" >&2
34
+ echo "usage: specialist-install node|wrappers|arcane-cli|trendarr|gcloud|bw|jq <version> | python <x.y.z+YYYYMMDD> | rollback node|wrappers|arcane-cli|trendarr|python|gcloud|bw|jq | status" >&2
26
35
  exit 2
27
36
  }
28
37
 
@@ -50,17 +59,34 @@ version_lt() {
50
59
  [ "$3" -lt "$6" ]
51
60
  }
52
61
 
62
+ # A python-build-standalone release: CPython x.y.z and the build date that names the release.
63
+ valid_python_version() {
64
+ case "$1" in '' | *[!0-9.+]*) return 1 ;; esac
65
+ printf '%s\n' "$1" | /usr/bin/grep -Eq '^(0|[1-9][0-9]{0,5})\.(0|[1-9][0-9]{0,5})\.(0|[1-9][0-9]{0,5})\+[0-9]{8}$'
66
+ }
67
+
68
+ # True when python release $1 is older than $2: CPython version first, then build date.
69
+ python_lt() {
70
+ version_lt "${1%+*}" "${2%+*}" && return 0
71
+ version_lt "${2%+*}" "${1%+*}" && return 1
72
+ [ "${1#*+}" -lt "${2#*+}" ]
73
+ }
74
+
53
75
  component_ok() {
54
- case "$1" in node | wrappers | arcane-cli | trendarr) return 0 ;; esac
76
+ case "$1" in node | wrappers | arcane-cli | trendarr | python | gcloud | bw | jq) return 0 ;; esac
55
77
  return 1
56
78
  }
57
79
 
58
80
  command=${1:-}
59
81
  case "$command" in
60
- node | wrappers | arcane-cli | trendarr)
82
+ node | wrappers | arcane-cli | trendarr | gcloud | bw | jq)
61
83
  [ $# -eq 2 ] || usage
62
84
  valid_version "$2" || usage
63
85
  ;;
86
+ python)
87
+ [ $# -eq 2 ] || usage
88
+ valid_python_version "$2" || usage
89
+ ;;
64
90
  rollback)
65
91
  [ $# -eq 2 ] || usage
66
92
  component_ok "$2" || usage
@@ -230,6 +256,217 @@ rollback_node() {
230
256
  echo "node $previous is current; previous $current"
231
257
  }
232
258
 
259
+ # Root-owned tools for the specialist: each version in its own folder under
260
+ # /usr/local/libexec/specialist-<tool>, behind the links current and previous.
261
+ check_tool_base() {
262
+ check_root_dir /
263
+ check_root_dir /usr
264
+ check_root_dir /usr/local
265
+ ensure_root_dir /usr/local/libexec
266
+ ensure_root_dir "$1"
267
+ }
268
+
269
+ # $1 base, $2 link name, $3 version check, $4 executable inside a version folder.
270
+ tool_link() {
271
+ target=$(/usr/bin/readlink "$1/$2" 2>/dev/null) || return 0
272
+ "$3" "$target" && [ -x "$1/$target/$4" ] && printf '%s' "$target"
273
+ return 0
274
+ }
275
+
276
+ switch_tool_link() {
277
+ /bin/rm -f "$1/.$2.tmp"
278
+ (umask 022 && /bin/ln -s "$3" "$1/.$2.tmp")
279
+ /bin/mv -h "$1/.$2.tmp" "$1/$2"
280
+ }
281
+
282
+ new_tool_folder() {
283
+ /bin/rm -rf "$1.partial"
284
+ /bin/mkdir -m 0755 "$1.partial"
285
+ }
286
+
287
+ seal_tool_folder() {
288
+ /usr/sbin/chown -R root:wheel "$1.partial"
289
+ /bin/chmod -R u=rwX,go=rX "$1.partial"
290
+ }
291
+
292
+ # Moves the checked $base/$version.partial into place, makes it current, keeps the old current as previous.
293
+ activate_tool() {
294
+ /bin/rm -rf "$1/$2"
295
+ /bin/mv "$1/$2.partial" "$1/$2"
296
+ [ -z "$3" ] || switch_tool_link "$1" previous "$3"
297
+ switch_tool_link "$1" current "$2"
298
+ for d in "$1"/*; do
299
+ n=${d##*/}
300
+ case "$n" in current | previous | "$2" | "$3") continue ;; esac
301
+ /bin/rm -rf "$d"
302
+ done
303
+ }
304
+
305
+ # $1 base, $2 label, $3 version check, $4 executable, $5 smoke function taking the executable's path.
306
+ rollback_tool() {
307
+ check_tool_base "$1"
308
+ current=$(tool_link "$1" current "$3" "$4")
309
+ previous=$(tool_link "$1" previous "$3" "$4")
310
+ [ -n "$current" ] && [ -n "$previous" ] || fail "no previous $2 to roll back to"
311
+ "$5" "$1/$previous/$4" "$previous" || fail "$2 $previous fails its check"
312
+ switch_tool_link "$1" current "$previous"
313
+ switch_tool_link "$1" previous "$current"
314
+ echo "$2 $previous is current; previous $current"
315
+ }
316
+
317
+ # Python from python-build-standalone: gcloud needs Python 3.10 to 3.15, and its macOS
318
+ # installer offers only Homebrew's (owned by cortex) or an unchecked python.org package.
319
+ SMOKE_PYTHON='"$1" -c "import ssl, sqlite3, sys; print(\"%d.%d.%d\" % sys.version_info[:3])"'
320
+ SMOKE_GCLOUD='cfg=$(/usr/bin/mktemp -d); trap "/bin/rm -rf \"\$cfg\"" EXIT
321
+ CLOUDSDK_PYTHON=$2 CLOUDSDK_CONFIG=$cfg CLOUDSDK_CORE_DISABLE_PROMPTS=1 "$1" version 2>/dev/null | /usr/bin/head -n 1'
322
+ SMOKE_BW='app=$(/usr/bin/mktemp -d); trap "/bin/rm -rf \"\$app\"" EXIT
323
+ BITWARDENCLI_APPDATA_DIR=$app "$1" --version 2>/dev/null'
324
+ SMOKE_JQ='"$1" --version'
325
+
326
+ smoke_python() {
327
+ [ "$(as_specialist "$SMOKE_PYTHON" "$1")" = "${2%+*}" ]
328
+ }
329
+
330
+ current_python() {
331
+ python=$(tool_link "$PYTHON_BASE" current valid_python_version bin/python3)
332
+ [ -n "$python" ] || fail "no specialist Python yet: run specialist-install python <x.y.z+YYYYMMDD> first"
333
+ printf '%s' "$PYTHON_BASE/$python/bin/python3"
334
+ }
335
+
336
+ # gcloud $1 (bin/gcloud) answers with version $2, run by Python $3.
337
+ gcloud_answers() {
338
+ [ "$(as_specialist "$SMOKE_GCLOUD" "$1" "$3")" = "Google Cloud SDK $2" ]
339
+ }
340
+
341
+ smoke_gcloud() {
342
+ gcloud_answers "$1" "$2" "$(current_python)"
343
+ }
344
+
345
+ smoke_bw() {
346
+ [ "$(as_specialist "$SMOKE_BW" "$1")" = "$2" ]
347
+ }
348
+
349
+ smoke_jq() {
350
+ [ "$(as_specialist "$SMOKE_JQ" "$1")" = "jq-$2" ]
351
+ }
352
+
353
+ install_python() {
354
+ version=$1
355
+ check_tool_base "$PYTHON_BASE"
356
+ current=$(tool_link "$PYTHON_BASE" current valid_python_version bin/python3)
357
+ if [ -n "$current" ] && python_lt "$version" "$current"; then fail "python $version is older than the installed $current; use rollback"; fi
358
+ [ "$current" != "$version" ] || { echo "python $version is already current"; return 0; }
359
+ tag=${version#*+}
360
+ name="cpython-$version-aarch64-apple-darwin-install_only_stripped.tar.gz"
361
+ base="https://github.com/astral-sh/python-build-standalone/releases/download/$tag"
362
+ fetch "$base/$name" "$work/$name"
363
+ fetch "$base/SHA256SUMS" "$work/SHA256SUMS"
364
+ check_sha256 "$work" "$name" "$work/SHA256SUMS"
365
+ dest="$PYTHON_BASE/$version"
366
+ new_tool_folder "$dest"
367
+ /usr/bin/tar -xzf "$work/$name" -C "$dest.partial" --strip-components 1
368
+ seal_tool_folder "$dest"
369
+ smoke_python "$dest.partial/bin/python3" "$version" || fail "python $version does not run"
370
+ gcloud=$(tool_link "$GCLOUD_BASE" current valid_version bin/gcloud)
371
+ if [ -n "$gcloud" ]; then
372
+ gcloud_answers "$GCLOUD_BASE/$gcloud/bin/gcloud" "$gcloud" "$dest.partial/bin/python3" || fail "gcloud $gcloud fails with python $version; current stays ${current:-unset}"
373
+ fi
374
+ activate_tool "$PYTHON_BASE" "$version" "$current"
375
+ echo "python $version is current${current:+; previous $current}"
376
+ }
377
+
378
+ # gcloud from the components of Google's release snapshot, each archive checked against
379
+ # the sha256 the snapshot lists; the snapshot and the archives come from dl.google.com.
380
+ install_gcloud() {
381
+ version=$1
382
+ python=$(current_python)
383
+ check_tool_base "$GCLOUD_BASE"
384
+ current=$(tool_link "$GCLOUD_BASE" current valid_version bin/gcloud)
385
+ if [ -n "$current" ] && version_lt "$version" "$current"; then fail "gcloud $version is older than the installed $current; use rollback"; fi
386
+ [ "$current" != "$version" ] || { echo "gcloud $version is already current"; return 0; }
387
+ channel=https://dl.google.com/dl/cloudsdk/channels/rapid
388
+ snapshot="$work/components.json"
389
+ fetch "$channel/components-v$version.json" "$snapshot"
390
+ [ "$(json_field "$snapshot" version)" = "$version" ] || fail "gcloud $version has no release snapshot"
391
+ dest="$GCLOUD_BASE/$version"
392
+ new_tool_folder "$dest"
393
+ for want in $GCLOUD_COMPONENTS; do
394
+ i=0
395
+ found=""
396
+ while [ "$i" -lt 1000 ]; do
397
+ id=$(json_field "$snapshot" "components.$i.id") || break
398
+ if [ "$id" = "$want" ]; then found=$i; break; fi
399
+ i=$((i + 1))
400
+ done
401
+ [ -n "$found" ] || fail "gcloud $version has no component $want"
402
+ source=$(json_field "$snapshot" "components.$found.data.source") || fail "component $want has no archive"
403
+ sum=$(json_field "$snapshot" "components.$found.data.checksum") || fail "component $want has no checksum"
404
+ printf '%s\n' "$source" | /usr/bin/grep -Eq '^components/google-cloud-sdk-[a-z0-9-]+-[0-9]{14}\.tar\.gz$' || fail "unexpected archive for component $want"
405
+ printf '%s\n' "$sum" | /usr/bin/grep -Eq '^[0-9a-f]{64}$' || fail "unexpected checksum for component $want"
406
+ fetch "$channel/$source" "$work/$want.tgz"
407
+ [ "$(/usr/bin/shasum -a 256 "$work/$want.tgz" | /usr/bin/awk '{ print $1 }')" = "$sum" ] || fail "checksum mismatch for component $want"
408
+ /usr/bin/tar -xzf "$work/$want.tgz" -C "$dest.partial"
409
+ done
410
+ seal_tool_folder "$dest"
411
+ gcloud_answers "$dest.partial/bin/gcloud" "$version" "$python" || fail "gcloud $version does not run"
412
+ activate_tool "$GCLOUD_BASE" "$version" "$current"
413
+ echo "gcloud $version is current${current:+; previous $current}"
414
+ }
415
+
416
+ # Bitwarden CLI: the native macOS arm64 build, checked against the sha256 digest GitHub publishes for the release asset.
417
+ install_bw() {
418
+ version=$1
419
+ check_tool_base "$BW_BASE"
420
+ current=$(tool_link "$BW_BASE" current valid_version bw)
421
+ if [ -n "$current" ] && version_lt "$version" "$current"; then fail "bw $version is older than the installed $current; use rollback"; fi
422
+ [ "$current" != "$version" ] || { echo "bw $version is already current"; return 0; }
423
+ name="bw-macos-arm64-$version.zip"
424
+ release="$work/bw-release.json"
425
+ fetch "https://api.github.com/repos/bitwarden/clients/releases/tags/cli-v$version" "$release"
426
+ i=0
427
+ found=""
428
+ while [ "$i" -lt 100 ]; do
429
+ asset=$(json_field "$release" "assets.$i.name") || break
430
+ if [ "$asset" = "$name" ]; then found=$i; break; fi
431
+ i=$((i + 1))
432
+ done
433
+ [ -n "$found" ] || fail "bw $version has no $name"
434
+ digest=$(json_field "$release" "assets.$found.digest") || fail "$name has no published digest"
435
+ printf '%s\n' "$digest" | /usr/bin/grep -Eq '^sha256:[0-9a-f]{64}$' || fail "unexpected digest for $name"
436
+ url="https://github.com/bitwarden/clients/releases/download/cli-v$version/$name"
437
+ [ "$(json_field "$release" "assets.$found.browser_download_url")" = "$url" ] || fail "unexpected download location for $name"
438
+ fetch "$url" "$work/$name"
439
+ [ "sha256:$(/usr/bin/shasum -a 256 "$work/$name" | /usr/bin/awk '{ print $1 }')" = "$digest" ] || fail "checksum mismatch for $name"
440
+ dest="$BW_BASE/$version"
441
+ new_tool_folder "$dest"
442
+ /usr/bin/unzip -q "$work/$name" bw -d "$dest.partial" || fail "$name has no bw"
443
+ /bin/chmod 0755 "$dest.partial/bw"
444
+ seal_tool_folder "$dest"
445
+ smoke_bw "$dest.partial/bw" "$version" || fail "bw $version does not run"
446
+ activate_tool "$BW_BASE" "$version" "$current"
447
+ echo "bw $version is current${current:+; previous $current}"
448
+ }
449
+
450
+ # jq: the macOS arm64 binary, checked against the release's sha256sum.txt.
451
+ install_jq() {
452
+ version=$1
453
+ check_tool_base "$JQ_BASE"
454
+ current=$(tool_link "$JQ_BASE" current valid_version jq)
455
+ if [ -n "$current" ] && version_lt "$version" "$current"; then fail "jq $version is older than the installed $current; use rollback"; fi
456
+ [ "$current" != "$version" ] || { echo "jq $version is already current"; return 0; }
457
+ base="https://github.com/jqlang/jq/releases/download/jq-$version"
458
+ fetch "$base/jq-macos-arm64" "$work/jq-macos-arm64"
459
+ fetch "$base/sha256sum.txt" "$work/sha256sum.txt"
460
+ check_sha256 "$work" jq-macos-arm64 "$work/sha256sum.txt"
461
+ dest="$JQ_BASE/$version"
462
+ new_tool_folder "$dest"
463
+ /usr/bin/install -m 0755 "$work/jq-macos-arm64" "$dest.partial/jq"
464
+ seal_tool_folder "$dest"
465
+ smoke_jq "$dest.partial/jq" "$version" || fail "jq $version does not run"
466
+ activate_tool "$JQ_BASE" "$version" "$current"
467
+ echo "jq $version is current${current:+; previous $current}"
468
+ }
469
+
233
470
  WRAPPER_VERSION='v=$(cat "$HOME/.specialists/bin/VERSION" 2>/dev/null) || exit 0; printf "%s" "$v"'
234
471
 
235
472
  # Moves the new wrapper files in and the old ones aside, two renames per item, then
@@ -433,13 +670,26 @@ json_version() {
433
670
  if valid_version "$1"; then printf '"%s"' "$1"; else printf 'null'; fi
434
671
  }
435
672
 
673
+ json_python() {
674
+ if valid_python_version "$1"; then printf '"%s"' "$1"; else printf 'null'; fi
675
+ }
676
+
677
+ # {"current":..,"previous":..} of one root-owned tool: $1 base, $2 version check, $3 executable, $4 JSON writer.
678
+ json_tool() {
679
+ printf '{"current":%s,"previous":%s}' "$("$4" "$(tool_link "$1" current "$2" "$3")")" "$("$4" "$(tool_link "$1" previous "$2" "$3")")"
680
+ }
681
+
436
682
  status() {
437
683
  wrappers=$(as_specialist "$WRAPPER_VERSION" || true)
438
684
  arcane=$(as_specialist "$ARCANE_VERSION" || true)
439
685
  trendarr=$(as_specialist "$TRENDARR_VERSION" || true)
440
- printf '{"node":{"current":%s,"previous":%s},"wrappers":%s,"arcaneCli":%s,"trendarr":%s}\n' \
686
+ printf '{"node":{"current":%s,"previous":%s},"wrappers":%s,"arcaneCli":%s,"trendarr":%s,"python":%s,"gcloud":%s,"bw":%s,"jq":%s}\n' \
441
687
  "$(json_version "$(node_link current)")" "$(json_version "$(node_link previous)")" \
442
- "$(json_version "$wrappers")" "$(json_version "$arcane")" "$(json_version "$trendarr")"
688
+ "$(json_version "$wrappers")" "$(json_version "$arcane")" "$(json_version "$trendarr")" \
689
+ "$(json_tool "$PYTHON_BASE" valid_python_version bin/python3 json_python)" \
690
+ "$(json_tool "$GCLOUD_BASE" valid_version bin/gcloud json_version)" \
691
+ "$(json_tool "$BW_BASE" valid_version bw json_version)" \
692
+ "$(json_tool "$JQ_BASE" valid_version jq json_version)"
443
693
  }
444
694
 
445
695
  case "$command" in
@@ -447,12 +697,20 @@ case "$command" in
447
697
  wrappers) install_wrappers "$2" ;;
448
698
  arcane-cli) install_arcane_cli "$2" ;;
449
699
  trendarr) install_trendarr "$2" ;;
700
+ python) install_python "$2" ;;
701
+ gcloud) install_gcloud "$2" ;;
702
+ bw) install_bw "$2" ;;
703
+ jq) install_jq "$2" ;;
450
704
  rollback)
451
705
  case "$2" in
452
706
  node) rollback_node ;;
453
707
  wrappers) rollback_wrappers ;;
454
708
  arcane-cli) rollback_arcane_cli ;;
455
709
  trendarr) rollback_trendarr ;;
710
+ python) rollback_tool "$PYTHON_BASE" python valid_python_version bin/python3 smoke_python ;;
711
+ gcloud) rollback_tool "$GCLOUD_BASE" gcloud valid_version bin/gcloud smoke_gcloud ;;
712
+ bw) rollback_tool "$BW_BASE" bw valid_version bw smoke_bw ;;
713
+ jq) rollback_tool "$JQ_BASE" jq valid_version jq smoke_jq ;;
456
714
  esac
457
715
  ;;
458
716
  status) status ;;