@futurelastic/colab-handbook 1.16.0-rc.15 → 1.16.0-rc.17

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json CHANGED
@@ -33,5 +33,5 @@
33
33
  "engines": {
34
34
  "node": ">=18"
35
35
  },
36
- "version": "1.16.0-rc.15"
36
+ "version": "1.16.0-rc.17"
37
37
  }
@@ -86,6 +86,7 @@ header naming its new home ([`docs/compat-surfaces.md`](../docs/compat-surfaces.
86
86
  | `ci-node.yml`, `ci-python.yml`, `ci-laravel.yml` (with the #625 `Migration round-trip` job), `ci-*-heavy.yml` | your forge's CI; `colab ship` / `promote` read or dispatch the runs; `colab template --roundtrip` writes the round-trip job | the adopter's CI layer; until it ships (#608), an optional starting file here |
87
87
  | `release-tag.yml`, `release-auto.yml` | your forge's CI; `release-auto.yml` runs `colab release cut` / `finalize --auto` | the release commands' new home (#608); an optional starting file here until then |
88
88
  | `dist-refs.yml`, `npx-launcher.mjs`, `npx-service.mjs` | your own release workflow and package `bin` | stays here, as an optional starting file |
89
+ | `publish-downloads.yml`, `publish-downloads.mjs` | your own release workflow, as its last job | stays here, as an optional starting file |
89
90
  | `deploy-xserver.yml`, `deploy-container.yml`, `deploy-container-run.mjs`, `deploy-adapter-portainer.mjs` | your repo's deploy workflow | the adopter's deploy tooling; here as an optional starting file |
90
91
  | `pre-push-guard` | git, honouring `COLAB_SHIP=1` from `colab ship` | the tool that lands branches (#608); an optional starting file here until then |
91
92
  | `pre-commit-dispatch`, `pre-commit-identity`, `identity-vocabulary.example` | git (this repo's own `.githooks/` run them in place) | stays here, as an optional starting file |
@@ -109,6 +110,8 @@ check that reads one of these files is that tool's behaviour, documented with th
109
110
  | `dist-refs.yml` | `.github/workflows/dist-refs.yml` | A **private** repo shipping a **compiled** tool, installed as `npx github:<org>/<repo>#vX.Y.Z` ([CONVENTIONS §6, *Distribution*](../CONVENTIONS.md#distribution--one-install-surface-npx-442)) | `workflow_call`, called from the same run that cut the release tag (a tag pushed with `GITHUB_TOKEN` triggers nothing else). Takes one build artifact per `<os>-<arch>` and pushes each as an orphan commit at `refs/tags/dist/vX.Y.Z/<os>-<arch>` with a `SHA256SUMS`. Refuses a public or unreadable repository (a public compiled tool ships npm platform packages instead), a version that is not an existing tag, and a missing platform — before pushing any. Never moves a dist ref that exists. |
110
111
  | `npx-launcher.mjs` | anywhere in-repo (e.g. `bin/launch.mjs`), named by the root `package.json`'s `bin` | Pairs with `dist-refs.yml` | Node ≥ 18, zero dependencies, needs `git`. Reads the `#vX.Y.Z` npx installed, fetches that one dist ref with the same URL and credentials npx cloned with, verifies `SHA256SUMS`, installs to a stable per-user path (`$XDG_DATA_HOME`/`%LOCALAPPDATA%`), and execs it; `--print-path` prints that path. A **channel** committish (`#stable`, `#next`, #465) is resolved to the release tag on the commit npx installed, then installed as that version; any other branch is refused. Also the library `npx-service.mjs` imports. `*.mjs`, so `colab template npx-launcher.mjs --dest <path>` copies it stamped, `--dest` required. One edit point: `BIN` (defaults to the package name without its scope). |
111
112
  | `npx-service.mjs` | the same directory as your `npx-launcher.mjs` copy (e.g. `bin/service.mjs`), named by the root `package.json`'s `bin` | A tool that runs as a **per-machine service** (a dashboard, a daemon, a JS app a launch agent serves), installed with npx ([CONVENTIONS §6, *Services over npx*](../CONVENTIONS.md#services-over-npx--init-update-rollback-465)) | `npx github:<org>/<repo>#vX.Y.Z init [--auto-update 6h \| --no-auto-update]`, then `<tool> update [--check] [--version vX.Y.Z] [--channel next\|stable]`, `<tool> rollback`, `<tool> status`, `<tool> uninstall [--purge]` (#472: unit, timer, shim and every version go; config and state stay unless `--purge`); anything else runs the current version's payload. Versions side by side under `$XDG_DATA_HOME/<tool>/` with `current`/`previous` links the unit and the `~/.local/bin` shim go through; update installs beside, switches, restarts, health-checks, and **switches back** on failure. Config/secrets under `$XDG_CONFIG_HOME/<tool>/`, state and logs under `$XDG_STATE_HOME/<tool>/` — the unit carries their locations, never a value. launchd agent on macOS, systemd user unit on Linux, Windows refused; `<TOOL>_SERVICE_MANAGER=none` skips the manager. `--auto-update` (#471) adds a timer (launchd `StartInterval` agent / systemd user `.timer`) that runs the same `update`. Node ≥ 18, zero dependencies; needs `git`, `npm` for `update`, `tar` for `dist`. **Imports `./npx-launcher.mjs`** — copy both, or edit the import. Edit points: `KIND` (`binary` = a compiled tool from dist refs, `package` = a JS app run with node, `dist` = a prebuilt JS app unpacked from one verified tarball in the `dist/vX.Y.Z/any` ref, #470), `LABEL`, `SERVICE_ARGS`, `ENTRY`, `DIST_FILE`, `HEALTH_URL`, `KEEP`, `AUTO_UPDATE`, `VERBS`. The package's `files` must include both copies (and a `package` app's built output). `colab template npx-service.mjs --dest <path>`, `--dest` required. |
113
+ | `publish-downloads.yml` | `.github/workflows/publish-downloads.yml` | A repo whose **finals** carry files people install on their own machines, published to a **self-hosted downloads site** ([CONVENTIONS §6, *Downloads*](../CONVENTIONS.md#downloads--finals-to-a-self-hosted-site-648); contract in [`docs/downloads.md`](../docs/downloads.md)) | `workflow_call` from your release workflow's last job (a tag or Release made with `GITHUB_TOKEN` triggers nothing else), and `workflow_dispatch` with a tag to retry, finish a partial run, or `dry_run`. Finals only. Downloads the Release's files the `downloads:` block in `project.yml` names and runs the script below. Inputs: `runs-on` — a runner that can **reach the site** (a private-network site is unreachable from a GitHub-hosted runner; an unresolvable name fails the run before anything is sent) — and `reserved`, names the site keeps for itself. Secrets `DOWNLOADS_SSH_KEY` / `_SSH_HOST` / `_SSH_PORT` / `_SSH_USER` / `DOWNLOADS_KNOWN_HOSTS`, variable `DOWNLOADS_URL` (the site root). Edit point: `PUBLISH_SCRIPT`. |
114
+ | `publish-downloads.mjs` | `.github/downloads/publish-downloads.mjs` (the workflow's `PUBLISH_SCRIPT`) | Pairs with `publish-downloads.yml` | `check` · `names vX.Y.Z` · `publish vX.Y.Z <dir> [--dry-run]`. rsync over SSH in a fixed order — the version folder, each file read back over HTTPS against its checksum, `SHA256SUMS`, installers (`@DOWNLOADS_URL@` stamped), `manifest.json`, `latest.json` **last** (never moved back to an older final) — then reads it all back. Never replaces a published file; a different `SHA256SUMS` for a published version refuses; a re-run sends nothing. Validates the manifest the portal lists. Node ≥ 18, zero dependencies. `colab template publish-downloads.mjs --dest <path>`, `--dest` required. |
112
115
  | `deploy-xserver.yml` | `.github/workflows/deploy-xserver.yml` | PHP-framework + Vite apps shipped to **shared hosting over SSH** (no root, no Docker): build on a runner, rsync, migrate on the server | Derived from three independently-written copies. Resolves Node the same way the CI templates do — all three hardcoded it, and one shipped on a different major than its CI built on. Migrates **production**; keeps a **mandatory** smoke test. Does **not** change your tier. |
113
116
  | `deploy-container.yml` | `.github/workflows/deploy-container.yml` | A **container** app on a host whose platform has an API (Portainer today), `deploy: tag` ([CONVENTIONS §6, *A container deploy*](../CONVENTIONS.md#6-releases), #452) | One contract on every host: build each listed image **once** per final tag (`vX.Y.Z` + commit sha), run the pre-deploy hook, have the platform run exactly that tag (all images, one call), green only on a **verified running version** (stack settled, commit and container images checked, then `release.health-url` reporting `X.Y.Z`), roll back to the previous final on a failure after the platform accepted. **Two jobs (#460):** `publish` builds and pushes on **every** final, ungated — so a repo not yet cut over still gets each final's image in the registry; `deploy` `needs:` it, never builds (a missing image fails before the platform is touched), and is the one job you gate on "is the platform switched on". Images and build args sit in the workflow-level `env`, written once for both jobs. Triggers: a final tag push (route `deploy-tag`), `workflow_dispatch` with a tag (a manual deploy or rollback), and `workflow_call` — route `deploy-tag-fast` reaches it from `release-auto.yml`'s commented `deploy` job, like `dist-refs.yml` from its release run. Needs its two scripts below. One required secret: `PORTAINER_API_KEY`, a non-admin key owning only this app's stack; optional `HEALTH_AUTH_HEADER` (`Name: value`) or `HEALTH_BASIC_AUTH` (`user:password`) when the version URL sits behind auth — masked, never printed. **Every redeploy recreates the containers.** Does **not** change your exposure. |
114
117
  | `deploy-container-run.mjs` | `.github/deploy/deploy-container-run.mjs` | Pairs with `deploy-container.yml` | The driver: builds, runs the hook, calls the adapter, verifies, rolls back, records. `--publish-only` = build and push only (no platform config, no version URL needed); `--deploy-only` = everything after, refusing a tag whose image is not in the registry. **Build args:** `DEPLOY_BUILD_ARGS`, one per line — `KEY=VALUE` for every image or `<image-ref> KEY=VALUE` for one; `${TAG}` (`vX.Y.Z`), `${VERSION}` (`X.Y.Z`) and `${SHA}` are filled in by the driver, so a static site can bake public config and its version into a served file. They end up in the image — public values only, never a secret. Node ≥ 18, zero dependencies; shells out to `git`, `docker`, `gh`. `colab template deploy-container-run.mjs --dest <path>`, `--dest` required, stamped on line 2. |
@@ -272,6 +275,8 @@ directory into ~45 repos and was wrong — fixed 2026-08-20.
272
275
  colab template deploy-container --dest .github/workflows/deploy-container.yml
273
276
  colab template deploy-container-run.mjs --dest .github/deploy/deploy-container-run.mjs
274
277
  colab template deploy-adapter-portainer.mjs --dest .github/deploy/deploy-adapter-portainer.mjs
278
+ colab template publish-downloads --dest .github/workflows/publish-downloads.yml
279
+ colab template publish-downloads.mjs --dest .github/downloads/publish-downloads.mjs
275
280
  colab template docs-lint.mjs --dest tools/docs-lint.mjs # --dest is required for this one
276
281
  ```
277
282
 
@@ -0,0 +1,574 @@
1
+ #!/usr/bin/env node
2
+ // Publish one FINAL version's files to a self-hosted downloads site — TEMPLATE (colab-handbook #648).
3
+ // Copy me into your repo beside the workflow (`colab template publish-downloads.mjs --dest
4
+ // .github/downloads/publish-downloads.mjs`); from then on YOU OWN THIS FILE. Node >= 18, zero dependencies,
5
+ // shells out to `rsync` (and `ssh` through it). Docs: docs/downloads.md in the handbook.
6
+ //
7
+ // node publish-downloads.mjs check validate the downloads: block and the manifest
8
+ // node publish-downloads.mjs names vX.Y.Z the Release files this publishes, one per line
9
+ // node publish-downloads.mjs publish vX.Y.Z <assetsDir> [--dry-run] stage, upload, read back
10
+ // (every command also takes --root <repo root>, default the current directory)
11
+ //
12
+ // WHAT IS PUBLISHED comes from the repo's `.github/project.yml`:
13
+ //
14
+ // downloads:
15
+ // app: myapp # the app's folder on the site: [a-z0-9][a-z0-9-]*
16
+ // files: # every final carries these; {v} is X.Y.Z
17
+ // - myapp-{v}.tgz
18
+ // - MyApp_{v}_aarch64.dmg
19
+ // optional: [MyApp_{v}_x64.msi] # published when the Release has it (optional key)
20
+ // installers: # repo paths, uploaded to /<app>/ with @DOWNLOADS_URL@ stamped (optional key)
21
+ // - scripts/install.sh
22
+ // manifest: .github/downloads/manifest.json # optional key, this is the default path
23
+ //
24
+ // Only that subset of YAML is read here (scalars, `- item` lists, `[a, b]` lists, comments): this script has no
25
+ // YAML library on purpose, and refuses — naming the line — anything else inside the block.
26
+ //
27
+ // EVERYTHING ABOUT THE SITE COMES FROM THE ENVIRONMENT; nothing about it is written in the repository:
28
+ //
29
+ // DOWNLOADS_URL the site's root address (a repo variable); /<app> is appended here
30
+ // DOWNLOADS_SSH_HOST/PORT/USER where rsync goes (repo secrets). The account is restricted to the app's
31
+ // folder server-side and cannot delete: its root IS /<app>/
32
+ // DOWNLOADS_KEY_FILE the private key file the workflow wrote from DOWNLOADS_SSH_KEY
33
+ // DOWNLOADS_KNOWN_HOSTS_FILE the known-hosts file the workflow wrote from DOWNLOADS_KNOWN_HOSTS
34
+ // DOWNLOADS_RESERVED names the site keeps for itself, refused as an app slug (space/comma list)
35
+ //
36
+ // THE ORDER ON THE SITE, which is what makes a half-finished run harmless:
37
+ // 1. <X.Y.Z>/ with every file except SHA256SUMS (rsync --ignore-existing: a published file is never replaced)
38
+ // 2. every file read back over HTTPS against the SHA256SUMS this build computed
39
+ // 3. <X.Y.Z>/SHA256SUMS (its presence = the folder is complete)
40
+ // 4. the installers (address stamped in), then manifest.json
41
+ // 5. latest.json LAST (unless the site already names a newer version)
42
+ // 6. SHA256SUMS and every file sent in 4–5 read back from the site
43
+ // A published version is immutable: a folder whose SHA256SUMS differs from this build's is an error and nothing is
44
+ // changed. A complete folder is verified and left alone; a partial one gets only its missing files. Installers,
45
+ // manifest.json and latest.json are sent only when the site's copy differs, so a re-run is a verified no-op.
46
+ // Never a candidate: the tag must be `vX.Y.Z`.
47
+ import { spawnSync } from 'node:child_process'
48
+ import { createHash } from 'node:crypto'
49
+ import { lookup } from 'node:dns/promises'
50
+ import {
51
+ copyFileSync,
52
+ createReadStream,
53
+ existsSync,
54
+ mkdirSync,
55
+ mkdtempSync,
56
+ readFileSync,
57
+ readdirSync,
58
+ realpathSync,
59
+ statSync,
60
+ writeFileSync,
61
+ } from 'node:fs'
62
+ import { tmpdir } from 'node:os'
63
+ import { basename, join, resolve } from 'node:path'
64
+ import { pathToFileURL } from 'node:url'
65
+
66
+ const FINAL_TAG = /^v(\d+\.\d+\.\d+)$/
67
+ const SLUG = /^[a-z0-9][a-z0-9-]*$/
68
+ const PLATFORMS = ['macos', 'windows', 'linux']
69
+ export const PLACEHOLDER = '@DOWNLOADS_URL@'
70
+ export const DEFAULT_MANIFEST = '.github/downloads/manifest.json'
71
+ /** Names the script itself writes at the app's root; an installer may not take one. */
72
+ const ROOT_NAMES = new Set(['latest.json', 'manifest.json'])
73
+
74
+ /** The version of a final tag. A candidate (`vX.Y.Z-rc.N`) and anything else is refused. */
75
+ export function finalVersion(tag) {
76
+ const m = typeof tag === 'string' ? FINAL_TAG.exec(tag) : null
77
+ if (!m)
78
+ throw new Error(
79
+ `${JSON.stringify(tag)} is not a final tag: expected vX.Y.Z (a candidate is never published)`,
80
+ )
81
+ return m[1]
82
+ }
83
+
84
+ // ── the downloads: block ─────────────────────────────────────────────────────────────────────────
85
+
86
+ function scalar(raw, where) {
87
+ let s = raw.trim()
88
+ if (s.startsWith('"') || s.startsWith("'")) {
89
+ const q = s[0]
90
+ const end = s.indexOf(q, 1)
91
+ if (end < 0) throw new Error(`${where}: unterminated quote`)
92
+ const rest = s.slice(end + 1).trim()
93
+ if (rest && !rest.startsWith('#')) throw new Error(`${where}: text after the closing quote`)
94
+ return s.slice(1, end)
95
+ }
96
+ const hash = s.search(/\s#/)
97
+ if (hash >= 0) s = s.slice(0, hash).trim()
98
+ if (s.startsWith('#')) s = ''
99
+ return s
100
+ }
101
+
102
+ function flowList(raw, where) {
103
+ const s = scalar(raw.replace(/\s+#.*$/, ''), where)
104
+ if (!s.startsWith('[') || !s.endsWith(']')) return null
105
+ const inner = s.slice(1, -1).trim()
106
+ return inner ? inner.split(',').map((x) => scalar(x, where)) : []
107
+ }
108
+
109
+ /**
110
+ * The `downloads:` block of project.yml text, as a plain object — or null when the repo declares none. Reads the
111
+ * subset documented in the header and throws, naming the line, on anything else inside the block.
112
+ */
113
+ export function parseDownloadsBlock(text) {
114
+ const lines = text.split(/\r?\n/)
115
+ const start = lines.findIndex((l) => /^downloads:\s*(#.*)?$/.test(l))
116
+ if (start < 0) {
117
+ const inline = lines.findIndex((l) => /^downloads:\s*\S/.test(l))
118
+ if (inline >= 0)
119
+ throw new Error(
120
+ `project.yml line ${inline + 1}: write downloads: as a block (one key per line), not inline`,
121
+ )
122
+ return null
123
+ }
124
+ const out = {}
125
+ let keyIndent = null
126
+ let listKey = null
127
+ for (let i = start + 1; i < lines.length; i++) {
128
+ const line = lines[i]
129
+ const where = `project.yml line ${i + 1}`
130
+ if (/^\s*(#.*)?$/.test(line)) continue
131
+ const indent = line.match(/^ */)[0].length
132
+ if (indent === 0) break // the next top-level key ends the block
133
+ if (line.includes('\t')) throw new Error(`${where}: a tab in the downloads: block`)
134
+ const body = line.slice(indent)
135
+ if (keyIndent === null) keyIndent = indent
136
+ if (indent > keyIndent && listKey && body.startsWith('- ')) {
137
+ out[listKey].push(scalar(body.slice(2), where))
138
+ continue
139
+ }
140
+ if (indent === keyIndent && listKey && body.startsWith('- ')) {
141
+ // `- item` at the key's own indent is valid YAML for a list under that key
142
+ out[listKey].push(scalar(body.slice(2), where))
143
+ continue
144
+ }
145
+ const m = indent === keyIndent ? /^([a-z][a-z-]*):(.*)$/.exec(body) : null
146
+ if (!m) throw new Error(`${where}: not a form this script reads inside downloads: (see its header)`)
147
+ const [, key, rest] = m
148
+ if (key in out) throw new Error(`${where}: downloads.${key} is given twice`)
149
+ const value = scalar(rest, where)
150
+ if (value === '') {
151
+ out[key] = []
152
+ listKey = key
153
+ continue
154
+ }
155
+ listKey = null
156
+ const list = flowList(rest, where)
157
+ out[key] = list ?? value
158
+ }
159
+ return out
160
+ }
161
+
162
+ const KEYS = ['app', 'files', 'optional', 'installers', 'manifest']
163
+
164
+ function fileName(pattern, where) {
165
+ if (typeof pattern !== 'string' || pattern === '') throw new Error(`${where}: an empty name`)
166
+ if (/[/\\]/.test(pattern) || pattern.startsWith('.') || pattern === 'SHA256SUMS')
167
+ throw new Error(`${where}: ${JSON.stringify(pattern)} is not a plain file name`)
168
+ return pattern
169
+ }
170
+
171
+ /** Validates a parsed block (see parseDownloadsBlock) and returns it normalised. */
172
+ export function readConfig(block, { reserved = [] } = {}) {
173
+ if (!block) throw new Error('.github/project.yml has no downloads: block — this repo publishes nothing')
174
+ for (const k of Object.keys(block))
175
+ if (!KEYS.includes(k)) throw new Error(`downloads.${k} is not a key (known: ${KEYS.join(', ')})`)
176
+ const app = block.app
177
+ if (typeof app !== 'string' || app === '') throw new Error('downloads.app is required')
178
+ if (app.startsWith('_')) throw new Error(`downloads.app ${JSON.stringify(app)}: names starting with _ belong to the site`)
179
+ if (!SLUG.test(app)) throw new Error(`downloads.app ${JSON.stringify(app)} is not a slug: [a-z0-9][a-z0-9-]*`)
180
+ if (reserved.includes(app)) throw new Error(`downloads.app ${JSON.stringify(app)} is a name the site reserves for itself`)
181
+ const list = (k, required) => {
182
+ const v = block[k]
183
+ if (v === undefined && !required) return []
184
+ if (!Array.isArray(v) || (required && v.length === 0))
185
+ throw new Error(`downloads.${k} must be a ${required ? 'non-empty ' : ''}list`)
186
+ return v
187
+ }
188
+ const files = list('files', true).map((p) => fileName(p, 'downloads.files'))
189
+ const optional = list('optional', false).map((p) => fileName(p, 'downloads.optional'))
190
+ const all = [...files, ...optional]
191
+ const dup = all.find((n, i) => all.indexOf(n) !== i)
192
+ if (dup) throw new Error(`downloads: ${dup} is listed twice`)
193
+ const installers = list('installers', false)
194
+ const seen = new Set()
195
+ for (const p of installers) {
196
+ const b = basename(String(p))
197
+ if (!p || p.startsWith('/') || String(p).split(/[/\\]/).includes('..'))
198
+ throw new Error(`downloads.installers: ${JSON.stringify(p)} is not a path inside the repo`)
199
+ if (ROOT_NAMES.has(b) || /^\d+\.\d+\.\d+$/.test(b) || seen.has(b))
200
+ throw new Error(`downloads.installers: ${b} would collide with another file at the app's root`)
201
+ seen.add(b)
202
+ }
203
+ const manifest = block.manifest ?? DEFAULT_MANIFEST
204
+ if (typeof manifest !== 'string' || manifest.startsWith('/') || manifest.split(/[/\\]/).includes('..'))
205
+ throw new Error('downloads.manifest must be a path inside the repo')
206
+ return { app, files, optional, installers, manifest }
207
+ }
208
+
209
+ export const namesFor = (patterns, v) => patterns.map((p) => p.split('{v}').join(v))
210
+
211
+ // ── manifest.json ────────────────────────────────────────────────────────────────────────────────
212
+
213
+ /** A text the portal shows: a string, or one string per language code (`{"en": "…", "vi": "…"}`). */
214
+ function isText(v, oneLine = false) {
215
+ const ok = (s) => typeof s === 'string' && s.trim() !== '' && (!oneLine || !/[\r\n]/.test(s))
216
+ if (typeof v === 'string') return ok(v)
217
+ return (
218
+ v !== null &&
219
+ typeof v === 'object' &&
220
+ !Array.isArray(v) &&
221
+ Object.keys(v).length > 0 &&
222
+ Object.entries(v).every(([k, s]) => /^[a-z]{2}(-[A-Z]{2})?$/.test(k) && ok(s))
223
+ )
224
+ }
225
+
226
+ /** Throws naming every problem with a manifest object; returns it unchanged when valid. Fields: docs/downloads.md. */
227
+ export function validateManifest(m) {
228
+ const problems = []
229
+ if (m === null || typeof m !== 'object' || Array.isArray(m)) throw new Error('manifest.json is not a JSON object')
230
+ for (const k of Object.keys(m))
231
+ if (!['schema', 'name', 'summary', 'install'].includes(k)) problems.push(`unknown field "${k}"`)
232
+ if (m.schema !== undefined && m.schema !== 1) problems.push('schema must be 1')
233
+ if (!isText(m.name, true)) problems.push('name: a one-line text is required')
234
+ if (!isText(m.summary, true)) problems.push('summary: a one-line text is required')
235
+ const inst = m.install
236
+ if (inst === null || typeof inst !== 'object' || Array.isArray(inst) || Object.keys(inst).length === 0) {
237
+ problems.push(`install: an object keyed by platform (${PLATFORMS.join(', ')}) is required`)
238
+ } else {
239
+ for (const [os, how] of Object.entries(inst)) {
240
+ if (!PLATFORMS.includes(os)) {
241
+ problems.push(`install.${os}: not a platform (${PLATFORMS.join(', ')})`)
242
+ continue
243
+ }
244
+ if (how === null || typeof how !== 'object' || Array.isArray(how)) {
245
+ problems.push(`install.${os}: an object with steps (and optional prompts)`)
246
+ continue
247
+ }
248
+ for (const k of Object.keys(how))
249
+ if (!['steps', 'prompts'].includes(k)) problems.push(`install.${os}.${k}: unknown field`)
250
+ if (!Array.isArray(how.steps) || how.steps.length === 0 || !how.steps.every((s) => isText(s)))
251
+ problems.push(`install.${os}.steps: a non-empty list of texts`)
252
+ if (how.prompts !== undefined && (!Array.isArray(how.prompts) || !how.prompts.every((s) => isText(s))))
253
+ problems.push(`install.${os}.prompts: a list of texts`)
254
+ }
255
+ }
256
+ if (problems.length) throw new Error(`manifest.json: ${problems.join('; ')}`)
257
+ return m
258
+ }
259
+
260
+ /** Reads and validates the repo's config + manifest. */
261
+ export function loadRepo(root, env = {}) {
262
+ const yml = join(root, '.github', 'project.yml')
263
+ if (!existsSync(yml)) throw new Error(`${yml} does not exist`)
264
+ const reserved = String(env.DOWNLOADS_RESERVED ?? '')
265
+ .split(/[\s,]+/)
266
+ .filter(Boolean)
267
+ const config = readConfig(parseDownloadsBlock(readFileSync(yml, 'utf8')), { reserved })
268
+ const mPath = join(root, config.manifest)
269
+ if (!existsSync(mPath)) throw new Error(`${config.manifest} does not exist — every app on the site carries a manifest`)
270
+ let manifest
271
+ try {
272
+ manifest = JSON.parse(readFileSync(mPath, 'utf8'))
273
+ } catch (error) {
274
+ throw new Error(`${config.manifest} is not JSON: ${error.message}`)
275
+ }
276
+ validateManifest(manifest)
277
+ for (const p of config.installers) {
278
+ const f = join(root, p)
279
+ if (!existsSync(f)) throw new Error(`downloads.installers: ${p} does not exist`)
280
+ if (!readFileSync(f, 'utf8').includes(PLACEHOLDER))
281
+ throw new Error(`${p} has no ${PLACEHOLDER} to stamp the downloads address into`)
282
+ }
283
+ return { config, manifest }
284
+ }
285
+
286
+ // ── staging ──────────────────────────────────────────────────────────────────────────────────────
287
+
288
+ export function sha256File(path) {
289
+ return new Promise((resolveHash, reject) => {
290
+ const h = createHash('sha256')
291
+ createReadStream(path)
292
+ .on('data', (c) => h.update(c))
293
+ .on('error', reject)
294
+ .on('end', () => resolveHash(h.digest('hex')))
295
+ })
296
+ }
297
+
298
+ /** Compares X.Y.Z versions: negative when a < b. */
299
+ export function compareVersions(a, b) {
300
+ const pa = a.split('.').map(Number)
301
+ const pb = b.split('.').map(Number)
302
+ for (let i = 0; i < 3; i++) if (pa[i] !== pb[i]) return pa[i] - pb[i]
303
+ return 0
304
+ }
305
+
306
+ export function siteRoot(baseUrl) {
307
+ if (typeof baseUrl !== 'string' || !/^https:\/\/[^\s'"\\]+$/.test(baseUrl)) {
308
+ throw new Error('DOWNLOADS_URL must be an https address (the repo variable is missing or malformed)')
309
+ }
310
+ return baseUrl.replace(/\/+$/, '')
311
+ }
312
+
313
+ /**
314
+ * Lays out what goes to the site under `outDir`: `<X.Y.Z>/` (the files + a combined SHA256SUMS), the installers with
315
+ * the site root stamped in, manifest.json and latest.json. Throws, uploading nothing, when a required file is missing.
316
+ */
317
+ export async function stage({ tag, assetsDir, outDir, baseUrl, root, config, manifest }) {
318
+ const version = finalVersion(tag)
319
+ const base = siteRoot(baseUrl)
320
+ const present = new Set(readdirSync(assetsDir))
321
+ const missing = namesFor(config.files, version).filter((n) => !present.has(n))
322
+ if (missing.length > 0) {
323
+ throw new Error(
324
+ `the Release of ${tag} lacks ${missing.join(', ')} — nothing was published; finish the builds, then dispatch the publish again`,
325
+ )
326
+ }
327
+ const names = [...namesFor(config.files, version), ...namesFor(config.optional, version).filter((n) => present.has(n))]
328
+ const folder = join(outDir, version)
329
+ mkdirSync(folder, { recursive: true })
330
+ const files = {}
331
+ for (const name of names) {
332
+ copyFileSync(join(assetsDir, name), join(folder, name))
333
+ files[name] = { sha256: await sha256File(join(folder, name)), size: statSync(join(folder, name)).size }
334
+ }
335
+ const sums = Object.keys(files)
336
+ .sort()
337
+ .map((n) => `${files[n].sha256} ${n}\n`)
338
+ .join('')
339
+ writeFileSync(join(folder, 'SHA256SUMS'), sums)
340
+ const rootFiles = []
341
+ for (const p of config.installers) {
342
+ const source = readFileSync(join(root, p), 'utf8')
343
+ writeFileSync(join(outDir, basename(p)), source.split(PLACEHOLDER).join(base))
344
+ rootFiles.push(basename(p))
345
+ }
346
+ writeFileSync(join(outDir, 'manifest.json'), `${JSON.stringify(manifest, null, 2)}\n`)
347
+ rootFiles.push('manifest.json')
348
+ writeFileSync(join(outDir, 'latest.json'), `${JSON.stringify({ version, files }, null, 2)}\n`)
349
+ return { version, files, sums, rootFiles }
350
+ }
351
+
352
+ // ── the site ─────────────────────────────────────────────────────────────────────────────────────
353
+
354
+ function runRsync({ rsync, sshArgs, args, log }) {
355
+ const full = [...(sshArgs ? ['-e', sshArgs] : []), ...args]
356
+ log(`$ rsync ${full.join(' ')}`)
357
+ const r = spawnSync(rsync, full, { encoding: 'utf8' })
358
+ if (r.status !== 0) throw new Error(`rsync failed (${r.status ?? r.signal}): ${(r.stderr || r.stdout || '').trim()}`)
359
+ }
360
+
361
+ async function get(fetchImpl, url, attempts = 3) {
362
+ let last
363
+ for (let i = 0; i < attempts; i++) {
364
+ try {
365
+ const res = await fetchImpl(url, { headers: { 'cache-control': 'no-cache' } })
366
+ if (res.status < 500) return res
367
+ last = new Error(`${url} answered ${res.status}`)
368
+ } catch (error) {
369
+ last = new Error(`${url} could not be read (${error.cause?.code ?? error.message}) — is the site reachable from this runner?`)
370
+ }
371
+ if (i + 1 < attempts) await new Promise((r) => setTimeout(r, 1000 * (i + 1)))
372
+ }
373
+ throw last
374
+ }
375
+
376
+ /** The bytes at `url`, or null on 404. Any other failure throws. */
377
+ async function readBytes(fetchImpl, url) {
378
+ const res = await get(fetchImpl, url)
379
+ if (res.status === 404) return null
380
+ if (!res.ok) throw new Error(`${url} answered ${res.status}`)
381
+ return Buffer.from(await res.arrayBuffer())
382
+ }
383
+
384
+ async function hashUrl(fetchImpl, url) {
385
+ const res = await get(fetchImpl, url)
386
+ if (!res.ok) return { status: res.status }
387
+ const h = createHash('sha256')
388
+ for await (const chunk of res.body) h.update(chunk)
389
+ return { status: res.status, sha256: h.digest('hex') }
390
+ }
391
+
392
+ /**
393
+ * Fails clearly, before anything else is tried, when the site's host does not resolve from this runner — the site
394
+ * may be reachable only on a private network, and only a runner on it can publish and read back.
395
+ */
396
+ export async function assertResolves(baseUrl, resolver = lookup) {
397
+ const host = new URL(siteRoot(baseUrl)).hostname
398
+ try {
399
+ await resolver(host)
400
+ } catch (error) {
401
+ throw new Error(
402
+ `${host} does not resolve from this runner (${error.code ?? error.message}) — the site may be private-network only: run the job on a runner that can reach it (the workflow's runs-on input)`,
403
+ )
404
+ }
405
+ }
406
+
407
+ /**
408
+ * Uploads a staged tree (see `stage`) in the order the header gives. `dest` is the rsync destination (`user@host:`
409
+ * in production — the account's root is the app's folder), `sshArgs` the `-e` command line.
410
+ */
411
+ export async function publish({
412
+ outDir,
413
+ app,
414
+ dest,
415
+ sshArgs,
416
+ baseUrl,
417
+ dryRun = false,
418
+ rsync = 'rsync',
419
+ fetchImpl = fetch,
420
+ log = console.log,
421
+ }) {
422
+ const root = `${siteRoot(baseUrl)}/${app}`
423
+ const latestBytes = readFileSync(join(outDir, 'latest.json'))
424
+ const { version, files } = JSON.parse(latestBytes.toString('utf8'))
425
+ const localSums = readFileSync(join(outDir, version, 'SHA256SUMS'))
426
+ const rootFiles = readdirSync(outDir).filter((n) => n !== 'latest.json' && n !== version && n !== 'manifest.json')
427
+ rootFiles.sort()
428
+ rootFiles.push('manifest.json') // installers first, manifest after them, latest.json last
429
+ const out = { version, uploaded: [], unchanged: [], skippedLatest: false, dryRun }
430
+ const slash = /[/:]$/.test(dest) ? '' : '/'
431
+
432
+ const remoteSums = await readBytes(fetchImpl, `${root}/${version}/SHA256SUMS`)
433
+ if (remoteSums !== null && !remoteSums.equals(localSums)) {
434
+ throw new Error(
435
+ `${version} is already published and its SHA256SUMS differs from this build's — versions are immutable, nothing was changed; publish a new version instead`,
436
+ )
437
+ }
438
+ log(remoteSums === null ? `${version}: not complete on the site yet` : `${version}: complete on the site — verifying only`)
439
+
440
+ const remoteLatest = await readBytes(fetchImpl, `${root}/latest.json`)
441
+ let remoteVersion = null
442
+ if (remoteLatest !== null) {
443
+ try {
444
+ const v = JSON.parse(remoteLatest.toString('utf8')).version
445
+ if (typeof v === 'string' && /^\d+\.\d+\.\d+$/.test(v)) remoteVersion = v
446
+ } catch {
447
+ remoteVersion = null
448
+ }
449
+ }
450
+ out.skippedLatest = remoteVersion !== null && compareVersions(remoteVersion, version) > 0
451
+
452
+ const common = { rsync, sshArgs, log }
453
+ if (dryRun) {
454
+ log('dry run — nothing is uploaded. Would send, in this order:')
455
+ if (remoteSums === null) {
456
+ for (const n of Object.keys(files)) log(` ${version}/${n}`)
457
+ log(` ${version}/SHA256SUMS`)
458
+ }
459
+ for (const n of rootFiles) log(` ${n} (when the site's copy differs)`)
460
+ log(out.skippedLatest ? ` (latest.json stays: the site names ${remoteVersion})` : ' latest.json (when the site\'s copy differs)')
461
+ return out
462
+ }
463
+
464
+ if (remoteSums === null) {
465
+ runRsync({ ...common, args: ['-rt', '--ignore-existing', '--exclude', 'SHA256SUMS', join(outDir, version), dest] })
466
+ // Check every file as a visitor would get it before the folder is declared complete.
467
+ for (const [name, want] of Object.entries(files)) {
468
+ const got = await hashUrl(fetchImpl, `${root}/${version}/${name}`)
469
+ if (got.sha256 !== want.sha256) {
470
+ throw new Error(
471
+ `${version}/${name} on the site is ${got.status === 200 ? 'a different file' : `answering ${got.status}`} than this build's — the folder was left without SHA256SUMS; a person removes that file on the server, then dispatch the same tag again`,
472
+ )
473
+ }
474
+ out.uploaded.push(`${version}/${name}`)
475
+ }
476
+ runRsync({ ...common, args: ['-t', join(outDir, version, 'SHA256SUMS'), `${dest}${slash}${version}/`] })
477
+ out.uploaded.push(`${version}/SHA256SUMS`)
478
+ } else {
479
+ for (const [name, want] of Object.entries(files)) {
480
+ const got = await hashUrl(fetchImpl, `${root}/${version}/${name}`)
481
+ if (got.sha256 !== want.sha256)
482
+ throw new Error(`${version}/${name} on the site does not match its SHA256SUMS (${got.status})`)
483
+ }
484
+ }
485
+
486
+ // The app-root files are replaced only when they differ, so a re-run sends nothing.
487
+ const mutable = out.skippedLatest ? rootFiles : [...rootFiles, 'latest.json']
488
+ for (const name of mutable) {
489
+ const local = readFileSync(join(outDir, name))
490
+ const remote = await readBytes(fetchImpl, `${root}/${name}`)
491
+ if (remote !== null && remote.equals(local)) {
492
+ out.unchanged.push(name)
493
+ continue
494
+ }
495
+ runRsync({ ...common, args: ['-t', join(outDir, name), dest] })
496
+ out.uploaded.push(name)
497
+ }
498
+ if (out.skippedLatest) log(`latest.json stays: the site already names ${remoteVersion}, newer than ${version}`)
499
+
500
+ // Read back what a visitor reads.
501
+ const sumsBack = await readBytes(fetchImpl, `${root}/${version}/SHA256SUMS`)
502
+ if (sumsBack === null || !sumsBack.equals(localSums))
503
+ throw new Error(`${root}/${version}/SHA256SUMS does not read back as this build's`)
504
+ for (const name of mutable) {
505
+ const back = await readBytes(fetchImpl, `${root}/${name}`)
506
+ if (back === null || !back.equals(readFileSync(join(outDir, name))))
507
+ throw new Error(`${root}/${name} does not read back as the file just sent`)
508
+ }
509
+ return out
510
+ }
511
+
512
+ function sshCommand(env) {
513
+ const need = (k) => {
514
+ if (!env[k]) throw new Error(`${k} is not set — the upload secrets are missing on this repo`)
515
+ return env[k]
516
+ }
517
+ const port = need('DOWNLOADS_SSH_PORT')
518
+ if (!/^\d+$/.test(port)) throw new Error('DOWNLOADS_SSH_PORT is not a port number')
519
+ const key = need('DOWNLOADS_KEY_FILE')
520
+ const known = need('DOWNLOADS_KNOWN_HOSTS_FILE')
521
+ for (const f of [key, known]) if (!existsSync(f)) throw new Error(`${f} does not exist`)
522
+ return {
523
+ dest: `${need('DOWNLOADS_SSH_USER')}@${need('DOWNLOADS_SSH_HOST')}:`,
524
+ sshArgs: `ssh -p ${port} -i ${key} -o UserKnownHostsFile=${known} -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes`,
525
+ }
526
+ }
527
+
528
+ async function main(argv, env) {
529
+ const rootAt = argv.indexOf('--root')
530
+ const root = resolve(rootAt >= 0 ? argv[rootAt + 1] ?? '' : '.')
531
+ const args = argv.filter((a, i) => !(rootAt >= 0 && (i === rootAt || i === rootAt + 1)))
532
+ const [command, tag, ...rest] = args
533
+ const usage = 'usage: publish-downloads.mjs check | names <tag> | publish <tag> <assetsDir> [--dry-run] [--root <repo>]'
534
+ if (!['check', 'names', 'publish'].includes(command)) throw new Error(usage)
535
+ const { config, manifest } = loadRepo(root, env)
536
+ if (command === 'check') {
537
+ console.log(`downloads: ${config.app} — ${config.files.length} file(s), ${config.optional.length} optional, ${config.installers.length} installer(s), manifest ${config.manifest} valid`)
538
+ return 0
539
+ }
540
+ if (command === 'names') {
541
+ const v = finalVersion(tag)
542
+ console.log([...namesFor(config.files, v), ...namesFor(config.optional, v)].join('\n'))
543
+ return 0
544
+ }
545
+ const dryRun = rest.includes('--dry-run')
546
+ const assetsDir = resolve(rest.filter((a) => !a.startsWith('--'))[0] ?? '')
547
+ const baseUrl = env.DOWNLOADS_URL
548
+ const outDir = mkdtempSync(join(tmpdir(), 'publish-downloads-'))
549
+ const staged = await stage({ tag, assetsDir, outDir, baseUrl, root, config, manifest })
550
+ console.log(`Staged ${config.app} ${staged.version}: ${Object.keys(staged.files).join(', ')}`)
551
+ await assertResolves(baseUrl)
552
+ const target = dryRun ? { dest: 'DRY-RUN:', sshArgs: undefined } : sshCommand(env)
553
+ const result = await publish({ outDir, app: config.app, ...target, baseUrl, dryRun })
554
+ if (dryRun) console.log(`Dry run of ${result.version} done — nothing was uploaded`)
555
+ else
556
+ console.log(
557
+ `Published ${config.app} ${result.version}: sent ${result.uploaded.length ? result.uploaded.join(', ') : 'nothing (a verified no-op)'}` +
558
+ `${result.unchanged.length ? `; unchanged ${result.unchanged.join(', ')}` : ''}${result.skippedLatest ? ' (latest.json names a newer version)' : ''}`,
559
+ )
560
+ return 0
561
+ }
562
+
563
+ // realpath: a script run through a symlinked path (macOS /var → /private/var) is still this script.
564
+ if (process.argv[1] && import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href) {
565
+ main(process.argv.slice(2), process.env).then(
566
+ (code) => {
567
+ process.exitCode = code
568
+ },
569
+ (error) => {
570
+ console.error(error instanceof Error ? error.message : String(error))
571
+ process.exitCode = 1
572
+ },
573
+ )
574
+ }
@@ -0,0 +1,181 @@
1
+ # Publish downloads — TEMPLATE. Copy me into your repo at .github/workflows/publish-downloads.yml.
2
+ #
3
+ # This is a starting point, not a dependency. Once you copy it, YOU OWN THIS FILE.
4
+ # Edit it freely; nothing calls this file remotely and nothing will update it for you.
5
+ #
6
+ # EDIT POINTS are marked `# EDIT:` — walk them once after copying.
7
+ #
8
+ # It needs its script in your repo (copy-and-own as well), at the path PUBLISH_SCRIPT names below:
9
+ # colab template publish-downloads.mjs --dest .github/downloads/publish-downloads.mjs
10
+ #
11
+ # For: a repo whose FINAL releases carry files people install on their own machines (a .dmg, a .deb, an
12
+ # .msi, a tarball …), published to a self-hosted downloads site — one folder per app, the newest finals kept
13
+ # by the site (colab-handbook #648; docs/downloads.md has the whole contract). What is published comes from
14
+ # the repo's `.github/project.yml` `downloads:` block plus a manifest.json the site's portal lists the app
15
+ # from; the script's header documents both.
16
+ #
17
+ # WHAT IT DOES for a final tag vX.Y.Z whose files are on its GitHub Release: uploads them with rsync over SSH
18
+ # in the order that keeps a half-finished run harmless — the version folder (with a combined SHA256SUMS last
19
+ # inside it, written only after every file read back over HTTPS against it), then the installers with the
20
+ # site address stamped in, then manifest.json, then latest.json LAST — and reads it all back. A published
21
+ # version is immutable: nothing is ever replaced, and a build whose SHA256SUMS differs from the published
22
+ # one is refused. A re-run of a complete version verifies and sends nothing. NEVER a candidate (-rc.N).
23
+ #
24
+ # WHEN IT RUNS.
25
+ # - Chained: your release/build workflow calls this one as its LAST job for a final tag, after the jobs
26
+ # that attach the files to the Release, and only when none of them failed:
27
+ # downloads:
28
+ # needs: [build-macos, build-linux] # the jobs that upload the Release's files
29
+ # if: ${{ !contains(github.ref_name, '-') }} # finals only
30
+ # uses: ./.github/workflows/publish-downloads.yml
31
+ # with: { tag: '${{ github.ref_name }}' }
32
+ # secrets: inherit
33
+ # (A tag pushed or a Release published with GITHUB_TOKEN starts no other workflow, so a chained call is
34
+ # the reliable trigger — not `on: release`.) Nothing is published from an incomplete Release: the script
35
+ # names what is missing and uploads nothing.
36
+ # - By hand (Actions → Run workflow) with a tag: retries a failed publish, finishes a partial one (only the
37
+ # missing files are sent), re-checks a complete one. `dry_run` stages and lists — it uploads nothing.
38
+ # The project.yml, manifest, installers and script are read from THIS run's checkout: the tag's commit when
39
+ # chained, the branch you dispatch from when run by hand.
40
+ #
41
+ # THE RUNNER. The read-back goes over HTTPS to the site, which may be reachable only on a private network —
42
+ # a GitHub-hosted runner then cannot read anything back. `runs-on` is an input: a bare label or a JSON label
43
+ # list of a runner that can reach the site. If the site's name does not resolve there, the run fails saying
44
+ # so, before anything is sent. The runner needs `gh`, `rsync` and `ssh`; Node is set up by the job.
45
+ #
46
+ # CONFIGURATION — per repo, never shared between repos:
47
+ # secrets DOWNLOADS_SSH_KEY · DOWNLOADS_SSH_HOST · DOWNLOADS_SSH_PORT · DOWNLOADS_SSH_USER ·
48
+ # DOWNLOADS_KNOWN_HOSTS — an upload account restricted to this app's folder, that cannot delete
49
+ # variable DOWNLOADS_URL — the site's root address (https). The script appends /<app>; no host is ever
50
+ # written in the repository.
51
+ #
52
+ # THIS FILE DOES NOT CHANGE YOUR REPO'S EXPOSURE (OR LEGACY TIER): it publishes files a final already made.
53
+ name: Publish downloads
54
+
55
+ on:
56
+ workflow_call:
57
+ inputs:
58
+ tag:
59
+ type: string
60
+ required: true
61
+ dry_run:
62
+ type: boolean
63
+ default: false
64
+ runs-on:
65
+ type: string
66
+ default: '["self-hosted","linux"]' # EDIT: a runner that can reach the downloads site
67
+ reserved:
68
+ type: string
69
+ default: '' # EDIT: names the site keeps for itself, refused as an app slug (space-separated)
70
+ workflow_dispatch:
71
+ inputs:
72
+ tag:
73
+ description: 'Existing FINAL tag to publish, e.g. v1.4.0 (a candidate is refused)'
74
+ type: string
75
+ required: true
76
+ dry_run:
77
+ description: 'Dry run: stage and list what would be sent — nothing is uploaded'
78
+ type: boolean
79
+ default: false
80
+ runs-on:
81
+ description: 'Runner: a bare label or a JSON label list of a runner that can reach the site'
82
+ type: string
83
+ default: '["self-hosted","linux"]' # EDIT: keep in step with the workflow_call default
84
+ reserved:
85
+ description: 'Names the site reserves for itself, refused as an app slug (space-separated)'
86
+ type: string
87
+ default: '' # EDIT: keep in step with the workflow_call default
88
+
89
+ permissions: {}
90
+
91
+ concurrency:
92
+ group: publish-downloads
93
+ cancel-in-progress: false
94
+
95
+ env:
96
+ PUBLISH_SCRIPT: .github/downloads/publish-downloads.mjs # EDIT: where you copied the script
97
+
98
+ jobs:
99
+ publish:
100
+ name: Publish ${{ inputs.tag }}
101
+ runs-on: ${{ startsWith(inputs.runs-on, '[') && fromJSON(inputs.runs-on) || inputs.runs-on }}
102
+ timeout-minutes: 45 # EDIT: above your own p99 — large installers over a slow link take a while
103
+ permissions:
104
+ contents: read
105
+ env:
106
+ GH_TOKEN: ${{ github.token }}
107
+ TAG: ${{ inputs.tag }}
108
+ DRY_RUN: ${{ inputs.dry_run }}
109
+ DOWNLOADS_URL: ${{ vars.DOWNLOADS_URL }}
110
+ DOWNLOADS_RESERVED: ${{ inputs.reserved }}
111
+ DOWNLOADS_SSH_HOST: ${{ secrets.DOWNLOADS_SSH_HOST }}
112
+ DOWNLOADS_SSH_PORT: ${{ secrets.DOWNLOADS_SSH_PORT }}
113
+ DOWNLOADS_SSH_USER: ${{ secrets.DOWNLOADS_SSH_USER }}
114
+ steps:
115
+ - name: A final tag only
116
+ run: |
117
+ set -euo pipefail
118
+ [[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || { echo "::error::$TAG is not a final tag (vX.Y.Z): only finals are published"; exit 1; }
119
+
120
+ - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
121
+ with:
122
+ persist-credentials: false
123
+
124
+ - name: Setup Node
125
+ uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
126
+ with:
127
+ node-version: '22' # EDIT: any Node >= 18 runs the script
128
+
129
+ - name: The tools the run needs
130
+ run: |
131
+ set -euo pipefail
132
+ need="gh"; [ "$DRY_RUN" = "true" ] || need="gh rsync ssh"
133
+ for t in $need; do command -v "$t" >/dev/null || { echo "::error::$t is not on this runner — pick one that has it (the runs-on input)"; exit 1; }; done
134
+ node "$PUBLISH_SCRIPT" check
135
+
136
+ - name: Download the Release's files
137
+ id: assets
138
+ run: |
139
+ set -euo pipefail
140
+ DIR="$RUNNER_TEMP/assets"; rm -rf "$DIR"; mkdir -p "$DIR"
141
+ gh release view "$TAG" --repo "$GITHUB_REPOSITORY" --json assets -q '.assets[].name' > "$RUNNER_TEMP/release-assets.txt"
142
+ while IFS= read -r name; do
143
+ if grep -qxF -- "$name" "$RUNNER_TEMP/release-assets.txt"; then
144
+ gh release download "$TAG" --repo "$GITHUB_REPOSITORY" --pattern "$name" --dir "$DIR"
145
+ fi
146
+ done < <(node "$PUBLISH_SCRIPT" names "$TAG")
147
+ ls -l "$DIR"
148
+ echo "dir=$DIR" >> "$GITHUB_OUTPUT"
149
+
150
+ - name: Write the upload key (this run only)
151
+ if: inputs.dry_run != true
152
+ env:
153
+ KEY: ${{ secrets.DOWNLOADS_SSH_KEY }}
154
+ KNOWN_HOSTS: ${{ secrets.DOWNLOADS_KNOWN_HOSTS }}
155
+ run: |
156
+ set -euo pipefail
157
+ umask 077
158
+ [ -n "$KEY" ] && [ -n "$KNOWN_HOSTS" ] || { echo "::error::the downloads upload secrets are not set on this repo"; exit 1; }
159
+ mkdir -p "$RUNNER_TEMP/downloads-ssh"
160
+ printf '%s\n' "$KEY" > "$RUNNER_TEMP/downloads-ssh/key"
161
+ printf '%s\n' "$KNOWN_HOSTS" > "$RUNNER_TEMP/downloads-ssh/known_hosts"
162
+ chmod 600 "$RUNNER_TEMP/downloads-ssh/key" "$RUNNER_TEMP/downloads-ssh/known_hosts"
163
+
164
+ - name: Publish
165
+ env:
166
+ ASSETS: ${{ steps.assets.outputs.dir }}
167
+ DOWNLOADS_KEY_FILE: ${{ runner.temp }}/downloads-ssh/key
168
+ DOWNLOADS_KNOWN_HOSTS_FILE: ${{ runner.temp }}/downloads-ssh/known_hosts
169
+ run: |
170
+ set -euo pipefail
171
+ FLAG=""; [ "$DRY_RUN" = "true" ] && FLAG="--dry-run"
172
+ # The script's own status is kept: a `{ …; echo; } | tee` group would end on the echo and go green.
173
+ status=0
174
+ { echo "### Downloads → $TAG"; echo; echo '```'; } >> "$GITHUB_STEP_SUMMARY"
175
+ node "$PUBLISH_SCRIPT" publish "$TAG" "$ASSETS" $FLAG 2>&1 | tee -a "$GITHUB_STEP_SUMMARY" || status=$?
176
+ echo '```' >> "$GITHUB_STEP_SUMMARY"
177
+ exit "$status"
178
+
179
+ - name: Remove the upload key
180
+ if: always()
181
+ run: rm -rf "$RUNNER_TEMP/downloads-ssh"
package/tools/colab CHANGED
@@ -11515,8 +11515,9 @@ function resolveCloseRefsSplit(repoAbs, gh, issues, refsFromFlag, declaredHolds
11515
11515
  if (gh) {
11516
11516
  for (const num of issues) {
11517
11517
  if (!refsFromFlag.has(num)) continue;
11518
- const info = git.ghIssueView(repoAbs, num, ['body', 'labels']);
11518
+ const info = git.ghIssueView(repoAbs, num, ['body', 'labels', 'state']);
11519
11519
  if (!info) continue; // read failed — a reminder never fires on a read we could not attempt
11520
+ if (String(info.state || '').toUpperCase() === 'CLOSED') continue; // #647: nothing re-takes a closed issue
11520
11521
  const f = refsBrake.refsBrakeFinding({ issue: num, body: info.body, labels: info.labels, declaredHolds });
11521
11522
  if (f) brakeFindings.push(f);
11522
11523
  }
@@ -12729,9 +12730,7 @@ function cmdShipDryJson(opts) {
12729
12730
 
12730
12731
  // #74/#263: same close-gate split the prose path runs, reported via the additive fields above —
12731
12732
  // AND, since #263, folded into `checks` as its own precondition row (below).
12732
- const refsFromFlagJson = new Set(
12733
- String(opts.refs || '').split(',').map((s) => s.trim().replace(/^#/, '')).filter(Boolean).map(Number),
12734
- );
12733
+ const refsFromFlagJson = shipRefsFromFlag(opts, sess); // #647: + the hand-off's closed-before issues
12735
12734
  const split = resolveCloseRefsSplit(repoAbs, gh, sess.issues, refsFromFlagJson, descriptorHolds(doc));
12736
12735
  closeIssues = split.closeIssues; refsIssues = split.refsIssues; checklistFindings = split.gateFindings;
12737
12736
  refsBrakeFindings = split.brakeFindings;
@@ -13746,27 +13745,48 @@ function applyShipHandoff(repoAbs, sess, opts, trunk) {
13746
13745
  syncChain = shipHandoff.syncMergeChain((a) => git.git(a, repoAbs), refs.remoteSha, `refs/remotes/${remote}/${trunk}`);
13747
13746
  }
13748
13747
  const claimsByIssue = {};
13748
+ const issueStates = {}; // #647: an issue closed before the hand-off needs no claim — read state too
13749
13749
  if (u && comment) {
13750
13750
  for (const n of shipguard.branchIssueNumbers(sess.branch)) {
13751
- const info = git.ghIssueView(repoAbs, n, ['comments']);
13751
+ const info = git.ghIssueView(repoAbs, n, ['comments', 'state', 'closedAt']);
13752
13752
  claimsByIssue[n] = info ? liveClaimComments(info.comments || []).map((c) => ({ ...c })) : null;
13753
+ if (info) issueStates[n] = { state: info.state || '', closedAt: info.closedAt || '' };
13753
13754
  }
13754
13755
  }
13755
- const v = shipHandoff.handoffVerdict({ url: opts.handoff, comment, branch: sess.branch, refs, claimsByIssue, syncChain });
13756
+ const v = shipHandoff.handoffVerdict({ url: opts.handoff, comment, branch: sess.branch, refs, claimsByIssue, syncChain,
13757
+ issueStates, refNumbers: refsFlagNumbers(opts.refs) });
13756
13758
  if (!v.ok) return v;
13757
13759
  sess.issues = v.issues;
13758
13760
  sess.handoff = v;
13761
+ // #647: the closed-before-hand-off issues — Refs in the squash, no 🚢/🔖 comment in the tail.
13762
+ sess.handoffRefs = new Set(v.refIssues || []);
13759
13763
  // #372: the implementer is the executor that handed off — name it, never leave it reading as the shipper.
13760
- const ex = (claimsByIssue[v.issues[0]] || []).find((c) => c.branch === sess.branch && c.login === v.executor.login);
13764
+ const claimed = v.issues.find((n) => !sess.handoffRefs.has(n));
13765
+ const ex = (claimsByIssue[claimed] || []).find((c) => c.branch === sess.branch && c.login === v.executor.login);
13761
13766
  if (ex && (ex.session || ex.sessionName)) sess.holder = { session: ex.session || '', sessionName: ex.sessionName || '' };
13762
13767
  return v;
13763
13768
  }
13764
13769
 
13770
+ /** The `--refs` value as issue numbers (`12,#13` → [12, 13]). */
13771
+ function refsFlagNumbers(refs) {
13772
+ return String(refs || '').split(',').map((x) => x.trim().replace(/^#/, '')).filter(Boolean).map(Number);
13773
+ }
13774
+
13775
+ /** #647: --refs ∪ the hand-off's closed-before issues — the set resolveCloseRefsSplit writes as Refs. */
13776
+ function shipRefsFromFlag(opts, sess) {
13777
+ const set = new Set(refsFlagNumbers(opts.refs));
13778
+ for (const n of (sess && sess.handoffRefs) || []) set.add(n);
13779
+ return set;
13780
+ }
13781
+
13765
13782
  function handoffDetail(v) {
13766
13783
  const head = v.syncMerges
13767
13784
  ? `${shortSha(v.handedOff)}, under ${v.syncMerges} clean base-sync merge${v.syncMerges === 1 ? '' : 's'} to the remote head ${shortSha(v.sha)} (#579)`
13768
13785
  : `${shortSha(v.sha)} = the remote head`;
13769
- return `${v.url} hands off ${head}; executor ${v.executor.login}'s live claim on ${v.issues.map((n) => '#' + n).join(' ')}`;
13786
+ const refd = new Set(v.refIssues || []);
13787
+ const closed = v.issues.filter((n) => !refd.has(n));
13788
+ const tail = refd.size ? `; ${[...refd].map((n) => '#' + n).join(' ')} closed before the hand-off — Refs only, no claim needed (#647)` : '';
13789
+ return `${v.url} hands off ${head}; executor ${v.executor.login}'s live claim on ${closed.map((n) => '#' + n).join(' ')}${tail}`;
13770
13790
  }
13771
13791
 
13772
13792
  /** The clause the 🚢 comment carries for a hand-off land — the executor still holds its claim. */
@@ -13922,9 +13942,7 @@ function cmdShip(argv) {
13922
13942
  // resolveCloseRefsSplit (#48, #74): --refs, the tracking label, and #74's close gate (an
13923
13943
  // unticked ## Plan checklist with no declared Remainder: #M). The claim is still released
13924
13944
  // unconditionally either way (B3, unchanged) — only Closes→Refs changes.
13925
- const refsFromFlag = new Set(
13926
- String(opts.refs || '').split(',').map((s) => s.trim().replace(/^#/, '')).filter(Boolean).map(Number),
13927
- );
13945
+ const refsFromFlag = shipRefsFromFlag(opts, sess); // #647: + the hand-off's closed-before issues
13928
13946
  for (const n of refsFromFlag) {
13929
13947
  if (!sess.issues.includes(n)) warn(`--refs #${n} is not a claimed issue on this branch — ignored`);
13930
13948
  }
@@ -14985,6 +15003,8 @@ function shipLandedTail({ repoAbs, sess, target, trunk, gh, sha, closeIssues, re
14985
15003
  const closedByShip = []; // the issues this ship actually closed — for (i2) below (#371)
14986
15004
  if (gh) {
14987
15005
  for (const num of sess.issues) {
15006
+ // #647: closed before the hand-off — history the branch name carries, not work this ship did.
15007
+ if (sess.handoffRefs && sess.handoffRefs.has(num)) continue;
14988
15008
  const isRef = refsSet.has(num);
14989
15009
  const info = git.ghIssueView(repoAbs, num, ['state', 'labels']);
14990
15010
  // A closed issue must have auto-closed; a ref'd (tracking) issue must have STAYED open. The
@@ -15834,6 +15854,14 @@ function shipBatchLand(repoAbs, doc, trunk, remote, base, existing, verdict, opt
15834
15854
  const sess = sessions[i];
15835
15855
  const closeIssues = sess.issues.filter((n) => closed.has(n));
15836
15856
  const refsIssues = sess.issues.filter((n) => !closed.has(n));
15857
+ // #647: a hand-off member's Refs issue that is already CLOSED is one the build exempted as
15858
+ // closed before the hand-off (an open one needed a claim) — history, so no 🔖 comment on it.
15859
+ if (sess.handoff && gh && refsIssues.length) {
15860
+ sess.handoffRefs = new Set(refsIssues.filter((n) => {
15861
+ const info = git.ghIssueView(repoAbs, n, ['state']);
15862
+ return info && String(info.state || '').toUpperCase() === 'CLOSED';
15863
+ }));
15864
+ }
15837
15865
  shipBatchMemberTail({ repoAbs, doc, trunk, sess, sha: shortSha(c.sha), closeIssues, refsIssues, suffix, opts, gh });
15838
15866
  });
15839
15867
 
@@ -35,6 +35,13 @@
35
35
  * 7. every issue the branch carries has a LIVE `🔒 Claimed` comment naming this branch, posted by
36
36
  * the hand-off's author before the hand-off — the claim really does live elsewhere, and the
37
37
  * party handing off is the party holding it.
38
+ * ONE exemption (#647): an issue the trailing run carries that was already CLOSED before the
39
+ * hand-off was posted only names history (a flaky test named after it) — it needs no claim, and
40
+ * goes back in `refIssues` so the squash says `Refs #A` and ship neither closes it nor posts a
41
+ * ship comment on it. Closed AFTER the hand-off is no exemption: closing an issue must never
42
+ * launder away the claim it lacked. An issue whose state could not be read is not exempt. Every
43
+ * OPEN issue still needs its claim, `--refs` or not — `--refs` decides Closes vs Refs, never who
44
+ * may land; and at least one issue must remain to close, or there is nothing handed off.
38
45
  *
39
46
  * Pure: every input is already read. tools/colab does the IO (`gh api`, `git fetch`, issue views).
40
47
  */
@@ -72,11 +79,15 @@ function namedShas(body) {
72
79
  * or null for an issue whose comments could not be read
73
80
  * syncChain — (#579) the first-parent ancestors of the remote head reached through pure
74
81
  * base-sync merges only, nearest first (syncMergeChain's output); [] or absent = none
75
- * Returns { ok: true, issues, sha, handedOff, syncMerges, executor, url } or { ok: false, reason, detail }.
82
+ * issueStates — (#647) { [n]: { state, closedAt } } per issue as read now; a missing entry is
83
+ * unknown and earns no exemption
84
+ * refNumbers — (#647) the --refs numbers; used only to say why they do not waive a claim
85
+ * Returns { ok: true, issues, refIssues, sha, handedOff, syncMerges, executor, url } or { ok: false, reason, detail }.
86
+ * `issues` is the whole trailing run; `refIssues` is its closed-before-the-hand-off subset (#647).
76
87
  * `sha` is what lands (the remote head); `handedOff` is the sha the hand-off named — equal to `sha`
77
88
  * unless the head was synced, in which case `syncMerges` counts the merges between them.
78
89
  */
79
- function handoffVerdict({ url, comment, branch, refs, claimsByIssue, syncChain }) {
90
+ function handoffVerdict({ url, comment, branch, refs, claimsByIssue, syncChain, issueStates, refNumbers }) {
80
91
  const no = (reason, detail) => ({ ok: false, reason, detail });
81
92
  const u = parseHandoffUrl(url);
82
93
  if (!u) return no('bad-url', `--handoff ${url || '(empty)'} is not an issue-comment URL (…/issues/<N>#issuecomment-<id>)`);
@@ -109,18 +120,36 @@ function handoffVerdict({ url, comment, branch, refs, claimsByIssue, syncChain }
109
120
  ? `the hand-off names ${shas.map((s) => s.slice(0, 8)).join(', ')}, but the remote ${branch} is at ${head.slice(0, 8)}${synced} — the branch moved after it was handed off by more than a clean merge of its base; ask its executor for a new hand-off`
110
121
  : `the hand-off names no head sha — it must say which commit it hands off (the remote is at ${head.slice(0, 8)})`);
111
122
  }
123
+ const refIssues = issues.filter((n) => closedBefore(issueStates && issueStates[n], comment.createdAt));
124
+ if (refIssues.length === issues.length) {
125
+ return no('nothing-to-close', `every issue ${branch} carries (${issues.map((n) => '#' + n).join(' ')}) was closed before the hand-off — there is no open, claimed issue for it to land`);
126
+ }
127
+ const waived = new Set((refNumbers || []).map(Number));
112
128
  let executor = null;
113
129
  for (const n of issues) {
130
+ if (refIssues.includes(n)) continue;
114
131
  const live = claimsByIssue ? claimsByIssue[n] : null;
115
132
  if (!live) return no('claims-unreadable', `#${n}'s comments could not be read — the executor's claim cannot be checked`);
116
133
  const match = live.find((c) => c.branch === branch && c.login === comment.login && c.at < comment.createdAt);
117
134
  if (!match) {
118
- return no('no-executor-claim', `#${n} has no live 🔒 claim on ${branch} by ${comment.login || 'the hand-off author'} posted before the hand-off — ` +
135
+ const st = issueStates && issueStates[n];
136
+ const why = st && String(st.state).toUpperCase() === 'CLOSED'
137
+ ? ` (it was closed at ${st.closedAt || 'an unreadable time'}, not before the hand-off at ${comment.createdAt} — only an issue already closed when the hand-off was posted needs no claim)`
138
+ : waived.has(n) ? ' (--refs keeps it open in the squash; it does not waive the claim of an open issue)' : '';
139
+ return no('no-executor-claim', `#${n} has no live 🔒 claim on ${branch} by ${comment.login || 'the hand-off author'} posted before the hand-off${why} — ` +
119
140
  'the hand-off must come from the party holding the claim');
120
141
  }
121
142
  if (!executor) executor = { login: match.login, host: match.host, machine: match.machine || '' };
122
143
  }
123
- return { ok: true, issues, sha: refs.remoteSha, handedOff: lineage[at], syncMerges: at, executor, url: String(url).trim() };
144
+ return { ok: true, issues, refIssues, sha: refs.remoteSha, handedOff: lineage[at], syncMerges: at, executor, url: String(url).trim() };
145
+ }
146
+
147
+ /** #647: was this issue CLOSED strictly before `at` (the hand-off's createdAt)? Unknown ⇒ false. */
148
+ function closedBefore(st, at) {
149
+ if (!st || String(st.state || '').toUpperCase() !== 'CLOSED') return false;
150
+ const c = Date.parse(st.closedAt || '');
151
+ const h = Date.parse(at || '');
152
+ return Number.isFinite(c) && Number.isFinite(h) && c < h;
124
153
  }
125
154
 
126
155
  /**
@@ -209,4 +238,4 @@ function parseHandoffTrailer(body) {
209
238
  return m ? { url: m[1], handedOff: m[2] } : null;
210
239
  }
211
240
 
212
- module.exports = { parseHandoffUrl, namesBranch, namedShas, handoffVerdict, handoffTrailer, syncMergeChain, parseBatchEntries, parseHandoffTrailer };
241
+ module.exports = { parseHandoffUrl, namesBranch, namedShas, handoffVerdict, closedBefore, handoffTrailer, syncMergeChain, parseBatchEntries, parseHandoffTrailer };