@futurelastic/colab-handbook 1.12.0-rc.28 → 1.12.0-rc.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +1 -1
- package/templates/README.md +2 -2
- package/templates/deploy-container-run.mjs +117 -17
- package/templates/deploy-container.yml +71 -13
- package/templates/release-auto.yml +13 -1
- package/tools/colab +13 -1
- package/tools/lib/git.js +73 -3
- package/tools/lib/ship-batch.js +3 -0
- package/tools/lib/stamp.js +6 -4
package/package.json
CHANGED
package/templates/README.md
CHANGED
|
@@ -24,8 +24,8 @@ Starting points you **copy into your own repo**. That is the entire model.
|
|
|
24
24
|
| `dist-refs.yml` | `.github/workflows/dist-refs.yml` | A **private** repo shipping a **compiled** tool, installed as `npx github:<org>/<repo>#vX.Y.Z` ([CONVENTIONS §6, *Distribution*](../CONVENTIONS.md#distribution--one-install-surface-npx-442)) | `workflow_call`, called from the same run that cut the release tag (a tag pushed with `GITHUB_TOKEN` triggers nothing else). Takes one build artifact per `<os>-<arch>` and pushes each as an orphan commit at `refs/tags/dist/vX.Y.Z/<os>-<arch>` with a `SHA256SUMS`. Refuses a public or unreadable repository (a public compiled tool ships npm platform packages instead), a version that is not an existing tag, and a missing platform — before pushing any. Never moves a dist ref that exists. |
|
|
25
25
|
| `npx-launcher.mjs` | anywhere in-repo (e.g. `bin/launch.mjs`), named by the root `package.json`'s `bin` | Pairs with `dist-refs.yml` | Node ≥ 18, zero dependencies, needs `git`. Reads the `#vX.Y.Z` npx installed, fetches that one dist ref with the same URL and credentials npx cloned with, verifies `SHA256SUMS`, installs to a stable per-user path (`$XDG_DATA_HOME`/`%LOCALAPPDATA%`), and execs it; `--print-path` prints that path for a service's `init`. `*.mjs`, so `colab template npx-launcher.mjs --dest <path>` copies it stamped, `--dest` required. One edit point: `BIN` (defaults to the package name without its scope). |
|
|
26
26
|
| `deploy-xserver.yml` | `.github/workflows/deploy-xserver.yml` | PHP-framework + Vite apps shipped to **shared hosting over SSH** (no root, no Docker): build on a runner, rsync, migrate on the server | Derived from three independently-written copies. Resolves Node the same way the CI templates do — all three hardcoded it, and one shipped on a different major than its CI built on. Migrates **production**; keeps a **mandatory** smoke test. Does **not** change your tier. |
|
|
27
|
-
| `deploy-container.yml` | `.github/workflows/deploy-container.yml` | A **container** app on a host whose platform has an API (Portainer today), `deploy: tag` ([CONVENTIONS §6, *A container deploy*](../CONVENTIONS.md#6-releases), #452) | One contract on every host: build each listed image **once** per final tag (`vX.Y.Z` + commit sha), run the pre-deploy hook, have the platform run exactly that tag (all images, one call), green only on a **verified running version** (stack settled, commit and container images checked, then `release.health-url` reporting `X.Y.Z`), roll back to the previous final on a failure after the platform accepted. Triggers: a final tag push (route `deploy-tag`), `workflow_dispatch` with a tag (a manual deploy or rollback), and `workflow_call` — route `deploy-tag-fast` reaches it from `release-auto.yml`'s commented `deploy` job, like `dist-refs.yml` from its release run. Needs its two scripts below. One secret: `PORTAINER_API_KEY`, a non-admin key owning only this app's stack. **Every redeploy recreates the containers.** Does **not** change your exposure. |
|
|
28
|
-
| `deploy-container-run.mjs` | `.github/deploy/deploy-container-run.mjs` | Pairs with `deploy-container.yml` | The driver: builds, runs the hook, calls the adapter, verifies, rolls back, records. Node ≥ 18, zero dependencies; shells out to `git`, `docker`, `gh`. `colab template deploy-container-run.mjs --dest <path>`, `--dest` required, stamped on line 2. |
|
|
27
|
+
| `deploy-container.yml` | `.github/workflows/deploy-container.yml` | A **container** app on a host whose platform has an API (Portainer today), `deploy: tag` ([CONVENTIONS §6, *A container deploy*](../CONVENTIONS.md#6-releases), #452) | One contract on every host: build each listed image **once** per final tag (`vX.Y.Z` + commit sha), run the pre-deploy hook, have the platform run exactly that tag (all images, one call), green only on a **verified running version** (stack settled, commit and container images checked, then `release.health-url` reporting `X.Y.Z`), roll back to the previous final on a failure after the platform accepted. **Two jobs (#460):** `publish` builds and pushes on **every** final, ungated — so a repo not yet cut over still gets each final's image in the registry; `deploy` `needs:` it, never builds (a missing image fails before the platform is touched), and is the one job you gate on "is the platform switched on". Images and build args sit in the workflow-level `env`, written once for both jobs. Triggers: a final tag push (route `deploy-tag`), `workflow_dispatch` with a tag (a manual deploy or rollback), and `workflow_call` — route `deploy-tag-fast` reaches it from `release-auto.yml`'s commented `deploy` job, like `dist-refs.yml` from its release run. Needs its two scripts below. One required secret: `PORTAINER_API_KEY`, a non-admin key owning only this app's stack; optional `HEALTH_AUTH_HEADER` (`Name: value`) or `HEALTH_BASIC_AUTH` (`user:password`) when the version URL sits behind auth — masked, never printed. **Every redeploy recreates the containers.** Does **not** change your exposure. |
|
|
28
|
+
| `deploy-container-run.mjs` | `.github/deploy/deploy-container-run.mjs` | Pairs with `deploy-container.yml` | The driver: builds, runs the hook, calls the adapter, verifies, rolls back, records. `--publish-only` = build and push only (no platform config, no version URL needed); `--deploy-only` = everything after, refusing a tag whose image is not in the registry. **Build args:** `DEPLOY_BUILD_ARGS`, one per line — `KEY=VALUE` for every image or `<image-ref> KEY=VALUE` for one; `${TAG}` (`vX.Y.Z`), `${VERSION}` (`X.Y.Z`) and `${SHA}` are filled in by the driver, so a static site can bake public config and its version into a served file. They end up in the image — public values only, never a secret. Node ≥ 18, zero dependencies; shells out to `git`, `docker`, `gh`. `colab template deploy-container-run.mjs --dest <path>`, `--dest` required, stamped on line 2. |
|
|
29
29
|
| `deploy-adapter-portainer.mjs` | `.github/deploy/deploy-adapter-portainer.mjs` — **beside** the driver, which loads `./deploy-adapter-<DEPLOY_ADAPTER>.mjs` | Pairs with `deploy-container.yml` | The Portainer adapter: git stacks redeploy `refs/tags/vX.Y.Z`, file stacks update with the compose file; the stack's **whole** `Env` is sent back with `IMAGE_TAG` replaced; 409 waited out and retried, any other error a hard failure (nothing changed). Another platform = another file exporting the same five names (`name`, `configFromEnv`, `snapshot`, `deploy`, `restore`). |
|
|
30
30
|
| `repo-CLAUDE-block.md` | *paste into* `CLAUDE.md` | Every adopting repo | The discovery hook — how an agent finds the handbook at all. |
|
|
31
31
|
| `gotchas-d-README.md` | `docs/gotchas.d/README.md` | Any repo starting a `docs/gotchas.d/` directory ([`code-wrap`](../skills/code-wrap/SKILL.md)'s A2 step) | Naming rule (`<issue>-<slug>.md`) and the don't-copy-back rule against an existing `docs/gotchas.md`. Optional — writing the first gotcha entry does not require copying this in first. |
|
|
@@ -19,15 +19,30 @@
|
|
|
19
19
|
// CONFIGURATION (env; the workflow's one env block):
|
|
20
20
|
// DEPLOY_ADAPTER portainer (default) — loads ./deploy-adapter-<name>.mjs beside this file
|
|
21
21
|
// DEPLOY_IMAGES one image per line: `<image-ref> <build-context> [<dockerfile>]`
|
|
22
|
+
// DEPLOY_BUILD_ARGS optional, one build arg per line: `KEY=VALUE` for every image, or
|
|
23
|
+
// `<image-ref> KEY=VALUE` for one. A value may use ${TAG} (vX.Y.Z),
|
|
24
|
+
// ${VERSION} (X.Y.Z) and ${SHA} (the tag's commit) — substituted here,
|
|
25
|
+
// not by a shell. Build args are baked into the image: never a secret.
|
|
22
26
|
// PRE_DEPLOY optional shell command; gets TAG and PREVIOUS_TAG in its env
|
|
23
27
|
// HEALTH_URL the version URL; else --health-url; else release.health-url in
|
|
24
28
|
// .github/project.yml
|
|
29
|
+
// HEALTH_AUTH_HEADER optional, `Name: value` sent with every version-URL request — or
|
|
30
|
+
// HEALTH_BASIC_AUTH optional, `user:password` sent as HTTP Basic. From a secret; masked
|
|
31
|
+
// in the log and never printed. Set at most one.
|
|
25
32
|
// VERIFY_TIMEOUT_SECONDS 600 · POLL_INTERVAL_MS 5000 · DEPLOY_TIMEOUT_SECONDS 600 ·
|
|
26
33
|
// BUSY_TIMEOUT_SECONDS 600
|
|
27
34
|
// GH_TOKEN, GITHUB_REPOSITORY — to record on the release issue (optional)
|
|
28
35
|
//
|
|
29
36
|
// Usage: node deploy-container-run.mjs --tag vX.Y.Z [--health-url URL] [--tag-sha SHA]
|
|
30
|
-
//
|
|
37
|
+
// [--publish-only | --deploy-only]
|
|
38
|
+
// --publish-only step 1 alone: build and push every image, then stop. Needs no platform
|
|
39
|
+
// configuration and no version URL — so a final publishes its images even
|
|
40
|
+
// before the platform is switched on (the workflow's ungated `publish` job).
|
|
41
|
+
// --deploy-only steps 2-5 alone: every image must already be in the registry for the tag
|
|
42
|
+
// (the `publish` job pushed it); a missing one fails before the platform is
|
|
43
|
+
// touched, never a rebuild here.
|
|
44
|
+
// Exit: 0 = running vX.Y.Z, verified (or, --publish-only, every image pushed) · 1 = not deployed
|
|
45
|
+
// (rolled back where something changed).
|
|
31
46
|
|
|
32
47
|
import { spawnSync } from 'node:child_process';
|
|
33
48
|
import fs from 'node:fs';
|
|
@@ -49,6 +64,60 @@ export function parseImages(text) {
|
|
|
49
64
|
return out;
|
|
50
65
|
}
|
|
51
66
|
|
|
67
|
+
/**
|
|
68
|
+
* DEPLOY_BUILD_ARGS → per-image `--build-arg` lists. `KEY=VALUE` applies to every image,
|
|
69
|
+
* `<image-ref> KEY=VALUE` to that one (a later line for the same key wins, per-image over shared).
|
|
70
|
+
* ${TAG} / ${VERSION} / ${SHA} are substituted; any other ${NAME} is a typo and refused.
|
|
71
|
+
*/
|
|
72
|
+
export function parseBuildArgs(text, images, release) {
|
|
73
|
+
const vars = { TAG: release.tag, VERSION: String(release.tag).replace(/^v/, ''), SHA: release.sha };
|
|
74
|
+
const refs = new Set(images.map((i) => i.ref));
|
|
75
|
+
const shared = new Map();
|
|
76
|
+
const per = new Map(images.map((i) => [i.ref, new Map()]));
|
|
77
|
+
for (const raw of String(text || '').split('\n')) {
|
|
78
|
+
const line = raw.trim();
|
|
79
|
+
if (!line || line.startsWith('#')) continue;
|
|
80
|
+
let target = null; let pair = line;
|
|
81
|
+
const first = line.split(/\s+/, 1)[0];
|
|
82
|
+
if (!first.includes('=')) {
|
|
83
|
+
target = first;
|
|
84
|
+
pair = line.slice(first.length).trim();
|
|
85
|
+
if (!refs.has(target)) throw new Error(`DEPLOY_BUILD_ARGS line "${line}" names ${target}, which DEPLOY_IMAGES does not list`);
|
|
86
|
+
}
|
|
87
|
+
const m = /^([A-Za-z_][A-Za-z0-9_]*)=(.*)$/.exec(pair);
|
|
88
|
+
if (!m) throw new Error(`DEPLOY_BUILD_ARGS line "${line}" needs KEY=VALUE or <image-ref> KEY=VALUE`);
|
|
89
|
+
const value = m[2].replace(/\$\{([A-Za-z_][A-Za-z0-9_]*)\}/g, (_, k) => {
|
|
90
|
+
if (!(k in vars)) throw new Error(`DEPLOY_BUILD_ARGS: \${${k}} is not one of \${TAG}, \${VERSION}, \${SHA}`);
|
|
91
|
+
return vars[k];
|
|
92
|
+
});
|
|
93
|
+
(target ? per.get(target) : shared).set(m[1], value);
|
|
94
|
+
}
|
|
95
|
+
const out = {};
|
|
96
|
+
for (const img of images) {
|
|
97
|
+
const merged = new Map([...shared, ...per.get(img.ref)]);
|
|
98
|
+
out[img.ref] = [...merged].flatMap(([k, v]) => ['--build-arg', `${k}=${v}`]);
|
|
99
|
+
}
|
|
100
|
+
return out;
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
/** HEALTH_AUTH_HEADER / HEALTH_BASIC_AUTH → { headers, secret }; the secret is what to mask. */
|
|
104
|
+
export function healthAuth(env) {
|
|
105
|
+
const header = (env.HEALTH_AUTH_HEADER || '').trim();
|
|
106
|
+
const basic = env.HEALTH_BASIC_AUTH || '';
|
|
107
|
+
if (header && basic) throw new Error('set HEALTH_AUTH_HEADER or HEALTH_BASIC_AUTH, not both');
|
|
108
|
+
if (basic) {
|
|
109
|
+
if (!basic.includes(':')) throw new Error('HEALTH_BASIC_AUTH must be user:password');
|
|
110
|
+
const token = Buffer.from(basic, 'utf8').toString('base64');
|
|
111
|
+
return { headers: { Authorization: `Basic ${token}` }, secrets: [basic, token] };
|
|
112
|
+
}
|
|
113
|
+
if (header) {
|
|
114
|
+
const m = /^([A-Za-z0-9-]+):\s*(.+)$/.exec(header);
|
|
115
|
+
if (!m) throw new Error('HEALTH_AUTH_HEADER must be "Name: value"');
|
|
116
|
+
return { headers: { [m[1]]: m[2] }, secrets: [m[2]] };
|
|
117
|
+
}
|
|
118
|
+
return { headers: {}, secrets: [] };
|
|
119
|
+
}
|
|
120
|
+
|
|
52
121
|
/** Does `body` report `version` — as a whole version, so 1.2.1 never matches 1.2.10 or 11.2.1? */
|
|
53
122
|
export function reportsVersion(body, version) {
|
|
54
123
|
const v = String(version).replace(/^v/, '').replace(/\./g, '\\.');
|
|
@@ -96,6 +165,7 @@ function defaultIo(env) {
|
|
|
96
165
|
sleep: (ms) => new Promise((r) => setTimeout(r, ms)),
|
|
97
166
|
now: () => Date.now(),
|
|
98
167
|
log: (m) => console.log(m),
|
|
168
|
+
mask: (v) => { if (env.GITHUB_ACTIONS) console.log(`::add-mask::${v}`); },
|
|
99
169
|
error: (m) => console.log(`::error::${m}`),
|
|
100
170
|
sh,
|
|
101
171
|
summary: (line) => { if (env.GITHUB_STEP_SUMMARY) fs.appendFileSync(env.GITHUB_STEP_SUMMARY, `${line}\n`); },
|
|
@@ -103,12 +173,12 @@ function defaultIo(env) {
|
|
|
103
173
|
};
|
|
104
174
|
}
|
|
105
175
|
|
|
106
|
-
async function waitForVersion(url, version, io, timeoutS) {
|
|
176
|
+
async function waitForVersion(url, version, io, timeoutS, auth = {}) {
|
|
107
177
|
const until = io.now() + timeoutS * 1000;
|
|
108
178
|
let last = '';
|
|
109
179
|
for (;;) {
|
|
110
180
|
try {
|
|
111
|
-
const res = await io.fetch(url, { headers: { 'Cache-Control': 'no-cache' } });
|
|
181
|
+
const res = await io.fetch(url, { headers: { 'Cache-Control': 'no-cache', ...auth } });
|
|
112
182
|
last = await res.text();
|
|
113
183
|
if (res.ok && reportsVersion(last, version)) return true;
|
|
114
184
|
} catch (e) { last = String(e && e.message); }
|
|
@@ -128,14 +198,14 @@ function record(io, tag, line) {
|
|
|
128
198
|
if (c.status !== 0) io.log(`could not comment on release issue #${n}: ${c.stderr.trim()}`);
|
|
129
199
|
}
|
|
130
200
|
|
|
131
|
-
function buildOnce(images, tag, sha, io) {
|
|
201
|
+
function buildOnce(images, tag, sha, io, buildArgs = {}) {
|
|
132
202
|
for (const img of images) {
|
|
133
203
|
if (io.sh('docker', ['manifest', 'inspect', `${img.ref}:${tag}`]).status === 0) {
|
|
134
204
|
io.log(`${img.ref}:${tag} is already pushed — built once per tag, not rebuilt`);
|
|
135
205
|
continue;
|
|
136
206
|
}
|
|
137
207
|
const steps = [
|
|
138
|
-
['build', '-f', img.dockerfile, '-t', `${img.ref}:${tag}`, '-t', `${img.ref}:${sha}`, img.context],
|
|
208
|
+
['build', '-f', img.dockerfile, ...(buildArgs[img.ref] || []), '-t', `${img.ref}:${tag}`, '-t', `${img.ref}:${sha}`, img.context],
|
|
139
209
|
['push', `${img.ref}:${tag}`],
|
|
140
210
|
['push', `${img.ref}:${sha}`],
|
|
141
211
|
];
|
|
@@ -151,27 +221,55 @@ export async function run(opts, io) {
|
|
|
151
221
|
const env = io.env;
|
|
152
222
|
const tag = opts.tag;
|
|
153
223
|
const fail = (m) => { io.error(m); return 1; };
|
|
224
|
+
if (opts.publishOnly && opts.deployOnly) return fail('--publish-only and --deploy-only together would do nothing — pass at most one');
|
|
154
225
|
if (!FINAL_TAG.test(tag || '')) return fail(`"${tag}" is not a final tag vX.Y.Z — this deploys finals only`);
|
|
155
226
|
const version = tag.slice(1);
|
|
156
|
-
const healthUrl = opts.healthUrl || env.HEALTH_URL || readHealthUrl(io.readFile('.github/project.yml'));
|
|
157
|
-
if (!healthUrl) return fail('no version URL — set release.health-url in .github/project.yml (or HEALTH_URL); a deploy is green only on a verified running version');
|
|
227
|
+
const healthUrl = opts.publishOnly ? null : opts.healthUrl || env.HEALTH_URL || readHealthUrl(io.readFile('.github/project.yml'));
|
|
228
|
+
if (!opts.publishOnly && !healthUrl) return fail('no version URL — set release.health-url in .github/project.yml (or HEALTH_URL); a deploy is green only on a verified running version');
|
|
158
229
|
|
|
159
|
-
let images; let adapter; let cfg;
|
|
230
|
+
let images; let adapter; let cfg; let auth = {};
|
|
160
231
|
try {
|
|
161
232
|
images = parseImages(env.DEPLOY_IMAGES);
|
|
162
233
|
if (!images.length) throw new Error('DEPLOY_IMAGES lists no image — list every image this stack runs');
|
|
163
|
-
|
|
164
|
-
|
|
234
|
+
if (!opts.publishOnly) {
|
|
235
|
+
const a = healthAuth(env);
|
|
236
|
+
for (const s of a.secrets) if (io.mask) io.mask(s);
|
|
237
|
+
auth = a.headers;
|
|
238
|
+
adapter = await loadAdapter(env.DEPLOY_ADAPTER || 'portainer');
|
|
239
|
+
cfg = adapter.configFromEnv(env);
|
|
240
|
+
}
|
|
165
241
|
} catch (e) { return fail(e.message); }
|
|
166
242
|
|
|
167
243
|
const tagSha = opts.tagSha || io.sh('git', ['rev-parse', `${tag}^{commit}`]).stdout.trim();
|
|
168
244
|
if (!/^[0-9a-f]{40}$/.test(tagSha)) return fail(`could not resolve ${tag} to a commit`);
|
|
169
|
-
const tags = io.sh('git', ['tag', '--list', 'v*']).stdout.split('\n').map((t) => t.trim()).filter(Boolean);
|
|
170
|
-
const previous = previousFinalTag(tags, tag);
|
|
171
245
|
const at = () => new Date(io.now()).toISOString();
|
|
172
246
|
|
|
173
|
-
// 1. Build once.
|
|
174
|
-
|
|
247
|
+
// 1. Build once — or, --deploy-only, check the publish job left every image in the registry.
|
|
248
|
+
if (opts.deployOnly) {
|
|
249
|
+
const missing = images.filter((img) => io.sh('docker', ['manifest', 'inspect', `${img.ref}:${tag}`]).status !== 0).map((img) => `${img.ref}:${tag}`);
|
|
250
|
+
if (missing.length) {
|
|
251
|
+
const m = `not in the registry: ${missing.join(', ')} — the publish job pushes every image before the deploy; nothing deployed`;
|
|
252
|
+
record(io, tag, `deploy of ${tag} failed at ${at()}: ${m}`);
|
|
253
|
+
return fail(m);
|
|
254
|
+
}
|
|
255
|
+
} else {
|
|
256
|
+
let buildArgs;
|
|
257
|
+
try { buildArgs = parseBuildArgs(env.DEPLOY_BUILD_ARGS, images, { tag, sha: tagSha }); } catch (e) { return fail(e.message); }
|
|
258
|
+
try { buildOnce(images, tag, tagSha, io, buildArgs); } catch (e) {
|
|
259
|
+
const what = opts.publishOnly ? 'publish' : 'deploy';
|
|
260
|
+
if (opts.publishOnly) io.summary(`${what} of ${tag} failed at ${at()}: ${e.message}`);
|
|
261
|
+
else record(io, tag, `${what} of ${tag} failed at ${at()}: ${e.message} — nothing deployed`);
|
|
262
|
+
return fail(e.message);
|
|
263
|
+
}
|
|
264
|
+
if (opts.publishOnly) {
|
|
265
|
+
const line = `published ${tag} at ${at()} — ${images.map((i) => `${i.ref}:${tag}`).join(', ')}`;
|
|
266
|
+
io.log(line);
|
|
267
|
+
io.summary(line);
|
|
268
|
+
return 0;
|
|
269
|
+
}
|
|
270
|
+
}
|
|
271
|
+
const tags = io.sh('git', ['tag', '--list', 'v*']).stdout.split('\n').map((t) => t.trim()).filter(Boolean);
|
|
272
|
+
const previous = previousFinalTag(tags, tag);
|
|
175
273
|
|
|
176
274
|
// 2. Pre-deploy hook — before anything on the platform changes.
|
|
177
275
|
if (env.PRE_DEPLOY && env.PRE_DEPLOY.trim()) {
|
|
@@ -195,7 +293,7 @@ export async function run(opts, io) {
|
|
|
195
293
|
let reason = null;
|
|
196
294
|
try {
|
|
197
295
|
await adapter.deploy(cfg, { tag, tagSha, images: refs, fileContent }, io, snap);
|
|
198
|
-
if (!(await waitForVersion(healthUrl, version, io, io.timing.verifyTimeoutS))) {
|
|
296
|
+
if (!(await waitForVersion(healthUrl, version, io, io.timing.verifyTimeoutS, auth))) {
|
|
199
297
|
throw Object.assign(new Error(`${healthUrl} did not report ${version} within ${io.timing.verifyTimeoutS}s`), { changed: true });
|
|
200
298
|
}
|
|
201
299
|
} catch (e) {
|
|
@@ -226,7 +324,7 @@ export async function run(opts, io) {
|
|
|
226
324
|
const prevSha = io.sh('git', ['rev-parse', `${previous}^{commit}`]).stdout.trim();
|
|
227
325
|
await adapter.deploy(cfg, { tag: previous, tagSha: prevSha, images: refs, fileContent: snap.fileContent }, io, snap);
|
|
228
326
|
}
|
|
229
|
-
if (back) verified = await waitForVersion(healthUrl, back.slice(1), io, io.timing.verifyTimeoutS);
|
|
327
|
+
if (back) verified = await waitForVersion(healthUrl, back.slice(1), io, io.timing.verifyTimeoutS, auth);
|
|
230
328
|
} catch (e) {
|
|
231
329
|
io.log(`rollback to ${back} failed: ${e.message}`);
|
|
232
330
|
}
|
|
@@ -237,10 +335,12 @@ export async function run(opts, io) {
|
|
|
237
335
|
return fail(line);
|
|
238
336
|
}
|
|
239
337
|
|
|
240
|
-
function parseArgs(argv) {
|
|
338
|
+
export function parseArgs(argv) {
|
|
241
339
|
const o = {};
|
|
242
340
|
for (let i = 0; i < argv.length; i++) {
|
|
243
341
|
const a = argv[i];
|
|
342
|
+
if (a === '--publish-only') { o.publishOnly = true; continue; }
|
|
343
|
+
if (a === '--deploy-only') { o.deployOnly = true; continue; }
|
|
244
344
|
const m = /^--([a-z-]+)(?:=(.*))?$/.exec(a);
|
|
245
345
|
if (!m) continue;
|
|
246
346
|
const v = m[2] !== undefined ? m[2] : argv[++i];
|
|
@@ -43,7 +43,15 @@
|
|
|
43
43
|
# that owns only this app's stack. Never an admin key: a leaked deploy key must reach one app.
|
|
44
44
|
# - `secrets.REGISTRY_PASSWORD` — optional; the registry login falls back to the job's
|
|
45
45
|
# `github.token`, which is enough for GHCR with `packages: write`.
|
|
46
|
-
#
|
|
46
|
+
# - `secrets.HEALTH_AUTH_HEADER` (`Name: value`) or `secrets.HEALTH_BASIC_AUTH` (`user:password`)
|
|
47
|
+
# — optional, only when the version URL sits behind auth. Masked; never printed.
|
|
48
|
+
# Everything else is configuration: the workflow-level `env` (images, build args — both jobs read
|
|
49
|
+
# them) and the env block of the deploy job's last step (the platform).
|
|
50
|
+
#
|
|
51
|
+
# TWO JOBS (#460). `publish` builds and pushes every image on EVERY final, whatever the platform's
|
|
52
|
+
# state — so a repo not yet cut over still has each final's image in the registry, ready to stage.
|
|
53
|
+
# `deploy` needs it and only talks to the platform; it never builds (a missing image fails before
|
|
54
|
+
# the platform is touched). Gate `deploy`, never `publish`, on "is the platform switched on".
|
|
47
55
|
#
|
|
48
56
|
# TOPOLOGY. Every server runs its own Portainer — there is no central panel and no Portainer
|
|
49
57
|
# Agent. So the per-app configuration is THAT host's Portainer URL + endpointId + stack id + key.
|
|
@@ -84,6 +92,10 @@ on:
|
|
|
84
92
|
required: false
|
|
85
93
|
REGISTRY_PASSWORD:
|
|
86
94
|
required: false
|
|
95
|
+
HEALTH_AUTH_HEADER:
|
|
96
|
+
required: false
|
|
97
|
+
HEALTH_BASIC_AUTH:
|
|
98
|
+
required: false
|
|
87
99
|
|
|
88
100
|
# One deploy at a time per repo, never cancelled half-way: a cancelled deploy is a stack in an
|
|
89
101
|
# unknown state. A fixed group, because under workflow_call `github.workflow` is the CALLER's name.
|
|
@@ -94,16 +106,28 @@ concurrency:
|
|
|
94
106
|
permissions:
|
|
95
107
|
contents: read
|
|
96
108
|
|
|
109
|
+
# Read by both jobs. Workflow-level, so the list is written once: `publish` builds exactly what
|
|
110
|
+
# `deploy` then checks is in the registry.
|
|
111
|
+
env:
|
|
112
|
+
# EDIT: every image this stack runs, one per line: <image-ref> <build-context> [<dockerfile>].
|
|
113
|
+
DEPLOY_IMAGES: |
|
|
114
|
+
ghcr.io/${{ github.repository }} .
|
|
115
|
+
# EDIT: optional build args, one per line — `KEY=VALUE` for every image, `<image-ref> KEY=VALUE`
|
|
116
|
+
# for one. ${TAG} (vX.Y.Z), ${VERSION} (X.Y.Z) and ${SHA} (the tag's commit) are filled in by the
|
|
117
|
+
# driver. Baked into the image, so public values only (`vars.*`), never a secret.
|
|
118
|
+
DEPLOY_BUILD_ARGS: ""
|
|
119
|
+
|
|
97
120
|
jobs:
|
|
98
|
-
|
|
99
|
-
name:
|
|
100
|
-
|
|
101
|
-
|
|
121
|
+
publish:
|
|
122
|
+
name: Publish the final's images
|
|
123
|
+
# Never gated on the platform: a final publishes its images before cutover too.
|
|
124
|
+
runs-on: ubuntu-latest # EDIT: a runner with docker.
|
|
102
125
|
permissions:
|
|
103
126
|
contents: read # check out the tag
|
|
104
127
|
packages: write # push the images (GHCR); drop it for another registry
|
|
105
|
-
issues: write # record the outcome on the release issue
|
|
106
128
|
timeout-minutes: 45
|
|
129
|
+
outputs:
|
|
130
|
+
tag: ${{ steps.tag.outputs.tag }}
|
|
107
131
|
steps:
|
|
108
132
|
- name: Resolve the tag to deploy
|
|
109
133
|
id: tag
|
|
@@ -120,7 +144,42 @@ jobs:
|
|
|
120
144
|
|
|
121
145
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
122
146
|
with:
|
|
123
|
-
ref: refs/tags/${{ steps.tag.outputs.tag }}
|
|
147
|
+
ref: refs/tags/${{ steps.tag.outputs.tag }} # the tag ref resolves to the commit it builds
|
|
148
|
+
persist-credentials: false # nothing here pushes to git
|
|
149
|
+
|
|
150
|
+
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6
|
|
151
|
+
with:
|
|
152
|
+
node-version: "22"
|
|
153
|
+
|
|
154
|
+
- name: Log in to the image registry
|
|
155
|
+
env:
|
|
156
|
+
REGISTRY: ghcr.io # EDIT: your registry host
|
|
157
|
+
REGISTRY_USER: ${{ github.actor }} # EDIT: the registry user, when not GHCR
|
|
158
|
+
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD || github.token }}
|
|
159
|
+
run: |
|
|
160
|
+
set -euo pipefail
|
|
161
|
+
printf '%s' "$REGISTRY_PASSWORD" | docker login "$REGISTRY" -u "$REGISTRY_USER" --password-stdin
|
|
162
|
+
|
|
163
|
+
- name: Build once and push (deploy-container-run --publish-only)
|
|
164
|
+
env:
|
|
165
|
+
TAG: ${{ steps.tag.outputs.tag }}
|
|
166
|
+
run: node .github/deploy/deploy-container-run.mjs --tag "$TAG" --publish-only
|
|
167
|
+
|
|
168
|
+
deploy:
|
|
169
|
+
name: Deploy the final
|
|
170
|
+
needs: publish
|
|
171
|
+
# EDIT: gate the platform half here, e.g. `if: vars.PLATFORM_LIVE == 'true'` — never `publish`.
|
|
172
|
+
runs-on: ubuntu-latest # EDIT: a runner that reaches your platform (LAN or VPN) and has docker.
|
|
173
|
+
environment: production # EDIT: the GitHub environment holding PORTAINER_API_KEY, or remove.
|
|
174
|
+
permissions:
|
|
175
|
+
contents: read # check out the tag
|
|
176
|
+
packages: read # confirm the images are in the registry (GHCR); drop it for another registry
|
|
177
|
+
issues: write # record the outcome on the release issue
|
|
178
|
+
timeout-minutes: 45
|
|
179
|
+
steps:
|
|
180
|
+
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
181
|
+
with:
|
|
182
|
+
ref: refs/tags/${{ needs.publish.outputs.tag }}
|
|
124
183
|
fetch-depth: 0 # every tag: the rollback target is the previous final
|
|
125
184
|
persist-credentials: false # nothing here pushes to git
|
|
126
185
|
|
|
@@ -137,12 +196,9 @@ jobs:
|
|
|
137
196
|
set -euo pipefail
|
|
138
197
|
printf '%s' "$REGISTRY_PASSWORD" | docker login "$REGISTRY" -u "$REGISTRY_USER" --password-stdin
|
|
139
198
|
|
|
140
|
-
- name:
|
|
199
|
+
- name: Run exactly this tag, verify (deploy-container-run --deploy-only)
|
|
141
200
|
env:
|
|
142
201
|
DEPLOY_ADAPTER: portainer
|
|
143
|
-
# EDIT: every image this stack runs, one per line: <image-ref> <build-context> [<dockerfile>].
|
|
144
|
-
DEPLOY_IMAGES: |
|
|
145
|
-
ghcr.io/${{ github.repository }} .
|
|
146
202
|
# EDIT: optional — runs before the platform is called, with TAG and PREVIOUS_TAG set; a
|
|
147
203
|
# non-zero exit stops the deploy before anything changes (a database snapshot, say).
|
|
148
204
|
PRE_DEPLOY: ""
|
|
@@ -153,9 +209,11 @@ jobs:
|
|
|
153
209
|
PORTAINER_COMPOSE_FILE: docker-compose.yml # file stacks only: sent as the stack content
|
|
154
210
|
PORTAINER_API_KEY: ${{ secrets.PORTAINER_API_KEY }}
|
|
155
211
|
HEALTH_URL: ${{ inputs.health-url }} # empty → release.health-url in .github/project.yml
|
|
212
|
+
HEALTH_AUTH_HEADER: ${{ secrets.HEALTH_AUTH_HEADER }} # optional: `Name: value`
|
|
213
|
+
HEALTH_BASIC_AUTH: ${{ secrets.HEALTH_BASIC_AUTH }} # optional: `user:password`
|
|
156
214
|
VERIFY_TIMEOUT_SECONDS: "600" # EDIT: how long the new version may take to report itself
|
|
157
215
|
DEPLOY_TIMEOUT_SECONDS: "600"
|
|
158
216
|
BUSY_TIMEOUT_SECONDS: "600"
|
|
159
217
|
GH_TOKEN: ${{ github.token }}
|
|
160
|
-
TAG: ${{
|
|
161
|
-
run: node .github/deploy/deploy-container-run.mjs --tag "$TAG"
|
|
218
|
+
TAG: ${{ needs.publish.outputs.tag }}
|
|
219
|
+
run: node .github/deploy/deploy-container-run.mjs --tag "$TAG" --deploy-only
|
|
@@ -403,7 +403,19 @@ jobs:
|
|
|
403
403
|
*-*) KIND=(--prerelease --latest=false) ;;
|
|
404
404
|
*) KIND=(--latest) ;;
|
|
405
405
|
esac
|
|
406
|
-
|
|
406
|
+
# The view above is a check, not a lock: `colab release cut` publishes a candidate's
|
|
407
|
+
# pre-release itself, and the API can still answer 404 for it seconds later (#462 — a
|
|
408
|
+
# trunk went red on exactly that 422). So a create refused because the Release already
|
|
409
|
+
# exists is the skip case too, decided from the create's own answer.
|
|
410
|
+
ERR="$RUNNER_TEMP/create-$TAG.err"
|
|
411
|
+
if ! gh release create "$TAG" --verify-tag --title "$TAG" --notes-file "$NOTES" --generate-notes "${KIND[@]}" 2> "$ERR"; then
|
|
412
|
+
cat "$ERR" >&2
|
|
413
|
+
if grep -qiE 'already[ _]exists' "$ERR"; then
|
|
414
|
+
echo "$TAG already has a Release (published while this step ran) — skipping"
|
|
415
|
+
continue
|
|
416
|
+
fi
|
|
417
|
+
exit 1
|
|
418
|
+
fi
|
|
407
419
|
echo "Published $TAG" | tee -a "$GITHUB_STEP_SUMMARY"
|
|
408
420
|
done
|
|
409
421
|
|
package/tools/colab
CHANGED
|
@@ -10179,7 +10179,19 @@ function shipCiCheck(repoAbs, gh, target, trunk, grantThunk, cureThunk) {
|
|
|
10179
10179
|
*/
|
|
10180
10180
|
function classifyCiRun(repoAbs, run, at) {
|
|
10181
10181
|
const v = classifyCiRunRaw(repoAbs, run, at);
|
|
10182
|
-
return { ...v, detail: v.detail + excludedRunsNote(run) };
|
|
10182
|
+
return { ...v, detail: v.detail + excludedRunsNote(run) + supersededRunsNote(run) };
|
|
10183
|
+
}
|
|
10184
|
+
|
|
10185
|
+
/**
|
|
10186
|
+
* #461: earlier runs of the same workflow at this sha that a newer run of it superseded — set aside
|
|
10187
|
+
* by git.summarizeRunsForCommit before the verdict, and named here so the reader sees what the
|
|
10188
|
+
* verdict did not judge. '' when none.
|
|
10189
|
+
*/
|
|
10190
|
+
function supersededRunsNote(run) {
|
|
10191
|
+
const sup = run && Array.isArray(run.superseded) ? run.superseded : [];
|
|
10192
|
+
if (!sup.length) return '';
|
|
10193
|
+
const names = sup.map((x) => `${x.workflowName || 'unnamed'} (${x.conclusion || x.status || '?'}${x.databaseId ? `, run ${x.databaseId}` : ''})`);
|
|
10194
|
+
return ` — set aside ${sup.length} superseded run${sup.length === 1 ? '' : 's'} (a newer run of the same workflow decides, #461): ${names.join(', ')}`;
|
|
10183
10195
|
}
|
|
10184
10196
|
|
|
10185
10197
|
/**
|
package/tools/lib/git.js
CHANGED
|
@@ -984,16 +984,84 @@ function excludedRunSummary(rows) {
|
|
|
984
984
|
* were, the result carries an additive `excluded` list naming them, so a caller can say what it
|
|
985
985
|
* did not count. When nothing is left the sha reads `none`, exactly as if no run had existed —
|
|
986
986
|
* a repo-owned red is never affected, since only `dynamic` rows are dropped.
|
|
987
|
+
*
|
|
988
|
+
* #461: the repo-owned rows are then reduced to the newest run per workflow (newestRunPerWorkflow,
|
|
989
|
+
* below); the rows that reduction set aside ride along as an additive `superseded` list, same shape
|
|
990
|
+
* as `excluded`, present only when non-empty. Every quantifier below runs over the reduced set.
|
|
987
991
|
*/
|
|
988
992
|
function summarizeRunsForCommit(allForSha, sha) {
|
|
989
993
|
if (allForSha === null) return null;
|
|
990
994
|
const dropped = allForSha.filter((x) => !isRepoOwnedRun(x));
|
|
991
|
-
const
|
|
992
|
-
const
|
|
993
|
-
const
|
|
995
|
+
const owned = dropped.length ? allForSha.filter(isRepoOwnedRun) : allForSha;
|
|
996
|
+
const { heads, superseded } = newestRunPerWorkflow(owned);
|
|
997
|
+
const extra = {
|
|
998
|
+
...(dropped.length ? { excluded: excludedRunSummary(dropped) } : {}),
|
|
999
|
+
...(superseded.length ? { superseded: excludedRunSummary(superseded) } : {}),
|
|
1000
|
+
};
|
|
1001
|
+
const r = summarizeRepoOwnedRuns(heads, sha);
|
|
994
1002
|
return { ...r, ...extra };
|
|
995
1003
|
}
|
|
996
1004
|
|
|
1005
|
+
/** Sort key for "which run of one workflow is newer": createdAt, then databaseId; null = unknown. */
|
|
1006
|
+
function runRecency(x) {
|
|
1007
|
+
const t = x && x.createdAt ? Date.parse(x.createdAt) : NaN;
|
|
1008
|
+
const id = x && x.databaseId != null ? Number(x.databaseId) : NaN;
|
|
1009
|
+
return { t: Number.isFinite(t) ? t : null, id: Number.isFinite(id) ? id : null };
|
|
1010
|
+
}
|
|
1011
|
+
|
|
1012
|
+
/** True when `a` is provably newer than `b`; false when older OR when neither key can tell. */
|
|
1013
|
+
function isNewerRun(a, b) {
|
|
1014
|
+
const ra = runRecency(a); const rb = runRecency(b);
|
|
1015
|
+
if (ra.t !== null && rb.t !== null && ra.t !== rb.t) return ra.t > rb.t;
|
|
1016
|
+
if (ra.id !== null && rb.id !== null && ra.id !== rb.id) return ra.id > rb.id;
|
|
1017
|
+
return false;
|
|
1018
|
+
}
|
|
1019
|
+
|
|
1020
|
+
/**
|
|
1021
|
+
* #461: each workflow is judged by its NEWEST run at the sha — an earlier attempt of the SAME
|
|
1022
|
+
* workflow that a later one superseded no longer vetoes it. Measured: a `workflow_run`-triggered
|
|
1023
|
+
* workflow fired once per CI attempt at one trunk sha; the two red CI attempts gave it two
|
|
1024
|
+
* `skipped` runs, the green third attempt a `success`, and the two stale rows held the sha
|
|
1025
|
+
* not-green forever (a re-run replays the original payload and skips again).
|
|
1026
|
+
*
|
|
1027
|
+
* Grouped by `workflowName` ONLY. A row with no workflowName (an older read, a hand-built fixture)
|
|
1028
|
+
* is its own group — absence of the field never collapses anything, so every verdict that could be
|
|
1029
|
+
* computed before #461 without the field is computed identically now. Rows of DIFFERENT workflows
|
|
1030
|
+
* are never reduced: a failing workflow beside a passing one still reads not-green (#146/#162).
|
|
1031
|
+
*
|
|
1032
|
+
* `cancelled` never supersedes: a cancelled run executed nothing and has always been neutral here
|
|
1033
|
+
* (#92), so it cannot stand for its workflow while that workflow has any non-cancelled run at the
|
|
1034
|
+
* sha — otherwise a cancel-in-progress straggler created after the passing run would turn #92's
|
|
1035
|
+
* green back into "every run was cancelled". It still counts when it is all a workflow has.
|
|
1036
|
+
*
|
|
1037
|
+
* Newest = later createdAt, then higher databaseId; when neither key separates two rows, gh's
|
|
1038
|
+
* newest-first order decides (the first row seen is kept). An unfinished newest run still blocks
|
|
1039
|
+
* green (#307): it is the head, so the all-finished quantifier sees it.
|
|
1040
|
+
*
|
|
1041
|
+
* Returns the heads in their original order (callers' newest-first `forSha[0]` pick relies on it)
|
|
1042
|
+
* plus the rows they superseded.
|
|
1043
|
+
*/
|
|
1044
|
+
function newestRunPerWorkflow(rows) {
|
|
1045
|
+
const best = new Map();
|
|
1046
|
+
rows.forEach((x, i) => {
|
|
1047
|
+
const name = x && x.workflowName;
|
|
1048
|
+
if (!name) return;
|
|
1049
|
+
const cur = best.get(name);
|
|
1050
|
+
if (cur === undefined) { best.set(name, i); return; }
|
|
1051
|
+
const c = rows[cur];
|
|
1052
|
+
const xCancelled = x.conclusion === 'cancelled';
|
|
1053
|
+
const cCancelled = c.conclusion === 'cancelled';
|
|
1054
|
+
if (xCancelled !== cCancelled) { if (cCancelled) best.set(name, i); return; }
|
|
1055
|
+
if (isNewerRun(x, c)) best.set(name, i);
|
|
1056
|
+
});
|
|
1057
|
+
const heads = []; const superseded = [];
|
|
1058
|
+
rows.forEach((x, i) => {
|
|
1059
|
+
const name = x && x.workflowName;
|
|
1060
|
+
if (!name || best.get(name) === i) heads.push(x); else superseded.push(x);
|
|
1061
|
+
});
|
|
1062
|
+
return { heads, superseded };
|
|
1063
|
+
}
|
|
1064
|
+
|
|
997
1065
|
function summarizeRepoOwnedRuns(forSha, sha) {
|
|
998
1066
|
if (forSha.length === 0) return { status: 'none', conclusion: null, sha, createdAt: null, databaseId: null, runCount: 0 };
|
|
999
1067
|
|
|
@@ -1001,6 +1069,8 @@ function summarizeRepoOwnedRuns(forSha, sha) {
|
|
|
1001
1069
|
// report "N runs at <sha>: all success" instead of a singular verdict that hides how many
|
|
1002
1070
|
// workflows actually agreed. It is the row count BEFORE any of the picks below, not "how many
|
|
1003
1071
|
// succeeded" — the picked row already tells the caller the conclusion; this tells it the sample size.
|
|
1072
|
+
// Since #461 that is the count of rows JUDGED (one per named workflow, after superseded attempts
|
|
1073
|
+
// are set aside) — "3 runs: all success" must never count two skipped attempts it did not judge.
|
|
1004
1074
|
const runCount = forSha.length;
|
|
1005
1075
|
|
|
1006
1076
|
// A completed sibling whose conclusion is anything but success or cancelled makes the sha
|
package/tools/lib/ship-batch.js
CHANGED
|
@@ -179,6 +179,9 @@ function wiring(fires) {
|
|
|
179
179
|
* databaseId}`). Same all-runs rule as summarizeRunsForCommit: any finished not-green row (cancelled
|
|
180
180
|
* aside) makes it red; any unfinished row keeps it pending; green needs every row finished and at
|
|
181
181
|
* least one success. `attempt` is the highest run attempt seen — how many times it was re-run.
|
|
182
|
+
* Unlike summarizeRunsForCommit it does NOT reduce to the newest run per workflow (#461): a batch
|
|
183
|
+
* ref is built fresh and re-run in place (same run, higher `attempt`), so it carries no superseded
|
|
184
|
+
* attempts of a workflow for that reduction to set aside.
|
|
182
185
|
*/
|
|
183
186
|
function combinedVerdict(rows) {
|
|
184
187
|
const list = Array.isArray(rows) ? rows : null;
|
package/tools/lib/stamp.js
CHANGED
|
@@ -504,10 +504,12 @@ const WORKFLOW_FINGERPRINTS = [
|
|
|
504
504
|
// overwrite. Verified against all three before being added here.
|
|
505
505
|
{ marker: 'Resolve deploy toolchain', kind: 'step', template: 'deploy-xserver' },
|
|
506
506
|
{ marker: 'Verify the deployed site answers', kind: 'step', template: 'deploy-xserver' },
|
|
507
|
-
// deploy-container (#452
|
|
508
|
-
// marker — a phrase a hand-written deploy
|
|
509
|
-
// an overwrite.
|
|
510
|
-
|
|
507
|
+
// deploy-container (#452, split into publish + deploy jobs by #460): its two coined steps.
|
|
508
|
+
// "Resolve the tag to deploy" is deliberately not a marker — a phrase a hand-written deploy
|
|
509
|
+
// workflow could carry, and an over-eager marker invites an overwrite. A copy taken before #460
|
|
510
|
+
// (one step, "Build once, run exactly this tag, verify …") still matches on the header text marker.
|
|
511
|
+
{ marker: 'Build once and push (deploy-container-run --publish-only)', kind: 'step', template: 'deploy-container' },
|
|
512
|
+
{ marker: 'Run exactly this tag, verify (deploy-container-run --deploy-only)', kind: 'step', template: 'deploy-container' },
|
|
511
513
|
// branch-name (#348): its one step.
|
|
512
514
|
{ marker: 'Check branch name (CONVENTIONS §4)', kind: 'step', template: 'branch-name' },
|
|
513
515
|
// Shared by all three ci-* templates: proves derivation, cannot say from which. Note the exact
|