@norskvideo/ctl-dev-kit 0.2.60 → 0.2.62
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/build/flake.sdk-app.nix +5 -1
- package/conventions/CLAUDE.core.md +6 -0
- package/conventions/bot-prs.yml +133 -16
- package/conventions/build-docs.yml +7 -3
- package/conventions/build-image.yml +25 -8
- package/conventions/check-drift.ts +2 -2
- package/conventions/integration.yml +32 -7
- package/conventions/publish-docs.yml +7 -3
- package/conventions/smoke.yml +22 -6
- package/conventions/sync-ctl-packages.yml +7 -6
- package/conventions/sync-dev-kit.yml +73 -73
- package/conventions/upgrade-latest.sdk-app.yml +112 -78
- package/conventions/upgrade-latest.yml +101 -93
- package/docs-theme/README.md +2 -2
- package/package.json +1 -1
package/build/flake.sdk-app.nix
CHANGED
|
@@ -8,7 +8,9 @@
|
|
|
8
8
|
#
|
|
9
9
|
# Unlike the Studio-product flake this does NOT fetch the norsk-ctl daemon —
|
|
10
10
|
# an sdk-app's dev loop is its own SDK app plus a Norsk engine, not a
|
|
11
|
-
# ctl-launched product container.
|
|
11
|
+
# ctl-launched product container. It still PINS one: `ctlVersion` below is the
|
|
12
|
+
# norsk-ctl the product's CI tiers (smoke) launch it with, read from here so a
|
|
13
|
+
# result belongs to a commit, and moved nightly by upgrade-latest. Decision: bun for tooling (install / build /
|
|
12
14
|
# test), Node for the shipped runtime (the SDK's blessed runtime).
|
|
13
15
|
#
|
|
14
16
|
# nixpkgs is pinned to the same rev the Studio flake uses, so bun matches the
|
|
@@ -27,6 +29,8 @@
|
|
|
27
29
|
|
|
28
30
|
outputs = { self, nixpkgs, nixpkgs-biome }:
|
|
29
31
|
let
|
|
32
|
+
# The norsk-ctl CI runs (see above); upgrade-latest moves it to `latest`.
|
|
33
|
+
ctlVersion = "0.1.0-2026-09-15-81bbde8";
|
|
30
34
|
systems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ];
|
|
31
35
|
forAllSystems = f: builtins.listToAttrs (map (s: { name = s; value = f s; }) systems);
|
|
32
36
|
in {
|
|
@@ -84,6 +84,12 @@
|
|
|
84
84
|
same as this core block and `nix/ctl-shell.nix`; `check:drift` fails CI if a copy
|
|
85
85
|
diverges. Only the `product:` key is per-repo. Never hand-edit the copy — edit
|
|
86
86
|
the dev-kit source and re-sync.
|
|
87
|
+
- **CI runs the norsk-ctl this repo pins** (`flake.nix` `ctlVersion`), never S3's
|
|
88
|
+
moving `latest`; only a `ctl-candidate` run tests a channel (`rc`). The bump bots
|
|
89
|
+
(`upgrade-latest`, `sync-dev-kit`, `sync-ctl-packages`) never push to main: each
|
|
90
|
+
opens a PR, and `bot-prs` merges it once checks, smoke, integration and the
|
|
91
|
+
guides gate (build-image) are green on it. A red bot PR is the alarm for a bad
|
|
92
|
+
upstream change; main stays green.
|
|
87
93
|
- **A config-driven workflow needs a parity-checked invariant contract.** If this
|
|
88
94
|
product emits its Studio graph dynamically from config, its load-bearing rules
|
|
89
95
|
are documented as stable-ID statements (`INV-<AREA>-<NNN>`) in an
|
package/conventions/bot-prs.yml
CHANGED
|
@@ -1,25 +1,37 @@
|
|
|
1
1
|
# Land the pin bots' green PRs; put the red ones on the product dashboard.
|
|
2
2
|
#
|
|
3
|
-
# Three bots keep a product fresh: upgrade-latest (the Norsk nightly line
|
|
4
|
-
# sync-dev-kit (this package) and sync-ctl-packages (the
|
|
5
|
-
#
|
|
6
|
-
#
|
|
7
|
-
#
|
|
3
|
+
# Three bots keep a product fresh: upgrade-latest (the Norsk nightly line and
|
|
4
|
+
# the norsk-ctl pin), sync-dev-kit (this package) and sync-ctl-packages (the
|
|
5
|
+
# ctl-* / moq-* npm family). All three ALWAYS open a PR and never push to main
|
|
6
|
+
# (ci-workflows docs/dashboard/ci-fragility.md, proposal B): a bot's own gate
|
|
7
|
+
# is lint, typecheck and unit tests, and the tiers that break on an upstream
|
|
8
|
+
# change -- smoke, integration and the guides that gate build-image's publish
|
|
9
|
+
# -- run on the PR. This workflow is what lands them. Nothing used to act on
|
|
10
|
+
# those PRs: green
|
|
8
11
|
# ones sat unmerged (studio's ctl-sdk was twenty releases behind on 2026-10-03,
|
|
9
12
|
# its PR open since 09-03) and red ones were invisible, because the bot that
|
|
10
13
|
# opened them reports success. With a product count that only goes up, "someone
|
|
11
14
|
# will notice" is not a process.
|
|
12
15
|
#
|
|
13
16
|
# So, for every open PR whose branch a bot owns:
|
|
14
|
-
# - every check on its head has passed
|
|
15
|
-
#
|
|
17
|
+
# - every check on its head has passed, every TIER that runs on a bot PR has a
|
|
18
|
+
# green run on that head, and it merges cleanly -> squash-merge it. The
|
|
19
|
+
# tiers are checks, smoke, integration and build-image (whose gate-guides
|
|
20
|
+
# job runs the guides): each of those the head carries whose workflow runs
|
|
21
|
+
# on pull_request. integration and build-image run on bot branches only, so
|
|
22
|
+
# human PRs do not pay for them. A hand-owned integration.yml that runs on
|
|
23
|
+
# push alone cannot gate, and still runs on main after the merge. "Nothing
|
|
24
|
+
# failed yet" is not enough: a slow tier that has not started has no check
|
|
25
|
+
# run at all, so bot-prs used to merge before integration had begun;
|
|
16
26
|
# - it conflicts with main -> re-run the bot that owns the branch, ONCE per
|
|
17
27
|
# head, so it rebuilds the branch from today's main (see below);
|
|
18
28
|
# - a check failed, it still conflicts after that rebuild, or it is a draft
|
|
19
29
|
# (the bot's own gate was red) -> list it, and report this repo's `bot-prs`
|
|
20
30
|
# cell RED on the product dashboard, which is the Discord message people
|
|
21
31
|
# actually read;
|
|
22
|
-
# - checks still running -> leave it for the next trigger
|
|
32
|
+
# - checks or a tier still running -> leave it for the next trigger; a tier
|
|
33
|
+
# that has not started an hour after checks finished never will (its
|
|
34
|
+
# workflow is broken, or does not run on PRs), so that is red too.
|
|
23
35
|
# The cell goes green again once nothing is red.
|
|
24
36
|
#
|
|
25
37
|
# Why the conflict rebuild: the three bots fire within minutes of each other
|
|
@@ -28,13 +40,12 @@
|
|
|
28
40
|
# bot only rebuilds its branch on its next DAILY run -- so on 2026-10-09
|
|
29
41
|
# commentary, probe and reuters each had a sync-ctl-packages PR "needing a
|
|
30
42
|
# human" that a re-run of sync-ctl-packages fixed outright. Every bot rebuilds
|
|
31
|
-
# from main when run:
|
|
32
|
-
#
|
|
33
|
-
# superseded, or force-push their branch. The marker comment caps it at one
|
|
43
|
+
# from main when run: each force-pushes its branch from today's main and
|
|
44
|
+
# closes any older PR of its own family. The marker comment caps it at one
|
|
34
45
|
# rebuild per head, so a rebuild that still conflicts goes to a human instead
|
|
35
46
|
# of looping every hour.
|
|
36
47
|
#
|
|
37
|
-
# Runs when
|
|
48
|
+
# Runs when any gating tier finishes (the prompt path), hourly as a sweep,
|
|
38
49
|
# and on demand. No checkout: it only talks to the GitHub API, so it needs none
|
|
39
50
|
# of the runner-workspace steps the building workflows carry.
|
|
40
51
|
#
|
|
@@ -46,7 +57,7 @@ name: bot-prs
|
|
|
46
57
|
|
|
47
58
|
on:
|
|
48
59
|
workflow_run:
|
|
49
|
-
workflows: [checks, smoke]
|
|
60
|
+
workflows: [checks, smoke, integration, build-image]
|
|
50
61
|
types: [completed]
|
|
51
62
|
schedule:
|
|
52
63
|
- cron: "23 * * * *"
|
|
@@ -56,8 +67,13 @@ permissions:
|
|
|
56
67
|
contents: read
|
|
57
68
|
checks: read
|
|
58
69
|
pull-requests: read
|
|
59
|
-
#
|
|
60
|
-
#
|
|
70
|
+
# Reads which gating tiers have passed on a PR's head, and writes Actions:
|
|
71
|
+
# re-running a cancelled tier and dispatching a bot's workflow to rebuild a
|
|
72
|
+
# conflicting PR. Those went through the PAT, which has no Actions write, and
|
|
73
|
+
# failed with HTTP 403 in every repo (2026-10-10). A workflow_dispatch made
|
|
74
|
+
# with this token does start a run (exempt from the recursion guard). Merges,
|
|
75
|
+
# branch deletes and comments stay on the PAT (WRITE_TOKEN).
|
|
76
|
+
actions: write
|
|
61
77
|
|
|
62
78
|
concurrency:
|
|
63
79
|
# One at a time per repo, and never cancelled: a merge half-done by a run that
|
|
@@ -86,6 +102,9 @@ jobs:
|
|
|
86
102
|
curl_as() { local tok="$1"; shift; curl -sS --retry 3 --retry-all-errors -H "Authorization: Bearer $tok" -H "Accept: application/vnd.github+json" "$@"; }
|
|
87
103
|
gh_api() { curl_as "$READ_TOKEN" "$@"; }
|
|
88
104
|
gh_write() { curl_as "$WRITE_TOKEN" "$@"; }
|
|
105
|
+
# Actions writes (re-run a tier, dispatch a bot's workflow) use the
|
|
106
|
+
# job token, which permissions grant actions: write; the PAT cannot.
|
|
107
|
+
gh_actions() { curl_as "$READ_TOKEN" "$@"; }
|
|
89
108
|
# An API error is a JSON object with a message; say what it was rather
|
|
90
109
|
# than letting jq fail later on a missing field.
|
|
91
110
|
expect() {
|
|
@@ -103,6 +122,88 @@ jobs:
|
|
|
103
122
|
| select(.head.ref | test("^(upgrade-latest/|sync-dev-kit/|chore/sync-ctl-packages$)"))
|
|
104
123
|
| {number, ref: .head.ref, sha: .head.sha, draft, url: .html_url}]' <<<"$pulls")"
|
|
105
124
|
|
|
125
|
+
# Does this workflow (on stdin) run on pull_request? Only its `on:` block
|
|
126
|
+
# counts. The convention integration.yml and build-image.yml do, on bot
|
|
127
|
+
# branches only; a hand-owned integration.yml that runs on push alone
|
|
128
|
+
# cannot gate a PR.
|
|
129
|
+
gates_bot_prs() {
|
|
130
|
+
awk '/^on:/ { on = 1; next } on && /^[^ #]/ { on = 0 } on && /^ pull_request:/ { found = 1 } END { exit !found }'
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
# The tiers a bump must pass on its head before it reaches main: each
|
|
134
|
+
# of these workflows the HEAD carries that runs on a bot PR. Read from
|
|
135
|
+
# the head, not main, so a dev-kit sync that brings a tier gates on it.
|
|
136
|
+
# Each file is read whole BEFORE it is judged: piped straight into
|
|
137
|
+
# gates_bot_prs under pipefail, a file larger than the pipe buffer
|
|
138
|
+
# failed the pipeline whenever the reader finished first, and
|
|
139
|
+
# build-image.yml read as "does not run on bot PRs" (commentary #36).
|
|
140
|
+
tiers_for() {
|
|
141
|
+
local sha="$1" listing f text
|
|
142
|
+
listing="$(gh_api "$api/contents/.github/workflows?ref=$sha")"
|
|
143
|
+
expect "listing the workflows at $sha" 'type == "array"' "$listing"
|
|
144
|
+
for f in checks.yml smoke.yml integration.yml build-image.yml; do
|
|
145
|
+
jq -e --arg f "$f" 'any(.[]; .name == $f)' >/dev/null <<<"$listing" || continue
|
|
146
|
+
# A tier it cannot read must not silently stop gating: fail instead.
|
|
147
|
+
text="$(curl -fsS --retry 3 --retry-all-errors -H "Authorization: Bearer $READ_TOKEN" \
|
|
148
|
+
-H "Accept: application/vnd.github.raw" "$api/contents/.github/workflows/$f?ref=$sha")" ||
|
|
149
|
+
{ echo "::error::could not read $f at $sha" >&2; return 1; }
|
|
150
|
+
if gates_bot_prs <<<"$text"; then
|
|
151
|
+
echo "$f"
|
|
152
|
+
else
|
|
153
|
+
echo "${f%.yml} does not run on bot PRs at ${sha:0:8}, so it does not gate the merge" >&2
|
|
154
|
+
fi
|
|
155
|
+
done
|
|
156
|
+
}
|
|
157
|
+
|
|
158
|
+
# tier_verdict REQUIRED RUNS NOW -> "ok", "wait <why>", "rerun <run ids>"
|
|
159
|
+
# or "red <why>". REQUIRED: workflow files, one per line. RUNS: the
|
|
160
|
+
# head's actions/runs listing. The newest pull_request run of each
|
|
161
|
+
# counts, so a re-run after a red is honoured. A tier with no run yet is
|
|
162
|
+
# waited for, unless checks finished over an hour ago: then it never
|
|
163
|
+
# will start. A CANCELLED tier is re-run, up to its third attempt: the
|
|
164
|
+
# guides queue on one concurrency group per repo and GitHub keeps one
|
|
165
|
+
# pending run per group, cancelling the older, so three bot PRs opening
|
|
166
|
+
# within the hour can cancel a guides run with nothing wrong in it.
|
|
167
|
+
tier_verdict() {
|
|
168
|
+
local required="$1" runs="$2" now="$3" f run status conclusion checks_age
|
|
169
|
+
local -a waiting=() failed=() rerun=()
|
|
170
|
+
checks_age="$(jq -r --argjson now "$now" '
|
|
171
|
+
[.workflow_runs[] | select((.path | split("@")[0]) == ".github/workflows/checks.yml"
|
|
172
|
+
and .event == "pull_request" and .status == "completed")] | max_by(.id)
|
|
173
|
+
| if . == null then "" else ($now - (.updated_at | fromdateiso8601)) end' <<<"$runs")"
|
|
174
|
+
while IFS= read -r f; do
|
|
175
|
+
[ -n "$f" ] || continue
|
|
176
|
+
run="$(jq -c --arg p ".github/workflows/$f" '
|
|
177
|
+
[.workflow_runs[] | select((.path | split("@")[0]) == $p and .event == "pull_request")] | max_by(.id) // empty' <<<"$runs")"
|
|
178
|
+
if [ -z "$run" ]; then
|
|
179
|
+
if [ -n "$checks_age" ] && [ "$checks_age" -gt 3600 ]; then
|
|
180
|
+
failed+=("${f%.yml} never ran on this head (does its workflow run on bot PRs?)")
|
|
181
|
+
else
|
|
182
|
+
waiting+=("${f%.yml} has not started")
|
|
183
|
+
fi
|
|
184
|
+
continue
|
|
185
|
+
fi
|
|
186
|
+
status="$(jq -r .status <<<"$run")"
|
|
187
|
+
conclusion="$(jq -r '.conclusion // ""' <<<"$run")"
|
|
188
|
+
if [ "$status" != "completed" ]; then
|
|
189
|
+
waiting+=("${f%.yml} $status")
|
|
190
|
+
elif [ "$conclusion" = "cancelled" ] && [ "$(jq -r '.run_attempt // 1' <<<"$run")" -lt 3 ]; then
|
|
191
|
+
rerun+=("$(jq -r .id <<<"$run")")
|
|
192
|
+
elif [ "$conclusion" != "success" ]; then
|
|
193
|
+
failed+=("${f%.yml} $conclusion")
|
|
194
|
+
fi
|
|
195
|
+
done <<<"$required"
|
|
196
|
+
if [ "${#failed[@]}" -gt 0 ]; then
|
|
197
|
+
printf 'red %s\n' "$(printf '%s; ' "${failed[@]}" | sed 's/; $//')"
|
|
198
|
+
elif [ "${#rerun[@]}" -gt 0 ]; then
|
|
199
|
+
echo "rerun ${rerun[*]}"
|
|
200
|
+
elif [ "${#waiting[@]}" -gt 0 ]; then
|
|
201
|
+
printf 'wait %s\n' "$(printf '%s; ' "${waiting[@]}" | sed 's/; $//')"
|
|
202
|
+
else
|
|
203
|
+
echo ok
|
|
204
|
+
fi
|
|
205
|
+
}
|
|
206
|
+
|
|
106
207
|
# The workflow that owns each bot branch, i.e. the one that rebuilds it.
|
|
107
208
|
producer() {
|
|
108
209
|
case "$1" in
|
|
@@ -127,6 +228,22 @@ jobs:
|
|
|
127
228
|
if [ -n "$failed" ]; then red+=("#$n $ref: failing $failed"); continue; fi
|
|
128
229
|
if [ "$(jq -r .draft <<<"$pr")" = "true" ]; then red+=("#$n $ref: draft (its bot's gate was red)"); continue; fi
|
|
129
230
|
if [ "$pending" -gt 0 ] || [ "$passed" -eq 0 ]; then echo "#$n $ref: checks not finished; next time"; continue; fi
|
|
231
|
+
# Every tier that runs on a bot PR must have PASSED on this head.
|
|
232
|
+
required="$(tiers_for "$sha")"
|
|
233
|
+
wf_runs="$(gh_api "$api/actions/runs?head_sha=$sha&per_page=100")"
|
|
234
|
+
expect "reading workflow runs for #$n" '.workflow_runs | type == "array"' "$wf_runs"
|
|
235
|
+
verdict="$(tier_verdict "$required" "$wf_runs" "$(date +%s)")"
|
|
236
|
+
case "$verdict" in
|
|
237
|
+
ok) ;;
|
|
238
|
+
red\ *) red+=("#$n $ref: ${verdict#red }"); continue ;;
|
|
239
|
+
rerun\ *)
|
|
240
|
+
for id in ${verdict#rerun }; do
|
|
241
|
+
code="$(gh_actions -o /dev/null -w '%{http_code}' -X POST "$api/actions/runs/$id/rerun")"
|
|
242
|
+
echo "#$n $ref: a gating tier (run $id) was cancelled; re-running it (HTTP $code)"
|
|
243
|
+
done
|
|
244
|
+
continue ;;
|
|
245
|
+
*) echo "#$n $ref: ${verdict#wait }; next time"; continue ;;
|
|
246
|
+
esac
|
|
130
247
|
# GitHub computes mergeability lazily; ask until it has an answer.
|
|
131
248
|
mergeable=null
|
|
132
249
|
for _ in 1 2 3 4 5; do
|
|
@@ -145,7 +262,7 @@ jobs:
|
|
|
145
262
|
elif jq -e --arg m "$marker" 'any(.[]; .body | contains($m))' >/dev/null <<<"$comments"; then
|
|
146
263
|
red+=("#$n $ref: still does not merge cleanly with main after $wf rebuilt it")
|
|
147
264
|
else
|
|
148
|
-
code="$(
|
|
265
|
+
code="$(gh_actions -o /dev/null -w '%{http_code}' -X POST "$api/actions/workflows/$wf/dispatches" -d '{"ref":"main"}')"
|
|
149
266
|
if [ "$code" = "204" ]; then
|
|
150
267
|
body="$marker -- it conflicted with main (another bot landed first), so $wf was re-run to rebuild it from today's main. If the rebuilt branch conflicts too, this PR is reported for a human."
|
|
151
268
|
gh_write -o /dev/null -X POST "$api/issues/$n/comments" -d "$(jq -nc --arg b "$body" '{body:$b}')" || true
|
|
@@ -164,12 +164,16 @@ jobs:
|
|
|
164
164
|
NORSK_LICENSE_V2: ${{ secrets.NORSK_LICENSE_V2 }}
|
|
165
165
|
run: printf '%s' "$NORSK_LICENSE_V2" > "$RUNNER_TEMP/norsk-license.json"
|
|
166
166
|
|
|
167
|
-
|
|
167
|
+
# The norsk-ctl this commit pins (flake.nix ctlVersion, moved by
|
|
168
|
+
# upgrade-latest's bump PR), never S3's moving `latest`: a guide run is then
|
|
169
|
+
# a real repeat, and a red belongs to a commit (ci-fragility proposal A).
|
|
170
|
+
- name: Download the pinned norsk-ctl binary (flake.nix ctlVersion)
|
|
168
171
|
run: |
|
|
169
172
|
set -euo pipefail
|
|
170
173
|
S3="https://s3.eu-west-1.amazonaws.com/norsk.video/norsk-ctl"
|
|
171
|
-
ver="$(
|
|
172
|
-
echo "norsk-ctl latest
|
|
174
|
+
ver="$( { sed -nE 's/^[[:space:]]*ctlVersion = "([^"]+)";.*/\1/p' flake.nix 2>/dev/null || true; } | head -1)"
|
|
175
|
+
[ -n "$ver" ] || { echo "::error::flake.nix pins no ctlVersion. CI runs the norsk-ctl this commit pins (upgrade-latest moves it); add ctlVersion = \"<version>\"; to flake.nix."; exit 1; }
|
|
176
|
+
echo "norsk-ctl pinned by flake.nix ctlVersion -> $ver"
|
|
173
177
|
curl -fsSL --retry 3 --retry-all-errors --retry-delay 2 "$S3/$ver/norsk-ctl-$ver-linux-x64" -o "$RUNNER_TEMP/norsk-ctl"
|
|
174
178
|
chmod +x "$RUNNER_TEMP/norsk-ctl"
|
|
175
179
|
"$RUNNER_TEMP/norsk-ctl" --version || true
|
|
@@ -21,6 +21,16 @@
|
|
|
21
21
|
# (rc -> latest promotion) lands, flip the push default to `nightly` and let the
|
|
22
22
|
# promotion gate move `latest`.
|
|
23
23
|
#
|
|
24
|
+
# BOT PRs: on a pull request from a bot branch (upgrade-latest/*,
|
|
25
|
+
# sync-dev-kit/*, chore/sync-ctl-packages) the workflow builds both arches and
|
|
26
|
+
# runs the guides gate exactly as a push would, and publishes, records and
|
|
27
|
+
# reports nothing. A bump reaches main only once that has passed on it, and
|
|
28
|
+
# bot-prs merges on it (ci-workflows docs/dashboard/ci-fragility.md, proposal
|
|
29
|
+
# B): a dev-kit sync that brought a newer copy of THIS file broke
|
|
30
|
+
# oracle-reasoning's build on main on 2026-10-09, after a green unit gate
|
|
31
|
+
# landed it. A human PR does not pay for the build; gate-checks is skipped on a
|
|
32
|
+
# PR because the PR's own checks run already covers it.
|
|
33
|
+
#
|
|
24
34
|
# Repo-local for now; to be hoisted into id3as/ci-workflows as a reusable
|
|
25
35
|
# workflow (RFC 0001 Workstream E).
|
|
26
36
|
name: build-image
|
|
@@ -28,6 +38,7 @@ name: build-image
|
|
|
28
38
|
on:
|
|
29
39
|
push:
|
|
30
40
|
branches: [main]
|
|
41
|
+
pull_request:
|
|
31
42
|
workflow_dispatch:
|
|
32
43
|
inputs:
|
|
33
44
|
channel:
|
|
@@ -57,6 +68,7 @@ jobs:
|
|
|
57
68
|
# Compute the immutable label ONCE, so both arch builds and the manifest share
|
|
58
69
|
# it (a per-job `date` could straddle midnight and disagree).
|
|
59
70
|
label:
|
|
71
|
+
if: ${{ github.event_name != 'pull_request' || (github.event.pull_request.head.repo.full_name == github.repository && (startsWith(github.head_ref, 'upgrade-latest/') || startsWith(github.head_ref, 'sync-dev-kit/') || github.head_ref == 'chore/sync-ctl-packages')) }}
|
|
60
72
|
runs-on: x64
|
|
61
73
|
outputs:
|
|
62
74
|
label: ${{ steps.gen.outputs.label }}
|
|
@@ -241,14 +253,14 @@ jobs:
|
|
|
241
253
|
'
|
|
242
254
|
|
|
243
255
|
- name: Log in to Docker Hub
|
|
244
|
-
if: ${{ env.PUBLISH_IMAGE != '' }}
|
|
256
|
+
if: ${{ env.PUBLISH_IMAGE != '' && github.event_name != 'pull_request' }}
|
|
245
257
|
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
|
|
246
258
|
with:
|
|
247
259
|
username: norskvideo
|
|
248
260
|
password: ${{ secrets.DOCKER_PAT }}
|
|
249
261
|
|
|
250
262
|
- name: Push the per-arch image to Docker Hub
|
|
251
|
-
if: ${{ env.PUBLISH_IMAGE != '' }}
|
|
263
|
+
if: ${{ env.PUBLISH_IMAGE != '' && github.event_name != 'pull_request' }}
|
|
252
264
|
run: docker push "$IMAGE_TAG"
|
|
253
265
|
- name: Reap the containers this job launched
|
|
254
266
|
# Whatever the tests did. A teardown cut short by a test timeout leaves
|
|
@@ -276,6 +288,7 @@ jobs:
|
|
|
276
288
|
# tests shipped a probe whose preview was broken for four days, while the one
|
|
277
289
|
# test that knew sat red in a nightly nobody saw (task 57).
|
|
278
290
|
gate-checks:
|
|
291
|
+
if: ${{ github.event_name != 'pull_request' }}
|
|
279
292
|
uses: ./.github/workflows/checks.yml
|
|
280
293
|
with:
|
|
281
294
|
report: false
|
|
@@ -388,13 +401,17 @@ jobs:
|
|
|
388
401
|
NORSK_LICENSE_V2: ${{ secrets.NORSK_LICENSE_V2 }}
|
|
389
402
|
run: printf '%s' "$NORSK_LICENSE_V2" > "$RUNNER_TEMP/norsk-license.json"
|
|
390
403
|
|
|
391
|
-
|
|
404
|
+
# The norsk-ctl this commit pins (flake.nix ctlVersion, moved by
|
|
405
|
+
# upgrade-latest's bump PR), never S3's moving `latest`: a guide run is then
|
|
406
|
+
# a real repeat, and a red belongs to a commit (ci-fragility proposal A).
|
|
407
|
+
- name: Download the pinned norsk-ctl binary (flake.nix ctlVersion)
|
|
392
408
|
if: ${{ steps.plan.outputs.guides == 'run' }}
|
|
393
409
|
run: |
|
|
394
410
|
set -euo pipefail
|
|
395
411
|
S3="https://s3.eu-west-1.amazonaws.com/norsk.video/norsk-ctl"
|
|
396
|
-
ver="$(
|
|
397
|
-
echo "norsk-ctl latest
|
|
412
|
+
ver="$( { sed -nE 's/^[[:space:]]*ctlVersion = "([^"]+)";.*/\1/p' flake.nix 2>/dev/null || true; } | head -1)"
|
|
413
|
+
[ -n "$ver" ] || { echo "::error::flake.nix pins no ctlVersion. CI runs the norsk-ctl this commit pins (upgrade-latest moves it); add ctlVersion = \"<version>\"; to flake.nix."; exit 1; }
|
|
414
|
+
echo "norsk-ctl pinned by flake.nix ctlVersion -> $ver"
|
|
398
415
|
curl -fsSL --retry 3 --retry-all-errors --retry-delay 2 "$S3/$ver/norsk-ctl-$ver-linux-x64" -o "$RUNNER_TEMP/norsk-ctl"
|
|
399
416
|
chmod +x "$RUNNER_TEMP/norsk-ctl"
|
|
400
417
|
"$RUNNER_TEMP/norsk-ctl" --version || true
|
|
@@ -474,7 +491,7 @@ jobs:
|
|
|
474
491
|
# a dry run.
|
|
475
492
|
publish:
|
|
476
493
|
needs: [label, build, gate-checks, gate-guides]
|
|
477
|
-
if: ${{ needs.label.outputs.publish == 'true' && !inputs.dry_run }}
|
|
494
|
+
if: ${{ needs.label.outputs.publish == 'true' && !inputs.dry_run && github.event_name != 'pull_request' }}
|
|
478
495
|
runs-on: x64
|
|
479
496
|
steps:
|
|
480
497
|
# Runs BEFORE checkout, and does two things: points the harness's temp base
|
|
@@ -583,7 +600,7 @@ jobs:
|
|
|
583
600
|
# exists this warns and records nothing, rather than failing a publish.
|
|
584
601
|
record:
|
|
585
602
|
needs: [label, gate-guides, publish]
|
|
586
|
-
if: ${{ needs.publish.result == 'success' }}
|
|
603
|
+
if: ${{ needs.publish.result == 'success' && github.event_name != 'pull_request' }}
|
|
587
604
|
runs-on: x64
|
|
588
605
|
permissions:
|
|
589
606
|
contents: read
|
|
@@ -717,7 +734,7 @@ jobs:
|
|
|
717
734
|
# still reports.
|
|
718
735
|
notify:
|
|
719
736
|
needs: [label, build, gate-checks, gate-guides, publish, record]
|
|
720
|
-
if: ${{ !cancelled() }}
|
|
737
|
+
if: ${{ !cancelled() && github.event_name != 'pull_request' }}
|
|
721
738
|
runs-on: x64
|
|
722
739
|
steps:
|
|
723
740
|
# Runs BEFORE checkout, and does two things: points the harness's temp base
|
|
@@ -70,8 +70,8 @@ export const PUBLISH_IMAGE_LINE = /^(\s*PUBLISH_IMAGE:\s*).*$/gm;
|
|
|
70
70
|
|
|
71
71
|
// integration.yml (04c s9 #1) is universally present and was hand-owned in six
|
|
72
72
|
// shapes, so on the day it joined the conventions set the gate could not demand
|
|
73
|
-
// the canonical of every copy: sync-dev-kit
|
|
74
|
-
// checks
|
|
73
|
+
// the canonical of every copy: sync-dev-kit then committed straight to main
|
|
74
|
+
// when checks were green, and would have rewritten six tiers before the hooks their
|
|
75
75
|
// suites need existed. A copy therefore declares itself converged by carrying
|
|
76
76
|
// the canonical's own first line; one without it is hand-owned and ungated
|
|
77
77
|
// until its repo converges deliberately. When every repo carries the marker,
|
|
@@ -38,6 +38,13 @@
|
|
|
38
38
|
# host siblings and the harness reaches them by compose name on norsk-net (the
|
|
39
39
|
# topology step below says so in NORSK_TEST_HOST/NORSK_TEST_NET); the GPU box
|
|
40
40
|
# is '["ARM64","nvidia"]'.
|
|
41
|
+
#
|
|
42
|
+
# On a pull request the suite runs for a BOT branch only (upgrade-latest/*,
|
|
43
|
+
# sync-dev-kit/*, chore/sync-ctl-packages): a bump reaches main only once this
|
|
44
|
+
# tier has passed on it, and bot-prs merges on that (ci-workflows
|
|
45
|
+
# docs/dashboard/ci-fragility.md, proposal B). A human PR does not pay for the
|
|
46
|
+
# slow tier; it runs on main after the merge, as before. A PR's result is the
|
|
47
|
+
# PR's, so it is not reported as the product's integration cell.
|
|
41
48
|
name: integration
|
|
42
49
|
|
|
43
50
|
on:
|
|
@@ -45,12 +52,13 @@ on:
|
|
|
45
52
|
types: [ctl-candidate]
|
|
46
53
|
push:
|
|
47
54
|
branches: [main]
|
|
55
|
+
pull_request:
|
|
48
56
|
workflow_dispatch:
|
|
49
57
|
inputs:
|
|
50
58
|
ctl_channel:
|
|
51
|
-
description: "norsk-ctl S3 channel to test against (release-gate dispatches 'rc' to validate a candidate before promoting latest
|
|
59
|
+
description: "norsk-ctl S3 channel to test against in place of the repo's pin (flake.nix ctlVersion). Empty runs the pin; release-gate dispatches 'rc' to validate a candidate before promoting latest."
|
|
52
60
|
type: string
|
|
53
|
-
default:
|
|
61
|
+
default: ""
|
|
54
62
|
|
|
55
63
|
permissions:
|
|
56
64
|
contents: read
|
|
@@ -83,6 +91,7 @@ concurrency:
|
|
|
83
91
|
|
|
84
92
|
jobs:
|
|
85
93
|
integration:
|
|
94
|
+
if: ${{ github.event_name != 'pull_request' || (github.event.pull_request.head.repo.full_name == github.repository && (startsWith(github.head_ref, 'upgrade-latest/') || startsWith(github.head_ref, 'sync-dev-kit/') || github.head_ref == 'chore/sync-ctl-packages')) }}
|
|
86
95
|
strategy:
|
|
87
96
|
matrix:
|
|
88
97
|
include:
|
|
@@ -277,18 +286,34 @@ jobs:
|
|
|
277
286
|
# Pick the binary for THIS runner's architecture: the GPU box is aarch64,
|
|
278
287
|
# the x64 pool is x86_64, and the wrong one dies with "Exec format error"
|
|
279
288
|
# deep in the suite. No `|| true` on --version: a bad binary must fail here.
|
|
280
|
-
|
|
289
|
+
#
|
|
290
|
+
# WHICH binary: the version this commit pins, flake.nix ctlVersion -- the
|
|
291
|
+
# daemon the dev shell launches, moved by upgrade-latest's bump PR. It used
|
|
292
|
+
# to be S3's moving `latest`, read at run time, so a result could change
|
|
293
|
+
# with no product commit and a re-run was never a real repeat (funke went
|
|
294
|
+
# red and green again on 2026-10-09 without a commit either time;
|
|
295
|
+
# ci-workflows docs/dashboard/ci-fragility.md, proposal A). A ctl-candidate
|
|
296
|
+
# dispatch still tests its payload's channel (rc) -- that is what it is
|
|
297
|
+
# for -- and a by-hand ctl_channel may too. ctl-pinned-in-repo.test.ts.
|
|
298
|
+
- name: Download the pinned norsk-ctl binary
|
|
299
|
+
env:
|
|
300
|
+
CTL_CHANNEL: ${{ github.event.client_payload.channel || inputs.ctl_channel }}
|
|
281
301
|
run: |
|
|
282
302
|
set -euo pipefail
|
|
283
|
-
channel="${{ github.event.client_payload.channel || inputs.ctl_channel || 'latest' }}"
|
|
284
303
|
case "$(uname -m)" in
|
|
285
304
|
x86_64) arch=linux-x64 ;;
|
|
286
305
|
aarch64 | arm64) arch=linux-arm64 ;;
|
|
287
306
|
*) echo "unsupported runner arch: $(uname -m)" >&2; exit 1 ;;
|
|
288
307
|
esac
|
|
289
308
|
S3="https://s3.eu-west-1.amazonaws.com/norsk.video/norsk-ctl"
|
|
290
|
-
|
|
291
|
-
|
|
309
|
+
if [ -n "${CTL_CHANNEL:-}" ]; then
|
|
310
|
+
ver="$(curl -fsSL --retry 3 --retry-all-errors --retry-delay 2 "$S3/$CTL_CHANNEL")"
|
|
311
|
+
echo "norsk-ctl $CTL_CHANNEL channel -> $ver ($arch), in place of the repo pin"
|
|
312
|
+
else
|
|
313
|
+
ver="$( { sed -nE 's/^[[:space:]]*ctlVersion = "([^"]+)";.*/\1/p' flake.nix 2>/dev/null || true; } | head -1)"
|
|
314
|
+
[ -n "$ver" ] || { echo "::error::flake.nix pins no ctlVersion. CI runs the norsk-ctl this commit pins (upgrade-latest moves it); add ctlVersion = \"<version>\"; to flake.nix."; exit 1; }
|
|
315
|
+
echo "norsk-ctl pinned by flake.nix ctlVersion -> $ver ($arch)"
|
|
316
|
+
fi
|
|
292
317
|
curl -fsSL --retry 3 --retry-all-errors --retry-delay 2 "$S3/$ver/norsk-ctl-$ver-$arch" -o "$RUNNER_TEMP/norsk-ctl"
|
|
293
318
|
chmod +x "$RUNNER_TEMP/norsk-ctl"
|
|
294
319
|
"$RUNNER_TEMP/norsk-ctl" --version
|
|
@@ -435,7 +460,7 @@ jobs:
|
|
|
435
460
|
# would false-red the board. Candidate validations report to ctl instead.
|
|
436
461
|
notify:
|
|
437
462
|
needs: integration
|
|
438
|
-
if: ${{ !cancelled() && github.event_name != 'repository_dispatch' }}
|
|
463
|
+
if: ${{ !cancelled() && github.event_name != 'repository_dispatch' && github.event_name != 'pull_request' }}
|
|
439
464
|
runs-on: x64
|
|
440
465
|
steps:
|
|
441
466
|
# Runs BEFORE checkout, and does two things: points the harness's temp base
|
|
@@ -174,12 +174,16 @@ jobs:
|
|
|
174
174
|
NORSK_LICENSE_V2: ${{ secrets.NORSK_LICENSE_V2 }}
|
|
175
175
|
run: printf '%s' "$NORSK_LICENSE_V2" > "$RUNNER_TEMP/norsk-license.json"
|
|
176
176
|
|
|
177
|
-
|
|
177
|
+
# The norsk-ctl this commit pins (flake.nix ctlVersion, moved by
|
|
178
|
+
# upgrade-latest's bump PR), never S3's moving `latest`: a guide run is then
|
|
179
|
+
# a real repeat, and a red belongs to a commit (ci-fragility proposal A).
|
|
180
|
+
- name: Download the pinned norsk-ctl binary (flake.nix ctlVersion)
|
|
178
181
|
run: |
|
|
179
182
|
set -euo pipefail
|
|
180
183
|
S3="https://s3.eu-west-1.amazonaws.com/norsk.video/norsk-ctl"
|
|
181
|
-
ver="$(
|
|
182
|
-
echo "norsk-ctl latest
|
|
184
|
+
ver="$( { sed -nE 's/^[[:space:]]*ctlVersion = "([^"]+)";.*/\1/p' flake.nix 2>/dev/null || true; } | head -1)"
|
|
185
|
+
[ -n "$ver" ] || { echo "::error::flake.nix pins no ctlVersion. CI runs the norsk-ctl this commit pins (upgrade-latest moves it); add ctlVersion = \"<version>\"; to flake.nix."; exit 1; }
|
|
186
|
+
echo "norsk-ctl pinned by flake.nix ctlVersion -> $ver"
|
|
183
187
|
curl -fsSL --retry 3 --retry-all-errors --retry-delay 2 "$S3/$ver/norsk-ctl-$ver-linux-x64" -o "$RUNNER_TEMP/norsk-ctl"
|
|
184
188
|
chmod +x "$RUNNER_TEMP/norsk-ctl"
|
|
185
189
|
"$RUNNER_TEMP/norsk-ctl" --version || true
|
package/conventions/smoke.yml
CHANGED
|
@@ -30,9 +30,9 @@ on:
|
|
|
30
30
|
workflow_dispatch:
|
|
31
31
|
inputs:
|
|
32
32
|
ctl_channel:
|
|
33
|
-
description: "norsk-ctl S3 channel to test against (release-gate dispatches 'rc' to validate a candidate before promoting latest
|
|
33
|
+
description: "norsk-ctl S3 channel to test against in place of the repo's pin (flake.nix ctlVersion). Empty runs the pin; release-gate dispatches 'rc' to validate a candidate before promoting latest."
|
|
34
34
|
type: string
|
|
35
|
-
default:
|
|
35
|
+
default: ""
|
|
36
36
|
|
|
37
37
|
permissions:
|
|
38
38
|
contents: read
|
|
@@ -196,18 +196,34 @@ jobs:
|
|
|
196
196
|
# Pick the binary for THIS runner's architecture: the GPU box is aarch64,
|
|
197
197
|
# the x64 pool is x86_64, and the wrong one dies with "Exec format error"
|
|
198
198
|
# deep in the suite. No `|| true` on --version: a bad binary must fail here.
|
|
199
|
-
|
|
199
|
+
#
|
|
200
|
+
# WHICH binary: the version this commit pins, flake.nix ctlVersion -- the
|
|
201
|
+
# daemon the dev shell launches, moved by upgrade-latest's bump PR. It used
|
|
202
|
+
# to be S3's moving `latest`, read at run time, so a result could change
|
|
203
|
+
# with no product commit and a re-run was never a real repeat (funke went
|
|
204
|
+
# red and green again on 2026-10-09 without a commit either time;
|
|
205
|
+
# ci-workflows docs/dashboard/ci-fragility.md, proposal A). A ctl-candidate
|
|
206
|
+
# dispatch still tests its payload's channel (rc) -- that is what it is
|
|
207
|
+
# for -- and a by-hand ctl_channel may too. ctl-pinned-in-repo.test.ts.
|
|
208
|
+
- name: Download the pinned norsk-ctl binary
|
|
209
|
+
env:
|
|
210
|
+
CTL_CHANNEL: ${{ github.event.client_payload.channel || inputs.ctl_channel }}
|
|
200
211
|
run: |
|
|
201
212
|
set -euo pipefail
|
|
202
|
-
channel="${{ github.event.client_payload.channel || inputs.ctl_channel || 'latest' }}"
|
|
203
213
|
case "$(uname -m)" in
|
|
204
214
|
x86_64) arch=linux-x64 ;;
|
|
205
215
|
aarch64 | arm64) arch=linux-arm64 ;;
|
|
206
216
|
*) echo "unsupported runner arch: $(uname -m)" >&2; exit 1 ;;
|
|
207
217
|
esac
|
|
208
218
|
S3="https://s3.eu-west-1.amazonaws.com/norsk.video/norsk-ctl"
|
|
209
|
-
|
|
210
|
-
|
|
219
|
+
if [ -n "${CTL_CHANNEL:-}" ]; then
|
|
220
|
+
ver="$(curl -fsSL --retry 3 --retry-all-errors --retry-delay 2 "$S3/$CTL_CHANNEL")"
|
|
221
|
+
echo "norsk-ctl $CTL_CHANNEL channel -> $ver ($arch), in place of the repo pin"
|
|
222
|
+
else
|
|
223
|
+
ver="$( { sed -nE 's/^[[:space:]]*ctlVersion = "([^"]+)";.*/\1/p' flake.nix 2>/dev/null || true; } | head -1)"
|
|
224
|
+
[ -n "$ver" ] || { echo "::error::flake.nix pins no ctlVersion. CI runs the norsk-ctl this commit pins (upgrade-latest moves it); add ctlVersion = \"<version>\"; to flake.nix."; exit 1; }
|
|
225
|
+
echo "norsk-ctl pinned by flake.nix ctlVersion -> $ver ($arch)"
|
|
226
|
+
fi
|
|
211
227
|
curl -fsSL --retry 3 --retry-all-errors --retry-delay 2 "$S3/$ver/norsk-ctl-$ver-$arch" -o "$RUNNER_TEMP/norsk-ctl"
|
|
212
228
|
chmod +x "$RUNNER_TEMP/norsk-ctl"
|
|
213
229
|
"$RUNNER_TEMP/norsk-ctl" --version
|
|
@@ -18,11 +18,12 @@
|
|
|
18
18
|
# nothing that morning. It is picked up the next time any pin moves, which for
|
|
19
19
|
# this family is most days.
|
|
20
20
|
#
|
|
21
|
-
#
|
|
22
|
-
#
|
|
23
|
-
#
|
|
24
|
-
#
|
|
25
|
-
#
|
|
21
|
+
# This NEVER pushes to main -- and since ci-fragility proposal B neither do
|
|
22
|
+
# upgrade-latest or sync-dev-kit, which once landed on a green gate. Its gate
|
|
23
|
+
# is drift + lint + typecheck + unit, and a bump to the test harness changes
|
|
24
|
+
# the code that drives smoke and integration -- neither of which that gate
|
|
25
|
+
# runs, because both need Docker and a licence; the PR's own tiers do, and
|
|
26
|
+
# bot-prs merges it only once they pass. A red gate
|
|
26
27
|
# still opens one, as a draft: adopting a breaking change is real engineering,
|
|
27
28
|
# not a copy-paste, and the PR is where that work belongs.
|
|
28
29
|
#
|
|
@@ -116,7 +117,7 @@ jobs:
|
|
|
116
117
|
set -euo pipefail
|
|
117
118
|
reg="https://registry.npmjs.org"
|
|
118
119
|
# The conventions package is deliberately absent: sync-dev-kit owns it,
|
|
119
|
-
#
|
|
120
|
+
# bumps it in its own PR, and re-syncs the copied files with it.
|
|
120
121
|
# The moq-* members are the @norskvideo fork of moq-dev/moq that the
|
|
121
122
|
# dashboards play through. They pin each other at one version, so all
|
|
122
123
|
# nine move together, including the ones a product only gets
|