@biffo/cli 0.226.0 → 0.228.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -0,0 +1,306 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Is the integration branch actually healthy — including the deploy, and
|
|
4
|
+
# including who broke it?
|
|
5
|
+
#
|
|
6
|
+
# ## Why this exists
|
|
7
|
+
#
|
|
8
|
+
# On 2026-08-02 an instance's `dev` deploy went red at 10:43 and nobody noticed
|
|
9
|
+
# until 12:36. **1h53m, four further merges**, each of which also failed, and
|
|
10
|
+
# development was effectively blocked for 2h25m. Three separate things had to be
|
|
11
|
+
# true at once for that to happen, and this script answers all three (#1133).
|
|
12
|
+
#
|
|
13
|
+
# **1. The obvious command hides the deploy.** An instance runs five workflows on
|
|
14
|
+
# a merge to `dev` — `CI`, `CodeQL`, `Core Version Tag`, `Deploy Application`,
|
|
15
|
+
# `RLS Tests` — and the reflexive check is:
|
|
16
|
+
#
|
|
17
|
+
# gh run list --branch dev --limit 3
|
|
18
|
+
#
|
|
19
|
+
# which returns three that are *not* the deploy. "dev CI green" was reported
|
|
20
|
+
# truthfully and repeatedly by an agent following AGENTS.md, while the deploy was
|
|
21
|
+
# red the whole time. A truncated list is not a status; it is a sample. This
|
|
22
|
+
# script enumerates **every** workflow that ran on the branch and reports the
|
|
23
|
+
# latest conclusion of each, so nothing can fall off the bottom.
|
|
24
|
+
#
|
|
25
|
+
# **2. A red post-merge deploy has no audience.** A failing PR check is noticed
|
|
26
|
+
# because someone is watching their PR. A failing *post-merge* deploy is watched
|
|
27
|
+
# by nobody: the author has moved on, and the next person only discovers it by
|
|
28
|
+
# merging into it. So this notifies, reusing the desktop-alert channel
|
|
29
|
+
# `practices-daily.sh` established (`_notify`) — same opt-OUT posture, because an
|
|
30
|
+
# opt-in alert is how that notification once spent months existing and never
|
|
31
|
+
# firing.
|
|
32
|
+
#
|
|
33
|
+
# **3. A poisoned branch blames the wrong person.** Once `dev` is broken every
|
|
34
|
+
# subsequent merge fails too, so four people each saw *their* change fail. The
|
|
35
|
+
# expensive part was not the breakage, it was four independent diagnoses of an
|
|
36
|
+
# innocent change. On failure this walks the run history back to the **first
|
|
37
|
+
# failing run** and names its commit, author and time — so the fifth person in
|
|
38
|
+
# starts at the real cause instead of their own diff.
|
|
39
|
+
#
|
|
40
|
+
# ## Exit codes
|
|
41
|
+
#
|
|
42
|
+
# 0 every workflow observed on the branch concluded, none failed
|
|
43
|
+
# 1 something failed — the workflow names and the first bad commit are printed
|
|
44
|
+
# 2 cannot determine: no runs found, or the repo/branch is unreadable
|
|
45
|
+
#
|
|
46
|
+
# 2 is deliberately not 0, matching `wait-for-checks.sh` and `ci-wiring-audit.sh`.
|
|
47
|
+
# A check that cannot see its input must say so rather than passing: this
|
|
48
|
+
# estate's most repeated defect is a zero that means "could not look", and
|
|
49
|
+
# `protection-audit.sh` was reporting `27 branches, all protected` while silently
|
|
50
|
+
# dropping the four repos least likely to be protected (#1145).
|
|
51
|
+
#
|
|
52
|
+
# `cancelled` is reported but does not fail the branch. On self-hosted runners it
|
|
53
|
+
# is usually spot reclamation or a `cancel-in-progress` concurrency group — two
|
|
54
|
+
# merges landing seconds apart cancel the first run by design. It is called out
|
|
55
|
+
# by name so nobody debugs a phantom.
|
|
56
|
+
#
|
|
57
|
+
# ## Usage
|
|
58
|
+
#
|
|
59
|
+
# sh scripts/branch-health.sh [-R owner/repo] [--branch dev] [--quiet]
|
|
60
|
+
#
|
|
61
|
+
# Requires `gh`, authenticated. Uses gh's embedded jq, so no jq binary is needed.
|
|
62
|
+
|
|
63
|
+
set -uo pipefail
|
|
64
|
+
|
|
65
|
+
REPO=""
|
|
66
|
+
BRANCH=""
|
|
67
|
+
QUIET=""
|
|
68
|
+
|
|
69
|
+
usage() {
|
|
70
|
+
sed -n '2,60p' "$0" | sed 's/^# \{0,1\}//'
|
|
71
|
+
exit 2
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
while [ $# -gt 0 ]; do
|
|
75
|
+
case "$1" in
|
|
76
|
+
-R | --repo)
|
|
77
|
+
REPO="${2:-}"
|
|
78
|
+
shift 2
|
|
79
|
+
;;
|
|
80
|
+
--branch)
|
|
81
|
+
BRANCH="${2:-}"
|
|
82
|
+
shift 2
|
|
83
|
+
;;
|
|
84
|
+
--quiet)
|
|
85
|
+
QUIET=1
|
|
86
|
+
shift
|
|
87
|
+
;;
|
|
88
|
+
-h | --help) usage ;;
|
|
89
|
+
*)
|
|
90
|
+
echo "branch-health: unexpected argument '$1'" >&2
|
|
91
|
+
usage
|
|
92
|
+
;;
|
|
93
|
+
esac
|
|
94
|
+
done
|
|
95
|
+
|
|
96
|
+
RED=$(printf '\033[31m')
|
|
97
|
+
GREEN=$(printf '\033[32m')
|
|
98
|
+
YELLOW=$(printf '\033[33m')
|
|
99
|
+
DIM=$(printf '\033[90m')
|
|
100
|
+
OFF=$(printf '\033[0m')
|
|
101
|
+
|
|
102
|
+
# A real tab, built the same way the colours above are, and used as `IFS="$TAB"`.
|
|
103
|
+
#
|
|
104
|
+
# AGENTS.md invokes these scripts as `sh scripts/...`, and /bin/sh is dash here.
|
|
105
|
+
# The bash spelling `IFS=$'\t'` is NOT a syntax error under dash — it is read as
|
|
106
|
+
# the four literal characters `$ ' \ t`, so the field split then happens on any
|
|
107
|
+
# of them. The first draft of this script did exactly that and reported a
|
|
108
|
+
# workflow called "Deploy Applica", having split "Deploy Application" at its
|
|
109
|
+
# 't'. It printed one row instead of four and still exited 0.
|
|
110
|
+
#
|
|
111
|
+
# Same reason the colours use `$(printf '\033[31m')` rather than `$'\e[31m'`.
|
|
112
|
+
TAB=$(printf '\t')
|
|
113
|
+
|
|
114
|
+
gh_run() {
|
|
115
|
+
if [ -n "$REPO" ]; then gh run "$@" --repo "$REPO"; else gh run "$@"; fi
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
# The integration branch is `dev` in every Biffo repo (AGENTS.md §2). Resolved
|
|
119
|
+
# from the repo rather than hardcoded so this still works in the rare repo whose
|
|
120
|
+
# default has not been migrated — and so the failure is "cannot read repo"
|
|
121
|
+
# rather than a confident answer about a branch that does not exist.
|
|
122
|
+
if [ -z "$BRANCH" ]; then
|
|
123
|
+
if [ -n "$REPO" ]; then
|
|
124
|
+
BRANCH=$(gh repo view "$REPO" --json defaultBranchRef --jq .defaultBranchRef.name 2>/dev/null)
|
|
125
|
+
else
|
|
126
|
+
BRANCH=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name 2>/dev/null)
|
|
127
|
+
fi
|
|
128
|
+
fi
|
|
129
|
+
|
|
130
|
+
if [ -z "$BRANCH" ]; then
|
|
131
|
+
echo "${RED}branch-health: cannot determine the integration branch.${OFF}" >&2
|
|
132
|
+
exit 2
|
|
133
|
+
fi
|
|
134
|
+
|
|
135
|
+
label=${REPO:-$(basename "$(git rev-parse --show-toplevel 2>/dev/null || pwd)")}
|
|
136
|
+
|
|
137
|
+
# --- Every workflow that ran on the branch, not the first three --------------
|
|
138
|
+
#
|
|
139
|
+
# 200 runs is deep enough to reach every workflow's latest run even when a busy
|
|
140
|
+
# one (CI) dominates the head of the list. `--json` on `gh run list` returns them
|
|
141
|
+
# newest-first, so the first row per workflow name IS its latest run.
|
|
142
|
+
|
|
143
|
+
# `group_by | max_by(.createdAt)` rather than "first occurrence in a newest-first
|
|
144
|
+
# list": the ordering is gh's to change, and a status tool that quietly reports
|
|
145
|
+
# an older run because an API changed its sort is the same class of defect as the
|
|
146
|
+
# truncated list this replaces. Ask for the newest explicitly.
|
|
147
|
+
summary=$(gh_run list --branch "$BRANCH" --limit 200 \
|
|
148
|
+
--json workflowName,status,conclusion,headSha,createdAt,url \
|
|
149
|
+
--jq 'group_by(.workflowName)
|
|
150
|
+
| map(max_by(.createdAt))
|
|
151
|
+
| .[]
|
|
152
|
+
| [ (if .status == "completed" then (.conclusion // "unknown") else .status end),
|
|
153
|
+
.workflowName, .headSha[0:8], .createdAt[0:16], .url ]
|
|
154
|
+
| @tsv' 2>/dev/null)
|
|
155
|
+
|
|
156
|
+
if [ -z "$summary" ]; then
|
|
157
|
+
echo "${RED}branch-health: no workflow runs readable on '$BRANCH' in $label.${OFF}" >&2
|
|
158
|
+
echo "${DIM} That is 'cannot tell', not 'healthy' — exiting 2.${OFF}" >&2
|
|
159
|
+
exit 2
|
|
160
|
+
fi
|
|
161
|
+
|
|
162
|
+
failed=""
|
|
163
|
+
pending=""
|
|
164
|
+
cancelled=""
|
|
165
|
+
skipped=""
|
|
166
|
+
ok=""
|
|
167
|
+
|
|
168
|
+
while IFS="$TAB" read -r state name sha when url; do
|
|
169
|
+
[ -n "$name" ] || continue
|
|
170
|
+
case "$state" in
|
|
171
|
+
success) ok="${ok}${name}\n" ;;
|
|
172
|
+
failure | timed_out | startup_failure)
|
|
173
|
+
failed="${failed}${state}\t${name}\t${sha}\t${when}\t${url}\n"
|
|
174
|
+
;;
|
|
175
|
+
cancelled) cancelled="${cancelled}${name}\n" ;;
|
|
176
|
+
skipped) skipped="${skipped}${name}\n" ;;
|
|
177
|
+
*) pending="${pending}${state}\t${name}\n" ;;
|
|
178
|
+
esac
|
|
179
|
+
done <<EOF
|
|
180
|
+
$summary
|
|
181
|
+
EOF
|
|
182
|
+
|
|
183
|
+
# --- Report -------------------------------------------------------------------
|
|
184
|
+
|
|
185
|
+
echo "${DIM}$label — branch '$BRANCH', latest run per workflow${OFF}"
|
|
186
|
+
echo
|
|
187
|
+
|
|
188
|
+
# `printf '%b'` FIRST, then sed. These lists are accumulated as strings holding
|
|
189
|
+
# literal `\n` two-character sequences (POSIX sh has no clean way to append a
|
|
190
|
+
# real newline to a variable), so piping them straight to sed hands it a single
|
|
191
|
+
# line and only the first entry gets its prefix — which read as a workflow with
|
|
192
|
+
# no status at all. Expand the escapes, then prefix each real line.
|
|
193
|
+
[ -n "$ok" ] && printf '%b' "$ok" | sed "s/^/ ${GREEN}ok${OFF} /"
|
|
194
|
+
[ -n "$skipped" ] && printf '%b' "$skipped" | sed "s/^/ ${DIM}skipped${OFF} /"
|
|
195
|
+
[ -n "$cancelled" ] && printf '%b' "$cancelled" | sed "s/^/ ${YELLOW}cancelled${OFF} /"
|
|
196
|
+
|
|
197
|
+
if [ -n "$pending" ]; then
|
|
198
|
+
printf '%b' "$pending" | awk -F'\t' -v d="$YELLOW" -v o="$OFF" 'NF{printf " %srunning%s %s (%s)\n", d, o, $2, $1}'
|
|
199
|
+
fi
|
|
200
|
+
|
|
201
|
+
if [ -z "$failed" ]; then
|
|
202
|
+
if [ -n "$cancelled" ]; then
|
|
203
|
+
echo
|
|
204
|
+
echo "${DIM}A cancelled run is usually spot reclamation or a superseded concurrency${OFF}"
|
|
205
|
+
echo "${DIM}group, not the code. Re-run it rather than debugging it.${OFF}"
|
|
206
|
+
fi
|
|
207
|
+
echo
|
|
208
|
+
echo "${GREEN}Nothing on '$BRANCH' is failing.${OFF}"
|
|
209
|
+
exit 0
|
|
210
|
+
fi
|
|
211
|
+
|
|
212
|
+
echo
|
|
213
|
+
printf '%b' "$failed" | awk -F'\t' -v r="$RED" -v o="$OFF" 'NF{printf " %s%s%s %s at %s %s\n", r, $1, o, $2, $3, $5}'
|
|
214
|
+
|
|
215
|
+
# --- Who actually broke it ----------------------------------------------------
|
|
216
|
+
#
|
|
217
|
+
# The whole point of #1133's third defect. Walk this workflow's runs on this
|
|
218
|
+
# branch backwards from the newest failure through consecutive failures, and
|
|
219
|
+
# report the OLDEST one in that unbroken streak. That run's commit is where the
|
|
220
|
+
# breakage started, which is very often not the person now reading this.
|
|
221
|
+
|
|
222
|
+
echo
|
|
223
|
+
printf '%b' "$failed" | while IFS="$TAB" read -r state name sha when url; do
|
|
224
|
+
[ -n "$name" ] || continue
|
|
225
|
+
|
|
226
|
+
# Sort newest-first ourselves, cut the list at the most recent SUCCESS, and
|
|
227
|
+
# take the oldest failure still inside that streak. Anything before a green run
|
|
228
|
+
# is a different, already-fixed breakage and must not be blamed for this one.
|
|
229
|
+
first=$(gh_run list --branch "$BRANCH" --workflow "$name" --limit 60 \
|
|
230
|
+
--json conclusion,headSha,createdAt,displayTitle,url \
|
|
231
|
+
--jq 'sort_by(.createdAt) | reverse
|
|
232
|
+
| (map(.conclusion == "success") | index(true)) as $green
|
|
233
|
+
| .[0: (if $green == null then length else $green end)]
|
|
234
|
+
| map(select(.conclusion == "failure"
|
|
235
|
+
or .conclusion == "timed_out"
|
|
236
|
+
or .conclusion == "startup_failure"))
|
|
237
|
+
| last
|
|
238
|
+
| select(. != null)
|
|
239
|
+
| [ .headSha[0:8], .createdAt[0:16], (.displayTitle // "")[0:72], .url ]
|
|
240
|
+
| @tsv' 2>/dev/null)
|
|
241
|
+
|
|
242
|
+
if [ -n "$first" ]; then
|
|
243
|
+
f_sha=$(printf '%s' "$first" | cut -f1)
|
|
244
|
+
f_when=$(printf '%s' "$first" | cut -f2)
|
|
245
|
+
f_title=$(printf '%s' "$first" | cut -f3)
|
|
246
|
+
f_url=$(printf '%s' "$first" | cut -f4)
|
|
247
|
+
echo " ${RED}$name${OFF} has been failing since ${YELLOW}$f_sha${OFF} ($f_when)"
|
|
248
|
+
echo " $f_title"
|
|
249
|
+
echo " ${DIM}$f_url${OFF}"
|
|
250
|
+
if [ "$f_sha" != "$sha" ]; then
|
|
251
|
+
echo " ${DIM}The newest failure is at $sha — but it is NOT where this started.${OFF}"
|
|
252
|
+
echo " ${DIM}Diagnose $f_sha, not your own merge.${OFF}"
|
|
253
|
+
fi
|
|
254
|
+
else
|
|
255
|
+
echo " ${RED}$name${OFF} is failing at $sha ${DIM}(could not establish when it started)${OFF}"
|
|
256
|
+
fi
|
|
257
|
+
done
|
|
258
|
+
|
|
259
|
+
# --- Tell somebody ------------------------------------------------------------
|
|
260
|
+
#
|
|
261
|
+
# Copied in posture, deliberately, from practices-daily.sh's `_notify`: opt-OUT
|
|
262
|
+
# via an env var rather than opt-in, because an opt-in alert is one that never
|
|
263
|
+
# fires. Replaces its own previous card rather than stacking, so a branch red for
|
|
264
|
+
# three days is one notification and not three.
|
|
265
|
+
|
|
266
|
+
_notify() {
|
|
267
|
+
[ -z "$QUIET" ] || return 0
|
|
268
|
+
command -v notify-send >/dev/null 2>&1 || return 0
|
|
269
|
+
[ -z "${BRANCH_HEALTH_NO_DESKTOP_ALERT:-}" ] || return 0
|
|
270
|
+
|
|
271
|
+
if [ -z "${DBUS_SESSION_BUS_ADDRESS:-}" ]; then
|
|
272
|
+
_bus="/run/user/$(id -u)/bus"
|
|
273
|
+
[ -S "$_bus" ] || return 0
|
|
274
|
+
DBUS_SESSION_BUS_ADDRESS="unix:path=$_bus"
|
|
275
|
+
export DBUS_SESSION_BUS_ADDRESS
|
|
276
|
+
fi
|
|
277
|
+
|
|
278
|
+
_slug=$(printf '%s' "$label-$BRANCH" | tr -c 'a-zA-Z0-9' '-')
|
|
279
|
+
_idfile="${XDG_RUNTIME_DIR:-/tmp}/biffo-branch-health-${_slug}.id"
|
|
280
|
+
_prev=""
|
|
281
|
+
[ -f "$_idfile" ] && _prev=$(cat "$_idfile" 2>/dev/null)
|
|
282
|
+
|
|
283
|
+
# Built as a plain variable rather than `${_prev:+--replace-id="$_prev"}`.
|
|
284
|
+
# That form nests double quotes inside a parameter expansion, which dash
|
|
285
|
+
# refuses to parse — and because the script is run as `sh`, the failure lands
|
|
286
|
+
# at RUNTIME, after all the useful output has already printed, turning a
|
|
287
|
+
# correct exit 1 into a confusing exit 2.
|
|
288
|
+
_replace=""
|
|
289
|
+
[ -n "$_prev" ] && _replace="--replace-id=$_prev"
|
|
290
|
+
|
|
291
|
+
_names=$(printf '%b' "$failed" | awk -F'\t' 'NF{printf "%s ", $2}')
|
|
292
|
+
# Deliberately unquoted: empty must expand to no argument at all.
|
|
293
|
+
# shellcheck disable=SC2086
|
|
294
|
+
_new=$(notify-send --print-id $_replace \
|
|
295
|
+
-u critical -a "biffo" \
|
|
296
|
+
"$label: $BRANCH is red" \
|
|
297
|
+
"$_names— nobody is watching a post-merge failure. sh scripts/branch-health.sh" 2>/dev/null)
|
|
298
|
+
[ -n "$_new" ] && printf '%s' "$_new" > "$_idfile" 2>/dev/null
|
|
299
|
+
return 0
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
_notify
|
|
303
|
+
|
|
304
|
+
echo
|
|
305
|
+
echo "${RED}'$BRANCH' is red. It blocks everyone — fixing it is the next task (AGENTS.md §6).${OFF}"
|
|
306
|
+
exit 1
|
|
@@ -0,0 +1,306 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
#
|
|
3
|
+
# Is the integration branch actually healthy — including the deploy, and
|
|
4
|
+
# including who broke it?
|
|
5
|
+
#
|
|
6
|
+
# ## Why this exists
|
|
7
|
+
#
|
|
8
|
+
# On 2026-08-02 an instance's `dev` deploy went red at 10:43 and nobody noticed
|
|
9
|
+
# until 12:36. **1h53m, four further merges**, each of which also failed, and
|
|
10
|
+
# development was effectively blocked for 2h25m. Three separate things had to be
|
|
11
|
+
# true at once for that to happen, and this script answers all three (#1133).
|
|
12
|
+
#
|
|
13
|
+
# **1. The obvious command hides the deploy.** An instance runs five workflows on
|
|
14
|
+
# a merge to `dev` — `CI`, `CodeQL`, `Core Version Tag`, `Deploy Application`,
|
|
15
|
+
# `RLS Tests` — and the reflexive check is:
|
|
16
|
+
#
|
|
17
|
+
# gh run list --branch dev --limit 3
|
|
18
|
+
#
|
|
19
|
+
# which returns three that are *not* the deploy. "dev CI green" was reported
|
|
20
|
+
# truthfully and repeatedly by an agent following AGENTS.md, while the deploy was
|
|
21
|
+
# red the whole time. A truncated list is not a status; it is a sample. This
|
|
22
|
+
# script enumerates **every** workflow that ran on the branch and reports the
|
|
23
|
+
# latest conclusion of each, so nothing can fall off the bottom.
|
|
24
|
+
#
|
|
25
|
+
# **2. A red post-merge deploy has no audience.** A failing PR check is noticed
|
|
26
|
+
# because someone is watching their PR. A failing *post-merge* deploy is watched
|
|
27
|
+
# by nobody: the author has moved on, and the next person only discovers it by
|
|
28
|
+
# merging into it. So this notifies, reusing the desktop-alert channel
|
|
29
|
+
# `practices-daily.sh` established (`_notify`) — same opt-OUT posture, because an
|
|
30
|
+
# opt-in alert is how that notification once spent months existing and never
|
|
31
|
+
# firing.
|
|
32
|
+
#
|
|
33
|
+
# **3. A poisoned branch blames the wrong person.** Once `dev` is broken every
|
|
34
|
+
# subsequent merge fails too, so four people each saw *their* change fail. The
|
|
35
|
+
# expensive part was not the breakage, it was four independent diagnoses of an
|
|
36
|
+
# innocent change. On failure this walks the run history back to the **first
|
|
37
|
+
# failing run** and names its commit, author and time — so the fifth person in
|
|
38
|
+
# starts at the real cause instead of their own diff.
|
|
39
|
+
#
|
|
40
|
+
# ## Exit codes
|
|
41
|
+
#
|
|
42
|
+
# 0 every workflow observed on the branch concluded, none failed
|
|
43
|
+
# 1 something failed — the workflow names and the first bad commit are printed
|
|
44
|
+
# 2 cannot determine: no runs found, or the repo/branch is unreadable
|
|
45
|
+
#
|
|
46
|
+
# 2 is deliberately not 0, matching `wait-for-checks.sh` and `ci-wiring-audit.sh`.
|
|
47
|
+
# A check that cannot see its input must say so rather than passing: this
|
|
48
|
+
# estate's most repeated defect is a zero that means "could not look", and
|
|
49
|
+
# `protection-audit.sh` was reporting `27 branches, all protected` while silently
|
|
50
|
+
# dropping the four repos least likely to be protected (#1145).
|
|
51
|
+
#
|
|
52
|
+
# `cancelled` is reported but does not fail the branch. On self-hosted runners it
|
|
53
|
+
# is usually spot reclamation or a `cancel-in-progress` concurrency group — two
|
|
54
|
+
# merges landing seconds apart cancel the first run by design. It is called out
|
|
55
|
+
# by name so nobody debugs a phantom.
|
|
56
|
+
#
|
|
57
|
+
# ## Usage
|
|
58
|
+
#
|
|
59
|
+
# sh scripts/branch-health.sh [-R owner/repo] [--branch dev] [--quiet]
|
|
60
|
+
#
|
|
61
|
+
# Requires `gh`, authenticated. Uses gh's embedded jq, so no jq binary is needed.
|
|
62
|
+
|
|
63
|
+
set -uo pipefail
|
|
64
|
+
|
|
65
|
+
REPO=""
|
|
66
|
+
BRANCH=""
|
|
67
|
+
QUIET=""
|
|
68
|
+
|
|
69
|
+
usage() {
|
|
70
|
+
sed -n '2,60p' "$0" | sed 's/^# \{0,1\}//'
|
|
71
|
+
exit 2
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
while [ $# -gt 0 ]; do
|
|
75
|
+
case "$1" in
|
|
76
|
+
-R | --repo)
|
|
77
|
+
REPO="${2:-}"
|
|
78
|
+
shift 2
|
|
79
|
+
;;
|
|
80
|
+
--branch)
|
|
81
|
+
BRANCH="${2:-}"
|
|
82
|
+
shift 2
|
|
83
|
+
;;
|
|
84
|
+
--quiet)
|
|
85
|
+
QUIET=1
|
|
86
|
+
shift
|
|
87
|
+
;;
|
|
88
|
+
-h | --help) usage ;;
|
|
89
|
+
*)
|
|
90
|
+
echo "branch-health: unexpected argument '$1'" >&2
|
|
91
|
+
usage
|
|
92
|
+
;;
|
|
93
|
+
esac
|
|
94
|
+
done
|
|
95
|
+
|
|
96
|
+
RED=$(printf '\033[31m')
|
|
97
|
+
GREEN=$(printf '\033[32m')
|
|
98
|
+
YELLOW=$(printf '\033[33m')
|
|
99
|
+
DIM=$(printf '\033[90m')
|
|
100
|
+
OFF=$(printf '\033[0m')
|
|
101
|
+
|
|
102
|
+
# A real tab, built the same way the colours above are, and used as `IFS="$TAB"`.
|
|
103
|
+
#
|
|
104
|
+
# AGENTS.md invokes these scripts as `sh scripts/...`, and /bin/sh is dash here.
|
|
105
|
+
# The bash spelling `IFS=$'\t'` is NOT a syntax error under dash — it is read as
|
|
106
|
+
# the four literal characters `$ ' \ t`, so the field split then happens on any
|
|
107
|
+
# of them. The first draft of this script did exactly that and reported a
|
|
108
|
+
# workflow called "Deploy Applica", having split "Deploy Application" at its
|
|
109
|
+
# 't'. It printed one row instead of four and still exited 0.
|
|
110
|
+
#
|
|
111
|
+
# Same reason the colours use `$(printf '\033[31m')` rather than `$'\e[31m'`.
|
|
112
|
+
TAB=$(printf '\t')
|
|
113
|
+
|
|
114
|
+
gh_run() {
|
|
115
|
+
if [ -n "$REPO" ]; then gh run "$@" --repo "$REPO"; else gh run "$@"; fi
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
# The integration branch is `dev` in every Biffo repo (AGENTS.md §2). Resolved
|
|
119
|
+
# from the repo rather than hardcoded so this still works in the rare repo whose
|
|
120
|
+
# default has not been migrated — and so the failure is "cannot read repo"
|
|
121
|
+
# rather than a confident answer about a branch that does not exist.
|
|
122
|
+
if [ -z "$BRANCH" ]; then
|
|
123
|
+
if [ -n "$REPO" ]; then
|
|
124
|
+
BRANCH=$(gh repo view "$REPO" --json defaultBranchRef --jq .defaultBranchRef.name 2>/dev/null)
|
|
125
|
+
else
|
|
126
|
+
BRANCH=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name 2>/dev/null)
|
|
127
|
+
fi
|
|
128
|
+
fi
|
|
129
|
+
|
|
130
|
+
if [ -z "$BRANCH" ]; then
|
|
131
|
+
echo "${RED}branch-health: cannot determine the integration branch.${OFF}" >&2
|
|
132
|
+
exit 2
|
|
133
|
+
fi
|
|
134
|
+
|
|
135
|
+
label=${REPO:-$(basename "$(git rev-parse --show-toplevel 2>/dev/null || pwd)")}
|
|
136
|
+
|
|
137
|
+
# --- Every workflow that ran on the branch, not the first three --------------
|
|
138
|
+
#
|
|
139
|
+
# 200 runs is deep enough to reach every workflow's latest run even when a busy
|
|
140
|
+
# one (CI) dominates the head of the list. `--json` on `gh run list` returns them
|
|
141
|
+
# newest-first, so the first row per workflow name IS its latest run.
|
|
142
|
+
|
|
143
|
+
# `group_by | max_by(.createdAt)` rather than "first occurrence in a newest-first
|
|
144
|
+
# list": the ordering is gh's to change, and a status tool that quietly reports
|
|
145
|
+
# an older run because an API changed its sort is the same class of defect as the
|
|
146
|
+
# truncated list this replaces. Ask for the newest explicitly.
|
|
147
|
+
summary=$(gh_run list --branch "$BRANCH" --limit 200 \
|
|
148
|
+
--json workflowName,status,conclusion,headSha,createdAt,url \
|
|
149
|
+
--jq 'group_by(.workflowName)
|
|
150
|
+
| map(max_by(.createdAt))
|
|
151
|
+
| .[]
|
|
152
|
+
| [ (if .status == "completed" then (.conclusion // "unknown") else .status end),
|
|
153
|
+
.workflowName, .headSha[0:8], .createdAt[0:16], .url ]
|
|
154
|
+
| @tsv' 2>/dev/null)
|
|
155
|
+
|
|
156
|
+
if [ -z "$summary" ]; then
|
|
157
|
+
echo "${RED}branch-health: no workflow runs readable on '$BRANCH' in $label.${OFF}" >&2
|
|
158
|
+
echo "${DIM} That is 'cannot tell', not 'healthy' — exiting 2.${OFF}" >&2
|
|
159
|
+
exit 2
|
|
160
|
+
fi
|
|
161
|
+
|
|
162
|
+
failed=""
|
|
163
|
+
pending=""
|
|
164
|
+
cancelled=""
|
|
165
|
+
skipped=""
|
|
166
|
+
ok=""
|
|
167
|
+
|
|
168
|
+
while IFS="$TAB" read -r state name sha when url; do
|
|
169
|
+
[ -n "$name" ] || continue
|
|
170
|
+
case "$state" in
|
|
171
|
+
success) ok="${ok}${name}\n" ;;
|
|
172
|
+
failure | timed_out | startup_failure)
|
|
173
|
+
failed="${failed}${state}\t${name}\t${sha}\t${when}\t${url}\n"
|
|
174
|
+
;;
|
|
175
|
+
cancelled) cancelled="${cancelled}${name}\n" ;;
|
|
176
|
+
skipped) skipped="${skipped}${name}\n" ;;
|
|
177
|
+
*) pending="${pending}${state}\t${name}\n" ;;
|
|
178
|
+
esac
|
|
179
|
+
done <<EOF
|
|
180
|
+
$summary
|
|
181
|
+
EOF
|
|
182
|
+
|
|
183
|
+
# --- Report -------------------------------------------------------------------
|
|
184
|
+
|
|
185
|
+
echo "${DIM}$label — branch '$BRANCH', latest run per workflow${OFF}"
|
|
186
|
+
echo
|
|
187
|
+
|
|
188
|
+
# `printf '%b'` FIRST, then sed. These lists are accumulated as strings holding
|
|
189
|
+
# literal `\n` two-character sequences (POSIX sh has no clean way to append a
|
|
190
|
+
# real newline to a variable), so piping them straight to sed hands it a single
|
|
191
|
+
# line and only the first entry gets its prefix — which read as a workflow with
|
|
192
|
+
# no status at all. Expand the escapes, then prefix each real line.
|
|
193
|
+
[ -n "$ok" ] && printf '%b' "$ok" | sed "s/^/ ${GREEN}ok${OFF} /"
|
|
194
|
+
[ -n "$skipped" ] && printf '%b' "$skipped" | sed "s/^/ ${DIM}skipped${OFF} /"
|
|
195
|
+
[ -n "$cancelled" ] && printf '%b' "$cancelled" | sed "s/^/ ${YELLOW}cancelled${OFF} /"
|
|
196
|
+
|
|
197
|
+
if [ -n "$pending" ]; then
|
|
198
|
+
printf '%b' "$pending" | awk -F'\t' -v d="$YELLOW" -v o="$OFF" 'NF{printf " %srunning%s %s (%s)\n", d, o, $2, $1}'
|
|
199
|
+
fi
|
|
200
|
+
|
|
201
|
+
if [ -z "$failed" ]; then
|
|
202
|
+
if [ -n "$cancelled" ]; then
|
|
203
|
+
echo
|
|
204
|
+
echo "${DIM}A cancelled run is usually spot reclamation or a superseded concurrency${OFF}"
|
|
205
|
+
echo "${DIM}group, not the code. Re-run it rather than debugging it.${OFF}"
|
|
206
|
+
fi
|
|
207
|
+
echo
|
|
208
|
+
echo "${GREEN}Nothing on '$BRANCH' is failing.${OFF}"
|
|
209
|
+
exit 0
|
|
210
|
+
fi
|
|
211
|
+
|
|
212
|
+
echo
|
|
213
|
+
printf '%b' "$failed" | awk -F'\t' -v r="$RED" -v o="$OFF" 'NF{printf " %s%s%s %s at %s %s\n", r, $1, o, $2, $3, $5}'
|
|
214
|
+
|
|
215
|
+
# --- Who actually broke it ----------------------------------------------------
|
|
216
|
+
#
|
|
217
|
+
# The whole point of #1133's third defect. Walk this workflow's runs on this
|
|
218
|
+
# branch backwards from the newest failure through consecutive failures, and
|
|
219
|
+
# report the OLDEST one in that unbroken streak. That run's commit is where the
|
|
220
|
+
# breakage started, which is very often not the person now reading this.
|
|
221
|
+
|
|
222
|
+
echo
|
|
223
|
+
printf '%b' "$failed" | while IFS="$TAB" read -r state name sha when url; do
|
|
224
|
+
[ -n "$name" ] || continue
|
|
225
|
+
|
|
226
|
+
# Sort newest-first ourselves, cut the list at the most recent SUCCESS, and
|
|
227
|
+
# take the oldest failure still inside that streak. Anything before a green run
|
|
228
|
+
# is a different, already-fixed breakage and must not be blamed for this one.
|
|
229
|
+
first=$(gh_run list --branch "$BRANCH" --workflow "$name" --limit 60 \
|
|
230
|
+
--json conclusion,headSha,createdAt,displayTitle,url \
|
|
231
|
+
--jq 'sort_by(.createdAt) | reverse
|
|
232
|
+
| (map(.conclusion == "success") | index(true)) as $green
|
|
233
|
+
| .[0: (if $green == null then length else $green end)]
|
|
234
|
+
| map(select(.conclusion == "failure"
|
|
235
|
+
or .conclusion == "timed_out"
|
|
236
|
+
or .conclusion == "startup_failure"))
|
|
237
|
+
| last
|
|
238
|
+
| select(. != null)
|
|
239
|
+
| [ .headSha[0:8], .createdAt[0:16], (.displayTitle // "")[0:72], .url ]
|
|
240
|
+
| @tsv' 2>/dev/null)
|
|
241
|
+
|
|
242
|
+
if [ -n "$first" ]; then
|
|
243
|
+
f_sha=$(printf '%s' "$first" | cut -f1)
|
|
244
|
+
f_when=$(printf '%s' "$first" | cut -f2)
|
|
245
|
+
f_title=$(printf '%s' "$first" | cut -f3)
|
|
246
|
+
f_url=$(printf '%s' "$first" | cut -f4)
|
|
247
|
+
echo " ${RED}$name${OFF} has been failing since ${YELLOW}$f_sha${OFF} ($f_when)"
|
|
248
|
+
echo " $f_title"
|
|
249
|
+
echo " ${DIM}$f_url${OFF}"
|
|
250
|
+
if [ "$f_sha" != "$sha" ]; then
|
|
251
|
+
echo " ${DIM}The newest failure is at $sha — but it is NOT where this started.${OFF}"
|
|
252
|
+
echo " ${DIM}Diagnose $f_sha, not your own merge.${OFF}"
|
|
253
|
+
fi
|
|
254
|
+
else
|
|
255
|
+
echo " ${RED}$name${OFF} is failing at $sha ${DIM}(could not establish when it started)${OFF}"
|
|
256
|
+
fi
|
|
257
|
+
done
|
|
258
|
+
|
|
259
|
+
# --- Tell somebody ------------------------------------------------------------
|
|
260
|
+
#
|
|
261
|
+
# Copied in posture, deliberately, from practices-daily.sh's `_notify`: opt-OUT
|
|
262
|
+
# via an env var rather than opt-in, because an opt-in alert is one that never
|
|
263
|
+
# fires. Replaces its own previous card rather than stacking, so a branch red for
|
|
264
|
+
# three days is one notification and not three.
|
|
265
|
+
|
|
266
|
+
_notify() {
|
|
267
|
+
[ -z "$QUIET" ] || return 0
|
|
268
|
+
command -v notify-send >/dev/null 2>&1 || return 0
|
|
269
|
+
[ -z "${BRANCH_HEALTH_NO_DESKTOP_ALERT:-}" ] || return 0
|
|
270
|
+
|
|
271
|
+
if [ -z "${DBUS_SESSION_BUS_ADDRESS:-}" ]; then
|
|
272
|
+
_bus="/run/user/$(id -u)/bus"
|
|
273
|
+
[ -S "$_bus" ] || return 0
|
|
274
|
+
DBUS_SESSION_BUS_ADDRESS="unix:path=$_bus"
|
|
275
|
+
export DBUS_SESSION_BUS_ADDRESS
|
|
276
|
+
fi
|
|
277
|
+
|
|
278
|
+
_slug=$(printf '%s' "$label-$BRANCH" | tr -c 'a-zA-Z0-9' '-')
|
|
279
|
+
_idfile="${XDG_RUNTIME_DIR:-/tmp}/biffo-branch-health-${_slug}.id"
|
|
280
|
+
_prev=""
|
|
281
|
+
[ -f "$_idfile" ] && _prev=$(cat "$_idfile" 2>/dev/null)
|
|
282
|
+
|
|
283
|
+
# Built as a plain variable rather than `${_prev:+--replace-id="$_prev"}`.
|
|
284
|
+
# That form nests double quotes inside a parameter expansion, which dash
|
|
285
|
+
# refuses to parse — and because the script is run as `sh`, the failure lands
|
|
286
|
+
# at RUNTIME, after all the useful output has already printed, turning a
|
|
287
|
+
# correct exit 1 into a confusing exit 2.
|
|
288
|
+
_replace=""
|
|
289
|
+
[ -n "$_prev" ] && _replace="--replace-id=$_prev"
|
|
290
|
+
|
|
291
|
+
_names=$(printf '%b' "$failed" | awk -F'\t' 'NF{printf "%s ", $2}')
|
|
292
|
+
# Deliberately unquoted: empty must expand to no argument at all.
|
|
293
|
+
# shellcheck disable=SC2086
|
|
294
|
+
_new=$(notify-send --print-id $_replace \
|
|
295
|
+
-u critical -a "biffo" \
|
|
296
|
+
"$label: $BRANCH is red" \
|
|
297
|
+
"$_names— nobody is watching a post-merge failure. sh scripts/branch-health.sh" 2>/dev/null)
|
|
298
|
+
[ -n "$_new" ] && printf '%s' "$_new" > "$_idfile" 2>/dev/null
|
|
299
|
+
return 0
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
_notify
|
|
303
|
+
|
|
304
|
+
echo
|
|
305
|
+
echo "${RED}'$BRANCH' is red. It blocks everyone — fixing it is the next task (AGENTS.md §6).${OFF}"
|
|
306
|
+
exit 1
|
package/dist/index.js
CHANGED
|
@@ -3390,6 +3390,18 @@ async function restoreCallerBranch(git, cwd, callerBranch, upgradeBranch) {
|
|
|
3390
3390
|
);
|
|
3391
3391
|
return;
|
|
3392
3392
|
}
|
|
3393
|
+
let dirty;
|
|
3394
|
+
try {
|
|
3395
|
+
dirty = await git.hasUncommittedChanges(cwd);
|
|
3396
|
+
} catch {
|
|
3397
|
+
dirty = true;
|
|
3398
|
+
}
|
|
3399
|
+
if (dirty) {
|
|
3400
|
+
log.warn(
|
|
3401
|
+
`Left ${cwd} on ${upgradeBranch}: it has uncommitted changes from this run, and switching back to ${callerBranch} would silently carry them onto it rather than losing them \u2014 which is worse (#1137). Resolve them on ${upgradeBranch} first: commit them there, or discard the failed attempt with \`git reset --hard\` / \`git clean -fd\`, THEN \`git switch ${callerBranch}\`.`
|
|
3402
|
+
);
|
|
3403
|
+
return;
|
|
3404
|
+
}
|
|
3393
3405
|
try {
|
|
3394
3406
|
await git.switchBranch(cwd, callerBranch);
|
|
3395
3407
|
} catch {
|