@memnexus-ai/mx-agent-cli 0.1.285 → 0.1.287
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/agent-config/agents/qa-tester.md +30 -105
- package/dist/agent-config/agents/security-reviewer.md +30 -104
- package/dist/agent-config/coordination-rosters.generated.json +2 -0
- package/dist/agent-config/hooks/memory-checkpoint-reminder.sh +210 -29
- package/dist/agent-config/shared/review-run-validity.md +141 -0
- package/package.json +1 -1
|
@@ -107,114 +107,39 @@ mx-agent-system/docs/testing-conventions.md.
|
|
|
107
107
|
If existing tests fail *before* your changes, that's a pre-existing issue. Note it but
|
|
108
108
|
don't fix it in this scope.
|
|
109
109
|
|
|
110
|
-
**Verify you are testing current bytes.**
|
|
110
|
+
**Verify you are testing current bytes.** The failure modes below produced a complete,
|
|
111
111
|
confident, wrong picture with no error anywhere — checking the identity of what you read is
|
|
112
112
|
the only defense.
|
|
113
113
|
|
|
114
|
-
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
Do not take the SHA from `gh pr view --json headRefOid` either — that is the same API
|
|
144
|
-
surface that served a stale head on #5086. Every one of these failures yields a real,
|
|
145
|
-
head-shaped SHA, so the verdict still *looks* SHA-scoped. A SHA-scoped verdict stays
|
|
146
|
-
honest after the next force-push; a PR-scoped one does not, and a wrongly-captured one
|
|
147
|
-
only looks scoped.
|
|
148
|
-
- When checks were re-run to pick up a fix on main: a `pull_request` run executes
|
|
149
|
-
`refs/pull/N/merge`, not main, and reopening a PR does not reliably refresh that ref.
|
|
150
|
-
Establish which base the runner actually used by reading the job log, not the API summary:
|
|
151
|
-
|
|
152
|
-
```bash
|
|
153
|
-
# 0. This check applies to `pull_request` runs only — they are the ones that
|
|
154
|
-
# execute refs/pull/N/merge. Confirm the trigger before anything else.
|
|
155
|
-
gh run view <run_id> --json event -q .event # expect: pull_request
|
|
156
|
-
|
|
157
|
-
# 1. A run URL gives you a run ID; jobs have their own IDs. Require COMPLETED,
|
|
158
|
-
# and exclude jobs that never ran — a skipped job's log endpoint returns 404
|
|
159
|
-
# with an XML body that greps as silence.
|
|
160
|
-
gh api --paginate repos/OWNER/REPO/actions/runs/<run_id>/jobs \
|
|
161
|
-
--jq '.jobs[] | select(.status == "completed" and .conclusion != "skipped") | {id, name}'
|
|
162
|
-
# Choose the job by NAME: the one behind the check you actually care about.
|
|
163
|
-
# A matrix run has several and they are not guaranteed to share a checkout.
|
|
164
|
-
|
|
165
|
-
# 2. Assert the base THAT job checked out. This must SUCCEED — it is not a
|
|
166
|
-
# scan for a failure line. Run it against a job that actually checked out
|
|
167
|
-
# code: a `skipped` job and a `push`-triggered run both produce NO line, and
|
|
168
|
-
# no line is NOT a pass. It means you cannot judge this run from this job —
|
|
169
|
-
# pick another job, or another run.
|
|
170
|
-
BASE=$(gh api repos/OWNER/REPO/actions/jobs/<job_id>/logs \
|
|
171
|
-
| grep -ao 'Merge [0-9a-f]\{40\} into [0-9a-f]\{40\}' \
|
|
172
|
-
| sed 's/.* into //' | sort -u)
|
|
173
|
-
[ "$(printf '%s' "$BASE" | grep -c .)" -eq 1 ] \
|
|
174
|
-
|| { echo "STUCK: found $(printf '%s' "$BASE" | grep -c .) checkout lines, need exactly 1"; exit 1; }
|
|
175
|
-
|
|
176
|
-
# 3. Compare against a FRESHLY fetched main. Skip the fetch and you compare
|
|
177
|
-
# against a stale local ref, which can agree by accident.
|
|
178
|
-
git fetch origin main
|
|
179
|
-
[ "$BASE" = "$(git rev-parse origin/main)" ] || { echo "STALE BASE — run proved nothing"; exit 1; }
|
|
180
|
-
```
|
|
181
|
-
|
|
182
|
-
A stale base means the run proved nothing — close/reopen and re-check the next run's log.
|
|
183
|
-
"It ran after the fix landed" is not evidence it ran the fix (#5116).
|
|
184
|
-
- **No line found is not a pass.** State the check as an assertion that must succeed, never
|
|
185
|
-
as a scan for a failure line, because empty output is the *default* result in three
|
|
186
|
-
reproducible cases: a skipped job, a `push`-triggered run, and a 404 log body (stderr
|
|
187
|
-
only, empty stdout, exit 1). In all three a mismatch test never fires and the reviewer
|
|
188
|
-
sees nothing that fails. A `push` run checks out the branch tip directly and logs
|
|
189
|
-
`Switched to a new branch` — the merge-ref rule does not apply to it, and neither does
|
|
190
|
-
its reassurance. Match the anchored `Merge <40hex> into <40hex>` form rather than the
|
|
191
|
-
prose prefix: this document itself contains that prose, and an unanchored substring grep
|
|
192
|
-
can eventually match our own guidance instead of a runner's log.
|
|
193
|
-
- **Require the job to have COMPLETED, not merely to be un-skipped.** A running job has
|
|
194
|
-
`conclusion: null` and passes a not-skipped test, so the old filter returned it (measured
|
|
195
|
-
on job `92503438573`). It does not produce a wrong pass: `/actions/jobs/<id>/logs` serves
|
|
196
|
-
a COMPLETED log archive, never a partial one. Measured on the same job before and after
|
|
197
|
-
it finished — HTTP 404 `BlobNotFound` while running, 180 KB containing the base line once
|
|
198
|
-
complete — and confirmed not to be a checkout-timing artifact: a job 213 seconds in, with
|
|
199
|
-
`Checkout code` and seven later steps already `completed`, still returned the 404. So a
|
|
200
|
-
running job greps as silence and lands in the STUCK branch above.
|
|
201
|
-
Require `completed` anyway, because STUCK does not tell you WHICH problem you have. A
|
|
202
|
-
skipped job, a running job, and a purged log all return HTTP 404 with a near-identical
|
|
203
|
-
`BlobNotFound` body — and they call for three different actions (pick another job; wait;
|
|
204
|
-
treat the run as unauditable). Filtering on `completed` removes one of the three up front.
|
|
205
|
-
A job whose conclusion is unknown cannot tell you whether the run validated the fix,
|
|
206
|
-
whatever its log says. If the filter returns nothing, every job is still running — wait.
|
|
207
|
-
- **Retraction — do not use `merge_commit_sha`.** An earlier version of this rule said to
|
|
208
|
-
poll `gh api repos/OWNER/REPO/pulls/N --jq .merge_commit_sha` until it changed. That
|
|
209
|
-
check is wrong and is withdrawn. Measured on #4942: the API reported a *fresh*
|
|
210
|
-
`merge_commit_sha` while the runner still checked out a stale base — run `31063081630`,
|
|
211
|
-
base `3c226df8c`, 36 commits behind the good run's base `5f2bc52d3`, still carrying the
|
|
212
|
-
broken step body — and the run went green, 2.5 minutes and identical check names apart
|
|
213
|
-
from the real one. If you learned the polling rule, unlearn it.
|
|
214
|
-
- Nothing else about the run is load-bearing. On #4942 the check name, the conclusion, the
|
|
215
|
-
check rollup, the timestamp, and `merge_commit_sha` were all **identical** between the
|
|
216
|
-
worthless run and the real one. The job log's `into <base>` was the only field that
|
|
217
|
-
distinguished them.
|
|
114
|
+
**Read `review-run-validity.md` IN FULL before reviewing or testing any PR — it is part of
|
|
115
|
+
this definition.** Find it at `mx-agent-system/agent-config/shared/` (in this repo) or at
|
|
116
|
+
`dist/agent-config/shared/` inside the installed `@memnexus-ai/mx-agent-cli` package (any
|
|
117
|
+
other project — it ships in the bundle but is not copied into `.claude/`). It carries the
|
|
118
|
+
commands and the measured evidence for the rules summarized here: how to pin a PR head
|
|
119
|
+
into your own ref, why `FETCH_HEAD` and `headRefOid` both yield a real-looking but wrong
|
|
120
|
+
SHA, the three-step job-log procedure for proving which base a CI run checked out, why an
|
|
121
|
+
empty grep result is the *default* rather than a pass, and the withdrawn
|
|
122
|
+
`merge_commit_sha` check (#4942).
|
|
123
|
+
|
|
124
|
+
These two rules are load-bearing enough to state here as well as there:
|
|
125
|
+
|
|
126
|
+
- **Stale API view (#5086).** After any force-push, amend, or rebase: `git fetch origin
|
|
127
|
+
<branch>` and diff the ref locally. Never work from `gh pr view` / `gh pr diff` alone —
|
|
128
|
+
the API has served a pre-force-push head, zero comments, and no force-push event on a PR
|
|
129
|
+
that had all three. A comment count or timeline thinner than you expect is a staleness
|
|
130
|
+
signal, not a quiet PR. Name the head SHA you actually tested in the QA report — "PASS at
|
|
131
|
+
2dd3845d1", not "PASS on #5046" — on **every** test run, not only after a force-push.
|
|
132
|
+
Capture that SHA from a ref you pinned yourself (`git fetch origin --force
|
|
133
|
+
refs/pull/<N>/head:refs/pr/<N> && git rev-parse refs/pr/<N>`); never from `FETCH_HEAD` or
|
|
134
|
+
`headRefOid` — both yield a real-looking wrong SHA.
|
|
135
|
+
- **Stale merge ref (#5116).** When checks were re-run to pick up a fix on main: a
|
|
136
|
+
`pull_request` run executes `refs/pull/N/merge`, not main, and reopening a PR does not
|
|
137
|
+
reliably refresh that ref. Prove which base the runner used by reading the job log's
|
|
138
|
+
`Merge <40hex> into <40hex>` line and comparing it to a freshly fetched `origin/main` —
|
|
139
|
+
not the API summary. A stale base means the run proved nothing. "It ran after the fix
|
|
140
|
+
landed" is not evidence it ran the fix, and **no line found is not a pass**. The
|
|
141
|
+
`merge_commit_sha` polling check was disproved on #4942 — do not use it; details in the
|
|
142
|
+
shared file.
|
|
218
143
|
|
|
219
144
|
### 3. Review Generated Tests
|
|
220
145
|
|
|
@@ -79,114 +79,40 @@ Before diving into code, understand what you're protecting:
|
|
|
79
79
|
|
|
80
80
|
### 2. Review Changed Files
|
|
81
81
|
|
|
82
|
-
**First, verify you are reading current bytes.**
|
|
82
|
+
**First, verify you are reading current bytes.** The failure modes below produced a
|
|
83
83
|
complete, confident, wrong picture with no error anywhere — checking the identity of what
|
|
84
84
|
you read is the only defense.
|
|
85
85
|
|
|
86
|
-
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
86
|
+
**Read `review-run-validity.md` IN FULL before reviewing any PR — it is part of this
|
|
87
|
+
definition.** Find it at `mx-agent-system/agent-config/shared/` (in this repo) or at
|
|
88
|
+
`dist/agent-config/shared/` inside the installed `@memnexus-ai/mx-agent-cli` package (any
|
|
89
|
+
other project — it ships in the bundle but is not copied into `.claude/`). It carries the
|
|
90
|
+
commands and the measured evidence for the rules summarized here: how to pin a PR head
|
|
91
|
+
into your own ref, why `FETCH_HEAD` and `headRefOid` both yield a real-looking but wrong
|
|
92
|
+
SHA, the three-step job-log procedure for proving which base a CI run checked out, why an
|
|
93
|
+
empty grep result is the *default* rather than a pass, and the withdrawn
|
|
94
|
+
`merge_commit_sha` check (#4942).
|
|
95
|
+
|
|
96
|
+
These two rules are load-bearing enough to state here as well as there:
|
|
97
|
+
|
|
98
|
+
- **Stale API view (#5086).** After any force-push, amend, or rebase: `git fetch origin
|
|
99
|
+
<branch>` and diff the ref locally. Never review from `gh pr view` / `gh pr diff` alone —
|
|
100
|
+
the API has served a pre-force-push head, zero comments, and no force-push event on a PR
|
|
101
|
+
that had all three. A comment count or timeline thinner than you expect is a staleness
|
|
102
|
+
signal, not a quiet PR. Name the head SHA you actually read in the verdict — "PASS at
|
|
103
|
+
2dd3845d1", not "PASS on #5046" — on **every** review, not only after a force-push.
|
|
104
|
+
Capture that SHA from a ref you pinned yourself (`git fetch origin --force
|
|
105
|
+
refs/pull/<N>/head:refs/pr/<N> && git rev-parse refs/pr/<N>`); never from `FETCH_HEAD` or
|
|
106
|
+
`headRefOid` — both yield a real-looking wrong SHA.
|
|
107
|
+
- **Stale merge ref (#5116).** When checks were re-run to pick up a fix on main: a
|
|
108
|
+
`pull_request` run executes `refs/pull/N/merge`, not main, and reopening a PR does not
|
|
109
|
+
reliably refresh that ref. Prove which base the runner used by reading the job log's
|
|
110
|
+
`Merge <40hex> into <40hex>` line and comparing it to a freshly fetched `origin/main` —
|
|
111
|
+
not the API summary. A stale base means the run proved nothing. "It ran after the fix
|
|
112
|
+
landed" is not evidence it ran the fix, and **no line found is not a pass**. The
|
|
113
|
+
`merge_commit_sha` polling check was disproved on #4942 — do not use it; details in the
|
|
114
|
+
shared file.
|
|
94
115
|
|
|
95
|
-
```bash
|
|
96
|
-
# Pin the ref you review into a name of your own. Never carry a verdict on FETCH_HEAD.
|
|
97
|
-
git fetch origin --force "refs/pull/<N>/head:refs/pr/<N>" && \
|
|
98
|
-
HEAD_SHA=$(git rev-parse "refs/pr/<N>") # --force: this ref is not remote-tracking,
|
|
99
|
-
# so without it a re-review after a force-push
|
|
100
|
-
# is REJECTED and rev-parse returns the OLD head
|
|
101
|
-
```
|
|
102
|
-
|
|
103
|
-
**Never `git rev-parse FETCH_HEAD` for a verdict SHA.** `FETCH_HEAD` is one mutable file
|
|
104
|
-
per worktree, rewritten by the next fetch from any source, and it fails three ways that
|
|
105
|
-
are all routine rather than exotic:
|
|
106
|
-
- The `git fetch origin main` in the run-validity block below overwrites it. Do not
|
|
107
|
-
interleave these two procedures while holding an unpinned SHA.
|
|
108
|
-
- A bare `git fetch origin` with no refspec — what most agents run at session start —
|
|
109
|
-
writes every remote branch into it. Measured here: 158 entries, all `not-for-merge`,
|
|
110
|
-
and `git rev-parse FETCH_HEAD` returned line 155, an unrelated worktree branch. You
|
|
111
|
-
sit on your own branch and review other teams' branches, so this is the normal case.
|
|
112
|
-
- A second agent fetching in the same worktree reassigns it between your fetch and your
|
|
113
|
-
read. That happened during the review of this very file, with two reviewers active.
|
|
114
|
-
|
|
115
|
-
Do not take the SHA from `gh pr view --json headRefOid` either — that is the same API
|
|
116
|
-
surface that served a stale head on #5086. Every one of these failures yields a real,
|
|
117
|
-
head-shaped SHA, so the verdict still *looks* SHA-scoped. A SHA-scoped verdict stays
|
|
118
|
-
honest after the next force-push; a PR-scoped one does not, and a wrongly-captured one
|
|
119
|
-
only looks scoped.
|
|
120
|
-
- When checks were re-run to pick up a fix on main: a `pull_request` run executes
|
|
121
|
-
`refs/pull/N/merge`, not main, and reopening a PR does not reliably refresh that ref.
|
|
122
|
-
Establish which base the runner actually used by reading the job log, not the API summary:
|
|
123
|
-
|
|
124
|
-
```bash
|
|
125
|
-
# 0. This check applies to `pull_request` runs only — they are the ones that
|
|
126
|
-
# execute refs/pull/N/merge. Confirm the trigger before anything else.
|
|
127
|
-
gh run view <run_id> --json event -q .event # expect: pull_request
|
|
128
|
-
|
|
129
|
-
# 1. A run URL gives you a run ID; jobs have their own IDs. Require COMPLETED,
|
|
130
|
-
# and exclude jobs that never ran — a skipped job's log endpoint returns 404
|
|
131
|
-
# with an XML body that greps as silence.
|
|
132
|
-
gh api --paginate repos/OWNER/REPO/actions/runs/<run_id>/jobs \
|
|
133
|
-
--jq '.jobs[] | select(.status == "completed" and .conclusion != "skipped") | {id, name}'
|
|
134
|
-
# Choose the job by NAME: the one behind the check you actually care about.
|
|
135
|
-
# A matrix run has several and they are not guaranteed to share a checkout.
|
|
136
|
-
|
|
137
|
-
# 2. Assert the base THAT job checked out. This must SUCCEED — it is not a
|
|
138
|
-
# scan for a failure line. Run it against a job that actually checked out
|
|
139
|
-
# code: a `skipped` job and a `push`-triggered run both produce NO line, and
|
|
140
|
-
# no line is NOT a pass. It means you cannot judge this run from this job —
|
|
141
|
-
# pick another job, or another run.
|
|
142
|
-
BASE=$(gh api repos/OWNER/REPO/actions/jobs/<job_id>/logs \
|
|
143
|
-
| grep -ao 'Merge [0-9a-f]\{40\} into [0-9a-f]\{40\}' \
|
|
144
|
-
| sed 's/.* into //' | sort -u)
|
|
145
|
-
[ "$(printf '%s' "$BASE" | grep -c .)" -eq 1 ] \
|
|
146
|
-
|| { echo "STUCK: found $(printf '%s' "$BASE" | grep -c .) checkout lines, need exactly 1"; exit 1; }
|
|
147
|
-
|
|
148
|
-
# 3. Compare against a FRESHLY fetched main. Skip the fetch and you compare
|
|
149
|
-
# against a stale local ref, which can agree by accident.
|
|
150
|
-
git fetch origin main
|
|
151
|
-
[ "$BASE" = "$(git rev-parse origin/main)" ] || { echo "STALE BASE — run proved nothing"; exit 1; }
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
A stale base means the run proved nothing — close/reopen and re-check the next run's log.
|
|
155
|
-
"It ran after the fix landed" is not evidence it ran the fix (#5116).
|
|
156
|
-
- **No line found is not a pass.** State the check as an assertion that must succeed, never
|
|
157
|
-
as a scan for a failure line, because empty output is the *default* result in three
|
|
158
|
-
reproducible cases: a skipped job, a `push`-triggered run, and a 404 log body (stderr
|
|
159
|
-
only, empty stdout, exit 1). In all three a mismatch test never fires and the reviewer
|
|
160
|
-
sees nothing that fails. A `push` run checks out the branch tip directly and logs
|
|
161
|
-
`Switched to a new branch` — the merge-ref rule does not apply to it, and neither does
|
|
162
|
-
its reassurance. Match the anchored `Merge <40hex> into <40hex>` form rather than the
|
|
163
|
-
prose prefix: this document itself contains that prose, and an unanchored substring grep
|
|
164
|
-
can eventually match our own guidance instead of a runner's log.
|
|
165
|
-
- **Require the job to have COMPLETED, not merely to be un-skipped.** A running job has
|
|
166
|
-
`conclusion: null` and passes a not-skipped test, so the old filter returned it (measured
|
|
167
|
-
on job `92503438573`). It does not produce a wrong pass: `/actions/jobs/<id>/logs` serves
|
|
168
|
-
a COMPLETED log archive, never a partial one. Measured on the same job before and after
|
|
169
|
-
it finished — HTTP 404 `BlobNotFound` while running, 180 KB containing the base line once
|
|
170
|
-
complete — and confirmed not to be a checkout-timing artifact: a job 213 seconds in, with
|
|
171
|
-
`Checkout code` and seven later steps already `completed`, still returned the 404. So a
|
|
172
|
-
running job greps as silence and lands in the STUCK branch above.
|
|
173
|
-
Require `completed` anyway, because STUCK does not tell you WHICH problem you have. A
|
|
174
|
-
skipped job, a running job, and a purged log all return HTTP 404 with a near-identical
|
|
175
|
-
`BlobNotFound` body — and they call for three different actions (pick another job; wait;
|
|
176
|
-
treat the run as unauditable). Filtering on `completed` removes one of the three up front.
|
|
177
|
-
A job whose conclusion is unknown cannot tell you whether the run validated the fix,
|
|
178
|
-
whatever its log says. If the filter returns nothing, every job is still running — wait.
|
|
179
|
-
- **Retraction — do not use `merge_commit_sha`.** An earlier version of this rule said to
|
|
180
|
-
poll `gh api repos/OWNER/REPO/pulls/N --jq .merge_commit_sha` until it changed. That
|
|
181
|
-
check is wrong and is withdrawn. Measured on #4942: the API reported a *fresh*
|
|
182
|
-
`merge_commit_sha` while the runner still checked out a stale base — run `31063081630`,
|
|
183
|
-
base `3c226df8c`, 36 commits behind the good run's base `5f2bc52d3`, still carrying the
|
|
184
|
-
broken step body — and the run went green, 2.5 minutes and identical check names apart
|
|
185
|
-
from the real one. If you learned the polling rule, unlearn it.
|
|
186
|
-
- Nothing else about the run is load-bearing. On #4942 the check name, the conclusion, the
|
|
187
|
-
check rollup, the timestamp, and `merge_commit_sha` were all **identical** between the
|
|
188
|
-
worthless run and the real one. The job log's `into <base>` was the only field that
|
|
189
|
-
distinguished them.
|
|
190
116
|
- The Codex differential review is a locally-run script, not a CI job, so "a comment
|
|
191
117
|
exists" is not evidence it ran clean. Check the **exit status**, and record the SHA
|
|
192
118
|
yourself:
|
|
@@ -6,8 +6,9 @@
|
|
|
6
6
|
# Fires on: PreToolUse (empty matcher — all tool calls)
|
|
7
7
|
# Input: JSON on stdin with tool_name and tool_input
|
|
8
8
|
# Output: reminder message on stderr (fd 3) when counter >= 50 — a required-save
|
|
9
|
-
# demand for a session observed to have a memory-write tool, otherwise a
|
|
10
|
-
# once-per-session advisory that demands nothing (#4863, see CAPABILITY below
|
|
9
|
+
# demand for a MAIN-THREAD session observed to have a memory-write tool, otherwise a
|
|
10
|
+
# once-per-session advisory that demands nothing (#4863, see CAPABILITY below;
|
|
11
|
+
# #5136, see DISPATCHED SUBAGENTS below — a subagent is never given the demand)
|
|
11
12
|
# Exit: 2 when either message is emitted, 0 otherwise.
|
|
12
13
|
# The previous header here read "always 0 (never blocks tool calls)". That has
|
|
13
14
|
# not matched the code since the reminder moved to exit 2 (see the exit at the
|
|
@@ -17,9 +18,80 @@
|
|
|
17
18
|
# the same defect class as the misattributed reminder #4734 addresses — in a
|
|
18
19
|
# file that runs on EVERY tool call for every agent.
|
|
19
20
|
#
|
|
20
|
-
#
|
|
21
|
-
#
|
|
22
|
-
#
|
|
21
|
+
# DISPATCHED SUBAGENTS (#5136). A dispatched subagent is never given the demand. It
|
|
22
|
+
# gets the advisory branch instead, on the same threshold, worded for its situation.
|
|
23
|
+
#
|
|
24
|
+
# The defect this closes: capability was being read as AUTHORITY. Having used Bash
|
|
25
|
+
# marks a session CAPABLE, and CAPABLE selected the demanding branch — which also
|
|
26
|
+
# DROPPED the advisory's "checkpointing is the dispatching session's responsibility"
|
|
27
|
+
# line, at exactly the sessions that most needed it. A subagent that runs one Bash
|
|
28
|
+
# call is capable of writing a memory and is still the wrong party to write one: its
|
|
29
|
+
# brief carries no conversationId, and it has no synchronous channel to ask its
|
|
30
|
+
# dispatcher for one. On 2026-08-07 this hook demanded a checkpoint from four
|
|
31
|
+
# dispatched subagents in one day. One complied with --conversation-id NEW, which
|
|
32
|
+
# opened a fresh conversation per checkpoint and split one iteration's record across
|
|
33
|
+
# several; three declined and flagged the omission. Both outcomes are costs the hook
|
|
34
|
+
# created. Being able to perform an action is not the same as being the party
|
|
35
|
+
# responsible for it, and a demand the receiving session cannot correctly satisfy is
|
|
36
|
+
# the #4863 failure mode in a new place.
|
|
37
|
+
#
|
|
38
|
+
# THE SIGNAL IS .agent_id, and it is authoritative, not inferred. Two sources, both
|
|
39
|
+
# checkable, quoted verbatim rather than summarised — this file's whole discipline is
|
|
40
|
+
# that a claim in a comment can be verified against the thing it describes.
|
|
41
|
+
#
|
|
42
|
+
# 1. The published hook reference, https://code.claude.com/docs/en/hooks, hook input
|
|
43
|
+
# fields table (checked 2026-08-07):
|
|
44
|
+
# agent_id — "Unique identifier for the subagent. Present only when the hook
|
|
45
|
+
# fires inside a subagent call. Use this to distinguish subagent
|
|
46
|
+
# hook calls from main-thread calls."
|
|
47
|
+
# agent_type — "Agent name (for example, "Explore" or "security-reviewer").
|
|
48
|
+
# Present when the session uses --agent or the hook fires inside a
|
|
49
|
+
# subagent. For subagents, the subagent's type takes precedence
|
|
50
|
+
# over the session's --agent value. See SubagentStart for the
|
|
51
|
+
# values custom and plugin subagents report and how to write a
|
|
52
|
+
# matcher against a plugin-scoped name."
|
|
53
|
+
# Both rows are the complete cell, to the end. The agent_type cell's last sentence
|
|
54
|
+
# is a cross-reference this hook does not use, and an earlier revision dropped it
|
|
55
|
+
# as irrelevant without marking the elision — in a block claiming verbatim
|
|
56
|
+
# quotation, which makes the judgement invisible to the reader checking it. Quoted
|
|
57
|
+
# whole instead of elided, so there is no judgement left to check.
|
|
58
|
+
# 2. The runtime's own payload schema, which is stricter about the main-thread case.
|
|
59
|
+
# In the installed bundle (checked against version 2.1.224, file
|
|
60
|
+
# ~/.local/share/claude/versions/2.1.224, the zod .describe() attached to
|
|
61
|
+
# agent_id) it reads: "Subagent identifier. Present only when the hook fires from
|
|
62
|
+
# within a subagent (e.g., a tool called by an AgentTool worker). Absent for the
|
|
63
|
+
# main thread, even in --agent sessions. Use this field (not agent_type) to
|
|
64
|
+
# distinguish subagent calls from main-thread calls."
|
|
65
|
+
#
|
|
66
|
+
# Source 2 is a version-pinned string inside a shipped binary, so it can change without
|
|
67
|
+
# notice and is cited as corroboration, not as the contract; source 1 is the contract.
|
|
68
|
+
# They agree on the load-bearing point, which is why .agent_id is read and .agent_type
|
|
69
|
+
# is deliberately NOT: agent_type is ALSO set on the main thread of an --agent session,
|
|
70
|
+
# so branching on it would silence the demand for a legitimate lead. A test pins that.
|
|
71
|
+
#
|
|
72
|
+
# (An earlier revision of this block presented source 2's wording as though it were the
|
|
73
|
+
# published documentation. The semantics were right and the string was real, but a
|
|
74
|
+
# reader checking it against the docs would not find those sentences — which is exactly
|
|
75
|
+
# the misattribution failure #4734 is about, committed in a comment instead of a
|
|
76
|
+
# message. Recorded rather than quietly deleted.)
|
|
77
|
+
#
|
|
78
|
+
# COUNTING SCOPE (#4802, extended by #5136). The counter is keyed by SESSION
|
|
79
|
+
# (harness-supplied .session_id) AND, inside a subagent, by .agent_id.
|
|
80
|
+
#
|
|
81
|
+
# Session alone was not enough, and this is the STALE-COUNTER root cause (#5136): a
|
|
82
|
+
# dispatched subagent runs under its DISPATCHER'S session_id. Verified in the fleet's
|
|
83
|
+
# own transcripts — a subagent's records live at
|
|
84
|
+
# <projectDir>/<sessionId>/subagents/agent-<agentId>.jsonl and every line in the child
|
|
85
|
+
# file carries the parent's sessionId. So one leader plus its concurrent subagents all
|
|
86
|
+
# shared ONE per-session counter: the lead's. A subagent's FIRST tool call read a
|
|
87
|
+
# counter already at 49 and was told "50 tool calls since your last memory save" about
|
|
88
|
+
# work it had not done. That is the same false premise #5013 removed from the
|
|
89
|
+
# worktree-wide fallback key, still alive one level down, because #4802 keyed on the
|
|
90
|
+
# session and a session is not one agent. Adding .agent_id to the key makes the count
|
|
91
|
+
# a fact about the agent being handed it, which is the whole property #4802 was after.
|
|
92
|
+
#
|
|
93
|
+
# The original #4802 rationale, unchanged and still the reason the session component
|
|
94
|
+
# exists: the key used to be the worktree alone, which made this guard resolve its subject
|
|
23
95
|
# from ambient context rather than from the agent it was actually guarding: a
|
|
24
96
|
# session that had made two tool calls itself was told "50 tool calls since last
|
|
25
97
|
# memory save" and required to write a checkpoint because OTHER sessions in the
|
|
@@ -81,6 +153,11 @@
|
|
|
81
153
|
# Either observation marks the session capable (a marker file beside its counter) and
|
|
82
154
|
# the demand is unchanged for it.
|
|
83
155
|
#
|
|
156
|
+
# Capability selects between the two messages. It does NOT establish that the session is
|
|
157
|
+
# the right party to save (#5136): a dispatched subagent takes the advisory branch even
|
|
158
|
+
# when it has been observed using Bash, because the question the demand asks — "will you
|
|
159
|
+
# write this checkpoint?" — has a different answer from "could you?".
|
|
160
|
+
#
|
|
84
161
|
# A session with NO such observation after 50 tool calls gets an advisory instead: same
|
|
85
162
|
# information, no demand, and an explicit "no action required if you cannot save" —
|
|
86
163
|
# emitted ONCE per session, because repeating a no-action notice to a session that
|
|
@@ -224,6 +301,45 @@ fi
|
|
|
224
301
|
# so no harness-supplied text reaches agent context through it.
|
|
225
302
|
SESSION_ID=$(echo "$INPUT" | jq -r '.session_id // empty' 2>/dev/null) || SESSION_ID=""
|
|
226
303
|
SESSION_ID="${SESSION_ID//[^a-zA-Z0-9_-]/}"
|
|
304
|
+
# BOUND THE LENGTH, not just the character set (#5136 review). Sanitizing for path
|
|
305
|
+
# traversal is not enough: these ids become filename components, and NAME_MAX is 255
|
|
306
|
+
# bytes on every filesystem this runs on. An over-long id pushes the counter name past
|
|
307
|
+
# that limit, every write to it fails ENAMETOOLONG into a `|| true`, the counter can
|
|
308
|
+
# never be read back or incremented, and the hook goes PERMANENTLY SILENT for that
|
|
309
|
+
# session — no counter, no threshold, no reminder, and no error either. Reproduced both
|
|
310
|
+
# ways before fixing: a 400-character session_id and a 400-character agent_id each
|
|
311
|
+
# produced zero counter files across repeated calls.
|
|
312
|
+
#
|
|
313
|
+
# That is the same silent-disable this file rejects everywhere else, reached through
|
|
314
|
+
# the filesystem instead of through a code path, and for session_id it PRE-DATES the
|
|
315
|
+
# subagent work. 64 is far above any real id — session ids are 36-character UUIDs and
|
|
316
|
+
# agent ids are ~17 hex characters — so this truncates nothing the harness actually
|
|
317
|
+
# sends, and truncation losing an unlikely collision to a shared counter is strictly
|
|
318
|
+
# better than the gate silently switching itself off.
|
|
319
|
+
SESSION_ID="${SESSION_ID:0:64}"
|
|
320
|
+
|
|
321
|
+
# Subagent attribution (#5136). Same parse, same sanitization, same load-bearing
|
|
322
|
+
# `|| AGENT_ID=""` as the two captures above — a broken jq must degrade, not return
|
|
323
|
+
# non-zero into the ERR trap. The value is used only as a filename component and as a
|
|
324
|
+
# 0/1 branch selector; it is never printed into the message, so no harness-supplied
|
|
325
|
+
# text reaches agent context through it.
|
|
326
|
+
#
|
|
327
|
+
# .agent_id, not .agent_type: the harness sets agent_id only inside a dispatched
|
|
328
|
+
# subagent, while agent_type is ALSO set on the main thread of an --agent session.
|
|
329
|
+
# Branching on agent_type would drop the demand for a lead that happens to have been
|
|
330
|
+
# started with --agent — a silent, invisible weakening of the gate for a real leader.
|
|
331
|
+
# See DISPATCHED SUBAGENTS in the header for the two sources this rests on, quoted.
|
|
332
|
+
AGENT_ID=$(echo "$INPUT" | jq -r '.agent_id // empty' 2>/dev/null) || AGENT_ID=""
|
|
333
|
+
AGENT_ID="${AGENT_ID//[^a-zA-Z0-9_-]/}"
|
|
334
|
+
# Same NAME_MAX bound as SESSION_ID above, same reason — and here it compounds, since
|
|
335
|
+
# a subagent's counter name carries BOTH components.
|
|
336
|
+
AGENT_ID="${AGENT_ID:0:64}"
|
|
337
|
+
# `if`, not `[[ ... ]] && VAR=1` — FL-13, never leave a failing test in reach of the
|
|
338
|
+
# ERR trap.
|
|
339
|
+
IS_SUBAGENT=0
|
|
340
|
+
if [[ -n "$AGENT_ID" ]]; then
|
|
341
|
+
IS_SUBAGENT=1
|
|
342
|
+
fi
|
|
227
343
|
|
|
228
344
|
# NO SESSION ID → NO REMINDER (#5013). Everything below this line — the counter, the
|
|
229
345
|
# capability marker, the advisory, the demand — is a statement about ONE session. With
|
|
@@ -251,6 +367,21 @@ fi
|
|
|
251
367
|
WORKTREE_NAME="${CLAUDE_WORKTREE_NAME:-default}"
|
|
252
368
|
# Sanitize: alphanumeric, hyphens, underscores only — prevent path traversal
|
|
253
369
|
WORKTREE_NAME="${WORKTREE_NAME//[^a-zA-Z0-9_-]/}"
|
|
370
|
+
# Bound the length too, for the same NAME_MAX reason as SESSION_ID and AGENT_ID above
|
|
371
|
+
# (#5136 review). This is the THIRD filename component and the last one left uncapped:
|
|
372
|
+
# an over-long worktree name overruns NAME_MAX on its own, every write to the counter
|
|
373
|
+
# fails ENAMETOOLONG into a `|| true`, and the gate goes permanently silent with no
|
|
374
|
+
# counter and no error. Reproduced before fixing — a 400-character CLAUDE_WORKTREE_NAME
|
|
375
|
+
# produced zero counter files across repeated calls, exactly as the two id fields did.
|
|
376
|
+
#
|
|
377
|
+
# Lower exposure than the other two, since this is a locally-set environment variable
|
|
378
|
+
# rather than a harness-supplied payload field, and that is precisely why it is a cap
|
|
379
|
+
# and not a rejection: it is not an attack to defend against, it is a sibling of a
|
|
380
|
+
# defect just fixed one screen up, and leaving it would mean shipping a known instance
|
|
381
|
+
# of the failure this file rejects everywhere else. Truncation cannot empty a non-empty
|
|
382
|
+
# name, so the `:-default` fallback below still only catches the all-invalid-chars case
|
|
383
|
+
# and must stay AFTER this line.
|
|
384
|
+
WORKTREE_NAME="${WORKTREE_NAME:0:64}"
|
|
254
385
|
WORKTREE_NAME="${WORKTREE_NAME:-default}"
|
|
255
386
|
COUNTER_FILE="/tmp/.mx-checkpoint-counter-${WORKTREE_NAME}"
|
|
256
387
|
# Per-session counter file. Naming follows #4758's `${STATE_FILE}.notice-${SESSION_ID}`
|
|
@@ -263,6 +394,15 @@ COUNTER_FILE="/tmp/.mx-checkpoint-counter-${WORKTREE_NAME}"
|
|
|
263
394
|
if [[ -n "$SESSION_ID" ]]; then
|
|
264
395
|
COUNTER_FILE="${COUNTER_FILE}.session-${SESSION_ID}"
|
|
265
396
|
fi
|
|
397
|
+
# Per-subagent counter (#5136). A dispatched subagent runs under its dispatcher's
|
|
398
|
+
# session_id, so the session suffix alone puts the lead and every concurrent subagent
|
|
399
|
+
# on ONE counter — see COUNTING SCOPE in the header. Appending the agent id splits
|
|
400
|
+
# them. Same derive-from-the-base convention as the session suffix, and appended
|
|
401
|
+
# AFTER it so the existing sweep pattern (`*.session-*`) still matches the whole
|
|
402
|
+
# family; the .cap and .advised companions below inherit the split for free.
|
|
403
|
+
if [[ "$IS_SUBAGENT" -eq 1 ]]; then
|
|
404
|
+
COUNTER_FILE="${COUNTER_FILE}.agent-${AGENT_ID}"
|
|
405
|
+
fi
|
|
266
406
|
THRESHOLD=50
|
|
267
407
|
# Companion state, named off the counter base for the same reason the per-session
|
|
268
408
|
# counter is (#4758's convention): one family of files, one sweep pattern.
|
|
@@ -364,6 +504,21 @@ if [[ "$CURRENT" -ge "$THRESHOLD" ]]; then
|
|
|
364
504
|
CAPABLE=1
|
|
365
505
|
fi
|
|
366
506
|
|
|
507
|
+
# Branch selection (#5136). Two independent reasons to advise rather than demand,
|
|
508
|
+
# and either one is sufficient:
|
|
509
|
+
# - nothing observed → the session may have no tool to satisfy a demand (#4863)
|
|
510
|
+
# - dispatched subagent → the session has a tool but is not the responsible party
|
|
511
|
+
# Under an opaque payload neither can be read (AGENT_ID is empty for the same reason
|
|
512
|
+
# TOOL_NAME is), so IS_SUBAGENT is 0 and the demand fires — the over-report direction
|
|
513
|
+
# every other degradation in this file picks.
|
|
514
|
+
ADVISORY=0
|
|
515
|
+
if [[ "$CAPABLE" -eq 0 ]]; then
|
|
516
|
+
ADVISORY=1
|
|
517
|
+
fi
|
|
518
|
+
if [[ "$IS_SUBAGENT" -eq 1 ]]; then
|
|
519
|
+
ADVISORY=1
|
|
520
|
+
fi
|
|
521
|
+
|
|
367
522
|
# A session with no observed memory-write tool is advised at most once. Past
|
|
368
523
|
# that, staying quiet is the correct behaviour: it has already been told, it
|
|
369
524
|
# cannot act, and every further interruption is cost with no possible outcome.
|
|
@@ -379,7 +534,10 @@ if [[ "$CURRENT" -ge "$THRESHOLD" ]]; then
|
|
|
379
534
|
# one builtin test on a path that runs once per 50 calls, and it keeps this branch
|
|
380
535
|
# correct on its own terms if the top-of-file guard is ever moved or relaxed. Do
|
|
381
536
|
# not read it as evidence that a sessionless call can reach this branch.
|
|
382
|
-
|
|
537
|
+
# ADVISORY, not CAPABLE, gates the quiet period as of #5136 — a subagent that is
|
|
538
|
+
# advised once must not be re-advised every 50 calls either. The suppression state
|
|
539
|
+
# is keyed to the counter, so for a subagent it is per-agent, not per-session.
|
|
540
|
+
if [[ "$ADVISORY" -eq 1 && -n "$SESSION_ID" && -f "$ADVISED_FILE" ]]; then
|
|
383
541
|
exit 0
|
|
384
542
|
fi
|
|
385
543
|
|
|
@@ -527,30 +685,53 @@ if [[ "$CURRENT" -ge "$THRESHOLD" ]]; then
|
|
|
527
685
|
# Route to the preserved real stderr (fd 3) — exit-2 PreToolUse messages
|
|
528
686
|
# surface via stderr, not stdout (KI-HOOKS-STDERR-SYSTEMIC). fd 2 may have been
|
|
529
687
|
# redirected to /dev/null by the flock exec above, so fd 3 is used here.
|
|
530
|
-
if [[ "$
|
|
531
|
-
# ADVISORY —
|
|
532
|
-
#
|
|
533
|
-
#
|
|
688
|
+
if [[ "$ADVISORY" -eq 1 ]]; then
|
|
689
|
+
# ADVISORY — nothing here is required of the receiving session. Two shapes,
|
|
690
|
+
# because the two reasons for advising are different facts and a message that
|
|
691
|
+
# states the wrong one is a false premise of the kind this file exists to avoid:
|
|
692
|
+
# - subagent (#5136): the session probably CAN save and should not be the one to
|
|
693
|
+
# - read-only (#4863): the hook has no evidence the session can save at all
|
|
534
694
|
: > "$ADVISED_FILE" 2>/dev/null || true
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
echo "
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
695
|
+
if [[ "$IS_SUBAGENT" -eq 1 ]]; then
|
|
696
|
+
{
|
|
697
|
+
echo ""
|
|
698
|
+
echo "ℹ️ Memory checkpoint advisory: ${CURRENT} tool calls in this subagent since the last memory save."
|
|
699
|
+
echo " NO ACTION IS REQUIRED, AND NO CHECKPOINT IS BEING ASKED OF YOU. The hook payload"
|
|
700
|
+
echo " carries an agent_id, which the harness sets only inside a dispatched subagent, so"
|
|
701
|
+
echo " this session is one. Checkpointing this work belongs to the session that dispatched"
|
|
702
|
+
echo " you, which holds the conversation the work is recorded in."
|
|
703
|
+
echo " - Do NOT save with --conversation-id NEW to satisfy this notice. A new conversation"
|
|
704
|
+
echo " per checkpoint splits one iteration's record across several, and you have no"
|
|
705
|
+
echo " synchronous channel to ask your dispatcher for the right id mid-task."
|
|
706
|
+
echo " - Put your progress in your final report instead. That is what your dispatcher"
|
|
707
|
+
echo " reads, and it is where this information is supposed to go."
|
|
708
|
+
echo " - If your brief DID give you a conversation id and told you to checkpoint, follow"
|
|
709
|
+
echo " the brief — it outranks this notice, which knows nothing about your task."
|
|
710
|
+
echo " Retry your original tool call to proceed. This advisory is emitted once per subagent."
|
|
711
|
+
print_provenance
|
|
712
|
+
echo ""
|
|
713
|
+
} >&3
|
|
714
|
+
else
|
|
715
|
+
{
|
|
716
|
+
echo ""
|
|
717
|
+
echo "ℹ️ Memory checkpoint advisory: ${CURRENT} tool calls in this session since the last memory save."
|
|
718
|
+
echo " NO ACTION IS REQUIRED IF YOU HAVE NO MEMORY-WRITE TOOL. This hook cannot see"
|
|
719
|
+
echo " your session's toolset, and it has not observed this session use Bash or a"
|
|
720
|
+
echo " MemNexus MCP tool, so it does not assume you can save a checkpoint."
|
|
721
|
+
echo " - If you CAN save one, this is the command:"
|
|
722
|
+
echo "$SAVE_COMMAND"
|
|
723
|
+
echo " (or the create_memory MCP tool, with the same codeContext)."
|
|
724
|
+
echo "$CONV_ID_NOTE"
|
|
725
|
+
if [[ -n "$CODE_CONTEXT_NOTE" ]]; then
|
|
726
|
+
echo "$CODE_CONTEXT_NOTE"
|
|
727
|
+
fi
|
|
728
|
+
echo " - If you CANNOT, ignore this and continue. Report your findings to the session"
|
|
729
|
+
echo " that dispatched you; checkpointing is that session's responsibility, not yours."
|
|
730
|
+
echo " Retry your original tool call to proceed. This advisory is emitted once per session."
|
|
731
|
+
print_provenance
|
|
732
|
+
echo ""
|
|
733
|
+
} >&3
|
|
734
|
+
fi
|
|
554
735
|
exit 2 # Surface this message to the agent — exit 2 is required for PreToolUse visibility
|
|
555
736
|
fi
|
|
556
737
|
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
# Shared: Stale-Head and Run-Validity Defenses
|
|
2
|
+
|
|
3
|
+
**What this is.** The procedural detail behind three measured failure modes in which a
|
|
4
|
+
reviewer or tester formed a complete, confident, wrong picture of a PR with no error
|
|
5
|
+
anywhere: a stale API view of the head (#5086), a CI run that checked out a stale base
|
|
6
|
+
(#5116), and an API field that reported freshness the runner did not have (#4942). All
|
|
7
|
+
three issues are OPEN.
|
|
8
|
+
|
|
9
|
+
**Which agents include it.** `security-reviewer` and `qa-tester`. Each of those definitions
|
|
10
|
+
carries the two most load-bearing rules INLINE — the #5086 stale-API-view rule and the
|
|
11
|
+
#5116 stale-merge-ref rule — and points here for the commands and the reasoning. Reading
|
|
12
|
+
this file is part of those definitions, not optional background.
|
|
13
|
+
|
|
14
|
+
**Where it resolves.** `mx-agent-system/agent-config/shared/` in this repo;
|
|
15
|
+
`dist/agent-config/shared/` inside an installed `@memnexus-ai/mx-agent-cli`. It is NOT
|
|
16
|
+
copied into `.claude/`: `generate-claude.mjs` materializes only `MANAGED_SUBDIRS`
|
|
17
|
+
(`hooks`, `agents`, `skills`, `commands`, `rules`), and `shared` is not one of them, so
|
|
18
|
+
both pointers name both locations. Adding `shared` to `MANAGED_SUBDIRS` would give a
|
|
19
|
+
single `.claude/`-relative path, but it changes the generator and the deployed-tree shape
|
|
20
|
+
for every team at once — deliberately not done here.
|
|
21
|
+
|
|
22
|
+
**Editing rule: edits happen HERE ONLY.** This content was duplicated verbatim across both
|
|
23
|
+
agent definitions and had already nearly drifted once. Do not fix a rule in one agent
|
|
24
|
+
definition and leave the other behind, and do not re-inline this material. If a rule
|
|
25
|
+
changes, change it in this file. If the change alters one of the inline summaries, update
|
|
26
|
+
both definitions in the same commit.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Verify you are examining current bytes
|
|
31
|
+
|
|
32
|
+
Both failure modes below produced a complete, confident, wrong picture with no error
|
|
33
|
+
anywhere — checking the identity of what you read is the only defense.
|
|
34
|
+
|
|
35
|
+
- After any force-push, amend, or rebase: `git fetch origin <branch>` and diff the ref
|
|
36
|
+
locally. Never work from `gh pr view` / `gh pr diff` alone — the API has served a
|
|
37
|
+
pre-force-push head, zero comments, and no force-push event on a PR that had all three
|
|
38
|
+
(#5086).
|
|
39
|
+
- A comment count or timeline thinner than you expect is a staleness signal, not a quiet PR.
|
|
40
|
+
- Name the head SHA you actually examined in the verdict: "PASS at 2dd3845d1", not "PASS on
|
|
41
|
+
#5046". Fetch the PR head and capture the SHA in the same breath — on **every** run, not
|
|
42
|
+
only after a force-push, because every verdict needs it:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
# Pin the ref you examine into a name of your own. Never carry a verdict on FETCH_HEAD.
|
|
46
|
+
git fetch origin --force "refs/pull/<N>/head:refs/pr/<N>" && \
|
|
47
|
+
HEAD_SHA=$(git rev-parse "refs/pr/<N>") # --force: this ref is not remote-tracking,
|
|
48
|
+
# so without it a re-review after a force-push
|
|
49
|
+
# is REJECTED and rev-parse returns the OLD head
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
**Never `git rev-parse FETCH_HEAD` for a verdict SHA.** `FETCH_HEAD` is one mutable file
|
|
53
|
+
per worktree, rewritten by the next fetch from any source, and it fails three ways that
|
|
54
|
+
are all routine rather than exotic:
|
|
55
|
+
- The `git fetch origin main` in the run-validity block below overwrites it. Do not
|
|
56
|
+
interleave these two procedures while holding an unpinned SHA.
|
|
57
|
+
- A bare `git fetch origin` with no refspec — what most agents run at session start —
|
|
58
|
+
writes every remote branch into it. Measured here: 158 entries, all `not-for-merge`,
|
|
59
|
+
and `git rev-parse FETCH_HEAD` returned line 155, an unrelated worktree branch. You
|
|
60
|
+
sit on your own branch and examine other teams' branches, so this is the normal case.
|
|
61
|
+
- A second agent fetching in the same worktree reassigns it between your fetch and your
|
|
62
|
+
read. That happened during the review of this very file, with two reviewers active.
|
|
63
|
+
|
|
64
|
+
Do not take the SHA from `gh pr view --json headRefOid` either — that is the same API
|
|
65
|
+
surface that served a stale head on #5086. Every one of these failures yields a real,
|
|
66
|
+
head-shaped SHA, so the verdict still *looks* SHA-scoped. A SHA-scoped verdict stays
|
|
67
|
+
honest after the next force-push; a PR-scoped one does not, and a wrongly-captured one
|
|
68
|
+
only looks scoped.
|
|
69
|
+
|
|
70
|
+
## Establish which base the runner actually used
|
|
71
|
+
|
|
72
|
+
- When checks were re-run to pick up a fix on main: a `pull_request` run executes
|
|
73
|
+
`refs/pull/N/merge`, not main, and reopening a PR does not reliably refresh that ref.
|
|
74
|
+
Establish which base the runner actually used by reading the job log, not the API summary:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
# 0. This check applies to `pull_request` runs only — they are the ones that
|
|
78
|
+
# execute refs/pull/N/merge. Confirm the trigger before anything else.
|
|
79
|
+
gh run view <run_id> --json event -q .event # expect: pull_request
|
|
80
|
+
|
|
81
|
+
# 1. A run URL gives you a run ID; jobs have their own IDs. Require COMPLETED,
|
|
82
|
+
# and exclude jobs that never ran — a skipped job's log endpoint returns 404
|
|
83
|
+
# with an XML body that greps as silence.
|
|
84
|
+
gh api --paginate repos/OWNER/REPO/actions/runs/<run_id>/jobs \
|
|
85
|
+
--jq '.jobs[] | select(.status == "completed" and .conclusion != "skipped") | {id, name}'
|
|
86
|
+
# Choose the job by NAME: the one behind the check you actually care about.
|
|
87
|
+
# A matrix run has several and they are not guaranteed to share a checkout.
|
|
88
|
+
|
|
89
|
+
# 2. Assert the base THAT job checked out. This must SUCCEED — it is not a
|
|
90
|
+
# scan for a failure line. Run it against a job that actually checked out
|
|
91
|
+
# code: a `skipped` job and a `push`-triggered run both produce NO line, and
|
|
92
|
+
# no line is NOT a pass. It means you cannot judge this run from this job —
|
|
93
|
+
# pick another job, or another run.
|
|
94
|
+
BASE=$(gh api repos/OWNER/REPO/actions/jobs/<job_id>/logs \
|
|
95
|
+
| grep -ao 'Merge [0-9a-f]\{40\} into [0-9a-f]\{40\}' \
|
|
96
|
+
| sed 's/.* into //' | sort -u)
|
|
97
|
+
[ "$(printf '%s' "$BASE" | grep -c .)" -eq 1 ] \
|
|
98
|
+
|| { echo "STUCK: found $(printf '%s' "$BASE" | grep -c .) checkout lines, need exactly 1"; exit 1; }
|
|
99
|
+
|
|
100
|
+
# 3. Compare against a FRESHLY fetched main. Skip the fetch and you compare
|
|
101
|
+
# against a stale local ref, which can agree by accident.
|
|
102
|
+
git fetch origin main
|
|
103
|
+
[ "$BASE" = "$(git rev-parse origin/main)" ] || { echo "STALE BASE — run proved nothing"; exit 1; }
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
A stale base means the run proved nothing — close/reopen and re-check the next run's log.
|
|
107
|
+
"It ran after the fix landed" is not evidence it ran the fix (#5116).
|
|
108
|
+
- **No line found is not a pass.** State the check as an assertion that must succeed, never
|
|
109
|
+
as a scan for a failure line, because empty output is the *default* result in three
|
|
110
|
+
reproducible cases: a skipped job, a `push`-triggered run, and a 404 log body (stderr
|
|
111
|
+
only, empty stdout, exit 1). In all three a mismatch test never fires and the reviewer
|
|
112
|
+
sees nothing that fails. A `push` run checks out the branch tip directly and logs
|
|
113
|
+
`Switched to a new branch` — the merge-ref rule does not apply to it, and neither does
|
|
114
|
+
its reassurance. Match the anchored `Merge <40hex> into <40hex>` form rather than the
|
|
115
|
+
prose prefix: this document itself contains that prose, and an unanchored substring grep
|
|
116
|
+
can eventually match our own guidance instead of a runner's log.
|
|
117
|
+
- **Require the job to have COMPLETED, not merely to be un-skipped.** A running job has
|
|
118
|
+
`conclusion: null` and passes a not-skipped test, so the old filter returned it (measured
|
|
119
|
+
on job `92503438573`). It does not produce a wrong pass: `/actions/jobs/<id>/logs` serves
|
|
120
|
+
a COMPLETED log archive, never a partial one. Measured on the same job before and after
|
|
121
|
+
it finished — HTTP 404 `BlobNotFound` while running, 180 KB containing the base line once
|
|
122
|
+
complete — and confirmed not to be a checkout-timing artifact: a job 213 seconds in, with
|
|
123
|
+
`Checkout code` and seven later steps already `completed`, still returned the 404. So a
|
|
124
|
+
running job greps as silence and lands in the STUCK branch above.
|
|
125
|
+
Require `completed` anyway, because STUCK does not tell you WHICH problem you have. A
|
|
126
|
+
skipped job, a running job, and a purged log all return HTTP 404 with a near-identical
|
|
127
|
+
`BlobNotFound` body — and they call for three different actions (pick another job; wait;
|
|
128
|
+
treat the run as unauditable). Filtering on `completed` removes one of the three up front.
|
|
129
|
+
A job whose conclusion is unknown cannot tell you whether the run validated the fix,
|
|
130
|
+
whatever its log says. If the filter returns nothing, every job is still running — wait.
|
|
131
|
+
- **Retraction — do not use `merge_commit_sha`.** An earlier version of this rule said to
|
|
132
|
+
poll `gh api repos/OWNER/REPO/pulls/N --jq .merge_commit_sha` until it changed. That
|
|
133
|
+
check is wrong and is withdrawn. Measured on #4942: the API reported a *fresh*
|
|
134
|
+
`merge_commit_sha` while the runner still checked out a stale base — run `31063081630`,
|
|
135
|
+
base `3c226df8c`, 36 commits behind the good run's base `5f2bc52d3`, still carrying the
|
|
136
|
+
broken step body — and the run went green, 2.5 minutes and identical check names apart
|
|
137
|
+
from the real one. If you learned the polling rule, unlearn it.
|
|
138
|
+
- Nothing else about the run is load-bearing. On #4942 the check name, the conclusion, the
|
|
139
|
+
check rollup, the timestamp, and `merge_commit_sha` were all **identical** between the
|
|
140
|
+
worthless run and the real one. The job log's `into <base>` was the only field that
|
|
141
|
+
distinguished them.
|