mandrel 2.29.0 → 2.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/rules/ci-remediation.md +44 -1
- package/.agents/scripts/lib/crap-utils.js +78 -5
- package/.agents/scripts/lib/orchestration/single-story-close/phases/push.js +17 -2
- package/.agents/scripts/lib/orchestration/single-story-close/runner.js +7 -1
- package/.agents/skills/skills.index.json +2 -2
- package/.agents/skills/stack/qa/playwright/SKILL.md +48 -0
- package/docs/CHANGELOG.md +14 -0
- package/package.json +1 -1
|
@@ -51,6 +51,7 @@ the two options above. Name the verdict you reached in the `friction` comment.
|
|
|
51
51
|
| **defect-in-diff** | The failure reproduces on the branch and not on an unmodified `main` | Option 1 — fix at source |
|
|
52
52
|
| **pre-existing** | The same check fails on an unmodified `main` too | Option 2 — file `meta::framework-gap`; remediate here only if it blocks this delivery |
|
|
53
53
|
| **capacity** | Proven exhaustion of a runner resource, not a property of the diff (see below) | Option 2 — file `meta::framework-gap` **and** escalate to the operator |
|
|
54
|
+
| **unreproducible-tier** | The tier cannot be exercised in this sandbox at all, proven by an attempted attach (see below) | Option 2 — file `meta::framework-gap` **and** escalate on first encounter |
|
|
54
55
|
|
|
55
56
|
### The `capacity` verdict
|
|
56
57
|
|
|
@@ -82,10 +83,46 @@ the operator, who owns the runner pool. Do not sit in a retry loop waiting for
|
|
|
82
83
|
capacity to return.
|
|
83
84
|
|
|
84
85
|
**Rerunning a failed job to reach green stays forbidden under every verdict,
|
|
85
|
-
`capacity` included.** The verdict changes who owns the fix and where it is
|
|
86
|
+
`capacity` and `unreproducible-tier` included.** The verdict changes who owns the fix and where it is
|
|
86
87
|
filed; it never licenses a re-run, and it is not a route to a green bar. A
|
|
87
88
|
capacity-blocked delivery ends `agent::blocked` — not merged.
|
|
88
89
|
|
|
90
|
+
### The `unreproducible-tier` verdict
|
|
91
|
+
|
|
92
|
+
A check can fail on a tier the sandbox cannot run at all — most often a
|
|
93
|
+
browser suite whose Playwright `webServer` block supervises a dev server the
|
|
94
|
+
local process manager daemonizes, which aborts the run with
|
|
95
|
+
`Process from config.webServer exited early` before any test executes. The
|
|
96
|
+
failure is a property of the sandbox's ability to *host* the suite, not of the
|
|
97
|
+
diff.
|
|
98
|
+
|
|
99
|
+
This is the same structural hole the `capacity` verdict was added to fill, one
|
|
100
|
+
step earlier in the loop. Without it the honest reading is `flaky`, which routes
|
|
101
|
+
to Option 1 — and fix-at-source requires reproducing the failure, which is the
|
|
102
|
+
one thing that cannot be done. The agent then spends the full timebox
|
|
103
|
+
rediscovering that before escalating anyway, and any fix it does author is
|
|
104
|
+
written blind against a tier it never ran.
|
|
105
|
+
|
|
106
|
+
**Unreproducible must be proven, not inferred.** "The suite did not run for me"
|
|
107
|
+
is not the verdict — it is the symptom every misconfiguration produces. Cite
|
|
108
|
+
both:
|
|
109
|
+
|
|
110
|
+
- **The attempted attach.** Work the attach-don't-boot seam in the
|
|
111
|
+
[`playwright`](../skills/stack/qa/playwright/SKILL.md) skill — boot the server
|
|
112
|
+
out-of-band, point the suite at the running origin, set `reuseExistingServer`
|
|
113
|
+
— and name which step failed and how. A tier that runs once attached was never
|
|
114
|
+
unreproducible.
|
|
115
|
+
- **The observed signature.** The verbatim line the runner aborted on, so a
|
|
116
|
+
later reader can tell a lifetime-ownership mismatch from a genuine boot
|
|
117
|
+
failure in the app under test.
|
|
118
|
+
|
|
119
|
+
Absent both readings the verdict is unavailable and the failure routes as it did
|
|
120
|
+
before. On the verdict: file the `meta::framework-gap` issue with the run link,
|
|
121
|
+
the failure signature, and the attach attempt; flip the Story to
|
|
122
|
+
`agent::blocked` with a `friction` comment naming the verdict; and hand back to
|
|
123
|
+
the operator, who owns the sandbox. Do not author a fix for a tier you could not
|
|
124
|
+
run — a blind fix to a suite nobody exercised is how the gap compounds.
|
|
125
|
+
|
|
89
126
|
## Verifier
|
|
90
127
|
|
|
91
128
|
The check is resolved only when it is **green with zero reruns of the failed
|
|
@@ -131,3 +168,9 @@ operator under **any** of:
|
|
|
131
168
|
signature) and escalate on the first encounter rather than burning iterations
|
|
132
169
|
trying to code around it. A proven-capacity failure is this case: reach the
|
|
133
170
|
`capacity` verdict above and escalate on the first encounter.
|
|
171
|
+
- **Unrunnable tier → escalate immediately.** A tier the sandbox cannot host at
|
|
172
|
+
all is this case too: work the attach seam once, reach the
|
|
173
|
+
`unreproducible-tier` verdict above with its two readings, and escalate on the
|
|
174
|
+
**first encounter**. The 30-minute timebox is a ceiling here, never a budget
|
|
175
|
+
to spend — every minute past the failed attach buys nothing, because no
|
|
176
|
+
iteration can make an unhostable suite run.
|
|
@@ -106,6 +106,73 @@ function resolveBaselinePath({ cwd = process.cwd(), baselinePath } = {}) {
|
|
|
106
106
|
* rows: Array<{file: string, method: string, startLine: number, crap: number}>,
|
|
107
107
|
* }|null}
|
|
108
108
|
*/
|
|
109
|
+
/**
|
|
110
|
+
* The envelope-level fields the CRAP compat axes read off a *loaded* baseline
|
|
111
|
+
* — the set every read path owes `assertBaselineCompatible`.
|
|
112
|
+
*
|
|
113
|
+
* It exists because there are TWO read paths and they have now diverged three
|
|
114
|
+
* times. `check-baselines` loads through `baselines/reader.js`; the
|
|
115
|
+
* `quality-preview` pre-commit arm loads through `projectCrapEnvelopeToLegacy`
|
|
116
|
+
* below. Both are ALLOW-LISTS, both feed the same axes, and a stamp added to
|
|
117
|
+
* one and not the other yields two opposite verdicts on one file: Story #4866
|
|
118
|
+
* (`scoringSemantics`, `tsTranspilerVersion`), Story #4969 (`rows[].anonymous`)
|
|
119
|
+
* and Story #4986 (`provenanceStamped`, half-fixed by #4973) were each that
|
|
120
|
+
* same half-landing.
|
|
121
|
+
*
|
|
122
|
+
* Every one of those axes keys on a POSITIVE marker, so a dropped field reads
|
|
123
|
+
* `undefined` and fails the baseline closed with a remedy that cannot work —
|
|
124
|
+
* re-deriving it writes the stamp the read path then discards. The projection
|
|
125
|
+
* below is DERIVED from this list rather than repeating it, so a new stamp is
|
|
126
|
+
* carried here the moment it is named; a parity test holds the reader to the
|
|
127
|
+
* same set and names whichever path forgot one.
|
|
128
|
+
*
|
|
129
|
+
* Row-level markers are deliberately out: they are projected per-row, not as
|
|
130
|
+
* envelope stamps.
|
|
131
|
+
*
|
|
132
|
+
* Deliberately module-local, mirroring `SCORING_SEMANTICS` in
|
|
133
|
+
* `baselines/kinds/crap.js`: the writer's `envelopeExtras()` is the single
|
|
134
|
+
* production door to the stamp set, and exporting this list would add a second
|
|
135
|
+
* one that only a test reaches. The parity test holds this projection to
|
|
136
|
+
* `Object.keys(envelopeExtras())` instead, so a stamp the writer starts
|
|
137
|
+
* emitting fails the test here until it is named — which is the enforcement
|
|
138
|
+
* this list needs, not an export.
|
|
139
|
+
*/
|
|
140
|
+
const COMPAT_STAMP_FIELDS = Object.freeze([
|
|
141
|
+
'scoringSemantics',
|
|
142
|
+
'tsTranspilerVersion',
|
|
143
|
+
'provenanceStamped',
|
|
144
|
+
]);
|
|
145
|
+
|
|
146
|
+
/**
|
|
147
|
+
* Per-stamp coercion applied on the way through the legacy projection. A field
|
|
148
|
+
* with no entry is carried VERBATIM, which is the correct default: the axes
|
|
149
|
+
* distinguish "stamped" from "absent", so inventing a value for a stamp the
|
|
150
|
+
* envelope never wrote is the one thing a read path must not do.
|
|
151
|
+
*/
|
|
152
|
+
const COMPAT_STAMP_NORMALIZERS = {
|
|
153
|
+
// `null` (not the running value) when unstamped, so `ts-transpiler-drift`
|
|
154
|
+
// can tell "written by a different transpiler" from "written before the
|
|
155
|
+
// stamp existed" instead of comparing a value against itself.
|
|
156
|
+
tsTranspilerVersion: (value) => (typeof value === 'string' ? value : null),
|
|
157
|
+
scoringSemantics: (value) => value ?? null,
|
|
158
|
+
};
|
|
159
|
+
|
|
160
|
+
/**
|
|
161
|
+
* Project the compat stamps off a v2 envelope, driven by
|
|
162
|
+
* `COMPAT_STAMP_FIELDS`.
|
|
163
|
+
*
|
|
164
|
+
* @param {Record<string, unknown>} parsed
|
|
165
|
+
* @returns {Record<string, unknown>}
|
|
166
|
+
*/
|
|
167
|
+
function projectCompatStamps(parsed) {
|
|
168
|
+
const stamps = {};
|
|
169
|
+
for (const field of COMPAT_STAMP_FIELDS) {
|
|
170
|
+
const normalize = COMPAT_STAMP_NORMALIZERS[field];
|
|
171
|
+
stamps[field] = normalize ? normalize(parsed[field]) : parsed[field];
|
|
172
|
+
}
|
|
173
|
+
return stamps;
|
|
174
|
+
}
|
|
175
|
+
|
|
109
176
|
/**
|
|
110
177
|
* Story #1895: shipped baseline switched to the canonical envelope shape
|
|
111
178
|
* (`$schema`, `kernelVersion`, `generatedAt`, `rollup`, `rows` keyed on
|
|
@@ -122,6 +189,16 @@ function resolveBaselinePath({ cwd = process.cwd(), baselinePath } = {}) {
|
|
|
122
189
|
* They are carried verbatim now, `null` when the envelope never stamped them,
|
|
123
190
|
* so an axis can tell "written by a different transpiler" apart from "written
|
|
124
191
|
* before the stamp existed" instead of guessing.
|
|
192
|
+
*
|
|
193
|
+
* **This projection is one of TWO (Story #4986).** `check-baselines` reads a
|
|
194
|
+
* baseline through `baselines/reader.js`; `quality-preview` reads the same file
|
|
195
|
+
* through here. Both feed `assertBaselineCompatible`, so a stamp added to one
|
|
196
|
+
* allow-list and not the other produces two opposite verdicts on one envelope —
|
|
197
|
+
* which is what happened to `provenanceStamped`: #4973 added it to the reader
|
|
198
|
+
* and left this projection dropping it, so the pre-commit CRAP arm rejected
|
|
199
|
+
* every stamped baseline with an un-satisfiable "re-seed" remedy while the
|
|
200
|
+
* authoritative gate passed. The stamp block is derived from
|
|
201
|
+
* `COMPAT_STAMP_FIELDS` above so this projection cannot fall behind again.
|
|
125
202
|
*/
|
|
126
203
|
function projectCrapEnvelopeToLegacy(parsed) {
|
|
127
204
|
if (
|
|
@@ -134,11 +211,7 @@ function projectCrapEnvelopeToLegacy(parsed) {
|
|
|
134
211
|
return {
|
|
135
212
|
kernelVersion: parsed.kernelVersion,
|
|
136
213
|
escomplexVersion: resolveEscomplexVersion(),
|
|
137
|
-
|
|
138
|
-
typeof parsed.tsTranspilerVersion === 'string'
|
|
139
|
-
? parsed.tsTranspilerVersion
|
|
140
|
-
: null,
|
|
141
|
-
scoringSemantics: parsed.scoringSemantics ?? null,
|
|
214
|
+
...projectCompatStamps(parsed),
|
|
142
215
|
rows: parsed.rows.map((row) => ({
|
|
143
216
|
crap: row.crap,
|
|
144
217
|
file: row.path,
|
|
@@ -15,6 +15,18 @@
|
|
|
15
15
|
* added because it would re-ask a question the push exit code already
|
|
16
16
|
* answered authoritatively.
|
|
17
17
|
*
|
|
18
|
+
* Issue #4990 — the push runs in the Story worktree, not the main
|
|
19
|
+
* checkout. `core.hooksPath` is the relative `.husky/_`, so the
|
|
20
|
+
* invocation directory is what selects which tree `pre-push` resolves
|
|
21
|
+
* and measures. Pushing from the main checkout therefore validated a
|
|
22
|
+
* tree other than the one being sent — false red when that checkout sat
|
|
23
|
+
* on unrelated work, and worse, false green whenever it happened to be
|
|
24
|
+
* clean. `worktreePath` falls back to `cwd` because single-tree mode and
|
|
25
|
+
* a disabled `delivery.worktreeIsolation` both legitimately have no
|
|
26
|
+
* worktree, and there the main checkout IS the tree being pushed. The
|
|
27
|
+
* two sibling phases resolve their tree the same way
|
|
28
|
+
* (`base-sync.js`, `close-validation.js`).
|
|
29
|
+
*
|
|
18
30
|
* `gitSync` is accepted as an injected dependency rather than statically
|
|
19
31
|
* imported so the caller's (cache-busted) binding wins. The
|
|
20
32
|
* `single-story-close.js` orchestrator owns the static import; test
|
|
@@ -29,6 +41,7 @@ import { gitSync as defaultGitSync } from '../../../git-utils.js';
|
|
|
29
41
|
*
|
|
30
42
|
* @param {{
|
|
31
43
|
* cwd: string,
|
|
44
|
+
* worktreePath?: string|null,
|
|
32
45
|
* storyBranch: string,
|
|
33
46
|
* gitSync?: typeof defaultGitSync,
|
|
34
47
|
* progress: (tag: string, msg: string) => void,
|
|
@@ -36,6 +49,7 @@ import { gitSync as defaultGitSync } from '../../../git-utils.js';
|
|
|
36
49
|
*/
|
|
37
50
|
export function pushStoryBranch({
|
|
38
51
|
cwd,
|
|
52
|
+
worktreePath = null,
|
|
39
53
|
storyBranch,
|
|
40
54
|
gitSync = defaultGitSync,
|
|
41
55
|
progress,
|
|
@@ -46,8 +60,9 @@ export function pushStoryBranch({
|
|
|
46
60
|
// before this point, but `--skip-validation` skips that chain, and the
|
|
47
61
|
// bypass then left nothing running at all. `pre-push` is the backstop,
|
|
48
62
|
// and it only became reachable once hooks were materialized into
|
|
49
|
-
// worktrees — which is where every Story branch is built
|
|
50
|
-
|
|
63
|
+
// worktrees — which is where every Story branch is built, and where the
|
|
64
|
+
// cwd below now resolves so the hook reads that tree.
|
|
65
|
+
gitSync(worktreePath ?? cwd, 'push', '-u', 'origin', storyBranch);
|
|
51
66
|
progress('GIT', `✅ Pushed ${storyBranch}.`);
|
|
52
67
|
} catch (err) {
|
|
53
68
|
throw new Error(
|
|
@@ -201,6 +201,7 @@ async function runPrePushPhases({
|
|
|
201
201
|
|
|
202
202
|
async function openAndReviewPr({
|
|
203
203
|
cwd,
|
|
204
|
+
worktreePath,
|
|
204
205
|
story,
|
|
205
206
|
storyId,
|
|
206
207
|
storyBranch,
|
|
@@ -211,7 +212,11 @@ async function openAndReviewPr({
|
|
|
211
212
|
setPhase = () => {},
|
|
212
213
|
}) {
|
|
213
214
|
setPhase('push');
|
|
214
|
-
|
|
215
|
+
// Issue #4990 — push from the Story worktree so `pre-push` measures the
|
|
216
|
+
// tree being sent. `gh` and the review's ref-based diffs below keep the
|
|
217
|
+
// caller's `cwd`: they read the shared `.git`, so they resolve identically
|
|
218
|
+
// from either tree.
|
|
219
|
+
pushStoryBranch({ cwd, worktreePath, storyBranch, gitSync, progress });
|
|
215
220
|
setPhase('pull-request');
|
|
216
221
|
const { url: prUrl, alreadyMerged } = await ensurePullRequestWith({
|
|
217
222
|
cwd,
|
|
@@ -729,6 +734,7 @@ async function runClosePipeline({
|
|
|
729
734
|
() =>
|
|
730
735
|
openAndReviewPr({
|
|
731
736
|
cwd: options.cwd,
|
|
737
|
+
worktreePath,
|
|
732
738
|
story,
|
|
733
739
|
storyId: options.storyId,
|
|
734
740
|
storyBranch,
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
{
|
|
2
|
-
"generatedAt": "2026-08-
|
|
2
|
+
"generatedAt": "2026-08-05T00:59:17.768Z",
|
|
3
3
|
"generator": "generate-skills-index.js@1",
|
|
4
4
|
"skills": [
|
|
5
5
|
{
|
|
@@ -148,7 +148,7 @@
|
|
|
148
148
|
"category": "qa",
|
|
149
149
|
"path": ".agents/skills/stack/qa/playwright/SKILL.md",
|
|
150
150
|
"description": "Robust E2E browser testing with Playwright. Use when writing browser-driven tests — leverage auto-waiting (no `waitForTimeout`), prefer user-visible locators (`getByRole`, `getByText`, `getByLabel`) over CSS/XPath, reuse `storageState` for auth, and enable trace-on-first-retry for CI debugging.",
|
|
151
|
-
"policyCapsuleBullets":
|
|
151
|
+
"policyCapsuleBullets": 8,
|
|
152
152
|
"allowedTools": null,
|
|
153
153
|
"vendor": "playwright"
|
|
154
154
|
},
|
|
@@ -19,3 +19,51 @@ vendor: playwright
|
|
|
19
19
|
- Write tests independent of one another so they run in parallel; clean up shared state in fixtures, not afterwards.
|
|
20
20
|
- Enable `trace: 'on-first-retry'` (or `'retain-on-failure'`) so CI failures are debuggable in the Trace Viewer.
|
|
21
21
|
- Use a unique data set per test run, or tear down state explicitly, to prevent cross-test contamination.
|
|
22
|
+
- Never let Playwright own the lifetime of a dev server it did not start: boot the server out-of-band, point the suite at the running origin, and set `reuseExistingServer` so `webServer` only probes readiness.
|
|
23
|
+
|
|
24
|
+
## Running a `webServer`-backed suite outside CI
|
|
25
|
+
|
|
26
|
+
Playwright's `webServer` block **watches the process it spawned**. That
|
|
27
|
+
assumption holds for a dev server that stays in the foreground, and breaks for
|
|
28
|
+
any manager that daemonizes one — the foreground process exits `0` while the
|
|
29
|
+
server keeps serving, Playwright reads the exit as a crash, and the run aborts
|
|
30
|
+
before a single test executes:
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
Process from config.webServer exited early
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Read that line as a **lifetime-ownership mismatch, not a flake**. It reproduces
|
|
37
|
+
on every invocation, clean tree or not, and no amount of retrying, tree-cleaning
|
|
38
|
+
or timeout-raising changes it. Agent sandboxes and IDE harnesses commonly manage
|
|
39
|
+
dev servers this way (`Dev server already running at … (pid N)`), so an agent
|
|
40
|
+
meets this far more often than a developer does.
|
|
41
|
+
|
|
42
|
+
### Attach, don't boot
|
|
43
|
+
|
|
44
|
+
Invert the ownership instead of fighting it — the manager owns the process,
|
|
45
|
+
Playwright owns only the probe:
|
|
46
|
+
|
|
47
|
+
1. **Boot the server out-of-band** through whatever manages it, and confirm it
|
|
48
|
+
is serving. Its lifetime is now the manager's concern, not the runner's.
|
|
49
|
+
2. **Point the suite at the already-running origin** — set the config's
|
|
50
|
+
`baseURL` (or the `webServer.url` the block probes) to that origin, via the
|
|
51
|
+
project's own environment seam rather than an edit to committed config.
|
|
52
|
+
3. **Set `reuseExistingServer: true`** so Playwright probes the URL, finds it
|
|
53
|
+
live, and never spawns or supervises a process of its own.
|
|
54
|
+
|
|
55
|
+
This is the same convention the QA harness already encodes as
|
|
56
|
+
`qa.environments[].baseUrl`: attach to a running origin, never boot one. A suite
|
|
57
|
+
run this way exercises identical browser behavior — only the process supervision
|
|
58
|
+
differs.
|
|
59
|
+
|
|
60
|
+
### When no attachable origin exists
|
|
61
|
+
|
|
62
|
+
Some apps genuinely cannot be reached this way — the server is unreachable from
|
|
63
|
+
the sandbox, or the suite depends on a build step the sandbox cannot run. Do
|
|
64
|
+
**not** burn a timebox rediscovering that. Record the observed signature, state
|
|
65
|
+
which of the three steps above failed, and escalate on the first encounter:
|
|
66
|
+
that evidence is exactly what the `unreproducible-tier` verdict in
|
|
67
|
+
[`ci-remediation.md`](../../../../rules/ci-remediation.md) requires, and it is
|
|
68
|
+
the only verdict that lets an unrunnable tier route somewhere other than a dead
|
|
69
|
+
end.
|
package/docs/CHANGELOG.md
CHANGED
|
@@ -15,6 +15,20 @@ All notable changes to this project will be documented in this file.
|
|
|
15
15
|
-->
|
|
16
16
|
<!-- markdownlint-disable-file MD004 MD012 MD037 -->
|
|
17
17
|
|
|
18
|
+
## [2.31.0](https://github.com/dsj1984/mandrel/compare/mandrel-v2.30.0...mandrel-v2.31.0) (2026-08-05)
|
|
19
|
+
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
* **qa:** add an attach-don't-boot seam and an unreproducible-tier verdict (refs [#4994](https://github.com/dsj1984/mandrel/issues/4994)) ([#4995](https://github.com/dsj1984/mandrel/issues/4995)) ([dac7986](https://github.com/dsj1984/mandrel/commit/dac79868a305e34db01005f06f638fc384c21c17)), closes [#4993](https://github.com/dsj1984/mandrel/issues/4993)
|
|
24
|
+
|
|
25
|
+
## [2.30.0](https://github.com/dsj1984/mandrel/compare/mandrel-v2.29.0...mandrel-v2.30.0) (2026-08-04)
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
### Fixed
|
|
29
|
+
|
|
30
|
+
* **baselines:** carry provenanceStamped through the legacy CRAP projection ([#4986](https://github.com/dsj1984/mandrel/issues/4986)) ([#4987](https://github.com/dsj1984/mandrel/issues/4987)) ([921b521](https://github.com/dsj1984/mandrel/commit/921b521f2094c961fec84acca1f90e068cb77798))
|
|
31
|
+
|
|
18
32
|
## [2.29.0](https://github.com/dsj1984/mandrel/compare/mandrel-v2.28.0...mandrel-v2.29.0) (2026-08-03)
|
|
19
33
|
|
|
20
34
|
|
package/package.json
CHANGED