@muggleai/works 5.28.0-staging.249 → 5.28.0-staging.251
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/dist/{chunk-HBCASFGN.js → chunk-JRYEMMKU.js} +39 -0
- package/dist/{chunk-USALT4LJ.js → chunk-XEFALLR5.js} +27 -48
- package/dist/cli.js +2 -2
- package/dist/index.js +2 -2
- package/dist/plugin/.claude-plugin/plugin.json +1 -1
- package/dist/plugin/.codex-mcp.json +1 -1
- package/dist/plugin/.codex-plugin/plugin.json +1 -1
- package/dist/plugin/.cursor-plugin/plugin.json +1 -1
- package/dist/plugin/.mcp.json +1 -1
- package/dist/plugin/agents/acceptance-tester.md +4 -0
- package/dist/plugin/skills/_shared/deliverable-categories/audio.md +29 -0
- package/dist/plugin/skills/_shared/deliverable-categories/data.md +31 -0
- package/dist/plugin/skills/_shared/deliverable-categories/image.md +34 -0
- package/dist/plugin/skills/_shared/deliverable-categories/research.md +28 -0
- package/dist/plugin/skills/_shared/deliverable-categories/video.md +32 -0
- package/dist/plugin/skills/_shared/deliverable-categories/web.md +25 -0
- package/dist/plugin/skills/_shared/deliverable-categories.md +69 -0
- package/dist/plugin/skills/_shared/dev-loop/failures.md +2 -1
- package/dist/plugin/skills/_shared/failure-mode-handling.md +5 -0
- package/dist/plugin/skills/do/e2e-acceptance.md +14 -3
- package/dist/plugin/skills/do/e2e-repair.md +3 -1
- package/dist/plugin/skills/do/impact-analysis.md +17 -3
- package/dist/plugin/skills/do/open-prs/forward.md +2 -2
- package/dist/plugin/skills/do/pre-flight.md +1 -1
- package/dist/plugin/skills/do/requirements.md +3 -3
- package/dist/plugin/skills/muggle-do/SKILL.md +2 -1
- package/dist/plugin/skills/muggle-pr-visual-walkthrough/e2e-report-assembly.md +2 -0
- package/dist/release-manifest.json +3 -3
- package/dist/{src-ZOCXV6PT.js → src-7OY7RXTE.js} +1 -1
- package/package.json +1 -1
- package/plugin/.claude-plugin/plugin.json +1 -1
- package/plugin/.codex-mcp.json +1 -1
- package/plugin/.codex-plugin/plugin.json +1 -1
- package/plugin/.cursor-plugin/plugin.json +1 -1
- package/plugin/.mcp.json +1 -1
- package/plugin/agents/acceptance-tester.md +4 -0
- package/plugin/skills/_shared/deliverable-categories/audio.md +29 -0
- package/plugin/skills/_shared/deliverable-categories/data.md +31 -0
- package/plugin/skills/_shared/deliverable-categories/image.md +34 -0
- package/plugin/skills/_shared/deliverable-categories/research.md +28 -0
- package/plugin/skills/_shared/deliverable-categories/video.md +32 -0
- package/plugin/skills/_shared/deliverable-categories/web.md +25 -0
- package/plugin/skills/_shared/deliverable-categories.md +69 -0
- package/plugin/skills/_shared/dev-loop/failures.md +2 -1
- package/plugin/skills/_shared/failure-mode-handling.md +5 -0
- package/plugin/skills/do/e2e-acceptance.md +14 -3
- package/plugin/skills/do/e2e-repair.md +3 -1
- package/plugin/skills/do/impact-analysis.md +17 -3
- package/plugin/skills/do/open-prs/forward.md +2 -2
- package/plugin/skills/do/pre-flight.md +1 -1
- package/plugin/skills/do/requirements.md +3 -3
- package/plugin/skills/muggle-do/SKILL.md +2 -1
- package/plugin/skills/muggle-pr-visual-walkthrough/e2e-report-assembly.md +2 -0
|
@@ -41,10 +41,13 @@ When the subagent aggregates per-script results from sections B and C into one v
|
|
|
41
41
|
| `infra` (replay or regen) | B / C | **BLOCKED** |
|
|
42
42
|
| `stale-script` (replay) | B | **INCONCLUSIVE** |
|
|
43
43
|
| `product-defect` (replay) | B | **FAIL** |
|
|
44
|
+
| studio verdict `bug_detected` (replay or regen) | B / C | **FAIL** |
|
|
44
45
|
| `product-uxux` (regen) | C | **FAIL** |
|
|
45
46
|
| `agent-course` (regen) | C | **INCONCLUSIVE** (agent couldn't complete, not a PR defect) |
|
|
46
47
|
| `transient` (regen) | C | retry once; if still transient → **INCONCLUSIVE** |
|
|
47
48
|
|
|
49
|
+
A deliverable check ([`deliverable-categories.md`](deliverable-categories.md)) carries no bucket; its own `FAIL` or `INCONCLUSIVE` verdict counts as a test case's **FAIL** or **INCONCLUSIVE** here.
|
|
50
|
+
|
|
48
51
|
When a run mixes buckets across multiple test cases, take the most-significant verdict in this priority order: **FAIL > PARTIAL > BLOCKED > INCONCLUSIVE > SKIPPED > PASS**. If any one test case maps to FAIL, the run is FAIL. If some passed and some were INCONCLUSIVE, the run is PARTIAL.
|
|
49
52
|
|
|
50
53
|
### How orchestrators consume the verdict
|
|
@@ -123,6 +126,7 @@ Call `muggle-local-run-result-get` (local) or the remote equivalent and read **s
|
|
|
123
126
|
- **infra** signals: `electron-crash`, `chromium-error`, `click-no-effect-on-clickable-element`, `timeout-on-trivial-wait`, `internal-error-in-mcp-output`.
|
|
124
127
|
- **stale-script** signals: `element-not-found`, `selector-timeout`, `label-path-mismatch`, `nav-target-404`, `aria-label-changed`.
|
|
125
128
|
- **product-defect** signals: `assertion-failed-on-passing-step`, `unexpected-error-toast`, `wrong-page-after-action`, `network-500-from-app`, `form-validation-rejected-valid-input`.
|
|
129
|
+
- Studio verdict `bug_detected` (`**Studio Verdict:** Bug Detected (bug_detected)` on `muggle-local-run-result-get`) settles the bucket as **product-defect**, status `failed`: it is a product bug the run proved. Surface it to the user as a bug in their app; do not retry, regenerate, or debug the test.
|
|
126
130
|
|
|
127
131
|
If signals span multiple buckets, pick the most specific one and list all signals in telemetry — the user can override.
|
|
128
132
|
|
|
@@ -213,6 +217,7 @@ Order:
|
|
|
213
217
|
- **infra**: `electron-mcp-handler-crash`, `internal-validation-error`, `pipeline-stuck`, identical failure repeated more than twice.
|
|
214
218
|
- **agent-course**: `goal_not_achievable` with summary mentioning the agent picked a different element, looped on a modal, kept trying the same wrong action; many steps but no real progress.
|
|
215
219
|
- **product-uxux**: server 5xx in step screenshots, "page not found" reached repeatedly, expected element provably absent (visible in screenshot), product clearly broken in the artifact log.
|
|
220
|
+
- Studio verdict `bug_detected` (`**Studio Verdict:** Bug Detected (bug_detected)` on `muggle-local-run-result-get`) settles the bucket as **product-uxux**, status `failed`: it is a product bug the run proved. Surface it to the user as a bug in their app; do not retry, regenerate, or debug the test.
|
|
216
221
|
|
|
217
222
|
### Suggestions per bucket
|
|
218
223
|
|
|
@@ -73,7 +73,7 @@ When other work is in front of the session as this change reaches Stage 6, the m
|
|
|
73
73
|
|
|
74
74
|
### Step 0.5: Readiness check
|
|
75
75
|
|
|
76
|
-
A `local-e2e` cycle runs these before any `muggle-local-execute-*` call, every cycle — even when pre-flight already reported the app ready, since a server can die while stages 2–5 run and a healthy one fast-exits:
|
|
76
|
+
A scope with deliverable checks and no browser case runs none of these and goes to Step 5. Otherwise a `local-e2e` cycle runs these before any `muggle-local-execute-*` call, every cycle — even when pre-flight already reported the app ready, since a server can die while stages 2–5 run and a healthy one fast-exits:
|
|
77
77
|
|
|
78
78
|
1. **Dev-server + backend readiness** — invoke [`muggle-test-prepare`](../muggle-test-prepare/SKILL.md) from the cycle's worktree, passing the PR target as `base` — the readiness/service-start owner (idempotent fast-exit when healthy; probes via `dev-server-readiness.md` and starts what is down). It syncs the worktree onto the latest base before starting anything, and serves every other repo from its latest default branch. Its readiness table must show the cycle's worktree as the `Source` of the changed repo's services, at the worktree's current HEAD; any other source is not ready. Still not ready after its start attempts → the readiness halt below.
|
|
79
79
|
2. **Auth** — `muggle-remote-auth-status` must be `authenticated`; else escalate.
|
|
@@ -87,7 +87,7 @@ Only the app not answering is this halt. A run the runner itself lost while read
|
|
|
87
87
|
|
|
88
88
|
Pre-flight handled auth. If `muggle-remote-auth-status` somehow shows expired here (session clock skew, etc.), re-auth silently via `muggle-remote-auth-login` + `muggle-remote-auth-poll` — but do not ask the user "continue with this account?" again.
|
|
89
89
|
|
|
90
|
-
If validation is `unit-only` or `skip`, emit a `SKIPPED` report citing the code that holds — `NO_WEB_SURFACE` when the repo has nothing to drive, `DEV_SERVER_UNREACHABLE` when it does but nothing answers — and exit cleanly. This is the only exit `DEV_SERVER_UNREACHABLE` belongs to, because pre-flight never planned a browser run; on a `local-e2e` cycle an unreachable server is the Step 0.5 readiness halt.
|
|
90
|
+
If validation is `unit-only` or `skip`, the scope's test cases — the criteria whose category flow is **Verified by** test cases ([`../_shared/deliverable-categories.md`](../_shared/deliverable-categories.md#category-flows)) — get no run: emit a `SKIPPED` report citing the code that holds — `NO_WEB_SURFACE` when the repo has nothing to drive, `DEV_SERVER_UNREACHABLE` when it does but nothing answers — and exit cleanly. Every other criterion follows its own flow: under `unit-only`, a scope that also holds deliverable checks records that code as the report's `**Browser skip:**` line instead and goes straight to Step 5 rather than exiting. This is the only exit `DEV_SERVER_UNREACHABLE` belongs to, because pre-flight never planned a browser run; on a `local-e2e` cycle an unreachable server is the Step 0.5 readiness halt.
|
|
91
91
|
|
|
92
92
|
If validation is `staging-replay`, emit `INCONCLUSIVE` with reason `staging replay not handled in Stage 6 local runner path` and exit cleanly.
|
|
93
93
|
|
|
@@ -141,11 +141,18 @@ Inputs to the loop: `mode` from the script-exists check, `localUrl`/project from
|
|
|
141
141
|
|
|
142
142
|
Each record's run is its own report entry carrying its `runSettingsId` and `resolutionLabel`. A record skipped because its primary failed is an Inconclusive entry with reason `primary failed`, not a skipped test case.
|
|
143
143
|
|
|
144
|
+
### Step 5: Run the deliverable checks
|
|
145
|
+
|
|
146
|
+
Run every `deliverableChecks` entry of the scope per [`../_shared/deliverable-categories.md`](../_shared/deliverable-categories.md): obtain the artifact from the working tree recorded in `state.md`, run each planned probe plus the category's review, and record one report entry per check in the **Deliverables** block. Probe output, extracted frames, and transcripts go in a scratch directory outside the worktree, recorded as the entry's `evidenceDir`. A check that could not run is an `INCONCLUSIVE` entry with its reason, never absent.
|
|
147
|
+
|
|
148
|
+
Deliverable checks do not satisfy the session's browser E2E gate. A cycle that ran no browser case still declares its browser skip with `echo "MUGGLE_E2E_SKIP: <CODE>: <detail>"`: `NO_WEB_SURFACE` when that holds, else `NOT_APPLICABLE` with a detail naming the categories checked instead, e.g. `NOT_APPLICABLE: change ships only video and image deliverables, verified by deliverable checks DC1-DC2`.
|
|
149
|
+
|
|
144
150
|
### Step 7: Collect Results
|
|
145
151
|
|
|
146
152
|
For each test case:
|
|
147
153
|
- Record pass or fail from the run result
|
|
148
154
|
- If failed, capture the error message, failure step index, and `artifactsDir` for local debugging
|
|
155
|
+
- A studio verdict of `bug_detected` (`**Studio Verdict:** Bug Detected (bug_detected)` on `muggle-local-run-result-get`) is failed: it is a product bug the run proved. Surface it to the user as a bug in their app; do not retry, regenerate, or debug the test
|
|
149
156
|
- In `local-e2e` mode, every scoped test case must be executed — generate a new script if none exists (no skips)
|
|
150
157
|
|
|
151
158
|
## Output
|
|
@@ -185,6 +192,10 @@ For each test case:
|
|
|
185
192
|
- reason: `<one short sentence>`
|
|
186
193
|
- steps: `[{ stepIndex, action, description, screenshotUrl }, ...]` (empty only when the run produced no steps)
|
|
187
194
|
|
|
195
|
+
**Deliverables:** (count) — one entry per deliverable check from Step 5, in the report-entry shape of [`../_shared/deliverable-categories.md`](../_shared/deliverable-categories.md#report-entry). Omitted when the scope planned none.
|
|
196
|
+
|
|
197
|
+
**Browser skip:** `<CODE>: <detail>` — only when Step 1 skipped the browser part of a cycle that still ran deliverable checks.
|
|
198
|
+
|
|
188
199
|
**Scope:** (from `e2eScope`, after Step 3)
|
|
189
200
|
- impacted: `<testCaseId>`, ...
|
|
190
201
|
- new: `<key>` → `<testCaseId>`, ...
|
|
@@ -194,7 +205,7 @@ For each test case:
|
|
|
194
205
|
**Metadata:**
|
|
195
206
|
- projectId: `<projectId>`
|
|
196
207
|
|
|
197
|
-
**Overall:** PASS | FAIL | PARTIAL | INCONCLUSIVE | BLOCKED | SKIPPED — see [`../_shared/failure-mode-handling.md`](../_shared/failure-mode-handling.md) section F for the canonical taxonomy.
|
|
208
|
+
**Overall:** PASS | FAIL | PARTIAL | INCONCLUSIVE | BLOCKED | SKIPPED — see [`../_shared/failure-mode-handling.md`](../_shared/failure-mode-handling.md) section F for the canonical taxonomy. Deliverable entries count toward it like test cases: a `FAIL` entry fails the run, an `INCONCLUSIVE` one is a gap. A browser skip beside deliverable entries is not `SKIPPED` — the run's verdict comes from the entries that ran.
|
|
198
209
|
|
|
199
210
|
A non-passing verdict does not end the cycle. [`e2e-repair.md`](e2e-repair.md) consumes this report at Stage 7.5 and root-causes every Failed and Inconclusive entry before a watcher is armed, so write those blocks in full — the bucket, failing step, and `artifactsDir` recorded here are that stage's entire input.
|
|
200
211
|
|
|
@@ -26,7 +26,7 @@ Read the Stage 6 run-level verdict, defined in [`../_shared/failure-mode-handlin
|
|
|
26
26
|
- `PASS` or `SKIPPED` → nothing to repair. Append `e2e-repair: clear (<verdict>)` to the iteration log and hand to Stage 8.
|
|
27
27
|
- A verdict the user waived earlier in this session → clear on the recorded waiver; never re-ask.
|
|
28
28
|
- An entry whose run failed because the app did not answer — connection refused, page never loaded — is the readiness halt of [`e2e-acceptance.md`](e2e-acceptance.md) Step 0.5 arriving late. Nothing ran, so there is nothing to repair and nothing to waive, attended or not: re-check readiness there, and if the app is still down, stop as Step 4's **Stop here** and report the halt. A run the runner lost while the app answers — an Electron, MCP or Studio crash, even before the first step — is not this: it is an `infra` failure and takes the loop below.
|
|
29
|
-
- `FAIL`, `PARTIAL`, `INCONCLUSIVE`, or `BLOCKED` → the loop runs over every entry in the Failed and Inconclusive blocks.
|
|
29
|
+
- `FAIL`, `PARTIAL`, `INCONCLUSIVE`, or `BLOCKED` → the loop runs over every entry in the Failed and Inconclusive blocks, and every non-`PASS` entry in the Deliverables block.
|
|
30
30
|
|
|
31
31
|
## Step 1: Root-cause each failure
|
|
32
32
|
|
|
@@ -56,6 +56,8 @@ A `repair` disposition requires a named defect: the failing step, expected-versu
|
|
|
56
56
|
|
|
57
57
|
**A failure the diff cannot explain may predate it.** When a `product-defect` or `product-uxux` failure names no code path the diff touched, replay the same test case against a checkout of the base before repairing anything: a worktree at the base tip's full SHA, with its own stack claim and services, under the same runSettings record. If that run fails the same way, its disposition is `baseline` — the defect is on the base, not this change's to fix — and the entry carries that run's id as `baselineRunId` into the walkthrough report ([`../muggle-pr-visual-walkthrough/e2e-report-assembly.md`](../muggle-pr-visual-walkthrough/e2e-report-assembly.md)). If it passes, the change caused the failure and the original disposition stands. A `baseline` entry is never re-run or waived; when every remaining failure is `baseline`, the loop clears.
|
|
58
58
|
|
|
59
|
+
A Deliverables entry carries no bucket. A `FAIL` entry is already a named defect — the failed probe or review with its expected and observed values — so its disposition is `repair`. An `INCONCLUSIVE` entry is `ask`, naming its reason: a missing tool or unreachable source is the machine's to fix, not the diff's. A Step 5 re-run re-checks only the deliverable entries the repair touched.
|
|
60
|
+
|
|
59
61
|
An Inconclusive entry Stage 6 recorded as `prerequisites unmet` never executed. It carries no bucket and none may be derived from it — there is no run to read. Its disposition is `ask`, and the question names the missing prerequisite: what failed is the project's test-plan chain, not the diff.
|
|
60
62
|
|
|
61
63
|
An Inconclusive entry Stage 6 recorded as `primary failed` never executed either: it is a record held back by its test case's failed primary. It takes no disposition of its own; the primary's entry is the one repaired, and re-running the test case runs it again.
|
|
@@ -101,6 +101,8 @@ The case selection [`e2e-acceptance.md`](e2e-acceptance.md) runs. Stage 6 execut
|
|
|
101
101
|
|
|
102
102
|
Scope only when this cycle runs E2E: `Validation` is `local-e2e`, `pathClassification` is not `none`, and in a watcher cycle `cycleSize` is `full`. Otherwise emit `e2eScope: not-scoped — <the condition that failed>`.
|
|
103
103
|
|
|
104
|
+
Deliverable checks ([`../_shared/deliverable-categories.md`](../_shared/deliverable-categories.md)) need no browser, dev server, or Muggle session, so they are scoped on their own rule: whenever `Validation` is not `skip`, `pathClassification` is not `none`, and in a watcher cycle `cycleSize` is `full`. A cycle that scopes deliverable checks but no browser cases emits only `projectId: none`, `deliverableChecks`, and `criteria`; the browser lists are written empty.
|
|
105
|
+
|
|
104
106
|
The criteria are the acceptance criteria in the session's `requirements.md`, plus each review-derived change item that alters behavior in an address-reviews cycle. A session with no `requirements.md` (an adopted PR) scopes from the diff alone and writes `criteria: []`.
|
|
105
107
|
|
|
106
108
|
### Procedure
|
|
@@ -113,9 +115,11 @@ The criteria are the acceptance criteria in the session's `requirements.md`, plu
|
|
|
113
115
|
- **Removed** — the change removes the feature it verifies. `replacedBy: none`.
|
|
114
116
|
|
|
115
117
|
A use case is retired only when the change removes its whole flow **and** every test case under it is retired: deleting a use case cascades to all its test cases and scripts, so one surviving test case keeps the use case. Retired cases are deleted after the PR merges ([`cleanup.md`](cleanup.md)), never during the cycle — an unmerged change must leave the project's suite intact for the default branch.
|
|
116
|
-
5. **Map every criterion.**
|
|
118
|
+
5. **Map every criterion.** A criterion's mapping follows its category's flow file ([`../_shared/deliverable-categories.md`](../_shared/deliverable-categories.md#category-flows)), never the category's name. A flow **Verified by** test cases gives the criterion `coveredBy` (impacted or new test cases that verify it); a flow **Verified by** a deliverable check gives it `verifiedBy` (the deliverable checks that verify it). Either may instead carry `unobservable` (why no check can observe it: a log line, a unit-test requirement, a config value). A criterion with none of the three is a scoping failure.
|
|
117
119
|
6. **Plan the new cases.** Each uncovered criterion gets a new test case under the impacted use case its behavior belongs to, or under a new use case when none fits. A test case instruction names one atomic behavior to verify, with no signup, login, navigation, or seed-data steps, per [`../_shared/test-case-design.md`](../_shared/test-case-design.md); prerequisites come from the test-plan graph at run time ([`../_shared/test-case-chain-readiness.md`](../_shared/test-case-chain-readiness.md)).
|
|
118
120
|
|
|
121
|
+
7. **Plan the deliverable checks.** One check per artifact that the criteria mapped to `verifiedBy` are about, keyed `DC1`, `DC2`, …: its category, the artifact and how it is obtained (committed path, or the build command), the probes with their expected values, and the criteria it verifies. A changed artifact that no criterion names is still checked when the diff touches it — the probes alone, with expected values from its base-branch version.
|
|
122
|
+
|
|
119
123
|
This stage plans new cases and creates nothing. Stage 6 creates them, so a cycle that never reaches E2E leaves no cases behind in the project.
|
|
120
124
|
|
|
121
125
|
### Shape
|
|
@@ -148,11 +152,21 @@ e2eScope:
|
|
|
148
152
|
- key: NTC1
|
|
149
153
|
useCase: <useCaseId | new use case key>
|
|
150
154
|
instruction: <one atomic behavior to verify>
|
|
155
|
+
deliverableChecks:
|
|
156
|
+
- key: DC1
|
|
157
|
+
category: <video | image | audio | data | research>
|
|
158
|
+
artifact: <repo-relative path | build: <command> → <output>>
|
|
159
|
+
probes:
|
|
160
|
+
- probe: <property>
|
|
161
|
+
expected: <value or range>
|
|
162
|
+
criteria: [<criterion>, ...]
|
|
151
163
|
criteria:
|
|
152
|
-
- criterion: <acceptance criterion>
|
|
164
|
+
- criterion: <[<category verified by test cases>] acceptance criterion>
|
|
153
165
|
coveredBy: [<testCaseId | new test case key>, ...]
|
|
166
|
+
- criterion: <[<category verified by a deliverable check>] acceptance criterion>
|
|
167
|
+
verifiedBy: [<DC key>, ...]
|
|
154
168
|
- criterion: <acceptance criterion>
|
|
155
|
-
unobservable: <why no
|
|
169
|
+
unobservable: <why no check can observe it>
|
|
156
170
|
```
|
|
157
171
|
|
|
158
172
|
An empty list is written empty, never omitted; an absent list reads as "not scoped". Record the scope in this stage's marker in `iterations/<NNN>.md`, so Stage 6, a Stage 7.5 re-run, and a resumed session read the same one.
|
|
@@ -39,7 +39,7 @@ Forward pipeline's Stage 7. Invoked by `/muggle-do` after stages 1–6 of a fres
|
|
|
39
39
|
- `## Goal` — from requirements.
|
|
40
40
|
- `## Acceptance Criteria` — bulleted; omit if empty.
|
|
41
41
|
- `## Changes` — summary of what changed in this repo.
|
|
42
|
-
- `## Validation` — one line: link to E2E report, `unit-only`, or `skip — <reason>`.
|
|
42
|
+
- `## Validation` — one line: link to E2E report, `unit-only`, or `skip — <reason>`. A report with a **Deliverables** block adds its tables below that line, per [`../../_shared/deliverable-categories.md`](../../_shared/deliverable-categories.md#evidence-on-the-pr).
|
|
43
43
|
- The walkthrough does **not** go in the body. It settles the PR's designated walkthrough comment in Step 6, so a rerun updates one comment instead of rewriting the description.
|
|
44
44
|
- **Signature** — write the assembled body to a file and sign it with `--command /muggle-do --mode editable` per [`../../_shared/vcs/post-signature.md`](../../_shared/vcs/post-signature.md). The signature lands last, after the walkthrough block; `editable` is the mode a description carries so later refreshes replace it instead of stacking.
|
|
45
45
|
|
|
@@ -49,7 +49,7 @@ Forward pipeline's Stage 7. Invoked by `/muggle-do` after stages 1–6 of a fres
|
|
|
49
49
|
|
|
50
50
|
6. **Settle the designated walkthrough comment.** Creating the PR reserves one comment for the walkthrough, marked `muggle-pr-walkthrough` and empty until settled — the PR's walkthrough check fails while it stays that way.
|
|
51
51
|
- E2E report exists → fire [`postPRVisualWalkthrough`](../../muggle-preferences/preference-gates/postPRVisualWalkthrough.md); on skip, leave the comment to the gate's own skip record. Otherwise invoke [`../../muggle-pr-visual-walkthrough/SKILL.md`](../../muggle-pr-visual-walkthrough/SKILL.md) Mode A with the PR number and repo passed explicitly — it fills the reserved comment in place. Explicit because a session carrying several changes may sit in another change's worktree, where the current branch's PR is the wrong one.
|
|
52
|
-
- No E2E report (validation was `unit-only` or `skip`),
|
|
52
|
+
- No E2E report (validation was `unit-only` or `skip`), a `SKIPPED` report, or a report whose browser part was skipped (a `**Browser skip:**` line, or deliverable entries and no test cases) → now that the PR exists, re-declare the skip Stage 6 recorded with the same code, from this PR's worktree: `echo "MUGGLE_E2E_SKIP: <CODE>: <detail>"`, codes per [`../e2e-acceptance.md`](../e2e-acceptance.md). An accepted code settles only the PRs of that worktree's change — the PRs opened from it and its checked-out branch's PR; a free-text reason is rejected and leaves it unsettled. A PR whose title says `[UNIT-ONLY]` still owes reviewers the why. The write runs detached, so confirm it landed: the walkthrough Stop gate holds the turn while the PR still shows the placeholder, and its block names a foreground command that prints `{"settled":true}` once the PR reads back the skip — run it, never a hand-written PATCH.
|
|
53
53
|
|
|
54
54
|
7. **Overflow comment:** if the walkthrough skill returned a non-null `comment`, post it once using the provider resolved in Step 5 — `github` per [`../../_shared/vcs/github/top-level-comment.md`](../../_shared/vcs/github/top-level-comment.md), `gitlab` per [`../../_shared/vcs/gitlab/mr-note.md`](../../_shared/vcs/gitlab/mr-note.md). End the posted body with the signature line (command `/muggle-do`) per [`../../_shared/vcs/post-signature.md`](../../_shared/vcs/post-signature.md). Never post when `comment` is `null`.
|
|
55
55
|
|
|
@@ -60,7 +60,7 @@ Present **one `AskUserQuestion`** (or the platform's structured-selection equiva
|
|
|
60
60
|
4. **Validation strategy** — the single most important question. Options:
|
|
61
61
|
- **Local E2E** (Muggle Test Electron against a running localhost) — default if a dev server was detected.
|
|
62
62
|
- **Staging replay** — for changes already deployed to a preview URL.
|
|
63
|
-
- **Unit tests only** — skip E2E, acceptable for pure refactors or backend-only changes.
|
|
63
|
+
- **Unit tests only** — skip browser E2E, acceptable for pure refactors or backend-only changes. Deliverable checks on video, image, audio, data, or research artifacts still run.
|
|
64
64
|
- **Skip validation** — explicit opt-out; the PR title gets `[UNVERIFIED]`.
|
|
65
65
|
5. **Local URL** — only if validation is Local E2E. Default: the detected port. "Confirm `<detected>`" / "Type a different URL".
|
|
66
66
|
6. **Backend reachable?** — only if validation is Local E2E and a backend URL is declared. If the health probe failed, ask "Start the backend now and I'll re-probe" / "Proceed anyway" / "Skip to unit tests only".
|
|
@@ -22,7 +22,7 @@ You receive:
|
|
|
22
22
|
|
|
23
23
|
1. **Read the task description carefully.** Understand what the user wants to build, fix, or change.
|
|
24
24
|
2. **Extract the goal** — one clear sentence describing the outcome.
|
|
25
|
-
3. **Extract acceptance criteria** — specific, verifiable conditions that must be true when the task is done. Each criterion should be independently testable. If the user's description is vague, infer reasonable criteria but flag them as inferred.
|
|
25
|
+
3. **Extract acceptance criteria** — specific, verifiable conditions that must be true when the task is done. Each criterion should be independently testable. If the user's description is vague, infer reasonable criteria but flag them as inferred. Tag each criterion with its deliverable category per [`../_shared/deliverable-categories.md`](../_shared/deliverable-categories.md#tagging-a-criterion), splitting any criterion that spans two.
|
|
26
26
|
4. **Identify which repos are likely affected** — based on the task description and the repo names provided.
|
|
27
27
|
|
|
28
28
|
## Output
|
|
@@ -33,8 +33,8 @@ Write the findings to `~/.muggle-ai/muggle-do/sessions/<slug>/requirements.md`.
|
|
|
33
33
|
**Goal:** (one sentence)
|
|
34
34
|
|
|
35
35
|
**Acceptance Criteria:**
|
|
36
|
-
- (criterion 1)
|
|
37
|
-
- (criterion 2)
|
|
36
|
+
- [<category>] (criterion 1)
|
|
37
|
+
- [<category>] (criterion 2)
|
|
38
38
|
|
|
39
39
|
**Affected Repos:** (comma-separated list)
|
|
40
40
|
|
|
@@ -55,7 +55,8 @@ Do not create or update a PR until each line holds, or is waived by a one-line r
|
|
|
55
55
|
- Build clean — typecheck + lint on changed files
|
|
56
56
|
- New/changed logic carries unit tests (authored in Stage 3; Stage 5 only runs the suite)
|
|
57
57
|
- Unit suite run, PASS recorded
|
|
58
|
-
- Stage 4's `e2eScope` recorded, mapping every acceptance criterion to a test case or
|
|
58
|
+
- Stage 4's `e2eScope` recorded, mapping every acceptance criterion to a test case, a deliverable check, or `unobservable` — or `not-scoped` with the condition that failed
|
|
59
|
+
- A verdict recorded for every deliverable check in that scope
|
|
59
60
|
- E2E verdict recorded with `runId` for every case in that scope, per `autoE2ETest` — or `[E2E FAILING]`, or `SKIPPED` citing a verified skip code from [`../do/e2e-acceptance.md`](../do/e2e-acceptance.md), where each code's verification rule lives. Free-text reasons are rejected by the gate
|
|
60
61
|
- UI changes verified in a real browser with evidence (screenshot path or muggle `runId`); `curl` + `grep` is not verification
|
|
61
62
|
|
|
@@ -10,6 +10,8 @@ Include **all** runs — passed, failed, **and inconclusive**. Never drop a run.
|
|
|
10
10
|
| `failed` | Run completed and the assertion failed, or the product itself broke before the assertion could be made (server error, 500, broken page). | `failureStepIndex`, `error` |
|
|
11
11
|
| `inconclusive` | Run could not yield a pass/fail signal for reasons **outside the product**: no replayable script existed, environment precondition unmet, infra/Electron error, agent stalled on a cookie banner or login wall, missing secrets, agent went off-course. | `reason` (one short sentence). Include every step the run produced; `steps[]` is empty only when it produced none. |
|
|
12
12
|
|
|
13
|
+
A run whose studio verdict is `bug_detected` (`**Studio Verdict:** Bug Detected (bug_detected)` on `muggle-local-run-result-get`) is `failed`, never `inconclusive`: it is a product bug the run proved. Put what the app did against what was expected in `error`; do not retry, regenerate, or debug the test.
|
|
14
|
+
|
|
13
15
|
If a test should be inconclusive but you don't have a real `runId` / `viewUrl` (e.g., generation never ran), use the same project-level dashboard fallback as the "last-resort" failed branch below: synthesize a UUID-shaped runId and use the project runs page as `viewUrl`. The schema requires both fields — but a working dashboard link is better than nothing.
|
|
14
16
|
|
|
15
17
|
A record skipped because its test case's primary failed never ran either: an `inconclusive` entry with `reason: "primary failed"`, the `runSettingsId` and `resolutionLabel` the caller carried for it, `steps: []`, and that same synthesized `runId` and project-level `viewUrl`.
|