@muggleai/works 4.10.0 → 4.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/dist/{chunk-RK6XKSZK.js → chunk-Q7WNEMN4.js} +109 -43
- package/dist/{chunk-UVXOHXY2.js → chunk-ZKRHFTIC.js} +832 -89
- package/dist/cli.js +2 -2
- package/dist/index.js +2 -2
- package/dist/plugin/.claude-plugin/plugin.json +1 -1
- package/dist/plugin/.cursor-plugin/plugin.json +1 -1
- package/dist/plugin/README.md +4 -2
- package/dist/plugin/agents/acceptance-tester.md +24 -10
- package/dist/plugin/commands/m.md +7 -0
- package/dist/plugin/commands/mdo.md +7 -0
- package/dist/plugin/commands/mfeedback.md +7 -0
- package/dist/plugin/commands/mimport.md +7 -0
- package/dist/plugin/commands/mpr.md +7 -0
- package/dist/plugin/commands/mprefs.md +7 -0
- package/dist/plugin/commands/mregen.md +7 -0
- package/dist/plugin/commands/mrelease.md +7 -0
- package/dist/plugin/commands/mrepair.md +7 -0
- package/dist/plugin/commands/mstatus.md +7 -0
- package/dist/plugin/commands/mtest.md +7 -0
- package/dist/plugin/commands/mtestlocal.md +7 -0
- package/dist/plugin/commands/mtestprep.md +7 -0
- package/dist/plugin/commands/mupgrade.md +7 -0
- package/dist/plugin/scripts/ensure-electron-app.sh +35 -8
- package/dist/plugin/skills/_aliases.json +18 -0
- package/dist/plugin/skills/_shared/dev-server-readiness.md +231 -0
- package/dist/plugin/skills/_shared/failure-mode-handling.md +244 -0
- package/dist/plugin/skills/_shared/post-merge-cleanup.md +10 -0
- package/dist/plugin/skills/_shared/rebase-before-e2e.md +15 -0
- package/dist/plugin/skills/_shared/telemetry-emit.md +15 -0
- package/dist/plugin/skills/_shared/use-worktrees.md +53 -0
- package/dist/plugin/skills/do/e2e-acceptance.md +63 -18
- package/dist/plugin/skills/do/impact-analysis.md +45 -5
- package/dist/plugin/skills/do/open-prs.md +13 -4
- package/dist/plugin/skills/do/pr-followup.md +225 -0
- package/dist/plugin/skills/do/pre-flight.md +18 -11
- package/dist/plugin/skills/m/SKILL.md +8 -0
- package/dist/plugin/skills/mdo/SKILL.md +8 -0
- package/dist/plugin/skills/mfeedback/SKILL.md +8 -0
- package/dist/plugin/skills/mimport/SKILL.md +8 -0
- package/dist/plugin/skills/mpr/SKILL.md +8 -0
- package/dist/plugin/skills/mprefs/SKILL.md +8 -0
- package/dist/plugin/skills/mregen/SKILL.md +8 -0
- package/dist/plugin/skills/mrelease/SKILL.md +8 -0
- package/dist/plugin/skills/mrepair/SKILL.md +8 -0
- package/dist/plugin/skills/mstatus/SKILL.md +8 -0
- package/dist/plugin/skills/mtest/SKILL.md +8 -0
- package/dist/plugin/skills/mtestlocal/SKILL.md +8 -0
- package/dist/plugin/skills/mtestprep/SKILL.md +8 -0
- package/dist/plugin/skills/muggle/SKILL.md +9 -6
- package/dist/plugin/skills/muggle-do/SKILL.md +22 -4
- package/dist/plugin/skills/muggle-do-pr-followup/SKILL.md +37 -0
- package/dist/plugin/skills/muggle-do-task/SKILL.md +134 -0
- package/dist/plugin/skills/muggle-feedback/SKILL.md +38 -0
- package/dist/plugin/skills/muggle-feedback/ops/delete.md +49 -0
- package/dist/plugin/skills/muggle-feedback/ops/list.md +65 -0
- package/dist/plugin/skills/muggle-feedback/ops/submit.md +117 -0
- package/dist/plugin/skills/muggle-pr-visual-walkthrough/SKILL.md +68 -43
- package/dist/plugin/skills/muggle-pr-visual-walkthrough/e2e-report-assembly.md +86 -0
- package/dist/plugin/skills/muggle-preferences/SKILL.md +16 -108
- package/dist/plugin/skills/muggle-preferences/ops/change-one.md +10 -0
- package/dist/plugin/skills/muggle-preferences/ops/configure.md +49 -0
- package/dist/plugin/skills/muggle-preferences/ops/list.md +9 -0
- package/dist/plugin/skills/muggle-preferences/ops/reset.md +6 -0
- package/dist/plugin/skills/muggle-preferences/ops/set.md +10 -0
- package/dist/plugin/skills/muggle-preferences/preference-gates/README.md +1 -1
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoCleanup.md +11 -0
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoCreatePR.md +13 -0
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoDetectChanges.md +3 -3
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoE2ETest.md +11 -0
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoLogin.md +1 -1
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoPublishLocalResults.md +2 -2
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoRebase.md +11 -0
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoSelectLocalHost.md +19 -0
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoSelectProject.md +1 -2
- package/dist/plugin/skills/muggle-preferences/preference-gates/autoUseWorktree.md +11 -0
- package/dist/plugin/skills/muggle-preferences/preference-gates/checkForUpdates.md +4 -4
- package/dist/plugin/skills/muggle-preferences/preference-gates/defaultExecutionMode.md +1 -1
- package/dist/plugin/skills/muggle-preferences/preference-gates/openTestResultsAfterRun.md +1 -1
- package/dist/plugin/skills/muggle-preferences/preference-gates/postPRVisualWalkthrough.md +8 -25
- package/dist/plugin/skills/muggle-preferences/preference-gates/showElectronBrowser.md +1 -1
- package/dist/plugin/skills/muggle-preferences/preference-gates/suggestRelatedTestCases.md +4 -4
- package/dist/plugin/skills/muggle-preferences/preference-gates/suggestRelatedUseCases.md +4 -4
- package/dist/plugin/skills/muggle-repair/SKILL.md +4 -2
- package/dist/plugin/skills/muggle-status/SKILL.md +8 -5
- package/dist/plugin/skills/muggle-test/SKILL.md +110 -86
- package/dist/plugin/skills/muggle-test-feature-local/SKILL.md +73 -66
- package/dist/plugin/skills/muggle-test-import/SKILL.md +28 -23
- package/dist/plugin/skills/muggle-test-prepare/SKILL.md +79 -18
- package/dist/plugin/skills/muggle-test-regenerate-missing/SKILL.md +34 -18
- package/dist/plugin/skills/muggle-test-regenerate-missing/evals/evals.json +9 -9
- package/dist/plugin/skills/muggle-upgrade/SKILL.md +4 -2
- package/dist/plugin/skills/muggle-works-npm-release/SKILL.md +6 -4
- package/dist/plugin/skills/mupgrade/SKILL.md +8 -0
- package/dist/release-manifest.json +4 -4
- package/dist/{src-GD3YFCCW.js → src-2G7ENRGI.js} +1 -1
- package/package.json +9 -7
- package/plugin/.claude-plugin/plugin.json +1 -1
- package/plugin/.cursor-plugin/plugin.json +1 -1
- package/plugin/README.md +4 -2
- package/plugin/agents/acceptance-tester.md +24 -10
- package/plugin/commands/m.md +7 -0
- package/plugin/commands/mdo.md +7 -0
- package/plugin/commands/mfeedback.md +7 -0
- package/plugin/commands/mimport.md +7 -0
- package/plugin/commands/mpr.md +7 -0
- package/plugin/commands/mprefs.md +7 -0
- package/plugin/commands/mregen.md +7 -0
- package/plugin/commands/mrelease.md +7 -0
- package/plugin/commands/mrepair.md +7 -0
- package/plugin/commands/mstatus.md +7 -0
- package/plugin/commands/mtest.md +7 -0
- package/plugin/commands/mtestlocal.md +7 -0
- package/plugin/commands/mtestprep.md +7 -0
- package/plugin/commands/mupgrade.md +7 -0
- package/plugin/scripts/ensure-electron-app.sh +35 -8
- package/plugin/skills/_aliases.json +18 -0
- package/plugin/skills/_shared/dev-server-readiness.md +231 -0
- package/plugin/skills/_shared/failure-mode-handling.md +244 -0
- package/plugin/skills/_shared/post-merge-cleanup.md +10 -0
- package/plugin/skills/_shared/rebase-before-e2e.md +15 -0
- package/plugin/skills/_shared/telemetry-emit.md +15 -0
- package/plugin/skills/_shared/use-worktrees.md +53 -0
- package/plugin/skills/do/e2e-acceptance.md +63 -18
- package/plugin/skills/do/impact-analysis.md +45 -5
- package/plugin/skills/do/open-prs.md +13 -4
- package/plugin/skills/do/pr-followup.md +225 -0
- package/plugin/skills/do/pre-flight.md +18 -11
- package/plugin/skills/m/SKILL.md +8 -0
- package/plugin/skills/mdo/SKILL.md +8 -0
- package/plugin/skills/mfeedback/SKILL.md +8 -0
- package/plugin/skills/mimport/SKILL.md +8 -0
- package/plugin/skills/mpr/SKILL.md +8 -0
- package/plugin/skills/mprefs/SKILL.md +8 -0
- package/plugin/skills/mregen/SKILL.md +8 -0
- package/plugin/skills/mrelease/SKILL.md +8 -0
- package/plugin/skills/mrepair/SKILL.md +8 -0
- package/plugin/skills/mstatus/SKILL.md +8 -0
- package/plugin/skills/mtest/SKILL.md +8 -0
- package/plugin/skills/mtestlocal/SKILL.md +8 -0
- package/plugin/skills/mtestprep/SKILL.md +8 -0
- package/plugin/skills/muggle/SKILL.md +9 -6
- package/plugin/skills/muggle-do/SKILL.md +22 -4
- package/plugin/skills/muggle-do-pr-followup/SKILL.md +37 -0
- package/plugin/skills/muggle-do-task/SKILL.md +134 -0
- package/plugin/skills/muggle-feedback/SKILL.md +38 -0
- package/plugin/skills/muggle-feedback/ops/delete.md +49 -0
- package/plugin/skills/muggle-feedback/ops/list.md +65 -0
- package/plugin/skills/muggle-feedback/ops/submit.md +117 -0
- package/plugin/skills/muggle-pr-visual-walkthrough/SKILL.md +68 -43
- package/plugin/skills/muggle-pr-visual-walkthrough/e2e-report-assembly.md +86 -0
- package/plugin/skills/muggle-preferences/SKILL.md +16 -108
- package/plugin/skills/muggle-preferences/ops/change-one.md +10 -0
- package/plugin/skills/muggle-preferences/ops/configure.md +49 -0
- package/plugin/skills/muggle-preferences/ops/list.md +9 -0
- package/plugin/skills/muggle-preferences/ops/reset.md +6 -0
- package/plugin/skills/muggle-preferences/ops/set.md +10 -0
- package/plugin/skills/muggle-preferences/preference-gates/README.md +1 -1
- package/plugin/skills/muggle-preferences/preference-gates/autoCleanup.md +11 -0
- package/plugin/skills/muggle-preferences/preference-gates/autoCreatePR.md +13 -0
- package/plugin/skills/muggle-preferences/preference-gates/autoDetectChanges.md +3 -3
- package/plugin/skills/muggle-preferences/preference-gates/autoE2ETest.md +11 -0
- package/plugin/skills/muggle-preferences/preference-gates/autoLogin.md +1 -1
- package/plugin/skills/muggle-preferences/preference-gates/autoPublishLocalResults.md +2 -2
- package/plugin/skills/muggle-preferences/preference-gates/autoRebase.md +11 -0
- package/plugin/skills/muggle-preferences/preference-gates/autoSelectLocalHost.md +19 -0
- package/plugin/skills/muggle-preferences/preference-gates/autoSelectProject.md +1 -2
- package/plugin/skills/muggle-preferences/preference-gates/autoUseWorktree.md +11 -0
- package/plugin/skills/muggle-preferences/preference-gates/checkForUpdates.md +4 -4
- package/plugin/skills/muggle-preferences/preference-gates/defaultExecutionMode.md +1 -1
- package/plugin/skills/muggle-preferences/preference-gates/openTestResultsAfterRun.md +1 -1
- package/plugin/skills/muggle-preferences/preference-gates/postPRVisualWalkthrough.md +8 -25
- package/plugin/skills/muggle-preferences/preference-gates/showElectronBrowser.md +1 -1
- package/plugin/skills/muggle-preferences/preference-gates/suggestRelatedTestCases.md +4 -4
- package/plugin/skills/muggle-preferences/preference-gates/suggestRelatedUseCases.md +4 -4
- package/plugin/skills/muggle-repair/SKILL.md +4 -2
- package/plugin/skills/muggle-status/SKILL.md +8 -5
- package/plugin/skills/muggle-test/SKILL.md +110 -86
- package/plugin/skills/muggle-test-feature-local/SKILL.md +73 -66
- package/plugin/skills/muggle-test-import/SKILL.md +28 -23
- package/plugin/skills/muggle-test-prepare/SKILL.md +79 -18
- package/plugin/skills/muggle-test-regenerate-missing/SKILL.md +34 -18
- package/plugin/skills/muggle-test-regenerate-missing/evals/evals.json +9 -9
- package/plugin/skills/muggle-upgrade/SKILL.md +4 -2
- package/plugin/skills/muggle-works-npm-release/SKILL.md +6 -4
- package/plugin/skills/mupgrade/SKILL.md +8 -0
- package/scripts/postinstall.mjs +11 -1
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# `autoSelectLocalHost`
|
|
2
|
+
|
|
3
|
+
Reuse the saved local dev server URL for this repo, or pick one each run. Substitute `{lastHost}` (the URL used in the previous run for this repo — omit the option entirely when no cache exists) and `{suggestedHost}` (auto-detected from running ports, e.g. `http://localhost:3000`).
|
|
4
|
+
|
|
5
|
+
Cache lives at `<cwd>/.muggle-ai/last-host.json`. The calling skill **always** updates the cache after the user picks/types a URL — independent of Picker 2 — so `Use {lastHost}` reflects the most recent run.
|
|
6
|
+
|
|
7
|
+
**Picker 1** — header `Local server`, question `"Which local URL should the test target?"`
|
|
8
|
+
- `Use {lastHost}` — `From your last run in this repo.` → reuse cached URL. *Skip this option when no cache exists.*
|
|
9
|
+
- `Use {suggestedHost}` — `Detected from a port that's running.` → use the suggestion
|
|
10
|
+
- `Type a URL` — `Paste any URL.` → calling skill prompts for free-text input
|
|
11
|
+
|
|
12
|
+
**Picker 2 — overrides shared template.** Fires after the user picks a URL.
|
|
13
|
+
- Header `Remember this URL?`, question `"Always use {chosenHost} for this repo from now on, without asking?"`
|
|
14
|
+
- `Yes, always` (sub: `You can change this later in muggle preferences.`) → `muggle-local-preferences-set` (`autoSelectLocalHost=always`, global). The cache is already up to date.
|
|
15
|
+
- `Just this once` (sub: `I'll ask again next time.`) → don't save the preference. The cache still updates.
|
|
16
|
+
|
|
17
|
+
**Silent action**
|
|
18
|
+
- `always` (cache used) → `Using saved local URL {lastHost}`
|
|
19
|
+
- `never` → no footer; the picker is the visible step.
|
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
# `autoSelectProject`
|
|
2
2
|
|
|
3
|
-
Reuse the
|
|
4
|
-
or pick from the list. Substitute `{projectName}`.
|
|
3
|
+
Reuse the saved project for this repo, or pick from the list. Substitute `{projectName}`.
|
|
5
4
|
|
|
6
5
|
Picker 1 *is* the project list (rendered by the calling skill — format and
|
|
7
6
|
tail options like "Show full list" / "Create new project" are skill-defined).
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# `autoUseWorktree`
|
|
2
|
+
|
|
3
|
+
Create a worktree for the change, or work in the current checkout.
|
|
4
|
+
|
|
5
|
+
**Picker 1** — header `Worktree`, question `"Create a git worktree for this change, or work in the current checkout?"`
|
|
6
|
+
- `Create a worktree` — `Isolate the change so your current checkout stays untouched.` → `always`
|
|
7
|
+
- `Use current checkout` — `Edit in place.` → `never`
|
|
8
|
+
|
|
9
|
+
**Silent action**
|
|
10
|
+
- `always` → `Creating a worktree for this change`
|
|
11
|
+
- `never` → `Working in the current checkout`
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# `checkForUpdates`
|
|
2
2
|
|
|
3
|
-
Check npm for a newer Muggle version at session start.
|
|
3
|
+
Check npm for a newer Muggle Test version at session start.
|
|
4
4
|
|
|
5
|
-
**Picker 1** — header `Update check`, question `"Check
|
|
6
|
-
- `
|
|
7
|
-
- `
|
|
5
|
+
**Picker 1** — header `Update check`, question `"Check for a newer Muggle Test version (one quick network call)?"`
|
|
6
|
+
- `Check now` — `Flags if you're behind.` → `always`
|
|
7
|
+
- `Skip check` — `Saves a network call at session start.` → `never`
|
|
8
8
|
|
|
9
9
|
**Silent action**
|
|
10
10
|
- `always` → `Checked for updates`
|
|
@@ -4,7 +4,7 @@ Default place to run tests when the user's request is ambiguous.
|
|
|
4
4
|
|
|
5
5
|
**Picker 1** — header `Where to run tests?`, question `"On your computer or in the cloud?"`
|
|
6
6
|
- `On my computer` — `Real browser on localhost. Faster feedback while developing.` → `local`
|
|
7
|
-
- `In the cloud` — `Muggle's cloud runs against a preview/staging URL.` → `remote`
|
|
7
|
+
- `In the cloud` — `Muggle Test's cloud runs against a preview/staging URL.` → `remote`
|
|
8
8
|
|
|
9
9
|
If the user's intent is already clear (e.g. "test on staging"), skip Picker 1
|
|
10
10
|
— confirm with `"Yes, proceed in <mode>"` / `"Switch to <other mode>"` and
|
|
@@ -1,30 +1,13 @@
|
|
|
1
1
|
# `postPRVisualWalkthrough`
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Preference gate. Controls whether the cycle posts the visual walkthrough comment (per-test-case dashboard links + step-by-step screenshots) to the PR after E2E results are available, or stops at "results ready". Fires once per E2E result set when a PR exists for the working branch (if it doesn't, callers fire [`autoCreatePR`](autoCreatePR.md) separately). Substitute `{prNumber}`.
|
|
4
4
|
|
|
5
|
-
**
|
|
6
|
-
|
|
5
|
+
**Picker 1** — header `Post to PR`, question `"Post the visual walkthrough to PR #{prNumber}?"`
|
|
6
|
+
- `Post to #{prNumber}` — `Comment with screenshots is added to the PR.` → `always`
|
|
7
|
+
- `Ask me each time` — `Decide per run.` → `ask`
|
|
8
|
+
- `Skip` — `Walkthrough is not posted.` → `never`
|
|
7
9
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
**Picker 1** — header `Share with the team`, question `"Post a visual walkthrough to PR #{prNumber} ({prTitle})?"`
|
|
11
|
-
- `Yes, post to #{prNumber}` — `Reviewers see clickable per-test screenshots and dashboard links.` → `always`
|
|
12
|
-
- `Skip` — `Keep it off the PR — you can post later from the dashboard.` → `never`
|
|
13
|
-
|
|
14
|
-
**Silent action (Case A)**
|
|
10
|
+
**Silent action**
|
|
15
11
|
- `always` → `Posting walkthrough to PR #{prNumber}`
|
|
16
|
-
- `
|
|
17
|
-
|
|
18
|
-
## Case B — no open PR
|
|
19
|
-
|
|
20
|
-
Situational fork — saved value is *not* updated from this picker.
|
|
21
|
-
|
|
22
|
-
**Picker 1** — header `No PR yet`, question `"This branch has no open PR. Create one and post the walkthrough, or skip?"`
|
|
23
|
-
- `Create a PR and post` — `I'll open a PR for this branch, then attach the walkthrough.` → run PR-creation flow (calling skill's responsibility), then post.
|
|
24
|
-
- `Skip` — `Skip the walkthrough this time — you can post later from the dashboard.` → continue.
|
|
25
|
-
|
|
26
|
-
**Picker 2** — skipped entirely.
|
|
27
|
-
|
|
28
|
-
**Silent action (Case B)** — when saved gate is `always` or `never` but no PR exists:
|
|
29
|
-
- `always` → fall through to Case B Picker 1 (don't auto-create silently). Print: `(`postPRVisualWalkthrough = always`, but this branch has no PR — asking what to do.)`
|
|
30
|
-
- `never` → `Skipping PR walkthrough — no open PR for this branch`
|
|
12
|
+
- `ask` → `Asking about PR walkthrough`
|
|
13
|
+
- `never` → `Skipping PR walkthrough`
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# `showElectronBrowser`
|
|
2
2
|
|
|
3
|
-
Show the
|
|
3
|
+
Show the browser during local tests, or run hidden.
|
|
4
4
|
|
|
5
5
|
**Picker 1** — header `Browser window`, question `"Show the test browser as it runs?"`
|
|
6
6
|
- `Show it` — `Watch the test live — useful when something's failing.` → `always` (omit `showUi`)
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# `suggestRelatedTestCases`
|
|
2
2
|
|
|
3
|
-
After creating/running a test case,
|
|
3
|
+
After creating/running a test case, show related ones already attached to the use case.
|
|
4
4
|
|
|
5
|
-
**Picker 1** — header `Related test cases`, question `"
|
|
6
|
-
- `
|
|
7
|
-
- `
|
|
5
|
+
**Picker 1** — header `Related test cases`, question `"Show related test cases already attached to this use case?"`
|
|
6
|
+
- `Show related` — `Catch test cases your import or change might have missed.` → `always`
|
|
7
|
+
- `Skip` — `You can ask later if you want them.` → `never`
|
|
8
8
|
|
|
9
9
|
**Silent action**
|
|
10
10
|
- `always` → `Showing related test cases below`
|
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# `suggestRelatedUseCases`
|
|
2
2
|
|
|
3
|
-
After creating/running a use case,
|
|
3
|
+
After creating/running a use case, show related ones already in the project.
|
|
4
4
|
|
|
5
|
-
**Picker 1** — header `Related use cases`, question `"
|
|
6
|
-
- `
|
|
7
|
-
- `
|
|
5
|
+
**Picker 1** — header `Related use cases`, question `"Show related use cases already in this project?"`
|
|
6
|
+
- `Show related` — `Catch use cases your import or change might have missed.` → `always`
|
|
7
|
+
- `Skip` — `You can ask later if you want them.` → `never`
|
|
8
8
|
|
|
9
9
|
**Silent action**
|
|
10
10
|
- `always` → `Showing related use cases below`
|
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: muggle-repair
|
|
3
|
-
description: Diagnose and fix a broken Muggle AI installation. Use when user types muggle repair or asks to fix Muggle setup.
|
|
3
|
+
description: Diagnose and fix a broken Muggle AI installation. Use when user types muggle repair or asks to fix Muggle Test setup.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Muggle Repair
|
|
6
|
+
# Muggle Test Repair
|
|
7
|
+
|
|
8
|
+
> Telemetry first step: see [`_shared/telemetry-emit.md`](../_shared/telemetry-emit.md). Use `skillName: "muggle-repair"`.
|
|
7
9
|
|
|
8
10
|
Automatically diagnose and fix broken components.
|
|
9
11
|
|
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: muggle-status
|
|
3
|
-
description: Check health of the Muggle AI installation. Use when user types muggle status, asks for Muggle health, MCP health, or auth validity.
|
|
3
|
+
description: Check health of the Muggle AI installation. Use when user types muggle status, asks for Muggle Test health, MCP health, or auth validity.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
# Muggle Status
|
|
6
|
+
# Muggle Test Status
|
|
7
|
+
|
|
8
|
+
> Telemetry first step: see [`_shared/telemetry-emit.md`](../_shared/telemetry-emit.md). Use `skillName: "muggle-status"`.
|
|
7
9
|
|
|
8
10
|
Run a full health check and report results.
|
|
9
11
|
|
|
@@ -13,7 +15,7 @@ Gates run per `preference-gates/README.md`.
|
|
|
13
15
|
|
|
14
16
|
| Preference | Step | Decision it gates |
|
|
15
17
|
|------------|------|-------------------|
|
|
16
|
-
| `checkForUpdates` | Check 4 | Check for newer Muggle version |
|
|
18
|
+
| `checkForUpdates` | Check 4 | Check for newer Muggle Test version |
|
|
17
19
|
|
|
18
20
|
## Checks
|
|
19
21
|
|
|
@@ -24,8 +26,9 @@ Gates run per `preference-gates/README.md`.
|
|
|
24
26
|
3. **Authentication** — call `muggle-remote-auth-status`. Report whether credentials are valid and when they expire.
|
|
25
27
|
|
|
26
28
|
4. **CLI version** — gate `checkForUpdates` (per `preference-gates/README.md`):
|
|
27
|
-
-
|
|
28
|
-
-
|
|
29
|
+
- `always` → run the check below.
|
|
30
|
+
- `never` → render the row as `[skip] check disabled by preference`.
|
|
31
|
+
- `ask` → run Picker 1 from `preference-gates/checkForUpdates.md` via `AskUserQuestion`; map the answer back to one of the actions above.
|
|
29
32
|
|
|
30
33
|
When the check runs: capture installed (`muggle --version`) and latest (`npm view @muggleai/works version`). Compare with `sort -V`; flag as out-of-date only when latest is strictly greater.
|
|
31
34
|
|
|
@@ -1,27 +1,29 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: muggle-test
|
|
3
|
-
description: "Run change-driven E2E acceptance testing using Muggle AI — detects local code changes, maps them to use cases, and generates test scripts either locally (real browser on localhost) or remotely (cloud execution on a preview/staging URL). Publishes results to Muggle dashboard, opens them in the browser, and posts E2E acceptance summaries with screenshots to the PR. Use this skill whenever the user wants to test their changes, run E2E acceptance tests on recent work, validate what they've been working on, or check if their code changes broke anything. Triggers on: 'test my changes', 'run tests on my changes', 'acceptance test my work', 'check my changes', 'validate my changes', 'test before I push', 'make sure my changes work', 'regression test my changes', 'test on preview', 'test on staging'. This is the go-to skill for change-driven E2E acceptance testing — it handles everything from change detection to test execution to result reporting."
|
|
3
|
+
description: "Run change-driven E2E acceptance testing using Muggle AI — detects local code changes, maps them to use cases, and generates test scripts either locally (real browser on localhost) or remotely (cloud execution on a preview/staging URL). Publishes results to Muggle Test dashboard, opens them in the browser, and posts E2E acceptance summaries with screenshots to the PR. Use this skill whenever the user wants to test their changes, run E2E acceptance tests on recent work, validate what they've been working on, or check if their code changes broke anything. Triggers on: 'test my changes', 'run tests on my changes', 'acceptance test my work', 'check my changes', 'validate my changes', 'test before I push', 'make sure my changes work', 'regression test my changes', 'test on preview', 'test on staging'. This is the go-to skill for change-driven E2E acceptance testing — it handles everything from change detection to test execution to result reporting."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Muggle Test — Change-Driven E2E Acceptance Router
|
|
7
7
|
|
|
8
|
+
> Telemetry first step: see [`_shared/telemetry-emit.md`](../_shared/telemetry-emit.md). Use `skillName: "muggle-test"`.
|
|
9
|
+
|
|
8
10
|
A router skill that detects code changes, resolves impacted test cases, executes them locally or remotely, publishes results to the Muggle AI dashboard, and posts E2E acceptance summaries to the PR. The user can invoke this at any moment, in any state.
|
|
9
11
|
|
|
10
12
|
## UX Guidelines — Minimize Typing
|
|
11
13
|
|
|
12
|
-
**Every selection-based question MUST use the `
|
|
14
|
+
**Every selection-based question MUST use the `AskUserQuestion` tool** (or the platform's equivalent structured selection tool). Never ask the user to "reply with a number" in a plain text message — always present clickable options.
|
|
13
15
|
|
|
14
|
-
- **Selections** (project, use case, test case, mode, approval): Use `
|
|
15
|
-
- **Multi-select** (use cases, test cases): Use `
|
|
16
|
+
- **Selections** (project, use case, test case, mode, approval): Use `AskUserQuestion` with labeled options the user can click.
|
|
17
|
+
- **Multi-select** (use cases, test cases): Use `AskUserQuestion` with `allow_multiple: true`.
|
|
16
18
|
- **Free-text inputs** (URLs, descriptions): Only use plain text prompts when there is no finite set of options. Even then, offer a detected/default value when possible.
|
|
17
|
-
- **Batch related questions**: If two questions are independent, present them together in a single `
|
|
19
|
+
- **Batch related questions**: If two questions are independent, present them together in a single `AskUserQuestion` call rather than asking sequentially.
|
|
18
20
|
- **Parallelize job-creation calls**: Whenever you're kicking off N independent cloud jobs — creating multiple use cases, generating/creating multiple test cases, fetching details for multiple test cases, starting multiple remote workflows, publishing multiple local runs, or fetching per-step screenshots for multiple runs — issue all N tool calls in a single message so they run in parallel. Never loop them sequentially unless there is a real ordering constraint (e.g. a single local Electron browser that can only run one test at a time).
|
|
19
21
|
|
|
20
22
|
## Test Case Design: One Atomic Behavior Per Test Case
|
|
21
23
|
|
|
22
24
|
Every test case verifies exactly **one** user-observable behavior. Never bundle multiple concerns, sequential flows, or bootstrap/setup into a single test case — even if you think it would be "cleaner" or "more efficient."
|
|
23
25
|
|
|
24
|
-
**Ordering, dependencies, and bootstrap are Muggle's service responsibility, not yours.** Muggle's cloud handles test case dependencies, prerequisite state, and execution ordering. Your job is to describe the *atomic behavior to verify* — never the flow that gets there.
|
|
26
|
+
**Ordering, dependencies, and bootstrap are Muggle Test's service responsibility, not yours.** Muggle Test's cloud handles test case dependencies, prerequisite state, and execution ordering. Your job is to describe the *atomic behavior to verify* — never the flow that gets there.
|
|
25
27
|
|
|
26
28
|
- ❌ Wrong: one test case that "signs up, logs in, navigates to the detail modal, verifies icon stacking, verifies tab order, verifies history format, and verifies reference layout."
|
|
27
29
|
- ✅ Right: four separate test cases — one per verifiable behavior — each with instruction text like "Verify the detail modal shows stacked pair of icons per card" with **no** signup / login / navigation / setup language.
|
|
@@ -39,12 +41,14 @@ Gates run per `preference-gates/README.md`.
|
|
|
39
41
|
| Preference | Step | Decision it gates |
|
|
40
42
|
|------------|------|-------------------|
|
|
41
43
|
| `autoLogin` | 3 | Reuse saved credentials when auth is required |
|
|
42
|
-
| `autoSelectProject` | 4 | Reuse last-used Muggle project for this repo |
|
|
44
|
+
| `autoSelectProject` | 4 | Reuse last-used Muggle Test project for this repo |
|
|
45
|
+
| `autoSelectLocalHost` | 7A | Reuse last-used local dev server URL for this repo |
|
|
43
46
|
| `autoDetectChanges` | 2 | Scan local git changes and map to affected test cases |
|
|
44
47
|
| `defaultExecutionMode` | 1 | Default to local or remote test execution |
|
|
45
|
-
| `autoPublishLocalResults` | 7A | Upload local results to Muggle cloud after run |
|
|
48
|
+
| `autoPublishLocalResults` | 7A | Upload local results to Muggle Test cloud after run |
|
|
46
49
|
| `showElectronBrowser` | 7A | Show the Electron browser window during local test execution (vs. run headless) |
|
|
47
50
|
| `postPRVisualWalkthrough` | 9 | Post visual walkthrough to PR after results are available |
|
|
51
|
+
| `autoCreatePR` | 9 (if no PR) | Auto-create the PR when posting the walkthrough has no PR to target |
|
|
48
52
|
|
|
49
53
|
## Step 1: Confirm Scope of Work (Always First)
|
|
50
54
|
|
|
@@ -58,7 +62,7 @@ Parse the user's query and explicitly confirm their expectation. There are exact
|
|
|
58
62
|
Signs the user wants this: mentions "localhost", "local", "my machine", "dev server", "my changes locally", or just "test my changes" in a repo context.
|
|
59
63
|
|
|
60
64
|
### Mode B: Remote Test Generation
|
|
61
|
-
> Ask Muggle's cloud to generate test scripts against a **preview/staging URL**.
|
|
65
|
+
> Ask Muggle Test's cloud to generate test scripts against a **preview/staging URL**.
|
|
62
66
|
>
|
|
63
67
|
> Execution tool: `muggle-remote-workflow-start-test-script-generation`
|
|
64
68
|
|
|
@@ -77,8 +81,9 @@ Only proceed after selection.
|
|
|
77
81
|
## Step 2: Detect Local Changes (gated by `autoDetectChanges`)
|
|
78
82
|
|
|
79
83
|
Gate `autoDetectChanges` (per `preference-gates/README.md`):
|
|
80
|
-
-
|
|
81
|
-
-
|
|
84
|
+
- `always` → run the scan and proceed to analysis below.
|
|
85
|
+
- `never` → ask "What would you like to test?" then jump to Step 3.
|
|
86
|
+
- `ask` → run Picker 1 from `preference-gates/autoDetectChanges.md` via `AskUserQuestion`; map the answer back to one of the actions above.
|
|
82
87
|
|
|
83
88
|
### Analysis (when scan is enabled)
|
|
84
89
|
|
|
@@ -101,8 +106,9 @@ If no changes detected (clean tree), tell the user and ask what they want to tes
|
|
|
101
106
|
|
|
102
107
|
1. Call `muggle-remote-auth-status`
|
|
103
108
|
2. If **authenticated and not expired** → gate `autoLogin` (per `preference-gates/README.md`):
|
|
104
|
-
-
|
|
105
|
-
-
|
|
109
|
+
- `always` → reuse saved session.
|
|
110
|
+
- `never` → `muggle-remote-auth-login` with `forceNewSession: true`, then `muggle-remote-auth-poll`.
|
|
111
|
+
- `ask` → run Picker 1 from `preference-gates/autoLogin.md` via `AskUserQuestion`; map the answer back to one of the actions above.
|
|
106
112
|
3. If **not authenticated or expired** → call `muggle-remote-auth-login`
|
|
107
113
|
4. If login pending → call `muggle-remote-auth-poll`
|
|
108
114
|
|
|
@@ -112,9 +118,9 @@ If auth fails repeatedly, suggest: `muggle logout && muggle login` from terminal
|
|
|
112
118
|
|
|
113
119
|
A **project** is where all your test results, use cases, and test scripts are grouped on the Muggle AI dashboard. Pick the project that matches what you're working on.
|
|
114
120
|
|
|
115
|
-
The per-repo cache lives at `<cwd>/.muggle-ai/last-project.json` (managed via the `muggle-local-last-project-get` / `muggle-local-last-project-set` MCP tools). Look for the `Muggle Last Project: id=… url=… name="…"` line in session context — if present, that's this repo's cached pick.
|
|
121
|
+
The per-repo cache lives at `<cwd>/.muggle-ai/last-project.json` (managed via the `muggle-local-last-project-get` / `muggle-local-last-project-set` MCP tools). Look for the `Muggle Test Last Project: id=… url=… name="…"` line in session context — if present, that's this repo's cached pick.
|
|
116
122
|
|
|
117
|
-
Gate `autoSelectProject` (per `preference-gates/README.md`). Cache: `Muggle Last Project` session line.
|
|
123
|
+
Gate `autoSelectProject` (per `preference-gates/README.md`). Cache: `Muggle Test Last Project` session line.
|
|
118
124
|
- `always` + cache → use cached `projectId`, skip to Step 5. No cache → fall through to `ask`.
|
|
119
125
|
- `never` → full project list; skip Picker 2.
|
|
120
126
|
- `ask` → project list picker (see gate file for spec + Picker 2 override). Skip Picker 2 if "Create new project".
|
|
@@ -143,7 +149,7 @@ If nothing looks like a confident match, fall back to asking the user which use
|
|
|
143
149
|
|
|
144
150
|
### 5c: Present the shortlist for confirmation
|
|
145
151
|
|
|
146
|
-
Use `
|
|
152
|
+
Use `AskUserQuestion` with `allow_multiple: true`:
|
|
147
153
|
|
|
148
154
|
Prompt: "These use cases look most relevant to your changes — confirm which to test:"
|
|
149
155
|
|
|
@@ -152,7 +158,7 @@ Prompt: "These use cases look most relevant to your changes — confirm which to
|
|
|
152
158
|
- Include "Create new use case" at the end
|
|
153
159
|
|
|
154
160
|
### 5d: If user picks "Pick a different use case"
|
|
155
|
-
Re-present the full list from 5a via `
|
|
161
|
+
Re-present the full list from 5a via `AskUserQuestion` with `allow_multiple: true`, then continue.
|
|
156
162
|
|
|
157
163
|
### 5e: If user chooses "Create new use case"
|
|
158
164
|
1. Ask the user to describe the use case(s) in plain English — they may want more than one
|
|
@@ -176,7 +182,7 @@ If nothing looks like a confident match, fall back to offering to run all test c
|
|
|
176
182
|
|
|
177
183
|
### 6c: Present the shortlist for confirmation
|
|
178
184
|
|
|
179
|
-
Use `
|
|
185
|
+
Use `AskUserQuestion` with `allow_multiple: true`:
|
|
180
186
|
|
|
181
187
|
Prompt: "These test cases look most relevant — confirm which to run:"
|
|
182
188
|
|
|
@@ -195,27 +201,46 @@ Prompt: "These test cases look most relevant — confirm which to run:"
|
|
|
195
201
|
|
|
196
202
|
### 6e: Confirm final selection
|
|
197
203
|
|
|
198
|
-
Use `
|
|
204
|
+
Use `AskUserQuestion` to confirm: "You selected [N] test case(s): [list titles]. Ready to proceed?"
|
|
199
205
|
- Option 1: "Yes, run them"
|
|
200
206
|
- Option 2: "No, let me re-select"
|
|
201
207
|
|
|
202
208
|
Wait for user confirmation before moving to execution.
|
|
203
209
|
|
|
210
|
+
### 6f: Classify execution mode per test case (replay vs regen)
|
|
211
|
+
|
|
212
|
+
For each selected test case, decide whether the run should be a **replay** of an existing script or a fresh **regen**, using the rules in [`_shared/failure-mode-handling.md`](../_shared/failure-mode-handling.md) section A. Inputs: the change summary from Step 2, the test case body, and the result of `muggle-remote-test-script-list` for that test case (last passing timestamp + whether any replayable script exists).
|
|
213
|
+
|
|
214
|
+
Per test case, fire one `muggle-local-telemetry-event-emit` with `eventType: "pre-execution-classification"` capturing the picked mode, the rule that fired, and the matched changed-file paths.
|
|
215
|
+
|
|
216
|
+
Then show the per-case decision in one `AskUserQuestion`:
|
|
217
|
+
|
|
218
|
+
> "Here's how I plan to run each test case — replay reuses the saved script, regen rebuilds it from scratch:
|
|
219
|
+
> - [REPLAY] Login with valid creds — selectors look unchanged
|
|
220
|
+
> - [REGEN] Sign up with valid email — last passed > 30 days ago
|
|
221
|
+
> - [REGEN] Add to cart — `app/cart/page.tsx` changed (UI/markup)"
|
|
222
|
+
>
|
|
223
|
+
> Options: "Looks good — proceed", "Override one or more", "Cancel"
|
|
224
|
+
|
|
225
|
+
If the user picks "Override one or more", let them flip the mode for any test case via a second multi-select `AskUserQuestion`. Emit a follow-up `pre-execution-classification` event with `userAction` set whenever the user overrides.
|
|
226
|
+
|
|
204
227
|
## Step 7A: Execute — Local Mode
|
|
205
228
|
|
|
206
|
-
### Pre-flight question — Local URL
|
|
229
|
+
### Pre-flight question — Local URL (gated by `autoSelectLocalHost`)
|
|
207
230
|
|
|
208
|
-
|
|
209
|
-
-
|
|
210
|
-
-
|
|
231
|
+
Skill responsibilities (the rest is in `preference-gates/autoSelectLocalHost.md`):
|
|
232
|
+
- **Read the cache**: `Muggle Test Last Host: <url>` session-context line, or `muggle-local-last-host-get`. Pass as `{lastHost}` substitution.
|
|
233
|
+
- **Auto-detect a suggested URL**: `lsof -iTCP -sTCP:LISTEN -nP | grep -E ':(3000|3001|4200|5173|8080)'`. Pass as `{suggestedHost}`.
|
|
234
|
+
- **Save the cache**: call `muggle-local-last-host-set` after the user picks (the gate file requires this on every pick).
|
|
211
235
|
|
|
212
|
-
|
|
236
|
+
Gate `autoSelectLocalHost` per `preference-gates/README.md` + `preference-gates/autoSelectLocalHost.md`.
|
|
213
237
|
|
|
214
238
|
### Pre-flight visibility (gated by `showElectronBrowser`)
|
|
215
239
|
|
|
216
240
|
Gate `showElectronBrowser` (per `preference-gates/README.md`). Resolve once; apply same `showUi` to every test case.
|
|
217
|
-
-
|
|
218
|
-
-
|
|
241
|
+
- `always` → omit `showUi` (defaults visible).
|
|
242
|
+
- `never` → pass `showUi: false`.
|
|
243
|
+
- `ask` → run Picker 1 from `preference-gates/showElectronBrowser.md` via `AskUserQuestion`; map the answer back to one of the actions above.
|
|
219
244
|
|
|
220
245
|
### Fetch test case details (in parallel)
|
|
221
246
|
|
|
@@ -234,16 +259,25 @@ If none of the above apply, omit `freshSession` (defaults to `false`, preserving
|
|
|
234
259
|
|
|
235
260
|
### Run sequentially (Electron constraint)
|
|
236
261
|
|
|
237
|
-
Execution itself **must** be sequential because there is only one local Electron browser. For each test case, in order:
|
|
262
|
+
Execution itself **must** be sequential because there is only one local Electron browser. For each test case, in the order chosen, branch on the mode picked in Step 6f:
|
|
238
263
|
|
|
264
|
+
**Regen-mode test case:**
|
|
239
265
|
1. Call `muggle-local-execute-test-generation`:
|
|
240
266
|
- `testCase`: Full test case object from the parallel fetch above
|
|
241
267
|
- `localUrl`: User's local URL from the pre-flight question
|
|
242
268
|
- `showUi`: from the `showElectronBrowser` resolution — omit (default visible) for `always`, pass `false` for `never`
|
|
243
269
|
- `freshSession`: `true` if the test case requires a clean browser state (see above), omit otherwise
|
|
244
|
-
2. Store the returned `runId`
|
|
270
|
+
2. Store the returned `runId` and tag the result `mode: "regen"`.
|
|
271
|
+
|
|
272
|
+
**Replay-mode test case:**
|
|
273
|
+
1. Fetch the action script: `muggle-remote-test-script-get` (latest replayable script id) → `muggle-remote-action-script-get` (full `actionScript` — use as-is, never edit). For batches, fan these calls out in parallel before the sequential execution loop begins.
|
|
274
|
+
2. Call `muggle-local-execute-replay`:
|
|
275
|
+
- `testScript`: from `muggle-remote-test-script-get`
|
|
276
|
+
- `actionScript`: from `muggle-remote-action-script-get`
|
|
277
|
+
- `localUrl`, `showUi`, `freshSession`: same resolution as regen
|
|
278
|
+
3. Store the returned `runId` and tag the result `mode: "replay"`.
|
|
245
279
|
|
|
246
|
-
If a
|
|
280
|
+
If a run fails, log it and continue to the next — do not abort the batch. Failures are routed through Step 7C's post-failure handler after the batch completes.
|
|
247
281
|
|
|
248
282
|
### Collect results (in parallel)
|
|
249
283
|
|
|
@@ -252,8 +286,9 @@ For every `runId`, issue all `muggle-local-run-result-get` calls in parallel. Ex
|
|
|
252
286
|
### Publish each run to cloud (gated by `autoPublishLocalResults`)
|
|
253
287
|
|
|
254
288
|
Gate `autoPublishLocalResults` (per `preference-gates/README.md`):
|
|
255
|
-
-
|
|
256
|
-
-
|
|
289
|
+
- `always` → proceed to publish logic below.
|
|
290
|
+
- `never` → skip to report summary; tell user Steps 8/9 and per-step screenshots are unavailable without publishing.
|
|
291
|
+
- `ask` → run Picker 1 from `preference-gates/autoPublishLocalResults.md` via `AskUserQuestion`; map the answer back to one of the actions above.
|
|
257
292
|
|
|
258
293
|
### Publish logic (when publishing is enabled)
|
|
259
294
|
|
|
@@ -293,7 +328,9 @@ Issue all `muggle-remote-test-case-get` calls in parallel (single message, multi
|
|
|
293
328
|
|
|
294
329
|
### Trigger remote workflows (in parallel)
|
|
295
330
|
|
|
296
|
-
|
|
331
|
+
Branch each test case on the mode chosen in Step 6f, then issue **all** workflow-start calls in parallel — never loop them sequentially. Mix regen and replay starts in the same parallel batch.
|
|
332
|
+
|
|
333
|
+
**Regen-mode test case** — `muggle-remote-workflow-start-test-script-generation`:
|
|
297
334
|
|
|
298
335
|
- `projectId`: The project ID
|
|
299
336
|
- `useCaseId`: The use case ID
|
|
@@ -305,7 +342,9 @@ Once details are in hand, issue all `muggle-remote-workflow-start-test-script-ge
|
|
|
305
342
|
- `instructions`: From the test case
|
|
306
343
|
- `expectedResult`: From the test case
|
|
307
344
|
|
|
308
|
-
|
|
345
|
+
**Replay-mode test case** — `muggle-remote-workflow-start-test-script-replay` against the latest replayable script for that test case (resolve via `muggle-remote-test-script-list` if not already in hand from Step 6f). Tag results with `mode: "replay"` so Step 7C can route failures correctly.
|
|
346
|
+
|
|
347
|
+
Store each returned workflow runtime ID along with its mode tag.
|
|
309
348
|
|
|
310
349
|
### Monitor and report (in parallel)
|
|
311
350
|
|
|
@@ -319,6 +358,21 @@ Login with invalid creds COMPLETED rt-def456
|
|
|
319
358
|
Checkout flow QUEUED rt-ghi789
|
|
320
359
|
```
|
|
321
360
|
|
|
361
|
+
## Step 7C: Route failures through the failure-mode handler
|
|
362
|
+
|
|
363
|
+
For every run with `status: "failed"` (or any non-passing terminal state) from 7A or 7B, follow [`_shared/failure-mode-handling.md`](../_shared/failure-mode-handling.md):
|
|
364
|
+
|
|
365
|
+
- **Replay-mode failures** — section B (buckets: `infra` / `stale-script` / `product-defect`).
|
|
366
|
+
- **Regen-mode failures** — section C (buckets: `transient` / `infra` / `agent-course` / `product-uxux`).
|
|
367
|
+
|
|
368
|
+
For each failed run:
|
|
369
|
+
1. Read the run with `muggle-local-run-result-get` (local) or `muggle-remote-wf-get-ts-gen-latest-run` / `muggle-remote-wf-get-ts-replay-latest-run` (remote) and extract signals per the heuristics in the shared doc.
|
|
370
|
+
2. Emit `replay-failure-classified` or `regen-failure-classified` via `muggle-local-telemetry-event-emit` **before** asking the user.
|
|
371
|
+
3. Present the recommended action via `AskUserQuestion` along with the alternatives the shared doc lists for that bucket.
|
|
372
|
+
4. After the user picks, emit the matching `*-resolved` event with `userAction` set to what they chose.
|
|
373
|
+
|
|
374
|
+
Process failures one at a time so the user isn't drowning in pickers — but emit telemetry per failure regardless.
|
|
375
|
+
|
|
322
376
|
## Step 8: Open Results in Browser
|
|
323
377
|
|
|
324
378
|
After execution and publishing are complete, open the Muggle AI dashboard so the user can visually inspect results and screenshots.
|
|
@@ -346,59 +400,25 @@ Tell the user:
|
|
|
346
400
|
|
|
347
401
|
## Step 9: Offer to Post Visual Walkthrough to PR
|
|
348
402
|
|
|
349
|
-
After reporting results
|
|
350
|
-
|
|
351
|
-
### 9a: Gather per-step screenshots (required input for the shared skill)
|
|
352
|
-
|
|
353
|
-
The shared skill takes an **`E2eReport` JSON** that includes per-step screenshot URLs. You already have `projectId`, `testCaseId`, `runId`, `viewUrl`, and `status` from earlier steps — you still need the step-level data.
|
|
354
|
-
|
|
355
|
-
For the published runs from Step 7A, issue **all** `muggle-remote-test-script-get` calls in parallel (single message, multiple tool calls) — one per `testScriptId` returned by `muggle-local-publish-test-script`. Then, for each response:
|
|
356
|
-
|
|
357
|
-
1. Extract `steps[].operation.action` (description) and `steps[].operation.screenshotUrl` (cloud URL).
|
|
358
|
-
2. Build a `steps` array: `[{ stepIndex: 0, action: "...", screenshotUrl: "..." }, ...]`.
|
|
359
|
-
3. If the run failed, also capture `failureStepIndex`, `error`, and the local `artifactsDir` from `muggle-local-run-result-get`.
|
|
360
|
-
4. Populate `description` (test case title/description) and `useCaseName` (parent use case title) on each report entry — optional but strongly recommended; they drive the grouped overview and the per-test collapsible headers. Prefer values already in your conversation context from earlier steps (e.g. the test case you just created or selected, or the use case you confirmed); only call `muggle-remote-test-case-get` / `muggle-remote-use-case-get` for anything you don't already have.
|
|
361
|
-
|
|
362
|
-
Assemble the report:
|
|
363
|
-
|
|
364
|
-
```json
|
|
365
|
-
{
|
|
366
|
-
"projectId": "<projectId>",
|
|
367
|
-
"tests": [
|
|
368
|
-
{
|
|
369
|
-
"name": "<test case title>",
|
|
370
|
-
"description": "<one-line description of what this test verifies (optional but recommended)>",
|
|
371
|
-
"useCaseName": "<parent use case title (optional but recommended)>",
|
|
372
|
-
"testCaseId": "<id>",
|
|
373
|
-
"testScriptId": "<id>",
|
|
374
|
-
"runId": "<id>",
|
|
375
|
-
"viewUrl": "<publish response viewUrl>",
|
|
376
|
-
"status": "passed",
|
|
377
|
-
"steps": [{ "stepIndex": 0, "action": "...", "screenshotUrl": "..." }]
|
|
378
|
-
}
|
|
379
|
-
]
|
|
380
|
-
}
|
|
381
|
-
```
|
|
382
|
-
|
|
383
|
-
See the shared skill for the full schema (including the failed-test shape with `failureStepIndex` and `error`).
|
|
403
|
+
After reporting results:
|
|
384
404
|
|
|
385
|
-
|
|
405
|
+
1. Fire [`postPRVisualWalkthrough`](../muggle-preferences/preference-gates/postPRVisualWalkthrough.md). On skip → Step 10.
|
|
406
|
+
2. `gh pr view --json number,title,url 2>/dev/null` — find the PR.
|
|
407
|
+
3. If no PR: fire [`autoCreatePR`](../muggle-preferences/preference-gates/autoCreatePR.md). On skip → Step 10.
|
|
408
|
+
4. Assemble the `E2eReport` — see [`../muggle-pr-visual-walkthrough/e2e-report-assembly.md`](../muggle-pr-visual-walkthrough/e2e-report-assembly.md). Include all runs from Step 7A (passed and failed).
|
|
409
|
+
5. Invoke [`../muggle-pr-visual-walkthrough/SKILL.md`](../muggle-pr-visual-walkthrough/SKILL.md) Mode A with the `E2eReport`.
|
|
386
410
|
|
|
387
|
-
|
|
388
|
-
- **Case A (PR found)** — `always` → proceed to 9c; `never`/skip → stop.
|
|
389
|
-
- **Case B (no PR)** — always run Picker 1 regardless of saved value; "Create a PR and post" → create PR then proceed to 9c; "Skip" → stop.
|
|
411
|
+
## Step 10: Offer feedback on failures
|
|
390
412
|
|
|
391
|
-
|
|
413
|
+
After the report is complete, if **any** test in the run had a `failed` or unexpected status (or the user verbally flags something looked off), suggest the feedback skill:
|
|
392
414
|
|
|
393
|
-
|
|
415
|
+
> "Looks like `<N>` test(s) didn't go as expected. Want to leave feedback on what should've happened? It triggers regeneration on the affected scripts."
|
|
394
416
|
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
4. Post the overflow `comment` as a second comment (only if the CLI emitted one)
|
|
399
|
-
5. Confirm the PR URL to the user
|
|
417
|
+
Use `AskUserQuestion`:
|
|
418
|
+
- **Yes — give feedback** → invoke the `muggle-feedback` skill via the `Skill` tool. Pass the failed run's `runId` (local) or `testScriptId` (remote) as anchor context so the submit flow opens with the correct script already loaded.
|
|
419
|
+
- **No — skip**
|
|
400
420
|
|
|
401
|
-
This
|
|
421
|
+
This is a suggestion, not automatic invocation. Skip silently if every test passed cleanly.
|
|
402
422
|
|
|
403
423
|
## Tool Reference
|
|
404
424
|
|
|
@@ -415,10 +435,14 @@ This skill always uses **Mode A** (post to an existing PR); `muggle-do` is the o
|
|
|
415
435
|
| Test Case | `muggle-remote-test-case-generate-from-prompt` | Both |
|
|
416
436
|
| Test Case | `muggle-remote-test-case-create` | Both |
|
|
417
437
|
| Test Case | `muggle-remote-test-case-get` | Both |
|
|
418
|
-
| Execute | `muggle-local-execute-test-generation` | Local |
|
|
419
|
-
| Execute | `muggle-
|
|
438
|
+
| Execute (regen) | `muggle-local-execute-test-generation` | Local |
|
|
439
|
+
| Execute (replay) | `muggle-local-execute-replay` | Local |
|
|
440
|
+
| Replay action script fetch | `muggle-remote-test-script-get`, `muggle-remote-action-script-get` | Local replay |
|
|
441
|
+
| Execute (regen) | `muggle-remote-workflow-start-test-script-generation` | Remote |
|
|
442
|
+
| Execute (replay) | `muggle-remote-workflow-start-test-script-replay` | Remote |
|
|
443
|
+
| Failure-mode telemetry | `muggle-local-telemetry-event-emit` | Both |
|
|
420
444
|
| Results | `muggle-local-run-result-get` | Local |
|
|
421
|
-
| Results | `muggle-remote-wf-get-ts-gen-latest-run` | Remote |
|
|
445
|
+
| Results | `muggle-remote-wf-get-ts-gen-latest-run`, `muggle-remote-wf-get-ts-replay-latest-run` | Remote |
|
|
422
446
|
| Publish | `muggle-local-publish-test-script` | Local |
|
|
423
447
|
| Per-step screenshots (for walkthrough) | `muggle-remote-test-script-get` | Both |
|
|
424
448
|
| Browser | `open` (shell command) | Both |
|
|
@@ -427,13 +451,13 @@ This skill always uses **Mode A** (post to an existing PR); `muggle-do` is the o
|
|
|
427
451
|
## Guardrails
|
|
428
452
|
|
|
429
453
|
- **Always confirm intent first** — never assume local vs remote without asking
|
|
430
|
-
- **User MUST select project** — present clickable options via `
|
|
454
|
+
- **User MUST select project** — present clickable options via `AskUserQuestion`, wait for explicit choice, never auto-select
|
|
431
455
|
- **Best-effort shortlist use cases** — use the change summary to narrow the list to the most relevant 1–5 use cases and pre-check them; never dump every use case in the project on the user. Always leave an escape hatch to reveal the full list.
|
|
432
456
|
- **Best-effort shortlist test cases** — same idea: pre-check the test cases most relevant to the change summary; never enumerate every test case attached to a use case. Always leave an escape hatch to reveal the full list.
|
|
433
|
-
- **Use `
|
|
457
|
+
- **Use `AskUserQuestion` for every selection** — never ask the user to type a number; always present clickable options
|
|
434
458
|
- **Auto-detect localhost URL when possible**; only fall back to free-text when nothing is listening on a common port
|
|
435
459
|
- **Parallelize independent cloud jobs** — when creating N use cases, generating/creating N test cases, fetching N test case details, starting N remote workflows, polling N workflow runtimes, publishing N local runs, or fetching N per-step test scripts, issue all N calls in a single message so they fan out in parallel. The only tolerated sequential loop is local Electron execution (one browser, one test at a time). For use case creation specifically, use the native batch form of `muggle-remote-use-case-create-from-prompts` (all descriptions in one `instructions` array) instead of parallel calls.
|
|
436
|
-
- **One atomic behavior per test case** — every test case verifies exactly one user-observable behavior. Never bundle signup/login/navigation/bootstrap/teardown into a test case body. Ordering and dependencies are Muggle's service responsibility, not the skill's.
|
|
460
|
+
- **One atomic behavior per test case** — every test case verifies exactly one user-observable behavior. Never bundle signup/login/navigation/bootstrap/teardown into a test case body. Ordering and dependencies are Muggle Test's service responsibility, not the skill's.
|
|
437
461
|
- **Never consolidate the generator's output** — if `muggle-remote-test-case-generate-from-prompt` returns N micro-tests, accept all N; never merge them into fewer test cases, even if "the plan" says 4 UC / 4 TC.
|
|
438
462
|
- **Never skip the generate→review cycle** — always present generated test cases to the user before calling `muggle-remote-test-case-create`, even when you're confident. "I'll skip the review and create directly" is always wrong.
|
|
439
463
|
- **Never silently drop test cases** — log failures and continue, then report them
|