@muggleai/works 4.2.2 → 4.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +45 -37
- package/dist/{chunk-BZJXQZ5Q.js → chunk-PMI2DI3V.js} +524 -173
- package/dist/cli.js +1 -1
- package/dist/index.js +1 -1
- package/dist/plugin/.claude-plugin/plugin.json +4 -4
- package/dist/plugin/.cursor-plugin/plugin.json +3 -3
- package/dist/plugin/README.md +7 -5
- package/dist/plugin/scripts/ensure-electron-app.sh +3 -3
- package/dist/plugin/skills/do/e2e-acceptance.md +161 -0
- package/dist/plugin/skills/do/open-prs.md +86 -16
- package/dist/plugin/skills/muggle/SKILL.md +15 -13
- package/dist/plugin/skills/muggle-do/SKILL.md +6 -6
- package/dist/plugin/skills/muggle-test/SKILL.md +380 -0
- package/dist/plugin/skills/muggle-test-feature-local/SKILL.md +44 -27
- package/dist/plugin/skills/muggle-test-import/SKILL.md +272 -0
- package/dist/plugin/skills/muggle-upgrade/SKILL.md +1 -1
- package/dist/plugin/skills/optimize-descriptions/SKILL.md +8 -8
- package/package.json +15 -12
- package/plugin/.claude-plugin/plugin.json +4 -4
- package/plugin/.cursor-plugin/plugin.json +3 -3
- package/plugin/README.md +7 -5
- package/plugin/scripts/ensure-electron-app.sh +3 -3
- package/plugin/skills/do/e2e-acceptance.md +161 -0
- package/plugin/skills/do/open-prs.md +86 -16
- package/plugin/skills/muggle/SKILL.md +15 -13
- package/plugin/skills/muggle-do/SKILL.md +6 -6
- package/plugin/skills/muggle-test/SKILL.md +380 -0
- package/plugin/skills/muggle-test-feature-local/SKILL.md +44 -27
- package/plugin/skills/muggle-test-import/SKILL.md +272 -0
- package/plugin/skills/muggle-upgrade/SKILL.md +1 -1
- package/plugin/skills/optimize-descriptions/SKILL.md +8 -8
- package/dist/plugin/skills/do/qa.md +0 -89
- package/plugin/skills/do/qa.md +0 -89
|
@@ -0,0 +1,380 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: muggle-test
|
|
3
|
+
description: "Run change-driven E2E acceptance testing using Muggle AI — detects local code changes, maps them to use cases, and generates test scripts either locally (real browser on localhost) or remotely (cloud execution on a preview/staging URL). Publishes results to Muggle dashboard, opens them in the browser, and posts E2E acceptance summaries with screenshots to the PR. Use this skill whenever the user wants to test their changes, run E2E acceptance tests on recent work, validate what they've been working on, or check if their code changes broke anything. Triggers on: 'test my changes', 'run tests on my changes', 'acceptance test my work', 'check my changes', 'validate my changes', 'test before I push', 'make sure my changes work', 'regression test my changes', 'test on preview', 'test on staging'. This is the go-to skill for change-driven E2E acceptance testing — it handles everything from change detection to test execution to result reporting."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Muggle Test — Change-Driven E2E Acceptance Router
|
|
7
|
+
|
|
8
|
+
A router skill that detects code changes, resolves impacted test cases, executes them locally or remotely, publishes results to the Muggle AI dashboard, and posts E2E acceptance summaries to the PR. The user can invoke this at any moment, in any state.
|
|
9
|
+
|
|
10
|
+
## UX Guidelines — Minimize Typing
|
|
11
|
+
|
|
12
|
+
**Every selection-based question MUST use the `AskQuestion` tool** (or the platform's equivalent structured selection tool). Never ask the user to "reply with a number" in a plain text message — always present clickable options.
|
|
13
|
+
|
|
14
|
+
- **Selections** (project, use case, test case, mode, approval): Use `AskQuestion` with labeled options the user can click.
|
|
15
|
+
- **Multi-select** (use cases, test cases): Use `AskQuestion` with `allow_multiple: true`.
|
|
16
|
+
- **Free-text inputs** (URLs, descriptions): Only use plain text prompts when there is no finite set of options. Even then, offer a detected/default value when possible.
|
|
17
|
+
- **Batch related questions**: If two questions are independent, present them together in a single `AskQuestion` call rather than asking sequentially.
|
|
18
|
+
|
|
19
|
+
## Step 1: Confirm Scope of Work (Always First)
|
|
20
|
+
|
|
21
|
+
Parse the user's query and explicitly confirm their expectation. There are exactly two modes:
|
|
22
|
+
|
|
23
|
+
### Mode A: Local Test Generation
|
|
24
|
+
> Test impacted use cases/test cases against **localhost** using the Electron browser.
|
|
25
|
+
>
|
|
26
|
+
> Execution tool: `muggle-local-execute-test-generation`
|
|
27
|
+
|
|
28
|
+
Signs the user wants this: mentions "localhost", "local", "my machine", "dev server", "my changes locally", or just "test my changes" in a repo context.
|
|
29
|
+
|
|
30
|
+
### Mode B: Remote Test Generation
|
|
31
|
+
> Ask Muggle's cloud to generate test scripts against a **preview/staging URL**.
|
|
32
|
+
>
|
|
33
|
+
> Execution tool: `muggle-remote-workflow-start-test-script-generation`
|
|
34
|
+
|
|
35
|
+
Signs the user wants this: mentions "preview", "staging", "deployed", "preview URL", "test on preview", "test the deployment", or provides a non-localhost URL.
|
|
36
|
+
|
|
37
|
+
### Confirming
|
|
38
|
+
|
|
39
|
+
If the user's intent is clear, state back what you understood and use `AskQuestion` to confirm:
|
|
40
|
+
- Option 1: "Yes, proceed"
|
|
41
|
+
- Option 2: "Switch to [the other mode]"
|
|
42
|
+
|
|
43
|
+
If ambiguous, use `AskQuestion` to let the user choose:
|
|
44
|
+
- Option 1: "Local — launch browser on your machine against localhost"
|
|
45
|
+
- Option 2: "Remote — Muggle cloud tests against a preview/staging URL"
|
|
46
|
+
|
|
47
|
+
Only proceed after the user selects an option.
|
|
48
|
+
|
|
49
|
+
## Step 2: Detect Local Changes
|
|
50
|
+
|
|
51
|
+
Analyze the working directory to understand what changed.
|
|
52
|
+
|
|
53
|
+
1. Run `git status` and `git diff --stat` for an overview
|
|
54
|
+
2. Run `git diff` (or `git diff --cached` if staged) to read actual diffs
|
|
55
|
+
3. Identify impacted feature areas:
|
|
56
|
+
- Changed UI components, pages, routes
|
|
57
|
+
- Modified API endpoints or data flows
|
|
58
|
+
- Updated form fields, validation, user interactions
|
|
59
|
+
4. Produce a concise **change summary** — a list of impacted features
|
|
60
|
+
|
|
61
|
+
Present:
|
|
62
|
+
> "Here's what changed: [list]. I'll scope E2E acceptance testing to these areas."
|
|
63
|
+
|
|
64
|
+
If no changes detected (clean tree), tell the user and ask what they want to test.
|
|
65
|
+
|
|
66
|
+
## Step 3: Authenticate
|
|
67
|
+
|
|
68
|
+
1. Call `muggle-remote-auth-status`
|
|
69
|
+
2. If authenticated and not expired → proceed
|
|
70
|
+
3. If not authenticated or expired → call `muggle-remote-auth-login`
|
|
71
|
+
4. If login pending → call `muggle-remote-auth-poll`
|
|
72
|
+
|
|
73
|
+
If auth fails repeatedly, suggest: `muggle logout && muggle login` from terminal.
|
|
74
|
+
|
|
75
|
+
## Step 4: Select Project (User Must Choose)
|
|
76
|
+
|
|
77
|
+
A **project** is where all your test results, use cases, and test scripts are grouped on the Muggle AI dashboard. Pick the project that matches what you're working on.
|
|
78
|
+
|
|
79
|
+
1. Call `muggle-remote-project-list`
|
|
80
|
+
2. Use `AskQuestion` to present all projects as clickable options. Include the project URL in each label so the user can identify the right one. Always include a "Create new project" option at the end.
|
|
81
|
+
|
|
82
|
+
Example labels:
|
|
83
|
+
- "MUGGLE AI STAGING 1 — https://staging.muggle-ai.com/"
|
|
84
|
+
- "Tanka Testing — https://www.tanka.ai"
|
|
85
|
+
- "Create new project"
|
|
86
|
+
|
|
87
|
+
Prompt: "Pick the project to group this test run into:"
|
|
88
|
+
|
|
89
|
+
3. **Wait for the user to explicitly choose** — do NOT auto-select based on repo name or URL matching
|
|
90
|
+
4. **If user chooses "Create new project"**:
|
|
91
|
+
- Ask for `projectName`, `description`, and the production/preview URL
|
|
92
|
+
- Call `muggle-remote-project-create`
|
|
93
|
+
|
|
94
|
+
Store the `projectId` only after user confirms.
|
|
95
|
+
|
|
96
|
+
## Step 5: Select Use Case (User Must Choose)
|
|
97
|
+
|
|
98
|
+
### 5a: List existing use cases
|
|
99
|
+
Call `muggle-remote-use-case-list` with the project ID.
|
|
100
|
+
|
|
101
|
+
### 5b: Present ALL use cases for user selection
|
|
102
|
+
|
|
103
|
+
Use `AskQuestion` with `allow_multiple: true` to present all use cases as clickable options. Always include a "Create new use case" option at the end.
|
|
104
|
+
|
|
105
|
+
Prompt: "Which use case(s) do you want to test?"
|
|
106
|
+
|
|
107
|
+
### 5c: Wait for explicit user selection
|
|
108
|
+
|
|
109
|
+
**CRITICAL: Do NOT auto-select use cases** based on:
|
|
110
|
+
- Git changes analysis
|
|
111
|
+
- Use case title/description matching
|
|
112
|
+
- Any heuristic or inference
|
|
113
|
+
|
|
114
|
+
The user MUST explicitly tell you which use case(s) to use.
|
|
115
|
+
|
|
116
|
+
### 5d: If user chooses "Create new use case"
|
|
117
|
+
1. Ask the user to describe the use case in plain English
|
|
118
|
+
2. Call `muggle-remote-use-case-create-from-prompts`:
|
|
119
|
+
- `projectId`: The project ID
|
|
120
|
+
- `prompts`: Array of `{ instruction: "..." }` with the user's description
|
|
121
|
+
3. Present the created use case and confirm it's correct
|
|
122
|
+
|
|
123
|
+
## Step 6: Select Test Case (User Must Choose)
|
|
124
|
+
|
|
125
|
+
For the selected use case(s):
|
|
126
|
+
|
|
127
|
+
### 6a: List existing test cases
|
|
128
|
+
Call `muggle-remote-test-case-list-by-use-case` with each use case ID.
|
|
129
|
+
|
|
130
|
+
### 6b: Present ALL test cases for user selection
|
|
131
|
+
|
|
132
|
+
Use `AskQuestion` with `allow_multiple: true` to present all test cases as clickable options. Always include a "Generate new test case" option at the end.
|
|
133
|
+
|
|
134
|
+
Prompt: "Which test case(s) do you want to run?"
|
|
135
|
+
|
|
136
|
+
### 6c: Wait for explicit user selection
|
|
137
|
+
|
|
138
|
+
**CRITICAL: Do NOT auto-select test cases** — the user MUST explicitly choose which test case(s) to execute.
|
|
139
|
+
|
|
140
|
+
### 6d: If user chooses "Generate new test case"
|
|
141
|
+
1. Ask the user to describe what they want to test in plain English
|
|
142
|
+
2. Call `muggle-remote-test-case-generate-from-prompt`:
|
|
143
|
+
- `projectId`, `useCaseId`, `instruction` (the user's description)
|
|
144
|
+
3. Present the generated test case(s) for review
|
|
145
|
+
4. Call `muggle-remote-test-case-create` to save the ones the user approves
|
|
146
|
+
|
|
147
|
+
### 6e: Confirm final selection
|
|
148
|
+
|
|
149
|
+
Use `AskQuestion` to confirm: "You selected [N] test case(s): [list titles]. Ready to proceed?"
|
|
150
|
+
- Option 1: "Yes, run them"
|
|
151
|
+
- Option 2: "No, let me re-select"
|
|
152
|
+
|
|
153
|
+
Wait for user confirmation before moving to execution.
|
|
154
|
+
|
|
155
|
+
## Step 7A: Execute — Local Mode
|
|
156
|
+
|
|
157
|
+
### Pre-flight questions (batch where possible)
|
|
158
|
+
|
|
159
|
+
**Question 1 — Local URL:**
|
|
160
|
+
|
|
161
|
+
Try to auto-detect the dev server URL by checking running terminals or common ports (e.g., `lsof -iTCP -sTCP:LISTEN -nP | grep -E ':(3000|3001|4200|5173|8080)'`). If a likely URL is found, present it as a clickable default via `AskQuestion`:
|
|
162
|
+
- Option 1: "http://localhost:3000" (or whatever was detected)
|
|
163
|
+
- Option 2: "Other — let me type a URL"
|
|
164
|
+
|
|
165
|
+
If nothing detected, ask as free text: "Your local app should be running. What's the URL? (e.g., http://localhost:3000)"
|
|
166
|
+
|
|
167
|
+
**Question 2 — Electron launch + window visibility (ask together):**
|
|
168
|
+
|
|
169
|
+
After getting the URL, use a single `AskQuestion` call with two questions:
|
|
170
|
+
|
|
171
|
+
1. "Ready to launch the Muggle Electron browser for [N] test case(s)?"
|
|
172
|
+
- "Yes, launch it (visible — I want to watch)"
|
|
173
|
+
- "Yes, launch it (headless — run in background)"
|
|
174
|
+
- "No, cancel"
|
|
175
|
+
|
|
176
|
+
If user cancels, stop and ask what they want to do instead.
|
|
177
|
+
|
|
178
|
+
### Run sequentially
|
|
179
|
+
|
|
180
|
+
For each test case:
|
|
181
|
+
|
|
182
|
+
1. Call `muggle-remote-test-case-get` to fetch full details
|
|
183
|
+
2. Call `muggle-local-execute-test-generation`:
|
|
184
|
+
- `testCase`: Full test case object from step 1
|
|
185
|
+
- `localUrl`: User's local URL (from Question 1)
|
|
186
|
+
- `approveElectronAppLaunch`: `true` (only if user approved in Question 2)
|
|
187
|
+
- `showUi`: `true` if user chose "visible", `false` if "headless" (from Question 2)
|
|
188
|
+
3. Store the returned `runId`
|
|
189
|
+
|
|
190
|
+
If a generation fails, log it and continue to the next. Do not abort the batch.
|
|
191
|
+
|
|
192
|
+
### Collect results
|
|
193
|
+
|
|
194
|
+
For each `runId`, call `muggle-local-run-result-get`. Extract: status, duration, step count, `artifactsDir`.
|
|
195
|
+
|
|
196
|
+
### Publish each run to cloud
|
|
197
|
+
|
|
198
|
+
For each completed run, call `muggle-local-publish-test-script`:
|
|
199
|
+
- `runId`: The local run ID
|
|
200
|
+
- `cloudTestCaseId`: The cloud test case ID
|
|
201
|
+
|
|
202
|
+
This returns:
|
|
203
|
+
- `viewUrl`: Direct link to view this test run on the Muggle AI dashboard
|
|
204
|
+
- `testScriptId`, `actionScriptId`, `workflowRuntimeId`
|
|
205
|
+
|
|
206
|
+
Store every `viewUrl` — these are used in the next steps.
|
|
207
|
+
|
|
208
|
+
### Report summary
|
|
209
|
+
|
|
210
|
+
```
|
|
211
|
+
Test Case Status Duration Steps View on Muggle
|
|
212
|
+
─────────────────────────────────────────────────────────────────────────
|
|
213
|
+
Login with valid creds PASSED 12.3s 8 https://www.muggle-ai.com/...
|
|
214
|
+
Login with invalid creds PASSED 9.1s 6 https://www.muggle-ai.com/...
|
|
215
|
+
Checkout flow FAILED 15.7s 12 https://www.muggle-ai.com/...
|
|
216
|
+
─────────────────────────────────────────────────────────────────────────
|
|
217
|
+
Total: 3 tests | 2 passed | 1 failed | 37.1s
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
For failures: show which step failed, the local screenshot path, and a suggestion.
|
|
221
|
+
|
|
222
|
+
## Step 7B: Execute — Remote Mode
|
|
223
|
+
|
|
224
|
+
### Ask for target URL
|
|
225
|
+
|
|
226
|
+
> "What's the preview/staging URL to test against?"
|
|
227
|
+
|
|
228
|
+
### Trigger remote workflows
|
|
229
|
+
|
|
230
|
+
For each test case:
|
|
231
|
+
|
|
232
|
+
1. Call `muggle-remote-test-case-get` to fetch full details
|
|
233
|
+
2. Call `muggle-remote-workflow-start-test-script-generation`:
|
|
234
|
+
- `projectId`: The project ID
|
|
235
|
+
- `useCaseId`: The use case ID
|
|
236
|
+
- `testCaseId`: The test case ID
|
|
237
|
+
- `name`: `"muggle-test: {test case title}"`
|
|
238
|
+
- `url`: The preview/staging URL
|
|
239
|
+
- `goal`: From the test case
|
|
240
|
+
- `precondition`: From the test case (use `"None"` if empty)
|
|
241
|
+
- `instructions`: From the test case
|
|
242
|
+
- `expectedResult`: From the test case
|
|
243
|
+
3. Store the returned workflow runtime ID
|
|
244
|
+
|
|
245
|
+
### Monitor and report
|
|
246
|
+
|
|
247
|
+
For each workflow, call `muggle-remote-wf-get-ts-gen-latest-run` with the runtime ID.
|
|
248
|
+
|
|
249
|
+
```
|
|
250
|
+
Test Case Workflow Status Runtime ID
|
|
251
|
+
────────────────────────────────────────────────────────
|
|
252
|
+
Login with valid creds RUNNING rt-abc123
|
|
253
|
+
Login with invalid creds COMPLETED rt-def456
|
|
254
|
+
Checkout flow QUEUED rt-ghi789
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
## Step 8: Open Results in Browser
|
|
258
|
+
|
|
259
|
+
After execution and publishing are complete, open the Muggle AI dashboard so the user can visually inspect results and screenshots.
|
|
260
|
+
|
|
261
|
+
### Mode A (Local) — open each published viewUrl
|
|
262
|
+
|
|
263
|
+
For each published run's `viewUrl`:
|
|
264
|
+
```bash
|
|
265
|
+
open "https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/scripts?modal=script-details&testCaseId={testCaseId}"
|
|
266
|
+
```
|
|
267
|
+
|
|
268
|
+
If there are many runs (>3), open just the project-level runs page instead of individual tabs:
|
|
269
|
+
```bash
|
|
270
|
+
open "https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/runs"
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
### Mode B (Remote) — open the project runs page
|
|
274
|
+
|
|
275
|
+
```bash
|
|
276
|
+
open "https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/runs"
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
Tell the user:
|
|
280
|
+
> "I've opened the Muggle AI dashboard in your browser — you can see the test results, step-by-step screenshots, and action scripts there."
|
|
281
|
+
|
|
282
|
+
## Step 9: Post E2E Acceptance Results to PR
|
|
283
|
+
|
|
284
|
+
After reporting results, check if there's an open PR for the current branch and attach the E2E acceptance summary.
|
|
285
|
+
|
|
286
|
+
### 9a: Find the PR
|
|
287
|
+
|
|
288
|
+
```bash
|
|
289
|
+
gh pr view --json number,url,title 2>/dev/null
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
- If a PR exists → post results as a comment
|
|
293
|
+
- If no PR exists → use `AskQuestion`:
|
|
294
|
+
- "Create PR with E2E acceptance results"
|
|
295
|
+
- "Skip posting to PR"
|
|
296
|
+
|
|
297
|
+
### 9b: Build the E2E acceptance comment body
|
|
298
|
+
|
|
299
|
+
Construct a markdown comment with the full E2E acceptance breakdown. The format links each test case to its detail page on the Muggle AI dashboard, so PR reviewers can click through to see step-by-step screenshots and action scripts.
|
|
300
|
+
|
|
301
|
+
```markdown
|
|
302
|
+
## 🧪 Muggle AI — E2E Acceptance Results
|
|
303
|
+
|
|
304
|
+
**X passed / Y failed** | [View all on Muggle AI](https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/runs)
|
|
305
|
+
|
|
306
|
+
| Test Case | Status | Details |
|
|
307
|
+
|-----------|--------|---------|
|
|
308
|
+
| [Login with valid creds](https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/scripts?modal=script-details&testCaseId={testCaseId}) | ✅ PASSED | 8 steps, 12.3s |
|
|
309
|
+
| [Login with invalid creds](https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/scripts?modal=script-details&testCaseId={testCaseId}) | ✅ PASSED | 6 steps, 9.1s |
|
|
310
|
+
| [Checkout flow](https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/scripts?modal=script-details&testCaseId={testCaseId}) | ❌ FAILED | Step 7: "Click checkout button" — element not found |
|
|
311
|
+
|
|
312
|
+
<details>
|
|
313
|
+
<summary>Failed test details</summary>
|
|
314
|
+
|
|
315
|
+
### Checkout flow
|
|
316
|
+
- **Failed at**: Step 7 — "Click checkout button"
|
|
317
|
+
- **Error**: Element not found
|
|
318
|
+
- **Local artifacts**: `~/.muggle-ai/sessions/{runId}/`
|
|
319
|
+
- **Screenshots**: `~/.muggle-ai/sessions/{runId}/screenshots/`
|
|
320
|
+
|
|
321
|
+
</details>
|
|
322
|
+
|
|
323
|
+
---
|
|
324
|
+
*Generated by [Muggle AI](https://www.muggle-ai.com) — change-driven E2E acceptance testing*
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
### 9c: Post to the PR
|
|
328
|
+
|
|
329
|
+
If PR already exists — add as a comment:
|
|
330
|
+
```bash
|
|
331
|
+
gh pr comment {pr-number} --body "$(cat <<'EOF'
|
|
332
|
+
{the E2E acceptance comment body from 9b}
|
|
333
|
+
EOF
|
|
334
|
+
)"
|
|
335
|
+
```
|
|
336
|
+
|
|
337
|
+
If creating a new PR — include the E2E acceptance section in the PR body alongside the usual summary/changes sections.
|
|
338
|
+
|
|
339
|
+
### 9d: Confirm to user
|
|
340
|
+
|
|
341
|
+
> "E2E acceptance results posted to PR #{number}. Reviewers can click the test case links to see step-by-step screenshots on the Muggle AI dashboard."
|
|
342
|
+
|
|
343
|
+
## Tool Reference
|
|
344
|
+
|
|
345
|
+
| Phase | Tool | Mode |
|
|
346
|
+
|:------|:-----|:-----|
|
|
347
|
+
| Auth | `muggle-remote-auth-status` | Both |
|
|
348
|
+
| Auth | `muggle-remote-auth-login` | Both |
|
|
349
|
+
| Auth | `muggle-remote-auth-poll` | Both |
|
|
350
|
+
| Project | `muggle-remote-project-list` | Both |
|
|
351
|
+
| Project | `muggle-remote-project-create` | Both |
|
|
352
|
+
| Use Case | `muggle-remote-use-case-list` | Both |
|
|
353
|
+
| Use Case | `muggle-remote-use-case-create-from-prompts` | Both |
|
|
354
|
+
| Test Case | `muggle-remote-test-case-list-by-use-case` | Both |
|
|
355
|
+
| Test Case | `muggle-remote-test-case-generate-from-prompt` | Both |
|
|
356
|
+
| Test Case | `muggle-remote-test-case-create` | Both |
|
|
357
|
+
| Test Case | `muggle-remote-test-case-get` | Both |
|
|
358
|
+
| Execute | `muggle-local-execute-test-generation` | Local |
|
|
359
|
+
| Execute | `muggle-remote-workflow-start-test-script-generation` | Remote |
|
|
360
|
+
| Results | `muggle-local-run-result-get` | Local |
|
|
361
|
+
| Results | `muggle-remote-wf-get-ts-gen-latest-run` | Remote |
|
|
362
|
+
| Publish | `muggle-local-publish-test-script` | Local |
|
|
363
|
+
| Browser | `open` (shell command) | Both |
|
|
364
|
+
| PR | `gh pr view`, `gh pr comment`, `gh pr create` | Both |
|
|
365
|
+
|
|
366
|
+
## Guardrails
|
|
367
|
+
|
|
368
|
+
- **Always confirm intent first** — never assume local vs remote without asking
|
|
369
|
+
- **User MUST select project** — present clickable options via `AskQuestion`, wait for explicit choice, never auto-select
|
|
370
|
+
- **User MUST select use case(s)** — present clickable options via `AskQuestion`, wait for explicit choice, never auto-select based on git changes or heuristics
|
|
371
|
+
- **User MUST select test case(s)** — present clickable options via `AskQuestion`, wait for explicit choice, never auto-select
|
|
372
|
+
- **Use `AskQuestion` for every selection** — never ask the user to type a number; always present clickable options
|
|
373
|
+
- **Batch related questions** — combine Electron approval + visibility into one question; auto-detect localhost URL when possible
|
|
374
|
+
- **Never launch Electron without explicit user approval** (`approveElectronAppLaunch`)
|
|
375
|
+
- **Never silently drop test cases** — log failures and continue, then report them
|
|
376
|
+
- **Never guess the URL** — always ask the user for localhost or preview URL
|
|
377
|
+
- **Always publish before opening browser** — the dashboard needs the published data to show results
|
|
378
|
+
- **Use correct dashboard URL format** — `modal=script-details` (not `modal=details`)
|
|
379
|
+
- **Always check for PR before posting** — don't create a PR comment if there's no PR (ask user first)
|
|
380
|
+
- **Can be invoked at any state** — if the user already has a project or use cases set up, skip to the relevant step rather than re-doing everything
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: muggle-test-feature-local
|
|
3
|
-
description: Run a real-browser
|
|
3
|
+
description: Run a real-browser end-to-end (E2E) acceptance test against localhost to verify a feature works correctly — signup flows, checkout, form validation, UI interactions, or any user-facing behavior. Launches a browser that executes test steps and captures screenshots. Use this skill whenever the user asks to test, validate, or verify their web app, UI changes, user flows, or frontend behavior on localhost or a dev server — even if they don't mention 'muggle' or 'E2E' explicitly.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Muggle Test Feature Local
|
|
7
7
|
|
|
8
|
-
**Goal:** Run or generate an end-to-end test against a **local URL** using Muggle
|
|
8
|
+
**Goal:** Run or generate an end-to-end test against a **local URL** using Muggle's Electron browser.
|
|
9
9
|
|
|
10
10
|
| Scope | MCP tools |
|
|
11
11
|
| :---- | :-------- |
|
|
@@ -15,12 +15,19 @@ description: Run a real-browser QA test against localhost to verify a feature wo
|
|
|
15
15
|
|
|
16
16
|
The local URL only changes where the browser opens; it does not change the remote project or test definitions.
|
|
17
17
|
|
|
18
|
+
## UX Guidelines — Minimize Typing
|
|
19
|
+
|
|
20
|
+
**Every selection-based question MUST use the `AskQuestion` tool** (or the platform's equivalent structured selection tool). Never ask the user to "reply with a number" in a plain text message — always present clickable options.
|
|
21
|
+
|
|
22
|
+
- **Selections** (project, use case, test case, script, approval): Use `AskQuestion` with labeled options the user can click.
|
|
23
|
+
- **Free-text inputs** (URLs, descriptions): Only use plain text prompts when there is no finite set of options. Even then, offer a detected/default value when possible.
|
|
24
|
+
|
|
18
25
|
## Workflow
|
|
19
26
|
|
|
20
27
|
### 1. Auth
|
|
21
28
|
|
|
22
29
|
- `muggle-remote-auth-status`
|
|
23
|
-
- If not signed in: `muggle-remote-auth-login` then `muggle-remote-auth-poll`
|
|
30
|
+
- If not signed in: `muggle-remote-auth-login` then `muggle-remote-auth-poll`
|
|
24
31
|
Do not skip or assume auth.
|
|
25
32
|
|
|
26
33
|
### 2. Targets (user must confirm)
|
|
@@ -31,41 +38,45 @@ Ask the user to pick **project**, **use case**, and **test case** (do not infer)
|
|
|
31
38
|
- `muggle-remote-use-case-list` (with `projectId`)
|
|
32
39
|
- `muggle-remote-test-case-list-by-use-case` (with `useCaseId`)
|
|
33
40
|
|
|
34
|
-
**Selection UI (mandatory):**
|
|
41
|
+
**Selection UI (mandatory):** Every selection MUST use `AskQuestion` with clickable options. Never ask the user to "reply with the number" in plain text.
|
|
35
42
|
|
|
36
|
-
**
|
|
43
|
+
**Project selection context:** A **project** groups all your test results, use cases, and test scripts on the Muggle AI dashboard. Include the project URL in each option label so the user can identify the right one.
|
|
37
44
|
|
|
38
|
-
|
|
39
|
-
2. **Create new …** — user creates a new entity instead of picking an existing one. Label per step: **Create new project**, **Create new use case**, or **Create new test case**.
|
|
45
|
+
Prompt for projects: "Pick the project to group this test into:"
|
|
40
46
|
|
|
41
47
|
**Relevance-first filtering (mandatory for project, use case, and test case lists):**
|
|
42
48
|
|
|
43
49
|
- Do **not** dump the full list by default.
|
|
44
|
-
- Rank items by semantic relevance to the user
|
|
45
|
-
- Show only the **top 3
|
|
46
|
-
-
|
|
50
|
+
- Rank items by semantic relevance to the user's stated goal (title first, then description / user story / acceptance criteria).
|
|
51
|
+
- Show only the **top 3-5** most relevant options via `AskQuestion`, plus these fixed tail options:
|
|
52
|
+
- **"Show full list"** — present the complete list in a new `AskQuestion` call. **Skip this option** if the API returned zero rows.
|
|
53
|
+
- **"Create new ..."** — never omitted. Label per step: "Create new project", "Create new use case", or "Create new test case".
|
|
47
54
|
|
|
48
55
|
**Create new — tools and flow (use these MCP tools; preview before persist):**
|
|
49
56
|
|
|
50
57
|
- **Project — Create new project:** Collect `projectName`, `description`, and `url` (may be the local app URL, e.g. `http://localhost:3999`). Call `muggle-remote-project-create`. Use the returned `projectId` and continue.
|
|
51
|
-
- **Use case — Create new use case:** User provides a natural-language instruction (or you reuse their testing goal).
|
|
52
|
-
1. `muggle-remote-use-case-prompt-preview` with `projectId`, `instruction` — show preview; get confirmation
|
|
58
|
+
- **Use case — Create new use case:** User provides a natural-language instruction (or you reuse their testing goal).
|
|
59
|
+
1. `muggle-remote-use-case-prompt-preview` with `projectId`, `instruction` — show preview; get confirmation via `AskQuestion`.
|
|
53
60
|
2. `muggle-remote-use-case-create-from-prompts` with `projectId`, `prompts: [{ instruction }]` — persist. Use the created use case id and continue to test-case selection.
|
|
54
|
-
- **Test case — Create new test case** (requires a chosen `useCaseId`): User provides an instruction describing what to test.
|
|
55
|
-
1. `muggle-remote-test-case-generate-from-prompt` with `projectId`, `useCaseId`, `instruction` — **preview only** (server test-case prompt preview); show the returned draft(s); get confirmation
|
|
56
|
-
2. Persist the accepted draft with `muggle-remote-test-case-create`, mapping preview fields into the required properties (`title`, `description`, `goal`, `expectedResult`, `url`, etc.). Then continue from
|
|
61
|
+
- **Test case — Create new test case** (requires a chosen `useCaseId`): User provides an instruction describing what to test.
|
|
62
|
+
1. `muggle-remote-test-case-generate-from-prompt` with `projectId`, `useCaseId`, `instruction` — **preview only** (server test-case prompt preview); show the returned draft(s); get confirmation via `AskQuestion`.
|
|
63
|
+
2. Persist the accepted draft with `muggle-remote-test-case-create`, mapping preview fields into the required properties (`title`, `description`, `goal`, `expectedResult`, `url`, etc.). Then continue from **section 4** with that `testCaseId`.
|
|
57
64
|
|
|
58
65
|
### 3. Local URL
|
|
59
66
|
|
|
60
|
-
-
|
|
61
|
-
-
|
|
67
|
+
Try to auto-detect the dev server URL by checking running terminals or common ports (e.g., `lsof -iTCP -sTCP:LISTEN -nP | grep -E ':(3000|3001|4200|5173|8080)'`). If a likely URL is found, present it as a clickable default via `AskQuestion`:
|
|
68
|
+
- Option 1: "http://localhost:3000" (or whatever was detected)
|
|
69
|
+
- Option 2: "Other — let me type a URL"
|
|
70
|
+
|
|
71
|
+
If nothing detected, ask as free text: "Your local app should be running. What's the URL? (e.g., http://localhost:3000)"
|
|
72
|
+
|
|
73
|
+
Remind them: local URL is only the execution target, not tied to cloud project config.
|
|
62
74
|
|
|
63
75
|
### 4. Existing scripts vs new generation
|
|
64
76
|
|
|
65
77
|
`muggle-remote-test-script-list` with `testCaseId`.
|
|
66
78
|
|
|
67
|
-
- **If any replayable/succeeded scripts exist:**
|
|
68
|
-
Show: name, id, created/updated, step count. Include **`Generate new script`** as the **last** numbered option (e.g. last number) so it is selectable by number too.
|
|
79
|
+
- **If any replayable/succeeded scripts exist:** use `AskQuestion` to present them as clickable options. Show: name, created/updated, step count per option. Include **"Generate new script"** as the last option.
|
|
69
80
|
- **If none:** go straight to generation (no need to ask replay vs generate).
|
|
70
81
|
|
|
71
82
|
### 5. Load data for the chosen path
|
|
@@ -77,8 +88,8 @@ Ask the user to pick **project**, **use case**, and **test case** (do not infer)
|
|
|
77
88
|
|
|
78
89
|
**Replay**
|
|
79
90
|
|
|
80
|
-
1. `muggle-remote-test-script-get`
|
|
81
|
-
2. `muggle-remote-action-script-get` with that id
|
|
91
|
+
1. `muggle-remote-test-script-get` — note `actionScriptId`
|
|
92
|
+
2. `muggle-remote-action-script-get` with that id — full `actionScript`
|
|
82
93
|
**Use the API response as-is.** Do not edit, shorten, or rebuild `actionScript`; replay needs full `label` paths for element lookup.
|
|
83
94
|
3. `muggle-local-execute-replay` (after approval in step 6) with `testScript`, `actionScript`, `localUrl`, `approveElectronAppLaunch: true` (optional: `showUi: true`, **`timeoutMs`** — see below)
|
|
84
95
|
|
|
@@ -88,17 +99,22 @@ The MCP client often uses a **default wait of 300000 ms (5 minutes)** for `muggl
|
|
|
88
99
|
|
|
89
100
|
- **Always pass `timeoutMs`** for flows that may be long — for example **`600000` (10 min)** or **`900000` (15 min)** — unless the user explicitly wants a short cap.
|
|
90
101
|
- If the tool reports **`Electron execution timed out after 300000ms`** (or similar) **but** Electron logs show the run still progressing (steps, screenshots, LLM calls), treat it as **orchestration timeout**, not an Electron app defect: **increase `timeoutMs` and retry** (after user re-approves if your policy requires it).
|
|
91
|
-
- **Test case design:** Preconditions like
|
|
102
|
+
- **Test case design:** Preconditions like "a test run has already completed" on an **empty account** can force many steps (sign-up, new project, crawl). Prefer an account/project that **already has** the needed state, or narrow the test goal so generation does not try to create a full project from scratch unless that is intentional.
|
|
92
103
|
|
|
93
104
|
### Interpreting `failed` / non-zero Electron exit
|
|
94
105
|
|
|
95
106
|
- **`Electron execution timed out after 300000ms`:** Orchestration wait too short — see **`timeoutMs`** above.
|
|
96
|
-
- **Exit code 26** (and messages like **LLM failed to generate / replay action script**): Often corresponds to a completed exploration whose **outcome was goal not achievable** (`goal_not_achievable`, summary with `halt`) — e.g. verifying
|
|
107
|
+
- **Exit code 26** (and messages like **LLM failed to generate / replay action script**): Often corresponds to a completed exploration whose **outcome was goal not achievable** (`goal_not_achievable`, summary with `halt`) — e.g. verifying "view script after a successful run" when **no run or script exists yet** in the UI. Use `muggle-local-run-result-get` and read the **summary / structured summary**; do not assume an Electron crash. **Fix:** choose a **project that already has** completed runs and scripts, or **change the test case** so preconditions match what localhost can satisfy (e.g. include steps to create and run a test first, or assert only empty-state UI when no runs exist).
|
|
97
108
|
|
|
98
109
|
### 6. Approval before any local execution
|
|
99
110
|
|
|
100
|
-
|
|
101
|
-
|
|
111
|
+
Use `AskQuestion` to get explicit approval before launching Electron. State: replay vs generation, test case name, URL.
|
|
112
|
+
|
|
113
|
+
- "Yes, launch Electron (visible — I want to watch)"
|
|
114
|
+
- "Yes, launch Electron (headless — run in background)"
|
|
115
|
+
- "No, cancel"
|
|
116
|
+
|
|
117
|
+
Only call local execute tools with `approveElectronAppLaunch: true` after the user selects a "Yes" option. Map visible to `showUi: true`, headless to `showUi: false`.
|
|
102
118
|
|
|
103
119
|
### 7. After successful generation only
|
|
104
120
|
|
|
@@ -112,8 +128,9 @@ Only then call local execute tools with `approveElectronAppLaunch: true`.
|
|
|
112
128
|
|
|
113
129
|
## Non-negotiables
|
|
114
130
|
|
|
115
|
-
- No silent auth skip; no launching Electron without approval
|
|
131
|
+
- No silent auth skip; no launching Electron without approval via `AskQuestion`.
|
|
116
132
|
- If replayable scripts exist, do not default to generation without user choice.
|
|
117
133
|
- No hiding failures: surface errors and artifact paths.
|
|
118
134
|
- Replay: never hand-built or simplified `actionScript` — only from `muggle-remote-action-script-get`.
|
|
119
|
-
-
|
|
135
|
+
- Use `AskQuestion` for every selection — project, use case, test case, script, and approval. Never ask the user to type a number.
|
|
136
|
+
- Project, use case, and test case selection lists must always include "Create new ...". Include "Show full list" whenever the API returned at least one row for that step; omit "Show full list" when the list is empty (offer "Create new ..." only). For creates, use preview tools (`muggle-remote-use-case-prompt-preview`, `muggle-remote-test-case-generate-from-prompt`) before persisting.
|