@muggleai/works 4.4.0 → 4.6.0

package/plugin/skills/do/open-prs.md

@@ -23,80 +23,15 @@ For each repo with changes:
    - If E2E acceptance tests have failures: `[E2E FAILING] <goal>`
    - Otherwise: `<goal>`
    - Keep under 70 characters
-3. **Build the PR body** with these sections:
+3. **Render the E2E acceptance block** by invoking the shared `muggle-pr-visual-walkthrough` skill in **Mode B** (render-only for embedding). See "Rendering the E2E acceptance block via the shared skill" below. You receive `{body, comment}` where `body` is the E2E markdown block and `comment` is a non-null overflow comment only when the content exceeds the CLI's byte budget.
+4. **Build the PR body** by concatenating, in order:
    - `## Goal` — the requirements goal
    - `## Acceptance Criteria` — bulleted list (omit section if empty)
    - `## Changes` — summary of what changed in this repo
-   - E2E acceptance evidence block from `muggle build-pr-section` (see "Rendering the E2E acceptance results block" below)
-4. **Create the PR** using `gh pr create --title "..." --body "..." --head <branch>` in the repo directory.
-5. **Capture the PR URL** and extract the PR number.
-6. **Post the overflow comment only if `muggle build-pr-section` emitted one** (see "Rendering the E2E acceptance results block" below). In the common case, no comment is posted.
-
-## Rendering the E2E acceptance results block
-
-Do **not** hand-write the `## E2E Acceptance Results` markdown. Use the `muggle build-pr-section` CLI, which renders a deterministic block and decides whether the evidence fits in the PR description or needs to spill into an overflow comment.
-
-### Step A: Build the report JSON
-
-Assemble the e2e-acceptance report you collected in `e2e-acceptance.md` into a JSON object with this shape:
-
-```json
-{
-  "projectId": "<project UUID>",
-  "tests": [
-    {
-      "name": "<test case name>",
-      "testCaseId": "<UUID>",
-      "testScriptId": "<UUID or omitted>",
-      "runId": "<UUID>",
-      "viewUrl": "<muggle-ai.com run URL>",
-      "status": "passed",
-      "steps": [
-        { "stepIndex": 0, "action": "<action>", "screenshotUrl": "<URL>" }
-      ]
-    },
-    {
-      "name": "<test case name>",
-      "testCaseId": "<UUID>",
-      "runId": "<UUID>",
-      "viewUrl": "<muggle-ai.com run URL>",
-      "status": "failed",
-      "failureStepIndex": 2,
-      "error": "<error message>",
-      "artifactsDir": "<path, optional>",
-      "steps": [
-        { "stepIndex": 0, "action": "<action>", "screenshotUrl": "<URL>" }
-      ]
-    }
-  ]
-}
-```
-
-### Step B: Render the evidence block
-
-Pipe the JSON into `muggle build-pr-section`. It writes `{ "body": "...", "comment": "..." | null }` to stdout:
-
-```bash
-echo "$REPORT_JSON" | muggle build-pr-section > /tmp/muggle-pr-section.json
-```
-
-The command exits nonzero on malformed input and writes a descriptive error to stderr — do not swallow that error, surface it to the user.
-
-### Step C: Build the PR body
-
-Build the PR body by concatenating, in order:
-
-- `## Goal` — the requirements goal
-- `## Acceptance Criteria` — bulleted list (omit section if empty)
-- `## Changes` — summary of what changed in this repo
-- The `body` field from the CLI output (already contains its own `## E2E Acceptance Results` header)
-
-### Step D: Create the PR, then post the overflow comment only if present
-
-1. Create the PR with `gh pr create --title "..." --body "..." --head <branch>`.
-2. Capture the PR URL and extract the PR number.
-3. If the CLI output's `comment` field is `null`, **do not post a comment** — everything is already in the PR description.
-4. If the CLI output's `comment` field is a non-null string, post it as a follow-up comment:
+   - The `body` field from the skill output (already contains its own `## E2E Acceptance Results` header — do not add another)
+5. **Create the PR** using `gh pr create --title "..." --body "..." --head <branch>` in the repo directory.
+6. **Capture the PR URL** and extract the PR number.
+7. **Post the overflow `comment` only if it is non-null.** In the common case, `comment` is `null` and nothing is posted. Never post speculatively.
 
    ```bash
    gh pr comment <PR#> --body "$(cat <<'EOF'
@@ -105,11 +40,37 @@ Build the PR body by concatenating, in order:
    )"
    ```
 
+## Rendering the E2E acceptance block via the shared skill
+
+**Do not hand-write the `## E2E Acceptance Results` markdown, and do not call `muggle build-pr-section` directly from this stage.** The rendering workflow is owned by the shared **`muggle-pr-visual-walkthrough`** skill (see `plugin/skills/muggle-pr-visual-walkthrough/SKILL.md`), which wraps the CLI and enforces the `E2eReport` input contract with a Zod schema.
+
+### Input — the `E2eReport` JSON
+
+The `e2e-acceptance.md` stage already produces an `E2eReport` with the exact shape the skill expects (`projectId` + `tests[]` with per-test `name`, `testCaseId`, `testScriptId`, `runId`, `viewUrl`, `status`, and `steps[]` of `{stepIndex, action, screenshotUrl}`; failed tests additionally have `failureStepIndex`, `error`, and optionally `artifactsDir`). Pass it through unchanged — do not reshape it. The full schema is documented in the shared skill.
+
+### Invocation — Mode B (render-only)
+
+Invoke `muggle-pr-visual-walkthrough` via the `Skill` tool with the `E2eReport` already in context. The skill will:
+
+1. Validate the `E2eReport` and call `muggle build-pr-section` (piping the JSON to stdin).
+2. Parse the CLI's `{body, comment}` stdout.
+3. **Return `{body, comment}` to this stage's conversation** without posting anything — because Mode B is render-only. `body` is the E2E markdown block; `comment` is a non-null overflow follow-up comment only when content exceeds the byte budget, otherwise `null`.
+
+Mode A (where the skill itself finds an existing PR and posts a `gh pr comment`) is **not used by `muggle-do`** — it's for interactive callers like `muggle-test` that are mid-development with a PR already open. `muggle-do` always creates new PRs, so it always uses Mode B.
+
+### After rendering
+
+Back in this stage:
+
+- Embed `body` in the `gh pr create --body` body (see step 4 above).
+- Post the overflow `comment` as a follow-up **only when it is non-null** (see step 7 above).
+- If the CLI exited non-zero, the skill surfaces the stderr error — do not swallow it, surface it to the user.
+
 ### Notes on fit vs. overflow
 
-- **The common case is fit**: the full evidence (summary, per-test rows, collapsible failure details) lives in the PR description, no comment is posted.
-- **The overflow case** is triggered automatically when the full inline body would exceed the CLI's budget. In that case the PR description contains the summary, the per-test rows, and a pointer line; the full step-by-step failure details live in the follow-up comment.
-- You do not make the fit-vs-overflow decision — the CLI does. Never post the comment speculatively.
+- **Common case (fit):** the full evidence (summary, per-test rows, collapsible failure details) lives in the PR description, `comment` is `null`, no follow-up comment is posted.
+- **Overflow case:** the CLI detects the full body would exceed its byte budget; `body` contains the summary, per-test rows, and a pointer line; `comment` contains the overflow details. Post both.
+- You do not make the fit-vs-overflow decision — the CLI does. Never post the comment when it is `null`.
 
 ## Output
 

package/plugin/skills/muggle-pr-visual-walkthrough/SKILL.md

@@ -0,0 +1,181 @@
+---
+name: muggle-pr-visual-walkthrough
+description: Renders and posts a visual walkthrough of Muggle AI E2E acceptance test results to a PR — per-test-case dashboard links, step-by-step screenshots, and pass/fail summary — using the `muggle build-pr-section` CLI for deterministic formatting with automatic fit-vs-overflow. Use at the end of any Muggle test run (local or remote) to give PR reviewers clickable visual evidence that user flows work. Triggers on 'post results to PR', 'attach walkthrough to PR', 'share E2E screenshots on the PR', 'add visual walkthrough to PR'.
+---
+
+# Muggle PR Visual Walkthrough
+
+Renders a visual walkthrough of Muggle AI E2E acceptance test results and posts it to a PR. Each test case is linked to its detail page on the Muggle AI dashboard, so PR reviewers can click through to see step-by-step screenshots and action scripts — not just a pass/fail flag.
+
+This is the **canonical PR-walkthrough workflow** shared across every Muggle entry point:
+
+| Caller | Mode | When to invoke |
+| :--- | :--- | :--- |
+| `muggle-test` | **Mode A** (post to existing PR) | After publishing results, user opts in via `AskQuestion` |
+| `muggle-test-feature-local` | **Mode A** (post to existing PR) | After publishing the run, user opts in via `AskQuestion` |
+| `muggle-do` / `open-prs.md` | **Mode B** (render-only for embedding) | During PR creation — caller embeds `body` in `gh pr create` and posts `comment` as follow-up |
+
+Rendering is always done by `muggle build-pr-section`, a battle-tested CLI that handles deterministic markdown layout, per-step screenshots, and automatic fit-vs-overflow (oversized content spills into a follow-up comment). Never hand-write the walkthrough markdown.
+
+## Input contract: the `E2eReport` JSON
+
+Every caller must build an `E2eReport` JSON object and have it in conversation context before invoking this skill. The schema is defined in `src/cli/pr-section/types.ts` (`E2eReportSchema`) and enforced by the CLI with Zod — malformed input exits non-zero with a descriptive stderr message.
+
+```json
+{
+  "projectId": "<UUID>",
+  "tests": [
+    {
+      "name": "<test case name>",
+      "testCaseId": "<UUID>",
+      "testScriptId": "<UUID>",
+      "runId": "<UUID>",
+      "viewUrl": "https://www.muggle-ai.com/...",
+      "status": "passed",
+      "steps": [
+        { "stepIndex": 0, "action": "Click login button", "screenshotUrl": "https://..." },
+        { "stepIndex": 1, "action": "Type email", "screenshotUrl": "https://..." }
+      ]
+    },
+    {
+      "name": "Checkout flow",
+      "testCaseId": "<UUID>",
+      "testScriptId": "<UUID>",
+      "runId": "<UUID>",
+      "viewUrl": "https://www.muggle-ai.com/...",
+      "status": "failed",
+      "steps": [
+        { "stepIndex": 0, "action": "Open cart", "screenshotUrl": "https://..." }
+      ],
+      "failureStepIndex": 2,
+      "error": "Element not found: Click checkout button",
+      "artifactsDir": "/Users/.../~/.muggle-ai/sessions/<runId>"
+    }
+  ]
+}
+```
+
+Required fields per test: `name`, `testCaseId`, `runId`, `viewUrl`, `status`, `steps[]` with `{stepIndex, action, screenshotUrl}`. Failed tests additionally require `failureStepIndex` and `error`. `testScriptId` and `artifactsDir` are optional.
+
+If any required field is missing, stop and tell the caller exactly what's missing. Never fabricate data.
+
+## Step 1: Gather the `E2eReport`
+
+How to assemble the JSON depends on which caller you are:
+
+### From `muggle-test` / `muggle-test-feature-local` (local mode)
+
+After `muggle-local-publish-test-script` returns `{testScriptId, viewUrl, ...}` for each run:
+
+1. Call `muggle-remote-test-script-get` with `testScriptId` to fetch the published script.
+2. Extract `steps[]` — for each step, build `{stepIndex: <index>, action: operation.action, screenshotUrl: operation.screenshotUrl}`.
+3. Determine `status` from the local run result (`muggle-local-run-result-get`).
+4. For failures, read `failureStepIndex`, `error`, and `artifactsDir` from the run result.
+5. Assemble the `E2eReport` with `projectId` from the test run.
+
+### From `muggle-do` (`open-prs.md`)
+
+The `e2e-acceptance.md` stage already produces an `E2eReport` with the exact shape above — that is the report's native output format. Pass it through unchanged.
+
+### Direct invocation (user asked to post existing results)
+
+The caller must have already executed tests and published them. If the `E2eReport` is not in context, stop and tell the user to run `muggle-test` or `muggle-test-feature-local` first.
+
+## Step 2: Render via `muggle build-pr-section`
+
+Pipe the `E2eReport` JSON to the CLI. It writes `{"body": "...", "comment": "..." | null}` to stdout — the `body` is the E2E markdown block, and `comment` is a non-null overflow comment only when the full body exceeds the byte budget (default 60 KB).
+
+```bash
+echo "$REPORT_JSON" | muggle build-pr-section > /tmp/muggle-pr-section.json
+```
+
+- Exit **non-zero** → the CLI wrote a descriptive error to stderr. Surface it to the user; do not swallow it.
+- `comment` is **`null`** (fit case) → everything is inline in `body`. Post `body` only.
+- `comment` is a **non-null string** (overflow case) → `body` contains the summary + a pointer; `comment` contains the full per-step details. Post both, in order.
+
+Never hand-write the walkthrough markdown. Never modify the CLI's output before posting. The CLI owns the format.
+
+## Step 3 — Mode A: Post to an existing PR
+
+Used by `muggle-test` and `muggle-test-feature-local`, where the user is mid-development and a PR already exists on the current branch.
+
+### 3A.1: Find the PR
+
+```bash
+gh pr view --json number,url,title 2>/dev/null
+```
+
+- **PR exists** → continue to 3A.2
+- **No PR exists** → use `AskQuestion`:
+  - "Create a new PR with the visual walkthrough in the body"
+  - "Skip posting"
+  - If the user chooses to create a new PR, switch to Mode B and return the rendered `body`/`comment` to the caller for embedding in `gh pr create`. Do not create the PR directly from this skill unless the caller has no better way to do it.
+- **`gh` not installed or not authenticated** → tell the user, suggest `gh auth login`, stop.
+
+### 3A.2: Post the body as a PR comment
+
+```bash
+gh pr comment <pr-number> --body "$(cat <<'EOF'
+<contents of body field from CLI output>
+EOF
+)"
+```
+
+### 3A.3: Post the overflow comment only if the CLI emitted one
+
+```bash
+gh pr comment <pr-number> --body "$(cat <<'EOF'
+<contents of comment field from CLI output>
+EOF
+)"
+```
+
+**Skip this step entirely if `comment` is `null`** — do not post a placeholder. The CLI decides fit-vs-overflow; never post the overflow comment speculatively.
+
+### 3A.4: Confirm to the user
+
+> "Visual walkthrough posted to PR #<number>. Reviewers can click any test case link to see the step-by-step screenshots on the Muggle AI dashboard."
+
+Include the PR URL in the confirmation.
+
+## Step 3 — Mode B: Return rendered block for embedding in a new PR
+
+Used by `muggle-do`'s `open-prs.md`, where the PR does not exist yet and the caller is assembling the PR body from multiple sections (`## Goal`, `## Acceptance Criteria`, `## Changes`, plus this walkthrough).
+
+Instead of posting, **return** the CLI output to the caller's context so they can:
+
+1. **Embed `body`** in their PR body, concatenated after `## Changes`. `body` already includes its own `## E2E Acceptance Results` header — do not add another.
+2. **Create the PR** with `gh pr create --title "..." --body "..."` using the concatenated body.
+3. **Post `comment` as a follow-up only if the CLI emitted one:**
+
+   ```bash
+   gh pr comment <new-pr-number> --body "$(cat <<'EOF'
+   <contents of comment field>
+   EOF
+   )"
+   ```
+
+   Skip if `comment` is `null`.
+
+In Mode B, this skill does not call `gh pr comment` or `gh pr create` itself — the caller owns PR creation because it also owns branch pushing, title building (including `[E2E FAILING]` prefix on failures), and multi-repo orchestration.
+
+## Tool Reference
+
+| Phase | Tool |
+|:------|:-----|
+| Gather per-step data (muggle-test, muggle-test-feature-local) | `muggle-remote-test-script-get` |
+| Render the walkthrough markdown | `muggle build-pr-section` (shell) |
+| Find existing PR (Mode A) | `gh pr view` |
+| Post comment(s) (Mode A) | `gh pr comment` |
+| Create new PR (Mode B, caller handles) | `gh pr create` |
+| User confirmation (Mode A no-PR branch) | `AskQuestion` |
+
+## Guardrails
+
+- **Never hand-write the walkthrough markdown** — always call `muggle build-pr-section`. The CLI is the single source of truth for formatting.
+- **Never modify the CLI's output** — post `body` and (if present) `comment` verbatim. Any reformatting defeats the fit-vs-overflow budget math.
+- **Never invent report fields** — if `projectId`, a per-test `viewUrl`, or per-step `screenshotUrl` is missing, stop and report what's missing. Do not fabricate URLs or fill in placeholders.
+- **Never post the overflow comment when `comment` is `null`** — the CLI decides fit-vs-overflow.
+- **Never create a PR without confirmation in Mode A** — if no PR exists, ask the user or switch to Mode B and hand back to the caller.
+- **Don't run tests** — this skill only renders and posts existing results. If the `E2eReport` is not in context, redirect the caller to `muggle-test`, `muggle-test-feature-local`, or `muggle-do`.
+- **Mode A vs Mode B is chosen by the caller, not the user** — `muggle-test` always uses Mode A; `muggle-do` always uses Mode B. Don't ask the user which mode to use.