greenrun-cli 0.2.11 → 0.2.12

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "greenrun-cli",
-  "version": "0.2.11",
+  "version": "0.2.12",
   "description": "CLI and MCP server for Greenrun - browser test management for Claude Code",
   "type": "module",
   "main": "dist/server.js",

package/templates/commands/procedures.md

@@ -44,66 +44,61 @@ If all tests are scripted, skip to Step 4.
 
 ### Step 3: Generate scripts for unscripted tests
 
-For each **unscripted** test, one at a time:
+For each **unscripted** test, launch a Task agent sequentially (one at a time, wait for each to complete before starting the next). This keeps browser snapshot data out of the parent context.
 
-1. Call `get_test(test_id)` to fetch the full instructions
-2. Do a **scouting pass** — follow the test instructions in the browser to observe all UI states:
-   - Navigate to the test's starting page via `browser_navigate`
-   - Take a `browser_snapshot` to see initial elements
-   - Follow the test instructions step by step using Playwright MCP tools (`browser_click`, `browser_type`, `browser_snapshot` after each action)
-   - Snapshot after each state change to capture: validation errors, success banners, modal dialogs, redirected pages, dynamically loaded content
-   - Collect all observed elements and selectors as context
-
-#### Handling failures during scouting
-
-If a step doesn't work as expected during the scouting pass, investigate before moving on:
-
-1. **Determine the cause**: Is it a test problem (wrong instructions, bad selectors, missing prerequisite) or an application bug (form won't submit, unexpected error, broken functionality)?
+```
+Task tool with:
+- subagent_type: "general-purpose"
+- max_turns: 30
+- model: "sonnet"
+- prompt: (see agent prompt below)
+```
 
-2. **If the test is wrong** — fix and retry:
-   - Adjust the instructions to match what the UI actually requires (e.g. a required field the instructions missed, a different button label, an extra confirmation step)
-   - Update the test via `update_test` with corrected instructions
-   - Retry the failing step
+#### Script generation agent prompt
 
-3. **If it's an application bug** — work around it and record the bug:
-   - Find a way to make the original test pass by avoiding the broken path (e.g. if a discount code field breaks form submission, leave it blank)
-   - Update the original test instructions if needed to use the working path
-   - Create a **new bug test** that reproduces the specific failure:
-     ```
-     create_test(project_id, {
-       name: "BUG: [description of the failure]",
-       instructions: "[steps that reproduce the bug, ending with the expected vs actual behaviour]",
-       tags: ["bug"],
-       page_ids: [relevant page IDs],
-       credential_name: same as original test
-     })
-     ```
-   - Start a run for the bug test and immediately complete it as failed:
-     ```
-     start_run(bug_test_id) → complete_run(run_id, "failed", "description of what went wrong")
-     ```
-   - Continue scouting the original test with the workaround
+Include the following in the prompt, substituting the actual values:
 
-This ensures the original test captures the happy path while bugs are tracked as separate failing tests that will show up in future runs.
+```
+Greenrun script generation for test: {test_name}
+Test ID: {test_id}
+Project ID: {project_id}
 
-After each test's scouting pass, close the browser so the next test starts with a clean context (no leftover cookies, storage, or page state):
+Project auth: {auth_mode}, login_url: {login_url}
+Credentials: {credential_name} — email: {email}, password: {password}
 
-Call `browser_close` after collecting all observations. The next `browser_navigate` call will automatically open a fresh browser context.
+## Task
 
-Then generate a `.spec.ts` script using the observed elements:
+1. Call `get_test("{test_id}")` to fetch the full test instructions
+2. Authenticate: navigate to {login_url} and log in with the credential above using `browser_navigate`, `browser_snapshot`, `browser_click`, `browser_type`
+3. Do a scouting pass — follow the test instructions step by step in the browser:
+   - Navigate to the test's starting page via `browser_navigate`
+   - Take a `browser_snapshot` to see initial elements
+   - Follow each instruction using Playwright MCP tools (`browser_click`, `browser_type`, `browser_snapshot` after each action)
+   - Snapshot after each state change to capture selectors, validation errors, success banners, modal dialogs, redirected pages
+4. Handle failures:
+   - If a step fails because the test instructions are wrong (wrong field name, missing step, bad selector), fix the instructions and retry. Update the test via `update_test` with corrected instructions.
+   - If a step fails because of an application bug, work around it for the main test and create a new bug test:
+     `create_test({project_id}, { name: "BUG: [description]", instructions: "[repro steps]", tags: ["bug"], page_ids: [...], credential_name: "{credential_name}" })`
+     Then: `start_run(bug_test_id)` → `complete_run(run_id, "failed", "description")`
+5. After scouting, generate a Playwright `.spec.ts` script:
 
-```ts
 import { test, expect } from '@playwright/test';
 test('{test_name}', async ({ page }) => {
-  // If the test has a credential_name, include login steps using the matching
-  // credential from project.credentials (email + password) at the login_url
+  // Include login steps using the credential email + password at login_url
   await page.goto('{start_url}');
-  // Steps generated from scouting pass observations
+  // Steps from scouting observations
   // Use getByRole, getByText, getByLabel, getByPlaceholder for selectors
 });
+
+6. Save: `update_test("{test_id}", { script: <generated_script>, script_generated_at: "<ISO_now>" })`
+7. Close browser: `browser_close`
+
+## Return
+
+Return a one-line summary: {test_name} | script generated | or | {test_name} | failed | {reason}
 ```
 
-Save via `update_test(test_id, { script: <generated_script>, script_generated_at: <ISO_now> })`.
+After each agent completes, note the result and proceed to the next unscripted test.
 
 ### Step 4: Export auth state
 
@@ -165,23 +160,50 @@ After parsing all native results, walk through them in completion order. Track c
 
 ### Step 7: AI fallback for native failures
 
-For tests that **failed** in native execution (and circuit breaker has not tripped), execute them one at a time using the AI agent approach:
+For tests that **failed** in native execution (and circuit breaker has not tripped), execute them one at a time via Task agents. This keeps snapshot data out of the parent context.
 
-1. Close the current browser context with `browser_close` so the fallback starts fresh
-2. Re-authenticate by navigating to the login page and following the Authenticate procedure
-3. For each failed test:
-   - Call `get_test(test_id)` to fetch the full instructions
-   - Start a new run via `start_run(test_id)` (the original run was already completed in Step 5)
-   - Navigate to the test's starting page via `browser_navigate`
-   - Follow the test instructions step by step using Playwright MCP tools
-   - Determine if this is a stale script (UI changed) or an actual bug
-   - If the test passes manually, invalidate the cached script: `update_test(test_id, { script: null, script_generated_at: null })`
-   - Call `complete_run(run_id, status, brief_summary)`
-   - Call `browser_close` before the next test to reset state
+For each failed test, launch a Task agent sequentially (wait for each to complete before the next):
+
+```
+Task tool with:
+- subagent_type: "general-purpose"
+- max_turns: 25
+- model: "sonnet"
+- prompt: (see agent prompt below)
+```
+
+#### AI fallback agent prompt
+
+```
+Greenrun AI fallback test. Test: {test_name}
+Test ID: {test_id}
+
+Project auth: {auth_mode}, login_url: {login_url}
+Credentials: {credential_name} — email: {email}, password: {password}
+
+Native execution failed with: {failure_message}
+
+## Task
+
+1. Call `get_test("{test_id}")` to fetch the full test instructions
+2. Start a new run: `start_run("{test_id}")` — note the run_id
+3. Authenticate: navigate to {login_url} and log in with the credential above
+4. Follow the test instructions step by step using Playwright MCP tools (`browser_navigate`, `browser_snapshot`, `browser_click`, `browser_type`)
+5. Determine if the native failure was a stale script (UI changed) or an actual application bug
+6. If the test passes manually, invalidate the stale cached script: `update_test("{test_id}", { script: null, script_generated_at: null })`
+7. Call `complete_run(run_id, status, brief_summary)` — ALWAYS call this, even on error
+8. Call `browser_close`
+
+## Return
+
+Return: {test_name} | {status} | {summary}
+```
+
+After each agent completes, note the result. If the agent fails to call `complete_run`, call it yourself with status "error".
 
 ### Step 8: Handle unscripted tests without scripts
 
-Any tests that didn't get scripts generated in Step 3 (e.g. if script generation failed) need to be executed the same way as Step 7 — one at a time using the AI agent approach. Follow the same pattern: get instructions, start run, execute in browser, complete run, close browser.
+Any tests that didn't get scripts generated in Step 3 (e.g. if script generation failed) need to be executed the same way as Step 7 — launch a Task agent for each one sequentially using the AI fallback agent prompt above (omit the "Native execution failed with" line).
 
 ## Summarize