playwright-test-agent 0.1.3 → 0.1.4

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "playwright-test-agent",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "description": "Initialize Playwright Test agents and install the playwright-test-agent skill.",
   "type": "module",
   "bin": {

package/playwright-test-agent/SKILL.md

@@ -21,10 +21,10 @@ Use the names for the active platform. The role names below refer to these mappe
 
 ## Agent responsibilities
 
-- **Main agent:** Own the user conversation, inspect only the local project for context, gather the minimum non-discoverable information from the user, locate the default seed generated by `playwright init-agents`, dispatch each subagent with that seed, present the plan for confirmation, and report final results. It must not open, navigate, inspect, screenshot, or otherwise investigate the deployed website itself.
-- **Planner:** Run the generated seed to establish page context, global setup, dependencies, fixtures, and hooks. Explore the deployed application for the requested scenarios and user flows, compare observations with the request and optional PRD, and save a precise human-readable Markdown plan under `specs/`. It does not generate the formal test files.
-- **Generator:** Read the complete user-confirmed Markdown plan and the same generated seed, execute each scenario to validate locators and assertions against the live application, and generate executable Playwright tests under the configured test directory. It must not invent unconfirmed scenarios or expected behavior.
-- **Healer:** Run the failing test, replay its steps, inspect the current UI, propose and apply test-side patches, and rerun until the test passes or a guardrail stops the loop. It must report a likely broken product, environment/data problem, or plan conflict instead of weakening assertions merely to obtain a pass.
+- **Main agent:** Own the user conversation, inspect only the local project for context, gather the minimum non-discoverable information from the user, locate the default seed generated by `playwright init-agents`, start Planner, Generator, and Healer directly as leaf subagents, present the plan for confirmation, and report final results. It must not open, navigate, inspect, screenshot, or otherwise investigate the deployed website itself. It must not insert an intermediary or nested agent between itself and any Playwright Test agent.
+- **Planner:** Run the generated seed to establish page context, global setup, dependencies, fixtures, and hooks. Explore the deployed application for the requested scenarios and user flows, compare observations with the request and optional PRD, and save a precise human-readable Markdown plan under `specs/`. It does not generate the formal test files and must not spawn another agent. It must call its own injected `planner_setup_page` and `planner_save_plan` tools directly.
+- **Generator:** Read the complete user-confirmed Markdown plan and the same generated seed, execute each scenario to validate locators and assertions against the live application, and generate executable Playwright tests under the configured test directory. It must not invent unconfirmed scenarios or expected behavior, must not spawn another agent, and must call its own injected `generator_setup_page`, `generator_read_log`, and `generator_write_test` tools directly.
+- **Healer:** Run the failing test, replay its steps, inspect the current UI, propose and apply test-side patches, and rerun until the test passes or a guardrail stops the loop. It must not spawn another agent and must call its own injected `test_list`, `test_run`, `test_debug`, and relevant `browser_*` tools directly. It must report a likely broken product, environment/data problem, or plan conflict instead of weakening assertions merely to obtain a pass.
 
 For UI testing, keep these responsibilities and sequence:
 
@@ -33,6 +33,8 @@ request + deployed URL -> locate generated seed -> planner -> user confirms plan
                        -> generator -> healer for failures -> final report
 ```
 
+Planner, Generator, and Healer are terminal/leaf agents in this flow. The main agent starts the platform-specific agent for each role directly and passes all required inputs in that invocation. Never delegate their work to `live_planner`, `live_generator`, `live_healer`, a same-role nested agent, or another intermediary. If a directly started agent lacks any required MCP tool, stop with an MCP-injection diagnosis; do not create another subagent level and do not replace that agent's work in the main agent.
+
 ## Initialize once per project
 
 Inspect the target project first. If Playwright Test, its config, or either Codex/Claude agent set is missing, tell the user initialization will modify the project, then run from the target project:
@@ -74,9 +76,9 @@ Never put credentials or tokens in plans, source, screenshots, reports, or chat
 
 Always have the planner save its human-readable Markdown plan under `specs/` before invoking the generator. Include prerequisites, the generated seed path, test data, independent scenarios, steps or requests, expected observable results, and intended output test files.
 
-For UI work, invoke the Planner once the objective, deployed base URL, and generated seed path are available and no known mismatch already blocks access. Explicitly name the seed in its prompt and provide an optional PRD only when one is relevant. The Planner runs the seed to perform global setup, dependencies, fixtures, and hooks, then uses its ready page context to inspect the live application. Reconnaissance must not create, delete, submit, purchase, message, or otherwise mutate durable/shared data without authorization.
+For UI work, invoke the Planner directly (as the first and only Planner-level subagent) once the objective, deployed base URL, and generated seed path are available and no known mismatch already blocks access. Explicitly name the seed in its prompt and provide an optional PRD only when one is relevant. The Planner runs the seed to perform global setup, dependencies, fixtures, and hooks, then uses its ready page context to inspect the live application. Reconnaissance must not create, delete, submit, purchase, message, or otherwise mutate durable/shared data without authorization.
 
-If the planner discovers a new missing or incorrect prerequisite, the main agent stops that planning attempt, summarizes what was observed, and asks the user for clarification. Continue or reinvoke the planner only after the prerequisite is resolved; do not let it keep exploring around missing information.
+If the planner discovers a new missing or incorrect prerequisite, the main agent stops that planning attempt, summarizes what was observed, and asks the user for clarification. Continue or reinvoke the planner only after the prerequisite is resolved; do not let it keep exploring around missing information. If the Planner session itself lacks a required MCP tool, report that direct-session environment failure and wait for the environment to be repaired; do not create a nested fallback agent.
 
 Choose relevant positive, negative, empty, invalid, boundary, permission, persistence, and error scenarios. For login, normally consider valid credentials, empty username, empty password, both empty, wrong password, and unknown user; add MFA, lockout, recovery, or remember-me only when in scope.
 
@@ -84,11 +86,11 @@ Show the complete scenario list and exclusions to the user. Ask whether it is co
 
 ## Generate and run
 
-For confirmed UI scenarios, invoke the Generator once for the complete confirmed plan (or complete selected scenario set), explicitly naming the Markdown plan and the same generated seed; use the configured Playwright test directory for generated tests. The Generator executes scenarios against the live application to validate locators and assertions while generating. Its internal per-scenario setup does not mean starting a new Generator agent for every scenario. Require one independent test per file, semantic locators, an assertion for every expected result, and environment-based secrets.
+For confirmed UI scenarios, invoke the Generator directly once for the complete confirmed plan (or complete selected scenario set), explicitly naming the Markdown plan and the same generated seed; use the configured Playwright test directory for generated tests. The Generator executes scenarios against the live application to validate locators and assertions while generating. Its internal per-scenario setup does not mean starting a new Generator agent for every scenario, and it must not delegate execution to a nested Generator. If the direct Generator lacks `generator_setup_page`, `generator_read_log`, or `generator_write_test`, report an MCP-injection failure and stop. Require one independent test per file, semantic locators, an assertion for every expected result, and environment-based secrets.
 
 For confirmed API-only scenarios, write `APIRequestContext` tests directly under `playwright-tests/`. Assert status, headers, schema, and stable business invariants; define safe setup/cleanup for mutations.
 
-After generation, invoke the Healer with one failing UI test name at a time. The Healer runs the test, replays the failing steps, inspects the current UI for the equivalent element or flow, proposes a patch such as a locator, synchronization, or test-data repair, and reruns until it passes or a guardrail stops the loop. API failures may be diagnosed directly.
+After generation, invoke the Healer directly with one failing UI test name at a time. The Healer runs the test, replays the failing steps, inspects the current UI for the equivalent element or flow, proposes a patch such as a locator, synchronization, or test-data repair, and reruns until it passes or a guardrail stops the loop. It must not delegate diagnosis or repair to a nested Healer. If the direct Healer lacks `test_list`, `test_run`, `test_debug`, or the browser tools needed for diagnosis, report an MCP-injection failure and stop. API failures may be diagnosed directly.
 
 If observed product behavior conflicts with the confirmed plan, report the conflict and ask whether it is a regression or intended change. Do not weaken assertions, add arbitrary sleeps/`networkidle`, retry blindly, or skip/fixme tests merely to get green. If the healer concludes the feature is broken, preserve the failure evidence and report it as an application defect rather than treating a skipped test as success.
 
@@ -106,4 +108,6 @@ All browser-agent and test artifacts must stay under:
 
 Never save snapshots or screenshots in the project root. Automatic failure evidence is enabled. For key business states, save an explicit screenshot with `testInfo.outputPath('screenshots', '<meaningful-name>.png')` so it remains in `test-results/`. Avoid secrets and sensitive personal data.
 
+Keep trace DOM snapshots, sources, network data, and attachments, but disable trace screenshots (`use.trace.screenshots: false`). Trace screenshots are screencast frames and can create hundreds of JPEG files under trace `resources/`; they are separate from explicit test screenshots and are not needed for ordinary DOM-based diagnosis.
+
 Finish by reporting the plan path, generated test paths, environment without secrets, pass/fail/flaky/skipped counts, and evidence paths. Classify failures as application defect, test defect, environment/data problem, or unresolved product decision.

package/playwright-test-agent/scripts/init-playwright.mjs

@@ -12,7 +12,9 @@ const CODEX_INSTRUCTIONS = `${BLOCK_START}
 
 For any website, browser workflow, HTTP API, or application-feature testing request, load and follow \`.agents/skills/playwright-test-agent/SKILL.md\` before planning or using browser tools. This skill has priority over ad hoc browser automation.
 
-For UI testing, the Codex subagents are \`playwright_test_planner\`, \`playwright_test_generator\`, and \`playwright_test_healer\`. The main agent may inspect only the local project; it must not open, navigate, inspect, screenshot, or otherwise investigate the deployed website itself. Before starting Planner, the main agent may ask for necessary information that materially affects test purpose, scope, expected outcomes, roles, allowed side effects, data, or environment constraints. It must not ask for UI details Planner can safely discover. Once the objective, deployed URL, and necessary non-discoverable constraints are available, start \`playwright_test_planner\` to perform all live page investigation.
+For UI testing, the Codex subagents are \`playwright_test_planner\`, \`playwright_test_generator\`, and \`playwright_test_healer\`. The main agent may inspect only the local project; it must not open, navigate, inspect, screenshot, or otherwise investigate the deployed website itself. Before starting Planner, the main agent may ask for necessary information that materially affects test purpose, scope, expected outcomes, roles, allowed side effects, data, or environment constraints. It must not ask for UI details Planner can safely discover. Once the objective, deployed URL, and necessary non-discoverable constraints are available, start \`playwright_test_planner\` directly as the live-investigation leaf agent.
+
+The main agent must start each Playwright Test agent directly. Planner, Generator, and Healer are leaf agents: do not ask them to spawn \`live_planner\`, \`live_generator\`, \`live_healer\`, or any other nested subagent. Each must use its own injected Playwright Test MCP tools directly. If a directly started agent does not expose its required tools, stop and report the Codex MCP-injection problem; do not retry by creating a second- or third-level agent.
 
 If Planner reports that required test information is missing, invalid, or inconsistent, stop that planning attempt and ask the user for the specific missing information. After the user responds, start \`playwright_test_planner\` again with the corrected information and the previous blocking observation. Do not continue guessing or exploring around the blocker.
 ${BLOCK_END}`;
@@ -22,7 +24,9 @@ const CLAUDE_INSTRUCTIONS = `${BLOCK_START}
 
 For any website, browser workflow, HTTP API, or application-feature testing request, load and follow \`.claude/skills/playwright-test-agent/SKILL.md\` before planning or using browser tools. This skill has priority over ad hoc browser automation.
 
-For UI testing, the Claude Code subagents are \`playwright-test-planner\`, \`playwright-test-generator\`, and \`playwright-test-healer\`. The main agent may inspect only the local project; it must not open, navigate, inspect, screenshot, or otherwise investigate the deployed website itself. Before starting Planner, the main agent may ask for necessary information that materially affects test purpose, scope, expected outcomes, roles, allowed side effects, data, or environment constraints. It must not ask for UI details Planner can safely discover. Once the objective, deployed URL, and necessary non-discoverable constraints are available, start \`playwright-test-planner\` to perform all live page investigation.
+For UI testing, the Claude Code subagents are \`playwright-test-planner\`, \`playwright-test-generator\`, and \`playwright-test-healer\`. The main agent may inspect only the local project; it must not open, navigate, inspect, screenshot, or otherwise investigate the deployed website itself. Before starting Planner, the main agent may ask for necessary information that materially affects test purpose, scope, expected outcomes, roles, allowed side effects, data, or environment constraints. It must not ask for UI details Planner can safely discover. Once the objective, deployed URL, and necessary non-discoverable constraints are available, start \`playwright-test-planner\` directly as the live-investigation leaf agent.
+
+The main agent must start each Playwright Test agent directly. Planner, Generator, and Healer are leaf agents: do not ask them to spawn \`live_planner\`, \`live_generator\`, \`live_healer\`, or any other nested subagent. Each must use its own injected Playwright Test MCP tools directly. If a directly started agent does not expose its required tools, stop and report the MCP-injection problem; do not retry by creating a second- or third-level agent.
 
 If Planner reports that required test information is missing, invalid, or inconsistent, stop that planning attempt and ask the user for the specific missing information. After the user responds, start \`playwright-test-planner\` again with the corrected information and the previous blocking observation. Do not continue guessing or exploring around the blocker.
 ${BLOCK_END}`;
@@ -98,10 +102,38 @@ async function patchPlaywrightConfig(projectDir) {
     /use:\s*\{/,
     "outputDir: '.playwright-evidence/test-results',\n  snapshotPathTemplate: '.playwright-evidence/snapshots/{testFilePath}/{arg}{ext}',\n  use: {\n    screenshot: 'only-on-failure',\n    video: 'retain-on-failure',",
   );
-  source = source.replace(/trace:\s*['"][^'"]+['"]/, "trace: 'retain-on-failure'");
+  source = source.replace(
+    /trace:\s*['"][^'"]+['"]/,
+    "trace: { mode: 'retain-on-failure', screenshots: false, snapshots: true, sources: true, attachments: true }",
+  );
   await writeFile(configPath, source, 'utf8');
 }
 
+function withoutTraceScreenshots(source) {
+  const replacement = "trace: { mode: 'retain-on-failure', screenshots: false, snapshots: true, sources: true, attachments: true }";
+  const fromString = source.replace(/trace:\s*['"][^'"]+['"]/, replacement);
+  if (fromString !== source) return fromString;
+
+  return source.replace(/trace:\s*\{([\s\S]*?)\}/, (setting, body) => {
+    if (/screenshots\s*:/.test(body)) {
+      return setting.replace(/screenshots\s*:\s*(?:true|false)/, 'screenshots: false');
+    }
+    return setting.replace('{', '{ screenshots: false,');
+  });
+}
+
+async function disableTraceScreenshots(projectDir) {
+  const configPath = path.join(projectDir, 'playwright.config.ts');
+  if (!await exists(configPath)) return;
+
+  let source = await readFile(configPath, 'utf8');
+  const updated = withoutTraceScreenshots(source);
+
+  if (updated !== source) {
+    await writeFile(configPath, updated, 'utf8');
+  }
+}
+
 async function configureClaudeMcp(projectDir) {
   const file = path.join(projectDir, '.mcp.json');
   if (!await exists(file)) return;
@@ -149,6 +181,8 @@ async function initializePlaywright(projectDir, run) {
     await patchPlaywrightConfig(projectDir);
   }
 
+  await disableTraceScreenshots(projectDir);
+
   await run(projectDir, 'npx', ['--no-install', 'playwright', 'init-agents', '--loop=codex']);
   await run(projectDir, 'npx', ['--no-install', 'playwright', 'init-agents', '--loop=claude']);
   await mkdir(path.join(projectDir, '.playwright-evidence', 'mcp'), { recursive: true });