playwright-test-agent 0.1.1 → 0.1.2

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "playwright-test-agent",
-  "version": "0.1.1",
+  "version": "0.1.2",
   "description": "Initialize Playwright Test agents and install the playwright-test-agent skill.",
   "type": "module",
   "bin": {

package/playwright-test-agent/SKILL.md

@@ -5,7 +5,33 @@ description: Use first when a user asks to test a website, browser workflow, HTT
 
 # Playwright Test Agent
 
-Use Playwright Test for durable automation. Browser tests use Playwright's official planner, generator, and healer subagents. Pure HTTP API tests may be written directly with Playwright `APIRequestContext` without those subagents.
+Use Playwright Test for durable automation. The main agent orchestrates Playwright's official planner, generator, and healer subagents instead of replacing their work with ad hoc browser exploration. Pure HTTP API tests may be written directly with Playwright `APIRequestContext` without those subagents.
+
+This file is the single source of truth for the testing workflow. Project-level `AGENTS.md` and `CLAUDE.md` only force this skill to be loaded and name the platform-specific subagents; do not duplicate or infer the detailed workflow from those routing files.
+
+## Platform agent names
+
+| Role | Codex | Claude Code |
+|---|---|---|
+| Planner | `playwright_test_planner` | `playwright-test-planner` |
+| Generator | `playwright_test_generator` | `playwright-test-generator` |
+| Healer | `playwright_test_healer` | `playwright-test-healer` |
+
+Use the names for the active platform. The role names below refer to these mapped agents.
+
+## Agent responsibilities
+
+- **Main agent:** Own the user conversation, gather the minimum entry information, locate the default seed generated by `playwright init-agents`, dispatch each subagent with that seed, present the plan for confirmation, and report final results. It must not ask the user to provide a seed or investigate UI details itself when the planner can discover them.
+- **Planner:** Run the generated seed to establish page context, global setup, dependencies, fixtures, and hooks. Explore the deployed application for the requested scenarios and user flows, compare observations with the request and optional PRD, and save a precise human-readable Markdown plan under `specs/`. It does not generate the formal test files.
+- **Generator:** Read the complete user-confirmed Markdown plan and the same generated seed, execute each scenario to validate locators and assertions against the live application, and generate executable Playwright tests under the configured test directory. It must not invent unconfirmed scenarios or expected behavior.
+- **Healer:** Run the failing test, replay its steps, inspect the current UI, propose and apply test-side patches, and rerun until the test passes or a guardrail stops the loop. It must report a likely broken product, environment/data problem, or plan conflict instead of weakening assertions merely to obtain a pass.
+
+For UI testing, keep these responsibilities and sequence:
+
+```text
+request + deployed URL -> locate generated seed -> planner -> user confirms plan
+                       -> generator -> healer for failures -> final report
+```
 
 ## Initialize once per project
 
@@ -24,23 +50,29 @@ npx playwright init-agents --loop=claude
 
 It uses the project-local Playwright command. Never install a global/additional Playwright CLI or run `playwright init-skills`.
 
+`playwright init-agents` also creates the project's default `seed.spec.ts`. Its directory follows the selected Playwright project's `testDir`, so it may be under `playwright-tests/`, `e2e/`, `tests/`, or another configured directory. Do not hardcode the path. Locate it from `playwright.config.*`, the generated agent definitions, or the filesystem before invoking planner or generator.
+
 ## Understand the request
 
-Before testing, inspect the target project for relevant source, routes/API clients, environment examples, existing tests, and run instructions. The project may be empty or may contain no useful application context; in that case, do not keep searching the filesystem or invent missing details. Use the user's description and the live target as the available evidence.
+Before testing, briefly inspect the target project for relevant source, routes/API clients, environment examples, existing tests, fixtures, the generated seed, and run instructions. The project may otherwise be empty or contain no useful application context; in that case, do not keep searching the filesystem or invent missing details. Use the user's description, generated seed, and live target as the available evidence.
 
-Establish the objective, reachable environment/base URL, allowed side effects, authentication/roles, required test data/variables, and observable success criteria before invoking the planner. Ask the user for any material fact that cannot be verified from the project or target application.
+Invoke the Planner as soon as the user has supplied a UI test objective and a deployed base URL. For example, "test this project's login feature" plus its deployment URL is sufficient: the Planner, not the main agent, investigates the login page, fields, controls, navigation, and observable outcomes. Do not delay Planner dispatch by asking for UI facts it can safely discover from the live application.
 
-If reconnaissance shows that provided information is missing, invalid, stale, or contradictory—for example, the URL is unreachable, a page or control does not exist, credentials fail, the observed role differs, a required variable is absent, or the application behavior conflicts with the stated workflow—stop exploring. Report the exact mismatch without exposing secrets and ask the user to correct or complete the information. Do not repeatedly try nearby URLs, guess credentials, wander through unrelated pages, or invoke the planner while blocked. Resume reconnaissance and invoke the planner only after the user supplies enough corrected information.
+Pass the planner the exact objective, deployed URL, generated seed path, known authentication/roles, allowed side effects, supplied test data, and relevant project or PRD context. The seed is produced by initialization, not supplied by the user. It may remain the default page-context seed or be customized for project fixtures, global setup, dependencies, or authenticated context. Credentials are required before attempting an authenticated-only flow, but not merely to investigate a publicly reachable login page.
+
+If no seed can be located after initialization, treat the Playwright agent setup as incomplete. Check the configured test directory and generated agent definitions, then rerun `npx playwright-test-agent init` when regeneration is needed. Do not ask the user to author a seed, invent an arbitrary path, or continue to planner without the initialized project context.
+
+If planner reconnaissance shows that information is missing, invalid, stale, or contradictory—for example, the URL is unreachable, a page or control does not exist, credentials fail, the observed role differs, a required variable is absent, or application behavior conflicts with the request—stop that planning attempt. Report the exact mismatch without exposing secrets and ask the user to correct or complete it. Do not repeatedly try nearby URLs, guess credentials, wander through unrelated pages, or substitute the main agent's own exploration. Resume with the planner only after the user supplies enough corrected information.
 
 Never put credentials or tokens in plans, source, screenshots, reports, or chat output. Use environment variables or an ignored secret file.
 
 ## Plan and confirm
 
-Always save a plan to `specs/<feature>.plan.md` before generating or running tests. Include prerequisites, seed, test data, independent scenarios, steps or requests, expected observable results, and output test files.
+Always have the planner save its human-readable Markdown plan under `specs/` before invoking the generator. Include prerequisites, the generated seed path, test data, independent scenarios, steps or requests, expected observable results, and intended output test files.
 
-For UI work, invoke `playwright_test_planner` only after the required project/user information has been checked and no known mismatch is blocking exploration. Have it inspect the live application and save the plan. Reconnaissance must not create, delete, submit, purchase, message, or otherwise mutate durable/shared data without authorization.
+For UI work, invoke the Planner once the objective, deployed base URL, and generated seed path are available and no known mismatch already blocks access. Explicitly name the seed in its prompt and provide an optional PRD only when one is relevant. The Planner runs the seed to perform global setup, dependencies, fixtures, and hooks, then uses its ready page context to inspect the live application. Reconnaissance must not create, delete, submit, purchase, message, or otherwise mutate durable/shared data without authorization.
 
-If the planner discovers a new missing or incorrect prerequisite, stop that planning attempt, summarize what was observed, and ask the user for clarification. Continue with the planner only after the prerequisite is resolved; do not let the planner keep exploring around the missing information.
+If the planner discovers a new missing or incorrect prerequisite, the main agent stops that planning attempt, summarizes what was observed, and asks the user for clarification. Continue or reinvoke the planner only after the prerequisite is resolved; do not let it keep exploring around missing information.
 
 Choose relevant positive, negative, empty, invalid, boundary, permission, persistence, and error scenarios. For login, normally consider valid credentials, empty username, empty password, both empty, wrong password, and unknown user; add MFA, lockout, recovery, or remember-me only when in scope.
 
@@ -48,11 +80,13 @@ Show the complete scenario list and exclusions to the user. Ask whether it is co
 
 ## Generate and run
 
-For confirmed UI scenarios, invoke `playwright_test_generator` once for the complete confirmed plan (or the complete selected set of scenarios), passing the exact plan content, seed, and destination under `playwright-tests/`. The generator processes the scenarios in plan order in that single invocation; its internal per-scenario setup does not mean starting a new generator agent for each scenario. Require one independent test per file, semantic locators, an assertion for every expected result, and environment-based secrets.
+For confirmed UI scenarios, invoke the Generator once for the complete confirmed plan (or complete selected scenario set), explicitly naming the Markdown plan and the same generated seed; use the configured Playwright test directory for generated tests. The Generator executes scenarios against the live application to validate locators and assertions while generating. Its internal per-scenario setup does not mean starting a new Generator agent for every scenario. Require one independent test per file, semantic locators, an assertion for every expected result, and environment-based secrets.
 
 For confirmed API-only scenarios, write `APIRequestContext` tests directly under `playwright-tests/`. Assert status, headers, schema, and stable business invariants; define safe setup/cleanup for mutations.
 
-Run the generated tests with the project-local command. On failure, use `playwright_test_healer` one failure at a time for UI tests; API failures may be diagnosed directly. If observed product behavior conflicts with the confirmed plan, ask whether it is a regression or intended change. Do not weaken assertions, add sleeps/`networkidle`, retry blindly, or skip/fixme tests merely to get green.
+After generation, invoke the Healer with one failing UI test name at a time. The Healer runs the test, replays the failing steps, inspects the current UI for the equivalent element or flow, proposes a patch such as a locator, synchronization, or test-data repair, and reruns until it passes or a guardrail stops the loop. API failures may be diagnosed directly.
+
+If observed product behavior conflicts with the confirmed plan, report the conflict and ask whether it is a regression or intended change. Do not weaken assertions, add arbitrary sleeps/`networkidle`, retry blindly, or skip/fixme tests merely to get green. If the healer concludes the feature is broken, preserve the failure evidence and report it as an application defect rather than treating a skipped test as success.
 
 ## Preserve evidence
 

package/playwright-test-agent/scripts/init-playwright.mjs

@@ -7,10 +7,24 @@ import { fileURLToPath } from 'node:url';
 
 const BLOCK_START = '<!-- playwright-test-agent:start -->';
 const BLOCK_END = '<!-- playwright-test-agent:end -->';
-const PROJECT_INSTRUCTIONS = `${BLOCK_START}
+const CODEX_INSTRUCTIONS = `${BLOCK_START}
 ## Playwright Test Agent
 
-When asked to test a website, browser workflow, HTTP API, or application feature, load and follow the project-installed \`playwright-test-agent\` skill first. For UI tests, use its Playwright Test planner, generator, and healer workflow instead of ad hoc browser automation. Keep its confirmation, credential, side-effect, and evidence rules in force.
+For any website, browser workflow, HTTP API, or application-feature testing request, load and follow \`.agents/skills/playwright-test-agent/SKILL.md\` before planning or using browser tools. This skill has priority over ad hoc browser automation.
+
+For UI testing, the Codex subagents are \`playwright_test_planner\`, \`playwright_test_generator\`, and \`playwright_test_healer\`. When the user provides a UI test objective and deployed URL, the main agent must start \`playwright_test_planner\` to investigate the live page; it must not replace Planner work with its own ad hoc browser exploration.
+
+If Planner reports that required test information is missing, invalid, or inconsistent, stop that planning attempt and ask the user for the specific missing information. After the user responds, start \`playwright_test_planner\` again with the corrected information and the previous blocking observation. Do not continue guessing or exploring around the blocker.
+${BLOCK_END}`;
+
+const CLAUDE_INSTRUCTIONS = `${BLOCK_START}
+## Playwright Test Agent
+
+For any website, browser workflow, HTTP API, or application-feature testing request, load and follow \`.claude/skills/playwright-test-agent/SKILL.md\` before planning or using browser tools. This skill has priority over ad hoc browser automation.
+
+For UI testing, the Claude Code subagents are \`playwright-test-planner\`, \`playwright-test-generator\`, and \`playwright-test-healer\`. When the user provides a UI test objective and deployed URL, the main agent must start \`playwright-test-planner\` to investigate the live page; it must not replace Planner work with its own ad hoc browser exploration.
+
+If Planner reports that required test information is missing, invalid, or inconsistent, stop that planning attempt and ask the user for the specific missing information. After the user responds, start \`playwright-test-planner\` again with the corrected information and the previous blocking observation. Do not continue guessing or exploring around the blocker.
 ${BLOCK_END}`;
 
 const exists = async (file) => {
@@ -35,7 +49,7 @@ const defaultRun = (projectDir, command, args) => new Promise((resolve, reject)
     : reject(new Error(`${command} ${args.join(' ')} exited with ${code}`)));
 });
 
-function withManagedBlock(source) {
+function withManagedBlock(source, managedBlock) {
   const start = source.indexOf(BLOCK_START);
   const end = source.indexOf(BLOCK_END);
 
@@ -44,12 +58,12 @@ function withManagedBlock(source) {
   }
 
   if (start !== -1) {
-    return source.slice(0, start) + PROJECT_INSTRUCTIONS + source.slice(end + BLOCK_END.length);
+    return source.slice(0, start) + managedBlock + source.slice(end + BLOCK_END.length);
   }
 
-  if (source.length === 0) return `${PROJECT_INSTRUCTIONS}\n`;
+  if (source.length === 0) return `${managedBlock}\n`;
   const separator = source.endsWith('\n') ? '\n' : '\n\n';
-  return `${source}${separator}${PROJECT_INSTRUCTIONS}\n`;
+  return `${source}${separator}${managedBlock}\n`;
 }
 
 async function installSkill(projectDir, skillSourceDir) {
@@ -61,10 +75,14 @@ async function installSkill(projectDir, skillSourceDir) {
 }
 
 async function updateInstructionFiles(projectDir) {
-  for (const name of ['AGENTS.md', 'CLAUDE.md']) {
+  const files = [
+    ['AGENTS.md', CODEX_INSTRUCTIONS],
+    ['CLAUDE.md', CLAUDE_INSTRUCTIONS],
+  ];
+  for (const [name, managedBlock] of files) {
     const file = path.join(projectDir, name);
     const source = await exists(file) ? await readFile(file, 'utf8') : '';
-    await writeFile(file, withManagedBlock(source), 'utf8');
+    await writeFile(file, withManagedBlock(source, managedBlock), 'utf8');
   }
 }