playwright-test-agent 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -12,7 +12,10 @@ Use Planner -> Generator -> Healer for durable tests. `AGENTS.md` or `CLAUDE.md`
|
|
|
12
12
|
1. Planner investigates the application and saves a Markdown plan under `specs/`.
|
|
13
13
|
2. The parent presents that plan and waits for user confirmation.
|
|
14
14
|
3. Generator validates the confirmed plan and writes executable tests.
|
|
15
|
-
4.
|
|
15
|
+
4. The parent executes the generated tests with `npx playwright test`.
|
|
16
|
+
5. If all generated tests pass, report the result. If any fail, Healer diagnoses, fixes justified test defects, and reruns the affected tests until they pass or a genuine blocker is identified.
|
|
17
|
+
|
|
18
|
+
Every phase is required in that order. Planner exploration without a saved plan is incomplete. Generator output without test files is incomplete. Generated test files that have not been executed are not a successful test result. Healer must never start before a real generated-test failure exists.
|
|
16
19
|
|
|
17
20
|
On Claude Code, the parent must start `playwright-test-planner`; only that Planner may open the website. The parent must not call `playwright-cli`, Chrome DevTools, browser MCP tools, or another browser integration. On Codex, the main agent does not spawn a Playwright subagent; it explicitly assumes one phase at a time, and may call CLI only while acting in that phase. Preserve the official Playwright Test MCP configuration and role tools; CLI is the preferred live-browser path and prevents missing MCP tools from blocking the workflow.
|
|
18
21
|
|
|
@@ -24,21 +27,23 @@ Keep credentials and tokens in environment variables or ignored secret files. Ne
|
|
|
24
27
|
|
|
25
28
|
## Planner
|
|
26
29
|
|
|
27
|
-
Planner—not the parent—opens the supplied URL with `playwright-cli open <deployed-url>`, investigates the application,
|
|
30
|
+
Planner—not the parent—opens the supplied URL with `playwright-cli open <deployed-url>`, investigates the application, converts its findings into a human-readable Markdown test plan, and saves it under `specs/`. Once the parent has supplied the objective, URL, access information, relevant project context, and constraints, Planner's first browser action must be that CLI command. It then uses compact `snapshot` or `find` output and refs for interaction. A seed is not a prerequisite and must not be located before exploration.
|
|
28
31
|
|
|
29
32
|
The plan contains prerequisites, test data, independent scenarios, steps, observable expected results, exclusions, and intended output files. Reconnaissance must not mutate durable/shared data or perform consequential actions unless authorized.
|
|
30
33
|
|
|
31
|
-
|
|
34
|
+
Planner returns the saved plan path and a scenario summary. The parent shows the plan and exclusions to the user. Generator starts only after the user explicitly confirms that plan.
|
|
32
35
|
|
|
33
36
|
## Generator
|
|
34
37
|
|
|
35
|
-
Generator starts only after confirmation and converts the confirmed plan into executable
|
|
38
|
+
Generator starts only after confirmation and converts the confirmed plan into executable Playwright test code. Generator—not the parent—uses `playwright-cli` to validate flows, locators, and assertions, writes the test files, and returns their paths. Generate only confirmed scenarios; prefer independent tests, semantic locators, explicit assertions, and environment-based secrets.
|
|
39
|
+
|
|
40
|
+
After Generator returns, the parent executes the generated test files with `npx playwright test`. This execution step is mandatory. If all tests pass, report the executed result. If any fail, pass the failed test names, failure output, confirmed plan, and generated file paths to Healer.
|
|
36
41
|
|
|
37
42
|
For API-only scenarios, use Playwright `APIRequestContext` directly and assert status, headers, schema, stable business invariants, and safe mutation cleanup.
|
|
38
43
|
|
|
39
44
|
## Healer
|
|
40
45
|
|
|
41
|
-
Use Healer only for failures. Healer—not the parent—
|
|
46
|
+
Use Healer only for real failures from executing generated tests. Healer—not the parent—reproduces the failing tests with `npx playwright test`, inspects the UI with `playwright-cli`, applies justified test-side changes, and reruns the affected tests within guardrails until they pass or a genuine blocker is identified.
|
|
42
47
|
|
|
43
48
|
Do not weaken assertions, add arbitrary sleeps, retry blindly, or skip merely to pass. If behavior conflicts with the confirmed plan, preserve evidence and ask whether it is a regression or intended change. Classify unresolved failures as application defect, test defect, environment/data problem, or product decision.
|
|
44
49
|
|
|
@@ -17,7 +17,7 @@ Use the installed \`playwright-cli\` command as the primary browser interface fo
|
|
|
17
17
|
|
|
18
18
|
Keep the official Playwright Test MCP configuration and tools generated for this role; do not remove or disable them. Prefer CLI for live page interaction and use it when those MCP tools are unavailable, so missing \`planner_*\`, \`generator_*\`, \`test_*\`, or \`browser_*\` tools must not block the phase. Do not call Chrome DevTools or an unrelated browser integration, and do not spawn a nested or same-role agent.
|
|
19
19
|
|
|
20
|
-
${role === 'planner' ? 'FAST START: you are the Planner. The parent may pass relevant local-project context and answers gathered before this phase. When the request contains the test objective, deployed URL, and any information required to access the target, your first browser action must be `playwright-cli open <url>`; do not call `planner_setup_page` or any other browser/MCP action before this CLI open. After the session is open, use compact CLI `snapshot`/`find` output for exploration; MCP tools remain available as an optional supplement. Do not wait for the parent agent to open a browser, and do not first locate or read a seed file, Playwright config, test directory, agent definition, source tree, fixtures, or existing tests. Ask only when the supplied information is still missing, incorrect, contradictory, or exploration reaches a real blocker such as required credentials, ambiguous expected behavior that changes assertions, or authorization for a consequential action.
|
|
20
|
+
${role === 'planner' ? 'FAST START: you are the Planner. The parent may pass relevant local-project context and answers gathered before this phase. When the request contains the test objective, deployed URL, and any information required to access the target, your first browser action must be `playwright-cli open <url>`; do not call `planner_setup_page` or any other browser/MCP action before this CLI open. After the session is open, use compact CLI `snapshot`/`find` output for exploration; MCP tools remain available as an optional supplement. Do not wait for the parent agent to open a browser, and do not first locate or read a seed file, Playwright config, test directory, agent definition, source tree, fixtures, or existing tests. Ask only when the supplied information is still missing, incorrect, contradictory, or exploration reaches a real blocker such as required credentials, ambiguous expected behavior that changes assertions, or authorization for a consequential action. Investigation is not the final output: you must turn the findings into a complete Markdown test plan and save it under `specs/` using filesystem tools or the available planner save tool. Return the saved plan path and scenario summary to the parent.' : ''}${role === 'generator' ? 'You are the Generator. Start only after the user confirms the saved test plan. Begin live validation with `playwright-cli open`/`attach` and use CLI snapshots/find to verify the confirmed plan; MCP setup and browser tools remain optional. Generate executable Playwright test code for the confirmed scenarios and write the test files using filesystem tools or the available generator write tool. Return the generated test file paths to the parent; generating code does not complete the workflow because the parent must execute the generated tests next.' : ''}${role === 'healer' ? 'You are the Healer. Start only after execution of the generated tests reports failures. Receive the failing test names and failure output, reproduce them with `npx playwright test`, begin UI diagnosis with `playwright-cli open`/`attach`, and use CLI snapshots/find to inspect the current UI; MCP tools remain optional. Diagnose and patch justified test defects, rerun the affected tests, and continue within the healer guardrails until they pass or a genuine application/environment/product blocker is identified. Return the final run result and classification to the parent.' : ''}
|
|
21
21
|
|
|
22
22
|
${ROLE_BLOCK_END}`;
|
|
23
23
|
const CODEX_INSTRUCTIONS = `${BLOCK_START}
|
|
@@ -29,7 +29,9 @@ Codex routing is mandatory for website, browser workflow, HTTP API, or applicati
|
|
|
29
29
|
2. Before Planner, the main agent may read relevant local project information and ask for information that materially affects the test, such as the deployed URL, account or role, expected behavior, environment, and authorization boundary. Do not perform a seed/config/generated-file audit or open the website during preparation.
|
|
30
30
|
3. Do not start Playwright subagents. Once the required context is available, the main agent explicitly assumes the Planner role in this session.
|
|
31
31
|
4. Planner's first live browser action is \`playwright-cli open <url>\`. Prefer CLI while preserving the generated Playwright Test MCP configuration and tools. Do not call Chrome DevTools or an unrelated browser integration.
|
|
32
|
-
5.
|
|
32
|
+
5. Planner must convert its investigation into a complete Markdown test plan under \`specs/\`. Show that saved plan to the user and wait for explicit confirmation; investigation alone is not completion.
|
|
33
|
+
6. After confirmation, assume Generator and generate executable Playwright test code for the confirmed plan. When generation finishes, execute the generated tests with \`npx playwright test\`; generating files alone is not completion.
|
|
34
|
+
7. If every generated test passes, report the result. If any generated test fails, assume Healer with the failed test names and output; diagnose, patch, and rerun until the tests pass or a genuine application/environment/product blocker is identified. Never skip execution and never report generated-but-unexecuted tests as passing.
|
|
33
35
|
|
|
34
36
|
Ask only for information that is necessary or could materially change scope, assertions, access, or safety. Do not guess or create nested agents.
|
|
35
37
|
|
|
@@ -43,8 +45,10 @@ Claude Code routing is mandatory for website, browser workflow, HTTP API, or app
|
|
|
43
45
|
1. Load \`.claude/skills/playwright-test-agent/SKILL.md\`.
|
|
44
46
|
2. Before starting Planner, the main agent may read relevant local project information and ask for information that materially affects the test, such as the deployed URL, account or role, expected behavior, environment, and authorization boundary. Do not perform a seed/config/generated-file audit.
|
|
45
47
|
3. Once the required context is available, directly start \`playwright-test-planner\` with the objective, deployed URL, supplied access information, relevant project context, and constraints. The main agent must not open or inspect the website itself and must not call \`playwright-cli\`, Chrome DevTools, browser MCP tools, or another browser integration.
|
|
46
|
-
4. Planner opens and investigates the website with \`playwright-cli\` and
|
|
47
|
-
5. Only after confirmation start \`playwright-test-generator
|
|
48
|
+
4. Planner opens and investigates the website with \`playwright-cli\`, converts its findings into a complete Markdown test plan under \`specs/\`, and returns the saved plan path. After Planner returns, show that plan to the user and wait for explicit confirmation; investigation alone is not completion.
|
|
49
|
+
5. Only after confirmation, directly start \`playwright-test-generator\` with the confirmed plan. Generator must validate the confirmed scenarios and write executable Playwright test files, then return their paths.
|
|
50
|
+
6. After Generator returns, the main agent must execute the generated tests with \`npx playwright test\`. Generating test files alone never completes the workflow.
|
|
51
|
+
7. If every generated test passes, report the result. If any generated test fails, directly start \`playwright-test-healer\` with the failed test names, failure output, confirmed plan, and generated file paths. Healer diagnoses and patches justified test defects and reruns the affected tests until they pass or it identifies a genuine application/environment/product blocker. Never report generated-but-unexecuted tests as passing.
|
|
48
52
|
|
|
49
53
|
Start each role directly and never create a nested or same-role intermediary. Do not locate a seed file or make setup-file discovery a prerequisite to Planner. If exploration finds missing, incorrect, or contradictory required information, let Planner pause and return the precise question; after the user answers, start or resume Planner with that answer and the blocking observation. Do not guess.
|
|
50
54
|
|