@bastani/atomic 0.9.7-alpha.1 → 0.9.8-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +31 -0
- package/dist/builtin/cursor/CHANGELOG.md +12 -0
- package/dist/builtin/cursor/package.json +2 -2
- package/dist/builtin/intercom/CHANGELOG.md +16 -0
- package/dist/builtin/intercom/package.json +1 -1
- package/dist/builtin/mcp/CHANGELOG.md +12 -0
- package/dist/builtin/mcp/package.json +1 -1
- package/dist/builtin/subagents/CHANGELOG.md +13 -0
- package/dist/builtin/subagents/package.json +1 -1
- package/dist/builtin/subagents/skills/{effective-liteparse → liteparse}/SKILL.md +4 -4
- package/dist/builtin/subagents/skills/playwright-cli/SKILL.md +21 -5
- package/dist/builtin/subagents/skills/playwright-cli/references/test-generation.md +311 -12
- package/dist/builtin/web-access/CHANGELOG.md +12 -0
- package/dist/builtin/web-access/package.json +1 -1
- package/dist/builtin/workflows/CHANGELOG.md +32 -0
- package/dist/builtin/workflows/README.md +2 -2
- package/dist/builtin/workflows/builtin/goal-prompts.ts +9 -7
- package/dist/builtin/workflows/builtin/goal-reducer.ts +9 -24
- package/dist/builtin/workflows/builtin/goal-review.ts +17 -36
- package/dist/builtin/workflows/builtin/ralph-core.ts +1 -3
- package/dist/builtin/workflows/builtin/ralph-review-gate.ts +31 -53
- package/dist/builtin/workflows/builtin/ralph-reviewer-prompt.ts +10 -4
- package/dist/builtin/workflows/builtin/shared-prompts.ts +5 -4
- package/dist/builtin/workflows/package.json +1 -1
- package/dist/builtin/workflows/skills/impeccable/reference/hooks.md +2 -2
- package/dist/builtin/workflows/skills/impeccable/reference/init.md +37 -10
- package/dist/builtin/workflows/skills/impeccable/scripts/context-signals.mjs +1 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/context.mjs +2 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/critique-storage.mjs +2 -2
- package/dist/builtin/workflows/skills/impeccable/scripts/detector/detect-antipatterns-browser.js +20 -4
- package/dist/builtin/workflows/skills/impeccable/scripts/detector/engines/regex/detect-text.mjs +3 -6
- package/dist/builtin/workflows/skills/impeccable/scripts/detector/rules/checks.mjs +2 -4
- package/dist/builtin/workflows/skills/impeccable/scripts/detector/shared/page.mjs +31 -8
- package/dist/builtin/workflows/skills/impeccable/scripts/hook-admin.mjs +7 -6
- package/dist/builtin/workflows/skills/impeccable/scripts/hook-lib.mjs +11 -10
- package/dist/builtin/workflows/skills/impeccable/scripts/lib/impeccable-paths.mjs +1 -0
- package/dist/builtin/workflows/skills/impeccable/scripts/lib/is-generated.mjs +1 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/lib/provider.mjs +4 -0
- package/dist/builtin/workflows/skills/impeccable/scripts/live/browser-script-parts.mjs +2 -1
- package/dist/builtin/workflows/skills/impeccable/scripts/live/svelte-component.mjs +38 -16
- package/dist/builtin/workflows/skills/impeccable/scripts/live-accept.mjs +33 -19
- package/dist/builtin/workflows/skills/impeccable/scripts/live-browser.js +9 -9
- package/dist/builtin/workflows/skills/impeccable/scripts/live-server.mjs +2 -0
- package/dist/builtin/workflows/skills/impeccable/scripts/pin.mjs +16 -9
- package/dist/builtin/workflows/src/tui/overlay-adapter.ts +59 -14
- package/docs/quickstart.md +1 -1
- package/docs/workflows.md +4 -4
- package/npm-shrinkwrap.json +23 -23
- package/package.json +2 -2
- package/dist/builtin/subagents/skills/playwright-cli/references/spec-driven-testing.md +0 -305
- /package/dist/builtin/subagents/skills/{effective-liteparse → liteparse}/scripts/search.py +0 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,37 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [0.9.8-alpha.1] - 2026-07-12
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- Changed builtin `goal` and `ralph` reviewer approval to be deterministic on the reviewer's self-reported `stop_review_loop` boolean: a reviewer approves exactly when it returns `stop_review_loop=true` with no `reviewer_error`, parse failures still count as non-approval, and Goal's reducer completes on quorum of those booleans without recomputing approval from findings arrays, priorities, or `requirements_traceability` statuses. Recomputing approval from those arrays could deadlock runs whose acceptance criteria referenced the review process itself (for example "three reviewers approve" or "an unmerged PR is created"): no individual reviewer can prove such clauses, so traceability never became fully `proven` even when every reviewer explicitly approved, and the loop burned worker/review turns until `needs_human`. Reviewer prompts now state that the boolean is the single authoritative convergence signal, spell out how to derive it (blocking P0/P1/P2 findings and `required_by_objective` findings at any priority mean `false`; in-scope P3 nice-to-haves, `beyond_objective`/`contradicts_objective` observations, the reviewer-quorum process itself, and the authorized post-approval PR/MR/review final action must never hold it at `false`), and keep findings/traceability as required audit evidence.
|
|
10
|
+
- Synchronized the builtin `playwright-cli` skill with microsoft/playwright-cli at `793cfb32572733cbcb401e6f28d05a7a914ce408`, including current installation, snapshot search, mobile emulation, and test generation guidance.
|
|
11
|
+
- Renamed the builtin `effective-liteparse` skill to `liteparse` and synchronized it with run-llama/llamaparse-agent-skills at `2dcef7c62417bd2ec4671fce4621bb1e8cce48d0`; existing `/skill:effective-liteparse` references must migrate to `/skill:liteparse`.
|
|
12
|
+
- Synchronized the complete builtin `impeccable` skill tree with pbakaus/impeccable at `630fc2682a5bd39b25a8e61f74b6b3f14f2b1e21`, including its latest references, detector libraries, live-review scripts, and provider integrations.
|
|
13
|
+
|
|
14
|
+
### Fixed
|
|
15
|
+
|
|
16
|
+
- Disabled terminal autowrap while the fullscreen workflow graph overlay is visible on Windows and restored the previous terminal mode when the overlay closes, preventing wrapped graph rows and stale terminal state ([#1760](https://github.com/bastani-inc/atomic/issues/1760)).
|
|
17
|
+
- Hardened synced Impeccable HTML filtering and preview selector escaping against nested sanitizer inputs, permissive script/style closing tags, HTML comment end-bang syntax, and backslash-containing session identifiers; also removed an ineffective CSS property replacement flagged by CodeQL.
|
|
18
|
+
|
|
19
|
+
## [0.9.7] - 2026-07-12
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- Added a shared convergence contract to the bundled `goal` and `ralph` workflows: implementation starts from an observable acceptance/contract matrix derived from the literal objective/acceptance criteria (with explicit state/transition/invariant modeling for stateful work), reviewers independently derive adversarial checks from the literal contract before relying on worker receipts or worker-authored tests, reproduced findings require durable regression evidence before they count as resolved, and each review round persists a deduplicated cross-reviewer `consolidated_findings` batch that the next worker turn repairs together instead of one finding per turn. Literal-contract scope controls are preserved throughout, so nothing beyond the user's requirements is forced.
|
|
24
|
+
- Added a "Choosing an Execution Shape" section to `docs/workflows.md`: an agent-facing decision ladder covering inline work, inline subagent delegation, direct one-off `task`/`tasks`/`chain` shapes, named/builtin workflows, custom TypeScript workflows, and composed/nested workflows; a six-dimension scoring rubric (structure, verifiability, iteration, risk, duration, isolation) with hard-signal overrides; a "Task queues and software factories" playbook for fire-and-forget requests like "address all open issues" (enumerate and dependency-classify first, fan out independent items as separate per-item workflow runs in bounded waves with per-item worktrees/PRs, compose dependent items into one parent graph that nests proven children, and mix both for clustered queues); and a "Prompting the choice" guide listing the user prompt levers (naming the shape, acceptance criteria, loop wording, evidence requests, scope boundaries, and queue policy) that steer the agent's execution-shape decision.
|
|
25
|
+
- Added a "Context-Mode-Aware Prompt Text" section to `docs/workflows.md` documenting that stage prompts must not describe their own context mode, fresh stages must not reference invisible context (prior conversation, sibling stages, graph topology), and forked continuation prompts should send only the delta with a pointer back to guidance already established in the forked history.
|
|
26
|
+
|
|
27
|
+
### Changed
|
|
28
|
+
|
|
29
|
+
- Accelerated PR/main CI by running platform-independent validation once on Linux while retaining installed-package Node integration and release-archive smoke coverage on Linux and Windows, and by reusing caller-installed dependencies and package builds during binary assembly. Test suites now have one bounded, observable flake-recovery attempt with preserved logs, environment/resource diagnostics, CI annotations, and no retries for deterministic workflow/release/package/publish gates. Release publication no longer reruns the full PR suite: a protected-default-branch integrity gate proves the release commit is generated from a parent already integrated into `main`, contains exactly the expected version and shrinkwrap material, and pins that immutable SHA across release jobs before preserving all release-specific metadata, docs, native, binary, package, and npm provenance checks.
|
|
30
|
+
- Changed bundled `goal` completion to evidence closure rather than reviewer agreement alone: reviewer quorum can only complete the run when no objective-relevant blocking finding from any reviewer in the current round remains unresolved, unresolved findings are recorded in the inspectable reducer decision reason, and the bounded loop still stops at `max_turns` as `needs_human`. Severity labels alone no longer dismiss objective-relevant findings in Goal or Ralph: `required_by_objective` findings block at any priority (P3 included), while `consistent_with_objective` P3 nice-to-haves stay non-blocking.
|
|
31
|
+
|
|
32
|
+
### Fixed
|
|
33
|
+
|
|
34
|
+
- Fixed retryable-failure classification across main-chat retry/fallback, workflow stage fallback, and subagent fallback to treat provider usage-limit exhaustion (for example `Codex error: The usage limit has been reached`, plus `usage_limit_reached`/`insufficient_quota`-style codes) as a retryable quota/rate-limit failure, so configured `fallbackModels` advance to the next candidate provider/model instead of dead-ending the turn, stage, or run. Provider messages that flatten the token into free text (for example `usage_limit_reached` or `usage-limit`, matched with space/underscore/hyphen/joined separators) classify the same as the structured codes across all three paths. Nested cause/diagnostic and session-shaped error payloads classify the same way; cancellations, safety refusals, task/tool failures, and unrelated errors remain non-retryable.
|
|
35
|
+
|
|
5
36
|
## [0.9.7-alpha.1] - 2026-07-12
|
|
6
37
|
|
|
7
38
|
### Added
|
|
@@ -2,6 +2,18 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [0.9.8-alpha.1] - 2026-07-12
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- Published a synchronized Atomic 0.9.8-alpha.1 prerelease for the Cursor provider package; no functional Cursor provider changes were made after 0.9.7.
|
|
10
|
+
|
|
11
|
+
## [0.9.7] - 2026-07-12
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
|
|
15
|
+
- Published the stable Atomic 0.9.7 release for the Cursor provider package; no functional Cursor provider changes were made after 0.9.6.
|
|
16
|
+
|
|
5
17
|
## [0.9.7-alpha.1] - 2026-07-12
|
|
6
18
|
|
|
7
19
|
### Changed
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bastani/cursor",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.8-alpha.1",
|
|
4
4
|
"private": true,
|
|
5
5
|
"description": "Experimental first-party Atomic extension for Cursor OAuth, model discovery, and streaming provider registration.",
|
|
6
6
|
"contributors": [
|
|
@@ -40,7 +40,7 @@
|
|
|
40
40
|
}
|
|
41
41
|
},
|
|
42
42
|
"dependencies": {
|
|
43
|
-
"@bastani/atomic-natives": "0.9.
|
|
43
|
+
"@bastani/atomic-natives": "0.9.8-alpha.1",
|
|
44
44
|
"@bufbuild/protobuf": "^2.12.1",
|
|
45
45
|
"@earendil-works/pi-ai": "^0.80.6"
|
|
46
46
|
}
|
|
@@ -4,6 +4,22 @@ All notable changes to the `pi-intercom` extension will be documented in this fi
|
|
|
4
4
|
|
|
5
5
|
## [Unreleased]
|
|
6
6
|
|
|
7
|
+
## [0.9.8-alpha.1] - 2026-07-12
|
|
8
|
+
|
|
9
|
+
### Changed
|
|
10
|
+
|
|
11
|
+
- Published a synchronized Atomic 0.9.8-alpha.1 prerelease for the intercom extension; no functional intercom changes were made after 0.9.7.
|
|
12
|
+
|
|
13
|
+
## [0.9.7] - 2026-07-12
|
|
14
|
+
|
|
15
|
+
### Fixed
|
|
16
|
+
|
|
17
|
+
- Made blocking reply waits race-safe under concurrent tool calls. Waiter admission is now an atomic synchronous check-and-reserve shared by `intercom` `ask` and `contact_supervisor`: when several blocking requests race (parallel tool calls in one turn, same-tool or cross-tool), exactly one wins the reservation and every other call returns a normal structured "Already waiting for a reply" tool error. Previously the loser received an already-rejected promise that could sit unhandled while the winner's question was still being sent, crashing the whole agent process with an unhandled `Error: Already waiting for a reply` rejection.
|
|
18
|
+
- Scoped blocking-ask cleanup to the owning call: cancellation, send failures, and delivery errors now settle only that call's own reply waiter instead of rejecting whichever waiter happened to be pending, so a losing or failing concurrent request can no longer tear down another call's in-flight ask. The reply-wait timeout also rejects only its own waiter, threaded replies still resolve the exact winning request, and session shutdown/replacement and broker disconnects continue to reject the currently pending waiter.
|
|
19
|
+
- Made the pending reply-wait promise unhandled-rejection-proof: rejections that fire between the owner's awaits (for example a delivery failure racing dispatch) are pre-handled, so blocking coordination can never terminate the process.
|
|
20
|
+
- Initialized the lazy Intercom runtime from the most recent turn/tool/model lifecycle context when a subagent result or control relay arrives in a session that never emitted `session_start` to extensions (for example non-interactive in-process child sessions). Such sessions now deliver self-addressed subagent result announcements locally instead of failing every relay.
|
|
21
|
+
- Stopped recording misleading `intercom_result_error`/`intercom_control_error` "Intercom shutting down" entries when a relay fires while the runtime is uninitialized and the target is not the local session. The relay now acknowledges the message as undelivered (callers fall back to inline results) without attempting a broker connection that can only fail.
|
|
22
|
+
|
|
7
23
|
## [0.9.7-alpha.1] - 2026-07-12
|
|
8
24
|
|
|
9
25
|
### Fixed
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bastani/intercom",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.8-alpha.1",
|
|
4
4
|
"private": true,
|
|
5
5
|
"description": "Atomic extension providing a private coordination channel between parent and child agent sessions. Fork of: https://github.com/nicobailon/pi-intercom",
|
|
6
6
|
"contributors": [
|
|
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.9.8-alpha.1] - 2026-07-12
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- Published a synchronized Atomic 0.9.8-alpha.1 prerelease for the MCP extension; no functional MCP changes were made after 0.9.7.
|
|
15
|
+
|
|
16
|
+
## [0.9.7] - 2026-07-12
|
|
17
|
+
|
|
18
|
+
### Changed
|
|
19
|
+
|
|
20
|
+
- Published the stable Atomic 0.9.7 release for the MCP extension; no functional MCP changes were made after 0.9.6.
|
|
21
|
+
|
|
10
22
|
## [0.9.7-alpha.1] - 2026-07-12
|
|
11
23
|
|
|
12
24
|
### Changed
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bastani/mcp",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.8-alpha.1",
|
|
4
4
|
"private": true,
|
|
5
5
|
"description": "Atomic extension that adapts MCP (Model Context Protocol) servers into the coding agent. Fork of: https://github.com/nicobailon/pi-mcp-adapter",
|
|
6
6
|
"contributors": [
|
|
@@ -2,6 +2,19 @@
|
|
|
2
2
|
|
|
3
3
|
## [Unreleased]
|
|
4
4
|
|
|
5
|
+
## [0.9.8-alpha.1] - 2026-07-12
|
|
6
|
+
|
|
7
|
+
### Changed
|
|
8
|
+
|
|
9
|
+
- Synchronized the builtin `playwright-cli` skill with microsoft/playwright-cli at `793cfb32572733cbcb401e6f28d05a7a914ce408`, including current installation, snapshot search, mobile emulation, and test generation guidance.
|
|
10
|
+
- Renamed the builtin `effective-liteparse` skill to `liteparse` and synchronized it with run-llama/llamaparse-agent-skills at `2dcef7c62417bd2ec4671fce4621bb1e8cce48d0`; existing `/skill:effective-liteparse` references must migrate to `/skill:liteparse`.
|
|
11
|
+
|
|
12
|
+
## [0.9.7] - 2026-07-12
|
|
13
|
+
|
|
14
|
+
### Fixed
|
|
15
|
+
|
|
16
|
+
- Fixed subagent model fallback to classify provider usage-limit exhaustion (for example `Codex error: The usage limit has been reached`, plus `usage_limit`/`usage_limit_reached`/`usage_limit_exceeded`/`insufficient_quota` codes) as a retryable quota/rate-limit failure, so configured `fallbackModels` advance to the next candidate provider/model instead of failing the run. The message matcher tolerates the same space/underscore/hyphen/joined separators as the code path, so provider errors that flatten the token into free text (for example `usage_limit_reached` or `usage-limit`) also advance the chain. Nested cause/diagnostic and session-shaped errors classify the same way, while cancellations, safety refusals, task/tool failures, and unrelated errors remain non-retryable; the shared conformance corpus keeps this rule in lockstep with the workflows classifier.
|
|
17
|
+
|
|
5
18
|
## [0.9.7-alpha.1] - 2026-07-12
|
|
6
19
|
|
|
7
20
|
### Fixed
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bastani/subagents",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.8-alpha.1",
|
|
4
4
|
"private": true,
|
|
5
5
|
"description": "Atomic extension for delegating tasks to subagents with chains, parallel execution, and background runs. Fork of: https://github.com/nicobailon/pi-subagents",
|
|
6
6
|
"contributors": [
|
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
---
|
|
2
|
-
name:
|
|
2
|
+
name: liteparse
|
|
3
3
|
description: Use this skill whenever a task involves a document file (PDF, DOCX, PPTX, XLSX, or image) and you need to read it or pull text, tables, or specific values out of it — to answer a question about its contents, look up a figure, or extract data. Provides fast, local, model-free extraction via the `lit` CLI with disciplined, low-cost search patterns.
|
|
4
4
|
compatibility: Requires Node 18+ and `@llamaindex/liteparse` (`npm i -g @llamaindex/liteparse`, verify `lit --version`). LibreOffice for Office files; ImageMagick for images. The bundled search.py helper needs `uv`.
|
|
5
5
|
license: MIT
|
|
6
6
|
metadata:
|
|
7
7
|
author: LlamaIndex
|
|
8
|
-
version: "1.0.
|
|
8
|
+
version: "1.0.1"
|
|
9
9
|
---
|
|
10
10
|
|
|
11
|
-
#
|
|
11
|
+
# LiteParse
|
|
12
12
|
|
|
13
13
|
Extract text from documents locally with the `lit` CLI — a fast, model-free parser. This skill is
|
|
14
14
|
about using it **cheaply**: each `lit parse` re-runs full extraction, and every line you dump into
|
|
@@ -68,7 +68,7 @@ one turn at a time. Run the bundled BM25 ranker ONCE to surface the most relevan
|
|
|
68
68
|
single command:
|
|
69
69
|
|
|
70
70
|
```bash
|
|
71
|
-
|
|
71
|
+
scripts/search.py /tmp/doc.txt -q "materiality assessment priority topics" -k 8 -e 5
|
|
72
72
|
```
|
|
73
73
|
|
|
74
74
|
`-k` = number of matches, `-e` = lines of context around each (so the window comes back inline — no
|
|
@@ -47,6 +47,11 @@ playwright-cli upload ./document.pdf
|
|
|
47
47
|
playwright-cli check e12
|
|
48
48
|
playwright-cli uncheck e12
|
|
49
49
|
playwright-cli snapshot
|
|
50
|
+
# search the snapshot for text or a regexp, returns matching nodes with surrounding context
|
|
51
|
+
playwright-cli find "Sign in"
|
|
52
|
+
playwright-cli find --regex "Sign (in|up)"
|
|
53
|
+
# wrap the regexp in slashes to add flags, e.g. /i for case-insensitive
|
|
54
|
+
playwright-cli find --regex "/sign (in|up)/i"
|
|
50
55
|
playwright-cli eval "document.title"
|
|
51
56
|
playwright-cli eval "el => el.textContent" e5
|
|
52
57
|
# get element id, class, or any attribute not visible in the snapshot
|
|
@@ -93,6 +98,7 @@ playwright-cli mousewheel 0 100
|
|
|
93
98
|
playwright-cli screenshot
|
|
94
99
|
playwright-cli screenshot e5
|
|
95
100
|
playwright-cli screenshot --filename=page.png
|
|
101
|
+
playwright-cli screenshot --hires
|
|
96
102
|
playwright-cli pdf --filename=page.pdf
|
|
97
103
|
```
|
|
98
104
|
|
|
@@ -209,6 +215,12 @@ playwright-cli open --browser=firefox
|
|
|
209
215
|
playwright-cli open --browser=webkit
|
|
210
216
|
playwright-cli open --browser=msedge
|
|
211
217
|
|
|
218
|
+
# Emulate a generic mobile device (Pixel 10 for Chromium, iPhone 17 for WebKit).
|
|
219
|
+
# Prefer this when a mobile layout is acceptable: mobile pages are usually
|
|
220
|
+
# lighter, so snapshots are smaller and cheaper.
|
|
221
|
+
playwright-cli open --mobile
|
|
222
|
+
playwright-cli open --device="iPhone 15"
|
|
223
|
+
|
|
212
224
|
# Use persistent profile (by default profile is in-memory)
|
|
213
225
|
playwright-cli open --persistent
|
|
214
226
|
# Use persistent profile with custom directory
|
|
@@ -278,6 +290,11 @@ playwright-cli snapshot e34
|
|
|
278
290
|
|
|
279
291
|
# include each element's bounding box as [box=x,y,width,height]
|
|
280
292
|
playwright-cli snapshot --boxes
|
|
293
|
+
|
|
294
|
+
# search a large snapshot instead of capturing it all — returns matching nodes
|
|
295
|
+
# with 3 lines of context around each match (like grep -C)
|
|
296
|
+
playwright-cli find "Add to cart"
|
|
297
|
+
playwright-cli find --regex "\\$[0-9]+\\.[0-9]{2}"
|
|
281
298
|
```
|
|
282
299
|
|
|
283
300
|
## Targeting elements
|
|
@@ -325,13 +342,13 @@ playwright-cli kill-all
|
|
|
325
342
|
|
|
326
343
|
## Installation
|
|
327
344
|
|
|
328
|
-
If global `playwright-cli` command is not available, try a local version via `npx playwright
|
|
345
|
+
If global `playwright-cli` command is not available, try a local version via `npx playwright cli`:
|
|
329
346
|
|
|
330
347
|
```bash
|
|
331
|
-
npx --no-install playwright
|
|
348
|
+
npx --no-install playwright --version
|
|
332
349
|
```
|
|
333
350
|
|
|
334
|
-
When local version is available, use `npx playwright
|
|
351
|
+
When local version is available, use `npx playwright cli` in all commands. Otherwise, install `playwright-cli` as a global command:
|
|
335
352
|
|
|
336
353
|
```bash
|
|
337
354
|
npm install -g @playwright/cli@latest
|
|
@@ -396,9 +413,8 @@ playwright-cli show --annotate
|
|
|
396
413
|
* **Request mocking** [references/request-mocking.md](references/request-mocking.md)
|
|
397
414
|
* **Running Playwright code** [references/running-code.md](references/running-code.md)
|
|
398
415
|
* **Browser session management** [references/session-management.md](references/session-management.md)
|
|
399
|
-
* **Spec-driven testing (plan / generate / heal)** [references/spec-driven-testing.md](references/spec-driven-testing.md)
|
|
400
416
|
* **Storage state (cookies, localStorage)** [references/storage-state.md](references/storage-state.md)
|
|
401
|
-
* **Test generation** [references/test-generation.md](references/test-generation.md)
|
|
417
|
+
* **Test generation (plan / generate / heal)** [references/test-generation.md](references/test-generation.md)
|
|
402
418
|
* **Tracing** [references/tracing.md](references/tracing.md)
|
|
403
419
|
* **Video recording** [references/video-recording.md](references/video-recording.md)
|
|
404
420
|
* **Inspecting element attributes** [references/element-attributes.md](references/element-attributes.md)
|
|
@@ -1,13 +1,19 @@
|
|
|
1
|
-
# Test
|
|
1
|
+
# Test generation (plan → generate → heal)
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
End-to-end workflow for authoring and maintaining Playwright tests with `playwright-cli`. Every `playwright-cli` action emits the equivalent Playwright TypeScript, and that generated code is the raw material for every test. The sections below can be used independently:
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
- **How generation works** — the core mechanic everything else relies on: actions become TypeScript, plus how to add assertions.
|
|
6
|
+
- **Plan** — explore the app, produce a spec file describing what to test.
|
|
7
|
+
- **Generate** — turn a spec into Playwright test files. Update the spec if it's vague or stale.
|
|
8
|
+
- **Heal** — diagnose failing tests, fix the code, reconcile the spec with reality.
|
|
6
9
|
|
|
7
|
-
|
|
8
|
-
This code appears in the output and can be copied directly into your test files.
|
|
10
|
+
Plan / generate / heal lean on the same mechanic: run `npx playwright test --debug=cli` in the background, then `playwright-cli attach tw-XXXX` to drive the paused page interactively. See [playwright-tests.md](playwright-tests.md) for the debug/attach mechanics.
|
|
9
11
|
|
|
10
|
-
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## 0. How generation works
|
|
15
|
+
|
|
16
|
+
Every action you perform with `playwright-cli` generates corresponding Playwright TypeScript code. This code appears in the output and can be copied directly into your test files.
|
|
11
17
|
|
|
12
18
|
```bash
|
|
13
19
|
# Start a session
|
|
@@ -31,7 +37,7 @@ playwright-cli click e3
|
|
|
31
37
|
# await page.getByRole('button', { name: 'Sign In' }).click();
|
|
32
38
|
```
|
|
33
39
|
|
|
34
|
-
|
|
40
|
+
### Building a test file
|
|
35
41
|
|
|
36
42
|
Collect the generated code into a Playwright test:
|
|
37
43
|
|
|
@@ -50,9 +56,7 @@ test('login flow', async ({ page }) => {
|
|
|
50
56
|
});
|
|
51
57
|
```
|
|
52
58
|
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
### 1. Use Semantic Locators
|
|
59
|
+
### Use semantic locators
|
|
56
60
|
|
|
57
61
|
The generated code uses role-based locators when possible, which are more resilient:
|
|
58
62
|
|
|
@@ -64,7 +68,7 @@ await page.getByRole('button', { name: 'Submit' }).click();
|
|
|
64
68
|
await page.locator('#submit-btn').click();
|
|
65
69
|
```
|
|
66
70
|
|
|
67
|
-
###
|
|
71
|
+
### Explore before recording
|
|
68
72
|
|
|
69
73
|
Take snapshots to understand the page structure before recording actions:
|
|
70
74
|
|
|
@@ -75,7 +79,7 @@ playwright-cli snapshot
|
|
|
75
79
|
playwright-cli click e5
|
|
76
80
|
```
|
|
77
81
|
|
|
78
|
-
###
|
|
82
|
+
### Add assertions manually
|
|
79
83
|
|
|
80
84
|
Generated code captures actions but not assertions. Add expectations in your test using one of the recommended matchers:
|
|
81
85
|
|
|
@@ -132,3 +136,298 @@ await expect(page.getByRole('navigation')).toMatchAriaSnapshot(`
|
|
|
132
136
|
- link "Profile"
|
|
133
137
|
`);
|
|
134
138
|
```
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## 1. Planning
|
|
143
|
+
|
|
144
|
+
Goal: produce a spec file (e.g. `specs/<feature>.plan.md`) that enumerates the scenarios to test. **Always** write the spec to a file.
|
|
145
|
+
|
|
146
|
+
### 1.1 Prerequisite: workspace
|
|
147
|
+
|
|
148
|
+
Check the workspace has Playwright installed before anything else:
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
# Either of these confirms a workspace:
|
|
152
|
+
test -f playwright.config.ts || test -f playwright.config.js
|
|
153
|
+
npx --no-install playwright --version
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
If there is no Playwright install, bootstrap one and let the user pick the defaults:
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
npm init playwright@latest
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
### 1.2 Prerequisite: seed test
|
|
163
|
+
|
|
164
|
+
A **seed test** is a minimal test that lands the page in the state every scenario starts from: navigation to the app, any required login, feature flags, etc. Scenarios assume a fresh start *after* the seed. `--debug=cli` pauses *inside* this test, so the seed is where every planning and generation session begins.
|
|
165
|
+
|
|
166
|
+
Minimum viable seed:
|
|
167
|
+
|
|
168
|
+
```ts
|
|
169
|
+
// tests/seed.spec.ts
|
|
170
|
+
import { test } from '@playwright/test';
|
|
171
|
+
|
|
172
|
+
test('seed', async ({ page }) => {
|
|
173
|
+
await page.goto('https://example.com/');
|
|
174
|
+
});
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
Preferred — push navigation into a fixture so scenario tests reuse it:
|
|
178
|
+
|
|
179
|
+
```ts
|
|
180
|
+
// tests/fixtures.ts
|
|
181
|
+
import { test as baseTest } from '@playwright/test';
|
|
182
|
+
export { expect } from '@playwright/test';
|
|
183
|
+
|
|
184
|
+
export const test = baseTest.extend({
|
|
185
|
+
page: async ({ page }, use) => {
|
|
186
|
+
await page.goto('https://example.com/');
|
|
187
|
+
await use(page);
|
|
188
|
+
},
|
|
189
|
+
});
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
```ts
|
|
193
|
+
// tests/seed.spec.ts
|
|
194
|
+
import { test } from './fixtures';
|
|
195
|
+
|
|
196
|
+
test('seed', async ({ page }) => {
|
|
197
|
+
// Fixture already navigates. This empty body tells agents where to start.
|
|
198
|
+
});
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
If no seed exists, create one that at least navigates to the app.
|
|
202
|
+
|
|
203
|
+
### 1.3 Explore the app
|
|
204
|
+
|
|
205
|
+
Launch the app via the seed in the background and attach:
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/seed.spec.ts --debug=cli
|
|
209
|
+
# wait for "Debugging Instructions" and the session name tw-XXXX
|
|
210
|
+
playwright-cli attach tw-XXXX
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
Resume so the seed runs, then probe the app:
|
|
214
|
+
|
|
215
|
+
```bash
|
|
216
|
+
playwright-cli resume # resume so that seed test runs fully
|
|
217
|
+
playwright-cli snapshot # inventory of interactive elements
|
|
218
|
+
playwright-cli click e5 # follow a flow
|
|
219
|
+
playwright-cli eval "location.href" # read URL / state
|
|
220
|
+
playwright-cli show --annotate # ask the user to point at something
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
Map out:
|
|
224
|
+
|
|
225
|
+
- Interactive surfaces (forms, buttons, lists, filters, modals).
|
|
226
|
+
- Primary user journeys end-to-end.
|
|
227
|
+
- Edge cases: empty states, validation errors, very long input, boundary values.
|
|
228
|
+
- Persistence: reload, local/session storage, URL fragments.
|
|
229
|
+
- Navigation: which controls change the URL, back/forward behaviour.
|
|
230
|
+
|
|
231
|
+
**Important**: Do not just open the app url with playwright-cli, always go through the test to capture any custom setup done there.
|
|
232
|
+
**Important**: Stop the background test when done exploring.
|
|
233
|
+
|
|
234
|
+
### 1.4 Write the spec file
|
|
235
|
+
|
|
236
|
+
Save under `specs/<feature>.plan.md`. Use this structure:
|
|
237
|
+
|
|
238
|
+
```markdown
|
|
239
|
+
# <Feature> Test Plan
|
|
240
|
+
|
|
241
|
+
## Application Overview
|
|
242
|
+
|
|
243
|
+
<One paragraph describing what the feature does and why it matters.>
|
|
244
|
+
|
|
245
|
+
## Test Scenarios
|
|
246
|
+
|
|
247
|
+
### 1. <Group Name>
|
|
248
|
+
|
|
249
|
+
**Seed:** `tests/seed.spec.ts`
|
|
250
|
+
|
|
251
|
+
#### 1.1. <kebab-case-scenario-name>
|
|
252
|
+
|
|
253
|
+
**File:** `tests/<group>/<kebab-case-scenario-name>.spec.ts`
|
|
254
|
+
|
|
255
|
+
**Steps:**
|
|
256
|
+
1. <Concrete user step>
|
|
257
|
+
- expect: <observable outcome>
|
|
258
|
+
- expect: <another observable outcome>
|
|
259
|
+
2. <Next step>
|
|
260
|
+
- expect: <outcome>
|
|
261
|
+
|
|
262
|
+
#### 1.2. <next-scenario>
|
|
263
|
+
...
|
|
264
|
+
|
|
265
|
+
### 2. <Next Group>
|
|
266
|
+
|
|
267
|
+
**Seed:** `tests/seed.spec.ts`
|
|
268
|
+
...
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
Guidelines:
|
|
272
|
+
|
|
273
|
+
- Each scenario is independent and starts from the seed's fresh state — never chain scenarios.
|
|
274
|
+
- Scenario names are kebab-case and match the test file name (`should-add-single-todo` → `should-add-single-todo.spec.ts`).
|
|
275
|
+
- Cover happy path, edge cases, validation, negative flows, persistence.
|
|
276
|
+
- Write steps at the user level ("Type 'Buy milk' into the input"), not the API level ("call `fill`").
|
|
277
|
+
- Put observable outcomes in `- expect:` bullets; each becomes an assertion during generation.
|
|
278
|
+
|
|
279
|
+
---
|
|
280
|
+
|
|
281
|
+
## 2. Generate
|
|
282
|
+
|
|
283
|
+
Goal: take a spec file and produce Playwright test files. Optionally update the spec if it has drifted.
|
|
284
|
+
|
|
285
|
+
### 2.1 Inputs
|
|
286
|
+
|
|
287
|
+
- **Spec file**, e.g. `specs/basic-operations.plan.md`.
|
|
288
|
+
- **Target**: either a single scenario (e.g. `1.2`), a whole group (`1`), or all.
|
|
289
|
+
- **Seed file**, read from the `**Seed:**` line of the scenario's group.
|
|
290
|
+
|
|
291
|
+
### 2.2 Generate one scenario
|
|
292
|
+
|
|
293
|
+
For each target scenario, in sequence (never in parallel — scenarios share the seed session):
|
|
294
|
+
|
|
295
|
+
```bash
|
|
296
|
+
PLAYWRIGHT_HTML_OPEN=never npx playwright test <seed-file> --debug=cli # background
|
|
297
|
+
playwright-cli attach tw-XXXX
|
|
298
|
+
# resume
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
**Do not** just open the app url with playwright-cli, always go through the test to capture any custom setup done there.
|
|
302
|
+
|
|
303
|
+
Walk the scenario's `Steps:` one by one with `playwright-cli`, treating the spec as the plan and the live app as the source of truth. If a step is vague ("click the button" — which button?), references an element that no longer exists, or contradicts the app's actual behaviour, use your judgement: update the spec to match what the app really does, then keep going. Editing the spec mid-generation is expected.
|
|
304
|
+
|
|
305
|
+
Every action prints the equivalent Playwright TypeScript (see [How generation works](#0-how-generation-works)):
|
|
306
|
+
|
|
307
|
+
```bash
|
|
308
|
+
playwright-cli snapshot # find refs
|
|
309
|
+
playwright-cli fill e3 "John Doe" # -> page.getByRole('textbox', {...}).fill(...)
|
|
310
|
+
playwright-cli press Enter
|
|
311
|
+
playwright-cli click e7
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
For each `- expect:` bullet, add an explicit assertion. See [How generation works](#0-how-generation-works) for details.
|
|
315
|
+
|
|
316
|
+
Collect the generated code and write the test file at the path given in the spec:
|
|
317
|
+
|
|
318
|
+
```ts
|
|
319
|
+
// spec: specs/basic-operations.plan.md
|
|
320
|
+
// seed: tests/seed.spec.ts
|
|
321
|
+
import { test, expect } from './fixtures'; // or '@playwright/test' if no fixtures file
|
|
322
|
+
|
|
323
|
+
test.describe('Signing in and out', () => {
|
|
324
|
+
test('should sign in', async ({ page }) => {
|
|
325
|
+
// 1. Navigate to the application
|
|
326
|
+
// (handled by the seed fixture)
|
|
327
|
+
|
|
328
|
+
// 2. Type 'John Doe' into the username field
|
|
329
|
+
await page.getByRole('textbox', { name: 'username' }).fill('John Doe');
|
|
330
|
+
|
|
331
|
+
// 3. Type password
|
|
332
|
+
await page.getByRole('textbox', { name: 'password' }).fill('TestPassword');
|
|
333
|
+
|
|
334
|
+
// 4. Press Enter to submit
|
|
335
|
+
await page.getByRole('textbox', { name: 'password' }).press('Enter');
|
|
336
|
+
|
|
337
|
+
await expect(page.getByRole('heading')).toContainText('Welcome, John Doe!');
|
|
338
|
+
});
|
|
339
|
+
});
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
Rules:
|
|
343
|
+
|
|
344
|
+
- **One test per file.** File path, describe name, and test name come verbatim from the spec (minus the ordinal).
|
|
345
|
+
- Prefix each numbered step with a `// N. <step text>` comment before its actions.
|
|
346
|
+
- Use the describe group name verbatim from the spec (no `1.` ordinal).
|
|
347
|
+
- Import from `./fixtures` if the project has one; otherwise `@playwright/test`.
|
|
348
|
+
- **Important**: close the CLI session and stop the background test before moving to the next scenario.
|
|
349
|
+
|
|
350
|
+
### 2.3 Generate multiple scenarios
|
|
351
|
+
|
|
352
|
+
Loop 2.2 over the targeted scenarios one at a time, restarting the seed between each so every test starts from a clean page. This is safe to parallelise due to unique generated session names - just make sure each test run is stopped.
|
|
353
|
+
|
|
354
|
+
### 2.4 Run generated tests
|
|
355
|
+
|
|
356
|
+
After generation, run the new tests once:
|
|
357
|
+
|
|
358
|
+
```bash
|
|
359
|
+
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/<group>/<scenario>.spec.ts
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
Any failure goes to Section 3.
|
|
363
|
+
|
|
364
|
+
---
|
|
365
|
+
|
|
366
|
+
## 3. Heal
|
|
367
|
+
|
|
368
|
+
Goal: fix failing tests, and update the spec if the app's intended behaviour changed.
|
|
369
|
+
|
|
370
|
+
### 3.1 Find failing tests
|
|
371
|
+
|
|
372
|
+
```bash
|
|
373
|
+
PLAYWRIGHT_HTML_OPEN=never npx playwright test
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
Record the list of failing `<file>:<line>` entries and process them one at a time. Do not attempt parallel fixes — shared state and the single CLI session make that fragile.
|
|
377
|
+
|
|
378
|
+
### 3.2 Debug one failure
|
|
379
|
+
|
|
380
|
+
Run the single failing test in debug mode in the background, then attach:
|
|
381
|
+
|
|
382
|
+
```bash
|
|
383
|
+
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/<group>/<scenario>.spec.ts:<line> --debug=cli
|
|
384
|
+
# wait for "Debugging Instructions" and the tw-XXXX session name
|
|
385
|
+
playwright-cli attach tw-XXXX
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
The test is paused at the start. Step forward or run to until just before the failing action or assertion, then diagnose:
|
|
389
|
+
|
|
390
|
+
```bash
|
|
391
|
+
playwright-cli snapshot # did the element change / move / rename?
|
|
392
|
+
playwright-cli console # app-side errors?
|
|
393
|
+
playwright-cli requests # failed request? wrong payload?
|
|
394
|
+
playwright-cli show --annotate # ask the user to point somewhere
|
|
395
|
+
```
|
|
396
|
+
|
|
397
|
+
Common causes: selector drift, new wrapper element, label/ARIA rename, timing (transition, async load), assertion text updated in the app, test data leaking between runs.
|
|
398
|
+
|
|
399
|
+
Rehearse the corrected interaction with `playwright-cli` — the generated code in the output is what you paste back into the test.
|
|
400
|
+
|
|
401
|
+
### 3.3 Apply the fix
|
|
402
|
+
|
|
403
|
+
Edit the test file: update the locator, assertion, step order, or inputs to match the corrected behaviour. Stop the background debug run. Rerun the single test to confirm green.
|
|
404
|
+
|
|
405
|
+
Never skip hooks or add sleeps as a fix. Never use `networkidle`.
|
|
406
|
+
|
|
407
|
+
### 3.4 Reconcile with the spec
|
|
408
|
+
|
|
409
|
+
Open the spec referenced by the `// spec:` header in the test file and locate the scenario that matches the test.
|
|
410
|
+
|
|
411
|
+
- **Fix was purely technical** (locator drift, better assertion shape) and the spec's user-level behaviour still matches the app → leave the spec alone.
|
|
412
|
+
- **Fix changed user-visible steps, inputs, order, or expected outcomes** that the spec describes → update the spec to match reality. Keep the scenario id and file path stable; only the step / expect lines change.
|
|
413
|
+
- **Unclear whether the app change is intentional** (spec is stale) **or a regression** (test was right, app is wrong) → **stop and ask the user**. Provide:
|
|
414
|
+
- the scenario id (e.g. `2.3`),
|
|
415
|
+
- the spec lines that no longer match,
|
|
416
|
+
- the observed app behaviour (quote a snapshot excerpt or a concrete outcome).
|
|
417
|
+
|
|
418
|
+
Only after the user answers, either update the spec (intentional change) or file/flag the test as covering a bug (regression).
|
|
419
|
+
|
|
420
|
+
### 3.5 Iteration and giving up
|
|
421
|
+
|
|
422
|
+
- Fix failures one at a time; rerun after each.
|
|
423
|
+
- If after thorough investigation you are confident the test is correct but the app is wrong *and* the user has confirmed it's a bug: mark the test `test.fixme(...)` with a comment pointing at the user's decision or issue link. Never silently skip.
|
|
424
|
+
|
|
425
|
+
---
|
|
426
|
+
|
|
427
|
+
## Cross-references
|
|
428
|
+
|
|
429
|
+
| For... | See |
|
|
430
|
+
|---|---|
|
|
431
|
+
| `--debug=cli` / attach mechanics | [playwright-tests.md](playwright-tests.md) |
|
|
432
|
+
| Mocking requests during exploration/generation | [request-mocking.md](request-mocking.md) |
|
|
433
|
+
| Managing the CLI browser session | [session-management.md](session-management.md) |
|
|
@@ -4,6 +4,18 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
## [Unreleased]
|
|
6
6
|
|
|
7
|
+
## [0.9.8-alpha.1] - 2026-07-12
|
|
8
|
+
|
|
9
|
+
### Changed
|
|
10
|
+
|
|
11
|
+
- Published a synchronized Atomic 0.9.8-alpha.1 prerelease for the web-access extension; no functional web-access changes were made after 0.9.7.
|
|
12
|
+
|
|
13
|
+
## [0.9.7] - 2026-07-12
|
|
14
|
+
|
|
15
|
+
### Changed
|
|
16
|
+
|
|
17
|
+
- Published the stable Atomic 0.9.7 release for the web-access extension; no functional web-access changes were made after 0.9.6.
|
|
18
|
+
|
|
7
19
|
## [0.9.7-alpha.1] - 2026-07-12
|
|
8
20
|
|
|
9
21
|
### Changed
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@bastani/web-access",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.8-alpha.1",
|
|
4
4
|
"private": true,
|
|
5
5
|
"description": "Atomic extension for web search, URL fetching, GitHub repo cloning, PDF/video extraction. Fork of: https://github.com/nicobailon/pi-web-access",
|
|
6
6
|
"contributors": [
|