tamash-playwright 0.12.0-beta.1 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,452 +1,467 @@
1
- # Changelog
2
-
3
- All notable changes to this project are documented here. Format loosely follows
4
- [Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
5
-
6
- ## [0.12.0-beta.1] - 2026-09-08
7
-
8
- ### Added
9
-
10
- - **Playwright 1.63 support.** Dev/CI now tracks `@playwright/test` 1.63; `peerDependencies`
11
- stays `>=1.40.0`. Verified: existing iframe healing (`page.frameLocator('#id')`) still heals,
12
- and the full unit + representative e2e suite passes against 1.63.
13
-
14
- - **Healing through Playwright 1.63's no-argument `page.frameLocator()`.** 1.63 made
15
- `frameLocator()`'s selector optional — with no argument it matches inside *any* frame on the
16
- page. As a healing scope that's ambiguous the moment the page has more than one frame (the
17
- healer's `locator('body')` snapshot then throws *"frameLocator() matched elements in multiple
18
- frames"*, and healing silently no-op'd with `stage=no_snapshot`). Now, when the page has
19
- exactly one subframe, the healer resolves the no-arg `frameLocator()` to that concrete frame
20
- and heals normally. With zero or several subframes it can't know which frame was meant, so it
21
- steps aside cleanly the original error is re-thrown, never a wrong-frame guess. For healable
22
- work on a multi-frame page, pass an explicit selector: `page.frameLocator('#id')`.
23
-
24
- ### Fixed
25
-
26
- - **`describeFactoryCall` rendered `iframe "undefined"` for a no-arg `frameLocator()`.** The
27
- factory-call label helper assumed a selector argument was always present; with 1.63's optional
28
- selector it produced the literal string `iframe "undefined"`. Now renders `any iframe`. (Latent
29
- the label isn't currently surfaced to users or the model, but it would be the moment any
30
- "healing inside iframe X" context is added to a report or prompt.)
31
-
32
- ## [0.11.0] - 2026-08-31
33
-
34
- ### Fixed
35
-
36
- - **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
37
-
38
- - **`doctor`'s connectivity check gave false confidence for `claude-subscription`/`copilot-subscription`.**
39
- It tested against a fixed, generous 15s timeout, completely decoupled from the project's real
40
- configured `actionTimeout` — so a project using the README's own example value (`actionTimeout:
41
- 8000`) could get `[OK] Connected successfully` from `doctor`, then have its very first real heal
42
- in a fresh test run fail with a misleading "not authenticated?" warning, because these two
43
- providers spawn the vendor's own CLI as a subprocess and the first call in a process pays a
44
- one-time cold-start cost (spawning + authenticating) on top of the actual model call a cost
45
- `doctor`'s own generous timeout never had to absorb. Found via a real, unbriefed agent walkthrough
46
- of a from-scratch setup (see the skill entry below): it hit this exact failure, correctly
47
- diagnosed the root cause from a source-code comment nothing user-facing ever surfaced, and worked
48
- around it by raising `actionTimeout` to 20000ms. Fixed at the actual source: `doctor` now tests
49
- connectivity within the project's real `actionTimeout` when one's configured (falling back to the
50
- old generous default only when none is set yet), and a `[FAIL]` for either of these two providers
51
- now explicitly suggests it may be the cold-start cost rather than a real auth problem. Verified
52
- against the real `copilot-subscription` provider — confirmed the check now genuinely uses the
53
- configured timeout (not just that it compiles); could not force a deterministic before/after
54
- repro of the timeout itself in this environment, since the underlying race is timing/environment-
55
- dependent (a warm local CLI session can mask it, exactly as the original cold-start comment
56
- already described).
57
-
58
- - **The published package was silently carrying stale, orphaned compiled output.** `npm run build`
59
- (plain `tsc`, no `--build`/incremental mode) never deletes `dist/` output for a source file that
60
- was later removed found in the `0.11.0-beta.1` publish itself: `dist/healer/providers/
61
- claude-cli.js`/`copilot-cli.js` shipped in the tarball with no corresponding source file
62
- anywhere in the repo (from an abandoned, never-committed, never-wired-up piece of past work) —
63
- dead code nothing actually imports, but real, needless bloat in every install. Fixed by cleaning
64
- `dist/` automatically before every build (`prebuild` script), verified by confirming a full
65
- rebuild no longer produces those two files.
66
-
67
- - **A popup/new tab opened via `context.waitForEvent('page')`, `page.on('popup', ...)`, or
68
- `page.waitForEvent('popup')` was never healing-aware.** `bindContext`/`bindPageActions` already
69
- made `context.newPage()`/`context.pages()`/`context.on('page', ...)` heal correctly, but these
70
- three including `context.waitForEvent('page')`, the pattern most commonly taught for popup
71
- handling — fell through to a raw, unwrapped Playwright page. A broken locator inside the
72
- resulting page just threw a plain timeout, with no `[self-healer]` line at all: healing was never
73
- attempted, not just unsuccessful. Found while building the first-ever test for popup healing
74
- (previously implemented with zero coverage); fixed by extending both bindings to wrap whatever
75
- page these three hand back, the same way `newPage()` already did.
76
-
77
- - **`tamash` (the rule-based provider) declined on a real, common page pattern it should have
78
- resolved.** A floating label rendered as its own text node right next to a field whose accessible
79
- name is *also* that same text (confirmed on a real login page: a "Username" label next to a field
80
- that's itself named "Username") was miscounted as two competing candidates and declined, even
81
- though there's genuinely only one field. `findRuleBasedMatch` now only treats a match as a real
82
- competing candidate when it's independently resolvable on its own (has a ref and a plausible
83
- role) a bare text node repeating the same words is decorative, not a second option. Genuine
84
- ambiguity (two actually-independent candidates) still declines exactly as before.
85
-
86
- - **The HTML report's self-healing attachment was a wall of escape sequences on a failed
87
- heal.** Playwright's own error messages embed ANSI colour codes in their "Call log" section;
88
- the healer stored those raw in `report.reason` / `report.warning` / `attempts[].error`, and
89
- `JSON.stringify` (which builds the `self-healing-<action>` attachment) escapes every ESC byte in the JSON, so a *not-healed* report rendered as unreadable escape soup. A
90
- *successful* heal has short or absent error strings, which is why it only showed up on failures.
91
- Fixed by stripping ANSI CSI sequences in `normalizeError` — the single point every captured
92
- error passes through so the attachment, the console line, and `heals.jsonl` are all plain
93
- text. Verified against a real not-healed run.
94
-
95
- - **The three CLI-based subscription providers (`cursor-subscription`, `kiro-subscription`,
96
- `codex-subscription`) were verified against their real vendor CLIs for the first time and none
97
- of the three actually worked as shipped.** Four separate bugs, all found by running them, all now
98
- fixed:
99
- 1. **`cursor-subscription` never returned a heal.** `agent -p` refuses to run in any directory
100
- without workspace trust — it prints "Workspace Trust Required" and exits without answering.
101
- Now passes `--trust --mode ask`; `--mode ask` is Cursor's own read-only Q&A mode ("read-only"
102
- per its `--help`), which *removes* the old "has access to all tools, could edit a file"
103
- caveat this provider carried a heal call in `ask` mode cannot write or run anything.
104
- 2. **`codex-subscription` refused to run outside a Git repository** ("Not inside a trusted
105
- directory and --skip-git-repo-check was not specified"). Now passes `--skip-git-repo-check`;
106
- this only relaxes the where-may-I-run guard, not the sandbox `codex exec` still runs
107
- `approval: never` + read-only, the property this provider depends on.
108
- 3. **`codex exec` hung on stdin.** It drains stdin ("Reading additional input from stdin…") and
109
- blocks until EOF; `runCliPrompt` left the child's stdin as an open pipe, so every call ran to
110
- the timeout and was killed — confirmed: a prompt the CLI answers in ~8s "timed out" at 15s.
111
- `runCliPrompt` now gives every child an already-closed stdin (`stdio: ['ignore', …]`),
112
- harmless for `agent`/`kiro-cli`, the fix for `codex`.
113
- 4. **The JSON parsers failed on conversational output.** `parseSuggestion` /
114
- `parseVisionSuggestion` / `parseActionTacticSuggestion` extracted JSON from prose with a
115
- greedy first-`{`-to-last-`}` slice, which breaks the moment the response contains more than
116
- one JSON object exactly what cursor's `agent` produces (it echoes the system prompt's
117
- example objects back before its real answer). Replaced with a balanced-brace-span scan that
118
- takes the last valid object (a model states its answer last), old greedy slice kept as a last
119
- resort. Improves robustness for every provider, not just the CLI ones.
120
-
121
- Verified live, all authenticated: `doctor` reports `[OK] Connected` for all three; a real
122
- `npx playwright test` heal (broken locator, cache cleared) went green with
123
- `HEALED [provider=codex-subscription]`, and `kiro-subscription` healed the same spec.
124
-
125
- - **`cursor-subscription` reliability note.** Verifying it (see above) showed Cursor's `agent` CLI
126
- is an interactive assistant rather than a one-shot completion endpoint given the heal prompt it
127
- often answers conversationally instead of with the required JSON, so the heal is declined more
128
- often than with `kiro`/`codex`. It's now invoked in the safest/best-effort form
129
- (`agent -p --trust --mode ask --output-format json`, read-only) and is marked **experimental**;
130
- the docs point to `kiro`/`codex` as the steadier local-subscription options. Report issues.
131
- Also: the full e2e healing suite (12 tests) was run end to end against `copilot-subscription` and
132
- `ollama` **12/12 each** to confirm the parser rewrite and provider changes don't regress it.
133
-
134
- ### Added
135
-
136
- - **`npx tamash-playwright init-skill` one command to install the orchestration skill, plus a
137
- `doctor` check that flags when it's missing or stale.** The skill this package ships
138
- (`skills/tamash-playwright/`) was previously a copy-this-shell-one-liner-per-agent step buried in
139
- `SKILL.md`, with nothing verifying it had been done — so a project could `npm install` the
140
- package, never copy the skill, and its assistant would silently never load the workflow.
141
- `init-skill` copies the skill (`SKILL.md` + `references/`) into **both** standard locations
142
- `.claude/skills/tamash-playwright/` (Claude Code) and `.agents/skills/tamash-playwright/` (the
143
- emerging cross-tool standard, read by Cursor, GitHub Copilot, Windsurf, Kiro, Zed, dotnet Aspire,
144
- the same convention Playwright's own `playwright-cli install --skills` uses). Same content in
145
- both; no per-agent format conversion. `--target claude` / `--target agents` installs one; `--user`
146
- installs under your home directory; `--force` overwrites a hand-edited copy; `--dry-run` previews.
147
- Each install carries a `tamash-playwright-skill-version:` marker; `doctor`'s **Skill** section
148
- reads it and reports `[OK]` current / `[WARN]` behind-the-package-version / `[INFO]` not-installed
149
- or unmanaged, with the same one-line fix. Covered by unit tests (`skill.test.js`, real installs
150
- into a temp dir) and verified live end to end.
151
-
152
- *(0.11.0-beta.7 shipped an earlier `init-skill` that instead auto-detected the agent and wrote a
153
- Cursor-specific `.mdc` file / appended blocks to `.github/copilot-instructions.md` and
154
- `AGENTS.md`. That approach is gone `init-skill` and `doctor` now point out any such leftovers
155
- from a beta.7 install so you can delete them by hand.)*
156
-
157
- - **`doctor`'s AI Provider check now tells you *what kind* of failure it hit and what to do about
158
- it, instead of one generic "no valid response".** Every provider gained an optional, diagnostics-
159
- only `diagnose()` method (never on the healing hot path) that runs one trivial round trip and
160
- reports a structured category `not-installed`, `not-authenticated`, `timeout`, `bad-model`,
161
- `network`, `bad-response` each of which `doctor` maps to a specific next step: the missing
162
- `npm install`/CLI installer for a missing SDK; "run `claude login` / check your API key / confirm
163
- your subscription is active and within quota" for a rejected request; "raise `actionTimeout`" (plus
164
- the existing subprocess cold-start note) for a timeout; "check `<MODEL>` in your .env" for a
165
- rejected model id; a network/proxy/base-URL hint for a connection failure. The raw error line is
166
- still printed verbatim beneath the guidance. Previously `doctor` could only distinguish "worked"
167
- from "didn't" a missing SDK, an expired login, a slow network and a wrong model name all
168
- produced the same line. Providers that don't implement `diagnose()` fall back to the old
169
- `suggestSelector()` probe (worked/didn't, plus a timing-based timeout guess). Verified live:
170
- `openai` bad key -> `not-authenticated` (401 body shown), `openai` bad base URL -> `network`,
171
- `copilot-subscription` bad model id -> `bad-model`, 1ms `actionTimeout` -> `timeout` with the
172
- raise-the-timeout guidance, and all three not-installed shapes by moving the dependency out of
173
- resolution and restoring it: `@github/copilot-sdk` (CJS `MODULE_NOT_FOUND`),
174
- `@anthropic-ai/claude-agent-sdk` (ESM `ERR_MODULE_NOT_FOUND`), and a missing `kiro-cli` binary
175
- (`ENOENT`) -- each surfacing `not-installed` with its exact install command.
176
-
177
- - **Three new local-development-only subscription providers: `cursor-subscription`,
178
- `kiro-subscription`, `codex-subscription`.** Extends the same "use what you're already paying
179
- for" idea `claude-subscription`/`copilot-subscription` already give, to three more real
180
- subscriptions but each is explicitly scoped to local development, never documented or
181
- recommended for CI, for two genuinely different reasons found by checking real vendor docs before
182
- writing any code. Cursor's and Kiro's own headless-mode docs confirm neither has a way to fully
183
- disable tool/file/command access the way `claude-subscription`'s `tools: []`/`copilot-subscription`'s
184
- `availableTools: []` do — Cursor's `agent -p` "has access to all tools" by its own docs, and Kiro's
185
- headless mode requires `--trust-all-tools`/`--trust-tools` specifically because there's no one to
186
- approve a tool call unattended; `kiro-subscription` deliberately never grants that trust, so a
187
- tool-requiring response just times out and declines rather than being granted broad access.
188
- `codex-subscription` is local-only for a different, simpler reason: `codex exec` (used here, not
189
- the interactive REPL) defaults to a read-only sandbox with no approval prompts — already as safe
190
- as the two existing subscription providers but no long-lived, subscription-only token for
191
- unattended CI use is confirmed to exist for it yet. None of the three vendors ship a Node SDK, so
192
- all three are `child_process` wrappers around the vendor's own CLI binary (`agent`, `kiro-cli`,
193
- `codex`) via a new shared `runCliPrompt` helper, rather than the SDK-based design the existing two
194
- use meaning every call pays a fresh process-spawn cost, not just the first the way
195
- `copilot-subscription`'s warm shared client does; `doctor` now has a distinct hint explaining this
196
- when one of these three fails within a tight `actionTimeout`. Investigated and explicitly ruled
197
- out this round: Antigravity its dedicated `google-antigravity` SDK is Python-only (this is a
198
- Node/TypeScript package), has no documented tool-restricted mode at all, and its authentication
199
- story isn't published, so it isn't even confirmed to ride on an existing subscription. Verified:
200
- real graceful-decline behavior confirmed against genuinely uninstalled CLIs (no fabricated
201
- simulation), and the full `doctor` integration (including the new every-call-cost hint) confirmed
202
- against a real sample repo. **Not yet verified**: the actual happy path against a real installed
203
- and authenticated CLI for any of the three none were available in the development environment,
204
- and all three require a real paid subscription account.
205
-
206
- - **A skill for running this package's local workflow inside an AI coding assistant.** Ships at
207
- `skills/tamash-playwright/` a `SKILL.md` entry point that branches on `npx tamash-playwright
208
- doctor`'s actual output (never assumed), plus two reference docs: `onboarding.md` (bringing a
209
- project up to standard provider setup, `actionTimeout`, `.describe()` labels, Page Object
210
- extraction) and `heal.md` (a gated review/apply/verify/land loop over `apply-heals` and
211
- `verify-heals.cjs` most runs proceed start to finish unattended, pausing only for a genuinely
212
- ambiguous fix or anything after a failed verification, and never landing/committing/opening a PR
213
- without asking first, no matter how clean the run was). Pure orchestration over commands that
214
- already exist no new healing capability. Ships with adapters for Claude Code, Kiro (identical
215
- `SKILL.md` format, confirmed against Kiro's own docs), Cursor (`.mdc` rule), GitHub Copilot
216
- (`copilot-instructions.md` section), and an `AGENTS.md` covering the broader cross-tool standard
217
- (Antigravity, Gemini CLI, Windsurf, Zed, Aider, and others) none of which are auto-discovered
218
- from `node_modules` by any of these tools, confirmed rather than assumed, so every adapter's
219
- install step is one explicit copy command, documented in `SKILL.md` itself.
220
- - **A new `tamash` heal provider — rule-based healing, no AI at all.** `HEALER_PROVIDER=tamash`
221
- needs no API key, no subscription, and makes no network call: it resolves a broken locator by
222
- text-matching the same description an AI provider would receive (`.describe()`, or a decoded
223
- variable name see below) against the already-captured accessibility snapshot, then reuses the
224
- exact same structural widening (`near`/`adjacent`) the AI-backed path already uses once it finds
225
- the right anchor same output shape, same downstream code, zero duplicated logic. It shares the
226
- same non-negotiable discipline as every text-matching step in this package: zero or more than one
227
- match, at any point, means it declines rather than guesses verified with a dedicated permanent
228
- e2e suite covering direct matches, widened matches, and every decline path (including the known,
229
- accepted limitation that a misleading type hint can't be second-guessed against). Genuinely a
230
- different tool than the AI providers, not a free replacement for one: no vision fallback (nothing
231
- to reason over a screenshot with) and no action-recovery tactics (those require understanding
232
- *why* an action failed, which is inference this provider deliberately doesn't attempt) a fast,
233
- free, fully deterministic first line of defense, best suited to well-`.describe()`d,
234
- Page-Object-style suites.
235
- - **A new `ollama-local` heal provider for self-hosted Ollama servers.** `HEALER_PROVIDER=ollama-local`
236
- targets your own `ollama serve` instance or an internal company deployment, instead of Ollama
237
- Cloud a deliberately separate provider from `ollama`, not a flag on it, since the two have
238
- genuinely different auth defaults: Ollama Cloud always requires `OLLAMA_API_KEY`, while
239
- `OLLAMA_LOCAL_API_KEY` is optional, since a bare `ollama serve` has no authentication at all. Set
240
- it only if your internal deployment sits behind a reverse proxy or API gateway that requires a
241
- bearer token verified against a real HTTP server both ways, confirming the `Authorization`
242
- header is omitted entirely when no key is set, and sent correctly when one is. Prompted by a real
243
- support request from a team wanting to use their own internally-hosted `gpt-oss:120b`.
244
- - **Undescribed, POM-style locator variables now get a real description automatically.** When
245
- `.describe()` was never called, the healer already fell back to the locator's own variable/
246
- property name (`txtEmployeeId`) as its best guess at intent now that raw identifier is decoded
247
- into the same kind of human-readable phrase `.describe()` would give: `txtEmployeeId` becomes
248
- "Employee Id (textbox)", `submitButton` becomes "Submit (button)", recognizing both prefix-style
249
- (`btnSubmit`) and suffix-style (`submitBtn`) naming, camelCase/snake_case/kebab-case, and correct
250
- acronym boundaries (`employeeIDNumber` "Employee ID Number"). Falls back to the raw identifier,
251
- exactly as before, whenever nothing meaningful survives decoding (a placeholder name like `el1`,
252
- or a bare affix with nothing else) never a guess dressed up as a real description. Verified
253
- live against a real, unpublished build in both sample repos (a real `copilot-subscription` call,
254
- and after finding and fixing a genuinely stale API key along the way a real `ollama` call),
255
- confirming the decoded description actually reaches the configured AI provider unchanged.
256
-
257
- ## [0.10.0] - 2026-08-27
258
-
259
- ### Fixed
260
-
261
- - **`waitFor()` is never sent to the AI.** It's a state check, not an action — a timeout on it can
262
- mean a genuinely broken selector, or it can mean the element correctly never reached the expected
263
- state (verifying something does NOT appear, or a real app issue), and there's no way to tell those
264
- apart from the error alone. `expect(locator).toBeVisible()` was already permanently excluded from
265
- healing for exactly this reason, but never reached this code at all (a separate path from the
266
- Proxy-intercepted actions) `waitFor` just never got the same treatment. Real, reported case: a
267
- user's `waitFor` on a locator verifying an absence (where timing out was the *correct* outcome)
268
- still burned 8175 tokens across a failed text attempt and a failed vision attempt before giving
269
- up, for a heal that could never have succeeded. Now fails fast with a clear `state-wait-not-healed`
270
- stage and zero AI calls; a real action (fill/click/...) on the same kind of broken locator is
271
- unaffected.
272
- - **`claude-subscription` was burning a real, unnecessary amount of extra tokens.** The SDK's
273
- `effort` option silently defaults to `'high'` ("deep reasoning") when left unset, and this
274
- provider never set it — a small prompt-complexity increase (the `nearbyRef`/`nearbyText` addition
275
- below) pushed adaptive thinking to reason unpredictably harder for a task that only ever needs to
276
- return one line of JSON. Confirmed live via real CI runs: output tokens for the same heal went
277
- from a 188-445 baseline up to 422-881, inconsistently, run to run. `thinking: { type: 'disabled' }`
278
- + `effort: 'low'` removes the variability entirely verified repeatedly at a steady 28-30 output
279
- tokens, below even the pre-regression baseline, with no change to correctness.
280
- - **The primary ariaSnapshot capture no longer requests `boxes:true`.** Every node was paying for a
281
- `[box=x,y,w,h]` annotation that nothing on the text/`ref` path (including the `nearbyRef`/
282
- `adjacent`-strategy widening logic below) ever reads it's purely topological. The one real
283
- consumer (vision's own nearest-candidate lookup) already captures its own separate, fresh
284
- snapshot, so this is genuinely free: verified live, input tokens dropped ~23% on a large real
285
- page (3794 2937) with no loss of accuracy, and no change on small pages (box overhead scales
286
- with node count).
287
-
288
- ### Added
289
-
290
- - **Search-scoped snapshots**: before falling back to the full page, the healer now searches the
291
- already-captured snapshot for the description's identifying phrase and, only when it matches
292
- exactly one node, sends the AI a scoped excerpt (that node's own subtree plus every sibling
293
- branch's subtree at each ancestor level up to the root) instead of the whole page — zero extra
294
- browser round-trips, since it's pure processing on data already in memory. Falls back to the full
295
- snapshot automatically whenever the search is empty or ambiguous, never a guess. Verified live
296
- across every real failure mode found this session: 41% token reduction on a deeply-nested field,
297
- 66% when the target was inside what looked like an unrelated navigation menu (proving it finds
298
- wherever the relevant text actually is, not "excludes the nav"), ~29% on a pair of identical
299
- sibling fields that still had to be correctly disambiguated, and a clean, correct fallback when
300
- the description doesn't match the page's real text at all.
301
- - **A new `adjacent` selector strategy**, fixing a real ambiguity in the existing `near` strategy:
302
- when two fields with no identity of their own share a row/section (two dropdowns side by side,
303
- say), `near`'s "climb to a shared ancestor, then search it for any element of this role"
304
- approach matches both and gives up rather than risk the wrong one. The AI's `ref` response can
305
- now optionally report `nearbyRef`/`nearbyText`/`nearbyRole` for a nameless target it identified;
306
- `deriveDurableLocator` uses that hint to find the true common ancestor between the target and
307
- its label via each ref's own full ancestor chain, not by assuming either sits at a matching
308
- depth and, when they're proven to be immediate sibling branches, builds a precise CSS
309
- `:text() + *` sibling match (or an xpath climb-then-step, when the label text turns out to be
310
- nested below its own branch root). Verified live against a real configured provider, resolving
311
- the correct field and never its same-row neighbor in both directions.
312
- - **Full attempt-history logging**: `SelfHealingReport` now carries an `attempts[]` array one
313
- entry per cache/ref/text/vision/action-recovery attempt actually made, each with its own
314
- `succeeded`/`stage`/`error`. Previously only the *last* attempt's stage survived; an earlier
315
- attempt's real failure (and the specific error it threw) was silently discarded the moment a
316
- later attempt also failed the exact shape of a real user-reported bug, where a genuine
317
- candidate selector was shown next to an unrelated `vision_provider_error` with no way to tell
318
- why the candidate itself hadn't worked. The attempt history now also prints directly to
319
- console/CI output (not just the JSON attachment) whenever more than one attempt was made, on
320
- both pass and fail since that plain-text output is what most bug reports actually paste, not
321
- an attachment nobody opens.
322
- - **`ariaSnapshot` is attached to the test report on failure** the exact accessibility tree the
323
- AI reasoned over, so a confusing report can be diagnosed against real evidence instead of a
324
- separately-captured DevTools screenshot.
325
-
326
- ## [0.9.0] - 2026-08-26
327
-
328
- ### Fixed
329
-
330
- - **Heal-log visibility**: a heal with no reusable selector (a one-shot `ref` resolution, or a
331
- vision-tagged point) was silently dropped from `heals.jsonl` entirely, even though it genuinely
332
- fixed that run. Now logged for audit with a clear `reviewNote` whenever it has a suggestion
333
- *or* a review note, without letting an audit-only entry shadow an older, real cached fix for the
334
- same location.
335
- - **Vision support for `claude-subscription`/`copilot-subscription`**: both providers had
336
- `supportsVision` hardcoded to `false` regardless of model. Both SDKs genuinely support image
337
- input (Claude Agent SDK via an image content block; Copilot SDK via a `blob` attachment)
338
- implemented for real and verified live with actual screenshots, not just doctor's yes/no label.
339
- - **Argument forwarding in healed replays**: `replayAction()` only ever forwarded the first call
340
- argument. A trailing options object (`click({ modifiers: [...] })`, `fill(value, { timeout })`,
341
- `dispatchEvent`'s 3rd argument) was silently dropped on a healed replay — changing the action's
342
- real behavior while still reporting a clean `HEALED`.
343
- - **`ref`-strategy replays now act through the derived durable locator, not the raw ref**:
344
- `aria-ref=` locators never resolve for `dispatchEvent` at all (a Playwright-level limitation,
345
- confirmed directly via `DEBUG=pw:api`), even though the identical element resolves instantly via
346
- a `normalize()`-derived locator for that same action. Fixed by deriving the durable locator
347
- *before* replaying and acting through it, falling back to the raw ref only when nothing durable
348
- could be found. A related accuracy bug is fixed alongside it: the report/heal-log could
349
- previously claim a derived selector "worked" even when its own replay had actually failed and a
350
- fallback silently took over now only the locator that genuinely performed the action is
351
- reported or cached.
352
- - **`copilot-subscription` could hang a non-Playwright test runner indefinitely** its shared
353
- client keeps a connection open across calls for performance, which Playwright's own test runner
354
- tolerates by force-exiting regardless, but a runner like Cucumber does not, so the process never
355
- returned even after every test had already passed. Found live running a real Cucumber suite in
356
- CI. `closeCopilotSubscriptionClient()` is now exported from the package's own entry point so any
357
- non-Playwright consumer can call it from their own teardown hook (e.g. Cucumber's `AfterAll`).
358
-
359
- ### Added
360
-
361
- - **`locator.getDurable(action?)`** resolves any locator (most usefully one built from
362
- `aria-ref=...`) to a durable, reusable equivalent (`getByRole`/`getByLabel`/a css selector/…),
363
- using the same derivation logic self-healing already uses internally. Throws if nothing durable
364
- could be derived, rather than silently handing back something untrusted.
365
- - **`apply-heals` now previews before writing**: every run dry or real shows a styled table of
366
- exactly what it found (location, before, after, review status) instead of a plain scrolling log.
367
- A real run also asks for confirmation before writing anything, but *only* at a genuine
368
- interactive terminal CI and any non-interactive/piped invocation proceed automatically exactly
369
- as before, so no existing unattended CI workflow is affected. `--yes`/`-y` skips the prompt at a
370
- real terminal too.
371
- - **`doctor`'s output is styled** with colors and tables (including a new end-of-run Summary
372
- section) instead of a plain scrolling log.
373
- - **Exact locator code in console lines and reports**: a healed suggestion is now shown as the
374
- real, copy-pasteable Playwright call (`getByRole("textbox", { name: "Username" })`) instead of
375
- an abbreviated shorthand (`role:textbox:Username`) — the same code `apply-heals` would write to
376
- source, so what you see is never a lossier stand-in for what was actually used.
377
- - A canonical Page Object Model usage example (`tests/pages/` +
378
- `tests/example-orangehrm-add-employee.spec.ts`), verified live against a real OrangeHRM demo,
379
- including a genuine self-heal via the `near`/widening strategy on a real unlabeled field.
380
-
381
- ### Changed
382
-
383
- - README/usage.md's vision-fallback documentation now correctly lists `claude-subscription`/
384
- `copilot-subscription` as vision-capable (with a suitable model), not just the API-key providers.
385
-
386
- ## [0.8.0] - 2026-08-25
387
-
388
- ### Added
389
-
390
- - `claude-subscription` and `copilot-subscription` heal providers self-healing backed by a
391
- personal Claude or GitHub Copilot subscription instead of a pay-per-token API key, working both
392
- locally and unattended in CI (`CLAUDE_CODE_OAUTH_TOKEN` for Claude; the ambient `GITHUB_TOKEN` or
393
- a personal-account PAT for Copilot on GitHub Actions).
394
-
395
- ### Fixed
396
-
397
- - Missing CLI install step in setup docs for `claude-subscription`/`copilot-subscription` — the
398
- standalone `claude`/`copilot` CLIs are genuinely required for the login step even though the SDKs
399
- don't need them installed to function at runtime.
400
-
401
- ### Documentation
402
-
403
- - The org-vs-personal-account Copilot licensing gotcha in CI (a repo owned by an organization
404
- without its own Copilot enablement can't use a personal subscription via the ambient token).
405
- - Action Recovery (`HEALER_ACTION_RECOVERY_ENABLED`) hidden from docs and `doctor` output
406
- disabled by default already; this only reduced its visibility, no behavior change.
407
-
408
- ## [0.7.0] - 2026-08-21
409
-
410
- ### Added
411
-
412
- - `apply-heals` turns a runtime heal into a permanent source-code fix, plus an opportunistic
413
- cache (`heals.jsonl`) so a previously-confirmed selector is tried before a fresh AI call, and
414
- history archival so a run's report/log isn't silently overwritten by the next one.
415
- - Position/relationship-based locator strategies (`near`, `scoped`, `containing`) for elements with
416
- no accessible identity of their own.
417
- - Self-healing rebuilt around `ariaSnapshot({ mode: 'ai' })` + `aria-ref=` resolution and
418
- `Locator.normalize()`, replacing pixel-distance guessing for both the text path and the vision
419
- fallback's durability upgrade.
420
- - A real unit test suite (32 tests at the time, zero new dependencies).
421
-
422
- ### Fixed
423
-
424
- - `doctor`'s `actionTimeout` check being fooled by a commented-out config value.
425
- - `apply-heals --logs-dir` silently losing raw heal-log archival.
426
- - `apply-heals` only replacing the first call when re-healing an already-`near`-fixed line.
427
- - `apply-heals` `ENOENT` on a fresh CI checkout with no `.tamash-playwright/` directory yet the
428
- sharded "apply-heals" CI job checks out fresh and has never run tests itself, so the directory
429
- genuinely doesn't exist there the way it always does in every local recipe.
430
- - File path corruption (and a `require()` crash in the generated verification script) for ESM
431
- (`"type": "module"`) consumer projects `Error.stack` renders as a `file://` URL there, which
432
- broke path resolution and a plain `.js` verification script alike.
433
- - The AI being misled by the broken selector still present in its own prompt context; a failed
434
- label guess now auto-upgrades to a structural `near` match instead of giving up.
435
-
436
- ### Verified
437
-
438
- - All four API-key providers (Ollama, OpenAI, Anthropic, Gemini) confirmed working with real API
439
- calls, not just documented.
440
-
441
- ## [0.6.0] - 2026-08-17
442
-
443
- ### Added
444
-
445
- - iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
446
- enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
447
- - `doctor`'s `actionTimeout` configuration check.
448
-
449
- ### Initial release
450
-
451
- - Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
452
- with `.describe()` for human-readable context.
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here. Format loosely follows
4
+ [Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
5
+
6
+ ## [0.12.0] - 2026-09-13
7
+
8
+ ### Added
9
+
10
+ - **Playwright 1.63 support.** Dev/CI now tracks `@playwright/test` 1.63; `peerDependencies`
11
+ stays `>=1.40.0`. Verified: existing iframe healing (`page.frameLocator('#id')`) still heals,
12
+ and the full unit + representative e2e suite passes against 1.63.
13
+
14
+ - **Healing through Playwright 1.63's no-argument `page.frameLocator()`.** 1.63 made
15
+ `frameLocator()`'s selector optional — with no argument it matches inside *any* frame on the
16
+ page. As a healing scope that's ambiguous the moment the page has more than one frame (the
17
+ healer's `locator('body')` snapshot then throws *"frameLocator() matched elements in multiple
18
+ frames"*, and healing silently no-op'd with `stage=no_snapshot`). Now, when the page has
19
+ exactly one frame, the healer re-expresses the no-arg `frameLocator()` as an explicit
20
+ single-frame `FrameLocator` and heals normally including deriving a **durable, persistable**
21
+ cross-frame selector (`apply-heals` can write it back to source), identical to what
22
+ `page.frameLocator('#id')` produces. With zero or several frames it can't know which was meant,
23
+ so it steps aside cleanly — the original error is re-thrown, never a wrong-frame guess. For
24
+ healable work on a multi-frame page, pass an explicit selector: `page.frameLocator('#id')`.
25
+
26
+ *(`0.12.0-beta.1` collapsed the scope to a raw `page.frames()` `Frame` instead of a
27
+ `FrameLocator`. `Locator.normalize()` on a `Frame`-rooted locator returns a correct selector
28
+ string but an object that resolves to nothing, so the heal worked once at runtime but couldn't
29
+ be persisted it fell back to a transient one-shot element reference with a `needsReview`
30
+ note. Fixed here.)*
31
+
32
+ - **Documented [`tamash-playwright-dashboard`](https://www.npmjs.com/package/tamash-playwright-dashboard).**
33
+ A separate, zero-config reporter package: pass-rate trends, per-test history across runs, and —
34
+ specific to this package — a Self-Healing Analytics page (tests/elements healed, token usage
35
+ per run and cumulatively, every heal event across recorded history), read directly from the
36
+ `self-healing-<action>` JSON attachment this package already writes. No code change on this
37
+ side — README and usage.md gained a "Trends across runs" section pointing to it.
38
+
39
+ ### Fixed
40
+
41
+ - **`describeFactoryCall` rendered `iframe "undefined"` for a no-arg `frameLocator()`.** The
42
+ factory-call label helper assumed a selector argument was always present; with 1.63's optional
43
+ selector it produced the literal string `iframe "undefined"`. Now renders `any iframe`. (Latent
44
+ the label isn't currently surfaced to users or the model, but it would be the moment any
45
+ "healing inside iframe X" context is added to a report or prompt.)
46
+
47
+ ## [0.11.0] - 2026-08-31
48
+
49
+ ### Fixed
50
+
51
+ - **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
52
+
53
+ - **`doctor`'s connectivity check gave false confidence for `claude-subscription`/`copilot-subscription`.**
54
+ It tested against a fixed, generous 15s timeout, completely decoupled from the project's real
55
+ configured `actionTimeout` so a project using the README's own example value (`actionTimeout:
56
+ 8000`) could get `[OK] Connected successfully` from `doctor`, then have its very first real heal
57
+ in a fresh test run fail with a misleading "not authenticated?" warning, because these two
58
+ providers spawn the vendor's own CLI as a subprocess and the first call in a process pays a
59
+ one-time cold-start cost (spawning + authenticating) on top of the actual model call a cost
60
+ `doctor`'s own generous timeout never had to absorb. Found via a real, unbriefed agent walkthrough
61
+ of a from-scratch setup (see the skill entry below): it hit this exact failure, correctly
62
+ diagnosed the root cause from a source-code comment nothing user-facing ever surfaced, and worked
63
+ around it by raising `actionTimeout` to 20000ms. Fixed at the actual source: `doctor` now tests
64
+ connectivity within the project's real `actionTimeout` when one's configured (falling back to the
65
+ old generous default only when none is set yet), and a `[FAIL]` for either of these two providers
66
+ now explicitly suggests it may be the cold-start cost rather than a real auth problem. Verified
67
+ against the real `copilot-subscription` provider confirmed the check now genuinely uses the
68
+ configured timeout (not just that it compiles); could not force a deterministic before/after
69
+ repro of the timeout itself in this environment, since the underlying race is timing/environment-
70
+ dependent (a warm local CLI session can mask it, exactly as the original cold-start comment
71
+ already described).
72
+
73
+ - **The published package was silently carrying stale, orphaned compiled output.** `npm run build`
74
+ (plain `tsc`, no `--build`/incremental mode) never deletes `dist/` output for a source file that
75
+ was later removed found in the `0.11.0-beta.1` publish itself: `dist/healer/providers/
76
+ claude-cli.js`/`copilot-cli.js` shipped in the tarball with no corresponding source file
77
+ anywhere in the repo (from an abandoned, never-committed, never-wired-up piece of past work)
78
+ dead code nothing actually imports, but real, needless bloat in every install. Fixed by cleaning
79
+ `dist/` automatically before every build (`prebuild` script), verified by confirming a full
80
+ rebuild no longer produces those two files.
81
+
82
+ - **A popup/new tab opened via `context.waitForEvent('page')`, `page.on('popup', ...)`, or
83
+ `page.waitForEvent('popup')` was never healing-aware.** `bindContext`/`bindPageActions` already
84
+ made `context.newPage()`/`context.pages()`/`context.on('page', ...)` heal correctly, but these
85
+ three — including `context.waitForEvent('page')`, the pattern most commonly taught for popup
86
+ handling fell through to a raw, unwrapped Playwright page. A broken locator inside the
87
+ resulting page just threw a plain timeout, with no `[self-healer]` line at all: healing was never
88
+ attempted, not just unsuccessful. Found while building the first-ever test for popup healing
89
+ (previously implemented with zero coverage); fixed by extending both bindings to wrap whatever
90
+ page these three hand back, the same way `newPage()` already did.
91
+
92
+ - **`tamash` (the rule-based provider) declined on a real, common page pattern it should have
93
+ resolved.** A floating label rendered as its own text node right next to a field whose accessible
94
+ name is *also* that same text (confirmed on a real login page: a "Username" label next to a field
95
+ that's itself named "Username") was miscounted as two competing candidates and declined, even
96
+ though there's genuinely only one field. `findRuleBasedMatch` now only treats a match as a real
97
+ competing candidate when it's independently resolvable on its own (has a ref and a plausible
98
+ role) — a bare text node repeating the same words is decorative, not a second option. Genuine
99
+ ambiguity (two actually-independent candidates) still declines exactly as before.
100
+
101
+ - **The HTML report's self-healing attachment was a wall of escape sequences on a failed
102
+ heal.** Playwright's own error messages embed ANSI colour codes in their "Call log" section;
103
+ the healer stored those raw in `report.reason` / `report.warning` / `attempts[].error`, and
104
+ `JSON.stringify` (which builds the `self-healing-<action>` attachment) escapes every ESC byte in the JSON, so a *not-healed* report rendered as unreadable escape soup. A
105
+ *successful* heal has short or absent error strings, which is why it only showed up on failures.
106
+ Fixed by stripping ANSI CSI sequences in `normalizeError`the single point every captured
107
+ error passes through — so the attachment, the console line, and `heals.jsonl` are all plain
108
+ text. Verified against a real not-healed run.
109
+
110
+ - **The three CLI-based subscription providers (`cursor-subscription`, `kiro-subscription`,
111
+ `codex-subscription`) were verified against their real vendor CLIs for the first time — and none
112
+ of the three actually worked as shipped.** Four separate bugs, all found by running them, all now
113
+ fixed:
114
+ 1. **`cursor-subscription` never returned a heal.** `agent -p` refuses to run in any directory
115
+ without workspace trust it prints "Workspace Trust Required" and exits without answering.
116
+ Now passes `--trust --mode ask`; `--mode ask` is Cursor's own read-only Q&A mode ("read-only"
117
+ per its `--help`), which *removes* the old "has access to all tools, could edit a file"
118
+ caveat this provider carried a heal call in `ask` mode cannot write or run anything.
119
+ 2. **`codex-subscription` refused to run outside a Git repository** ("Not inside a trusted
120
+ directory and --skip-git-repo-check was not specified"). Now passes `--skip-git-repo-check`;
121
+ this only relaxes the where-may-I-run guard, not the sandbox `codex exec` still runs
122
+ `approval: never` + read-only, the property this provider depends on.
123
+ 3. **`codex exec` hung on stdin.** It drains stdin ("Reading additional input from stdin…") and
124
+ blocks until EOF; `runCliPrompt` left the child's stdin as an open pipe, so every call ran to
125
+ the timeout and was killed confirmed: a prompt the CLI answers in ~8s "timed out" at 15s.
126
+ `runCliPrompt` now gives every child an already-closed stdin (`stdio: ['ignore', …]`),
127
+ harmless for `agent`/`kiro-cli`, the fix for `codex`.
128
+ 4. **The JSON parsers failed on conversational output.** `parseSuggestion` /
129
+ `parseVisionSuggestion` / `parseActionTacticSuggestion` extracted JSON from prose with a
130
+ greedy first-`{`-to-last-`}` slice, which breaks the moment the response contains more than
131
+ one JSON object exactly what cursor's `agent` produces (it echoes the system prompt's
132
+ example objects back before its real answer). Replaced with a balanced-brace-span scan that
133
+ takes the last valid object (a model states its answer last), old greedy slice kept as a last
134
+ resort. Improves robustness for every provider, not just the CLI ones.
135
+
136
+ Verified live, all authenticated: `doctor` reports `[OK] Connected` for all three; a real
137
+ `npx playwright test` heal (broken locator, cache cleared) went green with
138
+ `HEALED [provider=codex-subscription]`, and `kiro-subscription` healed the same spec.
139
+
140
+ - **`cursor-subscription` reliability note.** Verifying it (see above) showed Cursor's `agent` CLI
141
+ is an interactive assistant rather than a one-shot completion endpoint given the heal prompt it
142
+ often answers conversationally instead of with the required JSON, so the heal is declined more
143
+ often than with `kiro`/`codex`. It's now invoked in the safest/best-effort form
144
+ (`agent -p --trust --mode ask --output-format json`, read-only) and is marked **experimental**;
145
+ the docs point to `kiro`/`codex` as the steadier local-subscription options. Report issues.
146
+ Also: the full e2e healing suite (12 tests) was run end to end against `copilot-subscription` and
147
+ `ollama` **12/12 each** to confirm the parser rewrite and provider changes don't regress it.
148
+
149
+ ### Added
150
+
151
+ - **`npx tamash-playwright init-skill` — one command to install the orchestration skill, plus a
152
+ `doctor` check that flags when it's missing or stale.** The skill this package ships
153
+ (`skills/tamash-playwright/`) was previously a copy-this-shell-one-liner-per-agent step buried in
154
+ `SKILL.md`, with nothing verifying it had been done so a project could `npm install` the
155
+ package, never copy the skill, and its assistant would silently never load the workflow.
156
+ `init-skill` copies the skill (`SKILL.md` + `references/`) into **both** standard locations —
157
+ `.claude/skills/tamash-playwright/` (Claude Code) and `.agents/skills/tamash-playwright/` (the
158
+ emerging cross-tool standard, read by Cursor, GitHub Copilot, Windsurf, Kiro, Zed, dotnet Aspire,
159
+ the same convention Playwright's own `playwright-cli install --skills` uses). Same content in
160
+ both; no per-agent format conversion. `--target claude` / `--target agents` installs one; `--user`
161
+ installs under your home directory; `--force` overwrites a hand-edited copy; `--dry-run` previews.
162
+ Each install carries a `tamash-playwright-skill-version:` marker; `doctor`'s **Skill** section
163
+ reads it and reports `[OK]` current / `[WARN]` behind-the-package-version / `[INFO]` not-installed
164
+ or unmanaged, with the same one-line fix. Covered by unit tests (`skill.test.js`, real installs
165
+ into a temp dir) and verified live end to end.
166
+
167
+ *(0.11.0-beta.7 shipped an earlier `init-skill` that instead auto-detected the agent and wrote a
168
+ Cursor-specific `.mdc` file / appended blocks to `.github/copilot-instructions.md` and
169
+ `AGENTS.md`. That approach is gone — `init-skill` and `doctor` now point out any such leftovers
170
+ from a beta.7 install so you can delete them by hand.)*
171
+
172
+ - **`doctor`'s AI Provider check now tells you *what kind* of failure it hit and what to do about
173
+ it, instead of one generic "no valid response".** Every provider gained an optional, diagnostics-
174
+ only `diagnose()` method (never on the healing hot path) that runs one trivial round trip and
175
+ reports a structured category `not-installed`, `not-authenticated`, `timeout`, `bad-model`,
176
+ `network`, `bad-response` — each of which `doctor` maps to a specific next step: the missing
177
+ `npm install`/CLI installer for a missing SDK; "run `claude login` / check your API key / confirm
178
+ your subscription is active and within quota" for a rejected request; "raise `actionTimeout`" (plus
179
+ the existing subprocess cold-start note) for a timeout; "check `<MODEL>` in your .env" for a
180
+ rejected model id; a network/proxy/base-URL hint for a connection failure. The raw error line is
181
+ still printed verbatim beneath the guidance. Previously `doctor` could only distinguish "worked"
182
+ from "didn't" a missing SDK, an expired login, a slow network and a wrong model name all
183
+ produced the same line. Providers that don't implement `diagnose()` fall back to the old
184
+ `suggestSelector()` probe (worked/didn't, plus a timing-based timeout guess). Verified live:
185
+ `openai` bad key -> `not-authenticated` (401 body shown), `openai` bad base URL -> `network`,
186
+ `copilot-subscription` bad model id -> `bad-model`, 1ms `actionTimeout` -> `timeout` with the
187
+ raise-the-timeout guidance, and all three not-installed shapes by moving the dependency out of
188
+ resolution and restoring it: `@github/copilot-sdk` (CJS `MODULE_NOT_FOUND`),
189
+ `@anthropic-ai/claude-agent-sdk` (ESM `ERR_MODULE_NOT_FOUND`), and a missing `kiro-cli` binary
190
+ (`ENOENT`) -- each surfacing `not-installed` with its exact install command.
191
+
192
+ - **Three new local-development-only subscription providers: `cursor-subscription`,
193
+ `kiro-subscription`, `codex-subscription`.** Extends the same "use what you're already paying
194
+ for" idea `claude-subscription`/`copilot-subscription` already give, to three more real
195
+ subscriptions but each is explicitly scoped to local development, never documented or
196
+ recommended for CI, for two genuinely different reasons found by checking real vendor docs before
197
+ writing any code. Cursor's and Kiro's own headless-mode docs confirm neither has a way to fully
198
+ disable tool/file/command access the way `claude-subscription`'s `tools: []`/`copilot-subscription`'s
199
+ `availableTools: []` do Cursor's `agent -p` "has access to all tools" by its own docs, and Kiro's
200
+ headless mode requires `--trust-all-tools`/`--trust-tools` specifically because there's no one to
201
+ approve a tool call unattended; `kiro-subscription` deliberately never grants that trust, so a
202
+ tool-requiring response just times out and declines rather than being granted broad access.
203
+ `codex-subscription` is local-only for a different, simpler reason: `codex exec` (used here, not
204
+ the interactive REPL) defaults to a read-only sandbox with no approval prompts — already as safe
205
+ as the two existing subscription providers — but no long-lived, subscription-only token for
206
+ unattended CI use is confirmed to exist for it yet. None of the three vendors ship a Node SDK, so
207
+ all three are `child_process` wrappers around the vendor's own CLI binary (`agent`, `kiro-cli`,
208
+ `codex`) via a new shared `runCliPrompt` helper, rather than the SDK-based design the existing two
209
+ use meaning every call pays a fresh process-spawn cost, not just the first the way
210
+ `copilot-subscription`'s warm shared client does; `doctor` now has a distinct hint explaining this
211
+ when one of these three fails within a tight `actionTimeout`. Investigated and explicitly ruled
212
+ out this round: Antigravity its dedicated `google-antigravity` SDK is Python-only (this is a
213
+ Node/TypeScript package), has no documented tool-restricted mode at all, and its authentication
214
+ story isn't published, so it isn't even confirmed to ride on an existing subscription. Verified:
215
+ real graceful-decline behavior confirmed against genuinely uninstalled CLIs (no fabricated
216
+ simulation), and the full `doctor` integration (including the new every-call-cost hint) confirmed
217
+ against a real sample repo. **Not yet verified**: the actual happy path against a real installed
218
+ and authenticated CLI for any of the three none were available in the development environment,
219
+ and all three require a real paid subscription account.
220
+
221
+ - **A skill for running this package's local workflow inside an AI coding assistant.** Ships at
222
+ `skills/tamash-playwright/` a `SKILL.md` entry point that branches on `npx tamash-playwright
223
+ doctor`'s actual output (never assumed), plus two reference docs: `onboarding.md` (bringing a
224
+ project up to standard provider setup, `actionTimeout`, `.describe()` labels, Page Object
225
+ extraction) and `heal.md` (a gated review/apply/verify/land loop over `apply-heals` and
226
+ `verify-heals.cjs` most runs proceed start to finish unattended, pausing only for a genuinely
227
+ ambiguous fix or anything after a failed verification, and never landing/committing/opening a PR
228
+ without asking first, no matter how clean the run was). Pure orchestration over commands that
229
+ already exist no new healing capability. Ships with adapters for Claude Code, Kiro (identical
230
+ `SKILL.md` format, confirmed against Kiro's own docs), Cursor (`.mdc` rule), GitHub Copilot
231
+ (`copilot-instructions.md` section), and an `AGENTS.md` covering the broader cross-tool standard
232
+ (Antigravity, Gemini CLI, Windsurf, Zed, Aider, and others) none of which are auto-discovered
233
+ from `node_modules` by any of these tools, confirmed rather than assumed, so every adapter's
234
+ install step is one explicit copy command, documented in `SKILL.md` itself.
235
+ - **A new `tamash` heal provider rule-based healing, no AI at all.** `HEALER_PROVIDER=tamash`
236
+ needs no API key, no subscription, and makes no network call: it resolves a broken locator by
237
+ text-matching the same description an AI provider would receive (`.describe()`, or a decoded
238
+ variable name see below) against the already-captured accessibility snapshot, then reuses the
239
+ exact same structural widening (`near`/`adjacent`) the AI-backed path already uses once it finds
240
+ the right anchor same output shape, same downstream code, zero duplicated logic. It shares the
241
+ same non-negotiable discipline as every text-matching step in this package: zero or more than one
242
+ match, at any point, means it declines rather than guesses verified with a dedicated permanent
243
+ e2e suite covering direct matches, widened matches, and every decline path (including the known,
244
+ accepted limitation that a misleading type hint can't be second-guessed against). Genuinely a
245
+ different tool than the AI providers, not a free replacement for one: no vision fallback (nothing
246
+ to reason over a screenshot with) and no action-recovery tactics (those require understanding
247
+ *why* an action failed, which is inference this provider deliberately doesn't attempt) a fast,
248
+ free, fully deterministic first line of defense, best suited to well-`.describe()`d,
249
+ Page-Object-style suites.
250
+ - **A new `ollama-local` heal provider for self-hosted Ollama servers.** `HEALER_PROVIDER=ollama-local`
251
+ targets your own `ollama serve` instance or an internal company deployment, instead of Ollama
252
+ Cloud a deliberately separate provider from `ollama`, not a flag on it, since the two have
253
+ genuinely different auth defaults: Ollama Cloud always requires `OLLAMA_API_KEY`, while
254
+ `OLLAMA_LOCAL_API_KEY` is optional, since a bare `ollama serve` has no authentication at all. Set
255
+ it only if your internal deployment sits behind a reverse proxy or API gateway that requires a
256
+ bearer token — verified against a real HTTP server both ways, confirming the `Authorization`
257
+ header is omitted entirely when no key is set, and sent correctly when one is. Prompted by a real
258
+ support request from a team wanting to use their own internally-hosted `gpt-oss:120b`.
259
+ - **Undescribed, POM-style locator variables now get a real description automatically.** When
260
+ `.describe()` was never called, the healer already fell back to the locator's own variable/
261
+ property name (`txtEmployeeId`) as its best guess at intent now that raw identifier is decoded
262
+ into the same kind of human-readable phrase `.describe()` would give: `txtEmployeeId` becomes
263
+ "Employee Id (textbox)", `submitButton` becomes "Submit (button)", recognizing both prefix-style
264
+ (`btnSubmit`) and suffix-style (`submitBtn`) naming, camelCase/snake_case/kebab-case, and correct
265
+ acronym boundaries (`employeeIDNumber` "Employee ID Number"). Falls back to the raw identifier,
266
+ exactly as before, whenever nothing meaningful survives decoding (a placeholder name like `el1`,
267
+ or a bare affix with nothing else) never a guess dressed up as a real description. Verified
268
+ live against a real, unpublished build in both sample repos (a real `copilot-subscription` call,
269
+ and after finding and fixing a genuinely stale API key along the way — a real `ollama` call),
270
+ confirming the decoded description actually reaches the configured AI provider unchanged.
271
+
272
+ ## [0.10.0] - 2026-08-27
273
+
274
+ ### Fixed
275
+
276
+ - **`waitFor()` is never sent to the AI.** It's a state check, not an action a timeout on it can
277
+ mean a genuinely broken selector, or it can mean the element correctly never reached the expected
278
+ state (verifying something does NOT appear, or a real app issue), and there's no way to tell those
279
+ apart from the error alone. `expect(locator).toBeVisible()` was already permanently excluded from
280
+ healing for exactly this reason, but never reached this code at all (a separate path from the
281
+ Proxy-intercepted actions) — `waitFor` just never got the same treatment. Real, reported case: a
282
+ user's `waitFor` on a locator verifying an absence (where timing out was the *correct* outcome)
283
+ still burned 8175 tokens across a failed text attempt and a failed vision attempt before giving
284
+ up, for a heal that could never have succeeded. Now fails fast with a clear `state-wait-not-healed`
285
+ stage and zero AI calls; a real action (fill/click/...) on the same kind of broken locator is
286
+ unaffected.
287
+ - **`claude-subscription` was burning a real, unnecessary amount of extra tokens.** The SDK's
288
+ `effort` option silently defaults to `'high'` ("deep reasoning") when left unset, and this
289
+ provider never set it — a small prompt-complexity increase (the `nearbyRef`/`nearbyText` addition
290
+ below) pushed adaptive thinking to reason unpredictably harder for a task that only ever needs to
291
+ return one line of JSON. Confirmed live via real CI runs: output tokens for the same heal went
292
+ from a 188-445 baseline up to 422-881, inconsistently, run to run. `thinking: { type: 'disabled' }`
293
+ + `effort: 'low'` removes the variability entirely verified repeatedly at a steady 28-30 output
294
+ tokens, below even the pre-regression baseline, with no change to correctness.
295
+ - **The primary ariaSnapshot capture no longer requests `boxes:true`.** Every node was paying for a
296
+ `[box=x,y,w,h]` annotation that nothing on the text/`ref` path (including the `nearbyRef`/
297
+ `adjacent`-strategy widening logic below) ever reads it's purely topological. The one real
298
+ consumer (vision's own nearest-candidate lookup) already captures its own separate, fresh
299
+ snapshot, so this is genuinely free: verified live, input tokens dropped ~23% on a large real
300
+ page (3794 2937) with no loss of accuracy, and no change on small pages (box overhead scales
301
+ with node count).
302
+
303
+ ### Added
304
+
305
+ - **Search-scoped snapshots**: before falling back to the full page, the healer now searches the
306
+ already-captured snapshot for the description's identifying phrase and, only when it matches
307
+ exactly one node, sends the AI a scoped excerpt (that node's own subtree plus every sibling
308
+ branch's subtree at each ancestor level up to the root) instead of the whole page — zero extra
309
+ browser round-trips, since it's pure processing on data already in memory. Falls back to the full
310
+ snapshot automatically whenever the search is empty or ambiguous, never a guess. Verified live
311
+ across every real failure mode found this session: 41% token reduction on a deeply-nested field,
312
+ 66% when the target was inside what looked like an unrelated navigation menu (proving it finds
313
+ wherever the relevant text actually is, not "excludes the nav"), ~29% on a pair of identical
314
+ sibling fields that still had to be correctly disambiguated, and a clean, correct fallback when
315
+ the description doesn't match the page's real text at all.
316
+ - **A new `adjacent` selector strategy**, fixing a real ambiguity in the existing `near` strategy:
317
+ when two fields with no identity of their own share a row/section (two dropdowns side by side,
318
+ say), `near`'s "climb to a shared ancestor, then search it for any element of this role"
319
+ approach matches both and gives up rather than risk the wrong one. The AI's `ref` response can
320
+ now optionally report `nearbyRef`/`nearbyText`/`nearbyRole` for a nameless target it identified;
321
+ `deriveDurableLocator` uses that hint to find the true common ancestor between the target and
322
+ its label via each ref's own full ancestor chain, not by assuming either sits at a matching
323
+ depth and, when they're proven to be immediate sibling branches, builds a precise CSS
324
+ `:text() + *` sibling match (or an xpath climb-then-step, when the label text turns out to be
325
+ nested below its own branch root). Verified live against a real configured provider, resolving
326
+ the correct field and never its same-row neighbor in both directions.
327
+ - **Full attempt-history logging**: `SelfHealingReport` now carries an `attempts[]` array — one
328
+ entry per cache/ref/text/vision/action-recovery attempt actually made, each with its own
329
+ `succeeded`/`stage`/`error`. Previously only the *last* attempt's stage survived; an earlier
330
+ attempt's real failure (and the specific error it threw) was silently discarded the moment a
331
+ later attempt also failed the exact shape of a real user-reported bug, where a genuine
332
+ candidate selector was shown next to an unrelated `vision_provider_error` with no way to tell
333
+ why the candidate itself hadn't worked. The attempt history now also prints directly to
334
+ console/CI output (not just the JSON attachment) whenever more than one attempt was made, on
335
+ both pass and fail — since that plain-text output is what most bug reports actually paste, not
336
+ an attachment nobody opens.
337
+ - **`ariaSnapshot` is attached to the test report on failure** the exact accessibility tree the
338
+ AI reasoned over, so a confusing report can be diagnosed against real evidence instead of a
339
+ separately-captured DevTools screenshot.
340
+
341
+ ## [0.9.0] - 2026-08-26
342
+
343
+ ### Fixed
344
+
345
+ - **Heal-log visibility**: a heal with no reusable selector (a one-shot `ref` resolution, or a
346
+ vision-tagged point) was silently dropped from `heals.jsonl` entirely, even though it genuinely
347
+ fixed that run. Now logged for audit with a clear `reviewNote` whenever it has a suggestion
348
+ *or* a review note, without letting an audit-only entry shadow an older, real cached fix for the
349
+ same location.
350
+ - **Vision support for `claude-subscription`/`copilot-subscription`**: both providers had
351
+ `supportsVision` hardcoded to `false` regardless of model. Both SDKs genuinely support image
352
+ input (Claude Agent SDK via an image content block; Copilot SDK via a `blob` attachment) —
353
+ implemented for real and verified live with actual screenshots, not just doctor's yes/no label.
354
+ - **Argument forwarding in healed replays**: `replayAction()` only ever forwarded the first call
355
+ argument. A trailing options object (`click({ modifiers: [...] })`, `fill(value, { timeout })`,
356
+ `dispatchEvent`'s 3rd argument) was silently dropped on a healed replay changing the action's
357
+ real behavior while still reporting a clean `HEALED`.
358
+ - **`ref`-strategy replays now act through the derived durable locator, not the raw ref**:
359
+ `aria-ref=` locators never resolve for `dispatchEvent` at all (a Playwright-level limitation,
360
+ confirmed directly via `DEBUG=pw:api`), even though the identical element resolves instantly via
361
+ a `normalize()`-derived locator for that same action. Fixed by deriving the durable locator
362
+ *before* replaying and acting through it, falling back to the raw ref only when nothing durable
363
+ could be found. A related accuracy bug is fixed alongside it: the report/heal-log could
364
+ previously claim a derived selector "worked" even when its own replay had actually failed and a
365
+ fallback silently took over now only the locator that genuinely performed the action is
366
+ reported or cached.
367
+ - **`copilot-subscription` could hang a non-Playwright test runner indefinitely** its shared
368
+ client keeps a connection open across calls for performance, which Playwright's own test runner
369
+ tolerates by force-exiting regardless, but a runner like Cucumber does not, so the process never
370
+ returned even after every test had already passed. Found live running a real Cucumber suite in
371
+ CI. `closeCopilotSubscriptionClient()` is now exported from the package's own entry point so any
372
+ non-Playwright consumer can call it from their own teardown hook (e.g. Cucumber's `AfterAll`).
373
+
374
+ ### Added
375
+
376
+ - **`locator.getDurable(action?)`** resolves any locator (most usefully one built from
377
+ `aria-ref=...`) to a durable, reusable equivalent (`getByRole`/`getByLabel`/a css selector/…),
378
+ using the same derivation logic self-healing already uses internally. Throws if nothing durable
379
+ could be derived, rather than silently handing back something untrusted.
380
+ - **`apply-heals` now previews before writing**: every run — dry or real — shows a styled table of
381
+ exactly what it found (location, before, after, review status) instead of a plain scrolling log.
382
+ A real run also asks for confirmation before writing anything, but *only* at a genuine
383
+ interactive terminal — CI and any non-interactive/piped invocation proceed automatically exactly
384
+ as before, so no existing unattended CI workflow is affected. `--yes`/`-y` skips the prompt at a
385
+ real terminal too.
386
+ - **`doctor`'s output is styled** with colors and tables (including a new end-of-run Summary
387
+ section) instead of a plain scrolling log.
388
+ - **Exact locator code in console lines and reports**: a healed suggestion is now shown as the
389
+ real, copy-pasteable Playwright call (`getByRole("textbox", { name: "Username" })`) instead of
390
+ an abbreviated shorthand (`role:textbox:Username`) the same code `apply-heals` would write to
391
+ source, so what you see is never a lossier stand-in for what was actually used.
392
+ - A canonical Page Object Model usage example (`tests/pages/` +
393
+ `tests/example-orangehrm-add-employee.spec.ts`), verified live against a real OrangeHRM demo,
394
+ including a genuine self-heal via the `near`/widening strategy on a real unlabeled field.
395
+
396
+ ### Changed
397
+
398
+ - README/usage.md's vision-fallback documentation now correctly lists `claude-subscription`/
399
+ `copilot-subscription` as vision-capable (with a suitable model), not just the API-key providers.
400
+
401
+ ## [0.8.0] - 2026-08-25
402
+
403
+ ### Added
404
+
405
+ - `claude-subscription` and `copilot-subscription` heal providers self-healing backed by a
406
+ personal Claude or GitHub Copilot subscription instead of a pay-per-token API key, working both
407
+ locally and unattended in CI (`CLAUDE_CODE_OAUTH_TOKEN` for Claude; the ambient `GITHUB_TOKEN` or
408
+ a personal-account PAT for Copilot on GitHub Actions).
409
+
410
+ ### Fixed
411
+
412
+ - Missing CLI install step in setup docs for `claude-subscription`/`copilot-subscription` the
413
+ standalone `claude`/`copilot` CLIs are genuinely required for the login step even though the SDKs
414
+ don't need them installed to function at runtime.
415
+
416
+ ### Documentation
417
+
418
+ - The org-vs-personal-account Copilot licensing gotcha in CI (a repo owned by an organization
419
+ without its own Copilot enablement can't use a personal subscription via the ambient token).
420
+ - Action Recovery (`HEALER_ACTION_RECOVERY_ENABLED`) hidden from docs and `doctor` output
421
+ disabled by default already; this only reduced its visibility, no behavior change.
422
+
423
+ ## [0.7.0] - 2026-08-21
424
+
425
+ ### Added
426
+
427
+ - `apply-heals` turns a runtime heal into a permanent source-code fix, plus an opportunistic
428
+ cache (`heals.jsonl`) so a previously-confirmed selector is tried before a fresh AI call, and
429
+ history archival so a run's report/log isn't silently overwritten by the next one.
430
+ - Position/relationship-based locator strategies (`near`, `scoped`, `containing`) for elements with
431
+ no accessible identity of their own.
432
+ - Self-healing rebuilt around `ariaSnapshot({ mode: 'ai' })` + `aria-ref=` resolution and
433
+ `Locator.normalize()`, replacing pixel-distance guessing for both the text path and the vision
434
+ fallback's durability upgrade.
435
+ - A real unit test suite (32 tests at the time, zero new dependencies).
436
+
437
+ ### Fixed
438
+
439
+ - `doctor`'s `actionTimeout` check being fooled by a commented-out config value.
440
+ - `apply-heals --logs-dir` silently losing raw heal-log archival.
441
+ - `apply-heals` only replacing the first call when re-healing an already-`near`-fixed line.
442
+ - `apply-heals` `ENOENT` on a fresh CI checkout with no `.tamash-playwright/` directory yet — the
443
+ sharded "apply-heals" CI job checks out fresh and has never run tests itself, so the directory
444
+ genuinely doesn't exist there the way it always does in every local recipe.
445
+ - File path corruption (and a `require()` crash in the generated verification script) for ESM
446
+ (`"type": "module"`) consumer projects `Error.stack` renders as a `file://` URL there, which
447
+ broke path resolution and a plain `.js` verification script alike.
448
+ - The AI being misled by the broken selector still present in its own prompt context; a failed
449
+ label guess now auto-upgrades to a structural `near` match instead of giving up.
450
+
451
+ ### Verified
452
+
453
+ - All four API-key providers (Ollama, OpenAI, Anthropic, Gemini) confirmed working with real API
454
+ calls, not just documented.
455
+
456
+ ## [0.6.0] - 2026-08-17
457
+
458
+ ### Added
459
+
460
+ - iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
461
+ enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
462
+ - `doctor`'s `actionTimeout` configuration check.
463
+
464
+ ### Initial release
465
+
466
+ - Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
467
+ with `.describe()` for human-readable context.