tamash-playwright 0.12.0-beta.1 → 0.12.0-beta.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,452 +1,460 @@
1
- # Changelog
2
-
3
- All notable changes to this project are documented here. Format loosely follows
4
- [Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
5
-
6
- ## [0.12.0-beta.1] - 2026-09-08
7
-
8
- ### Added
9
-
10
- - **Playwright 1.63 support.** Dev/CI now tracks `@playwright/test` 1.63; `peerDependencies`
11
- stays `>=1.40.0`. Verified: existing iframe healing (`page.frameLocator('#id')`) still heals,
12
- and the full unit + representative e2e suite passes against 1.63.
13
-
14
- - **Healing through Playwright 1.63's no-argument `page.frameLocator()`.** 1.63 made
15
- `frameLocator()`'s selector optional — with no argument it matches inside *any* frame on the
16
- page. As a healing scope that's ambiguous the moment the page has more than one frame (the
17
- healer's `locator('body')` snapshot then throws *"frameLocator() matched elements in multiple
18
- frames"*, and healing silently no-op'd with `stage=no_snapshot`). Now, when the page has
19
- exactly one subframe, the healer resolves the no-arg `frameLocator()` to that concrete frame
20
- and heals normally. With zero or several subframes it can't know which frame was meant, so it
21
- steps aside cleanly — the original error is re-thrown, never a wrong-frame guess. For healable
22
- work on a multi-frame page, pass an explicit selector: `page.frameLocator('#id')`.
23
-
24
- ### Fixed
25
-
26
- - **`describeFactoryCall` rendered `iframe "undefined"` for a no-arg `frameLocator()`.** The
27
- factory-call label helper assumed a selector argument was always present; with 1.63's optional
28
- selector it produced the literal string `iframe "undefined"`. Now renders `any iframe`. (Latent
29
- — the label isn't currently surfaced to users or the model, but it would be the moment any
30
- "healing inside iframe X" context is added to a report or prompt.)
31
-
32
- ## [0.11.0] - 2026-08-31
33
-
34
- ### Fixed
35
-
36
- - **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
37
-
38
- - **`doctor`'s connectivity check gave false confidence for `claude-subscription`/`copilot-subscription`.**
39
- It tested against a fixed, generous 15s timeout, completely decoupled from the project's real
40
- configured `actionTimeout` — so a project using the README's own example value (`actionTimeout:
41
- 8000`) could get `[OK] Connected successfully` from `doctor`, then have its very first real heal
42
- in a fresh test run fail with a misleading "not authenticated?" warning, because these two
43
- providers spawn the vendor's own CLI as a subprocess and the first call in a process pays a
44
- one-time cold-start cost (spawning + authenticating) on top of the actual model call — a cost
45
- `doctor`'s own generous timeout never had to absorb. Found via a real, unbriefed agent walkthrough
46
- of a from-scratch setup (see the skill entry below): it hit this exact failure, correctly
47
- diagnosed the root cause from a source-code comment nothing user-facing ever surfaced, and worked
48
- around it by raising `actionTimeout` to 20000ms. Fixed at the actual source: `doctor` now tests
49
- connectivity within the project's real `actionTimeout` when one's configured (falling back to the
50
- old generous default only when none is set yet), and a `[FAIL]` for either of these two providers
51
- now explicitly suggests it may be the cold-start cost rather than a real auth problem. Verified
52
- against the real `copilot-subscription` provider — confirmed the check now genuinely uses the
53
- configured timeout (not just that it compiles); could not force a deterministic before/after
54
- repro of the timeout itself in this environment, since the underlying race is timing/environment-
55
- dependent (a warm local CLI session can mask it, exactly as the original cold-start comment
56
- already described).
57
-
58
- - **The published package was silently carrying stale, orphaned compiled output.** `npm run build`
59
- (plain `tsc`, no `--build`/incremental mode) never deletes `dist/` output for a source file that
60
- was later removed — found in the `0.11.0-beta.1` publish itself: `dist/healer/providers/
61
- claude-cli.js`/`copilot-cli.js` shipped in the tarball with no corresponding source file
62
- anywhere in the repo (from an abandoned, never-committed, never-wired-up piece of past work) —
63
- dead code nothing actually imports, but real, needless bloat in every install. Fixed by cleaning
64
- `dist/` automatically before every build (`prebuild` script), verified by confirming a full
65
- rebuild no longer produces those two files.
66
-
67
- - **A popup/new tab opened via `context.waitForEvent('page')`, `page.on('popup', ...)`, or
68
- `page.waitForEvent('popup')` was never healing-aware.** `bindContext`/`bindPageActions` already
69
- made `context.newPage()`/`context.pages()`/`context.on('page', ...)` heal correctly, but these
70
- three — including `context.waitForEvent('page')`, the pattern most commonly taught for popup
71
- handling — fell through to a raw, unwrapped Playwright page. A broken locator inside the
72
- resulting page just threw a plain timeout, with no `[self-healer]` line at all: healing was never
73
- attempted, not just unsuccessful. Found while building the first-ever test for popup healing
74
- (previously implemented with zero coverage); fixed by extending both bindings to wrap whatever
75
- page these three hand back, the same way `newPage()` already did.
76
-
77
- - **`tamash` (the rule-based provider) declined on a real, common page pattern it should have
78
- resolved.** A floating label rendered as its own text node right next to a field whose accessible
79
- name is *also* that same text (confirmed on a real login page: a "Username" label next to a field
80
- that's itself named "Username") was miscounted as two competing candidates and declined, even
81
- though there's genuinely only one field. `findRuleBasedMatch` now only treats a match as a real
82
- competing candidate when it's independently resolvable on its own (has a ref and a plausible
83
- role) — a bare text node repeating the same words is decorative, not a second option. Genuine
84
- ambiguity (two actually-independent candidates) still declines exactly as before.
85
-
86
- - **The HTML report's self-healing attachment was a wall of escape sequences on a failed
87
- heal.** Playwright's own error messages embed ANSI colour codes in their "Call log" section;
88
- the healer stored those raw in `report.reason` / `report.warning` / `attempts[].error`, and
89
- `JSON.stringify` (which builds the `self-healing-<action>` attachment) escapes every ESC byte in the JSON, so a *not-healed* report rendered as unreadable escape soup. A
90
- *successful* heal has short or absent error strings, which is why it only showed up on failures.
91
- Fixed by stripping ANSI CSI sequences in `normalizeError` — the single point every captured
92
- error passes through — so the attachment, the console line, and `heals.jsonl` are all plain
93
- text. Verified against a real not-healed run.
94
-
95
- - **The three CLI-based subscription providers (`cursor-subscription`, `kiro-subscription`,
96
- `codex-subscription`) were verified against their real vendor CLIs for the first time — and none
97
- of the three actually worked as shipped.** Four separate bugs, all found by running them, all now
98
- fixed:
99
- 1. **`cursor-subscription` never returned a heal.** `agent -p` refuses to run in any directory
100
- without workspace trust — it prints "Workspace Trust Required" and exits without answering.
101
- Now passes `--trust --mode ask`; `--mode ask` is Cursor's own read-only Q&A mode ("read-only"
102
- per its `--help`), which *removes* the old "has access to all tools, could edit a file"
103
- caveat this provider carried — a heal call in `ask` mode cannot write or run anything.
104
- 2. **`codex-subscription` refused to run outside a Git repository** ("Not inside a trusted
105
- directory and --skip-git-repo-check was not specified"). Now passes `--skip-git-repo-check`;
106
- this only relaxes the where-may-I-run guard, not the sandbox — `codex exec` still runs
107
- `approval: never` + read-only, the property this provider depends on.
108
- 3. **`codex exec` hung on stdin.** It drains stdin ("Reading additional input from stdin…") and
109
- blocks until EOF; `runCliPrompt` left the child's stdin as an open pipe, so every call ran to
110
- the timeout and was killed — confirmed: a prompt the CLI answers in ~8s "timed out" at 15s.
111
- `runCliPrompt` now gives every child an already-closed stdin (`stdio: ['ignore', …]`),
112
- harmless for `agent`/`kiro-cli`, the fix for `codex`.
113
- 4. **The JSON parsers failed on conversational output.** `parseSuggestion` /
114
- `parseVisionSuggestion` / `parseActionTacticSuggestion` extracted JSON from prose with a
115
- greedy first-`{`-to-last-`}` slice, which breaks the moment the response contains more than
116
- one JSON object — exactly what cursor's `agent` produces (it echoes the system prompt's
117
- example objects back before its real answer). Replaced with a balanced-brace-span scan that
118
- takes the last valid object (a model states its answer last), old greedy slice kept as a last
119
- resort. Improves robustness for every provider, not just the CLI ones.
120
-
121
- Verified live, all authenticated: `doctor` reports `[OK] Connected` for all three; a real
122
- `npx playwright test` heal (broken locator, cache cleared) went green with
123
- `HEALED [provider=codex-subscription]`, and `kiro-subscription` healed the same spec.
124
-
125
- - **`cursor-subscription` reliability note.** Verifying it (see above) showed Cursor's `agent` CLI
126
- is an interactive assistant rather than a one-shot completion endpoint — given the heal prompt it
127
- often answers conversationally instead of with the required JSON, so the heal is declined more
128
- often than with `kiro`/`codex`. It's now invoked in the safest/best-effort form
129
- (`agent -p --trust --mode ask --output-format json`, read-only) and is marked **experimental**;
130
- the docs point to `kiro`/`codex` as the steadier local-subscription options. Report issues.
131
- Also: the full e2e healing suite (12 tests) was run end to end against `copilot-subscription` and
132
- `ollama` — **12/12 each** — to confirm the parser rewrite and provider changes don't regress it.
133
-
134
- ### Added
135
-
136
- - **`npx tamash-playwright init-skill` — one command to install the orchestration skill, plus a
137
- `doctor` check that flags when it's missing or stale.** The skill this package ships
138
- (`skills/tamash-playwright/`) was previously a copy-this-shell-one-liner-per-agent step buried in
139
- `SKILL.md`, with nothing verifying it had been done — so a project could `npm install` the
140
- package, never copy the skill, and its assistant would silently never load the workflow.
141
- `init-skill` copies the skill (`SKILL.md` + `references/`) into **both** standard locations —
142
- `.claude/skills/tamash-playwright/` (Claude Code) and `.agents/skills/tamash-playwright/` (the
143
- emerging cross-tool standard, read by Cursor, GitHub Copilot, Windsurf, Kiro, Zed, dotnet Aspire,
144
- … — the same convention Playwright's own `playwright-cli install --skills` uses). Same content in
145
- both; no per-agent format conversion. `--target claude` / `--target agents` installs one; `--user`
146
- installs under your home directory; `--force` overwrites a hand-edited copy; `--dry-run` previews.
147
- Each install carries a `tamash-playwright-skill-version:` marker; `doctor`'s **Skill** section
148
- reads it and reports `[OK]` current / `[WARN]` behind-the-package-version / `[INFO]` not-installed
149
- or unmanaged, with the same one-line fix. Covered by unit tests (`skill.test.js`, real installs
150
- into a temp dir) and verified live end to end.
151
-
152
- *(0.11.0-beta.7 shipped an earlier `init-skill` that instead auto-detected the agent and wrote a
153
- Cursor-specific `.mdc` file / appended blocks to `.github/copilot-instructions.md` and
154
- `AGENTS.md`. That approach is gone — `init-skill` and `doctor` now point out any such leftovers
155
- from a beta.7 install so you can delete them by hand.)*
156
-
157
- - **`doctor`'s AI Provider check now tells you *what kind* of failure it hit and what to do about
158
- it, instead of one generic "no valid response".** Every provider gained an optional, diagnostics-
159
- only `diagnose()` method (never on the healing hot path) that runs one trivial round trip and
160
- reports a structured category — `not-installed`, `not-authenticated`, `timeout`, `bad-model`,
161
- `network`, `bad-response` — each of which `doctor` maps to a specific next step: the missing
162
- `npm install`/CLI installer for a missing SDK; "run `claude login` / check your API key / confirm
163
- your subscription is active and within quota" for a rejected request; "raise `actionTimeout`" (plus
164
- the existing subprocess cold-start note) for a timeout; "check `<MODEL>` in your .env" for a
165
- rejected model id; a network/proxy/base-URL hint for a connection failure. The raw error line is
166
- still printed verbatim beneath the guidance. Previously `doctor` could only distinguish "worked"
167
- from "didn't" — a missing SDK, an expired login, a slow network and a wrong model name all
168
- produced the same line. Providers that don't implement `diagnose()` fall back to the old
169
- `suggestSelector()` probe (worked/didn't, plus a timing-based timeout guess). Verified live:
170
- `openai` bad key -> `not-authenticated` (401 body shown), `openai` bad base URL -> `network`,
171
- `copilot-subscription` bad model id -> `bad-model`, 1ms `actionTimeout` -> `timeout` with the
172
- raise-the-timeout guidance, and all three not-installed shapes by moving the dependency out of
173
- resolution and restoring it: `@github/copilot-sdk` (CJS `MODULE_NOT_FOUND`),
174
- `@anthropic-ai/claude-agent-sdk` (ESM `ERR_MODULE_NOT_FOUND`), and a missing `kiro-cli` binary
175
- (`ENOENT`) -- each surfacing `not-installed` with its exact install command.
176
-
177
- - **Three new local-development-only subscription providers: `cursor-subscription`,
178
- `kiro-subscription`, `codex-subscription`.** Extends the same "use what you're already paying
179
- for" idea `claude-subscription`/`copilot-subscription` already give, to three more real
180
- subscriptions — but each is explicitly scoped to local development, never documented or
181
- recommended for CI, for two genuinely different reasons found by checking real vendor docs before
182
- writing any code. Cursor's and Kiro's own headless-mode docs confirm neither has a way to fully
183
- disable tool/file/command access the way `claude-subscription`'s `tools: []`/`copilot-subscription`'s
184
- `availableTools: []` do — Cursor's `agent -p` "has access to all tools" by its own docs, and Kiro's
185
- headless mode requires `--trust-all-tools`/`--trust-tools` specifically because there's no one to
186
- approve a tool call unattended; `kiro-subscription` deliberately never grants that trust, so a
187
- tool-requiring response just times out and declines rather than being granted broad access.
188
- `codex-subscription` is local-only for a different, simpler reason: `codex exec` (used here, not
189
- the interactive REPL) defaults to a read-only sandbox with no approval prompts — already as safe
190
- as the two existing subscription providers — but no long-lived, subscription-only token for
191
- unattended CI use is confirmed to exist for it yet. None of the three vendors ship a Node SDK, so
192
- all three are `child_process` wrappers around the vendor's own CLI binary (`agent`, `kiro-cli`,
193
- `codex`) via a new shared `runCliPrompt` helper, rather than the SDK-based design the existing two
194
- use — meaning every call pays a fresh process-spawn cost, not just the first the way
195
- `copilot-subscription`'s warm shared client does; `doctor` now has a distinct hint explaining this
196
- when one of these three fails within a tight `actionTimeout`. Investigated and explicitly ruled
197
- out this round: Antigravity — its dedicated `google-antigravity` SDK is Python-only (this is a
198
- Node/TypeScript package), has no documented tool-restricted mode at all, and its authentication
199
- story isn't published, so it isn't even confirmed to ride on an existing subscription. Verified:
200
- real graceful-decline behavior confirmed against genuinely uninstalled CLIs (no fabricated
201
- simulation), and the full `doctor` integration (including the new every-call-cost hint) confirmed
202
- against a real sample repo. **Not yet verified**: the actual happy path against a real installed
203
- and authenticated CLI for any of the three — none were available in the development environment,
204
- and all three require a real paid subscription account.
205
-
206
- - **A skill for running this package's local workflow inside an AI coding assistant.** Ships at
207
- `skills/tamash-playwright/` — a `SKILL.md` entry point that branches on `npx tamash-playwright
208
- doctor`'s actual output (never assumed), plus two reference docs: `onboarding.md` (bringing a
209
- project up to standard — provider setup, `actionTimeout`, `.describe()` labels, Page Object
210
- extraction) and `heal.md` (a gated review/apply/verify/land loop over `apply-heals` and
211
- `verify-heals.cjs` — most runs proceed start to finish unattended, pausing only for a genuinely
212
- ambiguous fix or anything after a failed verification, and never landing/committing/opening a PR
213
- without asking first, no matter how clean the run was). Pure orchestration over commands that
214
- already exist — no new healing capability. Ships with adapters for Claude Code, Kiro (identical
215
- `SKILL.md` format, confirmed against Kiro's own docs), Cursor (`.mdc` rule), GitHub Copilot
216
- (`copilot-instructions.md` section), and an `AGENTS.md` covering the broader cross-tool standard
217
- (Antigravity, Gemini CLI, Windsurf, Zed, Aider, and others) — none of which are auto-discovered
218
- from `node_modules` by any of these tools, confirmed rather than assumed, so every adapter's
219
- install step is one explicit copy command, documented in `SKILL.md` itself.
220
- - **A new `tamash` heal provider — rule-based healing, no AI at all.** `HEALER_PROVIDER=tamash`
221
- needs no API key, no subscription, and makes no network call: it resolves a broken locator by
222
- text-matching the same description an AI provider would receive (`.describe()`, or a decoded
223
- variable name — see below) against the already-captured accessibility snapshot, then reuses the
224
- exact same structural widening (`near`/`adjacent`) the AI-backed path already uses once it finds
225
- the right anchor — same output shape, same downstream code, zero duplicated logic. It shares the
226
- same non-negotiable discipline as every text-matching step in this package: zero or more than one
227
- match, at any point, means it declines rather than guesses — verified with a dedicated permanent
228
- e2e suite covering direct matches, widened matches, and every decline path (including the known,
229
- accepted limitation that a misleading type hint can't be second-guessed against). Genuinely a
230
- different tool than the AI providers, not a free replacement for one: no vision fallback (nothing
231
- to reason over a screenshot with) and no action-recovery tactics (those require understanding
232
- *why* an action failed, which is inference this provider deliberately doesn't attempt) — a fast,
233
- free, fully deterministic first line of defense, best suited to well-`.describe()`d,
234
- Page-Object-style suites.
235
- - **A new `ollama-local` heal provider for self-hosted Ollama servers.** `HEALER_PROVIDER=ollama-local`
236
- targets your own `ollama serve` instance or an internal company deployment, instead of Ollama
237
- Cloud — a deliberately separate provider from `ollama`, not a flag on it, since the two have
238
- genuinely different auth defaults: Ollama Cloud always requires `OLLAMA_API_KEY`, while
239
- `OLLAMA_LOCAL_API_KEY` is optional, since a bare `ollama serve` has no authentication at all. Set
240
- it only if your internal deployment sits behind a reverse proxy or API gateway that requires a
241
- bearer token — verified against a real HTTP server both ways, confirming the `Authorization`
242
- header is omitted entirely when no key is set, and sent correctly when one is. Prompted by a real
243
- support request from a team wanting to use their own internally-hosted `gpt-oss:120b`.
244
- - **Undescribed, POM-style locator variables now get a real description automatically.** When
245
- `.describe()` was never called, the healer already fell back to the locator's own variable/
246
- property name (`txtEmployeeId`) as its best guess at intent — now that raw identifier is decoded
247
- into the same kind of human-readable phrase `.describe()` would give: `txtEmployeeId` becomes
248
- "Employee Id (textbox)", `submitButton` becomes "Submit (button)", recognizing both prefix-style
249
- (`btnSubmit`) and suffix-style (`submitBtn`) naming, camelCase/snake_case/kebab-case, and correct
250
- acronym boundaries (`employeeIDNumber` → "Employee ID Number"). Falls back to the raw identifier,
251
- exactly as before, whenever nothing meaningful survives decoding (a placeholder name like `el1`,
252
- or a bare affix with nothing else) — never a guess dressed up as a real description. Verified
253
- live against a real, unpublished build in both sample repos (a real `copilot-subscription` call,
254
- and — after finding and fixing a genuinely stale API key along the way — a real `ollama` call),
255
- confirming the decoded description actually reaches the configured AI provider unchanged.
256
-
257
- ## [0.10.0] - 2026-08-27
258
-
259
- ### Fixed
260
-
261
- - **`waitFor()` is never sent to the AI.** It's a state check, not an action — a timeout on it can
262
- mean a genuinely broken selector, or it can mean the element correctly never reached the expected
263
- state (verifying something does NOT appear, or a real app issue), and there's no way to tell those
264
- apart from the error alone. `expect(locator).toBeVisible()` was already permanently excluded from
265
- healing for exactly this reason, but never reached this code at all (a separate path from the
266
- Proxy-intercepted actions) — `waitFor` just never got the same treatment. Real, reported case: a
267
- user's `waitFor` on a locator verifying an absence (where timing out was the *correct* outcome)
268
- still burned 8175 tokens across a failed text attempt and a failed vision attempt before giving
269
- up, for a heal that could never have succeeded. Now fails fast with a clear `state-wait-not-healed`
270
- stage and zero AI calls; a real action (fill/click/...) on the same kind of broken locator is
271
- unaffected.
272
- - **`claude-subscription` was burning a real, unnecessary amount of extra tokens.** The SDK's
273
- `effort` option silently defaults to `'high'` ("deep reasoning") when left unset, and this
274
- provider never set it — a small prompt-complexity increase (the `nearbyRef`/`nearbyText` addition
275
- below) pushed adaptive thinking to reason unpredictably harder for a task that only ever needs to
276
- return one line of JSON. Confirmed live via real CI runs: output tokens for the same heal went
277
- from a 188-445 baseline up to 422-881, inconsistently, run to run. `thinking: { type: 'disabled' }`
278
- + `effort: 'low'` removes the variability entirely — verified repeatedly at a steady 28-30 output
279
- tokens, below even the pre-regression baseline, with no change to correctness.
280
- - **The primary ariaSnapshot capture no longer requests `boxes:true`.** Every node was paying for a
281
- `[box=x,y,w,h]` annotation that nothing on the text/`ref` path (including the `nearbyRef`/
282
- `adjacent`-strategy widening logic below) ever reads — it's purely topological. The one real
283
- consumer (vision's own nearest-candidate lookup) already captures its own separate, fresh
284
- snapshot, so this is genuinely free: verified live, input tokens dropped ~23% on a large real
285
- page (3794 → 2937) with no loss of accuracy, and no change on small pages (box overhead scales
286
- with node count).
287
-
288
- ### Added
289
-
290
- - **Search-scoped snapshots**: before falling back to the full page, the healer now searches the
291
- already-captured snapshot for the description's identifying phrase and, only when it matches
292
- exactly one node, sends the AI a scoped excerpt (that node's own subtree plus every sibling
293
- branch's subtree at each ancestor level up to the root) instead of the whole page — zero extra
294
- browser round-trips, since it's pure processing on data already in memory. Falls back to the full
295
- snapshot automatically whenever the search is empty or ambiguous, never a guess. Verified live
296
- across every real failure mode found this session: 41% token reduction on a deeply-nested field,
297
- 66% when the target was inside what looked like an unrelated navigation menu (proving it finds
298
- wherever the relevant text actually is, not "excludes the nav"), ~29% on a pair of identical
299
- sibling fields that still had to be correctly disambiguated, and a clean, correct fallback when
300
- the description doesn't match the page's real text at all.
301
- - **A new `adjacent` selector strategy**, fixing a real ambiguity in the existing `near` strategy:
302
- when two fields with no identity of their own share a row/section (two dropdowns side by side,
303
- say), `near`'s "climb to a shared ancestor, then search it for any element of this role"
304
- approach matches both and gives up rather than risk the wrong one. The AI's `ref` response can
305
- now optionally report `nearbyRef`/`nearbyText`/`nearbyRole` for a nameless target it identified;
306
- `deriveDurableLocator` uses that hint to find the true common ancestor between the target and
307
- its label — via each ref's own full ancestor chain, not by assuming either sits at a matching
308
- depth — and, when they're proven to be immediate sibling branches, builds a precise CSS
309
- `:text() + *` sibling match (or an xpath climb-then-step, when the label text turns out to be
310
- nested below its own branch root). Verified live against a real configured provider, resolving
311
- the correct field and never its same-row neighbor in both directions.
312
- - **Full attempt-history logging**: `SelfHealingReport` now carries an `attempts[]` array — one
313
- entry per cache/ref/text/vision/action-recovery attempt actually made, each with its own
314
- `succeeded`/`stage`/`error`. Previously only the *last* attempt's stage survived; an earlier
315
- attempt's real failure (and the specific error it threw) was silently discarded the moment a
316
- later attempt also failed — the exact shape of a real user-reported bug, where a genuine
317
- candidate selector was shown next to an unrelated `vision_provider_error` with no way to tell
318
- why the candidate itself hadn't worked. The attempt history now also prints directly to
319
- console/CI output (not just the JSON attachment) whenever more than one attempt was made, on
320
- both pass and fail — since that plain-text output is what most bug reports actually paste, not
321
- an attachment nobody opens.
322
- - **`ariaSnapshot` is attached to the test report on failure** — the exact accessibility tree the
323
- AI reasoned over, so a confusing report can be diagnosed against real evidence instead of a
324
- separately-captured DevTools screenshot.
325
-
326
- ## [0.9.0] - 2026-08-26
327
-
328
- ### Fixed
329
-
330
- - **Heal-log visibility**: a heal with no reusable selector (a one-shot `ref` resolution, or a
331
- vision-tagged point) was silently dropped from `heals.jsonl` entirely, even though it genuinely
332
- fixed that run. Now logged for audit — with a clear `reviewNote` — whenever it has a suggestion
333
- *or* a review note, without letting an audit-only entry shadow an older, real cached fix for the
334
- same location.
335
- - **Vision support for `claude-subscription`/`copilot-subscription`**: both providers had
336
- `supportsVision` hardcoded to `false` regardless of model. Both SDKs genuinely support image
337
- input (Claude Agent SDK via an image content block; Copilot SDK via a `blob` attachment) —
338
- implemented for real and verified live with actual screenshots, not just doctor's yes/no label.
339
- - **Argument forwarding in healed replays**: `replayAction()` only ever forwarded the first call
340
- argument. A trailing options object (`click({ modifiers: [...] })`, `fill(value, { timeout })`,
341
- `dispatchEvent`'s 3rd argument) was silently dropped on a healed replay — changing the action's
342
- real behavior while still reporting a clean `HEALED`.
343
- - **`ref`-strategy replays now act through the derived durable locator, not the raw ref**:
344
- `aria-ref=` locators never resolve for `dispatchEvent` at all (a Playwright-level limitation,
345
- confirmed directly via `DEBUG=pw:api`), even though the identical element resolves instantly via
346
- a `normalize()`-derived locator for that same action. Fixed by deriving the durable locator
347
- *before* replaying and acting through it, falling back to the raw ref only when nothing durable
348
- could be found. A related accuracy bug is fixed alongside it: the report/heal-log could
349
- previously claim a derived selector "worked" even when its own replay had actually failed and a
350
- fallback silently took over — now only the locator that genuinely performed the action is
351
- reported or cached.
352
- - **`copilot-subscription` could hang a non-Playwright test runner indefinitely** — its shared
353
- client keeps a connection open across calls for performance, which Playwright's own test runner
354
- tolerates by force-exiting regardless, but a runner like Cucumber does not, so the process never
355
- returned even after every test had already passed. Found live running a real Cucumber suite in
356
- CI. `closeCopilotSubscriptionClient()` is now exported from the package's own entry point so any
357
- non-Playwright consumer can call it from their own teardown hook (e.g. Cucumber's `AfterAll`).
358
-
359
- ### Added
360
-
361
- - **`locator.getDurable(action?)`** — resolves any locator (most usefully one built from
362
- `aria-ref=...`) to a durable, reusable equivalent (`getByRole`/`getByLabel`/a css selector/…),
363
- using the same derivation logic self-healing already uses internally. Throws if nothing durable
364
- could be derived, rather than silently handing back something untrusted.
365
- - **`apply-heals` now previews before writing**: every run — dry or real — shows a styled table of
366
- exactly what it found (location, before, after, review status) instead of a plain scrolling log.
367
- A real run also asks for confirmation before writing anything, but *only* at a genuine
368
- interactive terminal — CI and any non-interactive/piped invocation proceed automatically exactly
369
- as before, so no existing unattended CI workflow is affected. `--yes`/`-y` skips the prompt at a
370
- real terminal too.
371
- - **`doctor`'s output is styled** with colors and tables (including a new end-of-run Summary
372
- section) instead of a plain scrolling log.
373
- - **Exact locator code in console lines and reports**: a healed suggestion is now shown as the
374
- real, copy-pasteable Playwright call (`getByRole("textbox", { name: "Username" })`) instead of
375
- an abbreviated shorthand (`role:textbox:Username`) — the same code `apply-heals` would write to
376
- source, so what you see is never a lossier stand-in for what was actually used.
377
- - A canonical Page Object Model usage example (`tests/pages/` +
378
- `tests/example-orangehrm-add-employee.spec.ts`), verified live against a real OrangeHRM demo,
379
- including a genuine self-heal via the `near`/widening strategy on a real unlabeled field.
380
-
381
- ### Changed
382
-
383
- - README/usage.md's vision-fallback documentation now correctly lists `claude-subscription`/
384
- `copilot-subscription` as vision-capable (with a suitable model), not just the API-key providers.
385
-
386
- ## [0.8.0] - 2026-08-25
387
-
388
- ### Added
389
-
390
- - `claude-subscription` and `copilot-subscription` heal providers — self-healing backed by a
391
- personal Claude or GitHub Copilot subscription instead of a pay-per-token API key, working both
392
- locally and unattended in CI (`CLAUDE_CODE_OAUTH_TOKEN` for Claude; the ambient `GITHUB_TOKEN` or
393
- a personal-account PAT for Copilot on GitHub Actions).
394
-
395
- ### Fixed
396
-
397
- - Missing CLI install step in setup docs for `claude-subscription`/`copilot-subscription` — the
398
- standalone `claude`/`copilot` CLIs are genuinely required for the login step even though the SDKs
399
- don't need them installed to function at runtime.
400
-
401
- ### Documentation
402
-
403
- - The org-vs-personal-account Copilot licensing gotcha in CI (a repo owned by an organization
404
- without its own Copilot enablement can't use a personal subscription via the ambient token).
405
- - Action Recovery (`HEALER_ACTION_RECOVERY_ENABLED`) hidden from docs and `doctor` output —
406
- disabled by default already; this only reduced its visibility, no behavior change.
407
-
408
- ## [0.7.0] - 2026-08-21
409
-
410
- ### Added
411
-
412
- - `apply-heals` — turns a runtime heal into a permanent source-code fix, plus an opportunistic
413
- cache (`heals.jsonl`) so a previously-confirmed selector is tried before a fresh AI call, and
414
- history archival so a run's report/log isn't silently overwritten by the next one.
415
- - Position/relationship-based locator strategies (`near`, `scoped`, `containing`) for elements with
416
- no accessible identity of their own.
417
- - Self-healing rebuilt around `ariaSnapshot({ mode: 'ai' })` + `aria-ref=` resolution and
418
- `Locator.normalize()`, replacing pixel-distance guessing for both the text path and the vision
419
- fallback's durability upgrade.
420
- - A real unit test suite (32 tests at the time, zero new dependencies).
421
-
422
- ### Fixed
423
-
424
- - `doctor`'s `actionTimeout` check being fooled by a commented-out config value.
425
- - `apply-heals --logs-dir` silently losing raw heal-log archival.
426
- - `apply-heals` only replacing the first call when re-healing an already-`near`-fixed line.
427
- - `apply-heals` `ENOENT` on a fresh CI checkout with no `.tamash-playwright/` directory yet — the
428
- sharded "apply-heals" CI job checks out fresh and has never run tests itself, so the directory
429
- genuinely doesn't exist there the way it always does in every local recipe.
430
- - File path corruption (and a `require()` crash in the generated verification script) for ESM
431
- (`"type": "module"`) consumer projects — `Error.stack` renders as a `file://` URL there, which
432
- broke path resolution and a plain `.js` verification script alike.
433
- - The AI being misled by the broken selector still present in its own prompt context; a failed
434
- label guess now auto-upgrades to a structural `near` match instead of giving up.
435
-
436
- ### Verified
437
-
438
- - All four API-key providers (Ollama, OpenAI, Anthropic, Gemini) confirmed working with real API
439
- calls, not just documented.
440
-
441
- ## [0.6.0] - 2026-08-17
442
-
443
- ### Added
444
-
445
- - iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
446
- enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
447
- - `doctor`'s `actionTimeout` configuration check.
448
-
449
- ### Initial release
450
-
451
- - Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
452
- with `.describe()` for human-readable context.
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here. Format loosely follows
4
+ [Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
5
+
6
+ ## [0.12.0-beta.2] - 2026-09-08
7
+
8
+ ### Added
9
+
10
+ - **Playwright 1.63 support.** Dev/CI now tracks `@playwright/test` 1.63; `peerDependencies`
11
+ stays `>=1.40.0`. Verified: existing iframe healing (`page.frameLocator('#id')`) still heals,
12
+ and the full unit + representative e2e suite passes against 1.63.
13
+
14
+ - **Healing through Playwright 1.63's no-argument `page.frameLocator()`.** 1.63 made
15
+ `frameLocator()`'s selector optional — with no argument it matches inside *any* frame on the
16
+ page. As a healing scope that's ambiguous the moment the page has more than one frame (the
17
+ healer's `locator('body')` snapshot then throws *"frameLocator() matched elements in multiple
18
+ frames"*, and healing silently no-op'd with `stage=no_snapshot`). Now, when the page has
19
+ exactly one frame, the healer re-expresses the no-arg `frameLocator()` as an explicit
20
+ single-frame `FrameLocator` and heals normally — including deriving a **durable, persistable**
21
+ cross-frame selector (`apply-heals` can write it back to source), identical to what
22
+ `page.frameLocator('#id')` produces. With zero or several frames it can't know which was meant,
23
+ so it steps aside cleanly — the original error is re-thrown, never a wrong-frame guess. For
24
+ healable work on a multi-frame page, pass an explicit selector: `page.frameLocator('#id')`.
25
+
26
+ *(`0.12.0-beta.1` collapsed the scope to a raw `page.frames()` `Frame` instead of a
27
+ `FrameLocator`. `Locator.normalize()` on a `Frame`-rooted locator returns a correct selector
28
+ string but an object that resolves to nothing, so the heal worked once at runtime but couldn't
29
+ be persisted — it fell back to a transient one-shot element reference with a `needsReview`
30
+ note. Fixed here.)*
31
+
32
+ ### Fixed
33
+
34
+ - **`describeFactoryCall` rendered `iframe "undefined"` for a no-arg `frameLocator()`.** The
35
+ factory-call label helper assumed a selector argument was always present; with 1.63's optional
36
+ selector it produced the literal string `iframe "undefined"`. Now renders `any iframe`. (Latent
37
+ — the label isn't currently surfaced to users or the model, but it would be the moment any
38
+ "healing inside iframe X" context is added to a report or prompt.)
39
+
40
+ ## [0.11.0] - 2026-08-31
41
+
42
+ ### Fixed
43
+
44
+ - **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
45
+
46
+ - **`doctor`'s connectivity check gave false confidence for `claude-subscription`/`copilot-subscription`.**
47
+ It tested against a fixed, generous 15s timeout, completely decoupled from the project's real
48
+ configured `actionTimeout` — so a project using the README's own example value (`actionTimeout:
49
+ 8000`) could get `[OK] Connected successfully` from `doctor`, then have its very first real heal
50
+ in a fresh test run fail with a misleading "not authenticated?" warning, because these two
51
+ providers spawn the vendor's own CLI as a subprocess and the first call in a process pays a
52
+ one-time cold-start cost (spawning + authenticating) on top of the actual model call — a cost
53
+ `doctor`'s own generous timeout never had to absorb. Found via a real, unbriefed agent walkthrough
54
+ of a from-scratch setup (see the skill entry below): it hit this exact failure, correctly
55
+ diagnosed the root cause from a source-code comment nothing user-facing ever surfaced, and worked
56
+ around it by raising `actionTimeout` to 20000ms. Fixed at the actual source: `doctor` now tests
57
+ connectivity within the project's real `actionTimeout` when one's configured (falling back to the
58
+ old generous default only when none is set yet), and a `[FAIL]` for either of these two providers
59
+ now explicitly suggests it may be the cold-start cost rather than a real auth problem. Verified
60
+ against the real `copilot-subscription` provider — confirmed the check now genuinely uses the
61
+ configured timeout (not just that it compiles); could not force a deterministic before/after
62
+ repro of the timeout itself in this environment, since the underlying race is timing/environment-
63
+ dependent (a warm local CLI session can mask it, exactly as the original cold-start comment
64
+ already described).
65
+
66
+ - **The published package was silently carrying stale, orphaned compiled output.** `npm run build`
67
+ (plain `tsc`, no `--build`/incremental mode) never deletes `dist/` output for a source file that
68
+ was later removed — found in the `0.11.0-beta.1` publish itself: `dist/healer/providers/
69
+ claude-cli.js`/`copilot-cli.js` shipped in the tarball with no corresponding source file
70
+ anywhere in the repo (from an abandoned, never-committed, never-wired-up piece of past work) —
71
+ dead code nothing actually imports, but real, needless bloat in every install. Fixed by cleaning
72
+ `dist/` automatically before every build (`prebuild` script), verified by confirming a full
73
+ rebuild no longer produces those two files.
74
+
75
+ - **A popup/new tab opened via `context.waitForEvent('page')`, `page.on('popup', ...)`, or
76
+ `page.waitForEvent('popup')` was never healing-aware.** `bindContext`/`bindPageActions` already
77
+ made `context.newPage()`/`context.pages()`/`context.on('page', ...)` heal correctly, but these
78
+ three — including `context.waitForEvent('page')`, the pattern most commonly taught for popup
79
+ handling — fell through to a raw, unwrapped Playwright page. A broken locator inside the
80
+ resulting page just threw a plain timeout, with no `[self-healer]` line at all: healing was never
81
+ attempted, not just unsuccessful. Found while building the first-ever test for popup healing
82
+ (previously implemented with zero coverage); fixed by extending both bindings to wrap whatever
83
+ page these three hand back, the same way `newPage()` already did.
84
+
85
+ - **`tamash` (the rule-based provider) declined on a real, common page pattern it should have
86
+ resolved.** A floating label rendered as its own text node right next to a field whose accessible
87
+ name is *also* that same text (confirmed on a real login page: a "Username" label next to a field
88
+ that's itself named "Username") was miscounted as two competing candidates and declined, even
89
+ though there's genuinely only one field. `findRuleBasedMatch` now only treats a match as a real
90
+ competing candidate when it's independently resolvable on its own (has a ref and a plausible
91
+ role) — a bare text node repeating the same words is decorative, not a second option. Genuine
92
+ ambiguity (two actually-independent candidates) still declines exactly as before.
93
+
94
+ - **The HTML report's self-healing attachment was a wall of escape sequences on a failed
95
+ heal.** Playwright's own error messages embed ANSI colour codes in their "Call log" section;
96
+ the healer stored those raw in `report.reason` / `report.warning` / `attempts[].error`, and
97
+ `JSON.stringify` (which builds the `self-healing-<action>` attachment) escapes every ESC byte in the JSON, so a *not-healed* report rendered as unreadable escape soup. A
98
+ *successful* heal has short or absent error strings, which is why it only showed up on failures.
99
+ Fixed by stripping ANSI CSI sequences in `normalizeError` — the single point every captured
100
+ error passes through — so the attachment, the console line, and `heals.jsonl` are all plain
101
+ text. Verified against a real not-healed run.
102
+
103
+ - **The three CLI-based subscription providers (`cursor-subscription`, `kiro-subscription`,
104
+ `codex-subscription`) were verified against their real vendor CLIs for the first time — and none
105
+ of the three actually worked as shipped.** Four separate bugs, all found by running them, all now
106
+ fixed:
107
+ 1. **`cursor-subscription` never returned a heal.** `agent -p` refuses to run in any directory
108
+ without workspace trust — it prints "Workspace Trust Required" and exits without answering.
109
+ Now passes `--trust --mode ask`; `--mode ask` is Cursor's own read-only Q&A mode ("read-only"
110
+ per its `--help`), which *removes* the old "has access to all tools, could edit a file"
111
+ caveat this provider carried — a heal call in `ask` mode cannot write or run anything.
112
+ 2. **`codex-subscription` refused to run outside a Git repository** ("Not inside a trusted
113
+ directory and --skip-git-repo-check was not specified"). Now passes `--skip-git-repo-check`;
114
+ this only relaxes the where-may-I-run guard, not the sandbox — `codex exec` still runs
115
+ `approval: never` + read-only, the property this provider depends on.
116
+ 3. **`codex exec` hung on stdin.** It drains stdin ("Reading additional input from stdin…") and
117
+ blocks until EOF; `runCliPrompt` left the child's stdin as an open pipe, so every call ran to
118
+ the timeout and was killed — confirmed: a prompt the CLI answers in ~8s "timed out" at 15s.
119
+ `runCliPrompt` now gives every child an already-closed stdin (`stdio: ['ignore', …]`),
120
+ harmless for `agent`/`kiro-cli`, the fix for `codex`.
121
+ 4. **The JSON parsers failed on conversational output.** `parseSuggestion` /
122
+ `parseVisionSuggestion` / `parseActionTacticSuggestion` extracted JSON from prose with a
123
+ greedy first-`{`-to-last-`}` slice, which breaks the moment the response contains more than
124
+ one JSON object — exactly what cursor's `agent` produces (it echoes the system prompt's
125
+ example objects back before its real answer). Replaced with a balanced-brace-span scan that
126
+ takes the last valid object (a model states its answer last), old greedy slice kept as a last
127
+ resort. Improves robustness for every provider, not just the CLI ones.
128
+
129
+ Verified live, all authenticated: `doctor` reports `[OK] Connected` for all three; a real
130
+ `npx playwright test` heal (broken locator, cache cleared) went green with
131
+ `HEALED [provider=codex-subscription]`, and `kiro-subscription` healed the same spec.
132
+
133
+ - **`cursor-subscription` reliability note.** Verifying it (see above) showed Cursor's `agent` CLI
134
+ is an interactive assistant rather than a one-shot completion endpoint — given the heal prompt it
135
+ often answers conversationally instead of with the required JSON, so the heal is declined more
136
+ often than with `kiro`/`codex`. It's now invoked in the safest/best-effort form
137
+ (`agent -p --trust --mode ask --output-format json`, read-only) and is marked **experimental**;
138
+ the docs point to `kiro`/`codex` as the steadier local-subscription options. Report issues.
139
+ Also: the full e2e healing suite (12 tests) was run end to end against `copilot-subscription` and
140
+ `ollama` — **12/12 each** — to confirm the parser rewrite and provider changes don't regress it.
141
+
142
+ ### Added
143
+
144
+ - **`npx tamash-playwright init-skill` — one command to install the orchestration skill, plus a
145
+ `doctor` check that flags when it's missing or stale.** The skill this package ships
146
+ (`skills/tamash-playwright/`) was previously a copy-this-shell-one-liner-per-agent step buried in
147
+ `SKILL.md`, with nothing verifying it had been done — so a project could `npm install` the
148
+ package, never copy the skill, and its assistant would silently never load the workflow.
149
+ `init-skill` copies the skill (`SKILL.md` + `references/`) into **both** standard locations —
150
+ `.claude/skills/tamash-playwright/` (Claude Code) and `.agents/skills/tamash-playwright/` (the
151
+ emerging cross-tool standard, read by Cursor, GitHub Copilot, Windsurf, Kiro, Zed, dotnet Aspire,
152
+ … — the same convention Playwright's own `playwright-cli install --skills` uses). Same content in
153
+ both; no per-agent format conversion. `--target claude` / `--target agents` installs one; `--user`
154
+ installs under your home directory; `--force` overwrites a hand-edited copy; `--dry-run` previews.
155
+ Each install carries a `tamash-playwright-skill-version:` marker; `doctor`'s **Skill** section
156
+ reads it and reports `[OK]` current / `[WARN]` behind-the-package-version / `[INFO]` not-installed
157
+ or unmanaged, with the same one-line fix. Covered by unit tests (`skill.test.js`, real installs
158
+ into a temp dir) and verified live end to end.
159
+
160
+ *(0.11.0-beta.7 shipped an earlier `init-skill` that instead auto-detected the agent and wrote a
161
+ Cursor-specific `.mdc` file / appended blocks to `.github/copilot-instructions.md` and
162
+ `AGENTS.md`. That approach is gone — `init-skill` and `doctor` now point out any such leftovers
163
+ from a beta.7 install so you can delete them by hand.)*
164
+
165
+ - **`doctor`'s AI Provider check now tells you *what kind* of failure it hit and what to do about
166
+ it, instead of one generic "no valid response".** Every provider gained an optional, diagnostics-
167
+ only `diagnose()` method (never on the healing hot path) that runs one trivial round trip and
168
+ reports a structured category — `not-installed`, `not-authenticated`, `timeout`, `bad-model`,
169
+ `network`, `bad-response` — each of which `doctor` maps to a specific next step: the missing
170
+ `npm install`/CLI installer for a missing SDK; "run `claude login` / check your API key / confirm
171
+ your subscription is active and within quota" for a rejected request; "raise `actionTimeout`" (plus
172
+ the existing subprocess cold-start note) for a timeout; "check `<MODEL>` in your .env" for a
173
+ rejected model id; a network/proxy/base-URL hint for a connection failure. The raw error line is
174
+ still printed verbatim beneath the guidance. Previously `doctor` could only distinguish "worked"
175
+ from "didn't" — a missing SDK, an expired login, a slow network and a wrong model name all
176
+ produced the same line. Providers that don't implement `diagnose()` fall back to the old
177
+ `suggestSelector()` probe (worked/didn't, plus a timing-based timeout guess). Verified live:
178
+ `openai` bad key -> `not-authenticated` (401 body shown), `openai` bad base URL -> `network`,
179
+ `copilot-subscription` bad model id -> `bad-model`, 1ms `actionTimeout` -> `timeout` with the
180
+ raise-the-timeout guidance, and all three not-installed shapes by moving the dependency out of
181
+ resolution and restoring it: `@github/copilot-sdk` (CJS `MODULE_NOT_FOUND`),
182
+ `@anthropic-ai/claude-agent-sdk` (ESM `ERR_MODULE_NOT_FOUND`), and a missing `kiro-cli` binary
183
+ (`ENOENT`) -- each surfacing `not-installed` with its exact install command.
184
+
185
+ - **Three new local-development-only subscription providers: `cursor-subscription`,
186
+ `kiro-subscription`, `codex-subscription`.** Extends the same "use what you're already paying
187
+ for" idea `claude-subscription`/`copilot-subscription` already give, to three more real
188
+ subscriptions — but each is explicitly scoped to local development, never documented or
189
+ recommended for CI, for two genuinely different reasons found by checking real vendor docs before
190
+ writing any code. Cursor's and Kiro's own headless-mode docs confirm neither has a way to fully
191
+ disable tool/file/command access the way `claude-subscription`'s `tools: []`/`copilot-subscription`'s
192
+ `availableTools: []` do — Cursor's `agent -p` "has access to all tools" by its own docs, and Kiro's
193
+ headless mode requires `--trust-all-tools`/`--trust-tools` specifically because there's no one to
194
+ approve a tool call unattended; `kiro-subscription` deliberately never grants that trust, so a
195
+ tool-requiring response just times out and declines rather than being granted broad access.
196
+ `codex-subscription` is local-only for a different, simpler reason: `codex exec` (used here, not
197
+ the interactive REPL) defaults to a read-only sandbox with no approval prompts — already as safe
198
+ as the two existing subscription providers — but no long-lived, subscription-only token for
199
+ unattended CI use is confirmed to exist for it yet. None of the three vendors ship a Node SDK, so
200
+ all three are `child_process` wrappers around the vendor's own CLI binary (`agent`, `kiro-cli`,
201
+ `codex`) via a new shared `runCliPrompt` helper, rather than the SDK-based design the existing two
202
+ use — meaning every call pays a fresh process-spawn cost, not just the first the way
203
+ `copilot-subscription`'s warm shared client does; `doctor` now has a distinct hint explaining this
204
+ when one of these three fails within a tight `actionTimeout`. Investigated and explicitly ruled
205
+ out this round: Antigravity — its dedicated `google-antigravity` SDK is Python-only (this is a
206
+ Node/TypeScript package), has no documented tool-restricted mode at all, and its authentication
207
+ story isn't published, so it isn't even confirmed to ride on an existing subscription. Verified:
208
+ real graceful-decline behavior confirmed against genuinely uninstalled CLIs (no fabricated
209
+ simulation), and the full `doctor` integration (including the new every-call-cost hint) confirmed
210
+ against a real sample repo. **Not yet verified**: the actual happy path against a real installed
211
+ and authenticated CLI for any of the three — none were available in the development environment,
212
+ and all three require a real paid subscription account.
213
+
214
+ - **A skill for running this package's local workflow inside an AI coding assistant.** Ships at
215
+ `skills/tamash-playwright/` — a `SKILL.md` entry point that branches on `npx tamash-playwright
216
+ doctor`'s actual output (never assumed), plus two reference docs: `onboarding.md` (bringing a
217
+ project up to standard — provider setup, `actionTimeout`, `.describe()` labels, Page Object
218
+ extraction) and `heal.md` (a gated review/apply/verify/land loop over `apply-heals` and
219
+ `verify-heals.cjs` — most runs proceed start to finish unattended, pausing only for a genuinely
220
+ ambiguous fix or anything after a failed verification, and never landing/committing/opening a PR
221
+ without asking first, no matter how clean the run was). Pure orchestration over commands that
222
+ already exist — no new healing capability. Ships with adapters for Claude Code, Kiro (identical
223
+ `SKILL.md` format, confirmed against Kiro's own docs), Cursor (`.mdc` rule), GitHub Copilot
224
+ (`copilot-instructions.md` section), and an `AGENTS.md` covering the broader cross-tool standard
225
+ (Antigravity, Gemini CLI, Windsurf, Zed, Aider, and others) — none of which are auto-discovered
226
+ from `node_modules` by any of these tools, confirmed rather than assumed, so every adapter's
227
+ install step is one explicit copy command, documented in `SKILL.md` itself.
228
+ - **A new `tamash` heal provider — rule-based healing, no AI at all.** `HEALER_PROVIDER=tamash`
229
+ needs no API key, no subscription, and makes no network call: it resolves a broken locator by
230
+ text-matching the same description an AI provider would receive (`.describe()`, or a decoded
231
+ variable name — see below) against the already-captured accessibility snapshot, then reuses the
232
+ exact same structural widening (`near`/`adjacent`) the AI-backed path already uses once it finds
233
+ the right anchor — same output shape, same downstream code, zero duplicated logic. It shares the
234
+ same non-negotiable discipline as every text-matching step in this package: zero or more than one
235
+ match, at any point, means it declines rather than guesses — verified with a dedicated permanent
236
+ e2e suite covering direct matches, widened matches, and every decline path (including the known,
237
+ accepted limitation that a misleading type hint can't be second-guessed against). Genuinely a
238
+ different tool than the AI providers, not a free replacement for one: no vision fallback (nothing
239
+ to reason over a screenshot with) and no action-recovery tactics (those require understanding
240
+ *why* an action failed, which is inference this provider deliberately doesn't attempt) — a fast,
241
+ free, fully deterministic first line of defense, best suited to well-`.describe()`d,
242
+ Page-Object-style suites.
243
+ - **A new `ollama-local` heal provider for self-hosted Ollama servers.** `HEALER_PROVIDER=ollama-local`
244
+ targets your own `ollama serve` instance or an internal company deployment, instead of Ollama
245
+ Cloud — a deliberately separate provider from `ollama`, not a flag on it, since the two have
246
+ genuinely different auth defaults: Ollama Cloud always requires `OLLAMA_API_KEY`, while
247
+ `OLLAMA_LOCAL_API_KEY` is optional, since a bare `ollama serve` has no authentication at all. Set
248
+ it only if your internal deployment sits behind a reverse proxy or API gateway that requires a
249
+ bearer token — verified against a real HTTP server both ways, confirming the `Authorization`
250
+ header is omitted entirely when no key is set, and sent correctly when one is. Prompted by a real
251
+ support request from a team wanting to use their own internally-hosted `gpt-oss:120b`.
252
+ - **Undescribed, POM-style locator variables now get a real description automatically.** When
253
+ `.describe()` was never called, the healer already fell back to the locator's own variable/
254
+ property name (`txtEmployeeId`) as its best guess at intent — now that raw identifier is decoded
255
+ into the same kind of human-readable phrase `.describe()` would give: `txtEmployeeId` becomes
256
+ "Employee Id (textbox)", `submitButton` becomes "Submit (button)", recognizing both prefix-style
257
+ (`btnSubmit`) and suffix-style (`submitBtn`) naming, camelCase/snake_case/kebab-case, and correct
258
+ acronym boundaries (`employeeIDNumber` → "Employee ID Number"). Falls back to the raw identifier,
259
+ exactly as before, whenever nothing meaningful survives decoding (a placeholder name like `el1`,
260
+ or a bare affix with nothing else) — never a guess dressed up as a real description. Verified
261
+ live against a real, unpublished build in both sample repos (a real `copilot-subscription` call,
262
+ and — after finding and fixing a genuinely stale API key along the way — a real `ollama` call),
263
+ confirming the decoded description actually reaches the configured AI provider unchanged.
264
+
265
+ ## [0.10.0] - 2026-08-27
266
+
267
+ ### Fixed
268
+
269
+ - **`waitFor()` is never sent to the AI.** It's a state check, not an action — a timeout on it can
270
+ mean a genuinely broken selector, or it can mean the element correctly never reached the expected
271
+ state (verifying something does NOT appear, or a real app issue), and there's no way to tell those
272
+ apart from the error alone. `expect(locator).toBeVisible()` was already permanently excluded from
273
+ healing for exactly this reason, but never reached this code at all (a separate path from the
274
+ Proxy-intercepted actions) — `waitFor` just never got the same treatment. Real, reported case: a
275
+ user's `waitFor` on a locator verifying an absence (where timing out was the *correct* outcome)
276
+ still burned 8175 tokens across a failed text attempt and a failed vision attempt before giving
277
+ up, for a heal that could never have succeeded. Now fails fast with a clear `state-wait-not-healed`
278
+ stage and zero AI calls; a real action (fill/click/...) on the same kind of broken locator is
279
+ unaffected.
280
+ - **`claude-subscription` was burning a real, unnecessary amount of extra tokens.** The SDK's
281
+ `effort` option silently defaults to `'high'` ("deep reasoning") when left unset, and this
282
+ provider never set it — a small prompt-complexity increase (the `nearbyRef`/`nearbyText` addition
283
+ below) pushed adaptive thinking to reason unpredictably harder for a task that only ever needs to
284
+ return one line of JSON. Confirmed live via real CI runs: output tokens for the same heal went
285
+ from a 188-445 baseline up to 422-881, inconsistently, run to run. `thinking: { type: 'disabled' }`
286
+ + `effort: 'low'` removes the variability entirely — verified repeatedly at a steady 28-30 output
287
+ tokens, below even the pre-regression baseline, with no change to correctness.
288
+ - **The primary ariaSnapshot capture no longer requests `boxes:true`.** Every node was paying for a
289
+ `[box=x,y,w,h]` annotation that nothing on the text/`ref` path (including the `nearbyRef`/
290
+ `adjacent`-strategy widening logic below) ever reads — it's purely topological. The one real
291
+ consumer (vision's own nearest-candidate lookup) already captures its own separate, fresh
292
+ snapshot, so this is genuinely free: verified live, input tokens dropped ~23% on a large real
293
+ page (3794 → 2937) with no loss of accuracy, and no change on small pages (box overhead scales
294
+ with node count).
295
+
296
+ ### Added
297
+
298
+ - **Search-scoped snapshots**: before falling back to the full page, the healer now searches the
299
+ already-captured snapshot for the description's identifying phrase and, only when it matches
300
+ exactly one node, sends the AI a scoped excerpt (that node's own subtree plus every sibling
301
+ branch's subtree at each ancestor level up to the root) instead of the whole page — zero extra
302
+ browser round-trips, since it's pure processing on data already in memory. Falls back to the full
303
+ snapshot automatically whenever the search is empty or ambiguous, never a guess. Verified live
304
+ across every real failure mode found this session: 41% token reduction on a deeply-nested field,
305
+ 66% when the target was inside what looked like an unrelated navigation menu (proving it finds
306
+ wherever the relevant text actually is, not "excludes the nav"), ~29% on a pair of identical
307
+ sibling fields that still had to be correctly disambiguated, and a clean, correct fallback when
308
+ the description doesn't match the page's real text at all.
309
+ - **A new `adjacent` selector strategy**, fixing a real ambiguity in the existing `near` strategy:
310
+ when two fields with no identity of their own share a row/section (two dropdowns side by side,
311
+ say), `near`'s "climb to a shared ancestor, then search it for any element of this role"
312
+ approach matches both and gives up rather than risk the wrong one. The AI's `ref` response can
313
+ now optionally report `nearbyRef`/`nearbyText`/`nearbyRole` for a nameless target it identified;
314
+ `deriveDurableLocator` uses that hint to find the true common ancestor between the target and
315
+ its label — via each ref's own full ancestor chain, not by assuming either sits at a matching
316
+ depth — and, when they're proven to be immediate sibling branches, builds a precise CSS
317
+ `:text() + *` sibling match (or an xpath climb-then-step, when the label text turns out to be
318
+ nested below its own branch root). Verified live against a real configured provider, resolving
319
+ the correct field and never its same-row neighbor in both directions.
320
+ - **Full attempt-history logging**: `SelfHealingReport` now carries an `attempts[]` array — one
321
+ entry per cache/ref/text/vision/action-recovery attempt actually made, each with its own
322
+ `succeeded`/`stage`/`error`. Previously only the *last* attempt's stage survived; an earlier
323
+ attempt's real failure (and the specific error it threw) was silently discarded the moment a
324
+ later attempt also failed — the exact shape of a real user-reported bug, where a genuine
325
+ candidate selector was shown next to an unrelated `vision_provider_error` with no way to tell
326
+ why the candidate itself hadn't worked. The attempt history now also prints directly to
327
+ console/CI output (not just the JSON attachment) whenever more than one attempt was made, on
328
+ both pass and fail — since that plain-text output is what most bug reports actually paste, not
329
+ an attachment nobody opens.
330
+ - **`ariaSnapshot` is attached to the test report on failure** — the exact accessibility tree the
331
+ AI reasoned over, so a confusing report can be diagnosed against real evidence instead of a
332
+ separately-captured DevTools screenshot.
333
+
334
+ ## [0.9.0] - 2026-08-26
335
+
336
+ ### Fixed
337
+
338
+ - **Heal-log visibility**: a heal with no reusable selector (a one-shot `ref` resolution, or a
339
+ vision-tagged point) was silently dropped from `heals.jsonl` entirely, even though it genuinely
340
+ fixed that run. Now logged for audit — with a clear `reviewNote` — whenever it has a suggestion
341
+ *or* a review note, without letting an audit-only entry shadow an older, real cached fix for the
342
+ same location.
343
+ - **Vision support for `claude-subscription`/`copilot-subscription`**: both providers had
344
+ `supportsVision` hardcoded to `false` regardless of model. Both SDKs genuinely support image
345
+ input (Claude Agent SDK via an image content block; Copilot SDK via a `blob` attachment) —
346
+ implemented for real and verified live with actual screenshots, not just doctor's yes/no label.
347
+ - **Argument forwarding in healed replays**: `replayAction()` only ever forwarded the first call
348
+ argument. A trailing options object (`click({ modifiers: [...] })`, `fill(value, { timeout })`,
349
+ `dispatchEvent`'s 3rd argument) was silently dropped on a healed replay — changing the action's
350
+ real behavior while still reporting a clean `HEALED`.
351
+ - **`ref`-strategy replays now act through the derived durable locator, not the raw ref**:
352
+ `aria-ref=` locators never resolve for `dispatchEvent` at all (a Playwright-level limitation,
353
+ confirmed directly via `DEBUG=pw:api`), even though the identical element resolves instantly via
354
+ a `normalize()`-derived locator for that same action. Fixed by deriving the durable locator
355
+ *before* replaying and acting through it, falling back to the raw ref only when nothing durable
356
+ could be found. A related accuracy bug is fixed alongside it: the report/heal-log could
357
+ previously claim a derived selector "worked" even when its own replay had actually failed and a
358
+ fallback silently took over — now only the locator that genuinely performed the action is
359
+ reported or cached.
360
+ - **`copilot-subscription` could hang a non-Playwright test runner indefinitely** — its shared
361
+ client keeps a connection open across calls for performance, which Playwright's own test runner
362
+ tolerates by force-exiting regardless, but a runner like Cucumber does not, so the process never
363
+ returned even after every test had already passed. Found live running a real Cucumber suite in
364
+ CI. `closeCopilotSubscriptionClient()` is now exported from the package's own entry point so any
365
+ non-Playwright consumer can call it from their own teardown hook (e.g. Cucumber's `AfterAll`).
366
+
367
+ ### Added
368
+
369
+ - **`locator.getDurable(action?)`** — resolves any locator (most usefully one built from
370
+ `aria-ref=...`) to a durable, reusable equivalent (`getByRole`/`getByLabel`/a css selector/…),
371
+ using the same derivation logic self-healing already uses internally. Throws if nothing durable
372
+ could be derived, rather than silently handing back something untrusted.
373
+ - **`apply-heals` now previews before writing**: every run — dry or real — shows a styled table of
374
+ exactly what it found (location, before, after, review status) instead of a plain scrolling log.
375
+ A real run also asks for confirmation before writing anything, but *only* at a genuine
376
+ interactive terminal — CI and any non-interactive/piped invocation proceed automatically exactly
377
+ as before, so no existing unattended CI workflow is affected. `--yes`/`-y` skips the prompt at a
378
+ real terminal too.
379
+ - **`doctor`'s output is styled** with colors and tables (including a new end-of-run Summary
380
+ section) instead of a plain scrolling log.
381
+ - **Exact locator code in console lines and reports**: a healed suggestion is now shown as the
382
+ real, copy-pasteable Playwright call (`getByRole("textbox", { name: "Username" })`) instead of
383
+ an abbreviated shorthand (`role:textbox:Username`) — the same code `apply-heals` would write to
384
+ source, so what you see is never a lossier stand-in for what was actually used.
385
+ - A canonical Page Object Model usage example (`tests/pages/` +
386
+ `tests/example-orangehrm-add-employee.spec.ts`), verified live against a real OrangeHRM demo,
387
+ including a genuine self-heal via the `near`/widening strategy on a real unlabeled field.
388
+
389
+ ### Changed
390
+
391
+ - README/usage.md's vision-fallback documentation now correctly lists `claude-subscription`/
392
+ `copilot-subscription` as vision-capable (with a suitable model), not just the API-key providers.
393
+
394
+ ## [0.8.0] - 2026-08-25
395
+
396
+ ### Added
397
+
398
+ - `claude-subscription` and `copilot-subscription` heal providers — self-healing backed by a
399
+ personal Claude or GitHub Copilot subscription instead of a pay-per-token API key, working both
400
+ locally and unattended in CI (`CLAUDE_CODE_OAUTH_TOKEN` for Claude; the ambient `GITHUB_TOKEN` or
401
+ a personal-account PAT for Copilot on GitHub Actions).
402
+
403
+ ### Fixed
404
+
405
+ - Missing CLI install step in setup docs for `claude-subscription`/`copilot-subscription` — the
406
+ standalone `claude`/`copilot` CLIs are genuinely required for the login step even though the SDKs
407
+ don't need them installed to function at runtime.
408
+
409
+ ### Documentation
410
+
411
+ - The org-vs-personal-account Copilot licensing gotcha in CI (a repo owned by an organization
412
+ without its own Copilot enablement can't use a personal subscription via the ambient token).
413
+ - Action Recovery (`HEALER_ACTION_RECOVERY_ENABLED`) hidden from docs and `doctor` output —
414
+ disabled by default already; this only reduced its visibility, no behavior change.
415
+
416
+ ## [0.7.0] - 2026-08-21
417
+
418
+ ### Added
419
+
420
+ - `apply-heals` — turns a runtime heal into a permanent source-code fix, plus an opportunistic
421
+ cache (`heals.jsonl`) so a previously-confirmed selector is tried before a fresh AI call, and
422
+ history archival so a run's report/log isn't silently overwritten by the next one.
423
+ - Position/relationship-based locator strategies (`near`, `scoped`, `containing`) for elements with
424
+ no accessible identity of their own.
425
+ - Self-healing rebuilt around `ariaSnapshot({ mode: 'ai' })` + `aria-ref=` resolution and
426
+ `Locator.normalize()`, replacing pixel-distance guessing for both the text path and the vision
427
+ fallback's durability upgrade.
428
+ - A real unit test suite (32 tests at the time, zero new dependencies).
429
+
430
+ ### Fixed
431
+
432
+ - `doctor`'s `actionTimeout` check being fooled by a commented-out config value.
433
+ - `apply-heals --logs-dir` silently losing raw heal-log archival.
434
+ - `apply-heals` only replacing the first call when re-healing an already-`near`-fixed line.
435
+ - `apply-heals` `ENOENT` on a fresh CI checkout with no `.tamash-playwright/` directory yet — the
436
+ sharded "apply-heals" CI job checks out fresh and has never run tests itself, so the directory
437
+ genuinely doesn't exist there the way it always does in every local recipe.
438
+ - File path corruption (and a `require()` crash in the generated verification script) for ESM
439
+ (`"type": "module"`) consumer projects — `Error.stack` renders as a `file://` URL there, which
440
+ broke path resolution and a plain `.js` verification script alike.
441
+ - The AI being misled by the broken selector still present in its own prompt context; a failed
442
+ label guess now auto-upgrades to a structural `near` match instead of giving up.
443
+
444
+ ### Verified
445
+
446
+ - All four API-key providers (Ollama, OpenAI, Anthropic, Gemini) confirmed working with real API
447
+ calls, not just documented.
448
+
449
+ ## [0.6.0] - 2026-08-17
450
+
451
+ ### Added
452
+
453
+ - iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
454
+ enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
455
+ - `doctor`'s `actionTimeout` configuration check.
456
+
457
+ ### Initial release
458
+
459
+ - Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
460
+ with `.describe()` for human-readable context.