tamash-playwright 0.11.0 → 0.12.0-beta.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,426 +1,460 @@
1
- # Changelog
2
-
3
- All notable changes to this project are documented here. Format loosely follows
4
- [Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
5
-
6
- ## [0.11.0] - 2026-08-31
7
-
8
- ### Fixed
9
-
10
- - **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
11
-
12
- - **`doctor`'s connectivity check gave false confidence for `claude-subscription`/`copilot-subscription`.**
13
- It tested against a fixed, generous 15s timeout, completely decoupled from the project's real
14
- configured `actionTimeout` so a project using the README's own example value (`actionTimeout:
15
- 8000`) could get `[OK] Connected successfully` from `doctor`, then have its very first real heal
16
- in a fresh test run fail with a misleading "not authenticated?" warning, because these two
17
- providers spawn the vendor's own CLI as a subprocess and the first call in a process pays a
18
- one-time cold-start cost (spawning + authenticating) on top of the actual model call — a cost
19
- `doctor`'s own generous timeout never had to absorb. Found via a real, unbriefed agent walkthrough
20
- of a from-scratch setup (see the skill entry below): it hit this exact failure, correctly
21
- diagnosed the root cause from a source-code comment nothing user-facing ever surfaced, and worked
22
- around it by raising `actionTimeout` to 20000ms. Fixed at the actual source: `doctor` now tests
23
- connectivity within the project's real `actionTimeout` when one's configured (falling back to the
24
- old generous default only when none is set yet), and a `[FAIL]` for either of these two providers
25
- now explicitly suggests it may be the cold-start cost rather than a real auth problem. Verified
26
- against the real `copilot-subscription` provider confirmed the check now genuinely uses the
27
- configured timeout (not just that it compiles); could not force a deterministic before/after
28
- repro of the timeout itself in this environment, since the underlying race is timing/environment-
29
- dependent (a warm local CLI session can mask it, exactly as the original cold-start comment
30
- already described).
31
-
32
- - **The published package was silently carrying stale, orphaned compiled output.** `npm run build`
33
- (plain `tsc`, no `--build`/incremental mode) never deletes `dist/` output for a source file that
34
- was later removed found in the `0.11.0-beta.1` publish itself: `dist/healer/providers/
35
- claude-cli.js`/`copilot-cli.js` shipped in the tarball with no corresponding source file
36
- anywhere in the repo (from an abandoned, never-committed, never-wired-up piece of past work) —
37
- dead code nothing actually imports, but real, needless bloat in every install. Fixed by cleaning
38
- `dist/` automatically before every build (`prebuild` script), verified by confirming a full
39
- rebuild no longer produces those two files.
40
-
41
- - **A popup/new tab opened via `context.waitForEvent('page')`, `page.on('popup', ...)`, or
42
- `page.waitForEvent('popup')` was never healing-aware.** `bindContext`/`bindPageActions` already
43
- made `context.newPage()`/`context.pages()`/`context.on('page', ...)` heal correctly, but these
44
- three including `context.waitForEvent('page')`, the pattern most commonly taught for popup
45
- handling — fell through to a raw, unwrapped Playwright page. A broken locator inside the
46
- resulting page just threw a plain timeout, with no `[self-healer]` line at all: healing was never
47
- attempted, not just unsuccessful. Found while building the first-ever test for popup healing
48
- (previously implemented with zero coverage); fixed by extending both bindings to wrap whatever
49
- page these three hand back, the same way `newPage()` already did.
50
-
51
- - **`tamash` (the rule-based provider) declined on a real, common page pattern it should have
52
- resolved.** A floating label rendered as its own text node right next to a field whose accessible
53
- name is *also* that same text (confirmed on a real login page: a "Username" label next to a field
54
- that's itself named "Username") was miscounted as two competing candidates and declined, even
55
- though there's genuinely only one field. `findRuleBasedMatch` now only treats a match as a real
56
- competing candidate when it's independently resolvable on its own (has a ref and a plausible
57
- role) a bare text node repeating the same words is decorative, not a second option. Genuine
58
- ambiguity (two actually-independent candidates) still declines exactly as before.
59
-
60
- - **The HTML report's self-healing attachment was a wall of escape sequences on a failed
61
- heal.** Playwright's own error messages embed ANSI colour codes in their "Call log" section;
62
- the healer stored those raw in `report.reason` / `report.warning` / `attempts[].error`, and
63
- `JSON.stringify` (which builds the `self-healing-<action>` attachment) escapes every ESC byte in the JSON, so a *not-healed* report rendered as unreadable escape soup. A
64
- *successful* heal has short or absent error strings, which is why it only showed up on failures.
65
- Fixed by stripping ANSI CSI sequences in `normalizeError` — the single point every captured
66
- error passes through so the attachment, the console line, and `heals.jsonl` are all plain
67
- text. Verified against a real not-healed run.
68
-
69
- - **The three CLI-based subscription providers (`cursor-subscription`, `kiro-subscription`,
70
- `codex-subscription`) were verified against their real vendor CLIs for the first time and none
71
- of the three actually worked as shipped.** Four separate bugs, all found by running them, all now
72
- fixed:
73
- 1. **`cursor-subscription` never returned a heal.** `agent -p` refuses to run in any directory
74
- without workspace trust — it prints "Workspace Trust Required" and exits without answering.
75
- Now passes `--trust --mode ask`; `--mode ask` is Cursor's own read-only Q&A mode ("read-only"
76
- per its `--help`), which *removes* the old "has access to all tools, could edit a file"
77
- caveat this provider carried — a heal call in `ask` mode cannot write or run anything.
78
- 2. **`codex-subscription` refused to run outside a Git repository** ("Not inside a trusted
79
- directory and --skip-git-repo-check was not specified"). Now passes `--skip-git-repo-check`;
80
- this only relaxes the where-may-I-run guard, not the sandbox `codex exec` still runs
81
- `approval: never` + read-only, the property this provider depends on.
82
- 3. **`codex exec` hung on stdin.** It drains stdin ("Reading additional input from stdin…") and
83
- blocks until EOF; `runCliPrompt` left the child's stdin as an open pipe, so every call ran to
84
- the timeout and was killed — confirmed: a prompt the CLI answers in ~8s "timed out" at 15s.
85
- `runCliPrompt` now gives every child an already-closed stdin (`stdio: ['ignore', …]`),
86
- harmless for `agent`/`kiro-cli`, the fix for `codex`.
87
- 4. **The JSON parsers failed on conversational output.** `parseSuggestion` /
88
- `parseVisionSuggestion` / `parseActionTacticSuggestion` extracted JSON from prose with a
89
- greedy first-`{`-to-last-`}` slice, which breaks the moment the response contains more than
90
- one JSON object exactly what cursor's `agent` produces (it echoes the system prompt's
91
- example objects back before its real answer). Replaced with a balanced-brace-span scan that
92
- takes the last valid object (a model states its answer last), old greedy slice kept as a last
93
- resort. Improves robustness for every provider, not just the CLI ones.
94
-
95
- Verified live, all authenticated: `doctor` reports `[OK] Connected` for all three; a real
96
- `npx playwright test` heal (broken locator, cache cleared) went green with
97
- `HEALED [provider=codex-subscription]`, and `kiro-subscription` healed the same spec.
98
-
99
- - **`cursor-subscription` reliability note.** Verifying it (see above) showed Cursor's `agent` CLI
100
- is an interactive assistant rather than a one-shot completion endpoint given the heal prompt it
101
- often answers conversationally instead of with the required JSON, so the heal is declined more
102
- often than with `kiro`/`codex`. It's now invoked in the safest/best-effort form
103
- (`agent -p --trust --mode ask --output-format json`, read-only) and is marked **experimental**;
104
- the docs point to `kiro`/`codex` as the steadier local-subscription options. Report issues.
105
- Also: the full e2e healing suite (12 tests) was run end to end against `copilot-subscription` and
106
- `ollama` — **12/12 each** — to confirm the parser rewrite and provider changes don't regress it.
107
-
108
- ### Added
109
-
110
- - **`npx tamash-playwright init-skill` one command to install the orchestration skill, plus a
111
- `doctor` check that flags when it's missing or stale.** The skill this package ships
112
- (`skills/tamash-playwright/`) was previously a copy-this-shell-one-liner-per-agent step buried in
113
- `SKILL.md`, with nothing verifying it had been done — so a project could `npm install` the
114
- package, never copy the skill, and its assistant would silently never load the workflow.
115
- `init-skill` copies the skill (`SKILL.md` + `references/`) into **both** standard locations
116
- `.claude/skills/tamash-playwright/` (Claude Code) and `.agents/skills/tamash-playwright/` (the
117
- emerging cross-tool standard, read by Cursor, GitHub Copilot, Windsurf, Kiro, Zed, dotnet Aspire,
118
- the same convention Playwright's own `playwright-cli install --skills` uses). Same content in
119
- both; no per-agent format conversion. `--target claude` / `--target agents` installs one; `--user`
120
- installs under your home directory; `--force` overwrites a hand-edited copy; `--dry-run` previews.
121
- Each install carries a `tamash-playwright-skill-version:` marker; `doctor`'s **Skill** section
122
- reads it and reports `[OK]` current / `[WARN]` behind-the-package-version / `[INFO]` not-installed
123
- or unmanaged, with the same one-line fix. Covered by unit tests (`skill.test.js`, real installs
124
- into a temp dir) and verified live end to end.
125
-
126
- *(0.11.0-beta.7 shipped an earlier `init-skill` that instead auto-detected the agent and wrote a
127
- Cursor-specific `.mdc` file / appended blocks to `.github/copilot-instructions.md` and
128
- `AGENTS.md`. That approach is gone — `init-skill` and `doctor` now point out any such leftovers
129
- from a beta.7 install so you can delete them by hand.)*
130
-
131
- - **`doctor`'s AI Provider check now tells you *what kind* of failure it hit and what to do about
132
- it, instead of one generic "no valid response".** Every provider gained an optional, diagnostics-
133
- only `diagnose()` method (never on the healing hot path) that runs one trivial round trip and
134
- reports a structured category`not-installed`, `not-authenticated`, `timeout`, `bad-model`,
135
- `network`, `bad-response` each of which `doctor` maps to a specific next step: the missing
136
- `npm install`/CLI installer for a missing SDK; "run `claude login` / check your API key / confirm
137
- your subscription is active and within quota" for a rejected request; "raise `actionTimeout`" (plus
138
- the existing subprocess cold-start note) for a timeout; "check `<MODEL>` in your .env" for a
139
- rejected model id; a network/proxy/base-URL hint for a connection failure. The raw error line is
140
- still printed verbatim beneath the guidance. Previously `doctor` could only distinguish "worked"
141
- from "didn't" — a missing SDK, an expired login, a slow network and a wrong model name all
142
- produced the same line. Providers that don't implement `diagnose()` fall back to the old
143
- `suggestSelector()` probe (worked/didn't, plus a timing-based timeout guess). Verified live:
144
- `openai` bad key -> `not-authenticated` (401 body shown), `openai` bad base URL -> `network`,
145
- `copilot-subscription` bad model id -> `bad-model`, 1ms `actionTimeout` -> `timeout` with the
146
- raise-the-timeout guidance, and all three not-installed shapes by moving the dependency out of
147
- resolution and restoring it: `@github/copilot-sdk` (CJS `MODULE_NOT_FOUND`),
148
- `@anthropic-ai/claude-agent-sdk` (ESM `ERR_MODULE_NOT_FOUND`), and a missing `kiro-cli` binary
149
- (`ENOENT`) -- each surfacing `not-installed` with its exact install command.
150
-
151
- - **Three new local-development-only subscription providers: `cursor-subscription`,
152
- `kiro-subscription`, `codex-subscription`.** Extends the same "use what you're already paying
153
- for" idea `claude-subscription`/`copilot-subscription` already give, to three more real
154
- subscriptions but each is explicitly scoped to local development, never documented or
155
- recommended for CI, for two genuinely different reasons found by checking real vendor docs before
156
- writing any code. Cursor's and Kiro's own headless-mode docs confirm neither has a way to fully
157
- disable tool/file/command access the way `claude-subscription`'s `tools: []`/`copilot-subscription`'s
158
- `availableTools: []` do Cursor's `agent -p` "has access to all tools" by its own docs, and Kiro's
159
- headless mode requires `--trust-all-tools`/`--trust-tools` specifically because there's no one to
160
- approve a tool call unattended; `kiro-subscription` deliberately never grants that trust, so a
161
- tool-requiring response just times out and declines rather than being granted broad access.
162
- `codex-subscription` is local-only for a different, simpler reason: `codex exec` (used here, not
163
- the interactive REPL) defaults to a read-only sandbox with no approval prompts already as safe
164
- as the two existing subscription providers — but no long-lived, subscription-only token for
165
- unattended CI use is confirmed to exist for it yet. None of the three vendors ship a Node SDK, so
166
- all three are `child_process` wrappers around the vendor's own CLI binary (`agent`, `kiro-cli`,
167
- `codex`) via a new shared `runCliPrompt` helper, rather than the SDK-based design the existing two
168
- use meaning every call pays a fresh process-spawn cost, not just the first the way
169
- `copilot-subscription`'s warm shared client does; `doctor` now has a distinct hint explaining this
170
- when one of these three fails within a tight `actionTimeout`. Investigated and explicitly ruled
171
- out this round: Antigravity its dedicated `google-antigravity` SDK is Python-only (this is a
172
- Node/TypeScript package), has no documented tool-restricted mode at all, and its authentication
173
- story isn't published, so it isn't even confirmed to ride on an existing subscription. Verified:
174
- real graceful-decline behavior confirmed against genuinely uninstalled CLIs (no fabricated
175
- simulation), and the full `doctor` integration (including the new every-call-cost hint) confirmed
176
- against a real sample repo. **Not yet verified**: the actual happy path against a real installed
177
- and authenticated CLI for any of the three none were available in the development environment,
178
- and all three require a real paid subscription account.
179
-
180
- - **A skill for running this package's local workflow inside an AI coding assistant.** Ships at
181
- `skills/tamash-playwright/` a `SKILL.md` entry point that branches on `npx tamash-playwright
182
- doctor`'s actual output (never assumed), plus two reference docs: `onboarding.md` (bringing a
183
- project up to standard provider setup, `actionTimeout`, `.describe()` labels, Page Object
184
- extraction) and `heal.md` (a gated review/apply/verify/land loop over `apply-heals` and
185
- `verify-heals.cjs` most runs proceed start to finish unattended, pausing only for a genuinely
186
- ambiguous fix or anything after a failed verification, and never landing/committing/opening a PR
187
- without asking first, no matter how clean the run was). Pure orchestration over commands that
188
- already exist no new healing capability. Ships with adapters for Claude Code, Kiro (identical
189
- `SKILL.md` format, confirmed against Kiro's own docs), Cursor (`.mdc` rule), GitHub Copilot
190
- (`copilot-instructions.md` section), and an `AGENTS.md` covering the broader cross-tool standard
191
- (Antigravity, Gemini CLI, Windsurf, Zed, Aider, and others) — none of which are auto-discovered
192
- from `node_modules` by any of these tools, confirmed rather than assumed, so every adapter's
193
- install step is one explicit copy command, documented in `SKILL.md` itself.
194
- - **A new `tamash` heal provider rule-based healing, no AI at all.** `HEALER_PROVIDER=tamash`
195
- needs no API key, no subscription, and makes no network call: it resolves a broken locator by
196
- text-matching the same description an AI provider would receive (`.describe()`, or a decoded
197
- variable name see below) against the already-captured accessibility snapshot, then reuses the
198
- exact same structural widening (`near`/`adjacent`) the AI-backed path already uses once it finds
199
- the right anchor same output shape, same downstream code, zero duplicated logic. It shares the
200
- same non-negotiable discipline as every text-matching step in this package: zero or more than one
201
- match, at any point, means it declines rather than guesses verified with a dedicated permanent
202
- e2e suite covering direct matches, widened matches, and every decline path (including the known,
203
- accepted limitation that a misleading type hint can't be second-guessed against). Genuinely a
204
- different tool than the AI providers, not a free replacement for one: no vision fallback (nothing
205
- to reason over a screenshot with) and no action-recovery tactics (those require understanding
206
- *why* an action failed, which is inference this provider deliberately doesn't attempt) a fast,
207
- free, fully deterministic first line of defense, best suited to well-`.describe()`d,
208
- Page-Object-style suites.
209
- - **A new `ollama-local` heal provider for self-hosted Ollama servers.** `HEALER_PROVIDER=ollama-local`
210
- targets your own `ollama serve` instance or an internal company deployment, instead of Ollama
211
- Cloud a deliberately separate provider from `ollama`, not a flag on it, since the two have
212
- genuinely different auth defaults: Ollama Cloud always requires `OLLAMA_API_KEY`, while
213
- `OLLAMA_LOCAL_API_KEY` is optional, since a bare `ollama serve` has no authentication at all. Set
214
- it only if your internal deployment sits behind a reverse proxy or API gateway that requires a
215
- bearer token verified against a real HTTP server both ways, confirming the `Authorization`
216
- header is omitted entirely when no key is set, and sent correctly when one is. Prompted by a real
217
- support request from a team wanting to use their own internally-hosted `gpt-oss:120b`.
218
- - **Undescribed, POM-style locator variables now get a real description automatically.** When
219
- `.describe()` was never called, the healer already fell back to the locator's own variable/
220
- property name (`txtEmployeeId`) as its best guess at intent now that raw identifier is decoded
221
- into the same kind of human-readable phrase `.describe()` would give: `txtEmployeeId` becomes
222
- "Employee Id (textbox)", `submitButton` becomes "Submit (button)", recognizing both prefix-style
223
- (`btnSubmit`) and suffix-style (`submitBtn`) naming, camelCase/snake_case/kebab-case, and correct
224
- acronym boundaries (`employeeIDNumber` "Employee ID Number"). Falls back to the raw identifier,
225
- exactly as before, whenever nothing meaningful survives decoding (a placeholder name like `el1`,
226
- or a bare affix with nothing else) never a guess dressed up as a real description. Verified
227
- live against a real, unpublished build in both sample repos (a real `copilot-subscription` call,
228
- and after finding and fixing a genuinely stale API key along the way — a real `ollama` call),
229
- confirming the decoded description actually reaches the configured AI provider unchanged.
230
-
231
- ## [0.10.0] - 2026-08-27
232
-
233
- ### Fixed
234
-
235
- - **`waitFor()` is never sent to the AI.** It's a state check, not an action — a timeout on it can
236
- mean a genuinely broken selector, or it can mean the element correctly never reached the expected
237
- state (verifying something does NOT appear, or a real app issue), and there's no way to tell those
238
- apart from the error alone. `expect(locator).toBeVisible()` was already permanently excluded from
239
- healing for exactly this reason, but never reached this code at all (a separate path from the
240
- Proxy-intercepted actions) `waitFor` just never got the same treatment. Real, reported case: a
241
- user's `waitFor` on a locator verifying an absence (where timing out was the *correct* outcome)
242
- still burned 8175 tokens across a failed text attempt and a failed vision attempt before giving
243
- up, for a heal that could never have succeeded. Now fails fast with a clear `state-wait-not-healed`
244
- stage and zero AI calls; a real action (fill/click/...) on the same kind of broken locator is
245
- unaffected.
246
- - **`claude-subscription` was burning a real, unnecessary amount of extra tokens.** The SDK's
247
- `effort` option silently defaults to `'high'` ("deep reasoning") when left unset, and this
248
- provider never set it a small prompt-complexity increase (the `nearbyRef`/`nearbyText` addition
249
- below) pushed adaptive thinking to reason unpredictably harder for a task that only ever needs to
250
- return one line of JSON. Confirmed live via real CI runs: output tokens for the same heal went
251
- from a 188-445 baseline up to 422-881, inconsistently, run to run. `thinking: { type: 'disabled' }`
252
- + `effort: 'low'` removes the variability entirely verified repeatedly at a steady 28-30 output
253
- tokens, below even the pre-regression baseline, with no change to correctness.
254
- - **The primary ariaSnapshot capture no longer requests `boxes:true`.** Every node was paying for a
255
- `[box=x,y,w,h]` annotation that nothing on the text/`ref` path (including the `nearbyRef`/
256
- `adjacent`-strategy widening logic below) ever reads it's purely topological. The one real
257
- consumer (vision's own nearest-candidate lookup) already captures its own separate, fresh
258
- snapshot, so this is genuinely free: verified live, input tokens dropped ~23% on a large real
259
- page (3794 2937) with no loss of accuracy, and no change on small pages (box overhead scales
260
- with node count).
261
-
262
- ### Added
263
-
264
- - **Search-scoped snapshots**: before falling back to the full page, the healer now searches the
265
- already-captured snapshot for the description's identifying phrase and, only when it matches
266
- exactly one node, sends the AI a scoped excerpt (that node's own subtree plus every sibling
267
- branch's subtree at each ancestor level up to the root) instead of the whole page — zero extra
268
- browser round-trips, since it's pure processing on data already in memory. Falls back to the full
269
- snapshot automatically whenever the search is empty or ambiguous, never a guess. Verified live
270
- across every real failure mode found this session: 41% token reduction on a deeply-nested field,
271
- 66% when the target was inside what looked like an unrelated navigation menu (proving it finds
272
- wherever the relevant text actually is, not "excludes the nav"), ~29% on a pair of identical
273
- sibling fields that still had to be correctly disambiguated, and a clean, correct fallback when
274
- the description doesn't match the page's real text at all.
275
- - **A new `adjacent` selector strategy**, fixing a real ambiguity in the existing `near` strategy:
276
- when two fields with no identity of their own share a row/section (two dropdowns side by side,
277
- say), `near`'s "climb to a shared ancestor, then search it for any element of this role"
278
- approach matches both and gives up rather than risk the wrong one. The AI's `ref` response can
279
- now optionally report `nearbyRef`/`nearbyText`/`nearbyRole` for a nameless target it identified;
280
- `deriveDurableLocator` uses that hint to find the true common ancestor between the target and
281
- its label via each ref's own full ancestor chain, not by assuming either sits at a matching
282
- depth and, when they're proven to be immediate sibling branches, builds a precise CSS
283
- `:text() + *` sibling match (or an xpath climb-then-step, when the label text turns out to be
284
- nested below its own branch root). Verified live against a real configured provider, resolving
285
- the correct field and never its same-row neighbor in both directions.
286
- - **Full attempt-history logging**: `SelfHealingReport` now carries an `attempts[]` array one
287
- entry per cache/ref/text/vision/action-recovery attempt actually made, each with its own
288
- `succeeded`/`stage`/`error`. Previously only the *last* attempt's stage survived; an earlier
289
- attempt's real failure (and the specific error it threw) was silently discarded the moment a
290
- later attempt also failedthe exact shape of a real user-reported bug, where a genuine
291
- candidate selector was shown next to an unrelated `vision_provider_error` with no way to tell
292
- why the candidate itself hadn't worked. The attempt history now also prints directly to
293
- console/CI output (not just the JSON attachment) whenever more than one attempt was made, on
294
- both pass and fail — since that plain-text output is what most bug reports actually paste, not
295
- an attachment nobody opens.
296
- - **`ariaSnapshot` is attached to the test report on failure** — the exact accessibility tree the
297
- AI reasoned over, so a confusing report can be diagnosed against real evidence instead of a
298
- separately-captured DevTools screenshot.
299
-
300
- ## [0.9.0] - 2026-08-26
301
-
302
- ### Fixed
303
-
304
- - **Heal-log visibility**: a heal with no reusable selector (a one-shot `ref` resolution, or a
305
- vision-tagged point) was silently dropped from `heals.jsonl` entirely, even though it genuinely
306
- fixed that run. Now logged for audit with a clear `reviewNote` whenever it has a suggestion
307
- *or* a review note, without letting an audit-only entry shadow an older, real cached fix for the
308
- same location.
309
- - **Vision support for `claude-subscription`/`copilot-subscription`**: both providers had
310
- `supportsVision` hardcoded to `false` regardless of model. Both SDKs genuinely support image
311
- input (Claude Agent SDK via an image content block; Copilot SDK via a `blob` attachment)
312
- implemented for real and verified live with actual screenshots, not just doctor's yes/no label.
313
- - **Argument forwarding in healed replays**: `replayAction()` only ever forwarded the first call
314
- argument. A trailing options object (`click({ modifiers: [...] })`, `fill(value, { timeout })`,
315
- `dispatchEvent`'s 3rd argument) was silently dropped on a healed replay changing the action's
316
- real behavior while still reporting a clean `HEALED`.
317
- - **`ref`-strategy replays now act through the derived durable locator, not the raw ref**:
318
- `aria-ref=` locators never resolve for `dispatchEvent` at all (a Playwright-level limitation,
319
- confirmed directly via `DEBUG=pw:api`), even though the identical element resolves instantly via
320
- a `normalize()`-derived locator for that same action. Fixed by deriving the durable locator
321
- *before* replaying and acting through it, falling back to the raw ref only when nothing durable
322
- could be found. A related accuracy bug is fixed alongside it: the report/heal-log could
323
- previously claim a derived selector "worked" even when its own replay had actually failed and a
324
- fallback silently took overnow only the locator that genuinely performed the action is
325
- reported or cached.
326
- - **`copilot-subscription` could hang a non-Playwright test runner indefinitely** its shared
327
- client keeps a connection open across calls for performance, which Playwright's own test runner
328
- tolerates by force-exiting regardless, but a runner like Cucumber does not, so the process never
329
- returned even after every test had already passed. Found live running a real Cucumber suite in
330
- CI. `closeCopilotSubscriptionClient()` is now exported from the package's own entry point so any
331
- non-Playwright consumer can call it from their own teardown hook (e.g. Cucumber's `AfterAll`).
332
-
333
- ### Added
334
-
335
- - **`locator.getDurable(action?)`** — resolves any locator (most usefully one built from
336
- `aria-ref=...`) to a durable, reusable equivalent (`getByRole`/`getByLabel`/a css selector/…),
337
- using the same derivation logic self-healing already uses internally. Throws if nothing durable
338
- could be derived, rather than silently handing back something untrusted.
339
- - **`apply-heals` now previews before writing**: every run dry or real — shows a styled table of
340
- exactly what it found (location, before, after, review status) instead of a plain scrolling log.
341
- A real run also asks for confirmation before writing anything, but *only* at a genuine
342
- interactive terminal — CI and any non-interactive/piped invocation proceed automatically exactly
343
- as before, so no existing unattended CI workflow is affected. `--yes`/`-y` skips the prompt at a
344
- real terminal too.
345
- - **`doctor`'s output is styled** with colors and tables (including a new end-of-run Summary
346
- section) instead of a plain scrolling log.
347
- - **Exact locator code in console lines and reports**: a healed suggestion is now shown as the
348
- real, copy-pasteable Playwright call (`getByRole("textbox", { name: "Username" })`) instead of
349
- an abbreviated shorthand (`role:textbox:Username`) the same code `apply-heals` would write to
350
- source, so what you see is never a lossier stand-in for what was actually used.
351
- - A canonical Page Object Model usage example (`tests/pages/` +
352
- `tests/example-orangehrm-add-employee.spec.ts`), verified live against a real OrangeHRM demo,
353
- including a genuine self-heal via the `near`/widening strategy on a real unlabeled field.
354
-
355
- ### Changed
356
-
357
- - README/usage.md's vision-fallback documentation now correctly lists `claude-subscription`/
358
- `copilot-subscription` as vision-capable (with a suitable model), not just the API-key providers.
359
-
360
- ## [0.8.0] - 2026-08-25
361
-
362
- ### Added
363
-
364
- - `claude-subscription` and `copilot-subscription` heal providers self-healing backed by a
365
- personal Claude or GitHub Copilot subscription instead of a pay-per-token API key, working both
366
- locally and unattended in CI (`CLAUDE_CODE_OAUTH_TOKEN` for Claude; the ambient `GITHUB_TOKEN` or
367
- a personal-account PAT for Copilot on GitHub Actions).
368
-
369
- ### Fixed
370
-
371
- - Missing CLI install step in setup docs for `claude-subscription`/`copilot-subscription` the
372
- standalone `claude`/`copilot` CLIs are genuinely required for the login step even though the SDKs
373
- don't need them installed to function at runtime.
374
-
375
- ### Documentation
376
-
377
- - The org-vs-personal-account Copilot licensing gotcha in CI (a repo owned by an organization
378
- without its own Copilot enablement can't use a personal subscription via the ambient token).
379
- - Action Recovery (`HEALER_ACTION_RECOVERY_ENABLED`) hidden from docs and `doctor` output
380
- disabled by default already; this only reduced its visibility, no behavior change.
381
-
382
- ## [0.7.0] - 2026-08-21
383
-
384
- ### Added
385
-
386
- - `apply-heals` turns a runtime heal into a permanent source-code fix, plus an opportunistic
387
- cache (`heals.jsonl`) so a previously-confirmed selector is tried before a fresh AI call, and
388
- history archival so a run's report/log isn't silently overwritten by the next one.
389
- - Position/relationship-based locator strategies (`near`, `scoped`, `containing`) for elements with
390
- no accessible identity of their own.
391
- - Self-healing rebuilt around `ariaSnapshot({ mode: 'ai' })` + `aria-ref=` resolution and
392
- `Locator.normalize()`, replacing pixel-distance guessing for both the text path and the vision
393
- fallback's durability upgrade.
394
- - A real unit test suite (32 tests at the time, zero new dependencies).
395
-
396
- ### Fixed
397
-
398
- - `doctor`'s `actionTimeout` check being fooled by a commented-out config value.
399
- - `apply-heals --logs-dir` silently losing raw heal-log archival.
400
- - `apply-heals` only replacing the first call when re-healing an already-`near`-fixed line.
401
- - `apply-heals` `ENOENT` on a fresh CI checkout with no `.tamash-playwright/` directory yet — the
402
- sharded "apply-heals" CI job checks out fresh and has never run tests itself, so the directory
403
- genuinely doesn't exist there the way it always does in every local recipe.
404
- - File path corruption (and a `require()` crash in the generated verification script) for ESM
405
- (`"type": "module"`) consumer projects `Error.stack` renders as a `file://` URL there, which
406
- broke path resolution and a plain `.js` verification script alike.
407
- - The AI being misled by the broken selector still present in its own prompt context; a failed
408
- label guess now auto-upgrades to a structural `near` match instead of giving up.
409
-
410
- ### Verified
411
-
412
- - All four API-key providers (Ollama, OpenAI, Anthropic, Gemini) confirmed working with real API
413
- calls, not just documented.
414
-
415
- ## [0.6.0] - 2026-08-17
416
-
417
- ### Added
418
-
419
- - iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
420
- enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
421
- - `doctor`'s `actionTimeout` configuration check.
422
-
423
- ### Initial release
424
-
425
- - Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
426
- with `.describe()` for human-readable context.
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented here. Format loosely follows
4
+ [Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
5
+
6
+ ## [0.12.0-beta.2] - 2026-09-08
7
+
8
+ ### Added
9
+
10
+ - **Playwright 1.63 support.** Dev/CI now tracks `@playwright/test` 1.63; `peerDependencies`
11
+ stays `>=1.40.0`. Verified: existing iframe healing (`page.frameLocator('#id')`) still heals,
12
+ and the full unit + representative e2e suite passes against 1.63.
13
+
14
+ - **Healing through Playwright 1.63's no-argument `page.frameLocator()`.** 1.63 made
15
+ `frameLocator()`'s selector optional with no argument it matches inside *any* frame on the
16
+ page. As a healing scope that's ambiguous the moment the page has more than one frame (the
17
+ healer's `locator('body')` snapshot then throws *"frameLocator() matched elements in multiple
18
+ frames"*, and healing silently no-op'd with `stage=no_snapshot`). Now, when the page has
19
+ exactly one frame, the healer re-expresses the no-arg `frameLocator()` as an explicit
20
+ single-frame `FrameLocator` and heals normally including deriving a **durable, persistable**
21
+ cross-frame selector (`apply-heals` can write it back to source), identical to what
22
+ `page.frameLocator('#id')` produces. With zero or several frames it can't know which was meant,
23
+ so it steps aside cleanly — the original error is re-thrown, never a wrong-frame guess. For
24
+ healable work on a multi-frame page, pass an explicit selector: `page.frameLocator('#id')`.
25
+
26
+ *(`0.12.0-beta.1` collapsed the scope to a raw `page.frames()` `Frame` instead of a
27
+ `FrameLocator`. `Locator.normalize()` on a `Frame`-rooted locator returns a correct selector
28
+ string but an object that resolves to nothing, so the heal worked once at runtime but couldn't
29
+ be persisted it fell back to a transient one-shot element reference with a `needsReview`
30
+ note. Fixed here.)*
31
+
32
+ ### Fixed
33
+
34
+ - **`describeFactoryCall` rendered `iframe "undefined"` for a no-arg `frameLocator()`.** The
35
+ factory-call label helper assumed a selector argument was always present; with 1.63's optional
36
+ selector it produced the literal string `iframe "undefined"`. Now renders `any iframe`. (Latent
37
+ the label isn't currently surfaced to users or the model, but it would be the moment any
38
+ "healing inside iframe X" context is added to a report or prompt.)
39
+
40
+ ## [0.11.0] - 2026-08-31
41
+
42
+ ### Fixed
43
+
44
+ - **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
45
+
46
+ - **`doctor`'s connectivity check gave false confidence for `claude-subscription`/`copilot-subscription`.**
47
+ It tested against a fixed, generous 15s timeout, completely decoupled from the project's real
48
+ configured `actionTimeout` so a project using the README's own example value (`actionTimeout:
49
+ 8000`) could get `[OK] Connected successfully` from `doctor`, then have its very first real heal
50
+ in a fresh test run fail with a misleading "not authenticated?" warning, because these two
51
+ providers spawn the vendor's own CLI as a subprocess and the first call in a process pays a
52
+ one-time cold-start cost (spawning + authenticating) on top of the actual model call a cost
53
+ `doctor`'s own generous timeout never had to absorb. Found via a real, unbriefed agent walkthrough
54
+ of a from-scratch setup (see the skill entry below): it hit this exact failure, correctly
55
+ diagnosed the root cause from a source-code comment nothing user-facing ever surfaced, and worked
56
+ around it by raising `actionTimeout` to 20000ms. Fixed at the actual source: `doctor` now tests
57
+ connectivity within the project's real `actionTimeout` when one's configured (falling back to the
58
+ old generous default only when none is set yet), and a `[FAIL]` for either of these two providers
59
+ now explicitly suggests it may be the cold-start cost rather than a real auth problem. Verified
60
+ against the real `copilot-subscription` provider confirmed the check now genuinely uses the
61
+ configured timeout (not just that it compiles); could not force a deterministic before/after
62
+ repro of the timeout itself in this environment, since the underlying race is timing/environment-
63
+ dependent (a warm local CLI session can mask it, exactly as the original cold-start comment
64
+ already described).
65
+
66
+ - **The published package was silently carrying stale, orphaned compiled output.** `npm run build`
67
+ (plain `tsc`, no `--build`/incremental mode) never deletes `dist/` output for a source file that
68
+ was later removed — found in the `0.11.0-beta.1` publish itself: `dist/healer/providers/
69
+ claude-cli.js`/`copilot-cli.js` shipped in the tarball with no corresponding source file
70
+ anywhere in the repo (from an abandoned, never-committed, never-wired-up piece of past work)
71
+ dead code nothing actually imports, but real, needless bloat in every install. Fixed by cleaning
72
+ `dist/` automatically before every build (`prebuild` script), verified by confirming a full
73
+ rebuild no longer produces those two files.
74
+
75
+ - **A popup/new tab opened via `context.waitForEvent('page')`, `page.on('popup', ...)`, or
76
+ `page.waitForEvent('popup')` was never healing-aware.** `bindContext`/`bindPageActions` already
77
+ made `context.newPage()`/`context.pages()`/`context.on('page', ...)` heal correctly, but these
78
+ three including `context.waitForEvent('page')`, the pattern most commonly taught for popup
79
+ handling fell through to a raw, unwrapped Playwright page. A broken locator inside the
80
+ resulting page just threw a plain timeout, with no `[self-healer]` line at all: healing was never
81
+ attempted, not just unsuccessful. Found while building the first-ever test for popup healing
82
+ (previously implemented with zero coverage); fixed by extending both bindings to wrap whatever
83
+ page these three hand back, the same way `newPage()` already did.
84
+
85
+ - **`tamash` (the rule-based provider) declined on a real, common page pattern it should have
86
+ resolved.** A floating label rendered as its own text node right next to a field whose accessible
87
+ name is *also* that same text (confirmed on a real login page: a "Username" label next to a field
88
+ that's itself named "Username") was miscounted as two competing candidates and declined, even
89
+ though there's genuinely only one field. `findRuleBasedMatch` now only treats a match as a real
90
+ competing candidate when it's independently resolvable on its own (has a ref and a plausible
91
+ role) a bare text node repeating the same words is decorative, not a second option. Genuine
92
+ ambiguity (two actually-independent candidates) still declines exactly as before.
93
+
94
+ - **The HTML report's self-healing attachment was a wall of escape sequences on a failed
95
+ heal.** Playwright's own error messages embed ANSI colour codes in their "Call log" section;
96
+ the healer stored those raw in `report.reason` / `report.warning` / `attempts[].error`, and
97
+ `JSON.stringify` (which builds the `self-healing-<action>` attachment) escapes every ESC byte in the JSON, so a *not-healed* report rendered as unreadable escape soup. A
98
+ *successful* heal has short or absent error strings, which is why it only showed up on failures.
99
+ Fixed by stripping ANSI CSI sequences in `normalizeError` the single point every captured
100
+ error passes through so the attachment, the console line, and `heals.jsonl` are all plain
101
+ text. Verified against a real not-healed run.
102
+
103
+ - **The three CLI-based subscription providers (`cursor-subscription`, `kiro-subscription`,
104
+ `codex-subscription`) were verified against their real vendor CLIs for the first time and none
105
+ of the three actually worked as shipped.** Four separate bugs, all found by running them, all now
106
+ fixed:
107
+ 1. **`cursor-subscription` never returned a heal.** `agent -p` refuses to run in any directory
108
+ without workspace trust — it prints "Workspace Trust Required" and exits without answering.
109
+ Now passes `--trust --mode ask`; `--mode ask` is Cursor's own read-only Q&A mode ("read-only"
110
+ per its `--help`), which *removes* the old "has access to all tools, could edit a file"
111
+ caveat this provider carried a heal call in `ask` mode cannot write or run anything.
112
+ 2. **`codex-subscription` refused to run outside a Git repository** ("Not inside a trusted
113
+ directory and --skip-git-repo-check was not specified"). Now passes `--skip-git-repo-check`;
114
+ this only relaxes the where-may-I-run guard, not the sandbox `codex exec` still runs
115
+ `approval: never` + read-only, the property this provider depends on.
116
+ 3. **`codex exec` hung on stdin.** It drains stdin ("Reading additional input from stdin…") and
117
+ blocks until EOF; `runCliPrompt` left the child's stdin as an open pipe, so every call ran to
118
+ the timeout and was killed confirmed: a prompt the CLI answers in ~8s "timed out" at 15s.
119
+ `runCliPrompt` now gives every child an already-closed stdin (`stdio: ['ignore', …]`),
120
+ harmless for `agent`/`kiro-cli`, the fix for `codex`.
121
+ 4. **The JSON parsers failed on conversational output.** `parseSuggestion` /
122
+ `parseVisionSuggestion` / `parseActionTacticSuggestion` extracted JSON from prose with a
123
+ greedy first-`{`-to-last-`}` slice, which breaks the moment the response contains more than
124
+ one JSON object exactly what cursor's `agent` produces (it echoes the system prompt's
125
+ example objects back before its real answer). Replaced with a balanced-brace-span scan that
126
+ takes the last valid object (a model states its answer last), old greedy slice kept as a last
127
+ resort. Improves robustness for every provider, not just the CLI ones.
128
+
129
+ Verified live, all authenticated: `doctor` reports `[OK] Connected` for all three; a real
130
+ `npx playwright test` heal (broken locator, cache cleared) went green with
131
+ `HEALED [provider=codex-subscription]`, and `kiro-subscription` healed the same spec.
132
+
133
+ - **`cursor-subscription` reliability note.** Verifying it (see above) showed Cursor's `agent` CLI
134
+ is an interactive assistant rather than a one-shot completion endpoint given the heal prompt it
135
+ often answers conversationally instead of with the required JSON, so the heal is declined more
136
+ often than with `kiro`/`codex`. It's now invoked in the safest/best-effort form
137
+ (`agent -p --trust --mode ask --output-format json`, read-only) and is marked **experimental**;
138
+ the docs point to `kiro`/`codex` as the steadier local-subscription options. Report issues.
139
+ Also: the full e2e healing suite (12 tests) was run end to end against `copilot-subscription` and
140
+ `ollama` **12/12 each** — to confirm the parser rewrite and provider changes don't regress it.
141
+
142
+ ### Added
143
+
144
+ - **`npx tamash-playwright init-skill` one command to install the orchestration skill, plus a
145
+ `doctor` check that flags when it's missing or stale.** The skill this package ships
146
+ (`skills/tamash-playwright/`) was previously a copy-this-shell-one-liner-per-agent step buried in
147
+ `SKILL.md`, with nothing verifying it had been done — so a project could `npm install` the
148
+ package, never copy the skill, and its assistant would silently never load the workflow.
149
+ `init-skill` copies the skill (`SKILL.md` + `references/`) into **both** standard locations —
150
+ `.claude/skills/tamash-playwright/` (Claude Code) and `.agents/skills/tamash-playwright/` (the
151
+ emerging cross-tool standard, read by Cursor, GitHub Copilot, Windsurf, Kiro, Zed, dotnet Aspire,
152
+ the same convention Playwright's own `playwright-cli install --skills` uses). Same content in
153
+ both; no per-agent format conversion. `--target claude` / `--target agents` installs one; `--user`
154
+ installs under your home directory; `--force` overwrites a hand-edited copy; `--dry-run` previews.
155
+ Each install carries a `tamash-playwright-skill-version:` marker; `doctor`'s **Skill** section
156
+ reads it and reports `[OK]` current / `[WARN]` behind-the-package-version / `[INFO]` not-installed
157
+ or unmanaged, with the same one-line fix. Covered by unit tests (`skill.test.js`, real installs
158
+ into a temp dir) and verified live end to end.
159
+
160
+ *(0.11.0-beta.7 shipped an earlier `init-skill` that instead auto-detected the agent and wrote a
161
+ Cursor-specific `.mdc` file / appended blocks to `.github/copilot-instructions.md` and
162
+ `AGENTS.md`. That approach is gone `init-skill` and `doctor` now point out any such leftovers
163
+ from a beta.7 install so you can delete them by hand.)*
164
+
165
+ - **`doctor`'s AI Provider check now tells you *what kind* of failure it hit and what to do about
166
+ it, instead of one generic "no valid response".** Every provider gained an optional, diagnostics-
167
+ only `diagnose()` method (never on the healing hot path) that runs one trivial round trip and
168
+ reports a structured category `not-installed`, `not-authenticated`, `timeout`, `bad-model`,
169
+ `network`, `bad-response` each of which `doctor` maps to a specific next step: the missing
170
+ `npm install`/CLI installer for a missing SDK; "run `claude login` / check your API key / confirm
171
+ your subscription is active and within quota" for a rejected request; "raise `actionTimeout`" (plus
172
+ the existing subprocess cold-start note) for a timeout; "check `<MODEL>` in your .env" for a
173
+ rejected model id; a network/proxy/base-URL hint for a connection failure. The raw error line is
174
+ still printed verbatim beneath the guidance. Previously `doctor` could only distinguish "worked"
175
+ from "didn't" — a missing SDK, an expired login, a slow network and a wrong model name all
176
+ produced the same line. Providers that don't implement `diagnose()` fall back to the old
177
+ `suggestSelector()` probe (worked/didn't, plus a timing-based timeout guess). Verified live:
178
+ `openai` bad key -> `not-authenticated` (401 body shown), `openai` bad base URL -> `network`,
179
+ `copilot-subscription` bad model id -> `bad-model`, 1ms `actionTimeout` -> `timeout` with the
180
+ raise-the-timeout guidance, and all three not-installed shapes by moving the dependency out of
181
+ resolution and restoring it: `@github/copilot-sdk` (CJS `MODULE_NOT_FOUND`),
182
+ `@anthropic-ai/claude-agent-sdk` (ESM `ERR_MODULE_NOT_FOUND`), and a missing `kiro-cli` binary
183
+ (`ENOENT`) -- each surfacing `not-installed` with its exact install command.
184
+
185
+ - **Three new local-development-only subscription providers: `cursor-subscription`,
186
+ `kiro-subscription`, `codex-subscription`.** Extends the same "use what you're already paying
187
+ for" idea `claude-subscription`/`copilot-subscription` already give, to three more real
188
+ subscriptionsbut each is explicitly scoped to local development, never documented or
189
+ recommended for CI, for two genuinely different reasons found by checking real vendor docs before
190
+ writing any code. Cursor's and Kiro's own headless-mode docs confirm neither has a way to fully
191
+ disable tool/file/command access the way `claude-subscription`'s `tools: []`/`copilot-subscription`'s
192
+ `availableTools: []` do — Cursor's `agent -p` "has access to all tools" by its own docs, and Kiro's
193
+ headless mode requires `--trust-all-tools`/`--trust-tools` specifically because there's no one to
194
+ approve a tool call unattended; `kiro-subscription` deliberately never grants that trust, so a
195
+ tool-requiring response just times out and declines rather than being granted broad access.
196
+ `codex-subscription` is local-only for a different, simpler reason: `codex exec` (used here, not
197
+ the interactive REPL) defaults to a read-only sandbox with no approval prompts — already as safe
198
+ as the two existing subscription providers but no long-lived, subscription-only token for
199
+ unattended CI use is confirmed to exist for it yet. None of the three vendors ship a Node SDK, so
200
+ all three are `child_process` wrappers around the vendor's own CLI binary (`agent`, `kiro-cli`,
201
+ `codex`) via a new shared `runCliPrompt` helper, rather than the SDK-based design the existing two
202
+ use meaning every call pays a fresh process-spawn cost, not just the first the way
203
+ `copilot-subscription`'s warm shared client does; `doctor` now has a distinct hint explaining this
204
+ when one of these three fails within a tight `actionTimeout`. Investigated and explicitly ruled
205
+ out this round: Antigravity its dedicated `google-antigravity` SDK is Python-only (this is a
206
+ Node/TypeScript package), has no documented tool-restricted mode at all, and its authentication
207
+ story isn't published, so it isn't even confirmed to ride on an existing subscription. Verified:
208
+ real graceful-decline behavior confirmed against genuinely uninstalled CLIs (no fabricated
209
+ simulation), and the full `doctor` integration (including the new every-call-cost hint) confirmed
210
+ against a real sample repo. **Not yet verified**: the actual happy path against a real installed
211
+ and authenticated CLI for any of the three none were available in the development environment,
212
+ and all three require a real paid subscription account.
213
+
214
+ - **A skill for running this package's local workflow inside an AI coding assistant.** Ships at
215
+ `skills/tamash-playwright/` — a `SKILL.md` entry point that branches on `npx tamash-playwright
216
+ doctor`'s actual output (never assumed), plus two reference docs: `onboarding.md` (bringing a
217
+ project up to standard provider setup, `actionTimeout`, `.describe()` labels, Page Object
218
+ extraction) and `heal.md` (a gated review/apply/verify/land loop over `apply-heals` and
219
+ `verify-heals.cjs` most runs proceed start to finish unattended, pausing only for a genuinely
220
+ ambiguous fix or anything after a failed verification, and never landing/committing/opening a PR
221
+ without asking first, no matter how clean the run was). Pure orchestration over commands that
222
+ already exist no new healing capability. Ships with adapters for Claude Code, Kiro (identical
223
+ `SKILL.md` format, confirmed against Kiro's own docs), Cursor (`.mdc` rule), GitHub Copilot
224
+ (`copilot-instructions.md` section), and an `AGENTS.md` covering the broader cross-tool standard
225
+ (Antigravity, Gemini CLI, Windsurf, Zed, Aider, and others) none of which are auto-discovered
226
+ from `node_modules` by any of these tools, confirmed rather than assumed, so every adapter's
227
+ install step is one explicit copy command, documented in `SKILL.md` itself.
228
+ - **A new `tamash` heal provider rule-based healing, no AI at all.** `HEALER_PROVIDER=tamash`
229
+ needs no API key, no subscription, and makes no network call: it resolves a broken locator by
230
+ text-matching the same description an AI provider would receive (`.describe()`, or a decoded
231
+ variable name — see below) against the already-captured accessibility snapshot, then reuses the
232
+ exact same structural widening (`near`/`adjacent`) the AI-backed path already uses once it finds
233
+ the right anchor — same output shape, same downstream code, zero duplicated logic. It shares the
234
+ same non-negotiable discipline as every text-matching step in this package: zero or more than one
235
+ match, at any point, means it declines rather than guesses verified with a dedicated permanent
236
+ e2e suite covering direct matches, widened matches, and every decline path (including the known,
237
+ accepted limitation that a misleading type hint can't be second-guessed against). Genuinely a
238
+ different tool than the AI providers, not a free replacement for one: no vision fallback (nothing
239
+ to reason over a screenshot with) and no action-recovery tactics (those require understanding
240
+ *why* an action failed, which is inference this provider deliberately doesn't attempt) a fast,
241
+ free, fully deterministic first line of defense, best suited to well-`.describe()`d,
242
+ Page-Object-style suites.
243
+ - **A new `ollama-local` heal provider for self-hosted Ollama servers.** `HEALER_PROVIDER=ollama-local`
244
+ targets your own `ollama serve` instance or an internal company deployment, instead of Ollama
245
+ Cloud — a deliberately separate provider from `ollama`, not a flag on it, since the two have
246
+ genuinely different auth defaults: Ollama Cloud always requires `OLLAMA_API_KEY`, while
247
+ `OLLAMA_LOCAL_API_KEY` is optional, since a bare `ollama serve` has no authentication at all. Set
248
+ it only if your internal deployment sits behind a reverse proxy or API gateway that requires a
249
+ bearer token verified against a real HTTP server both ways, confirming the `Authorization`
250
+ header is omitted entirely when no key is set, and sent correctly when one is. Prompted by a real
251
+ support request from a team wanting to use their own internally-hosted `gpt-oss:120b`.
252
+ - **Undescribed, POM-style locator variables now get a real description automatically.** When
253
+ `.describe()` was never called, the healer already fell back to the locator's own variable/
254
+ property name (`txtEmployeeId`) as its best guess at intent now that raw identifier is decoded
255
+ into the same kind of human-readable phrase `.describe()` would give: `txtEmployeeId` becomes
256
+ "Employee Id (textbox)", `submitButton` becomes "Submit (button)", recognizing both prefix-style
257
+ (`btnSubmit`) and suffix-style (`submitBtn`) naming, camelCase/snake_case/kebab-case, and correct
258
+ acronym boundaries (`employeeIDNumber` "Employee ID Number"). Falls back to the raw identifier,
259
+ exactly as before, whenever nothing meaningful survives decoding (a placeholder name like `el1`,
260
+ or a bare affix with nothing else) — never a guess dressed up as a real description. Verified
261
+ live against a real, unpublished build in both sample repos (a real `copilot-subscription` call,
262
+ and — after finding and fixing a genuinely stale API key along the way — a real `ollama` call),
263
+ confirming the decoded description actually reaches the configured AI provider unchanged.
264
+
265
+ ## [0.10.0] - 2026-08-27
266
+
267
+ ### Fixed
268
+
269
+ - **`waitFor()` is never sent to the AI.** It's a state check, not an action — a timeout on it can
270
+ mean a genuinely broken selector, or it can mean the element correctly never reached the expected
271
+ state (verifying something does NOT appear, or a real app issue), and there's no way to tell those
272
+ apart from the error alone. `expect(locator).toBeVisible()` was already permanently excluded from
273
+ healing for exactly this reason, but never reached this code at all (a separate path from the
274
+ Proxy-intercepted actions) `waitFor` just never got the same treatment. Real, reported case: a
275
+ user's `waitFor` on a locator verifying an absence (where timing out was the *correct* outcome)
276
+ still burned 8175 tokens across a failed text attempt and a failed vision attempt before giving
277
+ up, for a heal that could never have succeeded. Now fails fast with a clear `state-wait-not-healed`
278
+ stage and zero AI calls; a real action (fill/click/...) on the same kind of broken locator is
279
+ unaffected.
280
+ - **`claude-subscription` was burning a real, unnecessary amount of extra tokens.** The SDK's
281
+ `effort` option silently defaults to `'high'` ("deep reasoning") when left unset, and this
282
+ provider never set it a small prompt-complexity increase (the `nearbyRef`/`nearbyText` addition
283
+ below) pushed adaptive thinking to reason unpredictably harder for a task that only ever needs to
284
+ return one line of JSON. Confirmed live via real CI runs: output tokens for the same heal went
285
+ from a 188-445 baseline up to 422-881, inconsistently, run to run. `thinking: { type: 'disabled' }`
286
+ + `effort: 'low'` removes the variability entirely verified repeatedly at a steady 28-30 output
287
+ tokens, below even the pre-regression baseline, with no change to correctness.
288
+ - **The primary ariaSnapshot capture no longer requests `boxes:true`.** Every node was paying for a
289
+ `[box=x,y,w,h]` annotation that nothing on the text/`ref` path (including the `nearbyRef`/
290
+ `adjacent`-strategy widening logic below) ever reads it's purely topological. The one real
291
+ consumer (vision's own nearest-candidate lookup) already captures its own separate, fresh
292
+ snapshot, so this is genuinely free: verified live, input tokens dropped ~23% on a large real
293
+ page (3794 2937) with no loss of accuracy, and no change on small pages (box overhead scales
294
+ with node count).
295
+
296
+ ### Added
297
+
298
+ - **Search-scoped snapshots**: before falling back to the full page, the healer now searches the
299
+ already-captured snapshot for the description's identifying phrase and, only when it matches
300
+ exactly one node, sends the AI a scoped excerpt (that node's own subtree plus every sibling
301
+ branch's subtree at each ancestor level up to the root) instead of the whole page — zero extra
302
+ browser round-trips, since it's pure processing on data already in memory. Falls back to the full
303
+ snapshot automatically whenever the search is empty or ambiguous, never a guess. Verified live
304
+ across every real failure mode found this session: 41% token reduction on a deeply-nested field,
305
+ 66% when the target was inside what looked like an unrelated navigation menu (proving it finds
306
+ wherever the relevant text actually is, not "excludes the nav"), ~29% on a pair of identical
307
+ sibling fields that still had to be correctly disambiguated, and a clean, correct fallback when
308
+ the description doesn't match the page's real text at all.
309
+ - **A new `adjacent` selector strategy**, fixing a real ambiguity in the existing `near` strategy:
310
+ when two fields with no identity of their own share a row/section (two dropdowns side by side,
311
+ say), `near`'s "climb to a shared ancestor, then search it for any element of this role"
312
+ approach matches both and gives up rather than risk the wrong one. The AI's `ref` response can
313
+ now optionally report `nearbyRef`/`nearbyText`/`nearbyRole` for a nameless target it identified;
314
+ `deriveDurableLocator` uses that hint to find the true common ancestor between the target and
315
+ its label — via each ref's own full ancestor chain, not by assuming either sits at a matching
316
+ depth and, when they're proven to be immediate sibling branches, builds a precise CSS
317
+ `:text() + *` sibling match (or an xpath climb-then-step, when the label text turns out to be
318
+ nested below its own branch root). Verified live against a real configured provider, resolving
319
+ the correct field and never its same-row neighbor in both directions.
320
+ - **Full attempt-history logging**: `SelfHealingReport` now carries an `attempts[]` array one
321
+ entry per cache/ref/text/vision/action-recovery attempt actually made, each with its own
322
+ `succeeded`/`stage`/`error`. Previously only the *last* attempt's stage survived; an earlier
323
+ attempt's real failure (and the specific error it threw) was silently discarded the moment a
324
+ later attempt also failedthe exact shape of a real user-reported bug, where a genuine
325
+ candidate selector was shown next to an unrelated `vision_provider_error` with no way to tell
326
+ why the candidate itself hadn't worked. The attempt history now also prints directly to
327
+ console/CI output (not just the JSON attachment) whenever more than one attempt was made, on
328
+ both pass and fail since that plain-text output is what most bug reports actually paste, not
329
+ an attachment nobody opens.
330
+ - **`ariaSnapshot` is attached to the test report on failure** the exact accessibility tree the
331
+ AI reasoned over, so a confusing report can be diagnosed against real evidence instead of a
332
+ separately-captured DevTools screenshot.
333
+
334
+ ## [0.9.0] - 2026-08-26
335
+
336
+ ### Fixed
337
+
338
+ - **Heal-log visibility**: a heal with no reusable selector (a one-shot `ref` resolution, or a
339
+ vision-tagged point) was silently dropped from `heals.jsonl` entirely, even though it genuinely
340
+ fixed that run. Now logged for audit with a clear `reviewNote` whenever it has a suggestion
341
+ *or* a review note, without letting an audit-only entry shadow an older, real cached fix for the
342
+ same location.
343
+ - **Vision support for `claude-subscription`/`copilot-subscription`**: both providers had
344
+ `supportsVision` hardcoded to `false` regardless of model. Both SDKs genuinely support image
345
+ input (Claude Agent SDK via an image content block; Copilot SDK via a `blob` attachment)
346
+ implemented for real and verified live with actual screenshots, not just doctor's yes/no label.
347
+ - **Argument forwarding in healed replays**: `replayAction()` only ever forwarded the first call
348
+ argument. A trailing options object (`click({ modifiers: [...] })`, `fill(value, { timeout })`,
349
+ `dispatchEvent`'s 3rd argument) was silently dropped on a healed replay — changing the action's
350
+ real behavior while still reporting a clean `HEALED`.
351
+ - **`ref`-strategy replays now act through the derived durable locator, not the raw ref**:
352
+ `aria-ref=` locators never resolve for `dispatchEvent` at all (a Playwright-level limitation,
353
+ confirmed directly via `DEBUG=pw:api`), even though the identical element resolves instantly via
354
+ a `normalize()`-derived locator for that same action. Fixed by deriving the durable locator
355
+ *before* replaying and acting through it, falling back to the raw ref only when nothing durable
356
+ could be found. A related accuracy bug is fixed alongside it: the report/heal-log could
357
+ previously claim a derived selector "worked" even when its own replay had actually failed and a
358
+ fallback silently took over now only the locator that genuinely performed the action is
359
+ reported or cached.
360
+ - **`copilot-subscription` could hang a non-Playwright test runner indefinitely** — its shared
361
+ client keeps a connection open across calls for performance, which Playwright's own test runner
362
+ tolerates by force-exiting regardless, but a runner like Cucumber does not, so the process never
363
+ returned even after every test had already passed. Found live running a real Cucumber suite in
364
+ CI. `closeCopilotSubscriptionClient()` is now exported from the package's own entry point so any
365
+ non-Playwright consumer can call it from their own teardown hook (e.g. Cucumber's `AfterAll`).
366
+
367
+ ### Added
368
+
369
+ - **`locator.getDurable(action?)`** — resolves any locator (most usefully one built from
370
+ `aria-ref=...`) to a durable, reusable equivalent (`getByRole`/`getByLabel`/a css selector/…),
371
+ using the same derivation logic self-healing already uses internally. Throws if nothing durable
372
+ could be derived, rather than silently handing back something untrusted.
373
+ - **`apply-heals` now previews before writing**: every run — dry or real — shows a styled table of
374
+ exactly what it found (location, before, after, review status) instead of a plain scrolling log.
375
+ A real run also asks for confirmation before writing anything, but *only* at a genuine
376
+ interactive terminal — CI and any non-interactive/piped invocation proceed automatically exactly
377
+ as before, so no existing unattended CI workflow is affected. `--yes`/`-y` skips the prompt at a
378
+ real terminal too.
379
+ - **`doctor`'s output is styled** with colors and tables (including a new end-of-run Summary
380
+ section) instead of a plain scrolling log.
381
+ - **Exact locator code in console lines and reports**: a healed suggestion is now shown as the
382
+ real, copy-pasteable Playwright call (`getByRole("textbox", { name: "Username" })`) instead of
383
+ an abbreviated shorthand (`role:textbox:Username`) — the same code `apply-heals` would write to
384
+ source, so what you see is never a lossier stand-in for what was actually used.
385
+ - A canonical Page Object Model usage example (`tests/pages/` +
386
+ `tests/example-orangehrm-add-employee.spec.ts`), verified live against a real OrangeHRM demo,
387
+ including a genuine self-heal via the `near`/widening strategy on a real unlabeled field.
388
+
389
+ ### Changed
390
+
391
+ - README/usage.md's vision-fallback documentation now correctly lists `claude-subscription`/
392
+ `copilot-subscription` as vision-capable (with a suitable model), not just the API-key providers.
393
+
394
+ ## [0.8.0] - 2026-08-25
395
+
396
+ ### Added
397
+
398
+ - `claude-subscription` and `copilot-subscription` heal providers self-healing backed by a
399
+ personal Claude or GitHub Copilot subscription instead of a pay-per-token API key, working both
400
+ locally and unattended in CI (`CLAUDE_CODE_OAUTH_TOKEN` for Claude; the ambient `GITHUB_TOKEN` or
401
+ a personal-account PAT for Copilot on GitHub Actions).
402
+
403
+ ### Fixed
404
+
405
+ - Missing CLI install step in setup docs for `claude-subscription`/`copilot-subscription` the
406
+ standalone `claude`/`copilot` CLIs are genuinely required for the login step even though the SDKs
407
+ don't need them installed to function at runtime.
408
+
409
+ ### Documentation
410
+
411
+ - The org-vs-personal-account Copilot licensing gotcha in CI (a repo owned by an organization
412
+ without its own Copilot enablement can't use a personal subscription via the ambient token).
413
+ - Action Recovery (`HEALER_ACTION_RECOVERY_ENABLED`) hidden from docs and `doctor` output —
414
+ disabled by default already; this only reduced its visibility, no behavior change.
415
+
416
+ ## [0.7.0] - 2026-08-21
417
+
418
+ ### Added
419
+
420
+ - `apply-heals` turns a runtime heal into a permanent source-code fix, plus an opportunistic
421
+ cache (`heals.jsonl`) so a previously-confirmed selector is tried before a fresh AI call, and
422
+ history archival so a run's report/log isn't silently overwritten by the next one.
423
+ - Position/relationship-based locator strategies (`near`, `scoped`, `containing`) for elements with
424
+ no accessible identity of their own.
425
+ - Self-healing rebuilt around `ariaSnapshot({ mode: 'ai' })` + `aria-ref=` resolution and
426
+ `Locator.normalize()`, replacing pixel-distance guessing for both the text path and the vision
427
+ fallback's durability upgrade.
428
+ - A real unit test suite (32 tests at the time, zero new dependencies).
429
+
430
+ ### Fixed
431
+
432
+ - `doctor`'s `actionTimeout` check being fooled by a commented-out config value.
433
+ - `apply-heals --logs-dir` silently losing raw heal-log archival.
434
+ - `apply-heals` only replacing the first call when re-healing an already-`near`-fixed line.
435
+ - `apply-heals` `ENOENT` on a fresh CI checkout with no `.tamash-playwright/` directory yet — the
436
+ sharded "apply-heals" CI job checks out fresh and has never run tests itself, so the directory
437
+ genuinely doesn't exist there the way it always does in every local recipe.
438
+ - File path corruption (and a `require()` crash in the generated verification script) for ESM
439
+ (`"type": "module"`) consumer projects — `Error.stack` renders as a `file://` URL there, which
440
+ broke path resolution and a plain `.js` verification script alike.
441
+ - The AI being misled by the broken selector still present in its own prompt context; a failed
442
+ label guess now auto-upgrades to a structural `near` match instead of giving up.
443
+
444
+ ### Verified
445
+
446
+ - All four API-key providers (Ollama, OpenAI, Anthropic, Gemini) confirmed working with real API
447
+ calls, not just documented.
448
+
449
+ ## [0.6.0] - 2026-08-17
450
+
451
+ ### Added
452
+
453
+ - iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
454
+ enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
455
+ - `doctor`'s `actionTimeout` configuration check.
456
+
457
+ ### Initial release
458
+
459
+ - Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
460
+ with `.describe()` for human-readable context.