tamash-playwright 0.13.0-beta.3 → 0.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +474 -485
- package/RELEASE-TESTING.md +406 -403
- package/package.json +1 -1
- package/usage.md +2 -4
package/CHANGELOG.md
CHANGED
|
@@ -1,485 +1,474 @@
|
|
|
1
|
-
# Changelog
|
|
2
|
-
|
|
3
|
-
All notable changes to this project are documented here. Format loosely follows
|
|
4
|
-
[Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
|
|
5
|
-
|
|
6
|
-
## [
|
|
7
|
-
|
|
8
|
-
### Added
|
|
9
|
-
|
|
10
|
-
- **AI-powered analysis of a
|
|
11
|
-
|
|
12
|
-
Four
|
|
13
|
-
|
|
14
|
-
### Fixed
|
|
15
|
-
|
|
16
|
-
- **A context/page built off `browser` in `test.beforeAll`
|
|
17
|
-
|
|
18
|
-
### Docs
|
|
19
|
-
|
|
20
|
-
- **README.md rewritten
|
|
21
|
-
|
|
22
|
-
- **README.md/usage.md/feature.md
|
|
23
|
-
|
|
24
|
-
## [0.12.0] - 2026-09-13
|
|
25
|
-
|
|
26
|
-
### Added
|
|
27
|
-
|
|
28
|
-
- **Playwright 1.63 support.** Dev/CI now tracks `@playwright/test` 1.63; `peerDependencies`
|
|
29
|
-
stays `>=1.40.0`. Verified: existing iframe healing (`page.frameLocator('#id')`) still heals,
|
|
30
|
-
and the full unit + representative e2e suite passes against 1.63.
|
|
31
|
-
|
|
32
|
-
- **Healing through Playwright 1.63's no-argument `page.frameLocator()`.** 1.63 made
|
|
33
|
-
`frameLocator()`'s selector optional — with no argument it matches inside *any* frame on the
|
|
34
|
-
page. As a healing scope that's ambiguous the moment the page has more than one frame (the
|
|
35
|
-
healer's `locator('body')` snapshot then throws *"frameLocator() matched elements in multiple
|
|
36
|
-
frames"*, and healing silently no-op'd with `stage=no_snapshot`). Now, when the page has
|
|
37
|
-
exactly one frame, the healer re-expresses the no-arg `frameLocator()` as an explicit
|
|
38
|
-
single-frame `FrameLocator` and heals normally — including deriving a **durable, persistable**
|
|
39
|
-
cross-frame selector (`apply-heals` can write it back to source), identical to what
|
|
40
|
-
`page.frameLocator('#id')` produces. With zero or several frames it can't know which was meant,
|
|
41
|
-
so it steps aside cleanly — the original error is re-thrown, never a wrong-frame guess. For
|
|
42
|
-
healable work on a multi-frame page, pass an explicit selector: `page.frameLocator('#id')`.
|
|
43
|
-
|
|
44
|
-
*(`0.12.0-beta.1` collapsed the scope to a raw `page.frames()` `Frame` instead of a
|
|
45
|
-
`FrameLocator`. `Locator.normalize()` on a `Frame`-rooted locator returns a correct selector
|
|
46
|
-
string but an object that resolves to nothing, so the heal worked once at runtime but couldn't
|
|
47
|
-
be persisted — it fell back to a transient one-shot element reference with a `needsReview`
|
|
48
|
-
note. Fixed here.)*
|
|
49
|
-
|
|
50
|
-
- **Documented [`tamash-playwright-dashboard`](https://www.npmjs.com/package/tamash-playwright-dashboard).**
|
|
51
|
-
A separate, zero-config reporter package: pass-rate trends, per-test history across runs, and —
|
|
52
|
-
specific to this package — a Self-Healing Analytics page (tests/elements healed, token usage
|
|
53
|
-
per run and cumulatively, every heal event across recorded history), read directly from the
|
|
54
|
-
`self-healing-<action>` JSON attachment this package already writes. No code change on this
|
|
55
|
-
side — README and usage.md gained a "Trends across runs" section pointing to it.
|
|
56
|
-
|
|
57
|
-
### Fixed
|
|
58
|
-
|
|
59
|
-
- **`describeFactoryCall` rendered `iframe "undefined"` for a no-arg `frameLocator()`.** The
|
|
60
|
-
factory-call label helper assumed a selector argument was always present; with 1.63's optional
|
|
61
|
-
selector it produced the literal string `iframe "undefined"`. Now renders `any iframe`. (Latent
|
|
62
|
-
— the label isn't currently surfaced to users or the model, but it would be the moment any
|
|
63
|
-
"healing inside iframe X" context is added to a report or prompt.)
|
|
64
|
-
|
|
65
|
-
## [0.11.0] - 2026-08-31
|
|
66
|
-
|
|
67
|
-
### Fixed
|
|
68
|
-
|
|
69
|
-
- **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
|
|
70
|
-
|
|
71
|
-
- **`doctor`'s connectivity check
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
- **
|
|
92
|
-
(
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
- **The
|
|
120
|
-
|
|
121
|
-
the
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
`
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
(`
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
`
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
`
|
|
195
|
-
`
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
`
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
`
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
`
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
- **
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
|
|
278
|
-
|
|
279
|
-
|
|
280
|
-
|
|
281
|
-
|
|
282
|
-
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
-
|
|
314
|
-
|
|
315
|
-
|
|
316
|
-
|
|
317
|
-
snapshot
|
|
318
|
-
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
- **
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
the
|
|
334
|
-
- **
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
353
|
-
|
|
354
|
-
|
|
355
|
-
|
|
356
|
-
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
|
|
360
|
-
|
|
361
|
-
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
|
|
390
|
-
|
|
391
|
-
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
412
|
-
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
a personal
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
431
|
-
|
|
432
|
-
|
|
433
|
-
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
459
|
-
|
|
460
|
-
-
|
|
461
|
-
|
|
462
|
-
|
|
463
|
-
|
|
464
|
-
|
|
465
|
-
|
|
466
|
-
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
### Added
|
|
477
|
-
|
|
478
|
-
- iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
|
|
479
|
-
enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
|
|
480
|
-
- `doctor`'s `actionTimeout` configuration check.
|
|
481
|
-
|
|
482
|
-
### Initial release
|
|
483
|
-
|
|
484
|
-
- Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
|
|
485
|
-
with `.describe()` for human-readable context.
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. Format loosely follows
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/); dates are when each version was published.
|
|
5
|
+
|
|
6
|
+
## [0.13.0] - 2026-09-17
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
|
|
10
|
+
- **AI-powered analysis of why a test failed.** Classifies a genuinely-failed test (every retry exhausted) as `likely-defect`, `likely-wrong-locator`, `likely-timing-or-environment`, or `inconclusive`, with a short explanation. Covers any final failure, not just `expect()` — an action failure healing already reported on is analyzed too, using healing's own diagnosis (provider, failure stage, reason) as context. On by default (`FAILURE_ANALYSIS_ENABLED`), independent of `HEALER_ENABLED`. No reporter to register — runs as a `test.extend()` auto-fixture, the same mechanism `bindContext`/`bindPageActions`/`bindBrowser` use. Reuses whichever `HEALER_PROVIDER` is configured; `tamash` always returns `inconclusive`. One AI call per genuinely-failed test when retries are set at the project level (`playwright.config.ts`'s `retries:`, or `--retries`); a per-file `test.describe.configure({ retries })` override is not visible to the fixture, so that case analyzes every failing attempt instead of one. Reported the same way healing's own reports are: a `failure-analysis` annotation, a `failure-analysis-tokens-used` annotation (separate from healing's `llm-tokens-used`), and a `failure-analysis` JSON attachment with the full attempt history. `expect()` itself is never touched.
|
|
11
|
+
|
|
12
|
+
Four bugs were fixed during development, each covered by a permanent regression test: (1) the working log was keyed by `testFile:testLine`, which is not unique for parameterized tests sharing a source line — fixed by keying on `testInfo.testId`. (2) A flaky test that passes on retry left a stale row in the working log — fixed by clearing it on the passing branch too. (3) An action failure that healing already reported on received no failure-analysis verdict — fixed by analyzing it too, passing healing's own diagnosis in as context. (4) `readFailureAttempts` dropped the `healingAttempt` field on every read despite writing it to disk correctly — fixed by adding the field to its return list.
|
|
13
|
+
|
|
14
|
+
### Fixed
|
|
15
|
+
|
|
16
|
+
- **A context/page built off `browser` in `test.beforeAll` was not healing-aware.** Only the `context`/`page` fixtures were wrapped with the healing proxy; `test.beforeAll(async ({ browser }) => { context = await browser.newContext(); page = await context.newPage(); })` bypassed both. Fixed: `browser.newContext()`, `browser.newPage()`, and `browser.contexts()` now return healing-aware objects, matching `context.newPage()`. Does not cover a browser obtained outside the fixture system (e.g. `chromium.launch()` in `globalSetup`) — see "What else it heals" in README.md/usage.md for the manual `bindContext()`/`bindPageActions()` option.
|
|
17
|
+
|
|
18
|
+
### Docs
|
|
19
|
+
|
|
20
|
+
- **README.md rewritten — 666 lines to ~220, no content removed.** Deep-dive content (subscription-provider setup, the full CI YAML example) moved into usage.md's existing sections; README links out to them. Fixed a gap: the `.env`/usage.md "pick one" provider comment was missing `cursor-subscription`/`kiro-subscription`/`codex-subscription`.
|
|
21
|
+
|
|
22
|
+
- **README.md/usage.md/feature.md state the fixture-healing rule directly.** "What else it heals" now leads with the rule: any browser/context/page from Playwright's own fixtures heals automatically, however many hops away; anything built outside the fixture system needs manual `bindContext()`/`bindPageActions()`.
|
|
23
|
+
|
|
24
|
+
## [0.12.0] - 2026-09-13
|
|
25
|
+
|
|
26
|
+
### Added
|
|
27
|
+
|
|
28
|
+
- **Playwright 1.63 support.** Dev/CI now tracks `@playwright/test` 1.63; `peerDependencies`
|
|
29
|
+
stays `>=1.40.0`. Verified: existing iframe healing (`page.frameLocator('#id')`) still heals,
|
|
30
|
+
and the full unit + representative e2e suite passes against 1.63.
|
|
31
|
+
|
|
32
|
+
- **Healing through Playwright 1.63's no-argument `page.frameLocator()`.** 1.63 made
|
|
33
|
+
`frameLocator()`'s selector optional — with no argument it matches inside *any* frame on the
|
|
34
|
+
page. As a healing scope that's ambiguous the moment the page has more than one frame (the
|
|
35
|
+
healer's `locator('body')` snapshot then throws *"frameLocator() matched elements in multiple
|
|
36
|
+
frames"*, and healing silently no-op'd with `stage=no_snapshot`). Now, when the page has
|
|
37
|
+
exactly one frame, the healer re-expresses the no-arg `frameLocator()` as an explicit
|
|
38
|
+
single-frame `FrameLocator` and heals normally — including deriving a **durable, persistable**
|
|
39
|
+
cross-frame selector (`apply-heals` can write it back to source), identical to what
|
|
40
|
+
`page.frameLocator('#id')` produces. With zero or several frames it can't know which was meant,
|
|
41
|
+
so it steps aside cleanly — the original error is re-thrown, never a wrong-frame guess. For
|
|
42
|
+
healable work on a multi-frame page, pass an explicit selector: `page.frameLocator('#id')`.
|
|
43
|
+
|
|
44
|
+
*(`0.12.0-beta.1` collapsed the scope to a raw `page.frames()` `Frame` instead of a
|
|
45
|
+
`FrameLocator`. `Locator.normalize()` on a `Frame`-rooted locator returns a correct selector
|
|
46
|
+
string but an object that resolves to nothing, so the heal worked once at runtime but couldn't
|
|
47
|
+
be persisted — it fell back to a transient one-shot element reference with a `needsReview`
|
|
48
|
+
note. Fixed here.)*
|
|
49
|
+
|
|
50
|
+
- **Documented [`tamash-playwright-dashboard`](https://www.npmjs.com/package/tamash-playwright-dashboard).**
|
|
51
|
+
A separate, zero-config reporter package: pass-rate trends, per-test history across runs, and —
|
|
52
|
+
specific to this package — a Self-Healing Analytics page (tests/elements healed, token usage
|
|
53
|
+
per run and cumulatively, every heal event across recorded history), read directly from the
|
|
54
|
+
`self-healing-<action>` JSON attachment this package already writes. No code change on this
|
|
55
|
+
side — README and usage.md gained a "Trends across runs" section pointing to it.
|
|
56
|
+
|
|
57
|
+
### Fixed
|
|
58
|
+
|
|
59
|
+
- **`describeFactoryCall` rendered `iframe "undefined"` for a no-arg `frameLocator()`.** The
|
|
60
|
+
factory-call label helper assumed a selector argument was always present; with 1.63's optional
|
|
61
|
+
selector it produced the literal string `iframe "undefined"`. Now renders `any iframe`. (Latent
|
|
62
|
+
— the label isn't currently surfaced to users or the model, but it would be the moment any
|
|
63
|
+
"healing inside iframe X" context is added to a report or prompt.)
|
|
64
|
+
|
|
65
|
+
## [0.11.0] - 2026-08-31
|
|
66
|
+
|
|
67
|
+
### Fixed
|
|
68
|
+
|
|
69
|
+
- **`skills/tamash-playwright/references/heal.md` never told the agent the Playwright HTML report exists at the point it matters.** It only pointed at `npx playwright show-report` at the very end (the REPORT step), after the loop was already finished — not right after RUN, where the report's per-attempt annotation and JSON attachment (provider, vision/action-recovery involvement, suggested selector, token cost, failure stage) would actually help decide what to do next. Found by directly auditing the skill against every documented user workflow step, not by running it. Fixed: RUN now explicitly says what the report contains and when it's worth opening.
|
|
70
|
+
|
|
71
|
+
- **`doctor`'s connectivity check used a fixed 15s timeout, not the project's own `actionTimeout`.**
|
|
72
|
+
A project using `actionTimeout: 8000` could get `[OK] Connected successfully` from `doctor`, then
|
|
73
|
+
have its first real heal fail with a misleading "not authenticated?" warning: `claude-subscription`/
|
|
74
|
+
`copilot-subscription` spawn the vendor's CLI as a subprocess, and the first call in a process pays
|
|
75
|
+
a one-time cold-start cost on top of the model call, which the old 15s check absorbed but a smaller
|
|
76
|
+
`actionTimeout` does not. Found during a from-scratch setup walkthrough (see the skill entry
|
|
77
|
+
below). Fixed: `doctor` now tests connectivity within the project's real `actionTimeout` when one
|
|
78
|
+
is configured, falling back to the 15s default only when none is set. A `[FAIL]` for either
|
|
79
|
+
provider now names the cold-start cost as a possible cause. Verified against `copilot-subscription`
|
|
80
|
+
that the check uses the configured timeout.
|
|
81
|
+
|
|
82
|
+
- **The published package was silently carrying stale, orphaned compiled output.** `npm run build`
|
|
83
|
+
(plain `tsc`, no `--build`/incremental mode) never deletes `dist/` output for a source file that
|
|
84
|
+
was later removed — found in the `0.11.0-beta.1` publish itself: `dist/healer/providers/
|
|
85
|
+
claude-cli.js`/`copilot-cli.js` shipped in the tarball with no corresponding source file
|
|
86
|
+
anywhere in the repo (from an abandoned, never-committed, never-wired-up piece of past work) —
|
|
87
|
+
dead code nothing actually imports, but real, needless bloat in every install. Fixed by cleaning
|
|
88
|
+
`dist/` automatically before every build (`prebuild` script), verified by confirming a full
|
|
89
|
+
rebuild no longer produces those two files.
|
|
90
|
+
|
|
91
|
+
- **A popup/new tab opened via `context.waitForEvent('page')`, `page.on('popup', ...)`, or
|
|
92
|
+
`page.waitForEvent('popup')` was never healing-aware.** `bindContext`/`bindPageActions` already
|
|
93
|
+
made `context.newPage()`/`context.pages()`/`context.on('page', ...)` heal correctly, but these
|
|
94
|
+
three — including `context.waitForEvent('page')`, the pattern most commonly taught for popup
|
|
95
|
+
handling — fell through to a raw, unwrapped Playwright page. A broken locator inside the
|
|
96
|
+
resulting page just threw a plain timeout, with no `[self-healer]` line at all: healing was never
|
|
97
|
+
attempted, not just unsuccessful. Found while building the first-ever test for popup healing
|
|
98
|
+
(previously implemented with zero coverage); fixed by extending both bindings to wrap whatever
|
|
99
|
+
page these three hand back, the same way `newPage()` already did.
|
|
100
|
+
|
|
101
|
+
- **`tamash` (the rule-based provider) declined on a real, common page pattern it should have
|
|
102
|
+
resolved.** A floating label rendered as its own text node right next to a field whose accessible
|
|
103
|
+
name is *also* that same text (confirmed on a real login page: a "Username" label next to a field
|
|
104
|
+
that's itself named "Username") was miscounted as two competing candidates and declined, even
|
|
105
|
+
though there's genuinely only one field. `findRuleBasedMatch` now only treats a match as a real
|
|
106
|
+
competing candidate when it's independently resolvable on its own (has a ref and a plausible
|
|
107
|
+
role) — a bare text node repeating the same words is decorative, not a second option. Genuine
|
|
108
|
+
ambiguity (two actually-independent candidates) still declines exactly as before.
|
|
109
|
+
|
|
110
|
+
- **The HTML report's self-healing attachment was a wall of escape sequences on a failed
|
|
111
|
+
heal.** Playwright's own error messages embed ANSI colour codes in their "Call log" section;
|
|
112
|
+
the healer stored those raw in `report.reason` / `report.warning` / `attempts[].error`, and
|
|
113
|
+
`JSON.stringify` (which builds the `self-healing-<action>` attachment) escapes every ESC byte in the JSON, so a *not-healed* report rendered as unreadable escape soup. A
|
|
114
|
+
*successful* heal has short or absent error strings, which is why it only showed up on failures.
|
|
115
|
+
Fixed by stripping ANSI CSI sequences in `normalizeError` — the single point every captured
|
|
116
|
+
error passes through — so the attachment, the console line, and `heals.jsonl` are all plain
|
|
117
|
+
text. Verified against a real not-healed run.
|
|
118
|
+
|
|
119
|
+
- **The three CLI-based subscription providers (`cursor-subscription`, `kiro-subscription`,
|
|
120
|
+
`codex-subscription`) were verified against their real vendor CLIs for the first time — and none
|
|
121
|
+
of the three actually worked as shipped.** Four separate bugs, all found by running them, all now
|
|
122
|
+
fixed:
|
|
123
|
+
1. **`cursor-subscription` never returned a heal.** `agent -p` refuses to run in any directory
|
|
124
|
+
without workspace trust — it prints "Workspace Trust Required" and exits without answering.
|
|
125
|
+
Now passes `--trust --mode ask`; `--mode ask` is Cursor's own read-only Q&A mode ("read-only"
|
|
126
|
+
per its `--help`), which *removes* the old "has access to all tools, could edit a file"
|
|
127
|
+
caveat this provider carried — a heal call in `ask` mode cannot write or run anything.
|
|
128
|
+
2. **`codex-subscription` refused to run outside a Git repository** ("Not inside a trusted
|
|
129
|
+
directory and --skip-git-repo-check was not specified"). Now passes `--skip-git-repo-check`;
|
|
130
|
+
this only relaxes the where-may-I-run guard, not the sandbox — `codex exec` still runs
|
|
131
|
+
`approval: never` + read-only, the property this provider depends on.
|
|
132
|
+
3. **`codex exec` hung on stdin.** It drains stdin ("Reading additional input from stdin…") and
|
|
133
|
+
blocks until EOF; `runCliPrompt` left the child's stdin as an open pipe, so every call ran to
|
|
134
|
+
the timeout and was killed — confirmed: a prompt the CLI answers in ~8s "timed out" at 15s.
|
|
135
|
+
`runCliPrompt` now gives every child an already-closed stdin (`stdio: ['ignore', …]`),
|
|
136
|
+
harmless for `agent`/`kiro-cli`, the fix for `codex`.
|
|
137
|
+
4. **The JSON parsers failed on conversational output.** `parseSuggestion` /
|
|
138
|
+
`parseVisionSuggestion` / `parseActionTacticSuggestion` extracted JSON from prose with a
|
|
139
|
+
greedy first-`{`-to-last-`}` slice, which breaks the moment the response contains more than
|
|
140
|
+
one JSON object — exactly what cursor's `agent` produces (it echoes the system prompt's
|
|
141
|
+
example objects back before its real answer). Replaced with a balanced-brace-span scan that
|
|
142
|
+
takes the last valid object (a model states its answer last), old greedy slice kept as a last
|
|
143
|
+
resort. Improves robustness for every provider, not just the CLI ones.
|
|
144
|
+
|
|
145
|
+
Verified, all authenticated: `doctor` reports `[OK] Connected` for all three; a real
|
|
146
|
+
`npx playwright test` heal (broken locator, cache cleared) went green with
|
|
147
|
+
`HEALED [provider=codex-subscription]`, and `kiro-subscription` healed the same spec.
|
|
148
|
+
|
|
149
|
+
- **`cursor-subscription` reliability note.** Verifying it (see above) showed Cursor's `agent` CLI
|
|
150
|
+
is an interactive assistant rather than a one-shot completion endpoint — given the heal prompt it
|
|
151
|
+
often answers conversationally instead of with the required JSON, so the heal is declined more
|
|
152
|
+
often than with `kiro`/`codex`. It's now invoked in the safest/best-effort form
|
|
153
|
+
(`agent -p --trust --mode ask --output-format json`, read-only) and is marked **experimental**;
|
|
154
|
+
the docs point to `kiro`/`codex` as the steadier local-subscription options. Report issues.
|
|
155
|
+
Also: the full e2e healing suite (12 tests) was run end to end against `copilot-subscription` and
|
|
156
|
+
`ollama` — **12/12 each** — to confirm the parser rewrite and provider changes don't regress it.
|
|
157
|
+
|
|
158
|
+
### Added
|
|
159
|
+
|
|
160
|
+
- **`npx tamash-playwright init-skill` — one command to install the orchestration skill, plus a
|
|
161
|
+
`doctor` check that flags when it's missing or stale.** The skill this package ships
|
|
162
|
+
(`skills/tamash-playwright/`) was previously a copy-this-shell-one-liner-per-agent step buried in
|
|
163
|
+
`SKILL.md`, with nothing verifying it had been done — so a project could `npm install` the
|
|
164
|
+
package, never copy the skill, and its assistant would silently never load the workflow.
|
|
165
|
+
`init-skill` copies the skill (`SKILL.md` + `references/`) into **both** standard locations —
|
|
166
|
+
`.claude/skills/tamash-playwright/` (Claude Code) and `.agents/skills/tamash-playwright/` (the
|
|
167
|
+
emerging cross-tool standard, read by Cursor, GitHub Copilot, Windsurf, Kiro, Zed, dotnet Aspire,
|
|
168
|
+
… — the same convention Playwright's own `playwright-cli install --skills` uses). Same content in
|
|
169
|
+
both; no per-agent format conversion. `--target claude` / `--target agents` installs one; `--user`
|
|
170
|
+
installs under your home directory; `--force` overwrites a hand-edited copy; `--dry-run` previews.
|
|
171
|
+
Each install carries a `tamash-playwright-skill-version:` marker; `doctor`'s **Skill** section
|
|
172
|
+
reads it and reports `[OK]` current / `[WARN]` behind-the-package-version / `[INFO]` not-installed
|
|
173
|
+
or unmanaged, with the same one-line fix. Covered by unit tests (`skill.test.js`, real installs
|
|
174
|
+
into a temp dir) and verified end to end.
|
|
175
|
+
|
|
176
|
+
*(0.11.0-beta.7 shipped an earlier `init-skill` that instead auto-detected the agent and wrote a
|
|
177
|
+
Cursor-specific `.mdc` file / appended blocks to `.github/copilot-instructions.md` and
|
|
178
|
+
`AGENTS.md`. That approach is gone — `init-skill` and `doctor` now point out any such leftovers
|
|
179
|
+
from a beta.7 install so you can delete them by hand.)*
|
|
180
|
+
|
|
181
|
+
- **`doctor`'s AI Provider check now tells you *what kind* of failure it hit and what to do about
|
|
182
|
+
it, instead of one generic "no valid response".** Every provider gained an optional, diagnostics-
|
|
183
|
+
only `diagnose()` method (never on the healing hot path) that runs one trivial round trip and
|
|
184
|
+
reports a structured category — `not-installed`, `not-authenticated`, `timeout`, `bad-model`,
|
|
185
|
+
`network`, `bad-response` — each of which `doctor` maps to a specific next step: the missing
|
|
186
|
+
`npm install`/CLI installer for a missing SDK; "run `claude login` / check your API key / confirm
|
|
187
|
+
your subscription is active and within quota" for a rejected request; "raise `actionTimeout`" (plus
|
|
188
|
+
the existing subprocess cold-start note) for a timeout; "check `<MODEL>` in your .env" for a
|
|
189
|
+
rejected model id; a network/proxy/base-URL hint for a connection failure. The raw error line is
|
|
190
|
+
still printed verbatim beneath the guidance. Previously `doctor` could only distinguish "worked"
|
|
191
|
+
from "didn't" — a missing SDK, an expired login, a slow network and a wrong model name all
|
|
192
|
+
produced the same line. Providers that don't implement `diagnose()` fall back to the old
|
|
193
|
+
`suggestSelector()` probe (worked/didn't, plus a timing-based timeout guess). Verified:
|
|
194
|
+
`openai` bad key -> `not-authenticated` (401 body shown), `openai` bad base URL -> `network`,
|
|
195
|
+
`copilot-subscription` bad model id -> `bad-model`, 1ms `actionTimeout` -> `timeout` with the
|
|
196
|
+
raise-the-timeout guidance, and all three not-installed shapes by moving the dependency out of
|
|
197
|
+
resolution and restoring it: `@github/copilot-sdk` (CJS `MODULE_NOT_FOUND`),
|
|
198
|
+
`@anthropic-ai/claude-agent-sdk` (ESM `ERR_MODULE_NOT_FOUND`), and a missing `kiro-cli` binary
|
|
199
|
+
(`ENOENT`) -- each surfacing `not-installed` with its exact install command.
|
|
200
|
+
|
|
201
|
+
- **Three new local-development-only subscription providers: `cursor-subscription`,
|
|
202
|
+
`kiro-subscription`, `codex-subscription`.** Extends the same "use what you're already paying
|
|
203
|
+
for" idea `claude-subscription`/`copilot-subscription` already give, to three more real
|
|
204
|
+
subscriptions — but each is explicitly scoped to local development, never documented or
|
|
205
|
+
recommended for CI, for two genuinely different reasons found by checking real vendor docs before
|
|
206
|
+
writing any code. Cursor's and Kiro's own headless-mode docs confirm neither has a way to fully
|
|
207
|
+
disable tool/file/command access the way `claude-subscription`'s `tools: []`/`copilot-subscription`'s
|
|
208
|
+
`availableTools: []` do — Cursor's `agent -p` "has access to all tools" by its own docs, and Kiro's
|
|
209
|
+
headless mode requires `--trust-all-tools`/`--trust-tools` specifically because there's no one to
|
|
210
|
+
approve a tool call unattended; `kiro-subscription` deliberately never grants that trust, so a
|
|
211
|
+
tool-requiring response just times out and declines rather than being granted broad access.
|
|
212
|
+
`codex-subscription` is local-only for a different, simpler reason: `codex exec` (used here, not
|
|
213
|
+
the interactive REPL) defaults to a read-only sandbox with no approval prompts — already as safe
|
|
214
|
+
as the two existing subscription providers — but no long-lived, subscription-only token for
|
|
215
|
+
unattended CI use is confirmed to exist for it yet. None of the three vendors ship a Node SDK, so
|
|
216
|
+
all three are `child_process` wrappers around the vendor's own CLI binary (`agent`, `kiro-cli`,
|
|
217
|
+
`codex`) via a new shared `runCliPrompt` helper, rather than the SDK-based design the existing two
|
|
218
|
+
use — meaning every call pays a fresh process-spawn cost, not just the first the way
|
|
219
|
+
`copilot-subscription`'s warm shared client does; `doctor` now has a distinct hint explaining this
|
|
220
|
+
when one of these three fails within a tight `actionTimeout`. Investigated and explicitly ruled
|
|
221
|
+
out this round: Antigravity — its dedicated `google-antigravity` SDK is Python-only (this is a
|
|
222
|
+
Node/TypeScript package), has no documented tool-restricted mode at all, and its authentication
|
|
223
|
+
story isn't published, so it isn't even confirmed to ride on an existing subscription. Verified:
|
|
224
|
+
real graceful-decline behavior confirmed against genuinely uninstalled CLIs (no fabricated
|
|
225
|
+
simulation), and the full `doctor` integration (including the new every-call-cost hint) confirmed
|
|
226
|
+
against a real sample repo. **Not yet verified**: the actual happy path against a real installed
|
|
227
|
+
and authenticated CLI for any of the three — none were available in the development environment,
|
|
228
|
+
and all three require a real paid subscription account.
|
|
229
|
+
|
|
230
|
+
- **A skill for running this package's local workflow inside an AI coding assistant.** Ships at
|
|
231
|
+
`skills/tamash-playwright/` — a `SKILL.md` entry point that branches on `npx tamash-playwright
|
|
232
|
+
doctor`'s actual output (never assumed), plus two reference docs: `onboarding.md` (bringing a
|
|
233
|
+
project up to standard — provider setup, `actionTimeout`, `.describe()` labels, Page Object
|
|
234
|
+
extraction) and `heal.md` (a gated review/apply/verify/land loop over `apply-heals` and
|
|
235
|
+
`verify-heals.cjs` — most runs proceed start to finish unattended, pausing only for a genuinely
|
|
236
|
+
ambiguous fix or anything after a failed verification, and never landing/committing/opening a PR
|
|
237
|
+
without asking first, no matter how clean the run was). Pure orchestration over commands that
|
|
238
|
+
already exist — no new healing capability. Ships with adapters for Claude Code, Kiro (identical
|
|
239
|
+
`SKILL.md` format, confirmed against Kiro's own docs), Cursor (`.mdc` rule), GitHub Copilot
|
|
240
|
+
(`copilot-instructions.md` section), and an `AGENTS.md` covering the broader cross-tool standard
|
|
241
|
+
(Antigravity, Gemini CLI, Windsurf, Zed, Aider, and others) — none of which are auto-discovered
|
|
242
|
+
from `node_modules` by any of these tools, confirmed rather than assumed, so every adapter's
|
|
243
|
+
install step is one explicit copy command, documented in `SKILL.md` itself.
|
|
244
|
+
- **A new `tamash` heal provider — rule-based healing, no AI at all.** `HEALER_PROVIDER=tamash`
|
|
245
|
+
needs no API key, no subscription, and makes no network call: it resolves a broken locator by
|
|
246
|
+
text-matching the same description an AI provider would receive (`.describe()`, or a decoded
|
|
247
|
+
variable name — see below) against the already-captured accessibility snapshot, then reuses the
|
|
248
|
+
exact same structural widening (`near`/`adjacent`) the AI-backed path already uses once it finds
|
|
249
|
+
the right anchor — same output shape, same downstream code, zero duplicated logic. It shares the
|
|
250
|
+
same non-negotiable discipline as every text-matching step in this package: zero or more than one
|
|
251
|
+
match, at any point, means it declines rather than guesses — verified with a dedicated permanent
|
|
252
|
+
e2e suite covering direct matches, widened matches, and every decline path (including the known,
|
|
253
|
+
accepted limitation that a misleading type hint can't be second-guessed against). Genuinely a
|
|
254
|
+
different tool than the AI providers, not a free replacement for one: no vision fallback (nothing
|
|
255
|
+
to reason over a screenshot with) and no action-recovery tactics (those require understanding
|
|
256
|
+
*why* an action failed, which is inference this provider deliberately doesn't attempt) — a fast,
|
|
257
|
+
free, fully deterministic first line of defense, best suited to well-`.describe()`d,
|
|
258
|
+
Page-Object-style suites.
|
|
259
|
+
- **A new `ollama-local` heal provider for self-hosted Ollama servers.** `HEALER_PROVIDER=ollama-local`
|
|
260
|
+
targets your own `ollama serve` instance or an internal company deployment, instead of Ollama
|
|
261
|
+
Cloud — a deliberately separate provider from `ollama`, not a flag on it, since the two have
|
|
262
|
+
genuinely different auth defaults: Ollama Cloud always requires `OLLAMA_API_KEY`, while
|
|
263
|
+
`OLLAMA_LOCAL_API_KEY` is optional, since a bare `ollama serve` has no authentication at all. Set
|
|
264
|
+
it only if your internal deployment sits behind a reverse proxy or API gateway that requires a
|
|
265
|
+
bearer token — verified against a real HTTP server both ways, confirming the `Authorization`
|
|
266
|
+
header is omitted entirely when no key is set, and sent correctly when one is. Prompted by a real
|
|
267
|
+
support request from a team wanting to use their own internally-hosted `gpt-oss:120b`.
|
|
268
|
+
- **Undescribed, POM-style locator variables now get a real description automatically.** When
|
|
269
|
+
`.describe()` was never called, the healer already fell back to the locator's own variable/
|
|
270
|
+
property name (`txtEmployeeId`) as its best guess at intent — now that raw identifier is decoded
|
|
271
|
+
into the same kind of human-readable phrase `.describe()` would give: `txtEmployeeId` becomes
|
|
272
|
+
"Employee Id (textbox)", `submitButton` becomes "Submit (button)", recognizing both prefix-style
|
|
273
|
+
(`btnSubmit`) and suffix-style (`submitBtn`) naming, camelCase/snake_case/kebab-case, and correct
|
|
274
|
+
acronym boundaries (`employeeIDNumber` → "Employee ID Number"). Falls back to the raw identifier,
|
|
275
|
+
exactly as before, whenever nothing meaningful survives decoding (a placeholder name like `el1`,
|
|
276
|
+
or a bare affix with nothing else) — never a guess dressed up as a real description. Verified
|
|
277
|
+
live against a real, unpublished build in both sample repos (a real `copilot-subscription` call,
|
|
278
|
+
and — after finding and fixing a genuinely stale API key along the way — a real `ollama` call),
|
|
279
|
+
confirming the decoded description actually reaches the configured AI provider unchanged.
|
|
280
|
+
|
|
281
|
+
## [0.10.0] - 2026-08-27
|
|
282
|
+
|
|
283
|
+
### Fixed
|
|
284
|
+
|
|
285
|
+
- **`waitFor()` is never sent to the AI.** It's a state check, not an action — a timeout on it can
|
|
286
|
+
mean a genuinely broken selector, or it can mean the element correctly never reached the expected
|
|
287
|
+
state (verifying something does NOT appear, or a real app issue), and there's no way to tell those
|
|
288
|
+
apart from the error alone. `expect(locator).toBeVisible()` was already permanently excluded from
|
|
289
|
+
healing for exactly this reason, but never reached this code at all (a separate path from the
|
|
290
|
+
Proxy-intercepted actions) — `waitFor` just never got the same treatment. Real, reported case: a
|
|
291
|
+
user's `waitFor` on a locator verifying an absence (where timing out was the *correct* outcome)
|
|
292
|
+
still burned 8175 tokens across a failed text attempt and a failed vision attempt before giving
|
|
293
|
+
up, for a heal that could never have succeeded. Now fails fast with a clear `state-wait-not-healed`
|
|
294
|
+
stage and zero AI calls; a real action (fill/click/...) on the same kind of broken locator is
|
|
295
|
+
unaffected.
|
|
296
|
+
- **`claude-subscription` used more tokens than necessary.** The SDK's `effort` option defaults to
|
|
297
|
+
`'high'` when left unset, and this provider never set it — a small prompt-complexity increase (the
|
|
298
|
+
`nearbyRef`/`nearbyText` addition below) pushed adaptive thinking higher for a task that only needs
|
|
299
|
+
to return one line of JSON. Output tokens for the same heal ranged 188-445 before, 422-881 after,
|
|
300
|
+
across real CI runs. `thinking: { type: 'disabled' }` + `effort: 'low'` fixes this: a steady 28-30
|
|
301
|
+
output tokens, below the original baseline, with no change to correctness.
|
|
302
|
+
- **The primary ariaSnapshot capture no longer requests `boxes:true`.** Every node was paying for a
|
|
303
|
+
`[box=x,y,w,h]` annotation that nothing on the text/`ref` path (including the `nearbyRef`/
|
|
304
|
+
`adjacent`-strategy widening logic below) ever reads — it's purely topological. The one real
|
|
305
|
+
consumer (vision's own nearest-candidate lookup) already captures its own separate, fresh
|
|
306
|
+
snapshot, so this is genuinely free: verified, input tokens dropped ~23% on a large real
|
|
307
|
+
page (3794 → 2937) with no loss of accuracy, and no change on small pages (box overhead scales
|
|
308
|
+
with node count).
|
|
309
|
+
|
|
310
|
+
### Added
|
|
311
|
+
|
|
312
|
+
- **Search-scoped snapshots**: before falling back to the full page, the healer now searches the
|
|
313
|
+
already-captured snapshot for the description's identifying phrase and, only when it matches
|
|
314
|
+
exactly one node, sends the AI a scoped excerpt (that node's own subtree plus every sibling
|
|
315
|
+
branch's subtree at each ancestor level up to the root) instead of the whole page — zero extra
|
|
316
|
+
browser round-trips, since it's pure processing on data already in memory. Falls back to the full
|
|
317
|
+
snapshot automatically whenever the search is empty or ambiguous, never a guess. Verified across
|
|
318
|
+
several real cases: 41% token reduction on a deeply-nested field,
|
|
319
|
+
66% when the target was inside what looked like an unrelated navigation menu (proving it finds
|
|
320
|
+
wherever the relevant text actually is, not "excludes the nav"), ~29% on a pair of identical
|
|
321
|
+
sibling fields that still had to be correctly disambiguated, and a clean, correct fallback when
|
|
322
|
+
the description doesn't match the page's real text at all.
|
|
323
|
+
- **A new `adjacent` selector strategy**, fixing a real ambiguity in the existing `near` strategy:
|
|
324
|
+
when two fields with no identity of their own share a row/section (two dropdowns side by side,
|
|
325
|
+
say), `near`'s "climb to a shared ancestor, then search it for any element of this role"
|
|
326
|
+
approach matches both and gives up rather than risk the wrong one. The AI's `ref` response can
|
|
327
|
+
now optionally report `nearbyRef`/`nearbyText`/`nearbyRole` for a nameless target it identified;
|
|
328
|
+
`deriveDurableLocator` uses that hint to find the true common ancestor between the target and
|
|
329
|
+
its label — via each ref's own full ancestor chain, not by assuming either sits at a matching
|
|
330
|
+
depth — and, when they're proven to be immediate sibling branches, builds a precise CSS
|
|
331
|
+
`:text() + *` sibling match (or an xpath climb-then-step, when the label text turns out to be
|
|
332
|
+
nested below its own branch root). Verified against a real configured provider, resolving
|
|
333
|
+
the correct field and never its same-row neighbor in both directions.
|
|
334
|
+
- **Full attempt-history logging**: `SelfHealingReport` now carries an `attempts[]` array — one
|
|
335
|
+
entry per cache/ref/text/vision/action-recovery attempt actually made, each with its own
|
|
336
|
+
`succeeded`/`stage`/`error`. Previously only the *last* attempt's stage survived; an earlier
|
|
337
|
+
attempt's real failure (and the specific error it threw) was silently discarded the moment a
|
|
338
|
+
later attempt also failed — the exact shape of a real user-reported bug, where a genuine
|
|
339
|
+
candidate selector was shown next to an unrelated `vision_provider_error` with no way to tell
|
|
340
|
+
why the candidate itself hadn't worked. The attempt history now also prints directly to
|
|
341
|
+
console/CI output (not just the JSON attachment) whenever more than one attempt was made, on
|
|
342
|
+
both pass and fail — since that plain-text output is what most bug reports actually paste, not
|
|
343
|
+
an attachment nobody opens.
|
|
344
|
+
- **`ariaSnapshot` is attached to the test report on failure** — the exact accessibility tree the
|
|
345
|
+
AI reasoned over, so a confusing report can be diagnosed against real evidence instead of a
|
|
346
|
+
separately-captured DevTools screenshot.
|
|
347
|
+
|
|
348
|
+
## [0.9.0] - 2026-08-26
|
|
349
|
+
|
|
350
|
+
### Fixed
|
|
351
|
+
|
|
352
|
+
- **Heal-log visibility**: a heal with no reusable selector (a one-shot `ref` resolution, or a
|
|
353
|
+
vision-tagged point) was silently dropped from `heals.jsonl` entirely, even though it genuinely
|
|
354
|
+
fixed that run. Now logged for audit — with a clear `reviewNote` — whenever it has a suggestion
|
|
355
|
+
*or* a review note, without letting an audit-only entry shadow an older, real cached fix for the
|
|
356
|
+
same location.
|
|
357
|
+
- **Vision support for `claude-subscription`/`copilot-subscription`**: both providers had
|
|
358
|
+
`supportsVision` hardcoded to `false` regardless of model. Both SDKs genuinely support image
|
|
359
|
+
input (Claude Agent SDK via an image content block; Copilot SDK via a `blob` attachment) —
|
|
360
|
+
implemented for real and verified with actual screenshots, not just doctor's yes/no label.
|
|
361
|
+
- **Argument forwarding in healed replays**: `replayAction()` only ever forwarded the first call
|
|
362
|
+
argument. A trailing options object (`click({ modifiers: [...] })`, `fill(value, { timeout })`,
|
|
363
|
+
`dispatchEvent`'s 3rd argument) was silently dropped on a healed replay — changing the action's
|
|
364
|
+
real behavior while still reporting a clean `HEALED`.
|
|
365
|
+
- **`ref`-strategy replays now act through the derived durable locator, not the raw ref**:
|
|
366
|
+
`aria-ref=` locators never resolve for `dispatchEvent` at all (a Playwright-level limitation,
|
|
367
|
+
confirmed directly via `DEBUG=pw:api`), even though the identical element resolves instantly via
|
|
368
|
+
a `normalize()`-derived locator for that same action. Fixed by deriving the durable locator
|
|
369
|
+
*before* replaying and acting through it, falling back to the raw ref only when nothing durable
|
|
370
|
+
could be found. A related accuracy bug is fixed alongside it: the report/heal-log could
|
|
371
|
+
previously claim a derived selector "worked" even when its own replay had actually failed and a
|
|
372
|
+
fallback silently took over — now only the locator that genuinely performed the action is
|
|
373
|
+
reported or cached.
|
|
374
|
+
- **`copilot-subscription` could hang a non-Playwright test runner indefinitely.** Its shared
|
|
375
|
+
client keeps a connection open across calls for performance; Playwright's own test runner
|
|
376
|
+
force-exits regardless, but a runner like Cucumber does not, so the process never returns even
|
|
377
|
+
after every test has passed. `closeCopilotSubscriptionClient()` is now exported from the
|
|
378
|
+
package's own entry point so a non-Playwright consumer can call it from their own teardown hook
|
|
379
|
+
(e.g. Cucumber's `AfterAll`).
|
|
380
|
+
|
|
381
|
+
### Added
|
|
382
|
+
|
|
383
|
+
- **`locator.getDurable(action?)`** — resolves any locator (most usefully one built from
|
|
384
|
+
`aria-ref=...`) to a durable, reusable equivalent (`getByRole`/`getByLabel`/a css selector/…),
|
|
385
|
+
using the same derivation logic self-healing already uses internally. Throws if nothing durable
|
|
386
|
+
could be derived, rather than silently handing back something untrusted.
|
|
387
|
+
- **`apply-heals` now previews before writing**: every run — dry or real — shows a styled table of
|
|
388
|
+
exactly what it found (location, before, after, review status) instead of a plain scrolling log.
|
|
389
|
+
A real run also asks for confirmation before writing anything, but *only* at a genuine
|
|
390
|
+
interactive terminal — CI and any non-interactive/piped invocation proceed automatically exactly
|
|
391
|
+
as before, so no existing unattended CI workflow is affected. `--yes`/`-y` skips the prompt at a
|
|
392
|
+
real terminal too.
|
|
393
|
+
- **`doctor`'s output is styled** with colors and tables (including a new end-of-run Summary
|
|
394
|
+
section) instead of a plain scrolling log.
|
|
395
|
+
- **Exact locator code in console lines and reports**: a healed suggestion is now shown as the
|
|
396
|
+
real, copy-pasteable Playwright call (`getByRole("textbox", { name: "Username" })`) instead of
|
|
397
|
+
an abbreviated shorthand (`role:textbox:Username`) — the same code `apply-heals` would write to
|
|
398
|
+
source, so what you see is never a lossier stand-in for what was actually used.
|
|
399
|
+
- A canonical Page Object Model usage example (`tests/pages/` +
|
|
400
|
+
`tests/example-orangehrm-add-employee.spec.ts`), verified against a real OrangeHRM demo,
|
|
401
|
+
including a genuine self-heal via the `near`/widening strategy on a real unlabeled field.
|
|
402
|
+
|
|
403
|
+
### Changed
|
|
404
|
+
|
|
405
|
+
- README/usage.md's vision-fallback documentation now correctly lists `claude-subscription`/
|
|
406
|
+
`copilot-subscription` as vision-capable (with a suitable model), not just the API-key providers.
|
|
407
|
+
|
|
408
|
+
## [0.8.0] - 2026-08-25
|
|
409
|
+
|
|
410
|
+
### Added
|
|
411
|
+
|
|
412
|
+
- `claude-subscription` and `copilot-subscription` heal providers — self-healing backed by a
|
|
413
|
+
personal Claude or GitHub Copilot subscription instead of a pay-per-token API key, working both
|
|
414
|
+
locally and unattended in CI (`CLAUDE_CODE_OAUTH_TOKEN` for Claude; the ambient `GITHUB_TOKEN` or
|
|
415
|
+
a personal-account PAT for Copilot on GitHub Actions).
|
|
416
|
+
|
|
417
|
+
### Fixed
|
|
418
|
+
|
|
419
|
+
- Missing CLI install step in setup docs for `claude-subscription`/`copilot-subscription` — the
|
|
420
|
+
standalone `claude`/`copilot` CLIs are genuinely required for the login step even though the SDKs
|
|
421
|
+
don't need them installed to function at runtime.
|
|
422
|
+
|
|
423
|
+
### Documentation
|
|
424
|
+
|
|
425
|
+
- The org-vs-personal-account Copilot licensing gotcha in CI (a repo owned by an organization
|
|
426
|
+
without its own Copilot enablement can't use a personal subscription via the ambient token).
|
|
427
|
+
- Action Recovery (`HEALER_ACTION_RECOVERY_ENABLED`) hidden from docs and `doctor` output —
|
|
428
|
+
disabled by default already; this only reduced its visibility, no behavior change.
|
|
429
|
+
|
|
430
|
+
## [0.7.0] - 2026-08-21
|
|
431
|
+
|
|
432
|
+
### Added
|
|
433
|
+
|
|
434
|
+
- `apply-heals` — turns a runtime heal into a permanent source-code fix, plus an opportunistic
|
|
435
|
+
cache (`heals.jsonl`) so a previously-confirmed selector is tried before a fresh AI call, and
|
|
436
|
+
history archival so a run's report/log isn't silently overwritten by the next one.
|
|
437
|
+
- Position/relationship-based locator strategies (`near`, `scoped`, `containing`) for elements with
|
|
438
|
+
no accessible identity of their own.
|
|
439
|
+
- Self-healing rebuilt around `ariaSnapshot({ mode: 'ai' })` + `aria-ref=` resolution and
|
|
440
|
+
`Locator.normalize()`, replacing pixel-distance guessing for both the text path and the vision
|
|
441
|
+
fallback's durability upgrade.
|
|
442
|
+
- A real unit test suite (32 tests at the time, zero new dependencies).
|
|
443
|
+
|
|
444
|
+
### Fixed
|
|
445
|
+
|
|
446
|
+
- `doctor`'s `actionTimeout` check being fooled by a commented-out config value.
|
|
447
|
+
- `apply-heals --logs-dir` silently losing raw heal-log archival.
|
|
448
|
+
- `apply-heals` only replacing the first call when re-healing an already-`near`-fixed line.
|
|
449
|
+
- `apply-heals` `ENOENT` on a fresh CI checkout with no `.tamash-playwright/` directory yet — the
|
|
450
|
+
sharded "apply-heals" CI job checks out fresh and has never run tests itself, so the directory
|
|
451
|
+
genuinely doesn't exist there the way it always does in every local recipe.
|
|
452
|
+
- File path corruption (and a `require()` crash in the generated verification script) for ESM
|
|
453
|
+
(`"type": "module"`) consumer projects — `Error.stack` renders as a `file://` URL there, which
|
|
454
|
+
broke path resolution and a plain `.js` verification script alike.
|
|
455
|
+
- The AI being misled by the broken selector still present in its own prompt context; a failed
|
|
456
|
+
label guess now auto-upgrades to a structural `near` match instead of giving up.
|
|
457
|
+
|
|
458
|
+
### Verified
|
|
459
|
+
|
|
460
|
+
- All four API-key providers (Ollama, OpenAI, Anthropic, Gemini) confirmed working with real API
|
|
461
|
+
calls, not just documented.
|
|
462
|
+
|
|
463
|
+
## [0.6.0] - 2026-08-17
|
|
464
|
+
|
|
465
|
+
### Added
|
|
466
|
+
|
|
467
|
+
- iframe and popup healing, vision fallback (screenshot-based recovery when text alone isn't
|
|
468
|
+
enough), AI-driven action recovery (scroll/force/wait/dispatch), and source-location reporting.
|
|
469
|
+
- `doctor`'s `actionTimeout` configuration check.
|
|
470
|
+
|
|
471
|
+
### Initial release
|
|
472
|
+
|
|
473
|
+
- Self-healing Playwright bindings: broken locators are recovered at runtime via an AI provider,
|
|
474
|
+
with `.describe()` for human-readable context.
|