playwright-mutation-gate 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +67 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -209,6 +209,60 @@ npx playwright-mutation-gate "tests/**/*.spec.ts" --behavior
|
|
|
209
209
|
(a visibility check, a client-computed string, an API-request-fixture call) is
|
|
210
210
|
expected, not a defect.
|
|
211
211
|
|
|
212
|
+
### What the gate does for you
|
|
213
|
+
|
|
214
|
+
You write an ordinary test and mark its main assertion. The gate reads the value
|
|
215
|
+
out of that assertion, writes a throwaway copy of the spec with a `page.route()`
|
|
216
|
+
preamble injected at the top of the test, runs it, and deletes the copy. Your
|
|
217
|
+
spec and the app under test are never touched. The interception happens in the
|
|
218
|
+
browser at the network layer, so it works against any URL the test already drives
|
|
219
|
+
(local, staging, a deployed site) with no access to the app's source.
|
|
220
|
+
|
|
221
|
+
The preamble it generates per assertion is just this:
|
|
222
|
+
|
|
223
|
+
```ts
|
|
224
|
+
await page.route('**/*', async (route) => {
|
|
225
|
+
const resp = await route.fetch();
|
|
226
|
+
let body = await resp.text();
|
|
227
|
+
if (body.includes('Thanks, Ada!')) {
|
|
228
|
+
body = body.split('Thanks, Ada!').join('PMG_MUTATED_VALUE'); // break the named value in transit
|
|
229
|
+
}
|
|
230
|
+
await route.fulfill({ response: resp, body });
|
|
231
|
+
});
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
### A worked example
|
|
235
|
+
|
|
236
|
+
A test that reads fine on review but checks the wrong layer:
|
|
237
|
+
|
|
238
|
+
```ts
|
|
239
|
+
// The confirmation greeting is drawn on the client from a name cached at checkout
|
|
240
|
+
// (localStorage), not from this response. So the assertion checks the cache.
|
|
241
|
+
await expect(page.getByTestId('receipt-greeting')).toHaveText('Thanks, Ada!'); // @primary-assert
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
Inversion kills it (flip the text and the test goes red), so it clears the
|
|
245
|
+
standard gate. Behavior mutation corrupts `Thanks, Ada!` in the response; the
|
|
246
|
+
client rebuilds it from the cached name, the assertion stays green:
|
|
247
|
+
|
|
248
|
+
```text
|
|
249
|
+
tests/receipt.spec.ts
|
|
250
|
+
behavior:
|
|
251
|
+
weak Greets the customer by name on the confirmation page :22
|
|
252
|
+
the test stayed green while the value it names was broken
|
|
253
|
+
|
|
254
|
+
behavior: 0 killed, 1 weak, 0 cannot-locate
|
|
255
|
+
Gate FAILED: hollow, unverified or broken assertions above.
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
A runnable version ships in the
|
|
259
|
+
[ai-qa-pipeline](https://github.com/VladyslavDmitriiev/ai-qa-pipeline) demo repo.
|
|
260
|
+
Clone it and run:
|
|
261
|
+
|
|
262
|
+
```bash
|
|
263
|
+
npx playwright-mutation-gate tests/behavior-demo/receipt.spec.ts --behavior
|
|
264
|
+
```
|
|
265
|
+
|
|
212
266
|
The two modes compose and neither replaces the other. Inversion is cheap and
|
|
213
267
|
catches assertions that cannot fail at all (dead selectors, missing `await`,
|
|
214
268
|
silent skips). Behavior mutation is expensive and catches weak assertions.
|
|
@@ -225,6 +279,19 @@ on the client (common on SPAs) also report `cannot-locate`, since the literal
|
|
|
225
279
|
never reaches an interceptable response. As with inversion, the gate reports
|
|
226
280
|
what it could not do rather than guessing.
|
|
227
281
|
|
|
282
|
+
**Which stacks it fits.** Behavior mutation needs the asserted string to appear
|
|
283
|
+
verbatim in an HTTP response it can intercept. It fits server-rendered HTML
|
|
284
|
+
(Rails, Django, Laravel, Express templates, Next.js and Nuxt SSR) and any app
|
|
285
|
+
whose API returns the exact string the UI shows. It goes mostly blind on
|
|
286
|
+
client-rendered SPAs that build the text in the browser: if a React or Vue app
|
|
287
|
+
fetches `{"totalCents": 3400}` and renders `$34.00`, or assembles the string
|
|
288
|
+
through i18n or concatenation, the literal never crosses the wire and every such
|
|
289
|
+
assertion comes back `cannot-locate`. The same is true of formatted dates,
|
|
290
|
+
pluralized counts, and anything derived client-side. A mismatched stack costs you
|
|
291
|
+
nothing, since `cannot-locate` does not fail the gate, but it will not gain you
|
|
292
|
+
much either. Inversion still works everywhere; only the behavior pass is
|
|
293
|
+
stack-sensitive.
|
|
294
|
+
|
|
228
295
|
## CLI reference
|
|
229
296
|
|
|
230
297
|
```
|
package/package.json
CHANGED