playwright-mutation-gate 0.3.0 → 0.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +67 -0
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -209,6 +209,60 @@ npx playwright-mutation-gate "tests/**/*.spec.ts" --behavior
209
209
  (a visibility check, a client-computed string, an API-request-fixture call) is
210
210
  expected, not a defect.
211
211
 
212
+ ### What the gate does for you
213
+
214
+ You write an ordinary test and mark its main assertion. The gate reads the value
215
+ out of that assertion, writes a throwaway copy of the spec with a `page.route()`
216
+ preamble injected at the top of the test, runs it, and deletes the copy. Your
217
+ spec and the app under test are never touched. The interception happens in the
218
+ browser at the network layer, so it works against any URL the test already drives
219
+ (local, staging, a deployed site) with no access to the app's source.
220
+
221
+ The preamble it generates per assertion is just this:
222
+
223
+ ```ts
224
+ await page.route('**/*', async (route) => {
225
+ const resp = await route.fetch();
226
+ let body = await resp.text();
227
+ if (body.includes('Thanks, Ada!')) {
228
+ body = body.split('Thanks, Ada!').join('PMG_MUTATED_VALUE'); // break the named value in transit
229
+ }
230
+ await route.fulfill({ response: resp, body });
231
+ });
232
+ ```
233
+
234
+ ### A worked example
235
+
236
+ A test that reads fine on review but checks the wrong layer:
237
+
238
+ ```ts
239
+ // The confirmation greeting is drawn on the client from a name cached at checkout
240
+ // (localStorage), not from this response. So the assertion checks the cache.
241
+ await expect(page.getByTestId('receipt-greeting')).toHaveText('Thanks, Ada!'); // @primary-assert
242
+ ```
243
+
244
+ Inversion kills it (flip the text and the test goes red), so it clears the
245
+ standard gate. Behavior mutation corrupts `Thanks, Ada!` in the response; the
246
+ client rebuilds it from the cached name, the assertion stays green:
247
+
248
+ ```text
249
+ tests/receipt.spec.ts
250
+ behavior:
251
+ weak Greets the customer by name on the confirmation page :22
252
+ the test stayed green while the value it names was broken
253
+
254
+ behavior: 0 killed, 1 weak, 0 cannot-locate
255
+ Gate FAILED: hollow, unverified or broken assertions above.
256
+ ```
257
+
258
+ A runnable version ships in the
259
+ [ai-qa-pipeline](https://github.com/VladyslavDmitriiev/ai-qa-pipeline) demo repo.
260
+ Clone it and run:
261
+
262
+ ```bash
263
+ npx playwright-mutation-gate tests/behavior-demo/receipt.spec.ts --behavior
264
+ ```
265
+
212
266
  The two modes compose and neither replaces the other. Inversion is cheap and
213
267
  catches assertions that cannot fail at all (dead selectors, missing `await`,
214
268
  silent skips). Behavior mutation is expensive and catches weak assertions.
@@ -225,6 +279,19 @@ on the client (common on SPAs) also report `cannot-locate`, since the literal
225
279
  never reaches an interceptable response. As with inversion, the gate reports
226
280
  what it could not do rather than guessing.
227
281
 
282
+ **Which stacks it fits.** Behavior mutation needs the asserted string to appear
283
+ verbatim in an HTTP response it can intercept. It fits server-rendered HTML
284
+ (Rails, Django, Laravel, Express templates, Next.js and Nuxt SSR) and any app
285
+ whose API returns the exact string the UI shows. It goes mostly blind on
286
+ client-rendered SPAs that build the text in the browser: if a React or Vue app
287
+ fetches `{"totalCents": 3400}` and renders `$34.00`, or assembles the string
288
+ through i18n or concatenation, the literal never crosses the wire and every such
289
+ assertion comes back `cannot-locate`. The same is true of formatted dates,
290
+ pluralized counts, and anything derived client-side. A mismatched stack costs you
291
+ nothing, since `cannot-locate` does not fail the gate, but it will not gain you
292
+ much either. Inversion still works everywhere; only the behavior pass is
293
+ stack-sensitive.
294
+
228
295
  ## CLI reference
229
296
 
230
297
  ```
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "playwright-mutation-gate",
3
- "version": "0.3.0",
3
+ "version": "0.3.2",
4
4
  "description": "Mutation testing for Playwright assertions: invert each test's primary assertion and fail the run if the test stays green.",
5
5
  "keywords": [
6
6
  "playwright",