@skyramp/mcp 0.4.1 → 0.4.2-rc.2

package/plugin/skills/enhance-assertions/reference/shared-rules.md

@@ -0,0 +1,220 @@
+# Shared assertion rules
+
+Read this file first for `contract` and `integration` tests. For `ui` tests only the "Scope by mode" section below applies; the UI rule set in `reference/ui.md` replaces the rest.
+
+## What to assert
+
+Each test evaluates ONE behaviour — the one the recommendation it came from asked for. Assert that behaviour, and the values it depends on, exactly. Leave the rest of the response alone.
+
+A field belongs in the test when a wrong value in it would mean the behaviour is broken. A field does not belong when its value can change for reasons that have nothing to do with the behaviour — a generated id, a timestamp, an unrelated sibling field. Asserting those does not make the test stronger. It makes the test fail for reasons the test is not about, and it buries the one assertion a reader needs to understand what the test is for.
+
+So the rules below say how to assert a value, not how many values to assert. Apply a rule to the fields the behaviour depends on. `[]` is the correct answer for a rule whose fields this test does not depend on.
+
+If nothing in the response distinguishes the behaviour working from the behaviour broken, the test is exercising the wrong request or reading the wrong response. Say so; do not pad it with assertions on whatever else the body carries.
+
+## Scope by mode
+
+### When enhancing a freshly generated test (`$1` = `generation`)
+
+Apply the rules to every test function in `$2`.
+
+### When maintaining an existing test (`$1` = `maintenance`)
+
+Apply only to new test functions you are adding and existing test functions affected by changes in the diff. Do NOT modify test functions unrelated to the diff. A test function you do touch is held to the same standard as a new one: the behaviour the diff changed must be asserted by its exact value there too, so an existing `is not None` on a field the diff gives a knowable value becomes that value.
+
+## Pre-Edit Assertion Analysis
+
+Before editing `$2`, you must output a `<thinking>` block. It states the behaviour each test evaluates, then works through the rules for the responses that behaviour depends on, so no rule that applies is overlooked. The JSON array must match the `rule_checklist` template that accompanies the `$0` rules. Use the template's keys verbatim; each key is the slug shown in that rule's heading. Every key must appear — `[]` where the rule does not apply.
+
+1. For each test function in scope, name the behaviour it evaluates in one line, taken from the recommendation it came from. A generated test function may carry a name that states its behaviour — trust that name when it does. A function still named `test_integration` or `testUi` claims nothing, so use the name only when it is specific enough to add something, and never treat a generic one as evidence the test has no single behaviour. Then scan `$2` and the expected responses for the code change tested.
+2. Classify each response first by its response status type and then assign the applicable assertion rules to the response.
+   - Success with body (2xx with a response body): all assertion rules below may apply — echo-back of request fields, computed response fields, array / items validation, and chained values across steps.
+   - Success with no body (200/202/204 with an empty body — e.g. logout/cancel/submit action endpoints): assert the status code only. Also apply chained-values rules if a follow-up step uses this response's ID.
+   - Error response (the status is not 2xx, or the body is an error object such as `{"error": "..."}` with no resource payload): assert that the request was refused, as the Error path rule below states. Read the record back unchanged where the flow allows. Assert the exact status code or the exact error text only when a source states it. Where a source does state the wording, apply array / items validation to the `errors[]` array (exact length + per-item fields + next index null).
+3. For each in-scope response, output one JSON object using that `rule_checklist` template. The output is an array — one object per in-scope response.
+   Classify by the response the recorded trace or `expected_response_body` shows this request returning. The one exception is a test the plan declared `expected.outcome: fail` (typically a `bug_caught` or `requirement_conflict` candidate; read the declaration first and fall back to the category only when none was made): there the expected response is the intended behaviour established by the diff, the PR statement, or the code's evident intent, and the test is expected to fail until the code is fixed.
+   - `step`: the HTTP method, path, and response variable name for this request (e.g. `POST /products → products_POST_response`).
+   - `response_status`: one of `success`, `no_body`, or `error` based on the classification in step 2.
+   - `behaviour_under_test`: the one-line behaviour from step 1 that this response is evidence for. When a response is evidence for no behaviour any test in scope claims, write `none` and add nothing for it.
+   - `rule_checklist`: an object that MUST contain every rule title — every title in the shared rules plus every title in the `$0` rules — as a key. For each rule, the value is the assertion lines you will add for this response under that rule. `[]` means the rule does not apply here, which is a normal and expected answer: it is the right one whenever the rule's fields are not part of `behaviour_under_test`. Listing every key keeps a rule that DOES apply from being skipped; it is not a quota to fill.
+
+## SDK Helpers
+
+Access response body fields via the SDK helper already imported in the generated file (never dict/attribute access on the response variable): Python `skyramp.get_response_value(response, "json.path")`; TypeScript / JavaScript `getValue(response, "json.path")` or `getResponseValue(response, "json.path")` from `@skyramp/skyramp`; Java `getValue(response, "json.path")`.
+
+## Assertion Rules with Examples
+
+### Assert the behaviour under test (`behaviour-under-test`)
+
+Every test needs at least one assertion that fails when the behaviour it claims is broken. This is the rule the others support; a test that satisfies every rule below and still does not distinguish the behaviour working from the behaviour broken has not been enhanced.
+
+- Name the behaviour, then ask what value in the response changes when it breaks. Assert that value exactly.
+- When the test claims a state change ("Updates X", "Increments Y", "rejects Z"), the assertion must prove the change, not merely that the call succeeded.
+- When the PR changed the behaviour, the expected value is the one the diff, the PR description, or the referenced spec establishes — even when the running service still returns the old value. The test is expected to fail until the code is fixed.
+
+```javascript
+// "PATCH /orders applies the coupon discount" — the discount is the behaviour.
+expect(getResponseValue(patchResponse, "discount_amount"), 'discount applied').toBe(20);
+expect(getResponseValue(patchResponse, "total_amount"), 'total after discount').toBe(80);
+```
+
+### Echo-back the request fields (`req-fields-echo-back`)
+
+For every field returned unchanged from the request body, assert the exact sent value.
+
+- `is not None` / `not.toBeNull()` is only acceptable when the value is genuinely unknown — for server-generated timestamps or opaque IDs. This rule does not apply to computed fields.
+- When a response field's value equals the value sent in the request body, path, or query, assert that exact value rather than a null-check — the sent value is known and reproducible.
+- Assert the exact value of a response-only field when the request this test sends determines it (`filename_download`, `content_type`, `size`, an enum status after creation). When the behaviour is that a resource was created with the fields sent, the id alone does not prove it — assert those fields.
+- Range matchers like `toBeGreaterThanOrEqual(0)` and type-only checks like `typeof X === 'number'` are not acceptable for fields whose exact values are known.
+- On a SUCCESSFUL response, assert the exact status code from the recorded trace or `expected_response_body` — for example, `201` for resource-creation endpoints. (A refused request is judged by the Error path rule below, which does not pin today's code.) Action-style endpoints with verb path segments like `/run`, `/trigger`, `/execute`, `/deploy`, `/submit`, `/start`, `/stop`, or `/cancel` often use `200`, but assert the recorded or expected code. Permissive status matchers are never acceptable.
+
+```javascript
+expect(getValue(response, "data.filename_download")).toBe("invoice.pdf");
+expect(getValue(response, "data.type")).toBe("application/pdf");
+expect(response.statusCode).toBe(201);
+```
+
+```javascript
+expect(getResponseValue(productsPostResponse, "name"), 'name').toBe("Skyramp Tester");
+```
+
+### Error path (HTTP 4xx/5xx, or a 2xx carrying only an error body) (`error-path`)
+
+A request is refused when the status is not 2xx, or when the body is an error object (`error`, `message`, `errors[]`) and the resource fields carry no value. Assert the refusal: assert that the body carries the error field, and that the resource fields carry no value. Read the record back and assert it is unchanged, where the flow allows. Assert the exact status code or the exact error text only when a source states it — the plan's test case, the pull request, or a requirements file.
+
+- Where a source states the wording, also apply the array-validation rule to the errors array.
+- For a response with no body — a successful DELETE (204), or a refusal that returns nothing — assert the status code only. The status is the only evidence such a response carries, so assert it exactly even when no source states it.
+- For a 2xx error body, assert that the error field is present and that the resource fields carry no value (`toBeNull()` / `is None` — the SDK helper reads a missing path and an explicit null the same way, so this asserts "no payload", not "key absent"; use `checkSchema` when the two must be told apart). Assert the exact status code and the error field's exact value only when a source states them. Never assert the success shape (a non-empty collection, `error` absent) against a response the recorded trace or `expected_response_body` shows returning an error or no-match body — classify the response by what this request returns, not by the outcome the test was meant to check for. The one exception is a test the plan declared `expected.outcome: fail` (typically a `bug_caught` or `requirement_conflict` candidate; the category is only the fallback when nothing was declared): it asserts the intended behaviour established by the diff, the PR statement, or the code's evident intent — keep that assertion and let the test fail.
+
+```javascript
+// No source states the code or the wording, so neither is pinned
+expect(getValue(response, "errors.0.message")).not.toBeNull();
+expect(getValue(response, "data")).toBeNull();
+expect(getValue(itemGetResponse, "data.name")).toBe("Wireless keyboard");
+```
+
+```javascript
+// The plan's test case states 400 and the message
+expect(recordPostResponse.statusCode, 'status code').toBe(400);
+expect(getResponseValue(recordPostResponse, "error"), 'error').toBe("level must be one of Intern, Junior, Senior");
+expect(getResponseValue(recordPostResponse, "insertedId"), 'insertedId carries no value').toBeNull();
+```
+
+```python
+# A 2xx error body. The pull request states the wording; no source states the code.
+assert skyramp.get_response_value(search_post_response, "error") == "No items matched the criteria."
+assert skyramp.get_response_value(search_post_response, "results") is None
+```
+
+### Specific known values (`known-values`)
+
+For enum or status fields where only one outcome is valid for this flow, assert the exact expected value.
+
+- When the OpenAPI schema defines allowed values, assert the schema-defined value.
+- Also assert cross-field invariants when one field encodes the count or summary of another (e.g. `total_count == results.length`).
+
+```javascript
+expect(getResponseValue(ordersPostResponse, "status"), 'status').toBe("pending");
+```
+
+### Array / Items validation (`array-items`)
+
+Only assert indices that exist in the recorded response — never infer array length from the request or scenario name.
+
+- When the response is an empty array: assert the length is 0 and that the first index is absent. Never assert a field on index 0 of an empty array.
+- When the response is a non-empty array: assert the exact length, key fields on each item, and that the index after the last item is absent.
+- Shape-only checks such as `Array.isArray` or `typeof` are not sufficient when the response contains actual values.
+- If the response is sorted or ordered: assert the ordering direction across the first two items.
+- When the response is an empty or minimal body (an empty object, empty array, null, or only a few keys): assert the empty or minimal shape and the absence of error fields — do not stop at the status code. A minimal body that is itself an error object falls under the Error path rule instead.
+- When the request includes pagination or filter parameters, assert the response reflects them.
+
+```javascript
+expect(getResponseValue(patchResponse, "items.0.product_id")).toBe(getResponseValue(productPostResponse, "id"));
+expect(getResponseValue(patchResponse, "items.0.quantity")).toBe(2);
+expect(getResponseValue(patchResponse, "items.1.product_id")).toBeNull();
+expect(getValue(response, "data").length).toBeLessThanOrEqual(10);  // request sent limit=10
+```
+
+```javascript
+expect(getValue(response, "results").length).toBe(2);
+expect(getValue(response, "results.0.id")).toBe("prod_001");
+expect(getValue(response, "results.2")).toBeNull();
+```
+
+```javascript
+expect(getValue(response, "data").length).toBe(0);
+expect(getValue(response, "data.0")).toBeNull();
+```
+
+### Fields whose value varies across runs (`varying-values`)
+
+For a field whose exact value varies across runs — an ISO timestamp, an IP address, an email, a URL, or a number with no knowable exact value — assert its format or its range instead of its value, and only when the behaviour under test depends on that field.
+
+- ISO timestamp matches `/^\d{4}-\d{2}-\d{2}T/`.
+- IP address matches IPv4 or IPv6 format.
+- Email contains `@` and a domain.
+- URL starts with `http://` or `https://`.
+- Use format info from the OpenAPI schema (`format: date-time`) to identify these fields.
+- A numeric field with no knowable exact value takes a range from the field name, the domain, or the OpenAPI `minimum` / `maximum` — a range is the weakest assertion in this rule set, so use it only where no exact value exists and the behaviour depends on the field.
+
+```javascript
+expect(getResponseValue(productsPostResponse, "created_at")).toMatch(/^\d{4}-\d{2}-\d{2}T/);
+```
+
+### Use exact recorded JSON paths (`exact-json-paths`)
+
+Read every field at the path the recorded body actually uses. A wrapper prefix the response does not have makes the assertion read null and the test fail for a reason that is not the behaviour — if the body is flat (`{"id": "acc_...", "tax_exempt": true}`), the path is `"tax_exempt"`, never `"account.tax_exempt"`.
+
+```javascript
+expect(getResponseValue(accountResponse, "tax_exempt")).toBe(true);
+```
+
+### Computed response fields, derived using dynamic formulas (`computed-fields`)
+
+When the response contains a field whose value is derived from a calculation, assert the result using a formula built from prior response values and request inputs, not a hardcoded literal.
+
+- Use a dynamic formula when the formula is visible in source.
+- Fall back to the exact pre-computed value from `expected_response_body` only when no formula is knowable — all inputs must come from the inline request body or the response.
+
+```javascript
+expect(getResponseValue(patchResponse, "discount_amount")).toBe(getResponseValue(patchResponse, "total_amount") * (getResponseValue(patchResponse, "discount_value") / 100));
+```
+
+```javascript
+expect(getResponseValue(patchResponse, "total_amount")).toBe(getResponseValue(productPostResponse, "price") * quantitySent);
+```
+
+### Assert a nullable field is present, not merely null (`nullable-present`)
+
+Reading a field's value cannot tell a null field from a missing one — `get_response_value` / `getValue` return the same result for both, so `is None` does not prove the field exists. Use a schema check with an example body instead.
+
+- An example carrying `null` for the field asserts presence and nullness together, and fails when the field is absent.
+- An example carrying a concrete value only asserts that value's type, so a nullable field checked against a string example fails once the real value is null.
+- `check_schema` / `checkSchema` takes the RESPONSE OBJECT, not a parsed body. The `@skyramp/skyramp` type declaration says otherwise and is wrong — passing a plain object makes every check silently return false (SKYR-4189).
+
+```python
+assert skyramp.check_schema(order_response, '{"cancelled_at": null}')
+```
+
+```javascript
+expect(checkSchema(orderResponse, '{"cancelled_at": null}')).toBe(true);
+```
+
+## What not to do
+
+- Do not access response fields via dict syntax (`response["field"]`) or attribute access (`response.field`) — always use the SDK helper.
+- Do not assert `not.toBeNull()` / `is not None` on a field whose exact value is in the request body or a prior response.
+- Do not assert an exact value you learned only from a request you sent yourself while writing the test. Take the value from the diff, from what the request determines, or from a response this test received earlier.
+- Do not assert a field only because the body carries it. If a wrong value there would not mean the behaviour under test is broken, leave it alone.
+- Do not leave a test whose assertions would all still pass with the behaviour under test broken.
+- Do not use permissive status matchers (`.toMatch(/^2/)`, `.toBeGreaterThanOrEqual(200)`, `checkStatusCode(response, '20x')`).
+- Do not use shape-only or type-only assertions as a substitute for exact value validation. Forbidden patterns: `Array.isArray(...)`, `typeof X === '...'`, `X instanceof Array`, `Object.keys(X).length > 0`. When a field's value comes from one of those sources, assert it exactly.
+- Do not use shape-only, containment-only, range-only, or weak-length as the sole assertion on a populated array.
+- Do not swap between `getValue` and `getResponseValue` — keep whichever SDK helper the file already imports.
+- Do not restructure, reformat, reorder, or modify existing code; do not add comments or docstrings.
+- Do not change function signatures or variable names. You may add an import when a new assertion or a fix needs a helper — from `@skyramp/skyramp` or from the test's own helper module.
+- Do not remove existing assertions.
+
+## Verification of Assertions
+
+After adding the assertion lines in `$2`, re-read each test and check two things: every rule that applies to the behaviour under test has been applied correctly, and the test now fails if that behaviour breaks. Fix anything missing or weakly applied before completing. Remove any assertion you added that the behaviour does not depend on.

package/plugin/skills/enhance-assertions/reference/ui.md

@@ -0,0 +1,373 @@
+# UI test assertions
+
+## What to assert
+
+Each test evaluates ONE behaviour — the one the recommendation it came from asked for. Assert that behaviour on the page where it shows, and the rendered values it depends on, exactly. A page the test only passes through, and an element the behaviour does not touch, need no assertion: asserting them makes the test fail for reasons the test is not about and buries what the test is for. The rules below say how to assert a value, not how many values to assert.
+
+## Scope of this file
+
+This file is the complete rule set for UI tests. It replaces the response-body rules in `reference/shared-rules.md`; only that file's "Scope by mode" section also applies.
+
+## First Check
+
+If `$2` has no `expect()` assertions, you MUST manually add them before anything else. Use `import { expect } from '@skyramp/skyramp';` — never from `@playwright/test`. If an existing import pulls `expect` from `@playwright/test`, move it to `@skyramp/skyramp` (keep `test` on the playwright line).
+
+## Flutter Web Check
+
+If the test targets a Flutter web app (detected by `flt-semantics`, `flt-glass-pane`, or `flt-semantics-placeholder` appearing anywhere in `$2`, OR if the test navigates to a URL that served a Flutter app during recording), you MUST add the following two blocks immediately after `page.setDefaultTimeout(pageTimeout);` and before the `newSkyrampPlaywrightPage` wrap:
+
+```typescript
+// Flutter headless fix: patch Event.isTrusted so synthetic clicks activate Flutter semantics
+await page.context().addInitScript(() => {
+    Object.defineProperty(Event.prototype, 'isTrusted', { get: () => true, configurable: true });
+});
+```
+
+And immediately after **every** `page.goto()` call in the test, add exactly these three statements — do NOT substitute a different activation approach or different `waitForFunction` bodies:
+
+```typescript
+// Wait for Flutter to bootstrap (confirms Flutter engine is ready before activating)
+await page.waitForFunction(
+    () => !!document.querySelector('flt-semantics-placeholder, flt-glass-pane, flutter-view'),
+    undefined,
+    { timeout: 30000 }
+);
+// Activate Flutter accessibility semantics by clicking the hidden placeholder
+await page.evaluate(() => { const el = document.querySelector('flt-semantics-placeholder'); if (el) el.dispatchEvent(new MouseEvent('click', { isTrusted: true, bubbles: true, cancelable: true })); });
+// Wait for the semantics tree to be live: the activation placeholder is gone AND
+// at least one flt-semantics node exists. Requiring a populated tree (not just the
+// placeholder's absence) guards against a vacuous pass when the bootstrap wait
+// above resolved on flt-glass-pane/flutter-view before the placeholder ever mounted.
+await page.waitForFunction(
+    () => document.querySelector('flt-semantics-placeholder') === null
+        && document.querySelector('flt-semantics') !== null,
+    undefined,
+    { timeout: 20000 }
+);
+```
+
+Use exactly the three statements above. Do NOT use the two-argument form `waitForFunction(fn, { timeout })` — the timeout must be the third argument with `undefined` as the second. If you see an existing body click, `locator.click`, or old-style `waitForFunction` after a goto, replace the entire block with the exact three statements above.
+
+This is required because Flutter web renders to a `<canvas>` element (no traditional DOM). The first `waitForFunction` ensures Flutter has bootstrapped before activation; the `dispatchEvent` activates semantics; the second `waitForFunction` confirms the semantics tree is live (placeholder removed **and** `flt-semantics` nodes present). The combined gate handles both modes — when accessibility is pre-enabled (semantics active from frame 1, no placeholder ever shown) and when activation depends on the click — without passing prematurely on a frame where neither has happened yet.
+
+**Then wait for the first element you interact with.** The gate above confirms the semantics tree *started* building, but Flutter populates it incrementally — individual widgets (a textbox, a button) can still be a beat behind when the gate passes, especially on a cold start where accessibility was not pre-enabled. The wrapped Skyramp page also enforces a short internal action timeout, so a lagging widget fails fast rather than waiting out the page default. Guard against this: immediately after the activation block (and after any later `page.goto()`'s block), add an explicit `waitFor` on the **first locator the test interacts with on that page**, before the interaction itself. Example:
+
+```typescript
+// Wait for the specific target to be present before interacting (tree may still be filling in)
+await page.getByRole('textbox', { name: 'Username' }).waitFor({ state: 'visible', timeout: 20000 });
+await page.getByRole('textbox', { name: 'Username' }).fill('admin');
+```
+
+Use the **same locator** that the next interaction uses — do not invent a new selector. This waits for exactly the widget the test needs, which is more reliable than any generic tree-node count (those are app-specific magic numbers); it is the per-target wait that Flutter-web automation guidance recommends over a one-size-fits-all readiness check.
+
+**Also check for brittle flt-semantic-node-* selectors:** If the test uses `page.locator("#flt-semantic-node-N")` for any interaction (clicks, navigations), replace it with a semantic selector (`getByRole`, `getByText`, or `getByLabel`) — `flt-semantic-node-N` IDs change between sessions and across devices, making the test brittle. Example: replace `page.locator("#flt-semantic-node-78").click()` with `page.getByText("Settings").click()` using the visible label from the same line's context.
+
+**Flutter assertion grounding rule:** For Flutter web tests, use the following priority order when adding `expect()` assertions:
+
+1. **Existing selectors in the test file** — prefer these first; they were captured from the live accessibility tree during recording.
+2. **`browser_blueprint` snapshot** — if a blueprint was captured, use element roles, names, and text that appear verbatim in the snapshot.
+3. **Source code / diff** — only consult source code after exhausting the above. Flutter's accessibility tree frequently omits or transforms text (e.g. CJK characters split into individual nodes, `ExcludeSemantics` widgets, icon-only buttons), so what appears in the Dart source may not exist in the ARIA tree at runtime. If a selector derived from source cannot be confirmed in the blueprint or recorded trace, skip the assertion rather than risk a false positive.
+
+## Idempotent Setup Check
+
+Apply this check when — and only when — the test body contains a create-entity sequence: a "Create X" / "New X" / "Add X" button click that opens a dialog or form, followed by filling in a name or identifier, followed by a confirm/submit action whose result subsequent steps depend on.
+
+**Why:** Multiple generated tests run sequentially against a single shared SUT instance within the same CI job. The agent's own exploration phase may also create the entity before tests execute. Whichever test runs first succeeds; all later tests collide with the already-existing entity and time out on the "Create X" button.
+
+**This is the one exception to the "do not restructure" rule.** Wrapping entity creation in an existence check does not change the test's intent — it makes it robust to pre-existing state.
+
+**How to wrap it:**
+
+1. Before the create sequence, check whether the entity already exists (using `isVisible()` on the entity's list item or link, wrapped in `.catch(() => false)` to avoid throwing on timeout).
+2. If it exists: navigate to it directly.
+3. If it does not exist: run the original create sequence.
+
+For the existence check, use selectors already present in `$2`. If no list-item selector exists, fall back to `getByRole('link', { name: <entityName> })` or `getByText(<entityName>)` where `<entityName>` is the value filled into the creation form.
+
+**Example — entity located by link:**
+
+```typescript
+// Before (fragile — times out if entity already exists):
+await page.getByRole('button', { name: 'Create Project' }).click();
+await page.getByPlaceholder('Project name').fill('TestProject');
+await page.getByRole('button', { name: 'Create Project' }).nth(1).click();
+
+// After (idempotent — works whether entity exists or not):
+const existingProject = page.getByRole('link', { name: 'TestProject' });
+if (await existingProject.isVisible().catch(() => false)) {
+  await existingProject.click();
+} else {
+  await page.getByRole('button', { name: 'Create Project' }).click();
+  await page.getByPlaceholder('Project name').fill('TestProject');
+  await page.getByRole('button', { name: 'Create Project' }).nth(1).click();
+}
+```
+
+**Example — entity located by table row:**
+
+```typescript
+// Before (fragile):
+await page.getByRole('button', { name: 'Add Secret' }).click();
+await page.getByLabel('Secret name').fill('API_KEY');
+await page.getByRole('button', { name: 'Save' }).click();
+
+// After (idempotent):
+const existingSecret = page.getByRole('row', { name: 'API_KEY' });
+if (await existingSecret.isVisible().catch(() => false)) {
+  await existingSecret.click();
+} else {
+  await page.getByRole('button', { name: 'Add Secret' }).click();
+  await page.getByLabel('Secret name').fill('API_KEY');
+  await page.getByRole('button', { name: 'Save' }).click();
+}
+```
+
+## Pre-Edit Assertion Analysis
+
+Before editing `$2`, you must output a `<thinking>` block. It states the behaviour each test evaluates, then works through the rules for the items that behaviour depends on, so no rule that applies is overlooked. The JSON array should match the template below — every assertion category and every rule title under it must appear as a key, with `[]` where the rule does not apply.
+
+1. For each test in scope, name the behaviour it evaluates in one line, taken from the recommendation it came from, and name the page where that behaviour shows. A generated spec may carry a title that names its behaviour — trust that title when it does. A spec still named `testUi` claims nothing, so use the name only when it is specific enough to add something.
+2. Selector inventory — list every selector already present in `$2` (`data-testid`, role + name, text, label, etc.). New assertions may use only selectors from this list. An assertion the recorder dropped because it failed during recording may use the selector of the element named in that recording's blueprint capture. Do not invent `data-testid` values, role names, or aria attributes. Also note captured network responses, repeated element patterns, exact rendered text/value/attribute from trace/source, and existing `toBeVisible()` assertions whose exact text is knowable.
+3. Process — Replay → Identify → Fix or Add — walk through these three steps explicitly.
+   a. Replay the scenario mentally. At each state-changing action (form submit, item add/edit/delete), ask: "What is the EXPECTED outcome based on the action performed?"
+   b. Identify expectation mismatches. If the recorded trace shows a result that contradicts the action (e.g. removing 1 of 2 items but the page shows 3, submitting a form but getting a blank page, editing a field but the old value persists), that is an app bug the test should catch. List every mismatch you find.
+   c. Fix or add assertions for each mismatch.
+      - If an existing assertion uses the wrong (buggy) value, edit it to assert the correct expected value.
+      - If no assertion exists for the buggy behavior, add one immediately after the action that triggers it.
+4. Classify each in-scope action / selector / captured response by its applicable assertion category, marking each category APPLICABLE or NOT APPLICABLE.
+   - Critical UI Assertions applies when there is a collection / repeated element (`toHaveCount`), a pageerror handler, a captured network response, a negative-only or URL-only test needing a positive-path companion, a tautological locator-by-text + assert-text to replace, or a colour / class / size / element-order change the diff states (Appearance and layout changes), or an element the PR removed (Removed element guard).
+   - Computed Values applies when there is a knowable exact text / value / attribute, including a definition-page displayed value or an image `src`.
+   - Post-edit State applies whenever EITHER (a) the test contains a state-changing action (form fill+submit, save / delete / create / toggle, checkbox click, hover / mouseout, refresh, reload, JS update, form edit), OR (b) the PR changes interactive behavior — in case (b), add the triggering action and assert the before/after transition (see "Exercise the changed interaction"). Mark NOT APPLICABLE only when the PR changes no interactive behavior and the test has no state-changing action.
+5. For each in-scope item, output one JSON object using the template below. The output is an array — repeat the object template below once per in-scope item.
+   - `action_or_selector_or_response`: the selector, action, or captured network response this entry covers.
+   - `behaviour_under_test`: the one-line behaviour from step 1 that this item is evidence for. When an item is evidence for no behaviour any test in scope claims, write `none` and add nothing for it.
+   - `assertion_categories`: an object that MUST contain every category name below as a key. The value of each category is itself an object that MUST contain every rule title under that category as a key. For each rule, the value is the assertion lines you will add for this item under that rule. `[]` means the rule does not apply here, which is a normal and expected answer: it is the right one whenever the rule's elements are not part of `behaviour_under_test`. Listing every key keeps a rule that DOES apply from being skipped; it is not a quota to fill.
+
+```json
+[{
+  "action_or_selector_or_response": "<selector | action | response>",
+  "behaviour_under_test": "<the one-line behaviour this item is evidence for, or none>",
+  "assertion_categories": {
+    "Critical UI Assertions": {
+      "Exercise the changed interaction (dynamic behavior)": [],
+      "Appearance and layout changes": [],
+      "Selector constraints": [],
+      "Removed element guard": [],
+      "Page errors": [],
+      "Correct intended behavior": [],
+      "Tests target the changed behavior introduced by the pull request": [],
+      "Collection / repeated UI elements": [],
+      "Positive-path companion": [],
+      "Captured network responses": []
+    },
+    "Computed Values": {
+      "Exact rendered values": []
+    },
+    "Post-edit State": {
+      "Post-action visible state": []
+    }
+  }
+}]
+```
+
+## Assertion Rules with Examples
+
+Most-violated patterns — whenever the behaviour under test involves one of these, this is the shape it takes. (1) repeated elements → `toHaveCount(N)` + per-item `toHaveText`/`toHaveValue`/`toHaveAttribute` (Collection / repeated UI elements). (2) post-action state → `toHaveText`/`toHaveValue`/`toHaveAttribute`/`toBeChecked`, not `toBeVisible` when exact value is knowable (Post-action visible state). (3) routing/URL-only tests → at least one rendered-element exact text (Positive-path companion).
+
+Strength order. `toHaveCount`/`toHaveText`/`toHaveValue`/`toHaveAttribute` > `toContainText` (only when string is genuinely dynamic) > `toBeVisible` (only when presence is the actual test).
+
+### Critical UI Assertions
+
+#### Exercise the changed interaction (dynamic behavior)
+
+When the PR introduces or changes interactive behavior (toggle, theme switch, expand/collapse, dialog open/close, focus management, optimistic update), the test MUST perform the triggering action and assert BOTH the state before AND after the action — not only the default or resting state. WHY: the PR's change IS the transition; a test that asserts only the initial/resting state would still pass even if the interaction were completely broken, so it gives zero coverage of what changed. If the test asserts only the resting state, add the action that triggers the change and assert the resulting transition — never mark this not applicable merely because the generated test lacks an action.
+
+- Toggle / switch: click the control and assert the observable flips (e.g. data-theme 'light' -> 'dark'), then click again and assert it reverses.
+- Dialog / modal: open it and assert focus moves into the dialog; close via ESC or Cancel and assert focus returns to the trigger.
+- The after-state must differ from the before-state — a single assertion of the unchanged default value does not verify the change.
+
+```javascript
+await expect(page.locator('html')).toHaveAttribute('data-theme', 'light');
+await page.getByTestId('theme-toggle').click();
+await expect(page.locator('html')).toHaveAttribute('data-theme', 'dark');
+```
+
+```python
+trigger = page.get_by_test_id('delete-recipe')
+trigger.click()
+expect(page.get_by_role('dialog')).to_be_focused()
+page.keyboard.press('Escape')
+# focus returns to the trigger
+expect(trigger).to_be_focused()
+```
+
+#### Appearance and layout changes
+
+When the PR changes how an element looks or where it sits (colour, class, size, `flex-direction`, a moved block), assert the changed property on the element that OWNS it — `flex-direction` belongs to the flex container, not the clicked child, and an order check needs both compared elements. Use a selector already in the file; if the owner has none, report a recording gap in your `<thinking>` block — do not invent one or screenshot it.
+
+- Colour / class: a class assertion recorded at capture is already written as a bounded regex (`toHaveClass(/(^|\s)bg-blue-600(\s|$)/)`) — leave it as generated. Use `toContainClass('bg-blue-600')` only for a class assertion you add yourself (1.52+; it matches one token, unlike a bare `/bg-blue-600/` which also passes on `hover:bg-blue-600`). For a style, `toHaveCSS('background-color', <the value the app computes>)` — take it from the app, do not convert the diff's hex: Tailwind v4 serializes `oklch(...)`, not `rgb(...)`.
+- Order: compare both elements' `boundingBox()` on the changed axis (holds for `flex-direction` / `order` / `dir` / grid); `toHaveCSS('flex-direction', 'row-reverse')` only when the container is `display: flex` with an unconditional class.
+- Size: `toHaveCSS('width', <the computed width>)`. Name the value; never copy a `...` or `/…/` placeholder.
+
+A `toHaveScreenshot()` baseline is right only for a look no property expresses (image, gradient, chart) or to refresh an already-committed baseline (SKYR-4298) — never a new one for a colour, class or order change.
+
+#### Selector constraints
+
+Every assertion uses a selector already in the file. Never invent `data-testid`, role names, or classes. One exception: a `data-*` attribute + value listed in `uiContext.removedElements` (from the `skyramp_analyze_changes` output in your context) may be asserted `toBeHidden()` even though no element carries it any more — `getByTestId(value)` for `data-testid`, `page.locator('[data-cy="value"]')` for any other attribute — see Removed element guard. A `-` line in the raw diff is not enough on its own: renames and moved elements also appear there.
+
+- No tautological assertions — locating an element by text X, then asserting it contains X.
+
+```javascript
+await expect(page.getByTestId('badge-count')).toHaveText('3');
+```
+
+#### Removed element guard
+
+When the PR deletes a rendered element (a `data-*` attribute + value listed in `uiContext.removedElements`) and the test visits the page that rendered it, assert `toBeHidden()` on it — `getByTestId(value)` when the attribute is `data-testid`, otherwise `page.locator('[<attribute>="<value>"]')` (never `getByTestId` for `data-cy`/`data-qa`: Playwright resolves it against `data-testid` and the guard would pass vacuously). `toBeHidden` passes when the locator matches nothing or a single non-visible node (it is strict on multiple matches, like every locator assertion), which is the right contract for a removed element; do not substitute a `toBeVisible` on an unrelated element. Pair it with at least one positive assertion on a retained element of the same page so the test cannot pass on a blank or un-rendered page.
+
+- Anchor on the removed attribute+value or role+name, never on text or position — a text-based absence check breaks when unrelated siblings change.
+- Do not add a guard for an element whose whole page or route was removed — that is maintenance of the tests that covered it, not a new assertion; a spec that opens a deleted route must be anchored on a surviving page first (a retained element asserted on a live page in the same spec) — a spec whose only visited page is the dead URL is not a valid guard.
+- A renamed test id or accessible name (a removed `aria-label` included) is not a removed element: assert the element under its new identifier and never assert the retired identifier `toBeHidden()` — it passes today and turns into a false failure the day the name is reused.
+- An element that exists only inside a control the page has to open — an `option` of a `select`, an item of a closed menu or dropdown — takes `toHaveCount(0)`, not `toBeHidden()`. A closed control's contents have no bounding box, so `toBeHidden()` passes whether the element was removed or not, and the guard stays green the day it comes back. If the spec opens the control before asserting, the element would have had a box had it survived, so assert both.
+- The positive companion for that guard is counted or read, never `toBeVisible`: the same missing bounding box that makes `toBeHidden()` a false green on the removed entry makes `toBeVisible()` a false red on the one that stayed. Assert a retained sibling with `toHaveCount(1)` or `toHaveText`, or assert the control itself, which does have a box.
+
+```javascript
+await expect(page.getByTestId('navbar-sessionid-row')).toBeHidden();
+await expect(page.locator('[data-cy="checkout-btn"]')).toBeHidden();
+await expect(page.getByTestId('navbar-products')).toHaveText('Products');
+```
+
+```python
+expect(page.get_by_test_id('navbar-sessionid-row')).to_be_hidden()
+expect(page.get_by_test_id('navbar-products')).to_have_text('Products')
+```
+
+#### Page errors
+
+Register `page.on('pageerror', ...)` before the first navigation and assert `expect(errors).toHaveLength(0)` at the end of the test.
+
+#### Correct intended behavior
+
+Assertions encode the expected outcome of the action, not the buggy runtime output observed in the trace.
+
+```javascript
+await expect(page.locator('[data-testid="session-row"]')).toHaveCount(3);
+```
+
+#### Tests target the changed behavior introduced by the pull request
+
+Every test needs at least one assertion that fails when the behaviour it claims is broken. This is the rule the others support: a test that satisfies every rule below and still does not distinguish the behaviour working from the behaviour broken has not been enhanced. Target it on populated or updated state, not only on empty or zero state, and assert it on the page where the behaviour shows.
+
+When the PR changed the behaviour, the expected value is the one the diff, the PR description, or the referenced spec establishes — even when the running app still renders the old value. The test is expected to fail until the code is fixed.
+
+#### Collection / repeated UI elements
+
+For every collection page or selector matching multiple rendered nodes (items, rows, cards, badges, tabs, chips, definition rows, nav items), assert the exact count plus per-item content.
+
+- Use `toHaveCount(N)` where N comes from the trace array length or rendered DOM (`0` for empty state).
+- Add at least one concrete per-item content assertion (`toHaveText`, `toHaveValue`, `toHaveAttribute`).
+- Empty trace → `toHaveCount(0)` paired with the empty-state text.
+
+```javascript
+await expect(page.getByRole('row')).toHaveCount(2);
+await expect(page.getByRole('cell', { name: 'admin@example.com' })).toHaveText('admin@example.com');
+```
+
+```javascript
+await expect(page.getByTestId('notification-badge')).toHaveText('3');
+await expect(page.getByTestId('notification-row')).toHaveCount(3);
+```
+
+```javascript
+await expect(page.getByTestId('definition-row')).toHaveCount(3);
+await expect(page.getByTestId('definition-row').nth(0)).toHaveText('Version 10.2.1');
+```
+
+#### Positive-path companion
+
+Negative-only checks and URL-only routing tests must include at least one positive rendered-element assertion such as a heading, breadcrumb, title, link, count, or exact text.
+
+```javascript
+await expect(page).toHaveURL(/\/users/);
+await expect(page.getByRole('heading')).toHaveText('User Directory');
+```
+
+#### Captured network responses
+
+For every `page.waitForResponse`, `page.on`, or `page.route`, assert at least one status, body, or header field. Body values used downstream must also appear as visible UI text.
+
+```javascript
+const loginResp = await page.waitForResponse('**/login');
+expect(loginResp.status()).toBe(200);
+expect((await loginResp.json()).user.email).toBe('admin@example.com');
+```
+
+### Computed Values
+
+#### Exact rendered values
+
+When the exact text, value, or attribute is knowable from the trace or source, use `toHaveText`, `toHaveValue`, or `toHaveAttribute` — never `toBeVisible` or `toContainText`.
+
+- Definition / info pages: assert the actual displayed values (version, count, date), not just the static label text.
+- Images: assert `toHaveAttribute('src', ...)` with the exact src — never visibility alone.
+
+```javascript
+await expect(page.getByTestId('status')).toHaveText('Active');
+```
+
+```javascript
+await expect(page.getByTestId('product-image')).toHaveAttribute('src', /expected-pattern/);
+```
+
+### Post-edit State
+
+Apply only when the test contains a state-changing action. Otherwise mark NOT APPLICABLE.
+
+#### Post-action visible state
+
+After a state-changing action (form fill+submit, save / delete / create / toggle, checkbox click, hover / mouseout, refresh, reload, JS update, form edit), assert the visible updated outcome via `toHaveText`, `toHaveValue`, `toHaveAttribute`, or `toBeChecked`. The displayed values must reflect the edit.
+
+- Toggle / checkbox renames: assert the new label or state, not `toBeVisible` on the renamed element.
+- Hover / mouseout: assert the overlay or state visible / hidden after the event.
+- List edit: when the test submits N items, assert exactly N rows. When the count is unknown before the action, read it before and assert the delta after.
+
+```javascript
+await expect(page.getByTestId('description-checkbox')).toBeChecked();
+await expect(page.getByTestId('description-label')).toHaveText('check_box Description');
+```
+
+```javascript
+await expect(page.getByTestId('total')).toHaveText('\$19.98');
+```
+
+```javascript
+await expect(page.getByTestId('version')).toHaveText('10.2.1');
+```
+
+## What Not to Do
+
+- Do not invent selectors, values, attributes, or assertions
+- Do not use `toBeVisible` when exact text/value/state is knowable
+- Do not use tautological locator-by-text + assert-text
+- Do not use `toContainText` when the full string is known
+- Do not assert static page headings, intermediate states, or values guaranteed by the action you just took
+- Do not assert the same value with multiple selectors
+- Do not import `expect` from `@playwright/test`
+- Do not restructure, reorder, or modify existing code/imports/signatures
+- Do not add comments or docstrings
+- Do not remove or modify existing assertions
+- Do not assert buggy/error text as expected (unless the intended UX is an error state)
+- Do not add a `toHaveScreenshot()` baseline for a colour, class, size or order change the diff states — assert the property (Appearance and layout changes); a baseline created on this run cannot fail on it, unless that page already has a committed baseline PNG (then it is a rebaseline, not a new one)
+
+## Verification of Assertions
+
+After adding the assertion lines in `$2`, re-read each test and check two things: every rule that applies to the behaviour under test has been applied correctly, and the test now fails if that behaviour breaks. Fix anything missing or weakly applied before completing. Remove any assertion you added that the behaviour does not depend on.
+
+The goal is tests that FAIL when the app has bugs, not tests that simply replay what happened.
+
+## Additional checks in Testbot runs
+
+Apply this section only inside a Skyramp Testbot run.
+
+- If no suitable selector exists in `$2` for an assertion you need to add, go back and call `browser_assert` on the live page to record it with a valid selector, then re-export and regenerate.
+- **After executing a UI test that documents a bug from `issuesFound`**: if it passed when you expected it to fail, the assertions are too weak — add a stronger `expect()` that directly targets the buggy behavior. This counts as the single allowed retry under the 2-attempt cap — do NOT re-run more than once.