@zappie/verify 0.2.2 → 0.2.4

package/CHANGELOG.md

@@ -1,5 +1,54 @@
 # @zappie/verify
 
+## 0.2.4
+
+### Patch Changes
+
+- 53f28fb: V14, V15 and V16 leave the page the previous check opened before they score the CPU and start timing. A delayed step on that page, such as an item dialog's history update, no longer aborts V14's page load or skews the CPU score.
+
+  V13 is steadier:
+
+  - It scans once a few frames pass with no entry animation running, so an overlay that starts its animation a frame after it opens is read once it has settled.
+  - When checkout is already enabled and the phone sits behind a `data-zappie-field-open="phone"` row, as after a signed-in pre-fill, V13 doesn't type a number. It starts the keyboard walk at that row instead.
+  - A typed value that a sheet still mounting drops is typed again.
+
+- Updated dependencies [488e47d]
+  - @zappie/sdk@0.1.10
+
+## 0.2.3
+
+### Patch Changes
+
+- 8f75ba3: V10 and V13 reach the address and phone fields when a template opens them from a row.
+
+  - Mark the row with `data-zappie-field-open="address"` or `"phone"`. Verify taps it when the field isn't on the page.
+  - When the tap sends a signed-out shopper to sign in, verify signs in with the test session, returns to checkout, repeats the step before the row and taps it again.
+  - A phone field in a dialog is saved through `data-zappie-field-confirm`, or else the open dialog's submit or footer button, before V13 checks the checkout control.
+
+  V13 also waits for entry animations to finish before each accessibility scan, so an overlay that fades in is read as the shopper sees it once it has opened, not halfway through the fade. Animations that loop forever, such as a loading spinner, are not waited for.
+
+  V13's phone touch-target rule now asks for 24×24 px, the WCAG 2.2 AA size (2.5.8), instead of 44×44 px, the AAA size.
+
+  On a screen 768 px or wider, V13 accepts a cart the page always shows as a column, with no control to open it. It scans the cart with the page instead of opening it. A phone still needs a cart control.
+
+- 22ccfe2: Verify scores the CPU again just before each timed page in V14, V15 and V16, not only at the start of the run, because a shared runner's speed changes during a run. The new `overlayOpenRates` metric records the throttling rate each V16 sample ran at.
+- b932f64: The checks no longer assume one storefront design.
+
+  - V05 and V13 open an item through its `data-zappie-item` marker instead of a "View details" link. When the marked element is a link or button, as on a card that is the link, they tap it. Otherwise they tap a link inside the card, then a link beside it.
+  - V13 closes the item by a dialog button whose name contains "Close", or by Escape, instead of a button named "Close item". On a menu without quick add, it reaches Add from the keyboard through an item.
+  - V05 accepts an Add that stays enabled before a required choice, when a tap adds nothing and flags the group with a visible message. A tap that flags nothing fails with `required_option_unflagged`.
+  - V06 finds the quoted total in the bag or, when the bag hands over to a checkout page, on that page.
+  - V10 confirms a picked address when the template asks for it, through `data-zappie-address-confirm` or the open dialog's submit or footer button.
+  - V16 and V17 add an item without required options first when the bag trigger shows only once the cart has lines, then time and walk the bag as before.
+  - V06, V07 and V13 open the cart through `data-zappie-overlay-trigger="bag"` when no `data-zappie-cart-trigger` is marked.
+
+  The README lists the markers each journey relies on.
+
+- 5a4f4de: V16 times each overlay's first open in a new browser context instead of on a reloaded menu, so neither the checks before it nor its own earlier samples can warm the overlay. It still opens each overlay three times, now each in its own context, and judges the median against the 100 ms budget. V17 checks each overlay in a new browser context too.
+- Updated dependencies [e7cc5f0]
+- Updated dependencies [1c3cbbf]
+  - @zappie/sdk@0.1.9
+
 ## 0.2.2
 
 ### Patch Changes

package/README.md

@@ -42,9 +42,34 @@ console.log(printSummary(report));
 
 The report's shape is the JSON Schema at `@zappie/verify/schema/verify-report.v1.json`.
 
+## Markers
+
+The checks find shopping controls by their `data-zappie-*` markers, the ones the `zappie/verify-markers` lint rule and the template standards ask for, not by their labels, so your copy is yours to write. Where no marker names a control, a check uses its role. Text is read in three places only: V03 counts an item by its name when its card has no marker, V10 picks the address it typed from the suggestions, and V13 closes an item by a button whose name contains "Close".
+
+They accept either way of building a journey where templates differ:
+
+- **Opening an item.** When the element marked `data-zappie-item` is a link or button, as on a card that is the link, the checks tap it. Otherwise they tap the first link inside the marked card, then the first link beside it.
+- **The cart control.** `data-zappie-cart-trigger`, or on a template that marks only its overlays, the visible `data-zappie-overlay-trigger="bag"` control.
+- **Required options (V05).** Add can be disabled until every required choice is made. Or it can stay enabled: a tap then adds nothing, keeps the item open and marks the group with `aria-invalid` or `data-invalid` and a visible message, in the group, in its `aria-describedby` or `aria-errormessage` target, or in a `role="alert"` in the dialog.
+- **The quoted total (V06).** `data-zappie-total` in the bag, or, when the bag hands over to a checkout page through `data-zappie-cart-checkout` or a link to `/checkout`, on that page.
+- **A cart column (V13).** On a screen 768 px or wider, a cart the page always shows, with no visible cart control, is scanned with the page. V13 then skips opening it and the focus-return check for it. On a phone the cart control is still required.
+- **A checkout field behind a row (V10, V13).** The address and phone fields can be on the page, or open from a row marked `data-zappie-field-open="address"` or `"phone"`. Verify taps the row. When the tap sends a signed-out shopper to sign in, on the site or another origin, verify signs in with the test session, returns to the page, repeats the step before the row (choosing Delivery for the address) and taps it again. A phone field in a dialog is saved with `data-zappie-field-confirm`, or else the enabled submit or footer button of the dialog.
+- **A picked address (V10).** It can be stored as soon as the shopper picks it. Or the template can ask them to confirm it first: verify then taps `data-zappie-address-confirm`, or else the enabled submit or footer button of the open dialog.
+- **A bag trigger shown only once the cart has lines (V16, V17).** When the menu shows no `data-zappie-overlay-trigger="bag"`, they first add an item without required options, the way V04 does, open the menu again and then time or walk the bag as before.
+
+V13 scans each page and overlay once its entry animations have finished, so a fade-in is read as it ends. V13 relies on these markers:
+
+| Marker                             | What V13 does with it                                                                                                                                               |
+| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `data-zappie-item`                 | Opens the first available item, as above, and checks that focus returns to its card when it closes.                                                                 |
+| `role="dialog"`                    | Finds the open item and closes it by a button whose name contains "Close", or else by Escape.                                                                       |
+| `data-zappie-cart-trigger`         | Opens the cart for its scan, and checks that focus returns to it when the cart closes.                                                                              |
+| `data-zappie-action="add-to-cart"` | Checks that the keyboard reaches an Add control. On a menu without quick add, it tabs to an item without required options, opens it with Enter and tabs to its Add. |
+| `data-zappie-checkout`             | Checks that the keyboard reaches the enabled checkout control on `/checkout`.                                                                                       |
+
 ## Overlays
 
-Mark each control that opens an overlay with `data-zappie-overlay-trigger="bag"`, `"branch"`, `"search"` or `"item"`, and the overlay's `role="dialog"` element with `data-zappie-overlay` and the same name. V16 waits until the menu has hydrated and its idle work, such as overlay preloads, and the downloads that work starts are done, as they would be for a shopper who has read the menu. Then it taps each marked trigger on a phone profile with the CPU slowed four times, and fails when the overlay takes more than 100 ms to paint. Anything the tap itself loads counts toward that time. It opens each overlay three times, each on a freshly loaded and settled menu, and judges the median, so one slow sample on a busy machine doesn't fail it. The CPU slowdown is calibrated to the machine: before the checks run, verify scores the browser's CPU with a short fixed benchmark and sets the throttling rate so that V14, V15 and V16 see the same phone everywhere. A developer laptop gets about four times, a slower CI runner less, between one and eight times. The report's `environment` records `cpuBenchmark` and the `cpuThrottlingRate` the checks used. The report's `overlayOpenMs` holds the medians and `overlayOpenSamplesMs` holds every sample, in the order they ran. V17 opens each one from the keyboard and checks that focus moves in, Tab stays in, the page behind doesn't scroll, Escape closes it and focus returns to the trigger. Both are skipped while the menu has no markers.
+Mark each control that opens an overlay with `data-zappie-overlay-trigger="bag"`, `"branch"`, `"search"` or `"item"`, and the overlay's `role="dialog"` element with `data-zappie-overlay` and the same name. V16 waits until the menu has hydrated and its idle work, such as overlay preloads, and the downloads that work starts are done, as they would be for a shopper who has read the menu. Then it taps each marked trigger on a phone profile with the CPU slowed four times, and fails when the overlay takes more than 100 ms to paint. Anything the tap itself loads counts toward that time. Each tap is a shopper's first open on a first visit: V16 opens every overlay in a new browser context, so the checks before it can't warm it. It opens each overlay three times, each in its own context, and judges the median, so one slow sample on a busy machine doesn't fail it. The CPU slowdown is calibrated to the machine: before the checks run, verify scores the browser's CPU with a short fixed benchmark and sets the throttling rate so that V14, V15 and V16 see the same phone everywhere. A developer laptop gets about four times, a slower CI runner less, between one and eight times. The report's `environment` records `cpuBenchmark` and the `cpuThrottlingRate` the checks used. The report's `overlayOpenMs` holds the medians and `overlayOpenSamplesMs` holds every sample, in the order they ran. V17 opens each one from the keyboard, in a new browser context too, and checks that focus moves in, Tab stays in, the page behind doesn't scroll, Escape closes it and focus returns to the trigger. Both are skipped while the menu has no markers.
 
 ## Compare with a baseline
 

package/dist/index.d.ts

@@ -59,8 +59,10 @@ type VerifyReport = {
         menuCssKbGz?: number;
         /** Tap to first paint of each marked overlay, by its `data-zappie-overlay` name: the median of its samples. */
         overlayOpenMs?: Record<string, number>;
-        /** Each overlay's timed opens, in the order they ran. */
+        /** Each overlay's timed first opens, one per fresh browser context, in the order they ran. */
         overlayOpenSamplesMs?: Record<string, number[]>;
+        /** The CPU throttling rate each of those samples ran at, calibrated just before it. */
+        overlayOpenRates?: Record<string, number[]>;
     };
     signature?: string;
 };